repowisevectifyai/pageindex

Architecture of vectifyai/pageindex

vectifyai/pageindex is 140 source files holding 2,040 symbols across 4 modules, written mainly in Python, JavaScript, Shell. The largest module is pageindex, with 96 files, and 2 files are entry points into the dependency graph.

How pageindex fits together

Documents enter as files such as PDFs and are passed to PageIndexClient, which is imported from pageindex. pageindex/flash/main.py is one of the two entry points and runs the PageIndex Flash pipeline that builds the tree index for text-based PDFs. The pageindex/flash/clustering directory groups visual and positional evidence into stable regions and proposes candidates that later assembly can use as structural anchors. The pageindex/flash/labels directory models blocks, lines, spans and caption-like areas as bounded regions, so captions are handled as regions rather than stray text. pageindex/local_chat.py is one of the hotspot files for chat in local mode, where an LLM answers questions from the retrieved parts of the tree, and each finished index is saved as a JSON file that later questions reuse.

Largest modules

  • pageindex: 96 files and 23,137 lines of code
  • tests: 12 files and 10,169 lines of code
  • .github: 2 files and 250 lines of code
  • root: 1 file and 172 lines of code

Languages

By file count: Python 109 files, JavaScript 1 file, Shell 1 file.

Documentation

Repowise documents pageindex in 246 pages generated from the source, 15 of them written explanations of the repository and its modules. Module guides include:

  • pageindex/flash/parser_pdfium_charlevel: Char-Level PDF Content Walking
  • pageindex/flash: Flash API and Build Orchestration
  • pageindex/flash/clustering: Flash Clustering and Layout Grouping

More on pageindex: summary and FAQ, codebase overview, code health.

Related guide: Best dependency graph tools for monorepos