repowisevortex-data/vortex

Architecture of vortex-data/vortex

vortex-data/vortex is 2,724 source files holding 37,846 symbols across 51 modules, written mainly in Rust, Python, C++. The largest module is vortex-array, with 769 files, and 99 files are entry points into the dependency graph.

How vortex fits together

Logical types and schema are defined separately from the physical layout, which handles encoding and storage. The encodings directory holds the encoding implementations, which are the pluggable physical layouts that compress column data. The vortex-layout module handles how encoded data is arranged and read, and the vortex-array module is the largest module, with 769 files. The vortex-python module provides the Python package, and vortex-duckdb connects vortex to DuckDB. Data read from a vortex file keeps its encoded form, and compute kernels operate on that encoded data directly.

Largest modules

  • vortex-array: 769 files and 141,087 lines of code
  • encodings: 279 files and 44,683 lines of code
  • vortex-bench: 221 files and 17,964 lines of code
  • vortex-python: 161 files and 15,114 lines of code
  • vortex-cuda: 93 files and 22,803 lines of code
  • vortex-duckdb: 92 files and 15,154 lines of code
  • vortex-layout: 82 files and 18,534 lines of code
  • java: 81 files and 9,380 lines of code

Languages

By file count: Rust 1,908 files, Python 149 files, C++ 75 files, Java 74 files, TypeScript 59 files, Shell 13 files.

Documentation

Repowise documents vortex in 3,317 pages generated from the source.

More on vortex: summary and FAQ, codebase overview, code health.

Related guide: Best dependency graph tools for monorepos