vortex-data / vortex
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
github.com/vortex-data/vortexIndexed at a272f51Up to date with upstream
About this codebase
2,724 files and 37,846 symbols in 51 modules, led by Rust (1,908 files) and Python (149).
Built from the index
Code health
This codebase scores 7.5 out of 10 for code health, which we rate good. It also scores maintainability 8.0 and static performance 9.9 out of 10. The three are scored separately and never blended into one number. Risk is concentrated, as it usually is: 427 of 2,724 files are git hotspots, and they average 6.1 — which is where the fixes pay off most.
On the leaderboards:#2 of 10 Rust repos
Do next
1 thing worth doing this quarter.
Worth planning
Delete 371 unused symbols and files (2,726 lines)
Nothing in the graph reaches them, across 58 files; every reader and agent pays to skip them.
- symbols
- 371
- lines
- 2,726
- safe to delete
- Yes(inferred)
Medium effort
Improve what Repowise can see
Add a test coverage report
Without one, Repowise cannot tell tested code from untested code, so every test-related action says “unknown”.
- line coverage
- Unknown, no report
Small effortReview the 36 proposed decisions
None is accepted yet, so nothing can drift from one and agents get no enforced guidance from them.
- proposed
- 36
- accepted
- 0
Medium effort
Use vortex-data/vortex in your agent
Point Claude Code, Cursor, Codex or VS Code at this repository. Your agent gets the index this page is built from: cited answers, callers, history and health for any file.
https://api.repowise.dev/mcp/vortex-data/vortexOne command, run anywhere. Adds the server to your local Claude Code config.
Docsclaude mcp add --transport http repowise https://api.repowise.dev/mcp/vortex-data/vortex \ --header "Authorization: Bearer rw_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
This repo is public, so anyone with a Repowise account and an API key can connect.
Try asking
- get_overview()
- get_answer("how is vortex put together?")
- get_why("why is vortex structured this way?")
Prefer local? pip install repowise && repowise init indexes your own checkout, no account needed.
Go deeper
- OverviewEverything the index found, what changed this week, commits and decisions.
- DocumentationModule-by-module documentation, 3,317 pages, 0 written by a model.
- ArchitectureDependency graph, layers and third-party dependencies.
- Code healthEvery file's score, the findings behind it, tests and refactoring plans.
- CommitsEach commit's effect on code health, with agent provenance.
- ContributorsWho owns what, and where one person holds most of the knowledge.
- DecisionsArchitectural decisions mined from commits and pull requests.
- Ask this codebaseSign inGrounded answers over the indexed code.
Maintain this repo? Claim the page
Sign in with GitHub and we check that you are an admin or maintainer of vortex-data/vortex. Then add a badge to the README that links readers here.
Use this repo from Claude or ChatGPT
Add this URL as a custom connector and sign in with repowise when asked. No API key to copy.
https://api.repowise.dev/mcp/vortex-data/vortexFor maintainers
Add a badge to your README
Every badge is public, cached, and updates on its own after every index. Nothing to install.
Repowise badge
Links to this page, with no score on it. Adding it also re-indexes the repo every week.
[](https://repowise.dev/repo/vortex-data/vortex?src=badge_wiki)Code health badge
The repo's health score out of 10, from the latest index.
[](https://repowise.dev/repo/vortex-data/vortex?src=badge_health)Code health card
A larger card with the score, its change since the last index, and when it was measured. The HTML version follows the reader's light or dark theme.
[](https://repowise.dev/repo/vortex-data/vortex?src=badge_card)What is vortex?
vortex is a columnar file format and toolkit for high-performance data processing. It separates logical schemas from physical storage so systems can use pluggable encodings, compression strategies and layout strategies. The project is primarily written in Rust and also provides Python, C++, Java and TypeScript components.
Written by a model from the README and the index
Key features of vortex
- Vortex separates logical types and schemas from physical data layouts.
- Vortex supports pluggable encodings, compression strategies and layout strategies.
- Vortex provides zero-copy conversion to and from Apache Arrow arrays.
- Vortex supports cascading compression through nested encoding schemes.
- Vortex provides compute kernels for encoded data and lazy-loaded summary statistics.
- Vortex integrates with Arrow, DataFusion, DuckDB, Spark, Pandas and Polars.
How vortex-data/vortex works
Vortex accepts columnar data through its logical type and schema layer, then maps that data to physical layouts for encoding and storage. The vortex-array module provides array representations and compute operations, while encodings supplies pluggable physical encodings and compression schemes. The vortex-layout module handles layout strategies, and vortex-file provides file-level operations and tests. The vortex-ffi/cinclude/vortex.h header exposes a C interface, while vortex-python provides Python bindings. The vx command-line tool browses Vortex file structures, and vortex-duckdb supports DuckDB integration.
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation. This page is a map of the vortex-data/vortex repository, written primarily in Rust, rebuilt from the source each time it is indexed. Repowise parses every symbol, computes a dependency graph, scores per-file code health from complexity, duplication, test coverage and churn, mines git history for hotspots and ownership, and lifts the architectural decisions into documentation you can read here or query through MCP.
The codebase has 2,724 files and 37,846 symbols in 51 modules, led by Rust, Python and C++. Code health is 7.5 out of 10, rated good.
Use the links above to open each view, or connect this repository to your agent for grounded answers inside Claude, Cursor, Codex or VS Code.
Getting started with vortex
- Add the Rust crate to a project.
cargo add vortex - Add the Python package to a project.
uv add vortex-data - Install the vx command-line tool from source.
cargo install vortex-tui --locked
Commands copied from the README
vortex-data/vortex FAQ
What is vortex?
vortex is a columnar file format and toolkit for high-performance data processing. It uses separate logical and physical layers for schemas, encodings and storage.
How does vortex work?
vortex maps logical columnar data to physical layouts through pluggable encodings and compression strategies. Its array, file, layout and integration modules connect storage operations with compute and external interfaces.
How do I install vortex?
vortex can be added to a Rust project with cargo add vortex or to a Python project with uv add vortex-data. The vx tool can be installed with cargo install vortex-tui --locked.
What language is vortex written in?
vortex is primarily written in Rust. The repository also includes Python, C++, Java, TypeScript and shell code.
How is the vortex code organized?
vortex is organized into modules including vortex-array, encodings, vortex-file, vortex-layout, vortex-python and vortex-duckdb. Cargo, npm and PyPI ecosystems are represented in the repository.
Who is vortex for?
vortex is for developers building data systems and high-performance processing workflows backed by object storage. It supports integrations with Arrow, DataFusion, DuckDB, Spark, Pandas and Polars.
What license does vortex use?
vortex uses the Apache-2.0 license.
Answers written by a model from the README and the index