Architecture of activeloopai/deeplake
activeloopai/deeplake is 856 source files holding 13,494 symbols across 5 modules, written mainly in C++, Python, C. The largest module is cpp, with 612 files, and 4 files are entry points into the dependency graph.
How deeplake fits together
The python/ directory holds 51 Python files, and the README describes one API for uploading, downloading and streaming datasets. The cpp/ directory holds 612 files, the largest module, and cpp/CMakeLists.txt is its CMake build manifest. The postgres/ directory holds 131 files for the Postgres side, with tests under postgres/tests/py_tests, and .github/workflows/pg-extension-build.yaml builds that extension in continuous integration. Datasets are then read back for querying, vector search, streaming to PyTorch or TensorFlow dataloaders, or visualization. The docs/ directory holds 49 documentation files.
Largest modules
- cpp: 612 files and 154,896 lines of code
- postgres: 131 files and 17,857 lines of code
- python: 51 files and 12,081 lines of code
- docs: 49 files and 6,583 lines of code
- root: 5 files and 427 lines of code
- .github: 4 files and 468 lines of code
- scripts: 4 files and 620 lines of code
Languages
By file count: C++ 478 files, Python 123 files, C 66 files, Shell 13 files, Makefile 4 files, JavaScript 3 files.
Documentation
Repowise documents deeplake in 1,536 pages generated from the source.
More on deeplake: summary and FAQ, codebase overview, code health.
Related guide: Best dependency graph tools for monorepos