repowiseactiveloopai/deeplake

Architecture of activeloopai/deeplake

activeloopai/deeplake is 856 source files holding 13,494 symbols across 5 modules, written mainly in C++, Python, C. The largest module is cpp, with 612 files, and 4 files are entry points into the dependency graph.

How deeplake fits together

The python/ directory holds 51 Python files, and the README describes one API for uploading, downloading and streaming datasets. The cpp/ directory holds 612 files, the largest module, and cpp/CMakeLists.txt is its CMake build manifest. The postgres/ directory holds 131 files for the Postgres side, with tests under postgres/tests/py_tests, and .github/workflows/pg-extension-build.yaml builds that extension in continuous integration. Datasets are then read back for querying, vector search, streaming to PyTorch or TensorFlow dataloaders, or visualization. The docs/ directory holds 49 documentation files.

Largest modules

  • cpp: 612 files and 154,896 lines of code
  • postgres: 131 files and 17,857 lines of code
  • python: 51 files and 12,081 lines of code
  • docs: 49 files and 6,583 lines of code
  • root: 5 files and 427 lines of code
  • .github: 4 files and 468 lines of code
  • scripts: 4 files and 620 lines of code

Languages

By file count: C++ 478 files, Python 123 files, C 66 files, Shell 13 files, Makefile 4 files, JavaScript 3 files.

Documentation

Repowise documents deeplake in 1,536 pages generated from the source.

More on deeplake: summary and FAQ, codebase overview, code health.

Related guide: Best dependency graph tools for monorepos