Naive RAG build using fully open-source materials meant to be run with a local model (ie. Ollama - Qwen3.5:4B)
Local-first RAG prototype using Ollama, FAISS, and FastAPI
LORP is an early-stage local retrieval-augmented generation (RAG) project. The implemented path uses:
The project is a prototype, not a production-hardened or multi-agent platform. Framework integrations and features that are not implemented are identified as planned below.
The current implementation demonstrates:
Multi-agent orchestration, richer document ingestion, evaluation, monitoring, and deployment improvements remain planned work.
flowchart LR
A[User question] --> B[API]
B --> C[RAG agent]
C --> D[Retrieve context]
D --> E[Local LLM]
E --> F[Grounded answer]
classDef input fill:#14314f,stroke:#60a5fa,stroke-width:2px,color:#e0f2fe;
classDef process fill:#123a2b,stroke:#34d399,stroke-width:2px,color:#d1fae5;
classDef retrieval fill:#3a2a17,stroke:#fbbf24,stroke-width:2px,color:#fef3c7;
classDef output fill:#2d1d45,stroke:#a78bfa,stroke-width:2px,color:#f3e8ff;
class A,B input;
class C process;
class D retrieval;
class E,F output;
For the implemented API, agent, retrieval, model, and CLI paths, see the detailed runtime view.
.txt and .md files.nomic-embed-text by default.IndexFlatIP index using the CPU FAISS package.The loader can discover PDF, DOCX, and HTML paths, but the index builder skips those formats. They are not yet supported end to end.
KnowledgeAgent uses retrieved context to answer questions and includes source citations.src.prompt_flow provides a command-line question loop./health, /ready, /query, and /source endpoints, plus a basic browser UI at /.This interactive tree lists every tracked project file, excluding Git’s internal metadata. Entries marked planned are placeholders or stubs, not active implementations.
local-rag/├── .gitignore├── _config.yml├── LICENSE├── README.md├── requirements.txt├── _layouts/│ └── default.html├── ⚙️ config/│ ├── agents.yaml│ ├── models.yaml│ └── settings.yaml├── 💾 data/│ ├── indexed/│ │ ├── .gitkeep│ │ └── FAISS.txt│ ├── indexes/ — planned: generated FAISS indexes are created here│ │ └── .gitkeep│ ├── processed/│ │ ├── .gitkeep│ │ └── test.csv│ └── raw/│ ├── .gitkeep│ └── test.txt├── 🐳 docker/│ ├── api.Dockerfile│ ├── docker-compose.yaml│ ├── docker-compose.yml│ ├── ollama.Dockerfile│ └── webui.Dockerfile├── docs/│ └── lorp_runtime_flow.md├── src/│ ├── __init__.py│ ├── prompt_flow.py│ ├── 🤖 agents/│ │ ├── __init__.py│ │ ├── base_agent.py│ │ ├── knowledge_agent.py│ │ ├── report_agent.py — planned: structured report agent│ │ └── research_agent.py — planned: multi-step research agent│ ├── api/│ │ ├── __init__.py│ │ └── server.py│ ├── eval/│ │ ├── __init__.py│ │ └── rag_eval.py — planned: evaluation integration is a placeholder│ ├── ingestion/│ │ ├── __init__.py│ │ ├── build_index.py│ │ ├── loaders.py│ │ └── preprocess.py│ ├── llm/│ │ ├── __init__.py│ │ ├── embeddings.py│ │ └── ollama_client.py│ ├── retrieval/│ │ ├── __init__.py│ │ ├── llamaindex_client.py│ │ └── retrievers.py│ ├── 🧰 tools/ — planned: tool implementations are currently stubs│ │ ├── __init__.py│ │ ├── file_tool.py│ │ ├── retrieval_tool.py│ │ ├── sql_tool.py│ │ └── web_search_tool.py│ └── 🖥️ ui/│ ├── .gitkeep│ └── webui_integration.md — planned: UI integration has notes only└── 🧪 tests/ ├── test_agent.py ├── test_api.py ├── test_ingestion.py ├── test_prompt_flow.py └── test_retrieval.pyInstall Ollama, then pull a chat model and the default embedding model:
ollama pull qwen3.5:4b
ollama pull nomic-embed-text
The direct Python/API defaults use qwen3.5:4b. Docker Compose defaults CHAT_MODEL to qwen2.5:14b; pull that model instead or set CHAT_MODEL to another model available in Ollama.
pip install -r requirements.txt
python src/ingestion/build_index.py
The index command reads supported .txt and .md files from data/raw and also indexes the project README.md. It writes a FAISS index and metadata JSON under data/indexes/.
uvicorn src.api.server:app --reload
The interactive CLI is also available:
python -m src.prompt_flow
From the repository root, start the Ollama, API, and Open WebUI containers:
docker compose -f docker/docker-compose.yml up --build
The Compose Ollama service has its own model volume and does not automatically pull models or build an index. Build the index with the direct Python setup first, then stop the host Ollama service before starting Compose because both use port 11434. Pull the required models into the Compose Ollama container:
docker compose -f docker/docker-compose.yml exec ollama ollama pull qwen2.5:14b
docker compose -f docker/docker-compose.yml exec ollama ollama pull nomic-embed-text
The API container reads the generated index from the mounted project data directory.
GET /health — process healthGET /ready — reports whether an index and model clients are configuredPOST /query — accepts {"question": "..."} and returns an answer, sources, and available metricsGET /source?path=... — serves a source file only when it belongs to the loaded indexThe knowledge agent does not provide a separate document-summarization workflow. Open WebUI is included as a container; a project-specific RAG plugin or configured integration is not implemented.
.txt and .md)The document loader can discover PDF, DOCX, and HTML files, but the index-building pipeline currently skips those formats.
src/eval/rag_eval.py is a placeholder. RAGAS, LlamaIndex evaluation, DeepEval, and quality metrics such as faithfulness or answer correctness are not currently wired up.
Dockerfiles and a Compose definition are provided for Ollama, the API, and Open WebUI. This is a local-development setup, not a production deployment: authentication, a production CORS policy, and a custom WebUI integration are not configured.
The API can return retrieval/generation timing and Ollama token metrics with query responses. A Prometheus/Grafana stack, GPU/DCGM monitoring, and persistent metrics collection are not implemented.
data/raw/, data/processed/, and data/indexed/..gitignore excludes future local raw/processed data, generated index files under data/indexes/, GGUF model files, and .env secrets; it does not remove sample files already tracked in Git.GNU General Public License v3.0. See LICENSE.