Skip to main content
Fëanor's Code
Back to journal

Deep dive · 9 min read

System Overview

Fëanor's Code

June 28, 2026

EN

Table of Contents

  • Executive Summary

  • Architecture Diagram

  • Architecture Principles

  • System Modes

  • Service Inventory

  • SignalR Hubs

  • Communication Patterns

  • Data Stores

Executive Summary

TENGWAR is an on-premises enterprise RAG (Retrieval-Augmented Generation) platform designed for single GPU node deployment. It combines a .NET 10 backend with Python microservices, five vLLM model servers (chat, RAG-helper auxiliary, embedding, reranker, GLM-OCR), a background memory worker, and three client surfaces -- all orchestrated through Docker Compose as 18 Docker services (17 long-running + 1 init).

The platform provides:

  • Three client surfaces: Windows desktop (MAUI Blazor Hybrid), macOS desktop (Mac Catalyst), and a web fallback (Blazor Server).

  • 17-phase document ingestion pipeline (schema 2.0): Docling parsing, GLM-OCR vision, chunking, embedding, and vectorization coordinated via RabbitMQ.

  • Hybrid retrieval: Dense + BM25 sparse + visual search with RRF fusion, cross-encoder reranking, and context budget allocation.

  • Agent-driven chat: Microsoft Agent Framework with autonomous tool invocation, streaming responses via SignalR, and a middleware chain for security, audit, and citation capture.

  • Security-aware retrieval: RBAC, security clearance, and department-based document filtering propagated into the retrieval layer.

  • Mode-based orchestration: System modes (Online, DocumentProcessing, Transitioning, Updating) gate request handling at the HTTP level without stopping containers.

All 18 Compose services share a single Docker bridge network (app-network) except for the host-network UDP discovery sidecar. GPU-backed model services share a single NVIDIA GPU via vLLM's --gpu-memory-utilization 0.01 + --kv-cache-memory-bytes configuration.

Architecture Diagram

flowchart TB
    desktop[Windows / macOS desktop<br/>MAUI Blazor Hybrid] -->|HTTPS + SignalR| api[Backend API<br/>.NET 10<br/>Ports 5000 / 5001]
    web[Web fallback<br/>Blazor Server frontend] -->|HTTPS + SignalR| api

    api --> pg[(PostgreSQL<br/>metadata, auth, state)]
    api --> mq[(RabbitMQ<br/>async jobs)]
    api --> minio[(MinIO<br/>object storage)]
    api --> qdrant[(Qdrant<br/>vector DB)]
    api --> searx[SearXNG<br/>web search]

    mq --> ingester[Ingester<br/>17-phase Docling pipeline (schema 2.0)<br/>Port 8108]
    ingester --> tokenizer[Tokenizer<br/>BM25 sparse vectors<br/>Port 8104]
    ingester --> translator[Translator<br/>M2M-100 language service<br/>Port 8105]
    ingester --> embedding[Embedding<br/>BGE-M3<br/>Port 8101]
    ingester --> ocr[GLM-OCR<br/>vision OCR<br/>Port 8103]

    api --> chat[Chat vLLM<br/>Qwen3.6 35B-A3B FP8<br/>Port 8100]
    api --> raghelper[RAG-helper vLLM<br/>Qwen3.5-4B FP8<br/>Port 8000 internal / 8106 dev]
    api --> reranker[Reranker<br/>BGE-Reranker-v2-m3<br/>Port 8102]
    api --> memory[Memory worker<br/>per-user extraction]
    mq --> memory
    memory --> pg
    memory --> qdrant
    minio --> init[minio-init<br/>one-shot bucket setup]

Architecture Principles

1. Contracts-First Boundaries

The Contracts/ project is the shared schema source of truth. All DTOs, requests, responses, configuration interfaces, enums, and authorization constants live there (~390 files). Any schema change must be synchronized across all consumers -- backend, desktop, and web.

2. Layered Backend Design

The backend follows clean architecture: Api (controllers, hubs, background services, middleware) depends on Application (interfaces, service contracts), which depends on Domain (entities, enums, value objects). Infrastructure (implementations, EF Core, agent tools) depends on Application and Domain but never on Api.

3. Asynchronous Ingestion

Document processing is fully asynchronous. Uploads are queued via RabbitMQ, processed by the ingester through a 17-phase pipeline (schema 2.0), and results flow back through RabbitMQ progress messages. The backend broadcasts progress in real time via SignalR.

4. Security-Aware Retrieval

Every retrieval query is filtered by the requesting user's RBAC role, security clearance level, and department assignment. The security context is extracted from JWT claims by middleware and propagated into the retrieval pipeline.

5. Operational Resilience

All services include Docker health checks with configurable intervals and start periods. The backend runs health monitoring, auto-recovery, heartbeat timeout detection, and document state reconciliation as background services. Mode transitions use a semaphore and SignalR broadcast pattern for safe coordination. Failure scenarios and operator procedures live in the canonical runbook: `15-operations-runbook.md` §Failure Scenarios.

6. Cross-Platform Parity

All UI components, pages, services, and state management live in a shared Blazor RCL (Tengwar.Ui). Desktop-specific services (mDNS discovery, folder sync, TLS trust, secure storage) live in Tengwar.Desktop. Each host project (Windows, macOS, Web) is a thin shell that composes these shared layers.

System Modes

The system uses mode-based request blocking -- all containers run continuously, and the backend gates chat requests at the HTTP level based on the current mode.

| Mode | Behavior | |------|----------| | Online | Normal operation. Chat requests are served. | | DocumentProcessing | Ingestion is active. Chat requests are blocked with HTTP 409 Conflict. | | Transitioning | Mode change in progress. All requests are held until the mode settles. | | Updating | System update in progress (image pulls, container restarts). Requests blocked. See `13-update-system.md`. |

Transition Flow

  1. Acquire semaphore -- prevents concurrent mode transitions.

  2. Set `Transitioning` -- database flag updated.

  3. SignalR broadcast -- SystemHub notifies all connected clients of the pending transition.

  4. Drain in-flight requests -- ChatRequestRegistry waits for active chat requests to complete.

  5. Set target mode -- database flag updated to the final mode (Online or DocumentProcessing).

  6. SignalR broadcast -- clients receive the final mode.

Auto-Transitions

  • Upload triggers DocumentProcessing: When documents are uploaded, the system transitions to DocumentProcessing mode automatically.

  • Batch completion triggers Online: BatchCompletionBackgroundService polls every 15 seconds. When all documents in a batch are processed, it transitions back to Online. DocumentProgressBackgroundService also monitors RabbitMQ events for faster detection.

Service Inventory

18 Docker services: 17 long-running + 1 init container (minio-init).

| Service | Image / Build | Dev Port (host:container) | Role | Health Check | |---------|---------------|----------------------|------|-------------| | rabbitmq | rabbitmq:4.3.1-management-alpine | 5672:5672, 15672:15672 | Message broker for async document processing | rabbitmq-diagnostics -q ping | | qdrant | qdrant/qdrant:v1.18.2 | 6333:6333, 6334:6334 | Vector database for dense, sparse, and visual embeddings | TCP check on port 6333 | | ingester | Build: infra/docker/Dockerfile.Ingester | 8108:8000 | 17-phase document processing pipeline (schema 2.0; Docling + GLM-OCR) | HTTP GET /health | | tokenizer | Build: infra/docker/Dockerfile.Tokenizer | 8104:8000 | BM25 sparse vector tokenization for hybrid search | HTTP GET /health | | translator | Build: infra/docker/Dockerfile.Translator | 8105:8001 | Language detection and translation (M2M-100 1.2B) | HTTP GET /health | | postgres | postgres:17.10-alpine | 5432:5432 | Relational database for metadata, auth, state, and settings | pg_isready | | minio | quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z | Variable (env) | Object storage for documents, processed artifacts, generated files | HTTP /minio/health/live | | minio-init | quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z | None | One-shot init: creates MinIO buckets at first start | None (exits after completion) | | chat | Build: infra/docker/Dockerfile.Qwen36Chat | 8100:8000 | vLLM chat LLM (Qwen/Qwen3.6-35B-A3B-FP8) | HTTP GET /health | | rag-helper | Build: infra/docker/Dockerfile.RagHelper | 8106:8000 (dev) | vLLM auxiliary LLM (Qwen3.5-4B FP8) for HyDE, Self-RAG reflection, and query expansion -- offloads small-model calls from the main chat container. See `rag-helper-auxiliary-generation.md`. | HTTP GET /health | | embedding | Build: infra/docker/Dockerfile.BgeEmbedding | 8101:8000 | vLLM BGE-M3 embedding model (1024 dimensions) | HTTP GET /health | | reranker | Build: infra/docker/Dockerfile.BgeReranker | 8102:8000 | vLLM BGE-Reranker-v2-m3 cross-encoder reranker | HTTP GET /health | | glm-ocr | Build: infra/docker/Dockerfile.GLMOCR | 8103:8000 | vLLM vision OCR model (GLM-OCR 0.9B) | HTTP GET /health | | backend | Build: infra/docker/Dockerfile.Backend | 5000:5000, 5001:5001 | .NET 10 API: orchestration, auth, REST, SignalR, agent framework | ASP.NET health endpoints | | frontend | Build: infra/docker/Dockerfile.Frontend | 8080:8080, 8443:8443 | Blazor Server web fallback (hosts shared Tengwar.Ui RCL) | HTTP GET /_framework/blazor.server.js | | memory-worker | Build: infra/docker/Dockerfile.MemoryWorker | None | Background per-user memory extraction consumer | HTTP GET /health | | discovery | Build: infra/docker/Dockerfile.Discovery | 40123/udp (host network) | LAN broadcast discovery beacon for desktop clients | None (UDP responder) | | searxng | searxng/searxng:latest | None (internal only) | Self-hosted metasearch engine for agent web search tool | HTTP GET / |

Startup Order

Services start in dependency order enforced by Docker Compose depends_on with condition: service_healthy:

  1. Data stores: postgres, qdrant, rabbitmq, minio (independent, start in parallel)

  2. Init: minio-init (depends on minio)

  3. Python services: tokenizer, translator (independent)

  4. AI models: chat first (GPU memory reservation), then rag-helper, embedding, reranker (all depend on chat)

  5. AI models (parallel): glm-ocr (independent of other models)

  6. Ingester: depends on rabbitmq, minio, tokenizer, translator, qdrant, embedding, glm-ocr

  7. Memory worker: depends on postgres, rabbitmq, qdrant, minio, chat, and embedding

  8. Backend: depends on all API-facing dependencies

  9. Frontend: depends on backend

  10. SearXNG and discovery: independent sidecars

SignalR Hubs

Five SignalR hubs provide real-time communication between the backend and connected clients. All hubs require JWT authentication and are mapped under /hubs/.

| Hub | Path | Purpose | |-----|------|---------| | StatusHub | /hubs/status | System status, processing progress, health updates, statistics, mode changes, and syncing-knowledge broadcasts. Buffer: 16 KB. | | ChatHub | /hubs/chat | Streaming chat tokens to user-specific groups. Each user receives tokens in their own group (chat:{userId}). Buffer: 64 KB. | | DocumentHub | /hubs/document | Document processing progress updates and group management. Clients join/leave document-specific groups (doc-progress:{batchId}). Buffer: 16 KB. | | SystemHub | /hubs/system | Real-time system metrics (CPU, GPU, memory, disk), health alerts, and admin-only service status updates. Supports subscription-based groups: SystemMetrics, HealthAlerts, AdminUpdates. Buffer: 32 KB. | | MemoryHub | /hubs/memory | Per-user memory change notifications (memory subsystem). |

SignalR Configuration

  • Maximum receive message size: 1 MB (default is 32 KB)

  • Client timeout interval: 5 minutes (for bulk import scenarios with 10000+ files)

  • Keep-alive interval: 30 seconds

  • Handshake timeout: 30 seconds

  • User identity resolved from JWT NameIdentifier claim via NameUserIdProvider

Communication Patterns

REST (Client to Backend)

All client surfaces communicate with the backend via REST over HTTPS. The API supports versioned (api/v{version}/...) and unversioned (api/...) route prefixes. Responses follow the ApiResponse hierarchy defined in Contracts/.

SignalR (Real-Time Push)

The backend pushes real-time updates to connected clients via five SignalR hubs. Use cases include streaming chat tokens, document processing progress, system mode changes, health alerts, and metrics. SignalR connections authenticate via JWT tokens passed as query parameters.

RabbitMQ (Async Document Processing)

Document processing uses a producer-consumer pattern over RabbitMQ:

  • Backend publishes processing requests when documents are uploaded.

  • Ingester consumes requests and processes documents through the 17-phase pipeline (schema 2.0).

  • Ingester publishes progress messages back to RabbitMQ.

  • Backend consumes progress messages via DocumentProgressBackgroundService and broadcasts via SignalR.

Additional queues handle cancellation, heartbeat, and dead-letter scenarios.

HTTP (Backend to AI Services)

The backend communicates with AI services (chat, rag-helper, embedding, reranker, tokenizer, translator, SearXNG) via HTTP over the Docker bridge network. All outbound HTTP calls propagate the X-Correlation-Id header via CorrelationIdDelegatingHandler.

Data Stores

PostgreSQL (Relational)

  • Image: postgres:17.10-alpine

  • Purpose: Document metadata, user accounts, authentication state, system settings, mode flags, chat session history, approval workflows, sync state, agent audit logs.

  • Performance tuning: shared_buffers=256MB, effective_cache_size=512MB, work_mem=16MB, max_connections=25.

  • Access: Backend connects via EF Core with connection string from environment variables.

Qdrant (Vector)

  • Image: qdrant/qdrant:v1.18.2

  • Purpose: Dense embeddings (BGE-M3, 1024 dimensions, L2-normalized), BM25 sparse vectors, and optional visual embeddings for hybrid search.

  • Access: Backend uses QdrantVectorService via the Qdrant REST/gRPC API (ports 6333/6334).

MinIO (Object Storage)

  • Image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z

  • Purpose: Three buckets -- documents (originals), processed (artifacts/thumbnails), and generated documents (Word/Excel/PDF output from chat). The minio-init container creates these buckets on first start.

  • Access: Backend uses presigned URLs for client downloads. Ingester accesses MinIO directly for document retrieval during processing.

RabbitMQ (Message Queues)

  • Image: rabbitmq:4.3.1-management-alpine

  • Purpose: Async coordination between backend and ingester. Queues include document processing requests, progress updates, cancellation signals, heartbeats, and dead-letter handling.

  • Management UI: Available on port 15672 for queue inspection and diagnostics.

Last verified against commit `2a1034c` (2026-05-17).

Fëanor's Code

Engineering studio behind on-premises AI platforms and custom enterprise software.

System Overview