Architecture

System architecture

The research pipeline, the dual TypeScript + Python ecosystems, and the Docker topology on one VPS.

Clawlas runs as a research pipeline: market data flows through collectors and feature engineering into the quant engine and the multi-agent research layer, gets scored, passes human governance, and only then can reach execution — every step journaled and remembered. It is built on two ecosystems and fully self-hosted on one VPS.

The research pipeline

External data sourcesExchanges, DEXes, chains, social
Collectors & WebSocket workersOne independent collector per source
Raw event streamRedis Streams · Parquet archive · Convex records
Normalization & feature engineeringCommon event schema → features
Quantitative & statistical engineSignal · risk · execution scores · regime
Agentic research layerAI & memory scores · strategy · debate
Decision engineIgnore · watch · simulate · review · buy · reduce · sell
Execution engineIsolated · deterministic validation
Trade journal & memoryFrozen evidence → SQLite + Convex
Continuous evaluationFeeds the next cycle

From the whole stream to one decision

The operating principle is a funnel: deterministic systems detect and control, reducing the full data stream to a few candidates before any model call; AI interprets and assists on that small set; and the risk engine decides whether execution is permitted. No layer bypasses the one below it, and LLMs never process every tick.

Raw market, social & on-chain data

Everything, continuously

Deterministic collection & normalization

Identity, quality, dedup

Rules, indicators & anomaly detection

Produces a candidate event

Cheap AI classification

Only when a candidate warrants it

Deep analysis

A small number of candidates

Risk & execution-cost validation

Deterministic gate

Approval / controlled execution → reconcile & learn

One decision

The infrastructure

The TypeScript web layer and the Python research/execution core run as separate containers behind Traefik, sharing Convex (operational truth), SQLite (agent memory), Redis (events) and a Parquet/DuckDB lake (analytics).

┌──────────────────────────────────────────────────────────────────┐
│                          VPS · Docker                            │
│                  Traefik (proxy + TLS, shared)                   │
│                                                                  │
│  ┌────────────┐   ┌──────────────────┐   ┌────────────────────┐  │
│  │  Web (UI)  │   │ Self-host Convex │   │  Python core       │  │
│  │ TanStack   │◄─►│  operational     │◄─►│  FastAPI           │  │
│  │ Start/React│   │  state + journal │   │  ┌──────────────┐  │  │
│  └────────────┘   └──────────────────┘   │  │ LangGraph    │  │  │
│        ▲                  ▲               │  │ agents       │  │  │
│        │                  │               │  ├──────────────┤  │  │
│        │                  │               │  │ quant engine │  │  │
│  ┌─────┴───────┐   ┌──────┴───────┐       │  ├──────────────┤  │  │
│  │ Redis       │   │ SQLite memory│◄──────┤  │ execution    │  │  │
│  │ Streams     │   │ (volume)     │       │  │ (isolated)   │  │  │
│  └─────────────┘   └──────────────┘       │  └──────┬───────┘  │  │
│  ┌─────────────────────────────┐          │         │          │  │
│  │ Parquet lake + DuckDB        │◄─────────┘         │          │  │
│  └─────────────────────────────┘   collectors        │          │  │
│  ┌─────────────────────────────┐                     │ read-only │  │
│  │ Prometheus · Grafana · Loki  │                     ▼          │  │
│  └─────────────────────────────┘        Exchanges + data feeds   │  │
└──────────────────────────────────────────────────────────────────┘

One reproducible Docker topology behind Traefik: the web UI, self-hosted Convex, the Python core (FastAPI + LangGraph agents + quant + isolated execution), SQLite memory, Redis Streams, and the Parquet/DuckDB lake.

Layer → tech mapping

Layer	Tech	Hosting
Web (dashboard, journal, approvals)	TanStack Start + React + Tailwind	VPS container
Operational backend	Self-hosted Convex	VPS container
API & services	Python + FastAPI + Pydantic	VPS container
Agents	LangGraph + provider-agnostic LLM adapter	Python workers
Quant engine	NumPy / pandas / SciPy / statsmodels / scikit-learn	Python workers
Execution (isolated)	CCXT + direct adapters + web3.py	Execution container
Agent memory	SQLite (+ FTS5 / sqlite-vec)	Docker volume
Analytics	Parquet + DuckDB	Docker volume
Events	Redis Streams + workers	VPS container
Proxy / TLS	Traefik (existing, shared)	VPS
Observability	Prometheus · Grafana · Loki	VPS containers

Each store has one job

Convex = operational state + real-time UI. Agent memory → Convex (vector search) — SQLite for agent memory is parked. Parquet/DuckDB = historical analytics. Redis = ephemeral event transport. The same raw data is never duplicated — see the data boundary and data-model pages.