MAX3 · TECHNOLOGY EVALUATION

Hardware & Software Tech-Stack Evaluation

Specifications, alternatives (OSS / free / paid), and recommendations

2026-06-14 RTX 3090 Ti · 64 GB · local-first

00

Headline

The stack is well-chosen. Keep ~90 %. Three changes are worth piloting (TTS naturalness, embeddings quality, object detector), one upgrade is worth planning (vector store), and a handful of tempting-but-wrong options are explicitly rejected because they break a hard constraint. Nothing here argues for a rewrite, and no new hardware is needed for the design as specified.

01

Hardware specifications (the box this runs on)

THE FOUR BINDING CONSTRAINTS (PRE-FILTER EVERY OPTION)

■ Local-first. Routine use works with the network down; cloud is an upgrade path, not a default.

■ Open-source dependencies. MIT/BSD/Apache preferred; GPL/AGPL avoided; proprietary/telephone-home not acceptable — the one exception is cloud LLM APIs as a paid fallback.

■ Single core process, no microservices. The voice-turn latency budget (1500–2000 ms) will not survive an IPC hop per faculty.

■ No heavyweight agent frameworks. No LangChain / LangGraph / CrewAI / AutoGen; a custom thin orchestration layer we own.

02

Software scoreboard — the whole stack at a glance

Legend: Keep · Pilot a change · Plan an upgrade · Reject (tempting-but-breaks-a-rule).

03

LLM serving — the most-scrutinised choice

Verdict: keep Ollama. For a single-user, local-first resident, throughput engines optimise a dimension we don't have (concurrency) at the cost of dimensions we do (simplicity, single-process, VRAM headroom for Whisper + ArcFace + TTS to coexist). Drop to llama.cpp-direct only if you want speculative decoding.

Which models, given 24 GB

04

The three pilots worth doing

TTS — pilot Kokoro-82M (Apache)

Piper is fast, CPU-friendly, MIT, and tunable on six axes — all still valuable — but its ceiling is 'clear and robotic'. Kokoro-82M (Apache) punches far above its size, sounds dramatically more natural, and is small enough to stream on the 3090 Ti. Keep Piper as the default + low-latency path and the six-axis DSP persona chain; add Kokoro behind a setting and A/B the persona. Do NOT clone a voice or adopt cloud TTS — both are out of bounds.

Vision — pilot RT-DETR (Apache)

DETR was chosen to avoid Ultralytics YOLO's AGPL — correct and non-negotiable — but vanilla DETR is slow, which matters because vision shares the GPU with the voice critical path. RT-DETR is the same HF family and licence posture with real-time speed. Keep InsightFace for faces.

Embeddings — pilot bge-m3 / mxbai-embed-large

nomic-embed-text is solid and local, but retrieval quality is the lever under GraphRAG + curiosity + memory. bge-m3 (MIT, multilingual) and mxbai-embed-large (Apache, top English) are both Ollama-pullable; A/B against MAX's own recall set and switch only on a measurable win (cost: a background re-embed pass, scheduled off-peak).

05

Plan one consolidation — and what to reject

PLAN

■ Vector store — converge sqlite-vec → duckdb-vss when its WAL recovery is proven (two engines collapse into one). Hold Qdrant as the documented escape hatch only if embedded search becomes the measured bottleneck.

EXPLICITLY REJECTED (AND WHY)

■ vLLM / TGI / NIM — optimise multi-user concurrency we don't have, at the cost of the single-process VRAM headroom Whisper + ArcFace + TTS need to coexist.

■ Ultralytics YOLO — AGPL-3.0; forbidden by the OSS-deps rule.

■ XTTS / cloud TTS / cloud STT / cloud vector — send audio, frames, or embeddings off-box; break local-first.

■ LM Studio / NVIDIA NIM — closed-source / telephone-home posture.

■ Pinecone / Neo4j — cloud SaaS / licence + server shape that don't fit a single-box local-first resident.

06

Hardware — what would actually move the needle

Bottom line on hardware: hold. 24 GB is enough for the design as specified; the cheapest real latency win is a fast NVMe; spend on the cloud fallback, not a card, for the rare hard turn.

07

Recommendation summary

The through-line: this stack was chosen by the same constraints that should govern any change — local-first, open-source, single-process, auditable. The alternatives that beat it on a benchmark almost always lose on one of those; the ones that don't (Kokoro, RT-DETR, bge-m3) are model swaps inside the existing architecture, not replacements of it. The design is ambitious; the stack is already the right shape to carry it.