Gosoft Connect - AI chat grounded in your documents

A multi-tenant widget SaaS whose AI chat answers from each customer's own knowledge base - the entire retrieval stack self-hosted on one CPU-only node.

What it is

Gosoft Connect is a multi-tenant SaaS I designed and built solo at GoSoft, my company: embeddable website widgets - AI chat, live chat, forms, popups - and an admin portal. Each customer workspace has a knowledge base filled four ways: paste text, fetch a URL, upload files, or crawl a whole sitemap. The AI chat on the customer's site answers visitor questions grounded in that knowledge base, streamed token by token, with tools for looking up company details, capturing leads and handing over to a human. Around the chat sit digest emails, AI summaries of form submissions, conversation categorization, nightly reports of questions the bot couldn't answer, and moderation on both input and output.

The retrieval pipeline

Answers are grounded by hybrid search: dense vectors and full-text ranking fused with reciprocal rank fusion, then reranked by a cross-encoder, with a per-document diversity cap and language-aware retrieval across Finnish, Swedish and English content. Chunking is paragraph-aware with contextual headers, and content hashing means unchanged documents are never re-embedded.

Stack & infrastructure

Deliberately small, again: everything runs on one Linux node, CPU only. The embedding model (bge-m3) and the reranker (bge-reranker-v2-m3) run locally via ONNX - no GPU, no embedding API. Customer documents leave the box for exactly one thing: the LLM call itself (OpenAI, model selectable per workspace). The vector store is plain PostgreSQL with pgvector - one database for everything. Bun and Hono on the backend, a Solid.js portal, nginx in front.

The engineering

The numbers I can publish are engineering numbers. Quantizing the reranker took retrieval from 14 seconds to 4.1 on CPU, the rerank step itself from 5.6 to 1.1. An early load test dropped 2 of 10 concurrent chats and killed the process at 10.4 GB of memory; after an inference semaphore and batch chunking, the same test served 10 of 10 at a 6.8 GB peak with zero restarts. In production since June 2026.