Architecture Overview

Architecture Overview#

Gnosly is engineered as an end-to-end intelligence pipeline that bridges distributed data collection with focused, noise-free analytical reasoning. The system isolates signal from web noise across four distinct architectural stages: Ingestion, Processing & Structuring, Storage & Indexing, and the Synthesis & Delivery Layer.

High-Level System Architecture#

Gnosly - Architecture Overview

1. Ingestion Layer#

The platform relies on two concurrent collection vectors to balance automated coverage with user-driven curation:

  • Automated Crawler Network: Background workers continuously crawl public news outlets, wire services, macroeconomic data releases, and official regulatory platforms. Crawlers strip tracking query parameters, extract raw body content, and normalize metadata timestamps to UTC.

  • Client-Side Browser Agent: A specialized browser extension allowing authenticated users to clip context directly from protected or dynamic browser sessions (including feeds like X, LinkedIn, webmail clients, and WhatsApp Web). Instead of downloading raw DOM noise, the extension captures precise text selections along with canonical URLs, authors, and source stamps.

2. Processing & Structuring Pipeline#

Once ingested, raw events pass through an automated parsing pipeline prior to persistent storage:

  • Normalization & Deduplication: URLs and message bodies are hashed to detect duplication across multiple syndication networks.

  • AI Parsing & Extraction: Core extraction models parse the raw event to generate concise, factual summaries, separating opinion or commentary from primary facts.

  • Classification & Tagging: Unstructured content is categorized across predefined verticals (such as Economics, Geopolitics, and Technology) and indexed with entity tags (tickers, institutional bodies, geographic references).

3. Storage & Indexing#

Data persistence is separated into structured entities and searchable indexes:

  • Relational Storage (PostgreSQL): Stores core user identities, thematic chat session metadata, workspace permissions, and full ingested event records.

  • Search & Metadata Index: Powers the real-time /feeds and Synthesized Search interfaces. Events are indexed by topic, entity tag, timestamp, and verification level to allow low-latency querying and filtering.

4. Analytical Workspaces & Thematic Chats#

The operational tier where structured data converts into intelligence briefs:

  • Workspace Sandboxing: Users isolate analysis inside Thematic Chats (e.g., energy logistics, policy tracking). Each chat maintains an isolated context window with its selected data streams.

  • Document Ingestion: Chats accept multi-format document uploads (PDFs, transcripts, CSV data, plain text) to augment crawled signals with private internal research.

  • Synthesis Engine: Integrated AI models query the combined context (stream signals + uploaded files) to answer specific prompts, identify cross-source contradictions, and assemble chronological briefs.

  • Public Stream Serialization: Chat operators can flip an analytical workspace to a public read-only state, serializing real-time analytical logs for guest access without granting write or query permissions.