AIHOT

### AIHOT: Automated Domain-Specific News Aggregation Framework AIHOT is an open-source framework designed to automate industry-specific news gathering, scoring, event clustering, and daily report generation. Instead of relying on manual content curation or basic RSS aggregators, AIHOT builds a domain-tailored hot-topic portal driven by custom prompts and domain expertise. --- ### Pipeline & Architectural Overview The core architecture operates as a six-stage sequential processing engine: 1. Ingestion & Deduplication: Ingests raw content feeds across various source formats—including RSS feeds, webpage lists, JSON APIs, and X accounts—and immediately drops duplicate items. 2. Pre-screening & Dual Scoring: Raw items pass through an initial heuristic filter before entering a dual-pass evaluation step. Two independent scoring rounds evaluate the item's relevance against quality thresholds defined in `industry/prompts/`. Only content exceeding the threshold enters the curated pipeline. 3. Localization & Summarization: The LLM generates concise Chinese titles and summaries for qualifying content. 4. Vector + LLM Hybrid Clustering: To avoid presenting multiple reports covering the exact same news item, AIHOT clusters related articles into unified Events: * Uses vector embeddings of titles and summaries to query candidates across a 14-day sliding window. * Passes candidates to a primary LLM to determine if the item is a direct duplicate, a progress update, or a separate story. * Routes uncertain edge cases through a secondary model for cross-verification. 5. Event Heat Calculation: Heat scores belong to aggregated events, not individual articles. The scoring algorithm runs on a 48-hour rolling window with a 24-hour half-life decay. To prevent media manipulation, each independent source counts only once per event—publishing ten articles on the same topic yields a single source vote. 6. Publication: Top-ranked clusters compile directly into the daily brief. --- ### Core Stack & Performance Metrics The stack is lightweight and optimized for low-latency delivery: * Runtime: Node.js 24 * Database: PostgreSQL 17 * Containerization: Docker Compose Live production benchmarks demonstrate tight response times across the board: * Page Load Latency: 10 ms median (P95 < 50 ms) * API Endpoints: 6 ms median (P95 < 12 ms) * Article Pages: P95 < 14 ms --- ### Setup & Customization The system runs via Docker Compose with Node.js 24 and PostgreSQL 17. Adapting the engine for a new domain (e.g., legal, HR, finance) requires zero core code changes. You configure domain logic by updating the selection thresholds and system instructions inside `industry/prompts/`, alongside swapping the default content sources. The repository includes 18 sample overseas AI feeds to test the end-to-end pipeline immediately upon deployment.