AI Investing Machine: Part II

Claude Paugh
4 days ago
4 min read

After thinking that graph analytics of funds and instruments in a knowledge graph could provide some value to risk use cases, and potentially longer term ML potential, I moved on to what else goes with this? My initial thought was not necessarily to use the tools available from Neo4J, but to have a UX that could feature more.

I decided why not try to wrap an LLM chat bot with a grid style display, as well as graph navigation, and perhaps time-delayed and/or real-time market data. So it turned quickly into a larger suite of functions.

So I summarized my use cases:

grid style availability of graph data with filter/sort columns
LLM chat bot for user interactions
Faster user analytics
Graph display and drill-through
Recent history and active market data
Machine Learning for chart pattern discover (known and unknown)
Machine Learning market analysis
Integration of Interactive Brokers API functionality (basic then advanced)

I had previously used a series of Python scripts to fetch SEC Filings for the graph, and market data from FMP, so I had a little start of some of this. But some of the rest I was thin on expertise, so I decided to do some re-search using both ChatGPT and Claude to try to fill some gaps. I also engaged the Junie code assistance in Pycharm for some help. But I did have a few things I wanted to leverage:

1. Couchbase warehouse document store of SEC Filings

2. Postgres store of symbol, pricing, and financial statements

3. Interactive brokers API (I had done some preliminary work with API)

4. FMP subscription for market data API in addition to what I have with #3

So after determining what functional pieces I needed/wanted, and some other ideas on how to engage users, processing architecture, and persistence strategy, I came up with the below:

Summary highlights:

Browser front end with chatbot using charts, graphs, and visualizations
LLM and Orchestration layer, augmenting the LLM capabilities with a state machine for asynchronous tracking. ML Classification for decisions that are more complex.
MCP Toolbox and custom Query agent (using Python multi-processing) for dedicated data access layer
Agent processors that asynchronously perform data synchronization, aggregation, ingestion, asset pricing technical analysis, and an agent governor that monitors agent activity and ensures they are running and functioning
Data storage layer which include Couchbase as a document store, Neo4J for relationship graphs, PostgreSQL for specific financial data, and Redis for caching and event launching

I recognized that these were ambitious plans, but I decided to start with something functionally 'basic', which was putting a chatbot UI, data grid, and graph rending together to get data from the SEC NPORT filings and LEI. Its far from technology basic, but I did a deep dive and researched to educate myself on aspects that I needed more knowledge on. The below activity diagram details the design of the chatbot, LLM, and supporting or intended components. It includes some additional infrastructure to ensure certain scaling capabilities exist.

Chat Gateway ('chat_gateway/')

The entry point of the system. It handles state management, session context, and the primary orchestration loop.

Two-Stage Orchestration:

Analysis Turn: Asks the LLM to decide which tool to use and what filters to apply
Execution/Summarization Turn: Once data is retrieved, it sends the raw data back to the LLM for a natural language summary (RAG)

Conversational Logic: Enforces a "Talk-First" rule if identifiers are missing, prompting the user for clarification before executing heavy queries.

ML Classifier & Identifiers ('classifier.py')

To reduce LLM hallucinations and latency, the system uses a Scikit-Learn Random Forest classifier, which functions as:

Pre-Processing: It extracts canonical identifiers (ISIN, CIK, CUSIP, LEI) using regex before the LLM sees the request
Routing: The classifier predicts the most likely 'tool' based on the presence of these identifiers and specific financial keywords, e.g. holdings, profile, notional, etc.

Agent-to-Agent (A2A) Protocol ('util/adk/')

The system uses a standardized messaging envelope ('A2AMessage') for all internal communications.

Asynchronous Task Hub: Redis acts as the central broker. The gateway pushes tasks to agent inboxes, and agents push results back.
Multi-Process Workers: The 'query agent' offloads heavy database operations to separate worker processes. This prevents the FASTAPI event loop from blocking while serializing large Neo4j or Couchbase results

Specialized LLM Prompts

The system uses dynamic prompt templates tailored for different user intents.

System Orchestrator: Defines tools, schema rules, and JSON outputs
RAG Summarization: Instructs the LLM to synthesize data into human-readable insights without repeating the raw data

Key Design Patterns

Defensive Parsing: Validation of all LLM-generated decisions against a pydantic model (MCPDecision) before execution
Fingerprinted Caching: Query results are cached in Redis using an MD5 hash of the tool name and filters used, ensuring sub-second response times for repeated questions
Content Distillation: To prevent token overflow, the system strips large data grids from the conversation history, keeping on the distilled insights for subsequent turns