ReportAI is a local-first BI client that runs its LLM inference entirely on-device through Ollama on an RTX 4050, instead of calling a cloud API. Two decisions made that viable: keeping sensitive business data off third-party servers entirely, and replacing one broad "act as a business analyst" system prompt with 10+ narrow, single-purpose skill files — one for RFM segmentation, one for pricing waterfalls, one for churn risk — each with its own scoped instructions instead of asking one prompt to be good at everything.
Business intelligence tools built on cloud LLM APIs have a data problem that doesn't show up in the demo: every query you send — customer lists, margin structures, supplier terms — leaves the building. For an Indian MSME, that's not a compliance checkbox, it's the actual competitive data of the business, and "trust us, it's encrypted in transit" is a hard sell to an owner who's never used an API in their life.
So ReportAI runs the model locally. Ollama on an RTX 4050 is enough to run useful analytical inference on-device, which means the data never has to leave the machine it's already sitting on. No API bill per query either — the cost is a one-time GPU, not a recurring per-token charge that scales with how much the business actually uses its own data.
The prompt that didn't scale
The first version of ReportAI had one system prompt: something like "you are an expert business analyst, help the user understand their data." It handled simple questions fine and quietly produced wrong numbers on anything structured — asking it for customer lifetime value segments and getting back plausible-sounding categories that didn't actually match any consistent RFM methodology, because the model was pattern-matching "sounds like a segmentation answer" rather than executing a defined calculation.
A generalist prompt guesses at every task. A narrow skill file only has to be right about one.
The fix wasn't a bigger model — it was smaller, scoped prompts. ReportAI now ships 10+ .skill files, each one owning exactly one analytical framework: customer segmentation on defined RFM/CLV rules, pricing margin waterfalls, supply chain analytics, risk scoring, competitor benchmarking, structured SOP generation. Each skill file carries its own methodology and output format, so the model isn't reasoning about "what kind of answer does this sound like" — it's executing one defined task with the actual formula spelled out.
A model asked to be a generalist will always sound confident. A model given one scoped task and the actual methodology to follow will be right.
The actual lesson
"Local-first" sounds like a privacy feature, but the bigger win turned out to be an accuracy one. Narrowing the prompt surface — one skill file, one job, one defined output — did more for correctness than any amount of retrieval or context stuffing would have on a single do-everything prompt. Privacy was the reason to go local; the skill-file architecture is the reason the answers are actually trustworthy once you're there.
It's the same principle as the tool-calling constraint in Hostwise: don't make the model smarter, make it structurally harder for it to be wrong.
Sitting on data you don't want leaving your building?
If your BI needs are blocked by "we can't send this to the cloud," a local-first architecture might be the actual unlock — not a compliance workaround, just the right default.
Talk to me about local-first BI arrow_forward