Adding AI search and summaries to an existing SaaS product
AI that answers from the customer's own records, respects their permissions and can be tested before it ships.
A B2B SaaS product where customers store large volumes of documents, tickets or records and struggle to find what they need.
How does CodeWorx approach adding AI search and summaries to an existing SaaS product?
AI that answers from the customer's own records, respects their permissions and can be tested before it ships. The product calls an AI service that retrieves only what the user may see, sends it to the model with a controlled prompt and returns an answer with sources.
- Users find information in their own records faster
- Answers can be traced to the records they came from
- Changes are tested against known questions before they ship
- Cost per request is visible from day one
The starting point
Customers ask for AI features. The product team knows search and summaries would help, but not how to ship them safely.
Records belong to tenants and users with different permissions. An AI feature that ignores those permissions is a data leak.
Nobody can say in advance what a good answer looks like, so there is no way to know whether a change made it better or worse.
- Model output varies, so normal tests are not enough on their own.
- Costs grow with usage and are hard to predict.
- Customers in regulated sectors will ask where their data goes.
5 stages, each useful on its own.
- 01
Pick one job for AI
Start with a single, well-defined task, such as answering questions from a customer's own documents, rather than a general assistant.
- 02
Retrieve within permissions
Index content per tenant and filter every retrieval by the user's existing permissions before anything reaches the model.
- 03
Build an evaluation set first
Collect real questions with known good answers. Every prompt or model change is measured against them before release.
- 04
Show sources, allow review
Answers cite the records they came from. Where the output drives an action, a person confirms it first.
- 05
Measure cost and quality in production
Track cost per request, latency and user feedback, so the feature can be tuned rather than guessed at.
The product calls an AI service that retrieves only what the user may see, sends it to the model with a controlled prompt and returns an answer with sources.
- Per-tenant index with permission filtering at retrieval
- Model access through one service, so providers can change
- Evaluation set run on every change
- Logging of prompts, sources, cost and feedback
The calls that shape the outcome.
Why retrieval instead of fine-tuning?
Customer content changes daily and must respect permissions. Retrieval keeps the model current and the access rules intact.
Why one provider abstraction?
Models, prices and terms change quickly. The product should not be rebuilt each time.
What would change the plan
If customers cannot allow data to leave their region, model hosting and provider choice come first.
What this improves
- Users find information in their own records faster
- Answers can be traced to the records they came from
- Changes are tested against known questions before they ship
- Cost per request is visible from day one
Typical technology
- TypeScript
- Python
- PostgreSQL
- RAG
- Vector retrieval
- OpenAI
- Anthropic
Client projects using the same disciplines.
Facing a problem like this?
Tell us where your system is today. We will tell you which parts of this blueprint apply.