Eduardo González

Choose a style

Same content in ten visual styles. Every one passes automated WCAG 2.2 AA checks.

Default
Quiet
Bold
Interface

Work SitimmVito search

Backend case · October 2026

Teaching an assistant to find the right answer.

SitimmVito is the assistant inside SITIMM's app. Over one weekend it went from a fixed folder of documents to searching everything the union publishes, inside the Postgres database the platform already had.

My role
Design, decisions and review. AI agents wrote the code.
When
One weekend, 24–25 October 2026
Stack
FastAPI, PostgreSQL full-text search, pgvector, Celery, Claude Haiku
Result
The right source for 13 of 16 test questions, up from 9

Screen

SitimmVito, the app's AI assistant, answering a question about member discounts.
SitimmVito in the app, from the launch video. The member is a test account.

It started with an outage

On a Saturday the assistant's AI provider account ran out of credit. SitimmVito could still say hello and answer exact FAQ matches. Nothing else.

The first fix took a few hours: another model, the cheapest one that did the job, with keyword search over a fixed folder of documents. It worked.

It also showed the real problem. The assistant only knew what was in that folder, and the union publishes every day.

The decision: no new infrastructure

I had three options: go back to a hosted vector store, add a dedicated vector database, or use what was already running. I chose Postgres. It has full-text search and, with pgvector, vector search. For a team of one, one less service to run matters more than a slightly better ranking.

The cost of that choice: ranking quality became my problem.

  1. Cut into passagesNews, events, agreements, magazines, FAQs, laws and video transcripts.
  2. Index twiceA Spanish keyword index that ignores accents, and a vector.
  3. RefreshEvery 15 minutes, with the public API's filters. An event under embargo can't leak through the chat.
  4. Merge by rankKeyword and vector results together. If the vectors fail, keywords still answer.

Five ways it was wrong

Search failures, the cause of each one and the fix
What I sawWhyThe fix
Common words beat the right answerAsked about the Christmas bonus, magazine pages full of “days” and “December” outranked the labour-law article that answers it. Postgres's ranking has no idea which words are rare.Each matching word scores by how rare it is in the index. The built-in rank only breaks ties.
That fix took 4 to 15 secondsPostgres inlined a subquery and recounted the index for every matching row.Two words: AS MATERIALIZED. 36 to 83 ms on a test database of 18,700 passages.
Filtered searches came back emptyA vector index takes its nearest matches first and filters afterwards. Asking only for agreements, 114 among about 7,000 passages, returned nothing.pgvector's iterative scan keeps reading until the filter is satisfied.
Old news beat current factsA dentist's offer from a 2016 magazine outranked the agreement in force.Current sources get a boost when results are merged, and the prompt treats old magazine pages as history.
It refused without lookingA safety rule, “only answer SITIMM questions”, made it turn away questions about allied unions whose answers were in the index.Search first, then decide what's off-topic. Unrelated requests and attempts to extract the prompt are still refused.

How I knew

Measures before and after
MeasureBeforeAfter
Test questions that found the expected source9 of 1613 of 16
Questions about allied organisations answered0 of 66 of 6
Questions end to end through the public API17 of 1818 of 18
Keyword query, test database4–15 s36–83 ms

Unit tests with fake models were green the whole time. One bug only showed up against the real API: on its last round the model wrote a tool call as plain text until it ran out of tokens. Only the evaluation against the real thing caught it.

What I'd do differently

  • Write the evaluation first. I tuned ranking by feel for a few hours before I had 16 questions to measure against.
  • Grow the evaluation. Sixteen questions catch the big failures, not slow drift. Three of them still miss.
  • Plan for size. Word rarity is counted on every query. That holds to around 100,000 passages; after that it needs a statistics table.

Want the details?

I'm happy to walk through the queries, the evaluation and the parts that went wrong.