Semantic search
Semantic search finds messages by meaning, not only by matching words. A search for "when is the invoice due?" also finds "the payment for March is overdue". ByoTalk turns each message into an embedding (a list of numbers) with an embedding provider you choose, and stores it in your own PostgreSQL database with the pgvector extension.
What you get
chat.search(query)in your apps: results from the conversations the user is a member of, best match first.chatServer.search()on your server, for one user or one conversation.- New and edited messages are indexed automatically. Deleted messages are removed from the index.
- Existing messages are indexed when you turn search on.
- Any OpenAI-compatible embeddings endpoint: OpenAI, or a self-hosted Ollama or vLLM server.
Where your data goes
- Embeddings are stored only in your database, in the
message_embeddingstable next to your messages. ByoTalk does not keep them. - The embedding provider receives the text of every message in the environment (livestreams excepted) and every search query. Pick a provider you trust with that text, or host one yourself.
- Message text is never written to our logs or error messages.
- Your provider's API key is stored encrypted, and is shown masked after you save it.
Requirements
- Your own database is PostgreSQL (Bring your own database). MySQL, MongoDB and HTTP endpoints are not supported.
- The pgvector extension. Most managed PostgreSQL services include it: Amazon RDS and Aurora, Google Cloud SQL, Azure Database for PostgreSQL, Supabase and Neon.
- An embedding model with at most 2,000 dimensions. For example, OpenAI's
text-embedding-3-smallhas 1,536, and Ollama'snomic-embed-texthas 768.
Set it up
- Open Dashboard → Storage → Semantic search. Enter the embeddings URL, the model, its dimensions and, if the provider needs one, the API key. Save.
- Create the table. Use one of these three ways:
- Run the SQL the dashboard shows, as an admin user.
- Run the CLI on your machine:
npx byotalk db migrate --print-sql --engine postgres --search 1536prints it, andnpx -p byotalk -p pg byotalk db migrate --url "$ADMIN_DATABASE_URL" --search 1536runs it. - Re-run Create the tables with an admin URL. With search turned on, it creates the table too.
- Click Test. The test embeds a short text and checks the vector size, the table and the writer's permissions.
Messages are indexed once your own database is enabled. The dashboard shows the indexing status and the last error.
CREATE EXTENSION vector needs a superuser or your provider's admin role, for example rds_superuser on RDS or the postgres user on Supabase. The ByoTalk writer can read, insert, update and delete rows in message_embeddings only. It cannot delete anywhere else.
The settings can also be managed with a secret key:
curl -X PUT https://api.byotalk.com/v1/storage/search \
-H "Authorization: Bearer $BYOTALK_SECRET_KEY" -H "Content-Type: application/json" \
-d '{"embeddingUrl":"https://api.openai.com/v1/embeddings","model":"text-embedding-3-small","apiKey":"sk-...","dimensions":1536}'
Search from your app
const results = await chat.search("when is the invoice due?");
for (const { message, score } of results) {
console.log(score.toFixed(2), message.conversationId, message.text);
}
// One conversation, at most 5 results (the limit is 1 to 50, default 20)
await chat.search("invoice", { conversationId: conv.id, limit: 5 });
A user only gets results from conversations they are a member of right now. Removing someone from a conversation removes its messages from their results at once. Livestreams are not indexed.
Search from your server
import { ChatServer } from "byotalk/server";
const chatServer = new ChatServer({ secretKey: process.env.BYOTALK_SECRET_KEY! });
const { data } = await chatServer.search({ query: "refund", userId: "user_123" });
// or: { query: "refund", conversationId: "conv_01J9ZK..." }
Pass userId, conversationId, or both.
Errors and limits
| Situation | Result |
|---|---|
| Search is not set up | 400 search_disabled |
| The provider or your database cannot be reached | 503 search_unavailable (retry later) |
| Too many searches | 429 rate_limited: 30 per minute per user, 600 per minute per environment for secret keys |
Indexing never holds up replication. If the provider is down, messages still reach your database. Indexing retries with backoff and shows the error in the dashboard. A passing Test makes it retry at once.
Changing the model
If you change the URL, the model or the dimensions, ByoTalk deletes the stored embeddings and indexes every message again. Vectors from different models can't be compared. If the dimensions change, recreate the table with the new size first: drop message_embeddings and run the new SQL.
Tuning
pgvector's HNSW index looks at the 40 nearest rows first and then applies the membership filter. A user who belongs to a small share of a very large environment can get fewer results than the limit. On pgvector 0.8 or later, turn on iterative scans for the writer:
ALTER ROLE byotalk_writer SET hnsw.iterative_scan = relaxed_order;