I am trying to build this independent of APIs provided by specific providers (such as cache), so this isn't an option.

Ask HN: How to Build RAG?

I am building another 'ask PDF a question' RAG.

I have successfully converted PDF to markdown.

Then I used Jina segmenter to split it into chunks.

Each chunk is ~1000 characters long, but sometimes it is as short as just the title of the section.

I have then stored all of these chunks in a vector database and use cosine distance to sort chunks, pick the first 100, and include the associated chunks into LLM prompt that's used to answer user's question.

However...

I feel like I am missing a step.

The chunks that are returned by the query, while mostly relevant, they ...

* do not include the full recipe * include snippets of unrelated recipes

Is there a step am I missing?

Depending on your use case, there is also a HTML table https://glama.ai/model-prices and also an API that you can call to get more accurate cost https://glama.ai/blog/2024-09-25-chat-cost-calculator-api. The API uses the appropriate tokenization technique to accurately calculate the cost.

Show HN: JSON Price Table for Top Large Language Models