Yeah LLMs were the easiest way to get a proof of context running, but replacing it with a specialized distilled model/classifier should hopefully make it way quicker.

As for the results, it's tough because we've made the deliberate decision to have no control over the reranking. What that means is that if your criteria is "written by a woman", for instance, then any result that meets that will be ranked equally at the top. In all engines I've built for myself, I have a relevance criteria that's weighted relative to how much I care that the result is exactly what I'm looking for. It's probably important to make that clearer to the end user.

Credit to @ziftface — I should’ve included more examples in the original post. MatterRank is useful when you want results with specific qualitative traits that go beyond keyword matching. You can ask for stuff like “written by a woman,” “mentions these specific lines from a movie,” or “talks about X/Y/Z but avoids A/B.” Since it reads the full content, not just metadata or SEO signals, it lets you be a lot more precise in ways that traditional search engines just don’t support.

I think in a lot of cases that's because the meta with LLMs right now is to "have them do things for you", which generally means that obfuscating what's actually happening behind the scenes can make them seem "smarter" to the average user. Also, engineers are used to full control over deterministic input-output pipelines, which is a framework they try to force on LLM applications that fails miserably for the reasons you've listed.

In my opinion, the best applications of LLM UX will have full clarity for the end user (something we're trying to do with MatterRank). The non-determinism should be something the user can control to get better results, not something the engineer has prompted that takes control away from the user.

Now, if the use case you're looking for is "give me results with x text", then yes I agree with you that LLMs are just getting in the way. But that's not always the case.

Yeah that's fair, "doesn't know what we want" might have been oversimplifying. Better phrasing would have been that there is a very hard limit on the context you're able to give when using a search engine. It's mainly keywords, and then maybe some tricks like `site:` or quotes.

You don't have to create an account