Immediately after switching the page, it will work with CSR.
Please reload your browser to see how it works.

Source:https://github.com/SoraKumo001/next-streaming

⬅️ Show HN: OpenLIT – Open-Source LLM Observability with OpenTelemetry

Eridrus 16 daysReload

It would be great for this to actually explain what sorts of metrics are being computed here beyond what you get for free by instrumenting the requests library.

From looking at the screenshots, it looks like it can monitor number of tokens, which seems useful, but I'm not clear why that needed a whole big project.

I feel like the stuff you actually want to monitor in prod for ML that you don't get from infra monitoring are things that are not trivial to drop in because you want a sense for how well the ML components are working, which is generally pretty application specific. Having a general framework for that seems useful, but not really what we have here, at least for the moment.

Also, it just seems a bit weird for this to have it's own UI. Part of the point of OTEL is so that you can send all your metrics to one place. Not totally possible all the time and turning metrics into dashboards takes time, but the point of OTEL seems to be to separate these concerns.

LLMtools 15 daysReload

Very cool. QQ: How does this differ from Langtrace (https://www.langtrace.ai) / (https://github.com/Scale3-Labs/langtrace)?

jackmpcollins 16 daysReload

Does the dashboard/UI support traces? I would love a tool in which to view opentelemetry traces, that can neatly display full prompt and response for the spans that represent LLM queries. I'm planning to add opentelemetry instrumentation to magentic [1] and looking for a UI that is easy to run locally that makes it easy to see what an agent is doing (via OTEL traces). I have more of my thoughts on the github issue: https://github.com/jackmpcollins/magentic/issues/136

[1] https://github.com/jackmpcollins/magentic

talboren 16 daysReload

How does this differ from Traceloop’s Openllmetry? https://github.com/traceloop/openllmetry

asabla 16 daysReload

So, do I understand this right. But is this supposed to be a centralized (in a way) application for collecting all OpenTelemtry data related to LLM's? And it's supposed to support various of related services and/or services today?

I see how this can be really useful. But at the same time, how do you guys see your self differentiate your self from cloud hosted equivalents? (e.g dashboard-like services in Azure and similar).

Anyhow, interesting project. I'll keep an eye on it for future use