Install
Prompting, Guardrails & Eval
Prompt tools, safety filters, red-teaming, evals, and regression benchmarks.
- 6 Tracked terms
- Last 30 days Feed window
What this topic collects on
An article joins this feed when it matches these terms. Each one is also a search of its own.
Related topics
Latest in Prompting, Guardrails & Eval
178 reports in one afternoon: what a publish burst does to an LLM pipeline
5+ hour, 7+ min ago (314+ words) On a Thursday afternoon a traveller published 178 trip reports at once. Two years on the road, Canada... Tagged with ai, llm, webdev, buildinpublic....
Incident Context: tracing production failures without guessing
10+ hour, 9+ min ago (200+ words) This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content.... Tagged with devchallenge, sanitychallenge, sanity, ai....
CitrusFi: A subscription audit that actually understands billing cycles
10+ hour, 39+ min ago (23+ words) CitrusFi is a small subscription tracking app that answers a question most trackers get wrong: "how... Tagged with devchallenge, sanitychallenge, sanity, ai....
Your AI agent failed. The model might not be the problem.
13+ hour, 4+ min ago (477+ words) Nvidia explains why debugging AI agents means tracing decisions, not just logging errors — and backs a shared failure-reporting system called SAFE....
Unraveling the Complexity of AI Agents with MLflow: How LLM Workflow Observability Transforms Debugging, Evaluation, and Operations｜tam2
3+ day, 8+ hour ago (124+ words) This video, published by IBM Technology, focuses on the challenges of observability in multi-agent LLM (Large Language Model) applications. It explains how MLflow identifies and resolves AI-specific issues that are often overlooked by traditional monitoring. It suggests that this improves…...
What Retrieval Still Hasn't Decided
15+ hour, 34+ min ago (1785+ words) Started building a reranker CLI on top of Jev for the obvious reason: get the relevant documents to the top. Ended up with three modes in jev-reranker, and one I decided not to ship. The split appeared while I was…...
Build a Read-Only Eval Slice Before Giving Free Inference Write Authority
16+ hour, 32+ min ago (594+ words) So I refuse to give a completion write authority until a read-only eval slice can fail in public. The slice uses the same provider protocol the production route will use, minus the database mutation. Think of it like a dress…...
We spent the first twenty three minutes of an outage reading staging's graphs
17+ hour, 32+ min ago (792+ words) Checkout was returning errors for about one request in three and the dashboard in front of four engineers was entirely healthy. Request rate normal, error rate flat, latency at a hundred and eighty milliseconds. Everything on the screen was accurate,…...
JobMaster achieving horizontal scaling for.NET jobs while maintaining a centralised audit log
17+ hour, 54+ min ago (50+ words) How JobMaster's bucket system solves the tradeoff between horizontal scaling and a centralized audit trail for.NET background jobs. Tagged with dotnet, csharp, architecture, opensource....
OpenTelemetry tracing for Dagster, without monkeypatching or giving up @op/@asset
1+ day, 4+ min ago (177+ words) Dagster doesn't have a built-in way to get OpenTelemetry traces out of a run. There's an open issue on the main repo asking for it (dagster-io/dagster#11191), and a more specific one asking for trace/span IDs correlated into log…...