traceloop/openllmetry
Open-source observability for GenAI/LLM applications based on OpenTelemetry with 25+ integration backends.
vectordotdev/vector
A high-performance observability data pipeline for collecting, transforming, and routing logs and metrics. Real-time data processing with 50+ sources and sinks including Kafka, S3, and Elasticsearch. Ideal for AI/ML log processing and data ingestion. MPL 2.0 licensed.
pezzolabs/pezzo
Cloud-native LLMOps platform with prompt management, versioning, and observability. Features collaborative prompt editing, A/B testing, and cost analytics. Apache 2.0 licensed.
openlit/openlit
OpenTelemetry-native LLM observability platform with GPU monitoring, evaluations, prompt management, and guardrails.
kubestellar/console
AI-powered multi-cluster Kubernetes dashboard with GPU workload monitoring, AI pipeline observability, and CNCF ecosystem integrations. Apache 2.0 licensed.
thedotmack/claude-mem
Persistent agent-memory system that captures session observations, summarizes them, and retrieves relevant context across coding-agent sessions. Apache-2.0 licensed.
pydantic/pydantic-ai
Type-safe AI agent framework from the creators of Pydantic. Model-agnostic with 20+ providers, built-in observability via Logfire, MCP/A2A protocol support, and YAML/JSON agent definitions. MIT licensed.
mastra-ai/mastra
TypeScript-first agent framework with built-in RAG, workflows, tool integrations, observability and observational memory.
comet-ml/opik
Production-ready LLM evaluation platform.
lindixu6-hash/awesome-agentic-engineering
Zero-dependency AI agent production-readiness toolkit with GitHub Actions score and risk-profile gates, prompt-injection fixtures, machine-readable results, and reproducible LangGraph traces. MIT licensed.
Arize-ai/phoenix
AI observability & evaluation platform.
quickwit-oss/quickwit
Cloud-native search engine for observability. Open-source alternative to Datadog, Elasticsearch, Loki, and Tempo with native vector search support.
prefecthq/prefect
Workflow orchestration framework for building resilient data and ML pipelines. Python-native with modern observability and 200+ integrations. Apache 2.0 licensed.
morluto/flameox
Runtime-evidence toolkit for agents that coordinates PyTorch Profiler and Nsight Systems captures, preserves native traces, and compares GPU-kernel and inference runs. MIT licensed.
quarto-dev/quarto-cli
Open-source scientific and technical publishing system built on Pandoc. Create dynamic content with Python, R, Julia, and Observable. MIT licensed.
ianwieds/awesome-ai-agents
Curated list of AI agent frameworks, coding agents, browser agents, tools, platforms and research.
dagster-io/dagster
Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
langfuse/langfuse
#1 open-source LLM observability platform.
helicone/helicone
Open-source LLM observability platform with request logging, caching, prompt management, rate limiting, and cost analytics. Apache 2.0 licensed.
maximhq/bifrost
LLM gateway with a unified OpenAI-compatible API across providers, routing, load balancing, fallbacks, guardrails, and observability. Apache 2.0 licensed.
open-metadata/OpenMetadata
Unified metadata platform for data discovery, observability, and governance. Column-level lineage, semantic search, and team collaboration with 70+ data service connectors. Apache 2.0 licensed.
Portkey-AI/gateway
Blazing fast AI Gateway to route 200+ LLMs with unified API. Integrated guardrails, load balancing, fallbacks, and cost tracking. MIT licensed.
Giskard-AI/giskard-oss
Open-source evaluation and testing library for LLM agents. Red teaming, vulnerability scanning, RAG evaluation, and safety testing with modular architecture. Apache 2.0 licensed.
latitude-dev/latitude-llm
Open-source agent engineering platform with prompt management, evaluations, and optimization. Features prompt playground, LLM-as-judge evals, and GEPA prompt optimizer for production LLM features. LGPL-3.0 licensed.