# Langfuse Tutorial: LLM Observability Setup and Production Patterns

> Step-by-step Langfuse tutorial: LLM call tracing, prompt versioning, cost tracking, output evaluations. Self-hosted setup in 15 minutes with Docker Compose.
> Author: Roman Belov · Published: 2026-02-22 · Source: https://futurecraft.pro/blog/llm-observability-langfuse/

LLM calls in production need tracing, [prompts need versioning](/blog/prompt-engineering-system/), costs need tracking. Standard APM tools won't help here: responses are non-deterministic, every call costs money, and a 200 OK says nothing about output quality.

Langfuse is an open-source platform (MIT) for LLM observability. Self-hosted with no limits, 22,000+ GitHub stars, 23 million SDK installs per month. This guide covers installation, tracing, prompt management, cost tracking, and evaluations.

> **Note:** This article covers self-hosted Langfuse v2. For a current v3 installation, see the [step-by-step guide](/blog/langfuse-step-by-step/).

## LLM Observability: Four Components

If you've worked with APM (Application Performance Monitoring), LLM observability will feel familiar. But there are three important differences:

**Non-determinism.** The same prompt with temperature > 0 produces different outputs. You can't simply compare expected vs actual — quality has to be evaluated statistically.

**Per-call cost.** An HTTP request to your own API costs fractions of a cent. A single GPT-5.4 call with 10,000 tokens of context costs 3-5 cents. With 1,000 daily users, the difference between a good and bad prompt is hundreds of dollars per month.

**Output quality.** A 200 OK tells you nothing about the response quality. The model can return grammatically correct but factually useless text. You need quality metrics, not just availability metrics.

Four components of LLM observability:

```
+---------------------------------------------------------+
|                  LLM Observability                       |
+----------+----------+--------------+--------------------+
| Tracing  |   Cost   |   Prompt     |  Evaluation        |
|          | Tracking |  Management  |                    |
+----------+----------+--------------+--------------------+
| What     | How much | Which prompt | How good           |
| happened | it cost  | is in prod   | is the output      |
+----------+----------+--------------+--------------------+
```

Langfuse covers all four. You can start with one (usually tracing) and add the rest as you grow.

## Why Langfuse

There are five or six LLM observability platforms on the market. Here's what sets Langfuse apart:

**Open-source (MIT).** Code on GitHub, self-hosted version with no limits. No vendor lock-in — you always have the source code and data.

**Self-hosted for free.** No caps on trace count. For small and mid-size teams, self-hosting on a single VM is cheaper than any SaaS.

**Not tied to a framework.** LangSmith is built for LangChain — it's harder to use with other stacks. Langfuse works with any LLM provider: OpenAI, Anthropic, open-source models via LiteLLM, Vercel AI SDK.

**Actively developed.** Over the past year: SDK v3 on OpenTelemetry (native compatibility with any OTEL instrumentation), built-in [MCP server](/blog/mcp-production-custom-servers/) (prompt management right from your IDE), observation-level evaluations, dataset versioning. Growth from 10,000 to 22,000 stars in 10 months.

**Comparison (brief):**

| | Langfuse | LangSmith | Helicone |
|---|---------|-----------|----------|
| Open-source | MIT | No | Apache 2.0 |
| Self-hosted | Free, no limits | Enterprise only | Yes |
| Framework lock-in | No | LangChain | OpenAI-first |
| Prompt management | Yes + MCP | Yes | Yes (beta) |
| Free tier (cloud) | 50k obs/mo | 5k traces/mo | 100k req/mo |

## Installation in 15 Minutes: Self-Hosted

Minimum requirements: Docker and 2 GB of RAM. Langfuse bundles everything into a single docker-compose.

```yaml
# docker-compose.yml
services:
  langfuse:
    image: langfuse/langfuse:2
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://postgres:postgres@db:5432/langfuse
      NEXTAUTH_SECRET: your-secret-key-change-me
      SALT: your-salt-change-me
      NEXTAUTH_URL: http://localhost:3000
    depends_on:
      - db

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: langfuse
    volumes:
      - langfuse_data:/var/lib/postgresql/data

volumes:
  langfuse_data:
```

```bash
docker compose up -d
```

Langfuse will be available at `localhost:3000` within a minute. Create an account, a project, and copy the API keys.

### First Traces via Python SDK

```bash
pip install langfuse
```

```python
from langfuse import Langfuse

langfuse = Langfuse(
    public_key="pk-lf-...",
    secret_key="sk-lf-...",
    host="http://localhost:3000"  # your self-hosted URL
)

# Create a trace
trace = langfuse.trace(name="chat-response", user_id="user-123")

# Log an LLM call
generation = trace.generation(
    name="gpt-5.4-response",
    model="gpt-5.4",
    input=[{"role": "user", "content": "Recommend cafes in Moscow"}],
    output="Here are some options...",
    usage={"input": 42, "output": 128},
)

langfuse.flush()
```

Open the Langfuse UI — the trace will appear in the list with model, token, and latency data.

### Automatic Instrumentation via LiteLLM

Manual logging is for understanding the mechanics. In production, automation is better. If you use [LiteLLM as a proxy to LLM providers](/blog/multi-provider-llm-architecture/), Langfuse hooks in with a single line:

```python
import litellm
litellm.success_callback = ["langfuse"]

# Every call is automatically sent to Langfuse
response = litellm.completion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello"}],
    metadata={
        "trace_name": "chat-response",
        "trace_user_id": "user-123",
        "generation_name": "greeting",
        "tags": ["production", "chat"],
    }
)
```

LiteLLM sends everything to Langfuse: model, tokens, cost, latency, input/output. You write a regular LLM call, Langfuse fills up automatically.

For OpenAI SDK, there's a drop-in replacement:

```python
from langfuse.openai import openai

# Use it like the regular OpenAI SDK
# All calls are automatically traced
client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello"}],
)
```

## Tracing: What's Inside an LLM Call

A trace in Langfuse is a tree of operations. At the top level is the trace (one user request). Inside are generations (LLM calls) and spans (intermediate steps: retrieval, preprocessing, postprocessing).

```
Trace: "generate-itinerary"
|
+-- Span: "validate-input" (12ms)
|
+-- Generation: "analyze-request" (GPT-5.4, 340 tokens, $0.003)
|
+-- Span: "search-places" (Foursquare API, 800ms)
|
+-- Generation: "build-itinerary" (GPT-5.4, 2100 tokens, $0.018)
|
+-- Generation: "validate-result" (GPT-5.4-mini, 450 tokens, $0.001)
```

What you see in the UI for each generation:
- **Input/Output** — the full prompt and response
- **Model** — which model was used
- **Tokens** — input, output, total
- **Cost** — dollar amount (calculated automatically by model)
- **Latency** — response time
- **Metadata** — arbitrary fields (user ID, feature, session)

Three things tracing reveals immediately:

**Pattern 1: hidden retries.** Retry logic in code can call the model 2-3 times per user request. Without tracing, you only see the final response. With tracing — every call with its cost.

**Pattern 2: model mismatch.** GPT-5.4-mini is expected for quick responses, but one endpoint pulls GPT-5.4. Tracing shows the model per call — filtering by the model field immediately surfaces the discrepancy.

**Pattern 3: latency bottleneck.** In a chain of four LLM calls, one takes 80% of the time. Without spans, you only know the total. With spans — you see exactly where to optimize.

## Prompt Management: Versioning Without Deploys

The standard approach: prompts are hardcoded. A change requires PR, code review, merge, deploy. A/B testing two prompts means two deploys.

Langfuse externalizes prompts: your code loads them by name through an API. Changing a prompt in the UI takes effect on the next request — no deploy needed.

```python
# Load a prompt from Langfuse
prompt = langfuse.get_prompt("travel-assistant")

# Compile with variables
compiled = prompt.compile(
    destination="Moscow",
    preferences="vegetarian cuisine"
)

# Use in an LLM call
response = litellm.completion(
    model="gpt-5.4",
    messages=compiled,
    metadata={"langfuse_prompt_name": "travel-assistant"}
)
```

Every prompt change creates a new version. Langfuse stores the full history: who changed what and when. Roll back with a single click.

### Fallback Pattern

Langfuse is an external service. It can go down. The prompt is always needed.

```python
try:
    prompt = langfuse.get_prompt("travel-assistant", label="production")
    messages = prompt.compile(destination=city)
except Exception:
    # If Langfuse is unavailable — fall back to hardcoded prompt
    messages = [
        {"role": "system", "content": FALLBACK_SYSTEM_PROMPT},
        {"role": "user", "content": f"Recommend cafes in {city}"},
    ]
```

This pattern is mandatory for production. Langfuse speeds up prompt iteration, but it shouldn't be a single point of failure.

### MCP Server: Prompts from Your IDE

Since November 2025, Langfuse supports MCP (Model Context Protocol). By connecting the MCP server to Claude Code, Cursor, or another AI assistant, you can read and edit prompts directly from your IDE.

```json
{
  "mcpServers": {
    "langfuse": {
      "type": "http",
      "url": "https://your-langfuse.com/api/public/mcp",
      "headers": {
        "Authorization": "Basic base64(publicKey:secretKey)"
      }
    }
  }
}
```

Instead of switching between your IDE and a browser, the AI assistant sees the current prompts, suggests changes, and applies them via MCP — all without leaving the editor.

## Cost Tracking: How Much Each Feature Costs

Langfuse automatically calculates the cost of each LLM call based on the model and token count. The built-in pricing table updates with every release (GPT-5.5, Claude Opus 4.8 — supported on launch day).

In the dashboard you see:
- **Total cost** for a given period
- **Cost per trace** — average cost of a single request
- **Cost per user** — how much a specific user costs
- **Cost per model** — cost distribution across models

### Pattern: Per-Feature Cost Tracking

By adding tags or metadata to traces, you group costs by feature:

```python
trace = langfuse.trace(
    name="itinerary-generation",
    tags=["feature:itinerary", "tier:premium"],
    user_id="user-123",
)
```

Filtering by the `feature:itinerary` tag in the dashboard shows what itinerary generation specifically costs. Chat costs go separately, recommendations separately, summarization separately.

### Pattern: Syncing Costs to Your Database

The Langfuse API lets you programmatically retrieve costs:

```python
# Get all traces for a user
traces = langfuse.fetch_traces(user_id="user-123")

# Sum the cost
total_cost = sum(t.total_cost or 0 for t in traces.data)

# Write to your database for limits, billing, analytics
db.update_user_spending(user_id="user-123", amount=total_cost)
```

Useful if you have per-user LLM spending limits (freemium, credits) or want to show users their consumption.

## Evaluations: Output Quality in Numbers

Tracing shows what happened. Evaluations show how well it went.

### LLM-as-a-Judge

Automated evaluation via LLM: one model scores the outputs of another. Langfuse supports this out of the box.

Configure it in the UI: pick an evaluator template (relevance, helpfulness, toxicity), select the target set of traces — Langfuse runs the evaluation on each one. The result is a score from 0 to 1 attached to the trace.

### Observation-Level Evaluations (New in 2026)

Previously, scores could only be attached to the trace as a whole. But in a chain of four LLM calls, the problem might be in one specific step. Since February 2026, you can evaluate each observation (generation, span) individually.

Example: in a pipeline "analyze, search, generate, validate" the evaluator for `generate` checks factual accuracy, while the one for `validate` checks format compliance. Different evaluators for different steps.

### Datasets: Regression Testing for Prompts

A dataset in Langfuse is a set of input/expected_output pairs. Changed a prompt — run the dataset, compare scores with the previous version. If quality dropped — roll back.

Since December 2025, Langfuse versions datasets: each change (adding, deleting, updating an item) creates a new version. You can run an experiment on a specific historical version for reproducibility.

```python
# Create or update a dataset
langfuse.create_dataset(name="travel-queries")

langfuse.create_dataset_item(
    dataset_name="travel-queries",
    input={"query": "Cafes in central Moscow"},
    expected_output="A list of 5+ cafes with addresses",
)

# Run an experiment
dataset = langfuse.get_dataset("travel-queries")
for item in dataset.items:
    response = run_my_pipeline(item.input)
    item.link(
        trace_id=response.trace_id,
        run_name="prompt-v3-test",
    )
```

In the Langfuse UI, you'll see a comparison of runs: prompt v2 vs v3, with scores per item.

## Production Checklist

### Self-Hosted vs Cloud

| Criterion | Self-hosted | Cloud |
|-----------|-----------|-------|
| Cost | Infrastructure only (~$10-20/mo VPS) | Core $29/mo, Pro $199/mo |
| Data | Stays with you | On Langfuse servers |
| Maintenance | You update, back up, and monitor | All included |
| Limits | None | 50k obs/mo (free), then by plan |
| Best for | Teams with DevOps, compliance requirements | Quick start, small teams |

For most production projects, self-hosted is cheaper: $15/mo for a VPS vs Cloud Core at $29+/mo. But self-hosted means you're responsible for uptime and backups.

### Monitoring Langfuse Itself

Langfuse is an external dependency for your LLM calls (prompt management). If it goes down, prompts don't load. Two patterns:

1. **Health check.** `/api/public/health` returns the status. Add it to your monitoring (Zabbix, Uptime Robot, Grafana).
2. **Fallback prompts.** Every `get_prompt()` is wrapped in try/catch with a hardcoded fallback. Langfuse can be down — the app keeps running.

### Retention and Cleanup

Traces take up space in PostgreSQL. 1,000 traces per day means roughly 300–400 MB per month (depending on prompt and response size). Set up automatic deletion of old traces:

```sql
-- Delete traces older than 90 days
DELETE FROM traces WHERE created_at < NOW() - INTERVAL '90 days';
```

Or use the built-in retention settings in Langfuse Cloud.

### Minimum Production Setup

```
+-------------------------------------------+
|            Your Application                |
|                                            |
|  LLM call -> LiteLLM -> OpenAI/Anthropic  |
|       |                                    |
|       | metadata (trace_name,              |
|       |  user_id, tags)                    |
|       v                                    |
|  LiteLLM callback -> Langfuse             |
+-------------------------------------------+
         |
         v
+--------------------+
| Langfuse (Docker)  |
|                    |
| - Traces           |  <- automatic
| - Cost             |  <- automatic
| - Prompts          |  <- managed in UI
| - Evaluations      |  <- configured in UI
+--------------------+
```

Prompts are loaded from Langfuse on each call (with caching). Traces are sent asynchronously — they don't block the main thread. Cost is calculated automatically by model and token count.

## Conclusion

Langfuse covers four tasks with a single tool: tracing, cost tracking, prompt management, and evaluations. The self-hosted version is free and deploys in 15 minutes. Integration via LiteLLM or OpenAI drop-in takes one line of code.

The minimum path: hook up tracing to a single endpoint, look at the data after a week. The first call chain with a cost breakdown will make the value obvious.

**Links:**
- [Langfuse GitHub](https://github.com/langfuse/langfuse) — source code and self-hosted setup
- [Langfuse Docs](https://langfuse.com/docs) — documentation
- [LiteLLM + Langfuse](https://langfuse.com/docs/integrations/litellm) — integration
- [Langfuse Changelog](https://langfuse.com/changelog) — all updates

---

*Need help with LLM observability? I help startups build AI products and automate processes — [belov.works](https://belov.works).*