RubyLLM

Setup

Install the Braintrust and RubyLLM gems, then configure your API keys.

Install gems

gem install braintrust ruby_llm

Set environment variables

.env

BRAINTRUST_API_KEY=your-api-key
BRAINTRUST_DEFAULT_PROJECT=your-project-name  # Project that spans are logged to

# Configure your preferred provider(s)
OPENAI_API_KEY=your-openai-key
# ANTHROPIC_API_KEY=your-anthropic-key
# GOOGLE_API_KEY=your-google-key

Auto-instrumentation

To trace RubyLLM without modifying how you construct chats, load braintrust/setup before you require ruby_llm. Braintrust intercepts the require and patches RubyLLM::Chat, so every chat, tool, and streaming call emits a span. Loading braintrust/setup also initializes Braintrust for you and logs to the project set by BRAINTRUST_DEFAULT_PROJECT.

require 'braintrust/setup'
require 'ruby_llm'

# Configure RubyLLM with your provider
RubyLLM.configure do |config|
  config.openai_api_key = ENV['OPENAI_API_KEY']
end

# Create a chat and make requests (traced automatically)
chat = RubyLLM.chat(model: 'gpt-4o-mini')
response = chat.ask('What is machine learning?')

puts response.content

In a Rails app, add gem "braintrust", require: "braintrust/setup" to your Gemfile to enable auto-instrumentation without an explicit require line.

Manual instrumentation

To trace RubyLLM calls manually, enable instrumentation yourself with Braintrust.instrument!(:ruby_llm). Once enabled, every chat, tool, and streaming call emits a span. The following app.rb initializes Braintrust, instruments RubyLLM, and makes a request:

require 'braintrust'
require 'ruby_llm'

# Initialize Braintrust
Braintrust.init(default_project: 'My Project')

# Enable Braintrust tracing for RubyLLM
Braintrust.instrument!(:ruby_llm)

# Configure RubyLLM with your provider
RubyLLM.configure do |config|
  config.openai_api_key = ENV['OPENAI_API_KEY']
end

# Create a chat and make requests (traced)
chat = RubyLLM.chat(model: 'gpt-4o-mini')
response = chat.ask('What is machine learning?')

puts response.content

To instrument a single chat instead of every RubyLLM chat, pass the instance as a target: Braintrust.instrument!(:ruby_llm, target: chat).

What Braintrust traces

Braintrust captures:

Chat completion spans (ruby_llm.chat), with input messages, the model, and provider metadata, plus the assistant response and finish reason.
Tool execution spans (ruby_llm.tool.<name>), with the tool name, call ID, arguments, and returned result. Tool schemas passed to the model are captured on the chat span.
Token usage metrics (prompt, completion, total, and cached tokens) for chat completions.
Streaming responses, aggregated onto the chat span with time to first token.
Multimodal message content, including text with image and file attachments.
Errors, recorded on the span when a request raises.

AI providers

SDKs

Developer tools

Setup

Auto-instrumentation

Manual instrumentation

What Braintrust traces

Resources

​Setup

​Auto-instrumentation

​Manual instrumentation

​What Braintrust traces

​Resources

Setup

Auto-instrumentation

Manual instrumentation

What Braintrust traces

Resources