本 notebook 展示如何使用 OpenAI Responses LLM。

如果你在 Colab 中打开这个 Notebook，可能需要先安装 LlamaIndex 🦙。

%pip install llama-index llama-index-llms-openai

基本用法

import os

os.environ["OPENAI_API_KEY"] = "..."

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini",
    # api_key="some key",  # uses OPENAI_API_KEY env var by default
)

使用提示词调用 complete

from llama_index.llms.openai import OpenAI

resp = llm.complete("Paul Graham is ")

print(resp)

Paul Graham is a prominent computer scientist, entrepreneur, and venture capitalist, best known for co-founding the startup accelerator Y Combinator. He is also recognized for his essays on technology, startups, and programming, which have influenced many in the tech community. Graham has a background in programming languages and artificial intelligence, having earned a Ph.D. from Harvard University. His work has significantly shaped the startup ecosystem, particularly in Silicon Valley. Would you like to know more about a specific aspect of his work or ideas?

使用消息列表调用 chat

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)

print(resp)

assistant: Ahoy, matey! Ye can call me Captain Jollybeard, the most colorful pirate to sail the seven seas! What brings ye to me ship today? Arrr!

流式传输

使用 stream_complete 端点

resp = llm.stream_complete("Paul Graham is ")

for r in resp:
    print(r.delta, end="")

Paul Graham is a prominent computer scientist, entrepreneur, and venture capitalist, best known for co-founding the startup accelerator Y Combinator. He is also recognized for his essays on technology, startups, and programming, which have influenced many in the tech community. Graham has a background in programming languages and artificial intelligence and has authored several influential works, including "Hackers and Painters." His insights on entrepreneurship and innovation have made him a respected figure in Silicon Valley.

使用 stream_chat 端点

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)

for r in resp:
    print(r.delta, end="")

Ahoy there! Ye can call me Captain Jollybeard, the most colorful pirate to sail the seven seas! What brings ye to me ship today?

配置参数

Responses API 支持许多选项：

• 设置模型名称
• 温度、top_p、max_output_tokens 等生成参数
• 启用内置工具调用
• 为 O 系列模型设置推理强度
• 跟踪之前的响应以自动维护对话历史
• 以及更多！

基本参数

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini",
    temperature=0.5,  # default is 0.1
    max_output_tokens=100,  # default is None
    top_p=0.95,  # default is 1.0
)

内置工具调用

Responses API 支持内置工具调用，你可以在此处了解更多。

配置此项意味着 LLM 将自动调用工具并用它来增强响应。

工具定义为字典列表，每个字典包含一个工具的设置。

以下是使用内置网络搜索工具的示例。

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="gpt-4o-mini",
    built_in_tools=[{"type": "web_search_preview"}],
)

resp = llm.chat(
    [ChatMessage(role="user", content="What is the weather in San Francisco?")]
)
print(resp)
print("========" * 2)
print(resp.additional_kwargs)

assistant: As of 12:18 AM on Friday, March 28, 2025, in San Francisco, the current weather is partly sunny with a temperature of 61°F (16°C).

## Weather for San Francisco, CA:
Current Conditions: Partly sunny, 61°F (16°C)

Daily Forecast:
* Thursday, March 27: Low: 52°F (11°C), High: 61°F (16°C), Description: Periods of rain and drizzle beginning in the late morning; breezy this afternoon
* Friday, March 28: Low: 47°F (8°C), High: 61°F (16°C), Description: A shower in the area in the morning; otherwise, clouds giving way to some sun
* Saturday, March 29: Low: 50°F (10°C), High: 60°F (15°C), Description: Mostly sunny
* Sunday, March 30: Low: 51°F (11°C), High: 59°F (15°C), Description: Cloudy; periods of rain in the morning followed by a shower in spots in the afternoon
* Monday, March 31: Low: 49°F (10°C), High: 58°F (14°C), Description: Cloudy and cool; a couple of showers in the afternoon
* Tuesday, April 01: Low: 53°F (12°C), High: 58°F (14°C), Description: Some sunshine giving way to clouds, breezy and cool; occasional rain in the afternoon
* Wednesday, April 02: Low: 52°F (11°C), High: 56°F (13°C), Description: A couple of showers in the morning; otherwise, cloudy and remaining cool

In March, San Francisco typically experiences daytime temperatures around 61°F (16°C) and nighttime temperatures around 47°F (8°C). The city usually receives about 3.5 inches (89 mm) of rainfall over approximately 11 days during the month. ([weather2visit.com](https://www.weather2visit.com/north-america/united-states/san-francisco-march.htm?utm_source=openai))
================
{'built_in_tool_calls': [ResponseFunctionWebSearch(id='ws_67e5eaecce088191ab2edce452ef25420a24041ef7e917b2', status='completed', type='web_search_call')], 'annotations': [AnnotationURLCitation(end_index=1561, start_index=1439, title='San Francisco Weather in March 2025 | United States Averages | Weather-2-Visit', type='url_citation', url='https://www.weather2visit.com/north-america/united-states/san-francisco-march.htm?utm_source=openai')], 'usage': ResponseUsage(input_tokens=327, output_tokens=462, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=789, input_tokens_details={'cached_tokens': 0})}

推理强度

对于 O 系列模型，你可以设置推理强度来控制模型用于推理的时间。

更多信息请参阅 OpenAI API 文档。

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="o3-mini",
    reasoning_options={"effort": "high"},
)

resp = llm.chat(
    [ChatMessage(role="user", content="What is the meaning of life?")]
)
print(resp)
print("========" * 2)
print(resp.additional_kwargs)

assistant: The question “What is the meaning of life?” has been asked by philosophers, theologians, scientists, and countless individuals throughout history—and there isn’t one definitive answer that satisfies everyone. Here are a few perspectives that help illustrate why the answer is so open-ended:

1. Philosophical and Existential Views:
 • Some existentialist thinkers, such as Jean-Paul Sartre and Albert Camus, argue that life doesn’t come with a preordained meaning. Instead, they suggest that it’s up to each individual to create their own purpose through choices, relationships, and personal projects.
 • Other philosophies, like those in classical philosophy, have emphasized the pursuit of virtue, knowledge, or happiness as key to a meaningful life.

2. Religious and Spiritual Perspectives:
 • Many religious traditions offer meanings tied to their spiritual beliefs. In these views, life might be seen as a journey of growth, moral development, or fulfilling a divine plan—whether that means living according to God’s commandments, achieving enlightenment, or learning lessons through reincarnation.
 • Spiritual traditions often encourage adherents to find meaning in love, compassion, and service to others.

3. Scientific and Naturalistic Approaches:
 • From a biological standpoint, one might say that the purpose of life is simply to survive and reproduce—the basic mechanisms that drive natural selection and evolution.
 • Some argue that while biology defines life’s mechanics, human consciousness allows us to transcend mere survival and imbue our existence with personal and cultural meaning.

4. Personal and Cultural Interpretations:
 • For many, meaning is found in everyday connections: love, creativity, learning, and contributing to the community or society at large.
 • Different cultures and individuals may prioritize various values (such as achievement, compassion, or exploration), which means the “meaning” can vary widely based on one’s upbringing, experiences, and personal reflections.

5. A Humorous Take:
 • Popular culture, notably in Douglas Adams’ The Hitchhiker’s Guide to the Galaxy, famously suggests that the answer to the ultimate question of life is “42”—a playful reminder that the real significance often lies in the journey of questioning itself rather than arriving at a single answer.

In the end, the meaning of life might be considered less of an absolute truth and more of an invitation to explore, question, and define what matters most to you personally. Whether through relationships, creative pursuits, intellectual challenges, or spiritual practices, many believe we have the power to create our own meaning in a vast and complex universe.
================
{'built_in_tool_calls': [], 'reasoning': ResponseReasoningItem(id='rs_683e2dde0e308198a72c5e7f2e9bf52a0dd2faa5908183b8', summary=[], type='reasoning', encrypted_content=None, status=None), 'annotations': [], 'usage': ResponseUsage(input_tokens=13, input_tokens_details=InputTokensDetails(cached_tokens=0), output_tokens=841, output_tokens_details=OutputTokensDetails(reasoning_tokens=320), total_tokens=854)}

图像支持

OpenAI 的许多模型支持在聊天消息输入中使用图像。

使用聊天消息的内容块功能，你可以轻松地在单个 LLM 提示词中组合文本和图像。

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock
from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o")

messages = [
    ChatMessage(
        role="user",
        blocks=[
            ImageBlock(path="image.png"),
            TextBlock(text="Describe the image in a few sentences."),
        ],
    )
]

resp = llm.chat(messages)
print(resp.message.content)

The image shows three white dice with black dots, captured in mid-air above a checkered surface. The dice are in various orientations, displaying different numbers of dots. The background is dark, with a subtle light illuminating the dice, creating a dramatic effect. The checkered surface resembles a chess or checkerboard.

使用函数/工具调用

OpenAI 模型原生支持函数调用。它可以方便地与 LlamaIndex 工具抽象集成，让你可以将任意 Python 函数插入 LLM。

在下面的示例中，我们定义一个函数来生成 Song 对象。

from pydantic import BaseModel
from llama_index.core.tools import FunctionTool

class Song(BaseModel):
    """A song with name and artist"""

    name: str
    artist: str

def generate_song(name: str, artist: str) -> Song:
    """Generates a song with provided name and artist."""
    return Song(name=name, artist=artist)

tool = FunctionTool.from_defaults(fn=generate_song)

strict 参数告诉 OpenAI 在生成工具调用/结构化输出时是否使用约束采样。这意味着生成的工具调用 schema 将始终包含预期字段。

由于这似乎会增加延迟，它默认设为 false。

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o-mini", strict=True)
response = llm.predict_and_call(
    [tool],
    "Write a random song for me",
    # strict=True  # can also be set at the function level to override the class
)
print(str(response))

name='Chasing Stars' artist='Luna Sky'

我们也可以进行多个函数调用。

llm = OpenAIResponses(model="gpt-4o-mini")
response = llm.predict_and_call(
    [tool],
    "Generate five songs from the Beatles",
    allow_parallel_tool_calls=True,
)
for s in response.sources:
    print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")

Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Hey Jude', 'artist': 'The Beatles'}}, Output: name='Hey Jude' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Let It Be', 'artist': 'The Beatles'}}, Output: name='Let It Be' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Come Together', 'artist': 'The Beatles'}}, Output: name='Come Together' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Yesterday', 'artist': 'The Beatles'}}, Output: name='Yesterday' artist='The Beatles'
Name: generate_song, Input: {'args': (), 'kwargs': {'name': 'Twist and Shout', 'artist': 'The Beatles'}}, Output: name='Twist and Shout' artist='The Beatles'

手动工具调用

如果你想控制工具的调用方式，也可以将工具调用和工具选择拆分为各自独立的步骤。

首先，让我们选择一个工具。

from llama_index.core.llms import ChatMessage
from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o-mini")

chat_history = [ChatMessage(role="user", content="Write a random song for me")]

resp = llm.chat_with_tools([tool], chat_history=chat_history)

现在，让我们调用 LLM 选择的工具（如果有）。

如果发生了工具调用，我们应该将结果发送给 LLM 以生成最终响应（或另一次工具调用！）。

tools_by_name = {t.metadata.name: t for t in [tool]}
tool_calls = llm.get_tool_calls_from_response(
    resp, error_on_no_tool_call=False
)

while tool_calls:
    # add the LLM's response to the chat history
    chat_history.append(resp.message)

    for tool_call in tool_calls:
        tool_name = tool_call.tool_name
        tool_kwargs = tool_call.tool_kwargs

        print(f"Calling {tool_name} with {tool_kwargs}")
        tool_output = tool(**tool_kwargs)
        chat_history.append(
            ChatMessage(
                role="tool",
                content=str(tool_output),
                # most LLMs like OpenAI need to know the tool call id
                additional_kwargs={"call_id": tool_call.tool_id},
            )
        )

        resp = llm.chat_with_tools([tool], chat_history=chat_history)
        tool_calls = llm.get_tool_calls_from_response(
            resp, error_on_no_tool_call=False
        )

Calling generate_song with {'name': 'Chasing Stars', 'artist': 'Luna Sky'}

现在，我们应该得到最终响应了！

print(resp.message.content)

Here's a song for you titled **"Chasing Stars"** by **Luna Sky**!

### Chasing Stars

**Verse 1**
In the midnight glow, we wander free,
With dreams like fireflies, lighting up the sea.
Whispers of the night, calling out our names,
Together we’ll ignite, this wild, untamed flame.

**Chorus**
We’re chasing stars, through the endless night,
With every heartbeat, we’ll take flight.
Hand in hand, we’ll break the dark,
In this cosmic dance, we’ll leave our mark.

**Verse 2**
Underneath the moon, secrets softly shared,
Every glance a promise, every touch a dare.
The universe is ours, let the journey start,
With every step we take, we’re painting art.

**Chorus**
We’re chasing stars, through the endless night,
With every heartbeat, we’ll take flight.
Hand in hand, we’ll break the dark,
In this cosmic dance, we’ll leave our mark.

**Bridge**
And when the dawn arrives, we’ll still be here,
With the echoes of our laughter, crystal clear.
No matter where we go, no matter how far,
Forever in our hearts, we’ll chase those stars.

**Chorus**
We’re chasing stars, through the endless night,
With every heartbeat, we’ll take flight.
Hand in hand, we’ll break the dark,
In this cosmic dance, we’ll leave our mark.

**Outro**
So let’s chase the stars, let’s light the way,
In this beautiful journey, we’ll never stray.
With dreams as our compass, love as our guide,
Together we’ll soar, side by side.

Feel free to let me know if you'd like any changes or another song!

结构化预测

函数调用的一个重要用例是提取结构化对象。LlamaIndex 提供直观的接口，可将任意 LLM 转换为结构化 LLM——只需定义目标 Pydantic 类（可以嵌套），然后给定提示词，我们就会提取出所需对象。

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.prompts import PromptTemplate
from pydantic import BaseModel
from typing import List

class MenuItem(BaseModel):
    """A menu item in a restaurant."""

    course_name: str
    is_vegetarian: bool

class Restaurant(BaseModel):
    """A restaurant with name, city, and cuisine."""

    name: str
    city: str
    cuisine: str
    menu_items: List[MenuItem]

llm = OpenAIResponses(model="gpt-4o-mini")
prompt_tmpl = PromptTemplate(
    "Generate a restaurant in a given city {city_name}"
)
# Option 1: Use `as_structured_llm`
restaurant_obj = (
    llm.as_structured_llm(Restaurant)
    .complete(prompt_tmpl.format(city_name="Dallas"))
    .raw
)
# Option 2: Use `structured_predict`
# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name="Miami")

restaurant_obj

Restaurant(name='Tex-Mex Delight', city='Dallas', cuisine='Tex-Mex', menu_items=[MenuItem(course_name='Tacos', is_vegetarian=False), MenuItem(course_name='Vegetarian Enchiladas', is_vegetarian=True), MenuItem(course_name='Fajitas', is_vegetarian=False), MenuItem(course_name='Chips and Salsa', is_vegetarian=True), MenuItem(course_name='Queso Dip', is_vegetarian=True)])

异步

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(model="gpt-4o")

resp = await llm.acomplete("Paul Graham is ")

print(resp)

Paul Graham is a British-American entrepreneur, venture capitalist, and essayist. He is best known for co-founding Viaweb, one of the first web-based applications, which was later sold to Yahoo and became Yahoo Store. Graham is also a co-founder of Y Combinator, a highly influential startup accelerator that has funded and supported numerous successful startups, including Dropbox, Airbnb, and Reddit. In addition to his work in technology and startups, Graham is known for his insightful essays on topics such as programming, entrepreneurship, and the philosophy of work.

resp = await llm.astream_complete("Paul Graham is ")

async for delta in resp:
    print(delta.delta, end="")

Paul Graham is a British-American entrepreneur, venture capitalist, and essayist. He is best known for co-founding Viaweb, one of the first web-based applications, which was later sold to Yahoo and became Yahoo Store. Graham is also a co-founder of Y Combinator, a highly influential startup accelerator that has funded and supported numerous successful startups, including Dropbox, Airbnb, and Reddit. In addition to his work in technology and startups, Graham is known for his insightful essays on topics related to entrepreneurship, technology, and society.

也支持异步函数调用。

llm = OpenAIResponses(model="gpt-4o-mini")
response = await llm.apredict_and_call([tool], "Generate a random song")
print(str(response))

name='Chasing Stars' artist='Luna Sky'

其他 kwargs

如果构造函数中不存在额外的 kwargs，你可以通过 additional_kwargs 在实例级别设置它们。

这些参数将传入对 LLM 的每次调用。

from llama_index.llms.openai import OpenAIResponses

llm = OpenAIResponses(
    model="gpt-4o-mini", additional_kwargs={"user": "your_user_id"}
)
resp = llm.complete("Paul Graham is ")
print(resp)

图像生成

你可以通过将 {'type': 'image_generation'} 作为内置工具传入来使用图像生成；或者，如果你想启用流式输出，传入 {'type': 'image_generation', 'partial_images': 2}：

import base64
from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage, ImageBlock, TextBlock

# run without streaming
llm = OpenAIResponses(
    model="gpt-4.1-mini", built_in_tools=[{"type": "image_generation"}]
)
messages = [
    ChatMessage.from_str(
        content="A llama dancing with a cat in a meadow", role="user"
    )
]
response = llm.chat(
    messages
)  # response = await llm.achat(messages) for an async implementation
for block in response.message.blocks:
    if isinstance(block, ImageBlock):
        with open("llama_and_cat_dancing.png", "wb") as f:
            f.write(bas64.b64decode(block.image))
    elif isinstance(block, TextBlock):
        print(block.text)

# run with streaming
llm_stream = OpenAIResponses(
    model="gpt-4.1-mini",
    built_in_tools=[{"type": "image_generation", "partial_images": 2}],
)
response = llm_stream.stream_chat(
    messages
)  # response = await llm_stream.asteam_chat(messages) for an async implementation
for event in response:
    for block in event.message.blocks:
        if isinstance(block, ImageBlock):
            # block.detail contains the ID of the image
            with open(f"llama_and_cat_dancing_{block.detail}.png", "wb") as f:
                f.write(bas64.b64decode(block.image))
        elif isinstance(block, TextBlock):
            print(block.text)

MCP 远程调用

你可以通过 OpenAI Responses API 调用任何远程 MCP，只需将 MCP 的具体信息作为内置工具传给 LLM

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="gpt-4.1",
    built_in_tools=[
        {
            "type": "mcp",
            "server_label": "deepwiki",
            "server_url": "https://mcp.deepwiki.com/mcp",
            "require_approval": "never",
        }
    ],
)
messages = [
    ChatMessage.from_str(
        content="What transport protocols are supported in the 2025-03-26 version of the MCP spec?",
        role="user",
    )
]
response = llm.chat(messages)
# see the textual output
print(response.message.content)
# see the MCP tool call
print(response.raw.output[0])

代码解释器

你可以通过将 "type": "code_interpreter", "container": { "type": "auto" } 设置为内置工具来使用 Code Interpreter。

from llama_index.llms.openai import OpenAIResponses
from llama_index.core.llms import ChatMessage

llm = OpenAIResponses(
    model="gpt-4.1",
    built_in_tools=[
        {
            "type": "code_interpreter",
            "container": {"type": "auto"},
        }
    ],
)
messages = messages = [
    ChatMessage.from_str(
        content="I need to solve the equation 3x + 11 = 14. Can you help me?",
        role="user",
    )
]
response = llm.chat(messages)
# see the textual output
print(response.message.content)
# see the MCP tool call
print(response.raw.output[0])