WatsonxLLM 是 IBM watsonx.ai 基础模型的封装。

这些示例旨在展示如何使用 LlamaIndex LLM API 与 watsonx.ai 模型通信。

设置

安装 llama-index-llms-ibm 包：

!pip install -qU llama-index-llms-ibm

下面的单元格定义了使用 watsonx Foundation Model 推理所需的凭据。

操作： 提供 IBM Cloud 用户 API 密钥。详情请参阅管理用户 API 密钥。

import os
from getpass import getpass

watsonx_api_key = getpass()
os.environ["WATSONX_APIKEY"] = watsonx_api_key

此外，你还可以通过环境变量传入额外的密钥：

import os

os.environ["WATSONX_URL"] = "your service instance url"
os.environ["WATSONX_TOKEN"] = "your token for accessing the CPD cluster"
os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster"
os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster"
os.environ[
    "WATSONX_INSTANCE_ID"
] = "your instance_id for accessing the CPD cluster"

加载模型

你可能需要针对不同模型或任务调整模型 parameters。详情请参阅可用的 MetaNames。

temperature = 0.5
max_new_tokens = 50
additional_params = {
    "decoding_method": "sample",
    "min_new_tokens": 1,
    "top_k": 50,
    "top_p": 1,
}

使用之前设置的参数初始化 WatsonxLLM 类。

注意：

• 为 API 调用提供上下文时，必须传入 project_id 或 space_id。要获取项目或空间 ID，请打开你的项目或空间，转到 Manage 标签页，然后点击 General。更多信息请参阅：项目文档 或 部署空间文档。
• 根据已配置服务实例所在的区域，使用 watsonx.ai API Authentication 中列出的某个 URL。

在本示例中，我们将使用 project_id 和 Dallas URL。

你需要指定用于推理的 model_id。可以在受支持的基础模型 中找到所有可用模型的列表。

from llama_index.llms.ibm import WatsonxLLM

watsonx_llm = WatsonxLLM(
    model_id="ibm/granite-13b-instruct-v2",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="PASTE YOUR PROJECT_ID HERE",
    temperature=temperature,
    max_new_tokens=max_new_tokens,
    additional_params=additional_params,
)

或者，你也可以使用 Cloud Pak for Data 凭据。详情请参阅 watsonx.ai 软件设置。

watsonx_llm = WatsonxLLM(
    model_id="ibm/granite-13b-instruct-v2",
    url="PASTE YOUR URL HERE",
    username="PASTE YOUR USERNAME HERE",
    password="PASTE YOUR PASSWORD HERE",
    instance_id="openshift",
    version="4.8",
    project_id="PASTE YOUR PROJECT_ID HERE",
    temperature=temperature,
    max_new_tokens=max_new_tokens,
    additional_params=additional_params,
)

除了 model_id，你也可以传入之前调优模型的 deployment_id。完整的模型调优流程在使用 TuneExperiment 和 PromptTuner中描述。

watsonx_llm = WatsonxLLM(
    deployment_id="PASTE YOUR DEPLOYMENT_ID HERE",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="PASTE YOUR PROJECT_ID HERE",
    temperature=temperature,
    max_new_tokens=max_new_tokens,
    additional_params=additional_params,
)

创建补全

使用字符串类型的提示词直接调用模型：

response = watsonx_llm.complete("What is a Generative AI?")
print(response)

A generative AI is a computer program that can create new text, images, or other types of content. These programs are trained on large datasets of existing content, and they use that data to generate new content that is similar to the training data.

从 CompletionResponse 中，你还可以获取服务返回的原始响应：

print(response.raw)

{'model_id': 'ibm/granite-13b-instruct-v2', 'created_at': '2024-05-20T07:11:57.984Z', 'results': [{'generated_text': 'A generative AI is a computer program that can create new text, images, or other types of content. These programs are trained on large datasets of existing content, and they use that data to generate new content that is similar to the training data.', 'generated_token_count': 50, 'input_token_count': 7, 'stop_reason': 'max_tokens', 'seed': 494448017}]}

你也可以调用提供提示词模板的模型：

from llama_index.core import PromptTemplate

template = "What is {object} and how does it work?"
prompt_template = PromptTemplate(template=template)

prompt = prompt_template.format(object="a loan")

response = watsonx_llm.complete(prompt)
print(response)

A loan is a sum of money that is borrowed to buy something, such as a house or a car. The borrower must repay the loan plus interest. The interest is a fee charged for using the money. The interest rate is the amount of

使用消息列表调用 chat

通过提供消息列表创建 chat 补全：

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are an AI assistant"),
    ChatMessage(role="user", content="Who are you?"),
]
response = watsonx_llm.chat(
    messages, max_new_tokens=20, decoding_method="greedy"
)
print(response)

assistant: I am an AI assistant.

请注意，我们将 max_new_tokens 参数改为 20，将 decoding_method 参数改为 greedy。

流式输出模型结果

流式输出模型的响应：

for chunk in watsonx_llm.stream_complete(
    "Describe your favorite city and why it is your favorite."
):
    print(chunk.delta, end="")

I like New York because it is the city of dreams. You can achieve anything you want here.

类似地，要流式输出 chat 补全，请使用以下代码：

messages = [
    ChatMessage(role="system", content="You are an AI assistant"),
    ChatMessage(role="user", content="Who are you?"),
]

for chunk in watsonx_llm.stream_chat(messages):
    print(chunk.delta, end="")

I am an AI assistant.