主题
llama-index-llms-nvidia 包包含 LlamaIndex 集成，用于在 NVIDIA NIM 推理微服务上使用模型构建应用。NIM 支持跨领域的模型，例如来自社区和 NVIDIA 的聊天、嵌入和重排序模型。这些模型由 NVIDIA 优化，可在 NVIDIA 加速基础设施上提供最佳性能，并作为 NIM 部署——NIM 是易于使用、预构建的容器，可在 NVIDIA 加速基础设施上通过单条命令部署到任何地方。
NVIDIA 托管的 NIM 部署可在 NVIDIA API 目录 中试用。测试完成后，可以使用 NVIDIA AI Enterprise 许可证从 NVIDIA API 目录导出 NIM，并在本地或云端运行，让企业拥有并完全控制其 IP 和 AI 应用。
NIM 按模型打包为容器镜像，并通过 NVIDIA NGC Catalog 以 NGC 容器镜像的形式分发。NIM 的核心是提供简单、一致且熟悉的 API，用于在 AI 模型上运行推理。
%pip install --upgrade --quiet llama-index-core llama-index-readers-file llama-index-llms-nvidia llama-index-embeddings-nvidia llama-index-postprocessor-nvidia-rerank引入一个测试数据集，一份关于 2021 年旧金山住房建设的 PDF。
!mkdir data
!wget "https://www.dropbox.com/scl/fi/p33j9112y0ysgwg77fdjz/2021_Housing_Inventory.pdf?rlkey=yyok6bb18s5o31snjd2dxkxz3&dl=0" -O "data/housing_data.pdf"--2024-05-28 17:42:44-- https://www.dropbox.com/scl/fi/p33j9112y0ysgwg77fdjz/2021_Housing_Inventory.pdf?rlkey=yyok6bb18s5o31snjd2dxkxz3&dl=0
Resolving www.dropbox.com (www.dropbox.com)... 162.125.1.18, 2620:100:6016:18::a27d:112
Connecting to www.dropbox.com (www.dropbox.com)|162.125.1.18|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com/cd/0/inline/CTzJ0ZeHC3AFIV3iv1bv9v0oMNXW03OW2waLdeKJNs0X6Tto0MSewm9RZBHwSLhqk4jWFaCmbhMGVXeWa6xPO4mAR4hC3xflJfwgS9Z4lpPUyE4AtlDXpnfsltjEaNeFCSY/file# [following]
--2024-05-28 17:42:45-- https://ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com/cd/0/inline/CTzJ0ZeHC3AFIV3iv1bv9v0oMNXW03OW2waLdeKJNs0X6Tto0MSewm9RZBHwSLhqk4jWFaCmbhMGVXeWa6xPO4mAR4hC3xflJfwgS9Z4lpPUyE4AtlDXpnfsltjEaNeFCSY/file
Resolving ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com (ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com)... 162.125.4.15, 2620:100:6016:15::a27d:10f
Connecting to ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com (ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com)|162.125.4.15|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /cd/0/inline2/CTySzMwupnuXzKpccOeYJ-7RI0NK0f7XMKBkpicHxSBuuwqAvFly51Fm0oCOwFctgeTqmD3thJsTqfFNOFHNe2JSIkJerj3mMr4Du3C7x1BcSy8t5raSfHQ_qSXF1eHrhdFII8Ou59jbofYVLe0punOl-RIa9k_v722SwkxVbg0KL9MrRL48XjX7JbsYHKTHq-gZSdAmpXpIGqS22eJavcSTuYMIy_GSZtDIs3quHM3PGU4849rG34RjpvAa-XkYDBdE996CxWupZ1C2Red9jEc5Tc6miGgt8-4LbGoxKwKF5I_Q3EqHCbvkibVR8OuKSKPtQZcNJSjsvIImzDLJ2WB6BAp2CBxz8szFF3jF3Gp6Iw/file [following]
--2024-05-28 17:42:45-- https://ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com/cd/0/inline2/CTySzMwupnuXzKpccOeYJ-7RI0NK0f7XMKBkpicHxSBuuwqAvFly51Fm0oCOwFctgeTqmD3thJsTqfFNOFHNe2JSIkJerj3mMr4Du3C7x1BcSy8t5raSfHQ_qSXF1eHrhdFII8Ou59jbofYVLe0punOl-RIa9k_v722SwkxVbg0KL9MrRL48XjX7JbsYHKTHq-gZSdAmpXpIGqS22eJavcSTuYMIy_GSZtDIs3quHM3PGU4849rG34RjpvAa-XkYDBdE996CxWupZ1C2Red9jEc5Tc6miGgt8-4LbGoxKwKF5I_Q3EqHCbvkibVR8OuKSKPtQZcNJSjsvIImzDLJ2WB6BAp2CBxz8szFF3jF3Gp6Iw/file
Reusing existing connection to ucc6b49e945b8d71944c85f4a76d.dl.dropboxusercontent.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 4808625 (4.6M) [application/pdf]
Saving to: ‘data/housing_data.pdf’
data/housing_data.p 100%[===================>] 4.58M 8.26MB/s in 0.6s
2024-05-28 17:42:47 (8.26 MB/s) - ‘data/housing_data.pdf’ saved [4808625/4808625]设置
导入依赖项，并从 NVIDIA API Catalog 设置你的 NVIDIA API 密钥。
开始使用：
在 NVIDIA 创建免费账户，该平台托管 NVIDIA AI Foundation 模型。
点击你选择的模型。
在 Input 下，选择 Python 选项卡，然后点击 Get API Key。接着点击 Generate Key。
复制并保存生成的密钥为
NVIDIA_API_KEY。完成后，你就可以访问这些端点。
from llama_index.core import SimpleDirectoryReader, Settings, VectorStoreIndex
from llama_index.embeddings.nvidia import NVIDIAEmbedding
from llama_index.llms.nvidia import NVIDIA
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core import Settings
from google.colab import userdata
import os
os.environ["NVIDIA_API_KEY"] = userdata.get("nvidia-api-key")使用 NVIDIA 托管的 NIM 作为嵌入模型。
NVIDIA 的默认嵌入只嵌入前 512 个 token，因此将分块大小设置为 500，以最大化嵌入的准确性。
Settings.text_splitter = SentenceSplitter(chunk_size=500)
documents = SimpleDirectoryReader("./data").load_data()将嵌入模型设置为 NVIDIA 的默认模型。如果分块超过模型可编码的 token 数，默认行为是抛出错误，因此将 truncate="END" 设置为丢弃超出限制的 token。
Settings.embed_model = NVIDIAEmbedding(model="NV-Embed-QA", truncate="END")
index = VectorStoreIndex.from_documents(documents)现在数据已在内存中嵌入并建立索引，接下来设置 LLM。按照 NIM 快速入门指南，可以在 5 分钟内使用 Docker 在本地托管 NIM。
以下示例展示如何：
- 将 Meta 的开源
meta/llama-3.1-8b-instruct模型用作本地 NIM。 - 将
meta/llama-3.1-70b-instruct用作由 NVIDIA 托管的 API Catalog 中的 NIM。
如果你使用的是本地 NIM，请将 base_url 改为你部署的 NIM URL。
该示例检索前 20 个最相关的分块来回答问题。
# Self-hosted NIM: uncomment the line below and comment the API Catalog line
# Settings.llm = NVIDIA(model="meta/llama-3.1-8b-instruct", base_url="http://your-nim-host-address:8000/v1")
# API Catalog NIM: comment this line if using a self-hosted NIM
Settings.llm = NVIDIA(model="meta/llama-3.1-70b-instruct")
query_engine = index.as_query_engine(similarity_top_k=20)问一个在文档中某处（第 18 页）有答案的简单问题。
response = query_engine.query(
"How many new housing units were built in San Francisco in 2021?"
)
print(response)There was a net addition of 4,649 units to the City’s housing stock in 2021.现在问一个更复杂的问题，需要阅读表格（文档第 41 页）：
response = query_engine.query(
"What was the net gain in housing units in the Mission in 2021?"
)
print(response)There is no specific information about the net gain in housing units in the Mission in 2021. The provided data is about the city's overall housing stock and production, but it does not provide a breakdown by neighborhood, including the Mission.这不是正确答案。试试更高级的 PDF 解析器 LlamaParse：
!pip install llama-parsefrom llama_parse import LlamaParse
# in a notebook, LlamaParse requires this to work
import nest_asyncio
nest_asyncio.apply()
# you can get a key at cloud.llamaindex.ai
os.environ["LLAMA_CLOUD_API_KEY"] = userdata.get("llama-cloud-key")
# set up parser
parser = LlamaParse(
result_type="markdown" # "markdown" and "text" are available
)
# use SimpleDirectoryReader to parse our file
file_extractor = {".pdf": parser}
documents2 = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()Started parsing the file under job_id 84cb91f7-45ec-4b99-8281-0f4beef6a892index2 = VectorStoreIndex.from_documents(documents2)
query_engine2 = index2.as_query_engine(similarity_top_k=20)response = query_engine2.query(
"What was the net gain in housing units in the Mission in 2021?"
)
print(response)The net gain in housing units in the Mission in 2021 was 1,305 units.使用更好的解析器后，LLM 能够正确回答该问题。
现在试一个更棘手的问题：
response = query_engine2.query(
"How many affordable housing units were completed in 2021?"
)
print(response)Repeat: 110LLM 开始混淆了；这似乎是住房单元的百分比增长。
尝试给 LLM 更多上下文（40 而不是 20），然后用重排序器对这些分块排序。为此使用 NVIDIA 的重排序器：
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
query_engine3 = index2.as_query_engine(
similarity_top_k=40, node_postprocessors=[NVIDIARerank(top_n=10)]
)response = query_engine3.query(
"How many affordable housing units were completed in 2021?"
)
print(response)1,495太好了！现在数字正确了（如果你想知道的话，这在第 35 页）。