主题
如果你在 Colab 中打开这个 Notebook,可能需要先安装 LlamaIndex 🦙。
%pip install llama-index-embeddings-huggingface
%pip install llama-index-vector-stores-awadb!pip install llama-index创建 Awadb 索引
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))加载文档,构建 VectorStoreIndex
from llama_index.core import (
SimpleDirectoryReader,
VectorStoreIndex,
StorageContext,
)
from IPython.display import Markdown, display
import openai
openai.api_key = ""INFO:numexpr.utils:Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
NumExpr defaulting to 8 threads.下载数据
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'加载数据
# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.vector_stores.awadb import AwaDBVectorStore
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
vector_store = AwaDBVectorStore()
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context, embed_model=embed_model
)查询索引
# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")display(Markdown(f"<b>{response}</b>"))成长过程中,作者写短篇小说,在 IBM 1401 上尝试编程,缠着父亲买一台 TRS-80 电脑,编写简单游戏、一个用于预测模型火箭飞行高度的程序以及一个文字处理器。他还在大学学习哲学,后来转向 AI,并致力于构建 Web 基础设施。他写文章并在网上发表,每周四晚上为朋友们举办晚餐聚会,画画,并在剑桥买了一栋楼。
# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine()
response = query_engine.query(
"What did the author do after his time at Y Combinator?"
)display(Markdown(f"<b>{response}</b>"))在 Y Combinator 工作之后,作者写文章、研究 Lisp 并画画。他还去俄勒冈州看望了母亲,并帮助她离开了疗养院。