主题
在本 notebook 中，我们将快速演示如何将 Dragonfly 与向量存储一起使用。
如果你在 Colab 中打开这个 Notebook，可能需要先安装 LlamaIndex 🦙。
%pip install -U llama-index llama-index-vector-stores-redis llama-index-embeddings-cohere llama-index-embeddings-openai1
import os
import getpass
import sys
import logging
import textwrap
import warnings
warnings.filterwarnings("ignore")
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.redis import RedisVectorStore1
2
3
4
5
6
7
8
9
10
11
12
13
2
3
4
5
6
7
8
9
10
11
12
13
启动 Dragonfly
启动 Dragonfly 最简单的方法是使用 Dragonfly Docker 镜像，或快速注册一个 Dragonfly Cloud 演示实例。
要遵循本教程的每一步，请按如下方式启动镜像：
终端窗口
docker run -d -p 6379:6379 --name dragonfly docker.dragonflydb.io/dragonflydb/dragonfly1
设置 OpenAI
首先，让我们添加 openai api key。这将使我们能够访问 openai 以进行嵌入并使用 chatgpt。
oai_api_key = getpass.getpass("OpenAI API Key:")
os.environ["OPENAI_API_KEY"] = oai_api_key1
2
2
下载数据
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'1
2
2
--2025-06-30 14:41:20-- https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 75042 (73K) [text/plain]
Saving to: ‘data/paul_graham/paul_graham_essay.txt’
data/paul_graham/pa 100%[===================>] 73.28K --.-KB/s in 0.04s
2025-06-30 14:41:20 (2.00 MB/s) - ‘data/paul_graham/paul_graham_essay.txt’ saved [75042/75042]1
2
3
4
5
6
7
8
9
10
2
3
4
5
6
7
8
9
10
读取数据集
在这里，我们将使用一组 Paul Graham 的文章作为文本，将其转换为嵌入，存储在向量存储中，并进行查询，为我们的 LLM 问答循环寻找上下文。
# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
print(
"Document ID:",
documents[0].id_,
"Document Filename:",
documents[0].metadata["file_name"],
)1
2
3
4
5
6
7
8
2
3
4
5
6
7
8
Document ID: a5cae17c-27eb-411e-8967-fb6ef98bcdcf Document Filename: paul_graham_essay.txt1
初始化默认向量存储
现在文档已准备好，我们可以使用默认设置初始化向量存储。这将使我们能够将向量存储在 Dragonfly 中，并创建用于实时搜索的索引。
from llama_index.core import StorageContext
from redis import Redis
# create a client connection
redis_client = Redis.from_url("redis://localhost:6379")
# create the vector store wrapper
vector_store = RedisVectorStore(redis_client=redis_client, overwrite=True)
# load storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# build and load index from documents and storage context
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context
)1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
14:41:29 llama_index.vector_stores.redis.base INFO Using default RedisVectorStore schema.
14:41:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
14:41:31 llama_index.vector_stores.redis.base INFO Added 22 documents to index llama_index1
2
3
2
3
查询默认向量存储
现在数据已存储在索引中，我们可以针对索引提问了。
该索引将把这些数据用作 LLM 的知识库。as\_query\_engine() 的默认设置使用 OpenAI 嵌入和 GPT 作为语言模型。因此，除非你选择自定义或本地语言模型，否则需要一个 OpenAI 密钥。
下面我们将测试针对索引的搜索，然后使用 LLM 进行完整的 RAG。
query_engine = index.as_query_engine()
retriever = index.as_retriever()1
2
2
result_nodes = retriever.retrieve("What did the author learn?")
for node in result_nodes:
print(node)1
2
3
2
3
14:41:40 httpx INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
14:41:40 llama_index.vector_stores.redis.base INFO Querying index llama_index with query *=>[KNN 2 @vector $vector AS vector_distance] RETURN 5 id doc_id text _node_content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 2
14:41:40 llama_index.vector_stores.redis.base INFO Found 2 results for query with id ['llama_index/vector_f12d31cc-d154-4ae2-9511-81a1e0b2c185', 'llama_index/vector_a67c3af9-14cc-45fd-a2dd-142753a61d79']
Node ID: f12d31cc-d154-4ae2-9511-81a1e0b2c185
Text: What I Worked On February 2021 Before college the two main
things I worked on, outside of school, were writing and programming. I
didn't write essays. I wrote what beginning writers were supposed to
write then, and probably still are: short stories. My stories were
awful. They had hardly any plot, just characters with strong feelings,
which I ...
Score: 0.819
Node ID: a67c3af9-14cc-45fd-a2dd-142753a61d79
Text: In the summer of 2016 we moved to England. We wanted our kids to
see what it was like living in another country, and since I was a
British citizen by birth, that seemed the obvious choice. We only
meant to stay for a year, but we liked it so much that we still live
there. So most of Bel was written in England. In the fall of 2019,
Bel was final...
Score: 0.8151
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
response = query_engine.query("What did the author learn?")
print(textwrap.fill(str(response), 100))1
2
2
14:41:44 httpx INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
14:41:44 llama_index.vector_stores.redis.base INFO Querying index llama_index with query *=>[KNN 2 @vector $vector AS vector_distance] RETURN 5 id doc_id text _node_content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 2
14:41:44 llama_index.vector_stores.redis.base INFO Found 2 results for query with id ['llama_index/vector_f12d31cc-d154-4ae2-9511-81a1e0b2c185', 'llama_index/vector_a67c3af9-14cc-45fd-a2dd-142753a61d79']
14:41:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
The author learned that philosophy courses in college were boring to him, leading him to switch his
focus to studying AI.1
2
3
4
5
6
2
3
4
5
6
result_nodes = retriever.retrieve("What was a hard moment for the author?")
for node in result_nodes:
print(node)1
2
3
2
3
14:41:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
14:41:47 llama_index.vector_stores.redis.base INFO Querying index llama_index with query *=>[KNN 2 @vector $vector AS vector_distance] RETURN 5 id doc_id text _node_content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 2
14:41:47 llama_index.vector_stores.redis.base INFO Found 2 results for query with id ['llama_index/vector_8c02f420-3cfc-4da6-859b-97469872ef46', 'llama_index/vector_f12d31cc-d154-4ae2-9511-81a1e0b2c185']
Node ID: 8c02f420-3cfc-4da6-859b-97469872ef46
Text: HN was no doubt good for YC, but it was also by far the biggest
source of stress for me. If all I'd had to do was select and help
founders, life would have been so easy. And that implies that HN was a
mistake. Surely the biggest source of stress in one's work should at
least be something close to the core of the work. Whereas I was like
someone ...
Score: 0.804
Node ID: f12d31cc-d154-4ae2-9511-81a1e0b2c185
Text: What I Worked On February 2021 Before college the two main
things I worked on, outside of school, were writing and programming. I
didn't write essays. I wrote what beginning writers were supposed to
write then, and probably still are: short stories. My stories were
awful. They had hardly any plot, just characters with strong feelings,
which I ...
Score: 0.8021
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
response = query_engine.query("What was a hard moment for the author?")
print(textwrap.fill(str(response), 100))1
2
2
14:41:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
14:41:51 llama_index.vector_stores.redis.base INFO Querying index llama_index with query *=>[KNN 2 @vector $vector AS vector_distance] RETURN 5 id doc_id text _node_content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 2
14:41:51 llama_index.vector_stores.redis.base INFO Found 2 results for query with id ['llama_index/vector_8c02f420-3cfc-4da6-859b-97469872ef46', 'llama_index/vector_f12d31cc-d154-4ae2-9511-81a1e0b2c185']
14:41:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
Dealing with urgent problems related to Hacker News (HN) was a significant source of stress for the
author.1
2
3
4
5
6
2
3
4
5
6
index.vector_store.delete_index()1
14:41:55 llama_index.vector_stores.redis.base INFO Deleting index llama_index1
使用自定义索引 schema
在大多数用例中，你需要能够自定义底层索引的配置和规范。例如，当你希望启用特定的元数据过滤器时，这会非常方便。
使用 Dragonfly 时，这非常简单：只需定义一个索引 schema 对象（来自文件或 dict），并将其传给向量存储客户端封装。
在本示例中，我们将：
- 将嵌入模型切换为 Cohere
- 为文档的
updated_at时间戳添加一个额外的元数据字段 - 为现有的
file_name元数据字段建立索引
from llama_index.core.settings import Settings
from llama_index.embeddings.cohere import CohereEmbedding
# set up Cohere Key
co_api_key = getpass.getpass("Cohere API Key:")
Settings.embed_model = CohereEmbedding(api_key=co_api_key)1
2
3
4
5
6
7
2
3
4
5
6
7
from redisvl.schema import IndexSchema
custom_schema = IndexSchema.from_dict(
{
# customize basic index specs
"index": {
"name": "paul_graham",
"prefix": "essay",
"key_separator": ":",
},
# customize fields that are indexed
"fields": [
# required fields for llamaindex
{"type": "tag", "name": "id"},
{"type": "tag", "name": "doc_id"},
{"type": "text", "name": "text"},
# custom metadata fields
{"type": "numeric", "name": "updated_at"},
{"type": "tag", "name": "file_name"},
# custom vector field definition for cohere embeddings
{
"type": "vector",
"name": "vector",
"attrs": {
"dims": 1024,
"algorithm": "hnsw",
"distance_metric": "cosine",
},
},
],
}
)1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
custom_schema.index1
IndexInfo(name='paul_graham', prefix='essay', key_separator=':', storage_type=<StorageType.HASH: 'hash'>)1
custom_schema.fields1
{'id': TagField(name='id', type=<FieldTypes.TAG: 'tag'>, path=None, attrs=TagFieldAttributes(sortable=False, separator=',', case_sensitive=False, withsuffixtrie=False)),
'doc_id': TagField(name='doc_id', type=<FieldTypes.TAG: 'tag'>, path=None, attrs=TagFieldAttributes(sortable=False, separator=',', case_sensitive=False, withsuffixtrie=False)),
'text': TextField(name='text', type=<FieldTypes.TEXT: 'text'>, path=None, attrs=TextFieldAttributes(sortable=False, weight=1, no_stem=False, withsuffixtrie=False, phonetic_matcher=None)),
'updated_at': NumericField(name='updated_at', type=<FieldTypes.NUMERIC: 'numeric'>, path=None, attrs=NumericFieldAttributes(sortable=False)),
'file_name': TagField(name='file_name', type=<FieldTypes.TAG: 'tag'>, path=None, attrs=TagFieldAttributes(sortable=False, separator=',', case_sensitive=False, withsuffixtrie=False)),
'vector': HNSWVectorField(name='vector', type='vector', path=None, attrs=HNSWVectorFieldAttributes(dims=1024, algorithm=<VectorIndexAlgorithm.HNSW: 'HNSW'>, datatype=<VectorDataType.FLOAT32: 'FLOAT32'>, distance_metric=<VectorDistanceMetric.COSINE: 'COSINE'>, initial_cap=None, m=16, ef_construction=200, ef_runtime=10, epsilon=0.01))}1
2
3
4
5
6
2
3
4
5
6
from datetime import datetime
def date_to_timestamp(date_string: str) -> int:
date_format: str = "%Y-%m-%d"
return int(datetime.strptime(date_string, date_format).timestamp())
# iterate through documents and add new field
for document in documents:
document.metadata["updated_at"] = date_to_timestamp(
document.metadata["last_modified_date"]
)1
2
3
4
5
6
7
8
9
10
11
2
3
4
5
6
7
8
9
10
11
vector_store = RedisVectorStore(
schema=custom_schema, # provide customized schema
redis_client=redis_client,
overwrite=True,
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# build and load index from documents and storage context
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context
)1
2
3
4
5
6
7
8
9
10
11
12
2
3
4
5
6
7
8
9
10
11
12
14:42:26 httpx INFO HTTP Request: POST https://api.cohere.com/v2/embed "HTTP/1.1 200 OK"
14:42:26 httpx INFO HTTP Request: POST https://api.cohere.com/v2/embed "HTTP/1.1 200 OK"
14:42:27 httpx INFO HTTP Request: POST https://api.cohere.com/v2/embed "HTTP/1.1 200 OK"
14:42:27 llama_index.vector_stores.redis.base INFO Added 22 documents to index paul_graham1
2
3
4
2
3
4
查询向量存储并按元数据过滤
现在我们已经将额外的元数据索引到 Dragonfly 中，接下来尝试一些带过滤器的查询。
from llama_index.core.vector_stores import (
MetadataFilters,
MetadataFilter,
ExactMatchFilter,
)
retriever = index.as_retriever(
similarity_top_k=3,
filters=MetadataFilters(
filters=[
ExactMatchFilter(key="file_name", value="paul_graham_essay.txt"),
MetadataFilter(
key="updated_at",
value=date_to_timestamp("2023-01-01"),
operator=">=",
),
MetadataFilter(
key="text",
value="learn",
operator="text_match",
),
],
condition="and",
),
)1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
result_nodes = retriever.retrieve("What did the author learn?")
for node in result_nodes:
print(node)1
2
3
4
2
3
4
14:42:37 httpx INFO HTTP Request: POST https://api.cohere.com/v2/embed "HTTP/1.1 200 OK"
14:42:37 llama_index.vector_stores.redis.base INFO Querying index paul_graham with query ((@file_name:{paul_graham_essay\.txt} @updated_at:[1672524000 +inf]) @text:(learn))=>[KNN 3 @vector $vector AS vector_distance] RETURN 5 id doc_id text _node_content vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 3
14:42:37 llama_index.vector_stores.redis.base INFO Found 3 results for query with id ['essay:30148f62-13c6-4edb-b09f-1cf3054c5c98', 'essay:054f9488-83c7-4bf6-a408-9ef17eea0446', 'essay:608adb71-a995-489d-81dc-0deab7bbe656']
Node ID: 30148f62-13c6-4edb-b09f-1cf3054c5c98
Text: If he even knew about the strange classes I was taking, he never
said anything. So now I was in a PhD program in computer science, yet
planning to be an artist, yet also genuinely in love with Lisp hacking
and working away at On Lisp. In other words, like many a grad student,
I was working energetically on multiple projects that were not my
the...
Score: 0.404
Node ID: 054f9488-83c7-4bf6-a408-9ef17eea0446
Text: I wanted to go back to RISD, but I was now broke and RISD was
very expensive, so I decided to get a job for a year and then return
to RISD the next fall. I got one at a company called Interleaf, which
made software for creating documents. You mean like Microsoft Word?
Exactly. That was how I learned that low end software tends to eat
high end so...
Score: 0.396
Node ID: 608adb71-a995-489d-81dc-0deab7bbe656
Text: All that seemed left for philosophy were edge cases that people
in other fields felt could safely be ignored. I couldn't have put
this into words when I was 18. All I knew at the time was that I kept
taking philosophy courses and they kept being boring. So I decided to
switch to AI. AI was in the air in the mid 1980s, but there were two
things...
Score: 0.3941
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
删除文档或整个索引
有时删除文档或整个索引可能会很有用。这可以通过 delete 和 delete_index 方法完成。
document_id = documents[0].doc_id
document_id1
2
2
print("Number of documents before deleting", redis_client.dbsize())
vector_store.delete(document_id)
print("Number of documents after deleting", redis_client.dbsize())1
2
3
2
3
但是，索引仍然存在（只是没有关联的文档）。
vector_store.index_exists()1
# now lets delete the index entirely
# this will delete all the documents and the index
vector_store.delete_index()1
2
3
2
3
print("Number of documents after deleting", redis_client.dbsize())1