Oracle AI Vector Search 专为人工智能（AI）工作负载设计，允许你基于语义而非关键字查询数据。Oracle AI Vector Search 的最大优势之一，是可以在单一系统中将非结构化数据的语义搜索与业务数据的关系搜索结合起来。这不仅强大，而且效果显著更好，因为无需添加专门的向量数据库，消除了多系统间数据碎片化的痛苦。

此外，你的向量还可以受益于 Oracle Database 最强大的各项功能，例如：

• 分区支持
• Real Application Clusters 可扩展性
• Exadata 智能扫描
• 跨地理分布式数据库的分片处理
• 事务
• 并行 SQL
• 灾难恢复
• 安全
• Oracle Machine Learning
• Oracle 图数据库
• Oracle Spatial and Graph
• Oracle 区块链
• JSON

本指南演示如何在 Oracle AI Vector Search 中使用向量能力。

如果你刚开始使用 Oracle Database，可以考虑探索免费的 Oracle 23 AI，它提供了设置数据库环境的良好入门。在使用数据库时，通常建议避免默认使用 system 用户；相反，你可以创建自己的用户，以增强安全性和可定制性。有关创建用户的详细步骤，请参阅我们的端到端指南，其中还展示了如何在 Oracle 中设置用户。此外，理解用户权限对于有效管理数据库安全至关重要。你可以在官方的 Oracle 指南 中了解更多关于管理用户账户和安全的内容。

前提条件

请安装 Oracle Python 客户端驱动，以便将 LlamaIndex 与 Oracle AI Vector Search 配合使用。

%pip install llama-index-vector-stores-oracledb

连接到 Oracle AI Vector Search

以下示例代码展示如何连接 Oracle Database。默认情况下，python-oracledb 以“Thin”模式运行，直接连接 Oracle Database。该模式不需要 Oracle Client 库。不过，当 python-oracledb 使用这些库时，可以使用一些额外功能。使用 Oracle Client 库时，python-oracledb 被称为处于“Thick”模式。两种模式都具备支持 Python Database API v2.0 规范的完整功能。请参阅以下指南，了解各模式支持的功能。如果无法使用 thin 模式，你可能需要切换到 thick 模式。

import oracledb

# please update with your username, password, hostname and service_name
username = "<username>"
password = "<password>"
dsn = "<hostname>/<service_name>"

try:
    connection = oracledb.connect(user=username, password=password, dsn=dsn)
    print("Connection successful!")
except Exception as ex:
    print("Exception occurred while index creation", ex)

导入使用 Oracle AI Vector Search 所需的依赖项

import sys
import os

from llama_index.core.schema import NodeRelationship, RelatedNodeInfo, TextNode
from llama_index.core.vector_stores.types import (
    ExactMatchFilter,
    MetadataFilters,
    VectorStoreQuery,
)

from llama_index.vector_stores.oracledb import base as orallamavs
from llama_index.vector_stores.oracledb import OraLlamaVS, DistanceStrategy

加载文档

# Define a list of documents (These dummy examples are 4 random documents )

text_json_list = [
    {
        "text": "If the answer to any preceding questions is yes, then the database stops the search and allocates space from the specified tablespace; otherwise, space is allocated from the database default shared temporary tablespace.",
        "id_": "cncpt_15.5.3.2.2_P4",
        "embedding": [1.0, 0.0],
        "relationships": "test-0",
        "metadata": {
            "weight": 1.0,
            "rank": "a",
            "url": "https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/logical-storage-structures.html#GUID-5387D7B2-C0CA-4C1E-811B-C7EB9B636442",
        },
    },
    {
        "text": "A tablespace can be online (accessible) or offline (not accessible) whenever the database is open.\nA tablespace is usually online so that its data is available to users. The SYSTEM tablespace and temporary tablespaces cannot be taken offline.",
        "id_": "cncpt_15.5.5_P1",
        "embedding": [0.0, 1.0],
        "relationships": "test-1",
        "metadata": {
            "weight": 2.0,
            "rank": "c",
            "url": "https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/logical-storage-structures.html#GUID-D02B2220-E6F5-40D9-AFB5-BC69BCEF6CD4",
        },
    },
    {
        "text": "The database stores LOBs differently from other data types. Creating a LOB column implicitly creates a LOB segment and a LOB index. The tablespace containing the LOB segment and LOB index, which are always stored together, may be different from the tablespace containing the table.\nSometimes the database can store small amounts of LOB data in the table itself rather than in a separate LOB segment.",
        "id_": "cncpt_22.3.4.3.1_P2",
        "embedding": [1.0, 1.0],
        "relationships": "test-2",
        "metadata": {
            "weight": 3.0,
            "rank": "d",
            "url": "https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/concepts-for-database-developers.html#GUID-3C50EAB8-FC39-4BB3-B680-4EACCE49E866",
        },
    },
    {
        "text": "The LOB segment stores data in pieces called chunks. A chunk is a logically contiguous set of data blocks and is the smallest unit of allocation for a LOB. A row in the table stores a pointer called a LOB locator, which points to the LOB index. When the table is queried, the database uses the LOB index to quickly locate the LOB chunks.",
        "id_": "cncpt_22.3.4.3.1_P3",
        "embedding": [2.0, 1.0],
        "relationships": "test-3",
        "metadata": {
            "weight": 4.0,
            "rank": "e",
            "url": "https://docs.oracle.com/en/database/oracle/oracle-database/23/cncpt/concepts-for-database-developers.html#GUID-3C50EAB8-FC39-4BB3-B680-4EACCE49E866",
        },
    },
]

# Create Llama Text Nodes
text_nodes = []
for text_json in text_json_list:
    # Construct the relationships using RelatedNodeInfo
    relationships = {
        NodeRelationship.SOURCE: RelatedNodeInfo(
            node_id=text_json["relationships"]
        )
    }

    # Prepare the metadata dictionary; you might want to exclude certain metadata fields if necessary
    metadata = {
        "weight": text_json["metadata"]["weight"],
        "rank": text_json["metadata"]["rank"],
    }

    # Create a TextNode instance
    text_node = TextNode(
        text=text_json["text"],
        id_=text_json["id_"],
        embedding=text_json["embedding"],
        relationships=relationships,
        metadata=metadata,
    )

    text_nodes.append(text_node)
print(text_nodes)

使用 AI 向量搜索创建一组采用不同距离策略的向量存储

首先，我们将创建三个向量存储，每个使用不同的距离函数。由于我们尚未在其中创建索引，它们目前只会创建表。稍后我们将使用这些向量存储创建 HNSW 索引。

你可以手动连接到 Oracle Database，你将看到三个表 Documents_DOT、Documents_COSINE 和 Documents_EUCLIDEAN。

然后，我们将再创建三个表 Documents_DOT_IVF、Documents_COSINE_IVF 和 Documents_EUCLIDEAN_IVF，用于在这些表上创建 IVF 索引而不是 HNSW 索引。

要了解 Oracle AI Vector Search 支持的不同类型索引的更多信息，请参阅以下指南

# Ingest documents into Oracle Vector Store using different distance strategies

vector_store_dot = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_DOT",
    client=connection,
    distance_strategy=DistanceStrategy.DOT_PRODUCT,
)
vector_store_max = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_COSINE",
    client=connection,
    distance_strategy=DistanceStrategy.COSINE,
)
vector_store_euclidean = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_EUCLIDEAN",
    client=connection,
    distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,
)

# Ingest documents into Oracle Vector Store using different distance strategies
vector_store_dot_ivf = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_DOT_IVF",
    client=connection,
    distance_strategy=DistanceStrategy.DOT_PRODUCT,
)
vector_store_max_ivf = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_COSINE_IVF",
    client=connection,
    distance_strategy=DistanceStrategy.COSINE,
)
vector_store_euclidean_ivf = OraLlamaVS.from_documents(
    text_nodes,
    table_name="Documents_EUCLIDEAN_IVF",
    client=connection,
    distance_strategy=DistanceStrategy.EUCLIDEAN_DISTANCE,
)

演示文本的添加、删除操作以及基本相似度搜索

def manage_texts(vector_stores):
    """
    Adds texts to each vector store, demonstrates error handling for duplicate additions,
    and performs deletion of texts. Showcases similarity searches and index creation for each vector store.

    Args:
    - vector_stores (list): A list of OracleVS instances.
    """
    for i, vs in enumerate(vector_stores, start=1):
        # Deleting texts using the value of 'id'
        vs.delete("test-1")
        print(f"\n\n\nDelete texts complete for vector store {i}\n\n\n")

        # Similarity search
        query = VectorStoreQuery(
            query_embedding=[1.0, 1.0], similarity_top_k=3
        )
        results = vs.query(query=query)
        print(
            f"\n\n\nSimilarity search results for vector store {i}: {results}\n\n\n"
        )

vector_store_list = [
    vector_store_dot,
    vector_store_max,
    vector_store_euclidean,
    vector_store_dot_ivf,
    vector_store_max_ivf,
    vector_store_euclidean_ivf,
]
manage_texts(vector_store_list)

演示为每种距离策略使用特定参数创建索引

def create_search_indices(connection):
    """
    Creates search indices for the vector stores, each with specific parameters tailored to their distance strategy.
    """
    # Index for DOT_PRODUCT strategy
    # Notice we are creating a HNSW index with default parameters
    # This will default to creating a HNSW index with 8 Parallel Workers and use the Default Accuracy used by Oracle AI Vector Search
    orallamavs.create_index(
        connection,
        vector_store_dot,
        params={"idx_name": "hnsw_idx1", "idx_type": "HNSW"},
    )

    # Index for COSINE strategy with specific parameters
    # Notice we are creating a HNSW index with parallel 16 and Target Accuracy Specification as 97 percent
    orallamavs.create_index(
        connection,
        vector_store_max,
        params={
            "idx_name": "hnsw_idx2",
            "idx_type": "HNSW",
            "accuracy": 97,
            "parallel": 16,
        },
    )

    # Index for EUCLIDEAN_DISTANCE strategy with specific parameters
    # Notice we are creating a HNSW index by specifying Power User Parameters which are neighbors = 64 and efConstruction = 100
    orallamavs.create_index(
        connection,
        vector_store_euclidean,
        params={
            "idx_name": "hnsw_idx3",
            "idx_type": "HNSW",
            "neighbors": 64,
            "efConstruction": 100,
        },
    )

    # Index for DOT_PRODUCT strategy with specific parameters
    # Notice we are creating an IVF index with default parameters
    # This will default to creating an IVF index with 8 Parallel Workers and use the Default Accuracy used by Oracle AI Vector Search
    orallamavs.create_index(
        connection,
        vector_store_dot_ivf,
        params={
            "idx_name": "ivf_idx1",
            "idx_type": "IVF",
        },
    )

    # Index for COSINE strategy with specific parameters
    # Notice we are creating an IVF index with parallel 32 and Target Accuracy Specification as 90 percent
    orallamavs.create_index(
        connection,
        vector_store_max_ivf,
        params={
            "idx_name": "ivf_idx2",
            "idx_type": "IVF",
            "accuracy": 90,
            "parallel": 32,
        },
    )

    # Index for EUCLIDEAN_DISTANCE strategy with specific parameters
    # Notice we are creating an IVF index by specifying Power User Parameters which is neighbor_part = 64
    orallamavs.create_index(
        connection,
        vector_store_euclidean_ivf,
        params={
            "idx_name": "ivf_idx3",
            "idx_type": "IVF",
            "neighbor_part": 64,
        },
    )

    print("Index creation complete.")

create_search_indices(connection)

现在，我们将对所有六个向量存储执行一系列高级搜索。这三种搜索各有带过滤器和无过滤器两个版本。过滤器仅选出 id 为 101 的文档，并过滤掉其他所有内容

# Conduct advanced searches after creating the indices
def conduct_advanced_searches(vector_stores):
    # Constructing a filter for direct comparison against document metadata
    # This filter aims to include documents whose metadata 'id' is exactly '2'

    for i, vs in enumerate(vector_stores, start=1):

        def query_without_filters_returns_all_rows_sorted_by_similarity():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search without a filter
            print("\nSimilarity search results without filter:")
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], similarity_top_k=3
            )
            print(vs.query(query=query))

        query_without_filters_returns_all_rows_sorted_by_similarity()

        def query_with_filters_returns_multiple_matches():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search with filter
            print("\nSimilarity search results without filter:")
            filters = MetadataFilters(
                filters=[ExactMatchFilter(key="rank", value="c")]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], filters=filters, similarity_top_k=1
            )
            result = vs.query(query)
            print(result.ids)

        query_with_filters_returns_multiple_matches()

        def query_with_filter_applies_top_k():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search with a filter
            print("\nSimilarity search results with filter:")
            filters = MetadataFilters(
                filters=[ExactMatchFilter(key="rank", value="c")]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], filters=filters, similarity_top_k=1
            )
            result = vs.query(query)
            print(result.ids)

        query_with_filter_applies_top_k()

        def query_with_filter_applies_node_id_filter():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search with a filter
            print("\nSimilarity search results with filter:")
            filters = MetadataFilters(
                filters=[ExactMatchFilter(key="rank", value="c")]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0],
                filters=filters,
                similarity_top_k=3,
                node_ids=["452D24AB-F185-414C-A352-590B4B9EE51B"],
            )
            result = vs.query(query)
            print(result.ids)

        query_with_filter_applies_node_id_filter()

        def query_with_exact_filters_returns_single_match():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search with a filter
            print("\nSimilarity search results with filter:")
            filters = MetadataFilters(
                filters=[
                    ExactMatchFilter(key="rank", value="c"),
                    ExactMatchFilter(key="weight", value=2),
                ]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], filters=filters
            )
            result = vs.query(query)
            print(result.ids)

        query_with_exact_filters_returns_single_match()

        def query_with_contradictive_filter_returns_no_matches():
            filters = MetadataFilters(
                filters=[
                    ExactMatchFilter(key="weight", value=2),
                    ExactMatchFilter(key="weight", value=3),
                ]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], filters=filters
            )
            result = vs.query(query)
            print(result.ids)

        query_with_contradictive_filter_returns_no_matches()

        def query_with_filter_on_unknown_field_returns_no_matches():
            print(f"\n--- Vector Store {i} Advanced Searches ---")
            # Similarity search with a filter
            print("\nSimilarity search results with filter:")
            filters = MetadataFilters(
                filters=[ExactMatchFilter(key="unknown_field", value="c")]
            )
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], filters=filters
            )
            result = vs.query(query)
            print(result.ids)

        query_with_filter_on_unknown_field_returns_no_matches()

        def delete_removes_document_from_query_results():
            vs.delete("test-1")
            query = VectorStoreQuery(
                query_embedding=[1.0, 1.0], similarity_top_k=2
            )
            result = vs.query(query)
            print(result.ids)

        delete_removes_document_from_query_results()

conduct_advanced_searches(vector_store_list)

端到端演示

请参阅我们的完整演示指南 Oracle AI Vector Search 端到端演示指南，借助 Oracle AI Vector Search 构建端到端 RAG 流水线。