本 notebook 演示如何将 Vertex AI Vector Search v2.0 与 LlamaIndex 配合使用。

Vertex AI Vector Search v2.0 引入了简化的基于集合的架构，无需单独创建索引和部署端点。

v2.0 与 v1.0

特性	v1.0	v2.0
架构	索引 + 端点	集合
设置步骤	创建索引 → 部署到端点	创建集合
GCS 存储桶	批量更新时需要	不需要
混合搜索	不支持	支持

注意：有关 v1.0 的用法，请参阅 VertexAIVectorSearchDemo.ipynb

安装依赖项

安装支持 v2 的 LlamaIndex：

注意：V2 支持需要 llama-index-vector-stores-vertexaivectorsearch 版本支持 Vertex AI Vector Search v2.0 API。

# Install with v2 support (the [v2] extra installs google-cloud-vectorsearch)
# !pip install 'llama-index-vector-stores-vertexaivectorsearch[v2]' llama-index-embeddings-vertex llama-index-llms-vertex

身份验证（如果使用 Colab）：

# Colab authentication.
import sys

if "google.colab" in sys.modules:
    from google.colab import auth

    auth.authenticate_user()
    print("Authenticated")

配置

设置你的 Google Cloud 项目详细信息：

# Google Cloud Configuration
PROJECT_ID = "your-project-id"  # @param {type:"string"}
REGION = "us-central1"  # @param {type:"string"}
COLLECTION_ID = "llamaindex-demo-collection"  # @param {type:"string"}

# Embedding dimensions (768 for text-embedding-004)
EMBEDDING_DIMENSION = 768

创建 v2 集合

与需要创建索引并将其部署到端点的 v1.0 不同，v2.0 只需创建一个集合。

混合搜索的重要说明： 下面的集合 schema 已配置为支持所有混合搜索功能：

• 文本搜索：字符串字段（text、category、color）可以使用关键词搜索
• 语义搜索：vertex_embedding_config 为 SEMANTIC_HYBRID 模式启用自动嵌入
• 过滤：数值（price）和字符串字段支持元数据过滤

from google.cloud import vectorsearch_v1beta

# Initialize the client
client = vectorsearch_v1beta.VectorSearchServiceClient()

# Check if collection already exists
parent = f"projects/{PROJECT_ID}/locations/{REGION}"
collection_name = f"{parent}/collections/{COLLECTION_ID}"

# Collection schema that supports:
# - Dense vector search (embedding field)
# - Text search on text, category, color fields (for HYBRID mode)
# - Semantic search with auto-embeddings (for SEMANTIC_HYBRID mode)
# - Filtering on price, category, color fields
collection_config = {
    "data_schema": {
        "type": "object",
        "properties": {
            "text": {"type": "string"},  # Main text content (searchable)
            "ref_doc_id": {"type": "string"},  # Document reference
            "price": {"type": "number"},  # Filterable numeric field
            "color": {"type": "string"},  # Filterable/searchable string
            "category": {"type": "string"},  # Filterable/searchable string
        },
    },
    "vector_schema": {
        "embedding": {
            "dense_vector": {
                "dimensions": EMBEDDING_DIMENSION,
                # Auto-embedding config enables SEMANTIC_HYBRID mode
                # Vertex AI will auto-generate embeddings for semantic search
                "vertex_embedding_config": {
                    "model_id": "text-embedding-004",
                    "text_template": "{text}",
                    "task_type": "RETRIEVAL_DOCUMENT",
                },
            }
        },
    },
}

try:
    request = vectorsearch_v1beta.GetCollectionRequest(name=collection_name)
    collection = client.get_collection(request=request)
    print(f"Collection already exists: {collection.name}")
    print(
        "Note: If you need hybrid search features, delete and recreate with the schema above."
    )
except Exception as e:
    if "404" in str(e) or "NotFound" in str(e):
        print(f"Creating collection: {COLLECTION_ID}")
        print(
            "Schema includes: text search fields, auto-embedding config for semantic search"
        )

        request = vectorsearch_v1beta.CreateCollectionRequest(
            parent=parent,
            collection_id=COLLECTION_ID,
            collection=collection_config,
        )
        operation = client.create_collection(request=request)
        collection = operation.result()
        print(f"Collection created: {collection.name}")
    else:
        raise e

设置 LlamaIndex 组件

# Imports
from llama_index.core import Settings, StorageContext, VectorStoreIndex
from llama_index.core.schema import TextNode
from llama_index.core.vector_stores.types import (
    MetadataFilters,
    MetadataFilter,
    FilterOperator,
)
from llama_index.embeddings.vertex import VertexTextEmbedding
from llama_index.llms.vertex import Vertex
from llama_index.vector_stores.vertexaivectorsearch import VertexAIVectorStore

# Authentication - get default credentials
import google.auth

credentials, project = google.auth.default()
print(f"Authenticated with project: {project}")

# Configure embedding model
embed_model = VertexTextEmbedding(
    model_name="text-embedding-004",
    project=PROJECT_ID,
    location=REGION,
    credentials=credentials,
)

# Configure LLM
llm = Vertex(
    model="gemini-2.0-flash",
    project=PROJECT_ID,
    location=REGION,
    credentials=credentials,
)

# Set as defaults
Settings.embed_model = embed_model
Settings.llm = llm

print("Embedding model and LLM configured successfully!")

创建 v2 向量存储

创建 v2 向量存储很简单——只需指定 api_version="v2" 和你的 collection_id：

# Create v2 vector store
vector_store = VertexAIVectorStore(
    api_version="v2",  # Use v2 API
    project_id=PROJECT_ID,
    region=REGION,
    collection_id=COLLECTION_ID,
    # No index_id, endpoint_id, or gcs_bucket_name needed!
)

print(f"Vector store created with api_version={vector_store.api_version}")

添加文档

简单文本节点

# Create some sample text nodes
texts = [
    "LlamaIndex is a data framework for LLM applications.",
    "Vertex AI Vector Search provides scalable vector similarity search.",
    "RAG combines retrieval with generation for better AI responses.",
    "Embeddings convert text into numerical vectors for similarity matching.",
]

# Create nodes with embeddings
nodes = [
    TextNode(text=text, embedding=embed_model.get_text_embedding(text))
    for text in texts
]

# Add to vector store
ids = vector_store.add(nodes)
print(f"Added {len(ids)} nodes to vector store")

带元数据的节点

添加带元数据的节点，以用于过滤和混合搜索：

# Sample product data with metadata
products = [
    {
        "text": "Comfortable blue cotton t-shirt, perfect for casual wear",
        "color": "blue",
        "category": "tops",
        "price": 29.99,
    },
    {
        "text": "Professional black dress pants for office meetings",
        "color": "black",
        "category": "bottoms",
        "price": 79.99,
    },
    {
        "text": "Warm green wool sweater for cold winter days",
        "color": "green",
        "category": "tops",
        "price": 59.99,
    },
    {
        "text": "Lightweight blue running shorts with pockets",
        "color": "blue",
        "category": "bottoms",
        "price": 34.99,
    },
    {
        "text": "Elegant red silk blouse for formal occasions",
        "color": "red",
        "category": "tops",
        "price": 89.99,
    },
]

# Create nodes with metadata
# Note: Include "text" in metadata for TEXT_SEARCH to work on the text field
product_nodes = [
    TextNode(
        text=p["text"],
        embedding=embed_model.get_text_embedding(p["text"]),
        metadata={
            "text": p["text"],
            "color": p["color"],
            "category": p["category"],
            "price": p["price"],
        },
    )
    for p in products
]

# Add to vector store
product_ids = vector_store.add(product_nodes)
print(f"Added {len(product_ids)} product nodes with metadata")

查询向量存储

简单相似度搜索

# Create index from vector store
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_vector_store(
    vector_store=vector_store, embed_model=embed_model
)

# Create retriever
retriever = index.as_retriever(similarity_top_k=3)

# Query
results = retriever.retrieve("comfortable clothing for everyday wear")

print("Search Results:")
print("-" * 60)
for result in results:
    print(f"Score: {result.get_score():.3f}")
    print(f"Text: {result.get_text()[:100]}...")
    print(f"Metadata: {result.metadata}")
    print("-" * 60)

使用元数据过滤器搜索

# Filter by color
filters = MetadataFilters(filters=[MetadataFilter(key="color", value="blue")])

retriever = index.as_retriever(filters=filters, similarity_top_k=3)
results = retriever.retrieve("casual clothing")

print("Blue items only:")
print("-" * 60)
for result in results:
    print(
        f"Score: {result.get_score():.3f} | Color: {result.metadata.get('color')}"
    )
    print(f"Text: {result.get_text()[:80]}...")
    print("-" * 60)

# Filter by price range
filters = MetadataFilters(
    filters=[
        MetadataFilter(key="price", operator=FilterOperator.LT, value=50.0),
    ]
)

retriever = index.as_retriever(filters=filters, similarity_top_k=3)
results = retriever.retrieve("clothing")

print("Items under $50:")
print("-" * 60)
for result in results:
    print(
        f"Score: {result.get_score():.3f} | Price: ${result.metadata.get('price')}"
    )
    print(f"Text: {result.get_text()[:80]}...")
    print("-" * 60)

使用 LLM 进行 RAG 查询

将向量存储与 LLM 配合使用，进行检索增强生成：

# Create query engine
query_engine = index.as_query_engine(similarity_top_k=3)

# Ask a question
response = query_engine.query(
    "What blue clothing items do you have and what are their prices?"
)

print(
    "Question: What blue clothing items do you have and what are their prices?"
)
print("-" * 60)
print(f"Answer: {response.response}")
print("-" * 60)
print("Sources:")
for node in response.source_nodes:
    print(f"  - {node.text[:60]}... (score: {node.score:.3f})")

仅 v2 支持的功能

删除特定节点

# Delete specific nodes by ID
# vector_store.delete_nodes(node_ids=["node_id_1", "node_id_2"])
print("delete_nodes() - Delete specific nodes by their IDs")

清除所有数据（仅 v2）

v2 支持清除集合中的所有数据——这在 v1 中不可用：

# Clear all data from the collection
# WARNING: This deletes ALL data in the collection!
# vector_store.clear()
print("clear() - Clears all data from collection (v2 only!)")

混合搜索（仅 v2）

v2 支持混合搜索，将向量相似度与基于文本的搜索相结合，以提高检索质量。当你希望同时利用语义理解（向量）和精确关键词匹配（文本搜索）时，这尤其有用。

支持的查询模式

模式	说明	用例
DEFAULT	稠密向量相似度搜索	标准语义搜索
TEXT_SEARCH	全文关键词搜索	精确关键词匹配
HYBRID	向量 + 文本，使用 RRF 融合	兼顾两者优势
SEMANTIC_HYBRID	向量 + 语义搜索	自动嵌入语义搜索

排序器选项

排序器	说明
rrf（默认）	带 alpha 权重的倒数排名融合
vertex	通过 Vertex AI 提供 AI 驱动的语义排序

创建启用混合搜索的向量存储

要启用混合搜索，请设置 enable_hybrid=True，并指定用于文本搜索的字段：

# Create hybrid-enabled vector store
hybrid_vector_store = VertexAIVectorStore(
    api_version="v2",
    project_id=PROJECT_ID,
    region=REGION,
    collection_id=COLLECTION_ID,
    enable_hybrid=True,
    text_search_fields=["text", "category", "color"],  # Fields for text search
    default_hybrid_alpha=0.5,  # Balance between vector and text (0=text only, 1=vector only)
)

print(
    f"Hybrid vector store created with enable_hybrid={hybrid_vector_store.enable_hybrid}"
)

TEXT_SEARCH 模式——关键词搜索

使用 TEXT_SEARCH 模式进行全文关键词匹配。当你需要精确关键词匹配而非语义相似度时，这非常有用：

from llama_index.core.vector_stores.types import (
    VectorStoreQuery,
    VectorStoreQueryMode,
)

# TEXT_SEARCH: Full-text keyword search only
text_query = VectorStoreQuery(
    query_str="blue cotton",  # Keywords to search for
    mode=VectorStoreQueryMode.TEXT_SEARCH,
    similarity_top_k=3,
)

results = hybrid_vector_store.query(text_query)

print("TEXT_SEARCH Results (keyword matching):")
print("-" * 60)
for i, node in enumerate(results.nodes):
    print(f"{i+1}. {node.text[:80]}...")
    print(f"   Metadata: {node.metadata}")
    print()

HYBRID 模式——向量 + 文本搜索

使用 HYBRID 模式将向量相似度与文本搜索相结合。alpha 参数控制平衡：

• alpha=1.0——纯向量搜索
• alpha=0.5——平衡
• alpha=0.0——纯文本搜索

# Compare different alpha values
print("Comparing alpha values:")
print("=" * 60)

for alpha in [1.0, 0.5, 0.0]:
    query = VectorStoreQuery(
        query_embedding=embed_model.get_query_embedding(
            "warm winter clothing"
        ),
        query_str="sweater green",  # Keywords favor green sweater
        mode=VectorStoreQueryMode.HYBRID,
        alpha=alpha,
        similarity_top_k=2,
    )
    results = hybrid_vector_store.query(query)

    label = (
        "vector only"
        if alpha == 1.0
        else "text only"
        if alpha == 0.0
        else "balanced"
    )
    print(f"\nalpha={alpha} ({label}):")
    for node in results.nodes:
        print(f"  - {node.text[:60]}... | color={node.metadata.get('color')}")

SEMANTIC_HYBRID 模式

SEMANTIC_HYBRID 将你的稠密向量搜索与 Vertex AI 内置的语义搜索相结合。这要求集合已配置 vertex_embedding_config 以进行自动嵌入。

# SEMANTIC_HYBRID: Vector + Vertex AI's semantic search
# Note: Requires collection with vertex_embedding_config

semantic_query = VectorStoreQuery(
    query_embedding=embed_model.get_query_embedding(
        "professional work attire"
    ),
    query_str="formal office clothing",  # Semantic search text
    mode=VectorStoreQueryMode.SEMANTIC_HYBRID,
    similarity_top_k=3,
)

try:
    results = hybrid_vector_store.query(semantic_query)
    print("SEMANTIC_HYBRID Results:")
    print("-" * 60)
    for i, node in enumerate(results.nodes):
        print(f"{i+1}. {node.text[:80]}...")
except Exception as e:
    print(
        f"SEMANTIC_HYBRID requires collection with vertex_embedding_config: {e}"
    )

使用 VertexRanker

除了 RRF，你还可以使用 Vertex AI 的语义排序器进行 AI 驱动的结果排序：

# Create vector store with VertexRanker
vertex_ranker_store = VertexAIVectorStore(
    api_version="v2",
    project_id=PROJECT_ID,
    region=REGION,
    collection_id=COLLECTION_ID,
    enable_hybrid=True,
    text_search_fields=["text", "category"],
    hybrid_ranker="vertex",  # Use Vertex AI's semantic ranker
    vertex_ranker_model="semantic-ranker-default@latest",
    vertex_ranker_title_field="category",  # Field to use as title
    vertex_ranker_content_field="text",  # Field to use as content
)

# Query with VertexRanker
query = VectorStoreQuery(
    query_embedding=embed_model.get_query_embedding("casual everyday wear"),
    query_str="comfortable shirt",
    mode=VectorStoreQueryMode.HYBRID,
    similarity_top_k=3,
)

results = vertex_ranker_store.query(query)

print("HYBRID with VertexRanker Results:")
print("-" * 60)
for i, node in enumerate(results.nodes):
    print(f"{i+1}. {node.text[:80]}...")
    print(f"   Category: {node.metadata.get('category')}")
    print()

使用查询引擎进行混合搜索（RAG）

你还可以将混合搜索与标准 LlamaIndex 查询引擎配合使用，以构建 RAG 应用：

# Create index with hybrid-enabled vector store
hybrid_storage_context = StorageContext.from_defaults(
    vector_store=hybrid_vector_store
)
hybrid_index = VectorStoreIndex.from_vector_store(
    vector_store=hybrid_vector_store,
    embed_model=embed_model,
)

# Create query engine with hybrid mode
hybrid_query_engine = hybrid_index.as_query_engine(
    vector_store_query_mode=VectorStoreQueryMode.HYBRID,
    similarity_top_k=3,
    alpha=0.7,  # Favor vector search slightly
)

# Ask a question using RAG with hybrid retrieval
response = hybrid_query_engine.query(
    "What comfortable blue clothing options are available?"
)

print("Question: What comfortable blue clothing options are available?")
print("-" * 60)
print(f"Answer: {response.response}")
print("-" * 60)
print("Sources:")
for node in response.source_nodes:
    print(f"  - {node.text[:60]}... (score: {node.score:.3f})")

混合搜索参数参考

参数	类型	默认值	说明
enable_hybrid	bool	False	启用混合搜索模式
text_search_fields	List[str]	None	用于文本搜索的数据字段
embedding_field	str	"embedding"	向量字段名称
default_hybrid_alpha	float	0.5	默认 RRF 权重（0=文本，1=向量）
hybrid_ranker	str	"rrf"	排序器：“rrf”或“vertex”
semantic_task_type	str	"RETRIEVAL_QUERY"	SemanticSearch 的任务类型
vertex_ranker_model	str	"semantic-ranker-default@latest"	VertexRanker 模型
vertex_ranker_title_field	str	None	VertexRanker 的标题字段
vertex_ranker_content_field	str	None	VertexRanker 的内容字段

清理

完成后删除集合以避免产生费用：

CLEANUP = False  # Set to True to delete the collection

if CLEANUP:
    from google.cloud import vectorsearch_v1beta

    client = vectorsearch_v1beta.VectorSearchServiceClient()
    collection_name = (
        f"projects/{PROJECT_ID}/locations/{REGION}/collections/{COLLECTION_ID}"
    )

    print(f"Deleting collection: {collection_name}")
    client.delete_collection(name=collection_name)
    print("Collection deleted.")

总结

本 notebook 演示了：

1. 简单设置：v2 只需一个集合——无需索引/端点部署

2. 易于集成：只需添加 api_version="v2" 即可使用新 API

3. 相同接口：所有 LlamaIndex 操作（添加、查询、删除）的工作方式相同

4. 新功能：v2 增加了 v1 中不可用的 clear() 方法

5. 混合搜索：结合向量和文本搜索以获得更好的检索效果：

   • TEXT_SEARCH——全文关键词搜索
   • HYBRID——向量 + 文本，使用 RRF 融合
   • SEMANTIC_HYBRID——向量 + 语义搜索
   • 用于 AI 驱动排序的 VertexRanker

从 v1 迁移

# v1 (old)
vector_store = VertexAIVectorStore(
    project_id="...",
    region="...",
    index_id="projects/.../indexes/123",
    endpoint_id="projects/.../indexEndpoints/456",
    gcs_bucket_name="my-bucket"
)

# v2 (new)
vector_store = VertexAIVectorStore(
    api_version="v2",
    project_id="...",
    region="...",
    collection_id="my-collection"
)

# v2 with hybrid search
vector_store = VertexAIVectorStore(
    api_version="v2",
    project_id="...",
    region="...",
    collection_id="my-collection",
    enable_hybrid=True,
    text_search_fields=["text", "title"],
)

有关详细的迁移说明，请参阅 V2_MIGRATION.md