引言

本 notebook 演示如何使用 MyMagicAI 对存储在云存储桶中的海量数据进行批量推理。仅实现了 complete 和 acomplete 端点，它们可用于许多用例，包括补全、摘要和提取。要使用本 notebook，你需要 MyMagicAI 的 API 密钥（个人访问令牌）以及存储在云存储桶中的数据。点击 MyMagicAI 网站上的 Get Started 注册以获取 API 密钥。

设置

要设置存储桶并授予 MyMagic API 对你的云存储的安全访问权限，请访问 MyMagic 文档以供参考。如果你在 Colab 中打开这个 Notebook，可能需要安装 LlamaIndex 🦙。

%pip install llama-index-llms-mymagic

!pip install llama-index

from llama_index.llms.mymagic import MyMagicAI

llm = MyMagicAI(
    api_key="your-api-key",
    storage_provider="s3",  # s3, gcs
    bucket_name="your-bucket-name",
    session="your-session-name",  # files should be located in this folder on which batch inference will be run
    role_arn="your-role-arn",
    system_prompt="your-system-prompt",
    region="your-bucket-region",
    return_output=False,  # Whether you want MyMagic API to return the output json
    input_json_file=None,  # name of the input file (stored on the bucket)
    list_inputs=None,  # Option to provide inputs as a list in case of small batch
    structured_output=None,  # json schema of the output
)

注意：如果上面将 return_output 设为 True，则 max_tokens 应至少设置为 100

resp = llm.complete(
    question="your-question",
    model="chhoose-model",  # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come...
    max_tokens=5,  # number of tokens to generate, default is 10
)

# The response indicated that the final output is stored in your bucket or raises an exception if the job failed
print(resp)

使用 acomplete 端点进行异步请求

对于异步操作，请使用以下方法。

import asyncio

async def main():
    response = await llm.acomplete(
        question="your-question",
        model="choose-model",  # supported models constantly updated and are listed at docs.mymagic.ai
        max_tokens=5,  # number of tokens to generate, default is 10
    )

    print("Async completion response:", response)

await main()