追踪生成器函数

在大多数 LLM 应用中，你都会希望流式输出，以尽量减少用户看到第一个 token 的等待时间。

LangSmith 的追踪功能通过 generator 函数原生支持流式输出。下面是一个示例。

Python

from langsmith import traceable
@traceable
def my_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk
# Stream to the user
for output in my_generator():
  print(output)
# It also works with async functions
import asyncio
@traceable
async def my_async_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk
# Stream to the user
async def main():
  async for output in my_async_generator():
      print(output)
asyncio.run(main())

TypeScript

import { traceable } from "langsmith/traceable";
const myGenerator = traceable(function* () {
  for (const chunk of ["Hello", "World", "!"]) {
      yield chunk;
  }
});
for (const output of myGenerator()) {
  console.log(output);
}

聚合结果

默认情况下，被追踪函数的 outputs 会在 LangSmith 中聚合为单个数组。如果你想自定义存储方式（例如将输出拼接为单个字符串），可以使用 aggregate 选项（Python 中为 reduce_fn）。这在聚合流式 LLM 输出时尤其有用。

NOTE

聚合输出只会影响输出在追踪中的呈现，不会改变函数返回的值。

Python

from langsmith import traceable
def concatenate_strings(outputs: list):
  return "".join(outputs)
@traceable(reduce_fn=concatenate_strings)
def my_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk
# Stream to the user
for output in my_generator():
  print(output)
# It also works with async functions
import asyncio
@traceable(reduce_fn=concatenate_strings)
async def my_async_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk
# Stream to the user
async def main():
  async for output in my_async_generator():
      print(output)
asyncio.run(main())

TypeScript

import { traceable } from "langsmith/traceable";
const concatenateStrings = (outputs: string[]) => outputs.join("");
const myGenerator = traceable(function* () {
  for (const chunk of ["Hello", "World", "!"]) {
      yield chunk;
  }
}, { aggregator: concatenateStrings });
for (const output of await myGenerator()) {
  console.log(output);
}

---