主题
文本转语音（TTS）API
Spring AI 通过 TextToSpeechModel 和 StreamingTextToSpeechModel 接口为文本转语音（TTS）提供统一 API。这让你能编写跨不同 TTS 服务商工作的可移植代码。
受支持的服务商
通用接口
所有 TTS 服务商都实现以下共享接口：
TextToSpeechModel
TextToSpeechModel 接口提供了把文本转换为语音的方法：
java
public interface TextToSpeechModel extends Model<TextToSpeechPrompt, TextToSpeechResponse>, StreamingTextToSpeechModel {
/**
* Converts text to speech with default options.
*/
default byte[] call(String text) {
// Default implementation
}
/**
* Converts text to speech with custom options.
*/
TextToSpeechResponse call(TextToSpeechPrompt prompt);
/**
* Returns the default options for this model.
*/
default TextToSpeechOptions getOptions() {
// Default implementation
}
}StreamingTextToSpeechModel
StreamingTextToSpeechModel 接口提供了实时流式传输音频的方法：
java
@FunctionalInterface
public interface StreamingTextToSpeechModel extends StreamingModel<TextToSpeechPrompt, TextToSpeechResponse> {
/**
* Streams text-to-speech responses with metadata.
*/
Flux<TextToSpeechResponse> stream(TextToSpeechPrompt prompt);
/**
* Streams audio bytes for the given text.
*/
default Flux<byte[]> stream(String text) {
// Default implementation
}
}TextToSpeechPrompt
TextToSpeechPrompt 类封装输入文本和选项：
java
TextToSpeechPrompt prompt = new TextToSpeechPrompt(
"Hello, this is a text-to-speech example.",
options
);TextToSpeechResponse
TextToSpeechResponse 类包含生成的音频和元数据：
java
TextToSpeechResponse response = model.call(prompt);
byte[] audioBytes = response.getResult().getOutput();
TextToSpeechResponseMetadata metadata = response.getMetadata();编写服务商无关的代码
共享 TTS 接口的一大关键好处，是让你能编写无需修改即可与任何 TTS 服务商协作的代码。实际的服务商（OpenAI、ElevenLabs 等）由你的 Spring Boot 配置决定，让你无需改动应用代码就能切换服务商。
基本服务示例
共享接口让你能编写与任何 TTS 服务商协作的代码：
java
@Service
public class NarrationService {
private final TextToSpeechModel textToSpeechModel;
public NarrationService(TextToSpeechModel textToSpeechModel) {
this.textToSpeechModel = textToSpeechModel;
}
public byte[] narrate(String text) {
// Works with any TTS provider
return textToSpeechModel.call(text);
}
public byte[] narrateWithOptions(String text, TextToSpeechOptions options) {
TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
TextToSpeechResponse response = textToSpeechModel.call(prompt);
return response.getResult().getOutput();
}
}该服务能与 OpenAI、ElevenLabs 或任何其他 TTS 服务商无缝协作，实际实现由你的 Spring Boot 配置决定。
高级示例：多服务商支持
你可以构建同时支持多个 TTS 服务商的应用：
java
@Service
public class MultiProviderNarrationService {
private final Map<String, TextToSpeechModel> providers;
public MultiProviderNarrationService(List<TextToSpeechModel> models) {
// Spring will inject all available TextToSpeechModel beans
this.providers = models.stream()
.collect(Collectors.toMap(
model -> model.getClass().getSimpleName(),
model -> model
));
}
public byte[] narrateWithProvider(String text, String providerName) {
TextToSpeechModel model = providers.get(providerName);
if (model == null) {
throw new IllegalArgumentException("Unknown provider: " + providerName);
}
return model.call(text);
}
public Set<String> getAvailableProviders() {
return providers.keySet();
}
}流式音频示例
共享接口也支持用于实时音频生成的流式：
java
@Service
public class StreamingNarrationService {
private final TextToSpeechModel textToSpeechModel;
public StreamingNarrationService(TextToSpeechModel textToSpeechModel) {
this.textToSpeechModel = textToSpeechModel;
}
public Flux<byte[]> streamNarration(String text) {
// TextToSpeechModel extends StreamingTextToSpeechModel
return textToSpeechModel.stream(text);
}
public Flux<TextToSpeechResponse> streamWithMetadata(String text, TextToSpeechOptions options) {
TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
return textToSpeechModel.stream(prompt);
}
}REST 控制器示例
用与服务商无关的 TTS 构建一个 REST API：
java
@RestController
@RequestMapping("/api/tts")
public class TextToSpeechController {
private final TextToSpeechModel textToSpeechModel;
public TextToSpeechController(TextToSpeechModel textToSpeechModel) {
this.textToSpeechModel = textToSpeechModel;
}
@PostMapping(value = "/synthesize", produces = "audio/mpeg")
public ResponseEntity<byte[]> synthesize(@RequestBody SynthesisRequest request) {
byte[] audio = textToSpeechModel.call(request.text());
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType("audio/mpeg"))
.header("Content-Disposition", "attachment; filename=\"speech.mp3\"")
.body(audio);
}
@GetMapping(value = "/stream", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public Flux<byte[]> streamSynthesis(@RequestParam String text) {
return textToSpeechModel.stream(text);
}
record SynthesisRequest(String text) {}
}把流作为阻塞式 InputStream 消费
有些 API 只接受 java.io.InputStream，而非响应式的 Flux。一个音频播放库，或用阻塞 I/O 增量写入磁盘的代码，就是典型的例子。把一个 Flux<byte[]> 桥接进一个 InputStream 很直接，但值得精确说明这样一个桥接实际保证了什么。
一个行为良好的桥接应每次从 Flux 请求一个块，因此它绝不会在 InputStream 消费者已读内容之外预缓冲超过一个已接收的块。它还应在 InputStream 于到达流末尾前被关闭时取消底层订阅，从而使一个提前停止（例如中止播放）的消费者立即释放服务商连接，而非让它一直运行到响应本会完成为止。基于 Flux#toIterable() 构建的桥接无法提供第二个特性，因为其迭代器不暴露任何取消订阅的方式。通过一个 reactor.core.publisher.BaseSubscriber 桥接能给你这两个特性：从 hookOnNext 每次请求一个元素，并从 close() 经 Subscriber.dispose() 取消。
暂停 InputStream 消费者确实会把真实的背压传播给那些客户端使用 HTTP/2 的服务商，例如 OpenAI。底层传输 OkHttp 只在应用代码实际读取缓冲的字节之后，才发送告知服务端继续发数据的 WINDOW_UPDATE 帧，因此一个停滞的消费者最终会阻止服务端在该连接上继续发送数据。这一效应真实但粗粒度：它要等到累积了整整一个 HTTP/2 流控窗口的未读数据（通常是数十 KB）后才生效，而非在一个未读块之后。它也只保证传输停止发送；某个服务商的后端在其发送缓冲区填满后是否真的放慢音频生成，而非继续在服务端缓冲，是服务商特定的，也不是客户端侧桥接所能控制或核实的。
TIP
一个完整、经过测试的参考实现 FluxInputStream，可在 spring-ai-openai 模块测试源码里的 OpenAiAudioSpeechModelIT 获得。
基于配置的服务商选择
用 Spring profiles 或属性在服务商之间切换：
yaml
# application-openai.yml
spring:
ai:
model:
audio:
speech: openai
openai:
api-key: ${OPENAI_API_KEY}
audio:
speech:
options:
model: gpt-4o-mini-tts
voice: alloy
# application-elevenlabs.yml
spring:
ai:
model:
audio:
speech: elevenlabs
elevenlabs:
api-key: ${ELEVENLABS_API_KEY}
tts:
options:
model-id: eleven_turbo_v2_5
voice-id: your_voice_id然后激活所需的服务商：
bash
# Use OpenAI
java -jar app.jar --spring.profiles.active=openai
# Use ElevenLabs
java -jar app.jar --spring.profiles.active=elevenlabs使用可移植选项
为获得最大可移植性，只用通用的 TextToSpeechOptions 接口方法：
java
@Service
public class PortableNarrationService {
private final TextToSpeechModel textToSpeechModel;
public PortableNarrationService(TextToSpeechModel textToSpeechModel) {
this.textToSpeechModel = textToSpeechModel;
}
public byte[] createPortableNarration(String text) {
// Use provider's default options for maximum portability
TextToSpeechOptions options = textToSpeechModel.getOptions();
TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
TextToSpeechResponse response = textToSpeechModel.call(prompt);
return response.getResult().getOutput();
}
}处理服务商特定特性
当你需要服务商特定特性时，仍可在使用它们的同时保持代码库可移植：
java
@Service
public class FlexibleNarrationService {
private final TextToSpeechModel textToSpeechModel;
public FlexibleNarrationService(TextToSpeechModel textToSpeechModel) {
this.textToSpeechModel = textToSpeechModel;
}
public byte[] narrate(String text, TextToSpeechOptions baseOptions) {
TextToSpeechOptions options = baseOptions;
// Apply provider-specific optimizations if available
if (textToSpeechModel instanceof OpenAiAudioSpeechModel) {
options = OpenAiAudioSpeechOptions.builder()
.from(baseOptions)
.model("gpt-4o-tts") // OpenAI-specific: use high-quality model
.speed(1.0)
.build();
} else if (textToSpeechModel instanceof ElevenLabsTextToSpeechModel) {
// ElevenLabs-specific options could go here
}
TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
TextToSpeechResponse response = textToSpeechModel.call(prompt);
return response.getResult().getOutput();
}
}可移植代码的最佳实践
- 依赖接口：总是注入
TextToSpeechModel而非具体实现 - 使用通用选项：坚持用
TextToSpeechOptions接口方法以获得最大可移植性 - 优雅处理元数据：不同服务商返回不同的元数据；泛化地处理它
- 用多个服务商测试：确保你的代码至少能与两个 TTS 服务商协作
- 记录服务商假设：若你依赖特定的服务商行为，清晰地记录它
服务商特定特性
虽然共享接口提供了可移植性，但每个服务商也通过服务商特定的选项类（例如 OpenAiAudioSpeechOptions、ElevenLabsSpeechOptions）提供特定功能。这些类实现 TextToSpeechOptions 接口，同时添加服务商特定的能力。