代码优化
This commit is contained in:
179
openspec/changes/add-siliconflow-voice-provider/design.md
Normal file
179
openspec/changes/add-siliconflow-voice-provider/design.md
Normal file
@@ -0,0 +1,179 @@
|
||||
# Technical Design: SiliconFlow Voice Provider
|
||||
|
||||
## Context
|
||||
|
||||
硅基流动(SiliconFlow)是一个提供多种 AI 服务的平台,包括语音合成、语音克隆等功能。本次设计将其作为新的语音供应商集成到现有的多供应商架构中。
|
||||
|
||||
**约束条件**:
|
||||
- 必须兼容现有的 `VoiceCloneProvider` 接口
|
||||
- 不能影响现有的 CosyVoice 供应商功能
|
||||
- 需要适配硅基流动的 API 差异
|
||||
|
||||
**关键 API 差异**:
|
||||
1. **语音克隆**: 硅基流动需要先上传参考音频,返回 `uri` 作为音色 ID
|
||||
2. **TTS 合成**: 使用 `/v1/audio/speech` 端点,返回二进制音频数据
|
||||
3. **认证**: 使用 Bearer Token 格式的 API Key
|
||||
4. **模型**: 使用 `IndexTeam/IndexTTS-2` 模型
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
### Goals
|
||||
- 实现 `SiliconFlowProvider` 完整支持语音克隆和 TTS
|
||||
- 支持硅基流动 `IndexTeam/IndexTTS-2` 模型
|
||||
- 提供完整的配置支持,可独立开关
|
||||
- 处理 API 差异,提供统一的服务接口
|
||||
|
||||
### Non-Goals
|
||||
- 不实现语音转文字(STT)功能(已有其他服务)
|
||||
- 不修改现有的 `VoiceCloneProvider` 接口定义
|
||||
- 不改变前端 API 契约
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. 实现类结构
|
||||
|
||||
**架构**:
|
||||
```
|
||||
SiliconFlowProvider (implements VoiceCloneProvider)
|
||||
├── SiliconFlowApi (HTTP 客户端)
|
||||
├── SiliconFlowProviderConfig (配置类)
|
||||
└── DTO 类 (请求/响应适配)
|
||||
```
|
||||
|
||||
**Why**:
|
||||
- 遵循现有的 `CosyVoiceProvider` 模式
|
||||
- 分离 API 调用逻辑,便于测试和维护
|
||||
- 专用 DTO 处理硅基流动 API 差异
|
||||
|
||||
### 2. 语音克隆流程适配
|
||||
|
||||
**硅基流动语音克隆 API**:
|
||||
- 端点: `POST /v1/uploads/audio/voice`
|
||||
- 请求参数: `model`, `customName`, `text`, `audio` (base64)
|
||||
- 响应: `{"uri": "speech:customName:xxx:xxx"}`
|
||||
|
||||
**适配策略**:
|
||||
1. 将统一请求的 `audioUrl` 下载并转换为 base64
|
||||
2. 使用 `prefix` 作为 `customName`
|
||||
3. 使用 `audioUrl` 对应的转录文本作为 `text` 参数
|
||||
4. 返回的 `uri` 存储为 `voiceId`
|
||||
|
||||
**代码示例**:
|
||||
```java
|
||||
@Override
|
||||
public VoiceCloneResult cloneVoice(VoiceCloneRequest request) {
|
||||
// 1. 下载音频文件
|
||||
byte[] audioData = downloadAudio(request.getAudioUrl());
|
||||
String base64Audio = Base64.getEncoder().encodeToString(audioData);
|
||||
|
||||
// 2. 构建硅基流动请求
|
||||
SiliconFlowVoiceUploadRequest sfRequest = new SiliconFlowVoiceUploadRequest();
|
||||
sfRequest.setModel("IndexTeam/IndexTTS-2");
|
||||
sfRequest.setCustomName(request.getPrefix());
|
||||
sfRequest.setText(getTranscriptionText(request.getAudioUrl()));
|
||||
sfRequest.setAudio("data:audio/mpeg;base64," + base64Audio);
|
||||
|
||||
// 3. 调用 API
|
||||
SiliconFlowVoiceUploadResponse sfResponse = siliconFlowApi.uploadVoice(sfRequest);
|
||||
|
||||
// 4. 适配返回结果
|
||||
VoiceCloneResult result = new VoiceCloneResult();
|
||||
result.setVoiceId(sfResponse.getUri());
|
||||
return result;
|
||||
}
|
||||
```
|
||||
|
||||
### 3. TTS 合成流程适配
|
||||
|
||||
**硅基流动 TTS API**:
|
||||
- 端点: `POST /v1/audio/speech`
|
||||
- 请求参数: `model`, `input`, `voice`, `speed`, `sample_rate`, `response_format`
|
||||
- 响应: 二进制音频数据
|
||||
|
||||
**适配策略**:
|
||||
1. 使用 `voiceId` (uri 格式) 作为 `voice` 参数
|
||||
2. 支持语速调节 (`speed`)
|
||||
3. 将二进制响应转换为 Base64 返回
|
||||
|
||||
### 4. 配置设计
|
||||
|
||||
**配置结构**:
|
||||
```yaml
|
||||
yudao:
|
||||
voice:
|
||||
providers:
|
||||
siliconflow:
|
||||
enabled: false
|
||||
api-key: ${SILICONFLOW_API_KEY}
|
||||
base-url: https://api.siliconflow.cn
|
||||
default-model: IndexTeam/IndexTTS-2
|
||||
audio-format: mp3
|
||||
sample-rate: 24000
|
||||
connect-timeout: 10s
|
||||
read-timeout: 180s
|
||||
```
|
||||
|
||||
**配置类设计**:
|
||||
```java
|
||||
@Data
|
||||
@EqualsAndHashCode(callSuper = true)
|
||||
public class SiliconFlowProviderConfig extends VoiceProviderProperties.ProviderConfig {
|
||||
private String baseUrl = "https://api.siliconflow.cn";
|
||||
private String defaultModel = "IndexTeam/IndexTTS-2";
|
||||
private String audioFormat = "mp3";
|
||||
private Integer sampleRate = 24000;
|
||||
private Duration connectTimeout = Duration.ofSeconds(10);
|
||||
private Duration readTimeout = Duration.ofSeconds(180);
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 错误处理策略
|
||||
|
||||
- API 调用失败时记录详细日志
|
||||
- 统一转换为 `VOICE_TTS_FAILED` 业务异常
|
||||
- 不暴露硅基流动技术细节给上层
|
||||
- 支持重试机制(网络错误)
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| 硅基流动 API 变更 | 封装在独立的 API 客户端中,便于更新 |
|
||||
| 语音克隆需要转录文本 | 在创建配音时已有转录流程,复用该文本 |
|
||||
| 音频下载增加延迟 | 可考虑配置是否需要下载,或使用异步处理 |
|
||||
| Base64 编码增加内存占用 | 限制音频文件大小(已有 50MB 限制) |
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### 阶段一:后端实现
|
||||
1. 创建 `SiliconFlowApi` HTTP 客户端
|
||||
2. 创建 `SiliconFlowProviderConfig` 配置类
|
||||
3. 创建硅基流动专用 DTO 类
|
||||
4. 实现 `SiliconFlowProvider`
|
||||
5. 更新 `application.yaml` 配置
|
||||
|
||||
### 阶段二:测试验证
|
||||
1. 单元测试:`SiliconFlowApi` 调用
|
||||
2. 单元测试:`SiliconFlowProvider` 适配逻辑
|
||||
3. 集成测试:语音克隆完整流程
|
||||
4. 集成测试:TTS 合成完整流程
|
||||
|
||||
### 阶段三:前端支持(已有基础)
|
||||
1. 验证 `voiceConfig.js` 已支持 `siliconflow` 类型
|
||||
2. 验证 API 请求已支持 `providerType` 参数
|
||||
|
||||
### 回滚方案
|
||||
- 通过配置 `enabled: false` 禁用硅基流动
|
||||
- 删除 `SiliconFlowProvider` 相关代码
|
||||
- 恢复 `application.yaml` 配置
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Q**: 语音克隆时是否必须提供转录文本?
|
||||
**A**: 硅基流动 API 需要 `text` 参数,使用配音创建时的转录文本
|
||||
|
||||
2. **Q**: 是否需要支持硅基流动的其他模型?
|
||||
**A**: 本次仅支持 `IndexTeam/IndexTTS-2`,后续可扩展
|
||||
|
||||
3. **Q**: 音频下载失败如何处理?
|
||||
**A**: 抛出业务异常,提示用户检查音频 URL
|
||||
36
openspec/changes/add-siliconflow-voice-provider/proposal.md
Normal file
36
openspec/changes/add-siliconflow-voice-provider/proposal.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Change: Add SiliconFlow Voice Provider
|
||||
|
||||
## Why
|
||||
|
||||
当前语音克隆功能已完成策略模式重构,支持多供应商架构。CosyVoice 供应商已实现并投入使用。为提供更多选择和降低对单一供应商的依赖,需要添加硅基流动(SiliconFlow)作为新的语音供应商,支持 IndexTeam/IndexTTS-2 模型的语音克隆和 TTS 合成。
|
||||
|
||||
## What Changes
|
||||
|
||||
- **ADDED** 新增 `SiliconFlowProvider` 实现类,实现 `VoiceCloneProvider` 接口
|
||||
- **ADDED** 新增 `SiliconFlowProviderConfig` 配置类
|
||||
- **ADDED** 新增 `SiliconFlowApi` API 客户端类
|
||||
- **ADDED** 新增硅基流动专用 DTO 类
|
||||
- **MODIFIED** 更新 `VoiceProviderProperties` 支持硅基流动配置
|
||||
- **MODIFIED** 更新 `application.yaml` 添加硅基流动配置项
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected specs**:
|
||||
- `voice-clone` - 扩展支持新的语音供应商
|
||||
- **Affected code**:
|
||||
- 新增 `yudao-module-tik/.../voice/client/SiliconFlowProvider.java`
|
||||
- 新增 `yudao-module-tik/.../voice/client/SiliconFlowApi.java`
|
||||
- 新增 `yudao-module-tik/.../voice/config/SiliconFlowProviderConfig.java`
|
||||
- 新增 `yudao-module-tik/.../voice/client/dto/SiliconFlow*.java` (DTO 类)
|
||||
- 更新 `yudao-server/src/main/resources/application.yaml`
|
||||
|
||||
## Dependencies
|
||||
|
||||
- 依赖已完成的多供应商架构重构(`VoiceCloneProvider` 接口和工厂模式)
|
||||
- 硅基流动 API Key 需要在配置中提供
|
||||
|
||||
## Migration
|
||||
|
||||
- 无需数据迁移,新功能为纯新增
|
||||
- 现有 CosyVoice 供应商功能不受影响
|
||||
- 硅基流动默认禁用,需通过配置启用
|
||||
@@ -0,0 +1,117 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: SiliconFlow 语音克隆支持
|
||||
|
||||
系统 MUST 支持使用硅基流动(SiliconFlow)作为语音克隆供应商,允许用户上传参考音频并生成可复用的音色 ID。
|
||||
|
||||
#### Scenario: 成功使用硅基流动克隆音色
|
||||
|
||||
- **GIVEN** 用户已上传参考音频文件并获得文件 URL
|
||||
- **AND** 参考音频已成功转录为文本
|
||||
- **AND** 硅基流动供应商已启用并配置有效的 API Key
|
||||
- **WHEN** 用户通过 API 发起语音克隆请求,指定 `providerType` 为 `siliconflow`
|
||||
- **THEN** 系统应当下载参考音频文件并转换为 base64 格式
|
||||
- **AND** 调用硅基流动 `/v1/uploads/audio/voice` API,使用 `IndexTeam/IndexTTS-2` 模型
|
||||
- **AND** 将返回的 `uri`(格式如 `speech:customName:xxx:xxx`)存储为 `voiceId`
|
||||
- **AND** 返回克隆成功的响应,包含生成的 `voiceId`
|
||||
|
||||
#### Scenario: 硅基流动 API 调用失败
|
||||
|
||||
- **GIVEN** 硅基流动供应商已启用
|
||||
- **WHEN** 调用硅基流动 API 时发生网络错误或认证失败
|
||||
- **THEN** 系统应当记录详细的错误日志
|
||||
- **AND** 返回统一错误码 `VOICE_TTS_FAILED`,不暴露底层技术细节
|
||||
|
||||
#### Scenario: 硅基流动供应商未配置
|
||||
|
||||
- **GIVEN** 硅基流动供应商未启用或 API Key 未配置
|
||||
- **WHEN** 用户尝试使用硅基流动进行语音克隆
|
||||
- **THEN** 系统应当返回友好的错误提示,要求管理员先配置硅基流动
|
||||
|
||||
---
|
||||
|
||||
### Requirement: SiliconFlow 文本转语音支持
|
||||
|
||||
系统 MUST 支持使用硅基流动进行文本转语音合成,允许用户使用已克隆的音色 ID 或系统默认音色合成语音。
|
||||
|
||||
#### Scenario: 使用克隆音色合成语音
|
||||
|
||||
- **GIVEN** 用户已通过硅基流动成功克隆音色,获得 `voiceId`
|
||||
- **AND** 硅基流动供应商已启用
|
||||
- **WHEN** 用户发起 TTS 请求,指定 `providerType` 为 `siliconflow` 和有效的 `voiceId`
|
||||
- **THEN** 系统应当调用硅基流动 `/v1/audio/speech` API
|
||||
- **AND** 使用 `IndexTeam/IndexTTS-2` 模型和指定的 `voiceId`
|
||||
- **AND** 将返回的音频二进制数据转换为 base64 格式
|
||||
- **AND** 返回包含音频数据、格式和采样率的响应
|
||||
|
||||
#### Scenario: 使用默认音色合成语音
|
||||
|
||||
- **GIVEN** 硅基流动供应商已启用
|
||||
- **AND** 用户未指定 `voiceId`
|
||||
- **WHEN** 用户发起 TTS 请求,指定 `providerType` 为 `siliconflow`
|
||||
- **THEN** 系统应当使用硅基流动提供的默认音色进行合成
|
||||
- **AND** 返回合成结果
|
||||
|
||||
#### Scenario: TTS 合成参数支持
|
||||
|
||||
- **GIVEN** 硅基流动供应商已启用
|
||||
- **WHEN** 用户发起 TTS 请求,包含可选参数(语速、采样率、音频格式)
|
||||
- **THEN** 系统应当将这些参数适配为硅基流动 API 格式
|
||||
- **AND** 支持的参数包括:`speed`(0.25-4.0)、`sample_rate`、`response_format`(mp3/wav/pcm)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 供应商动态切换
|
||||
|
||||
系统 SHALL 支持在请求时动态指定语音供应商,无需重启服务。
|
||||
|
||||
#### Scenario: 通过 providerType 切换供应商
|
||||
|
||||
- **GIVEN** 系统已配置 CosyVoice 和 SiliconFlow 两个供应商
|
||||
- **AND** 默认供应商为 `cosyvoice`
|
||||
- **WHEN** 用户在 API 请求中指定 `providerType` 为 `siliconflow`
|
||||
- **THEN** 系统应当使用 `SiliconFlowProvider` 处理请求
|
||||
- **AND** 不影响其他使用默认供应商的请求
|
||||
|
||||
#### Scenario: 不支持的供应商类型
|
||||
|
||||
- **GIVEN** 系统仅配置了 CosyVoice 和 SiliconFlow 供应商
|
||||
- **WHEN** 用户指定 `providerType` 为不存在的值(如 `other`)
|
||||
- **THEN** 系统应当返回错误提示 "不支持的语音克隆供应商: other"
|
||||
|
||||
---
|
||||
|
||||
### Requirement: SiliconFlow 配置管理
|
||||
|
||||
系统 MUST 支持通过配置文件管理硅基流动供应商的启用状态和连接参数。
|
||||
|
||||
#### Scenario: 配置硅基流动供应商
|
||||
|
||||
- **GIVEN** 管理员希望在系统中启用硅基流动
|
||||
- **WHEN** 管理员在 `application.yaml` 中配置以下参数:
|
||||
```yaml
|
||||
yudao:
|
||||
voice:
|
||||
providers:
|
||||
siliconflow:
|
||||
enabled: true
|
||||
api-key: sk-xxxxx
|
||||
base-url: https://api.siliconflow.cn
|
||||
default-model: IndexTeam/IndexTTS-2
|
||||
```
|
||||
- **THEN** 系统应当在启动时注册 `SiliconFlowProvider`
|
||||
- **AND** 用户可以通过指定 `providerType` 为 `siliconflow` 使用该供应商
|
||||
|
||||
#### Scenario: 禁用硅基流动供应商
|
||||
|
||||
- **GIVEN** 硅基流动供应商已配置
|
||||
- **WHEN** 管理员将 `enabled` 设置为 `false` 或移除配置
|
||||
- **THEN** 系统启动时不应注册 `SiliconFlowProvider`
|
||||
- **AND** 用户请求硅基流动服务时应当返回错误提示
|
||||
|
||||
#### Scenario: 向后兼容旧配置
|
||||
|
||||
- **GIVEN** 系统已从旧版本升级,存在 `yudao.cosyvoice.*` 配置
|
||||
- **WHEN** 系统启动时检测到旧配置
|
||||
- **THEN** 系统应当自动将旧配置迁移到 `yudao.voice.providers.cosyvoice.*` 结构
|
||||
- **AND** 优先使用新配置,旧配置作为 fallback
|
||||
66
openspec/changes/add-siliconflow-voice-provider/tasks.md
Normal file
66
openspec/changes/add-siliconflow-voice-provider/tasks.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Implementation Tasks
|
||||
|
||||
## 1. 配置类实现
|
||||
- [ ] 1.1 创建 `SiliconFlowProviderConfig` 配置类
|
||||
- 继承 `VoiceProviderProperties.ProviderConfig`
|
||||
- 添加硅基流动特有配置字段
|
||||
- 添加默认值和配置前缀
|
||||
|
||||
## 2. API 客户端实现
|
||||
- [ ] 2.1 创建 `SiliconFlowApi` HTTP 客户端
|
||||
- 实现 `uploadVoice()` 方法 - 上传参考音频
|
||||
- 实现 `synthesize()` 方法 - 文本转语音
|
||||
- 实现 `transcribe()` 方法 - 语音转文本(可选)
|
||||
- 配置 RestTemplate/ WebClient
|
||||
- 添加请求/响应日志
|
||||
|
||||
## 3. DTO 类实现
|
||||
- [ ] 3.1 创建 `SiliconFlowVoiceUploadRequest` - 上传参考音频请求
|
||||
- [ ] 3.2 创建 `SiliconFlowVoiceUploadResponse` - 上传参考音频响应
|
||||
- [ ] 3.3 创建 `SiliconFlowTtsRequest` - 文本转语音请求
|
||||
- [ ] 3.4 创建 `SiliconFlowTtsResponse` - 文本转语音响应(二进制处理)
|
||||
|
||||
## 4. Provider 实现类
|
||||
- [ ] 4.1 创建 `SiliconFlowProvider` 实现类
|
||||
- 实现 `VoiceCloneProvider` 接口
|
||||
- 实现 `cloneVoice()` 方法
|
||||
- 实现 `synthesize()` 方法
|
||||
- 实现 `supports()` 方法
|
||||
- 实现 `getProviderType()` 方法
|
||||
- 添加 `@Component` 注解注册为 Spring Bean
|
||||
|
||||
## 5. 配置文件更新
|
||||
- [ ] 5.1 更新 `application.yaml`
|
||||
- 添加 `yudao.voice.providers.siliconflow` 配置节
|
||||
- 配置 API Key、base URL、模型等参数
|
||||
- 默认设置为 `enabled: false`
|
||||
|
||||
## 6. 测试与验证
|
||||
- [ ] 6.1 编写单元测试
|
||||
- 测试 `SiliconFlowApi` HTTP 调用
|
||||
- 测试 `SiliconFlowProvider` 适配逻辑
|
||||
- Mock 硅基流动 API 响应
|
||||
- [ ] 6.2 集成测试
|
||||
- 测试语音克隆完整流程
|
||||
- 测试 TTS 合成完整流程
|
||||
- 测试供应商切换功能
|
||||
- [ ] 6.3 验证前端兼容性
|
||||
- 验证 `voiceConfig.js` 支持 `siliconflow` 类型
|
||||
- 验证 API 请求支持 `providerType` 参数
|
||||
|
||||
## 7. 文档与清理
|
||||
- [ ] 7.1 更新相关文档
|
||||
- 添加硅基流动配置说明
|
||||
- 添加硅基流动使用示例
|
||||
- [ ] 7.2 代码审查与清理
|
||||
- 检查代码规范
|
||||
- 移除调试代码
|
||||
- 确保日志级别正确
|
||||
|
||||
---
|
||||
|
||||
**总计**: 20 项任务
|
||||
|
||||
**预计工作量**: 2-3 天
|
||||
|
||||
**依赖**: 多供应商架构重构已完成
|
||||
@@ -1,133 +0,0 @@
|
||||
# Technical Design: Voice Clone Provider Refactoring
|
||||
|
||||
## Context
|
||||
|
||||
当前语音克隆功能直接依赖阿里云 CosyVoice 的 SDK 和 API。Service 层直接调用 `CosyVoiceClient`,导致:
|
||||
|
||||
1. **强耦合**:无法轻松切换或添加其他供应商
|
||||
2. **测试困难**:难以 mock 外部依赖
|
||||
3. **扩展性差**:添加新供应商需要修改 Service 层
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
### Goals
|
||||
- 解耦 Service 层与具体供应商实现
|
||||
- 支持多供应商并存和动态切换
|
||||
- 保持现有功能完全兼容
|
||||
- 为添加硅基流动 IndexTTS-2 打下基础
|
||||
|
||||
### Non-Goals
|
||||
- 不改变现有 API 行为
|
||||
- 不修改数据库结构
|
||||
- 不改变前端交互
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. 采用策略模式 + 工厂模式
|
||||
|
||||
**Why**:
|
||||
- 策略模式:定义统一接口,各供应商独立实现
|
||||
- 工厂模式:根据配置动态获取 Provider 实例
|
||||
- 符合开闭原则,扩展时无需修改现有代码
|
||||
|
||||
**架构**:
|
||||
```
|
||||
VoiceCloneProvider (interface)
|
||||
├── CosyVoiceProvider (impl) - 阿里云 CosyVoice (DashScope)
|
||||
├── SiliconFlowProvider (impl) - 阶段二:硅基流动 IndexTTS-2
|
||||
└── VoiceCloneProviderFactory
|
||||
```
|
||||
|
||||
**说明**:
|
||||
- `CosyVoiceProvider` 对应阿里云 DashScope 的语音服务
|
||||
- 默认模型:`cosyvoice-v3-flash`
|
||||
- 扩展时添加新的 Provider 实现
|
||||
|
||||
### 2. 统一 DTO 设计
|
||||
|
||||
**Why**: 屏蔽不同供应商的 API 差异
|
||||
|
||||
```java
|
||||
// 统一请求
|
||||
VoiceCloneRequest {
|
||||
String audioUrl; // 音频 URL
|
||||
String prefix; // 音色前缀
|
||||
String targetModel; // 目标模型
|
||||
}
|
||||
|
||||
// 统一响应
|
||||
VoiceCloneResult {
|
||||
String voiceId; // 生成的音色 ID
|
||||
String requestId; // 请求 ID
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 配置结构设计
|
||||
|
||||
**新配置结构**:
|
||||
```yaml
|
||||
yudao:
|
||||
voice:
|
||||
# 默认供应商
|
||||
default-provider: cosyvoice
|
||||
|
||||
# 供应商配置
|
||||
providers:
|
||||
cosyvoice: # 阿里云 CosyVoice
|
||||
enabled: true
|
||||
api-key: ${DASHSCOPE_API_KEY}
|
||||
default-model: cosyvoice-v3-flash
|
||||
# ... 其他配置
|
||||
|
||||
siliconflow: # 阶段二添加
|
||||
enabled: false
|
||||
api-key: ${SILICONFLOW_API_KEY}
|
||||
base-url: https://api.siliconflow.cn
|
||||
default-model: indextts-2
|
||||
```
|
||||
|
||||
**向后兼容**:
|
||||
- 读取旧配置 `yudao.cosyvoice.*` 并合并到新结构
|
||||
- 优先使用新配置,旧配置作为 fallback
|
||||
|
||||
### 4. 错误处理策略
|
||||
|
||||
- Provider 调用失败时,记录详细日志
|
||||
- 返回统一的业务异常 `VOICE_TTS_FAILED`
|
||||
- 不暴露底层供应商的技术细节
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| 破坏现有功能 | 充分测试,保持 DTO 兼容 |
|
||||
| 配置迁移复杂 | 支持旧配置自动映射 |
|
||||
| 性能开销 | 工厂缓存 Provider 实例 |
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### 阶段一:CosyVoice 重构
|
||||
1. 创建接口和工厂
|
||||
2. 重构 CosyVoice 为 Provider 实现
|
||||
3. 更新 Service 层使用接口
|
||||
4. 测试验证
|
||||
|
||||
### 阶段二:添加 SiliconFlow
|
||||
1. 实现 SiliconFlowProvider
|
||||
2. 添加配置支持
|
||||
3. 集成测试
|
||||
|
||||
### 回滚方案
|
||||
- 保留原有配置支持
|
||||
- Feature Flag 控制新逻辑
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Q**: 是否需要支持运行时动态切换供应商?
|
||||
**A**: 初期不支持,通过配置切换即可
|
||||
|
||||
2. **Q**: 是否需要 Provider 健康检查?
|
||||
**A**: 阶段二考虑添加
|
||||
|
||||
3. **Q**: DTO 字段差异如何处理?
|
||||
**A**: 使用公共字段,扩展字段放 `Map<String, Object> extensions`
|
||||
@@ -1,35 +0,0 @@
|
||||
# Change: Refactor Voice Clone Provider
|
||||
|
||||
## Why
|
||||
|
||||
当前语音克隆功能直接依赖阿里云 CosyVoice 实现,代码强耦合,扩展性差。添加新供应商(如硅基流动 IndexTTS-2)需要修改 Service 层代码,违反开闭原则。
|
||||
|
||||
**说明**: CosyVoice 是阿里云的语音合成服务(DashScope 平台),支持语音克隆和 TTS。当前代码使用 `cosyvoice-v3-flash` 模型。
|
||||
|
||||
## What Changes
|
||||
|
||||
- **ADDED** 引入策略模式,定义 `VoiceCloneProvider` 统一接口
|
||||
- **ADDED** 创建工厂类 `VoiceCloneProviderFactory` 管理多供应商
|
||||
- **MODIFIED** 将现有 `CosyVoiceClient` 改造为 `CosyVoiceProvider`
|
||||
- **MODIFIED** 更新 `TikUserVoiceServiceImpl` 使用 Provider 接口
|
||||
- **ADDED** 新增配置类支持多供应商配置和切换
|
||||
- **BREAKING** 配置项从 `yudao.cosyvoice` 迁移到 `yudao.voice.providers`
|
||||
|
||||
## Impact
|
||||
|
||||
- **Affected specs**:
|
||||
- `voice-clone` (新增能力规范)
|
||||
- **Affected code**:
|
||||
- `TikUserVoiceServiceImpl.java` - Service 层改为依赖注入 Provider
|
||||
- `CosyVoiceClient.java` → `CosyVoiceProvider.java` - 重命名并实现接口
|
||||
- `CosyVoiceProperties.java` → `VoiceProviderProperties.java` - 配置结构重组
|
||||
- 新增 `VoiceCloneProvider.java` - 统一接口定义
|
||||
- 新增 `VoiceCloneProviderFactory.java` - 工厂类
|
||||
- 新增 `SiliconFlowProvider.java` - 硅基流动实现(阶段二)
|
||||
|
||||
## Migration
|
||||
|
||||
- 现有配置自动迁移:`yudao.cosyvoice.*` → `yudao.voice.providers.cosyvoice.*`
|
||||
- 默认供应商保持为 `cosyvoice`
|
||||
- 默认行为保持不变,向后兼容
|
||||
- 支持通过配置切换供应商:`yudao.voice.default-provider`
|
||||
@@ -1,132 +0,0 @@
|
||||
# Voice Clone Capability Specification
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Provider Abstraction Layer
|
||||
The system SHALL provide a unified provider abstraction layer for voice cloning services, supporting multiple vendors through a common interface.
|
||||
|
||||
#### Scenario: Get provider by type
|
||||
- **GIVEN** the system is configured with multiple voice clone providers
|
||||
- **WHEN** requesting a provider by type
|
||||
- **THEN** the system SHALL return the corresponding provider instance
|
||||
- **AND** the provider SHALL implement the `VoiceCloneProvider` interface
|
||||
|
||||
#### Scenario: Provider not found
|
||||
- **GIVEN** the system is configured with a default provider
|
||||
- **WHEN** requesting a non-existent provider type
|
||||
- **THEN** the system SHALL fallback to the default provider
|
||||
- **AND** log a warning message
|
||||
|
||||
### Requirement: Voice Cloning
|
||||
The system SHALL support voice cloning through the provider interface, accepting an audio file URL and returning a unique voice ID.
|
||||
|
||||
#### Scenario: Successful voice cloning with CosyVoice
|
||||
- **GIVEN** a valid CosyVoice provider is configured
|
||||
- **WHEN** submitting a voice clone request with audio URL
|
||||
- **THEN** the system SHALL return a voice ID
|
||||
- **AND** the voice ID SHALL be usable for subsequent TTS synthesis
|
||||
|
||||
#### Scenario: Voice cloning failure
|
||||
- **GIVEN** the provider API is unavailable or returns an error
|
||||
- **WHEN** submitting a voice clone request
|
||||
- **THEN** the system SHALL throw a `VOICE_TTS_FAILED` exception
|
||||
- **AND** log the error details for debugging
|
||||
|
||||
### Requirement: Text-to-Speech Synthesis
|
||||
The system SHALL support TTS synthesis through cloned voices or system voices, accepting text input and returning audio data.
|
||||
|
||||
#### Scenario: TTS with cloned voice
|
||||
- **GIVEN** a valid voice ID from a previous clone operation
|
||||
- **WHEN** submitting a TTS request with text and voice ID
|
||||
- **THEN** the system SHALL return audio data in the specified format
|
||||
- **AND** the audio SHALL match the cloned voice characteristics
|
||||
|
||||
#### Scenario: TTS with system voice
|
||||
- **GIVEN** a system voice ID is configured
|
||||
- **WHEN** submitting a TTS request with text and system voice ID
|
||||
- **THEN** the system SHALL return audio data using the system voice
|
||||
- **AND** the audio SHALL match the system voice characteristics
|
||||
|
||||
#### Scenario: TTS with reference audio (file URL)
|
||||
- **GIVEN** a reference audio URL and transcription text
|
||||
- **WHEN** submitting a TTS request with file URL
|
||||
- **THEN** the system SHALL perform on-the-fly voice cloning
|
||||
- **AND** return audio data matching the reference voice
|
||||
|
||||
### Requirement: Configuration Management
|
||||
The system SHALL support multi-provider configuration through a unified configuration structure.
|
||||
|
||||
#### Scenario: Configure multiple providers
|
||||
- **GIVEN** the application configuration file
|
||||
- **WHEN** configuring multiple voice providers
|
||||
- **THEN** each provider SHALL have independent `enabled` flag
|
||||
- **AND** the system SHALL only use enabled providers
|
||||
|
||||
#### Scenario: Default provider selection
|
||||
- **GIVEN** the configuration specifies a `default-provider`
|
||||
- **WHEN** no provider is explicitly specified
|
||||
- **THEN** the system SHALL use the default provider
|
||||
- **AND** fallback to `cosyvoice` if default is not configured
|
||||
|
||||
#### Scenario: Backward compatibility
|
||||
- **GIVEN** existing configuration using `yudao.cosyvoice.*`
|
||||
- **WHEN** the system starts
|
||||
- **THEN** the system SHALL automatically migrate to new config structure
|
||||
- **AND** existing functionality SHALL remain unchanged
|
||||
|
||||
### Requirement: Provider Factory
|
||||
The system SHALL provide a factory component for managing provider instances and resolving providers by type.
|
||||
|
||||
#### Scenario: Factory resolves provider
|
||||
- **GIVEN** the factory is initialized with provider configurations
|
||||
- **WHEN** calling `factory.getProvider("cosyvoice")`
|
||||
- **THEN** the factory SHALL return the CosyVoiceProvider instance
|
||||
- **AND** cache the instance for subsequent requests
|
||||
|
||||
#### Scenario: Factory returns default
|
||||
- **GIVEN** the factory is configured with default provider
|
||||
- **WHEN** calling `factory.getProvider(null)`
|
||||
- **THEN** the factory SHALL return the default provider instance
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Voice Creation Flow
|
||||
The voice creation process SHALL use the provider abstraction layer instead of directly calling CosyVoice client.
|
||||
|
||||
#### Scenario: Create voice with CosyVoice
|
||||
- **GIVEN** a user uploads a voice audio file
|
||||
- **WHEN** creating a voice configuration through the API
|
||||
- **THEN** the system SHALL:
|
||||
1. Validate the file exists and belongs to voice category
|
||||
2. Call `provider.cloneVoice()` with the audio URL
|
||||
3. Store the returned `voiceId` in the database
|
||||
4. Return success response with voice configuration ID
|
||||
|
||||
#### Scenario: Create voice with transcription
|
||||
- **GIVEN** a voice configuration is created without transcription
|
||||
- **WHEN** the user triggers transcription
|
||||
- **THEN** the system SHALL:
|
||||
1. Fetch the audio file URL
|
||||
2. Call the transcription service
|
||||
3. Store the transcription text
|
||||
4. Update the voice configuration
|
||||
|
||||
### Requirement: Voice Preview
|
||||
The voice preview functionality SHALL work with both cloned voices (voiceId) and reference audio (file URL).
|
||||
|
||||
#### Scenario: Preview cloned voice
|
||||
- **GIVEN** a voice configuration with a valid `voiceId`
|
||||
- **WHEN** requesting a preview with custom text
|
||||
- **THEN** the system SHALL call `provider.synthesize()` with the voiceId
|
||||
- **AND** return audio data in Base64 format
|
||||
|
||||
#### Scenario: Preview with reference audio
|
||||
- **GIVEN** a voice configuration without `voiceId` but with audio file
|
||||
- **WHEN** requesting a preview
|
||||
- **THEN** the system SHALL call `provider.synthesize()` with the file URL
|
||||
- **AND** use the stored transcription as reference text
|
||||
- **AND** return audio data in Base64 format
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
None. This change is additive and refactoring only.
|
||||
@@ -1,53 +0,0 @@
|
||||
# Implementation Tasks
|
||||
|
||||
## 1. 接口与基础结构
|
||||
- [ ] 1.1 创建 `VoiceCloneProvider` 接口
|
||||
- 定义 `cloneVoice(VoiceCloneRequest)` 方法
|
||||
- 定义 `synthesize(VoiceTtsRequest)` 方法
|
||||
- 定义 `supports(String providerType)` 方法
|
||||
- [ ] 1.2 创建统一 DTO 类
|
||||
- `VoiceCloneRequest` - 语音克隆请求
|
||||
- `VoiceCloneResult` - 语音克隆响应
|
||||
- `VoiceTtsRequest` - 语音合成请求
|
||||
- `VoiceTtsResult` - 语音合成响应
|
||||
- [ ] 1.3 创建 `VoiceCloneProviderFactory` 工厂类
|
||||
- 根据配置获取 Provider 实例
|
||||
- 支持动态切换供应商
|
||||
|
||||
## 2. CosyVoice 重构(保持现有功能)
|
||||
- [ ] 2.1 重命名 `CosyVoiceClient` → `CosyVoiceProvider`
|
||||
- [ ] 2.2 `CosyVoiceProvider` 实现 `VoiceCloneProvider` 接口
|
||||
- [ ] 2.3 适配现有 DTO 到新的统一 DTO
|
||||
- [ ] 2.4 保持现有 DashScope SDK 调用逻辑不变
|
||||
|
||||
## 3. 配置重构
|
||||
- [ ] 3.1 创建 `VoiceProviderProperties` 配置类
|
||||
- 支持多供应商配置结构
|
||||
- 添加 `default-provider` 配置项
|
||||
- [ ] 3.2 创建 `CosyVoiceProviderConfig` (嵌套配置)
|
||||
- [ ] 3.3 保持向后兼容:支持读取旧的 `yudao.cosyvoice.*` 配置
|
||||
|
||||
## 4. Service 层改造
|
||||
- [ ] 4.1 修改 `TikUserVoiceServiceImpl`
|
||||
- 注入 `VoiceCloneProvider` 而非 `CosyVoiceClient`
|
||||
- 使用工厂获取 Provider 实例
|
||||
- [ ] 4.2 更新方法调用
|
||||
- `createVoice()` - 使用 `provider.cloneVoice()`
|
||||
- `synthesizeVoice()` - 使用 `provider.synthesize()`
|
||||
- `previewVoice()` - 使用 `provider.synthesize()`
|
||||
|
||||
## 5. 测试与验证
|
||||
- [ ] 5.1 单元测试:CosyVoiceProvider
|
||||
- [ ] 5.2 单元测试:VoiceCloneProviderFactory
|
||||
- [ ] 5.3 集成测试:TikUserVoiceServiceImpl
|
||||
- [ ] 5.4 验证现有功能正常运行
|
||||
|
||||
## 6. 文档与配置迁移
|
||||
- [ ] 6.1 更新 `application.yaml` 配置示例
|
||||
- [ ] 6.2 添加配置迁移说明文档
|
||||
|
||||
---
|
||||
|
||||
**总计**: 20 项任务
|
||||
|
||||
**预计工作量**: 2-3 天
|
||||
Reference in New Issue
Block a user