优化
This commit is contained in:
@@ -16,8 +16,8 @@
|
||||
"veoModel": "veo3-fast-frames",
|
||||
"veoEnhancePrompt": true,
|
||||
"veoEnableUpsample": true,
|
||||
"kelingApiBaseUrl": "https://yunwu.ai",
|
||||
"kelingApiKey": "sk-m5inhwXqrbcBL6NNKOe7kTdhX8M31azvAvDvtSPGS71rRzd8",
|
||||
"kelingApiBaseUrl": "https://api-beijing.klingai.com",
|
||||
"kelingApiKey": "nreeg9bbKekdeenAma4KA3bkHMQkG4ND",
|
||||
"kelingModel": "Kling-V2-5-Turbo",
|
||||
"ossRegion": "oss-cn-hangzhou",
|
||||
"ossAccessKeyId": "LTAI5tPV9Ag3csf41GZjaLTA",
|
||||
|
||||
@@ -37,14 +37,14 @@ description: 素材生产路由。根据用户意图分发到对应子技能:i
|
||||
| "生图"、"批量图片" | 生图 | `image-generator` |
|
||||
| "图片成片"、"图片轮播" | 已有图片 → 组装 | `capcut` |
|
||||
| "图文成片"、"生图+成片" | 生图 → TTS+字幕+组装 | `image-generator` → `capcut` |
|
||||
| "图生视频"、"图片转视频" | 生图 → AI视频 → 组装 | `image-generator` → Grok/VEO → `capcut` |
|
||||
| "首尾帧"、"帧动画"、"关键帧" | 生图(成对) → VEO视频 → 组装 | `image-generator`(帧对) → VEO → `capcut` |
|
||||
| "图生视频"、"图片转视频" | 生图 → AI视频 → 组装 | `image-generator` → Grok/VEO/Kling → `capcut` |
|
||||
| "首尾帧"、"帧动画"、"关键帧" | 生图(成对) → 视频过渡 → 组装 | `image-generator`(帧对) → VEO/Kling → `capcut` |
|
||||
| "文案转视频"、"配音视频" | 生图 → TTS+字幕+组装 | `image-generator` → `capcut` |
|
||||
| 只说"做视频" | **询问**:图文成片 / 图生视频(单图/首尾帧)? | — |
|
||||
|
||||
**"图生视频"的后续追问**:用户说"图生视频"时,追问视频模式:
|
||||
- **单图模式**:一张图 → 一段视频(Grok 或 VEO)
|
||||
- **首尾帧模式**:起始帧+结束帧 → 一段过渡视频(仅 VEO)
|
||||
- **单图模式**:一张图 → 一段视频(Grok / VEO / Kling)
|
||||
- **首尾帧模式**:起始帧+结束帧 → 一段过渡视频(VEO / Kling)
|
||||
|
||||
---
|
||||
|
||||
@@ -73,8 +73,8 @@ Step -1: 意图确认(进入任何步骤前必须完成,逐项确认,缺
|
||||
- 有参考图/风格参考?
|
||||
|
||||
3. 视频模式(涉及视频时必问):
|
||||
- 单图模式:1 张图 → 1 段视频(Grok 或 VEO)
|
||||
- 首尾帧模式:2 张图 → 过渡视频(仅 VEO)
|
||||
- 单图模式:1 张图 → 1 段视频(Grok / VEO / Kling)
|
||||
- 首尾帧模式:2 张图 → 过渡视频(VEO / Kling)
|
||||
|
||||
4. 账号确认:
|
||||
- 扫描 accounts/*/account.json 获取最新账号列表
|
||||
@@ -84,7 +84,7 @@ Step -1: 意图确认(进入任何步骤前必须完成,逐项确认,缺
|
||||
- 确认后记住 account ID
|
||||
|
||||
5. 参数确认:
|
||||
- 画幅(9:16 / 16:9)、生图模型(Gemini / MJ)、视频模型(VEO / Grok)
|
||||
- 画幅(9:16 / 16:9)、生图模型(Gemini / MJ)、视频模型(VEO / Grok / Kling)
|
||||
- 有账号时从 account.json 继承默认值,只问是否覆盖
|
||||
|
||||
→ 以上 5 项全部确认后,agent 写出完整执行计划,让用户最终确认:
|
||||
@@ -121,7 +121,7 @@ Step 2: Prompt 生成 + Manifest 初始化(分镜 + 风格 → 英文 prompts
|
||||
- 输入:分镜表 + style.md + account.json
|
||||
- 子 Agent 将每个 shot 的中文画面描述结合风格文件,生成:
|
||||
· imagePrompt(英文画面描述,给 Gemini/MJ)
|
||||
· videoPrompt(英文运动描述,给 Grok/VEO)
|
||||
· videoPrompt(英文运动描述,给 Grok/VEO/Kling)
|
||||
· keyword, keywordColor
|
||||
- **禁止 AI 手写 manifest.json**,必须通过脚本初始化:
|
||||
```bash
|
||||
@@ -196,18 +196,21 @@ digraph single_image {
|
||||
prompt [label="videoPrompt"]
|
||||
grok [label="Grok\n6s 视频", fillcolor="#fff3e0"]
|
||||
veo [label="VEO\n6-8s 视频", fillcolor="#e8f5e9"]
|
||||
kling [label="Kling\n6s 视频", fillcolor="#e1f5fe"]
|
||||
result [label="视频输出", shape=oval, fillcolor="#e3f2fd"]
|
||||
|
||||
img -> prompt
|
||||
prompt -> grok
|
||||
prompt -> veo
|
||||
prompt -> kling
|
||||
grok -> result
|
||||
veo -> result
|
||||
kling -> result
|
||||
}
|
||||
```
|
||||
|
||||
- 每条文案生成 1 张图 + 1 个 videoPrompt
|
||||
- Grok 和 VEO 都支持
|
||||
- Grok、VEO 和 Kling 都支持
|
||||
- 提示词描述运动:"slow zoom in on subject"
|
||||
|
||||
### 首尾帧模式
|
||||
@@ -221,17 +224,22 @@ digraph frame_pair {
|
||||
last [label="结束帧"]
|
||||
prompt [label="videoPrompt"]
|
||||
veo [label="VEO\n6-8s 过渡视频", fillcolor="#e8f5e9"]
|
||||
kling [label="Kling\n6s 过渡视频", fillcolor="#e1f5fe"]
|
||||
result [label="视频输出", shape=oval, fillcolor="#e3f2fd"]
|
||||
|
||||
first -> veo
|
||||
last -> veo
|
||||
prompt -> veo
|
||||
first -> kling
|
||||
last -> kling
|
||||
prompt -> kling
|
||||
veo -> result
|
||||
kling -> result
|
||||
}
|
||||
```
|
||||
|
||||
- 每条文案生成 **2 张图**(firstFrame + lastFrame)+ 1 个 videoPrompt
|
||||
- **仅 VEO 支持**(images 数组传两张图)
|
||||
- **VEO 和 Kling 支持**(images 数组传两张图)
|
||||
- 起始帧和结束帧必须是**同一场景的不同状态**
|
||||
- 提示词描述过渡:"transition from idle machines to active production"
|
||||
|
||||
@@ -240,7 +248,7 @@ digraph frame_pair {
|
||||
| 图片数量 | N 张 | 2N 张 |
|
||||
| 生图耗时 | 标准 | ~2 倍(可并行) |
|
||||
| 视频连贯性 | 仅运动 | 场景变化(更强) |
|
||||
| 可用模型 | Grok + VEO | 仅 VEO |
|
||||
| 可用模型 | Grok + VEO + Kling | VEO + Kling |
|
||||
| 适用场景 | 风景、人物展示 | 状态变化、叙事过渡 |
|
||||
|
||||
---
|
||||
@@ -288,7 +296,7 @@ digraph single_image_video {
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n生图 + videoPrompt"]
|
||||
gate1 [label="人工卡点\n用户挑选图片", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent2 [label="Agent 2\nGrok / VEO\n单图输入,并行生成视频"]
|
||||
agent2 [label="Agent 2\nGrok / VEO / Kling\n单图输入,并行生成视频"]
|
||||
agent3 [label="Agent 3\ncapcut\n视频片段 + 字幕 → 成片"]
|
||||
|
||||
agent1 -> gate1 -> agent2 -> agent3
|
||||
@@ -303,7 +311,7 @@ digraph frame_pair_video {
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n成对生图\n(firstFrame + lastFrame)\n可并行"]
|
||||
gate1 [label="人工卡点\n检查首尾帧连贯性\n同一场景/相似视角", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent2 [label="Agent 2\nVEO\n双图输入\nimages:[first, last]"]
|
||||
agent2 [label="Agent 2\nVEO / Kling\n双图输入\nimages:[first, last]"]
|
||||
agent3 [label="Agent 3\ncapcut\n视频片段 + 字幕 → 成片"]
|
||||
|
||||
agent1 -> gate1 -> agent2 -> agent3
|
||||
@@ -317,6 +325,7 @@ digraph frame_pair_video {
|
||||
| Grok | 6s | 任意 | ✅ | ❌ | 快、稳定 | yunwu.ai |
|
||||
| Veo3-fast | ~8s | 16:9, 9:16 | ✅ | ✅ | 超分、中文增强 | jimmyai.cn |
|
||||
| Veo3-fast-frames | ~8s | 16:9, 9:16 | ✅ | ✅ | 多帧、质量最高 | jimmyai.cn |
|
||||
| Kling | 6s | 任意 | ✅ | ✅ | 快、首尾帧支持 | yunwu.ai |
|
||||
|
||||
图生视频注意事项:
|
||||
- **并行执行**:先同时提交所有任务(并发 3),再并行轮询结果
|
||||
@@ -328,22 +337,24 @@ digraph frame_pair_video {
|
||||
|
||||
### 视频大小一致性
|
||||
|
||||
- **同批次同模型**,不混合 Grok(720P/6s)和 VEO(超分/8s)
|
||||
- **同批次同模型**,不混合 Grok(720P/6s)、VEO(超分/8s)和 Kling(6s)
|
||||
- 画幅统一跟随 manifest 顶层 `format`(默认 `9:16`)
|
||||
- 个别 item 降级到备用模型时,在 manifest 中标记 `"videoModel"` 以便追踪
|
||||
|
||||
### 视频生成失败降级
|
||||
|
||||
**降级链**: `Grok ↔ VEO → 可灵(Kling)`
|
||||
**降级链**: `Grok ↔ VEO ↔ Kling`
|
||||
|
||||
**触发**: 同一 item 重试 5 次仍失败 → 用备用模型单独补生成
|
||||
|
||||
```bash
|
||||
# Grok 失败 → VEO 补
|
||||
# Grok 失败 → VEO/Kling 补
|
||||
node veo-video-generator.js --image <url> --prompt <prompt> -o ./videos
|
||||
node kling-video-generator.js --image <url> --prompt <prompt> -o ./videos
|
||||
|
||||
# VEO 失败 → Grok 补
|
||||
# VEO 失败 → Grok/Kling 补
|
||||
node grok-video-generator.js --image <url> --prompt <prompt> -o ./videos
|
||||
node kling-video-generator.js --image <url> --prompt <prompt> -o ./videos
|
||||
```
|
||||
|
||||
**规则**: 逐 item 降级,不卡整批次。补完后上传 OSS,回写 `videoUrl`,继续 `tts → assemble`。
|
||||
@@ -375,7 +386,7 @@ output/{account}_{YYYYMMDD}_{NNN}/
|
||||
|
||||
**核心规则**:
|
||||
- 脚本检测 `lastFrameUrl` → 首尾帧模式(传 images:[url, lastFrameUrl]);否则 → 单图模式(传 images:[url])
|
||||
- 顶层 `format` 自动传给 VEO/Grok 作为画幅比例
|
||||
- 顶层 `format` 自动传给 VEO/Grok/Kling 作为画幅比例
|
||||
- `account` 字段驱动 capcut_assemble 读取对应 account.json 的字幕风格配置
|
||||
|
||||
---
|
||||
@@ -448,7 +459,7 @@ shot 3-N(交替): close-up(压)→ wide(松)→ close-up(压)→
|
||||
|
||||
每条文案生成:
|
||||
- `imagePrompt`:画面描述(英文,给 Gemini/MJ)
|
||||
- `videoPrompt`:运动描述(英文,给 Grok/VEO)
|
||||
- `videoPrompt`:运动描述(英文,给 Grok/VEO/Kling)
|
||||
|
||||
videoPrompt 规则:
|
||||
- 描述**运动**而非内容("zoom in" 而非 "a cat")
|
||||
@@ -457,14 +468,14 @@ videoPrompt 规则:
|
||||
- **收敛原则**:基于图片已有内容,仅描述镜头运动和微动效果
|
||||
- **禁止**:大幅度环境切换、场景变化、人物位置跳变
|
||||
- **推荐写法**:镜头运动(slow zoom/pan/dolly)+ 星座/光效微动 + 保持静止氛围
|
||||
- **画幅继承**:manifest.json 顶层 `format` 字段(如 `"9:16"`)会自动传给 VEO,无需命令行 `-a`
|
||||
- **画幅继承**:manifest.json 顶层 `format` 字段(如 `"9:16"`)会自动传给 VEO/Kling,无需命令行 `-a`
|
||||
|
||||
### 首尾帧模式提示词
|
||||
|
||||
每条文案生成:
|
||||
- `imagePrompt`:起始帧画面(英文,与 single 模式复用同一字段)
|
||||
- `lastFramePrompt`:结束帧画面(英文)
|
||||
- `videoPrompt`:过渡描述(英文,给 VEO)
|
||||
- `videoPrompt`:过渡描述(英文,给 VEO/Kling)
|
||||
|
||||
**首尾帧提示词设计原则**:
|
||||
|
||||
|
||||
@@ -151,7 +151,7 @@ node scripts/pipeline.js create-account \
|
||||
--id military \
|
||||
--name "军事账号" \
|
||||
--desc "军事主题短视频,暗黑漫画风格" \
|
||||
--video-model veo3-fast \
|
||||
--video-model veo3-fast-frames \
|
||||
--references ./ref1.png,./ref2.png
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,544 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Kling Video Generator - 图生视频工具(可灵模型)
|
||||
*
|
||||
* 功能:
|
||||
* - 提交图生视频任务(Kling 模型)
|
||||
* - 支持单图和首尾帧模式
|
||||
* - 轮询直到完成(60-300秒)
|
||||
* - 失败自动优化提示词重试(最多3次)
|
||||
* - 批量并行生成 + manifest.json 文案透传
|
||||
*
|
||||
* 用法:
|
||||
* node kling-video-generator.js --image ./ref.jpg --prompt "zoom in slowly" -o ./output
|
||||
* node kling-video-generator.js --image ./first.jpg --last-frame ./last.jpg --prompt "transition" -o ./output
|
||||
* node kling-video-generator.js batch ./manifest.json -o ./output
|
||||
*/
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
const https = require('https')
|
||||
const http = require('http')
|
||||
|
||||
// ============================================================================
|
||||
// 配置
|
||||
// ============================================================================
|
||||
|
||||
function loadConfig() {
|
||||
const configPath = path.join(__dirname, '..', '..', 'config.json')
|
||||
if (fs.existsSync(configPath)) {
|
||||
return JSON.parse(fs.readFileSync(configPath, 'utf-8'))
|
||||
}
|
||||
return {}
|
||||
}
|
||||
|
||||
const cfg = loadConfig()
|
||||
|
||||
const Config = {
|
||||
baseUrl: cfg.kelingApiBaseUrl ,
|
||||
apiKey: cfg.kelingApiKey || '',
|
||||
model: cfg.kelingModel || 'Kling-V2-5-Turbo',
|
||||
pollInterval: 10000,
|
||||
maxPollTime: 600000,
|
||||
maxRetries: 3,
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 提示词优化(失败时自动调整)
|
||||
// ============================================================================
|
||||
|
||||
const PromptOptimizer = {
|
||||
optimize(prompt, failReason, attempt) {
|
||||
let optimized = prompt
|
||||
|
||||
if (attempt === 1) {
|
||||
optimized = simplifyPrompt(prompt)
|
||||
console.log(` 重试策略: 简化提示词`)
|
||||
}
|
||||
|
||||
if (attempt === 2) {
|
||||
optimized = `${simplifyPrompt(prompt)}, smooth motion, high quality`
|
||||
console.log(` 重试策略: 简化 + 安全后缀`)
|
||||
}
|
||||
|
||||
if (attempt >= 3) {
|
||||
optimized = extractCoreSubject(prompt)
|
||||
console.log(` 重试策略: 极简提示词`)
|
||||
}
|
||||
|
||||
return optimized
|
||||
}
|
||||
}
|
||||
|
||||
function simplifyPrompt(prompt) {
|
||||
const parts = prompt.split(',').map(s => s.trim())
|
||||
return parts.slice(0, 3).join(', ')
|
||||
}
|
||||
|
||||
function extractCoreSubject(prompt) {
|
||||
const match = prompt.match(/^([^.!,]+)/)
|
||||
return match ? match[1].trim() : 'cinematic motion'
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// API
|
||||
// ============================================================================
|
||||
|
||||
const KlingApi = {
|
||||
async create(imageUrl, prompt, options = {}) {
|
||||
const {
|
||||
aspectRatio = '9:16',
|
||||
model = Config.model,
|
||||
lastFrameUrl = '',
|
||||
} = options
|
||||
|
||||
const images = []
|
||||
if (imageUrl) images.push(imageUrl)
|
||||
if (lastFrameUrl) images.push(lastFrameUrl)
|
||||
|
||||
const mode = lastFrameUrl ? '首尾帧' : '单图'
|
||||
|
||||
const body = {
|
||||
model,
|
||||
prompt,
|
||||
images,
|
||||
aspect_ratio: aspectRatio,
|
||||
}
|
||||
|
||||
console.log(`\n📡 提交 Kling 视频任务 [${mode}]`)
|
||||
console.log(` 模型: ${model}`)
|
||||
console.log(` 提示词: ${prompt.substring(0, 80)}...`)
|
||||
if (lastFrameUrl) {
|
||||
console.log(` 起始帧: ${imageUrl.substring(0, 60)}...`)
|
||||
console.log(` 结束帧: ${lastFrameUrl.substring(0, 60)}...`)
|
||||
} else {
|
||||
console.log(` 参考图: ${imageUrl ? imageUrl.substring(0, 60) + '...' : '无'}`)
|
||||
}
|
||||
console.log(` 画幅: ${aspectRatio}`)
|
||||
|
||||
const res = await fetch(`${Config.baseUrl}/v1/video/create`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Accept': 'application/json',
|
||||
'Authorization': `Bearer ${Config.apiKey}`,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
})
|
||||
|
||||
const result = await res.json()
|
||||
|
||||
if (!result.id) {
|
||||
throw new Error(`Kling 提交失败: ${JSON.stringify(result)}`)
|
||||
}
|
||||
|
||||
console.log(` 任务 ID: ${result.id}`)
|
||||
return result.id
|
||||
},
|
||||
|
||||
async query(taskId) {
|
||||
const res = await fetch(`${Config.baseUrl}/v1/video/query?id=${taskId}`, {
|
||||
headers: {
|
||||
'Authorization': `Bearer ${Config.apiKey}`,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
})
|
||||
|
||||
return await res.json()
|
||||
},
|
||||
|
||||
async poll(taskId) {
|
||||
const startTime = Date.now()
|
||||
let lastProgress = 0
|
||||
|
||||
console.log(`\n⏳ 等待 Kling 视频生成(预计 60-300 秒)...`)
|
||||
|
||||
while (Date.now() - startTime < Config.maxPollTime) {
|
||||
const task = await KlingApi.query(taskId)
|
||||
|
||||
if (task.status === 'completed') {
|
||||
console.log(`\n✅ 视频生成完成!`)
|
||||
console.log(` 视频: ${task.video_url}`)
|
||||
return {
|
||||
success: true,
|
||||
videoUrl: task.video_url,
|
||||
}
|
||||
}
|
||||
|
||||
if (task.status === 'failed') {
|
||||
throw new Error(task.error || task.message || 'Kling 生成失败')
|
||||
}
|
||||
|
||||
const progress = task.progress || 0
|
||||
if (progress !== lastProgress) {
|
||||
lastProgress = progress
|
||||
const elapsed = Math.round((Date.now() - startTime) / 1000)
|
||||
process.stdout.write(` 进度: ${progress}% 已等待: ${elapsed}s 状态: ${task.status}\r`)
|
||||
}
|
||||
|
||||
await new Promise(r => setTimeout(r, Config.pollInterval))
|
||||
}
|
||||
|
||||
throw new Error(`Kling 生成超时 (${Config.maxPollTime / 1000}s)`)
|
||||
},
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 图片下载工具
|
||||
// ============================================================================
|
||||
|
||||
async function download(url, outputPath) {
|
||||
const protocol = url.startsWith('https') ? https : http
|
||||
return new Promise((resolve, reject) => {
|
||||
const file = fs.createWriteStream(outputPath)
|
||||
protocol.get(url, (response) => {
|
||||
if (response.statusCode >= 300 && response.statusCode < 400 && response.headers.location) {
|
||||
file.close()
|
||||
fs.unlinkSync(outputPath)
|
||||
return download(response.headers.location, outputPath).then(resolve).catch(reject)
|
||||
}
|
||||
response.pipe(file)
|
||||
file.on('finish', () => { file.close(); resolve(outputPath) })
|
||||
}).on('error', (err) => {
|
||||
file.close()
|
||||
if (fs.existsSync(outputPath)) fs.unlinkSync(outputPath)
|
||||
reject(err)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 核心流程(单任务带重试)
|
||||
// ============================================================================
|
||||
|
||||
async function generate(imageUrl, prompt, options = {}) {
|
||||
const { outputDir = './output', aspectRatio = '9:16' } = options
|
||||
|
||||
if (!Config.apiKey) throw new Error('未配置 kelingApiKey,请在 config.json 中添加')
|
||||
fs.mkdirSync(outputDir, { recursive: true })
|
||||
|
||||
let currentPrompt = prompt
|
||||
let lastError = null
|
||||
|
||||
for (let attempt = 0; attempt <= Config.maxRetries; attempt++) {
|
||||
try {
|
||||
if (attempt > 0) {
|
||||
currentPrompt = PromptOptimizer.optimize(prompt, lastError, attempt)
|
||||
console.log(`\n🔄 第 ${attempt} 次重试`)
|
||||
console.log(` 新提示词: ${currentPrompt}`)
|
||||
}
|
||||
|
||||
const taskId = await KlingApi.create(imageUrl, currentPrompt, { aspectRatio, lastFrameUrl: options.lastFrameUrl })
|
||||
const result = await KlingApi.poll(taskId)
|
||||
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-')
|
||||
const videoFile = path.join(outputDir, `${timestamp}_kling.mp4`)
|
||||
await download(result.videoUrl, videoFile)
|
||||
console.log(` 下载完成: ${videoFile}`)
|
||||
|
||||
return {
|
||||
success: true,
|
||||
taskId,
|
||||
prompt: currentPrompt,
|
||||
originalPrompt: prompt,
|
||||
attempts: attempt + 1,
|
||||
files: [videoFile],
|
||||
}
|
||||
|
||||
} catch (err) {
|
||||
lastError = err.message
|
||||
console.error(` ❌ 第 ${attempt + 1} 次失败: ${err.message}`)
|
||||
|
||||
if (attempt < Config.maxRetries) {
|
||||
console.log(` 等待 5 秒后重试...`)
|
||||
await new Promise(r => setTimeout(r, 5000))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`Kling 视频生成失败(已重试 ${Config.maxRetries} 次): ${lastError}`)
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 批量并行生成(支持 manifest.json 输入输出)
|
||||
// ============================================================================
|
||||
|
||||
async function batchGenerate(tasks, options = {}) {
|
||||
const { outputDir = './output' } = options
|
||||
let aspectRatio = options.aspectRatio || '9:16'
|
||||
const concurrency = options.concurrency || 2
|
||||
|
||||
if (!Config.apiKey) throw new Error('未配置 kelingApiKey,请在 config.json 中添加')
|
||||
fs.mkdirSync(outputDir, { recursive: true })
|
||||
|
||||
// 支持 manifest 格式
|
||||
if (tasks.items && Array.isArray(tasks.items)) {
|
||||
if (tasks.format || tasks.defaultFormat) {
|
||||
aspectRatio = tasks.format || tasks.defaultFormat || aspectRatio
|
||||
}
|
||||
tasks = tasks.items.map(item => ({
|
||||
image: item.url || item.image || '',
|
||||
prompt: item.videoPrompt || item.prompt || 'cinematic motion',
|
||||
text: item.text || item.caption || '',
|
||||
keyword: item.keyword || '',
|
||||
keywordColor: item.keywordColor || '',
|
||||
file: item.file || '',
|
||||
lastFrameUrl: item.lastFrameUrl || '',
|
||||
}))
|
||||
}
|
||||
|
||||
// Phase 1: 并行提交
|
||||
const mode = tasks.some(t => t.lastFrameUrl) ? '首尾帧' : '单图'
|
||||
console.log(`\n📡 并行提交 ${tasks.length} 个 Kling 视频任务(并发: ${concurrency},模式: ${mode})...`)
|
||||
|
||||
const submitted = []
|
||||
for (let i = 0; i < tasks.length; i += concurrency) {
|
||||
const batch = tasks.slice(i, i + concurrency)
|
||||
const batchResults = await Promise.allSettled(
|
||||
batch.map(async (task, j) => {
|
||||
const idx = i + j
|
||||
const prompt = task.videoPrompt || task.prompt
|
||||
console.log(` [${idx + 1}/${tasks.length}] 提交: ${prompt.substring(0, 50)}...`)
|
||||
try {
|
||||
const taskId = await KlingApi.create(task.image, prompt, { aspectRatio, lastFrameUrl: task.lastFrameUrl })
|
||||
return { idx, taskId, task, error: null }
|
||||
} catch (err) {
|
||||
console.error(` [${idx + 1}] 提交失败: ${err.message}`)
|
||||
return { idx, taskId: null, task, error: err.message }
|
||||
}
|
||||
})
|
||||
)
|
||||
submitted.push(...batchResults.map(r => r.value || r.reason))
|
||||
}
|
||||
|
||||
const pendingTasks = submitted.filter(s => s.taskId)
|
||||
|
||||
if (pendingTasks.length === 0) {
|
||||
console.error('\n❌ 所有任务提交失败')
|
||||
return tasks.map((task, idx) => ({
|
||||
success: false, ...task,
|
||||
error: (submitted.find(s => s.idx === idx) || {}).error || '提交失败',
|
||||
}))
|
||||
}
|
||||
|
||||
// Phase 2: 并行轮询
|
||||
console.log(`\n⏳ 并行等待 ${pendingTasks.length} 个视频生成...`)
|
||||
|
||||
const pollResults = await Promise.allSettled(
|
||||
pendingTasks.map(async ({ idx, taskId, task }) => {
|
||||
const prompt = task.videoPrompt || task.prompt
|
||||
const result = await pollWithRetry(taskId, prompt, { outputDir, aspectRatio, imageUrl: task.image, lastFrameUrl: task.lastFrameUrl })
|
||||
return { idx, ...result, task }
|
||||
})
|
||||
)
|
||||
|
||||
// 合并结果
|
||||
const results = []
|
||||
for (let i = 0; i < tasks.length; i++) {
|
||||
const submittedInfo = submitted.find(s => s.idx === i)
|
||||
if (!submittedInfo || !submittedInfo.taskId) {
|
||||
results.push({ success: false, ...tasks[i], error: submittedInfo?.error || '提交失败' })
|
||||
continue
|
||||
}
|
||||
const pollResult = pollResults.find(r => {
|
||||
if (r.status === 'fulfilled') return r.value.idx === i
|
||||
return false
|
||||
})
|
||||
if (pollResult && pollResult.status === 'fulfilled') {
|
||||
results.push({ success: true, ...tasks[i], ...pollResult.value })
|
||||
} else {
|
||||
const reason = pollResult?.reason?.message || '生成失败'
|
||||
results.push({ success: false, ...tasks[i], error: reason })
|
||||
}
|
||||
}
|
||||
|
||||
const ok = results.filter(r => r.success).length
|
||||
console.log(`\n✨ 批量完成: ${ok}/${tasks.length} 成功`)
|
||||
|
||||
// 输出 manifest.json
|
||||
const manifestItems = results
|
||||
.filter(r => r.success && r.files && r.files.length > 0)
|
||||
.map(r => {
|
||||
const item = {
|
||||
file: path.basename(r.files[0]),
|
||||
duration: 6,
|
||||
}
|
||||
if (r.text) item.text = r.text
|
||||
if (r.caption) item.caption = r.caption
|
||||
if (r.keyword) item.keyword = r.keyword
|
||||
if (r.keywordColor) item.keywordColor = r.keywordColor
|
||||
return item
|
||||
})
|
||||
|
||||
if (manifestItems.length > 0 && !options.skipManifestWrite) {
|
||||
const manifestPath = path.join(outputDir, 'manifest.json')
|
||||
fs.writeFileSync(manifestPath, JSON.stringify({ items: manifestItems }, null, 2))
|
||||
console.log(` 已生成 manifest.json(${manifestItems.length} 条,文案与视频对应)`)
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
|
||||
/**
|
||||
* 轮询 + 失败重试(单任务)
|
||||
*/
|
||||
async function pollWithRetry(taskId, prompt, options = {}) {
|
||||
let currentTaskId = taskId
|
||||
let currentPrompt = prompt
|
||||
let lastError = null
|
||||
|
||||
for (let attempt = 0; attempt <= Config.maxRetries; attempt++) {
|
||||
try {
|
||||
if (attempt > 0) {
|
||||
currentPrompt = PromptOptimizer.optimize(prompt, lastError, attempt)
|
||||
console.log(`\n 🔄 重试 (任务 ${currentTaskId.substring(0, 8)}...): ${currentPrompt.substring(0, 50)}`)
|
||||
currentTaskId = await KlingApi.create(
|
||||
options.imageUrl || '',
|
||||
currentPrompt,
|
||||
{ aspectRatio: options.aspectRatio, lastFrameUrl: options.lastFrameUrl || '' }
|
||||
)
|
||||
}
|
||||
|
||||
const result = await KlingApi.poll(currentTaskId)
|
||||
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-')
|
||||
const videoFile = path.join(options.outputDir || './output', `${timestamp}_kling.mp4`)
|
||||
await download(result.videoUrl, videoFile)
|
||||
|
||||
return {
|
||||
taskId: currentTaskId,
|
||||
prompt: currentPrompt,
|
||||
originalPrompt: prompt,
|
||||
attempts: attempt + 1,
|
||||
file: videoFile,
|
||||
files: [videoFile],
|
||||
duration: 6,
|
||||
}
|
||||
} catch (err) {
|
||||
lastError = err.message
|
||||
if (attempt < Config.maxRetries) {
|
||||
await new Promise(r => setTimeout(r, 5000))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`重试 ${Config.maxRetries} 次后仍失败: ${lastError}`)
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// CLI
|
||||
// ============================================================================
|
||||
|
||||
function showHelp() {
|
||||
console.log(`
|
||||
🎬 Kling Video Generator - 图生视频工具(可灵模型)
|
||||
|
||||
用法:
|
||||
node kling-video-generator.js --image <url> --prompt "指令" [options]
|
||||
node kling-video-generator.js --image <url> --last-frame <url> --prompt "过渡" [options]
|
||||
node kling-video-generator.js batch <manifest.json> [options]
|
||||
|
||||
选项:
|
||||
-o, --output <dir> 输出目录 (默认: ./output)
|
||||
-a, --ar <ratio> 宽高比 (默认: 9:16)
|
||||
--model <model> 模型名称 (默认: ${Config.model})
|
||||
--last-frame <url> 结束帧 URL(首尾帧模式)
|
||||
--retries <n> 失败重试次数 (默认: 3)
|
||||
-h, --help 帮助
|
||||
|
||||
模式:
|
||||
单图模式: --image <url> --prompt "运动描述"
|
||||
首尾帧模式: --image <首帧url> --last-frame <尾帧url> --prompt "过渡描述"
|
||||
|
||||
示例:
|
||||
# 单图
|
||||
node kling-video-generator.js --image http://img.com/ref.jpg --prompt "zoom in" -a 16:9
|
||||
|
||||
# 首尾帧
|
||||
node kling-video-generator.js --image http://img.com/first.jpg --last-frame http://img.com/last.jpg --prompt "过渡" -a 16:9
|
||||
|
||||
# 批量(自动检测单图/首尾帧)
|
||||
node kling-video-generator.js batch ./manifest.json -o ./videos
|
||||
`)
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2)
|
||||
|
||||
if (args.includes('-h') || args.includes('--help') || args.length === 0) {
|
||||
showHelp()
|
||||
return
|
||||
}
|
||||
|
||||
let command = 'single'
|
||||
let params = []
|
||||
const options = {
|
||||
outputDir: './output',
|
||||
aspectRatio: '9:16',
|
||||
imageUrl: '',
|
||||
lastFrameUrl: '',
|
||||
prompt: '',
|
||||
}
|
||||
|
||||
let i = 0
|
||||
if (args[0] === 'batch') {
|
||||
command = 'batch'
|
||||
i = 1
|
||||
}
|
||||
|
||||
while (i < args.length) {
|
||||
const arg = args[i]
|
||||
if (arg === '-o' || arg === '--output') {
|
||||
options.outputDir = args[++i]
|
||||
} else if (arg === '-a' || arg === '--ar') {
|
||||
options.aspectRatio = args[++i]
|
||||
} else if (arg === '--model') {
|
||||
Config.model = args[++i]
|
||||
} else if (arg === '--image') {
|
||||
options.imageUrl = args[++i]
|
||||
} else if (arg === '--last-frame') {
|
||||
options.lastFrameUrl = args[++i]
|
||||
} else if (arg === '--prompt') {
|
||||
options.prompt = args[++i]
|
||||
} else if (arg === '--retries') {
|
||||
Config.maxRetries = parseInt(args[++i], 10)
|
||||
} else {
|
||||
params.push(arg)
|
||||
}
|
||||
i++
|
||||
}
|
||||
|
||||
if (command === 'batch') {
|
||||
const filePath = params[0]
|
||||
if (!filePath || !fs.existsSync(filePath)) {
|
||||
console.error('请提供 manifest.json 路径')
|
||||
process.exit(1)
|
||||
}
|
||||
const tasks = JSON.parse(fs.readFileSync(filePath, 'utf-8'))
|
||||
await batchGenerate(tasks, options)
|
||||
} else {
|
||||
if (!options.imageUrl) {
|
||||
console.error('请提供 --image 参数(图片 URL)')
|
||||
process.exit(1)
|
||||
}
|
||||
if (!options.prompt) {
|
||||
console.error('请提供 --prompt 参数')
|
||||
process.exit(1)
|
||||
}
|
||||
await generate(options.imageUrl, options.prompt, options)
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 导出
|
||||
// ============================================================================
|
||||
|
||||
module.exports = { generate, batchGenerate, pollWithRetry, KlingApi, PromptOptimizer }
|
||||
|
||||
if (require.main === module) {
|
||||
main().catch(err => {
|
||||
console.error(`\n❌ 错误: ${err.message}`)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
@@ -337,7 +337,7 @@ async function phaseVideos(manifest, manifestPath, options) {
|
||||
ensureDir(videosDir)
|
||||
|
||||
const accountConfig = options.accountConfig || {}
|
||||
const videoModel = manifest.videoModel || accountConfig.videoModel || 'veo3-fast'
|
||||
const videoModel = manifest.videoModel || accountConfig.videoModel || 'veo3-fast-frames'
|
||||
|
||||
const items = manifest.items.filter(it =>
|
||||
it.status === 'done' && it.url && it.videoPrompt && !it.video
|
||||
@@ -346,8 +346,11 @@ async function phaseVideos(manifest, manifestPath, options) {
|
||||
|
||||
// 选择生成器
|
||||
let generator
|
||||
if (videoModel.includes('grok')) {
|
||||
const modelLower = videoModel.toLowerCase()
|
||||
if (modelLower.includes('grok')) {
|
||||
generator = require('./grok-video-generator')
|
||||
} else if (modelLower.includes('kling')) {
|
||||
generator = require('./kling-video-generator')
|
||||
} else {
|
||||
generator = require('./veo-video-generator')
|
||||
}
|
||||
@@ -703,7 +706,7 @@ function initManifest(options) {
|
||||
const manifest = {
|
||||
account: accountId,
|
||||
imageModel: accountConfig.imageModel || 'gemini',
|
||||
videoModel: accountConfig.videoModel || 'veo3-fast',
|
||||
videoModel: accountConfig.videoModel || 'veo3-fast-frames',
|
||||
format: accountConfig.defaultFormat || '9:16',
|
||||
mode: resolvedMode,
|
||||
references,
|
||||
|
||||
Reference in New Issue
Block a user