feat(video-from-script): 重构工作流为子Agent分步执行并新增提示词模板系统
将视频制作工作流拆分为独立子步骤:分镜 → 图片提示词 → 生图 → 视频提示词 → 生视频 → 成片,每步由子Agent独立执行。引入prompts/目录统一管理提示词模板(分镜.md、图片提示词.md、视频提示词.md),通过account.json的storyboardPrompt/imageStylePrompt/videoStylePrompt字段引用。 变更内容: - 新增confirmed机制和pipeline.js confirm命令,生图后必须人工确认才能继续 - manifest schema改用shotDesc/narration/duration/directorRef替代旧字段 - 文件命名规则从keyword改为slug(从shotDesc/narration派生) - 删除旧的storyboard-rules.md和prompt-rules.md - pipeline.js脚本拆分为lib/目录下的独立模块(cmd-init/cmd-confirm/cmd-validate/phase-*) - 新增cmd-create-account支持一键创建带prompts目录的账号 - capcut_assemble支持narration字段替代text作为字幕源 - 新增.gitclaude/settings.json权限配置
This commit is contained in:
@@ -7,12 +7,13 @@ description: 素材生产路由。根据用户意图分发到对应子技能:i
|
||||
|
||||
## 强制规则
|
||||
|
||||
1. **工作流不可跳步**:分镜(纯叙事)→ Prompt 生成(分镜+风格)→ Pipeline 执行。每阶段之间必须审查结果
|
||||
1. **工作流不可跳步**:分镜 → 图片提示词 → 生图 → 视频提示词 → 生视频 → TTS+成片。每阶段之间必须审查结果
|
||||
2. **manifest.json 是唯一状态源**:任何操作(生图、上传、替换素材)完成后必须立即回写 manifest
|
||||
3. **禁止 curl 调用生图/生视频 API**:必须通过 `pipeline.js` 或对应 generator 脚本执行
|
||||
4. **并行优先**:多个独立子任务必须用子 agent 并行,不要在主对话中串行完成
|
||||
5. **prompts/*.md 只被子 Agent 读取**:主 Agent 读 account.json + styles/*.md 获取风格信息,不读子 Agent 提示词模板
|
||||
|
||||
**禁止**:跳过分镜 / 分镜阶段读风格 / 不更新 manifest 就继续 / 一口气跑完 pipeline 不审查
|
||||
**禁止**:跳过分镜 / 不更新 manifest 就继续 / 一口气跑完 pipeline 不审查 / 主 Agent 替代子 Agent 生成提示词
|
||||
|
||||
---
|
||||
|
||||
@@ -56,7 +57,7 @@ Agent 创建 manifest.json 后,用 `pipeline.js` 分阶段执行。**不要一
|
||||
|
||||
| 角色 | 职责 |
|
||||
|------|------|
|
||||
| **Agent**(你) | 读取 account.json + style.md → **分镜规划** → 从分镜生成 imagePrompt/videoPrompt → 写出 manifest.json → 审查每阶段结果 |
|
||||
| **Agent**(你) | 读取 account.json + styles/*.md → **分镜规划** → 图片提示词生成 → 视频提示词生成 → 审查每阶段结果 |
|
||||
| **Pipeline** | 机械执行:生图 → 上传 → 生视频 → TTS → 成片。每完成一个 item 写盘,支持断点续跑 |
|
||||
|
||||
### 执行步骤
|
||||
@@ -90,65 +91,81 @@ Step -1: 意图确认(进入任何步骤前必须完成,逐项确认,缺
|
||||
→ 以上 5 项全部确认后,agent 写出完整执行计划,让用户最终确认:
|
||||
|
||||
执行计划示例(根据实际任务调整):
|
||||
1. 读取 {account} 账号配置(id = 目录名)+ 风格文件(style.md)
|
||||
2. 根据用户文案生成分镜表(N shot)
|
||||
3. 分镜 + 风格 → 生成英文 prompts(imagePrompt + videoPrompt)
|
||||
1. 读取 {account} 账号配置(id = 目录名)+ styles/*.md
|
||||
2. 子 Agent 读取 prompts/分镜.md → 根据用户文案生成分镜表(N shot)
|
||||
3. 子 Agent 读取 prompts/图片提示词.md → 为每个 shot 生成 imagePrompt
|
||||
4. pipeline.js init → 创建 manifest.json + 输出目录
|
||||
5. pipeline.js run --phase images → 生图 → 人工审查
|
||||
6. pipeline.js run --phase upload,videos → 上传 + 生成视频
|
||||
7. pipeline.js run --phase tts,assemble → TTS + 成片
|
||||
5. pipeline.js run --phase images → 生图 → 人工审查确认(可选)
|
||||
6. 子 Agent 读取 prompts/视频提示词.md → 为每个 shot 生成 videoPrompt
|
||||
7. pipeline.js run --phase upload,videos → 上传 + 生成视频
|
||||
8. pipeline.js run --phase tts,assemble → TTS + 成片
|
||||
|
||||
用户确认 "开始" → 进入 Step 0
|
||||
用户修改 → 调整计划后重新输出
|
||||
→ 禁止在用户未确认执行计划的情况下进入 Step 0
|
||||
|
||||
Step 0: 前置检查(账号+风格校验)
|
||||
- 读取 根目录 accounts/{account}/account.json,检查 styles 字段是否配置了风格文件
|
||||
- 如果账号不存在或没有风格:
|
||||
Step 0: 前置检查(账号+风格+提示词模板校验)
|
||||
- 读取 根目录 accounts/{account}/account.json
|
||||
- 检查 prompts/ 目录下的提示词模板是否存在(分镜.md、图片提示词.md、视频提示词.md)
|
||||
- 检查 styles/ 目录下是否有风格文件
|
||||
- 如果账号不存在或缺少模板/风格:
|
||||
→ 暂停流程,通过 CLI 创建:`pipeline.js create-account --id <id> --name <名称> --references ./ref.png`
|
||||
→ 然后编辑 `styles/*.md` 完善提示词策略
|
||||
→ 然后编辑 prompts/*.md 和 styles/*.md
|
||||
- 校验账号完整性:`pipeline.js validate-account --account <id>`
|
||||
- 有风格则继续 Step 1
|
||||
- 全部就绪则继续 Step 1
|
||||
|
||||
Step 1: 分镜规划(子 Agent 执行)
|
||||
- 主 Agent 将用户文案 + 约束交给子 Agent
|
||||
- 子 Agent 读取 references/storyboard-rules.md,按要求输出分镜表
|
||||
- 主 Agent 审查分镜表(景别交替、hook 设置、时长合理)
|
||||
- 展示给用户确认,确认后进入 Step 2
|
||||
Step 1: 分镜脚本生成(子 Agent 执行)
|
||||
- 读取 account.json 中的 storyboardPrompt 字段,定位分镜模板文件(如 prompts/分镜.md)
|
||||
- 主 Agent 将用户文案 + 模板交给子 Agent
|
||||
- 子 Agent 按模板要求输出分镜表 JSON:
|
||||
```json
|
||||
[{"id":1,"shotDesc":"英文画面描述,40-80词","narration":"中文口播旁白,≤22字","duration":5,"directorRef":"tarantino"}]
|
||||
```
|
||||
- 主 Agent 审查分镜表(时长合理、隐性动势完整、directorRef 已填)
|
||||
- 展示给用户确认,确认后进入 Step 2-A
|
||||
|
||||
Step 2: Prompt 生成 + Manifest 初始化(分镜 + 风格 → 英文 prompts → pipeline.js init)
|
||||
- 输入:分镜表 + style.md + account.json
|
||||
- 子 Agent 将每个 shot 的中文画面描述结合风格文件,生成:
|
||||
· imagePrompt(英文画面描述,给 Gemini/MJ)
|
||||
· videoPrompt(英文运动描述,给 Grok/VEO/Kling)
|
||||
· keyword, keywordColor
|
||||
- **禁止 AI 手写 manifest.json**,必须通过脚本初始化:
|
||||
Step 2-A: 生成图片提示词(子 Agent 执行)
|
||||
- 读取 account.json 中的 imageStylePrompt 字段,定位图片提示词模板(如 prompts/图片提示词.md)
|
||||
- 子 Agent 为每个 shot 生成 imagePrompt:
|
||||
- 入参:shotDesc + narration(情绪参考)+ directorRef(光影策略)+ 目标模型
|
||||
- 出参:imagePrompt(可直接送给图片模型的英文提示词)
|
||||
- 主 Agent 审查 imagePrompt 质量(shotDesc 内容完整保留、光影词库对应 directorRef)
|
||||
|
||||
Step 2-B: 生成静态分镜图 + Manifest 初始化
|
||||
- 组装 items 并初始化 manifest(**不含 videoPrompt**):
|
||||
```bash
|
||||
node pipeline.js init --account <id> --mode <single|framePair> \
|
||||
--items '[{"text":"文案","imagePrompt":"...","videoPrompt":"...","keyword":"关键词","keywordColor":"#FF6B35"}]'
|
||||
--items '[{"shotDesc":"...","narration":"...","duration":5,"imagePrompt":"...","directorRef":"tarantino"}]'
|
||||
```
|
||||
- 脚本自动从 account.json 继承:imageModel、videoModel、format、references
|
||||
- 脚本自动创建目录、校验必填字段、设置 status=pending
|
||||
- AI 只负责创意内容(text、imagePrompt、videoPrompt、keyword),不碰结构字段
|
||||
- 首尾帧模式额外要求:每个 item 必须有 `lastFramePrompt`(`imagePrompt` 作为第一帧,不需要单独的 `firstFramePrompt`)
|
||||
- init 返回 manifest 路径,后续命令使用该路径
|
||||
- 所有 item.confirmed = false
|
||||
- 生成分镜图:`pipeline.js run --manifest <path> --phase images`
|
||||
- 参考图在此阶段介入(Gemini 图生图 / MJ --sref)
|
||||
- 首尾帧模式额外要求:每个 item 必须有 `lastFramePrompt`
|
||||
|
||||
Step 3: 生图 → 审查
|
||||
跑 images 阶段。完成后:
|
||||
- 用户指定"自行选图"→ Agent 自动检查数量对上文案数量就通过继续
|
||||
- 否则 → 暂停,等用户审查。不合格则删除/调 prompt 重跑,不进入下一步
|
||||
Step 2-C: 人工确认(可选卡点)
|
||||
- 展示所有分镜图给用户
|
||||
- 用户可:确认全部 / 替换 MJ 候选图(改 item.file = item.candidates[N]) / 删除不合格 item / 跳过确认直接继续
|
||||
- 用户确认后:`node pipeline.js confirm --manifest <path> --all`
|
||||
- 跳过确认时:批量设置 `confirmed = true`,直接进入 Step 3
|
||||
|
||||
生图模型
|
||||
- 支持模型:gemini / mj / kling
|
||||
- 降级链:gemini → mj → kling → gemini(循环)
|
||||
- 触发:连续失败→ Agent 换下一个模型重跑失败项
|
||||
- 操作:`pipeline.js run --manifest <path> --phase images --retry-failed --image-model <新模型>`
|
||||
Step 3-A: 生成视频提示词(子 Agent 执行)
|
||||
- 读取 account.json 中的 videoStylePrompt 字段,定位视频提示词模板(如 prompts/视频提示词.md)
|
||||
- 子 Agent 为每个 shot 生成 videoPrompt:
|
||||
- 入参:shotDesc + directorRef(运动策略)+ 已确认的分镜图 + 目标模型
|
||||
- 出参:videoPrompt(描述镜头运动的英文提示词)
|
||||
- Agent 将 videoPrompt 回写到 manifest items(直接编辑 manifest.json 的每个 item)
|
||||
- 主 Agent 审查 videoPrompt 质量(描述运动而非内容、字数≤50)
|
||||
|
||||
Step 4: 上传 + 生视频(可选,图文成片跳过此步)
|
||||
跑 upload + videos 阶段。首尾帧模式检查过渡连贯性。
|
||||
Step 3-B: 生成视频片段
|
||||
- 上传 + 生成视频:`pipeline.js run --manifest <path> --phase upload,videos`
|
||||
- 跳过确认时由 Step 2-C 自动批量设置 confirmed=true
|
||||
- 首尾帧模式检查过渡连贯性
|
||||
|
||||
Step 5: TTS + 成片
|
||||
跑 tts + assemble 阶段。检查字幕准确、BGM 不盖配音。
|
||||
Step 4: TTS + 成片
|
||||
- 跑 tts + assemble 阶段:`pipeline.js run --manifest <path> --phase tts,assemble`
|
||||
- TTS 使用 narration 字段(口播旁白)
|
||||
- 检查字幕准确、BGM 不盖配音
|
||||
```
|
||||
|
||||
> 命令语法见下方「CLI 参考」,不在此处重复。
|
||||
@@ -163,15 +180,18 @@ node pipeline.js create-account --id <id> --name <名称> \
|
||||
# 校验账号完整性
|
||||
node pipeline.js validate-account --account <id>
|
||||
|
||||
# 初始化 manifest(Step 2 使用,AI 只提供创意内容)
|
||||
# 初始化 manifest(Step 2-B 使用,AI 只提供创意内容,不含 videoPrompt)
|
||||
node pipeline.js init --account <id> --mode <single|framePair> \
|
||||
--items '[{"text":"...","imagePrompt":"...","videoPrompt":"...","keyword":"...","keywordColor":"..."}]'
|
||||
--items '[{"shotDesc":"...","narration":"...","duration":5,"imagePrompt":"...","directorRef":"tarantino"}]'
|
||||
# 也可从文件读取 items(适合大量数据)
|
||||
node pipeline.js init --account <id> --mode single --items-file ./items.json
|
||||
|
||||
# 校验 manifest 完整性
|
||||
node pipeline.js validate --manifest <path>
|
||||
|
||||
# 人工确认分镜图(Step 2-C,可选:跳过时 Agent 批量设置 confirmed=true)
|
||||
node pipeline.js confirm --manifest <path> --all
|
||||
|
||||
# 跑指定阶段
|
||||
node pipeline.js run --manifest <path> --phase images
|
||||
node pipeline.js run --manifest <path> --phase upload,videos
|
||||
@@ -259,72 +279,9 @@ digraph frame_pair {
|
||||
|
||||
---
|
||||
|
||||
## 多阶段执行策略
|
||||
## 视频模型与执行策略
|
||||
|
||||
用 Agent 工具串行或并行执行子技能,**阶段间必须通过质量卡点**:
|
||||
|
||||
**生图+成片(串行+人工卡点)**:
|
||||
```dot
|
||||
digraph image_then_assemble {
|
||||
rankdir=LR
|
||||
node [shape=box, style=filled, fillcolor="#f5f5f5", fontsize=11]
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n生成图片到 output/"]
|
||||
gate1 [label="人工卡点\n用户挑选图片\n删除不合格的", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent2 [label="Agent 2\ncapcut\n读取精选素材 → 组装"]
|
||||
|
||||
agent1 -> gate1 -> agent2
|
||||
}
|
||||
```
|
||||
|
||||
**配音+生图(并行+自动校验)**:
|
||||
```dot
|
||||
digraph parallel_image_tts {
|
||||
rankdir=LR
|
||||
node [shape=box, style=filled, fillcolor="#f5f5f5", fontsize=11]
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n生图", fillcolor="#e8f5e9"]
|
||||
agent2 [label="Agent 2\ncapcut\nTTS 配音", fillcolor="#e8f5e9"]
|
||||
validate [label="自动校验\n分辨率>=1024\n画幅匹配\n音频时长匹配", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent3 [label="Agent 3\ncapcut\n组装全部素材 → 成片"]
|
||||
|
||||
agent1 -> validate
|
||||
agent2 -> validate
|
||||
validate -> agent3
|
||||
}
|
||||
```
|
||||
|
||||
**图生视频 - 单图模式**:
|
||||
```dot
|
||||
digraph single_image_video {
|
||||
rankdir=LR
|
||||
node [shape=box, style=filled, fillcolor="#f5f5f5", fontsize=11]
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n生图 + videoPrompt"]
|
||||
gate1 [label="人工卡点\n用户挑选图片", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent2 [label="Agent 2\nGrok / VEO / Kling\n单图输入,并行生成视频"]
|
||||
agent3 [label="Agent 3\ncapcut\n视频片段 + 字幕 → 成片"]
|
||||
|
||||
agent1 -> gate1 -> agent2 -> agent3
|
||||
}
|
||||
```
|
||||
|
||||
**图生视频 - 首尾帧模式**:
|
||||
```dot
|
||||
digraph frame_pair_video {
|
||||
rankdir=LR
|
||||
node [shape=box, style=filled, fillcolor="#f5f5f5", fontsize=11]
|
||||
|
||||
agent1 [label="Agent 1\nimage-generator\n成对生图\n(firstFrame + lastFrame)\n可并行"]
|
||||
gate1 [label="人工卡点\n检查首尾帧连贯性\n同一场景/相似视角", shape=diamond, fillcolor="#fff9c4"]
|
||||
agent2 [label="Agent 2\nVEO / Kling\n双图输入\nimages:[first, last]"]
|
||||
agent3 [label="Agent 3\ncapcut\n视频片段 + 字幕 → 成片"]
|
||||
|
||||
agent1 -> gate1 -> agent2 -> agent3
|
||||
}
|
||||
```
|
||||
|
||||
**视频模型选择**:
|
||||
### 视频模型选择
|
||||
|
||||
| 模型 | 时长 | 画幅 | 单图 | 首尾帧 | 特点 | API |
|
||||
|------|------|------|------|--------|------|-----|
|
||||
@@ -333,11 +290,11 @@ digraph frame_pair_video {
|
||||
| Veo3-fast-frames | ~8s | 16:9, 9:16 | ✅ | ✅ | 多帧、质量最高 | jimmyai.cn |
|
||||
| Kling | 6s | 任意 | ✅ | ✅ | 快、首尾帧支持 | yunwu.ai |
|
||||
|
||||
图生视频注意事项:
|
||||
### 视频生成注意事项
|
||||
|
||||
- **并行执行**:先同时提交所有任务(并发 3),再并行轮询结果
|
||||
- 单个视频生成耗时 60-300 秒
|
||||
- 脚本内置 3 次重试,每次自动简化提示词
|
||||
- **videoPrompt 在生图阶段一并生成**
|
||||
- VEO 独有:`enhance_prompt=true` 中文增强,`enable_upsample=true` 超分
|
||||
- 配置在 `config.json`
|
||||
|
||||
@@ -377,8 +334,8 @@ node kling-video-generator.js --image <url> --prompt <prompt> -o ./videos
|
||||
output/{name}_{YYYYMMDD}_{NNN}/
|
||||
├── manifest.json # 主清单(贯穿全流程)
|
||||
├── prompts.txt # 原始提示词存档
|
||||
├── images/ # scene_{NN}_{keyword}.jpeg(首尾帧加 _last 后缀)
|
||||
├── videos/ # scene_{NN}_{keyword}.mp4(与图片对应)
|
||||
├── images/ # scene_{NN}_{slug}.jpeg(slug 从 narration/shotDesc 派生,首尾帧加 _last 后缀)
|
||||
├── videos/ # scene_{NN}_{slug}.mp4(与图片对应)
|
||||
└── urls.json # OSS 公网 URL 映射
|
||||
```
|
||||
|
||||
@@ -397,16 +354,6 @@ output/{name}_{YYYYMMDD}_{NNN}/
|
||||
|
||||
---
|
||||
|
||||
## 分镜规划规则
|
||||
|
||||
完整规则见 [storyboard-rules.md](references/storyboard-rules.md)。由子 Agent 读取并执行,主 Agent 只审查输出。
|
||||
|
||||
---
|
||||
|
||||
## 提示词生成规则
|
||||
|
||||
完整规则见 [prompt-rules.md](references/prompt-rules.md)。由子 Agent 读取并执行,主 Agent 审核提示词质量,不合格则退回重做。
|
||||
|
||||
---
|
||||
|
||||
## 质量卡点(Agent 可执行)
|
||||
|
||||
@@ -11,12 +11,18 @@
|
||||
accounts/ # 项目根目录下
|
||||
├── _template/ # 新账号模板(复制此目录创建新账号)
|
||||
│ ├── account.json
|
||||
│ ├── prompts/ # 提示词模板目录(子 Agent 系统提示词)
|
||||
│ │ └── .gitkeep
|
||||
│ ├── references/ # 参考图目录
|
||||
│ │ └── .gitkeep
|
||||
│ └── styles/ # 风格文件目录(可多个)
|
||||
│ └── .gitkeep
|
||||
└── {account}/ # 用户创建的账号(目录名 = account.json 的 id,输出目录用 name 命名)
|
||||
├── account.json
|
||||
├── prompts/ # 提示词模板(账号专属)
|
||||
│ ├── 分镜.md # 口播文案 → 分镜表
|
||||
│ ├── 图片提示词.md # shotDesc → imagePrompt
|
||||
│ └── 视频提示词.md # shotDesc → videoPrompt
|
||||
├── references/ # 参考图(所有风格共用)
|
||||
│ ├── ref_001.png
|
||||
│ └── ref_002.png
|
||||
@@ -38,6 +44,9 @@ accounts/ # 项目根目录下
|
||||
"imageModel": "gemini",
|
||||
"videoModel": "kling",
|
||||
"batchSize": 30,
|
||||
"storyboardPrompt": "prompts/分镜.md",
|
||||
"imageStylePrompt": "prompts/图片提示词.md",
|
||||
"videoStylePrompt": "prompts/视频提示词.md",
|
||||
"capcut": {
|
||||
"effects": ["录制边框 III"],
|
||||
"filter": "电影感:40",
|
||||
@@ -61,6 +70,9 @@ accounts/ # 项目根目录下
|
||||
| `imageModel` | string | 默认图片模型 |
|
||||
| `videoModel` | string | 默认视频模型 |
|
||||
| `batchSize` | number | 默认批量生成数量 |
|
||||
| `storyboardPrompt` | string | 分镜提示词模板路径(相对于账号目录) |
|
||||
| `imageStylePrompt` | string | 图片提示词模板路径(相对于账号目录) |
|
||||
| `videoStylePrompt` | string | 视频提示词模板路径(相对于账号目录) |
|
||||
| `capcut.effects` | string[] | CapCut 特效名称列表 |
|
||||
| `capcut.filter` | string | CapCut 滤镜,格式 "名称:强度" |
|
||||
| `capcut.subtitleStyle` | object | 字幕样式(字号、颜色、高亮色、加粗) |
|
||||
@@ -68,66 +80,50 @@ accounts/ # 项目根目录下
|
||||
|
||||
---
|
||||
|
||||
## 提示词模板(prompts/)
|
||||
|
||||
每个账号在 `prompts/` 目录下维护三个子 Agent 提示词模板:
|
||||
|
||||
| 文件 | 用途 | Step |
|
||||
|------|------|------|
|
||||
| `分镜.md` | 口播文案 → 分镜表(shotDesc/narration/duration/directorRef) | Step 1 |
|
||||
| `图片提示词.md` | shotDesc + directorRef → imagePrompt | Step 2-A |
|
||||
| `视频提示词.md` | shotDesc + directorRef → videoPrompt | Step 3-A |
|
||||
|
||||
这些文件定义了子 Agent 的角色、入参、出参、词库和示例。路径在 account.json 中以相对路径引用,Agent 读取 account.json 后自动定位。
|
||||
|
||||
### 与 style 文件的关系
|
||||
|
||||
- **prompts/** — 子 Agent 的系统提示词(定义 HOW 生成提示词)
|
||||
- **styles/** — 视觉规则参考(定义 WHAT 风格应该长什么样:颜色、构图、禁止项)
|
||||
- prompts 模板内部会引用 styles 中定义的色彩体系、构图规则等
|
||||
|
||||
---
|
||||
|
||||
## 风格文件(styles/)
|
||||
|
||||
每种视觉风格一个文件,文件名即风格名。文件内同时包含图片和视频的提示词策略。
|
||||
每种视觉风格一个文件,文件名即风格名。文件内包含视觉规则参考。
|
||||
|
||||
### 风格文件结构
|
||||
|
||||
```markdown
|
||||
# 风格名称(英文短横线命名)
|
||||
# 风格名称
|
||||
|
||||
一句话描述风格。
|
||||
|
||||
---
|
||||
|
||||
## 图片提示词
|
||||
|
||||
### 核心视觉要素
|
||||
<!-- 必选的视觉元素 -->
|
||||
|
||||
### 场景/背景规则
|
||||
<!-- 背景要求 -->
|
||||
## 核心视觉要素
|
||||
|
||||
### 色调方案
|
||||
<!-- 可选的色彩组合 -->
|
||||
|
||||
### 构图模式
|
||||
<!-- 支持的构图类型 -->
|
||||
|
||||
### 图片 Prompt 模板
|
||||
<!-- 生成 prompt 时的固定结构 -->
|
||||
|
||||
### 示例
|
||||
<!-- 2-3 个完整示例 -->
|
||||
|
||||
### MJ/Gemini 参数
|
||||
<!-- 模型专用后缀参数 -->
|
||||
### 模型参数(MJ/Gemini/Kling)
|
||||
|
||||
### 图片禁止项
|
||||
<!-- 不得出现的元素 -->
|
||||
|
||||
---
|
||||
|
||||
## 视频提示词
|
||||
|
||||
### 运镜规则
|
||||
<!-- 运镜方式、节奏 -->
|
||||
|
||||
### 动态元素要求
|
||||
<!-- 光影、角色动作、环境氛围 -->
|
||||
|
||||
### 视频 Prompt 模板
|
||||
<!-- VEO/Grok prompt 结构 -->
|
||||
|
||||
### 示例
|
||||
<!-- 2-3 个完整示例 -->
|
||||
|
||||
### VEO/Grok 后缀
|
||||
<!-- 模型专用后缀 -->
|
||||
|
||||
### 视频禁止项
|
||||
<!-- 不得出现的元素 -->
|
||||
### 禁止项
|
||||
```
|
||||
|
||||
### 风格文件命名
|
||||
@@ -158,12 +154,13 @@ node scripts/pipeline.js create-account \
|
||||
### 手动创建
|
||||
|
||||
1. 复制 `_template/` 目录,重命名为账号 ID
|
||||
2. 编辑 `account.json` 填写账号信息
|
||||
3. 在 `references/` 中放入参考图(所有风格共用)
|
||||
4. 上传参考图到 OSS,URL 写入 account.json:
|
||||
2. 编辑 `account.json` 填写账号信息(含 prompts 路径)
|
||||
3. 在 `prompts/` 中放入提示词模板(从 `工作流程/` 草稿区复制)
|
||||
4. 在 `references/` 中放入参考图(所有风格共用)
|
||||
5. 上传参考图到 OSS,URL 写入 account.json:
|
||||
- `node scripts/oss-upload.js ../../accounts/{id}/references/{图片文件}`
|
||||
- 将返回的 URL 写入 `styles.{styleName}.references[].url`
|
||||
5. 在 `styles/` 中创建风格文件(至少一个)
|
||||
6. 在 `styles/` 中创建风格文件(至少一个)
|
||||
|
||||
### 校验账号
|
||||
|
||||
@@ -171,7 +168,7 @@ node scripts/pipeline.js create-account \
|
||||
node scripts/pipeline.js validate-account --account military
|
||||
```
|
||||
|
||||
检查:id 匹配、必填字段、参考图完整性、风格文件存在、OSS URL 有效。
|
||||
检查:id 匹配、必填字段、prompts 模板存在、参考图完整性、风格文件存在、OSS URL 有效。
|
||||
|
||||
## 添加新风格
|
||||
|
||||
|
||||
@@ -2,19 +2,23 @@
|
||||
|
||||
> `pipeline.js init` 创建,Pipeline 执行,Agent 审查。
|
||||
>
|
||||
> **禁止 AI 手写 manifest.json**,必须通过 `pipeline.js init` 初始化。脚本从 account.json 自动继承结构字段,AI 只提供创意内容(items 的 text/imagePrompt/videoPrompt/keyword)。
|
||||
> **禁止 AI 手写 manifest.json**,必须通过 `pipeline.js init` 初始化。脚本从 account.json 自动继承结构字段,AI 只提供创意内容(items 的 shotDesc/narration/imagePrompt 等)。
|
||||
|
||||
---
|
||||
|
||||
## 创建方式
|
||||
|
||||
```bash
|
||||
# AI 生成创意内容后,通过脚本初始化
|
||||
node pipeline.js init --account military --mode single \
|
||||
--items '[{"text":"中文文案","imagePrompt":"English prompt","videoPrompt":"motion prompt","keyword":"关键词","keywordColor":"#FF6B35"}]'
|
||||
# Step 2-A 生成 imagePrompt 后,通过脚本初始化(不含 videoPrompt)
|
||||
node pipeline.js init --account 军事账号 --mode single \
|
||||
--items '[{"shotDesc":"英文画面描述","narration":"中文口播旁白","duration":5,"imagePrompt":"English prompt","directorRef":"tarantino"}]'
|
||||
|
||||
# 或从文件读取
|
||||
node pipeline.js init --account military --mode single --items-file ./items.json
|
||||
node pipeline.js init --account 军事账号 --mode single --items-file ./items.json
|
||||
|
||||
# Step 2-C 人工确认
|
||||
node pipeline.js confirm --manifest <path> --all
|
||||
node pipeline.js confirm --manifest <path> --items 1,3,5
|
||||
|
||||
# 校验已有 manifest
|
||||
node pipeline.js validate --manifest <path>
|
||||
@@ -53,11 +57,18 @@ node pipeline.js validate --manifest <path>
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| `status` | 固定写 `"pending"` |
|
||||
| `text` | 中文字幕文案 |
|
||||
| `imagePrompt` | 英文画面描述(给 Gemini/MJ) |
|
||||
| `videoPrompt` | 英文运动描述(给 Grok/VEO),描述镜头运动而非内容 |
|
||||
| `keyword` | 字幕高亮关键词 |
|
||||
| `keywordColor` | 高亮颜色 |
|
||||
| `shotDesc` | 英文分镜描述(含隐性动势,40-80词) |
|
||||
| `narration` | 中文口播旁白(≤22字) |
|
||||
| `duration` | 计划视频时长(秒),来自分镜阶段 |
|
||||
| `imagePrompt` | 英文画面描述(给 Gemini/MJ),Step 2-A 生成 |
|
||||
| `directorRef` | 导演构图参考(tarantino / kitano / fincher),三层透传 |
|
||||
| `confirmed` | 人工确认状态,默认 `false` |
|
||||
|
||||
### Agent 后续回写(Step 3-A 视频提示词)
|
||||
|
||||
| 字段 | 说明 | 写入时机 |
|
||||
|------|------|---------|
|
||||
| `videoPrompt` | 英文运动描述(给 Grok/VEO),描述镜头运动而非内容 | Step 3-A 由 Agent 回写 |
|
||||
|
||||
### Pipeline 回写(执行后)
|
||||
|
||||
@@ -67,17 +78,19 @@ node pipeline.js validate --manifest <path>
|
||||
| `file` | 生成的图片路径(相对 manifest) | images |
|
||||
| `candidates` | MJ 拆分的 4 张候选图路径(Gemini 无此字段) | images |
|
||||
| `url` | 图片 OSS 公网 URL | upload |
|
||||
| `confirmed` | 人工确认后设为 `true` | confirm |
|
||||
| `video` | 生成的视频路径 | videos |
|
||||
| `videoDuration` | 视频时长(秒),Grok=6, VEO=8 | videos |
|
||||
| `videoUrl` | 视频 OSS 公网 URL | videos |
|
||||
| `audio` | TTS 音频路径 | tts |
|
||||
| `duration` | 音频时长(秒) | tts |
|
||||
| `audioDuration` | 音频时长(秒) | tts |
|
||||
|
||||
### Agent 审查时可操作
|
||||
|
||||
- MJ 换选:`item.file = item.candidates[2]`
|
||||
- 删除不合格 item:直接从 items 数组移除,重新跑 `--phase images`
|
||||
- 调整 prompt 重跑:改 `imagePrompt`,status 改回 `pending`
|
||||
- 人工确认:`node pipeline.js confirm --manifest <path> --all`
|
||||
|
||||
---
|
||||
|
||||
@@ -86,9 +99,9 @@ node pipeline.js validate --manifest <path>
|
||||
### item 生命周期
|
||||
|
||||
```
|
||||
pending → [images] → done → [upload: url填入] → done → [videos] → done → [tts] → done
|
||||
↓ ↓
|
||||
failed failed + error
|
||||
pending → [images] → done → [confirm] → confirmed=true → [upload: url填入] → [videos] → done → [tts] → done
|
||||
↓ ↓
|
||||
failed failed + error
|
||||
```
|
||||
|
||||
status 一旦进入 `done` 就不再回退。后续阶段通过检查"有前置字段 + 无后置字段"来识别待处理 item,不依赖 status 变化。
|
||||
@@ -101,8 +114,8 @@ Agent **不需要记住这些条件**,pipeline 内部自动匹配。仅供理
|
||||
|------|------------------|
|
||||
| images | `status=pending` + 有 `imagePrompt` |
|
||||
| upload | `status=done` + 有 `file` + 无 `url` |
|
||||
| videos | `status=done` + 有 `url` + 有 `videoPrompt` + 无 `video` |
|
||||
| tts | `status=done` + 有 `text` + 无 `audio` |
|
||||
| videos | `status=done` + `confirmed=true` + 有 `url` + 有 `videoPrompt` + 无 `video` |
|
||||
| tts | `status=done` + 有 `narration`(回退 `text`) + 无 `audio` |
|
||||
|
||||
### pipeline.phases 整体状态
|
||||
|
||||
@@ -180,7 +193,9 @@ node pipeline.js run --manifest <path> --retry-failed
|
||||
```
|
||||
output/{account}_{YYYYMMDD}_{NNN}/
|
||||
├── manifest.json # 主清单
|
||||
├── images/ # scene_{NN}_{keyword}.jpeg(首尾帧加 _last,MJ 候选加 _cand{1-4})
|
||||
├── videos/ # scene_{NN}_{keyword}.mp4
|
||||
├── images/ # scene_{NN}_{slug}.jpeg(首尾帧加 _last,MJ 候选加 _cand{1-4})
|
||||
├── videos/ # scene_{NN}_{slug}.mp4
|
||||
└── audio/ # seg_001.mp3
|
||||
```
|
||||
|
||||
slug 从 `shotDesc` 派生(slugify: 保留中文和字母数字,最多 20 字符)。
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
# 提示词生成规则
|
||||
|
||||
**前置条件**:账号必须有风格文件。无风格 → 提醒用户创建,不跳过。
|
||||
|
||||
**输入**:分镜表 + style.md + account.json
|
||||
**输出**:每个 shot 生成 imagePrompt / videoPrompt / keyword / keywordColor(英文)
|
||||
|
||||
## 单图模式提示词
|
||||
|
||||
每条文案生成:
|
||||
- `imagePrompt`:画面描述(英文,给 Gemini/MJ/Kling)
|
||||
- `videoPrompt`:运动描述(英文,给 Grok/VEO/Kling)
|
||||
|
||||
### imagePrompt 模型适配
|
||||
|
||||
| 模型 | 约束 |
|
||||
|------|------|
|
||||
| Gemini | 无特殊限制 |
|
||||
| MJ | **禁止写实人像描述**(face, body, skin, portrait 等触发审核);人物用抽象表达(silhouette, figure from behind, shadow)或用场景/物体代替 |
|
||||
| Kling | 无特殊限制,参考图走 style_image |
|
||||
|
||||
### videoPrompt 规则
|
||||
|
||||
- 描述**运动**而非内容("zoom in" 而非 "a cat")
|
||||
- 与 imagePrompt 画面内容对应
|
||||
- 简洁(1-2 句,不超过 50 词)
|
||||
- **收敛原则**:基于图片已有内容,仅描述镜头运动和微动效果
|
||||
- **禁止**:大幅度环境切换、场景变化、人物位置跳变
|
||||
- **推荐写法**:镜头运动(slow zoom/pan/dolly)+ 光效微动 + 保持静止氛围
|
||||
- **画幅继承**:manifest.json 顶层 `format` 字段(如 `"9:16"`)会自动传给 VEO/Kling,无需在 prompt 中指定
|
||||
|
||||
## 首尾帧模式提示词
|
||||
|
||||
每条文案生成:
|
||||
- `imagePrompt`:起始帧画面(英文,与 single 模式复用同一字段)
|
||||
- `lastFramePrompt`:结束帧画面(英文)
|
||||
- `videoPrompt`:过渡描述(英文,给 VEO/Kling)
|
||||
|
||||
### 首尾帧提示词设计原则
|
||||
|
||||
| 原则 | 说明 | 示例 |
|
||||
|------|------|------|
|
||||
| 同一场景 | 首尾帧是同一地点/主体的不同状态 | 都是工厂,不是两个地方 |
|
||||
| 视角一致 | 相机角度/高度/距离相同 | 都是 wide shot |
|
||||
| 状态对比 | imagePrompt"静止/之前",lastFramePrompt"运动/之后" | 空车间 → 生产线运转 |
|
||||
| 过渡自然 | videoPrompt 描述从首到尾的变化 | "machines start up rhythmically" |
|
||||
| 光照连贯 | 光源方向一致,可以有渐变 | 冷光 → 暖光可以,不能反转光源 |
|
||||
|
||||
### videoPrompt 规则(首尾帧)
|
||||
|
||||
- 描述**过渡过程**而非单帧状态
|
||||
- "from X to Y" 或 "X begins, Y happens" 格式
|
||||
- 必须同时呼应 imagePrompt(起始帧)和 lastFramePrompt(结束帧)中的元素
|
||||
- 简洁(1-2 句,不超过 50 词)
|
||||
@@ -1,59 +0,0 @@
|
||||
# 分镜规划规则
|
||||
|
||||
**分镜是纯叙事思考,与视觉风格无关。** 拿到文案后、读风格文件之前,先完成分镜。
|
||||
|
||||
短视频的画面节奏和文案节奏是脱钩的:TTS 配音连续流淌,画面在配音下面切换。分镜规划的是**视觉节拍**,不是文字断句。
|
||||
|
||||
## 输入输出
|
||||
|
||||
- **输入**:用户文案 + 约束(时长、shot 数、特殊要求)
|
||||
- **输出**:结构化分镜表(JSON 数组)
|
||||
|
||||
## 时长规划
|
||||
|
||||
分镜前先算数:
|
||||
- 短视频目标时长:20-60 秒
|
||||
- 每个 shot 时长:6-8 秒(由视频模型决定)
|
||||
- shot 数量 = 目标时长 / 6~8(取整,一般 4-8 个 shot)
|
||||
- 配音字数 ≈ shot 数 x 12-15 字(按正常语速)
|
||||
|
||||
## 分镜表字段
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `text` | string | 该 shot 覆盖的配音文案(可能不到一句,也可能跨句) |
|
||||
| `shotType` | enum | `wide` / `medium` / `close-up` / `extreme-close-up` |
|
||||
| `cameraMove` | enum | `static` / `zoom-in` / `zoom-out` / `pan-left` / `pan-right` / `dolly-in` / `tracking` |
|
||||
| `visualDesc` | string | 画面描述(中文),只写三件事:**主体是什么、什么状态/动作、视觉焦点在哪**。氛围和构图交给风格层 |
|
||||
| `hook` | boolean | 仅 shot 1 为 true,标记是否为开场钩子 |
|
||||
|
||||
## 景别节奏
|
||||
|
||||
```
|
||||
shot 1 (hook): close-up 或 extreme-close-up,强主体,抓眼球
|
||||
shot 2: wide 或 medium,展开场景,给上下文
|
||||
shot 3-N(交替): close-up(压)→ wide(松)→ close-up(压)→ ...
|
||||
最后一个 shot: medium 或 wide,收束,不过度设计
|
||||
```
|
||||
|
||||
不要用 extreme-close-up 收尾(太紧),不要用 tracking 滥用(信息密度低)。
|
||||
|
||||
## 镜头运动选择
|
||||
|
||||
| cameraMove | 情绪 | 典型场景 |
|
||||
|------------|------|---------|
|
||||
| `static` | 稳定、庄严 | 建筑、静物、仪式感 |
|
||||
| `zoom-in` | 聚焦、压迫 | 悬疑、揭秘、强调细节 |
|
||||
| `zoom-out` | 揭示、震撼 | 从局部拉出全景,揭示真相 |
|
||||
| `pan-left/right` | 环顾、流动 | 展示空间、物品陈列 |
|
||||
| `dolly-in` | 沉浸、紧张 | 人物面部、关键物件 |
|
||||
| `tracking` | 跟随、活力 | 运动场景、行走(少用,AI 生成的 tracking 质量不稳定) |
|
||||
|
||||
短视频默认转场是硬切,不需要单独字段。特殊转场(fade/dissolve)仅在判断需要情绪转换时标注在 `visualDesc` 里。
|
||||
|
||||
## 首尾帧额外规则
|
||||
|
||||
首尾帧模式下分镜需要额外注意:
|
||||
- 每个 shot 必须能拆为两个有状态对比的画面(起始帧 / 结束帧)
|
||||
- `visualDesc` 需描述状态变化方向:从什么状态到什么状态
|
||||
- 首尾帧必须在**同一场景**,仅状态不同
|
||||
@@ -132,7 +132,7 @@ function buildTimeline(items, defaultDurationUs) {
|
||||
// 音频为主轴,视频调速适配(≤2x 加速,>2x 截断)
|
||||
let offset = 0
|
||||
return items.map(item => {
|
||||
const audioDur = (item.duration != null) ? item.duration * US : 0
|
||||
const audioDur = (item.audioDuration != null) ? item.audioDuration * US : (item.duration != null) ? item.duration * US : 0
|
||||
const videoDur = (item.videoDuration != null) ? item.videoDuration * US : 0
|
||||
// 无 TTS:用视频时长或固定时长
|
||||
if (audioDur <= 0) {
|
||||
@@ -196,7 +196,7 @@ async function assemble(args) {
|
||||
// 统一时间线:由 duration 驱动(TTS 音频时长)或 fallback 到固定时长
|
||||
const timeline = buildTimeline(items, defaultDurationUs)
|
||||
const totalDurationUs = timeline.length > 0 ? timeline[timeline.length - 1].end : 0
|
||||
const hasTTS = items.some(item => item.audio && item.duration != null)
|
||||
const hasTTS = items.some(item => item.audio && (item.audioDuration != null || item.duration != null))
|
||||
|
||||
console.log(`\nCapCut 成片组装`)
|
||||
console.log(` 模式: ${mode} 画幅: ${format} (${width}x${height})`)
|
||||
@@ -280,7 +280,7 @@ async function assemble(args) {
|
||||
if (manifestFile) {
|
||||
try {
|
||||
const m = JSON.parse(fs.readFileSync(manifestFile, 'utf-8'))
|
||||
const mi = m.items.find(i => i.text === item.text)
|
||||
const mi = m.items.find(i => i.id === item.id || i.narration === (item.narration || item.text) || i.text === (item.narration || item.text))
|
||||
if (mi) { mi.videoUrl = url; fs.writeFileSync(manifestFile, JSON.stringify(m, null, 2)) }
|
||||
} catch (_) {}
|
||||
}
|
||||
@@ -316,7 +316,7 @@ async function assemble(args) {
|
||||
|
||||
// -- 添加字幕 --
|
||||
step++; console.log(`[${step}/${totalSteps}] 添加字幕...`)
|
||||
if (subtitles === 'true' && items.some(i => i.text)) {
|
||||
if (subtitles === 'true' && items.some(i => i.narration || i.text)) {
|
||||
await addSubtitles(draftUrl, items, timeline, subtitleStyle)
|
||||
} else {
|
||||
console.log(' 跳过')
|
||||
@@ -667,12 +667,12 @@ async function addSubtitles(draftUrl, items, timeline, style = {}) {
|
||||
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
const item = items[i]
|
||||
const text = item.text || item.caption || ''
|
||||
const text = item.narration || item.text || item.caption || ''
|
||||
if (!text) continue
|
||||
|
||||
const tl = timeline[i]
|
||||
const keyword = item.keyword || ''
|
||||
const keywordColor = style.highlightColor || item.keywordColor || style.color || '#FFFFFF'
|
||||
const keyword = ''
|
||||
const keywordColor = style.highlightColor || style.color || '#FFFFFF'
|
||||
|
||||
const cap = {
|
||||
start: tl.start,
|
||||
|
||||
36
.claude/skills/video-from-script/scripts/lib/cmd-confirm.js
Normal file
36
.claude/skills/video-from-script/scripts/lib/cmd-confirm.js
Normal file
@@ -0,0 +1,36 @@
|
||||
/**
|
||||
* Command: confirm — 人工确认分镜图
|
||||
*/
|
||||
|
||||
const { loadManifest, saveManifest } = require('./pipeline-utils')
|
||||
|
||||
function confirmManifest(options) {
|
||||
const { manifest: manifestPath, all } = options
|
||||
|
||||
if (!manifestPath) {
|
||||
console.error('用法: pipeline.js confirm --manifest <path> --all')
|
||||
process.exit(1)
|
||||
}
|
||||
if (!all) {
|
||||
console.error('错误: 必须指定 --all')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const manifest = loadManifest(manifestPath)
|
||||
|
||||
let count = 0
|
||||
for (const item of manifest.items) {
|
||||
if (item.file && item.status === 'done' && !item.confirmed) {
|
||||
item.confirmed = true
|
||||
count++
|
||||
}
|
||||
}
|
||||
|
||||
saveManifest(manifestPath, manifest)
|
||||
|
||||
const total = manifest.items.length
|
||||
const confirmed = manifest.items.filter(it => it.confirmed).length
|
||||
console.log(`已确认: ${count} items(共 ${confirmed}/${total} 已确认)`)
|
||||
}
|
||||
|
||||
module.exports = { confirmManifest }
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Command: create-account — 一键创建账号
|
||||
*
|
||||
* 创建目录结构 → 复制参考图 → 上传 OSS → 生成 account.json + 风格骨架
|
||||
*/
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
const { ensureDir, log, ACCOUNTS_DIR } = require('./pipeline-utils')
|
||||
|
||||
async function createAccount(args) {
|
||||
const { id, name, desc, format, imageModel, videoModel, references } = args
|
||||
|
||||
if (!id) { console.error('错误: 必须指定 --id <账号ID>'); process.exit(1) }
|
||||
if (!/^[a-z0-9_-]+$/.test(id)) { console.error('错误: id 只允许小写字母/数字/短横线/下划线'); process.exit(1) }
|
||||
if (!name) { console.error('错误: 必须指定 --name <账号名>'); process.exit(1) }
|
||||
|
||||
const accountDir = path.join(ACCOUNTS_DIR, id)
|
||||
if (fs.existsSync(accountDir)) { console.error(`错误: 账号已存在: ${accountDir}`); process.exit(1) }
|
||||
|
||||
ensureDir(accountDir)
|
||||
ensureDir(path.join(accountDir, 'prompts'))
|
||||
ensureDir(path.join(accountDir, 'references'))
|
||||
ensureDir(path.join(accountDir, 'styles'))
|
||||
|
||||
// 复制参考图到 references/ 并上传 OSS
|
||||
const refs = (references || '').split(',').filter(Boolean)
|
||||
const uploadedRefs = []
|
||||
|
||||
if (refs.length > 0) {
|
||||
const { uploadFile } = require('../oss-upload')
|
||||
for (const refPath of refs) {
|
||||
const absPath = path.resolve(refPath)
|
||||
if (!fs.existsSync(absPath)) { console.error(`参考图不存在: ${absPath}`); continue }
|
||||
const fileName = path.basename(absPath)
|
||||
const destPath = path.join(accountDir, 'references', fileName)
|
||||
fs.copyFileSync(absPath, destPath)
|
||||
try {
|
||||
const { url } = await uploadFile(destPath)
|
||||
uploadedRefs.push({ file: fileName, url })
|
||||
log('account', `参考图 ${fileName} → OK`)
|
||||
} catch (err) {
|
||||
uploadedRefs.push({ file: fileName })
|
||||
log('account', `参考图 ${fileName} 上传失败: ${err.message}(仅保存本地)`)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 生成 account.json
|
||||
const styleName = args.style || id
|
||||
const accountConfig = {
|
||||
id,
|
||||
name,
|
||||
description: desc || '',
|
||||
defaultFormat: format || '9:16',
|
||||
imageModel: imageModel || 'gemini',
|
||||
videoModel: videoModel || '',
|
||||
batchSize: 30,
|
||||
storyboardPrompt: 'prompts/分镜.md',
|
||||
imageStylePrompt: 'prompts/图片提示词.md',
|
||||
videoStylePrompt: 'prompts/视频提示词.md',
|
||||
capcut: {
|
||||
effects: [],
|
||||
filter: '',
|
||||
subtitleStyle: {
|
||||
fontSize: 36,
|
||||
color: '#FFFFFF',
|
||||
highlightColor: '#FF6B35',
|
||||
bold: true,
|
||||
},
|
||||
defaultBGM: '',
|
||||
},
|
||||
}
|
||||
|
||||
if (uploadedRefs.length > 0) {
|
||||
accountConfig.styles = {
|
||||
[styleName]: { references: uploadedRefs },
|
||||
}
|
||||
}
|
||||
|
||||
const accountPath = path.join(accountDir, 'account.json')
|
||||
fs.writeFileSync(accountPath, JSON.stringify(accountConfig, null, 2), 'utf-8')
|
||||
|
||||
// 生成默认风格骨架
|
||||
const stylePath = path.join(accountDir, 'styles', `${styleName}.md`)
|
||||
const styleContent = [
|
||||
`# ${styleName}`,
|
||||
'',
|
||||
`${desc || name} 的视觉风格。`,
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'## 图片提示词',
|
||||
'',
|
||||
'### 核心视觉要素',
|
||||
'',
|
||||
'(待填充:描述关键视觉元素)',
|
||||
'',
|
||||
'### 色调方案',
|
||||
'',
|
||||
'(待填充)',
|
||||
'',
|
||||
'### 图片 Prompt 模板',
|
||||
'',
|
||||
'(待填充)',
|
||||
'',
|
||||
'### 图片禁止项',
|
||||
'',
|
||||
'- 文字水印',
|
||||
'- 字幕覆盖',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'## 视频提示词',
|
||||
'',
|
||||
'### 运镜规则',
|
||||
'',
|
||||
'(待填充)',
|
||||
'',
|
||||
'### 视频 Prompt 模板',
|
||||
'',
|
||||
'(待填充)',
|
||||
'',
|
||||
].join('\n')
|
||||
fs.writeFileSync(stylePath, styleContent, 'utf-8')
|
||||
|
||||
console.log(`\n账号已创建: ${accountDir}`)
|
||||
console.log(` ID: ${id}`)
|
||||
console.log(` 名称: ${name}`)
|
||||
console.log(` 模型: ${accountConfig.imageModel} + ${accountConfig.videoModel || '(未指定)'}`)
|
||||
console.log(` 参考图: ${uploadedRefs.length} 张(${uploadedRefs.filter(r => r.url).length} 已上传)`)
|
||||
console.log(` 风格: ${styleName}`)
|
||||
console.log(`\n下一步: 编辑 ${stylePath} 完善提示词策略\n`)
|
||||
|
||||
return accountPath
|
||||
}
|
||||
|
||||
module.exports = { createAccount }
|
||||
138
.claude/skills/video-from-script/scripts/lib/cmd-init.js
Normal file
138
.claude/skills/video-from-script/scripts/lib/cmd-init.js
Normal file
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Command: init — 从 account.json + AI 创意内容生成规范 manifest.json
|
||||
*
|
||||
* 校验 items → 继承账号配置 → 创建输出目录 → 写出 manifest
|
||||
*/
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
const { loadAccountConfig, saveManifest, ensureDir, ACCOUNTS_DIR, SKILLS_DIR } = require('./pipeline-utils')
|
||||
|
||||
function initManifest(options) {
|
||||
const { account: accountId, mode, items: itemsJson, itemsFile } = options
|
||||
|
||||
if (!accountId) {
|
||||
console.error('错误: 必须指定 --account <账号ID>')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const accountConfig = loadAccountConfig(accountId)
|
||||
|
||||
// 解析 items
|
||||
let rawItems
|
||||
if (itemsFile) {
|
||||
const filePath = path.resolve(itemsFile)
|
||||
if (!fs.existsSync(filePath)) {
|
||||
console.error(`错误: items 文件不存在: ${filePath}`)
|
||||
process.exit(1)
|
||||
}
|
||||
rawItems = JSON.parse(fs.readFileSync(filePath, 'utf-8'))
|
||||
} else if (itemsJson) {
|
||||
rawItems = JSON.parse(itemsJson)
|
||||
} else {
|
||||
console.error('错误: 必须指定 --items <JSON> 或 --items-file <path>')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!Array.isArray(rawItems) || rawItems.length === 0) {
|
||||
console.error('错误: items 必须是非空数组')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// 校验必填字段
|
||||
const requiredFields = ['shotDesc', 'narration', 'imagePrompt']
|
||||
const resolvedMode = mode || 'single'
|
||||
|
||||
for (let i = 0; i < rawItems.length; i++) {
|
||||
const item = rawItems[i]
|
||||
for (const f of requiredFields) {
|
||||
if (!item[f]) {
|
||||
console.error(`错误: items[${i}] 缺少必填字段 "${f}"`)
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
if (resolvedMode === 'framePair' && !item.lastFramePrompt) {
|
||||
console.error(`错误: 首尾帧模式 items[${i}] 缺少 "lastFramePrompt"(imagePrompt 作为第一帧)`)
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// 从 account.json 继承参考图
|
||||
const styles = accountConfig.styles || {}
|
||||
const firstStyleKey = Object.keys(styles)[0]
|
||||
const styleRefs = firstStyleKey ? (styles[firstStyleKey].references || []) : []
|
||||
const references = styleRefs.map(ref => {
|
||||
const entry = {}
|
||||
if (ref.file) entry.file = path.join(ACCOUNTS_DIR, accountId, 'references', ref.file)
|
||||
if (ref.url) entry.url = ref.url
|
||||
return entry
|
||||
})
|
||||
|
||||
// 构建 items
|
||||
const items = rawItems.map((raw, i) => {
|
||||
const item = {
|
||||
id: i + 1,
|
||||
status: 'pending',
|
||||
shotDesc: raw.shotDesc || '',
|
||||
narration: raw.narration || raw.text || '',
|
||||
duration: raw.duration || 5,
|
||||
imagePrompt: raw.imagePrompt,
|
||||
confirmed: false,
|
||||
}
|
||||
if (raw.directorRef) item.directorRef = raw.directorRef
|
||||
if (raw.videoPrompt) item.videoPrompt = raw.videoPrompt
|
||||
if (resolvedMode === 'framePair') item.lastFramePrompt = raw.lastFramePrompt
|
||||
return item
|
||||
})
|
||||
|
||||
// 组装 manifest
|
||||
const manifest = {
|
||||
account: accountId,
|
||||
imageModel: accountConfig.imageModel || 'gemini',
|
||||
videoModel: accountConfig.videoModel || 'veo3-fast-frames',
|
||||
format: accountConfig.defaultFormat || '9:16',
|
||||
mode: resolvedMode,
|
||||
references,
|
||||
items,
|
||||
}
|
||||
|
||||
// 创建输出目录(自增序号)
|
||||
const date = new Date()
|
||||
const dateStr = [
|
||||
date.getFullYear(),
|
||||
String(date.getMonth() + 1).padStart(2, '0'),
|
||||
String(date.getDate()).padStart(2, '0'),
|
||||
].join('')
|
||||
const prefix = `${accountConfig.name}_${dateStr}`
|
||||
const outputBase = path.join(SKILLS_DIR, '..', '..', '..', 'output')
|
||||
ensureDir(outputBase)
|
||||
|
||||
let seq = 1
|
||||
while (fs.existsSync(path.join(outputBase, `${prefix}_${String(seq).padStart(3, '0')}`))) {
|
||||
seq++
|
||||
}
|
||||
const dirName = `${prefix}_${String(seq).padStart(3, '0')}`
|
||||
const outputDir = path.join(outputBase, dirName)
|
||||
ensureDir(outputDir)
|
||||
ensureDir(path.join(outputDir, 'images'))
|
||||
ensureDir(path.join(outputDir, 'videos'))
|
||||
ensureDir(path.join(outputDir, 'audio'))
|
||||
|
||||
const manifestPath = path.join(outputDir, 'manifest.json')
|
||||
saveManifest(manifestPath, manifest)
|
||||
|
||||
console.log(`\nManifest 已创建: ${manifestPath}`)
|
||||
console.log(` 账号: ${accountId} (${accountConfig.name})`)
|
||||
console.log(` 模型: ${manifest.imageModel} + ${manifest.videoModel}`)
|
||||
console.log(` 画幅: ${manifest.format}, 模式: ${manifest.mode}`)
|
||||
console.log(` Items: ${items.length}`)
|
||||
console.log(` 参考图: ${references.length}`)
|
||||
if (items.some(it => !it.videoPrompt)) {
|
||||
console.log(` ⚠ ${items.filter(it => !it.videoPrompt).length} 个 item 缺少 videoPrompt,生视频阶段将跳过`)
|
||||
}
|
||||
console.log()
|
||||
|
||||
return manifestPath
|
||||
}
|
||||
|
||||
module.exports = { initManifest }
|
||||
38
.claude/skills/video-from-script/scripts/lib/cmd-status.js
Normal file
38
.claude/skills/video-from-script/scripts/lib/cmd-status.js
Normal file
@@ -0,0 +1,38 @@
|
||||
/**
|
||||
* Command: status — 显示 manifest 执行状态
|
||||
*/
|
||||
|
||||
const { loadManifest } = require('./pipeline-utils')
|
||||
|
||||
const ALL_PHASES = ['images', 'upload', 'videos', 'tts', 'assemble']
|
||||
|
||||
function showStatus(manifestPath) {
|
||||
const manifest = loadManifest(manifestPath)
|
||||
const phases = manifest.pipeline?.phases || {}
|
||||
|
||||
console.log(`\nManifest: ${manifestPath}`)
|
||||
console.log(`Account: ${manifest.account || '(未指定)'}`)
|
||||
console.log(`\n阶段状态:`)
|
||||
for (const p of ALL_PHASES) {
|
||||
const status = phases[p] || 'pending'
|
||||
const icon = status === 'done' ? '✓' : status === 'running' ? '→' : status === 'failed' ? '✗' : status === 'partial' ? '~' : '·'
|
||||
console.log(` ${icon} ${p}: ${status}`)
|
||||
}
|
||||
|
||||
const items = manifest.items || []
|
||||
const done = items.filter(it => it.status === 'done').length
|
||||
const failed = items.filter(it => it.status === 'failed').length
|
||||
const pending = items.filter(it => !it.status || it.status === 'pending').length
|
||||
|
||||
console.log(`\nItems: ${items.length} 总计, ${done} 完成, ${failed} 失败, ${pending} 待处理`)
|
||||
|
||||
if (failed > 0) {
|
||||
console.log(`\n失败项:`)
|
||||
items.filter(it => it.status === 'failed').forEach((it, i) => {
|
||||
console.log(` [${it.id || i + 1}] ${it.error || '未知错误'}`)
|
||||
})
|
||||
}
|
||||
console.log()
|
||||
}
|
||||
|
||||
module.exports = { showStatus }
|
||||
110
.claude/skills/video-from-script/scripts/lib/cmd-validate.js
Normal file
110
.claude/skills/video-from-script/scripts/lib/cmd-validate.js
Normal file
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* Command: validate — 校验 manifest.json 完整性
|
||||
* Command: validate-account — 校验账号目录完整性
|
||||
*/
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
const { loadManifest, ACCOUNTS_DIR } = require('./pipeline-utils')
|
||||
|
||||
function validateManifest(manifestPath) {
|
||||
const issues = []
|
||||
|
||||
if (!fs.existsSync(manifestPath)) {
|
||||
console.error(`错误: manifest 不存在: ${manifestPath}`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
let manifest
|
||||
try {
|
||||
manifest = loadManifest(manifestPath)
|
||||
} catch (e) {
|
||||
console.error(`错误: JSON 解析失败: ${e.message}`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!manifest.account) issues.push('缺少顶层 account')
|
||||
if (!manifest.imageModel) issues.push('缺少顶层 imageModel(可选: gemini, mj)')
|
||||
if (!manifest.format) issues.push('缺少顶层 format(如 9:16)')
|
||||
if (!manifest.items || !Array.isArray(manifest.items)) issues.push('缺少顶层 items 数组')
|
||||
if (!manifest.mode) issues.push('缺少顶层 mode(single 或 framePair)')
|
||||
|
||||
if (manifest.items && Array.isArray(manifest.items)) {
|
||||
manifest.items.forEach((item, i) => {
|
||||
const prefix = `items[${i}]`
|
||||
if (!item.narration && !item.text) issues.push(`${prefix} 缺少 narration 或 text(中文旁白)`)
|
||||
if (!item.shotDesc) issues.push(`${prefix} 缺少 shotDesc(分镜描述)`)
|
||||
if (!item.imagePrompt) issues.push(`${prefix} 缺少 imagePrompt`)
|
||||
if (manifest.mode === 'framePair' && !item.lastFramePrompt) {
|
||||
issues.push(`${prefix} 首尾帧模式缺少 lastFramePrompt(imagePrompt 作为第一帧)`)
|
||||
}
|
||||
if (item.status && !['pending', 'generating', 'done', 'failed'].includes(item.status)) {
|
||||
issues.push(`${prefix} status 无效: ${item.status}`)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
if (issues.length === 0) {
|
||||
console.log(`✓ Manifest 校验通过: ${manifestPath}`)
|
||||
console.log(` ${manifest.items?.length || 0} items, account=${manifest.account}, mode=${manifest.mode}`)
|
||||
} else {
|
||||
console.error(`✗ 发现 ${issues.length} 个问题:`)
|
||||
issues.forEach(issue => console.error(` - ${issue}`))
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
function validateAccount(accountId) {
|
||||
const issues = []
|
||||
const accountDir = path.join(ACCOUNTS_DIR, accountId)
|
||||
|
||||
if (!fs.existsSync(accountDir)) { console.error(`错误: 账号不存在: ${accountDir}`); process.exit(1) }
|
||||
|
||||
const accountPath = path.join(accountDir, 'account.json')
|
||||
if (!fs.existsSync(accountPath)) { console.error('错误: 缺少 account.json'); process.exit(1) }
|
||||
|
||||
let config
|
||||
try { config = JSON.parse(fs.readFileSync(accountPath, 'utf-8')) }
|
||||
catch (e) { console.error(`错误: JSON 解析失败: ${e.message}`); process.exit(1) }
|
||||
|
||||
if (config.id !== accountId) issues.push(`id 不匹配: json="${config.id}" vs 目录="${accountId}"`)
|
||||
if (!config.name) issues.push('缺少 name')
|
||||
if (!config.imageModel) issues.push('缺少 imageModel')
|
||||
if (!config.defaultFormat) issues.push('缺少 defaultFormat')
|
||||
|
||||
const refDir = path.join(accountDir, 'references')
|
||||
const styles = config.styles || {}
|
||||
const hasStyleRefs = Object.values(styles).some(s => s.references && s.references.length > 0)
|
||||
const localRefs = fs.existsSync(refDir)
|
||||
? fs.readdirSync(refDir).filter(f => /\.(png|jpg|jpeg|webp)$/i.test(f))
|
||||
: []
|
||||
if (localRefs.length === 0 && !hasStyleRefs) {
|
||||
issues.push('无参考图(建议至少 1 张)')
|
||||
}
|
||||
|
||||
const stylesDir = path.join(accountDir, 'styles')
|
||||
const styleFiles = fs.existsSync(stylesDir)
|
||||
? fs.readdirSync(stylesDir).filter(f => f.endsWith('.md'))
|
||||
: []
|
||||
if (styleFiles.length === 0) {
|
||||
issues.push('无风格文件(styles/ 下至少 1 个 .md)')
|
||||
}
|
||||
|
||||
for (const [sName, sConf] of Object.entries(styles)) {
|
||||
for (const ref of (sConf.references || [])) {
|
||||
if (!ref.url) issues.push(`styles.${sName}: 参考图 ${ref.file} 缺少 url(未上传 OSS)`)
|
||||
}
|
||||
}
|
||||
|
||||
if (issues.length === 0) {
|
||||
console.log(`✓ 账号校验通过: ${accountId}`)
|
||||
console.log(` ${config.name}, 模型: ${config.imageModel}+${config.videoModel || '(未指定)'}`)
|
||||
console.log(` 参考图: ${localRefs.length} 本地, 风格: ${styleFiles.length} 个`)
|
||||
} else {
|
||||
console.error(`✗ 发现 ${issues.length} 个问题:`)
|
||||
issues.forEach(i => console.error(` - ${i}`))
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { validateManifest, validateAccount }
|
||||
@@ -0,0 +1,41 @@
|
||||
/**
|
||||
* Phase: assemble — CapCut 成片组装
|
||||
*
|
||||
* 图片/视频 + TTS → 剪映草稿
|
||||
*/
|
||||
|
||||
const { log, getManifestDir } = require('./pipeline-utils')
|
||||
|
||||
async function phaseAssemble(manifest, manifestPath, options) {
|
||||
const dir = getManifestDir(manifestPath)
|
||||
const accountConfig = options.accountConfig || {}
|
||||
const capcutConfig = accountConfig.capcut || {}
|
||||
|
||||
const videoItems = manifest.items.filter(it => it.video && it.status === 'done')
|
||||
const hasVideos = videoItems.length > 0
|
||||
const mode = hasVideos ? 'videos' : 'images'
|
||||
|
||||
const assembleArgs = {
|
||||
input: dir,
|
||||
manifest: manifestPath,
|
||||
mode,
|
||||
format: manifest.format || accountConfig.defaultFormat || '9:16',
|
||||
subtitles: mode === 'images' ? 'true' : 'false',
|
||||
voiceover: manifest.items.some(it => it.audio) ? 'true' : 'false',
|
||||
duration: '4',
|
||||
animation: 'kenburns-zoom',
|
||||
}
|
||||
|
||||
if (capcutConfig.defaultBGM) assembleArgs.bgm = capcutConfig.defaultBGM
|
||||
if (capcutConfig.effects) assembleArgs.effects = capcutConfig.effects.join(',')
|
||||
if (capcutConfig.filter) assembleArgs.filter = capcutConfig.filter
|
||||
|
||||
log('assemble', `模式: ${mode}, 字幕: true, 配音: ${assembleArgs.voiceover}`)
|
||||
|
||||
const { assemble } = require('../capcut_assemble')
|
||||
await assemble(assembleArgs)
|
||||
|
||||
log('assemble', '成片完成')
|
||||
}
|
||||
|
||||
module.exports = { phaseAssemble }
|
||||
173
.claude/skills/video-from-script/scripts/lib/phase-images.js
Normal file
173
.claude/skills/video-from-script/scripts/lib/phase-images.js
Normal file
@@ -0,0 +1,173 @@
|
||||
/**
|
||||
* Phase: images — 图片生成
|
||||
*
|
||||
* 支持 Gemini / MJ / Kling 三种模型,含首尾帧模式
|
||||
*/
|
||||
|
||||
const path = require('path')
|
||||
const { saveManifest, getReferences, ensureDir, renameGeneratedFile, log, getManifestDir } = require('./pipeline-utils')
|
||||
|
||||
async function phaseImages(manifest, manifestPath, options) {
|
||||
const dir = getManifestDir(manifestPath)
|
||||
const imagesDir = path.join(dir, 'images')
|
||||
ensureDir(imagesDir)
|
||||
|
||||
const items = manifest.items.filter(it =>
|
||||
(!it.status || it.status === 'pending' || it.status === 'generating') && it.imagePrompt
|
||||
)
|
||||
if (items.length === 0) { log('images', '无待处理 item,跳过'); return }
|
||||
|
||||
const accountConfig = options.accountConfig || {}
|
||||
let model = options.imageModel || manifest.imageModel || accountConfig.imageModel || 'gemini'
|
||||
const ratio = manifest.format || accountConfig.defaultFormat || '9:16'
|
||||
|
||||
// 首尾帧模式:MJ 降级为 Gemini(MJ 出4张候选图无法一一对应首尾帧)
|
||||
if (model === 'mj' && manifest.mode === 'framePair') {
|
||||
log('images', '首尾帧模式不支持 MJ,自动降级为 Gemini')
|
||||
model = 'gemini'
|
||||
}
|
||||
const refs = getReferences(manifest, accountConfig)
|
||||
|
||||
log('images', `共 ${items.length} 张, 模型: ${model}, 画幅: ${ratio}, 参考图: ${refs.localPaths.length}本地/${refs.urls.length}URL`)
|
||||
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
const item = items[i]
|
||||
const idx = i + 1
|
||||
try {
|
||||
item.status = 'generating'
|
||||
saveManifest(manifestPath, manifest)
|
||||
|
||||
let result
|
||||
if (model === 'gemini') {
|
||||
const { generate: geminiGen, edit: geminiEdit } = require('../gemini-image-generator')
|
||||
if (refs.localPaths.length > 0) {
|
||||
log('images', `[${idx}/${items.length}] Gemini 图生图: ${item.imagePrompt.substring(0, 60)}...`)
|
||||
result = await geminiEdit(item.imagePrompt, refs.localPaths, {
|
||||
outputDir: imagesDir,
|
||||
aspectRatio: ratio,
|
||||
})
|
||||
} else {
|
||||
log('images', `[${idx}/${items.length}] Gemini 文生图: ${item.imagePrompt.substring(0, 60)}...`)
|
||||
result = await geminiGen(item.imagePrompt, {
|
||||
outputDir: imagesDir,
|
||||
aspectRatio: ratio,
|
||||
})
|
||||
}
|
||||
if (result.savedFiles && result.savedFiles.length > 0) {
|
||||
item.file = renameGeneratedFile(
|
||||
path.relative(dir, result.savedFiles[0]).replace(/\\/g, '/'),
|
||||
dir, idx, item.narration || item.shotDesc, ''
|
||||
)
|
||||
}
|
||||
} else if (model === 'mj') {
|
||||
const { generate: mjGen } = require('../mj-image-generator')
|
||||
const mjOpts = { outputDir: imagesDir, aspectRatio: ratio, split: true }
|
||||
if (refs.urls.length > 0) {
|
||||
mjOpts.referenceImages = refs.urls
|
||||
mjOpts.styleWeight = 200
|
||||
}
|
||||
log('images', `[${idx}/${items.length}] MJ 生图: ${item.imagePrompt.substring(0, 60)}...`)
|
||||
result = await mjGen(item.imagePrompt, mjOpts)
|
||||
if (result.files && result.files.length > 0) {
|
||||
item.candidates = result.files.map((f, ci) =>
|
||||
renameGeneratedFile(
|
||||
path.relative(dir, f).replace(/\\/g, '/'),
|
||||
dir, idx, item.narration || item.shotDesc, `cand${ci + 1}`
|
||||
)
|
||||
)
|
||||
item.file = item.candidates[0]
|
||||
log('images', `[${idx}/${items.length}] ${result.files.length} 张候选,默认选第1张`)
|
||||
}
|
||||
} else if (model === 'kling') {
|
||||
const { generate: klingGen } = require('../kling-image-generator')
|
||||
const klingOpts = { outputDir: imagesDir, aspectRatio: ratio }
|
||||
if (refs.urls.length > 0) {
|
||||
klingOpts.styleImageUrl = refs.urls[0]
|
||||
}
|
||||
log('images', `[${idx}/${items.length}] 可灵生图: ${item.imagePrompt.substring(0, 60)}...`)
|
||||
result = await klingGen(item.imagePrompt, klingOpts)
|
||||
if (result.savedFiles && result.savedFiles.length > 0) {
|
||||
item.file = renameGeneratedFile(
|
||||
path.relative(dir, result.savedFiles[0]).replace(/\\/g, '/'),
|
||||
dir, idx, item.narration || item.shotDesc, ''
|
||||
)
|
||||
}
|
||||
} else {
|
||||
throw new Error(`不支持的模型: ${model}(支持: gemini, mj, kling)`)
|
||||
}
|
||||
|
||||
if (item.file) {
|
||||
item.status = 'done'
|
||||
log('images', `[${idx}/${items.length}] 完成: ${item.file}`)
|
||||
} else {
|
||||
item.status = 'failed'
|
||||
item.error = '生成器未返回文件'
|
||||
log('images', `[${idx}/${items.length}] 失败: 生成器未返回文件`)
|
||||
}
|
||||
|
||||
// 首尾帧模式:生成第二张图(lastFrame)
|
||||
if (item.status === 'done' && manifest.mode === 'framePair' && item.lastFramePrompt && !item.lastFrame) {
|
||||
await generateLastFrame(item, idx, items.length, manifest, dir, imagesDir, model, ratio, manifestPath)
|
||||
}
|
||||
} catch (err) {
|
||||
item.status = 'failed'
|
||||
item.error = err.message
|
||||
log('images', `[${idx}/${items.length}] 失败: ${err.message}`)
|
||||
}
|
||||
saveManifest(manifestPath, manifest)
|
||||
}
|
||||
}
|
||||
|
||||
async function generateLastFrame(item, idx, total, manifest, dir, imagesDir, model, ratio, manifestPath) {
|
||||
try {
|
||||
item.status = 'generating'
|
||||
saveManifest(manifestPath, manifest)
|
||||
|
||||
const firstFramePath = path.resolve(dir, item.file)
|
||||
let lastResult
|
||||
if (model === 'gemini') {
|
||||
const { edit: geminiEdit } = require('../gemini-image-generator')
|
||||
lastResult = await geminiEdit(item.lastFramePrompt, [firstFramePath], {
|
||||
outputDir: imagesDir,
|
||||
aspectRatio: ratio,
|
||||
})
|
||||
} else if (model === 'mj') {
|
||||
const { generate: mjGen } = require('../mj-image-generator')
|
||||
const mjOpts = { outputDir: imagesDir, aspectRatio: ratio, split: false }
|
||||
if (item.url) {
|
||||
mjOpts.referenceImages = [item.url]
|
||||
mjOpts.styleWeight = 200
|
||||
}
|
||||
lastResult = await mjGen(item.lastFramePrompt, mjOpts)
|
||||
} else if (model === 'kling') {
|
||||
const { generate: klingGen } = require('../kling-image-generator')
|
||||
lastResult = await klingGen(item.lastFramePrompt, {
|
||||
outputDir: imagesDir,
|
||||
styleImageUrl: item.url || '',
|
||||
aspectRatio: ratio,
|
||||
})
|
||||
}
|
||||
|
||||
if (lastResult) {
|
||||
const files = lastResult.savedFiles || lastResult.files || []
|
||||
if (files.length > 0) {
|
||||
item.lastFrame = renameGeneratedFile(
|
||||
path.relative(dir, files[0]).replace(/\\/g, '/'),
|
||||
dir, idx, item.narration || item.shotDesc, 'last'
|
||||
)
|
||||
item.status = 'done'
|
||||
log('images', `[${idx}/${total}] lastFrame 完成: ${item.lastFrame}`)
|
||||
} else {
|
||||
item.status = 'failed'
|
||||
item.error = 'lastFrame 生成器未返回文件'
|
||||
log('images', `[${idx}/${total}] lastFrame 失败: 未返回文件`)
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
item.status = 'failed'
|
||||
item.error = `lastFrame 失败: ${err.message}`
|
||||
log('images', `[${idx}/${total}] lastFrame 失败: ${err.message}`)
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { phaseImages }
|
||||
44
.claude/skills/video-from-script/scripts/lib/phase-tts.js
Normal file
44
.claude/skills/video-from-script/scripts/lib/phase-tts.js
Normal file
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* Phase: tts — 语音合成
|
||||
*
|
||||
* 使用通义千问 TTS 生成旁白音频
|
||||
*/
|
||||
|
||||
const path = require('path')
|
||||
const { saveManifest, ensureDir, log, getManifestDir } = require('./pipeline-utils')
|
||||
|
||||
async function phaseTts(manifest, manifestPath) {
|
||||
const dir = getManifestDir(manifestPath)
|
||||
const audioDir = path.join(dir, 'audio')
|
||||
ensureDir(audioDir)
|
||||
|
||||
const { synthesize } = require('../qwen-tts')
|
||||
|
||||
const items = manifest.items.filter(it =>
|
||||
it.status === 'done' && (it.narration || it.text) && !it.audio
|
||||
)
|
||||
if (items.length === 0) { log('tts', '无待处理 item,跳过'); return }
|
||||
|
||||
log('tts', `共 ${items.length} 段`)
|
||||
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
const item = items[i]
|
||||
const idx = i + 1
|
||||
try {
|
||||
const { filePath, duration } = await synthesize(item.narration || item.text, {
|
||||
outputDir: audioDir,
|
||||
id: item.id || idx,
|
||||
})
|
||||
item.audio = path.relative(dir, filePath).replace(/\\/g, '/')
|
||||
item.audioDuration = Math.round(duration * 1000) / 1000
|
||||
log('tts', `[${idx}/${items.length}] ${duration.toFixed(1)}s: ${(item.narration || item.text).substring(0, 30)}...`)
|
||||
} catch (err) {
|
||||
item.status = 'failed'
|
||||
item.error = `TTS失败: ${err.message}`
|
||||
log('tts', `[${idx}/${items.length}] 失败: ${err.message}`)
|
||||
}
|
||||
saveManifest(manifestPath, manifest)
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { phaseTts }
|
||||
46
.claude/skills/video-from-script/scripts/lib/phase-upload.js
Normal file
46
.claude/skills/video-from-script/scripts/lib/phase-upload.js
Normal file
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* Phase: upload — OSS 上传
|
||||
*
|
||||
* 将生成的图片(含首尾帧)上传到 OSS,回写 url
|
||||
*/
|
||||
|
||||
const path = require('path')
|
||||
const { saveManifest, log, getManifestDir } = require('./pipeline-utils')
|
||||
|
||||
async function phaseUpload(manifest, manifestPath) {
|
||||
const dir = getManifestDir(manifestPath)
|
||||
const { uploadFile } = require('../oss-upload')
|
||||
|
||||
const items = manifest.items.filter(it =>
|
||||
it.status === 'done' && it.file && !it.url
|
||||
)
|
||||
if (items.length === 0) { log('upload', '无待上传 item,跳过'); return }
|
||||
|
||||
log('upload', `共 ${items.length} 个文件`)
|
||||
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
const item = items[i]
|
||||
const filePath = path.resolve(dir, item.file)
|
||||
try {
|
||||
const { url } = await uploadFile(filePath)
|
||||
item.url = url
|
||||
log('upload', `[${i + 1}/${items.length}] ${item.file} → ${url.substring(0, 60)}...`)
|
||||
} catch (err) {
|
||||
item.error = `上传失败: ${err.message}`
|
||||
log('upload', `[${i + 1}/${items.length}] 失败: ${err.message}`)
|
||||
}
|
||||
if (item.url && item.lastFrame && !item.lastFrameUrl) {
|
||||
const lastPath = path.resolve(dir, item.lastFrame)
|
||||
try {
|
||||
const { url } = await uploadFile(lastPath)
|
||||
item.lastFrameUrl = url
|
||||
log('upload', `[${i + 1}/${items.length}] lastFrame → OK`)
|
||||
} catch (err) {
|
||||
log('upload', `[${i + 1}/${items.length}] lastFrame 上传失败: ${err.message}`)
|
||||
}
|
||||
}
|
||||
saveManifest(manifestPath, manifest)
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { phaseUpload }
|
||||
104
.claude/skills/video-from-script/scripts/lib/phase-videos.js
Normal file
104
.claude/skills/video-from-script/scripts/lib/phase-videos.js
Normal file
@@ -0,0 +1,104 @@
|
||||
/**
|
||||
* Phase: videos — 视频生成(VEO / Grok / Kling)
|
||||
*
|
||||
* 图生视频,批量提交,生成后自动上传 OSS
|
||||
*/
|
||||
|
||||
const path = require('path')
|
||||
const { saveManifest, ensureDir, log, getManifestDir } = require('./pipeline-utils')
|
||||
|
||||
async function phaseVideos(manifest, manifestPath, options) {
|
||||
const dir = getManifestDir(manifestPath)
|
||||
const videosDir = path.join(dir, 'videos')
|
||||
ensureDir(videosDir)
|
||||
|
||||
const accountConfig = options.accountConfig || {}
|
||||
const videoModel = manifest.videoModel || accountConfig.videoModel || 'veo3-fast-frames'
|
||||
|
||||
const items = manifest.items.filter(it =>
|
||||
it.status === 'done' && it.confirmed !== false && it.url && it.videoPrompt && !it.video
|
||||
)
|
||||
if (items.length === 0) { log('videos', '无待处理 item,跳过'); return }
|
||||
|
||||
// 选择生成器
|
||||
let generator
|
||||
const modelLower = videoModel.toLowerCase()
|
||||
if (modelLower.includes('grok')) {
|
||||
generator = require('../grok-video-generator')
|
||||
} else if (modelLower.includes('kling')) {
|
||||
generator = require('../kling-video-generator')
|
||||
} else {
|
||||
generator = require('../veo-video-generator')
|
||||
}
|
||||
|
||||
log('videos', `共 ${items.length} 个, 模型: ${videoModel}`)
|
||||
|
||||
const tasks = items.map((item, i) => {
|
||||
const task = {
|
||||
id: item.id || i + 1,
|
||||
prompt: item.videoPrompt,
|
||||
image: item.url,
|
||||
outputDir: videosDir,
|
||||
}
|
||||
if (item.lastFrameUrl) {
|
||||
task.images = [item.url, item.lastFrameUrl]
|
||||
task.lastFrameUrl = item.lastFrameUrl
|
||||
} else {
|
||||
task.images = [item.url]
|
||||
}
|
||||
return task
|
||||
})
|
||||
|
||||
try {
|
||||
const results = await generator.batchGenerate(tasks, {
|
||||
videoModel,
|
||||
aspectRatio: manifest.format || '9:16',
|
||||
outputDir: videosDir,
|
||||
skipManifestWrite: true,
|
||||
})
|
||||
|
||||
for (let i = 0; i < results.length; i++) {
|
||||
const result = results[i]
|
||||
const item = items[i]
|
||||
if (!item) continue
|
||||
if (result.success && result.file) {
|
||||
item.video = path.relative(dir, result.file).replace(/\\/g, '/')
|
||||
item.videoDuration = result.duration
|
||||
} else {
|
||||
item.status = 'failed'
|
||||
item.error = result.error || '视频生成失败'
|
||||
log('videos', ` item ${(item.id || '?')} 失败: ${item.error}`)
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
log('videos', `批量生成失败: ${err.message}`)
|
||||
for (const item of items) {
|
||||
if (!item.video) {
|
||||
item.status = 'failed'
|
||||
item.error = `批量生成异常: ${err.message}`
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 上传视频到 OSS
|
||||
const { uploadFile } = require('../oss-upload')
|
||||
const videoItems = manifest.items.filter(it => it.video && !it.videoUrl)
|
||||
if (videoItems.length > 0) {
|
||||
log('videos', `上传 ${videoItems.length} 个视频到 OSS...`)
|
||||
for (const item of videoItems) {
|
||||
const videoPath = path.resolve(dir, item.video)
|
||||
try {
|
||||
const { url } = await uploadFile(videoPath)
|
||||
item.videoUrl = url
|
||||
log('videos', ` ${item.video} → OK`)
|
||||
} catch (err) {
|
||||
log('videos', ` ${item.video} 上传失败: ${err.message}`)
|
||||
}
|
||||
saveManifest(manifestPath, manifest)
|
||||
}
|
||||
}
|
||||
|
||||
saveManifest(manifestPath, manifest)
|
||||
}
|
||||
|
||||
module.exports = { phaseVideos }
|
||||
170
.claude/skills/video-from-script/scripts/lib/pipeline-utils.js
Normal file
170
.claude/skills/video-from-script/scripts/lib/pipeline-utils.js
Normal file
@@ -0,0 +1,170 @@
|
||||
/**
|
||||
* Pipeline 共享工具函数与路径常量
|
||||
*
|
||||
* 所有 phase/command 模块共用:配置加载、文件操作、路径计算、日志
|
||||
*/
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
// 路径常量(基于 lib/ 的父目录 scripts/)
|
||||
const SCRIPTS_DIR = path.join(__dirname, '..')
|
||||
const SKILLS_DIR = path.join(SCRIPTS_DIR, '..')
|
||||
const PROJECT_ROOT = path.join(SKILLS_DIR, '..', '..')
|
||||
const CONFIG_PATH = path.join(SKILLS_DIR, 'config.json')
|
||||
const ACCOUNTS_DIR = path.join(PROJECT_ROOT, 'accounts')
|
||||
|
||||
// ============================================================================
|
||||
// 配置 & Manifest
|
||||
// ============================================================================
|
||||
|
||||
function loadConfig() {
|
||||
return JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf-8'))
|
||||
}
|
||||
|
||||
function loadManifest(manifestPath) {
|
||||
return JSON.parse(fs.readFileSync(manifestPath, 'utf-8'))
|
||||
}
|
||||
|
||||
function saveManifest(manifestPath, manifest) {
|
||||
const tmp = manifestPath + '.tmp'
|
||||
fs.writeFileSync(tmp, JSON.stringify(manifest, null, 2), 'utf-8')
|
||||
fs.renameSync(tmp, manifestPath)
|
||||
}
|
||||
|
||||
function loadAccountConfig(accountId) {
|
||||
const accountPath = path.join(ACCOUNTS_DIR, accountId, 'account.json')
|
||||
if (!fs.existsSync(accountPath)) throw new Error(`账号不存在: ${accountPath}`)
|
||||
return JSON.parse(fs.readFileSync(accountPath, 'utf-8'))
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 参考图解析
|
||||
// ============================================================================
|
||||
|
||||
function getReferences(manifest, accountConfig) {
|
||||
const result = { localPaths: [], urls: [] }
|
||||
const accountId = accountConfig.id || manifest.account || ''
|
||||
|
||||
// 优先读 manifest.references(agent 创建时写入)
|
||||
const refs = manifest.references || []
|
||||
if (refs.length > 0) {
|
||||
for (const ref of refs) {
|
||||
if (ref.url) result.urls.push(ref.url)
|
||||
if (ref.file) {
|
||||
const localPath = path.isAbsolute(ref.file) ? ref.file : path.resolve(ref.file)
|
||||
if (fs.existsSync(localPath)) {
|
||||
result.localPaths.push(localPath)
|
||||
} else {
|
||||
log('images', `参考图不存在: ${ref.file}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (result.localPaths.length > 0 || result.urls.length > 0) return result
|
||||
log('images', 'manifest.references 全部无效,尝试 account fallback')
|
||||
}
|
||||
|
||||
// Fallback 1: 从 account.json 的 styles.*.references 读取
|
||||
const styles = accountConfig.styles || {}
|
||||
for (const [, style] of Object.entries(styles)) {
|
||||
for (const ref of (style.references || [])) {
|
||||
if (ref.url) result.urls.push(ref.url)
|
||||
if (ref.file && accountId) {
|
||||
const localPath = path.join(ACCOUNTS_DIR, accountId, 'references', ref.file)
|
||||
if (fs.existsSync(localPath)) {
|
||||
result.localPaths.push(localPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (result.localPaths.length > 0 || result.urls.length > 0) return result
|
||||
|
||||
// Fallback 2: 扫描 account 的 references 目录
|
||||
if (accountId) {
|
||||
const refDir = path.join(ACCOUNTS_DIR, accountId, 'references')
|
||||
if (fs.existsSync(refDir)) {
|
||||
const files = fs.readdirSync(refDir).filter(f =>
|
||||
/\.(png|jpg|jpeg|webp)$/i.test(f)
|
||||
)
|
||||
for (const f of files) {
|
||||
result.localPaths.push(path.join(refDir, f))
|
||||
}
|
||||
if (files.length > 0) {
|
||||
log('images', `从 references 目录兜底扫描到 ${files.length} 个参考图`)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (result.localPaths.length === 0 && result.urls.length === 0) {
|
||||
log('images', '无参考图,将使用纯文生图模式')
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 文件操作
|
||||
// ============================================================================
|
||||
|
||||
function ensureDir(dir) {
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
}
|
||||
|
||||
function slugify(text) {
|
||||
return text
|
||||
.replace(/[^\w一-鿿]/g, '_')
|
||||
.replace(/_+/g, '_')
|
||||
.replace(/^_|_$/g, '')
|
||||
.substring(0, 20)
|
||||
}
|
||||
|
||||
function renameGeneratedFile(oldRelPath, dir, seq, nameHint, suffix) {
|
||||
if (!oldRelPath) return oldRelPath
|
||||
const oldAbs = path.resolve(dir, oldRelPath)
|
||||
if (!fs.existsSync(oldAbs)) return oldRelPath
|
||||
const ext = path.extname(oldAbs)
|
||||
const slug = nameHint ? slugify(nameHint) : ''
|
||||
const tag = suffix ? `_${suffix}` : ''
|
||||
const newName = slug
|
||||
? `scene_${String(seq).padStart(2, '0')}_${slug}${tag}${ext}`
|
||||
: `scene_${String(seq).padStart(2, '0')}${tag}${ext}`
|
||||
const newAbs = path.join(path.dirname(oldAbs), newName)
|
||||
if (oldAbs !== newAbs) {
|
||||
try { fs.renameSync(oldAbs, newAbs) } catch (_) { return oldRelPath }
|
||||
}
|
||||
return path.relative(dir, newAbs).replace(/\\/g, '/')
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// 日志 & 路径
|
||||
// ============================================================================
|
||||
|
||||
function log(phase, msg) {
|
||||
console.log(`[${phase}] ${msg}`)
|
||||
}
|
||||
|
||||
function getManifestDir(manifestPath) {
|
||||
return path.dirname(path.resolve(manifestPath))
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Exports
|
||||
// ============================================================================
|
||||
|
||||
module.exports = {
|
||||
SCRIPTS_DIR,
|
||||
SKILLS_DIR,
|
||||
PROJECT_ROOT,
|
||||
CONFIG_PATH,
|
||||
ACCOUNTS_DIR,
|
||||
loadConfig,
|
||||
loadManifest,
|
||||
saveManifest,
|
||||
loadAccountConfig,
|
||||
getReferences,
|
||||
ensureDir,
|
||||
slugify,
|
||||
renameGeneratedFile,
|
||||
log,
|
||||
getManifestDir,
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user