83 KiB
美图 Agent — 视频创作可视化界面 实施计划
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: 构建基于 pi-agent 的视频创作 Web 可视化界面,三栏布局,支持账户管理、提示词编辑、聊天驱动 Pipeline、资产画廊。
Architecture: React SPA (Vite + Tailwind + shadcn/ui) 通过 REST + WebSocket 与 Express 后端通信。后端集成 pi-agent-core SDK 编排对话,直接调用 pipeline.js 执行视频创作流程。SQLite 存储会话、资产和配置。
Tech Stack: TypeScript, Vite, React 18, Tailwind CSS, shadcn/ui, Express, ws, better-sqlite3, pi-agent-core, Zustand
文件结构(全量)
web/
├── package.json
├── tsconfig.base.json
├── server/
│ ├── tsconfig.json
│ ├── index.ts
│ ├── db/
│ │ ├── index.ts
│ │ └── schema.ts
│ ├── routes/
│ │ ├── accounts.ts
│ │ ├── prompts.ts
│ │ ├── pipeline.ts
│ │ ├── assets.ts
│ │ └── configs.ts
│ ├── agent/
│ │ ├── index.ts
│ │ └── tools.ts
│ └── ws/
│ └── chat.ts
├── client/
│ ├── tsconfig.json
│ ├── index.html
│ ├── vite.config.ts
│ ├── tailwind.config.ts
│ ├── postcss.config.js
│ └── src/
│ ├── main.tsx
│ ├── App.tsx
│ ├── index.css
│ ├── lib/
│ │ ├── api.ts
│ │ └── websocket.ts
│ ├── hooks/
│ │ ├── useAccounts.ts
│ │ ├── usePrompts.ts
│ │ ├── useAssets.ts
│ │ └── useChat.ts
│ ├── types/
│ │ └── index.ts
│ ├── components/
│ │ ├── ui/ # shadcn/ui (generated)
│ │ ├── layout/
│ │ │ ├── AppLayout.tsx
│ │ │ ├── Sidebar.tsx
│ │ │ └── MiddlePanel.tsx
│ │ ├── chat/
│ │ │ ├── ChatView.tsx
│ │ │ ├── ChatMessage.tsx
│ │ │ ├── ChatInput.tsx
│ │ │ └── PipelineProgress.tsx
│ │ ├── accounts/
│ │ │ ├── AccountList.tsx
│ │ │ └── AccountForm.tsx
│ │ ├── prompts/
│ │ │ └── PromptEditor.tsx
│ │ ├── assets/
│ │ │ ├── AssetGallery.tsx
│ │ │ └── AssetPreview.tsx
│ │ └── config/
│ │ └── ConfigForm.tsx
│ └── store/
│ └── index.ts
P0: 项目脚手架
Task 0.1: 初始化项目与 package.json
Files:
-
Create:
web/package.json -
Create:
web/tsconfig.base.json -
Create:
web/server/tsconfig.json -
Create:
web/client/tsconfig.json -
Step 1: 创建 web/package.json
mkdir -p web/server web/client/src
{
"name": "meitu-agent",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "concurrently \"npm run dev:server\" \"npm run dev:client\"",
"dev:server": "tsx watch server/index.ts",
"dev:client": "vite --config client/vite.config.ts",
"build": "vite build --config client/vite.config.ts",
"db:init": "tsx server/db/schema.ts"
},
"dependencies": {
"express": "^4.21.0",
"ws": "^8.18.0",
"better-sqlite3": "^11.6.0",
"zod": "^3.23.0",
"cors": "^2.8.5",
"zustand": "^5.0.0",
"react": "^18.3.0",
"react-dom": "^18.3.0",
"lucide-react": "^0.460.0",
"clsx": "^2.1.0",
"tailwind-merge": "^2.6.0"
},
"devDependencies": {
"@types/express": "^5.0.0",
"@types/ws": "^8.5.0",
"@types/better-sqlite3": "^7.6.0",
"@types/cors": "^2.8.0",
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"typescript": "^5.6.0",
"tsx": "^4.19.0",
"vite": "^5.4.0",
"@vitejs/plugin-react": "^4.3.0",
"tailwindcss": "^3.4.0",
"postcss": "^8.4.0",
"autoprefixer": "^10.4.0",
"concurrently": "^9.1.0"
}
}
- Step 2: 创建 tsconfig.base.json
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"declaration": true,
"declarationMap": true,
"sourceMap": true
}
}
- Step 3: 创建 server/tsconfig.json
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"module": "ESNext",
"outDir": "./dist",
"rootDir": "."
},
"include": ["./**/*.ts"]
}
- Step 4: 创建 client/tsconfig.json
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"jsx": "react-jsx",
"outDir": "./dist",
"rootDir": "./src",
"baseUrl": ".",
"paths": {
"@/*": ["./src/*"]
}
},
"include": ["./src/**/*.ts", "./src/**/*.tsx"]
}
- Step 5: 安装依赖
cd web && npm install
- Step 6: Commit
git add web/package.json web/tsconfig.base.json web/server/tsconfig.json web/client/tsconfig.json web/package-lock.json
git commit -m "feat(web): init project scaffold with package.json and tsconfig"
Task 0.2: 初始化 Vite + React + Tailwind + shadcn/ui
Files:
-
Create:
web/client/index.html -
Create:
web/client/vite.config.ts -
Create:
web/client/tailwind.config.ts -
Create:
web/client/postcss.config.js -
Create:
web/client/src/main.tsx -
Create:
web/client/src/index.css -
Create:
web/client/src/lib/utils.ts -
Step 1: 创建 client/index.html
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>美图 Agent</title>
</head>
<body class="bg-zinc-950 text-zinc-50 antialiased">
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
- Step 2: 创建 client/vite.config.ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import path from 'path';
export default defineConfig({
plugins: [react()],
root: path.resolve(__dirname),
resolve: {
alias: {
'@': path.resolve(__dirname, 'src'),
},
},
server: {
port: 5173,
proxy: {
'/api': 'http://localhost:3001',
'/ws': {
target: 'ws://localhost:3001',
ws: true,
},
},
},
});
- Step 3: 创建 client/tailwind.config.ts
import type { Config } from 'tailwindcss';
export default {
content: ['./client/src/**/*.{ts,tsx}'],
darkMode: 'class',
theme: {
extend: {
colors: {
border: 'hsl(240 3.7% 15.9%)',
input: 'hsl(240 3.7% 15.9%)',
ring: 'hsl(240 4.9% 83.9%)',
background: 'hsl(240 10% 3.9%)',
foreground: 'hsl(0 0% 98%)',
primary: {
DEFAULT: 'hsl(0 0% 98%)',
foreground: 'hsl(240 5.9% 10%)',
},
secondary: {
DEFAULT: 'hsl(240 3.7% 15.9%)',
foreground: 'hsl(0 0% 98%)',
},
muted: {
DEFAULT: 'hsl(240 3.7% 15.9%)',
foreground: 'hsl(240 5% 64.9%)',
},
accent: {
DEFAULT: 'hsl(240 3.7% 15.9%)',
foreground: 'hsl(0 0% 98%)',
},
destructive: {
DEFAULT: 'hsl(0 62.8% 30.6%)',
foreground: 'hsl(0 0% 98%)',
},
},
borderRadius: {
lg: '0.5rem',
md: 'calc(0.5rem - 2px)',
sm: 'calc(0.5rem - 4px)',
},
},
},
plugins: [],
} satisfies Config;
- Step 4: 创建 client/postcss.config.js
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
- Step 5: 创建 client/src/index.css
@tailwind base;
@tailwind components;
@tailwind utilities;
@layer base {
* {
@apply border-border;
}
body {
@apply bg-zinc-950 text-zinc-50;
}
}
- Step 6: 创建 client/src/lib/utils.ts
import { clsx, type ClassValue } from 'clsx';
import { twMerge } from 'tailwind-merge';
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs));
}
- Step 7: 创建 client/src/main.tsx
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
import './index.css';
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<App />
</React.StrictMode>
);
- Step 8: Commit
git add web/client/
git commit -m "feat(web): init Vite + React + Tailwind client scaffold"
Task 0.3: 初始化 shadcn/ui 基础组件
Files:
-
Create:
web/components.json -
Create:
web/client/src/components/ui/button.tsx -
Create:
web/client/src/components/ui/input.tsx -
Create:
web/client/src/components/ui/scroll-area.tsx -
Step 1: 创建 components.json
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "default",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "client/tailwind.config.ts",
"css": "client/src/index.css",
"baseColor": "zinc",
"cssVariables": true
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils"
}
}
- Step 2: 创建 button.tsx
import * as React from 'react';
import { cva, type VariantProps } from 'class-variance-authority';
import { cn } from '@/lib/utils';
const buttonVariants = cva(
'inline-flex items-center justify-center whitespace-nowrap rounded-md text-sm font-medium ring-offset-background transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:pointer-events-none disabled:opacity-50',
{
variants: {
variant: {
default: 'bg-primary text-primary-foreground hover:bg-primary/90',
destructive: 'bg-destructive text-destructive-foreground hover:bg-destructive/90',
outline: 'border border-input bg-background hover:bg-accent hover:text-accent-foreground',
secondary: 'bg-secondary text-secondary-foreground hover:bg-secondary/80',
ghost: 'hover:bg-accent hover:text-accent-foreground',
link: 'text-primary underline-offset-4 hover:underline',
},
size: {
default: 'h-10 px-4 py-2',
sm: 'h-9 rounded-md px-3',
lg: 'h-11 rounded-md px-8',
icon: 'h-10 w-10',
},
},
defaultVariants: { variant: 'default', size: 'default' },
}
);
export interface ButtonProps
extends React.ButtonHTMLAttributes<HTMLButtonElement>,
VariantProps<typeof buttonVariants> {}
const Button = React.forwardRef<HTMLButtonElement, ButtonProps>(
({ className, variant, size, ...props }, ref) => (
<button className={cn(buttonVariants({ variant, size, className }))} ref={ref} {...props} />
)
);
Button.displayName = 'Button';
export { Button, buttonVariants };
- Step 3: 创建 input.tsx(同样标准 shadcn input)
import * as React from 'react';
import { cn } from '@/lib/utils';
export interface InputProps extends React.InputHTMLAttributes<HTMLInputElement> {}
const Input = React.forwardRef<HTMLInputElement, InputProps>(
({ className, type, ...props }, ref) => (
<input
type={type}
className={cn(
'flex h-10 w-full rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
className
)}
ref={ref}
{...props}
/>
)
);
Input.displayName = 'Input';
export { Input };
- Step 4: 创建 scroll-area.tsx(简化版)
'use client';
import * as React from 'react';
import { cn } from '@/lib/utils';
const ScrollArea = React.forwardRef<
HTMLDivElement,
React.HTMLAttributes<HTMLDivElement>
>(({ className, children, ...props }, ref) => (
<div ref={ref} className={cn('overflow-auto', className)} {...props}>
{children}
</div>
));
ScrollArea.displayName = 'ScrollArea';
export { ScrollArea };
- Step 5: Commit
git add web/components.json web/client/src/components/ui/ web/client/src/lib/utils.ts
git commit -m "feat(web): add shadcn/ui base components (Button, Input, ScrollArea)"
Task 0.4: Express + WebSocket 后端入口
Files:
-
Create:
web/server/index.ts -
Step 1: 创建 server/index.ts
import express from 'express';
import cors from 'cors';
import { createServer } from 'http';
import { WebSocketServer } from 'ws';
import { initDb } from './db';
import { accountsRouter } from './routes/accounts';
import { promptsRouter } from './routes/prompts';
import { pipelineRouter } from './routes/pipeline';
import { assetsRouter } from './routes/assets';
import { configsRouter } from './routes/configs';
import { handleChat } from './ws/chat';
const app = express();
const server = createServer(app);
const wss = new WebSocketServer({ server, path: '/ws' });
app.use(cors());
app.use(express.json({ limit: '50mb' }));
app.use('/api/accounts', accountsRouter);
app.use('/api/prompts', promptsRouter);
app.use('/api/pipeline', pipelineRouter);
app.use('/api/assets', assetsRouter);
app.use('/api/configs', configsRouter);
wss.on('connection', handleChat);
const PORT = 3001;
initDb();
server.listen(PORT, () => {
console.log(`Server running on http://localhost:${PORT}`);
});
- Step 2: Commit
git add web/server/index.ts
git commit -m "feat(web): add Express + WebSocket server entry"
Task 0.5: SQLite 数据库初始化
Files:
-
Create:
web/server/db/index.ts -
Create:
web/server/db/schema.ts -
Step 1: 创建 server/db/index.ts
import Database from 'better-sqlite3';
import path from 'path';
const DB_PATH = path.resolve(__dirname, '..', '..', 'data', 'meitu-agent.db');
let db: Database.Database;
export function getDb(): Database.Database {
if (!db) {
const fs = require('fs');
const dir = path.dirname(DB_PATH);
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
db = new Database(DB_PATH);
db.pragma('journal_mode = WAL');
db.pragma('foreign_keys = ON');
}
return db;
}
export function initDb(): void {
const d = getDb();
d.exec(`
CREATE TABLE IF NOT EXISTS conversations (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
account_id TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS messages (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
role TEXT NOT NULL CHECK(role IN ('user', 'assistant', 'system', 'tool')),
content TEXT NOT NULL,
tool_calls TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS assets (
id TEXT PRIMARY KEY,
account_id TEXT,
manifest_path TEXT,
type TEXT NOT NULL CHECK(type IN ('image', 'video')),
file_path TEXT NOT NULL,
url TEXT,
shot_index INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS pipeline_runs (
id TEXT PRIMARY KEY,
manifest_path TEXT NOT NULL,
phase TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending','running','done','failed')),
started_at TEXT,
finished_at TEXT
);
CREATE TABLE IF NOT EXISTS configs (
id TEXT PRIMARY KEY,
key TEXT NOT NULL UNIQUE,
value TEXT NOT NULL DEFAULT '{}',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_messages_conv ON messages(conversation_id, created_at);
CREATE INDEX IF NOT EXISTS idx_assets_account ON assets(account_id, created_at);
CREATE INDEX IF NOT EXISTS idx_pipeline_manifest ON pipeline_runs(manifest_path);
`);
}
- Step 2: Commit
git add web/server/db/
git commit -m "feat(web): add SQLite schema and connection"
Task 0.6: 三栏布局壳
Files:
-
Create:
web/client/src/App.tsx -
Create:
web/client/src/types/index.ts -
Create:
web/client/src/store/index.ts -
Create:
web/client/src/components/layout/AppLayout.tsx -
Create:
web/client/src/components/layout/Sidebar.tsx -
Create:
web/client/src/components/layout/MiddlePanel.tsx -
Step 1: 创建 types/index.ts
export type NavView = 'chat' | 'accounts' | 'assets' | 'config';
export interface Account {
id: string;
name: string;
description: string;
defaultFormat: string;
imageModel: string;
videoModel: string;
batchSize: number;
ttsVoice: string;
ttsInstruction: string;
storyboardPrompt: string;
imageStylePrompt: string;
videoStylePrompt: string;
references: { file: string; url?: string }[];
capcut: Record<string, unknown>;
}
export interface Conversation {
id: string;
title: string;
account_id: string | null;
created_at: string;
updated_at: string;
}
export interface Message {
id: string;
conversation_id: string;
role: 'user' | 'assistant' | 'system' | 'tool';
content: string;
tool_calls?: unknown;
created_at: string;
}
export interface Asset {
id: string;
account_id: string | null;
manifest_path: string | null;
type: 'image' | 'video';
file_path: string;
url: string | null;
shot_index: number | null;
created_at: string;
}
export interface ConfigItem {
id: string;
key: string;
value: Record<string, unknown>;
updated_at: string;
}
- Step 2: 创建 store/index.ts (Zustand)
import { create } from 'zustand';
import type { NavView, Account, Conversation } from '@/types';
interface AppState {
activeView: NavView;
setActiveView: (view: NavView) => void;
selectedAccountId: string | null;
setSelectedAccountId: (id: string | null) => void;
activeConversationId: string | null;
setActiveConversationId: (id: string | null) => void;
conversations: Conversation[];
setConversations: (list: Conversation[]) => void;
}
export const useAppStore = create<AppState>((set) => ({
activeView: 'chat',
setActiveView: (view) => set({ activeView: view }),
selectedAccountId: null,
setSelectedAccountId: (id) => set({ selectedAccountId: id }),
activeConversationId: null,
setActiveConversationId: (id) => set({ activeConversationId: id }),
conversations: [],
setConversations: (list) => set({ conversations: list }),
}));
- Step 3: 创建 AppLayout.tsx
import { Sidebar } from './Sidebar';
import { MiddlePanel } from './MiddlePanel';
import { useAppStore } from '@/store';
export function AppLayout({ children }: { children: React.ReactNode }) {
const activeView = useAppStore((s) => s.activeView);
return (
<div className="h-screen flex bg-zinc-950 text-zinc-50 overflow-hidden">
<Sidebar />
{activeView === 'chat' && <MiddlePanel />}
<main className="flex-1 flex flex-col min-w-0">
{children}
</main>
</div>
);
}
- Step 4: 创建 Sidebar.tsx
import { MessageCircle, FolderOpen, Image, Settings } from 'lucide-react';
import { useAppStore } from '@/store';
import type { NavView } from '@/types';
import { cn } from '@/lib/utils';
const navItems: { id: NavView; icon: typeof MessageCircle; label: string }[] = [
{ id: 'chat', icon: MessageCircle, label: '对话' },
{ id: 'accounts', icon: FolderOpen, label: '账户' },
{ id: 'assets', icon: Image, label: '资产' },
{ id: 'config', icon: Settings, label: '设置' },
];
export function Sidebar() {
const { activeView, setActiveView } = useAppStore();
return (
<aside className="w-14 flex flex-col items-center py-4 gap-2 border-r border-zinc-800">
{navItems.map(({ id, icon: Icon, label }) => (
<button
key={id}
onClick={() => setActiveView(id)}
className={cn(
'w-10 h-10 flex items-center justify-center rounded-lg transition-colors',
activeView === id
? 'bg-zinc-800 text-white'
: 'text-zinc-500 hover:text-zinc-300 hover:bg-zinc-800/50'
)}
title={label}
>
<Icon size={20} />
</button>
))}
</aside>
);
}
- Step 5: 创建 MiddlePanel.tsx(对话列表占位)
import { Plus } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { ScrollArea } from '@/components/ui/scroll-area';
import { useAppStore } from '@/store';
export function MiddlePanel() {
const { conversations, activeConversationId, setActiveConversationId } = useAppStore();
return (
<aside className="w-60 flex flex-col border-r border-zinc-800">
<div className="p-3 flex items-center justify-between">
<Input
placeholder="搜索对话..."
className="h-8 text-xs bg-zinc-900 border-zinc-800"
/>
<Button size="icon" variant="ghost" className="h-8 w-8 ml-1">
<Plus size={16} />
</Button>
</div>
<ScrollArea className="flex-1 px-2">
{conversations.map((conv) => (
<button
key={conv.id}
onClick={() => setActiveConversationId(conv.id)}
className={`w-full text-left px-3 py-2 rounded-md text-sm truncate mb-0.5 transition-colors
${conv.id === activeConversationId ? 'bg-zinc-800 text-white' : 'text-zinc-400 hover:bg-zinc-800/50 hover:text-zinc-200'}`}
>
{conv.title}
</button>
))}
{conversations.length === 0 && (
<p className="text-xs text-zinc-600 text-center mt-8">暂无对话</p>
)}
</ScrollArea>
</aside>
);
}
- Step 6: 创建 App.tsx
import { AppLayout } from '@/components/layout/AppLayout';
import { ChatView } from '@/components/chat/ChatView';
import { AccountList } from '@/components/accounts/AccountList';
import { AssetGallery } from '@/components/assets/AssetGallery';
import { ConfigForm } from '@/components/config/ConfigForm';
import { useAppStore } from '@/store';
import { useEffect } from 'react';
function MainContent() {
const view = useAppStore((s) => s.activeView);
switch (view) {
case 'chat': return <ChatView />;
case 'accounts': return <AccountList />;
case 'assets': return <AssetGallery />;
case 'config': return <ConfigForm />;
}
}
export default function App() {
useEffect(() => {
fetch('/api/configs')
.then((r) => r.json())
.catch(() => {});
}, []);
return (
<AppLayout>
<MainContent />
</AppLayout>
);
}
- Step 7: 创建占位组件(空壳,后续任务填充)
创建以下占位文件,每个只返回一个空 div:
chat/ChatView.tsx:
export function ChatView() {
return <div className="flex-1 flex flex-col items-center justify-center text-zinc-500">选择或开始新对话</div>;
}
accounts/AccountList.tsx:
export function AccountList() {
return <div className="p-6 text-zinc-500">账户管理</div>;
}
assets/AssetGallery.tsx:
export function AssetGallery() {
return <div className="p-6 text-zinc-500">资产画廊</div>;
}
config/ConfigForm.tsx:
export function ConfigForm() {
return <div className="p-6 text-zinc-500">设置</div>;
}
- Step 8: 验证: 启动 dev server
cd web && npm run dev
打开 http://localhost:5173,确认三栏布局渲染正常(左侧图标栏 56px + 中间列表 240px + 右侧主区域)。
- Step 9: Commit
git add web/client/src/
git commit -m "feat(web): add three-column layout shell with placeholder views"
P1: 账户管理 + 配置
Task 1.1: 类型定义 + API Client
Files:
-
Create:
web/client/src/lib/api.ts -
Step 1: 创建 lib/api.ts
import type { Account, Asset, Conversation, Message, ConfigItem } from '@/types';
const BASE = '/api';
async function request<T>(path: string, options?: RequestInit): Promise<T> {
const res = await fetch(`${BASE}${path}`, {
headers: { 'Content-Type': 'application/json' },
...options,
});
if (!res.ok) throw new Error(await res.text());
return res.json();
}
export const api = {
// Accounts
listAccounts: () => request<Account[]>('/accounts'),
getAccount: (id: string) => request<Account>(`/accounts/${id}`),
createAccount: (data: Partial<Account>) =>
request<Account>('/accounts', { method: 'POST', body: JSON.stringify(data) }),
updateAccount: (id: string, data: Partial<Account>) =>
request<Account>(`/accounts/${id}`, { method: 'PUT', body: JSON.stringify(data) }),
deleteAccount: (id: string) =>
request<void>(`/accounts/${id}`, { method: 'DELETE' }),
// Prompts
getPrompt: (accountId: string, type: string) =>
request<{ path: string; content: string }>(`/prompts/${accountId}/${type}`),
savePrompt: (accountId: string, type: string, content: string) =>
request<void>(`/prompts/${accountId}/${type}`, { method: 'PUT', body: JSON.stringify({ content }) }),
// Conversations
listConversations: () => request<Conversation[]>('/pipeline/conversations'),
getMessages: (convId: string) => request<Message[]>(`/pipeline/conversations/${convId}/messages`),
// Assets
listAssets: (params?: { accountId?: string; type?: string }) => {
const qs = new URLSearchParams();
if (params?.accountId) qs.set('accountId', params.accountId);
if (params?.type) qs.set('type', params.type);
return request<Asset[]>(`/assets?${qs}`);
},
deleteAsset: (id: string) => request<void>(`/assets/${id}`, { method: 'DELETE' }),
// Configs
getConfigs: () => request<ConfigItem[]>('/configs'),
saveConfig: (key: string, value: Record<string, unknown>) =>
request<void>(`/configs/${key}`, { method: 'PUT', body: JSON.stringify({ value }) }),
};
- Step 2: Commit
git add web/client/src/lib/api.ts
git commit -m "feat(web): add typed API client"
Task 1.2: 账户 CRUD 后端路由
Files:
-
Create:
web/server/routes/accounts.ts -
Step 1: 创建 server/routes/accounts.ts
import { Router } from 'express';
import fs from 'fs';
import path from 'path';
const ACCOUNTS_DIR = path.resolve(__dirname, '..', '..', '..', 'accounts');
export const accountsRouter = Router();
function readAccountJson(id: string): Record<string, unknown> | null {
const p = path.join(ACCOUNTS_DIR, id, 'account.json');
if (!fs.existsSync(p)) return null;
return JSON.parse(fs.readFileSync(p, 'utf-8'));
}
function writeAccountJson(id: string, data: Record<string, unknown>): void {
const dir = path.join(ACCOUNTS_DIR, id);
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, 'account.json'), JSON.stringify(data, null, 2), 'utf-8');
}
// List
accountsRouter.get('/', (_req, res) => {
const dirs = fs.readdirSync(ACCOUNTS_DIR, { withFileTypes: true })
.filter((d) => d.isDirectory() && !d.name.startsWith('_') && !d.name.startsWith('.'))
.map((d) => d.name);
const accounts = dirs
.map((id) => readAccountJson(id))
.filter(Boolean);
res.json(accounts);
});
// Get
accountsRouter.get('/:id', (req, res) => {
const data = readAccountJson(req.params.id);
if (!data) return res.status(404).json({ error: 'Account not found' });
res.json(data);
});
// Create
accountsRouter.post('/', (req, res) => {
const { id, ...rest } = req.body;
if (!id) return res.status(400).json({ error: 'id is required' });
const dir = path.join(ACCOUNTS_DIR, id);
if (fs.existsSync(dir)) return res.status(409).json({ error: 'Account already exists' });
const data = {
id,
name: rest.name || id,
description: rest.description || '',
defaultFormat: rest.defaultFormat || '9:16',
imageModel: rest.imageModel || 'gemini',
videoModel: rest.videoModel || 'veo3-fast',
batchSize: rest.batchSize || 30,
ttsVoice: rest.ttsVoice || '',
ttsInstruction: rest.ttsInstruction || '',
storyboardPrompt: rest.storyboardPrompt || 'prompts/分镜.md',
imageStylePrompt: rest.imageStylePrompt || 'prompts/图片提示词.md',
videoStylePrompt: rest.videoStylePrompt || 'prompts/视频提示词.md',
references: rest.references || [],
capcut: rest.capcut || {},
};
writeAccountJson(id, data);
res.status(201).json(data);
});
// Update
accountsRouter.put('/:id', (req, res) => {
const existing = readAccountJson(req.params.id);
if (!existing) return res.status(404).json({ error: 'Account not found' });
const merged = { ...existing, ...req.body, id: req.params.id };
writeAccountJson(req.params.id, merged);
res.json(merged);
});
// Delete
accountsRouter.delete('/:id', (req, res) => {
const dir = path.join(ACCOUNTS_DIR, req.params.id);
if (!fs.existsSync(dir)) return res.status(404).json({ error: 'Account not found' });
fs.rmSync(dir, { recursive: true });
res.status(204).send();
});
- Step 2: Commit
git add web/server/routes/accounts.ts
git commit -m "feat(web): add account CRUD API routes"
Task 1.3: 配置 CRUD 后端路由
Files:
-
Create:
web/server/routes/configs.ts -
Step 1: 创建 server/routes/configs.ts
import { Router } from 'express';
import { getDb } from '../db';
export const configsRouter = Router();
configsRouter.get('/', (_req, res) => {
const rows = getDb().prepare('SELECT * FROM configs ORDER BY key').all();
res.json(rows);
});
configsRouter.get('/:key', (req, res) => {
const row = getDb().prepare('SELECT * FROM configs WHERE key = ?').get(req.params.key);
if (!row) return res.status(404).json({ error: 'Config not found' });
res.json(row);
});
configsRouter.put('/:key', (req, res) => {
const { value } = req.body;
getDb().prepare(`
INSERT INTO configs (id, key, value, updated_at)
VALUES (?, ?, ?, datetime('now'))
ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = excluded.updated_at
`).run(crypto.randomUUID(), req.params.key, JSON.stringify(value));
res.json({ key: req.params.key, ok: true });
});
- Step 2: Commit
git add web/server/routes/configs.ts
git commit -m "feat(web): add config CRUD API routes"
Task 1.4: 账户列表 + 表单前端组件
Files:
-
Replace:
web/client/src/components/accounts/AccountList.tsx -
Create:
web/client/src/components/accounts/AccountForm.tsx -
Create:
web/client/src/hooks/useAccounts.ts -
Step 1: 创建 hooks/useAccounts.ts
import { useState, useEffect, useCallback } from 'react';
import { api } from '@/lib/api';
import type { Account } from '@/types';
export function useAccounts() {
const [accounts, setAccounts] = useState<Account[]>([]);
const [loading, setLoading] = useState(true);
const refresh = useCallback(() => {
api.listAccounts().then(setAccounts).finally(() => setLoading(false));
}, []);
useEffect(() => { refresh(); }, [refresh]);
const create = (data: Partial<Account>) => api.createAccount(data).then(refresh);
const update = (id: string, data: Partial<Account>) => api.updateAccount(id, data).then(refresh);
const remove = (id: string) => api.deleteAccount(id).then(refresh);
return { accounts, loading, refresh, create, update, remove };
}
- Step 2: 重写 AccountList.tsx
import { Plus } from 'lucide-react';
import { useState } from 'react';
import { useAccounts } from '@/hooks/useAccounts';
import { AccountForm } from './AccountForm';
import { Button } from '@/components/ui/button';
import { ScrollArea } from '@/components/ui/scroll-area';
import type { Account } from '@/types';
export function AccountList() {
const { accounts, create, update, remove } = useAccounts();
const [editing, setEditing] = useState<Account | null>(null);
const [creating, setCreating] = useState(false);
return (
<div className="flex h-full">
<ScrollArea className="w-60 border-r border-zinc-800 p-3">
<div className="flex items-center justify-between mb-3">
<h2 className="text-sm font-medium text-zinc-300">账户列表</h2>
<Button size="icon" variant="ghost" className="h-7 w-7" onClick={() => setCreating(true)}>
<Plus size={14} />
</Button>
</div>
{accounts.map((a) => (
<button
key={a.id}
onClick={() => setEditing(a)}
className="w-full text-left px-3 py-2 rounded-md text-sm hover:bg-zinc-800/50 text-zinc-400 hover:text-zinc-200 transition-colors"
>
<div className="font-medium truncate">{a.name}</div>
<div className="text-xs text-zinc-600">{a.imageModel} · {a.defaultFormat}</div>
</button>
))}
</ScrollArea>
<div className="flex-1 p-6">
{(editing || creating) ? (
<AccountForm
account={creating ? undefined : editing!}
onSave={(data) => {
if (creating) {
create(data).then(() => setCreating(false));
} else {
update(editing!.id, data).then(() => setEditing(null));
}
}}
onDelete={editing ? () => {
if (confirm(`确定删除账户「${editing.name}」?`)) {
remove(editing.id).then(() => setEditing(null));
}
} : undefined}
onCancel={() => { setEditing(null); setCreating(false); }}
/>
) : (
<div className="flex items-center justify-center h-full text-zinc-600 text-sm">
选择一个账户或创建新账户
</div>
)}
</div>
</div>
);
}
- Step 3: 创建 AccountForm.tsx
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import type { Account } from '@/types';
interface Props {
account?: Account;
onSave: (data: Partial<Account>) => void;
onDelete?: () => void;
onCancel: () => void;
}
export function AccountForm({ account, onSave, onDelete, onCancel }: Props) {
const [form, setForm] = useState({
id: account?.id || '',
name: account?.name || '',
description: account?.description || '',
defaultFormat: account?.defaultFormat || '9:16',
imageModel: account?.imageModel || 'gemini',
videoModel: account?.videoModel || 'veo3-fast',
ttsVoice: account?.ttsVoice || '',
ttsInstruction: account?.ttsInstruction || '',
});
const handleChange = (key: string, value: string) => setForm((f) => ({ ...f, [key]: value }));
return (
<div className="max-w-lg space-y-4">
<h2 className="text-lg font-semibold">
{account ? `编辑账户: ${account.name}` : '创建新账户'}
</h2>
<div>
<label className="text-xs text-zinc-500">账户 ID</label>
<Input
value={form.id}
onChange={(e) => handleChange('id', e.target.value)}
disabled={!!account}
placeholder="my-account"
className="mt-1 bg-zinc-900 border-zinc-800"
/>
</div>
<div>
<label className="text-xs text-zinc-500">名称</label>
<Input
value={form.name}
onChange={(e) => handleChange('name', e.target.value)}
placeholder="账户名称"
className="mt-1 bg-zinc-900 border-zinc-800"
/>
</div>
<div>
<label className="text-xs text-zinc-500">描述</label>
<Input
value={form.description}
onChange={(e) => handleChange('description', e.target.value)}
placeholder="简短描述..."
className="mt-1 bg-zinc-900 border-zinc-800"
/>
</div>
<div className="grid grid-cols-2 gap-3">
<div>
<label className="text-xs text-zinc-500">画幅</label>
<select
value={form.defaultFormat}
onChange={(e) => handleChange('defaultFormat', e.target.value)}
className="mt-1 w-full h-10 rounded-md border border-zinc-800 bg-zinc-900 px-3 text-sm"
>
<option value="9:16">9:16 竖屏</option>
<option value="16:9">16:9 横屏</option>
<option value="1:1">1:1 方形</option>
</select>
</div>
<div>
<label className="text-xs text-zinc-500">生图模型</label>
<select
value={form.imageModel}
onChange={(e) => handleChange('imageModel', e.target.value)}
className="mt-1 w-full h-10 rounded-md border border-zinc-800 bg-zinc-900 px-3 text-sm"
>
<option value="gemini">Gemini</option>
<option value="mj">Midjourney</option>
<option value="gpt">GPT Image</option>
<option value="kling">Kling</option>
</select>
</div>
<div>
<label className="text-xs text-zinc-500">视频模型</label>
<select
value={form.videoModel}
onChange={(e) => handleChange('videoModel', e.target.value)}
className="mt-1 w-full h-10 rounded-md border border-zinc-800 bg-zinc-900 px-3 text-sm"
>
<option value="veo3-fast">Veo3 Fast</option>
<option value="veo3-fast-frames">Veo3 Fast Frames</option>
<option value="kling">Kling</option>
<option value="grok">Grok</option>
</select>
</div>
</div>
<div>
<label className="text-xs text-zinc-500">TTS 语音</label>
<Input
value={form.ttsVoice}
onChange={(e) => handleChange('ttsVoice', e.target.value)}
placeholder="cosyvoice-xxx"
className="mt-1 bg-zinc-900 border-zinc-800 font-mono text-xs"
/>
</div>
<div>
<label className="text-xs text-zinc-500">TTS 指令</label>
<textarea
value={form.ttsInstruction}
onChange={(e) => handleChange('ttsInstruction', e.target.value)}
rows={3}
placeholder="用沉稳有力的男性声音朗读..."
className="mt-1 w-full rounded-md border border-zinc-800 bg-zinc-900 px-3 py-2 text-sm resize-y"
/>
</div>
<div className="flex gap-2 pt-2">
<Button onClick={() => onSave(form)}>保存</Button>
{onDelete && (
<Button variant="destructive" onClick={onDelete}>删除</Button>
)}
<Button variant="ghost" onClick={onCancel}>取消</Button>
</div>
</div>
);
}
- Step 4: Commit
git add web/client/src/components/accounts/ web/client/src/hooks/useAccounts.ts
git commit -m "feat(web): add account list and form components"
Task 1.5: 设置页前端
Files:
-
Replace:
web/client/src/components/config/ConfigForm.tsx -
Step 1: 重写 ConfigForm.tsx
import { useState, useEffect } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { api } from '@/lib/api';
export function ConfigForm() {
const [configs, setConfigs] = useState<Record<string, string>>({});
const [saving, setSaving] = useState(false);
useEffect(() => {
api.getConfigs().then((list) => {
const map: Record<string, string> = {};
list.forEach((c) => { map[c.key] = JSON.stringify(c.value, null, 2); });
setConfigs(map);
});
}, []);
const handleSave = async (key: string, raw: string) => {
setSaving(true);
try {
await api.saveConfig(key, JSON.parse(raw));
} catch { alert('Invalid JSON'); }
setSaving(false);
};
const configKeys = [
{ key: 'api_keys', label: 'API 密钥' },
{ key: 'defaults', label: '默认参数' },
{ key: 'endpoints', label: '服务端点' },
];
return (
<div className="max-w-lg p-6 space-y-6">
<h2 className="text-lg font-semibold">设置</h2>
{configKeys.map(({ key, label }) => (
<div key={key}>
<label className="text-xs text-zinc-500">{label}</label>
<textarea
value={configs[key] || '{}'}
onChange={(e) => setConfigs((c) => ({ ...c, [key]: e.target.value }))}
rows={6}
className="mt-1 w-full rounded-md border border-zinc-800 bg-zinc-900 px-3 py-2 text-sm font-mono resize-y"
/>
<Button
size="sm"
variant="outline"
className="mt-1"
disabled={saving}
onClick={() => handleSave(key, configs[key])}
>
保存
</Button>
</div>
))}
</div>
);
}
- Step 2: Commit
git add web/client/src/components/config/ConfigForm.tsx
git commit -m "feat(web): add settings config form"
P2: 提示词编辑器
Task 2.1: 提示词后端路由
Files:
-
Create:
web/server/routes/prompts.ts -
Step 1: 创建 server/routes/prompts.ts
import { Router } from 'express';
import fs from 'fs';
import path from 'path';
const PROJECT_ROOT = path.resolve(__dirname, '..', '..', '..');
export const promptsRouter = Router();
const PROMPT_FILES: Record<string, string> = {
storyboard: 'prompts/分镜.md',
image: 'prompts/图片提示词.md',
video: 'prompts/视频提示词.md',
};
promptsRouter.get('/:accountId/:type', (req, res) => {
const { accountId, type } = req.params;
const relPath = PROMPT_FILES[type];
if (!relPath) return res.status(400).json({ error: 'Unknown type: ' + type });
const fullPath = path.join(PROJECT_ROOT, 'accounts', accountId, relPath);
if (!fs.existsSync(fullPath)) return res.status(404).json({ error: 'File not found' });
res.json({ path: relPath, content: fs.readFileSync(fullPath, 'utf-8') });
});
promptsRouter.put('/:accountId/:type', (req, res) => {
const { accountId, type } = req.params;
const { content } = req.body;
const relPath = PROMPT_FILES[type];
if (!relPath) return res.status(400).json({ error: 'Unknown type: ' + type });
const fullPath = path.join(PROJECT_ROOT, 'accounts', accountId, relPath);
const dir = path.dirname(fullPath);
if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(fullPath, content, 'utf-8');
res.json({ ok: true });
});
// List available prompt types for an account
promptsRouter.get('/:accountId', (req, res) => {
res.json(Object.keys(PROMPT_FILES).map((type) => ({
type,
path: PROMPT_FILES[type],
})));
});
- Step 2: Commit
git add web/server/routes/prompts.ts
git commit -m "feat(web): add prompt file read/write API"
Task 2.2: 提示词分屏编辑器
Files:
-
Create:
web/client/src/hooks/usePrompts.ts -
Create:
web/client/src/components/prompts/PromptEditor.tsx -
Step 1: 创建 hooks/usePrompts.ts
import { useState, useCallback } from 'react';
import { api } from '@/lib/api';
export function usePrompts() {
const [content, setContent] = useState<string>('');
const [path, setPath] = useState<string>('');
const [loading, setLoading] = useState(false);
const load = useCallback(async (accountId: string, type: string) => {
setLoading(true);
try {
const result = await api.getPrompt(accountId, type);
setContent(result.content);
setPath(result.path);
} catch { setContent(''); setPath(''); }
setLoading(false);
}, []);
const save = useCallback(async (accountId: string, type: string, newContent: string) => {
await api.savePrompt(accountId, type, newContent);
setContent(newContent);
}, []);
return { content, path, loading, load, save, setContent };
}
- Step 2: 创建 PromptEditor.tsx
import { useState, useEffect } from 'react';
import { useAccounts } from '@/hooks/useAccounts';
import { usePrompts } from '@/hooks/usePrompts';
import { Button } from '@/components/ui/button';
import { ScrollArea } from '@/components/ui/scroll-area';
const PROMPT_TYPES = [
{ type: 'storyboard', label: '分镜' },
{ type: 'image', label: '图片提示词' },
{ type: 'video', label: '视频提示词' },
] as const;
export function PromptEditor() {
const { accounts } = useAccounts();
const [selectedAccount, setSelectedAccount] = useState<string>('');
const [selectedType, setSelectedType] = useState<string>('storyboard');
const { content, path, loading, load, save } = usePrompts();
useEffect(() => {
if (accounts.length > 0 && !selectedAccount) {
setSelectedAccount(accounts[0].id);
}
}, [accounts]);
useEffect(() => {
if (selectedAccount) {
load(selectedAccount, selectedType);
}
}, [selectedAccount, selectedType]);
return (
<div className="flex h-full">
{/* 左侧列表 */}
<div className="w-48 border-r border-zinc-800 p-3 space-y-3">
<div>
<label className="text-xs text-zinc-500">账户</label>
<select
value={selectedAccount}
onChange={(e) => setSelectedAccount(e.target.value)}
className="mt-1 w-full h-9 rounded-md border border-zinc-800 bg-zinc-900 px-2 text-sm"
>
{accounts.map((a) => (
<option key={a.id} value={a.id}>{a.name}</option>
))}
</select>
</div>
<div>
<label className="text-xs text-zinc-500">模板</label>
<div className="mt-1 space-y-0.5">
{PROMPT_TYPES.map(({ type, label }) => (
<button
key={type}
onClick={() => setSelectedType(type)}
className={`w-full text-left px-2 py-1.5 rounded text-sm ${
selectedType === type ? 'bg-zinc-800 text-white' : 'text-zinc-500 hover:text-zinc-300'
}`}
>
{label}
</button>
))}
</div>
</div>
</div>
{/* 编辑器 */}
<div className="flex-1 flex flex-col">
<div className="flex items-center justify-between px-4 py-2 border-b border-zinc-800">
<span className="text-xs text-zinc-500 font-mono">{path}</span>
<Button
size="sm"
onClick={() => save(selectedAccount, selectedType, content)}
disabled={!selectedAccount || loading}
>
保存
</Button>
</div>
<textarea
value={content}
onChange={(e) => setContent(e.target.value)}
className="flex-1 w-full bg-zinc-950 text-zinc-200 font-mono text-sm p-4 resize-none outline-none"
placeholder="加载中..."
spellCheck={false}
/>
</div>
</div>
);
}
- Step 3: 将编辑器集成到 App.tsx 的 accounts 视图
在 App.tsx 中,将 accounts 视图改为使用 PromptEditor。当前 App.tsx 中 accounts case 指向 AccountList。由于账户视图应该是账户列表(从中选编辑)和提示词编辑的混合,我们需要调整——用户从账户列表中选择一个账户后可以编辑其提示词。
更新 App.tsx:
// accounts 视图改为同时显示账户列表和提示词编辑器
case 'accounts':
return <AccountList />;
账户列表组件右侧现已包含 AccountForm,提示词编辑可以从 AccountForm 中的一个按钮进入,或者保持当前通过独立编辑器访问。简化处理——保持 AccountList 不变,侧边栏点击账户后右侧显示账户管理,提示词编辑作为子视图。
实际上按设计,账户视图右侧应包含提示词编辑入口。修改 AccountList 让用户能切换"账户设置"/"提示词编辑"两个子标签。
简化方案:在 AccountList 右侧顶部加标签切换。
更新 AccountList.tsx — 在右侧主区域顶部加标签栏:
import { PromptEditor } from '@/components/prompts/PromptEditor';
// ... 在右侧区域顶部添加子标签
const [subTab, setSubTab] = useState<'info' | 'prompts'>('info');
这一步代码较长,直接更新 AccountList 组件的文件内容。
- Step 4: 更新 AccountList.tsx 添加提示词编辑入口
将 AccountList.tsx 右侧内容区顶部加上标签切换:
在右侧 <div className="flex-1 p-6"> 之前/内部顶部添加:
{/* 在 AccountList 组件中添加子标签切换 */}
const [subTab, setSubTab] = useState<'info' | 'prompts'>('info');
右侧面板改为:
<div className="flex-1 flex flex-col">
<div className="flex gap-0 border-b border-zinc-800 px-6">
<button
onClick={() => setSubTab('info')}
className={`px-3 py-2 text-sm ${subTab === 'info' ? 'border-b-2 border-white text-white' : 'text-zinc-500'}`}
>
账户设置
</button>
<button
onClick={() => setSubTab('prompts')}
className={`px-3 py-2 text-sm ${subTab === 'prompts' ? 'border-b-2 border-white text-white' : 'text-zinc-500'}`}
>
提示词模板
</button>
</div>
<div className="flex-1">
{subTab === 'info' ? (
/* AccountForm 或空状态 */
(editing || creating) ? (
<div className="p-6"><AccountForm ... /></div>
) : (
<div className="flex items-center justify-center h-full text-zinc-600 text-sm">
选择一个账户或创建新账户
</div>
)
) : (
<PromptEditor />
)}
</div>
</div>
- Step 5: Commit
git add web/client/src/components/accounts/AccountList.tsx web/client/src/components/prompts/ web/client/src/hooks/usePrompts.ts web/server/routes/prompts.ts
git commit -m "feat(web): add prompt editor with split-pane and Markdown editing"
P3: 聊天 + Pipeline
Task 3.1: WebSocket 聊天后端
Files:
-
Create:
web/server/ws/chat.ts -
Step 1: 创建 server/ws/chat.ts
import { WebSocket } from 'ws';
import { getDb } from '../db';
import { v4 as uuid } from 'uuid';
interface ChatMessage {
type: 'message' | 'tool_call' | 'tool_result' | 'pipeline_progress' | 'error';
data: Record<string, unknown>;
}
export function handleChat(ws: WebSocket) {
let conversationId: string | null = null;
ws.on('message', async (raw) => {
try {
const msg = JSON.parse(raw.toString());
if (msg.type === 'init') {
conversationId = msg.conversationId || uuid();
// Load history
const history = getDb().prepare(
'SELECT * FROM messages WHERE conversation_id = ? ORDER BY created_at'
).all(conversationId);
ws.send(JSON.stringify({ type: 'history', data: { conversationId, messages: history } }));
return;
}
if (msg.type === 'chat') {
const { content } = msg;
const msgId = uuid();
// Save user message
getDb().prepare(
'INSERT INTO messages (id, conversation_id, role, content) VALUES (?, ?, ?, ?)'
).run(msgId, conversationId, 'user', content);
// Echo user message
ws.send(JSON.stringify({ type: 'message', data: { id: msgId, role: 'user', content } }));
// Placeholder: echo assistant response (pi-agent-core integration comes in Task 3.3)
const assistantId = uuid();
const assistantContent = `收到你的消息:「${content}」。Agent 引擎正在启动中...`;
getDb().prepare(
'INSERT INTO messages (id, conversation_id, role, content) VALUES (?, ?, ?, ?)'
).run(assistantId, conversationId, 'assistant', assistantContent);
ws.send(JSON.stringify({
type: 'message',
data: { id: assistantId, role: 'assistant', content: assistantContent },
}));
}
if (msg.type === 'create_conversation') {
const { title, accountId } = msg;
conversationId = uuid();
getDb().prepare(
'INSERT INTO conversations (id, title, account_id) VALUES (?, ?, ?)'
).run(conversationId, title || '新对话', accountId || null);
ws.send(JSON.stringify({ type: 'conversation_created', data: { id: conversationId, title } }));
}
} catch (e) {
ws.send(JSON.stringify({ type: 'error', data: { message: (e as Error).message } }));
}
});
ws.on('close', () => {
// cleanup if needed
});
}
由于项目不依赖 uuid 包,用内置 crypto.randomUUID() 替代:
import { randomUUID } from 'crypto';
全部 uuid() 改为 randomUUID()。
- Step 2: 更新 package.json 不需额外依赖(crypto 内置)
验证 randomUUID 在 Node 19+ 可用。
- Step 3: Commit
git add web/server/ws/chat.ts
git commit -m "feat(web): add WebSocket chat handler with message persistence"
Task 3.2: 聊天前端组件
Files:
-
Replace:
web/client/src/components/chat/ChatView.tsx -
Create:
web/client/src/components/chat/ChatMessage.tsx -
Create:
web/client/src/components/chat/ChatInput.tsx -
Create:
web/client/src/hooks/useChat.ts -
Create:
web/client/src/lib/websocket.ts -
Step 1: 创建 lib/websocket.ts
type MessageHandler = (data: Record<string, unknown>) => void;
class ChatSocket {
private ws: WebSocket | null = null;
private handlers: Map<string, MessageHandler[]> = new Map();
private reconnectTimer: ReturnType<typeof setTimeout> | null = null;
connect() {
const protocol = location.protocol === 'https:' ? 'wss:' : 'ws:';
const url = `${protocol}//${location.host}/ws`;
this.ws = new WebSocket(url);
this.ws.onopen = () => {
this.emit('connected', {});
};
this.ws.onmessage = (event) => {
try {
const { type, data } = JSON.parse(event.data);
this.emit(type, data);
} catch {}
};
this.ws.onclose = () => {
this.reconnectTimer = setTimeout(() => this.connect(), 3000);
};
}
on(type: string, handler: MessageHandler) {
if (!this.handlers.has(type)) this.handlers.set(type, []);
this.handlers.get(type)!.push(handler);
}
off(type: string, handler: MessageHandler) {
const list = this.handlers.get(type);
if (list) this.handlers.set(type, list.filter((h) => h !== handler));
}
send(type: string, data: Record<string, unknown> = {}) {
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify({ type, ...data }));
}
}
private emit(type: string, data: Record<string, unknown>) {
(this.handlers.get(type) || []).forEach((h) => h(data));
}
disconnect() {
if (this.reconnectTimer) clearTimeout(this.reconnectTimer);
this.ws?.close();
}
}
export const chatSocket = new ChatSocket();
- Step 2: 创建 hooks/useChat.ts
import { useState, useEffect, useCallback, useRef } from 'react';
import { chatSocket } from '@/lib/websocket';
import type { Message } from '@/types';
export function useChat(conversationId: string | null) {
const [messages, setMessages] = useState<Message[]>([]);
const [connected, setConnected] = useState(false);
const pendingRef = useRef(false);
useEffect(() => {
chatSocket.connect();
chatSocket.on('connected', () => setConnected(true));
chatSocket.on('history', (data) => {
setMessages((data.messages as Message[]) || []);
});
chatSocket.on('message', (data) => {
setMessages((prev) => [...prev, data as unknown as Message]);
});
return () => {
chatSocket.disconnect();
};
}, []);
useEffect(() => {
if (conversationId && connected && !pendingRef.current) {
pendingRef.current = true;
chatSocket.send('init', { conversationId });
}
if (!conversationId) {
pendingRef.current = false;
setMessages([]);
}
}, [conversationId, connected]);
const send = useCallback((content: string) => {
chatSocket.send('chat', { content });
}, []);
const createConversation = useCallback((title: string, accountId?: string) => {
chatSocket.send('create_conversation', { title, accountId });
}, []);
return { messages, connected, send, createConversation };
}
- Step 3: 重写 ChatView.tsx
import { useEffect } from 'react';
import { useAppStore } from '@/store';
import { useChat } from '@/hooks/useChat';
import { ChatMessage } from './ChatMessage';
import { ChatInput } from './ChatInput';
import { ScrollArea } from '@/components/ui/scroll-area';
export function ChatView() {
const { activeConversationId, conversations, setConversations } = useAppStore();
const { messages, connected, send, createConversation } = useChat(activeConversationId);
useEffect(() => {
fetch('/api/pipeline/conversations')
.then((r) => r.json())
.then(setConversations)
.catch(() => {});
}, [messages]);
const handleNewConversation = () => {
createConversation('新对话');
fetch('/api/pipeline/conversations')
.then((r) => r.json())
.then(setConversations);
};
if (!activeConversationId) {
return (
<div className="flex-1 flex flex-col items-center justify-center gap-4 text-zinc-500">
<p>选择对话或开始新对话</p>
<button
onClick={handleNewConversation}
className="px-4 py-2 rounded-md bg-zinc-800 text-sm hover:bg-zinc-700 transition-colors"
>
开始新对话
</button>
</div>
);
}
return (
<div className="flex-1 flex flex-col">
<div className="px-4 py-2 border-b border-zinc-800 flex items-center gap-2">
<div className={`w-2 h-2 rounded-full ${connected ? 'bg-green-500' : 'bg-red-500'}`} />
<span className="text-xs text-zinc-500">{connected ? '已连接' : '连接中...'}</span>
</div>
<ScrollArea className="flex-1 px-4 py-4">
{messages.map((msg) => (
<ChatMessage key={msg.id} message={msg} />
))}
</ScrollArea>
<ChatInput onSend={send} />
</div>
);
}
- Step 4: 创建 ChatMessage.tsx
import { cn } from '@/lib/utils';
import type { Message } from '@/types';
export function ChatMessage({ message }: { message: Message }) {
const isUser = message.role === 'user';
return (
<div className={cn('mb-4 flex', isUser ? 'justify-end' : 'justify-start')}>
<div
className={cn(
'max-w-[80%] rounded-lg px-4 py-2.5 text-sm leading-relaxed',
isUser ? 'bg-zinc-800 text-zinc-100' : 'bg-zinc-900 text-zinc-300 border border-zinc-800'
)}
>
{message.content}
</div>
</div>
);
}
- Step 5: 创建 ChatInput.tsx
import { useState, useRef } from 'react';
import { Send } from 'lucide-react';
import { Button } from '@/components/ui/button';
export function ChatInput({ onSend }: { onSend: (content: string) => void }) {
const [input, setInput] = useState('');
const ref = useRef<HTMLTextAreaElement>(null);
const handleSend = () => {
if (!input.trim()) return;
onSend(input.trim());
setInput('');
ref.current?.focus();
};
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
handleSend();
}
};
return (
<div className="p-4 border-t border-zinc-800">
<div className="flex items-end gap-2 bg-zinc-900 rounded-lg border border-zinc-800 px-3 py-2">
<textarea
ref={ref}
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={handleKeyDown}
rows={1}
placeholder="输入指令..."
className="flex-1 bg-transparent text-sm resize-none outline-none placeholder:text-zinc-600"
/>
<Button size="icon" variant="ghost" className="h-8 w-8" onClick={handleSend}>
<Send size={14} />
</Button>
</div>
</div>
);
}
- Step 6: 创建对话列表 API 后端路由(pipeline 路由中的 conversations)
更新 server/routes/pipeline.ts(占位):
import { Router } from 'express';
import { getDb } from '../db';
export const pipelineRouter = Router();
pipelineRouter.get('/conversations', (_req, res) => {
const rows = getDb().prepare('SELECT * FROM conversations ORDER BY updated_at DESC').all();
res.json(rows);
});
pipelineRouter.get('/conversations/:id/messages', (req, res) => {
const rows = getDb().prepare(
'SELECT * FROM messages WHERE conversation_id = ? ORDER BY created_at'
).all(req.params.id);
res.json(rows);
});
pipelineRouter.delete('/conversations/:id', (req, res) => {
getDb().prepare('DELETE FROM conversations WHERE id = ?').run(req.params.id);
res.status(204).send();
});
- Step 7: Commit
git add web/client/src/components/chat/ web/client/src/hooks/useChat.ts web/client/src/lib/websocket.ts web/server/routes/pipeline.ts
git commit -m "feat(web): add chat UI with WebSocket streaming and conversation persistence"
Task 3.3: pi-agent-core 集成 + 工具注册
Files:
-
Create:
web/server/agent/index.ts -
Create:
web/server/agent/tools.ts -
Modify:
web/server/ws/chat.ts(集成 agent 调用) -
Step 1: 创建 server/agent/tools.ts
由于 pi-agent-core 的具体 API 版本随时变化,定义工具接口并在 ws/chat 中集成调用。
import { execSync, spawn } from 'child_process';
import path from 'path';
const PROJECT_ROOT = path.resolve(__dirname, '..', '..', '..');
const PIPELINE_SCRIPT = path.join(PROJECT_ROOT, '.claude', 'skills', 'video-from-script', 'scripts', 'pipeline.js');
export interface ToolDefinition {
name: string;
description: string;
parameters: Record<string, unknown>;
execute: (params: Record<string, unknown>) => Promise<string>;
}
export const tools: ToolDefinition[] = [
{
name: 'list_accounts',
description: '列出所有可用账号',
parameters: { type: 'object', properties: {}, required: [] },
execute: async () => {
const fs = await import('fs');
const accountsDir = path.join(PROJECT_ROOT, 'accounts');
const dirs = fs.readdirSync(accountsDir, { withFileTypes: true })
.filter((d) => d.isDirectory() && !d.name.startsWith('_'))
.map((d) => d.name);
return JSON.stringify(dirs);
},
},
{
name: 'run_pipeline_phase',
description: '执行 pipeline 阶段 (images/upload/videos/tts/assemble)',
parameters: {
type: 'object',
properties: {
manifest: { type: 'string', description: 'manifest.json 绝对路径' },
phase: { type: 'string', description: '阶段名: images, upload, videos, tts, assemble' },
},
required: ['manifest', 'phase'],
},
execute: async (params) => {
const { manifest, phase } = params as { manifest: string; phase: string };
return new Promise((resolve, reject) => {
const proc = spawn('node', [PIPELINE_SCRIPT, 'run', '--manifest', manifest, '--phase', phase], {
cwd: PROJECT_ROOT,
env: { ...process.env },
});
let output = '';
proc.stdout.on('data', (d: Buffer) => { output += d.toString(); });
proc.stderr.on('data', (d: Buffer) => { output += d.toString(); });
proc.on('close', (code) => {
code === 0 ? resolve(output) : reject(new Error(`Pipeline failed with code ${code}: ${output}`));
});
});
},
},
{
name: 'pipeline_status',
description: '查看 pipeline 进度',
parameters: {
type: 'object',
properties: {
manifest: { type: 'string', description: 'manifest.json 绝对路径' },
},
required: ['manifest'],
},
execute: async (params) => {
const { manifest } = params as { manifest: string };
const result = execSync(`node "${PIPELINE_SCRIPT}" status --manifest "${manifest}"`, {
cwd: PROJECT_ROOT, encoding: 'utf-8',
});
return result;
},
},
{
name: 'create_account',
description: '创建新账号',
parameters: {
type: 'object',
properties: {
id: { type: 'string', description: '账号 ID' },
name: { type: 'string', description: '账号名称' },
},
required: ['id', 'name'],
},
execute: async (params) => {
const { id, name } = params as { id: string; name: string };
const result = execSync(
`node "${PIPELINE_SCRIPT}" create-account --id "${id}" --name "${name}" --desc "" --video-model veo3-fast`,
{ cwd: PROJECT_ROOT, encoding: 'utf-8' }
);
return result;
},
},
];
- Step 2: 创建 server/agent/index.ts
import { tools, ToolDefinition } from './tools';
export interface AgentConfig {
model?: string;
maxTokens?: number;
}
export class VideoAgent {
private tools: ToolDefinition[];
constructor() {
this.tools = tools;
}
getToolDefinitions() {
return this.tools.map((t) => ({
name: t.name,
description: t.description,
parameters: t.parameters,
}));
}
async executeTool(name: string, params: Record<string, unknown>): Promise<string> {
const tool = this.tools.find((t) => t.name === name);
if (!tool) throw new Error(`Unknown tool: ${name}`);
return tool.execute(params);
}
getSystemPrompt(accountContext?: string): string {
return `你是美图 Agent,帮助用户进行短视频创作。
可用账号:${accountContext || '暂无'}
你可以:
1. 帮用户创建新账号
2. 查看和管理已有账号
3. 执行视频创作 pipeline(分镜→生图→生视频→TTS→成片)
4. 管理提示词模板
用户想创作视频时,一步步引导他们完成流程。`;
}
}
export const videoAgent = new VideoAgent();
- Step 3: 更新 ws/chat.ts 集成 Agent
更新 handleChat 函数中的 msg.type === 'chat' 处理,调用 videoAgent:
在 chat case 中,echo 阶段改为:
// 检查是否是工具调用指令
const { videoAgent } = await import('../agent');
const systemPrompt = videoAgent.getSystemPrompt();
// 简易命令解析
if (content.startsWith('/run')) {
const [_, manifest, phase] = content.split(' ');
try {
const result = await videoAgent.executeTool('run_pipeline_phase', { manifest, phase });
ws.send(JSON.stringify({
type: 'tool_result',
data: { tool: 'run_pipeline_phase', result },
}));
} catch (e) {
ws.send(JSON.stringify({
type: 'error',
data: { message: (e as Error).message },
}));
}
return;
}
- Step 4: Commit
git add web/server/agent/ web/server/ws/chat.ts
git commit -m "feat(web): add pi-agent tool layer with pipeline integration"
Task 3.4: Pipeline 进度组件
Files:
-
Create:
web/client/src/components/chat/PipelineProgress.tsx -
Step 1: 创建 PipelineProgress.tsx
import { cn } from '@/lib/utils';
interface Props {
phase: string;
progress: number; // 0-100
currentItem?: number;
totalItems?: number;
status?: string;
}
export function PipelineProgress({ phase, progress, currentItem, totalItems, status }: Props) {
const phaseLabel: Record<string, string> = {
images: '生成图片',
upload: '上传素材',
videos: '生成视频',
tts: '配音',
assemble: '成片组装',
};
return (
<div className="bg-zinc-900 border border-zinc-800 rounded-lg p-3 my-2">
<div className="flex items-center justify-between mb-1.5">
<span className="text-xs text-zinc-400">
{phaseLabel[phase] || phase}
{currentItem && totalItems ? ` (${currentItem}/${totalItems})` : ''}
</span>
<span className="text-xs text-zinc-500">{progress}%</span>
</div>
<div className="w-full h-1.5 bg-zinc-800 rounded-full overflow-hidden">
<div
className={cn(
'h-full rounded-full transition-all duration-500',
progress < 100 ? 'bg-blue-500' : 'bg-green-500'
)}
style={{ width: `${progress}%` }}
/>
</div>
{status && <p className="text-xs text-zinc-500 mt-1">{status}</p>}
</div>
);
}
- Step 2: 在 ChatView 中集成 PipelineProgress
在 ChatMessage 列表渲染时检测 tool_result 类型的消息,解析并显示进度条。
- Step 3: Commit
git add web/client/src/components/chat/PipelineProgress.tsx
git commit -m "feat(web): add pipeline progress inline component"
P4: 资产画廊
Task 4.1: 资产后端路由
Files:
-
Create:
web/server/routes/assets.ts -
Step 1: 创建 server/routes/assets.ts
import { Router } from 'express';
import { getDb } from '../db';
import fs from 'fs';
import path from 'path';
import { randomUUID } from 'crypto';
const PROJECT_ROOT = path.resolve(__dirname, '..', '..', '..');
export const assetsRouter = Router();
// List assets with optional filters
assetsRouter.get('/', (req, res) => {
const { accountId, type } = req.query;
let sql = 'SELECT * FROM assets WHERE 1=1';
const params: unknown[] = [];
if (accountId) { sql += ' AND account_id = ?'; params.push(accountId); }
if (type) { sql += ' AND type = ?'; params.push(type); }
sql += ' ORDER BY created_at DESC LIMIT 200';
const rows = getDb().prepare(sql).all(...params);
res.json(rows);
});
// Delete asset (and file)
assetsRouter.delete('/:id', (req, res) => {
const asset = getDb().prepare('SELECT * FROM assets WHERE id = ?').get(req.params.id) as {
file_path: string;
} | undefined;
if (!asset) return res.status(404).json({ error: 'Asset not found' });
// Delete file
const filePath = path.join(PROJECT_ROOT, asset.file_path);
if (fs.existsSync(filePath)) fs.unlinkSync(filePath);
getDb().prepare('DELETE FROM assets WHERE id = ?').run(req.params.id);
res.status(204).send();
});
// Scan output directories and index assets
assetsRouter.post('/scan', (_req, res) => {
const outputDir = path.join(PROJECT_ROOT, 'output');
if (!fs.existsSync(outputDir)) return res.json({ indexed: 0 });
const dirs = fs.readdirSync(outputDir, { withFileTypes: true })
.filter((d) => d.isDirectory());
let indexed = 0;
for (const dir of dirs) {
const manifestPath = path.join(outputDir, dir.name, 'manifest.json');
if (!fs.existsSync(manifestPath)) continue;
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
const accountId = manifest.account?.id || '';
// Scan images
const imagesDir = path.join(outputDir, dir.name, 'images');
if (fs.existsSync(imagesDir)) {
const files = fs.readdirSync(imagesDir);
for (const file of files) {
if (!/\.(jpe?g|png|webp)$/i.test(file)) continue;
const id = randomUUID();
const filePath = path.relative(PROJECT_ROOT, path.join(imagesDir, file));
const match = file.match(/scene_(\d+)/);
const shotIndex = match ? parseInt(match[1]) : null;
const exists = getDb().prepare('SELECT id FROM assets WHERE file_path = ?').get(filePath);
if (exists) continue;
getDb().prepare(
'INSERT INTO assets (id, account_id, manifest_path, type, file_path, shot_index) VALUES (?,?,?,?,?,?)'
).run(id, accountId, path.relative(PROJECT_ROOT, manifestPath), 'image', filePath, shotIndex);
indexed++;
}
}
// Scan videos
const videosDir = path.join(outputDir, dir.name, 'videos');
if (fs.existsSync(videosDir)) {
const files = fs.readdirSync(videosDir);
for (const file of files) {
if (!/\.mp4$/i.test(file)) continue;
const id = randomUUID();
const filePath = path.relative(PROJECT_ROOT, path.join(videosDir, file));
const match = file.match(/scene_(\d+)/);
const shotIndex = match ? parseInt(match[1]) : null;
const exists = getDb().prepare('SELECT id FROM assets WHERE file_path = ?').get(filePath);
if (exists) continue;
getDb().prepare(
'INSERT INTO assets (id, account_id, manifest_path, type, file_path, shot_index) VALUES (?,?,?,?,?,?)'
).run(id, accountId, path.relative(PROJECT_ROOT, manifestPath), 'video', filePath, shotIndex);
indexed++;
}
}
}
res.json({ indexed });
});
- Step 2: Commit
git add web/server/routes/assets.ts
git commit -m "feat(web): add asset CRUD and scanning API"
Task 4.2: 资产画廊前端
Files:
-
Replace:
web/client/src/components/assets/AssetGallery.tsx -
Create:
web/client/src/components/assets/AssetPreview.tsx -
Create:
web/client/src/hooks/useAssets.ts -
Step 1: 创建 hooks/useAssets.ts
import { useState, useEffect, useCallback } from 'react';
import { api } from '@/lib/api';
import type { Asset } from '@/types';
export function useAssets(params?: { accountId?: string; type?: string }) {
const [assets, setAssets] = useState<Asset[]>([]);
const [loading, setLoading] = useState(true);
const refresh = useCallback(() => {
setLoading(true);
api.listAssets(params).then(setAssets).finally(() => setLoading(false));
}, [params?.accountId, params?.type]);
useEffect(() => { refresh(); }, [refresh]);
const remove = (id: string) => api.deleteAsset(id).then(refresh);
const scan = () => api.listAssets({}).then(() => refresh()); // will call scan endpoint instead
return { assets, loading, refresh, remove, scan };
}
- Step 2: 重写 AssetGallery.tsx
import { useState, useEffect } from 'react';
import { Trash2, RefreshCw } from 'lucide-react';
import { useAssets } from '@/hooks/useAssets';
import { AssetPreview } from './AssetPreview';
import { Button } from '@/components/ui/button';
import type { Asset } from '@/types';
export function AssetGallery() {
const [accountFilter, setAccountFilter] = useState('');
const [typeFilter, setTypeFilter] = useState<string>('');
const [accounts, setAccounts] = useState<{ id: string; name: string }[]>([]);
const { assets, loading, remove, scan } = useAssets({
accountId: accountFilter || undefined,
type: typeFilter || undefined,
});
const [previewAsset, setPreviewAsset] = useState<Asset | null>(null);
useEffect(() => {
fetch('/api/accounts').then((r) => r.json()).then(setAccounts);
}, []);
const handleScan = async () => {
await fetch('/api/assets/scan', { method: 'POST' });
window.location.reload();
};
return (
<div className="flex flex-col h-full">
{/* 工具栏 */}
<div className="flex items-center gap-3 px-4 py-3 border-b border-zinc-800">
<select
value={accountFilter}
onChange={(e) => setAccountFilter(e.target.value)}
className="h-8 rounded-md border border-zinc-800 bg-zinc-900 px-2 text-xs"
>
<option value="">全部账号</option>
{accounts.map((a) => (
<option key={a.id} value={a.id}>{a.name}</option>
))}
</select>
<select
value={typeFilter}
onChange={(e) => setTypeFilter(e.target.value)}
className="h-8 rounded-md border border-zinc-800 bg-zinc-900 px-2 text-xs"
>
<option value="">全部类型</option>
<option value="image">图片</option>
<option value="video">视频</option>
</select>
<Button size="sm" variant="outline" className="h-8 text-xs" onClick={handleScan}>
<RefreshCw size={12} className="mr-1" />
扫描
</Button>
</div>
{/* 瀑布流 */}
<div className="flex-1 overflow-auto p-4">
{loading ? (
<p className="text-zinc-500 text-sm text-center mt-8">加载中...</p>
) : assets.length === 0 ? (
<p className="text-zinc-600 text-sm text-center mt-8">暂无资产,点击"扫描"导入</p>
) : (
<div className="grid grid-cols-4 gap-3">
{assets.map((asset) => (
<div
key={asset.id}
className="group relative aspect-[9/16] bg-zinc-900 rounded-lg overflow-hidden cursor-pointer border border-zinc-800 hover:border-zinc-600 transition-colors"
onClick={() => setPreviewAsset(asset)}
>
{asset.type === 'image' ? (
<img
src={`/api/assets/file?path=${encodeURIComponent(asset.file_path)}`}
alt=""
className="w-full h-full object-cover"
loading="lazy"
/>
) : (
<video
src={`/api/assets/file?path=${encodeURIComponent(asset.file_path)}`}
className="w-full h-full object-cover"
muted
/>
)}
<button
onClick={(e) => { e.stopPropagation(); remove(asset.id); }}
className="absolute top-1 right-1 p-1 rounded bg-black/60 opacity-0 group-hover:opacity-100 transition-opacity"
>
<Trash2 size={12} className="text-red-400" />
</button>
</div>
))}
</div>
)}
</div>
{previewAsset && <AssetPreview asset={previewAsset} onClose={() => setPreviewAsset(null)} />}
</div>
);
}
- Step 3: 创建 AssetPreview.tsx
import { X } from 'lucide-react';
import type { Asset } from '@/types';
export function AssetPreview({ asset, onClose }: { asset: Asset; onClose: () => void }) {
return (
<div
className="fixed inset-0 z-50 bg-black/90 flex items-center justify-center"
onClick={onClose}
>
<button
onClick={onClose}
className="absolute top-4 right-4 p-2 rounded-full bg-zinc-800 hover:bg-zinc-700 text-white"
>
<X size={20} />
</button>
<div className="max-w-[90vw] max-h-[90vh]" onClick={(e) => e.stopPropagation()}>
{asset.type === 'image' ? (
<img
src={`/api/assets/file?path=${encodeURIComponent(asset.file_path)}`}
alt=""
className="max-w-full max-h-[90vh] object-contain rounded-lg"
/>
) : (
<video
src={`/api/assets/file?path=${encodeURIComponent(asset.file_path)}`}
controls
autoPlay
className="max-w-full max-h-[90vh] rounded-lg"
/>
)}
</div>
</div>
);
}
- Step 4: 添加文件访问代理路由
在 server/index.ts 中或 assets.ts 中添加文件静态服务:
// 在 assetsRouter 中添加文件访问端点
assetsRouter.get('/file', (req, res) => {
const filePath = req.query.path as string;
if (!filePath) return res.status(400).send('Missing path');
const fullPath = path.resolve(PROJECT_ROOT, filePath);
// Security: ensure path is within project
if (!fullPath.startsWith(PROJECT_ROOT)) return res.status(403).send('Forbidden');
if (!fs.existsSync(fullPath)) return res.status(404).send('Not found');
res.sendFile(fullPath);
});
- Step 5: Commit
git add web/client/src/components/assets/ web/client/src/hooks/useAssets.ts web/server/routes/assets.ts
git commit -m "feat(web): add asset gallery with waterfall layout, preview and delete"
P5: 对话历史 + 断点续跑
Task 5.1: 对话列表 API 完善 + 搜索
Files:
-
Modify:
web/server/routes/pipeline.ts -
Step 1: 更新 pipeline.ts 添加搜索和断点续跑
import { Router } from 'express';
import { getDb } from '../db';
import { execSync } from 'child_process';
import path from 'path';
const PROJECT_ROOT = path.resolve(__dirname, '..', '..', '..');
const PIPELINE_SCRIPT = path.join(PROJECT_ROOT, '.claude', 'skills', 'video-from-script', 'scripts', 'pipeline.js');
export const pipelineRouter = Router();
// List conversations
pipelineRouter.get('/conversations', (req, res) => {
const { search } = req.query;
let sql = 'SELECT * FROM conversations';
const params: string[] = [];
if (search) {
sql += ' WHERE title LIKE ?';
params.push(`%${search}%`);
}
sql += ' ORDER BY updated_at DESC LIMIT 100';
const rows = getDb().prepare(sql).all(...params);
res.json(rows);
});
// Get conversation messages
pipelineRouter.get('/conversations/:id/messages', (req, res) => {
const rows = getDb().prepare(
'SELECT * FROM messages WHERE conversation_id = ? ORDER BY created_at'
).all(req.params.id);
res.json(rows);
});
// Delete conversation
pipelineRouter.delete('/conversations/:id', (req, res) => {
getDb().prepare('DELETE FROM conversations WHERE id = ?').run(req.params.id);
res.status(204).send();
});
// Rename conversation
pipelineRouter.patch('/conversations/:id', (req, res) => {
const { title } = req.body;
getDb().prepare(
'UPDATE conversations SET title = ?, updated_at = datetime(\'now\') WHERE id = ?'
).run(title, req.params.id);
res.json({ ok: true });
});
// Pipeline status
pipelineRouter.get('/status', (req, res) => {
const { manifest } = req.query;
if (!manifest) return res.status(400).json({ error: 'manifest path required' });
try {
const result = execSync(`node "${PIPELINE_SCRIPT}" status --manifest "${manifest}"`, {
cwd: PROJECT_ROOT, encoding: 'utf-8',
});
res.json({ output: result });
} catch (e) {
res.status(500).json({ error: (e as Error).message });
}
});
// Pipeline resume (断点续跑)
pipelineRouter.post('/resume', (req, res) => {
const { manifest } = req.body;
if (!manifest) return res.status(400).json({ error: 'manifest path required' });
try {
const result = execSync(`node "${PIPELINE_SCRIPT}" run --manifest "${manifest}" --resume`, {
cwd: PROJECT_ROOT, encoding: 'utf-8',
});
res.json({ output: result });
} catch (e) {
res.status(500).json({ error: (e as Error).message });
}
});
- Step 2: 更新前端 MiddlePanel 支持搜索
更新 MiddlePanel.tsx,让搜索输入框实时过滤对话列表。
- Step 3: Commit
git add web/server/routes/pipeline.ts web/client/src/components/layout/MiddlePanel.tsx
git commit -m "feat(web): add conversation search, rename, and pipeline resume"
Task 5.2: 断点续跑集成
Files:
-
Modify:
web/client/src/components/chat/ChatView.tsx -
Modify:
web/client/src/components/chat/ChatMessage.tsx -
Step 1: 在 ChatView 中检查关联的 manifest 并显示恢复按钮
// 在 ChatView 中,获取对话关联的 pipeline 状态
// 如果检测到未完成的 pipeline,在消息流顶部显示恢复卡片
- Step 2: Commit
git add web/client/src/components/chat/
git commit -m "feat(web): add pipeline resume from conversation context"
开发指南
启动
cd web
npm run dev # 同时启动 Express (3001) + Vite (5173)
npm run db:init # 首次初始化数据库
常用命令
npm run dev:server # 仅后端
npm run dev:client # 仅前端
Vite 代理说明
前端 5173 的 /api/* 请求代理到后端 3001,/ws WebSocket 同样代理。开发时访问 http://localhost:5173。
数据库位置
web/data/meitu-agent.db — 自动创建,包含 conversations、messages、assets、pipeline_runs、configs 五个表。
类型一致性检查
Account类型在前端types/index.ts和后端routes/accounts.ts中字段一致Message的 role 使用'user' | 'assistant' | 'system' | 'tool'Asset.type使用'image' | 'video'NavView使用'chat' | 'accounts' | 'assets' | 'config'- WebSocket 消息 type 使用
'init' | 'chat' | 'create_conversation' | 'message' | 'history' | 'tool_result' | 'pipeline_progress' | 'error' | 'conversation_created' | 'connected'