# FishStudio

**Repository Path**: kzc_666/fish-studio

## Basic Information

- **Project Name**: FishStudio
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-07-03
- **Last Updated**: 2026-07-03

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# VoxWeaver Studio

**[English](#english)** · **[中文](#中文)**

---

<a id="english"></a>

## English

VoxWeaver Studio is a local voice production WebUI that combines **Fish Audio** (`s2.1-pro-free`) and **DeepSeek** (`deepseek-v4-flash` / `deepseek-v4-pro`) for AI text optimization, multi-role scripting, voice cloning, batch TTS, and ffmpeg audio merging.

### Features

- **Bilingual UI** — Chinese default in the app; switch **中文 / English** from the top-right of the main area
- Single-text workspace with optional DeepSeek optimization
- Batch `.txt` upload with per-file AI optimization and ZIP export
- Multi-role Studio with role voice audition → lock as persistent clone, AI coverage review, preview/final queues, and merged output
- Voice binding: default, `reference_id`, instant clone (MessagePack), persistent clone
- Role clone history on the Voices page (style + sample text preserved)
- Full Fish `s2.1-pro-free` parameter panel with presets (collapsed by default)
- Emotion/style bracket cue toolbar for Fish S2 models
- Voice library CRUD and persistent cloning (`train_mode: fast`)
- History (latest **50** records, details collapsed by default)
- Settings with **manual** Fish / DeepSeek / ffmpeg health checks (no auto probe on page load)
- Timestamped audio download filenames (`YYYYMMDD-HHMMSS-first20chars.mp3`)

### Fish Audio API Key

1. Sign up at [https://fish.audio](https://fish.audio)
2. Open your developer dashboard and create an API key
3. Copy the key into `.env` as `FISH_API_KEY`

### DeepSeek API Key

1. Sign up at [https://platform.deepseek.com](https://platform.deepseek.com)
2. Create an API key in the API keys section
3. Copy the key into `.env` as `DEEPSEEK_API_KEY`

### Environment Configuration

Copy `.env.example` to `.env` and fill in your keys:

```env
FISH_API_KEY=your_fish_audio_key
DEEPSEEK_API_KEY=your_deepseek_key
FISH_BASE_URL=https://api.fish.audio
FISH_TTS_MODEL=s2.1-pro-free
DEEPSEEK_BASE_URL=https://api.deepseek.com
DEEPSEEK_DEFAULT_MODEL=deepseek-v4-flash
OUTPUT_DIR=outputs
UPLOAD_DIR=uploads
TEMP_DIR=temp
MAX_TXT_FILE_SIZE_MB=10
MAX_AUDIO_FILE_SIZE_MB=50
MAX_BATCH_FILES=50
MAX_PARALLEL_TTS=2
MAX_PARALLEL_AI=2
PORT=3001
CLIENT_URL=http://localhost:5173
```

API keys are read **only on the server** and must never be placed in client code or committed to git.

### Install Dependencies

Requirements: Node.js 18+, pnpm, ffmpeg on PATH.

```bash
pnpm install
pnpm -F @voxweaver/shared build
```

### ffmpeg Installation

- **macOS**: `brew install ffmpeg`
- **Ubuntu/Debian**: `sudo apt install ffmpeg`
- **Windows**: download from [https://ffmpeg.org/download.html](https://ffmpeg.org/download.html) and add to PATH

Verify: `ffmpeg -version`

### Start Frontend and Backend

Development (hot reload):

```bash
cp .env.example .env   # then edit with your keys
pnpm dev
```

- Frontend: [http://localhost:5173](http://localhost:5173)
- Backend API: [http://localhost:3001/api](http://localhost:3001/api)

Production-style (serve built client from Express):

```bash
pnpm build
pnpm start
```

Open [http://localhost:3001](http://localhost:3001)

### Single-Text Generation

1. Open **Workspace** (`/`)
2. Type text or upload a `.txt` file
3. Choose AI mode (or **No optimization**)
4. Click **AI Optimize** (optional), edit TTS text, insert emotion tags
5. Configure voice binding and Fish parameters
6. **Preview / Generate Audio** — download uses a timestamped filename

### Batch TXT Generation

1. Open **Batch** (`/batch`)
2. Upload multiple `.txt` files (per-file AI optimization)
3. Set global or per-file voice binding
4. **Batch generate & ZIP** → `batch-output.zip`

### Multi-Role Studio

1. Open **Studio** (`/studio`)
2. Paste long text → **AI split roles** (default model: `deepseek-v4-pro`)
3. Optional: auto or manual **AI review** for missing segments
4. Per role: **audition** → **regenerate** → **lock as role voice** (saved to clone history)
5. **Preview queue** / **Final queue + merge** → download merged audio and exports

### Voice Library

- **Role clone history** — entries created when locking voices in Studio
- **My clones** — Fish account voices
- Registry loads in stages; health/state sync is manual via **Refresh status**
- Removing a record deletes the local registry file only (not the Fish model)

### reference_id Usage

1. Copy a voice `_id` from **Voices**
2. Set Voice Binding type to `reference_id` and paste the ID

Example curl (use your server env key, never embed in frontend):

```bash
curl --request POST \
  --url https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro-free" \
  --data '{"text":"Hello","reference_id":"YOUR_VOICE_ID","format":"mp3"}'
```

### Instant Clone

1. Prepare reference audio with accurate transcripts
2. Voice Binding → **Instant clone** → upload samples
3. Backend uses MessagePack for direct Fish requests (no base64 in JSON)

### Persistent Clone

1. **Voices** → upload samples → **Create clone** (`train_mode: fast`)
2. Or lock from Studio audition via **Lock as role voice**
3. Wait until `state === trained` before batch/studio generation

### Fish s2.1-pro-free Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `format` | mp3 / wav / pcm / opus | mp3 |
| `sample_rate` | Format-dependent sample rate | 44100 |
| `mp3_bitrate` | 64 / 128 / 192 | 128 |
| `latency` | normal / balanced / low | normal |
| `temperature` | Expression variation 0–1 | 0.7 |
| `top_p` | Nucleus sampling 0–1 | 0.7 |
| `prosody.speed` | Speech speed | 1.0 |
| `chunk_length` | Text chunk size 100–300 | 300 |
| `normalize` | Text normalization | true |

UI presets: High-quality narration, Quick preview, Real-time, Long-text stable.

### DeepSeek Models

- Default: `deepseek-v4-flash`; Studio role-split default: `deepseek-v4-pro`
- OpenAI-compatible `POST /chat/completions` with `response_format: json_object`
- `mode: none` bypasses DeepSeek (passthrough project)

### Troubleshooting

| Issue | Fix |
|-------|-----|
| `FISH_API_KEY is not configured` | Add key to `.env`, restart server |
| `DEEPSEEK_API_KEY is not configured` | Add key to `.env`, restart server |
| `ffmpeg is not installed` | Install ffmpeg; verify `ffmpeg -version` |
| Fish 401/402 | Check key validity and account balance |
| Clone still `training` | Use **Refresh status** on Voices |
| Duplicate role in clone history | Remove extras; same `role_id` now upserts one record |
| Test data reappeared after `pnpm test` | Tests use an isolated temp dir (not project `outputs/`) |

### Security & Privacy

- **Never** commit `.env`, `.env.local`, or real API keys
- `.gitignore` excludes `outputs/`, `uploads/`, `temp/`, and all `.env*` except `.env.example`
- All Fish and DeepSeek traffic goes through the Express backend
- Persistent clones default to `visibility: private`
- Generated audio and registry JSON live under `outputs/` locally — treat as sensitive if your scripts contain private content
- If a key was ever committed or shared, **rotate it** in the Fish / DeepSeek dashboards immediately

### Project Structure

```
voxweaver-studio/
  client/     Vite + React + TypeScript UI (i18n)
  server/     Express + TypeScript API
  shared/     Types, zod schemas, pure logic
  outputs/    Generated audio, projects, voice-registry (gitignored)
  uploads/    Uploaded reference audio (gitignored)
  temp/       Temporary merge/silence files (gitignored)
```

### Tests

```bash
pnpm test
```

Integration tests mock Fish/DeepSeek with nock and write to a **temporary** output directory, not your project `outputs/`.

### License

Local development tool — comply with Fish Audio and DeepSeek terms of service.

---

<a id="中文"></a>

## 中文

VoxWeaver Studio 是本地语音制作 WebUI，整合 **Fish Audio**（`s2.1-pro-free`）与 **DeepSeek**（`deepseek-v4-flash` / `deepseek-v4-pro`），支持 AI 文本优化、多角色分镜、音色克隆、批量 TTS 与 ffmpeg 音频合并。

### 功能

- **中英文界面** — 应用内默认中文；主区域右上角可切换 **中文 / English**
- 单文本工作台（可选 DeepSeek 优化）
- 批量 `.txt` 上传、逐文件 AI 优化与 ZIP 导出
- 多角色工作室：试听 → 锁定角色音色、AI 覆盖复查、预览/终稿队列与合并输出
- 音色绑定：默认、`reference_id`、即时克隆（MessagePack）、持久克隆
- 音色库「角色克隆历史」（保留 Style 与试听文本）
- Fish 参数面板（默认折叠）与情绪标签工具栏
- 音色库增删改查与持久克隆（`train_mode: fast`）
- 历史记录（仅保留最近 **50** 条，详情默认折叠）
- 设置页健康检查为**手动触发**（进入页面不自动探测）
- 音频下载文件名带时间戳（`YYYYMMDD-HHMMSS-文本前20字.mp3`）

### 获取 Fish Audio API Key

1. 注册 [https://fish.audio](https://fish.audio)
2. 在开发者后台创建 API Key
3. 写入 `.env`：`FISH_API_KEY=...`

### 获取 DeepSeek API Key

1. 注册 [https://platform.deepseek.com](https://platform.deepseek.com)
2. 在 API Keys 页面创建密钥
3. 写入 `.env`：`DEEPSEEK_API_KEY=...`

### 环境配置

复制 `.env.example` 为 `.env` 并填入密钥：

```env
FISH_API_KEY=your_fish_audio_key
DEEPSEEK_API_KEY=your_deepseek_key
FISH_BASE_URL=https://api.fish.audio
FISH_TTS_MODEL=s2.1-pro-free
DEEPSEEK_BASE_URL=https://api.deepseek.com
DEEPSEEK_DEFAULT_MODEL=deepseek-v4-flash
OUTPUT_DIR=outputs
UPLOAD_DIR=uploads
TEMP_DIR=temp
MAX_TXT_FILE_SIZE_MB=10
MAX_AUDIO_FILE_SIZE_MB=50
MAX_BATCH_FILES=50
MAX_PARALLEL_TTS=2
MAX_PARALLEL_AI=2
PORT=3001
CLIENT_URL=http://localhost:5173
```

密钥**仅在后端读取**，切勿写入前端代码或提交到 git。

### 安装依赖

需要：Node.js 18+、pnpm、系统 PATH 中的 ffmpeg。

```bash
pnpm install
pnpm -F @voxweaver/shared build
```

### 安装 ffmpeg

- **macOS**：`brew install ffmpeg`
- **Ubuntu/Debian**：`sudo apt install ffmpeg`
- **Windows**：从 [ffmpeg.org](https://ffmpeg.org/download.html) 下载并加入 PATH

验证：`ffmpeg -version`

### 启动前后端

开发模式（热更新）：

```bash
cp .env.example .env   # 编辑填入密钥
pnpm dev
```

- 前端：[http://localhost:5173](http://localhost:5173)
- 后端 API：[http://localhost:3001/api](http://localhost:3001/api)

生产式启动（Express 托管构建后的前端）：

```bash
pnpm build
pnpm start
```

访问 [http://localhost:3001](http://localhost:3001)

### 单文本生成

1. 打开 **工作台**（`/`）
2. 输入文本或上传 `.txt`
3. 选择 AI 模式（可选 **不优化**）
4. **AI 优化**（可选）→ 编辑 TTS 文本、插入情绪标签
5. 配置音色绑定与 Fish 参数
6. **预览 / 生成音频**（下载文件名含时间戳）

### 批量 TXT 生成

1. 打开 **批量生成**（`/batch`）
2. 上传多个 `.txt`（逐文件 AI 优化）
3. 设置全局或单文件音色绑定
4. **批量生成并打包 ZIP**

### 多角色工作室

1. 打开 **多角色工作室**（`/studio`）
2. 粘贴长文本 → **AI 分角色**（默认 `deepseek-v4-pro`）
3. 可选：自动或手动 **AI 复查** 遗漏片段
4. 每角色：**试听** → **重新生成** → **锁定为角色音色**（写入克隆历史）
5. **预览队列** / **终稿队列 + 合并** → 下载合并音频与导出文件

### 音色库

- **角色克隆历史** — Studio 锁定音色后自动记录
- **我的克隆** — Fish 账户音色列表
- 列表分段加载；训练状态需手动 **刷新状态**
- **移除记录** 仅删除本地注册信息，不删除 Fish 云端模型

### reference_id 用法

1. 在 **音色库** 复制音色 `_id`
2. 音色绑定选 `reference_id` 并粘贴

curl 示例（密钥放在服务端环境变量，不要写进前端）：

```bash
curl --request POST \
  --url https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro-free" \
  --data '{"text":"你好","reference_id":"YOUR_VOICE_ID","format":"mp3"}'
```

### 即时克隆

1. 准备参考音频与准确转写
2. 音色绑定 → **即时克隆** → 上传样本
3. 后端通过 MessagePack 调用 Fish（JSON 中不使用 base64）

### 持久克隆

1. **音色库** 上传样本 → **创建克隆**（`train_mode: fast`）
2. 或在 Studio 试听后 **锁定为角色音色**
3. 等待 `state === trained` 后再用于批量/工作室生成

### Fish s2.1-pro-free 主要参数

| 参数 | 说明 | 默认 |
|------|------|------|
| `format` | mp3 / wav / pcm / opus | mp3 |
| `sample_rate` | 采样率 | 44100 |
| `mp3_bitrate` | 64 / 128 / 192 | 128 |
| `latency` | normal / balanced / low | normal |
| `temperature` | 表现力 0–1 | 0.7 |
| `top_p` | 核采样 0–1 | 0.7 |
| `prosody.speed` | 语速 | 1.0 |
| `chunk_length` | 分块长度 100–300 | 300 |
| `normalize` | 文本标准化 | true |

UI 预设：高质量旁白、快速预览、实时、长文本稳定。

### DeepSeek 模型

- 默认 `deepseek-v4-flash`；Studio 分角色默认 `deepseek-v4-pro`
- 兼容 OpenAI 的 `POST /chat/completions`，`response_format: json_object`
- `mode: none` 跳过 DeepSeek，直接透传文本

### 常见问题

| 现象 | 处理 |
|------|------|
| `FISH_API_KEY is not configured` | 在 `.env` 填入密钥并重启后端 |
| `DEEPSEEK_API_KEY is not configured` | 同上 |
| `ffmpeg is not installed` | 安装 ffmpeg 并验证 `ffmpeg -version` |
| Fish 401/402 | 检查密钥与账户余额 |
| 克隆一直 `training` | 在音色库点 **刷新状态** |
| 克隆历史出现重复角色 | 删除多余项；同 `role_id` 现已自动合并为一条 |
| 跑 `pnpm test` 后又出现假数据 | 测试已改用临时目录，不会写入项目 `outputs/` |

### 安全与隐私

- **切勿**提交 `.env`、`.env.local` 或真实 API 密钥
- `.gitignore` 已忽略 `outputs/`、`uploads/`、`temp/` 及除 `.env.example` 外的所有 `.env*`
- Fish / DeepSeek 请求均经 Express 后端转发
- 持久克隆默认 `visibility: private`
- 生成的音频与注册表 JSON 在本地 `outputs/` — 若文稿涉密请妥善保管
- 若密钥曾误提交或外泄，请立即在 Fish / DeepSeek 控制台**轮换密钥**

### 项目结构

```
voxweaver-studio/
  client/     Vite + React + TypeScript 前端（i18n）
  server/     Express + TypeScript API
  shared/     类型、zod 校验、纯逻辑
  outputs/    生成音频、项目、voice-registry（已 gitignore）
  uploads/    上传的参考音频（已 gitignore）
  temp/       临时合并/静音文件（已 gitignore）
```

### 测试

```bash
pnpm test
```

集成测试使用 nock 模拟 Fish/DeepSeek，输出写入**系统临时目录**，不会污染项目 `outputs/`。

### 许可

本地开发工具 — 请遵守 Fish Audio 与 DeepSeek 服务条款。