Inference Pipeline 推理管线

The Singing Voice Synthesis (SVS) pipeline is the heart of SXSEditor. It is implemented in src/inference/pipeline/index.js by the OnnxSVSPipeline class, which loads 9 ONNX models and runs SoulX-Singer diffusion-based synthesis.

歌声合成（SVS）管线是 SXSEditor 的核心。它由 src/inference/pipeline/index.js 中的 OnnxSVSPipeline 类实现，加载 9 个 ONNX 模型并运行基于扩散的 SoulX-Singer 合成。

ℹ️

All pipeline constants live in two files: src/inference/shared/constants.js (shared with the WebNN module) and src/inference/pipeline/constants.js (pipeline-specific). The two inference paths (main-process ONNX Runtime + DirectML, renderer WebNN) must use the same values — that file is the single source of truth.

所有管线常量分布在两个文件：src/inference/shared/constants.js（与 WebNN 模块共享）和 src/inference/pipeline/constants.js（管线专属）。两条推理路径（主进程 ONNX Runtime + DirectML、渲染进程 WebNN）必须使用相同的数值——该文件是唯一来源。

Pipeline Overview 管线总览

Notes + Lyrics + BPM
        │
        ▼
┌─ Stage 1: Text Processing (textProcessing.js) ─────────────┐
│  Chinese → pinyin-pro  │  English → ARPAbet G2P dict        │
│  Japanese → hiragana/katakana map                          │
│  Output: phoneme ID sequences (phone_set.json, 2820 entries)│
└────────────────────────────────────────────────────────────┘
        │
        ▼
┌─ Stage 2: Duration Statistics (durationStats.js) ──────────┐
│  Lazy-loaded en_phoneme_durations.json (~4.1MB)            │
│  Lookup chain: trigram_full → trigram → bigram → unigram    │
│  Falls back to vowel-priority for very short notes         │
└────────────────────────────────────────────────────────────┘
        │
        ▼
┌─ Stage 3: Preprocessing (preprocessing.js) ────────────────┐
│  Build F0 frame sequence, mel2token, note type ids         │
│  Pitch → MIDI → frequency (440 * 2^((midi-69)/12))         │
│  F0 quantization to F0_BIN=361 bins, F0_MIN=32.70Hz        │
└────────────────────────────────────────────────────────────┘
        │
        ▼
┌─ Stage 4: Encoders (parallel) ─────────────────────────────┐
│  note_text_encoder  →  text embedding     (EMBED_DIM=512)   │
│  note_pitch_encoder →  pitch embedding    (EMBED_DIM=512)   │
│  note_type_encoder  →  type embedding     (rest/vocal/slur) │
│  f0_encoder         →  quantized F0 emb                        │
│  preflow (ConvNeXtV2) + cond_emb → condition (COND_DIM=1024)│
└────────────────────────────────────────────────────────────┘
        │
        ▼
┌─ Stage 5: Diffusion (diffusion.js) ────────────────────────┐
│  Iterative denoising, DEFAULT_DIFF_STEPS=32                │
│  CFG: schedule-based (constant/linear/cosine/custom)       │
│  Per-step: cond + uncond branches, flow_pred output        │
│  NPU static shapes: pad to NPU_STATIC_SEQ_LEN=2048         │
└────────────────────────────────────────────────────────────┘
        │
        ▼
┌─ Stage 6: Vocoder / Postprocessing (postprocessing.js) ────┐
│  Mel (MEL_DIM=128) → waveform at SAMPLE_RATE=24000         │
│  Chunked vocoding: VOCODER_CHUNK_FRAMES=1008, overlap=32   │
│  Default vocoder: vocoder_dml.onnx (Vocos, DirectML)       │
│  Optional: SiFiGAN (SIFIGAN_HOP_SIZE=120, 200Hz mel)       │
│  Output validation: reject all-zero / NaN waveforms (OOM)  │
└────────────────────────────────────────────────────────────┘
        │
        ▼
Float32Array PCM audio (24kHz model rate, output upsampled to 48kHz)

Stage 1 — Text Processing 第 1 阶段 — 文本处理

src/inference/pipeline/textProcessing.js converts lyrics into phoneme ID sequences. The TextProcessing class loads phone_set.json (2820 entries) and en_g2p_dict.json (126k English words → ARPAbet) on construction.

src/inference/pipeline/textProcessing.js 将歌词转换为音素 ID 序列。TextProcessing 类在构造时加载 phone_set.json（2820 条目）与 en_g2p_dict.json（12.6 万英文词 → ARPAbet）。

Chinese — uses pinyin-pro to convert characters to pinyin, then maps pinyin syllables to zh_* phoneme IDs.中文— 使用 pinyin-pro 将汉字转换为拼音，再将拼音音节映射到 zh_* 音素 ID。
English — looks up the ARPAbet dictionary; unknown words fall back to a letter-based heuristic.英语— 查询 ARPAbet 词典；未知单词回退到基于字母的启发式规则。
Japanese — hiragana / katakana → phoneme map (built-in JP_HIRAGANA_MAP); a small kanji dictionary is also included. Requires the JP model variants in onnx_models/<precision>/JP/.日语— 平假名 / 片假名 → 音素映射（内置 JP_HIRAGANA_MAP）；还包含一个小型汉字字典。需要 onnx_models/<precision>/JP/ 下的日语模型变体。

Language auto-detection is done by OnnxSVSPipeline.detectJapanese(notes) (checks for jp_* phonemes or hiragana/katakana characters in lyrics).

语言自动检测由 OnnxSVSPipeline.detectJapanese(notes) 完成（检查歌词中是否含 jp_* 音素或平假名/片假名字符）。

Stage 2 — Duration Statistics 第 2 阶段 — 时长统计

durationStats.js lazily loads src/inference/en_phoneme_durations.json (~4.1MB), generated by build_en_phoneme_duration_stats.py from MFA-aligned LJSpeech. The lookup fallback chain (most precise → most robust):

durationStats.js 懒加载 src/inference/en_phoneme_durations.json（约 4.1MB），由 build_en_phoneme_duration_stats.py 从 MFA 对齐的 LJSpeech 生成。查表回退链（精度从高到低）：

trigram_full (prev | curr | next | pos | stress)（前 | 当前 | 后 | 位置 | 重音）
trigram (prev | curr | next)（前 | 当前 | 后）
bigram (prev | curr)（前 | 当前）
unigram (curr) — always hits（当前）— 总能命中

Application policy (in preprocessing.js via _allocateByStats):

应用策略（在 preprocessing.js 的 _allocateByStats 中实现）：

User phonemeAdjustments (manual) always wins.用户 phonemeAdjustments（手动）始终优先。
English long notes use the trigram_full → trigram → bigram → unigram chain.英文长音符使用 trigram_full → trigram → bigram → unigram 链。
Extremely short notes (innerFrames < phonemeCount) keep vowel-priority to prevent phoneme swallowing.极短音符（innerFrames < phonemeCount）保留元音优先策略，避免音素被吞。
Non-English or stats-not-loaded → linear interpolation (unchanged behavior).非英文或统计表未加载 → 线性插值（行为不变）。

💡

The 4.1MB stats file is loaded asynchronously in the Preprocessing constructor — startup is never blocked. If loading fails, the pipeline silently falls back to linear allocation.

4.1MB 统计表在 Preprocessing 构造函数中异步加载——绝不阻塞启动。若加载失败，管线会静默回退到线性分配。

Stage 3 — Preprocessing 第 3 阶段 — 预处理

preprocessing.js builds the frame-level inputs consumed by the encoders:

preprocessing.js 构建编码器消费的帧级输入：

F0 frame sequence — buildF0FrameSequence(notes, bpm, f0Envelope, pitchCurveF0). MIDI note → frequency via midiToFreq(pitch) = 440 * 2^((pitch-69)/12). Frame count = totalSeconds * SAMPLE_RATE / HOP_SIZE. Optional f0Envelope (pitch-bend keyframes) and pitchCurveF0 (per-frame F0 override) are applied.F0 帧序列— buildF0FrameSequence(notes, bpm, f0Envelope, pitchCurveF0)。MIDI 音符 → 频率：midiToFreq(pitch) = 440 * 2^((pitch-69)/12)。帧数 = totalSeconds * SAMPLE_RATE / HOP_SIZE。可选的 f0Envelope（音高弯曲关键帧）与 pitchCurveF0（逐帧 F0 覆盖）会被应用。
F0 quantization — continuous F0 (Hz) → F0_BIN=361 bins, anchored at F0_MIN=32.703Hz, for the f0_encoder.onnx lookup table.F0 量化— 连续 F0（Hz）→ F0_BIN=361 个 bin，锚点 F0_MIN=32.703Hz，供 f0_encoder.onnx 查表。
mel2token — maps each mel frame to the phoneme that owns it, derived from the duration allocation above.mel2token— 将每个 mel 帧映射到其所属音素，由上面的时长分配得到。
Note type ids — rest / vocal / slur classification, fed to note_type_encoder.onnx.音符类型 id— rest / vocal / slur 分类，送入 note_type_encoder.onnx。
Continuation normalization — both lyric editors accept the ASCII hyphen -. On save it is normalized to lyric: '', noteType: 3, isContinuation: true, and isSlur: true. Rendering converts that internal state back to -. Inference must test continuation flags before treating an empty lyric as <SP>.连音规范化——两个歌词编辑器都接受半角连字符 -。保存时规范化为 lyric: ''、noteType: 3、isContinuation: true 和 isSlur: true；渲染时再显示为 -。推理必须先判断连音标记，再把空歌词判断为 <SP>。

Stage 4 — Encoders 第 4 阶段 — 编码器

Five encoder models run in parallel to produce the conditioning tensor cond of dimension COND_DIM=1024:

五个编码器模型并行运行，生成维度为 COND_DIM=1024 的条件张量 cond：

Model模型	Input输入	Output dim输出维度
`note_text_encoder.onnx`	phoneme ids音素 id	EMBED_DIM=512
`note_pitch_encoder.onnx`	MIDI pitch idsMIDI 音高 id	EMBED_DIM=512
`note_type_encoder.onnx`	note type ids (rest/vocal/slur)音符类型 id（休止/人声/连音）	EMBED_DIM=512
`f0_encoder.onnx`	quantized F0 ids量化 F0 id	EMBED_DIM=512
`preflow.onnx`	sum of the four embeddings (ConvNeXtV2 pre-flow)四个嵌入之和（ConvNeXtV2 预流）	EMBED_DIM=512
`cond_emb.onnx`	preflow outputpreflow 输出	COND_DIM=1024

Stage 5 — Diffusion 第 5 阶段 — 扩散

diffusion.js implements the iterative denoising loop using diff_step_dml.onnx. Each step takes xt_input (current noisy mel), t (timestep), cond (conditioning), and xt_mask; it outputs flow_pred used to update xt. The loop is driven through a pluggable sampler interface (see Samplers Module below).

diffusion.js 使用 diff_step_dml.onnx 实现迭代去噪循环。每步接收 xt_input（当前带噪 mel）、t（时间步）、cond（条件）、xt_mask，输出 flow_pred 用于更新 xt。循环通过可插拔的 求解器 接口驱动（见下方求解器模块）。

Default steps: DEFAULT_DIFF_STEPS=32. The CLI synth command uses 4 steps by default for quick smoke tests.默认步数：DEFAULT_DIFF_STEPS=32。CLI synth 命令默认使用 4 步以快速冒烟测试。
Classifier-Free Guidance: CFG_STRENGTH=3.0, CFG_RESCALE=0.75. Each step runs a conditional and an unconditional branch; the combine() callback (shared by both inference paths) merges them into a single velocity vector. A single-pass Welford online variance algorithm replaces the former three-pass CFG combine. CFG strength now supports per-step scheduling (see CFG Schedule below).无分类器引导：CFG_STRENGTH=3.0、CFG_RESCALE=0.75。每步运行条件分支与无条件分支各一次；combine() 回调（两条推理路径共用）将二者合并为单一速度向量。单遍 Welford 在线方差算法替代了之前的三遍 CFG 合并。CFG 强度现支持逐步调度（见下方 CFG 调度）。
Sampler selection: runDiffusionLoop / runDiffusionLoopChunked take a samplerName arg ('euler' | 'heun' | 'extrap' | 'stork2', default 'euler'). The batch path (runBatchDiffusionLoop) keeps the batch=4 optimization for Euler and falls back to sequential single-segment calls for non-Euler samplers.求解器选择：runDiffusionLoop / runDiffusionLoopChunked 接收 samplerName 参数（'euler' | 'heun' | 'extrap' | 'stork2'，默认 'euler'）。批量路径（runBatchDiffusionLoop）对 Euler 保留 batch=4 优化，非 Euler 求解器回退为顺序单段调用。
Tensor lifecycle: input/output tensors are disposed immediately after each step to prevent GPU VRAM accumulation (32 steps × 2 branches × 5 tensors = 320 tensors per synthesis).张量生命周期：每步推理后立即释放输入/输出张量，避免 GPU 显存累积（32 步 × 2 分支 × 5 张量 = 每次合成 320 个张量）。
Cached tensors & shared buffers: _runDiffStepWithCachedTensors reuses cond / mask across steps (they don't change) to avoid 64× redundant FP16 conversion. The sampler is given a pre-allocated buffers object (vBuf / deltaBuf / v1Buf / xPredBuf) that is reused across all steps, eliminating per-step Float32Array allocations.缓存张量与共享缓冲：_runDiffStepWithCachedTensors 跨步复用 cond / mask（它们不变），避免 64 倍冗余的 FP16 转换。求解器获预分配的 buffers 对象（vBuf / deltaBuf / v1Buf / xPredBuf），跨所有步复用，消除每步 Float32Array 分配。
NFE accounting: both paths track totalNFE (number of function evaluations) and log it, making the cost difference between Euler (1 NFE/step), Heun (2 NFE/step), and Extrap/STORK-2 (1 NFE/step) visible.NFE 统计：两条路径均跟踪 totalNFE（模型评估次数）并打印日志，使 Euler（每步 1 次）、Heun（每步 2 次）、Extrap/STORK-2（每步 1 次）的开销差异可见。
NPU static shapes: when modelPrecision='int8-npu', inputs are padded to NPU_STATIC_SEQ_LEN=2048 and the output is sliced back to the real frame count.NPU 静态形状：当 modelPrecision='int8-npu' 时，输入被填充到 NPU_STATIC_SEQ_LEN=2048，输出再切片回真实帧数。
QDIT int8 diff_step: the pipeline now supports QDIT-quantized int8 diff_step models with a new ONNX signature: x (audio latent), diffusion_step (scalar), x_mask (bool mask). Legacy int8 models (without the diffusion_step input) are auto-detected at load time, and a user prompt suggests switching to the QDIT version for better quality.QDIT int8 diff_step：管线现支持 QDIT 量化的 int8 diff_step 模型，采用新的 ONNX 签名：x（音频隐变量）、diffusion_step（标量）、x_mask（布尔掩码）。加载时自动检测旧版 int8 模型（缺少 diffusion_step 输入），并提示用户切换到 QDIT 版本以获得更好质量。

Dynamic Thresholding 动态阈值

Per-frame quickselect percentile clipping (arXiv:2507.08965) is applied after CFG combine. The percentile is configurable (0.9–0.999). This technique clips extreme values in the diffusion output to a percentile band and then linearly remaps the clipped values back, preventing rare outlier frames from dominating the audio. It is integrated into both the DML (nativeSvsPipeline.js) and WebNN diffusion paths.

在 CFG 合并之后，应用逐帧 quickselect 百分位裁剪（arXiv:2507.08965）。百分位可配置（0.9–0.999）。该技术将扩散输出中的极值裁剪到百分位区间内，再将裁剪后的值线性重映射回去，防止罕见的离群帧主导音频。已集成到 DML（nativeSvsPipeline.js）与 WebNN 两条扩散路径中。

CFG Schedule CFG 调度

Classifier-Free Guidance now supports configurable schedule modes: constant, linear, cosine, and custom. This replaces the fixed CFG strength with a per-step schedule, allowing stronger guidance early in the diffusion process and gentler guidance near the end. The three-pass CFG combine (conditional + unconditional + variance) has been replaced by a single-pass Welford online variance algorithm, reducing computation per step.

无分类器引导现支持可配置的调度模式：恒定、线性、余弦与自定义。这用逐步调度替代了固定 CFG 强度，允许在扩散过程初期使用更强引导、末期使用更柔和引导。三遍 CFG 合并（条件 + 无条件 + 方差）已被单遍 Welford 在线方差算法替代，减少了每步计算量。

Samplers Module 求解器模块

src/inference/pipeline/samplers/ contains the pluggable ODE-solver abstraction for the flow-matching diffusion loop. The model outputs a velocity field flow_pred = v(x, t); sampling solves dx/dt = v(x, t) with t going 0 → 1 (equivalent to the paper's reverse integration). The solver only decides when to call diffStep and how to combine predictions into the xt delta; CFG / Rescale / tensor lifecycle stay with the caller so both inference paths share one algorithm.

src/inference/pipeline/samplers/ 包含 flow-matching 扩散循环的可插拔 ODE 求解器抽象。模型输出速度场 flow_pred = v(x, t)；采样即求解 dx/dt = v(x, t)，t 从 0 到 1（等价于论文的反向积分）。求解器只决定何时调用 diffStep与如何组合预测为 xt 增量；CFG / Rescale / 张量生命周期仍由调用方管理，保证两条推理路径共用同一份算法。

Unified interface — every solver implements:

统一接口——每个求解器实现：

async step({ evalDiffStep, combine, step, totalSteps, xtData, buffers }) → { nfe }
//   evalDiffStep(t, xtOverride?) → Promise<{condPred, uncondPred}>
//   combine(condPred, uncondPred) → Float32Array  // writes buffers.vBuf, returns it
//   buffers: { vBuf, deltaBuf, v1Buf, xPredBuf }  // caller-allocated, reused across steps
//   delta is written into buffers.deltaBuf; the caller accumulates it onto xt.data

Solver求解器	File文件	NFE / step每步 NFE	Algorithm算法
`euler` (default)（默认）	`euler.js`	1	First-order explicit, midpoint time `t = (step + 0.5) / totalSteps`. Equivalent to the pre-refactor loop.一阶显式，中点时间 `t = (step + 0.5) / totalSteps`。等价于重构前的循环。
`heun`	`heun.js`	2	RK2 trapezoidal: predict `x_pred = x + v1·dt`, then correct `delta = 0.5·(v1 + v2)·dt`. Final step degrades to Euler to avoid `t > 1`.RK2 梯形：预测 `x_pred = x + v1·dt`，再校正 `delta = 0.5·(v1 + v2)·dt`。末步退化为 Euler 以避免 `t > 1`。
`extrap`	`extrap.js`	1	Velocity-extrapolation heuristic inspired by STORK (ICLR 2026). `v2 = v1 + γ·(v1 − v_prev)` with γ=0.5; `delta = 0.5·dt·(v1 + v2)`. First step and unsafe extrapolation fall back to Euler. Stability guards: velocity-jump ratio > 2, `\|v2\|/\|v1\| > 3`, sign-flip with growing amplitude, and NaN/Inf.受 STORK（ICLR 2026）启发的速度外推启发式。`v2 = v1 + γ·(v1 − v_prev)`，γ=0.5；`delta = 0.5·dt·(v1 + v2)`。首步与不安全外推退化为 Euler。稳定性保护：速度突变比 > 2、`\|v2\|/\|v1\| > 3`、符号翻转且幅度增大、NaN/Inf。
`stork2`	`stork2.js`	1	Paper-faithful STORK-2 (Tan et al., ICLR 2026, arXiv:2505.24210). Runge-Kutta-Gegenbauer 2nd-order recurrence with `s=8` sub-stages and Taylor-expansion virtual NFE. First step bootstraps as Euler. `b(j)` coefficients via closed-form RKG formula. Designed for stiff ODEs (stability region ~2s² = 128×).论文原版 STORK-2（Tan et al., ICLR 2026, arXiv:2505.24210）。Runge-Kutta-Gegenbauer 二阶递推 + `s=8` 个 sub-stage + Taylor 展开 virtual NFE。首步以 Euler 启动。`b(j)` 系数用 RKG 闭式公式。专为刚性 ODE 设计（稳定性域约 2s²=128 倍）。

Registry & factory — samplers/index.js exports SOLVERS (id → {label, labelKey, descKey, create()}), DEFAULT_SOLVER='euler', resolveSamplerName() (validates + normalizes), and createSampler(). LEGACY_ALIASES = { stork: 'extrap' } keeps old user settings working. The dropdown options in src/renderer/exportDialog.js and the settings UI must stay aligned with SOLVERS.

注册表与工厂——samplers/index.js 导出 SOLVERS（id → {label, labelKey, descKey, create()}）、DEFAULT_SOLVER='euler'、resolveSamplerName()（校验 + 归一化）与 createSampler()。LEGACY_ALIASES = { stork: 'extrap' } 兼容旧用户设置。src/renderer/exportDialog.js 与设置 UI 的下拉选项必须与 SOLVERS 保持一致。

⚠️

Chunked inference caveat: extrap and stork2 keep cross-step velocity state (_vPrev / _velPreds). A fresh sampler instance is created per runDiffusionLoop call, so each vocoder chunk starts from Euler-equivalent behavior at its first step. For chunked previews this resets their advantage at every chunk boundary — prefer euler or heun there. reset() is provided for callers that reuse a sampler instance across runs.

分块推理注意事项：extrap 与 stork2 保留跨步速度状态（_vPrev / _velPreds）。每次 runDiffusionLoop 调用都会新建 sampler 实例，因此每个 vocoder 块的首步都相当于 Euler 行为。分块预览时每块边界都会重置其优势——建议分块预览使用 euler 或 heun。跨运行复用 sampler 实例的调用方可调用 reset()。

Stage 6 — Vocoder & Postprocessing 第 6 阶段 — 声码器与后处理

postprocessing.js converts the mel spectrogram (MEL_DIM=128) into a PCM waveform at the model rate SAMPLE_RATE=24000, which is then resampled to the target output rate (default 48 kHz). Because vocoding is the most VRAM-hungry stage, it runs in chunks:

postprocessing.js 将 mel 频谱（MEL_DIM=128）转换为模型速率 SAMPLE_RATE=24000 的 PCM 波形，随后重采样到目标输出速率（默认 48 kHz）。由于声码器是最耗显存的阶段，因此采用分块运行：

Chunk size: VOCODER_CHUNK_FRAMES=1008 mel frames per chunk, VOCODER_OVERLAP_FRAMES=32 frames of cross-fade overlap (user-adjustable 8–96).分块大小：每块 VOCODER_CHUNK_FRAMES=1008 个 mel 帧，VOCODER_OVERLAP_FRAMES=32 帧交叉淡入淡出重叠（用户可调 8–96）。
Smart chunk sizing: the actual chunk size is auto-derived from VRAM budget = (VRAM − resident weights − diff_step activations ~2GB − OS reserve ~1GB) × 0.7 safety factor. Manual override is available in Settings.智能分块：实际分块大小由显存预算自动推导 =（VRAM − 常驻权重 − diff_step 激活 ~2GB − OS 占用 ~1GB）× 0.7 安全系数。可在设置中手动覆盖。
Default vocoder: vocoder_dml.onnx (Vocos, DirectML-optimized).默认声码器：vocoder_dml.onnx（Vocos，DirectML 优化版）。
Optional SiFiGAN: sifigan_vocoder_dml_fp16.onnx or sifigan_vocoder_dml.onnx (ICASSP 2023). Uses SIFIGAN_HOP_SIZE=120 (200 Hz mel frame rate, 4× upsampling to 24 kHz). Auto-falls back to the default vocoder on load failure.可选 SiFiGAN：sifigan_vocoder_dml_fp16.onnx 或 sifigan_vocoder_dml.onnx（ICASSP 2023）。使用 SIFIGAN_HOP_SIZE=120（200Hz mel 帧率，4× 上采样到 24kHz）。加载失败时自动回退到默认声码器。
Output validation: validateVocoderOutput samples the waveform and throws a clear OOM error if it's all-zero or NaN — DirectML can fail silently on VRAM exhaustion.输出校验：validateVocoderOutput 对波形抽样，若全零或含 NaN 则抛出明确的 OOM 错误——DirectML 在显存耗尽时可能静默失败。
Minimal swap: switching vocoder (default ↔ SiFiGAN) only reloads the vocoder session; encoders/diffusion stay loaded (swapVocoder).最小化切换：切换声码器（默认 ↔ SiFiGAN）仅重载声码器会话，编码器/扩散保持已加载状态（swapVocoder）。
2× oversampling anti-aliasing: replaces the previous 1st-order Butterworth low-pass with a full 2× oversampling pipeline: zero-stuff → 2nd-order Butterworth anti-image LP → anti-alias LP → decimate. This improves audio quality during resampling from the 24 kHz model rate to the target output rate.2× 过采样抗混叠：用完整的 2× 过采样管线替代之前的一阶 Butterworth 低通：零填充 → 二阶 Butterworth 抗镜像低通 → 抗混叠低通 → 抽取。在从 24kHz 模型速率重采样到目标输出速率时提升音质。
WSOLA crossfade: replaces Hann overlap-add with WSOLA (Waveform Similarity Overlap-Add) crossfade at vocoder and diffusion chunk boundaries, producing smoother audio transitions.WSOLA 交叉淡入淡出：在声码器与扩散分块边界用 WSOLA（波形相似性重叠相加）交叉淡入淡出替代 Hann 重叠相加，产生更平滑的音频过渡。
EBU R128 loudness normalization: applies EBU R128 loudnorm (−14 LUFS) and a true-peak limiter (−1 dBTP) to the synthesis output, ensuring consistent loudness across different projects and singers.EBU R128 响度归一化：对合成输出应用 EBU R128 响度归一化（−14 LUFS）与真峰值限制器（−1 dBTP），确保不同项目与歌手之间的响度一致。

Key Constants 关键常量

Constant常量	Value值	Defined in定义于	Meaning含义
`SAMPLE_RATE`	24000 (model rate)（模型速率）	shared/constants.js	Model internal sample rate (Hz). Default output/playback rate is now 48000 Hz.模型内部采样率（Hz）。默认输出/播放采样率现为 48000 Hz。
`HOP_SIZE`	480	shared/constants.js	Mel frame hop (50 Hz frame rate)Mel 帧步长（50Hz 帧率）
`SIFIGAN_HOP_SIZE`	120	shared/constants.js	SiFiGAN mel hop (200 Hz)SiFiGAN mel 步长（200Hz）
`MEL_DIM`	128	shared/constants.js	Mel spectrogram dimensionMel 频谱维度
`EMBED_DIM`	512	shared/constants.js	Encoder embedding dimension编码器嵌入维度
`COND_DIM`	1024	shared/constants.js	Diffusion conditioning dimension扩散条件维度
`VOCODER_CHUNK_FRAMES`	1008	shared/constants.js	Default vocoder chunk size默认声码器分块大小
`VOCODER_OVERLAP_FRAMES`	32 (user-adjustable 8–96)（用户可调 8–96）	shared/constants.js	Cross-fade overlap frames (was 8)交叉淡入淡出重叠帧数（原为 8）
`NPU_STATIC_SEQ_LEN`	2048	shared/constants.js	NPU encoder/diffusion fixed seq lenNPU 编码器/扩散固定序列长度
`NPU_VOCODER_SEQ_LEN`	500	shared/constants.js	NPU vocoder fixed seq len (WebNN 2GB pad limit)NPU 声码器固定序列长度（受 WebNN 2GB pad 限制）
`N_FFT`	1920	pipeline/constants.js	FFT size for mel transformmel 变换的 FFT 大小
`NUM_MELS`	128	pipeline/constants.js	Mel filter banksmel 滤波器组数
`F0_BIN` / `F0_MIN`	361 / 32.703	pipeline/constants.js	F0 quantization bins / min Hz (C1)F0 量化 bin 数 / 最低 Hz（C1）
`CFG_STRENGTH` / `CFG_RESCALE`	3.0 / 0.75	pipeline/constants.js	Classifier-Free Guidance params无分类器引导参数
`DEFAULT_DIFF_STEPS`	32	pipeline/constants.js	Default diffusion denoising steps默认扩散去噪步数
`LONG_AUDIO_THRESHOLD_SEC`	30	pipeline/constants.js	Audio longer than this triggers segmentation超过该时长触发分段合成
`SEGMENT_MIN_SEC` / `SEGMENT_MAX_SEC` / `SEGMENT_OVERLAP_SEC`	15 / 30 / 2	pipeline/constants.js	Segment size range and overlap分段大小范围与重叠
`MAX_SAFE_FRAMES`	40000	pipeline/constants.js	Hard cap on mel frames per synthesis每次合成 mel 帧数硬上限
`SILENCE_THRESHOLD_SEC`	1.5	pipeline/constants.js	Silence detection threshold (was 20s/8s/20s multi-tier)静音检测阈值（原为 20s/8s/20s 多级）
`LONG_REST_SPLIT_SEC`	1.5	pipeline/constants.js	Rest notes ≥ this duration trigger segment splitting休止音符达到此时长即触发分段切分
`DEFAULT_OUTPUT_SAMPLE_RATE`	48000	shared/constants.js	Default audio output / playback sample rate (Hz). Export rate is user-selectable: 24 / 44.1 / 48 / 96 kHz.默认音频输出/播放采样率（Hz）。导出速率可选：24 / 44.1 / 48 / 96 kHz。

Audio Segmentation 音频分段

audioSegmentation.js splits long vocals (> LONG_AUDIO_THRESHOLD_SEC=30s) into segments of SEGMENT_MIN_SEC=15s–SEGMENT_MAX_SEC=30s with SEGMENT_OVERLAP_SEC=2s of overlap. Segment boundaries are chosen at rest-note midpoints to minimize stitching artifacts.

audioSegmentation.js 将长人声（> LONG_AUDIO_THRESHOLD_SEC=30s）切分为 SEGMENT_MIN_SEC=15s–SEGMENT_MAX_SEC=30s 的分段，重叠 SEGMENT_OVERLAP_SEC=2s。分段边界选择在休止音符中点，以最小化拼接瑕疵。

fillNoteGaps also inserts rest notes between non-adjacent notes so the diffusion model has silent reference frames.

fillNoteGaps 还会在不相邻的音符之间插入休止符，让扩散模型拥有静音参考帧。

The segment silence detection threshold has been unified to SILENCE_THRESHOLD_SEC=1.5s (previously a multi-tier 20s/8s/20s scheme). Rest notes ≥ LONG_REST_SPLIT_SEC=1.5s now trigger segment splitting, ensuring diffusion chunks are bounded by natural pauses.

分段静音检测阈值已统一为 SILENCE_THRESHOLD_SEC=1.5s（此前为 20s/8s/20s 多级方案）。休止音符 ≥ LONG_REST_SPLIT_SEC=1.5s 时触发分段切分，确保扩散分块以自然停顿为边界。

Float16 Patch Float16 补丁

float16Patch.js is loaded as a side-effect at the top of pipeline/index.js. It patches onnxruntime-common's NUMERIC_TENSOR_TYPE_TO_TYPEDARRAY_MAP so that float16 tensors use Uint16Array instead of the native Float16Array.

float16Patch.js 作为副作用在 pipeline/index.js 顶部加载。它修补 onnxruntime-common 的 NUMERIC_TENSOR_TYPE_TO_TYPEDARRAY_MAP，使 float16 张量使用 Uint16Array 而非原生 Float16Array。

⚠️

Node.js v24+ / Electron 42+ (Chromium 138) natively supports Float16Array, but the onnxruntime-node C++ binding cannot read its buffer, causing "not enough space: expected N, got 0". The patch is required for FP16 models to work. Under webpack, it uses __non_webpack_require__ to reach the native require.cache.

Node.js v24+ / Electron 42+（Chromium 138）原生支持 Float16Array，但 onnxruntime-node 的 C++ binding 无法读取其 buffer，会报 "not enough space: expected N, got 0"。该补丁是 FP16 模型正常工作的前提。在 webpack 下，它通过 __non_webpack_require__ 访问原生 require.cache。

Model Loading & Execution Providers 模型加载与执行提供者

modelLoader.js handles device enumeration, execution-provider selection, and session creation with validation:

modelLoader.js 负责设备枚举、执行提供者选择，以及带校验的会话创建：

Device enumeration: enumerateDMLDevices first tries systeminformation (cached); if it returns nothing, it falls back to creating a probe DML session on note_text_encoder.onnx and parsing ORT verbose stderr for Discovered OrtHardwareDevice lines.设备枚举：enumerateDMLDevices 优先使用 systeminformation（带缓存）；若无结果，回退到在 note_text_encoder.onnx 上创建探测 DML 会话，并解析 ORT verbose stderr 中的 Discovered OrtHardwareDevice 行。
Smart device selection: priority discrete-gpu > npu > integrated-gpu > cpu, weighted by VRAM. buildModelDeviceMapping places large model groups (>100MB) on GPU, small groups (<10MB) on NPU when available.智能设备选择：优先级 独显 > NPU > 核显 > CPU，并按显存加权。buildModelDeviceMapping 将大模型组（>100MB）放在 GPU，小模型组（<10MB）在 NPU 可用时放在 NPU。
Session creation: createSessionWithValidation tries DML first (with enableMemPattern: false + executionMode: 'sequential' to avoid DML over-allocation), runs a dummy inference to verify, and falls back to CPU on failure.会话创建：createSessionWithValidation 优先尝试 DML（设置 enableMemPattern: false + executionMode: 'sequential' 以避免 DML 过度分配），运行 dummy 推理校验，失败则回退到 CPU。
WebNN proxy: when inferenceProvider='ortweb' and NPU/GPU is available, a WebNNSessionProxy forwards session.run() calls to the renderer process via IPC (the main window renderer hosts onnxruntime-web).WebNN 代理：当 inferenceProvider='ortweb' 且 NPU/GPU 可用时，WebNNSessionProxy 通过 IPC 将 session.run() 调用转发到渲染进程（主窗口渲染进程承载 onnxruntime-web）。
W16A32 fallback: if a FP16 diff_step fails to load (e.g. partial FP16 support), the pipeline reloads it from the FP32 base directory while keeping other models in FP16.W16A32 回退：若 FP16 的 diff_step 加载失败（如 FP16 支持不完整），管线会从 FP32 基础目录重新加载它，其余模型保持 FP16。

Synthesis Cache 合成缓存

The pipeline keeps a segment-level LRU cache of recent synthesis results (_synthCacheMaxEntries=32, _synthCacheMaxBytes=300MB). Cache keys are derived from an FNV-1a hash of segment content (notes, bpm, and option fields). Switching vocoder type, SiFiGAN precision, or language invalidates the cache. During iterative editing, unchanged segments return cached audio without re-running the vocoder — only modified segments are re-synthesized.

管线维护一个分段级 LRU 合成结果缓存（_synthCacheMaxEntries=32、_synthCacheMaxBytes=300MB）。缓存键由分段内容（音符、bpm 与选项字段）的 FNV-1a 哈希派生。切换声码器类型、SiFiGAN 精度或语言会使缓存失效。迭代编辑时，未修改的分段直接返回缓存音频，不再运行声码器——仅修改过的分段会被重新合成。

ℹ️

Concurrent synthesize() calls are serialized via _synthPromise. This prevents a reload race where synthesis B starts while synthesis A is still rebuilding heavy sessions (diff_step 846MB / vocoder), which would double VRAM and trigger OOM.

并发的 synthesize() 调用通过 _synthPromise 串行化。这避免了一种重载竞态：合成 A 仍在重建重量级会话（diff_step 846MB / vocoder）时合成 B 启动，导致显存翻倍并触发 OOM。

Cooperative Synthesis Cancellation 协作式合成取消

Synthesis cancellation now uses AbortController-based cooperative cancellation instead of worker.terminate(). The GPU inference loop checks the abort signal at safe checkpoints between diffusion steps and exits cleanly, releasing tensor resources in an orderly fashion. The cancel command bypasses the serial worker queue for immediate response — the user does not have to wait for the current step to finish before cancellation takes effect.

合成取消现使用基于 AbortController 的协作式取消，而非 worker.terminate()。GPU 推理循环在扩散步之间的安全检查点检查中止信号并干净退出，有序释放张量资源。取消命令绕过串行 worker 队列以实现即时响应——用户无需等待当前步骤完成即可生效。

Pipeline API Surface 管线 API 表面

const pipeline = new OnnxSVSPipeline(modelDir, {
  modelPrecision,       // 'fp32' | 'fp16' | 'int8' | 'int8-npu'
  deviceId,             // DML adapter index (undefined = auto)
  preferredDeviceType,  // 'npu' | 'webnn-gpu' | undefined
  inferenceProvider,    // 'ortnode' (default) | 'ortweb'
  languageOverride,     // 'ja' | null
});

await pipeline.init();                 // loads all 9 sessions
await pipeline.swapLanguageModels('ja');   // hot-swap JP models
await pipeline.swapVocoder('sifigan');     // hot-swap vocoder only
await pipeline.swapSifiganPrecision('fp16');

const audio = await pipeline.synthesize(notes, bpm, {
  nSteps,               // default 32
  cfg, cfgRescale,      // default 3.0 / 0.75
  cfgSchedule,          // 'constant' | 'linear' | 'cosine' | 'custom'
  sampler,              // 'euler' (default) | 'heun' | 'extrap' | 'stork2'
  autoShift, pitchShift,
  f0Envelope, pitchCurveF0,
  refAudioWavBuffer,    // reference audio for prompt mel + autoShift
  outputSampleRate,     // 24000 | 44100 | 48000 (default) | 96000
  signal,               // AbortSignal for cooperative cancellation
  onProgress,           // 0..100
  onChunkAudio,         // streaming vocoder chunks
});

pipeline.dispose();                   // release all sessions