Performance & Inference Improvements 性能与推理改进

Over the last few releases we reworked the SXSEditor inference stack to run faster on NVIDIA GPUs and to be more robust across hardware. This article summarizes what changed, with a focus on the WindowsML TensorRT-RTX execution provider and the ~4× FP16 inference speedup it delivers on an NVIDIA GeForce RTX 5060 Laptop.

在最近几个版本中，我们重构了 SXSEditor 的推理栈，使其在 NVIDIA GPU 上运行更快、并在不同硬件上更加稳健。本文总结了这些改动，重点介绍 WindowsML TensorRT-RTX 执行提供器，以及它在 NVIDIA GeForce RTX 5060 Laptop 上带来的约 4 倍 FP16 推理加速。

WindowsML TensorRT-RTX: ~4× FP16 speedup WindowsML TensorRT-RTX：约 4 倍 FP16 加速

SXSEditor now ships FP16 ONNX models that can run on NVIDIA GPUs through the Windows ML (WinML) NvTensorRTRTXExecutionProvider (TensorRT via the MSIX WinML runtime, ORT 1.27 + TensorRT-RTX). When the TensorRT-RTX path is available and model files are present, preflow, diff_step and vocoder are loaded on the [winml:NvTensor] backend with DML (and CPU) as automatic fallbacks.

SXSEditor 现在提供 FP16 ONNX 模型，可通过 Windows ML (WinML) 的 NvTensorRTRTXExecutionProvider（基于 MSIX WinML 运行时封装 TensorRT，ORT 1.27 + TensorRT-RTX）在 NVIDIA GPU 上运行。当 TensorRT-RTX 路径可用且模型文件存在时，preflow、diff_step 与 vocoder 会在 [winml:NvTensor] 后端加载，并自动以 DML（以及 CPU）作为回退。

We measured both model-level p50 latency and end-to-end synthesis on an NVIDIA GeForce RTX 5060 Laptop (8 GB), Windows 11 24H2. The TensorRT-RTX FP16 path is roughly 4–5× faster than the DML FP32 baseline on individual models, and approx. 4× faster end-to-end for a single 16-step segment.

我们在 NVIDIA GeForce RTX 5060 Laptop（8 GB）、Windows 11 24H2 上测得了模型级 p50 延迟与端到端合成时间。TensorRT-RTX 的 FP16 路径在单个模型上比 DML FP32 基线快约 4–5 倍，对于单个 16 步乐段，端到端约快 4 倍。

TensorRT-RTX is opt-in: the app exposes a one-click TensorRT-RTX diagnostic and keeps the TRT dynamic profile generation optional. Model files are drop-in compatible with the existing fp16/ folder, so switching between DML and TensorRT-RTX requires no extra downloads.

TensorRT-RTX 为可选开启：应用提供一键 TensorRT-RTX 诊断，并保持 TRT 动态 profile 生成可选。模型文件与现有 fp16/ 目录直接兼容，因此在 DML 与 TensorRT-RTX 之间切换无需额外下载。

Native backend selection 原生后端选择

The old WAML-enabled checkbox has been replaced by a native inference backend selector in Settings. A single execution provider now serves multiple hardware paths — you pick a preferred backend (e.g. TensorRT-RTX, DML, WebNN NPU/GPU or CPU) and SXSEditor falls back automatically when a path is unavailable. This removes a whole class of "model loaded on CPU" surprises and makes hardware switching explicit.

设置中旧的 WinML 开关已被原生推理后端选择器取代。单一执行提供器现可服务多条硬件路径——你选择一个首选后端（如 TensorRT-RTX、DML、WebNN NPU/GPU 或 CPU），当某条路径不可用时 SXSEditor 会自动回退。这消除了一整类"模型意外跑到 CPU 上"的问题，也让硬件切换变得清晰。

Robustness: zero-output protection & D2H fallback 稳健性：全零输出保护与 D2H 回退

• Application-level zero-output protection — vendor execution providers occasionally return all-zero tensors. SXSEditor now detects all-zero outputs and falls back to a safe path instead of silently producing silence.应用级全零输出保护——第三方执行提供器偶尔会返回全零张量。SXSEditor 现在会检测全零输出并回退到安全路径，而不是静默产出静音。
• D2H ArrayBuffer copy fallback — device-to-host copies for vendor EPs now use a robust ArrayBuffer copy path with a d2h-copy-v3 marker in the native bridge, fixing all-zero outputs from vendor EPs and keeping TensorRT-RTX profiles opt-in.D2H ArrayBuffer 复制回退——第三方 EP 的设备到主机复制现改用稳健的 ArrayBuffer 复制路径，并在原生桥接模块中加入 d2h-copy-v3 标记，修复第三方 EP 的全零输出，同时保持 TensorRT-RTX profile 可选。
• Native ORT bridge rebuild — the native addon is rebuilt for Electron on each release and verified at CI time to carry the D2H copy path, so packaged builds always ship the hardened bridge.原生 ORT 桥接模块重建——每次发布时都会为 Electron 重建原生插件，并在 CI 中校验其携带 D2H 复制路径，确保打包版本始终包含加固后的桥接模块。

Latency & streaming improvements 延迟与流式改进

• GPU drain delay removal — the fixed per-chunk GPU drain waits are gone; waits are now applied only after an out-of-memory event, cutting idle time in the diffusion loop.移除 GPU 排空延迟——移除每 chunk 固定的 GPU 排空等待；仅在发生显存不足后才施加等待，减少扩散循环中的空闲时间。
• Streaming chunk-resume cache — long streams resume from cached chunks, so scrubbing back and replaying a section avoids re-running inference.流式 chunk 续播缓存——长流可从已缓存 chunk 续播，回拉播放头重放某段时无需重新推理。
• Vocal region splitting & long-rest fix — non-vocal regions (incl. long rests ≥ 1.5 s) are split out of model input, and streaming synthesis keeps absolute note starts so voices no longer pile up after a long rest.人声区域切分与长休止修复——非人声区域（含 ≥ 1.5 秒的长休止）从模型输入中切分出去，流式合成保持绝对音符起点，长休止后的人声不再重叠堆叠。

How to enable it 如何启用

1. Install the latest v2.0.2 build.
2. 安装最新的 v2.0.2 版本。
3. Open Settings → Backend and select TensorRT-RTX (NVIDIA) after running the built-in diagnostic.
4. 打开 设置 → 后端，运行内置诊断后选择 TensorRT-RTX (NVIDIA)。
5. Confirm the log shows [winml:NvTensor] for preflow / diff_step / vocoder.
6. 确认日志中 preflow / diff_step / vocoder 显示 [winml:NvTensor]。