近日，Bert-vits2-v2.2如约更新，该新版本v2.2主要把Emotion 模型换用CLAP多模态模型，推理支持输入text prompt提示词和audio prompt提示语音来进行引导风格化合成，让推理音色更具情感特色，并且推出了新的预处理webuI，操作上更加亲民和接地气。
更多情报请参见Bert-vits2官网：
https://github.com/fishaudio/Bert-VITS2/releases/tag/v2.2 与此同时，基于FastApi的推理web界面项目也同步适配了Bert-vits2-v2.2版本，官网如下：
https://github.com/jiangyuxiaoxiao/Bert-VITS2-UI 本次我们基于此两个项目来克隆原神角色八重神子的英文语音模型miko。
Bert-vits2-v2.2新的底模和情感模型
首先克隆Bert-vits2-v2.2官方项目：
git clone https://github.com/fishaudio/Bert-VITS2/tree/v2.2 安装依赖：
pip3 install -r requirements.txt 这里注意是v2.2的tag分支，因为官方随时都在更新，主分支可能会存在bug。
进入项目的目录：
cd /Bert-VITS2 随后下载新的底模和情感模型，下载地址：
https://openi.pcl.ac.cn/Stardust_minus/Bert-VITS2/modelmanage/show_model
将新的情感模型clap-hatsat-fused放入到项目的emotional目录，结构如下：
E:workBert-VITS2-v22emotional>tree /f Folder PATH listing for volume myssd Volume serial number is 7CE3-15AE E:. ├───clap-htsat-fused │ .gitattributes │ config.json │ merges.txt │ preprocessor_config.json │ pytorch_model.bin │ README.md │ special_tokens_map.json │ tokenizer.json │ tokenizer_config.json │ vocab.json │ └───wav2vec2-large-robust-12-ft-emotion-msp-dim .gitattributes config.json LICENSE preprocessor_config.json pytorch_model.bin README.md vocab.json 注意，wav2vec2-large-robust-12-ft-emotion-msp-dim是Bert-vits2-v2.1的情感模型，也需要保留，具体请移步：义无反顾马督工,Bert-vits2V210复刻马督工实践(Python3.10)， 这里不再赘述。
至此，新模型就配置好了。
Bert-vits2-v2.2模型训练
首先下载训练集，以原神角色八重神子的英文配音为例子，数据集下载地址：
https://github.com/AI-Hobbyist/Genshin_Datasets 随后新建miko角色目录
mkdir miko 将语音标注文件以esd.list命名，放入miko目录。
同时将分片语音素材放入raw目录。
最后新建miko/configs/config.json配置文件：
{ "train": { "log_interval": 50, "eval_interval": 50, "seed": 42, "epochs": 1000, "learning_rate": 0.0002, "betas": [ 0.8, 0.99 ], "eps": 1e-09, "batch_size": 6, "fp16_run": false, "lr_decay": 0.99995, "segment_size": 16384, "init_lr_ratio": 1, "warmup_epochs": 0, "c_mel": 45, "c_kl": 1.0, "skip_optimizer": false, "freeze_ZH_bert": false, "freeze_JP_bert": false, "freeze_EN_bert": false }, "data": { "training_files": "data/miko/train.list", "validation_files": "data/miko/val.list", "max_wav_value": 32768.0, "sampling_rate": 44100, "filter_length": 2048, "hop_length": 512, "win_length": 2048, "n_mel_channels": 128, "mel_fmin": 0.0, "mel_fmax": null, "add_blank": true, "n_speakers": 1, "cleaned_text": true, "spk2id": { "miko": 0 } }, "model": { "use_spk_conditioned_encoder": true, "use_noise_scaled_mas": true, "use_mel_posterior_encoder": false, "use_duration_discriminator": true, "inter_channels": 192, "hidden_channels": 192, "filter_channels": 768, "n_heads": 2, "n_layers": 6, "kernel_size": 3, "p_dropout": 0.1, "resblock": "1", "resblock_kernel_sizes": [ 3, 7, 11 ], "resblock_dilation_sizes": [ [ 1, 3, 5 ], [ 1, 3, 5 ], [ 1, 3, 5 ] ], "upsample_rates": [ 8, 8, 2, 2, 2 ], "upsample_initial_channel": 512, "upsample_kernel_sizes": [ 16, 16, 8, 2, 2 ], "n_layers_q": 3, "use_spectral_norm": false, "gin_channels": 256 }, "version": "2.2" } 这里注意"version": "2.2"，即版本号为最新的v2.2。
其他参数根据当前的设备环境酌情调整即可。
随后启动预处理页面：
python3 webui_preprocess.py 访问http://127.0.0.1:7860/:
按照页面的步骤进行操作即可，简单且方便。
操作完之后，运行训练命令：
python3 train_ms.py 训练好的模型放在data/miko/models目录，结构如下：
E:workBert-VITS2-v22Datamikomodels>tree /f Folder PATH listing for volume myssd Volume serial number is 7CE3-15AE E:. │ DUR_0.pth │ DUR_100.pth │ DUR_150.pth │ DUR_50.pth │ D_0.pth │ D_100.pth │ D_150.pth │ D_50.pth │ events.out.tfevents.1702457087.ly.13044.0 │ events.out.tfevents.1702458207.ly.12416.0 │ githash │ G_0.pth │ G_100.pth │ G_150.pth │ G_50.pth │ train.log │ └───eval events.out.tfevents.1702457087.ly.13044.1 events.out.tfevents.1702458207.ly.12416.1 至此，训练环节结束。
Bert-vits2-v2.2模型推理
推理我们使用Bert-vits2-UI项目的页面，克隆web项目：
git clone https://github.com/jiangyuxiaoxiao/Bert-VITS2-UI 将Web项目放入Bert-vits2-v2.2的根目录中，目录结构如下：
E:workBert-VITS2-v22_lilithWeb>tree /f Folder PATH listing for volume myssd Volume serial number is 7CE3-15AE E:. │ index.html │ ├───assets │ index-21bc6a28.css │ index-402c0217.js │ └───img helps1.png helps2.png Hiyori.ico 这里包含主页面、样式文件以及JS文件，基于Hiyori。
随后启动推理页面:
python3 server_fastapi.py
加载模型进行推理即可。
此外，还可以基于FastAPI的接口进行推理，换句话说，发送http请求即可获取推理音频，接口参数如下：
{ "openapi": "3.1.0", "info": { "title": "FastAPI", "version": "0.1.0" }, "paths": { "/": { "get": { "summary": "Index", "operationId": "index__get", "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } } } } }, "/voice": { "post": { "summary": "Voice", "description": "语音接口，若需要上传参考音频请仅使用post请求", "operationId": "voice_voice_post", "parameters": [ { "name": "model_id", "in": "query", "required": true, "schema": { "type": "integer", "description": "模型ID", "title": "Model Id" }, "description": "模型ID" }, { "name": "speaker_name", "in": "query", "required": false, "schema": { "type": "string", "description": "说话人名", "title": "Speaker Name" }, "description": "说话人名" }, { "name": "speaker_id", "in": "query", "required": false, "schema": { "type": "integer", "description": "说话人id，与speaker_name二选一", "title": "Speaker Id" }, "description": "说话人id，与speaker_name二选一" }, { "name": "sdp_ratio", "in": "query", "required": false, "schema": { "type": "number", "description": "SDP/DP混合比", "default": 0.2, "title": "Sdp Ratio" }, "description": "SDP/DP混合比" }, { "name": "noise", "in": "query", "required": false, "schema": { "type": "number", "description": "感情", "default": 0.2, "title": "Noise" }, "description": "感情" }, { "name": "noisew", "in": "query", "required": false, "schema": { "type": "number", "description": "音素长度", "default": 0.9, "title": "Noisew" }, "description": "音素长度" }, { "name": "length", "in": "query", "required": false, "schema": { "type": "number", "description": "语速", "default": 1, "title": "Length" }, "description": "语速" }, { "name": "language", "in": "query", "required": false, "schema": { "type": "string", "description": "语言", "title": "Language" }, "description": "语言" }, { "name": "auto_translate", "in": "query", "required": false, "schema": { "type": "boolean", "description": "自动翻译", "default": false, "title": "Auto Translate" }, "description": "自动翻译" }, { "name": "auto_split", "in": "query", "required": false, "schema": { "type": "boolean", "description": "自动切分", "default": false, "title": "Auto Split" }, "description": "自动切分" }, { "name": "emotion", "in": "query", "required": false, "schema": { "anyOf": [ { "type": "integer" }, { "type": "string" }, { "type": "null" } ], "description": "emo", "title": "Emotion" }, "description": "emo" } ], "requestBody": { "required": true, "content": { "multipart/form-data": { "schema": { "$ref": "#/components/schemas/Body_voice_voice_post" } } } }, "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } }, "get": { "summary": "Voice", "description": "语音接口", "operationId": "voice_voice_get", "parameters": [ { "name": "text", "in": "query", "required": true, "schema": { "type": "string", "description": "输入文字", "title": "Text" }, "description": "输入文字" }, { "name": "model_id", "in": "query", "required": true, "schema": { "type": "integer", "description": "模型ID", "title": "Model Id" }, "description": "模型ID" }, { "name": "speaker_name", "in": "query", "required": false, "schema": { "type": "string", "description": "说话人名", "title": "Speaker Name" }, "description": "说话人名" }, { "name": "speaker_id", "in": "query", "required": false, "schema": { "type": "integer", "description": "说话人id，与speaker_name二选一", "title": "Speaker Id" }, "description": "说话人id，与speaker_name二选一" }, { "name": "sdp_ratio", "in": "query", "required": false, "schema": { "type": "number", "description": "SDP/DP混合比", "default": 0.2, "title": "Sdp Ratio" }, "description": "SDP/DP混合比" }, { "name": "noise", "in": "query", "required": false, "schema": { "type": "number", "description": "感情", "default": 0.2, "title": "Noise" }, "description": "感情" }, { "name": "noisew", "in": "query", "required": false, "schema": { "type": "number", "description": "音素长度", "default": 0.9, "title": "Noisew" }, "description": "音素长度" }, { "name": "length", "in": "query", "required": false, "schema": { "type": "number", "description": "语速", "default": 1, "title": "Length" }, "description": "语速" }, { "name": "language", "in": "query", "required": false, "schema": { "type": "string", "description": "语言", "title": "Language" }, "description": "语言" }, { "name": "auto_translate", "in": "query", "required": false, "schema": { "type": "boolean", "description": "自动翻译", "default": false, "title": "Auto Translate" }, "description": "自动翻译" }, { "name": "auto_split", "in": "query", "required": false, "schema": { "type": "boolean", "description": "自动切分", "default": false, "title": "Auto Split" }, "description": "自动切分" }, { "name": "emotion", "in": "query", "required": false, "schema": { "anyOf": [ { "type": "integer" }, { "type": "string" }, { "type": "null" } ], "description": "emo", "title": "Emotion" }, "description": "emo" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/models/info": { "get": { "summary": "Get Loaded Models Info", "description": "获取已加载模型信息", "operationId": "get_loaded_models_info_models_info_get", "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } } } } }, "/models/delete": { "get": { "summary": "Delete Model", "description": "删除指定模型", "operationId": "delete_model_models_delete_get", "parameters": [ { "name": "model_id", "in": "query", "required": true, "schema": { "type": "integer", "description": "删除模型id", "title": "Model Id" }, "description": "删除模型id" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/models/add": { "get": { "summary": "Add Model", "description": "添加指定模型：允许重复添加相同路径模型，且不重复占用内存", "operationId": "add_model_models_add_get", "parameters": [ { "name": "model_path", "in": "query", "required": true, "schema": { "type": "string", "description": "添加模型路径", "title": "Model Path" }, "description": "添加模型路径" }, { "name": "config_path", "in": "query", "required": false, "schema": { "type": "string", "description": "添加模型配置文件路径，不填则使用./config.json或../config.json", "title": "Config Path" }, "description": "添加模型配置文件路径，不填则使用./config.json或../config.json" }, { "name": "device", "in": "query", "required": false, "schema": { "type": "string", "description": "推理使用设备", "default": "cuda", "title": "Device" }, "description": "推理使用设备" }, { "name": "language", "in": "query", "required": false, "schema": { "type": "string", "description": "模型默认语言", "default": "ZH", "title": "Language" }, "description": "模型默认语言" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/models/get_unloaded": { "get": { "summary": "Get Unloaded Models Info", "description": "获取未加载模型", "operationId": "get_unloaded_models_info_models_get_unloaded_get", "parameters": [ { "name": "root_dir", "in": "query", "required": false, "schema": { "type": "string", "description": "搜索根目录", "default": "Data", "title": "Root Dir" }, "description": "搜索根目录" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/models/get_local": { "get": { "summary": "Get Local Models Info", "description": "获取全部本地模型", "operationId": "get_local_models_info_models_get_local_get", "parameters": [ { "name": "root_dir", "in": "query", "required": false, "schema": { "type": "string", "description": "搜索根目录", "default": "Data", "title": "Root Dir" }, "description": "搜索根目录" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/status": { "get": { "summary": "Get Status", "description": "获取电脑运行状态", "operationId": "get_status_status_get", "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } } } } }, "/tools/translate": { "get": { "summary": "Translate", "description": "翻译", "operationId": "translate_tools_translate_get", "parameters": [ { "name": "texts", "in": "query", "required": true, "schema": { "type": "string", "description": "待翻译文本", "title": "Texts" }, "description": "待翻译文本" }, { "name": "to_language", "in": "query", "required": true, "schema": { "type": "string", "description": "翻译目标语言", "title": "To Language" }, "description": "翻译目标语言" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/tools/random_example": { "get": { "summary": "Random Example", "description": "获取一个随机音频+文本，用于对比，音频会从本地目录随机选择。", "operationId": "random_example_tools_random_example_get", "parameters": [ { "name": "language", "in": "query", "required": false, "schema": { "type": "string", "description": "指定语言，未指定则随机返回", "title": "Language" }, "description": "指定语言，未指定则随机返回" }, { "name": "root_dir", "in": "query", "required": false, "schema": { "type": "string", "description": "搜索根目录", "default": "Data", "title": "Root Dir" }, "description": "搜索根目录" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } }, "/tools/get_audio": { "get": { "summary": "Get Audio", "operationId": "get_audio_tools_get_audio_get", "parameters": [ { "name": "path", "in": "query", "required": true, "schema": { "type": "string", "description": "本地音频路径", "title": "Path" }, "description": "本地音频路径" } ], "responses": { "200": { "description": "Successful Response", "content": { "application/json": { "schema": {} } } }, "422": { "description": "Validation Error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/HTTPValidationError" } } } } } } } }, "components": { "schemas": { "Body_voice_voice_post": { "properties": { "text": { "type": "string", "title": "Text" }, "reference_audio": { "type": "string", "format": "binary", "title": "Reference Audio" } }, "type": "object", "required": [ "text" ], "title": "Body_voice_voice_post" }, "HTTPValidationError": { "properties": { "detail": { "items": { "$ref": "#/components/schemas/ValidationError" }, "type": "array", "title": "Detail" } }, "type": "object", "title": "HTTPValidationError" }, "ValidationError": { "properties": { "loc": { "items": { "anyOf": [ { "type": "string" }, { "type": "integer" } ] }, "type": "array", "title": "Location" }, "msg": { "type": "string", "title": "Message" }, "type": { "type": "string", "title": "Error Type" } }, "type": "object", "required": [ "loc", "msg", "type" ], "title": "ValidationError" } } } } 最后奉上Bert-vits2-v2.2本地训练推理整合包：
https://pan.baidu.com/s/1OVX9seRwZR6bZ-xsE_nRLg?pwd=v3uc 与众乡亲同飨。
![洛谷 P11345 [KTSC 2023 R2] 基地简化 题解](http://www.itfaba.com/wp-content/themes/kemi/timthumb.php?src=http://www.itfaba.com/wp-content/themes/kemi/img/random/2.jpg&w=218&h=124&zc=1)
