更新记录

1.5.4(2026-06-05)

  • 鸿蒙端离线识别模式增加punctuationModel标点恢复模型配置

1.5.3(2026-06-03)

  • 安卓端离线识别模式增加punctuationModel标点恢复模型配置

1.5.2(2026-06-03)

  • 细节优化
查看更多

平台兼容性

uni-app(4.87)

Vue2 Vue3 Chrome Safari app-vue app-nvue Android iOS iOS插件版本 鸿蒙 鸿蒙插件版本
- - - - - - 5.0 13 1.0.9 19 1.0.4
微信小程序 支付宝小程序 抖音小程序 百度小程序 快手小程序 京东小程序 鸿蒙元服务 QQ小程序 飞书小程序 小红书小程序 快应用-华为 快应用-联盟
- - - - - - - - - - - -

uni-app x(4.87)

Chrome Safari Android iOS iOS插件版本 鸿蒙 鸿蒙插件版本 微信小程序
- - 5.0 13 1.0.9 19 1.0.4 -

xwq-sherpa-onnx

开发文档说明

插件功能 (兼容安卓、鸿蒙端、iOS)

  • 实时语音识别
  • 离线语音识别(wav音频文件转文本识别)
  • 支持热词检测
  • 支持VAD检测
  • 支持sense_voice
  • 支持拼音词组替换fst
  • 支持关键字配置

ASR模型

  • ASR模型下载地址ASR模型下载
  • Transducer模型中英双语模型可以下载 sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20
  • Paraformer模型双语模型可以下载(推荐)https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2

KWS关键词模型(唤醒词场景专用)

  • KWS 模型下载地址KWS模型下载
  • 推荐使用 sherpa-onnx-kws-zipformer-wenetspeech-3.3M 模型
  • keyword 文件格式为:关键词 阈值,阈值范围 0-1,越小越灵敏

鸿蒙端使用特别说明

 * 鸿蒙端使用sherpa_onnx语音识别特别说明: 
 * 将路径:unpackage\dist\dev\app-harmony\entry\build-profile.json5文件拷贝到根目录harmony-configs/entry下(目录没有的需要手动创建)
 * 
 * 在harmony-configs/entry/build-profile.json5中配置参数字段:buildOption下面新增以下内容
 * "sourceOption": {
 *     "workers": [
 *      '../uni_modules/xwq-sherpa-onnx/utssdk/app-harmony/onnxWorkers.ets'
 *    ]
 *  }

 模型资源需要放在插件目录resources/rawfile/目录下
  • 初始化模型参数说明
属性 类型 默认值 必填 描述
model string - N 模型路径,单模型的情况使用
encoder string - N 编码器模型路径,多模型的情况使用,例如:transducer模型
decoder string - N 解码器模型路径,多模型的情况使用,例如:transducer模型
joiner string - N joiner模型路径,多模型的情况使用,例如:transducer模型
tokens string - N tokens文件路径,一般在多模型文件下才会存在,例如:transducer模型
isCopyFile boolean - N 模型文件的拷贝,默认不拷贝,由页面传入可访问的地址
numThreads number 1 N 执行线程数
hotwords string - N 热词内容文件路径(仅transducer模型支持)
bpeVocab string - N 热词解码文件路径(和热词文件配合使用)
gtcm string - N gtcm模型路径,用于处理麦克风降噪、回声(仅支持安卓/ios)
onlyOutputCurrently boolean false N 是否只输出当前说的内容,不包含历史
fst string - N 拼音词组必要文件路径(拼音词组只支持中文替换)
lexicon string - N 汉字转拼音的通用词典文件路径
keyword string - N 关键词文件路径,部分模型不支持
keywordsText string - N 关键词字符串内容;内容格式需与 keywords 文件原文完全一致,多条关键词使用换行分隔。当前 Android / iOS / Harmony 均支持;
vad string - N vad模型文件路径
punctuationModel string - N 标点恢复模型文件路径
success ()=>void - Y 初始化模型成功回调
fail ()=>void - Y 初始化模型失败回调
  • 初始化Onnx失败回调函数fail参数说明
errCode msg
-1 缺少模型文件地址
-2 实例已存在,请勿重复初始化
-3 模型文件不存在
-4 ASR 初始化失败
  • hotwords.txt热词文件内容如下设置
    
    打开豆包
    语音识别
    打开灯光
    关闭灯光
    关闭空调暖气
    频繁
    礼拜二

#### 实时语音识别使用步骤

* `1.初始化模型识别器`

```html
uesSherpaOnnx({
    mode: 'online',
    modelType: "transducer",
    model: '',
    tokens: staticPath + "tokens.txt",
    encoder: staticPath + "encoder-epoch-99-avg-1.int8.onnx",
    decoder: staticPath + "decoder-epoch-99-avg-1.onnx",
    joiner: staticPath + "joiner-epoch-99-avg-1.int8.onnx",
    success: () => {
        console.log('初始化成功')
        uni.hideLoading()
    },
    fail: (res) => {
        console.log(res)
    }
})
  • 2.设置识别结果回调监听函数
setListner() {
    setRecognizerResultListner((res, final) => {
        console.log('识别结果====', res)
        console.log('final====', final)
        this.content = res
    })
}
  • 3.开始语音识别
start() {
    startRecognizer()
}

离线语音识别(识别wav音频文件)使用步骤

  • 1.初始化模型识别器
initOfflineOnnx() {
    uni.showLoading({
        title: '模型加载中...',
    })
    let path = '/static/offlineOnnxModel';
    const staticPath = plus.io.convertLocalFileSystemURL(path);
    console.log('staticPath===', staticPath)
    uesSherpaOnnx({
        mode: 'offline',
        modelType: "zipformer2ctc",
        model: '',
        tokens: staticPath+'/' + "tokens.txt",
        encoder: staticPath+'/' + "encoder-epoch-34-avg-19.int8.onnx",
        decoder: staticPath+'/' + "decoder-epoch-34-avg-19.onnx",
        joiner: staticPath+'/' + "joiner-epoch-34-avg-19.int8.onnx",
        success: () => {
            console.log('初始化成功')
            uni.hideLoading()
        },
        fail: (res) => {
            console.log(res)
        }
    })
}
  • 2.开始识别wav音频文件(需要单声道、16kHz、16位 PCM 格式)
recognizerFile() {
    let path = '/static/offlineOnnxModel/';
    const filePath = plus.io.convertLocalFileSystemURL(path) + '1.wav';
    startOfflineRecognizerFile(filePath, (result) => {
        console.log('识别结果===', result)
        this.content = result
    })
}
  • 在不使用的时候可以停止识别
stopRecognizer()  //停止实时语音识别
stopOfflineRecognizerFile() //停止音频文件识别

destroyRecognizer()  //页面销毁时调用此方法销毁实例,释放内存占用

KWS唤醒词场景说明

  • keyword关键字说明:推荐直接使用下载模型里的keywords文件内容。若手动编写,请按 sherpa-onnx 的拼音 token 粒度书写,不是简单的单字符拆分;例如 zh、ch、sh 这类整体声母保持原样。
  • 格式说明:
  • 1. 每条关键词通常写成:拼音序列 + 空格 + @显示文本
  • 2. 多条关键词通常使用换行分隔
  • 3. 例如:y ǎn y uán @演员、zh ī m íng @知名,这里的 zh 不是写错,它本身就是整体声母 token
  • 4. 可以直接传关键词文件路径 keyword,也可以直接传关键词字符串内容 keywordsText;keywordsText 的内容格式必须与 keywords 文件原文完全一致
    n ǐ h ǎo j ūn g ē @你好军哥
    d àn g ē d àn g ē @蛋哥蛋哥
    x iǎo ài t óng x ué @小爱同学
    n ǐ h ǎo w èn w èn @你好问问
    x iǎo y ì x iǎo y ì @小艺小艺
    x iǎo m ǐ x iǎo m ǐ @小米小米
    l ín m ěi l ì @林美丽
    n ǐ h ǎo x ī x ī @你好西西
    x iǎo áo t óng x ué @小鳌同学
    n ǐ h ǎo x iǎo áo @你好小鳌
    x iǎo áo x iǎo áo @小鳌小鳌
  • 也支持直接传关键词字符串,字符串内容格式与 keywords 文件内容一致,多条关键词使用换行分隔
uesSherpaOnnx({
    mode: 'online',
    modelType: 'transducer',
    encoder: staticPath + 'encoder.onnx',
    decoder: staticPath + 'decoder.onnx',
    joiner: staticPath + 'joiner.onnx',
    tokens: staticPath + 'tokens.txt',
    keywordsText: 'y ǎn y uán @演员\nzh ī m íng @知名',
    success: () => {},
    fail: (err) => {}
})
  • 具体用法如下

<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
    </view>
</template>

<script>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner
    } from "@/uni_modules/xwq-sherpa-onnx";
    export default {
        data() {
            return {
                content: ""
            }
        },
        methods: {
            // 在线识别
            initOnnx() {
                let path = '/static/onlineOnnxModel/kws/';
                const staticPath = plus.io.convertLocalFileSystemURL(path);

                uesSherpaOnnx({
                    mode: 'online',
                    modelType:"zipformerCCC",
                    encoder:  staticPath +"encoder-epoch-12-avg-2-chunk-16-left-64.onnx",
                    decoder:  staticPath +"decoder-epoch-12-avg-2-chunk-16-left-64.onnx",
                    joiner:  staticPath +"joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx",
                    tokens:  staticPath +"tokens.txt",
                    keyword:staticPath + "keywords.txt",
                    success: () => {
                        console.log('***初始化成功****')
                    },
                    fail: (res) => {
                        console.log(res)
                    }
                })
            },
            //开始识别
            start() {
                startRecognizer()
            },
            //停止识别
            stop() {
                stopRecognizer()
            },
            //设置监听
            setListner() {
                setRecognizerResultListner((res, final) => {
                    console.log('识别结果====', res)
                    this.content = res
                })
            },
            //移除监听
            removeRecognizerResultListner() {
                removeOnRecognizerResultListner()
            }

        }
    }
</script>

<style>
    .title {
        margin: 20px 0 10px 0;
    }

    .content {
        height: 100%;
        border: 1px solid #ccc;
        background-color: #f2f2f2;
    }
</style>

uniappX项目中完整示例

<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
        <button @click="destroy">销毁实例</button>
        <button @click="initOfflineOnnx">初始化离线语音识别器</button>
        <button @click="recognizerFile">开始识别WAV File音频文件</button>
        <button @click="stopRecognizerFile">停止识别WAV File音频文件</button>
        <view style="padding:10px;">
            <view class="title">
                <text>识别结果:</text>
            </view>
            <view class="content">
                <textarea :value="content" disabled style="padding:15px;background-color: #f2f2f2;width: 100%;"></textarea>
                <!-- <text v-for="(i,k) in resultArr" :key="k"></text> -->
            </view>
        </view>
    </view>
</template>

<script setup>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner,
        startOfflineRecognizerFile,
        stopOfflineRecognizerFile,
        destroyRecognizer
    } from "@/uni_modules/xwq-sherpa-onnx";
    import { UseSherpaOnnxOptions } from "@/uni_modules/xwq-sherpa-onnx/utssdk/interface.uts";

    const content=ref('');

    // 在线识别
    const initOnnx=()=> {
        uni.showLoading({
            title: '模型加载中...',
        })

        uesSherpaOnnx({
            mode: 'online', //识别方式
            modelType: "transducer",//模型类型,支持'paraformer' | 'transducer' | 'zipformer'
            model: '',
            tokens: "/static/onlineOnnxModel/tokens.txt",
            encoder: "/static/onlineOnnxModel/encoder-epoch-99-avg-1.int8.onnx",
            decoder: "/static/onlineOnnxModel/decoder-epoch-99-avg-1.onnx",
            joiner: "/static/onlineOnnxModel/joiner-epoch-99-avg-1.int8.onnx",
            hotwords:'hotwords.txt', //热词内容文件
            bpeVocab:'bpe.vocab', //热词解码文件(和热词配合使用)
            success: () => {
                console.log('初始化成功')
                uni.hideLoading()
            },
            fail: (res) => {
                console.log(res)
            }
        } as UseSherpaOnnxOptions)
    };
    //离线识别
    const initOfflineOnnx=()=> {
        uni.showLoading({
            title: '模型加载中...',
        })
        uesSherpaOnnx({
            mode: 'offline',
            modelType: "zipformer2ctc",
            model: '',
            tokens: "/static/offlineOnnxModel/tokens.txt",
            encoder: "/static/offlineOnnxModel/encoder-epoch-34-avg-19.int8.onnx",
            decoder: "/static/offlineOnnxModel/decoder-epoch-34-avg-19.onnx",
            joiner: "/static/offlineOnnxModel/joiner-epoch-34-avg-19.int8.onnx",
            success: () => {
                console.log('初始化成功')
                uni.hideLoading()
            },
            fail: (res) => {
                console.log(res)
            }
        } as UseSherpaOnnxOptions)
    };
    //开始识别
    const start=()=> {
        startRecognizer()
    };
    //停止识别
    const stop=()=> {
        stopRecognizer()
    };
    //在不使用时或页面销毁时需调用此方法销毁实例
    const destroy=()=>{
        destroyRecognizer()
    };
    //设置监听
    const setListner=()=> {
        setRecognizerResultListner((res:string, final:boolean) => {
            console.log('识别结果====', res)
            console.log('final====', final)
            content.value = res
        })
    };
    //移除监听
    const removeRecognizerResultListner=()=> {
        removeOnRecognizerResultListner()
    };

    //离线识别file(wav格式音频)
    const recognizerFile=()=> {
        const filePath = '/static/offlineOnnxModel/1.wav';
        startOfflineRecognizerFile(filePath, (result) => {
            console.log('识别结果===', result)
            content.value = result
        })

    };

    //停止离线识别wav file
    const stopRecognizerFile=()=> {
        stopOfflineRecognizerFile()
    };

</script>

<style>

</style>

uniapp项目中完整示例

<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
        <button @click="destroy">销毁实例</button>
        <button @click="initOfflineOnnx">初始化离线语音识别器</button>
        <button @click="recognizerFile">开始识别WAV File音频文件</button>
        <button @click="stopRecognizerFile">停止识别WAV File音频文件</button>
        <view style="padding:10px;">
            <view class="title">
                <text>识别结果:</text>
            </view>
            <view class="content">
                <textarea :value="content" disabled style="padding:15px;"></textarea>
                <!-- <text v-for="(i,k) in resultArr" :key="k"></text> -->
            </view>
        </view>
    </view>
</template>

<script>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner,
        startOfflineRecognizerFile,
        stopOfflineRecognizerFile,
        destroyRecognizer
    } from "@/uni_modules/xwq-sherpa-onnx";
    export default {
        data() {
            return {
                resultArr: [],
                content: ""
            }
        },

        mounted() {},
        methods: {
            // 在线识别
            initOnnx() {
                uni.showLoading({
                    title: '模型加载中...',
                })
                let path = '/static/onlineOnnxModel';
                const staticPath = plus.io.convertLocalFileSystemURL(path);
                console.log('staticPath===', staticPath)

                uesSherpaOnnx({
                    mode: 'online',//识别方式
                    modelType: "transducer",//模型类型,支持'paraformer' | 'transducer' | 'zipformer'
                    model: '',
                    tokens: staticPath+'/' + "tokens.txt",
                    encoder: staticPath+'/' + "encoder-epoch-99-avg-1.int8.onnx",
                    decoder: staticPath+'/' + "decoder-epoch-99-avg-1.onnx",
                    joiner: staticPath+'/' + "joiner-epoch-99-avg-1.int8.onnx",
                    hotwords:'hotwords.txt', //热词内容文件
                    bpeVocab:'bpe.vocab', //热词解码文件(和热词配合使用)
                    success: () => {
                        console.log('初始化成功')
                        uni.hideLoading()
                    },
                    fail: (res) => {
                        console.log(res)
                    }
                })
            },
            //离线识别
            initOfflineOnnx() {
                uni.showLoading({
                    title: '模型加载中...',
                })
                let path = '/static/offlineOnnxModel';
                const staticPath = plus.io.convertLocalFileSystemURL(path);
                console.log('staticPath===', staticPath)
                uesSherpaOnnx({
                    mode: 'offline',
                    modelType: "zipformer2ctc",
                    model: '',
                    tokens: staticPath+'/' + "tokens.txt",
                    encoder: staticPath+'/' + "encoder-epoch-34-avg-19.int8.onnx",
                    decoder: staticPath+'/' + "decoder-epoch-34-avg-19.onnx",
                    joiner: staticPath+'/' + "joiner-epoch-34-avg-19.int8.onnx",
                    success: () => {
                        console.log('初始化成功11')
                        uni.hideLoading()
                    },
                    fail: (res) => {
                        console.log(res)
                    }
                })
            },
            //开始识别
            start() {
                startRecognizer()
            },
            //停止识别
            stop() {
                stopRecognizer()
            },
            //在不使用时或页面销毁时需调用此方法销毁实例
            destroy(){
                destroyRecognizer()
            },
            //设置监听
            setListner() {
                setRecognizerResultListner((res, final) => {
                    console.log('识别结果====', res)
                    console.log('final====', final)
                    this.content = res
                })
            },
            //移除监听
            removeRecognizerResultListner() {
                removeOnRecognizerResultListner()
            },

            //离线识别file(wav格式音频)
            recognizerFile() {
                let path = '/static/offlineOnnxModel/';
                const filePath = plus.io.convertLocalFileSystemURL(path) + '1.wav';
                startOfflineRecognizerFile(filePath, (result) => {
                    console.log('识别结果===', result)
                    this.content = result
                })

            },

            //停止离线识别wav file
            stopRecognizerFile() {
                stopOfflineRecognizerFile()
            }

        }
    }
</script>

<style>
    .title {
        margin: 20px 0 10px 0;
    }

    .content {
        height: 100%;
        border: 1px solid #ccc;
        background-color: #f2f2f2;
    }
</style>

VAD语音检测+paraformer模型用例

<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
        <button @click="destroy">销毁实例</button>
        <view style="padding:10px;">
            <view class="title">
                <text>识别结果:</text>
            </view>
            <view class="content">
                <textarea :value="content" disabled style="padding:15px;"></textarea>
            </view>
        </view>
    </view>
</template>

<script>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner,
        destroyRecognizer
    } from "@/uni_modules/xwq-sherpa-onnx";
    export default {
        data() {
            return {
                content: "", // textarea 最终展示文本
                finalLines: [], // 已确认结束的句子列表
                currentPartial: '', // 当前流式输出中的临时句子
                nextLineNumber: 1, // 下一条最终句子的显示序号
            }
        },
        methods: {
            resetDisplayState() {
                this.content = ''
                this.finalLines = []
                this.currentPartial = ''
                this.nextLineNumber = 1
            },
            renderContent() {
                // 已完成句子和当前 partial 分开渲染,避免流式结果重复叠加。
                const lines = this.finalLines.slice()
                if (this.currentPartial) {
                    lines.push(`${this.nextLineNumber}. ${this.currentPartial}`)
                }
                this.content = lines.join('\n')
                if (!this.currentPartial && this.finalLines.length > 0) {
                    this.content += '\n'
                }
            },
            // 在线识别
            initOnnx() {
                let path = '/static/paraformer';
                const staticPath = plus.io.convertLocalFileSystemURL(path);

                uesSherpaOnnx({
                    mode: 'online',
                    modelType: "paraformer",
                    tokens: staticPath+'/' + "tokens.txt",
                    encoder: staticPath+'/' + "encoder.int8.onnx",
                    decoder: staticPath+'/' + "decoder.int8.onnx",
                    joiner: "",
                    vad: staticPath+'/' + "silero_vad.onnx",
                    numThreads:2,
                    success: () => {
                        uni.hideLoading()
                    },
                    fail: (res) => {
                        console.log("初始化失败===",res)
                    }
                })
            },

            //开始识别
            start() {
                this.resetDisplayState()
                startRecognizer()
            },
            //停止识别
            stop() {
                stopRecognizer()
            },
            //销毁实例
            destroy(){
                this.resetDisplayState()
                destroyRecognizer()
            },
            //设置监听
            setListner() {
                // res: 当前回调文本;final: 是否为当前句的最终结果。
                setRecognizerResultListner((res, final) => {
                    console.log('识别结果====', res)
                    console.log('final====', final)
                    const text = (res || '').trim()
                    if (!text) {
                        return
                    }

                    if (!final) {
                        // partial 只替换当前行,不直接追加到历史内容。
                        this.currentPartial = text
                        this.renderContent()
                        return
                    }

                    const finalLine = `${this.nextLineNumber}. ${text}`
                    if (this.finalLines[this.finalLines.length - 1] !== finalLine) {
                        // final 才真正落行并递增序号。
                        this.finalLines.push(finalLine)
                        this.nextLineNumber += 1
                    }
                    this.currentPartial = ''
                    this.renderContent()

                })
            },
            //移除监听
            removeRecognizerResultListner() {
                removeOnRecognizerResultListner()
            },

        }
    }
</script>

<style>
    .title {
        margin: 20px 0 10px 0;
    }

    .content {
        height: 100%;
        border: 1px solid #ccc;
        background-color: #f2f2f2;
    }
</style>

实时语音识别+VAD语音检测+hotwords用法(针对语音唤醒场景)

hotwords.txt 文件内容说明:
格式:内容+空格+冒号+分数
例如:小度小度 :10
注意:后面的分数越高越容易触发,但太高也容易导致其他词被屏蔽无法触发,请设置合理值使用

silero_vad.onnx下载地址:https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx
<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
        <view style="padding:10px;">
            <view class="title">
                <text>识别结果:</text>
            </view>
            <view class="content">
                <textarea :value="content" :disabled="true" :auto-height="true" style="padding:15px;"></textarea>
            </view>
        </view>
    </view>
</template>

<script>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner
    } from "@/uni_modules/xwq-sherpa-onnx";
    export default {
        data() {
            return {
                content: ""
            }
        },

        methods: {
            // 在线识别
            initOnnx() {
                let path = '/static/onlineOnnxModel/transducer/';
                uesSherpaOnnx({
                    mode: 'online',
                    modelType: "transducer",
                    hotwords:path+'hotwords.txt', //热词内容文件
                    bpeVocab:path +'bpe.vocab', //热词解码文件(transducer模型自带)
                    encoder: path + "encoder-epoch-99-avg-1.int8.onnx",//transducer模型文件
                    decoder: path + "decoder-epoch-99-avg-1.onnx",//transducer模型文件
                    joiner: path + "joiner-epoch-99-avg-1.int8.onnx",//transducer模型文件
                    tokens: path + "tokens99.txt", //transducer模型tokens
                    vad:path + "silero_vad.onnx",//VAD模型文件
                    isCopyFile:false,
                    success: () => {
                        console.log('初始化成功')
                    },
                    fail: (res) => {
                        console.log(res)
                    }
                })
            },

            //开始识别
            start() {
                startRecognizer()
            },
            //停止识别
            stop() {
                stopRecognizer()
            },
            //设置监听
            setListner() {
                setRecognizerResultListner((res, final) => {
                    console.log('识别结果====', res)
                    console.log('final====', final)
                    this.content = res
                })
            },
            //移除监听
            removeRecognizerResultListner() {
                removeOnRecognizerResultListner()
            }
        }
    }
</script>

<style>
    .title {
        margin: 20px 0 10px 0;
    }

    .content {
        height: 100%;
        border: 1px solid #ccc;
        background-color: #f2f2f2;
    }
</style>

sense_voice模型+vad语音检测使用方式

silero_vad模型下载地址:https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx
silero_voice模型下载地址:https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2
<template>
    <view>
        <button @click="initOnnx">初始化Onnx</button>
        <button @click="setListner">设置结果监听</button>
        <button @click="removeRecognizerResultListner">移除结果监听</button>
        <button @click="start">开始识别</button>
        <button @click="stop">停止识别</button>
        <view style="padding:10px;">
            <view class="title">
                <text>识别结果:</text>
            </view>
            <view class="content">
                <textarea :value="content" :disabled="true" :auto-height="true" style="padding:15px;"></textarea>
            </view>
        </view>
    </view>
</template>

<script>
    import {
        startRecognizer,
        uesSherpaOnnx,
        stopRecognizer,
        setRecognizerResultListner,
        removeOnRecognizerResultListner
    } from "@/uni_modules/xwq-sherpa-onnx";
    export default {
        data() {
            return {
                content: ""
            }
        },

        methods: {
            // 在线识别
            initOnnx() {
                let path = '/static/onlineOnnxModel/transducer/';
                uesSherpaOnnx({
                    mode: 'online',
                    modelType: "sense_voice",
                    tokens: path + "tokens.txt",
                    model: path + "model.int8.onnx",// silero_voice模型模型文件
                    vad:path + "silero_vad.onnx",//silero_vad模型模型
                    isCopyFile:false,
                    success: () => {
                        console.log('初始化成功')
                        // uni.hideLoading()
                    },
                    fail: (res) => {
                        console.log(res)
                    }
                })
            },

            //开始识别
            start() {
                startRecognizer()
            },
            //停止识别
            stop() {
                stopRecognizer()
            },
            //设置监听
            setListner() {
                setRecognizerResultListner((res, final) => {
                    console.log('识别结果====', res)
                    console.log('final====', final)
                    this.content = res
                })
            },
            //移除监听
            removeRecognizerResultListner() {
                removeOnRecognizerResultListner()
            }
        }
    }
</script>

<style>
    .title {
        margin: 20px 0 10px 0;
    }

    .content {
        height: 100%;
        border: 1px solid #ccc;
        background-color: #f2f2f2;
    }
</style>

其他插件预览

隐私、权限声明

1. 本插件需要申请的系统权限列表:

麦克风权限

2. 本插件采集的数据、发送的服务器地址、以及数据用途说明:

3. 本插件是否包含广告,如包含需详细说明广告表达方式、展示频率: