有奖捉虫:行业应用 & 管理与支持文档专题 HOT

功能说明

输入音频数据,输出苹果 ARKit 标准的52表情数据,请参见 ARFaceAnchor。 您可以利用这些表情数据做一进步的开发,例如传到 Unity 中驱动您的模型。

接入方式

方式1:通过虚拟形象 SDK 接入

语音转表情集成在虚拟形象 SDK 中,因此第一步需要按照虚拟形象文档进行接入。

方式2:通过独立的语音转表情 SDK 接入

如果您只需要语音转表情的 AI 原子能力,不需要用到虚拟形象 SDK 的能力,则可以考虑使用独立的语音转表情 SDK,aar 包约6MB左右。请联系我们的架构师或销售获取此 SDK。

接入步骤

1. 设置 License,请参见 鉴权
2. 配置模型文件:请将必需的模型文件从 assets 拷贝到 app 的私有目录,例如:context.getFilesDir() + "/my_models_dir/audio2exp",然后在调用 Audio2ExpApi 的 init(String modelPath) 接口时,传入参数 context.getFilesDir() + "/my_models_dir" 模型文件在 SDK 包里,位置如下:
?
?

接口说明

接口
说明
public int Audio2ExpApi.init(String modelPath);
初始化,传入模型路径,见上文说明。返回值为0表示成功
public float[] Audio2ExpApi.parseAudio(float[] inputData);
输入的是音频数据,要求单通道,16K采样率,数组长度为267(即267个采样点),输出的数据是长度为52的float数组,表示52表情基,取值为0到1之间,顺序为 苹果标准顺序{"eyeBlinkLeft","eyeLookDownLeft","eyeLookInLeft","eyeLookOutLeft","eyeLookUpLeft","eyeSquintLeft","eyeWideLeft","eyeBlinkRight","eyeLookDownRight","eyeLookInRight","eyeLookOutRight","eyeLookUpRight","eyeSquintRight","eyeWideRight","jawForward","jawLeft","jawRight","jawOpen","mouthClose","mouthFunnel","mouthPucker","mouthRight","mouthLeft","mouthSmileLeft","mouthSmileRight","mouthFrownRight","mouthFrownLeft","mouthDimpleLeft","mouthDimpleRight","mouthStretchLeft","mouthStretchRight","mouthRollLower","mouthRollUpper","mouthShrugLower","mouthShrugUpper","mouthPressLeft","mouthPressRight","mouthLowerDownLeft","mouthLowerDownRight","mouthUpperUpLeft","mouthUpperUpRight","browDownLeft","browDownRight","browInnerUp","browOuterUpLeft","browOuterUpRight","cheekPuff","cheekSquintLeft","cheekSquintRight","noseSneerLeft","noseSneerRight","tongueOut"}
public int Audio2ExpApi.release();
使用完毕后调用,释放资源

集成代码示例

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
findViewById(R.id.button).setOnClickListener(new OnClickListener() {
@Override
public void onClick(View view) {
TELicenseCheck.getInstance().setTELicense(MainActivity.this, licenseUrl, licenseKey, new TELicenseCheckListener() {
@Override
public void onLicenseCheckFinish(int errorCode, String s) {
Log.d(TAG, "onLicenseCheckFinish: errorCode = "+errorCode+",msg="+s);
if (errorCode == TELicenseCheck.ERROR_OK) {
//license check success
Audio2ExpApi audio2ExpApi = new Audio2ExpApi();
int err = audio2ExpApi.init(MainActivity.this.getFilesDir() +"/models");
Log.d(TAG, "onLicenseCheckFinish: err="+err);
//TODO start record and parse audio data
} else {
// license check failed
}
}
});
}
});
}
说明
完整的示例代码请参考 虚拟形象 SDK demo 工程
录音可以参考 com.tencent.demo.avatar.audio.AudioCapturer
接口使用可以参考:com.tencent.demo.avatar.activity.Audio2ExpActivity 及其相关类。


http://www.vxiaotou.com