Speech Recognition (Speech-to-Text)

API Reference: SpeechRecognizer

SpeechRecognizer provides access to the speech recognition service.

Offline Speech Recognition On Android

To use offline speech recognizer, language package must be installed. If not installed, the speech recognizer throws SpeechRecognizer.Error.SERVER error. This screenshot is taken from a device that supports English and Spanish speech recognition.

onFinish Callback

onFinish callback is triggered in a few seconds after starting Speech Recognizer on Android devices. But it requires much more time on iOS devices.

onResult Callback

onResult is fired when partial recognition results are available. This callback can also be fired different number of times in Android and iOS. There is no guarantee onResult callback will be fired on each word.

Network changes on iOS

When the network changes, speech recognizer may not available. The speech recognizer throws SpeechRecognizer.Error.NETWORK error in this case.

Supported Locales on iOS

For iOS, you should check your current locale is supported from isLocaleSupported function. if doesn't support your current locale, you must set locale parameter in constructor function.

Android Permissions

For Android, RECORD_AUDIO permission is required for using SpeechRecognizer. Add the following permission to AndroidManifest.xml file in necessary, also take a look at Application.android.Permissions.RECORD_AUDIO.

See also Android official documentation about this permission.

iOS Permissions

Permission keys must be added info.plist for iOS

NSSpeechRecognitionUsageDescription\ $(PRODUCT_NAME) Speech Recognition access for development

NSMicrophoneUsageDescription\ $(PRODUCT_NAME) Microphone access for development

note

The components in the example are added from the code for better showcase purposes. To learn more about the subject you can refer to:

Adding Component From Code

As a best practice, Smartface recommends using the WYSIWYG editor in order to add components and styles to your page or library. To learn how to use UI Editor better, please refer to this documentation

UI Editor Basics

TypeScript

import PageSampleDesign from "generated/pages/pageSample";
import { Route, Router } from "@smartface/router";
import System from "@smartface/native/device/system";
import Button from "@smartface/native/ui/button";
import Application from "@smartface/native/application";
import { styleableComponentMixin, styleableContainerComponentMixin } from "@smartface/styling-context";
import Label from "@smartface/native/ui/label";
import FlexLayout from "@smartface/native/ui/flexlayout";
import SpeechRecognizer from "@smartface/native/speechrecognizer";

class StyleableButton extends styleableComponentMixin(Button) {}
class StyleableLabel extends styleableComponentMixin(Label) {}
class StyleableFlexLayout extends styleableContainerComponentMixin(FlexLayout){}

//You should create new Page from UI-Editor and extend with it.
export default class Sample extends PageSampleDesign {
    myButton: StyleableButton;
    myLabel: StyleableLabel;
    myFlexLayout: StyleableFlexLayout;
    constructor(private router?: Router, private route?: Route) {
    super({});
  }

  // The page design has been made from the code for better
  // showcase purposes. As a best practice, remove this and
  // use WYSIWYG editor to style your pages.
  centerizeTheChildrenLayout() {
    this.dispatch({
        type: "updateUserStyle",
        userStyle: {
            flexProps: {
              flexDirection: 'COLUMN',
              justifyContent: 'CENTER',
              alignItems: 'CENTER'
            }
        }
    })
  }

  startSpeechRecognizer(): void {
    SpeechRecognizer.start({
      locale: "en_US",
      onResult: (result) => {
        this.myLabel.text = result;
      },
      onFinish: (result) => {
        this.myButton.text = "Start Recording";
        alert("Finish : " + result);
      },
      onError: (error) => {
        this.myButton.text = "Start Recording";
        alert("Error : " + error);
      },
    });
  }

  onShow() {
    super.onShow();
    const { headerBar } = System.OS === System.OSType.ANDROID ? this : this.parentController;
    Application.statusBar.visible = true;
    headerBar.visible = true;
  }

  onLoad() {
    super.onLoad();
    this.centerizeTheChildrenLayout();
    
    this.myFlexLayout = new StyleableFlexLayout();

    this.myButton = new StyleableButton({
        text: "START SPEECH",
        onPress: () => {
        if (!SpeechRecognizer.isRunning()) {
            this.myButton.text = "Stop Recording";
            if (System.OS === System.OSType.IOS) {
            this.startSpeechRecognizer();
            } else if (System.OS === System.OSType.ANDROID) {
            const RECORD_AUDIO_CODE = 1002;
            Application.android.requestPermissions(
                RECORD_AUDIO_CODE,
                Application.Android.Permissions.RECORD_AUDIO
            );
            Application.android.onRequestPermissionsResult = (e) => {
                if (e.requestCode === RECORD_AUDIO_CODE && e.result) {
                this.startSpeechRecognizer();
                }
            };
            }
        } else {
            this.myButton.text = "Start Recording";
            SpeechRecognizer.stop();
        }
        },
    });

    this.addChild(this.myFlexLayout, "myFlexLayout", ".sf-flexLayout", {
        flexGrow: 1,
        flexProps: {
        flexDirection: "COLUMN",
        },
    });

    this.myLabel = new StyleableLabel({
        text: "Result is here",
    });

    this.myFlexLayout.addChild(this.myButton, "myButton", ".sf-button", {
        width: null,
        height: null,
        flexGrow: 1,
    });

    this.myFlexLayout.addChild(this.myLabel, "myLabel", ".sf-label", {
        width: null,
        height: null,
        flexGrow: 3,
    });
  }
}