Quick start

To start, create or open a TwinCAT HMI project.

Installation

Install the NuGet package "Beckhoff.TwinCAT.HMI.Speech" via the NuGet Package Manager.

Establishing a connection

The server must be announced first in TwinCAT Speech (see Documentation of TwinCAT Speech).

Basic configuration

By default, no Speech connection is established. For this purpose, a connection must be actively started by a TypeScript/JavaScript API call.

TcHmi.TcSpeech.openConnection({
   enableMicrophone: true, 
   enableSpeaker: true 
});

Further possible parameters of the call with the respective default value are:

defaultVolume: 1 This is the volume of the sound output when loading the application. This can be set from 0 to 1.

domain: 'TcHmiSpeech' Domain name under which the Speech Extension is reachable.

confidenceThreshold: 0.2 Each recognition event has a probability that the recognized text corresponds to the specified entry. This threshold value specifies the value from which this is treated as a genuine event. If, for example, 0.9 is specified here, an HMI event is only triggered in the case of a very precise match. (see also the Documentation from Microsoft)

Use of recognized events in the project

Recognized texts call events in the Speech system. These are transferred to the browser and can be used there in the same way as control events (such as myButton.onClick). The Recognition Tag from the SRGS file is taken here as the name.

It can thus be used as an event in the Actions and Conditions Editor in the HMI. The name is entered in the Custom area for this purpose. If, for example, SpeechOnCommand<SET_ENGLISH> is entered here, then an action is started on recognition of the Recognition Tag SET_ENGLISH.

Use via CodeBehind is also possible. Sample code with TypeScript:

TcHmi.EventProvider.register('SpeechOnCommand<SET_ENGLISH>', (e, data: TcHmi.TcSpeech.IEventCallbackParameter) => {
    // Will output:
    // 'The user said a text which triggered the recoginition tag: SET_ENGLISH'
    console.log('The user said a text which triggered the recoginition tag: ' + data.Command);
    if (data.Confidence > 0.9) {
        console.log('The speech recognition was pretty sure about that.');
    }
    if (data.Parameter) {
        console.log('We got a parameter for this recognition: ', data.Parameter);
    }
});