> For the complete documentation index, see [llms.txt](https://b-it-bots.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://b-it-bots.gitbook.io/wiki/gitbook-1/_posts/2018-05-06-speech.md).

# Speech

The speech package is run on Jenny's PC3 This will generate two files (.bnf and .fcf), which contain the speech commands in the form of a dictionary . The state machine receives one of this keys, and looks for the related words, while discarding the garbage.

## Packages

* `nuance_vocon_speech_recognition`: speech recognition
* `manyears_ros`: sound localization 8 sounds / Manyears

### Scripts

* `android_speech_recognition.py` define the path to the grammar file
* `command_states_speech.py` requires a string and just outputs the audio.

## Running the code

### Speech recognition

`roslaunch vocon_speech_recognizer vocon.launch`

The default grammar file is specified in:`vocon_speech_recognizer/data/general_purpose.bnf`.

Recognized speech is published on the topic:`/recognized_speech` Strings published to `/sound/say` are output to the speaker as audio

```
rostopic echo /recognized_speech
rostopic echo /sound/say
rostopic pub /sound/say std_msgs/String "hello"
```

To generate new grammar files you can:\
a) <https://gist.github.com/moriarty/20fbb331d90755a059ad7df30fba16ac>\
b) go to <http://edinburgh.uni-koblenz.de:8000/> and upload the grammar file (.txt).

#### In Jenny

Run the following:

```bash
cob3
roslaunch mdr_speech_and_audio speech_test.launch
```

In another terminal:

```
cob1
rosrun mdr_speech_and_audio speech_test
```

once you run the `speech_test`

Say "begin test" or "start test" and then ask one of the questions from `/mcr_speech/mcr_question_matching/ros/config/questions.txt`

### ManyEars sound localization

Sound source direction is published on the topic: `/manyears/source_direction`

Parameter file which specifies arrangement of microphones on Jenny and other config: `manyears_ros/data/cob3-1.mes`

#### In Jenny

In addition to the `roscore` and `robot.launch`, run:

```bash
cob1
rosrun mdr_speech_and_audio turn_and_answer_test
```

In another terminal:

```
cob2
roslaunch manyears_ros demo_8sounds.launch
```

And finally:

```
cob1
roslaunch mdr_speech_and_audio turn_and_answer_test.launch
```

Speech synthesis is done on cob3 using the following node. It is launched by default with bringup. In case you reboot cob3 while bringup is running, relaunch this node:

```bash
cob3
roslaunch cob_bringup sound.launch
```

#### Calibration

A description of the parameters can be found here: <https://sourceforge.net/p/manyears/wiki/SoftwareDocumentation/>