---
title: "Configuring for ToBI annotations"
author: "Jonathan Harrington"
date: "WiSe 2021"
output: 
  bookdown::html_document2:
    number_sections: TRUE
    toc: true
    theme: flatly
    highlight: pygments
---

<style>
div.gray {background-color: #e8e8e8; border-radius: 5px; padding: 20px;}
body {font-size: 16pt;}
h1 {font-size: 24pt;}
h2 {font-size: 22pt;}
p.caption {font-size: 12pt; text-align: justify;}
code.sourceCode {font-size: 16pt;}
</style>

# Preliminaries
If you have not already done so as a consequence of completing earlier modules, follow the
[setup instructions here](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/02_praat.html#preliminaries-and-starting-up-r)

```{r}
library(tidyverse)
library(emuR)
library(wrassp)
sourceDir = "./testsample"
targetDir = "./emu_databases"
```

# Overall aim

Starting out with just 
a [text collection](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/03_forced_alignment.html#converting-a-text-collection-into-an-emu-database)
the aim is to create Emu database that is useful for tones-and-break-indices annotation. The more specific aim is to configure the database to obtain a configuration as in Fig. \@ref(fig:figtobi). 
 
```{r figtobi, fig.align="center", fig.cap="A TOBI annotation", echo=FALSE}
knitr::include_graphics("./img/figtobi.png")
```

The starting point will be the text collection of Albanian data 
[analysed in the previous module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/03_forced_alignment.html#forced-alignment-albanian).

# Forced alignment
The first task is to get apply forced alignment to these Albanian data. The commands will be presented in this section without detailed comment given that these were discussed 
[in the previous module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/03_forced_alignment.html#forced-alignment-albanian)

The resulting Emu database will be stored as `alb2_DB` (to distinguish it from the one created in the last module)


```{r}
# the path of the text collection of Albanian utterances
path.albanian = file.path(sourceDir, "albanian")
# convert it to an Emu database called alb2_DB
convert_txtCollection(dbName = "alb2", 
                      sourceDir = path.albanian,targetDir = targetDir)
# load the emu database
alb2_DB = load_emuDB(file.path(targetDir, "alb2_emuDB"))
# run forced alignment
runBASwebservice_all(alb2_DB,
transcriptionAttributeDefinitionName = "transcription", 
language = "sqi-AL",  runMINNI = F)
```

# Configuring a database for TOBI (tones and break indices) annotation

The database currently has an `ITEM` tier `ORT` than dominates an `ITEM` tier `MAS` containing syllabifications than dominates the `SEGMENT` tier `MAU` containing the phonetic segmenation as shown by either `serve(alb2_DB, useViewer=F)` or by `summary(alb2_DB)`. 

The task is to add a  tier `Tone` for imarking ntonational events and to add a new  `SEGMENT` tier, `ORT2` **showing word segmentations and their times**. The tonal events of the `Tone` tier and annotations of the `ORT2` tier are to be linked automatically, just  as in Fig. \@ref(fig:figtobi). 
 

Here are the steps needed:

- Create the `SEGMENT` tier `ORT2` containing word annotations linked to times.
- Create a `Tone tier` and link it to `ORT2` 
- Calculate and display f0.
- Annotate the `Tone` tier and link the annotations automatically to word annotations.

## Creating and displaying the `SEGMENT` tier `ORT2` with word annotations.

This requires four steps:

- Make a `SEGMENT` tier called `ORT2`
- Make a segment list of all words in the database using `query()`.
- Add the word annotations to `ORT2`.
- Display the annotations of `ORT2` time-aligned to the signals

### Make a `SEGMENT` tier called `ORT2`
As discussed [in an earlier module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/praat.html#adding-an-event-tier), this can be done with the `add_levelDefinition()` function, thus:

```{r}
add_levelDefinition(alb2_DB, "ORT2", "SEGMENT")
# list the tiers (annotation levels):
list_levelDefinitions(alb2_DB)
```

### Make a segment list of all words in the database
This can be done with the `query()` function. 
```{r}
# find all words in the database and 
# get their start and end times.

text.s = query(alb2_DB, "[ORT =~ .*]")
text.s
```

The attribute and name of the tier from which this segment list was derived is `ORT` as shown by this:

```{r}
text.s$attribute
text.s$level
```

These all need to be changed to the name of the new `SEGMENT` tier `ORT2`. This can be done as follows:

```{r}
text.s$attribute = paste(text.s$attribute, "2", sep="")
text.s$level = text.s$attribute
text.s$attribute
text.s$level
```
### add the annotations to `ORT2`

The next step is to add  the information from `text.s` to the `ORT2` tier. This is done with the function `create_itemsInLevel()` thus:

```{r}
create_itemsInLevel(alb2_DB, 
                    itemsToCreate = text.s)
```

So it should now be possible to query the word annotations directly from `ORT2`:
```{r}
query(alb2_DB, "[ORT2 =~ .*]")
```

### Display the annotations of `ORT2` time-aligned to the signals

Entering `serve(alb2_DB, useViewer = F)` shows that the  `SEGMENT` tier `MAU` is displayed underneath the signals. As also discussed  [in the same earlier module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/praat.html#adding-an-event-tier), the reason for this is because of the setting in the `get/set_levelCanvasesOrder()` functions. Thus this confirms that the current display is to `MAU`

```{r}
get_levelCanvasesOrder(alb2_DB, perspectiveName = "default")
```

But the aim is to display only the newly created `ORT2` tier. This can be done as follows:
```{r}
set_levelCanvasesOrder(alb2_DB, 
                       perspectiveName = "default", 
                       order = "ORT2")
```
If you view the database now, it will show only the segment tier `ORT2` underneath the signals:

```{r, eval=F}
serve(alb2_DB, useViewer=F)
```
  
## Create a `Tone tier` and link it to `ORT2` 

### Creating the new `Tone` tier
This can be done using the `add_levelDefinition()` function used earlier in creating `ORT2`. In this case, `Tone` should be an `EVENT` tier for marking intonational targets as single points in time. The command is:

```{r}
add_levelDefinition(alb2_DB, 
                    name = "Tone", 
                    type = "EVENT")
```
### Linking `Tone` and `ORT2`
The tiers `Tone` and `ORT2` are currently unlinked as the following verifies:

```{r}
list_linkDefinitions(alb2_DB)
```

But they need to be linked in order that their annotations can be queried relatively to each other. On the assumption that a word can be associated with one or more tones, but that a given tone can only be associated with one word, then the association between the tiers is `ONE-TO-MANY`. The tiers should therefore be linked with the `add_linkDefinitions()` function in the following way:

```{r}
add_linkDefinition(alb2_DB, "ONE_TO_MANY", "ORT2", "Tone")
```

The same command as earlier now verifies that these tiers are linked (see the last line of the following output):

```{r}
list_linkDefinitions(alb2_DB)
```
### Displaying the `Tone` tier.
As before, this is done with the function `set_levelCanvasesOrder()`

```{r}
set_levelCanvasesOrder(alb2_DB, 
                       perspectiveName="default", 
                       order = c("Tone", "ORT2"))
```
You can verify the relationship that the `Tone` and `ORT2` tiers are being displayed underneath the signals with:

```{r, eval=FALSE}
serve(alb2_DB, useViewer=F)
```


## Calculating and displaying f0 

There are three steps here:

- Calculate f0
- Add the f0 information to the database
- Configure the database to display the f0 track.

Most of these steps [were covered in an earlier module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/02_praat.html#calculating-pitch-with-wrassp). Therefore, only a brief summary is given as follows.

### Calculating f0
There are four parts to this:

- Identifying the `.wav` files.
- Calculating pitch data.
- Adding the pitch data to the database.
- Displaying the pitch data.

#### Identifying the `.wav` files
Identify the `.wav` files for which fundamental frequency data is to be calculated. They are here:
```{r}
alb_wav_paths = list.files(path.albanian, 
        pattern = ".*wav$", recursive = T, full.names = T)
alb_wav_paths
```

#### Calculating pitch data
Calculate pitch for each `.wav` file and store the output:

```{r}
mhsF0(alb_wav_paths, outputDirectory = path.albanian)
```
#### Adding the pitch data to the database
Add these f0 files to the Emu database with the `add_files()` function

```{r}
add_files(alb2_DB, dir = path.albanian, 
          fileExtension = "pit", targetSessionName = "0000")
```

#### Diplaying the pitch data 
This requires two steps: 

- defining the pitch track.
- displaying the pitch data.

##### Defining the pitch track
Use the `add_ssffTrackDefinition()` function for this purpose. See also 
[this earlier module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/02_praat.html#calculating-pitch-with-wrassp) for further details.

```{r}
add_ssffTrackDefinition(alb2_DB,
                        name = "pitch",
                        columnName = "pitch",
                        fileExtension = "pit")
```

##### Displaying the pitch data
As 
[explained in an earlier module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/02_praat.html#displaying-the-pitch-files-in-the-webapp), this is done with the function `set_signalCanvasesOrder()`. Currently, the waveform and spectrogram are being displayed, as shown by:

```{r}
get_signalCanvasesOrder(alb2_DB, perspectiveName="default")
```

To set things up so that the pitch data is displayed underneath the spectrogram and without displaying the spectrogram:
```{r}
set_signalCanvasesOrder(alb2_DB, perspectiveName = "default",
                        order = c("SPEC",  "pitch"))
```

Overlaying the pitch on the spectrogram is a bit more intricate. For this, you will have to use a  text editor (outside of `R`) to edit `alb2_DBconfig.json` that is located here. 

```{r}
file.path(targetDir, "alb2_emuDB", "alb_DBconfig.json")
```
Then proceed as follows:

1. Optionally make a backup copy of `alb2_DBconfig.json` in case anything goes wrong.
2. Open `alb2_DBconfig.json` **with a plain text editor**.
3. Search for `"assign"`
4. Carefully replace

`"assign": [],`

with


`"assign": [{
"signalCanvasName": "SPEC",
"ssffTrackName": "pitch"
}],`


5. Save `alb2_DBconfig.json`
6. Look at the database again

```{r, eval=FALSE}
serve(alb2_DB, useViewer = F)
```

6. Finally, get rid of the pitch track. 

```{r, eval=FALSE}
set_signalCanvasesOrder(alb2_DB, 
                        perspectiveName = "default",
                        order = "SPEC")
```
 
And look again at the database. It should now be as in Fig. \@ref(fig:figtobi).

```{r, eval=FALSE}
serve(alb2_DB, useViewer = F)
```

## Automatically linking annotations at the `Tone` tier

The first task is to provide a couple of annotations. More specifically add two pitch targets (e.g., `L*` to *Lena*, `L*` to *leu*) for the utterance `0001BF_1syll_1` as in Fig. \@ref(fig:figtobi). See 
[the sub-section in this module](https://www.phonetik.uni-muenchen.de/~jmh/lehre/sem/ws2122/Emuintro/01_creating_database.html#creating-an-emu-database-from-scratch) (scroll down to the grey box) to see details of how to annotate in Emu.

```{r, eval=FALSE}
serve(alb2_DB, useViewer = F)
```

The next task is to link automatically the annotations at the `tonal level`Tone` tier to the corresponding words in `ORT2` using the function `autobuild_linkFromTimes()`. This function links an annotation at time *t* at the `Tone` tier to an annotation at the `ORT2` tier if `w_onset < t < w_offset` where `w_onset` and `w_offset` are the word's start and end times respectively (i.e. automatically link any tone to a word if the tone's time falls within the boundary times of the word). 

```{r, eval=FALSE}
autobuild_linkFromTimes(alb2_DB,
                        superlevelName = "ORT2",
                        sublevelName = "Tone")
```

Look at the database again: Check in the hierarchy view that the links between `ORT2` and `Tone` were created in the path `ORT2 -> Tone`). 

```{r, eval=FALSE}
serve(alb2_DB, useViewer = F)
```

It should now be possible to query tones with respect to words and vice-versa. For example:

```{r, eval=FALSE}
# Find all pitch-accented words 
# (i.e. find any word linked to any tone):
query(alb2_DB, "[Tone =~.* ^ #ORT2 =~.*]")
# A tibble: 2 × 16
 # labels start   end db_uuid         session bundle  start_item_id end_item_id level
 # <chr>  <dbl> <dbl> <chr>           <chr>   <chr>           <int>       <int> <chr>
#1 lena   1050. 1670. 589b69bd-e69a-… 0000    0001BF…            40          40 ORT2 
#2 leu    1670. 2100. 589b69bd-e69a-… 0000    0001BF…            41          41 ORT2 
```