---
title: "05 Querying Annotation Structures"
output:
  pdf_document: default
  html_document: default
word_document: default
---


First of all, let us once again create a temporary EMU-SDMS-database:

```{r, echo=TRUE,results="hide",message=FALSE}
# load package
library(emuR)
# create demo data in directory
# provided by tempdir()
create_emuRdemoData(dir = tempdir())
# create path to demo database
path2ae = file.path(tempdir(), "emuR_demoData", "ae_emuDB")
# load database
ae = load_emuDB(path2ae, verbose = F)
summary(ae)
```
In the link definitions, we can see a very rich annotation structure. This will allow us to perform even more complicated queries. Let us have a look at the annotation structure:

```{r, rexample:tutorial_serve, eval=FALSE}
serve(ae)
```

![Figure 1: Hierarchy of the first utterance of the database *ae*](pics/annot_struct.png)

However, let us start with very basic queries. The command for conducting queries is simply `query`, called with at least two arguments, `emuDBhandle` and `query`:

```{r}
query(emuDBhandle = ae,query = "Phonetic==V")
```

We could have been less verbose instead to achieve the very same result, e.g. by typing:

```{r,eval=FALSE}
query(ae,"Phonetic==V")
```

 
The expression `"Phonetic==V"` is a legal expression in the `EMU Query Language (EQL)` and could be translated into "which labels in the level Phonetic equal the label "V"".
However, before we start to learn the `EQL` in detail, we will discuss other parameters of the `query`-command, and learn the structure of a result of the query-command.
```{r,eval=FALSE}
help(query)
```
#Arguments to `query()`
```{r, eval=FALSE}
  query(emuDBhandle, + 
        query, + 
        sessionPattern = ".*", + 
        bundlePattern = ".*", + 
        queryLang = "EQL2", + 
        timeRefSegmentLevel = NULL, + 
        resultType = NULL, + 
        calcTimes = TRUE, verbose = FALSE)
```


| Argument | Meaning |
| :------------------ | :--------------------------------------------------------------------------------------|
|`emuDBhandle`|	emuDB handle object|
|`query`|	string (see `vignette("EQL")`)|
|`sessionPattern`|	A regular expression pattern matching session names to be searched from the database|
|`bundlePattern`|	A regular expression pattern matching bundle names to be searched from the database|
|`queryLang`|	query language used for evaluating the query string|
|`timeRefSegmentLevel`|	set time segment level from which to derive time information. It is only necessary to set this parameter if the queried parent level is of type `ITEM` and more than one child level contains time information.|
|`resultType`|	type (class name) of result|
|`calcTimes`|	calculate times for resulting segments (results in NA values for start and end times in emuseg/emuRsegs). As it can be very computationally expensive to calculate the times for large nested hierarchies it can be turned off via this boolian parameter.|
|`verbose`|	be verbose. Set this to `TRUE` if you wish to choose which path to traverse on intersecting hierarchies. If set to `FALSE` (the default) all paths will be traversed (= legacy EMU behaviour).|


We could e.g. restrict ourselves to only one `sessionPattern` (but there is only one in `ae`) or to only one `bundlePattern`; the latter case may be a not-too-unusual use case:
```{r}
query(emuDBhandle = ae,query = "Phonetic==V",bundlePattern = "msajc003")
```

#Results of a query: `emuRsegs` 
An emuR segment list is a list of segment descriptors. Each segment descriptor describes a sequence of annotation elements. The list is usually a result of an emuDB query using function `query`.

An `emuRsegs` object is an attributed data.frame, with one row per segment descriptor.

**Data frame columns**

+ *labels*: sequenced labels of segment concatenated by '->'

+ *start*: onset time in milliseconds

+ *end*: offset time in milliseconds

+ *session*: session name

+ *bundle*: bundle name

+ *level*: level name

+ *type*: type of "segment" row: `ITEM`: symbolic item, `EVENT`: event item, `SEGMENT`: segment

Additional **hidden columns**

+ *utts*: utterance name (for compatibility to emusegs class)

+ *db_uuid*: UUID of emuDB

+ *startItemID*: item ID of first element of sequence

+ *endItemID*: item ID of last element of sequence

+ *sampleStart*: start sample position

+ *sampleEnd*: end sample position

+ *sampleRate*: sample rate

**Attributes**

+ *database*: name of emuDB

+ *query*: Query string

+ *type*: type (`SEGMENT` or `EVENT`, but not `ITEM`) (for compatibility to `emusegs` class)

Assign our query above to an object that we call `V_first_utt`:
```{r}
(V_first_utt = query(emuDBhandle = ae,query = "Phonetic==V",bundlePattern = "msajc003"))
class(V_first_utt)
summary(V_first_utt)
```
Although every column can easily be called with the $-operator, there are some commands in emuR doing the very same job, e.g. the commands `label()`, `start()`, `end()`:
```{r}
#Get labels:
#either
V_first_utt$labels
#or
label(V_first_utt)
```
Other examples:
```{r}
#Get start times:
start(V_first_utt)

#Get end times:
end(V_first_utt)
```
Other more complex commands are simply a shorter way of doing rather simple things:
```{r}
#Get durations:
dur(V_first_utt)

#which is, of course, the same as:
end(V_first_utt) - start(V_first_utt)

#We can also use the $-operator to access the columns of V_first_utt, because it is a data.frame
#see chapter 07
V_first_utt$end - V_first_utt$start

```
**Calculated times**
You might recall from chapter 04, that start and end times are stored internally as `sampleStart`
and `sampleDur`. In the very same annot.json-file, there is also information about the sampling frequency:
```
{
    "name": "msajc003",
    "annotates": "msajc003.wav",
    "sampleRate": 20000,
...
        {
            "name": "Phonetic",
            "type": "SEGMENT",
            "items": [
                {
                    "id": 147,
                    "sampleStart": 3749,
                    "sampleDur": 1389,
                    "labels": [
                        {
                            "name": "Phonetic",
                            "value": "V"
                        }
                    ]
                },
```

This means, that out first "V" start at the 3749th sample of 20000 samples per second. We can calculate:
```{r}
3749/20000
```
... i.e., our first V starts a 187.45 milliseconds. Compare this to
```{r}
start(V_first_utt)[1]
```
In other words, `$start` and `$end` report times in milliseconds.
As we have mentioned earlier, `$sampleStart` and `$sampleDur` are available in the `emuRsegs`-object, but are hidden. Nevertheless, we could call them with

```{r}
V_first_utt$sampleStart
V_first_utt$sampleDur
```

As can be seen in the annot.json-file, our "V" on level "Phonetic" is of type `SEGMENT`, i.e. of a time-aligned type. This information can also be found in

```{r}
V_first_utt
```
What happens, if we were looking for a timeless `ITEM`?
```{r}
(amongst=query(emuDBhandle = ae,query = "Text==amongst",bundlePattern = "msajc003"))
```
We can see in `$type`, that the type of "amongst" is `ITEM`. However, we can see non-empty `$start` and `$end` columns, and 
```{r}
attributes(amongst)$type
```
says, that its type was "segment" (for compatibility with the former emusegs format).
So, although "amongst" is actually a timeless `ITEM` ...
```
        {
            "name": "Word",
            "type": "ITEM",
            "items": [
                {
                    "id": 2,
                    "labels": [
                        {
                            "name": "Word",
                            "value": "C"
                        },
                        {
                            "name": "Accent",
                            "value": "S"
                        },
                        {
                            "name": "Text",
                            "value": "amongst"
                        }
                    ]
                },
```
... times can be derived from this item's first and last time-aligned segments (our first "V" above is the first segment, and amongst end with a "t", that is aspirated (="H" on the Phonetic level, see Figure 1))
```{r}
start(query(emuDBhandle = ae,query = "Text==amongst",bundlePattern = "msajc003"))
start(query(emuDBhandle = ae,query = "Phonetic==V",bundlePattern = "msajc003")[1,])
end(query(emuDBhandle = ae,query = "Text==amongst",bundlePattern = "msajc003"))
end(query(emuDBhandle = ae,query = "Phonetic==H",bundlePattern = "msajc003")[1,])
```
This, of course, will only work if Text (one attribute of Word) and Phonetic levels are linked (and they are, see also Figure 1):
```{r}
list_linkDefinitions(ae)
```
If the `ITEM` we are interested in was linked to several time-aligned segments, we would have to use `query`'s parameter `timeRefSegmentLevel` to choose the segment level from which `query` derives time information.

#Preview to `requery_hier` and `requery_seq`
We already know that the first "V" in our results belongs to (and is linked to) the word "amongst".
By which words are the other "V"s dominated, then?
We could find our by a hierarchical re-query:
```{r}
#find all "V"-labels in `ae`
(V=query(emuDBhandle = ae,query = "Phonetic==V"))
```
Now put this segment list into requery_hier() and look for the linked `ITEM` in `Word:Text`:
```{r}
requery_hier(emuDBhandle = ae,seglist = V,level = "Text")
```
Your result will be the `ITEM` labels and calculated times (for the corresponding words).

You could also wish to know what "V"s contexts are, e.g. the subsequent segments. We use the sequential structure of the database, and the command `requery_seq()`:
```{r}
requery_seq(emuDBhandle = ae,seglist = V,offset = 1)
```

We will discuss both commands more extensively later in the seminar, but wanted to show that it is possible to use the annotation structure and a given segment list to retrieve additional information afterwards. We could use both commands to express more complex queries: e.g. we could look for all "V" within the word "amongst" by querying "V", then requery all linked words, and then deletin all "V" that are not linked to "amongst". However, this would be rather cumbersome. A much easier way to conduct more complicated queries is the use of all possibilities of emuR's query language `EQL` within the command `query`.

#The **E**mu **Q**uery **L**anguage `EQL`

To learn about the functionality of the `EQL`, you can always type
```{r, eval=FALSE}
vignette("EQL")
```
As we have seen above, any query must be placed within " ". You minimally have to give a level, and some sort of representation for a label (this may be a regular expression), unless you do not use one of the `position` and `count` functions. 

###Single argument queries
####Equality/inequality/matching/non-matching
In the examples above, we had looked for the equality of the labels to "V" on the level "Phonetic" (in the database `ae`):
```{r}
query(emuDBhandle = ae, query = "Phonetic == V")
```
So "==" is the equality operator. For backward compatibility, a single "=" is also allowed (but we ask you to prefer "==" instead):
```{r}
query(emuDBhandle = ae, query = "Phonetic = V")
```

We can also search everything *except* "V" by the use of `!=`

```{r, eval = FALSE}
query(emuDBhandle = ae, query = "Phonetic != V")
```
(We do not show the resulting segment list, because it is very long.) So one way to get 'everything' would be to query something that is not in your database, like "xyz". However, there is a much better way: Using regular expressions. To use these, you have to type "=~":

```{r}
Everything1 = query(emuDBhandle = ae, query = "Phonetic != xyz")
Everything2 = query(emuDBhandle = ae, query = "Phonetic =~ .*")
any(Everything1 != Everything2) # should result in FALSE if both are equal everywhere
```

You can also negate the latter operator by "!~". A not very useful example would be:
```{r}
(Nothing = query(emuDBhandle = ae, query = "Phonetic !~ .*"))
```
A more interesting example would be:
```{r, eval=FALSE}
# What is the query to retrieve all ITEMs in the “Text” level that don’t begin with ‘a’?
query(emuDBhandle = ae, query = "Text !~ a.*")
```

So, there are four similar operators, two for equality matching, and two for inequalitiy:

| Symbol | Meaning |
| :------------------ | :--------------------------------------------------------------------------------------|
|`==`|	equality|
|`=~`|	regular expression matching|
|`!=`|	inequality|
|`!~`|	regular expression non-matching|

#### The `OR` operator
Use "|" to look for one label and another one(s), e.g.
```{r}
query(emuDBhandle = ae, query = "Phonetic == m|n")
```
You can expand this as well:
```{r}
mnN = query(emuDBhandle = ae, query = "Phonetic == m|n|N")
summary(mnN)
```

##Complex queries
###Sequencial and dominance queries
####Bracketing
In all hierarchical queries, bracketing with "[" and "]" is required to structure your query. In simple queries, however, brackets are optional.
```{r, error=FALSE}
mnN = query(emuDBhandle = ae, query = "[Phonetic == m|n|N]")
summary(mnN)
```
However, this sequential query would fail, because of missing brackets:
```{r, eval = FALSE}
query(ae, "Phonetic == V -> Phonetic == m")
```

####Sequential queries
Use the "->" operator to find sequences of segments:
```{r, eval = TRUE}
query(ae, "[Phonetic == V -> Phonetic == m]")
```
Note: all row entries in the resulting segment list have the start time of "V", the end time of "m" and their labels will be "V->m". Change this with the so-called `result modifier` hash tag "#":
```{r, eval = TRUE}
query(ae, "[#Phonetic == V -> Phonetic == m]")
query(ae, "[Phonetic == V -> #Phonetic == m]")
```
Keep in mind that only one hash tag per query is allowed.

You can search sequences of sequences, however, you have to use bracketing; otherwise, you get an error

```{r, eval=FALSE}
query(ae, "[Phonetic == @ -> Phonetic == n  -> Phonetic == s]")
```

The correct code would be as follows:

```{r}
query(ae, "[[Phonetic == @ -> Phonetic == n ] -> Phonetic == s]")
```

```{r}
## What is the query to retrieve all sequences of ITEMs containing labels “offer” followed by two arbitrary labels followed by “resistance”?
query(ae, "[[[Text == offer -> Text =~ .*] -> Text =~ .* ] -> Text == resistance]")
```

####Domination queries
Use the operator "^" for all queries, in which two linked levels are involved.

```{r}
## What is the query to retrieve all ITEMs containing the label “p” in the “Phoneme” level that occur in strong syllables (i.e. dominated by / linked to ITEMs of the level “Syllable” that contain the label “S”)?
query(ae, "[Phoneme == p ^ Syllable == S]")
```
However, the operator is *not* directional; although "Syllable" dominates "Phoneme", you could have asked
```{r}
query(ae, "[Syllable == S ^ #Phoneme == p]")
```
So, "^" should not be translated with "is dominated by". However, you have to use the hash tag in order to get labels and times of the Phoneme level here. You can leave out the hash tag if the level you are interested in is the first one in your question.

You can query multiple dominations, however, like in the sequencing case, you have to use brackets:
```{r}
## What is the query to retrieve all ITEMs on the “Phonetic” level that are part of a strong syllable (labeled “S”) and belong to the words “amongst” or “beautiful”?
query(ae, "[[#Phonetic =~ .* ^ Syllable == S] ^ Text == amongst | beautiful]")
```
```{r, eval=FALSE}
# same as
query(ae, "[[Phonetic =~ .* ^ Syllable == S] ^ Text == amongst | beautiful]")
```
```{r}
## to get the "Text"-items instead, use
query(ae, "[[Phonetic =~ .* ^ Syllable == S] ^ #Text == amongst | beautiful]")
```




###Functions
The are three **position** and one **count** functions. As the latter function results in a number, queries involve a comparison with a number (by using one of "==", "!=", ">", ">=", "<", "<=", see below); The result of the position functions is logical; we therefore ask, whether a certain condition is `TRUE` or `FALSE`.

#### Position functions
There are three position functions, `Start()`, `Medial()`, and `End()`. Example queries are:
```{r}
## What is the query to retrieve all word-initial syllables?
query(ae, "[Start(Text, Syllable) == TRUE]", bundlePattern = "msajc003")

## be careful with the direction of the arguments; the following gives a result, which makes no sense (the word label in Text at every Start of a syllable):
query(ae, "[Start(Syllable, Text) == TRUE]", bundlePattern = "msajc003")
```

Examples for Medial() and End() are:

```{r}
## What is the query to retrieve all word-medial syllables?
query(ae, "[Medial(Word, Syllable) == TRUE]", bundlePattern = "msajc003")
## What is the query to retrieve all word-final syllables?
query(ae, "[End(Word, Syllable) == TRUE]", bundlePattern = "msajc003")
```
Everything not being first or last element is medial:
```{r}
query(ae, "[Medial(Word, Phoneme) == TRUE]", bundlePattern = "msajc003")
```

#### Count function
The count function's name is `Num()`. `Num(x,y)` counts, how many y are in x. You can therefore ask things like the following:
```{r}
## What is the query to retrieve all words that contain two syllables?
query(ae, "[Num(Text, Syllable) == 2]", bundlePattern = "msajc003")
## What is the query to retrieve all syllables that contain more than four phonemes?
query(ae, "[Num(Syllable, Phoneme) > 4]", bundlePattern = "msajc003")
```

###Conjunction
You can use "&" to search within several attribute definitions on the same level. For example, the level Word in `ae` has several attribute definitions
```{r}
list_attributeDefinitions(ae,level="Word")
```
We could, therefore, look for all accented ("S") words by ...
```{r}
query(ae, "[Text =~.* & Accent == S]")
```
Another usage of "&" is to combine a basic query with a function, e.g.
```{r}
## What is the query to retrieve all non-word-final “S” syllables?
query(ae, "[Syllable == S & End(Word, Syllable) == FALSE]", bundlePattern = "msajc003")
```