We will use the EMU-SDMS-demo-database which we had created last week:

# load packages
library(emuR)
library(dplyr)
library(ggplot2)
# create path to demo database
path2ae = file.path(mypath, "emuR_demoData", "ae_emuDB")
# load database
ae = load_emuDB(path2ae, verbose = F)

In the last chapter, we were interested in querying the rather complex hierarchy that can be seen at “Link definitions”. Today, we are interested in derived signals, i.e. in signals dervied from the signal, with which every utterance is associated with (this is usually an audio recording, but we could also have EMA-data, eletropalatographic data, and so on, possibly without any audio data at all). In an EMU-SMDS-data-base, derived data is stored in the so-called SSFF file format. SSFF stands for Simple Signal File Format. In the so-called SSFF track definitions, we can see two definitions, one for so-called dft-data, one for fm-data. To see these definitions, we could also have typed:

list_ssffTrackDefinitions(ae)
##   name columnName fileExtension
## 1  dft        dft           dft
## 2   fm         fm           fms

Be informed, that these data are signals derived from the audio data, and represent spectral analyses in the dft-tracks and calculated formants (and their bandwidths) in the fm-tracks. Keep in mind that every signal except audio data is stored in this format, so e.g. the EMA-tracks in the demo-data-bases ema or the electropalatographic data in the demo-data-base epgdorsal are stored in the SSFF format. See e.g. the aforementioned demo-data-bases in https://ips-lmu.github.io/EMU-webApp/ (open demo – ema/epgdorsal).

As the SSFF track definition informs us, these tracks are obviously saved in extra files, defined by their fileExtension, e.g. ‘dft’ or ‘fms’. If we open one of these files, we will notice, that we cannot see the data directly (because it is binary data), but we can read the header, e.g. in the case of the fms-files:

## SSFF -- (c) SHLRC
## Machine IBM-PC
## Record_Freq 200.0
## Start_Time 0.0025
## Column fm SHORT 4
## Column bw SHORT 4
## Original_Freq DOUBLE 20000.0
## -----------------

This means, that every fms-file contains four columns with formant values (recorded every 5 ms - as can be derived from the Record_Freq information) alongside with 4 columns which contain the bandwidths of the first four formants. We can read in these data into R with the following commands. First of all, we have to do a query, e.g. for the vowel /i:/, and then use the command get_trackdata():

# query loaded "ae" emuDB for all "i:" segments of the "Phonetic" level
sl = query(emuDBhandle = ae, 
           query = "Phonetic == i:")

# get the corresponding formant trackdata
ae_i_fm = get_trackdata(emuDBhandle = ae, 
                   seglist = sl, 
                   ssffTrackName = "fm")

In case there aren’t any pre-calculated derived signals, we will have to calculate these. We can do this on-the-fly (within the get_trackdata() function), or we can precalculate derived signals (in the function add_ssffTrackDefinition()). In both cases, we will actually use the package wrassp. wrassp was installed, when we installed the package emuR. The functions of wrassp can be used independently of emuR, but it is way easier to use the integration of wrassp functions into functions of package emuR.

The R package wrassp1

This passage gives an overview and introduction to the wrassp package. The wrassp package is a wrapper for R around Michel Scheffers’ libassp (Advanced Speech Signal Processor). The libassp library and therefore the wrassp package provide functionality for handling speech signales in most common audio formats and for performing signal analyses common in the phonetic and speech sciences. As such, wrassp fills a gap in the R package landscape as, to our knowledge, no previous packages provided this specialized functionality. The currently available signal processing functions provided by wrassp are:

Command Meaning
acfana() Analysis of short-term autocorrelation function
afdiff() Computes the first difference of the signal
affilter() Filters the audio signal (e.g., low-pass and high-pass)
cepstrum() Short-term cepstral analysis
cssSpectrum() Cepstral smoothed version of dftSpectrum()
dftSpectrum() Short-term DFT spectral analysis
forest() Formant estimation
ksvF0() F0 analysis of the signal
lpsSpectrum() Linear predictive smoothed version of dftSpectrum()
mhsF0() Pitch analysis of the speech signal using Michel Scheffers’ Modified Harmonic Sieve algorithm
rfcana() Linear prediction analysis
rmsana() Analysis of short-term Root Mean Square amplitude
zcrana() Analysis of the averages of the short-term positive and negative zero-crossing rates

As mentioned earlier, you could use these functions independently of the emuR package; however, we would like to advise you to use them as parameters in the emuR functions get_trackdata() and add_ssffTrackDefinition().

Extracting pre-defined tracks

To access data that are stored in files, the user has to define tracks for a database that point to sequences of samples that match a user-specified file extension. The user-defined name of such a track can then be used to reference the track in the signal data extraction process. Internally, emuR uses wrassp to read the appropriate files from disk, extract the sample sequences that match the result of a query and return values to the user for further inspection and evaluation.

# list currently available tracks
list_ssffTrackDefinitions(ae)
##   name columnName fileExtension
## 1  dft        dft           dft
## 2   fm         fm           fms

In ae, there are three tracks available, that can be read by get_trackdata(). We could e.g.

# query all "ai" phonetic segments
ai_segs = query(ae, "Phonetic == ai")
# get "fm" track data for these segments
# Note that verbose is set to FALSE
# only to avoid a progress bar
# being printed in this document.
ai_td_fm = get_trackdata(emuDBhandle = ae,
                         seglist = ai_segs,
                         ssffTrackName = "fm",
                         verbose = FALSE,
                         resultType = "tibble")
# show summary of ai_td_fm
ai_td_fm
## # A tibble: 183 x 24
##    sl_rowIdx labels start   end utts  db_uuid session bundle start_item_id
##        <int> <chr>  <dbl> <dbl> <chr> <chr>   <chr>   <chr>          <int>
##  1         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  2         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  3         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  4         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  5         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  6         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  7         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  8         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  9         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
## 10         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
## # ... with 173 more rows, and 15 more variables: end_item_id <int>,
## #   level <chr>, start_item_seq_idx <int>, end_item_seq_idx <int>,
## #   type <chr>, sample_start <int>, sample_end <int>, sample_rate <int>,
## #   times_orig <dbl>, times_rel <dbl>, times_norm <dbl>, T1 <int>,
## #   T2 <int>, T3 <int>, T4 <int>

So, we needed an emuDBhandle, a seglist, and the correct ssffTrackName to read the formant values from the fms-files. Being able to access data that is stored in files is important for two main reasons.

Firstly, it is possible to generate files using external programs such as VoiceSauce (Shue et al., 2011), which can export its calculated output to the general purpose SSFF file format. This file mechanism is also used to access data produced by EMA, EPG or any other form of signal data recordings. Secondly, it is possible to track, save and access manipulated data such as formant values that have been manually corrected. It is also worth noting that the get trackdata() function has a predefined track which is always available without it having to be defined. The name of this track is MEDIAFILE SAMPLES which references the actual samples of the audio files of the database. The next example shows how this predefined track can be used to access the audio samples belonging to the segments in ai_segs.

# get media file samples
ai_td_mfs = get_trackdata(ae,
        seglist = ai_segs,
        ssffTrackName = "MEDIAFILE_SAMPLES",
        verbose = FALSE,
        resultType = "tibble")
# plot ai_td_mfs$T1
ggplot(ai_td_mfs) + aes(y = T1, x = times_rel) + geom_line() + facet_wrap(~sl_rowIdx)

Adding new tracks

The signal processing routines provided by the wrassp package can be used to produce SSFF files containing various derived signal data (e.g., formants, fundamental frequency, etc.). The following example shows how the function add_ssffTrackDefinition() can be used to add a new track to the ae emuDB. Using the onTheFlyFunctionName parameter, the add_ssffTrackDefinition() function automatically executes the wrassp signal processing function ksvF0 (onTheFlyFunctionName= "ksvF0") and stores the results in SSFF files in the bundle directories.

# add new track and calculate
# .f0 files on-the-fly using wrassp::ksvF0()
add_ssffTrackDefinition(ae,
                        name = "F0",
                        onTheFlyFunctionName = "ksvF0",
                        verbose = FALSE)
# show newly added track
list_ssffTrackDefinitions(ae)
# show newly added files
list_files(ae, fileExtension = "f0")
# extract newly added trackdata
ai_td = get_trackdata(ae,
                      seglist = ai_segs,
                      ssffTrackName = "F0",
                      verbose = FALSE,
                      resultType = "tibble")

In the command add_ssffTrackDefinition(), we could have also added parameter values to the wrassp function ksvF0.

formals(wrassp::ksvF0)
## $listOfFiles
## NULL
## 
## $optLogFilePath
## NULL
## 
## $beginTime
## [1] 0
## 
## $endTime
## [1] 0
## 
## $windowShift
## [1] 5
## 
## $gender
## [1] "u"
## 
## $maxF
## [1] 600
## 
## $minF
## [1] 50
## 
## $minAmp
## [1] 50
## 
## $maxZCR
## [1] 3000
## 
## $toFile
## [1] TRUE
## 
## $explicitExt
## NULL
## 
## $outputDirectory
## NULL
## 
## $forceToLog
## useWrasspLogger
## 
## $verbose
## [1] TRUE

We can see, that the parameter gender is by default on “u” (undefined). As we know, that the ae data-base consists of male speech only, we could set this to “m” (male) in onThe FlyParams:

add_ssffTrackDefinition(ae,
                        name = "F0",
                        onTheFlyFunctionName = "ksvF0",
                        onTheFlyParams = list(gender = "m"),
                        verbose = FALSE)

Pre-calculated dervied signals can be shown and corrected in the EMU-webApp. Pre-calculated formants may be overlaid to the spectrogram. Learn more about this in chapter 08.

One disadvantage of this method may not be withholded: it is – until now – not possible to pre-calculate speaker-group-(e.g. gender)-specific data by setting different parameters for various speaker-groups. However, implementation of this feature is in the making.

Calculating tracks on-the-fly

The user is able to select one of the signal processing routines provided by wrassp and pass it on to the signal data extraction function. The signal data extraction function can then apply this wrassp function to each audio file as part of the signal data extraction process. This means that the user can quickly manipulate function parameters and evaluate the result without having to store to disk the files that would usually be generated by the various parameter experiments. In many cases this new functionality eliminates the need for defining a track definition for the entire database for temporary data analysis purposes. The following example shows how the onTheFlyFunctionName parameter of the get trackdata() function is used:

ai_td_pit = get_trackdata(ae,
seglist = ai_segs,
onTheFlyFunctionName = "mhsF0",
              verbose = FALSE,
              resultType = "tibble")
# show head of ai_td
ai_td_pit
## # A tibble: 183 x 21
##    sl_rowIdx labels start   end utts  db_uuid session bundle start_item_id
##        <int> <chr>  <dbl> <dbl> <chr> <chr>   <chr>   <chr>          <int>
##  1         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  2         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  3         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  4         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  5         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  6         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  7         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  8         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
##  9         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
## 10         1 ai      863. 1016. 0000… 0fc618… 0000    msajc…           161
## # ... with 173 more rows, and 12 more variables: end_item_id <int>,
## #   level <chr>, start_item_seq_idx <int>, end_item_seq_idx <int>,
## #   type <chr>, sample_start <int>, sample_end <int>, sample_rate <int>,
## #   times_orig <dbl>, times_rel <dbl>, times_norm <dbl>, T1 <dbl>

Reconsidering last week’s example:

# query A and V(front and back open vowels),
# i:and u: (front and back closed vowels), and
# E and o: (front and back mid vowels)
ae_vowels = query(emuDBhandle = ae,query = "[Phonetic== V|A|i:|u:|o:|E]")
#get the formants:
ae_formants = get_trackdata(ae, seglist = ae_vowels,ssffTrackName = "fm", resultType = "tibble")

Please never forget to set the resultType parameter of the get trackdata() function to tibble!

The emuRtrackdata object of type tibble is an amalgamation of both a segment list and trackdata. The first sl_rowIdx column of the ae_formants object indicates the row index of the segment list the current row belongs to,

ae_formants$sl_rowIdx
##   [1]  1  1  1  1  1  1  1  1  1  1  1  1  1  1  2  2  2  2  2  2  2  2  2
##  [24]  2  2  2  2  2  2  2  2  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3
##  [47]  3  4  4  4  4  4  4  4  4  4  5  5  5  5  5  5  5  5  5  5  5  5  5
##  [70]  5  5  6  6  6  6  6  6  6  6  6  6  6  6  6  7  7  7  7  7  7  7  7
##  [93]  7  7  7  7  7  7  7  7  7  7  8  8  8  8  8  8  8  8  8  8  8  8  8
## [116]  8  8  8  9  9  9  9  9  9  9  9  9  9  9  9  9  9  9 10 10 10 10 10
## [139] 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 11
## [162] 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11
## [185] 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 13 13 13 13 13 13 13 13
## [208] 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 15
## [231] 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 16 16 16 16 16 16
## [254] 16 16 16 16 16 16 16 16 16 16 16 17 17 17 17 17 17 17 17 17 17 17 17
## [277] 17 17 17 17 17 17 17 17 17 17 17 18 18 18 18 18 18 18 18 18 18 18 18
## [300] 18 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19
## [323] 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
## [346] 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 22 22 22 22
## [369] 22 22 22 22 22 22 22 22 22 22 22 22 23 23 23 23 23 23 23 23 23 23 23
## [392] 23 23 23 23 23 23 23 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24
## [415] 24 24 24 24 24 24 24 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25
## [438] 25 25 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26
## [461] 26

the times_rel, times_orig and times_norm columns represent the relative time,

ae_formants$times_rel
##   [1]   0   5  10  15  20  25  30  35  40  45  50  55  60  65   0   5  10
##  [18]  15  20  25  30  35  40  45  50  55  60  65  70  75  80   0   5  10
##  [35]  15  20  25  30  35  40  45  50  55  60  65  70  75   0   5  10  15
##  [52]  20  25  30  35  40   0   5  10  15  20  25  30  35  40  45  50  55
##  [69]  60  65  70   0   5  10  15  20  25  30  35  40  45  50  55  60   0
##  [86]   5  10  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85
## [103]   0   5  10  15  20  25  30  35  40  45  50  55  60  65  70  75   0
## [120]   5  10  15  20  25  30  35  40  45  50  55  60  65  70   0   5  10
## [137]  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85  90  95
## [154] 100 105 110 115 120 125 130   0   5  10  15  20  25  30  35  40  45
## [171]  50  55  60  65  70  75  80  85  90  95 100 105 110 115   0   5  10
## [188]  15  20  25  30  35  40  45  50  55  60  65  70   0   5  10  15  20
## [205]  25  30  35  40  45  50  55  60  65   0   5  10  15  20  25  30  35
## [222]  40  45  50  55  60  65  70  75   0   5  10  15  20  25  30  35  40
## [239]  45  50  55  60  65  70  75  80  85   0   5  10  15  20  25  30  35
## [256]  40  45  50  55  60  65  70  75  80   0   5  10  15  20  25  30  35
## [273]  40  45  50  55  60  65  70  75  80  85  90  95 100 105 110   0   5
## [290]  10  15  20  25  30  35  40  45  50  55  60   0   5  10  15  20  25
## [307]  30  35  40  45  50  55  60  65  70  75  80  85  90  95 100 105   0
## [324]   5  10  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85
## [341]  90  95 100 105 110   0   5  10  15  20  25  30  35  40  45  50  55
## [358]  60  65  70  75  80  85  90   0   5  10  15  20  25  30  35  40  45
## [375]  50  55  60  65  70  75   0   5  10  15  20  25  30  35  40  45  50
## [392]  55  60  65  70  75  80  85   0   5  10  15  20  25  30  35  40  45
## [409]  50  55  60  65  70  75  80  85  90  95 100 105 110   0   5  10  15
## [426]  20  25  30  35  40  45  50  55  60  65  70  75  80  85   0   5  10
## [443]  15  20  25  30  35  40  45  50  55  60  65  70  75  80  85  90  95
## [460] 100 105

the original time,

ae_formants$times_orig
##   [1]  187.5  192.5  197.5  202.5  207.5  212.5  217.5  222.5  227.5  232.5
##  [11]  237.5  242.5  247.5  252.5  342.5  347.5  352.5  357.5  362.5  367.5
##  [21]  372.5  377.5  382.5  387.5  392.5  397.5  402.5  407.5  412.5  417.5
##  [31]  422.5  952.5  957.5  962.5  967.5  972.5  977.5  982.5  987.5  992.5
##  [41]  997.5 1002.5 1007.5 1012.5 1017.5 1022.5 1027.5 1422.5 1427.5 1432.5
##  [51] 1437.5 1442.5 1447.5 1452.5 1457.5 1462.5 2212.5 2217.5 2222.5 2227.5
##  [61] 2232.5 2237.5 2242.5 2247.5 2252.5 2257.5 2262.5 2267.5 2272.5 2277.5
##  [71] 2282.5  737.5  742.5  747.5  752.5  757.5  762.5  767.5  772.5  777.5
##  [81]  782.5  787.5  792.5  797.5 1132.5 1137.5 1142.5 1147.5 1152.5 1157.5
##  [91] 1162.5 1167.5 1172.5 1177.5 1182.5 1187.5 1192.5 1197.5 1202.5 1207.5
## [101] 1212.5 1217.5 1437.5 1442.5 1447.5 1452.5 1457.5 1462.5 1467.5 1472.5
## [111] 1477.5 1482.5 1487.5 1492.5 1497.5 1502.5 1507.5 1512.5 1557.5 1562.5
## [121] 1567.5 1572.5 1577.5 1582.5 1587.5 1592.5 1597.5 1602.5 1607.5 1612.5
## [131] 1617.5 1622.5 1627.5 1187.5 1192.5 1197.5 1202.5 1207.5 1212.5 1217.5
## [141] 1222.5 1227.5 1232.5 1237.5 1242.5 1247.5 1252.5 1257.5 1262.5 1267.5
## [151] 1272.5 1277.5 1282.5 1287.5 1292.5 1297.5 1302.5 1307.5 1312.5 1317.5
## [161] 2572.5 2577.5 2582.5 2587.5 2592.5 2597.5 2602.5 2607.5 2612.5 2617.5
## [171] 2622.5 2627.5 2632.5 2637.5 2642.5 2647.5 2652.5 2657.5 2662.5 2667.5
## [181] 2672.5 2677.5 2682.5 2687.5  352.5  357.5  362.5  367.5  372.5  377.5
## [191]  382.5  387.5  392.5  397.5  402.5  407.5  412.5  417.5  422.5  427.5
## [201]  432.5  437.5  442.5  447.5  452.5  457.5  462.5  467.5  472.5  477.5
## [211]  482.5  487.5  492.5 1502.5 1507.5 1512.5 1517.5 1522.5 1527.5 1532.5
## [221] 1537.5 1542.5 1547.5 1552.5 1557.5 1562.5 1567.5 1572.5 1577.5 2412.5
## [231] 2417.5 2422.5 2427.5 2432.5 2437.5 2442.5 2447.5 2452.5 2457.5 2462.5
## [241] 2467.5 2472.5 2477.5 2482.5 2487.5 2492.5 2497.5 2877.5 2882.5 2887.5
## [251] 2892.5 2897.5 2902.5 2907.5 2912.5 2917.5 2922.5 2927.5 2932.5 2937.5
## [261] 2942.5 2947.5 2952.5 2957.5  777.5  782.5  787.5  792.5  797.5  802.5
## [271]  807.5  812.5  817.5  822.5  827.5  832.5  837.5  842.5  847.5  852.5
## [281]  857.5  862.5  867.5  872.5  877.5  882.5  887.5 1522.5 1527.5 1532.5
## [291] 1537.5 1542.5 1547.5 1552.5 1557.5 1562.5 1567.5 1572.5 1577.5 1582.5
## [301] 2147.5 2152.5 2157.5 2162.5 2167.5 2172.5 2177.5 2182.5 2187.5 2192.5
## [311] 2197.5 2202.5 2207.5 2212.5 2217.5 2222.5 2227.5 2232.5 2237.5 2242.5
## [321] 2247.5 2252.5  597.5  602.5  607.5  612.5  617.5  622.5  627.5  632.5
## [331]  637.5  642.5  647.5  652.5  657.5  662.5  667.5  672.5  677.5  682.5
## [341]  687.5  692.5  697.5  702.5  707.5 1137.5 1142.5 1147.5 1152.5 1157.5
## [351] 1162.5 1167.5 1172.5 1177.5 1182.5 1187.5 1192.5 1197.5 1202.5 1207.5
## [361] 1212.5 1217.5 1222.5 1227.5  587.5  592.5  597.5  602.5  607.5  612.5
## [371]  617.5  622.5  627.5  632.5  637.5  642.5  647.5  652.5  657.5  662.5
## [381] 1382.5 1387.5 1392.5 1397.5 1402.5 1407.5 1412.5 1417.5 1422.5 1427.5
## [391] 1432.5 1437.5 1442.5 1447.5 1452.5 1457.5 1462.5 1467.5 1712.5 1717.5
## [401] 1722.5 1727.5 1732.5 1737.5 1742.5 1747.5 1752.5 1757.5 1762.5 1767.5
## [411] 1772.5 1777.5 1782.5 1787.5 1792.5 1797.5 1802.5 1807.5 1812.5 1817.5
## [421] 1822.5 1947.5 1952.5 1957.5 1962.5 1967.5 1972.5 1977.5 1982.5 1987.5
## [431] 1992.5 1997.5 2002.5 2007.5 2012.5 2017.5 2022.5 2027.5 2032.5 2482.5
## [441] 2487.5 2492.5 2497.5 2502.5 2507.5 2512.5 2517.5 2522.5 2527.5 2532.5
## [451] 2537.5 2542.5 2547.5 2552.5 2557.5 2562.5 2567.5 2572.5 2577.5 2582.5
## [461] 2587.5

and the normalized time (ranging from 0 to 1) of the samples contained in the current row

ae_formants$times_norm
##   [1] 0.00000000 0.07692308 0.15384615 0.23076923 0.30769231 0.38461538
##   [7] 0.46153846 0.53846154 0.61538462 0.69230769 0.76923077 0.84615385
##  [13] 0.92307692 1.00000000 0.00000000 0.06250000 0.12500000 0.18750000
##  [19] 0.25000000 0.31250000 0.37500000 0.43750000 0.50000000 0.56250000
##  [25] 0.62500000 0.68750000 0.75000000 0.81250000 0.87500000 0.93750000
##  [31] 1.00000000 0.00000000 0.06666667 0.13333333 0.20000000 0.26666667
##  [37] 0.33333333 0.40000000 0.46666667 0.53333333 0.60000000 0.66666667
##  [43] 0.73333333 0.80000000 0.86666667 0.93333333 1.00000000 0.00000000
##  [49] 0.12500000 0.25000000 0.37500000 0.50000000 0.62500000 0.75000000
##  [55] 0.87500000 1.00000000 0.00000000 0.07142857 0.14285714 0.21428571
##  [61] 0.28571429 0.35714286 0.42857143 0.50000000 0.57142857 0.64285714
##  [67] 0.71428571 0.78571429 0.85714286 0.92857143 1.00000000 0.00000000
##  [73] 0.08333333 0.16666667 0.25000000 0.33333333 0.41666667 0.50000000
##  [79] 0.58333333 0.66666667 0.75000000 0.83333333 0.91666667 1.00000000
##  [85] 0.00000000 0.05882353 0.11764706 0.17647059 0.23529412 0.29411765
##  [91] 0.35294118 0.41176471 0.47058824 0.52941176 0.58823529 0.64705882
##  [97] 0.70588235 0.76470588 0.82352941 0.88235294 0.94117647 1.00000000
## [103] 0.00000000 0.06666667 0.13333333 0.20000000 0.26666667 0.33333333
## [109] 0.40000000 0.46666667 0.53333333 0.60000000 0.66666667 0.73333333
## [115] 0.80000000 0.86666667 0.93333333 1.00000000 0.00000000 0.07142857
## [121] 0.14285714 0.21428571 0.28571429 0.35714286 0.42857143 0.50000000
## [127] 0.57142857 0.64285714 0.71428571 0.78571429 0.85714286 0.92857143
## [133] 1.00000000 0.00000000 0.03846154 0.07692308 0.11538462 0.15384615
## [139] 0.19230769 0.23076923 0.26923077 0.30769231 0.34615385 0.38461538
## [145] 0.42307692 0.46153846 0.50000000 0.53846154 0.57692308 0.61538462
## [151] 0.65384615 0.69230769 0.73076923 0.76923077 0.80769231 0.84615385
## [157] 0.88461538 0.92307692 0.96153846 1.00000000 0.00000000 0.04347826
## [163] 0.08695652 0.13043478 0.17391304 0.21739130 0.26086957 0.30434783
## [169] 0.34782609 0.39130435 0.43478261 0.47826087 0.52173913 0.56521739
## [175] 0.60869565 0.65217391 0.69565217 0.73913043 0.78260870 0.82608696
## [181] 0.86956522 0.91304348 0.95652174 1.00000000 0.00000000 0.07142857
## [187] 0.14285714 0.21428571 0.28571429 0.35714286 0.42857143 0.50000000
## [193] 0.57142857 0.64285714 0.71428571 0.78571429 0.85714286 0.92857143
## [199] 1.00000000 0.00000000 0.07692308 0.15384615 0.23076923 0.30769231
## [205] 0.38461538 0.46153846 0.53846154 0.61538462 0.69230769 0.76923077
## [211] 0.84615385 0.92307692 1.00000000 0.00000000 0.06666667 0.13333333
## [217] 0.20000000 0.26666667 0.33333333 0.40000000 0.46666667 0.53333333
## [223] 0.60000000 0.66666667 0.73333333 0.80000000 0.86666667 0.93333333
## [229] 1.00000000 0.00000000 0.05882353 0.11764706 0.17647059 0.23529412
## [235] 0.29411765 0.35294118 0.41176471 0.47058824 0.52941176 0.58823529
## [241] 0.64705882 0.70588235 0.76470588 0.82352941 0.88235294 0.94117647
## [247] 1.00000000 0.00000000 0.06250000 0.12500000 0.18750000 0.25000000
## [253] 0.31250000 0.37500000 0.43750000 0.50000000 0.56250000 0.62500000
## [259] 0.68750000 0.75000000 0.81250000 0.87500000 0.93750000 1.00000000
## [265] 0.00000000 0.04545455 0.09090909 0.13636364 0.18181818 0.22727273
## [271] 0.27272727 0.31818182 0.36363636 0.40909091 0.45454545 0.50000000
## [277] 0.54545455 0.59090909 0.63636364 0.68181818 0.72727273 0.77272727
## [283] 0.81818182 0.86363636 0.90909091 0.95454545 1.00000000 0.00000000
## [289] 0.08333333 0.16666667 0.25000000 0.33333333 0.41666667 0.50000000
## [295] 0.58333333 0.66666667 0.75000000 0.83333333 0.91666667 1.00000000
## [301] 0.00000000 0.04761905 0.09523810 0.14285714 0.19047619 0.23809524
## [307] 0.28571429 0.33333333 0.38095238 0.42857143 0.47619048 0.52380952
## [313] 0.57142857 0.61904762 0.66666667 0.71428571 0.76190476 0.80952381
## [319] 0.85714286 0.90476190 0.95238095 1.00000000 0.00000000 0.04545455
## [325] 0.09090909 0.13636364 0.18181818 0.22727273 0.27272727 0.31818182
## [331] 0.36363636 0.40909091 0.45454545 0.50000000 0.54545455 0.59090909
## [337] 0.63636364 0.68181818 0.72727273 0.77272727 0.81818182 0.86363636
## [343] 0.90909091 0.95454545 1.00000000 0.00000000 0.05555556 0.11111111
## [349] 0.16666667 0.22222222 0.27777778 0.33333333 0.38888889 0.44444444
## [355] 0.50000000 0.55555556 0.61111111 0.66666667 0.72222222 0.77777778
## [361] 0.83333333 0.88888889 0.94444444 1.00000000 0.00000000 0.06666667
## [367] 0.13333333 0.20000000 0.26666667 0.33333333 0.40000000 0.46666667
## [373] 0.53333333 0.60000000 0.66666667 0.73333333 0.80000000 0.86666667
## [379] 0.93333333 1.00000000 0.00000000 0.05882353 0.11764706 0.17647059
## [385] 0.23529412 0.29411765 0.35294118 0.41176471 0.47058824 0.52941176
## [391] 0.58823529 0.64705882 0.70588235 0.76470588 0.82352941 0.88235294
## [397] 0.94117647 1.00000000 0.00000000 0.04545455 0.09090909 0.13636364
## [403] 0.18181818 0.22727273 0.27272727 0.31818182 0.36363636 0.40909091
## [409] 0.45454545 0.50000000 0.54545455 0.59090909 0.63636364 0.68181818
## [415] 0.72727273 0.77272727 0.81818182 0.86363636 0.90909091 0.95454545
## [421] 1.00000000 0.00000000 0.05882353 0.11764706 0.17647059 0.23529412
## [427] 0.29411765 0.35294118 0.41176471 0.47058824 0.52941176 0.58823529
## [433] 0.64705882 0.70588235 0.76470588 0.82352941 0.88235294 0.94117647
## [439] 1.00000000 0.00000000 0.04761905 0.09523810 0.14285714 0.19047619
## [445] 0.23809524 0.28571429 0.33333333 0.38095238 0.42857143 0.47619048
## [451] 0.52380952 0.57142857 0.61904762 0.66666667 0.71428571 0.76190476
## [457] 0.80952381 0.85714286 0.90476190 0.95238095 1.00000000

and T1 (to Tn in n dimensional trackdata) contains the actual signal sample values.

ae_formants$T1 # =first formant frequency
##   [1]   0 628 655 724 717 715 711 688 625 535 496 502 458 440 454 523 568
##  [18] 607 618 643 665 638 584 524 493 490 484 474 455 435 417 422 434 447
##  [35] 449 445 439 424 412 423 433 415 410 408 421 427 417 297 290 288 288
##  [52] 287 287 295 292 290 306 310 314 319 324 330 334 341 349 355 356 352
##  [69] 342 334 303 305 314 325 333 339 341 342 342 341 338 332 323 308 196
##  [86] 207 215 232 245 259 271 275 278 285 290 297 297 305 316 332 352 370
## [103] 330 333 333 330 335 340 352 352 348 342 336 327 314 308 286 263 284
## [120] 285 288 284 289 294 300 298 293 266 251 239 227 205 199 244 277 306
## [137] 329 337 340 343 345 349 357 364 371 377 380 381 382 382 380 379 375
## [154] 369 362 350 339 323 301 277 339 341 347 344 339 364 349 350 334 311
## [171] 338 329 321 295 261 317 296 239 213 195 187 162   0   0 153 184 240
## [188] 266 272 263 253 245 246 249 252 257 266 279 291 313 338 368 392 412
## [205] 419 420 424 425 423 419 419 412 388 419 443 456 458 473 467 416 374
## [222] 367 377 378 354 371 402 416 417 289 305 317 326 327 323 321 321 320
## [239] 320 319 316 308 298 296 300 305 311 284 302 320 328 335 335 331 328
## [256] 320 304 286 269 251 230 194 174 176 367 371 376 384 389 388 383 377
## [273] 373 367 361 357 355 354 352 352 353 355 355 356 356 354 351 211 286
## [290] 371 381 341 345 359 364 378 388 366 356 331 509 537 560 571 576 594
## [307] 625 635 633 632 629 625 612 616 609 577 578 567 572 591 570 438 120
## [324] 133 140 148 159 181 269 270 272 301 333 360 378 387 398 409 406 398
## [341] 385 363 326 296 279 339 385 406 423 432 438 443 454 463 470 473 476
## [358] 477 466 447 433 418 396 355 287 289 291 293 295 297 298 298 298 297
## [375] 296 295 292 289 284 281 519 517 539 573 577 569 580 585 570 563 554
## [392] 530 475 444 352 317 287 267 286 301 307 312 320 325 330 331 333 339
## [409] 352 374 396 411 417 415 409 395 370 337 305 272 263 408 434 481 530
## [426] 550 581 633 662 690 707 712 708 687 635 535   0 150   0 291 353 405
## [443] 423 434 444 449 456 462 467 461 456 455 459 463 462 452 440 428 400
## [460] 348 278
ae_formants$T2 # =second formant frequency
##   [1] 1293 1306 1303 1283 1266 1244 1229 1214 1189 1173 1146 1136 1121 1111
##  [15] 1156 1187 1201 1222 1239 1265 1278 1283 1288 1296 1314 1329 1340 1358
##  [29] 1370 1375 1414 1613 1651 1686 1703 1712 1730 1747 1765 1812 1845 1844
##  [43] 1846 1808 1793 1752 1728 2041 2034 2044 1998 1727 1633 1600 1478 1339
##  [57] 1985 1954 1901 1850 1783 1730 1693 1680 1663 1642 1596 1567 1535 1510
##  [71] 1471 1945 1907 1879 1864 1835 1780 1739 1723 1716 1700 1657 1576 1532
##  [85] 1227 1255 1258 1248 1222 1176 1118  999  950  923  901  892  861  852
##  [99]  835  826  827  831 1120 1147 1173 1200 1229 1257 1295 1335 1361 1407
## [113] 1456 1493 1531 1605 1661 1625 1927 2043 2185 2255 2270 2286 2285 2273
## [127] 2266 2244 2213 2162 2121 2016 1910  928  903  884  862  833  816  798
## [141]  768  745  745  750  760  779  801  819  832  841  849  855  860  865
## [155]  880  909  930  940  940  950 1307 1367 1425 1454 1515 1555 1673 1754
## [169] 1773 1825 1846 1878 1927 1935 1950 1968 1984 2031 2107 2154 2069 2049
## [183] 2053 2030 2362 2249 2182 2180 2176 2210 2262 2265 2254 2251 2252 2254
## [197] 2252 2226 2161 2114 2068 2035 1998 1958 1921 1883 1841 1795 1740 1682
## [211] 1594 1515 1426 1567 1611 1668 1737 1792 1836 1861 1884 1915 1942 1978
## [225] 1995 1963 1923 1859 1925 1517 1550 1605 1677 1741 1785 1834 1863 1894
## [239] 1925 1940 1946 1943 1921 1888 1822 1692 1586  970 1077 1227 1314 1403
## [253] 1473 1553 1725 1835 1914 2012 2078 2103 2140 2197 2197 2115  904  882
## [267]  855  835  817  801  790  783  777  772  763  737  706  691  685  676
## [281]  669  673  683  692  699  709  730 1747 1773 1776 1794 1796 1795 1794
## [295] 1786 1783 1785 1758 1665 1567 1435 1467 1504 1530 1564 1567 1572 1610
## [309] 1628 1630 1636 1638 1636 1641 1661 1661 1648 1636 1640 1638 1652 2314
## [323] 1708 1696 1761 2025 2113 2068 1942 1931 1920 1881 1870 1874 1865 1852
## [337] 1848 1844 1851 1865 1877 1882 1877 1854 1830 1578 1633 1672 1713 1742
## [351] 1757 1758 1757 1754 1750 1745 1744 1739 1712 1684 1667 1651 1624 1598
## [365] 2049 2059 2058 2034 2011 1998 1988 1955 1868 1817 1793 1775 1752 1727
## [379] 1701 1671 1481 1495 1505 1521 1546 1566 1584 1609 1636 1660 1688 1719
## [393] 1760 1779 1794 1804 1795 1772  751  748  742  730  719  712  700  683
## [407]  681  694  710  723  738  757  777  793  809  823  843  867  870  860
## [421]  866 1489 1472 1440 1411 1389 1371 1358 1342 1326 1311 1300 1298 1288
## [435] 1274 1268 1223 1192 1227 1485 1547 1577 1605 1629 1660 1688 1704 1713
## [449] 1717 1718 1714 1705 1693 1674 1648 1611 1564 1515 1470 1422 1376

ae_formants$labels contains the phonetic symbols:

ae_formants$labels 
##   [1] "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V" 
##  [15] "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V" 
##  [29] "V"  "V"  "V"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
##  [43] "E"  "E"  "E"  "E"  "E"  "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
##  [57] "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:"
##  [71] "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:"
##  [85] "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:"
##  [99] "u:" "u:" "u:" "u:" "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [113] "E"  "E"  "E"  "E"  "E"  "E"  "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [127] "i:" "i:" "i:" "i:" "i:" "i:" "i:" "o:" "o:" "o:" "o:" "o:" "o:" "o:"
## [141] "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:"
## [155] "o:" "o:" "o:" "o:" "o:" "o:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [169] "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [183] "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [197] "i:" "i:" "i:" "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [211] "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [225] "E"  "E"  "E"  "E"  "E"  "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [239] "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:"
## [253] "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "i:" "o:" "o:"
## [267] "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:"
## [281] "o:" "o:" "o:" "o:" "o:" "o:" "o:" "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [295] "E"  "E"  "E"  "E"  "E"  "E"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A" 
## [309] "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A" 
## [323] "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [337] "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [351] "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [365] "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:" "u:"
## [379] "u:" "u:" "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A"  "A" 
## [393] "A"  "A"  "A"  "A"  "A"  "A"  "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:"
## [407] "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:" "o:"
## [421] "o:" "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V"  "V" 
## [435] "V"  "V"  "V"  "V"  "V"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E" 
## [449] "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"  "E"

We can now plot the tracked formants, e.g. F2. as a function of time (of relative time, i.e. $times_rel). We can use plotting functions of the package ggplot2:

ggplot(ae_formants) +
  aes(x=times_rel,y=T2,col=labels) +
  geom_line()