Thumbnails: Working with Gesture Annotation
CLAN thumbnail files for multimodal transcription. Transcripts that include annotations of social actions that are not encoded in the verbal modality can be both difficult to produce, and incredibly challenging to read. Eye-gaze, gestures, posture-shifts, features oriented to in the local environment etc may all be relevant to the unfolding interaction, but describing such actions within the main body of the transcript can lead to a loss of ‘readability’.
The movie thumbnail feature in CLAN allows you to embed secondary transcription files within the main transcript, which can be activated when a reader/analyst chooses to look at a sequence in greater detail, but without the amount of detail always being visible in the transcript.
This is how you do it.
1 Select your sequence. In your CLAN transcript, find the line/sequence you wish to add a secondary annotation to (for example, a gesture you would like to highlight).
30 *ROS: i-i was hoping (0.2) i’ve i’ve worn them for you basically
31 i was hoping that you ⌈would⌉
32 *FRA: ⌊but some⌋ questions tell more about yourself
33 ⌈⁇as they:⁇⌉
34 *ROS: ⌊ah::⌋
35 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉
36 *ROS: ⌊go on as-⌋ ask me anything then
37 huh ⌈huh heh heh hhh hah hah⌉
38 *FRA: ⌊·hhhh er:: huhuh what about⌋ the colour red
39 *ROS: ·hhhh ah:: now that’s an interesting one
40 erm it’s definitely my favorite colour ehuh huh but (0.4)
2 Create a .cut file. Start by opening a new file in CLAN,
a. give it a name that refers to the sequence you’ve selected, save it to the same folder
b. but save as ‘.cut’ rather than ‘.cha’ (Example ‘Griffin iconic.cut’)
c. Add the header @Media, followed by a tab. Then add the name of the video file (without the .mov extension), followed by a comma, a space and ‘video’ (this should be the same media file as in the main transcript file).
d. Mode>Transcribe audio or video (or F5). This will play the
movie file. Click the main window to stop
e. Mode>Sonic Mode (or esc+0) will display the waveform
f. Note: these steps are exactly the same as how you usually open a CLAN file.
g. On another line, add the header %pic: followed by a Tab and ‘entire sequence’
h. In the waveform, select the sequence you’re going to work on, and add it as a bullet to ‘entire sequence’
1 @Media: JMSH_BLT_FW_20130513_YT_IN, video
2
3 %pic: Entire sequence •
i. Copy and paste the sequence you want to work on, from the main transcript into the .cut file
1 @Media: JMSH_BLT_FW_20130513_YT_IN, video
2
3 %pic: Entire sequence •
4 *ROS: i-i was hoping (0.2) i’ve i’ve worn them for you basically •
5 i was hoping that you ⌈would⌉ •
6 *FRA: ⌊but some⌋ questions tell more about yourself•
7 ⌈⁇as they:⁇⌉ •
8 *ROS ⌊ah::⌋
9 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉ •
10 *ROS: ⌊go on as-⌋ ask me anything then •
11 huh ⌈huh heh heh hhh hah hah⌉ •
12 *FRA: ⌊·hhhh er:: huhuh what about⌋ the colour red •
13 *ROS: hhhh ah:: now that’s an interesting one •
14 erm it’s definitely my favorite colour ehuh huh but (0.4) •
j. Note you can also open the bullets with esc+A. This allows you to adjust the bullets manually, for example if you want to ensure the ‘Entire sequence’ matches the sequence you have copied. Close them again by repeating the command esc+A.
3 %pic: Entire sequence •39279_56826•
4 *ROS: i-i was hoping (0.2) i’ve i’ve worn them for you basically •39279_42344•
5 i was hoping that you ⌈would⌉ •42355_43632•
6 *FRA: ⌊but some⌋ questions tell more about yourself•
7 ⌈⁇as they:⁇⌉ •43614_46725•
8 *ROS ⌊ah::⌋
9 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉ •46725_48037•
10 *ROS: ⌊go on as-⌋ ask me anything then •48026_49732•
11 huh ⌈huh heh heh hhh hah hah⌉ •49709_52437•
12 *FRA: ⌊·hhhh er:: huhuh what about⌋ the colour red •49674_52809•
13 *ROS: hhhh ah:: now that’s an interesting one •52751_54527•
14 erm it’s definitely my favorite colour ehuh huh but (0.4) •54515_56826•
k. You are now ready to add further annotations.
l. Insert a line below the line where you wish to add further annotation, and name the new line according to the type of action you’d like to annotate (%ges: – gesture; %gaz: gaze ; %pos: posture etc.)
m. Add an identifier at the position you choose at starting point (for example F1 for Frank1)
n. Select in the waveform the segment that matches the timing of this gesture, and add a bullet to the identifier (Ctrl-I). If you click on the bullet, it should now play the sequence you’ve selected. Again, you can adjust the bullet manually (esc+A).
6 *FRA: ⌊but some⌋ questions tell more about yourself
7 ⌈⁇as they:⁇⌉
8 *ROS: ⌊ah::⌋
9 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉
10 %ges: F1 •
o. Mark the unfolding gesture depending on the conventions you wish to use (for example: preparation …………; stroke ——–, hold ++++ , retraction,,,,,,,,,,,,,)
6 *FRA: ⌊but some⌋ questions tell more about yourself
7 ⌈⁇as they:⁇⌉
8 *ROS: ⌊ah::⌋
9 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉
10 %ges: F1 • +++++++,,,,,,,,
p. Beneath the copied sequence, you can now add a more detailed description of the actions involved in this sequence.
i. Start with %pic:, followed by a tab, the identifier (for example F1), and the bullet (these can simply be copy and pasted from above).
ii. Then add whichever lines of description you care to. In this example included, they consist of Action, Gaze, Classification, Meaning.
29 %pic: F1 •
30 action: two handed reach forward, retracted to rest on lap
31 gaz: to ROS
32 Classification: deictic
33 meaning: you
iii. Note: this example has only one gesture described. A sequence can contain as many as you choose. For the others, simply repeat the above procedures.
q. You have now created your .cut file. Save and return to the main transcription.
3 Embedding the .cut file in the main transcript.
a. In the main transcript, insert a line immediately prior to the sequence, and give it the header @T: followed by a Tab, and add an identifier (for example, ‘deictic’, ‘ROS gesture’, or ‘The Guardian’).
b. Add a bullet (Ctrl+I) (Note: make sure you are not in Sonic Mode). A window will pop up to ask you for the file you wish to link to. Choose the .cut file
29 @T: FW-RG-echoic-gesture-sequence-1 •
30 *ROS: i-i was hoping (0.2) i’ve i’ve worn them for you basically •
31 i was hoping that you ⌈would⌉ •
32 *FRA: ⌊but some⌋ questions tell more about yourself •
33 ⌈⁇as they:⁇⌉ •
34 *ROS: ⌊ah::⌋
35 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉ •
36 *ROS: ⌊go on as-⌋ ask me anything then •
c. If you would like to check that the bullet now contains a link to the right file, use esc+A to open the bullets. Close them with the same command.
29 @T: FW-RG-echoic-gesture-sequence-1 •%txt:”fw-rg-echoic-gesture-sequence-1″•
30 *ROS: i-i was hoping (0.2) i’ve i’ve worn them for you basically •39279_42344•
31 i was hoping that you ⌈would⌉ •42355_43632•
32 *FRA: ⌊but some⌋ questions tell more about yourself
33 ⌈⁇as they:⁇⌉ •43614_46725•
34 *ROS: ⌊ah::⌋
35 *FRA: ehuh huh huh huh huh ⌈huh yeah⌉ •46725_48037•
36 *ROS: ⌊go on as-⌋ ask me anything then •48026_49732•
d. You can now Ctrl-Click on the bullet and the .cut file will open in a new window.
e. Click on the movie thumbnails to see the selected gesture sequences you have annotated. When finished, simply close the .cut file
f. Note of caution: be careful not to have the @T: thumbnail tiers open when you use the F6 command. This will trigger the .cut files, and this has the habit of crashing the programme. You can still use the F5 command to play the file, without triggering the .cut file. Should you wish to use the F6 command, hide the @T: tiers by using the Mode>Hide tiers function (or Esc+4). Here you can select which tiers to display or hide.
Other CLAN tutorials:
Getting Started in 10 Steps Learn CLAN Course
Subtitles with CLAN and Handbrake Piping
Need To Know! Top 4 CLAN Troubles Timedur(ation)
Measuring Pause Length Embedding Still Images
Colouring Keywords Thumbnails: Gesture Annotation
Indent Searching and Coding with COMBO
Making Collections: GEMs Exporting to ELAN