Fone'ikka & fonologia. Ohala (1997:674) Ohala (1997:676) KÄSITTEIDEN ANALYYSIA. SVKS110 (kevät 2016) Tommi Jantunen

Transkriptio

1 Fone'ikka & fonologia SVKS110 (kevät 2016) Tommi Jantunen Osa 1 KÄSITTEIDEN ANALYYSIA Ohala (1997:674) Ohala (1997:676) The ques'on what is the rela'on between phone'cs and phonology, like any ques'on with historical and philosophical implica'ons, cannot be answered objec'vely. Whenever an answer to this ques'on is given it is inevitably subjec've, grounded in the experience, beliefs, and prejudices of the answerer. Phone2cs Concerned with discovering and describing the vocal sounds u'lizable by humans, and studying ar'cula'on, acous'cs, and percep'on. Deals with concrete, physical manifesta'ons of speech sounds. Uses the methods of natural sciences. Phonology Concerned with how the sounds used in language pazern or func'on, how they are represented and used in the mental grammar of speakers. Deals with abstract, psychological manifesta'ons. Uses the methods of social sciences and humani'es. 1

2 Sommerstein (1977:1) Anderson (1981) Phone2cs Not a branch of linguis'cs. Deals with the capabili'es of the human ar'culatory and auditory systems with respect to the sounds and prosodic features available for use in language, and with the acous'c characteris'cs of these sounds and features themselves. Phonology Branch of linguis'cs. Begins where phone'cs leaves off. Is concerned with the ways in which the sounds and prosodic features defined by phone'cs are actually used in natural languages. Phone'cs Language Psychology Ethology Socio- Cultural Factors Etc. Cited from Ohala (1997:684) Karlsson (1994:38, 56) Ojutkangas et al. (2009:83) Fone2ikka KäsiZelee puheen tuozamista ja tunnistamista sekä analysoi puheen akus'sta rakenneza. Keskeisenä tehtävänä on sellaisen järjestelmän luominen, jonka avulla maailman kielten äänteellisiä resursseja voidaan kuvata ja luokitella. Fonologia Tutkimuskohteena ovat kielten äännejärjestelmät. Keskeinen tarkastelukulma on foneehsten äänne- erojen tehtävien analyysi, tarkoituksena määrizää kielen rakenteen kannalta dis'nk'iviset äänne- erot. Fone2ikalla tarkoitetaan äänteiden ja ääntämisen tutkimusta, jossa kiinnostuksen kohteena ovat puheen tuozamiseen, puheen akus'seen välizymiseen ja puheen havaitsemiseen liizyvät ominaisuudet. Konkreehses' fone'ikassa tarkastellaan esimerkiksi ihmisen ääntöelimistön toimintaa erilaisia äänteitä tuotezaessa tai selvitellään ihmisen kielelliseen käyzöön tuozamien äänteiden vastaanozamista. Fone'ikka on lähellä luonnon'eteitä käyzämiensä kokeellisten tutkimusmenetelmien puolesta. 2

3 jatkuu Äännejärjestelmiä tutkivassa fonologiassa puolestaan halutaan selvizää, mitkä kielen äänteet ja äännepiirteet erozelevat merkityksiä eli mitkä ovat kielen foneemit ja kuinka ne voivat yhdistyä isommiksi rakenneyksiköiksi kuten tavuiksi ja sanoiksi. Sekä fone'ikassa ezä fonologiassa tutkitaan lisäksi prosodisia seikkoja, kuten sana- ja lausepainoa ja intonaa'ota (prosodia tarkoizaa sellaisia foneehsia tai fonologisia ilmiöitä, jotka koskevat useampaa kuin yhtä äännezä). Ohala (1997) There is a substan'al overlap between phonological, phone'c, and psychological studies in speech and, in fact, all of the current phonological work incorporate some phone'cs (at least Taxonomic Phone'cs) it is also evident that phonology benefits from phone'cs (for example, in defining basic no'ons). Karlsson (1994) Fone'ikan ja fonologian jatkumoluonne Fone'ikkaa ja fonologiaa ei voi erozaa toisistaan: fone'ikka ilman fonologista näkökulmaa saazaa hukkua yksityiskoh'in puheen jatkumoluonteen ja vaihtelun takia, fonologia ilman foneehsta näkökulmaa käy liian abtrak'ksi. Fone2ikka [jasit:e] /ja sit.ten/ Fonologia x x Si /T/en Signaali IPA Klassinen fonologia Autosegmentaalinen fonologia 3

4 Fonologisten teorioiden olemus Emergenhset teoriat [jasit:e] /ja sit.ten/ [jasit:e] Modulaariset teoriat x x Si /T/en Osa 2 VIITTOMAKIELEN NÄKÖKULMA Crasborn (2012:4-5) Despite the large importance that is oren (intui'vely) azributed to the phone'c difference between sign and speech, rela'vely lizle research within the field of sign language studies has focused on the area of sign language phone'cs, especially in comparison to the phonological analysis of sign languages. Crasborn (2012:7) The phone'c study of sign languages includes the low- level produc'on and percep'on of manual and non- manual signals. It is much less evident how such phone'c analysis of language relates to the phonological structure. 4

5 con'nued Perhaps the impression that we can see the ar'culators in sign languages has made it self- evident what the phonological form looks like, and in that way reduced the need for an accurate phone'c descrip'on. Crasborn (2012:8) What we consider to be phone'c and phonological descrip'ons and how these two interact depends on our model of these different components of language form. Brentari (1998:1-2; emphasis added) Phonology is the level of gramma'cal analysis where primi've structural units without meaning are recursively combined to create an infinite number of meaningful uzerances. It is the level of grammar that has a direct link with the ar2culatory and perceptual phone2c systems, either a visual/gestural pair or an auditory/vocal pair of peripheral systems. Johnson (2000) Phonology Phone'cs (descrip'on of the signal) Phonotac'cs (descrip'on of permissible co- occurrences among signals and among parts of signals) Phonological Processes (descrip'on of phone'c effect of one word on another within strings of words) 5

6 Johnson & Liddell (2010: ) [P]hone'c transcrip'on symbols are symbols for abstract categories of sounds, but the categories are not as abstract as phonemic categories. The primary difference between phone'c and phonemic categories is that phonemic categories are abstrac'ons across (abstract) phone'c categories. con'nued Ideally, a phone'c representa'on system for signs should accomplish the same goals as one for speech. It should provide a wrizen, categorical representa'on of what a signer actually produces. Such representa'ons could provide a basis for the analysis of various types of phonological processes, including assimila'on, persevera'on, and so on. Fone2ikka Fonologia Signaali Johnson & Liddell (2011) Stokoe (1960) Brentari (1998) KOHTI VIITOTTUA SIGNAALIA Osa 3 6

7 Täydellisessä maailmassani Foneehnen tutkimus olisi mahdollisimman empiiristä ja tuozaisi kielellistä signaalia ja sen osasia koskevia (lingvis'ses') alhaisen abstrak'otason faktoja. Fonologia perustuisi foneehsille faktoille tai olisi ainakin läpinäkyväs' kytkezävissä niihin eli tuozaisi kielellistä signaalia koskevia (lingvis'ses') korkeamman abstrak'otason faktoja. Peruskysymys Miten pääsemme luotezavas' käsiksi viizomakieliseen signaaliin, eli millaisille aineiston hankinta- ja käsizelymene- telmille viizomakielen foneehnen tutkimus voisi perustua? Esimerkki puhesignaalin tarkastelusta Praat- ohjelmalla Examples of laboratory techniques for inves'ga'ng speech ar'cula'on Imaging techniques X- ray Tomography (CT, MRI, ultrasound) Point- tracking techniques Electromagne'c MidsagiZal Ar'culator Optotrak Measures of complex behaviors Electropalatography Electromyography (EMG) Stone, M. (1997). Laboratory techniques for inves'ga'ng speech ar'cula'on. In Hardcastle & Laver (Eds.), The handbook of phone3c sciences, Oxford: Blackwell Publishing. 7

8 Menetelmiä foneemsen aineiston keräämiseen ja käsioelyyn viioomakielentutkimuksessa Liikkeenkaappaus laboratorio- oloissa (Mo'on Capture eli mocap) Liike'edon tallennus pelilaizeilla (Kinect, Wii) Tietokonenäköpohjainen videoanalyysi (SLMo'on) Mocap JyU Department of Music (Finnish Centre of Excellence in Interdisciplinary Music Research), University of Jyväskylä Qualisys Mo2on Tracking system with 8 ProReflex MCU120 cameras (120 Hz)......synchronized with a set of DV cameras (1+3) Data analysis with Mo2on Capture Toolbox (versions and 1.3; Toiviainen & Burger 2010, 2011) Toiviainen, P., & Burger, B. (2010, 2011). MoCap Toolbox Manual. University of Jyväskylä: Jyväskylä, Finland. Available at hzp:// mocaptoolbox/mctmanual Con'nuous mocap data on FinSL Demo: Monologue Monologue (1 set) 20 markers (head, upper torso, arms, hands) Ca. 45 seconds Dialogue (2 sets) Altogether 62 markers 31 per signer (head, face, torso, arms, hands) Ca. 60 seconds per set Also other sets Jantunen, T.; Burger, B.; De Weerdt, D.; Seilola, I. & Wainio, T. (2012). Experiences collec'ng mo'on capture data on con'nuous signing. In Proceedings of the 5th Workshop on the Representa3on and Processing of Sign Languages: Interac3ons Between Corpus and Lexicon), organized as a part of LREC 2012 in Istanbul, Turkey, 27 May,

9 Demo: Dialogue Mocap data preprocessed in ELAN and analyzed in Matlab with MoCap Toolbox Kaksi peräkkäistä transi2ivilauseoa Säännöllinen laajataajuuksinen rytmi Autokorrelaa2ofunk2on kuvaaja pään sivuoaissuun- taisesta liikkeestä kolmessa lausetyypissä (liiketunnistusdata 1) Topiikki- kommenm - rakenne Epäsäännöllinen rytmi (katkos rytmissä) Kielteinen luonneh2va lause Säännöllinen 'heätaajuuksinen rytmi 9

10 Linguis'c annota'on in ELAN Kinect data The Kinect data consist of a low- quality RGB video, augmented with a 16 Hz infrared video, and a skeleton model of the signer. Of these, the infrared video allows one to inves'gate the signers ac'vity in the dimension of depth to the precision of one millimeter. The skeleton data adds further value to the analysis of the signers movements as it provides data analogous to that collected with mo'on capture equipment. Puupponen, A.; T. Jantunen; R. Takkinen; T. Wainio & O. Pippuri (2014). Taking non- manuality into account in collec'ng and analyzing FinSL video data. In Proceedings of the 6th Workshop on the Representa3on and Processing of Sign Languages: Beyond the Manual Channel, Paris: ELRA, pp Demo: Kinect data recorded with NiRecorder Demo: Kinect data inspected with Skeletonviewer 10

11 SLMo'on Aalto in CoBaSiL Kinema'c analysis of the mo'on of the arms, hands, and the head with SLMo'on Two complementary tracking methods: feature point tracking (fpt) and active shape modeling (asm) Types of data produced by the methods per articulator: The number of tracked motion points (fpt only!) The amount (& direction) of horizontal and vertical motion Length of the velocity vector (i.e. speed) Length of the acceleration vector (currently fpt only!) Karppa, M.; V. Viitaniemi; M. Luzardo; J. Laaksonen & T. Jantunen (2014). SLMo'on: An extensible sign language oriented video analysis tool. In Proceedings of the 9th Interna3onal Conference on Language Resources and Evalua3on (LREC'14), Paris: ELRA, pp Processing steps KARPPA, M.; Jantunen, T.; Koskela, M.; Laaksonen, J. & Viitaniemi, V Method for visualisa'on and analysis of hand and head movements in sign language video. In C. Kirchhof, Z. Malisz & P. Wagner (Eds.), Proceedings of the 2nd Gesture and Speech in Interac3on conference (GESPIN 2011), organized in Bielefeld, Germany, September 5-7, Demo: fpt & asm ' how the deaf clubs would func'on without these posts?' 1. Face detection 2. Detection of skin-coloured regions 3. Skin blob detection 4. Corner point detection 5. Feature point tracking 6. Fitting of active shape models Feature point tracking ' but does this mean that deaf clubs are old fashioned?' Ac've shape modeling 11

12 Comparison of SLMo'on hand tracking with MoCap data SLMo'on detec'on of head pose angles Yaw: Pitch: Roll: Yaw: Pitch: Roll: Yaw: Pitch: Roll: See SLMo'on hzp://research.ics.aalto.fi/cbir/sorware/slmo'on/ KARPPA, M.; Jantunen, T.; Viitaniemi, V.; Laaksonen, J.; Burger, B. & De Weerdt, D. (2012). Comparing computer vision analysis of signed language video with mo'on capture recordings. In Proceedings of the 8th interna3onal conference on Language Resources and Evalua3on (LREC 2012), organized in Istanbul, Turkey, May 21-27, Demo KARPPA, M., V. Viitaniemi, M. Luzardo, J. Laaksonen and T. Jantunen, "SLMo'on - An extensible sign language oriented video analysis tool," in Proceedings of LREC Paris: ELRA, pp , LUZARDO, M., M. Karppa, J. Laaksonen, T. Jantunen, "Head pose es'ma'on for sign language video," in J.- K. Kamarainen and M. Koskela (eds.), Image Analysis. Springer, Lecture Notes in Computer Science, Vol. 7944, pp , Demo: silmän liikkeiden ja asentojen seuranta JANTUNEN, T. (2007). The equa've sentence in Finnish Sign Language. Sign Language & Linguis3cs 10:2, LUZARDO, Marcos; Viitaniemi, Ville; Karppa, Mah; Laaksonen, Jorma & Jantunen, Tommi (forthcoming in May 2014). Es'ma'ng Head Pose and State of Facial Elements for Sign Language Video. Proceedings of the 6th Workshop on the Representa'on and Processing of Sign Languages: Beyond the Manual Channel [to be organized as a part of LREC 2014 at Reykjavik, Iceland, 31 May, 2014]. 12

13 Mul'- camera recordings CFINSL JyU Puupponen, A.; T. Jantunen; R. Takkinen; T. Wainio & O. Pippuri (2014). Taking non- manuality into account in collec'ng and analyzing FinSL video data. In Proceedings of the 6th Workshop on the Representa3on and Processing of Sign Languages: Beyond the Manual Channel, Paris: ELRA, pp Jantunen, T.; Pippuri, O.; Wainio, T. & Puupponen, A. (2016). Annotated video corpus on FinSL with Kinect and computer- vision data. To appear in "Proceedings of 7th Workshop on the Representa'on and Processing of Sign Languages: Corpus Mining", to be organized as a part of LREC 2016 in Portoroz, Slovenia, May 28, Demo: mul'dimensional data in ELAN Kirjallisuus (jota ei ole listazu dioissa) Anderson, S. R. (1981). Why phonology isn t natural. Linguis3c Inquiry, 12: Brantari, D. (1998). A prosodic model of sign language phonology. Cambridge, MA.: The MIT Press. Crasborn, O. (2012). Phone'cs. Teoksessa R. Pfau, M. Steinbach & B. Woll (toim.), Sign language: An interna3onal handbook, Berlin: Mouton De Gruyter. Johnson, R. E. (2000). American Sign Language phonology. Luentomateriaali, , Jyväskylän yliopisto. Johnson, R. E. & Liddell, S. K. (2010). Toward a Phone'c Representa'on of Signs: Sequen'ality and Contrast. Sign Language Studies 11: Johnson, R. E. & Liddell, S. K. (2011). A segmental framework for represen'ng signs phone'cally. Sign Language Studies 11: Karlsson, F. (1994). Yleinen kieli3ede. Helsinki: Yliopistopaino. Ohala, J. (1997). The rela'on between phone'cs and phonology. Teoksessa W. Hardcastle & J. Laver, (toim.), The Handbook of Phone3c Sciences, Oxford: Blackwell Publishing. Ojutkangas, K., Larjavaara, M., Miestamo, M. & Ylikoski, J. (2009). Johdatus kieli3eteeseen. Helsinki: WSOY. Sommerstein, A. H. (1977). Modern phonology. Bal'more: University Park Press. Stokoe, W. C. (1960). Sign Language Structure: An Outline of the Visual Communica3on System of the American Deaf. Studies in Linguis'cs occasional papers, no. 8, Buffalo: Department of Anthropology and Linguis'cs, University of Buffalo. 13