Dissertations in Interactive Technology
The School of Information Sciences publishes the Dissertation in Interactive Technology series (ISSN 1795-9489). The series was started in 2005. Before that, dissertations in the HCI field were published in the report series of the (then) Department of Computer Sciences. Also after 2005 some dissertations have been published elsewhere, e.g. in the report series of the employer of the doctoral candidate.
This web page lists first in chronological order (starting from the most recent) the dissertations published in the series, and then the dissertations that deal with interactive technology but have been published in other series.
Date of defense: 2015-11-28
Opponent: Eva-Lotta Sallnäs Pysander
Reviewers: Anirudha Joshi, Thomas Olsson
Supervisor: Markku Turunen
Press release (in Finnish)
This dissertation considers user experience within the field of human-technology interaction from a practical perspective. The aim is to fulfill the research gap of how to evaluate user experience in practice. This dissertation focuses on evaluating the user experience of interactive systems utilizing new interaction techniques in challenging circumstances outside of laboratories. Seven interactive systems and especially their eight user experience evaluations are reported in detail. The main challenges in the case studies have arisen either from the context or the user group(s). The basis for the evaluations has been to focus on taking into account the different circumstances, and to design a user experience evaluation approach accordingly.
The main contribution of this dissertation is a process model on how to evaluate the user experience of interactive systems in practice. The model comprises the whole life cycle of user evaluations, i.e., what needs to be considered before, during, and after the evaluation situation itself. The model provides a set of practical guidelines, and it can be utilized in the design and execution of user experience evaluations in various circumstances.
This dissertation presents eye movement studies of reading dynamically rendered text. Technological development has made reading of electronic text commonplace. A number of studies have been conducted to find a suitable presentation format for text display on electronic devices. Nevertheless, improved text presentation formats, especially for dynamically presented text, are still being developed and studied. Eye movements in on-screen reading provide a useful source of information about reading patterns under different circumstances, such as content and context of the text. This thesis collects six publications on eye movements in reading on-screen text with diverse presentation formats and also with variety in contexts, such as reading print interpreted text and reading for translation.
The results show that the number of fixations and regressions are highly influenced by the design of the text presentation formats. Eye movement metrics are significantly affected when text rendering shrinks from the largest presentation unit (full paragraph) to the smallest unit (phrase of a few words) at a time and further when text appears letter-by-letter. Reading one’s own emerging dynamic text shows different gaze behavior than reading static text. Gaze can also be used as an active channel to instigate automatic scrolling of long pieces of texts. Our studies indicate similar readability with manual scrolling and the gaze-enhanced auto scrolling technique.
Date of defense: 2015-04-28
Opponent: Stephen Brewster
Reviewers: Satu Jumisko-Pyykkö, Eva-Lotta Sallnäs Pysander
Supervisor: Veikko Surakka
Press release (in Finnish)
Touch has an essential role in socio-emotional communication. Currently, different haptic technologies provide a high potential for scientific research to study the relationship between touch and emotions as they enable the accurate creation of stimulation. This thesis summarizes results of six publications that studied systematically emotion-related responses to different haptic stimuli (i.e., friction, vibrotactile, and thermal). For this purpose, subjective experiences (i.e., emotion-related ratings of the stimuli), psychophysiological measurements (i.e., activation of sweat glands), and behavioral responses (e.g., differentiation of the stimuli) were measured.
The results of the thesis showed that different haptic stimulations activated the human emotion system differently, as evidenced by subjective ratings and behavioral and physiological responses. In respect of the ratings, it can be concluded that friction and thermal stimulations were better at evoking changes in the ratings of pleasantness and approachability than vibrotactile stimuli. Vibrotactile stimuli were associated with a higher level of arousal and a feeling of being controlled by the stimulation. As there is growing interest in using stimulation of the sense of touch in human-technology interaction, it is likely that the results of the current thesis can be utilized in designing haptics-based affective computing.
Touch is essential in human-computer interaction with mobile devices. Devices such as smart phones sense input through touchscreens to provide users with information. However, this information is presented mainly via visual and auditory modalities. The use of touch output has been limited to vibration alerts and feedback of touchscreen buttons. This thesis focused on ways to use touch output for presenting alphabetical and emotional information. Special emphasis was put on studying how spatial touch output presented to different areas of the user’s hand could be utilized with mobile devices.
This is achieved by designing, implementing, and evaluating novel interaction methods based on handheld device prototypes. The results indicate that alphabetical information can be presented to users by vibrating the touchscreen of a mobile device. In addition, different emotional information could be communicated between people who used devices capable of presenting touch gestures such as squeezing and stroking with vibration. These findings can be utilized in future human-computer interaction research aiming at supporting more active use of touch with mobile devices.
The significance of eLearning solutions to overcome challenges facing the education sector in higher education in East Africa cannot be overstated. Appropriate use of eLearning solutions has the potential to reduce costs, to widen access, and to improve the quality of teaching and learning. Although these solutions have been succesfully implemented in many developed countries, the degree of acceptance and usage is low in the majority of higher education in East Africa.
The aim of this thesis was to investigate factors that influence acceptance and use of various eLearning solutions in higher education in East Africa. It is a compound thesis comprising five articles that describe four distinct case studies.
The results described in the thesis should enable institutions to find strategies that promote greater use and accceptance of eLearning solutions in higher education in East Africa. They also give developers tools to develop eLearning services that are relevant and acceptable to intended users in the region.
Date of defense: 2014-08-29
Opponent: Katrien Verbert
Reviewers: Dietmar Jannach, Martin Schmettow
Supervisor: Kari-Jouko Räihä
Press release (in Finnish)
Recommender systems have become omnipresent in e-commerce and many other domains, as the number of items available to us has exceeded our ability to consider them individually. Recommenders help us make better decisions with less effort and uncertainty by helping us find salient items (e.g. suggesting books) and decide which one(s) to choose (e.g. which book to buy). However, recommender system research has largely focused on system-centric aspects, e.g. algorithms, and ignored user-centric aspects. This is now seen as having been detrimental to the field, as it is user-centric aspects that determine the adoption and use of recommenders.
In this work, we study user-centric aspects of recommender systems in three domains, e-commerce, news recommending, and e-learning. All studies except one involved actual users using actual systems in authentic use contexts. Consequently, this work offers us windows into the actual use of recommender systems. The results underline that there are few universal truths as far as user-related factors in recommender systems are concerned. In fact, what is true in one context can be the opposite in another context due to changing user tasks and goals. Consequently, in developing recommender systems that truly serve their users, user-centric testing is a must.
Outi Tuisku: Face Interface
The use of facial information is imperative in human to human communication. For example, people naturally gaze at the person they are interacting with. Further, they use facial muscles to generate facial expressions in order to express emotions and intentions. In addition to the communicative purposes, gaze and facial muscle system can also be used for interacting with and controlling computers. The current thesis introduces a wearable prototype device called Face Interface that measures both gaze direction and facial expressions like frowning and smiling for pointing and selecting objects while interacting with graphical user interfaces. Three different versions of the prototype have been used in the course of this thesis by improving the prototype according to the objective and subjective results.
The results show that Face Interface functions promisingly as a pointing and selection technique. Along the iterations significant improvements have been achieved in the pointing task times (i.e. from 2.5 seconds (first prototype) to 1.3 seconds (third prototype)). Further the research has shown, for example, that it is easy to learn the use of these two different modalities together, and the use of them does not require much practice. These are clear indications that the use of facial information has great potential in human-computer interaction.
In spoken language, both the content of spoken words and the prosody of speech can mediate emotion related information. To study how the pure content of spoken words affects human emotions, speech synthesizers offer good opportunities as they allow for good controllability over the prosodic cues. This thesis summarizes five publications that investigated how the lexical emotion-related content of synthesized speech affected people’s emotional experiences and physiological responses (i.e. facial muscle activity, pupil size, and heart rate). The other aim was to study how the human-likeness of synthetic voice affects responses. Thirdly, the effects of emotional content on the perception of the quality of speech synthesis were studied.
The results presented in this thesis suggest that the synthesized lexical expressions of emotions can evoke emotions in people. The results suggest that the features of the voice also matter when evoking emotions through computers. Finally, the results showed that the lexical content of the messages had such a strong effect on people that the impression of the voice quality was affected by the content of the spoken message.
Mobile Web search is a rapidly growing form of everyday information access. Research suggests that new paradigms are needed for better support of mobile searchers. In this work, two such novel search interface techniques were designed, implemented, and evaluated. The first method is a clustering search interface that presents a categorized overview of the results. The findings from laboratory and longitudinal studies indicate that clustering can support exploratory search needs of mobile searchers. The second presentation method is a visualization of the instances of the user’s query phrase in the result document. Findings from user studies suggrest that the visualization can be useful in ruling out non-relevant results and can assist when the other result descriptors do not provide for a conclusive relevance assessment, although some learning is required. Finally, the contextual triggers and information behaviors of active mobile Internet users were studied, for understanding the role of Web search as a mobile information seeking activity. The results show that mobile Web search and browsing are important information seeking activities to resolve emerging information needs in various situations, whether at home or “on the go.” The results of this thesis underline the need for future mobile search interfaces to consider new result presentation methods and account for context-dependent information needs.
Emotions are intimately connected with our lives. They are essential in motivating behaviour, for reasoning effectively, and in facilitating interactions with other people. Consequently, the ability to regulate the tone and intensity of emotions is important for leading a life of success and well-being. Intelligent computer perception of human emotions and effective expression of virtual emotions provide a basis for assisting emotion regulation with technology. State-of-the-art technologies already allow computers to recognize and imitate human social and emotional cues accurately and in great detail. For example, in the present work a regular looking office chair was used to covertly measure human body movement responses to artificial expressions of proximity and facial cues. In general, such artificial cues from visual agents were found to significantly affect heart, sweat gland, and facial muscle activities, as well as subjective experiences of emotion and attention. The perceptual and expressive capabilities were combined in a setup where a person regulated her or his more spontaneous reactions by either smiling or frowning voluntarily to a virtual humanlike character. These results highlight the potential of future emotion-sensitive technologies for creating supportive and even healthy interactions between humans and computers.
Text entry methods enable input of written text to computing systems. As a logosyllabic language Chinese has unique characteristics that bring new challenges to the design and evaluation of Chinese text entry methods. This dissertation explores new interaction solutions and patterns of user behavior in the Chinese text entry process with various approaches. The work covers four means of Chinese text entry on mobile devices: Chinese handwriting recognition, Chinese indirect text entry with a rotator, Mandarin dictation, and Chinese pinyin input methods with a 12-key keypad. New design solutions for Chinese handwriting recognition and pinyin methods utilizing a rotator are proposed and shown to be well accepted by users with emprical studies. A Mandarin short message dictation application for mobile phones is also presented, with two associated studies on human factors. Two studies were also carried out on Chinese pinyin input methods that are based on the 12-key keypad. The comparative study of five phrasal pinyin input methods led to design guidelines for the advanced feature of phrasal input. The second study of pinyin input methods produced a predictive model addressing users’ error-free speeds.
Text entry by eye gaze is used by people with severe motor disabilities. This thesis provides an extensive review of the research conducted in the area of gaze-based text entry. It summarizes results from several experiments that study various aspects of text entry by gaze. An overview of different design solutions and guidelines derived from the research results are given. It is hoped that the thesis will provide a useful starting point for developers, researchers, and assistive technology professionals wishing to gain deeper insight into gaze-based text entry.
Date of defense: 2009-01-16
Opponent: Heikki Ailisto
Reviewers: Matti Pietikäinen, Marcos A. Rodrigues
Supervisor: Veikko Surakka
Press release (in Finnish)
Most human interaction takes place through face-to-face communication. The obvious importance of facial stimuli for humans motivates the idea of utilizing facial information in human-technology interaction. In this type of interaction it is required that facial information, among others, is automatically captured, analysed, and further processed in oreder to make the interaction more natural and intelligent. Computer vision capabilities are especially helpful for capturing and analyzing important visual cues from the user’s face.
In this dissertation, one aspect of automatic face analysis, namely, face and feature detection, was addressed. During the course of this research work a framework for automatic and expression-invariant localization of faces and prominent facial landmarks, such as eyes, eyebrows, nose, and mouth from static images and real-time video was developed. The performance of the framework was evaluated on several databases of facial expressions coded in terms of prototypical facial displays, like happiness and surprise, and facial muscle activations presented alone or in combinations. In general, the results showed that the framework allowed the face and facial landmarks to be located automatically, robustly, and efficiently from static images and streaming videos displaying facial expressions of varying complexity.
Date of defense: 2008-05-09
Opponent: Markku Tukiainen
Reviewers: John Paulin Hansen, Jukka Paakki
Supervisor: Kari-Jouko Räihä
Press release (in Finnish)
The growing number of publications that address human-computer interaction using eye movement highlights the increasing need for tools to investigate and analyse the behaviour of human eyes. Despite the fact that eye movement analysis tools are becoming more intelligent and advanced, there is still a lack of effective tools that allow recording and on-line use of eye movement data from various eye trackers. Also needed are tools for further analysis and visualisation of gaze paths that support the majority of methods already known.
This dissertation describes the iComponent software that was developed to fill this gap as well as the research projects where it has been used. iComponent has a highly flexible architecture, which allows easy development of dynamic plug-in modules to support eye tracking devices and experimental software. The unique data format and data transfer interfaces were developed on the basis of careful analysis of existing hardware. iComponent supports several types of gaze data visualization with high customization abilities. Experience has confirmed the effectiveness of the tool developed as well as pointed to several issues that were improved by further development of iComponent.
Vision has an important role when people communicate with each other. Face provides vast amount of information such as identity, gender, facial expressions, and age. On the other hand, traditionally people have interacted with computers using a mouse and a keyboard. This is about to change. New technologies are emerging that enable natural interaction with computers. One of these technologies is automatic face analysisthat enables computers to interpret human faces.
In this dissertation various face analysis techniques were studied and their applicability to human-computer interaction was considered. Some novel methods were presented and experiments were carried out for face detection and gender classification methods. The reliabilities of the methods were measured in conditions likely in real perceptual applications. The results of the experiments showed that about 80% gender classification accuracy can be achieved with a fully automatic face analysis system and web camera quality frontal face images. Applications where face analysis techniques have been used and ideas for future applications were also presented.
Date of defense: 2007-04-21
Opponent: Robert Spence
Reviewers: Kasper Hornbæk, Jonathan C. Roberts
Supervisor: Kari-Jouko Räihä
Press release (in Finnish)
Acquiring data is much easier than gaining insight into it. Interactive techniques for information visualization can aid us in understanding the data, making the acquisition of information easier. One of the challenging issues in information visualization is the treatment of multidimensional data, i.e., when we need to consider a large number of data variables and their relationships simultaneously, often without a well-defined understanding of what to look for in the data. This work studies interaction in three conceptually different multidimensional visualization techniques: the reorderable matrix, parallel coordinates, and interactive glyphs. The three techniques were investigated by implementing a number of interactive prototypes and performing controlled experiments with them. A number of interaction enhancements were developed and evaluated using an incremental development approach and by augmenting the controlled experiments with usability evaluation techniques. Contributions include a new technique for processing a reorderable matrix visualization, improvements to the user interface of parallel coordinate browsers, and a new visualization technique based on data glyphs and small multiple visualizations.
Speech has been in use as a computer user interface for some time already. For more than two decades, speech-recognition based services have been publicly available, and the number of available services has risen steadily. However, speech user interfaces have not reached the wide popularity sometimes hoped for or expected. This is due to the limitations of spoken human-computer interaction.For speech interfaces to be usable, users must receive some guidance to be able to act within the limits of the system. Existing systems either provide the necessary guidance to users as part of the user interface or require the users to explicitly read some sort of guidance material.
In this study, software tutoring has been applied for speech user interfaces. A software tutor is a software component that teaches the use of a software application. A tutor can monitor users’ actions and adapt appropriately. The teaching happens in situ; the users learn about the application while using it. The study presents two kinds of software tutors: a speech-based tutor and graphics-based tutors. The nature of the tutors, their technical solutions, the iterative development process, and formal evaluations are reported. The results show that the tutors can support initial use better than the previously used static text-based guidance materials.
Computer games are traditionally controlled with hand-operated input devices which might cause negative health effects. This thesis focuses on studying one particular computer and video game genre, referred to as physically interactive games. These games are controlled with body movements, and they aim at involving the player in a physical effort or developing motor abilities during the course of game play. However, this new way of controlling games poses challenges for game designers not only from the technological point of view but, more importantly, also from the user perspective. The core question is that of how physically interactive games should be designed so that they would be entertaining, usable, and physically suitable for players. The thesis consists of three case studies that illustrate the importance of the quality of physical game control, and discuss the methodological issues related to game design and evaluation. Furthermore, studies show that the embodied gaming context supports social interactions between players, enables players’ creative physical expression, and can improve cardiovascular fitness, muscle strength, coordination of movements, and reaction times.
Eyes can be used as a source of information when moving from human-computer interaction toward the more natural and effective. In particular, the focus of the user’s attention could be valuable for many computing applications. In this dissertation we report the experiences obtained during the design, implementation, and evaluation of a gaze-aware attentive application, iDict. iDict aims to help with electronic documents by tracking the reader’s eye movements and providing assistance automatically when the reader seems to be in need of help. The three main issues addressed in this work are the problems caused by the limited accuracy inherent to eye tracking, the problems with interpreting user’s eye movements, and the design principles of gaze-aware applications.
World Wide Web search engines are essential tools in today’s world, where more and more information is to be found only on the Web. This thesis focuses on the strategies employed by different user groups of search engines during three phases of the information search process: query formulation, evaluation of search results, and information re-access. The studies show that queries of more experienced users tend to be longer, more precise, and, if not successful, iterated frequently. More experienced users are also more efficient in evaluating the search results. In information re-access, the experienced users are innovative and not completely dependent on the tools specifically designed for information re-access. Surprisingly, even experienced users have misconceptions concerning their primary search engine. The elderly face several challenges with Web search engines, such as not understanding the required language and the functionality provided by the interfaces, as well as problems with text input. The thesis presents several design suggestions to place the benefits of successful strategies at the disposal of all users. For example, the thesis presents a novel style for presenting textual result summaries, a natural-language explanation tool for queries, a search interface for elderly users, and several ideas for facilitating information re-access.
World Wide Web is an enormous source of information, but finding the relevant bits can be challenging as the current search engines typically present search results as a long list. This work studies ways to enhance users’ result access with automatically formed categories and associated filtering user interface. The concept is implemented in a search user interface called Findex. The usefulness of the approach was evaluated in controlled experiments, in a longitudinal study, and with a theoretical test. The results show that finding relevant results is about 30-40% faster with the proposed user interface compared to the de facto standard, the ranked results user interface. The user attitudes favor the new user interface. The results of the experiments were complemented with a longitudinal study in real use situation. The results indicate that the categorization user interface becomes a part of the users’ search habits and is beneficial. In real settings, the categories are needed in about every fourth search. The usage patterns indicate that the categories help when result ranking does not bring relevant results to the top of the result list.
Human-computer interaction is a growing interdisciplinary research area, which aims at developing user-friendly methods for computer use. It has been traditionally studied mostly from the cognitive perspective. In his dissertation, Timo Partala studied the possibility of utilizing affective information in human-computer interaction. Specifically, he studied the possibilities of using pupil size variation and facial expressions in computer input, and the effects of affective synthetic speech messages and different agent proximities in computer output. The results suggested that pupil size variation and facial expressions can give the computer information about the user’s affective responses, and they could be potential input signals for human-computer interaction in the future. It was also found that synthetic speech messages used in computer output influenced the users’ affective physiological responses, behavior, and experienced valence. In addition, it was found that the different simulated proximities of a conversational agent affected the users’ experienced dominance.
Dissertations in Other Series
Pasi Välkkynen: Physical Selection in Ubiquitous Computing
Tatiana G. Evreinova: Alternative Visualization of Textual Information for People with Sensory Impairment
Poika Isokoski: Manual Text Input: Experiments, Models, and Systems
Markku Turunen: Jaspis – A Spoken Dialogue Architecture and its Applications
Juha Lehikoinen: Interacting with Wearable Computers: Techniques and Their Application in Wayfinding Using Digital Maps
Date of defense: 2002-09-27
Opponent: Bruce H. Thomas
Reviewers: Petri Pulli, Tapio Takala, Jukka Vanhala
Supervisors: Kari-Jouko Räihä, Hannu Nieminen
Press release (in Finnish)
Roope Raisamo: Multimodal Human-Computer Interaction: A Constructive and Empirical Study