Perception of Multimodal Referring Expressions in a Cross-Cultural Context

Perception of Multimodal Referring Expressions in a Cross-Cultural Context, ongoing research incorporating a 2 month research stay (February 2010 - March 2010), which was supported by the National Institute of Informatics at Tokyo Japan, comprises 2 academic partners (Trinity College Dublin, National Institute of Informatics at Tokyo), and 1 industrial partner (Dai Nippon Printing).

The main aim of the visit of Dr. Ielka van der Sluis, postdoctoral researcher at Trinity College Dublin (TCD), at the National Institute of Informatics (NII) is to finish the study that was started in 2009 in the collaboration with Prof. Helmut Prendinger and Mr. Werner Breitfuss at NII and Dr. Saturnino Luz at TCD, which was set up to

• Identify indicators for assessing a virtual character’s behaviour across cultures;
• Obtain cross-cultural evaluation metrics for assessing dynamically created content;
• Enhance dialogue strategies in human computer interaction;
• Prepare and test an environment for evaluating information presentation systems.

The data that was collected with this study that was carried out in both Tokyo and Japan will be analysed and the results will be discussed.

The tangible outcome of these activities will be a journal paper to be submitted to the International Journal of Human-Computer Studies (IJHCS). The paper will include the previously published material at the preCogsci workshop (Van der Sluis et al. 2009) and at the Intelligent Virtual Agent Conference (Breitfuss etal., 2009) last year. In addition, it will contain a description of the full study that was conducted in December 2009 and January 2010, as well as a paragraph on the cross-cultural aspects of the work. The latter paragraph requires some background reading on cross-cultural aspects, which will be carried out during the visit and which will also serve a paper about cultural dependencies in the evaluation of multimodal output generation which will be submitted to the MOG 2010 workshop (deadline 14 March).

Provided that some interesting results are obtained with the current study, we will look at the feasibility of setting up the same study in Finnish with Prof. Kristiina Jokinen (University of Helsinki). Prof. Kristiina Jokinen is staying in Japan in February and March 2010 and a meeting will be arranged. A visit to DNP will be scheduled to see if there is scope for implementation of the results in a real application.

Please contact

• Dr. Ielka van der Sluis or
• Dr. Saturnino Luz