Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
16-10-2024
Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
16-10-2024De Experience Tool: Mapping facts and practice to develop (spatial) experiences (Moes, Schrandt, Manuputty, Admiraal & van der Mark, 2019), is in eerste instantie ontwikkeld door docent-onderzoekers en een oud-student van het Amsterdam Fashion Institute (AMFI) om studenten beter onderbouwde afwegingen te laten maken over inrichting van bijvoorbeeld metrostations, winkels maar ook tentoonstellingen. De toolkit is dus niet ontwikkeld in het kader van Designing Experiencescapes of De Tentoonstellingsmaker van de 21ste Eeuw, maar deze onderzoeken hebben wel een belangrijke inhoudelijke basis gegeven voor de toolkit en zijn dus zeer relevant voor de (toekomstige) tentoonstellingsmaker. Het doel van deze tool is om spelers te inspireren bij en informeren over het creëren van belevingen in (hoofdzakelijk) fysieke ruimtes. De tool is voor iedereen die geïnteresseerd is in het creëren van belevingen en met name interessant voor studenten die een beleving willen neerzetten, in welke vorm dan ook en professionals uit de museale en de retailsector die invloed hebben op het inrichten van fysieke ruimtes.
MULTIFILE
31-12-2018Energy transition is key to achieving a sustainable future. In this transition, an often neglected pillar is raising awareness and educating youth on the benefits, complexities, and urgency of renewable energy supply and energy efficiency. The Master Energy for Society, and particularly the course “Society in Transition”, aims at providing a first overview on the urgency and complexities of the energy transition. However, educating on the energy transition brings challenges: it is a complex topic to understand for students, especially when they have diverse backgrounds. In the last years we have seen a growing interest in the use of gamification approaches in higher institutions. While most practices have been related to digital gaming approaches, there is a new trend: escape rooms. The intended output and proposed innovation is therefore the development and application of an escape room on energy transition to increase knowledge and raise motivation among our students by addressing both hard and soft skills in an innovative and original way. This project is interdisciplinary, multi-disciplinary and transdisciplinary due to the complexity of the topic; it consists of three different stages, including evaluation, and requires the involvement of students and colleagues from the master program. We are confident that this proposed innovation can lead to an improvement, based on relevant literature and previous experiences in other institutions, and has the potential to be successfully implemented in other higher education institutions in The Netherlands.
This project develops a European network for transdisciplinary innovation in artistic engagement as a catalyst for societal transformation, focusing on immersive art. It responds to the professionals in the field’s call for research into immersive art’s unique capacity to ‘move’ people through its multisensory, technosocial qualities towards collective change. The project brings together experts leading state-of-the-art research and practice in related fields with an aim to develop trajectories for artistic, methodological, and conceptual innovation for societal transformation. The nascent field of immersive art, including its potential impact on society, has been identified as a priority research area on all local-to-EU levels, but often suffers from the common (mis)perception as being technological spectacle prioritising entertainment values. Many practitioners create immersive art to enable novel forms of creative engagement to address societal issues and enact change, but have difficulty gaining recognition and support for this endeavour. A critical challenge is the lack of knowledge about how their predominantly sensuous and aesthetic experience actually lead to collective change, which remains unrecognised in the current systems of impact evaluation predicated on quantitative analysis. Recent psychological insights on awe as a profoundly transformative emotion signals a possibility to address this challenge, offering a new way to make sense of the transformational effect of directly interacting with such affective qualities of immersive art. In parallel, there is a renewed interest in the practice of cultural mediation, which brings together different stakeholders to facilitate negotiation towards collective change in diverse domains of civic life, often through creative engagements. Our project forms strategic grounds for transdisciplinary research at the intersection between these two developments. We bring together experts in immersive art, psychology, cultural mediation, digital humanities, and design across Europe to explore: How can awe-experiences be enacted in immersive art and be extended towards societal transformation?
Physical rehabilitation programs revolve around the repetitive execution of exercises since it has been proven to lead to better rehabilitation results. Although beginning the motor (re)learning process early is paramount to obtain good recovery outcomes, patients do not normally see/experience any short-term improvement, which has a toll on their motivation. Therefore, patients find it difficult to stay engaged in seemingly mundane exercises, not only in terms of adhering to the rehabilitation program, but also in terms of proper execution of the movements. One way in which this motivation problem has been tackled is to employ games in the rehabilitation process. These games are designed to reward patients for performing the exercises correctly or regularly. The rewards can take many forms, for instance providing an experience that is engaging (fun), one that is aesthetically pleasing (appealing visual and aural feedback), or one that employs gamification elements such as points, badges, or achievements. However, even though some of these serious game systems are designed together with physiotherapists and with the patients’ needs in mind, many of them end up not being used consistently during physical rehabilitation past the first few sessions (i.e. novelty effect). Thus, in this project, we aim to 1) Identify, by means of literature reviews, focus groups, and interviews with the involved stakeholders, why this is happening, 2) Develop a set of guidelines for the successful deployment of serious games for rehabilitation, and 3) Develop an initial implementation process and ideas for potential serious games. In a follow-up application, we intend to build on this knowledge and apply it in the design of a (set of) serious game for rehabilitation to be deployed at one of the partners centers and conduct a longitudinal evaluation to measure the success of the application of the deployment guidelines.
