iv4XR

Raihana Ferdous; Fitsum Kifetew; Davide Prandi; Angelo Susi

Computer game is a billion-dollar industry and is booming. Testing games has been recognized as a difficult task, which mainly relies on manual playing and scripting based testing. With the advances in technologies, computer games have become increasingly more interactive and complex, thus play-testing using human participants alone has become unfeasible. In recent days, play-testing of games via autonomous agents has shown great promise by accelerating and simplifying this process. Reinforcement Learning solutions have the potential of complementing current scripted and automated solutions by learning directly from playing the game without the need of human intervention. This paper presented an approach based on reinforcement learning for automated testing of 3D games. We make use of the notion of curiosity as a motivating factor to encourage an RL agent to explore its environment. The results from our exploratory study are promising and we have preliminary evidence that reinforcement learning can be adopted for automated testing of 3D games.

Paper on Zenodo

Diogo Rato; Rui Prada

Current architectures for social agents are designed around some specific units of social behavior that address particular challenges, such as modeling beliefs and motivations, establishing social relationships, or understanding group memberships. Although their performance might be adequate for controlled environments, deploying these agents in the wild is difficult. Moreover, the increasing demand for autonomous agents capable of living alongside humans calls for the design of more robust social agents that can cope with diverse social situations. We believe that to design such agents, their sociality and cognition should be conceived as one. This includes creating mechanisms for constructing social reality as an interpretation of the physical world with social meanings and selective deployment of cognitive resources adequate to the situation. We identify several design principles that should be considered while designing agent architectures for socio-cognitive systems. Taking these remarks into account, we propose a socio-cognitive agent model based on the concept of cognitive social frames that allow the adaptation of an agent’s cognition based on its interpretation of its surroundings, its social context. Our approach supports an agent’s reasoning about other social actors and its relationship with them. Cognitive social frames can be built around social groups, and form the basis for social group dynamics mechanisms and construct of social identity.

Paper on Zenodo

Aaron van der Brugge; Fernando Pastor Ricos; Pekka Aho; Beatriz Marín; Tanja E. J. Vos

Testing is of paramount importance in assuring the quality of software products. Nevertheless, it is not easy to judge which techniques or tools are the most effective. A commonly used surrogate metric to evaluate the effectiveness of testing tools is code coverage, which has been widely used for unit and integration testing. However, for GUI testing approaches, this metric has not been sufficiently investigated. To fill this gap, we run experiments with the TESTAR tool, a scriptless testing tool that automatically generates test cases at the Graphical User Interface (GUI) level. In the experiment, we analyze and compare the obtained code coverage when using four different action selection mechanisms (ASMs) in TESTAR that are used to test three SUTs.

Paper on Zenodo

“Scriptless Testing for Extended Reality Systems”

Fernando Pastor Ricós

Extended Reality (XR) systems are complex applications that have emerged in a wide variety of domains, such as computer games and medical practice. Testing XR software is mainly done manually by human testers, which implies a high cost in terms of time and money. Current automated testing approaches for XR systems consist of rudimentary capture and replay of scripts. However, this approach only works for simple test scenarios. Moreover, it is well-known that the scripts break easily each time the XR system is changed. There are research projects aimed at using autonomous agents that will follow scripted instructions to test XR functionalities. Nonetheless, using only scripted testing techniques, it is difficult and expensive to tackle the challenges of testing XR systems. This thesis is focus on the use of automated scriptless testing for XR systems. This way we help to reduce part of the manual testing effort and complement the scripted techniques.

Paper on Zenodo

Ferdous, Raihana; Hung, Chia-kang; Kifetew, Fitsum; Prandi, Davide; Susi, Angelo

EvoMBT is a model-based test generator that uses search algorithms to generate tests from a given extended finite state machine (EFSM). In the context of Cyber-physical systems (CPS) testing, and in particular self-driving cars, we model a set of road configurations as an EFSM and use EvoMBT to generate different roads for testing the car. This report briefly introduces EvoMBT and summarizes its results in the Cyber-physical systems testing competition at SBST 2022. Overall the results achieved by EvoMBT are promising where effectiveness and efficiency scores are quite good while the scores related to diversity need improvement.

Paper on Zenodo

Wishnu Prasetya; Fernando Pastor Ricós; Fitsum Kifetew; Davide Prandi; Samira Shirzadeh-hajimahmood; Tanja E. J. Vos; Premysl Paska; Karel Hovorska; Raihana Ferdous; Angelo Susi; Joseph Davidson

Computer games are very challenging to handle for traditional automated testing algorithms. In this paper we will look at intelligent agents as a solution. Agents are suitable for testing games, since they are reactive and able to reason about their environment to decide the action they want to take. This paper presents the experience of using an agent-based automated testing framework called iv4xr to test computer games. Three games will be discussed, including a sophisticated 3D game called Space Engineers. We will show how the framework can be used in different ways, either directly to drive a test agent, or as an intelligent functionality that can be driven by a traditional automated testing algorithm such as a random algorithm or a model based testing algorithm.

Paper on Zenodo

Tanja E. J. Vos; Pekka Aho; Fernando Pastor Ricós; Olivia Rodríguez-Valdés; Ad Mulders

Covering all the possible paths of the GUI with test scripts would take too much effort and result in serious maintenance issues. We propose complementing scripted testing with scriptless test automation using the open source TESTAR tool. This paper gives a comprehensive overview of TESTAR and its latest extensions together with the ongoing and future research. With this paper we hope we can help and encourage other researchers to use TESTAR for their GUI testing related research and pave the way for an international research agenda in GUI testing built upon stable and open source infrastructure.

Paper on Zenodo

Beatriz Marin; Marta Couto; Ian Saunter; Jeremy Cooke

White paper about the iv4XR solution, exploitation, valorization opportunities, and corresponding business models.

Paper on Zenodo

Anastasios Latos (Master Thesis)

In the current project we present our study on automated video game testing. For our research, we apply our approach of automated agent-based testing, on NethackClone, a 2D, grid-based video game. Our implementation utilizes the Iv4xr framework, a tool that is able to apply and generalize automated testing on multiple types of video games, enabling us in that way to perform agent-based testing on the game, by creating agents and assigning goals to them. Alongside the testing tasks we implemented for our project, we also perform a number of checks on the SUT, checking whether the game behaves as intended when specific actions take place in it. Checks are related to the interaction between the player and the main elements of the game. We created two different tests, with 7 goals and more than 25 actions, tactics and utilities, running our experiments on a total of 307 unique test cases (171 on Test 1, 136 for Test 2). We evaluated our approach based on 3 main factors: coverage, success ratio and time, while the time and effort the framework needs to adapt for a new game each time is also of interest to us. Results derived through the experiments proved not only that our approach performs efficiently at a considerable level, but also our system was even able to detect an actual, unknown bug in the game. The functionality and the ability of the framework to adjust and generalize for multiple games is also promising, considering factors such as updates and adjustments on a game, or similarities between video games. The effort and time we devoted to the framework proved out to be a one-time investment, as once the integration of the SUT into the framework is complete, it can be repeatedly used for creating new testing tasks, checking on different assets of the game. In this way it can assist testers save important time and effort in further, future tests on the same SUT. However, our study also pointed out existing malfunctions in our approach, since our research was limited in terms of time and computational power, proving the need for extended research on a huge number of tests and test cases, in possible future studies.

Paper on Zenodo

“An Online Agent-based Search Approach in Automated Computer Game Testing with Model Construction”

Samira Shirzadeh-hajimahmood; I. S. W. B. Prasteya; Frank Dignum; Mehdi Dastani

The complexity of computer games is ever increasing. In this setup, guiding an automated test algorithm to find a solution to solve a testing task in a game’s huge interaction space is very challenging. Having a model of a system to automatically generate test cases would have a strong impact on the effectiveness and efficiency of the algorithm. However, manually constructing a model turns out to be expensive and time-consuming. In this study, we propose an online agent-based search approach to solve common testing tasks when testing computer games that also constructs a model of the system on-the-fly based on the given task, which is then exploited to solve the task. To demonstrate the efficiency of our approach, a case study is conducted using a game called Lab Recruits and its simulation of another game called Dungeons and Dragons Online.

Paper on Zenodo

Gholizadeh Ansari, Saba; Prasetya, I. S. W. B.; Dastani, Mehdi; Dignum, Frank; Keller , Gabriele

Player experience (PX) evaluation has become a field of interest in the game industry. Several manual PX techniques have been introduced to assist developers to understand and evaluate the experience of players in computer games. However, automated testing of player experience still needs to be addressed. An automated player experience testing framework would allow designers to evaluate the PX requirements in the early development stages without the necessity of participating human players. In this paper, we propose an automated player experience testing approach by suggesting a formal model of event-based emotions. In particular, we discuss an event-based transition system to formalize relevant emotions using Ortony, Clore, & Collins (OCC) theory of emotions. A working prototype of the model is integrated on top of Aplib, a tactical agent programming library, to create intelligent PX test agents, capable of appraising emotions in a 3D game case study. The results are graphically shown e.g. as heat maps. Emotion visualization of the test agent would ultimately help game designers in creating content that evokes a certain experience in players.

Paper on Zenodo

Prasetya, Wishnu; Shirzadehhajimahmood, Samira; Gholizadeh Ansari, Saba; Fernandes, Pedro; Prada, Rui

This short paper presents an architectural overview of an agent-based framework called iv4XR for automated testing that is currently under development by an H2020 project with the same name. The framework’s intended main use case of is testing the family of Extended Reality (XR) based systems (e.g. 3D games, VR systems, AR systems), though the approach can indeed be adapted to target other types of interactive systems. The framework is unique in that it is an agent-based system. Agents are inherently reactive, and therefore are arguably a natural match to deal with interactive systems. Moreover, it is also a natural vessel for mounting and combining different AI capabilities, e.g. reasoning, navigation, and learning.

Paper on Zenodo

Olivia Rodríguez-Valdés; Tanja E. J. Vos; Pekka Aho; Beatriz Marín

Context: Over the last 30 years, GUIs have changed considerably, becoming everyday part of our lives through smart phones and other devices. More complex GUIs and multitude of platforms have increased the challenges when testing software through the GUI. Objective:To visualise how the field of automated GUI testing has evolved by studying the growth of the field; types of publications; influential events, papers and authors; collaboration among authors; and trends on GUI testing. Method: To conduct a bibliometric analysis of automated GUI testing by performing a systematic search of primary studies in Scopus from 1990 to 2020. Results: 744 publications were selected as primary studies. The majority of them were conference papers, the most cited paper was published on 2013, and the most published author has 53 papers. Conclusions: Automated GUI testing has continuously grown. Keywords show that testing applied to mobile interfaces will be the trend in next years, along with the integration of Artificial Intelligence and automated exploration techniques.

Paper on Zenodo

“Search-based Automated Play Testing of Computer Games: a model-based approach”

Ferdous, Raihana; Kifetew, Fitsum; Prandi, Davide; Prasetya, I. S. W. B; Shirzadehhajimahmood, Samira; Susi, Angelo

13th International Symposium on Search-Based Software Engineering, Bari, Italy, October 11-12, 2021

Computergametechnologyisincreasinglymorecomplexand applied in a wide variety of domains, beyond entertainment, such as training and educational scenarios. Testing games is a difficult task re- quiring a lot of manual effort since the interaction space in the game is very fine grained and requires a certain level of intelligence that cannot be easily automated. This makes testing a costly activity in the overall development of games.

This paper presents a model-based formulation of game play testing in such a way that search-based testing can be applied for test generation. An abstraction of the desired game behaviour is captured in an extended finite state machine (EFSM) and search-based algorithms are used to derive abstract tests from the model, which are then concretised into action sequences that are executed on the game under test.

The approach is implemented in a prototype tool EvoMBT. We carried out experiments on a 3D game to assess the suitability of the approach in general, and search-based test generation in particular. We applied 5 search algorithms for test generation on three different models of the game. Results show that search algorithms are able to achieve reasonable coverage on models: between 75% and 100% for the small and medium sized models, and between 29% and 56% for the bigger model. Mutation analysis shows that on the actual game application tests kill up to 99% of mutants. Tests have also revealed previously unknown faults.

Paper on Zenodo

Diogo Rato, Rui Prada

20th IFIP TC 14 International Conference, ICEC 2021, Coimbra, Portugal, November 2-5, 2021

Social agents have been used in games often, for example, to create a social dimension (e.g. the inhabitants of a village) or to provide challenges to players (e.g. the opponents players face). These agents have an essential role in the players’ experience, and, as such, their creation needs to carefully considered. In this paper we propose a taxonomy of social roles that agents can play in games as a step towards the formalization of the problem of the creation of social agents in games. We believe that this taxonomy can help researchers to reach some common ground on the subject and, therefore, promote common views of the research problems involved in the design and development of social agents for games. We discuss several open challenges in the creation of social agents for games and discuss some future directions of research that can be grounded on the analysis of the taxonomy. For instance, many of the social roles proposed are played by agents that do not have much agency or autonomy. Also, there is a large number of under-explored social roles in games at the moment. The taxonomy serves as inspiration to guide game design involving social interactions with game actors, promoting new kinds of gameplay built on the interactive space afforded by the social agents.

Paper on Zenodo

“Using an Agent-Based Approach for Robust Automated Testing of Computer Games”

Shirzadehhajimahmood, Samira; Prasetya, I. S. W. B.; Dignum, Frank; Dastani, Mehdi; Keller, Gabriele

12th International Workshop on Automating TEST Case Design, Selection, and Evaluation (A-TEST ’21), August 23–24, 2021

Modern computer games typically have a huge interaction spaces and non-deterministic environments. Automation in testing can provide a vital boost in development and it further improves the overall software’s reliability and efficiency. Moreover, layout and game logic may regularly change during development or consecutive releases which makes it difficult to test because the usage of the system continuously changes. To deal with the latter, tests also need to be robust. Unfortunately, existing game testing approaches are not capable of maintaining test robustness. To address these challenges, this paper presents an agent-based approach for robust automated testing based on the reasoning type of AI.

Paper on Zenodo

“Tool Demonstration: iv4XR Agent-based Testing Framework”

Prasetya, I. S. W. B.; Shirzadehhajimahmood, Samira; Ansari, Saba Gholizadeh

The iv4XR Framework is an open source agent-based Java framework for automated testing. Currently the main use case is for testing computer games. The main novelty of iv4XR is in its choice for an agent-based approach. Agents are inherently reactive programs, and hence a good match to test highly interactive systems such as computer games. Iv4XR uses BDI based agency, which allows an agent to have belief and desire. Desires are formulated as goals, e.g. to test a certain state in the SUT. Goal solvers can be composed from reasoning rules and rule combinators, and complex goals can be formulated through goal combinators. This approach combines a proactive dimension, that together with the reactive nature of agents provides more versatility and strength for automated testing.

Paper on Zenodo

“Validating the plot of interactive Narrative games”

Carolina Veloso, Rui Prada

The authoring of interactive dialogues in video games is an overwhelming and complex task for game writers. Developing an Interactive Narrative that balances authorial intent and players’ agency requires frequent in-depth testing. The limited range of tools to assist authors in verifying their story can limit the creation of more complex narratives. In this paper, we discuss the challenges of Interactive Story design and provide a model consisting of a set of metrics for testing interactive dialogues. Using this model, we developed a prototype for the Story Validator tool. This debugging tool allows game writers to experiment with different hypotheses and narrative properties in order to identify inconsistencies in the authored narrative and predict the output of different playthroughs with visual representation support. We conducted a series of user tests, Using the Story Validator, to investigate whether the tool adequately helps users identify problems that appear in the game’s story. The results showed that the tool enables content creators to easily test their stories, setting our model as a good step towards automated verification for assistance of authoring interactive narratives. 

Paper on Zenodo

“iv4XR – Intelligent Verification/Validation for Extended Reality Based System”
Wishnu Prasetya, Rui Prada, Tanja E. J. Vos, Fitsum Kifetew, Frank Dignum, Jason Lander, Jean-Yves Donnart, Alexandre Kazmierowski, Joseph Davidson, Fernando Pastor Ricos
RCIS’2020 – The 14th International Conference on Research Challenges in Information Science

Abstract:

“Extended Reality” (XR) systems are advanced interactive systems such as Virtual Reality (VR) and Augmented Reality (AR) systems. They have emerged in various domains, ranging from entertainment, cultural heritage, to combat train ing and mission critical applications. As the complexity of these systems keeps increasing, testing is getting more complex too. Current toolsets do no propose XR testing technology beyond rudimentary record and replay tools that only work for simple test scenarios. The following challenges need to be addressed:
1. Fine-grained interaction space. XR systems more accurately reflect the real world, so they allow fine grained, almost continuous, interactions. Also, XR worlds are inhabited by independent and dynamic entities simulating the corresponding real world entities. They interact with the user as well as with each other, and often lead to emerging behavior. These result in an interaction space far larger than in traditional interactive digital products, and intractable by existing automated testing approaches.
2. Assessing user experience (UX). High quality UX is very important for XR systems. If it is not smooth enough, is too boring, or too overwhelming, the users become unhappy, annoyed or can make mistakes. The latter is a serious concern for mission-critical XR applications. Since manually assessing the UX quality is very labour intensive, automation is needed. Unfortunately, existing tools are too simplistic and lack deeper models of human emotion and cognitive capabilities to be able to judge the different emotional states that an interaction event might evoke on users. Moreover, they are not able to deal with the diversity of users nor are they able to judge the progression of the UX that is built up over time as users engage in long term interactions.

Paper on Zenodo

“Tactical Agents for Testing Computer Games”
I. S. W. B. Prasetya, Mehdi Dastani, Rui Prada, Tanja E. J. Vos, Frank Dignum, Fitsum Kifetew
EMAS’2020 – Engineering Multi-Agent Systems workshop @ AAMAS’2020

Abstract:

Modern interactive software, such as computer games, employ complex user interfaces. Although these user interfaces make the games attractive and powerful, unfortunately they also make them extremely difficult to test. Not only do we have to deal with their functional complexity, but also the fine grained interactivity of their user interface blows up their interaction space, so that traditional automated testing techniques have trouble handling it. An agent-based testing approach offers an alternative solution: agents’ goal driven planning, adaptivity, and reasoning ability can provide an extra edge towards effective navigation in complex interaction space. This paper presents aplib, a Java library for programming intelligent test agents, featuring novel tactical programming as an abstract way to exert control over agents’ underlying reasoning-based behavior. This type of control is suitable for programming testing tasks. Aplib is implemented in such a way to provide the fluency of a Domain Specific Language (DSL). Its embedded DSL approach also means that aplib programmers will get all the advantages that Java programmers get: rich language features and a whole array of development tools.

Paper on Zenodo

“Adoption Dynamics and Societal Impact of AI Systems in Complex Networks”
Pedro M. Fernandes, Francisco C. Santos, Manuel Lopes
AIES’2020 – AAAI/ACM Conference on AI, Ethics, and Society

Abstract:

We propose a game-theoretical model to simulate the dynamics of AI adoption in adaptive networks. This formalism allows us to understand the impact of the adoption of AI systems for society as a whole, addressing some of the concerns on the need for regulation. Using this model we study the adoption of AI systems, the distribution of the different types of AI (from selfish to utilitarian), the appearance of clusters of specific AI types, and the impact on the fitness of each individual. We suggest that the entangled evolution of individual strategy and network structure constitutes a key mechanism for the sustainability of utilitarian and human-conscious AI. Differently, in the absence of rewiring, a minority of the population can easily foster the adoption of selfish AI and gains a benefit at the expense of the remaining majority.

Paper on Zenodo

“Deploying TESTAR to Enable Remote Testing in an Industrial CI Pipeline: A Case-Based Evaluation”

Pastor Ricos, Fernando; Aho, Pekka; Vos, Tanja; Torres Boigues, Ismael; Calas Blasco, Ernesto; Martinez Martinez, Hector

Leveraging Applications of Formal Methods, Verification and Validation: Verification Principles, 2020

Companies are facing constant pressure towards shorter release cycles while still maintaining a high level of quality. Agile development, continuous integration and testing are commonly used quality assurance techniques applied in industry. Increasing the level of test automation is a key ingredient to address the short release cycles. Testing at the graphical user interface (GUI) level is challenging to automate, and therefore many companies still do this manually. To help find solutions for better GUI test automation, academics are researching scriptless GUI testing to complement the script-based approach. In order to better match industrial problems with academic results, more academia-industry collaborations for case-based evaluations are needed. This paper describes such an initiative to improve, transfer and integrate an academic scriptless GUI testing tool TESTAR into the CI pipeline of a Spanish company Prodevelop. The paper describes the steps taken, the outcome, the challenges, and some lessons learned for successful industry-academia collaboration.

Paper on Zenodo

Gholizadeh Ansari, Saba

Abstract:

Designers of extended reality systems need to predict users feedback about designed elements to evaluate their systems. Manual user experience testing can not cover all preferences of users and user-system interactions. To improve and accelerate this process, automated user experience testing is a field of growing interest. Since users’ emotions affect their experience, the automated testing framework should represent users with different emotional states. In this study, we propose an approach to deploy an automated user experience testing framework using BDI test agents which work with a computational model of emotion to regulate their testing behavior.
Paper on Zenodo

“Agent-based Testing of Extended Reality Systems”
Rui Prada, I. S. W. B. Prasetya, Fitsum Kifetew, Frank Dignum, Tanja E. J. Vos, Jason Lander, Jean-yves Donnart, Alexandre Kazmierowski, Joseph Davidson, Pedro M. Fernandes
ICST-2020 – IEEE Conference on Software Testing, Validation and Verification

Abstract:

Testing for quality assurance (QA) is a crucial step in the development of Extended Reality (XR) systems that typically follow iterative design and development cycles. Bringing automation to these testing procedures will increase the productivity of XR developers. However, given the complexity of the XR environments and the User Experience (UX) demands, achieving this is highly challenging. We propose to address this issue through the creation of autonomous cognitive test agents that will have the ability to cope with the complexity of the interaction space by intelligently explore the most prominent interactions given a test goal and support the assessment of affective properties of the UX by playing the role of users.

Paper on Zenodo – Poster on Zenodo – Poster on Zenodo

“Navigation and Exploration in 3D-Game Automated Play Testing”

Prasetya, Wishnu; Voshol, Maurin; Tanis, Tom; Smits, Adam; Smit, Bram; van Mourik, Jacco; Klunder, Menno; Hoogmoed, Frank; Hinlopen, Stijn; van Casteren, August; van de Berg, Jesse; Prasetya, Naraenda; Shirzadehhajimahmood, Samira; Gholizadeh Ansari, Saba

International Workshop on Automating Test case Design, Selection and Evaluation, co-located with ESEC.FSE (ATEST), 9 November, 2020

Abstract:

To enable automated software testing, the ability to automatically navigate to a state of interest and to explore all, or at least sufficient number of, instances of such a state is fundamental. When test- ing a computer game the problem has an extra dimension, namely the virtual world where the game is played on. This world often plays a dominant role in constraining which logical states are reach- able, and how to reach them. So, any automated testing algorithm for computer games will inevitably need a layer that deals with navigation on a virtual world. Unlike e.g. navigating through the GUI of a typical web-based application, navigating over a virtual world is much more challenging. This paper discusses how concepts from geometry and graph-based path finding can be applied in the context of game testing to solve the problem of automated navigation and exploration. As a proof of concept, the paper also briefly discusses the implementation of the proposed approach.

“Aplib: An Agent Programming Library for Testing Games”

Prasetya, Wishnu; Dastani, Mehdi

International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Auckland, New Zealand, 9-13 May, 2020

Abstract:

Testing modern computer games is notoriously hard. Highly dynamic behavior, inherent non-determinism, and fine grained inter activity blow up their state space; too large for traditional auto- mated testing techniques. An agent-based testing approach offers an alternative as agents’ goal driven planning, adaptivity, and reasoning ability can provide an extra edge. This paper provides a summary of aplib, a Java library for programming intelligent test agents, featuring tactical programming as an abstract way to exert control on agents’ underlying reasoning based behavior. Aplib is implemented in such a way to provide the fluency of a Domain Specific Language (DSL) while still staying in Java, and hence aplib programmers will keep all the advantages that Java programmers get: rich language features and a whole array of development tools.

 Paper on Zenodo

“Agents for Automated User Experience Testing“

Pedro M. Fernandes, Manuel Lopes, Rui Prada

2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

Abstract:

The automation of functional testing in software has allowed developers to continuously check for negative impacts on functionality throughout the iterative phases of development. This is not the case for User eXperience (UX), which has hitherto relied almost exclusively on testing with real users. User testing is a slow endeavour that can become a bottleneck for development of interactive systems. To address this problem, we here propose an agent based approach for automatic UX testing. We develop agents with basic problem solving skills and a core affect model, allowing us to model an artificial affective state as they traverse different levels of a game. Although this research is still at a primordial state, we believe the results here presented make a strong case for the use of intelligent agents endowed with affective computing models for automating UX testing.

Paper on Zenodo