Intelligent Verification/Validation for Extended Reality Based Systems
“Extended Reality” (XR) systems are advanced interactive systems such as Virtual Reality (VR) and Augmented Reality (AR) systems. They have emerged in various domains, ranging from entertainment, cultural heritage, to combat training and mission critical applications. The development and authoring of such systems is an iterative process that also includes quality assurance to make sure that the resulting systems are correct and delivering a high quality user experience. As the complexity of these systems keeps increasing, the XR industry now finds itself confronting a soaring engineering challenge: paradoxically, XR’s fine grained and high level of interactivity and realism make such systems very hard and expensive to test. The current XR authoring and development toolset poses no XR testing technology beyond rudimentary record and replay tools that only work for simple test scenarios. This project aims to build a novel verification and validation technology for XR systems based on techniques from AI to provide learning and reasoning over a virtual world. With this technology XR developers can deploy powerful test agents to automatically explore and test the correct parameters of their virtual worlds as they iteratively develop and refine them. In addition, user experience is an equally important aspect for all XR systems. We will therefore also develop socio-emotional AI to enable test agents to conduct automated assessment of the quality of user experience and parameterization by different demographic and socio economic types, such as male, female, young, and elderly.
The tester/developer starts by formulating one or more test goals and gives them to the Framework. The latter then deploys one or more intelligent agents (step 2) to simulate human testers. These agents autonomously interact with the XR system under test to solve the test goals, and in doing so verify the predicates asserted by the test goals. The Framework will inform the tester of the agents’ findings (step 3), e.g. violations of asserted predicates, both as live feed and as a comprehensive report at the end. There will be two types of agents: functional test agents (FTAs) and socio-emotional test agents (SETAs).
OBJECTIVE 1: to develop an intelligent multi-agent based verification approach, suitably scalable to handle XR based systems.
OBJECTIVE 2: to develop a well founded computational approach to appraise the human emotions that would be induced by a series of interactions with a target XR system, hence enabling automated, systematic, and cost-effective assessment of the system UX qualities.
OBJECTIVE 3: to deliver an open multi agent verification technology framework for XR based systems.
Establish a sustainable competitive ecosystem of European technology and solution providers for interactive technologies.
Strengthening European research and industrial capacities to develop future interactive devices.
The current toolset of the XR industry lacks structural testing technology support as existing automated QA techniques are unable to handle XRs’ high interactivity and realism. Many new opportunities can burst out if we remove this impediment. The goal of iv4XR project is to develop advanced automated testing tools that will empower the toolset available to the XR industry.
Our position poster-paper is just accepted at the International Conference on Software Testing, Verification and Validation (ICST) 2020.