UCAAT Logo

Biographies & Abstracts

Speakers

 

   

Silvia Almagia

OSM DevOps: combining CI/CD and Interop Testing Best Practices

Abstract  

Open Source MANO (ETSI OSM) is a project developing an open source implementation of the Network Function Virtualization Management and Orchestration stack (NVF MANO) aligned with the ETSI NFV Information Model. OSM has a wide and diverse community of 65+ members with different competences and backgrounds in open source development, IT, Telecom and NFV spaces and counts with the support of ETSI’s Center for Testing and Interoperability. One of the main challenges the group faced was the design and setup of the Continuous Integration/Continuous Development (CI/CD) pipeline allowing to automatically test and validate code contributions to the project. The group developed a comprehensive and extensible CI/CD pipeline allowing to automate a wide range of tests going from simple OSS license compliance to exhaustive interoperability testing.

     
 

Silvia Almagia joined ETSI in 2012 as a Technical Expert in the Center for Testing and Interoperability. Her current activities include providing technical management and testing expertise in the areas of open source, interoperability testing and proofs of concept (PoC). Among others, she is in charge of the NFV Plugtest Program, the Open Source MANO project (ETSI OSM) and the NVF PoC Framework. Prior to this, Silvia was Lead Member of the Technical Staff at Ulticom where she was responsible for the design and development of software based carrier grade solutions enabling 3G and 4G services.

 

   

Matej Artač, Damian Andrew Tamburri, Vasilis Papanikolaou, Ismael Torres Boigues and Guiliano Casale

Testing Big Data applications at design time and runtime with DICE

Abstract  

An important segment of IoT research and practice explores the role of live data, which is only the beginning of innovative applications that then live in the world of Big Data. The providers of Data Intensive Applications (DIAs) face difficulties such as the question of performance of the application runtime. The challenges include how to design the DIA to uphold a desired quality or how to deploy the application without spending too much time on recurring manual operations. In the Horizon2020 project DICE which focuses on developing Open Source software and performance engineering methods for Big Data, we are addressing these problems. With DICE UML metamodel and profiles, the users design their application first by building UML models at three levels of abstraction: platform independent, technology specific and deployment specific level. Testing of the application design then already occurs at the design stage thanks to formal models and formal analysis tools such as DICE Simulation, DICE Verification and DICE Optimization. The novelty of DICE tools comes from integration of tools and industry good practices into a streamlined workflow. It complements functional quality assurance by providing non-functional feedback on the application.

     
 

Matej Artač is a researcher at the XLAB Research, Slovenia. He leads the DICE EU H2020’s workpackage on deployment and configuration. His experience includes work on distributed systems and smart energy data. His interests include improving software quality by automating testing deployment.

 

 

     
    Damian A. Tamburri is research fellow at Politecnico die Milano, Italy. He is deputy workpackage leader on data-intensive quality abstractions and automations. His research interests include empirical software engineering methods in social software and advanced software architectures (SOA, MDA/MDE).
     
    Vasilis Papanikolaou is a senior innovation manager at the Innovation Lab of ATC SA. He has extensive experience in New Product Development, Innovation Management and Technology Transfer from research to market through innovative ICT products and applications.
     
    Ismael Torres obtained his degree on Computer Sciences at Universitat Politecnica de València in 2000. Now, he has been working at Prodevelop S.L., the Spanish leader in the development of port solutions software. His interests are related with software engineering and model-driven architectures.
     
    Guiliano Casale is a Senior Lecturer in modelling and simulation within the Department of Computing at Imperial College London, UK. He coordinates the Horizon2020 project DICE. His research focuses on cloud computing and performance engineering.

 



 

Robin Bussenot, Hervé Leblanc and Christian Percebois

Integration testing based on Behavior Driven Development

Abstract  

Generally, integration tests are written in natural language and are executed on a test bench step-by-step manually. The lack of formalism introduces misinterpretations by testers. The step-by-step execution can be fastidious, expensive and error prone. The presented work is focused on the system integration testing phase for avionics systems and aims at introducing agile methods with a new generation of test benches. To manage system integration testing activities, concepts from the Behavior Driven Development approach (BDD) were reused: A new category of Domain Specific Language (DSLs), a Domain Specific Test Language (DSTL), was formalized. Furthermore, based on the DSTL an executable code for a test bench was generated via a pivot language. In the presented approach, the focus is on the description of test scenarios for ATA 21 (air conditioning and pressurization) and ATA 42 (Integrated Modular Avionics).

     
    Robin Bussenot is a Ph. D. student at the University of Toulouse (France). His thesis topic is about introducing agile methods in an avionic context by the formalization of integration test procedures.
     
    Hervé Leblanc has been associate professor of computer science at the University of Toulouse (France) since 2002. He worked on automatic restructuring class and interface hierarchies thanks to the Galois lattice. His main research interests are now linked to design patterns, refactoring and agile methods.
     
    Christian Percebois has been professor of computer science at the University of Toulouse (France) since 1992 where he pursues research on software engineering. Today his main topic concerns developer assistance that targets the interplay of tests and proofs for Hoare triples.

 

   

Michele Cargnani and Anthony Wiles

Using TDL in the development of standardized test specifications for IoT and other technologies

Abstract  

ETSI testing activities cover a wide range of technical domains including 3G and 5G mobile communications, Virtual Network Functions, Multi-access Edge Computing, Internet of Things, Machine-to-Machine and Intelligent Transport Systems. With limited resources and the need for a fast turn-around, ETSI is looking to find the most efficient ways to write these complex, large-scale test specifications in an agile yet consistent manner. The Test Description Language (TDL) has shown promises of simplification and efficiency by abstracting from the complexities of the final, executable test cases. This presentation is a report of ETSI’s experiences on the (early) use of TDL as a key component in the toolchain for the development of test specifications in the ETSI Technical Committees and Industry Specification Groups for NFV, MEC, M2M, ITS etc. both for protocols and API’s.

     
    Michele Carignani is Technical Expert within ETSI’s Centre for Testing and Interoperability (CTI). He has a M.Sc. in Computer Science and Networking. Before joining ETSI, he was research fellow in CNIT (University Consortium for Telecommunication in Italy) working mainly on IoT and ITS research projects.
     
    Anthony Wiles is currently the Director of ETSI’s Centre for Testing and Interoperability (CTI). His experience ranges from running his own business in the UK to managing Swedish operator Telia’s Requirements Engineering Lab in Stockholm. Prior to joining ETSI in 1995, he was with Ericsson, facilitating the introduction and deployment of advanced testing approaches in the company. His contribution to ETSI is many years of experience in industrial hardware and software development and testing, coupled with extensive participation as a delegate and official in standardization activities, including ISO, ITU-T and ETSI Technical Committees.

 

   

Christophe Darçot and Dr. Elizabeta Fourneret

The Power of visual representation for an efficient test design – feedback from an IT implementation project in an industrial company

Abstract  

STREIT Group is an industrial company specialized in the large-scale manufacturing of mechanical parts for the automotive industry. The Enterprise-Resource-Planning (ERP) implemented at STREIT is a strategic backbone to ensure quality services with clients and suppliers in a context highly regulated by ISO Standards (9001 & 14001). Initiated in 2016 on the key domains of procurement, sales, inventory management, quality and finance with SAGE X3 V7, the company decided to adopt an innovative testing methodology based on a lightweight and visual representation of the ERP business processes and rules. The presentation is a return on experience of this project, illustrating the method adopted to produce and maintain acceptance tests (functional and end-to-end). It shows how business analysts and testers may improve their collaborative work using a common language and how this can make the test process more efficient during the phases of test analysis, test design, test implementation and test execution.

     
    Christophe Darçot is IT Manager at STTREIT Group. He has more than 30 year’s career at STREIT crossing different domains and responsibilities: sales & marketing, management, IT. Today, he is leading the implementation of a new ERP module at STREIT, being deployed in the company’s sites across Europe.
     
    Dr. Elizabeta Fourneret is Consultant and Project Manager at Smartesting Solutions & Services. She is consultant for many clients at Smartesting on testing IT systems and leads several R&D projects on testing security-critical systems. Among other, she is testing consultant for the STREIT Group. She is also working in the context of large-scale projects for functional, conformance and security testing.

 

   

Alexis Despeyroux

From Manual Testing to Cognitive Test Automation

Abstract  

In a recent survey of the French Testing Board in 2017, 43% of the respondents replied that less than 10% of their testing is automated. This tutorial explains how companies can achieve complete automation of the end to end testing process for their Enterprise IT applications. Attendees will learn about the evolution of the software testing process over the past 20 years. The tutorial presents latest technology as applied in industrial test automation across various domains. It is about creating models for test generation by reusing various, already available assets like Gherkin feature files, business process models or manual tests. Furthermore, a sneak preview will be given how Artificial Intelligence (AI) can be effectively used to even further automate testing processes and which magnitude of change in test automation can be expected from AI in the not too distant future.

     
    Alexis Despeyroux is a Sales Manager and Solution Architect at Conformiq. He is Certified Advanced Level Test manager and has been working in testing for more than 10 years in various application domains such as telecom, automotive, enterprise IT applications, and now mobile applications.

 

   

Berk Dülger and M. Barış Sarıalioğlu

Sustainable Test Automation: Collaborate within Team

Abstract  

In IT area, one of the biggest challenges is to keep legacy and new systems alive at the same time. Each product added to portfolio or feature to an existing product might bring regression effects that break software in production. Test automation takes a step further freeing valuable human resource by transferring repetitive regression test activities to computers. As each level of automation has its own type of benefits, highest level UI Tests provide a vision for end-to-end system behavior. However, test automation has its disadvantages. Even solid architectures require continuous maintenance for a smooth execution. Considering it is not that easy neither to have programming skills nor business know-how at the same time, it is much more suitable to divide automation responsibility into layers and assign to project member. Driver Layer, Implementation Layer and Business Layer have different functions one can benefit from. This architecture also promotes re-usability with Keyword-Driven Test Design. It combines proven test design techniques with automated test ownership.

     
    Berk Dülger is an IT Professional who worked as a Developer, account manager, consultant and Software Development Engineer in test as well as DevOps Practitioner. He has experiences in international and local projects and believes that total quality improvement can be achieved only with focus on both managerial and engineering agility. Thus, he is very excited about the DevOps movement in the last few years.
     
    M. Barış Sarıalioğlu is an IT professional with more than years of experience as Information Technologies Consultant, Software Engineer, Software Developer and Software Tester for many different organizations. He has published a number of papers and books within the Software Testing profession and contributes to the field by regularly attending conferences as a speaker, panelist, lecturer, moderator and contributor.

 

   

Felix Elliger

On combining IaaS and configuration management into a test automation framework

Abstract  

Test automation evolves – its requirements change based on the system under test as well as organizational demands. On the one hand, the increasingly agile world of software development strives for continuous testing to provide immediate feedback on code changes. On the other hand, the Internet of Things (IoT) targets the secure interconnection of billions of devices, generating lots of data and a respective load on central IoT components. How can test automation be available on demand while scaling with ever-growing demands on test infrastructure and tooling of IoT? The presented solution will sketch the tasks, based on Infrastructure-as-a-Service (IaaS) and configuration management, creating a company-internal platform for automated testing.

     
    While studying IT Systems Engineering at Hasso Plattner Institute, Felix Elliger started working as a research engineer in 2009. After finishing his master’s degree in 2011, he became a software developer which finally made him follow his interest in testing in 2012. As a system engineer, he gained expertise in test automation and its integration into agile processes.

 

   

Dr. Omar Elloumi

The role of interoperability in building a profitable IoT: the case of oneM2M

Abstract  

Nearly 40% of IoT-related economic impact requires interoperability between IoT systems, according to a market research study from McKinsey. oneM2M partnership project developed a software framework that allows interoperability within and across IoT systems. It can be considered as an “Android” for the Internet of Things but it sits both on the field devices/gateways and cloud servers, and it is a voluntary standard that is influenced by oneM2M members at large. The keynote highlights the problem statement of interoperability for IoT as well as the value proposition of a standards-based approach to IoT interoperability: avoiding lock-in to vendors or cloud providers. The role of complementary open source approaches and interoperability events in improving the quality of the specifications and achieving time-to-market with interoperable products will also be addressed.

     
    Dr. Omar Elloumi leads oneM2M technical plenary, a partnership project chartered to advance interoperability standards for consumer, enterprise and industrial IoT. He joined Nokia in 1999 and held several positions including research, strategy and system architecture. He is currently with Nokia Bell-Labs and CTO group where he is responsible for standards and system architectures for IoT. Dr Elloumi is coeditor of books on M2M communications and Internet of Things published by Wiley and ETSI. He is also involved in program committees of several international conferences and magazines on M2M and IoT. He is a recognized IoT industry thought leader and is regularly invited as a panelist or keynote speaker at IoT industry events.

 

   

István Forgács

4Test, an Agile MBT Method

Abstract  

Model-based testing (MBT) is a test design method which starts from a model of the planned code on the basis of which test cases are then generated from the model. Models in MBT can be graphical, text-based or mixed. However, none of these methods support agile testing. Another problem is that traditional models are invented for program development, not for testing. 4Test is a new method which addresses these problems. Here, the model is a simple Gherkin-based text which is understandable by all stakeholders. The related tool is easy to learn and use. Starting point is a user/feature specification which is related to a model. Analyzing the feature, the agile team tries to collect the categories finding the preconditions, inputs, outputs and categories which can be further divided into choices. The key element of the method are the constraints which are relations among the choices. As this model extends the original Gherkin syntax for the description of the constraints, some new elements make it possible to reduce the work for model writing. A significant advantage is that models can be arranged in a hierarchy. The final output is a generated excel sheet where the tests with all the steps help manual testing.

     
    István Forgács PhD. has published more than 25 scientific articles in leading journals and conference proceedings. His research interests include test design, agile testing, model-based testing, debugging, code comprehension, static and dynamic analysis. He is the coauthor of the book: Agile Testing Foundations, An ISTQB Foundation Level Agile Tester guide. He is a member of Agile Extension Board of ISTQB. He led several EU R&D projects at 4D Soft. Dr. Forgács is the CEO of 4Test-Plus. He is the creator of 4Test MBT method by which model-based testing becomes agile, and easier to apply.

 

   

Marcus Friman

Automated Active Assurance of Critical SLAs for Dynamic 5G Network Slices

Abstract  

This presentation will look at the DevOps in Service Chains and 5G Network Slices PoC driven by Netrounds in collaboration with Telenor, Intel, RIFT.io and Arctos Labs, featuring technology from Open Source MANO. The PoC was created to address why DevOps and automation will be critical to network operation procedures of service providers. It also addresses questions about the kind of challenges which will occur with NFV in terms of increasing network complexity and subscriber demands on quality. The presentation is centered on a core solution of Netrounds, an automatable active test and service assurance tool, and how it can be completely orchestrated to assure 5G network slice SLAs and service KPIs in dynamic, virtual network environments.

     
    Marcus Friman co-founded Netrounds in 2007 and has succeeded in leading the company forward in a number of managerial, product management and R&D positions. Today, Marcus leads the strategy and development of the Netrounds product as Head of Engineering and is focused on developing solutions to assist network operators throughout their telco transformation journey. Prior to co-founding Netrounds, Marcus held positions as part of both the R&D and management teams at TeliaSonera and Telia Research. He is a Certified Ethernet Professional (MEF) and holds a Master of Science in Electrical Engineering from Luleå University of Technology (Sweden).

 

   

Sven Gröning, Jens Schmutzler and Christian Wietfeld

Conformance Testing of Electric Vehicle Charging Communication based on TTCN-3

Abstract  

Electric Vehicles (EVs) are a key component for green transportation and future smart grids. At the same time, they define a new, scalable but heterogeneous global market. Hence, interoperability of charging communication is a core enabler for user acceptance and economics of scale. However, its complexity increases by the number of use cases, e.g. for association, authentication and accounting used for Plug and Charge (PnC), demand management or bidirectional power flow. Thus, a considerable amount of testing is required in order to enable interoperability between independent implementations. EV charging communication brings together cross-domain vendors from the automotive-, utility- and communication domain. However, only a standardized approach based on a publicly available and formally defined testing language like TTCN-3 holds true for the required testing scope of the charging communication interface and a systematic approach for corresponding automated test procedures. The presentation gives an overview of requirement-based conformance testing of the Combined Charging System (CCS) using TTCN-3 and describes furthermore lessons learned using TTCN-3 as a formal test case description in a multi-domain standardization process and how automated testing based on TTCN-3 can improve interoperability of electric vehicle charging communication.

     
    Dipl.-Inf. Sven Gröning is senior researcher at the Communication Networks Institute (CNI) of TU Dortmund, Germany. He is an active member of DIN and responsible for the technical concept for conformance testing in ISO/IEC 15118. His research background is model-based testing of communication protocols using formal validation.
     
    Dipl.-Ing. Jens Schmutzler is senior researcher at the Communication Networks Institute (CNI) of TU Dortmund, Germany. He coordinates the research group ICT for Energy Systems at CNI and is Project Team leader for conformance testing in ISO/IEC 15118. His research background is embedded Web Services.
     
    Prof. Dr.-Ing. Christian Wietfeld is head of Communication Networks Institute (CNI) of TU Dortmund, Germany. He has published over 200 peer-reviewed publications and holds several patents. He received an ITU-T outstanding contribution award and is IEEE Senior Member. He chairs the VDE/ITG committee on Communication Networks and Systems.

 

   

Patrick Harms and Jens Grabowski

Automating Usability Testing for Prototypes of the Things in the Internet using Augmented and Virtual Reality

Abstract  

The Internet of Things (IoT) is not only facing challenges like technical aspects, but also the increasing complexity of devices. Moreover, users need to be able to configure and use IoT devices as well as to understand and solve occurring problems. To ensure this, developers need to apply usability techniques during the development process. Augmented and Virtual Reality (AR/VR) are possible vehicles to apply usability testing by using virtual prototypes of devices under development. Presenting such a virtual prototype to a user makes it possible to get an impression whether the users can carry out the intended actions and to simulate challenging user situations. Using a Google Cardboard or similar equipment, AR/VR can nowadays be experienced by almost any owner of a smartphone. This presentation describes the overall scenario for applying automated usability testing on virtual device prototypes by using AR/VR.

     
    Dr. Patrick Harms finished his PhD on “Automated Field Usability Evaluation Using Generated Task Trees” in 2015. Currently, he transfers this research to Augmented and Virtual Reality (AR/VR) and assesses the applicability of AR/VR for usability evaluations of real-world product prototypes.
     
    Prof. Dr. Jens Grabowski is vice director of the Institute of Computer Science at the University of Goettingen (Germany) and heading the Software Engineering for Distributed Systems group. He contributed to the development of TTCN-3 and the UML Testing Profile and works in standardization projects of ETSI.

 

   

Jani Haukinen, Jouni Rajala

Levering test automation to verify infrastructure

Abstract  

Large scale test automation solutions have tendency to have slow feedback cycle, unreliable test results and large root cause analysis scope. Due to this, there is considerable potential for improved efficiency by adding infrastructure specific automated tests. On large scale enterprise solutions, one of the challenges is maintaining test automation reliability. Initial infrastructure testing enables faster feedback cycle and considerable reduces the failures reported from application/service level functional tests that are caused by infrastructure problems. This limits the scope of root cause analysis because it is typically known if the failure is application code or in the infrastructure setup. Moreover, this solution has the potential for considerable time and money savings while using modern SDLC models such as Agile, DevOps, SAFe.

     
    Jani Haukinen is partner and CTO at Comiq Oy, the largest Finnish pure-play QA & test consultancy company. He is Board Member at Finnish Software Testing Board (FiSTB) and co-author for ISTQB Advanced Level Test Automation syllabus.
     
    Jouni Rajala is Senior DevOps consultant at Comiq and has worked over 20 years in the IT-industry. He believes in the benefits of high level of automation in deployment and testing. He works as a DevOps consultant helping customers in their DevOps transformation projects.

 

   

Jens Hempel and Dirk Reufsteck

Certification Testing for Communication in Virtual Power Plants

Abstract  

Testing is essential for decision making in the certification process of energy systems. Strict adherence to standards on communication/cyber security is the basis for operation of the critical energy infrastructure (including virtual power plants). The industry association VHPready choses the lean test process as basis for the certification of the components and systems of the related VPP’s. This common approach shall promote a standardized industry solution and hence the spread of this important Smart Grid technology. Through the membership of Fraunhofer FOKUS in the VHPready industry alliance, core elements of the TTCN-3 test automation language are used in the introduced test suite. Besides that, practices of test development for accredited operation were applied. The VHPready approach may inspire other multi-standard certification test developers directly. The introduced process bridges the gap between demand for accredited certification and a lean test process. This presentation provides insight into the requirements of accredited testing based on two independently developed WAN standards used in the energy environment.

     
    Jens Hempel is responsible for the Smart Grid Competence Center at TÜV Rheinland. In his function, he is (amongst others) engaged with KNX, the EEBus Initiative, the IP500 Alliance, the G3-PLC Alliance, VHPready and the UCA International Users Group.
     
    Dirk Reufsteck is engaged with the UCA International Users Group, the IP500 Alliance and VHPready. For the Competence Center Smart Grid Communication, he is responsible for the IEC 61850 conformance testing and acts as a Certifier, named by the DAkkS and member of the UCA lug testing committee.

 

   

Ivan Jovanovikj

Framework for Constructing Context-Specific Migration

Abstract  

Software migration is a proven and often applied technique for software reuse. Software testing is an important activity in software migration as it verifies whether the migrated system still provides the same functionality as the source system. Since there can be thousands of test cases, automated migration of test cases is also desired. Following the idea of model-driven software migration, the reference method extracts models out of existing test cases and reflects the system changes. At the end, in a model-based testing manner, it generates test cases for the migrated system. All in all, a novel approach for constructing context-specific co-evolution enabled migration methods for test cases is presented. This method was applied in an industrial project with the main goal to migrate parts of the well-known Eclipse Modelling Framework (EMF) along with the Object Constraint Language (OCL) from Java to C#. The approach is basically applicable in any migration scenario that includes existing test cases. Depending on the context information, an appropriate migration method gets tailored and it can be applied to migrate the test cases.

     
  Ivan Jovanovikj is a researcher in s-lab – Software Quality Lab at Paderborn University (Germany). After studying Computer Science in Skopje (Macedonia) and Paderborn, he is now a PhD student at the Chair of Databases and Information Systems under supervision of Prof. Dr. Gregor Engels at Paderborn University. H of interest are Model-based testing, software engineering and software migration.

 

   

Mika Katara

Ten Test Automation Pitfalls to avoid when introducing new tools

Abstract  

A project trying to introduce advanced test automation tools can fail due to non-tool related reasons hampering the adoption of the new, important technologies. These problems are common to all test automation projects introducing new tools. Some points which make the project fail often are unrealistic expectations while making too big promises, automating chaos, lack of manual testing skills as well as a lack of communication skills or too complex verification steps amongst others. Therefore, further discussions in this context should be promoted.

     
    Dr. Mika Katara is a Senior Quality Consultant at Qentinel. He has worked on advanced test automation since 2005 in different roles including test modeler, implementer, designer, reporter, scrum master and team leader. Currently, he works on test automation for fault tolerance in the embedded domain.

 

   

Anne Kramer and Martin Beißer

Strategies for safety-relevant vehicle tests at Continental

Abstract  

Developing safety-relevant components like air bags has never been easy, but the increasing degree of interactions between various components makes it harder than ever. Basically, each internal state of the component/controller has to be combined with each situation which may occur in the environment of the vehicle. The approach of the specific method for testing safety-relevant car functions combines model-based test design, automated test case generation and automated test execution in a simulated environment. While test execution is performed using a virtual test driving tool, model based scenarios are generated automatically using a test case generator. The exact tool chain consists of Enterprise Architect, MBTsuite, PTC Integrity and CarMaker. Because of the high flexibility, test execution is possible at every stage of development (MiL, SiL, HiL). The novelty of the approach lies in the continuous integration of artefacts and tools – from the test model to executable test scenarios in the virtual environment of the simulation system. Applying the method allows to decrease the effort for implementing and maintaining test cases.

     
  Anne Kramer works as senior process consultant at sepp.med GmbH, a service provider specialized in quality assurance for complex, safety-relevant systems. She also acted as test manager for a tier 1 supplier. Besides, she teaches various training programs including the ISTQB® Model-Based Tester.



     
  Martin Beißer is vice president at sepp.med. He is responsible for SW development, quality assurance and consulting projects in safety-critical industries (Automotive, Avionic, Automation and Medical). He often supports the roll-out of model based testing together with automated test generation.




 

   

Markus Lachenmayr and Florian Krautwurm

Testing a Fault-Tolerant Cyber-Physical System designed for Testability

Abstract  

A dependable cyber-physical system (CPS) responds correctly and timely in its intended environment. This requirement also applies while testing reliability, availability and performance properties. Testing such systems and properties pose various challenges. It includes stimulating selected CPS parts with correct as well as faulty data in critical situations and checking selected CPS parts at selected points in time or during time intervals. Other challenges are the differentiation of the CPS's treatment of temporary and permanent faults as well as the absolute non-disturbance of the tested system's functionality. Meeting these challenges requires to design the CPS architecture for testability. For the presentation, an integration platform of an X-by-wire eCar will be the example of a CPS. After deriving testable requirements, a series of design tactics enabling or improving the system’s testability will be enabled. Therefore, the focus is on test probes – Built-in Monitors and Maintenance Interfaces with particular demands in probe environments.

     

  Markus Lachenmayr has a strong background in stochastic schedulability analysis. Since joining Siemens AG, he works on modeling and simulation of software architectures for analyzing different extra-functional properties and their interactions in the context of safety-critical systems.





     

  Florian Krautwurm is Software Test Architect at Siemens AG. For more than 5 years he is working in developing, researching and consulting in the area of industrial information systems and cyber-physical systems and how they can be designed for testability as well as reliably tested.





 

   

Philip Makedonski, Gusztáv Adamis, Martti Käärik, Finn Kristoffersen and György Réthy

From TDL to TTCN-3: A Step by Step Tutorial

Abstract  

This tutorial will briefly introduce both TTCN-3 and TDL including their scope and limitations as well as semantic gaps between them. With the ultimate goal of showcasing how executable TTCN-3 code can be derived from a higher level TDL test description, necessary steps and assumptions, as well as common challenges associated with mapping languages at different levels of abstraction, will explained in detail and supported by illustrative examples.

     
    Philip Makedonski is a researcher at the Software Engineering for Distributed Systems Group at the University of Göttingen, Germany. He has been an integral part of the efforts by TC-MTS on the standardization and implementation of the Test Description Language (TDL) at ETSI within STFs 454, 476, 492, and 522. He is also involved in the development and maintenance of the TTCN-3 open-source tools TRex, T3Q, and T3D. His further interests include research and practice in the fields of software mining, human aspects in software evolution and maintenance, and software quality assurance.
     
    Gusztáv Adamis is an honorary associate professor at the Budapest University of Technology and Economics. Since 2010 he is an architect and product owner at Ericsson's Test Competence Centre. He is involved in the development and standardization of TDL at ETSI within STFs 476, 492, and 522.
     
    Martti Käärik is quality assurance project manager and test tool developer at Elvior. He is involved in the development and standardization of TDL at ETSI within STFs 454, 476, 492, and 522.
     
    Finn Kristoffersen is a software engineer at the tool vendor, Cinderella ApS. He has specialized in tool development for system specification and validation and has been involved in the standardization of specification languages and in development of standardized TTCN-3 test suites.
     
    György Réthy is working with testing for more than 20 years at Ericsson’s Test Competence Centre. He is a vice-chairman of ETSI TC MTS and member of oneM2M Testing Working Group. His expertise covers testing and descriptive languages, including ASN.1, XSD/XML, JSON, TTCN-3, MSC, and TDL.

 

   

Philip Makedonski, Gusztáv Adamis, Martti Käärik, Finn Kristoffersen and György Réthy

Mapping TDL to TTCN-3

Abstract  

While continuing development of the “Test Description Language” (TDL), the Technical Committee “Methods for Testing and Specification” (TC-MTS) at ETSI is focused on bridging the gap between high level test purposes and test cases. The process of stepwise development of tests from requirement specifications is well established and used in both, standardization and industry. This process is supported by methods and languages, such as TTCN-3 and TDL. TDL bridges the gap between declarative test purpose specifications and imperative test case specifications. The lack of a standardized mapping of TDL to TTCN-3 may result in a proliferation of different and possibly incompatible tool- and user-specific mappings of TDL test descriptions to executable test cases. Facing these challenges and common problems of bridging the gaps between languages and representations, users and tool vendors aiming to do the same with other technologies may benefit. This presentation will showcase experiences and the currently ongoing work on the standardized mapping of TDL to TTCN-3.

 

   

Klaus Moessner

IoT based services

Abstract  

Services that involve actuation, or rely on the availability of information and data obtained from sensors or from connected devices in general are often rather volatile. The reliability of connected devices is mostly sub-optimal, and such devices, especially if unsupervised, may have been tempered with. The communication links are mostly over the air and therefore may suffer from the various types of wireless communications problems. IoT based services will need to have built in means to overcome these obstacles; this speech will discuss a range of the issues that lead to problems in Service provision, highlight the need for automated testing measures to test IoT based Services, and will present techniques currently investigated that may help to overcome such problems.

     
    Klaus Moessner is a Professor in the Institute for Communication Systems, at the University of Surrey. Klaus was involved in many projects in the Cognitive Communications, Service provision and IoT areas. He was responsible for the work on cognitive decision making mechanisms in the CR project ORACLE, and led the work on radio awareness in the ICT FP7 project QoSMOS and is now leading the H2020 Speed5G project. In the past, Klaus was the founding chair of the IEEE DYSPAN Working Group (WG6) on Sensing Interfaces for future and cognitive communication systems. He was involved in the definition and evaluation of cooperation management between autonomous entities, in the UniverSelf project, and was technical manager of the iCore project and has the same role in the H2020 project CPaaS.io; he was project leader of IoT.est, SocIoTal, and at current leads the iKaaS and Speed-5G projects. His research interests include cognitive networks, IoT deployments and sensor data based knowledge generation.

 

   

Francesco Oppedisano, Sergio Borghese, Francesco Lamonica and Enrico La Vela

TDL for testing collaboration IT services: the NetResults experience

Abstract  

In the era of Industry 4.0, human labor is becoming “smarter” than ever and, as a consequence, collaboration IT services such as “desktop sharing” evolve to be of mission-critical importance for businesses. The specific action performed on a specific GUI can be separated from the coordination and communication that takes place among the automatic agents performing the test activity. This approach paved the way to a new set of “comparative” testing setups which are not only capable of testing the service itself, but also to benchmark it against other (homologous) services available in the market. Using ETSI TDL, interactions among testing agents can be described and drive the development of a set of mechanisms that are common to any service of this kind. The application specific components were developed using computer-vision frameworks such as Sikuli. The main purpose of the presentation is to share information about impressions and feelings about this TDL-based approach to test an IT service.

     
    Francesco Oppedisano graduated “cum laude” in Telecommunication Engineering at University of Pisa (Italy). In 2007, he received the PhD in Information Engineering working on traffic and QoE active measurement. In April 2006, he founded the academic spin-off NetResults S.r.l. He is currently CEO of NetResults.
     
    Sergio Borghese graduated in Electronic Engineering at University of Pisa (Italy) in 2002. He worked as a software developer in several companies ranging from user applications to device drivers. He joined NetResults S.r.l. in 2011 as a senior software engineer. He is a company shareholder and member of the Board of Directors.
     
    Francesco Lamonica graduated in Computer Science Engineering at University of Pisa (Italy). He worked as research assistant at the University of Pisa in the Telecommunication Engineering department and as senior consultant for major software companies in the Voip field. In 2006, he founded NetResults S.r.l. where he acts as Director of Software Development and as a member of the Board of Directors.
     
    Enrico La Vela graduated in Electronics Engineering at University of Pisa (Italy) in 2009. He worked as FPGA Designer, Automatic Test Developer and Innovation Driver at Ericsson Telecommunicazioni S.p.A. from February 2010 till November 2016. In December 2016, he joined NetResults S.r.l. as a Research Engineer.

 

   

Jorge Pascal, Patrick Meuth, Jörg Reiner and Mathias Helminger

Introducing and adopting Model-Based-Testing for development projects

Abstract  

As the automotive industry is facing the confluence of different new technologies, the implementation of highly integrated software functions in embedded systems is required. The effects of these changes on testing processes and methods reflect the need of higher software quality standards. Model-based Testing (MBT) addresses these challenges by reducing the time to market while maximizing quality standards such as test coverage. A further advantage of MBT is the possibility to derive test cases for subsets of a given model. Test suites can be derived for the different variants of the same version out of one single-source-model. Although MBT refers usually to all testing activities through the entire model-based-development process, this approach focuses on the Hardware-in-the-Loop (HiL) testing phase. The adoption of MBT allows to enhance the testing quality, to reduce the testing cycle time, to develop new testing strategies and to have accurate test metrics such as test coverage.

     
    Jorge Pascal is an industrial Engineer (CPS Zaragoza) and has an MBA from the IE Business School. He works as senior expert in the HiL Simulation & Test Automation areas at TKI Automotive. Through his 15 years of international experience in the automotive industry, Jorge Pascal has held different technical leadership positions at OEMs (VW Group, Manitowoc), Tier 1 suppliers (Lear Corporation) and Tool Vendors (dSPACE). In his current assignment as technical lead at TKI, he must procure, among other duties, testing methodologies and processes to meet the quality standards of highly integrated software functions in embedded systems. He also has published and presented a paper at the SAE World Congress (Detroit, 2006) in cooperation with General Motors.
     
   

Patrick Meuth obtained his Bachelor of engineering in Renewable Energies at the TU Ingolstadt (Germany). He works as test development Engineer at TKI Automotive with special focus on Model-based testing, an Audi AG and IPETRONIK GmbH joint venture founded in 2012.

 

     
    Jörg Reiner has studied electrical engineering at the University of the Federal Armed Forces Hamburg (Germany). After his service in the navy he worked in different companies as a software engineer for various domains. Since 2001 he is working for Assystem Germany GmbH as a software engineer, focused on technical software in the field of automotive and rail. Specialized on test processes, test design tools and testing of embedded systems, he is in charge for the product development of MESSINA (test automation and virtual validation) and MODICA (model-based test design).
     
   

Mathias Helminger is a systems engineer at Assystem Germany GmbH since 2014. In 2014 he obtained his MSc. in computer science at TU München in the field of testing in real-time contexts. During his studies he was working for BMW, re-designing a test automation framework. Since 2015 he is the lead developer of MODICA (model-based test design) but is also involved in broad activities around Assystem´s testing ecosystem and topics around IT-security.

 

 

   

Rafael Schirru and Anke Abromeit

Configuring a TTCN-3 Test System in a Complex Multi-System Environment

Abstract  

The telematics infrastructure (TI) is a complex environment including decentral products such as smart cards and card terminals as well as central products, i.e. servers in the TI network. It is the aim of Gematik to connect all parties involved in the German public health sector by building and operating the telematics infrastructure. Approval tests comprise product tests, performance tests, integration tests as well as end-to-end tests. In some product tests, each system with interfaces to the system under test is mocked using parallel test components (PTCs) in TTCN-3 or simulated outside of TTCN-3. When executing test cases, the test case configuration has to be adapted according to different criteria. ConfiqStore allows gematik to adapt a basic test configuration for different test cases, products and workplaces. The configuration data is held in XML files and organized in different layers. Eventually, the XML data is converted to TTCN-3 records and provided to the test scripts. Currently, ConfigStore cannot be used outside of gematik. This presentation aims at offering room for discussion about the topic of configuration of complex test systems for test designers using TTCN-3.

     
    Dr. Rafael Schirru is a test designer and tester at gematik since May 2014. In 2013, he obtained his PhD from the University of Kaiserslautern (Germany) in Computer Science. Currently, Rafael supports the development of a card terminal simulation as product owner in a scrum team.
     
    Anke Abromeit works as a test designer at gematik since 2014. She obtained her diploma in Electrical Engineering from TU Berlin (Germany). Anke is experienced in software development and testing. She worked as a trainer for installation and configuration of a Telephone Application and Control Server for IBM.

 

   

Martin Schneider and Marc-Florian Wendland

Testing Non-Functional Quality Characteristics of Cyber-Physical Systems

Abstract  

Cyber-physical systems have to deal with several challenges in contrast to traditional systems that are mainly related to their complex interaction with the cyber and their physical environment. These leads to challenges when testing those systems, in particular with respect to quality aspects such as reliability, security and safety aspects. Some of the key questions are: Which scenarios may have a negative impact on these quality characteristics? How to model those situations adequately and cost-efficiently? The presentation will discuss the abovementioned challenges and propose a novel solution based on uncertainty and search-based testing of cyber-physical systems. The integration of the proposed approach in a tool for model-based testing (Fokus!MBT) will be shown. The audience will get a new view on modeling and testing of non-functional quality characteristics for cyber-physical systems that will become essential for new, autonomous applications of cyber-physical systems such as autonomous cars or industrial robots interacting with humans.

     
    Martin Schneider is senior researcher at Fraunhofer FOKUS at System Quality Center (SQC) with a special focus on advanced testing methods and techniques, e.g. model-based testing techniques for security aspects based on different fuzzing techniques.
     
    Marc-Florian Wendland is an expert in formal test approaches, processes and in model-driven techniques. He is involved in several industrial and research projects related to model-based testing, an active member of several standardization working groups, in particular the UML Testing Profile and UML.

 

   

Harry Sneed

Testing Micro Services

Abstract  

Testing Micro Services is important because of an increasing number of applications are now being built with Micro Services. Converting legacy systems into micro services is a favored approach in software migration. The biggest cost factor is the testing of the services. The novelty of this approach is the technique of extracting test cases from the WSDL or REST interface, converting them into a test script to generate requests and to validate the responses. The outcome of this work is a tool supported process for modal testing of micro-services against a semi-formal service specification in natural language which is converted to an assertion script. The requests are generated from the pre-assertions, the responses are validated against the post assertions. In the end, the quality of the service is evaluated and presented in a tester’s dashboard. The key takeaway is the knowledge of how to approach the problem of defining, specifying and testing micro services.

     
    Harry M. Sneed has been working in testing since 1977 when he took over the position of test manager for the Siemens ITS project. Since then he has taken part in over 50 test and migration projects throughout Europe and developed some 36 tools for testing, measuring and re-engineering software systems. Sneed has received the Stevens Award from the IEEE, the first German Award for Quality Assurance from the ASQF and the International Software Testing Award from the ISTQB. He is currently teaching maintenance at the technical high schools of Hagenberg and Vienna in Austria and Reengineering at the Technical University of Dresden in Germany.

 

   

Markku Suominen and Tommi Pernila

Automated adaptive quality and security monitoring in 5G networks

Abstract  

As modern networks are becoming more complex with 5G, it has become quite impossible to fully understand them. In realistic, large scale installations there are a multitude on small errors such as occasional timeouts and unnecessary packet replications, which are not critical on their own. When combined, these small errors can form much larger issues over a time period and are almost impossible to spot by human eye beforehand. This approach enables automated testing and monitoring of security within networks and simulation of potential incidents due to security related anomalies or cumulative amount of minor bugs passed through earlier testing stages. As a part of a research program, an automated solution (sensor) has been developed. This solution consists of machine learning capabilities, neutral networks and other state of the art technologies that can automatically spot anomalies in complex environments. This solution has been developed for spotting security anomalies in telecommunication and enterprise networks.

     
  Markku Suominen is a seasoned IT professional with background of being programmer to architect to senior level security specialist covering banking and insurance business as well as mobile phone development. He has experience from low level HW to networking and architecting large scale security systems. Certifications CISSP 2002 and GSNA 2010.
     
  Tommi Pernila is an experienced senior security consultant with a history of working in the information technology and telecommunication industries for more than 10 years. He has extensive knowledge in Firewalls, Penetration Testing, TCP/IP, SS7 and security standards like ISO 27001. He has Bachelor of Engineering in Electronics from Turku University of Applied Sciences (Finland).

 

   

Marcin Szczukiewicz and Łukasz Wałach

Automated testing model for complex and highly configurable software systems in globally distributed organization

Abstract  

Nokia’s radio base station facilitates wireless communication between user terminal as IoT and a network. It is a complex system consisting of several components, either hardware or software, and can be configured for different operation settings. It is obvious that testing of large systems manually is an outdated technique, however traditional test automation within an international corporation causes creation of autonomous units formed. It leads to a situation in which the number of independently created test beds and their configuration variants increase at a high rate. To reduce time and manpower cost, it is necessary to think outside the box by treating testbed just as another software component. This presentation is about the process of reducing the number of manual tests to a minimum, of shortening the time needed to deliver the software to the customer. Furthermore, it will be shown how to reduce costs and to achieve Continuous Delivery Deployment Pipeline with automated acceptance tests in the cloud.

     
    Marcin Szczukiewicz is a ISTQB Certified Advanced Level Test Manager and is working at telecom industry for 16 years. For the past 8 years he is responsible for radio base station software testing at Nokia’s departments. Currently, he is Project Manager for global system deployed to offer test beds in the cloud to distributed R&D teams in Nokia.
     
    Łukasz Wałach is working for 8 years in Nokia’s R&D, including test automation area. Taking challenges as SW Integration Engineer, Fault Manager and currently R&D Manager, he is responsible for setting and executing test automation goals and tools implementation supporting Continuous Delivery Pipeline for global LTE organization.

 

   

Marc-Florian Wendland and Ina Schieferdecker

Suitability of UTP and TDL for model-based testing – Checking for compliance with ES 202 951

Abstract  

The main objective of MBT is to improve the efficiency by automating the test design activities. The efforts require to create maintain and update a test model are thus success-critical for MBT. These efforts are partly influenced by tooling, but mainly depend on the adequacy and applicability of (test) modelling languages. ETSI has developed a standard (ES 202 951) on requirements a language for MBT should support in order to be adequate for MBT approaches. ES 202 951 was written to effectively specify requirements that support automated test design in the first place. Although ETSI ES 202 951 is not a new standard, it attracted little attention from the industry so far. In the presentation, we demonstrate an application of ES 202 951 to two standardized test modelling languages (i.e., UTP and TDL) and a systematic evaluation of the conformance of these languages to ES 202 951.

     
    Marc-Florian Wendland is a senior researcher at the department System Quality Center (SQC) at Fraunhofer Institut FOKUS, Germany. He is an expert in formal test approaches, processes and in model-driven techniques. He is involved in several industrial and research projects related to model-based testing, an active member of several standardization working groups, in particular the UML Testing Profile and UML. Marc-Florian is leading the UTP working group at OMG for several years now and is a driving member of the UTP 2 specification.
     
    Prof. Dr.-Ing. Ina Schieferdecker is Director of Fraunhofer FOKUS, Berlin and is also professor for Quality Engineering of Open Distributed Systems at Technische Universität Berlin. Her research interests include ICT for smart cities, open data, critical infrastructures, networking, conformance, interoperability, security and certification as well as software engineering and testing. Ina Schieferdecker received a PhD in electrical engineering from TU Berlin and a diploma in Mathematical Computer Science from Humboldt-University Berlin. She is beside others president of the Association for Software Quality and Education (ASQF), member of the German Academy of Science and Engineering (acatech) and of the German Testing Board (GTB).

 

   

Hans-Werner Wiesbrock and Sadegh Sadeghipour

Reactive test of embedded systems using models

Abstract  

The focus in this presentation is on testing embedded systems with a high variability of interaction with the environment. Consequently, their cost-effective quality assurance is a crucial requirement of a successful development process. The solution presented here is a model-based testing approach enabling an automated test execution and evaluation. The novelty of the presented approach is using a Domain Specific Language (DSL) within the environment model as well as a parallel simulation/execution of the environment model and the test object. There are different time scales of embedded systems affecting their test process - very short time scale, short time scale and low time scale. This presentation is focused on cases of low time scale and recommends using a kind of reactive test specification. The approach presented has been implemented using the test framework ContinoProva, which performs automated test execution and evaluation across various tools deployed in the test environment. Reactive tests, specified in the UML tool Enterprise Architect using UML Testing Profile and the DSL extension mentioned above, were integrated into ContinoProva using the interface mentioned.

     
    Hans-Werner Wiesbrock achieved his PhD in Mathematics and in Theoretical Physics. In 1999 he started his work as a software quality expert in the industry. Since then he specialized in test methods and requirements engineering of embedded systems. He currently works as a technical director at ITPower Solutions GmbH.
     
    Sadegh Sadeghipour achieved his PhD in Computer Science. Currently he is the managing director of ITPower Solutions GmbH, which was founded by him in Berlin in 2000. His main fields of interest and activity are test methods and test process of embedded software.

 

 

Posters

 

 

   

Luca Berardinelli and Hong-Linh Truong

Combining Model-driven Engineering and Elastic Execution for Testing Uncertainty in CPS

Abstract  

Cyber Physical Systems incorporate many elements from the Internet of Things to interact with the environment. Simultaneously, they contain elements from cloud computing technologies in order to boost computational, networking, and storage capabilities. Therefore, CPSs have to be capable to adapt its structure and behavior at runtime – they have to be "elastic" (eCPS).
In such eCPS, failures can occur during scaling or during normal system operation. In this context, models can help dealing with complexity and driving the engineering process. Within the community of model-driven engineering (MDE), effort is spent to automate testing based on system models (model-based testing, MBT). In addition, this presentation is focused on uncertainty, a so far neglected aspect of CPS.
This presentation is about reporting experiences and challenges on combining model-driven engineering (MDE) methodologies with elastic execution models to design and test the uncertainty of real-world CPS.

     
    Luca Berardinelli is postdoc at TU Wien. He received his PhD in Computer Science from the University of L'Aquila, Italy. His interests are model-driven engineering and analysis of complex systems relying on IoT/cloud-based infrastructures.
     
    Hong-Linh Truong is an Assistant Professor at the Distributed Systems Group, TU Wien. He is leading the Service Engineering Analytics team in distributed systems at TU Wien. He received his PhD and a Habilitation from TU Wien, Austria..

 

   

Tamás Bőhm, Tibor Csöndes, György Réthy and Antal Wu-Hen-Chang

A flexible, multi-purpose, open source test platform for IoT testing

Abstract  

The Internet of Things (IoT) will have a profound impact in the future. It will enable the inter-connection of devices in various fields of application, from consumer devices, to utilities based meters to sensors in industries. Testing IoT is an important challenge. The challenges stem mainly from the extreme number of connected devices that are expected, highlighting the importance of non-functional testing. But it is not just the sheer number of devices that makes testing challenging but the wide range of functionalities these devices implement with a diverse set of typically open-source technologies, calling for functional testing. Considering the fierce competition in this field, testing frameworks need to support fast time-to-market through easy adaptation to changing technologies and use cases. In order to tackle these challenges, test automation is a must and Model-Based Testing (MBT) techniques are recommended. The presentation will describe the use case of free, open source TTCN-3-based test solutions allowing both functional and non-functional testing, supporting the client and server sides of a wide assortment of IoT solutions, and fulfilling the above test tool requirements.

     
    Dr. Tamás Bőhm is the System Architect for TitanSim at Ericsson’s R&D center in Budapest, Hungary. Earlier he was a consultant in a pilot project on Model-Based Testing. Before joining Ericsson, he was a researcher at the Hungarian Academy of Sciences and the Budapest University of Technology and Economics.
     
    Dr. Tibor Csöndes defended his PhD at the Technical University of Budapest (Hungary) at 2002. He participated in ETSI’s standardization work between 1998-1999 and 2008-2009. He is an honorary associate professor at TU Budapest and Vice President of the Hungarian Testing Board (HTB) a regional board of the ISTQB.
     
    Dr. György Réthy has graduated at the Moscow Telecommunication Institute (Russia) as PhD. He is working with testing for more than 20 years at Ericsson’s Test Competence Centre. Is a Vice-chairman of ETSI TC MTS and member of oneM2M Testing Working Group. Expert in testing and descriptive languages. Holds ISTQB foundation and advanced level certificates.
     
    Antal Wu-Hen-Chang has got his MSc at the Budapest University of Technology and Economics (Hungary) in 2002. He is expert in formal languages, test generation and model based testing. He designed several TTCN-3 function test frameworks and participated in the development of a TTCN-3 based load test framework.

 

   

Jens Grabowski, Tomas Urban, Kristóf Szabados, György Réthy, Julien Deltour and Jacob Wieland

Modernizing TTCN-3

Abstract  

The TTCN-3 testing language has intensively been developed by ETSI during the last decade and, by today, it consists of 14 ETSI standards. The language is also endorsed by ITU-T as the Z.16x and Z.17x Recommendation series. By now TTCN-3 is used exceptionally as the formal specification language of standardized test suites and has also become an important testing technology at various ETSI member companies and in several industrial domains and standards organizations. On evolving the language for more than a decade, ETSI TC MTS has recognized the need for the harmonization of language concepts as well as adding sophisticated features for application and test framework development including object oriented features. For doing this, ETSI TC MTS initiated the Specialist Task Force (STF) 533 on “TTCN-3 Evolution 2017”. This presentation will present and discuss the new object oriented features of TTCN-3 and report on the harmonization of TTCN-3 language concepts.

     
    Prof. Dr. Jens Grabowski is vice director of the Institute of Computer Science at the University of Göttingen (Germany) and heading the Software Engineering for Distributed Systems group. He is active in standardization projects of ETSI and is the leader of the ETSI Specialists Task Force (STF) 533 on “TTCN-3 Evolution 2017”.

 

   

Madis Taimre and Õnne Mets

Model-Based Testing of 3D video games

Abstract  

Currently, almost all game testing is done manually. Some automation is done using bots, capture and replay tools, using custom scripting framework or model-based testing. The video game industry is one of the fastest growing segments with compound annual growth rate 4.8% from $71.3 billion in 2015 to $90.1 billion in 2020. Every day, more than 500 new games are submitted to the iOS App Store. Despite the heavy development, there are very little options for automated system testing of games and there is definitely a desperate need for it. The presentation is based on a case study provided by Virtual Heritage where model-based testing was applied on a virtual reality game VR Quiz. The presentation proposes a model-based approach for automated system testing using a framework which was created as part of the case study for the most popular game development platform, Unity. It can be attached to different test automation platforms.

     
    Madis Taimre is a software engineer in Elvior and has a M.Sc. from Tallinn University of Technology (Estonia). He has over three years of professional experience in testing and software development. Furthermore, he has over two years of experience as a game developer.
     
    Õnne Mets is co-founder of Virtual Heritage (which developed VR Quiz) and CEO of InspireCom. She has over 10 years of experience in communication management and is also a PhD student at Università degli Studi di Milano-Bicocca (Italy).

 

   

Man Zhang, Shaukat Ali, Tao Yue and Phu H. Nguyen

Uncertainty-wise and model-based Testing of Industrial Cyber-Physical Systems

Abstract  

The industrial applications of cyber-physical systems (CPS) are visible in many domains like in robotics, aerospace or healthcare. Given the critical nature of their applications, it is important that a CPS performs its function normally even in an uncertain environment. In this poster, uncertainty-wise and model-based testing frameworks will be presented. The frameworks are specifically developed for testing CPS under uncertainty. They utilize model-based testing as well as search-based testing and include one uncertainty-wise prioritization solution to prioritize the sequence to execute test cases in a cost-effective way with the multi-objective search. To promote the application of these uncertainty-wise and model-based testing frameworks in industry, some have already been implemented in the Certifylt tool – a commercial model-based testing tool.

     
    Man Zhang is a PhD student at Simula Research Laborartory, Norway and at the Department of Informatics, University of Oslo. Previously, she obtained her Master in Computer Technology from Beihang University, Beijing, China. Her main research interests include model-based testing of cyber-physical systems, uncertainty modeling, model-based engineering and requirements engineering.
     
    Shaukat Ali is currently a senior research scientist at Simula Research Laboratory, Norway. He has been involved in several basic research, research-based innovation, and innovation projects in the capacity of PI/Co-PI related to model-based testing (MBT), search-based software engineering and model-based system engineering. He is also actively participating in defining international standards on software modeling in Object Management Group (OMG), notably a new standard on Uncertainty Modeling.
     
    Tao Yue is a chief research scientist at Simula Research Laboratory, Oslo, where she is leading the expertise area of Model Driven Engineering (MDE). She has around 20 years of experience of conducting industry-oriented research with a focus on MDE in various application domains such as Avionics, Maritime and Energy, and Communications in several countries including Canada, Norway and China. Her main research area is software engineering with specific interest in requirements engineering, model-based development, requirements-based testing, model-based configuration and variability modeling as well as software engineering.
     
    Phu H. Nguyen is currently a postdoctoral fellow at the Simula Research Laboratory, Norway. At Simula, he works in research projects about model-based (security) testing for cyber-physical systems under uncertainty. Before joining Simula, he completed his PhD study on the topic of model-driven security at the Interdisciplinary Center for Security, Reliability and Trust (SnT), University of Luxembourg. His current research interests focus on model-based security testing, model-based risk-driven testing, security design patterns and cyber-physical systems/IoT.

 

   

Mengxuan Zhao, Philippe Cousin and Hamza Baqa

A flexible approach for semantic testing in the context of Internet of Things

Abstract  

Internet of Things (IoT) is an emerging area that not only requires development of infrastructure but also deployment of new services capable of supporting multiple, scalable and interoperable applications taking benefit of data from multiple sources. One of the core aspect to achieve semantic interoperability is the ontology and the data annotation using ontology. Standardization bodies have already defined reference ontologies such as W3C-SSN ontology, ETSI-SAREF ontology and oneM2M base ontology. Once reference ontologies have been defined, an important step to ensure semantic interoperability is to test conformity against such reference ontologies. In this perspective, this work contributes to the ontology validation activity by proposing a web-based tool, independent of any ontology development environment, for detecting anomalies in semantic data. The EGM semantic data validation tool is a web service integrated within a web-based client-server architecture. The validator detects syntactic and semantic issues of an ontology and produces a detailed test report.

     
    Dr. Mengxuan Zhao has a PhD in computer science from University of Grenoble, France. With a strong background and research interest in the IoT domain and the semantic technologies, she joint Easy Global Market in June 2015 after 3 years of work of thesis preparation in Orange Labs in Grenoble. She also participates and contributes to standardization works.
     
    Philippe Cousin is CEO of Easy Global Market and an expert in ICT Standardisation, Validation and Interoperability issues for more than 20 years. He worked for Orange Labs for 17 years and as Project Officer at the European Commission for 5 years on Conformance Testing Services. After 4 years as Test house General Manager he joined ETSI where he launched the Plugtests Service. He is involved in more than 30 ICT research projects including a new H2020 project FIESTA to develop tests, tool and certification program for semantic interoperability.