In English

Component- based Capture & Replay vs. Visual GUI Testing: en Empirical Comparison in Industry

Anmar Khazal ; Ármann David Sigurdsson
Göteborg : Chalmers tekniska högskola, 2015. 59 s.
[Examensarbete på avancerad nivå]

Graphical user interfaces (GUIs) are becoming an essential part of all software, which presents new challenges for high-level testing, i.e. GUI based system and acceptance testing. Currently, these tests are primarily performed through manual practices that are associated with problems such as high cost, tediousness and error proneness. These problems have been proposed to be solvable with automated testing techniques. However, support for automated high-level testing is still limited as Component-based Capture & Replay (CC&R), the most commonly used automated GUI based testing technique in industry today, suffers from various limitations that affect its usability, robustness and foremost maintainability, which leads to higher costs. However, Visual GUI Testing (VGT), a novel technique with promising characteristics such as high script robustness, has therefore been proposed as a more suitable technique in industrial practice than its predecessor CC&R. However, the body of knowledge regarding the VGT technique's applicability is limited in regards to the technique's maintenance cost and robustness. This thesis will present an empirical comparison in industry between CC&R and VGT. The goal of the thesis is to compare the two techniques in an industrial context and to further bridge the gap in empirical knowledge concerning the VGT technique's long term applicability in industrial practice. The thesis work was conducted at CompanyX, which develops schedule and long term planning systems for the avionics industry and was conducted in two phases. The first phase was a pre-study with the goal of determining the industrial context of CompanyX. The second phase was an industrial study performed with a quasi-experimental design which compared and evaluated the two testing techniques, in terms of development cost, maintenance cost and robustness. The results from this thesis work showed that there exists a statistically significant difference between the techniques in terms of development costs and robustness. However, the results showed that there was no statistical difference between the techniques' maintenance costs. Furthermore, both techniques were found to be applicable in industry and are powerful techniques for automated GUI based testing. However, the techniques have different benefits and drawbacks in different contexts, which indicates that a combination of the techniques would be the most beneficial. Further research is however required to verify this claim.

Publikationen registrerades 2015-05-07.

CPL ID: 216713

Detta är en tjänst från Chalmers bibliotek