Security Testing: a Key Challenge for Software Engineering

Yves le Traon

While important efforts are dedicated to system functional testing, very few works study how to specifically and systematically test security mechanisms. In this talk, we will present two categories of approaches. The first ones aim at assessing security mechanisms compliance with declared policies. Any security policy is strongly connected to system functionality: testing function includes exercising many security mechanisms. However, testing functionality does not intend at exercizing all security mechanisms. We thus propose test selection criteria to produce tests from a security policy. Empirical results will be presented about access control policies and about Android apps permission checks.

The second ones concern the attack surface of web apps, with a particular focus on web browser sensitivity to XSS attacks. Indeed, one of the major threats against web applications is Cross-Site Scripting (XSS) that crosses several web components: web server, security components and finally the client's web browser. The final target is thus the client running a particular web browser. During this last decade, several competing web browsers (IE, Netscape, Chrome, Firefox) have been upgraded to add new features for the final users benefit. However, the improvement of web browsers is not related with systematic security regression testing. Beginning with an analysis of their current exposure degree to XSS, we extend the empirical study to a decade of most popular web browser versions.The results reveal a chaotic behavior in the evolution of most web browsers attack surface over time. This particularly shows an urgent need for regression testing strategies to ensure that security is not sacrificed when a new version is delivered.

In both cases, security must become a specific target for testing in order to get a satisfying level of confidence in security mechanisms