Tuesday, October 16th, 2:00PM - 3:30PM, R2: Journal-first Paper Presentations 

For the first time in the history of ISSRE, we are enriching the program with journal-first presentations, in cooperation with the Information and Software Technology journal. Journal-first papers report completely new contributions that have not been previously presented at any other conference or journal, and that do not extend any previous paper. The journal-first papers were selected among the ones recently published in the journal (2017 and mid-2018) and that intersect with the topics of interest of the ISSRE conference.

An architecture, system engineering, and acquisition approach for space system software resiliency. Dewanne M. Phillips (The Aerospace Corporation), Thomas A. Mazzuchi, Shahram Sarkani (George Washington University)​

Software metrics thresholds calculation techniques to predict fault-proneness: An empirical comparison.  Alexandre Boucher, Mourad Badri (University of Quebec)​

SPIRITuS: a SimPle Information Retrieval regressIon Test Selection approach. Simone Romano, Giuseppe Scanniello (University of Basilicata), Giuliano Antoniol (Ecole Polytech. de Montreal), Alessandro Marchetto 

An architecture, system engineering, and acquisition approach for space system software resiliency. Dewanne M. Phillips (The Aerospace Corporation), Thomas A. Mazzuchi, Shahram Sarkani (George Washington University)​

https://www.sciencedirect.com/science/article/abs/pii/S0950584917300575

Abstract 

Context: Software-intensive space systems can harbor defects and vulnerabilities that may enable external adversaries or malicious insiders to disrupt or disable system functions, risking mission compromise or loss. Mitigating this risk demands a sustained focus on the security and resiliency of the system architecture including software, hardware, and other components.

Objective: In this paper we offer methodical approaches for improving space system resiliency through software architecture design, system engineering, and increased software security, thereby reducing the risk of latent software defects and vulnerabilities.

Method: We conducted a systematic review of existing architectural practices, standards, security and coding practices, various threats, defects, and vulnerabilities that impact space systems from hundreds of relevant publications and interviews of subject matter experts. We expanded on the system-level body of knowledge for resiliency and identified a new software architecture framework and acquisition methodology to improve the resiliency of space systems from a software perspective with an emphasis on the early phases of the systems engineering life cycle. This methodology involves seven steps: 1)Define technical resiliency requirements, 1a) Identify standards/policy for software resiliency, 2)Develop a request for proposal (RFP)/statement of work (SOW) for resilient space systems software, 3)Define software resiliency goals for space systems, 4)Establish software resiliency quality attributes, 5)Perform architectural tradeoffs and identify risks, 6)Conduct architecture assessments as part of the procurement process, and 7)Ascertain space system software architecture resiliency metrics.

Results: Data illustrates that software vulnerabilities can lead to opportunities for malicious cyber activities, which could degrade the space mission capability for its user community. Reducing the number of vulnerabilities by improving architecture and software system engineering practices can contribute to making space systems more resilient.

Conclusion: Since cyber-attacks are enabled by shortfalls in software, robust software engineering practices and an architectural design are foundational to resiliency, which is a quality that allows the system to take a hit to a critical component and recover in a known, bounded, and generally acceptable period of time. To achieve software resiliency for space systems, acquirers and suppliers must identify relevant factors and systems engineering practices to apply across the life cycle, in software requirements analysis, architecture development, design, implementation, verification and validation, and maintenance phases.

Software metrics thresholds calculation techniques to predict fault-proneness: An empirical comparison. Alexandre Boucher, Mourad Badri (University of Quebec)​ 

https://www.sciencedirect.com/science/article/abs/pii/S0950584916303135​

Abstract

Context: Nowadays, fault-proneness prediction is an important field of software engineering. It can be used by developers and testers to prioritize tests. This would allow a better allocation of resources, reducing testing time and costs, and improving the effectiveness of software testing. Non-supervised fault-proneness prediction models, especially thresholds-based models, can easily be automated and give valuable insights to developers and testers on the classification performed.

Objective: In this paper, we investigated three thresholds calculation techniques that can be used for fault-proneness prediction: ROC Curves, VARL (Value of an Acceptable Risk Level) and Alves rankings. We compared the performance of these techniques with the performance of four machine learning and two clustering based models.

Method: Threshold values were calculated on a total of twelve different public datasets: eleven from the PROMISE Repository and another based on the Eclipse project. Thresholds-based models were then constructed using each thresholds calculation technique investigated. For comparison, results were also computed for supervised machine learning and clustering based models. Inter-dataset experimentation between different systems and versions of a same system was performed.

Results: Results show that ROC Curves is the best performing method among the three thresholds calculation methods investigated, closely followed by Alves Rankings. VARL method didn’t give valuable results for most of the datasets investigated and was easily outperformed by the two other methods. Results also show that thresholds-based models using ROC Curves outperformed machine learning and clustering based models.

Conclusion: The best of the three thresholds calculation techniques for fault-proneness prediction is ROC Curves, but Alves Rankings is a good choice too. In fact, the advantage of Alves Rankings over ROC Curves technique is that it is completely unsupervised and can therefore give pertinent threshold values when fault data is not available.

SPIRITuS: a SimPle Information Retrieval regressIon Test Selection approach. Simone Romano, Giuseppe Scanniello (University of Basilicata), Giuliano Antoniol (Ecole Polytech. de Montreal), Alessandro Marchetto

https://www.sciencedirect.com/science/article/pii/S0950584918300405​

Abstract

Context: Regression Test case Selection (RTS) approaches aim at selecting only those test cases of a test suite that exercise changed parts of the System Under Test (SUT) or parts affected by changes.

Objective: We present SPIRITuS (SimPle Information Retrieval regressIon Test Selection approach). It uses method code coverage information and a Vector Space Model to select test cases to be run. In a nutshell, the extent of a lexical modification to a method is used to decide if a test case has to be selected. The main design goals of SPIRITuS are to be: (i)easy to adapt to different programming languages and (ii) tunable via an easy to understand threshold.

Method: To assess SPIRITuS, we conducted a large experiment on 389 faulty versions of 14 open-source programs implemented in Java. We were mainly interested in investigating the tradeoff between the number of selected test cases from the original test suite and fault detection effectiveness. We also compared SPIRITuS against well-known RTS approaches.

Results: SPIRITuS selects a number of test cases significantly smaller than the number of test cases the other approaches select at the price of a slight reduction in fault detection capability.

Conclusions: SPIRITuS can be considered a viable competitor of existing test case selection approaches especially when the average number of test cases covering a modified method increases (such information can be easily derived before test case selection takes place).