Replication Package for the Paper on "Towards Building a Universal Defect Prediction Model with Rank Transformed Predictors"
Software defects can lead to undesired results. Correcting defects costs 50% to 75% of the total software development budgets. To predict defective files, a prediction model must be built with predictors (e.g., software metrics) obtained from either a project itself (within-project) or from other projects (cross-project). A universal defect prediction model that is built from a large set of diverse projects would relieve the need to build and tailor prediction models for an individual project. A formidable obstacle to build a universal model is the variations in the distribution of predictors among projects of diverse contexts (e.g., size and programming language). Hence, we propose to cluster projects based on the similarity of the distribution of predictors, and derive the rank transformations using quantiles of predictors for a cluster. We fit the universal model on the transformed data of 1,385 open source projects hosted on SourceForge and GoogleCode. The universal model obtains prediction performance comparable to the within-project models, yields similar results when applied on five external projects (one Apache and four Eclipse projects), and performs similarly among projects with different context factors. At last, we investigate what predictors should be included in the universal model. We expect that this work could form a basis for future work on building a universal model and would lead to software support tools that incorporate it into a regular development workflow.
Our empirical study uses SourceForge projects as our subject projects.
We have two types of data sources: CVS repositories by Audris Mockus and D'Ambros et al. dataset.
This dataset was collected from SourceForge and GoogleCode. In our study, we select 1,385 projects. As the original dataset is large, we provide the list of subject projects, raw metric values, and converted metric values. Our primary tool for computing metrics is a commercial tool called Understand.
|List of subject projects||Download link|
|Raw metric values of subject projects||Download link|
|Converted metric values of subject projects||Download link|
This dataset was collected by D'Ambros. It contains four Eclipse projects (i.e., Eclipse JDT Core, Eclipse PDE UI, Equinox Framework, and Mylyn), and one Apache project (i.e., Lucene). This dataset is publicly available at http://bug.inf.usi.ch/download.php. In our study, we selected the common metrics from both our dataset and their dataset. They are: ... . We also provide the raw and rank-transformed metric values of these five projects.
|Raw metric values of five external projects||Download link|
|Converted metric values of five external projects||Download link|
The description of the approach can be found in the paper. Here, we put the results of the predictions, i.e., the performance measures of log-transformed and rank-transformed values of within-project models for each project. The results can be downloaded from this link.
The description of the approach can be found in the paper. Here, we put the results of the performances. For RQ2.1, the performance of three different metric set (i.e., code, code and process, code and process and context factors) can be downloaded from this link. For RQ2.2, the performance of within-project and universal models can be downloaded from this link.
The description of the approach can be found in the paper. Here, we put the results of the performances. The performance of within-project and universal models on five external projects can be downloaded from this link.
The description of the approach can be found in the paper. Here, we put the results of the performance. For each project, we present its context factors, and performance of the universal model on each project (when it is as the target project). The results can be downloaded from this link.
The results have been described clearly in the paper. And it can be obtained using the metrics from this link. So we don't present results for RQ5 here.
TO BE UPDATED.
IV.2 Unified Model
TO BE UPDATED.