Impact of Developer Reputation on Code Review Outcomes in OSS Projects: An Empirical Investigation


People

Amiangshu Bosu, Jeffrey C. Carver
Department of Computer Science
University of Alabama
Tuscaloosa, AL USA

Summary

Gaining an identity and building a good reputation are important motivations for Open Source Software (OSS) developers. It is unclear whether these motivations have any actual impact on OSS project success. To identify how an OSS developerís reputation affects the outcome of his/her code review requests. Method: We conducted a social network analysis (SNA) of the code review data from eight popular OSS projects. Working on the assumption that core developers have better reputation than peripheral developers, we developed an approach, Core Identification using K-means (CIK) to divide the OSS developers into core and periphery groups based on six SNA centrality measures. We then compared the outcome of the code review process for members of the two groups. The results suggest that the core developers receive quicker first feedback on their review request, complete the review process in shorter time, and are more likely to have their code changes accepted into the project codebase. Peripheral developers may have to wait 2 - 19 times (or 12 - 96 hours) longer than core developers for the review process of their code to complete. We recommend that projects allocate resources or create tool support to triage the code review requests to motivate prospective developers through quick feedback.


Gerrit repositories


Project Gerrit URL
1. Chromium OS https://gerrit.chromium.org/gerrit/#/q/status:open,n,z
2. ITK/VTK http://review.source.kitware.com/#/q/status:open,n,z
3. LibreOffice https://gerrit.wikimedia.org/r/#/q/status:open,n,z
4. OmapZoom http://review.omapzoom.org/#/q/status:open,n,z
5. OpenStack http://gerrit.openafs.org/#q,status:open,n,z
6. oVirt http://gerrit.ovirt.org/#/q/status:open,n,z
7. Qt Project https://codereview.qt-project.org/#q,status:open,n,z
8. Typo3 https://review.typo3.org/#/q/status:open,n,z



Replication package


Database schema  |  Sample database mined from Gerrit  | Sample Social network file ( Gephi file)  |  Sample SPSS file  

Research Hypotheses


H1. Core developers have shorter first feedback interval than the peripheral developers.
H2. Core developers have shorter review interval than the peripheral developers.
H3. Core developers have higher code acceptance rate than the peripheral developers.
H4. Core developers are able to get code changes accepted through lower number of patchsets than the peripheral developers.


Detailed Results of the statistical tests



Publications


  1. Bosu, A. and Carver, J., "Impact of Developer Reputation on Code Review Outcomes in OSS Projects: An Empirical Investigation", Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2014 pp.33:1--33:10, Torino, Italy



Acknowledgements


This research is partially supported by the US National Science Foundation Grant No. 1322276.