Predicting Code Modifications using BERT Model and Code Histories

Abstract

The use of deep learning-based techniques has emerged in various code analysis tasks. One of the most important tasks for refactoring and vulnerability detection is to identify points in the code that are likely to change. This study aims to develop a deep learning-based system for code modification detection. A pre-trained BERT model and recurrent neural networks were used to detect the type of code modification. We created an experimental dataset by collecting code from open-source GitHub repositories. The proposed system achieves an average accuracy of about 87%, and the use of history information increases the accuracy by 1.5 points. These results suggest that the BERT model can improve the accuracy of code modification detection by using code history information. Thus, DNN techniques can guide developers through detecting software code changes.