Assessing the Refactorability of Software Clones

Abstract

The presence of duplicated code in software systems is significant and several studies have shown that clones can be potentially harmful with respect to the maintainability and evolution of the source code. Despite the significance of the problem, there is still limited support for eliminating software clones through refactoring, because the unification and merging of duplicated code is a very challenging problem, especially when software clones have gone through several modifications after their initial introduction. In this work, we propose an approach for automatically assessing whether a pair of clones can be safely refactored without changing the behavior of the program. In particular, our approach examines if the differences present between the clones can be safely parameterized without causing any side-effects. The evaluation results have shown that the clones assessed as refactorable by our approach can be indeed refactored without causing any compile errors or test failures. Additionally, the computational cost of the proposed approach is negligible (less than a second) in the vast majority of the examined cases. Finally, we perform a large-scale empirical study on over a million clone pairs detected by four different clone detection tools in nine open-source projects to investigate how refactorability is affected by different clone properties and tool configuration options.

@article{Tsantalis:2015,
	author={Tsantalis, Nikolaos and Mazinanian, Davood and Krishnan, Giri P.}, 
	journal={IEEE Transactions on Software Engineering}, 
	title={Assessing the Refactorability of Software Clones}, 
	year={2015}, 
	volume={41}, 
	number={11}, 
	pages={1055-1090}, 
	doi={10.1109/TSE.2015.2448531}, 
	ISSN={0098-5589}, 
	month={Nov},
}

Experiment data

We have provided the resulting files (input and output Excel files, HTML reports, CSV files and real code fragments) for every project and every tool separately in a .7z file. You can click on the corresponding cell to get the file.

Testing Results

To evaluate the correctness of our approach we refactored 610 clone pairs detected by Deckard in the src directory of JFreeChart 1.0.10 that were assessed as refactorable and were covered by unit tests.
We used the EclEmma Java code coverage Eclipse plugin to assess the test coverage of the clone fragments.
We provide the testing results in JFreeChart-1.0.10-tests.xls (that is located inside the .7z file).
For every clone fragment (i.e. row in the Excel file) we indicate whether it is fully covered, partially covered, or not covered by tests.

Gallery

• JFreeChart 1.0.10 -- Reordered statements.
• JFreeChart 1.0.10 -- Symmetrical if/else if chains, and symmetrical if/else match.
• JFreeChart 1.0.10 -- Reordered statements, symmetrical if/else if chains, and symmetrical if/else match.
• Apache Ant 1.9.0 -- Replacement of field accesses with getter method calls, replacement of field assignments with setter method calls, and replacement of this reference with a variable (i.e., task).
• Hibernate 3.3.2 -- Refactorable Type-3 clone pair.
• JFreeChart 1.0.10 -- Refactorable Type-3 clone pair.
• JFreeChart 1.0.10 -- Refactorable Type-3 clone pair.
• EMF 2.4.1 -- The largest clone found (314 statements)

Tools installation/usage guide

Installation

1. Download the Headless application and JDeodorant plug-in and import them as existing projects into your workspace.
2. Right-click on the eclipse.commandline project and select "Run As" > "Run Configurations..."
3. Click on "Eclipse Application" and then on the "New launch configuration" button. Give a name to the newly created launch configuration (e.g., eclipse.commandline as shown in the screenshot below).
4. In the "Main" tab
   a) In the "Workspace Data" setup the "Location" to point to the workspace containing the projects that you want to analyze in headless mode. This must be a workspace directory created by Eclipse. You can create such a workspace by clicking on "File" > "Switch Workspace" and specifying a new workspace directory.
   b) In the "Program to Run" select to "Run an application" and from the drop-down list select "ca.concordia.jdeodorant.eclipse.commandline.application" as shown in the screenshot below.
   
5. In the "Arguments" tab specify the "Program arguments" as explained in Section 5.2.
   Next, specify the "VM arguments" as -Xms128m -Xmx4096m -XX:PermSize=128m (you can increase the Xmx value, if more memory is available).
   

   Note: For your convenience, we have made available, for every project and every tool, the launch configuration necessary for running the tool. Inside each of the given 7z files in Table 1, there is a file with .launch extension. You can import this file in Eclipse from File > Import > Run/Debug > Launch Configurations. However, you will need to update the absolute paths in the arguments.

6. In the "Plug-ins" tab first select "plug-ins selected below only" in the "Launch with:" drop-down list. Then select "ca.concordia.jdeodorant.eclipse.commandline (1.0.0.qualifier)" and click on "Add Required Plug-ins" button.
   
7. Apply the changes in order to save the new Launch Configuration.
8. Click Run to test whether the headless plug-in works properly. If you are getting BundleExceptions, go back to the "Plug-ins" tab and select Launch with: "all workspace and enabled target plug-ins". Apply the changes and Run again the headless plug-in.

Command-line arguments

Mode of Operation

The headless application works in three different modes. These modes are explained in the following table. For running the tool in each of these modes, use appropriate value for --mode (or -m) argument.

The input (and output) Excel files

The input Excel file must be in Excel 97-2003 (.xls) format. Please note that, the tool cannot handle .xlsx files. This excel file will contain the columns, as shown in Table 5. The first row of the Excel file is used as header row. In Table 5, column names are sorted as they appear in the Excel file. For the analysis, the input Excel file must contain the information for some of the columns, while for other columns, the cells will be filled during the analysis.

In the Excel file, each row is for one clone. Each clone is a code fragment which is detected to be duplicated in another part of the system. Several clones in the consecutive rows belong to one clone group. Hence, each possible pair of clones inside a clone group are code fragments that are duplicated. The row corresponding to the first clone of every clone group contains some information about the clone group, including values for Clone Group Size, Clone Group Info and Connected columns. Please find the information about these columns in the following table.

A sample empty Excel file is provided here. Also, the the Excel files that we generated for this paper are provided in Table 1.

Using the output of clone detection tools

The output of a clone detection tool must be first converted to the desired Excel file. For convenience, we have provided parsers for the popular clone detection tools, as an internal feature in the command-line tool.

When the tool is executed in the PARSE or PARSE_AND_ANALYZE modes, user has to provide the tool with the output file of the clone detection tool, using --tooloutputfile (-i) argument. Also, the name of the clone detector must be specified using --tool (-t) argument. For example, the following arguments can be used to parse and analyze an output from CCFinder for project Apache Ant:

For the moment the tool supports four different clone detection tools, as shown in the table below. The value for --extra-args- (-xargs) argument depends upon the tool, and provides necessary information for parsing the input file. For instance, in this example we have provided two additional strings through this argument, separated by comma.

Output of the commandline tool

Tool generates an Excel file, with the same name (appended by -analyze) and in the same path as the input Excel file which contains the results of the analysis. You can find the information about the columns of this Excel file in Table 5. The HTML reports of the analysis can be found in a folder named html.reports which is located in the same folder as the input and output Excel files. You can see samples of these HTML reports in Section 4.

When the tool is used to parse the output of a clone detection tool, a folder named code-fragments in the same path as the input and output Excel files is created, which contains the real code fragments as reported by the clone detection tool. The names of these files are in the format {ID}-{CLONE_NUMBER}, where {ID} is the ID of the corresponding clone group to which this clone belongs, and {CLONE_NUMBER} is the clone's index in current clone group. This helps in mapping Excel file rows (clones) to these files.

For those who are interested in performing statistical analysis using tools such as R, Matlab, etc, the tool generates CSV files containing information gathered during analysis. Three CSV files are created, as explaned below. Please note that, the separator in these files is pipe ("|") character. The first row of these files is header.

{INPUT_EXCEL_FILE_NAME}.report.csv

Contains general information about the refactorability analysis results. Every row in these files corresponds to a single clone pair. The columns in the order they appear in the CSV files are:

{INPUT_EXCEL_FILE_NAME}.trees.csv

For every clone pair, more than one subtree may be found which could be refactorable or not. This file contains the information about every subtree. The columns in the order they appear in the CSV files are:

{INPUT_EXCEL_FILE_NAME}.precondviolations.csv

This file contains information about precondition violations for each subtree, if the subtree was not found to be refactorable. The columns in the order they appear in the CSV files are:

R Scripts

You can download the R scripts we developed for the analysis of the experiment data from here. The scripts and CSV files can be found inside the Scripts and CSVs folders, respectively. If you use R Studio, you can open the R Project by opening the file RProject.Rproj. The script files are:

• load.R: contains the function load() which loads all the necessary tables to the environment. By default, this function looks at a directory named CSVs in the current working directory and loads all the CSV files in that directory. You may provide the folder containing CSV files using the argument for this function.
• refactorable-clone-pairs.R: generates the information about the refactorability of clones. It contains the following functions:
  • printProductionCodeResultsTable() and printTestCodeResultsTable: Prints the table about refactorability of production and test code, respectively, for every project and every clone detection tool. For getting the information about all the results, use function printAllCodeResultsTable().
  • testVSProduction(): Displays the information about the refactorability of test vs production code for every clone detection tool.
  • printRefactorableLocation(): Displays the information about the refactorability of code in different locations for every clone detection tool.
  • printRefactorableCloneType(): Displays the information about the refactorability of different clone types for every clone detection tool.
• efficiency.R: contains the function plotNumberOfNodeComparisons() which generates a plot showing the number of node comparisons performed by our technique vs. a brute force approach.
• size.R: contains two functions:
  • plotSizes(): generates a plot showing the size of the clones for each clone detection tool.
  • plotSizeRefactorable(): generates a plot showing the size of the refactorable and non-refactorable clones for each clone detection tool.

Clone detection tools

In this experiment, we used the results of four popular clone detection tools for the experiments.

CCFinder

All you need to run CCFinder is to download and install CCFinderX, a major version of CCFinder, from the given URL. (There are some documentation for installing CCFinder on Ubuntu Linux here, while we installed and tested it on Windows. Note that, you will need Phyton 2.6 installed on your operating system (we used 2.6.5, Python 2.7 or later versions didn't work). Also, JRE version 5 or later must also be installed (we tried version 7 Update 51).

CCFinderX is a major version of CCFinder which provides a useful GUI for using CCFinder. After installation, you can run the tool by running /bin/gemx.bat (on Windows). From GUI, select File > Detect Clones. Select your desired programming language, add the folders of the source files of the project under investigation, and select Next. In the next window, you can configure CCFinder for clone detection. As mentioned in the paper, we used these options for our analysis:

Deckard

We compiled Deckard in Ubuntu 13.04 X86, using the steps given in the README on Deckard's Github page. The running instructions are also given in the same page. Note that, you only need to run Deckard using what is described in the first part of the Usage section in the mentioned page (For clone detection, using file deckard.sh).

Note: we were not able to run the version of Deckard on Github, it has a problem in the vector generation phase. We used the one given in the alternative URL and it worked.

To configure Deckard, you have to modify the config file in the same folder that deckard.sh file exists. You can follow the comments in the config file to make sure that you config it correctly. In this research, we have used the following values for the options in the condig file:

CloneDR

We run CloneDR on Windows 8.1. You should contact Semantic Designs to request an academic license for CloneDR, because the evaluation version reports only 10 clone groups. After installation and registration, run DMS Project Specifier tool, configure it with the following options, select the path of the project you are going to analyze and run the tool.

Nicad

We installed Nicad on Ubuntu Linux 14.04 LTS X64. Please note that, to run Nicad, you'll first need to download and install TXL from here.

The instructions found in the readme file in the Nicad's folder is straightforward for installing and running it. Please note that, as its authors have changed the name of the runnable file to "nicad3", in the "Testing Nicad3" and "Using Nicad3" sections of the readme file, instead of running nicad functions java... in the terminal, you should run nicad3 functions java....

For configuting Nicad, you have to make a copy of the default.cfg file in the config folder in the Nicad's directory, and change the desired parameters in it. Below are the parameters we changed from default config file, which were used for running Nicad in this experiment. Note that, you have to append the name of the config file (without extension) at the end of your command which runs Nicad (see the mentioned Readme file for the sample commands.)