Tutorial "Mining Programs and Processes": Assignments

Arrangements

• Send your solutions electronically (see instructions below) to Andreas Zeller <zeller@cs.uni-sb.de>
• This e-mail address also can be used for questions.
• Please turn in assignments by July 15, 2008.

Assignment 1: Simplifying Input

This project requires you to program in Python. Learning Python is not hard, though; and even if you include the time it takes you to learn Python, you will still complete your work faster than in most other languages.

Read this first

• Tutorial Slides
• Chapter 5 "Simplifying Problems" of the "Why Programs Fail" book (Slides)
• Paper: Simplifying and Isolating Failure-Inducing Input
• Python documentation and tutorial

The failure

1. Download the XMLProc parser. It should unpack into a directory called xmlproc.
2. In the xmlproc dictionary, the file xpcmd.py is a command-line interface to the XML parser. To have the parser parse the XML file demo/nstest1.xml, type

   $ cd xmlproc
   $ python xpcmd.py demo/nstest1.xml
   xmlproc version 0.70
   
   Parsing 'demo/nstest1.xml'
   Parse complete, 0 error(s) and 0 warning(s)
   $ _
   

3. The input file demo/urls.xml causes the parser to fail:

   $ python xpcmd.py demo/urls.xml
   xmlproc version 0.70
   
   Traceback (most recent call last):
     File "xpcmd.py", line 112, in ?
       p.parse_resource(sysid) [...]
     File "xmlproc/xml/parsers/xmlproc/xmlproc.py", line 401, in parse_charref
       if digs==None: return
   UnboundLocalError: local variable 'digs' referenced before assignment
   $ _
   

Your task

Step 1: Write a testing function.

To invoke the parser, you have two options:

• Invoke the parser as a separate program. Use Python's os module to execute the parser. You can use fork(), execv(), and wait() functions, as in Figure 5.11; however, for this particular task, system() or popen() are far easier to use. Python's built-in File Objects are useful in communicating with the parser via files.
• Invoke the parser directly from Python. Take a look at the xpcmd.py source to see how this works. The essential lines are

  from xml.parsers.xmlproc import xmlproc
  	
  app = xmlproc.Application()
  p = xmlproc.XMLProcessor()
  p.parse_resource(file)

  In order to catch the exception thrown by the parser, be sure to read and understand exception handling in Python.

• The parser completely parses the file (PASS), without errors or warnings;
• The parser fails as in the original failure (FAIL)
• The parser has another outcome, in particular syntax errors (UNRESOLVED)

Step 2: Write a splitting function.

In the long term, you may want to split the input along token delimiters, or even use syntactic simplification (Section 5.8.3).

Step 3: Attach Delta Debugging.

Finally, you need the ddmin() function from Figure 5.7. This ddmin module even comes with a self-contained test function.

Step 4: Choose a representation.

• If you want to simplify input, each delta might be a single character—thus, the string "defect" becomes the list ['d', 'e', 'f', 'e', 'c', 't']. However, this will cause trouble as you cannot distinguish between the same character at different locations. For instance, listminus(['d', 'e', 'f', 'e', 'c', 't'], ['e']) = ['d', 'f', 'c', 't']—which is not what you want.
• A better solution is to store characters with their indices, as in [('d', 1), ('e', 2), ('f', 3), ('e', 4), ('c', 5), ('t', 6)]—this way, each circumstance can be uniquely identified.
• For a more general solution, you may wish to set up Circumstance or Delta classes and to store lists of these objects.

Step 5: Run it.

Step 6: Document it.

Have a README file that describes how to invoke the simplification program.

Step 7 (optional): Make it more general.

Generalize the delta debugging function such that it can be parameterized with an arbitrary unit test and an arbitrary input—that is, it becomes independent from XMLProc and can easily be adapted to other input minimization tasks. (As examples of such tasks, see the paper Simplifying and Isolating Failure-Inducing Input.) Eventually, you will have a general stand-alone framework; to have it solve the XMLProc problem, instantiate it with the XMLProc unit test and the failing input.

Step 8 (optional): Improve efficiency.

Assignment 2: Mine a Bug Database

You do not interact with version and bug databases directly; instead, you operate on the iBugs data set which contains all the necessary data in XML format.

Read this first

• Tutorial Slides
• Chapter 4 "Predicting Bugs from History" of the "Software Evolution" book (PDF)
• The iBugs project (www.ibugs.org) and the papers referred therein.
• Learn how to process XML data in your favorite programming language.

The data

• The AspectJ source code (in aspectj.org)
• Defect data from iBugs (in repository.xml)

Each bug, among other characteristics, lists the file(s) where the bug was fixed, enclosed in <fixedFiles>...</fixedFiles>. Each <file> has a name attribute listing the file that was fixed.

A property name="severity" attribute lists the severity of the bug, from enhancement over minor, normal, and major to critical.

Task 1: Find the most defect-prone files.

A common observation in software systems is Pareto's Law: 80% of the defects can be found in 20% of the locations. Is this true for AspectJ, too?

Task 2: Find the most defect-prone directories.

Note: Be sure to count every bug only once per directory. That is, a single bug whose fix affects foo/a.c and foo/b.c should be counted only once for foo/.

Task 3: Hypothesize.

Please note: As the supplied AspectJ source code is just a snapshot, it does not contain every file referred in repository.xml, which records a multi-year version history.

Task 4 (optional): Where are the most critical bugs?