Compiling and Testing Your Code

Compiling your code

We will be using make, a tool that automatically builds executables from source code by reading a file (named Makefile) that specifies how to construct a target program. We will provide the Makefile. For the first short exercises, the Makefile has targets for compiling your code, for generating your score on our automated tests, and for cleaning up generated files. To use this file to compile your code, run one of the following commands from the Linux command-line:

$ make student_test_se1
$ make test_se1
$ make

(Remember to leave out the $ prompt.)

The first command will compile your code in src/se1.c along with any tests you have added to src/student_test_se1.c and if possible, generate an executable (bin/student_test_se1). The second will compile your code and, if possible, generate an executable (bin/test_se1) with the automated tests. The third will do both.

Testing your code

Once you’ve written a piece of code, you will want to test that it works as expected. There are broadly two ways of doing this:

  • Manual testing: This just involves running the code you wrote with some sample values, to check whether it behaves as expected. For example, if you were writing a function to compute whether a year is a leap year, you may try adding code to the main function in src/student_test_se1.c to call your function with a few leap years and a few non-leap years and print out the results to see whether your code correctly identifies the leap years. You would then compile this code and run it as follows:

    $ make student_test_se1
    $ bin/student_test_se1
    
  • Automated testing: There are automated testing frameworks that allow you to specify a series of tests you want to run on your code, and which make it easy to automatically re-run all those tests. For example, following the leap year example, you wouldn’t have to manually test each year value one by one; instead, the testing framework would test all these year values for you, and would report back how many produced the expected result.

All the exercises and programming assignments include a suite of automated tests that you can use to check whether your code is working correctly, Furthermore, these tests also factor into your score for an exercise or programming assignment.

Because the automated tests are easy to run (and affect your score for the assignment), you may be tempted to do the following: write some code, immediately try running the automated tests to find a test that fails, make a guess as to how to modify your code, and then repeat the process until all of the tests pass.

This is not a good way to test your code. Instead, you should start by doing some manual testing to get a sense of whether it is working before you try the automated tests. In this page, we describe how to do this.

Manual testing

You will find a file named student_test_se1.c in the src directory. You can add code to call your functions with sample input to this file. We encourage you to do some testing by hand before you run the automated tests.

In short exercises like the one above, it is sometimes enough to look at the code and figure our why it is not working. However, for more complex code, especially once you move on to the programming assignments, you will want to follow a more rigorous approach. You should make a hypothesis about what might be wrong and use printf statements to print out key values to help you verify or disprove your hypothesis. (Eventually, we will teach you how to use a debugger named lldb instead of using printf statements.) You can also find a lot of tips on how to debug your code in The Debugging Guide

After you’ve done some manual testing, and get the sense that your function seems to be working for, at least, a few simple inputs, you should try running the automated tests. The tests could reveal that there are still issues with your function and, at that point, you could repeat the same process we described above. The important thing is that you always have an idea in your head about how your code works, and don’t make random changes that you don’t fully understand.

Automated testing

We will be using the Criterion testing framework. The Makefile contains the information necessary to generate the automated tests. Just run make test_se1 to generate the executable for the automated tests.

To run all of the test code, run this command from within your se1 directory (after running make):

$ bin/test_se1 -f --verbose

The -f flag tells the test code to stop after the first failure. The --verbose flag generates information which specific tests passed and failed.

The tests are broken into suites. To run a specific test, you can add a filter. For example, to run all of the vec_length tests, you would run:

$ bin/test_se1 -f --verbose --filter="vec_length/*"

To run a specific test replace the * with the name of the test. For example, the first test for the vec_length suite, you would run:

$ bin/test_se1 -f --verbose --filter="vec_length/test0"

Obtaining your test score

Your score on the exercises, as well as the “Completeness” portion of the programming assignments, is determined by the automated tests. We have include a target in the Makefile to generate your score. To do so, simply run the following:

$ make gen_score