Wednesday, December 14th 2011, 12:00:00 am

Shame on me.

I know you caught it in that Let's Make a Language: C♭ - Part 2.

I didn’t write any tests. Not a one.

Shame on me.

Convert ignominy to esteem

Let’s rectify the situation by writing some tests to make sure that our code does what we want it to do and to safeguard against future side effects. You can download the 20111213.zip as a starting point.

Create the test project

If you’re following along and using Monodevelop, add a new C# “NUnit Library Project” named “cflat.tests”. This comes with NUnit 2.4.8, so the includes will differ from the newer version of NUnit.

If you’re following along and using Visual Studio 2010, download and install NUnit, create a new C# “Library” named “cflat.tests”, and add a reference to the NUnit library installed in the GAC or locally.

If you’re not following along, then go back to your baba ghanouj and skate.

Now, add to the “cflat.tests” project a reference to the “cflat” project.

Rename the default file to “Needs300LinesTests.cs” and ensure the contents of the file look a lot like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
using System;
using System.IO;
using NUnit.Framework;
using System.Diagnostics;
using NUnit.Framework.SyntaxHelpers;

namespace cflat.tests
{
  [TestFixture()]
  public class Needs300LinesTests
  {
  }
}

Black-box testing

I elect to perform black-box testing on the cflat project. This means that I won’t test the inner functions, just test it through the publicly exposed API. In this case, an executable, that means running a process.

This is very different from unit testing. I just want to make that clear.

Success scenario

First off, let’s test the good case, that a file of more than 300 lines exists, the C♭ transcompiler will consume it, and we should get no message on stderr and the return code should equal 0. Test-writing time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[Test()]
public void Compiler_does_not_err_with_file_containing_300_newline_chars()
{
  string sourcePath = Path.GetTempFileName();
  using(StreamWriter source = new StreamWriter(sourcePath)) {
    for(int i = 1; i <= 300; i += 1) {
      source.WriteLine();
    }
  }
  ProcessStartInfo startInfo = new ProcessStartInfo();
  startInfo.Arguments = sourcePath;
  startInfo.FileName = "cflat.exe";
  startInfo.UseShellExecute = false;
  startInfo.WorkingDirectory = Environment.CurrentDirectory;
  startInfo.RedirectStandardError = true;
  Process compiler = new Process();
  compiler.StartInfo = startInfo;
  compiler.Start();
  compiler.WaitForExit(10000);
  Assert.That(compiler.ExitCode, Is.EqualTo(0));
  Assert.That(compiler.StandardError.ReadToEnd().Length, Is.EqualTo(0));
}

I get a big ol’ green light when I run that test.

Failing scenario

Let’s do the same thing but with a file that contains 299 new-line characters. Copy-And-Paste Commando says:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
[Test()]
public void Compiler_errs_with_file_containing_less_than_300_newline_chars()
{
  string sourcePath = Path.GetTempFileName();
  using(StreamWriter source = new StreamWriter(sourcePath)) {
    for(int i = 1; i <= 299; i += 1) {
      source.WriteLine();
    }
  }
  ProcessStartInfo startInfo = new ProcessStartInfo();
  startInfo.Arguments = sourcePath;
  startInfo.FileName = "cflat.exe";
  startInfo.UseShellExecute = false;
  startInfo.WorkingDirectory = Environment.CurrentDirectory;
  startInfo.RedirectStandardError = true;
  Process compiler = new Process();
  compiler.StartInfo = startInfo;
  compiler.Start();
  compiler.WaitForExit(10000);
  Assert.That(compiler.ExitCode, Is.Not.EqualTo(0));
  Assert.That(compiler.StandardError.ReadToEnd().Length, Is.GreaterThan(0));
}

Again, big ol’ green light. Sweet!

Refactoring

Ok, Copy-And-Paste Commando: BAD. So, let’s fix the test file. We can extract the file generation and the process bootstrapping. Really, if we can test the content of the process’ stderr and its exit code, our assertions will change very little. So, move all of that repeated, boilerplate, file-generating, process-starting code out of both methods into a method named CreateFileAndRunCompilerOnIt(int). And, Shazam!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[Test()]
public void Compiler_does_not_err_with_file_containing_300_newline_chars()
{
  CreateFileAndRunCompilerOnIt(300);
  Assert.That(exitCode, Is.EqualTo(0));
  Assert.That(stderrContent, Is.EqualTo(""));
}

[Test()]
public void Compiler_errs_with_file_containing_less_than_300_newline_chars()
{
  CreateFileAndRunCompilerOnIt(299);
  Assert.That(exitCode, Is.Not.EqualTo(0));
  Assert.That(stderrContent.Length, Is.GreaterThan(0));
}

That improves the code’s comprehensibility by a gazillion.

You can download the final code for this post at 20111214.zip.