From time to time I get to teach and mentor Java developers on the fine art of unit testing. There are many similarities when unit testing in Java and .NET but more interesting are the differences between the two.

Faking objects in Java using Mockito

One of the well-used Java Isolation (Mocking) frameworks is called Mockito which is easy to use, and has an API that reminds me of FakeItEasy, Isolator or Moq (as well as others). Creating a new fake object using Mockito is as simple as writing mock(SomeDependecy.class).
Mockito also enables automatically creating fakes, spies (partial fakes) and classes with dependencies using annotations (similar to attributes in Java):
public class ArticleManagerTest extends SampleBaseTestCase {

    private ArticleCalculator calculator;

    @Mock(name = "database") 
    private ArticleDatabase dbMock; // note the mock name attribute
    private UserProvider userProvider = new ConsumerUserProvider();
    private ArticleManager manager;

    public void shouldDoSomething() {

public class SampleBaseTestCase {

 public void initMocks() {
In the example above annotations are used to create fake objects and a real object which uses them. The end result is very similar to using an AutoMocking container.

What about NUnit?

I wanted to see if I can create similar functionality in the .NET world. At first I’ve tried using PostSharp but unfortunately the test runners had problems finding the new injected code. Instead I’ve decided to use NUnit’s ITestAction - a simple AOP for unit tests that seemed like a good fit.

NUnit has had the ability to execute code upon these events by decorating fixture classes and methods with the appropriate NUnit- provided attributes. Action Attributes allow the user to create custom attributes to encapsulate specific actions for use before or after any test is run.

I’ve created a simple attribute to mark fields that I wanted faked and named it (FakeItAttribute) and create a new attribute to enable discovery and creation of fake object for fields with that attribute:
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Interface, AllowMultiple = false)]
public abstract class AutoFakeAttributeBase : Attribute, ITestAction
    private readonly IFakeHelper _fakeHelper;

    protected AutoFakeAttributeBase(IFakeHelper fakeHelper)
        _fakeHelper = fakeHelper;

    private IEnumerable<FieldInfo> _testFields;

    public void BeforeTest(TestDetails details)
        var isTestFixture = details.Method == null;

        if (isTestFixture)


        foreach (var testField in _testFields)
            var fakeObject = _fakeHelper.DynamicallyCreateFakeObject(testField.FieldType);

            testField.SetValue(details.Fixture, fakeObject);

    public void AfterTest(TestDetails details)

    private void DiscoverFieldsToFake(TestDetails details)
        _testFields = details.Fixture.GetType()
            .GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic)
            .Where(testField => testField.CustomAttributes.Any(data => data.AttributeType == typeof(FakeItAttribute)));

    public ActionTargets Targets
        get { return ActionTargets.Test | ActionTargets.Suite; }
I’ve inherited that attribute with AutoFakeItEasyAttribute that uses FakeItEasy and reflection to create fake objects.
And now I can write tests that automatically create fake objects by adding that attribute.
[TestFixture, AutoFakeItEasy]
public class UsingSimpleClassTests
    private IDependency _fakeDependency;

    // Not faked
    private IDependency _uninitializedDependency;

    public void FakesCreatedAutomatically()
        Assert.That(_fakeDependency, Is.Not.Null);

    public void FieldsWithoutAttributesAreNotInitialized()
        Assert.That(_uninitializedDependency, Is.Null);
Quite cool, and yes it’s on GitHub.

Now what

I’m not sure if I like the use of fields in unit tests. I’ve seen it misused to create unreadable (and unmaintainable) tests more often than not – but at least now the fake fields are will be initialized between test runs. Now I know it’s possible to create AutoFaking using attributes and I considering adding more features until I fully implement AutoFakes.

Until then – happy coding…
In my not-so-distant past I needed to write a component which would control an air conditioning unit.
My application needed to send commands to the external device (on, off, set temperature) and read information from it (room temperature, humidity etc.)
And so I came with the following:
It’s a bit simplistic but the idea was that the client would be responsible for communication with the device (protocols, timeouts) while the manager would handle the “business logic” end of things (if temp > 10c than turn on cooling unit).
Commands from the manager would get translated to and unholy stream of bytes and sent to the device (and re-sent – numerous times) while data arriving from the device would be converted to strongly typed C# classes and passed to the manager where additional actions can be taken. Some of that data would end up in the application's data model according to the business rules inside the manager.
Obviously I developed the client and manager using TDD. While writing my tests (and later my classes) I’ve kept a clear division between the manager and client – each had it’s own set of tests and each had clear inputs and outputs (or so I thought).
A week later a new requirement came which caused a changed in my client and so I needed to re-write both the client’s tests as well as the manager’s tests. a few days later a bug was found – one that my tests should have caught but missed since it was a collaboration between the client and the manager that caused it. After a while I‘ve noticed that I’m doing something wrong – my tests would break whenever I’ve changed one of the methods between my client and manager. And so I’ve decided to test my manager and client as one “unit” using fake objects only for faking the device out of my tests. And it worked! I’ve managed to reduce the amount of tests while increasing my functionality – and there was much rejoicing.
That is until one day a co-worker of mine seen my tests – and told me that “those are not unit tests since you’re testing more than a single class”. It made me think about the actual definition of unit tests.

Maybe we should not call them unit tests anymore?

It all came back to me yesterday when read a post by Simon Brown: “Unit and integration are ambiguous names for tests”. In his post Simon explains that there is no clear definition of unit/integration tests (which is dead on the money) and come up with his own definitions which is clear (or at least clearer) and also helps understand which automatic tests a developer should write for each scope of the project – nicely aligned with the architecture.
Simon’s definitions are great. They help the align the developer’s test with the project architecture – something most of the projects I witnessed lack.
But there’s a small problem with splitting the tests into class-, component- and system- tests. While it set clear rules about their scope, what to fake, test size and speed - it does not help avoid testing the wrong thing…
I think that’s “class tests” falls short of explaining what we want to test namely the behavior of the code and not the actual implementation. I fear that calling a test “class test” would result in developers writing unit tests for every single class (and every single method). Those over specified tests tends to test trivial functionality while being extremely fragile.

Is there a definition for unit tests?

Since I needed an actual definition I went to the definite unit testing encyclopedia – XUnit Test Patterns and got the following definition for my troubles:
A test that verifies the behavior of some small part of the overall system. What makes a test a unit test is that the system under test (SUT) is a very small subset of the overall system and may be unrecognizable to someone who is not involved in building the software. The actual SUT may be as small as a single object or method that is a consequence of one or more design decisions although its behavior may also be traced back to some aspect of the functional requirements. There is no need for unit tests to be readable, recognizable or verifiable by the customer or business domain expert. Contrast this with a customer test which is derived almost entirely from the requirements and which should be verifiable by the customer. In eXtreme Programming, unit tests are also called developer tests or programmer tests.
[From: XUnit test patterns – Unit test]
So… Nothing other than only special people (someone who is involved in building the software) can know what the subject under test is?
It does states that a unit test can test a single class – or more…
There’s an excellent blog post by Martin Fowler – titled simply: UnitTest in which he tries to define unit tests but it seems that the only definite there is that unit tests should be fast – how fast? it depends!

What I use

I don’t have a clear answer here but I seem to go back to three different types of developer tests:
Unit Tests
Test a single unit of work (one class or more), and employ fake objects to prevent running external dependencies (DB, server, 3rd party and other classes). They are fast – usually runs for a fraction of a second. And they are relatively short (tens LOC) and simple.
Unit tests are independent from one another and each run on any machine should provide the exact same result.
Integration tests
They are similar to my unit tests in look and used to test interaction with external dependencies. I usually use Integration Tests when in need to test logic inside external dependency (think stored procedures) or interaction between my code and that external dependency.
They usually require setup (and cleanup) between test runs and a specific environment in place to pass. Fake objects can be used to avoid running other dependencies. They will be slower than unit test (seconds to minutes) and might fail due to reasons outside of the scope of the test (permissions, server down etc.)
Acceptance tests
Those are application, system wide or scenario level tests that make sure that business requirement are implemented according to spec.
They are slow and cross component boundaries and usually do not employ fake objects of any kind.


What I like about Simon’s methodology is that it’s easy to explain and set very good ground rules to the kind of tests a good project should have. I still think that the definition of class tests does not change the original definition (or dis-definition) of unit tests – while it’s about focusing on the operation of a single class it does not mean that other classes are not exercises as well.
It’s hard to define what is a “good” or “correct” unit test. I’ve noticed that there is no one rule for every project out there. And so I’ll keep on using my flawed yet accurate definition of “unit tests” – until I find a better definition.

And until then – Happy coding…

Broken windows and software development

No comments

Sunday, June 14, 2015

In 1969 a car with no license plates was parked with it’s hood up in the Bronx. Within minutes it was vandalized and stripped. That car was part of an experiment to test a theory called “broken windows”.

The Broken windows theory

The original broken windows theory was introduced in 1982:
Consider a building with a few broken windows. If the windows are not repaired, the tendency is for vandals to break a few more windows. Eventually, they may even break into the building, and if it's unoccupied, perhaps become squatters or light fires inside.
Or consider a pavement. Some litter accumulates. Soon, more litter accumulates. Eventually, people even start leaving bags of refuse from take-out restaurants there or even break into cars.
[Wilson, James Q; Kelling, George L (Mar 1982), "Broken Windows: The police and neighborhood safety", The Atlantic]
The bottom line is that we need to fix the small issues quickly in order to prevent bigger issues down the road. And by fixing these problems we avoid escalation and make sure that the neighborhood does not become a crime ridden cesspool, or the inspiration of the next Fallout game. 

What does it have to do with my code?

Code can also have broken windows – bad design, leftover comments that doesn't mean anything anymore and dead code. I’ve seen it happen countless times – long methods become longer (instead of refactored to smaller methods) and messy code becomes harder to read and maintain as time go by.
I remember one code review in which the developer didn’t bother with writing unit tests for a new feature since “the existing code did not have any unit tests in place”…
When code issues are left unattended development of new features becomes harder (or even impossible) and every change introduces new and exotic bugs. Weird governing processes are created – I’ve seen one place where is was common practice to have the equivalent of two weeks of meetings before a single line of code was written.
Broken windows in code leads to fear – fear of changing code, fear of writing new code and overall
The funny thing is that usually fixing broken windows in code is simple – refactoring when applied correctly can solve most issues before they become and impossible mess. Since most modern IDEs support at least a subset of the common refactoring (such as rename and extract to method) – these refactorings can be done safely and without breaking existing code.

Other broken windows in software development

But there are other ways in which we can unintentionally create a broken window that has nothing to do with our code. At one job we had to use a tool that made building a new version impossible to automate, no once know why we’ve been using it and no one could explain the merits on using it but every single time I’ve tried to replace it I was met with fierce resistance – since it’s the tool that all the big players are using.
If releasing a new version takes three days and a hundred steps of manual work – this could be a problem that can escalate quickly, and before you know it a wrong version is shipped to your customer.
And it could be the small things such as a version number of another dependency that seem to change without any logic behind it (yesterday version was, today’s is – this pseudo randomness can cause big problems once you need to revisit a change (hotfix anyone?) or distribute a new version using automatic update tool.
The solution for most of these ALM related is education and automation which seems like a lot of work at beginning but once it place saves quite a lot of time every single day.  


Broken windows are everywhere – it’s the small things that can becomes huge problems if left unchecked. It becomes a real problem once the development team (or the whole organization) accept them as a fact of life (a.k.a it’s the way it was always done here).
In the book “Clean code” Bob Martin explain about the “boy scout rule”:
If we all checked-in our code a little cleaner than when we checked it out, the code simply could not rot. The cleanup doesn't have to be something big. Change one variable name for the better, break up one function that's a little too large, eliminate one small bit of duplication, clean up one composite if statement.
Can you imagine working on a project where the code simply got better as time passed? Do you believe that any other option is professional? Indeed, isn't continuous improvement an intrinsic part of professionalism?
So pay attention to the small pains – it usually means that you can fix something today instead of waiting until it would be impossible to change. At once company we had a rule – if something annoyed us more than three times – we would change it. I still find this to be a useful way of finding ways to improve my code as well as the development process.

Happy coding…
Related Posts Plugin for WordPress, Blogger...