Tag: testing

What is a bug?

I often find myself trying to explain what a bug is to people who have less experience with them, so this is an attempt to try and lift the lid.

Bugs are things that do not work as intended. Their name comes from the original bug, which was an insect that got squashed onto a punched card, which meant that the instructions that should have been followed were read incorrectly. In modern times they might be spelling errors in languages that are not strict about declaration, however the tools we use to write code, will normally be looking for this and highlighting where things look wrong.

Some bugs are simple

Simple bugs are things you can usually resolve within a two dimensional context. Everything you need to understand how they go wrong is in one file, you just need to read left to right (or RTL) and top to bottom.

These bugs might be as simple as a check for two things being the same x==y, mistyped so that it becomes a setter x=y.

Some bugs are complicated

As applications grow, they tend to be expressed in ways that make code easier to read, easier to understand, easier to reuse, but not always easier to debug. Code gets spread across multiple files, broken into units where the code in one file is somewhat related to each other. However your application is combining these groups of code together, so the runtime execution jumps around from file to file. When trying to find bugs, you’ve moved from 2D representations where everything is visible, onto 3D where you need to be in the right file to start looking down and right.

One common issue at this level might be where the usage of a function in one file, mismatches the definition in another. For example we create a method that subtracts one number from another Subtract(x,y), and then call it with the arguments the wrong way around Subtract(y,x).

Other bugs might be where we change a value somewhere but never update the original, or conversely update a value that everything expects not to change. Often we are starting to see side effects to our code being the cause of subtle problems.

Some bugs are complex

Even with code that works perfectly in one environment, there are other factors that can make it break in another. The first to consider is time, which counter intuitively is more than one issue.

Code might run where it has the machine to itself and have no issues, then run again later when other things are happening and hit limitations. These might be resource constraints, like running out of memory, or not being connected to the internet. Defining the conditions that caused the problem can often be a bigger challenge than recreating them. It’s very common for your code to be run by people who are less precise in their description of the problem.

Another problem is that often we make our code do more than one thing at once, for performance. The number of things that happen and speed that they happen can vary across systems or even on the same machine. These can mean things happen out of order and your bug is where you are not prepared for these. It becomes a race for which finishes first, hence these bugs are known as race conditions.

The next time related issue is where the version of the environment changes. Your code works perfectly with your current environment, but with older or newer dependencies, it breaks. Conversely someone might be running an old version of your code, so you have to consider what your code would have done previously and what state things might be in now. Did you write a file in an older format, but now your code is the newer version? Did someone use the newer version of your code to write a file and its been sent to a machine with the older version?

At this point we are starting to see that the number of factors to consider is expanding. We don’t just look up and down, left and right, ie. 2D. We don’t only look between files, 3D. We have to think about when it runs 4D, what version of it’s environment its in 5D, and when it’s data was produced 6D.

Talk about n-dimensional chess!

Functionality v Framework

This is going to sound so simple, so bear with me, but at the moment its becoming my personal mantra.

There is a difference between the functionality of a system and the framework it runs in.

Lets take a simple example, we have a process at work. Basically it gets a list of files from a database copies the files over and updates a different database to point to the new files. It is currently hosted in a windows service. Can you separate the functionality from the framework?

Functionality

It gets a list of files from a database copies the files over and updates a different database to point to the new files

Framework

Hosted as a service

Why does this matter?

Well the problem is that I also need to run the functionality on ad-hoc basis with some special parameters. So what do we do, well simply we split the functionality from the framework, put the functionality in a separate assembly, call it once from the service and again separately from a console application or a WinForms GUI.

But that’s only the first level

We can consider the same at a method level, where we have a for loop with a code body. It is impossible to test a single iteration of the logic in the for body without going through the framework. 

            foreach (string name in _list.Keys)
            {
                progress.DoOne("Adding " + name);

                TableRow tr = resultsTable.AddRow();
                tr.AddCell().Value = name;
                Dictionary<DateTime, TestResult> data = _list[name];
                //now add cells
                foreach (DateTime dt in _dates)
                {
                    TableCell cell = tr.AddCell();
                    if (data.ContainsKey(dt))
                    {
                        switch (data[dt])
                        {
                            case TestResult.Fail:
                                cell.Value = "Fail";
                                cell.BGColor = Color.Red;
                                break;

                            case TestResult.Pass:
                                cell.Value = "Ok";
                                cell.BGColor = Color.Green;
                                break;

                            case TestResult.Untested:
                                cell.Value = "";
                                cell.BGColor = Color.Yellow;
                                break;

                        }
                    }
                }

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

What is more is that we can ONLY call this functionality with its framework. We can refactor the body of a for loop out to a method that takes parameters or even a Visitor pattern. This lets us call it separately, for example we can now Unit Test the functionality.

At the class level we can also refactor so that we separate the multi-threading code out and leave the business logic. This lets us host the logic with or without the threading. So maybe, once again we can unit test the logic separately, or host in different applications as before.

So what does this mean?

I think it might be time for Al’s First law (see Haacked’s law).

The greatest barrier to reuse and testing is a mix of functionality and framework in the same unit.

I wonder if there will ever be 2nd?