Putting the Cart Before the Horse

On TDD, unit testing and version control

  As I have been growing my software skills I have heard the phrase, “Test Driven Development” (TDD) many times, but have never had a chance to explore it personally. For those who do not know what TDD is, it is a method for writing code that flips around the normal paradigm: tests before code instead of code before tests. For many developers, the development process starts with writing a chunk of code (often written in a manner not conducive to testing) and then applying tests (or not, for those who test in production) to see what worked and what didn’t. TDD is a paradigm that operates in reverse and forces the developer to write tests first, and then to write the code.

  In TDD, failure transforms from a dreaded possibility into an objective. The benefit of this is that it encourages code progression in a more concrete way as the tests are based off of the application’s requirements. This is compounded by how these tests are designed. TDD stresses bite-sized unit tests so that fixing bugs is easier as there is less room for things to go wrong.

  The complete cycle of development for TDD is “red”, “green”, and “refactor”. The red stage is where tests are made that are guaranteed to fail until a functional condition is met. Once it is made and fails properly, the next stage begins. In the green stage, the developer is tasked with fixing the bare minimum required to satisfy the test. Doing extra work increases the risk of having under-tested code, and having harder to fix issues later on.

  The final stage, “refactoring”, is where the tests and the production code can be optimized. The “refactoring” stage can involve removing redundant tests.  If you create a new test that checks a function’s output, then an earlier test that only asserts that the function can be called becomes unnecessary. Pruning these help keep the test cycle quick. Another example is finding sections of code that are common to multiple tests and making one copy accessible to all of them.

  In Python, I have been doing TDD with Pytest. In the previous example, where we need to deduplicate code, we can proceed in several ways, including using pytest fixtures, and xUnit setup and teardown functions. Pytest fixtures are decorators that can operate at class, module or session levels to give pre-defined information, or tools. They’re also a good example of dependency injectors, and can be used with the older xUnit functions that do a similar job if desired. 

As an example of Pytest and Pytest fixtures, consider this class that takes a dictionary and has a method that finds values by key prefixes (with corresponding test code): 

import pytest

class Foo(object):
   def __init__(self, my_dict=None):
      self.my_dict = my_dict or {}

   def find_values(self, starts_with):
      matching_values = []
      for key, value in self.my_dict.items():
          if key.startswith(starts_with):
              matching_values.append(value)
      return matching_values

# Creation of foo happens 3 times...we want to reduce this.
def test_non_existant_key():
    foo = Foo({'foo': 'bar', 'foobar': 'baz'})
    assert len(foo.find_values('hello')) == 0

def test_one_matching_key():
    foo = Foo({'foo': 'bar', 'foobar': 'baz'})
    values = foo.find_values('foobar')
    assert len(values) == 1
    assert 'baz' in values
    assert 'bar' not in values

def test_two_matching_keys():
    foo = Foo({'foo': 'bar', 'foobar': 'baz'})
    values = foo.find_values('foo')
    assert len(values) == 2
    assert 'bar' in values
    assert 'baz' in values

We can clean up the code a little bit by using a pytest fixture:

import pytest

class Foo(object):
   def __init__(self, my_dict=None):
      self.my_dict = my_dict or {}

   def find_values(self, starts_with):
      matching_values = []
      for key, value in self.my_dict.items():
          if key.startswith(starts_with):
              matching_values.append(value)
      return matching_values

# fixture function name must be the same as the argument
# to the test functions
@pytest.fixture
def foo():
   return Foo({'foo': 'bar', 'foobar': 'baz'})

def test_non_existant_key(foo):
    assert len(foo.find_values('hello')) == 0

def test_one_matching_key(foo):
    values = foo.find_values('foobar')
    assert len(values) == 1
    assert 'baz' in values
    assert 'bar' not in values

def test_two_matching_keys(foo):
    values = foo.find_values('foo')
    assert len(values) == 2
    assert 'bar' in values
    assert 'baz' in values

   The result of using these tools is that you make your tests quicker to run and easier to read. Another perk to using TDD, besides thorough testing, is that it gives comfortable checkpoints for version control. While completing an online course on TDD, commits felt organic after each cycle or two. While this may not be practical for larger projects, making a commit after a small number of tests would still be very useful. Operating like this gives a steady pace to work at. This helps avoid a tendency of writing bloated sections that may need to be re-written if part of them isn’t needed.

   While getting used to this new paradigm can feel odd, it has definite benefits for projects of any size. A good rule of thumb is that If the project is big enough to use version control, then it’s a suitable candidate for TDD. As with any skill, practice makes perfect and TDD is no exception. I was pleasantly surprised at how the TDD process improved my development cycles, and believe there’s a lot of value in using it in most programming ventures to help create clean, maintainable and correct code.

Further Reading

Uncle Bob Martin’s Three Rules of TDD

Test Driven Development: By Example, Kent Beck (2002)

PyOhio 2018

If you have never been to a convention before, then I would highly recommend taking the time to attend at least one. They are an amazing way to meet new people, learn more about a certain field, and have fun! PyOhio 2018 was in no short supply of these, boasting many events to connect with people, the chance to work on interesting projects to advance a common goal and develop new skills, and a variety of informational talks.

A particular interest of mine in attending PyOhio was to network with people who were software engineers to get a good feeling for what sort of skills were most useful, find places that might be hiring, and how to ace interviews for tech jobs. Two great ways the event fulfilled this interest was through sprints, and open spaces that hosted different topics.

If you are unfamiliar with what a “sprint” is in this context, a simple definition is that it is a laid back gathering where people can meet to ask for help with their projects or just show off them off. Sometimes it’s as small as an individual’s pet project, to a team’s large and complex application. These are a great occasion to meet new people and discuss their work and potentially gain some experience. I had decided to learn more about the Arcade sprint that was led by Paul Everitt. This project revolved around improving a 2D arcade game library. In doing so, I learned about operating in a team to test code and fix bugs.

Something novel I had not experienced much of at other conferences was the use of open spaces, where a topic could be discussed in an informal group setting. I was able to attend one on interviewing; I was a little surprised but the discussion was possibly at least as interesting and enlightening as any of the formal talks. I really appreciated the opportunity this gave attendees to ask more detailed questions from a speaker, though I believe these would have been better if there was a little more time allocated.

The subjects of the talks ran the gamut from new language features that had recently been developed, to interesting uses of Python with other tools like Raspberry Pi, and even how to be more successful in the interview process as a company or a job seeker. They ranged in length from longer 30 – 45 minute sessions to short 5 minute lightning talks. Longer tutorials were also available to those who had preregistered for them as well. While it was hard at times to choose which to attend as 4 – 5 occurred simultaneously, the choice was made easier by the fact that all of them would be posted on YouTube. I would highly recommend that if you are curious about the conference to watch them. They should give a good feel for what things may get presented at other similar conferences.

Having never been to a Python or any computer science conference before, I wasn’t sure quite what to expect. What I experienced was beyond comparison as an informational and networking event that had many fun moments as well. I would absolutely recommend this conference to anyone who is a beginner to the language or to grizzled veterans and I can’t wait till next year’s conference!

Current Projects and Adapting

Hello, welcome to my first blog post on WordPress. I intend to use this blog as a medium for sharing my thoughts, struggles, and successes as I continue to grow as a chemical engineer and novice software engineer.

One of the more interesting projects I’ve been working through recently is revisiting my statistics skills. I had seen and watched a lot of alternative educational endeavors from a distance while completing my bachelors of engineering degree at UK, and recently had the desire to learn more (without needing to resort to traditional university classes). A friend recommended Coursera’s Statistics with R set of courses, which have been extremely refreshing and have provided an excellent, less chemical engineering focused take on statistics. There’s certainly a lot to be said for traditional brick and mortar education and physical proximity, but it’s been really nice to have the opportunity to continue to advance myself. I am eager to make it through all five of the courses, and hopefully will be able to apply the skills to something fun like examining League of Legends data.
Similarly, in discussing with engineers in the field, the juxtaposition between what industry is apparently looking for and what is pushed in school is often interesting, if not a little worrying. Programming appears to be an increasingly important skill asked of engineers of all hats, and somewhere that I believe myself or any other enterprising engineer can stand out. While I’ve seen Python before, I’d really would like to apply mobile app development in a useful way to engineering, so I’ve set out to learn Java as best I can.

To achieve the goal of being a decent mobile and Java developer, I’ve been knee deep in evaluating a bunch of different resources. Again, while there’s been a few non-starters, I’ve been pleasantly surprised with some of the options I’ve had to learn: the “Introduction to Algorithms” in Java course taught by Dr. Sedgewick has also been an excellent occasion to grow. While the course might move a little quick if you’ve never programmed before, it’s been a lot of fun to explore some data structures in algorithms in detail I had not seen before. As a companion to the Coursera course, Dr. Sedgewicks Algorithms book has also been a very
good read.

My hope is that all of this helps me continue to grow and find a strong career path for myself. After having left college, it’s been a little jarring navigating forward sometimes, but I find it important to remain optimistic and stay strong, and feel fortunate to have a lot of resources at my disposal as I adapt and evolve to meet the challenges of the real world. Questions or comments? Feel free to post below.