Saturday, February 24, 2007

Test reuse trick



The problem


Tests are great!
Tests that don't get run are not so great.
Slow tests don't get run (not great)
Duplication in code is not great






Unit test definition

Michael Feathers unit test definition

A test is not a unit test if:

  • It talks to the database
  • It communicates across the network
  • It touches the file system
  • It can't run at the same time as any of your other unit tests
  • You have to do special things to your environment (such as editing config files) to run it.


  • This does not mean we don't write tests that touch the database ,etc.
    We just call them functional tests or integration tests.
    Violating any of these rules, usually makes tests slow and/or difficult to run.


    Example

    Let's say you were creating an application that needed a database.

    For example:

    http://ciber.com/jobs







    Example 1 - Separate (and duplicated) tests

    First, we'll write some tests that talk to a fake database object.
    test_fake.py

    Then we'll blindly copy the test class above and make minor changes so we can talk to a real database object.
    test_real.py

    Then we'll write the real database object.
    real.py

    Unit test framework refresher

    The unittest frameworks operate in a 4 step process.
    for every test* method found in the test file:
    1. run the test method's class setUp function, if it exists
    2. exercise the system under test (SUT)
    3. verify the SUT behaved as you expected
    4. run the test method's class tearDown function, if it exists




    test_fake.py version 1
    #!/usr/bin/env python
    import unittest

    class FakeJobsDB(object):
    def __init__(self):
    self.data= {}
    def get_users(self):
    return self.data.keys()
    def add_user(self, user_id):
    self.data[user_id] = None

    class TestUnit(unittest.TestCase):
    def setUp(self):
    self.db = FakeJobsDB()
    def test_empty_db(self):
    self.assertEquals([],self.db.get_users())
    def test_add_user(self):
    self.user_id = u"Tammy"
    self.db.add_user(self.user_id)
    self.assertEquals([self.user_id,], self.db.get_users())

    if __name__ == '__main__':
    unittest.main()


    test_real.py version 1
    #!/usr/bin/env python
    import unittest

    import real

    class TestInteg(unittest.TestCase):
    def setUp(self):
    self.conn = real.create_db()
    self.db = real.RealJobsDB(self.conn)
    def test_empty_db(self):
    self.assertEquals([], self.db.get_users())
    def test_add_user(self):
    self.user_id = "Tammy"
    self.db.add_user(self.user_id)
    self.assertEquals([(self.user_id,)], self.db.get_users())


    if __name__ == '__main__':
    unittest.main()


    real.py version 1
    #!/usr/bin/env python
    import sqlite3

    def create_db():
    conn = sqlite3.connect(':memory:')
    curr = conn.cursor()
    curr.execute('''create table jobs (id)''')
    return conn

    class RealJobsDB(object):
    def __init__(self,db_conn):
    self.conn = db_conn
    self.curr = self.conn.cursor()
    def get_users(self):
    self.curr.execute('''select id from jobs''')
    ids = self.curr.fetchall()
    return ids
    def add_user(self,user_id):
    self.curr.execute('''insert into jobs
    (id) values ("%s")''' % user_id)





    Example 2 - Reuse the tests (DRY)

    Nothing needs to change in here.
    test_fake.py

    Now let's get rid of the duplication in the tests.
    test_real.py

    Nothing needs to change in here either.
    real.py



    test_fake.py version 2
    Nothing changes in here.

    test_real.py version 2
    #!/usr/bin/env python
    import unittest
    import test_fake

    import real

    class TestInteg(test_fake.TestUnit):
    def setUp(self):
    self.conn = real.create_db()
    self.db = real.RealJobsDB(self.conn)

    if __name__ == '__main__':
    unittest.main()


    real.py version 2
    Nothing changes in here either.






    Example 3 - A slightly more complicated setup

    Imagine you needed to initialize the database with an admin user.


    The set up that is common to both test suites is put in a separate function and then the setUp function in each suite calls the common function.
    test_fake.py

    Now let's get rid of the duplication in the tests.
    test_real.py

    Nothing needs to change in here either.
    real.py



    test_fake.py version 3

    #!/use/bin/env python
    import unittest

    class FakeJobsDB(object):
    def __init__(self):
    self.data= {}
    def get_users(self):
    return self.data.keys()
    def add_user(self, user_id):
    self.data[user_id] = None

    class TestUnit(unittest.TestCase):
    def setUp(self):
    self.db = FakeJobsDB()
    self.setup_common()

    def setup_common(self):
    self.init_id = u"admin"
    self.db.add_user(self.init_id)

    def test_verify_db_type(self):
    self.assert_(isinstance(self.db, FakeJobsDB))

    def test_empty_db(self):
    self.assertEquals([self.init_id],self.db.get_users())
    def test_add_user(self):
    self.user_id = u"Tammy"
    self.db.add_user(self.user_id)
    self.assertEquals([self.init_id,self.user_id,], self.db.get_users())

    if __name__ == '__main__':
    unittest.main()


    test_real.py version 3
    #!/usr/bin/env python
    import unittest
    import test_fake

    import real

    class TestInteg(test_fake.TestUnit):
    def setUp(self):
    self.conn = real.create_db()
    self.db = real.RealJobsDB(self.conn)
    self.setup_common()

    def test_verify_db_type(self):
    self.assert_(isinstance(self.db, real.RealJobsDB))

    if __name__ == '__main__':
    unittest.main()


    real.py version 3
    Nothing changes in here.


    Summary




    Links

    Grig's blog agiletesting.blogspot.com (with links to Titus' blog, etc.)

    xunitpatterns.com and now in book form

    Pragmatic Programmer: Pragmatic Unit Testing: in Java with JUnit
    They have a great Unit testing summary in pdf.

    Buildbot (buildbot.net): Because tests that don't get run are worthless

    Did I mention ciber.com/jobs?

    Thursday, December 22, 2005

    Problem solvers vs. pain amplifiers

    My manager's manager at Pixelworks had a great saying, "Be a problem solver not a pain amplifier." I really liked that sound bite. I was thinking about that saying during my last post on negativity.

    This came to mind recently while I was tring to configure Buildbot 0.7.1. I got Buildbot 0.6.6 configured pretty well. I had to read the manual pretty much cover to cover, which a coworker said wasn't necessarily a bad thing, but I got it working. But I had a tough time with 0.7.1. There are a few new features that are definitely worth upgrading for. So I stuck with it and got close. I finally had to post a question the the buildbot email list, which got a very quick (1 hour and 23 minutes) and correct reply. I have to say, the buildbot email list is one of the best I have seen. A very freindly list with very detailed responses. Including very thorough posts by the main Buildbot developer, Brian Warner.

    All that to say, Buildbot is awesome and worth the effort to get up and runing.

    Warm fuzzies and cold pricklies

    I just wanted to say a little about what I do and don't want to write and see in my blog.

    There is so much negativity on the web. I've read that it has to do with the feeling of anonimity people "enjoy" online. But it has gotten to the point that I almost never read the comments on a blog. I don't even read comments for Slashdot posts because it seems to be a flameathon, or at the very least the signal to flame ratio is too high for me.

    Don't get me wrong, I enjoy a good debate. We don't all have to agree on everything and have some big 60's love-in.

    Even this post feels too negative, but I think having said it out loud from the beginning will keep me honest and trying to write in a positive style.

    All that to say, keep comments in the non-smoking section and I'll do the same.

    Friday, December 16, 2005

    First times

    One of the stories my family tells, is that my Grandma could remember her first car ride. I always thought that was so funny. But I can remember EXACTLY where I was when I first saw the web.

    I could take you to within feet of where I was standing, I remember it so well. I was working as a temp at HP's Deskjet printer division, in Vancouver, Washington. One of the HP Engineers started showing me various Internet services. He showed me archie, then looked up his cousin's phone number at U of Minn., or something, using gopher (yawn), and then a little bit of ftp, which I had seen in college. Then he brought up the old NCSA Mosaic browser and showed me a web page. I remember saying, "That's cool." That was around February or March of 2003.

    Now my son plays World of Warcraft (too much), while talking to his friends, who are 2000 miles away, using YIM's VoIP. My, eventual, grandchildren will probably be amazed, I remember my first Internet experience, just like I was about my Grandma. The strange thing is I always assumed that my grandkids would be amased I could remember my first aiplane flight, which I only vaguely remember.

    This is like the culture shock you get the first time you leave your own country. The best way I can describe that initial culture shock is with pizza. Imagine a Dad from the US going on a business trip to, say, Brazil or India. When he comes home his kids ask, "Daddy, what did they put on their pizza?" and the Dad replies, "They don't eat pizza." At which point you can hear the poor kid's brain grinding gears, grind-grind-grind. We tend to think of things as changes from what we already know, ala "The Matrix" "...are you saying I can dodge bullets?" "I'm saying ... you won't have to."

    The Japanese have a word for big changes. "Hoshin". HP used lots of Japanese words when I was there. All managers had to have yearly hoshins. Everyone wants the graph to go up and to the right. A hoshin was supposed to be a step function in addition to the continous improvement.

    All that to say, this is my first blog entry.