Timing parts of the application code in Python

Measuring application performance is a very important matter in the software development. Almost every framework comes with some tools to accomplish this task (such as Spring’s Stopwatch). Lately I was looking for such a class for Python and could not find any (for Google’a Appengine framework). There is a great timeit module, but it is useful to test only small portions of code.

I searched how the Java guys do that to eventually port their solution. On gustlik’s blog, there was a very interesting implementation of the Stopwatch class which prints logs in a way that you can see a hierarchical structure of time consumption of individual pieces of code. Something like:

post : 32 ms
  __init__ : 29 ms
  save_user : 3 ms

This was exactly what I wanted in my application. The Java code was self-explanable so it was not very difficult to move it to the Python’s world.

Below you can find my implementation and some examples of usage.

import logging
import threading
import time
from StringIO import StringIO

class Event(object):
  '''
  Event class for stopwatch.
  '''

  def __init__(self, index, depth, name):
    self.depth = depth
    self.index = index
    self.name = name
    self.start_tick = time.clock()
    self.ticks = 0

  def stop(self):
    self.ticks = time.clock() - self.start_tick

  def get_time(self):
    ''' Returns time in ms '''
    return self.ticks * 1000

  def __cmp__(self, other):
    return cmp(self.index, other.index)

class Stopwatch(object):
  '''
  Class used to measure application execution times.
  '''

  def __init__(self):
    '''
    Constructor
    '''
    self.event_ind = 0
    self.events = list()
    self.done_events = list()

  @staticmethod
  def get_instance():
    if not Stopwatch.__dict__.has_key('tdata') or not Stopwatch.tdata:
      Stopwatch.tdata = threading.local()
      Stopwatch.tdata.stopwatch = Stopwatch()
    return Stopwatch.tdata.stopwatch

  @staticmethod
  def start(event_name):
    Stopwatch.get_instance().__start_event(event_name)

  @staticmethod
  def stop():
    Stopwatch.get_instance().__stop_event()

  @staticmethod
  def pretty_print():
    return Stopwatch.get_instance().__pretty_print()

  @staticmethod
  def reset():
    ''' Resets stopwatch '''
    Stopwatch.tdata = None

  def __start_event(self, event_name):
    ''' Starts the event '''
    self.events.insert(0, Event(self.event_ind, len(self.events), event_name))
    self.event_ind += 1

  def __stop_event(self):
    ''' Stops the event '''
    if not self.events:
      return
    ev = self.events.pop(0)
    ev.stop()
    self.done_events.append(ev)

  def __pretty_print(self):
    buffer = StringIO()
    if (self.events):
      buffer.write('WARNING: same events have not been marked as closed, the summary might contain wrong data...\n')

    for ev in sorted(self.done_events):
      for i in range(0, ev.depth):
        buffer.write('  ')
      buffer.write('%s : %d ms\n' % (ev.name, ev.get_time()))
    return buffer.getvalue()

Now, when you have this code in one of your modules, you can simply type:

Stopwatch.start('doing something')
do_something() # let's assume it takes 1s
Stopwatch.start('doing something else')
do_something_else() # let's assume it takes 2s
Stopwatch.stop()
Stopwatch.stop()
print Stopwatch.pretty_print()

and you should receive on the output:

do_something : 3000ms
  do_something_else : 2000ms

Most often you would like to measure the specific method execution time, so it may be boring to write Stopwatch.start as the first method’s line and Stopwatch.stop() as the last one and then remember to print stats somewhere. There’s a simple solution to this approach. I wrote a stopwatch method decorator. With it you can decorate the method you want to test, passing in arguments information if the log should be printed to the output. Here’s the code:

def stopwatched(reset=False, logoutput=False):
""" Traces execution of the method """
def decorator(fun):
def wrapper(*args, **kw):
if reset:
Stopwatch.reset()
Stopwatch.start(fun.func_name)
res = fun(*args, **kw)
Stopwatch.stop()
if (logoutput):
logging.info("\n" + Stopwatch.pretty_print())
return res
return wrapper
return decorator

and example of usage (from Google’s Appengine):

class MainHandler(webapp.RequestHandler):
  """
  Index site
  """
  @stopwatched(True, True)
  def get(self):
    validator = Validator()
    validator.is_valid(request.get('param'))
    self.response.out.write('TEST')

class Validator(object):

  @stopwatched()
  def is_valid(param):
    flag = do_some_logic() # 20ms
    return flag

After execution in your log output you should receive something like:

get: 21ms
  is_valid : 20ms

I hope you find this code useful. And if you have suggestions on how to optimize or improve it in any way, I will be happy to hear from you!