running clean up jobs in django

Posted by Ian Holsman Tue, 22 Aug 2006 10:27:00 GMT

I submitted this to the django core a while back, and they weren’t interested…

So i have put it in my open SVN repo for others to use if they want to.

it’s called ‘run_jobs’.

what you do is in your [appname].management.py file you define a set of cleanup tasks that your application needs run on a regular basis.

for example.. here are two which clean up the session and cache tables

def session_cleanup():
    cursor = connection.cursor()
    cursor.execute("DELETE FROM %s WHERE %s < NOW()" % \
        (backend.quote_name('django_session'), backend.quote_name('expire_date')))
    transaction.commit_unless_managed()

def cache_cleanup():
    from django.conf import settings
    import os

    if settings.CACHE_BACKEND.startswith('db://'):
        table_name = settings.CACHE_BACKEND[5:]
        cursor = connection.cursor()
        cursor.execute("DELETE FROM %s WHERE %s < UTC_TIMESTAMP()" % \
            (backend.quote_name(table_name), backend.quote_name('expires')))
        transaction.commit_unless_managed()
you then hook up these function to be called when a certain signal is fired in our case run_daily_jobs, as I want these run once a day. there is a signal for hourly, weekly, and monthly as well. 

dispatcher.connect(session_cleanup, signal=signals.run_daily_jobs)
dispatcher.connect(cache_cleanup, signal=signals.run_daily_jobs)
then in my crontab I put: 
15 9 * * * /usr/local/src/magik/zilbo/bin/run_jobs.py --settings=zilbo.settings runjobs daily
15 7 * * * /usr/local/src/magik/zilbo/bin/run_jobs.py --settings=zilbo.site2 runjobs daily

for the 2 sites I have on that machine. which would fire off once a day, and then run the session and cache clean up jobs.

so far .. pretty mundane.

now lets say I wrote another application called ‘counter’ which counts how often people view a given object. I really only want to keep stats for 8-10 weeks.. so now all I need to do is create a counter.management.py file and add the write a similar function and connect it to the signal I care about (daily in this case as well).

now.. the next time I run my batch job.. my counter objects will get cleaned as well.

without having to remember to edit the crontab.

Posted in  | Tags ,  | 3 comments | no trackbacks