Posted by Ian Holsman
Tue, 01 Jul 2008 17:59:00 GMT
so on the RRD mailing list there is a discussion on how to write a RRD server/accelerator to help speed up RRD. which is a great tool, but when you abuse it and try to capture hundreds of thousands of metrics it kinda uses a bit too much disk I/O. (read swamps the system)
So imagine my surprise when I noticed that orbitz has recently open sourced their monitoring framework
- ERMA: the monitoring API
- Graphite: a graphing component on top of it
- Whisper: a fixed size db that stores the info
and imagine my surprise when I found out it was written in Django, my favorite framework.
and now I find out Theo Schlossnagle has just released reconnoiter (reconnoiter project home)
now.. to find a couple of hours in the day to actually get into them.
Tags django, metric, monitoring, reconnoiter, rrd, whisper | 1 comment
Posted by Ian Holsman
Wed, 16 Apr 2008 19:39:00 GMT
From "Don't be Evil" to "We don't need to be Evil".
because its true.
This yahoo/msft/aol/newscorp thing is just noise to them.
James pointed me to Latrz, which to me is a delicious clone. I'm guessing it was written in a day. My recent experience and It highilghts a couple of things to me
- Having the user api just makes sense for google in so many ways. I wish they would have chosen OpenID for openness, but If I was in thier seats I wouldn't have.
- the next great idea will be written with appengine (or on a clone). it's just too easy not to. I was on a conference call yesterday, and while listening I nearly coded up a django picture gallery and voting tool. now I would have been done if I actually remember WTF I was doing (I haven't coded a django app in about a year and a half now).
- it is easier to just upload it on appengine than go through operations, and even copy it to a hosted machine I run.
- I don't see the lock in anymore. If google announced this about a month ago, I would have a SoC project proposed to write a GFS-SQL parser for Hbase and Hypertable. (Anyone see the irony/joy of getting google to pay for it's competition?) and it would have been done.
Posted in Business Related | Tags appengine, django, google | 3 comments
Posted by Ian Holsman
Tue, 08 Apr 2008 07:13:00 GMT
Google has just announced their alternative to Amazon's s3 called 'App Engine'.
I think that if this is successful it will provide a shift in some of the basic web development economics and practices, even more than Amazon's s3 has.
why?
- Small hosting providers (ones that offer a shell account for $12/month) will be marginalized. why pay for something when you get it for free?
- M&A. It will create a 3rd platform to develop on. you currently have LAMP and Windows. The google app engine provides a 3rd. The major difference is you can't buy it. If we acquire a company who runs on this platform we have 2 choices. continue paying google for the infrastructure, or redevelop it onto LAMP. of course this suits google as their integration costs are lessened. Google might provide a 'open source' version of their infrastructure.. but I doubt it.
- Language choice. currently it only runs one language, python. They say they might support others in the future, but if not there will be a lot of people learning python (to the detriment of PHP, perl, and ruby), as well as new tools and utilities written in it. It's going to give python a huge boost in usage
- Database choice. Google's App Engine will be using 'bigtable' which is not a RDBMS, and uses a hacked up version of SQL. This impacts companies like mysql. you don't need to worry about replication here Krow ;-)
- Applications are integrated into Google's authentication system by default. you don't even have your own list of users.
As a python developer I love it. It even has django out of the box, but I would be a bit cautious to base my startup on a infrastructure which can only be provided by a single company.. when I get a invite I will be porting my applications over to it.. hopefully by then someone would have ported their blogging software to it so i won't have to.
Posted in Business Related | Tags django, google, mysql, python | 2 comments
Posted by Ian Holsman
Mon, 28 Jan 2008 18:54:00 GMT
What they need the most in the short term is someone who could do customer support between 3am and 11am, London time, so someone in Australia would be perfect
Do you happen to know anyone who might be interested in the job?
Ideally it would be someone who’s familiar with web technologies (web, DNS, e-mail servers), linux and maybe some web frameworks (Django, Rails,. ..).
The candidate would be doing customer support (answering tickets) but if they’re interested they could also do sysadmin work or even development work.
see the webfaction job page for more details
Posted in Lazyweb | Tags django, job, mysql, webfaction | no comments
Posted by Ian Holsman
Wed, 20 Dec 2006 16:49:00 GMT
can you guess the web service who was so stupid they stored the users password in plain text?
On a separate note, we want to make you aware that media of ours that contained a backup of a portion of the XXXXX database was stolen recently. Although the media did not contain any personally identifiable information about our users and we have no reason to believe that XXXX data was the target of the theft, we wanted to alert you to the possibility that your username, password, and – in some cases – e-mail address may have been compromised. If you use the user name and/or password for other purposes, we suggest that you change them in those other uses as soon as possible—just in case.
I guess the next stupidest thing is users like me who use the same password/logon details for free websites like this.
the sooner OpenID gets to be a household name the better.
aside: django automatically hashes the password and user creation so you don’t even have to think of it! (so the answer to this question is not a django application!)
Tags django, lame, password | 10 comments | no trackbacks
Posted by Ian Holsman
Wed, 30 Aug 2006 03:02:00 GMT
Brian Aker starts work on a memcache engine for mysql. so your memcache cache acts just like a table.
the big thing here which I’ve seen asked for a couple of times on the memcached list is the ability to see a list of keys.
mysql > select * from foo1 WHERE k=”mine”;
freaking amazing.. I love these kind of mashups.
and the 2nd important event.
Django is starting a branch to integrate SQLAlchemy
Posted in Development | Tags django, memcached, mysql, sqlalchemy | no comments | no trackbacks
Posted by Ian Holsman
Fri, 25 Aug 2006 13:06:00 GMT
My Two students Simon and Chris did wonderful jobs, and I’m happy with their work.
I think both projects were worthwhile, and fun.
I can’t wait to implement the Per Object Permissions on Zyons and my Chatter group of apps.
I even convinced Brian to lend me his domain (and his time) to create one…
ok I lie .. it’s his.. I’m just hosting it .. I’ll leave it up to brian to announce it when it is ready
Posted in General | Tags django, soc | 1 comment | no trackbacks
Posted by Ian Holsman
Thu, 24 Aug 2006 11:38:00 GMT
The California’s Governors Council on Sports and Fitness is live.
and it’s running Django.
Posted in Business Related | Tags django | 4 comments | no trackbacks
Posted by Ian Holsman
Tue, 22 Aug 2006 10:27:00 GMT
I submitted this to the django core a while back, and they weren’t interested…
So i have put it in my open SVN repo for others to use if they want to.
it’s called ‘run_jobs’.
what you do is in your [appname].management.py file you define a set of cleanup tasks
that your application needs run on a regular basis.
for example.. here are two which clean up the session and cache tables
def session_cleanup():
cursor = connection.cursor()
cursor.execute("DELETE FROM %s WHERE %s < NOW()" % \
(backend.quote_name('django_session'), backend.quote_name('expire_date')))
transaction.commit_unless_managed()
def cache_cleanup():
from django.conf import settings
import os
if settings.CACHE_BACKEND.startswith('db://'):
table_name = settings.CACHE_BACKEND[5:]
cursor = connection.cursor()
cursor.execute("DELETE FROM %s WHERE %s < UTC_TIMESTAMP()" % \
(backend.quote_name(table_name), backend.quote_name('expires')))
transaction.commit_unless_managed()
you then hook up these function to be called when a certain signal is fired in our case run_daily_jobs, as I want these run once a day. there is a signal for hourly, weekly, and monthly as well.
dispatcher.connect(session_cleanup, signal=signals.run_daily_jobs)
dispatcher.connect(cache_cleanup, signal=signals.run_daily_jobs)
then in my crontab I put:
15 9 * * * /usr/local/src/magik/zilbo/bin/run_jobs.py --settings=zilbo.settings runjobs daily
15 7 * * * /usr/local/src/magik/zilbo/bin/run_jobs.py --settings=zilbo.site2 runjobs daily
for the 2 sites I have on that machine. which would fire off once a day, and then run the session and cache clean up jobs.
so far .. pretty mundane.
now lets say I wrote another application called ‘counter’ which counts how often people view a given object. I really only want to keep stats for 8-10 weeks.. so now all I need to do is create a counter.management.py file and add the write a similar function and connect it to the signal I care about (daily in this case as well).
now.. the next time I run my batch job.. my counter objects will get cleaned as well.
without having to remember to edit the crontab.
Posted in Development | Tags batch, django | 3 comments | no trackbacks
Posted by Ian Holsman
Fri, 18 Aug 2006 06:10:00 GMT
on a similar vein to Coulix’s contribution, I have written a context_processor which allow you to see what city/region/etc your visitor is coming in from
It uses MaxMind’s GeoIP database.
warning.. this module is GPL due to MaxMind’s code being GPL. putting it into your codebase will make it catch a virus.
Posted in Development | Tags django, geoip | 1 comment | no trackbacks