Posted by Ian Holsman
Tue, 08 Apr 2008 07:13:00 GMT
Google has just announced their alternative to Amazon’s s3 called ‘App Engine’.
I think that if this is successful it will provide a shift in some of the basic web development economics and practices, even more than Amazon’s s3 has.
why?
- Small hosting providers (ones that offer a shell account for $12/month) will be marginalized. why pay for something when you get it for free?
- M&A. It will create a 3rd platform to develop on. you currently have LAMP and Windows. The google app engine provides a 3rd. The major difference is you can’t buy it. If we acquire a company who runs on this platform we have 2 choices. continue paying google for the infrastructure, or redevelop it onto LAMP. of course this suits google as their integration costs are lessened. Google might provide a ‘open source’ version of their infrastructure.. but I doubt it.
- Language choice. currently it only runs one language, python. They say they might support others in the future, but if not there will be a lot of people learning python (to the detriment of PHP, perl, and ruby), as well as new tools and utilities written in it. It’s going to give python a huge boost in usage
- Database choice. Google’s App Engine will be using ‘bigtable’ which is not a RDBMS, and uses a hacked up version of SQL. This impacts companies like mysql. you don’t need to worry about replication here Krow ;-)
- Applications are integrated into Google’s authentication system by default. you don’t even have your own list of users.
As a python developer I love it. It even has django out of the box, but I would be a bit cautious to base my startup on a infrastructure which can only be provided by a single company.. when I get a invite I will be porting my applications over to it.. hopefully by then someone would have ported their blogging software to it so i won’t have to.
Posted in Business Related | Tags django, google, mysql, python | 2 comments
Posted by Ian Holsman
Mon, 28 Jan 2008 18:54:00 GMT
What they need the most in the short term is someone who could do customer support between 3am and 11am, London time, so someone in Australia would be perfect
Do you happen to know anyone who might be interested in the job?
Ideally it would be someone who’s familiar with web technologies (web, DNS, e-mail servers), linux and maybe some web frameworks (Django, Rails,. ..).
The candidate would be doing customer support (answering tickets) but if they’re interested they could also do sysadmin work or even development work.
see the webfaction job page for more details
Posted in Lazyweb | Tags django, job, mysql, webfaction | no comments
Posted by Ian Holsman
Wed, 13 Jun 2007 17:52:00 GMT
It’s been a long time since I bought a book about mysql, so I thought I would ask what were some of the recent good books around about mysql.
The only good book I know is ‘high performance mysql’, but it is from 2004. so I’m concerned it’s a bit dated.
So.. what’s on your bookshelves?
Posted in Development | Tags mysql | 4 comments
Posted by Ian Holsman
Fri, 20 Oct 2006 09:18:00 GMT
Kaj Arnös announced today the create of 2 separate mysql products.
a ‘enterprise’ and a ‘community’ one.
this is exactly what redhat did a couple of years ago with it’s creation of ‘fedora’.
the thing i’m not sure I like is the following
Community users get new features at no-cost to them — funded by paying customers
that says guinea pig to me.
don’t get me wrong.. I’m not complaining .. MySQL has provided a excellent data source for many a year for next to nothing for most people, and I am grateful.
I’m more interested in how they can push more people into their enterprise product, when the community one is pretty damn good. The way redhat did it was to introduce lots of change into fedora. New versions with new features were constantly being introduced in a 3-6 month time window.. much too fast for a ‘enterprise’ to digest… and hence more corporates went to RHEL and sales increased.
but the risk of MySQL’s strategy also lies in what a percentage of redhat users did when they introduced the schism. They switched off redhat.
It will be interesting to see how many new users postgres gets from this, and if this will change what the alpha-geek uses for their development. (for example most of django’s core developers use postgres).
and a side note to the web design team of mysql.com.. I couldn’t find the link to download the mysql .. the ‘community’ tab felt more like a forum/support area so I didn’t bother, and the ‘products’ tab doesn’t really mention it either. The right hand side looked to me like a advertisement and my eyes didn’t read a word there.
Posted in Business Related | Tags mysql | no comments | no trackbacks
Posted by Ian Holsman
Wed, 30 Aug 2006 03:02:00 GMT
Brian Aker starts work on a memcache engine for mysql. so your memcache cache acts just like a table.
the big thing here which I’ve seen asked for a couple of times on the memcached list is the ability to see a list of keys.
mysql > select * from foo1 WHERE k=”mine”;
freaking amazing.. I love these kind of mashups.
and the 2nd important event.
Django is starting a branch to integrate SQLAlchemy
Posted in Development | Tags django, memcached, mysql, sqlalchemy | no comments | no trackbacks
Posted by Ian Holsman
Thu, 29 Jun 2006 04:18:00 GMT
two interesting posts arrived on the memcached list which might be interesting to performance people.
The first was a comparison of The fastest lanugage binding on which ‘P’ language performed better. To make a note the PHP version actually uses libmemcache a ‘C’ library which goes a bit of the way to explain the wild disparity in speeds.
The 2nd more interesting one (to me) was the discussion of how Digg switched from using mysql to memcached with v3 of their new interface to handle storing sessions, due to a hardware crash on their mysql server.
others mentioned using InnoDB for this instead of MyISAM, with the biggest issue being clearing out expired sessions (which memcached does for you with less overhead), but storing the sessions in the database still suffered due to OS-contention.
of course with django you can choose either, to cache your stuff.. but the session handling is stored directly in the database .. looks like I have a weekend project ;-)
Posted in Development | Tags django, memcached, mysql, performance, PHP | 4 comments | no trackbacks
Posted by Ian Holsman
Wed, 21 Jun 2006 17:38:00 GMT
are you a committer or a member of a large OSS project?
do you participate in any other large open source project?
I’m trying to get a understanding of how all the different major oss projects are being cross pollinated by having people participating in different projects.
feel free to comment, or mail me directly at ian at holsman.net
Thanks!
Posted in Development | Tags django, mysql | 4 comments | no trackbacks
Posted by Ian Holsman
Wed, 14 Jun 2006 03:29:44 GMT
continuing the recent thread about contenttypes in django I thought I would talk about a feature which got added in the magic removal branch, which doesn’t have as much attention as I think it deserves.
signals and the dispatcher.
signals are way of telling the rest of the world that something happened. If you are interested you simply listen for it (connect in django speak).
take for example my tagging application currently in use on zyons. one of it’s features is that it let’s users store their own tags.
One of the performance improvements I added to this was the creation of a ‘summary’ tag which aggregates which the users preferences into a single record.
Now, the first approach I could have taken was to call a ‘generate_summary_tag’ function every time I modify the user tag, but that was just messy, and it would be quite possible that I would forget somewhere.
Instead I did the following in the models.py:
dispatcher.connect( increment_tag_summary , signal=signals.pre_save, sender=TagUserObject )
dispatcher.connect( decrement_tag_summary , signal=signals.post_delete, sender=TagUserObject )
Now.. every time the django ORM updates a TagUserObject record my function will get called.
Other examples in the zyons code base include using signals to update the forum and conversation models to show the last-comment date and the number of posts. (instead of looking them up).
But you don’t need to only use django’s pre-defined signals. you can create your own.
For example, in my counter application (which is used to determine ‘popular’ conversations in the forums) uses a custom signal (object_viewed) to do it’s work.
Whenever a user views a forum or a conversation a object_view signal is sent.
ala
dispatcher.send(signal=signals.object_viewed, request=request, object = object )
At the moment I’m doing the heavy lifting at request time, but there is nothing stopping me just changing the logic of ‘increment_tag_summary’ to use ActiveMQ via Stomp and having a seperate batch job do it instead.
Other uses of the pre_save signal that I plan to do in the near future is to update a SolR lucene-based search server and use it instead of some complex/heavy MySQL that is currently done, by creating a ‘de-normalised’ version of some of the records and sticking it in SolR.
Oh… and a request.. zyons.com is looking for a new home. If you can provide a mod-python, mysql and shell access it would be appreciated.. my home machine (a dual pentium II 450) is beginning to show it’s age.
Posted in Development | Tags activemq, django, mysql, solr, stomp | 1 comment | no trackbacks
Posted by Ian Holsman
Fri, 07 Apr 2006 21:01:00 GMT
If you have been reading my previous entries the answer you will think is ‘not bloody well’.
After about 3 days of tuning we doubled the throughput, and got a much nicer picture, outperforming a x86-64 machine by 2.5 times in one case.
Thanks to Luojia Chen (Jenny) from Sun, Peter Zaitsev from Mysql, and Colm MacCárthaigh & Mads Toftum from the ASF.
oh.. the benchmark.. I nearly forgot ;-)
(Oh people..please link to the blog entry, and not the paper itself.. Thanks)
update:
people were having issues downloading the PDF.
so I placed a mirror of it here
Posted in Development | Tags mysql, sun, sunfire, t2000 | 11 comments | no trackbacks
Posted by Ian Holsman
Wed, 05 Apr 2006 22:13:00 GMT
this is the output of
dtrace -n 'pid$target:::entry{ @[probefunc] = count() }'
when I run mysqlslap -c 20 -i 20
pthread_getspecific 365
malloc_internal 377
my_malloc 377
free 378
malloc 378
my_wc_mb_filename 399
my_no_flags_free 420
strmake 444
pthread_self 468
memset 582
alloc_root 630
strmov 651
strlen 655
memcpy 897
mutex_trylock 936
my_pthread_fastmutex_lock 936
mutex_lock 1133
mutex_lock_impl 1133
my_utf8_uni 1365
mutex_unlock 2069
I’m about to prepare a more formal analysis (and benchmark) about this. but in order for me not to compete with colm I’m going to wait until the 24th to submit it.
I switched to using sysbench as I thought there might be a bug in actual mysqlslap code, but it is also showing the same results.
those being mysql runs 10x slower on the sunfire than on a linux x86-64 machine.
If you are from sun, mysql, or a apache member, and want to check out the configuration to see how I screwed up, send an email from your @sun.com/@mysql.com/@apache.org email address with a public key and I’ll mail you the IP#/account details (ian at holsman.net).
the machine is only available while I’m awake (I live in Australia so it probably means your nighttime), as I haven’t put it in a colo (until I win it of course) and my ‘server room’ is my walk in robe.
Posted in General | Tags mysql, sun, sunfire, t2000 | 5 comments | no trackbacks