Matt Croydon::Postneo

I haven’t had enough time to work up a proper hack for this, but I though I would pass along an interesting discovery that I made the other day before heading out to PyCon. After hearing about how great BeautifulSoup is at scraping HTML and making it easy to get little bits from it that you need, I thought I’d have a go at running it on my taco. You know what? It worked. I was expecting it to barf on import, but no, it chugged along just fine.

Now unfortunately BeautifulSoup won’t work out of the box with the standard .SIS install of Python for Series 60. It relies only on SGMLParser, string, and types, but those three libraries have some dependencies themselves. Here is what BeautifulSoup requires according to modulefinder.py running on my Debian box:

• array
• copy_reg
• markupbase
• re
• sgmllib
• sre
• sre_compile
• sre_constraints
• sre_parse
• string
• strop
• sys
• types

These dependencies can be easily taken care of by dropping the python modules from the source distro in the appropriate libs directory on the drive you installed Python on.

One reason that BeautfulSoup “just works” on Series 60 is that the author strives to keep imports to a minimum and that the author srives to keep BeautifulSoup backwards compatible all the way back to Python 1.5.2. There are probably many modules out there like BeautifulSoup that are designed to be backwards compatible and platform independent that should work just fine on Series 60. As I find them, I will definitely point them out. I also hope to do some hacking on a few screen scraping apps that use BeautifulSoup and appuifw to present web data using native widgets.