GetHotNewStuff (GHNS) support with a custom server and OCS rant

For up- and downloading documentation files in kdev-python, I wanted to implement support for the KDE GHNS dialog (you know, this thing). In addition to a good GUI dialog which makes it easy for people to generate raw versions of docfiles and edit them a bit, this could be quite effective in supporting binary python modules correctly in the IDE. Sounds easy enough to do, right? The widgets are there... so just use them. It's not quite as easy tough...
Most applications which use the GHNS widget host their data on kde-*.org (e.g. kde-look.org for wallpapers). However, I didn't want to do that, because

• I don't think this program's data is particularily valuable to be browsable (is that even a word, browsable?) on a website like that -- it's not what people are looking for, it's nothing you would see there and say "hey cool, I gotta download that and try it"
• The sites are not maintained by KDE, and are sort of closed -- I can't access the underlying data with a script easily, but I'd like to do that, in order to be able to e.g. ship some of it with my releases
• Other people have reported it to be a bit of an effort to get one's own category on those sites

So, what to do? I have a virtual server at my disposal anyways, so i tought I would just go install an OCS server (OCS is name of the protocol being used by GHNS) there and be done. But, turns out -- there isn't any (reasonable, maintained) free implementation of the server side of this protocol.

Sigh. Alright, this can't be so difficult, can it? So, I went ahead and created a very minimal implementation of the protocol in django + python, which only supports a small subset of the protocol. It implements exactly those methods which are needed to function with the KDE GHNS upload and download widgets (some fields are still missing, I'll add them in the following days, also data validation and user registration are incomplete, but other than that it works). And let me tell you... this OCS specification!

The joys of the OCS specification

In general having a specification for uploading and downloading user-shareable content is a fine thing. But... this! Just look at the arguments you provide to create a new content entry. If you don't want to open the link, it allows you to specify up to 12 download entities (why 12? I have no idea), with fields like "downloadbuyprice11" and "downloadpackagerepository9". It also allows up to 10 (why 10?) homepage links to be specified ("homepagetype4"). Needless to say, if you request the data to be displayed in return, you'll get (or, in my case rather, have to generate) XML like this (actual quote from the spec!):

XML, OCS style -- sorry for using an image, but posting stuff containing tags in blogger is sort of impossible

You might say, okay, yeah, it's not great but it was probably the easiest way to get stuff done. I agree, especially since I'm also that kind of person which at some point will say "alright, f** elegance, let's just get stuff done". But, there's a list of other things which are somewhat off in this specification, which just fit this kind of XML rather well:

• Every response contains a "status" field (which contains "ok" or "failed") and a "statuscode" field. The status code is always 100 if the requests succeeds, and if it's not 100 something went wrong. So, where's the point of the status field?
• The specification mainly works through providing examples. That's not a bad thing by itself, but it would be good if at least the field names and their possible values were always fully listed and explained outside of an example. This is not the case (example; example2 -- here, what is e.g. "latitude"? How am I supposed to format the data in that field? I mean you can guess from the values in the example for a similar call, but still... Also note how a person only has 6 homepages while a content has 10... and 12 downloads... and 3 previews). Generally, there's lots of stuff which is just unspecified and you can only guess it from the examples. Bad.
• Funny URL schemes: my favourite is "uploaddownload"
• In uploadfile, there's various error conditions, but none matches e.g. "the item to which the file should be added was not found". In return, there's "localfile not found" which means... um... whatever... maybe the server couldn't find the file on the client's computer?
• It has typos. E.g. in edit, it has a field named "downloadtyp1". This looks just weird, shouldn't it be "downloadtype1"? What do I do now, implement the weird one and rely on everyone else following the spec closely too? Or guess that this is a typo and that everyone else probably corrected it? I'll go for the former.
• It has nonsensical field values, e.g. this: donation - do you want donation for this entry? possible values are "yes" or "" for no donation. Sure, why allow "no" if you can just leave the field empty... I mean, specifying to treat empty as "no" is ok but this is just weird.
• In download, there's this confusing thing: contentid - Id of a content;  itemid - Id of a content -- Hm. You can guess that the itemid probably means like "the 3rd download item in this content entry", but what is it? Can I use my own IDs here? Probably not. Do they start at 0, or 1? Probably 1, but I don't know.
• It leaves quite a few other questions open, for example, can there be more than one entry with a given title?

I could continue this list for a while, and I only tried to implement about a tenth of the spec.

Alright, I'll stop whining now, I think my solution is roughly working now and I'll just hope the specification will improve by a lot until I have to deal with it again in the future ;)

So now what? Well, I don't know. It would in any case be a good idea to clarify this specification a bit to make it clear how it works exactly. Then, for what KDE uses it for it's way too much stuff, one could go with a far easier specification (I think the following methods would be enough: list content, upload file, add entry, edit entry, check user, new user, download content). Thus, another option would be to dump the specification altogether and create a new, slim one. Or, just go live with it. ;)

Stay tuned, I'll surely post about GHNS integration into kdev-python soon-ish.