GSoC: Collaborative text editing in kate + kde-telepathy

First of all, I've been accepted for GSoC this year! I'll be working on creating a collaborative text editor based on the KTextEditor interface and libinfinity library, and on integrating that editor into kde-telepathy. The point of integration with kde-telepathy is that it will allow for a nice user experience in setting up connections: instead of typing IP addresses, they can just select a person from their contact list.
"Integration" doesn't mean you'll be required to use it, though -- it'll work just fine with the old-fashioned way, too.
This post will give a rough overview over the project and what I have done so far.
First, here's a nice image so you don't get bored.

Collaborative editing in kate.

This is based on the prototype I made a while ago. As you can see, there's now many more nice features, such as:

• a status bar telling about the connection status and active users
• coloured text depending on which user wrote it
• a "Collaborative" menu (more on that later).

Most improvements are not visible in a screenshot, though; numerous issues were fixed which caused crashes or errors in synchronization.
In the following, I want to write down a few details about some of the components.

Using KTextEditor

What you see in the screenshots is a plugin for KTextEditor. KTextEditor is located in kdelibs and defines an interface for a text editor; but only for the text editor, not for e.g. managing multiple documents. The most popular (and only?) implementation of the interface is KatePart. Common applications which use KatePart include kate, kwrite, KDevelop, and kile. Those applications could use any other implementation of the KTextEditor interface, if there were any, though.
KTextEditor itself defines a plugin interface; in addition to that, each of the applications listed above define an extra plugin interface. A kate plugin is more powerful than a KTextEditor plugin, since it has some capabilities which a KTextEditor plugin can by definition not have -- such as "open a document", "add a toolview" etc. However, implementing this project as a kate plugin would mean that it can only be used in kate (obviously). Getting it to work in other applications, such as kile (where it would certainly be useful, too) would cause quite some extra work. Thus I decided I want to implement this project only as a KTextEditor plugin -- although it causes quite some difficulties in some places, I think it's worth the trouble. Over the course of the project, it will be necessary to add one or two interfaces to KTextEditor (and implement them in KatePart), but almost all of the things we need are in place already.

Apart from that "application plugin vs interface plugin" question, using KTextEditor at all gives us an excellent text editor with numerous features for free, and even makes implementation of some of the things required for collaborative editing very easy, such as changing the background colour of text depending on who wrote it. Especially, in contrast to other collaborative editing applications such as gobby or etherpad, KTextEditor provides the end user with a wealth of features and plugins (such as snippets, text-completion, block-based editing, and scripts). Last but not least, it's a well-known editor for many people, which is good for getting someone to actually use it :)

It also might be worth mentioning that this project is based on the Kobby application, originally written by Gregory Haynes, which -- although a bit outdated and buggy in some places -- provided an excellent starting point.

Using libinfinity

libinfinity, written by Armin Burgmeier, is a library which implements a collaborative editing protocol (called infinote) and all the difficult parts of collaborative editing, such as multi-user text merging algorithms. Using it is what makes this project possible at all, since writing such a library in addition to an editor on top of it would be well out of the scope of a GSoC project. In addition to having bug-free and well-tested algorithms right from the start, a benefit from using this library is compatibility with applications such as gobby. (Yeah, you get the idea, I'm not actually doing anything, I'm just putting pieces together!)
Why another editor on top of the same protocol, you might ask? I partially explained it in the previous section already -- gobby is an editor built around a collaborative editing library, and it is clear that such an application will have a difficult time competing with KTextEditor feature-wise (since the latter was developed for years by many people as a powerful text editor specifically). Just think about KTextEditor's VI mode, for example, and what huge amount of work it would be to get that in gobby. I think it's a great idea to re-use all that effort and all its achievements instead of trying to compete with it by writing another editor. Also, katepart rocks ;)

Integration into kde-telepathy

One of the main drawbacks of desktop (i.e. not etherpad or google docs or similar web-based interfaces) collaborative editing applications is that connecting to someone is difficult. Almost all network setups do not allow you to open ports without special configuration, and many users are either unwilling or incapable of performing the necessary changes. Web interfaces solve this problem, but introduce various other problems -- such as having to rely on third parties (server running the service), being slow, and generally being not very powerful editors.
By using telepathy tubes for connecting KTextEditor instances, we get the goodies of both worlds: the setup is very easy, and we have a powerful and fast editor application. In detail, telepathy gives us a nice API allowing to exchange arbitrary data with a contact, without firewalls or routers getting in the way. To achieve this, telepathy will try to establish a direct connection, then a connection over a proxy server, and if both fails transfer data base64-encoded inside jabber messages.

Thus instead of typing IP addresses, in the end of this project you will be able to just select an IM contact from your contact list, click "Share document", and telepathy will handle all the difficult setup for you. As a bonus, telepathy will -- as mentioned -- try the efficient peer-to-peer-connection method first -- so if you have actually done the network setup mentioned above, you will still profit from it (better performance).

"Collaborative" menu, which will (as soon as it's implemented *grin*) allow you to share documents with any XMPP IM contact, by just selecting the contact from a list.

What I have done -- week 1

Ok, enough cheap talk! Here's what I have done so far as part of GSoC.

Stabilization

The editor does no longer crash when you close a document, or disconnect the server. Other crashes relating to unicode characters were fixed too.

KIO slave

The KIO slave is now working much better; it'll verify that operations completed successfully, and throw appropriate error messages if not. It also allows to delete things now.

Editor part -- user interface

There's now text with coloured background depending on which user typed what. There's also a status bar, which shows the connection status and how many users are currently connected. Also, a menu was added which allows to perform actions such as disconnecting from the server, or changing your user name (*cough* which is a bit fragile currently).

Editor part -- backend

Various bugs were fixed in the editor backend which led to synchronization problems. As of today, I don't know of anything causing synchronization problems any more. However my tests are using a not-yet-merged interface to KTE, which I will keep un-merged until I know what other features I need (it is optional for compiling the application).

Unit tests!

There are now basic unit tests for the editor backend. They're already useful :)

Unit tests for collaborative editing, yay

Exploring telepathy tubes

I have written a few test applications which experiment with telepathy tubes. Locally, this works nicely already (you can even use it for collaborative editing already), but numerous tests with my patient GSoC mentor David Edmundson have shown that some more work is needed for actual remote contacts.

All in all, everything is going well and the project is progressing nicely. By the way, I'll be in Bilbao -- I'm looking forward to meeting you all there!