Collaborative editing in KTE (prototype)

I had a bit of spare time, so I decided I'd try to implement collaborative editing in KTextEditor, and see how far I can get in a week. That week is over now -- and this is how far I got:


(If your browser doesn't support HTML5 video, here's a direct link)

Technical implementation

In the background, all this is based on libinfinity. There has been a KDE project in the past which attempted to build a collaborative editor on top of that protocol/library, called kobby, which is based on KTE (KTextEditor -- the thing used by kate, kwrite, kdevelop, kile, ...) too, but builds a seperate application on top of KTE (like kate or kwrite). I decided to try fixing its bugs, and turning it into a KTE plugin instead of starting from scratch. Developed together with kobby there's a library called libqinfinity which is a Qt wrapper around libinfinity (which is written in gobject-based C). So: kobby-kte (work-name for this project) is based on libinfinity and libqinfinity, and is a continuation of the kobby code base.

All the functionality currently resides in a KTE plugin, and a kioslave. The KTE plugin watches the documents being opened, and if a document uses the inf:// (infinity) protocol, it tries to subscribe to a collaborative editing session. The reason for splitting the program into a kioslave and a KTE plugin is that a KTE plugin cannot do anything which is related to managing documents (such as, opening a document), because that is specific to the application (KDevelop manages open documents in a different way than kate does). Also, a KTE plugin cannot really add UI (like a toolview for example), because that, too, is specific to the application. Since the only UI this project really needs is a way to browse remote documents, it seemed like a good idea to just write a kio slave and use the application's "Filesystem" toolview or "Open" dialog for that purpose.

kio slave

Not much needs to be said about the kioslave; it's very basic, it can just list files and directories, and create empty files and new directories. But because I like pictures and videos, here's a picture and a video:

kioslave listing data from infinoted running on localhost

(If your browser doesn't support HTML5 video, here's a direct link)

KTE plugin

The remaining functionality, mainly synchronizing documents, is implemented in the KTE plugin. The code used in the plugin is mainly from kobby (modified sometimes slightly, sometimes considerably), with just the plugin-related stuff and the fancy popup widgets implemented in the plugin itself. Here's another video of it synchronizing stuff across kate and kdevelop:


(If your browser doesn't support HTML5 video, here's a direct link)
Oh, the popups are done in QML of course! I'm aware that they flicker a bit, I think it can be solved easily, I just didn't get around to do it yet.
Note how KDevelop's code completion synchronizes flawlessly into kate! Another nice way to convince people using such mundane editors of the powers of KDevelop ;)

Main remaining issues

There's a few major things which are broken with this (as you probably guessed for something that was glued together in a week). The most obvious ones are:

• There's no way to set the username, it'll be choosen randomly. I will implement this using the protocol's auth, so you can go to inf://myuser:mypassword@localhost:12345/foo.py and then you'll be editing foo.py with username myuser. That's easy, I just didn't do it yet.
• When the remote inserts or removes text, the document will jump to your current cursor location. I have noticed this behaviour of KTE before, I'm not sure why it does that. Could even be a bug in KTE.
• There's no proper undo/redo. Undo/redo does something, but it'll break in real-world use cases. The infinity library has this problem solved; it's just a matter of taking the undo/redo actions away from the KTE view, and re-assigning them to the library functions. That needs to be talked about, though, since there's currently no sane way to do that.
• When KTE changes something as a direct reaction to you typing text, and that happens only in the remote application (so, you type something, and {if your peer had typed it instead, something other than what was typed would have changed}*), stuff breaks. For example, if one person has "replace tabs by spaces" on and the other person has it off**, strange things will happen. Here's an even more subtle example (video). However I currently believe that all those issues are the same bug in the core, and can probably be fixed with one or two mutexes in the right place (it's just a matter of finding that place). Handling conflicts on the network / insertion order level is not a concern of ours, it's handled by the library automatically (and this is the far more difficult thing).
• There's barely error handling code in both the kioslave and the plugin. For example, if your user name is already taken, it'll just write a kWarning to stdout, instead of giving you a dialog to enter a new user name; there's no proper timeouts in the kioslave; etc. Lots of error conditions are just handeled with asserts or even crashes (in a week, you just don't have enough time to write lots of error boxes ;)

Code

If you really want, you can find the code in two KDE scratch repos of mine (called kobby and libqinfinity; see the kobby-plugin branch for the kobby repo). Be warned though, the code is a bit mixed up (there's about 3 layers of wrappers stacked on top of each other which makes everything a bit complicated). If you still want to work on this, it's very much appreciated, I'm not sure if I will have the time to finish and/or maintain it in the future. Although the code is a bit screwy at places, it's very little code (below 2500 SLOC, so it's easy to fix structural defects), and it's really fun to work on since you can get fancy stuff fairly easily (also it has QML!1!!1!). One more note, lots of documents in the repo are actually unused (and are not even being built), they're just there still because I didn't decide yet whether we need to integrate them into the plugin or not. Most of what is not being built currently can probably be removed (the difference is about 3000 SLOC, so out of 5500 lines in the repo only 2500 are being built and used).
There also has been some fuzz around GSoC projects similar to this... maybe someone wants to continue it in GSoC? I'm open for everything, just email me or ping me in IRC. (note that I'm not a GSoC mentor though)

Credits

The main work about this was done by Armin, who wrote libinfinity, and Gregory, the author of kobby. I just wrote some glue code to turn the existing stuff into something new.

______
* Wait, so you're telling me I can't use curly braces to mark the semantics in a sentence? Where do you come from? Go away.
** The issue with this is that it will replace a tab (one character) by four spaces (four characters), inside the "we're currently inserting text written by the remote, don't notify the remote of changes done just now" block. That's why it breaks.

P.S.: The definition of irony: A video explaining how to embed videos in HTML without flash -- in a flash video player.