KataSpace Post Mortem Part II

Ewen Cheslack-Postava

(This is Part II of the KataSpace post-mortem. Part I focuses on the visitors and browsers they visited with while this post focuses on technical details.)

Besides collecting useful information about how users access KataSpace and their experience with it, we also got a lot of good technical feedback from our initial deployment of KataSpace.

The Good

Overall, we were very happy with the way the system performed. There were 3 main components to consider: the KataJS-based client including the KataSpace code, the Sirikata space server on the back-end, and the “CDN” which is just the web server. The web server, of course, was not a problem. The space server held up very well. We saw a maximum of about 20 concurrent users and the space server didn’t break a sweat — it rarely even showed up in top. We expected this since the server has previously been tested under simulation with thousands of simulated objects, but a real world test always stresses the system in ways you might not have expected.

Finally, the client code also did quite well — users reported relatively good experience with a bit of performance issues, primarily complaining about jerkiness due to somewhat low frame rates. This was expected for two reasons. First, the code is relatively new and less tested than other parts of the system. In this case, we were pleasantly surprised with how rare functionality (as opposed to performance) issues were. Second, since we focused on functionality before this demo, we were aware of some performance issues. The good news is that since performance hasn’t yet been addressed, we know there’s a lot of room for improvement.

We feel we have validated that rich multiuser 3D experiences can be delivered though leading edge web tech. As we focus more on issues like performance, we hope to be able to even rival native application experiences.

Bugs

By far the most useful aspect of releasing KataSpace and running a public instance were the bug reports. Overall, we had surprisingly few bugs (although quite a few feature requests). KataSpace is built from a number of components, so we break the bugs down between the application and the supporting libraries and services. The short version: most of the bugs found are in the libraries and services, and as they get fixed they’ll improve all Sirikata-based applications, not just KataSpace.

KataSpace

As mentioned above, KataSpace itself had very few bugs. Most of the “bugs” we heard about are actually feature requests — physics and terrain, avatar customization, more complicated navigation controls, and so on. While these are features that Sirikata-based worlds should and can support, KataSpace is also intended as a simple demo and one which could be used as a skeleton for other applications. We won’t focus on getting these features into KataSpace’s master branch, but we are building other applications on the KataSpace base which will have many of these features.

KataJS

Because the KataSpace application code is relatively simple (about 1,500 lines), most of the bugs in KataSpace were actually bugs in KataJS. KataJS is the JavaScript library that handles connecting to and interacting with Sirikata servers as well as handling display, all from within a browser. It is a full, in-browser version of Sirikata’s “object host,” using JavaScript as the object scripting language. These bugs fell into two categories.

First, there were display bugs such as these two. These are potentially bugs in the underlying display library (GLGE, which has made building KataJS significantly easier). Some of these are fixed already, but compatibility with various user configurations and graphics drivers is an ongoing challenge for any application like this — WebGL applications are no different.

The second group of bugs were in the interaction with the Sirikata server. These were issues with location updates, proximity queries, and their interaction. These are mostly bugs which became more apparent with larger volume of users and it became clear that fixing them would significantly improve the user experience. All of these bugs have been fixed and any application built on KataJS will benefit from the fixes.

Sirikata Space Server

The final component is the Sirikata space server, which coordinates all the clients and allows them to interact. The space server held up pretty well as we over the course of those days had 5000 users connect to it. We only caught two new critical bugs which could cause the server to crash. One was easily fixed and the other is relatively rare.

The heavy traffic, especially the high churn of users logging in and out, revealed some memory usage problems. This was the main cause of instability since the space server’s process has its memory usage capped to avoid swapping to disk. Some memory usage issues have already been addressed and the situation has significantly improved — it takes significantly longer and more users to reach the same memory cap. However, there are still some memory issues to be worked on. Having a real deployment to test against is invaluable, and stresses the server in ways we aren’t able to otherwise.

Extending KataSpace

Since KataSpace is open source and intended as a sample application for web-based Sirikata worlds, we were also interested how people would pick up the code and run with it. Soon after the release we had a few people asking about how to build, deploy and extend the code. This helped us work out a few kinks in the script to run on EC2 (or really any Ubuntu installation). If you want to try extending KataSpace or build a new application using KataSpace as a template, the code is available here. Since the layout of the code hasn’t been well document in the repository itself, a thread on the mailing list describes the basic setup and how you go about modifying different parts of the demo.

Overall, we were very happy technically with the first public test of KataSpace — we discovered many bugs, but none of them were show stoppers and at least a few other developers have built and tested their own deployments. And of course, if you’ve discovered any bugs as you’ve worked with KataSpace, please let us know via the KataSpace, KataJS, or Sirikata bug trackers or on a mailing list or in IRC.


2 Responses to “KataSpace Post Mortem Part II”