Crash Reports
Sirikata is a large system and already requires a pretty large download. Even if you're a software developer and want to help by providing stack traces, it can be annoying since it would require a full development environment and mean even larger downloads -- a release build on windows with debug symbols (in .pdb files) can weigh in at as much as 700MB.
Instead, Sirikata uses Google Breakpad. Breakpad catches application crashes and generates a small file to report back to the developers with information about the crash. This makes it easy for everyone to help improve stability, whether you're an experiences C++ developer or not. Many other projects use Breakpad -- Google Chrome, Firefox, Google Picasa, Camino, Google Earth, as well as others.
Filling in a Report
When a crash occurs, Sirikata will attempt to upload the crash to a server (see below for how to change this). The server then returns a URL which Sirikata directs your browser to. It contains space to enter information about what you were doing when the crash occurred -- any information you can provide will help in debugging the problem.
The link you are directed to also contains a link to a simple status page which can show you information, including a backtrace, for the crash.
Turning Crash Reporting Off
Sometimes automatic crash reporting can get in the way, for example during development. Since debug builds are not generally distributed for end users, Breakpad is disabled by default in debug builds. For release builds, or those obtained from the Sirikata site, you can easily toggle it off (or have it report to your own server) by changing the crashreporter URL. Add the following option to your configuration file or command line:
--crashreport=none
The value of the option is a URL to report to (e.g. http://crashes.sirikata.com/report), so filling any bad value will simply cause reporting to fail.
Setting Up Your Own Crash Report Server
See the directory sirikata.git/tools/crashcollector. It contains a simple server which collects the reports, stores them, and can run the break pad processor to extract backtraces. It also has a utility script for generating the necessary symbol files from binaries (before stripping them on OSX and Linux, or from pdb files on Windows).
Getting Debug Symbols
If you do want to get the debug symbols to do some debugging yourself, they should be made available in parallel to the release. For example, the debug symbols for the release sirikata_win32.a9562bf05dc0f0eeae10ed992ac1738e9391e654.zip should be available in the same directory as sirikata_win32.a9562bf05dc0f0eeae10ed992ac1738e9391e654.dbg.zip (note the additional dbg at the end of the filename).
Generating Symbols and Debugging Crash Reports for Your Own Builds
To better understand how and why crash reporting works as it does, you should probably read the Getting Started With Breakpad document first. You'll be working on a single machine, but it's useful to understand how the components fit together. In short, to use a .dmp file, you need to:
- Dump some symbol information. To dump symbol information, we suggest you use a script we've built which works across multiple platforms: tools/crashcollector/symbolstore.py. Just run it in your build/cmake directory and it should result in a directory build/cmake/symbols with the symbol information for binaries. Note that this can take quite awhile, but after an initial scan it should report how many files it has processed and how many are left.
- Generate a backtrace using the .dmp and Breakpad's minidump_stackwalk. You should just need to execute the following command and it should spit out stack traces as well as a bit of summary information.
dependencies/installed-breakpad/bin/minidump_stackwalk minidump.dmp ./symbols
The stackwalker can generate a lot of output on stderr, so you may just want to direct stdout to a log and then use that:
dependencies/installed-breakpad/bin/minidump_stackwalk minidump.dmp ./symbols > minidump.stacktrace.log less minidump.stacktrace.log