Brrian to go.
jsprobes: cross-platform browser instrumentation using JavaScript

Introduction

Browser instrumentation is the technical basis for performance tuning and many web-related research projects. Such projects boil down to recording interesting data as the browser runs, and optionally modifying the behavior of the browser itself based on recorded data. In an extensible browser such as Firefox, there are a number of means to accomplish these ends, each with certain tradeoffs:

  • Extensions/addons are code written in high-level JavaScript and dynamically loaded at runtime. Extensions can record and modify the behavior of the browser to the extent allowed by the APIs of browser components (in Firefox, XPCOM components).

    Extensions are eminently portable, cross-platform and relatively easy to author, but are limited in the data and behavior to which they have access. Many low-level subsystems (such as the layout engine and JavaScript engine) are off-limits due to the performance and complexity costs of exposing certain functionality and data to a COM-like API. Furthermore, even if there are no performance considerations, many potentially interesting pieces of data remain unexposed for lack of (widespread) need.

  • Many platform/OS-level dynamic instrumentation frameworks (such as DTrace, SystemTap, and Windows ETW) allow for custom instrumentation in both kernel and user code. A common metaphor for these systems is the insertion of “probes” into the code, which perform some user-defined action.

    The main advantages of these probe frameworks is that they are incredibly low-overhead, and probes can be added and removed dynamically without recompiling or restarting. As a building block for browser research and tools, they are inadequate: probes are difficult to write correctly, are not cross-platform, and the obtained data does not easily feed back into the program being inspected.

  • Modifying the C++ source code of the browser is the brute-force method used by many performance analysts and researchers. The developer creates a fork of the browser source tree, makes local modifications, and optionally distributes a modified binary or source tarball to others who want to run the instrumented browser.

    "Hacking" the browser in this way provides the ultimate power to record and change any behavior, but has significant drawbacks. The researcher/developer must invest much time and energy to navigate and understand a large, foreign codebase just to figure out where to add instrumentation. Many independent developers do not have the resources to test their changes on all relevant platforms. Most importantly, these modifications almost always doom the forked codebase (and thus the research project) to become a transitory software artifact instead of a useful tool.

jsprobes

jsprobes is cross-platform, portable, and flexible browser instrumentation framework. It follows the event-driven idiom common to DTrace and many browser components: certain locations in the code (“probe points”), when executed, can run custom bits of instrumentation. A significant feature of jsprobes is that this instrumentation (“probe handlers”) is written in JavaScript. Browser data structures are exposed to instrumentation through a safe, easy-to-use data API.

The framework was created to address the weak points of existing approaches and build on their strengths. More specifically, the design goals include:

  • cross-platform: jsprobes is implemented as part of the browser platform, instead of beneath it. There are no dependencies on specific operating system features, architectures, kernel modules, or escalated privileges (root access/jailbeaks).

  • flexible: Probe handlers have access to the full JavaScript language, and can leverage existing libraries. Adding new probe points or exposing additional data to instrumentation code is simple and customizable.

    The handler execution model could be extended to serialize the probe event stream and data to disk, allowing for offline or remote evaluation of handlers.

  • easy-to-use: handlers are written in JavaScript, and handler-writers need not know the inner workings of Firefox data structures and architecture. Simple probes can be written using higher-level data API’s, while complex probes can use lower-level data API’s that more closely mirror the C++ view of browser data structures.

  • portable: Extensions are the de-facto way to share browser customizations; extensions can add, remove, and communicate with jsprobes-based probe handlers using a standard XPCOM interface.

  • cross-language: Firefox is implemented using a mix of C++ and JavaScript. jsprobes supports custom conversions between the data representations of C++ and JavaScript (or even JS-JS).

  • low overhead: The architecture of jsprobes minimizes time spent by the main thread to execute probe handler code. Most handlers are executed asynchronously on a side thread, and communication happens via asynchronous message passing.

    Although not yet implemented, the design is also conducive to future adoption of “zero probe effect” techniques used by other instrumentation frameworks like DTrace. These techniques are a prerequisite for jsprobes to land in Firefox trunk.

Current status

In it’s current incarnation, jsprobes is exposed via the nsIProbeService XPCOM interface, which provides a high-level API to control the core instrumentation engine implemented inside SpiderMonkey. Accompanying this service and the core instrumentation engine are dozens of stub calls throughout the Firefox code base. These source locations are well-known and are shared by other instrumentation frameworks such as DTrace and ETW.

Currently jsprobes is distributed as a patch series, and is available on my bitbucket account. The patches track mozilla-inbound fairly closely as of the time of this writing. See the bitbucket page for more specifics.

Example: understanding GC behavior

Suppose you would like to know which benchmarks cause garbage collection (GC), the duration of such collections, and how much garbage is collected. At a data level, one needs to know the start and end times of each GC cycle, and the heap size before and after each GC.

Registering your instrumentation

Instrumentation is added and removed via an event handler-like interface, which should be familiar to DOM/JavaScript programmers. Each place where instrumentation could be added (i.e., at GC start and end) is called a probe point. Each probe point has a fixed set of arguments: for example, the current JavaScript context, the current runtime, or the GC compartment to be collected.

The instrumentation that should be executed upon reaching a probe point is called a probe handler (to distinguish it from event handlers). Probe handlers have access to the probe point arguments, but the actual handler code is run asynchronously on a separate thread with its own JavaScript heap. So, the probe point arguments are serialized when the probe point is reached, and then later deserialized into the handler-heap when the handler runs.

In our example, the probe points of interest are GC_DID_START and GC_WILL_END. These probe points are constants defined in the nsIProbeService interface, and each are described in the (forthcoming) documentation. Their interfaces are:

GC_DID_START(runtime)
GC_WILL_END(runtime)

Each argument type has a specific API as well. Below, to the left of the arrow is the field name, and to the right of the arrow is the data type. Capitalized types are representable by instances of the equivalent JavaScript prototypes. (env is a special implicit argument to all probe points, and is always available.)

runtime:
    heapSize       -> Number
gcTriggerBytes -> Number
heapLastSize   -> Number
heapMaxSize    -> Number

env:
    currentTimeMS  -> Date

At each of these points, we want to record heap size and current time. Fortunately, this data is readily available from the probe point arguments above. So, our two handlers should look like so:

/* handler for GC_DID_START */
/* format: [startTime, stopTime, startHeap, stopHeap] */
record = [env.currentTimeMS, 0, runtime.heapSize, 0];

/* handler for GC_WILL_END */
record[1] = env.currentTimeMS;
record[3] = runtime.HeapSize;
pendingData.push(record);

There is one more piece required before we can register these probe handlers: a data specification. Probe handlers run asynchronously, but data must be captured and serialized on the main thread when the probe point is reached. Capturing all of the data is expensive—-Data specifications tell the instrumentation engine exactly which values need to be captured.

Data specifications are easy to write. Both of the above probe handlers have the same data specification:

using(env.currentTimeMS);
using(runtime.heapSize);

To put all of this together, we would make the following calls to nsIProbeService from an addon:

const Ci = Components.interfaces;
const Cc = Components.classes;
const probes = Cc['@mozilla.org/base/probes;1']
                 .getService(Ci.nsIProbeService);

var activeHandlers = [];
var cookie;

cookie = probes.addHandler(probes.GC_DID_START,
               "using(env.currentTimeMS);" +
               "using(runtime.heapSize);",
               "record = [env.currentTimeMS, 0, runtime.heapSize, 0];");
activeHandlers.push(cookie);

cookie = probes.addHandler(probes.GC_WILL_END,
               "using(env.currentTimeMS);" +
               "using(runtime.heapSize);",
               "record[1] = env.currentTimeMS;" +
               "record[3] = runtime.heapSize;" +
               "pendingData.push(record);");
activeHandlers.push(cookie);

Note that probes.addHandler returns a cookie, which is usable later as an argument to the corresponding probes.removeHandler API method.

Now that we’ve registered some probe handlers, subsequent garbage collections will trigger our instrumentation. This is great, but we eventually want to know the results of our instrumentation. nsIProbeService has one more method called asyncQuery. The arguments to asyncQuery are 1) some JavaScript source to run, and 2) an optional callback to invoke whenever a message is sent using postMessage from the handler thread to the probe service (main thread). For our example, we need two calls to asyncQuery: one for setup, and a periodic script that collects pending data records:

/* setup: run this before registering handlers */
probes.asyncQuery("pendingData = [];", function(){});

/* collection: run this periodically to marshal results */
probes.asyncQuery("while (pendingData.length) {        " + 
                  "    postMessage(pendingData.pop()); " +
                  "}                                   ",
                  function(msg) { results.push(msg); });

Once data starts rolling in via the callback, we can take action on the results array of records. For example, we could graph the data to show heap size over time, or plot GC pause length vs. garbage claimed. In fact, I have developed a demo extension called about:gc which does exactly this. It is available at my bitbucket.org/burg/aboutgc.

For now, you can view a screenshot of the about:gc prototype while running the Dromaeo benchmark suite.

  1. psdtohtmlshop reblogged this from brrian
  2. cahra-stewart reblogged this from brrian
  3. fooyeahcode reblogged this from brrian
  4. brrian posted this