tech blog // jared hirsch // ¯\_(ツ)_/¯

Gecko C++ notes

Fri, 10 Dec 2021 23:42:55 +0000

Probably not useful for anybody, but something I told myself I’d do: share out some C++ notes.

These notes are based on the 2016? Mozilla C++ and Gecko onboarding presentation by froydnj: https://mozilla.hosted.panopto.com/Panopto/Pages/Embed.aspx?id=65922cea-26e0-4d3b-a5da-ac1600012f45

I don’t know why this talk is set to Mozilla-private, I’m working on getting it made public.

Anyway here we go.

Basic topics to cover:

Subsystems in Gecko
XPCOM strings and datatypes
infallible alloc
coding style
C++ guidelines

Additional topics suggested by Gecko experts:

Refcounting
Smart pointers
IDLs: XPIDL, WebIDL, IPDL
Electrolysis (e10s) (TODO: this video predates fission and should be brought up to date)
NSPR
Threading
C++ visibility

Outline of the talk:

What happens when I click a link? (Intro to subsystems and overall network and rendering pipeline)
Nuts and bolts of Gecko C++
Code generation
Memory management
- Smart pointers
- Reference counting schemes
Gecko C++ data structures

What happens when I click a link?

Sequence: Network -> Parser -> DOM -> Styles -> Layout -> Display List -> Paint -> Composite

Docshell is where it all begins: Docshell sets up, then network loads

/docshell directory (see also /uriloader)
URI loading starts
Handles navigating between docs (back-forward movement)
Starts doc loading
Sets things up for network
Manages session history
See also uriloader (we probably should merge these eventually)

Networking code

/netwerk dir (the misspelling is deliberate and historical)
/netwerk/dns has DNS-related code
- We use the OS for DNS resolution
- We add in other stuff
- DNS is done off the main thread and parallelized
/netwerk/protocol has code for protocols (HTTP, FTP, WebSocket, …)
/netwerk/cache2 handles document caching
- We cache documents on disk for speed
- There’s old code in /netwerk/cache, unclear why it’s not deleted. Ignore it.
/netwerk/cookie manages cookies

Once the network returns data, DocShell sets up listeners to handle the results.

Next: the parser, which handles the network response.

Parsing

/parser directory
Multiple parsers:
- XML (/parser/xm, /parser/expat)
- old-style HTML /parser/htmlparser, only used for about:blank as of 2016
- HTML5 (/parser/html)
- Old-style parsers had a lot of quirks at handling parsing errors
- HTML5 is much better specified for error handling
- Expat is ~20 years old, we use it for XML parsing
  - It’s very simple and very fast

As the parser parses, it builds DOM nodes for all elements Loading other formats (CSS, JS, images) is driven by the parser, but handled elsewhere.

DOM

dom directory
Absolutely massive directory
- Documents, elements all live in dom/base, /dom/html
- Various web APIs often have individual subdirectories, like /dom/indexedDB
- WebAPI definitions are imported into /dom/webidl for spec conformance verification
  - These definitions are also used to code generate JS bindings in /dom/bindings

DOM builds up the linked tree of DOM nodes Next, we need to position nodes on a 2D page.

Layout

Translates the DOM tree to 2D canvas.
Every visible DOM node gets its own Frame, then Layout figures out how to position the Frames relative to each other
/layout directory
also massive
- Our CSS implementation is in /layout/style
- /base, /generic: Frames (box around each element), Display Lists (organizing Frames so we can paint them quickly)
- /layout/svg holds the SVG code
- /layout/xul XUL

Given a bunch of boxes, now we need to paint them to the screen.

Graphics

/gfx directory
also large, but mainly because we use third-party libraries
We have our own code for some 2D primitives in /gfx/2d, /gfx/thebes
Thebes: our old way of rendering to the screen
- old enough that it doesn’t map well to modern graphics APIs
- rewrite in progress (as of 2016)
Layers for compositing in /gfx/layers
- All the boxes we had get organized into Layers
- The static parts of the page are grouped together as one layer, paint that into a texture on the graphics card
- Things that are changing get their own layers, sent off as textures to the graphics card
- Ship all the layers off to the GPU to be squashed together and composited all at once
Lots of third-party graphics code managed by the gfx team:
- /angle, /cairo, /graphite2, /harfbuzz, /ots, /skia, /ycbcr

For this introductory talk, we’re skipping the final stages of rendering.

What else is in Gecko?

Storage in /storage

We use SQLite for many things
IndexedDB, localStorage use SQLite
Browser history (the Places DB)
Cookies
ServiceWorker cache API is backed by SQLite
Not an ORM. We run SQL strings everywhere.
MozStorage provides a nice multithreaded API over SQLite
- e.g. run query in background thread, then return results async

Images, /image

We wrote a .bmp decoder
For newer formats, we use third-party libraries: libpng, libjpeg, libgif
Decoding images & transforming into suitable form for the gfx code
Decoding is done off the main thread
We use a thread pool to parallelize up to the computer’s limits

Media in /media

Largely imported code for sound, image, video handling
Also where (imported) WebRTC implementation lives

Widget code in /widget

Interface between Gecko and OS native event and window handling
Abstraction layer (generic IDLs)
Separate directories for various platforms:
- /android, /cocoa (mac), /donk (B2G, now deleted as of 2021), /gtk (linux), uikit (iOS), /windows

IPC (inter-process communication) code lives in /ipc

Code for managing multiple processes
Electrolysis (e10s) separates out browsing duties:
- Parent process (handles user interaction)
- Child proc: web content code, JS, etc. for responsiveness & security
This code originally came from Chromium ~5 years ago (i.e. 2011 I guess?)
- It’s now been modified beyond all mergeability. An unintentional fork.
- If you hit bugs, likely worth looking upstream for existing fixes.
Big chunks here:
- sending / receiving messages
- Object serialization / deserialization
- A separate event loop (more on this in a bit)
- IPDL code generator

JavaScript engine in /js

Our JS implementation: parsing, interpretation, JIT compiler, GC, etc.
To encourage embedding, we have public interfaces in /js/public
- But we don’t invest in this right now
The source code lives in /js/src

XPConnect: /js/xpconnect

“Deep Magic”: manages reflection of objects between JS and C++
Also defines our security architecture for JS
- See Script Security in source docs for details

MFBT /mfbt

“Mozilla Framework Based on Templates” (backronym and quite a stretch)
Code shared between Gecko and the JS engine
- Classes we find useful
- Polyfills for C++ stdlib things
- The stdlib is inconsistent across platforms, so we have code to smooth it out
- Close to C++11, aiming to delete large chunks of /mfbt someday

XPCOM /xpcom

“Cross-platform COM” (component object model)
Idea: use code in multiple languages via shared language-agnostic interfaces
- This was inspired by some Microsoft ideas from the 90s
We tried to do this across platforms, with varying degrees of success.
Lots of building blocks in here:
- Data structures (arrays, hashtables, strings)
- Core event loop (timers, threads)
  - Main thread and nonmain threads
  - Interacts with IPC event loop in various ways
- Parts of the JS interface to chrome code (XPConnect uses this stuff)
  - Interacts with XPConnect code, mostly machine-dependent parts
- Not quite complete abstract IO interface (stream abstraction that provides bytes, etc)
  - Most implementations of the abstract interface live in /netwerk for historical reasons
- Cycle collector: part of the reference counting memory management code
  - Much more on this later in the talk

NSPR /nspr

Netscape Portable Runtime, 20+ years old
Abstracts threads, locks, I/O, logging, etc.
We use NSPR when it’s convenient
- Sometimes we have nice wrappers (XPCOM has other nice wrappers for some threading stuff)
- Sometimes we use things directly (I/O)
- Sometimes stdlib has better alternatives
Slowly working on removing NSPR
- NSPR is a separate repo and team from Gecko
- Separate release schedule, etc.

Part 2: Gecko C++

C++ Style Guide

We have a style guide
Not all code conforms to the style guide
- We’re working on it
Please use the style guide for new code
- Follow local conventions when necessary
- /dom, /layout are stricter than /gfx
- Use clang-format if you like

C++ Features

We don’t use exceptions
- Historically, exceptions weren’t good on some platforms
- They’ve gotten better, but converting Gecko to be exception-safe would be a huge chore
We don’t use RTTI (dynamic_cast, e.g.)
- Bloat, doesn’t work with JS-implemented things
- We already have an equivalent dynamic cast that’s JS compatible
- RTTI not supported on Android
We have a page with our C++ feature support matrix across C++ versions and different compilers

Brief note on portability

Across platforms, rate of new feature adoption differs, quality of implementation differs
Guarantees provided by interfaces may not be strong enough for us
- e.g. we need memory reporting, std::vector doesn’t support this
Hence, we reimplement the wheel in some places

C++ std:: classes

Historically, we avoid std:: things
We don’t use exceptions, std does
It’s not necessarily available everywhere
Please use Gecko versions if possible
Please file bugs on Gecko versions if you find an API annoying

Infallible allocation (new T(...) or new T[])

By convention, if out of memory, allocation will crash, rather than returning a null pointer
We don’t return null, so we never need to rely on consistent null-checking
Useful for debugging things
- e.g. if you try to allocate 100MB, crash immediately.
- Otherwise, you get a null pointer back, sometime later you may dereference a null pointer–but it’s very hard to see how/where that happened.
- So, this makes things simpler to debug
- Also shorter: no need for lots of null checks
If you need an array but don’t control the allocation amount, we do have options
- Fallible allocation: new (fallible) T(...)
- Similar to nothrow in stdlib, predates it

C++ Visibility

We ship one big shared library, xul.dll, /xul.so
Improves startup time
Exporting symbols from XUL is expensive
- Runtime penalties, size penalties
- Exposed interface penalties (malware hooks, etc)
Strive to export as little as possible
- Generally the Right Thing happens by default

Threads

Main (UI) thread
- Handles all user interaction, most content stuff
- Important to avoid long blocking operations
- Particularly important to avoid I/O on the main thread
Lots of (mostly idle) helper threads
- SQLite, media/image decoding, network cache, JS JIT compilation, JS GC, timers

3. Code Generation

Used extensively in Gecko
3 main kinds of code generation
1. XPIDL: old bindings between JS and C++
2. IPDL: multi-process communication
3. WebIDL: new way of doing bindings between JS and C++
We have other, one-off code generators throughout the tree
All written in Python

XPIDL files

Most useful for understanding how we get from JS to C++
“Cross-platform IDL”
Defining interfaces for either C++ or JS to implement
Generates C++ class boilerplate
Generates metadata for calling into C++ from JS (argument conversions)
Docs on MDN (RIP, TODO find them and pull into firefox docs somewhere)

XPIDL: Generated code

Generates headers for each .idl in $OBJDIR/dist/include
nsIFile.idl generates nsIFile.h
IDL can define multiple interfaces per file => generated header will have multiple class definitions
Each interface nsIFoo generates:
- nsIFoo class definition (empty methods)
- NS_DECL_IFOO macro for declaring all of nsIFoo’s interface in C++
- Every interface has a UUID, there are NS_IFILE_IID and NS_IFILE_STR macros for the UUID
- other macros, rarely needed

Some IDL -> C++ examples:

IDL: const unsigned long NORMAL_FILE_TYPE=0; (note: you don’t get strongly-typed enums) C++: enum { NORMAL_FILE_TYPE=0 };

IDL: void append(in AString node); C++: NS_IMETHOD Append(const nsAString&);

Note the IDL parameters need to be annotated with in or out to indicate if they are passed in or received out

IDL: bool exists(); C++: NS_IMETHOD Exists(bool*);

IDL: readonly attribute long long fileSize; C++: NS_IMETHOD GetFileSize(int64_t*);

By convention, readonly attributes return a getter

IDL: attribute AString leafName; C++: NS_IMETHOD GetLeafName(nsAString&); and NS_IMETHOD SetLeafName(const nsAString&)

By convention, modifiable attributes generate getter and setter in C++
By convention, return values are a pointer.
NS_IMETHOD is short for virtual nsresult + windows-specific goo
nsresult is our status code
- Success case: NS_OK
  - Please don’t use the other success values
- Failure: NS_ERROR_FAILURE and many others
  - Please return the most specific error available
- Test the return value with NS_SUCCEEDED(val) or NS_FAILED(val), don’t check directly
- Complete list of errors: /xpcom/base/ErrorList.h

Methods

Return values are passed through outparams
Only set outparam when you’re returning NS_OK
Don’t use the outparam at caller until you check NS_SUCCEEDED
Always propagate errors to caller(s)
XPConnect maps errors to JS exceptions automatically

Getters / Setters

Both can fail
Same outparam convention as methods

IPDL

IPDL files describe multi-process messaging “protocols
- List of messages sent by parent or child
Code generator generates the uninteresting, repetitive parts
- e.g. serialization / deserialization
Protocols are sorted into a hierarchy
- e.g. PContent does top-level e10s per tab
- PContent manages other protocols
- Similarly, child has its own hierarchy / tree of protocols
Provides C++ structure for the interesting parts (defines virtual handler methods)
Lives in ipc/ipdl/ipdl.py
- Not great fun to work with, infrequently modified
- C++ code goes to $OBJDIR/ipc/ipdl
- C++ headers to $OBJDIR/ipc/ipdl/_ipdlheaders
- Headers organized by C++ namespace

WebIDL

TODO

Memory Management

Owned pointers

std::unique_ptr - encapsulates lots of details of passing ownership of pointers around (for exclusive-ownership)

Not available because not all our platforms support it

But we do have Mozilla::UniquePtr (in mfbt)

Same interface as std::unique_ptr
Anywhere you might need to pass raw pointers around, use UniquePtr instead

The old way: (gone from Gecko as of 2021, it seems)

nsAutoPtr<T> and nsAutoArrayPtr<T>
- Similar to std::auto_ptr
- Like auto_ptr, these are deprecated too

Shared pointers

We do this by reference counting
AddRef() and Release() methods
- Doesn’t matter virtualness or not
- You just need the names and parameters, if any
Unlike std::shared_ptr, which manages proxies to objects, in Gecko, objects manage their own refcounts directly
If you just want to pass around shared data, std::shared_ptr is easier
For Gecko, we control all the data, so we just manage refcount directly

Reference counting: thread safety

Thread safety: thread-safe and non-threadsafe options are supported
std::shared_ptr only available in thread-safe version
If you’re passing objects among threads, please use thread-safe refcounting
DEBUG builds verify thread usage by:
- marking the owning thread that allocates the object using nonsafe reference counting
- all future refcount operations will check if they are on the owning thread, and crash if not
- else, bad things may happen in non-debug builds

Smart pointers

Used to avoid needing to manually call AddRef() / Release()
RefPtr<T>
- found in /mfbt
- Used for concrete classes
nsCOMPtr<T>
- Very old, XPCOM smart pointer
- Used with interface classes: XPIDL, nsI*
- Cooperates with many other things
  - Lots of do_Thing helpers

Cycles

Reference counting is not perfect
Easy to create cyclic references => memory leaks
Not just C++ <=> C++ cycles, either.
- Cycles can include JS objects!
Avoiding this: we tried avoiding cycles, using raw pointers (scary)

Refcounting: Cycle Collector

Rather than manually break cycles, periodically detect & collect cycles
- Integration with JS GC to trigger this
- JS GC triggers things, then cycle collector looks for cycles
If an object participates in cycle collection, AddRef() / Release() do extra work to notify cycle collector
Generally only for DOM things (not network or gfx)
Lots of obtuse macros for making this almost-nice

Refcounting: declaration

Macros handle lots of this
For nsISupports things (anything with XPIDL associated to it):
- NS_DECL_ISUPPORTS single-threaded
- NS_DECL_THREADSAFE_ISUPPORTS thread-safe
- Drop these macros in your class declaration

Refcounting: definition

In concrete nsThing.cpp, list concrete classname and list of implemented interfaces
- NS_IMPL_SUPPORTS(nsThing, nsIThing, ...);
Same syntax for thread-safe and non-thread-safe classes
For non-nsISupports things:
- NS_INLINE_DECL_REFCOUNTING single-threaded
- NS_INLINE_DECL_THREADSAFE_REFCOUNTING multi-threaded

Refcounting: implementation

Generally, the macros provide all you’ll need
See xpcom/glue/nsISupportsImpl.h for the grotty details

Leak Checking

DEBUG only
We have leak checking tooling for refcounted things
Not as complete as Valgrind / Asan
Refcounted classes are automatically leak checked
Individual malloc/new isn’t counted
- ASan / Valgrind test runs to check leaks, but totally separate from our debug builds

Leak Checking: macros

Also possible to leak check non-reference counted classes
Helpful to ensure new code manages memory correctly
Include reference to MOZ_COUNT_CTOR(klassname) in all constructors
In destructor, MOZ_COUNT_DTOR(klassname)

Limitations:

Checking hierarchies is difficult
- We generally avoid hierarchies in Gecko
- MOZ_COUNT_CTOR_INHERITED is available, but rarely used
- Generally just use MOZ_COUNT_CTOR in the base class
Checking templates classes is difficult
- Solution: use a non-templated base class
- Same approach as inherited / hierarchies

More info:

BloatView, MDN article (RIP TODO find this)
- Runs on DEBUG tests by default
- Also has info on leaks not reproducible locally

Refcounting: ownership transfers

Want to avoid passing raw pointers to refcounted classes
You have a smart pointer to some class, allocate it, do stuff with it
How do you get a pointer to that object out of your method?
Can’t just return the raw pointer: the ref will get released, you’ll be passing a pointer to freed memory 🙀

Ownership transfer strategies

As usual with Gecko, there are many ways to do it
Oldest: outparam T**
- Some class returns a pointer to T*, so, class pointer pointer
- Common with XPIDL-style interfaces
More common on C++ side: already_AddRefed<T> return value
- Generally indicates new object entirely yours
- Means: I’m giving you a pointer that has had AddRef() called on it
- Typically used in create interfaces as a return value
- Means also: need to Release() it at some point
- Can’t do much with it except assign it to a smart pointer
“Modern” way: return RefPtr<T> or nsCOMPtr<T>
- Pass around or return smart pointers themselves
- With recent C++ standard changes, now actually efficient
  - In the past, might or might not be efficient
- Not currently super safe with our current smart pointer implementation with current compilers
- Planning to make improvements

Ownership transfers: summary table

	Pointer outparam	`already_AddRefed<T>`	Smart pointer
Identifying feature	`T**` argument (usually last in args list, in XPIDL)	return type	return type
Passing ownership from a smart pointer	`p.forget(outparam);` (stores the wrapped pointer into the outparam)	`return p.forget();`	`return p;`
Caller (Starting with `RefPtr <T> p;`	`o->Method(getter_AddRefs(p));`	`p = o->Method();`	`p = o->Method();`
Use cases	XPIDL-style interfaces, especially for helper methods called by XPIDL methods	static `Create()` methods	many

QueryInterface: XPCOM’s dynamic_cast

Next couple sections: brief digression into nuts & bolts of XPIDL, XPCOM

Every class/interface in Gecko gets a UUID
- tied specifically to the interface
Given a random pointer, nsISupports pointer
Can check if the pointer supports a given interface by passing it a UUID
We have infrastructure to simplify these checks

`do_QueryInterface()`

Given pointer nsCOMPtr<nsIFoo> foo
If you want to know if it supports nsIBar
Just do nsCOMPtr<nsIBar> bar = do_QueryInterface(foo);
bar will be non-null if foo was non-null and supports nsIBar
Handles null-checking foo, looking up interface UUID for nsIBar, looking up list of interface UUIDs supported by foo
Because QI does the null-check for you, you can QI through multiple interfaces and only null-check once at the end.

`do_CreateInstance()`

Used for creating objects across C++ and JS (for XPIDL)
Because we don’t use exceptions in C++, the constructor cannot return an error
So, we construct a minimal object, then call an init() method to finish initializing
To avoid using UUIDs, we have a thing called “contract ID” (typically defined in IDL)
- Example: “@mozilla.org/foo;1”.
- Stays the same even if the interface changes (hence, the UUID must be updated)
- Reduces churn due to UUID evolution / interface evolution
Syntax enforces creation/initialization separation
- Makes it easy to get ns_result from init() and relay to JS as a JS Exception
C++ syntax:
- construction: nsCOMPtr<nsIFoo> f = do_CreateInstance("@mozilla.org/foo;1");
- initialization: nsresult rv = f->init();
JS syntax:
- construction: let f = Cc["@mozilla.org/foo;1"].createInstance(nsIFoo);
- initialization: f.init();
On the C++ side, typically better to just directly init via the constructor, instead of doing all this indirection
On the JS side, this machinery is unavoidable, so these patterns are necessary

`do_GetService()`

Services are XPCOM’s singletons
They are lazily created by do_GetService()
Enforced only by convention: if you do_CreateInstance instead, strange things may happen
- Weirdness is especially likely if you do_CreateInstance more than once
do_GetService is prohibitively slow for common services
Instead, we have a faster way to get common services: services:: (aka Services.jsm on JS side)
- defined in xpcom/build/ServiceList.h
Syntax examples:
- do_GetService: nsCOMPtr<nsICatchyService> service = do_GetService("@mozilla.org/catchy;1");
- faster services global: nsCOMPtr<nsICatchyService> service = services::GetCatchyService();

`do_GetInterface()`

Was meant to replace attribute access
nsIInterfaceRequestor is a fairly common interface
Used to ask an object if it has something corresponding to some interface UUID
Could return anything that conforms to the UUID:
- might return a direct member of the concrete class
- or the class itself
- or something else entirely that happens to implement that interface UUID among many others
This is not a good pattern
- e.g. browser code will QI to nsIInterfaceRequestor,
- then get some interface,
- then query that interface to nsIInterfaceRequestor, etc.
Long, ugly chain of operations to do something relatively simple
To make matters worse, it’s not always clear if you should do_GetInterface or QueryInterface
Don’t use this in new code.
Instead, just provide an attribute or method to access a concrete thing, and sidestep the UUID machinery
- Example: given nsCOMPtr<nsIFoo> foo. How to get an nsIBar from foo?
  1. nsCOMPtr<nsIInterfaceRequestor> iir = do_QueryInterface(foo);
  2. nsComPtr<nsIBar> bar = do_GetInterface(iir);
- bar is non-null if foo was non-null and supported nsIInterfaceRequestor and knew how to provide nsIBar things
- null-check bar just once at the end and you’re done
Again: you may see this, but don’t use it in new code.

A few final bits still need to be copied over from my paper notes:

Gecko C++ Data Structures
- XPCOM / MFBT Data Structures: Growable vectors, hashtables, lists, strings, string helpers
NSPR logging
Debugging: ./mach run, rr (time-traveling record replay debugger, linux only)

Test Pilot in 2018: Lessons Learned from Screenshots

Sat, 09 Dec 2017 00:36:00 +0000

Test Pilot in 2018: Lessons learned from graduating Screenshots into Firefox

Wil and I were talking about the Bugzilla vs Github question for Screenshots a couple of days ago, and I have to admit that I’ve come around to “let’s just use Bugzilla for everything, and just ride the trains, and work with the existing processes as much as possible.”

I think it’s important to point out that this is a complete departure from my original point of view. Getting an inside view of how and why Firefox works the way it does has changed my mind. Everything just moves slower, with so many good reasons for doing so (good topic for another blog post). Given that our goal is to hand Screenshots off, just going with the existing processes, minimizing friction, is the way to go.

If Test Pilot’s goals include landing stuff in Firefox, what does this mean for the way that we run earlier steps in the product development process?

Suggestions for experiments that the Test Pilot team will graduate into Firefox

Ship maximal, not minimal, features

I don’t think we should plan on meaningful iteration once a feature lands in Firefox. It’s just fundamentally too slow to iterate rapidly, and it’s way too hard for a very small team to ship features faster than that default speed (again, many good reasons for that friction).

The next time we graduate something into Firefox, we should plan to ship much more than a minimum viable product, because we likely won’t get far past that initial landing point.

When in Firefox, embrace the Firefox ways

Everything we do, once an experiment gets approval to graduate, should be totally Firefox-centric. Move to Bugzilla for bugs (except, maybe, for server bugs). Mozilla-central for code, starting with one huge import from Github (again, except for server code). Git-cinnabar works really well, if you prefer git over mercurial. We have committers within the team now, and relationships with reviewers, so the code side of things is pretty well sorted.

Similarly for processes: we should just go with the existing processes to the best of our ability, which is to say, constantly ask the gatekeepers if we’re doing it right. Embrace the fact that everything in Firefox is high-touch, and use existing personal relationships to ask early and often if we’ve missed any important steps. We will always miss something, whether it’s new rules for some step, or neglecting to get signoff from some newly-created team, but we can plan for that in the schedule. I think we’ve hit most of the big surprises in shipping Screenshots.

Aim for bigger audiences in Test Pilot

Because it’s difficult to iterate on features when code is inside Firefox, we should make our Test Pilot audience as close to a release audience as possible. We want to aim for the everyday users, not the early adopters. I think we can do this by just advertising Test Pilot experiments more heavily.

By gathering data from a larger audience, our data will be more representative of the release audience, and give us a better chance of feature success.

Aim for web-flavored features / avoid dramatic changes to the Firefox UI

Speaking as the person who did “the weird Gecko stuff” on both Universal Search and Min-Vid, doing novel things with the current Firefox UI is hard, for all the reasons Firefox things are hard: the learning curve is significant and it’s high-touch. Knowledge of how the code works is confined to a small group of people, the docs aren’t great, and learning how things work requires reading the source code, plus asking for help on IRC.

Given that our team’s strengths lie in web development, we will be more successful if our features focus on the webby things: cross-device, mobile, or cloud integration; bringing the ‘best of the web’ into the browser. This is stuff we’re already good at, stuff that could be a differentiator for Firefox just as much as new Firefox UI, and we can iterate much more quickly on server code.

This said, if Firefox Product wants to reshape the browser UI itself in powerful, unexpected, novel ways, we can do it, but we should have some Firefox or Platform devs committed to the team for a given experiment.

Bootstrapping Test Pilot Community through Public Decision Making

Wed, 05 Oct 2016 19:30:00 +0000

This proposal is pulled verbatim from a message I sent to the Test Pilot mailing list a few minutes ago.

The question is: how do we best begin to build a community around Test Pilot?

My answer: start by making decisions in public.

If this seems interesting to you, read on below.

Proposed Participation Timeline

Q4 2016: Start by making decisions in public, in the Discourse user forums.

Q4 2016 - Q1 2017: Once we’ve made our process accessible to contributors, ask active Mozillians to get involved. Build an awesome core community. Advertise idea submissions to active Mozillians & iterate on the submission system before a huge public influx.

Q1-Q2 2017: Once the core community is in place, and idea submission has been tweaked, get Marketing involved with a public call for ideas. Our awesome core community will help manage the influx: greet newcomers, bash trolls, de-dupe suggestions.

Background

To frame the discussion, I wrote up some thoughts on what a more open, community-centered product development cycle might look like. TL;DR: give community a seat at the table by offering equal visibility into the process, and opportunities to provide input at decision points.

See also Producing OSS, which makes interesting points on the importance of public decision-making.

Suggested Q4 plan

In Q4, we can set the stage for community by making our work public:

ask product and UX to move decision making discussions to Discourse
ask experiment teams to post in Discourse as they launch, measure, iterate, and wind down
make Discourse a secondary call to action on the Test Pilot experiment pages
deprecate mailing list in favor of Discourse
(if there’s time) ask vetted, active Mozillians to join the conversation
(if there’s time) ask Mozillians to share ideas

If we make these changes in Q4, then by Q1, we’ll have plenty of content in Discourse. New community members will tend to model their input on the tone and content of the existing discussion; for this reason, I think we should hold the open call until a bit later. I realize I’m contradicting my own recent suggestion, but I do think this approach will yield better results.

Key Result: 100 monthly active Discourse users

One natural number to measure would be the number of monthly active Discourse users (MADU), meaning: users who post at least once a month in the Test Pilot category. Right now, this number is probably below 10. If all the dev teams and product/UX get involved, that’ll jump to, maybe, 30 MADUs. If we get Mozillians engaged, we could see this number jump into the hundreds. 100 MADUs seems ambitious, but possible.

Q4 plan in detail

It’s a bit buried in the list above, but I think we should deprecate the mailing list, and move discussion to Discourse instead. Our current list has little traffic, and a primitive, plain text archive that doesn’t allow search. It would be easy to move the current traffic to Discourse. Finally, moving to Discourse would encourage the Test Pilot team to use it.

Going into a bit more detail on the types of content that should move to Discourse to help seed our community:

Design phase (product / UX):

Idea proposals & discussion
Decision making discussions: which ideas to invest in, and why those ideas.

Development / iteration phase (product / UX / dev):

Summaries of user interaction data (how are people using the product, how many keep using it)
High level discussion of which features to add, change, or remove, grounded in the public interaction data
I’m not yet sure how much development discussion should happen on Discourse vs. a specific mailing list; we can ask around to see what approaches different teams would like to try.

Graduation / end of cycle phase (product):

Discussions about whether to keep an experiment running, move into Firefox, or retire it.

Wrapping up

That does it for the mailing list post.

What do you think about this proposal? Is this a good way to build the foundations of a participatory Test Pilot community?

Let me know! Twitter and email are in the footer below.

Test Pilot and Open Product Development

Wed, 05 Oct 2016 00:36:00 +0000

Test Pilot started out as a better way for Mozilla to build ambitious new features in Firefox: prototype features as add-ons, share with an opt-in beta testing community, and iterate much more quickly than the Firefox release cycle allows. This all makes sense, and it’s amazing how much we’ve accomplished in just over a year–but this conception leaves out the community algother.

How could Test Pilot be more open, more participatory?

Well, what if the Test Pilot team asked the community for feature ideas?

Even better: what if Test Pilot shipped prototypes built by self-organized teams of community members (volunteer or staff)?

What if we think about Test Pilot as a community of contributors interested in discussing and building new Firefox features?

Yeah, that’s the stuff!

I don’t have the time or space to get deeper into this vision right now, but I will mention it in another post, soon.

Assuming it makes sense to build products in the open from the early stages, how might we get there?

Here’s a rough sketch of an open product development lifecycle for Firefox features, centered around Test Pilot.

Generating and discussing feature ideas

An inventor with an idea creates a Discourse account and creates a post describing their idea. This could be anyone: a product manager at Mozilla, or a new Mozillian located anywhere in the world.

Interested community members discuss the idea, and help the inventor improve their proposal.

Community members with user research skills can help generate qualitative or quantitative insights about ideas they are interested in.

Turning ideas into working prototypes

Community members with design and development skills self-organize to build prototypes of ideas they believe in.

Companies (like Mozilla) can sponsor development of ideas.

Deciding which prototypes belong in Firefox Test Pilot

Every few months, someone from the Firefox product org leads a public discussion about which of the in-progress prototypes might be a good fit for the Firefox roadmap.

The decision-making process is a public, open conversation based on publicly defined criteria (the public Firefox roadmap and Mozilla’s guiding principles).

Input from the community is welcome, the Firefox product org makes the final decision, and deliberations happen in the same public channels used by the rest of the community.

This is the part we’ve already built at https://testpilot.firefox.com.

Prototypes ship in the Test Pilot site, and Mozilla provides marketing support and a global audience. Firefox users interested in trying new features install different prototypes, and offer feedback on what they do or don’t like.

Like any other Firefox add-on, teams ship in AMO (or self-host) and self-promote.

It’s still possible to create a great feature that later makes it into Test Pilot, or becomes a great add-on on its own, and teams can continue to discuss features and updates with the Test Pilot community.

Iterating on the product concept

Once launched in Test Pilot, the product should evolve based on quantitative usage data, as well as qualitative user feedback.

Prototype building teams are encouraged to make their discussions public, so that interested community members can provide input.

From Test Pilot prototype to Firefox

The Firefox product org decides which Test Pilot experiments to turn into real browser features. Again, this is an open discussion, centered around criteria any contributor could understand (except for contractual / partner private factors).

From Test Pilot prototype to AMO

Ideas that fail to catch on with a big audience, at the end of their Test Pilot run, can still be supported by the original team, or handed off to other community members, and permanently hosted on AMO.

Next steps: I’m out of time for now, but I’d like to explore which parts of this theoretical open product dev cycle might make sense at Mozilla. In other words, stay tuned, more to come :-)

What do you think? Let me know! Twitter and email are in the footer below.

The Joy of the Hack: Gecko Focus Stealing Edition

Thu, 31 Mar 2016 23:42:55 +0000

I’ve been working on a prototype, on and off, over the past year. The idea is to do cool new stuff with the little popup menu that appears below the Firefox address bar as you type into it:

This prototype will be available to a beta testing audience as part of the new Test Pilot program at Mozilla, which is slated to launch in May. See the links section at the end of this article for more about Test Pilot.

There are lots of experiments we’d like to try with the urlbar, but the first one shows a server-generated recommendation above the existing history items and search suggestions. Because this is just a prototype, we want to get as much quality as we can with as little work as possible–which means we’re relying heavily on third-party services.

The recommendation is generated by a really simple algorithm:

You type something in the urlbar, like ‘lol wut p’, and the keystrokes are sent to a server.
The server looks up search suggestions for ‘lol wut p’, and finds that the top suggestion is ‘lol wut pear’.
The server sends the suggestion, ‘lol wut pear’, to a search engine service; the top result is a page about that meme.
The top result is sent back to the browser, where it’s shown in the urlbar popup:

Keyboard interactions with the popup are straightforward. The list of results can be traversed using the up and down keys; a blue highlight tells the user which item is currently selected. Hitting ‘Enter’ loads the URL of the highlighted item.

If none of the results are highlighted when the user hits ‘Enter’, the browser either surfs to the URL in the urlbar, or searches the default search engine for the string in the urlbar. To clarify this search-or-navigate behavior, a top row was recently added to the urlbar, highlighted by default, which explains the options visually:

If the urlbar contents look like a URL, the top row explains that you’ll visit that URL by hitting Enter:

Otherwise, the top row explains that you’ll search for the typed string by hitting Enter:

Fitting the recommendation into the existing interface

Due to technical constraints, we’re inserting our recommendation above the existing list of results. Rather than continue to highlight the top results row, it seemed best to remove the highlight completely.

This initially seemed like a pretty simple task: the highlight is set by adjusting the selectedIndex property on the panel, and setting it to -1 removes the highlight completely, so the stealHighlight function initially might look something like:

function stealHighlight() {
  gURLBar.popup.selectedIndex = -1;
}

Sadly, this approach doesn’t work:

The results list is stealing the highlight back! What’s going on?

It turns out that the list of results isn’t rendered all at once. To keep the urlbar responsive to additional keystrokes or other user input, the results list is rendered a little at a time, in chunks of 5 rows, up to 6 times, for a total of 30 results in a scrollable list.

In addition, each time a new set of rows is inserted, the highlight is reapplied to the selected item in the popup. This is why naively unsetting selectedIndex didn’t work: the recommendation finishes rendering before the results list, so the highlight is quickly stolen back.

What to do?

There are two problems to be solved: detecting when new results are inserted, and stealing the highlight back quickly.

Detecting when results are inserted

The first problem, detecting when results are inserted, was a tough one. The UI controller code that fetches and inserts results is written in C++ (ugh), and there’s really no way to break into that code from an add-on. The C++ controller code is a huge file, so although Gecko’s XPCOM component system does allow components to be written in JS or C++, porting all of its behavior to JS would have required a rewrite–not a viable option. The existing code also doesn’t fire any signals that could be used to detect when results are about to be inserted.

The only option I could come up with was to use a MutationObserver (MDN) to listen for changes in the popup’s XUL DOM when the results were inserted. Even this required some care: the results list recycles DOM nodes as a performance enhancement, so listening for changes in the popup’s childList wasn’t enough. Happily, XUL elements store visible data in attributes, as you can see in this example of a typical result:

<richlistitem
  xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
  url="https://www.bing.com/search?q=lol+wut&amp;pc=MOZI"
  title="lol wut - Bing"
  type="favicon"
  text="lol wut p"
  class="autocomplete-richlistitem"
  image="moz-anno:favicon:https://www.bing.com/s/a/bing_p.ico"/>

When a result row is reused, its url attribute is changed to reflect the new result’s URL, and attribute changes are visible to MutationObservers. Ultimately, listening for changes on the url attribute, along with changes in the number of child elements of the popup, led to fairly reliable signals.

Quickly stealing the highlight

The second problem, stealing the highlight quickly enough to prevent flickering, was a challenge.

We could try using setTimeout ourselves, to steal the highlight on a later turn, but this doesn’t quite work: depending on random timing issues, another set of results might be inserted after the second time we steal the highlight.

Another problem with relying on setTimeout is that rendering the results is slow, so our delayed highlight stealing code can sometimes get stuck behind delayed result rendering code in the timer queue–another nondeterministic race that adds up to noticeable blue flickering, or the highlight again getting stolen back by the results list.

There’s another way to schedule future code execution in JavaScript: requestAnimationFrame. Where setTimeout(fn, n) asks the browser to call fn at least n milliseconds in the future, requestAnimationFrame(fn) asks the browser to call fn on the very next frame, just before it flushes the rendered page to the screen. rAF allows the highlight stealing code to cut in line, undoing highlight changes at the last possible instant.

Using a single rAF wasn’t enough, for the same reasons that a single setTimeout wasn’t enough. However, chaining rAF calls allows multiple successive frames to be altered, and this is just aggressive enough to work fairly well.

The working highlight stealing code basically does this:

// In response to each observed mutation event,
// steal the highlight on the current and the next two frames.
function onMutation() {
  stealHighlight();
  requestAnimationFrame(() => {
    stealHighlight();
    requestAnimationFrame(() => {
      stealHighlight();
    });
  });
}

It’s a bit of a weird hack, but it certainly works well enough for beta testing:

If you look carefully at the video, there’s some blue flicker that appears when I hit the space bar. I think this is because the existing code trims the whitespace, detects that the trimmed query hasn’t changed, and immediately re-renders, while the prototype code isn’t quite that smart yet. The few milliseconds’ delay translates into the highlighted top row getting rendered before the recommendation, and just for a moment, that blue flicker is visible.

Links

The working highlight management code is here, if you’d like to read it. It’s fairly complex: highlight management requires adjusting the highlight in response to key or mouse events. That said, it’s heavily commented, and might be interesting reading.

If you’d like to learn more about this experiment, its wiki page is a good place to start: you’ll find links to the github repos, UX artifacts, some producty discussion, and contact info for the team, if you’d like to get involved.

If you’re interested in the new Test Pilot, I’d suggest checking out the wiki, or Wil’s blog post about it.

Comments?

If you have questions or have spotted a typo, feel free to email or tweet at me.

If you’d like to offer longer comments or a response, I’d like to encourage you to post your own response on your own website, and share the link with me over email or twitter.