Test Pilot started out as a better way for Mozilla to build ambitious new features in Firefox: prototype features as add-ons, share with an opt-in beta testing community, and iterate much more quickly than the Firefox release cycle allows. This all makes sense, and it’s amazing how much we’ve accomplished in just over a year–but this conception leaves out the community algother.

How could Test Pilot be more open, more participatory?

Well, what if the Test Pilot team asked the community for feature ideas?

Even better: what if Test Pilot shipped prototypes built by self-organized teams of community members (volunteer or staff)?

What if we think about Test Pilot as a community of contributors interested in discussing and building new Firefox features?

Yeah, that’s the stuff!

I don’t have the time or space to get deeper into this vision right now, but I will mention it in another post, soon.

Assuming it makes sense to build products in the open from the early stages, how might we get there?

Here’s a rough sketch of an open product development lifecycle for Firefox features, centered around Test Pilot.

Generating and discussing feature ideas

An inventor with an idea creates a Discourse account and creates a post describing their idea. This could be anyone: a product manager at Mozilla, or a new Mozillian located anywhere in the world.

Interested community members discuss the idea, and help the inventor improve their proposal.

Community members with user research skills can help generate qualitative or quantitative insights about ideas they are interested in.

Turning ideas into working prototypes

Community members with design and development skills self-organize to build prototypes of ideas they believe in.

Companies (like Mozilla) can sponsor development of ideas.

Deciding which prototypes belong in Firefox Test Pilot

Every few months, someone from the Firefox product org leads a public discussion about which of the in-progress prototypes might be a good fit for the Firefox roadmap.

The decision-making process is a public, open conversation based on publicly defined criteria (the public Firefox roadmap and Mozilla’s guiding principles).

Input from the community is welcome, the Firefox product org makes the final decision, and deliberations happen in the same public channels used by the rest of the community.

Sharing prototypes with the Firefox Test Pilot audience

This is the part we’ve already built at https://testpilot.firefox.com.

Prototypes ship in the Test Pilot site, and Mozilla provides marketing support and a global audience. Firefox users interested in trying new features install different prototypes, and offer feedback on what they do or don’t like.

Sharing prototypes that don’t make it into Test Pilot

Like any other Firefox add-on, teams ship in AMO (or self-host) and self-promote.

It’s still possible to create a great feature that later makes it into Test Pilot, or becomes a great add-on on its own, and teams can continue to discuss features and updates with the Test Pilot community.

Iterating on the product concept

Once launched in Test Pilot, the product should evolve based on quantitative usage data, as well as qualitative user feedback.

Prototype building teams are encouraged to make their discussions public, so that interested community members can provide input.

From Test Pilot prototype to Firefox

The Firefox product org decides which Test Pilot experiments to turn into real browser features. Again, this is an open discussion, centered around criteria any contributor could understand (except for contractual / partner private factors).

From Test Pilot prototype to AMO

Ideas that fail to catch on with a big audience, at the end of their Test Pilot run, can still be supported by the original team, or handed off to other community members, and permanently hosted on AMO.

Next steps: I’m out of time for now, but I’d like to explore which parts of this theoretical open product dev cycle might make sense at Mozilla. In other words, stay tuned, more to come :-)

What do you think? Let me know! Twitter and email are in the footer below.