Planet Bugzilla

April 13, 2018

Dylan Hardison

Happy BMO Push Day!

Happy BMO Push Day!

View On WordPress

April 13, 2018 06:56 PM

April 08, 2018

Dylan Hardison

Bugzilla Harmony Beta

Bugzilla Harmony Beta - Testers Wanted

In addition to the Mojolicious ❤️, we’re also focused on more near-term gains. Specifically getting the Bugzilla/Harmony branch running under PSGI, and being a thing you can download and use. I am happy to announce today that the master branch is in a beta-quality state as of today, At the moment, the following installation scenarios have been tested: checkout the code run cpanm –installdeps…

View On WordPress

April 08, 2018 07:58 PM

April 05, 2018

Dylan Hardison

happy bmo push day!

happy bmo push day!

release tag the following changes have been pushed to [1448681] Bugmail Message-ID header format changed without changing In-Reply-To/References, breaking threading [1440829] Bugzilla comment for Phabricator commit should include entire commit message, not just first line [1449413] Refactor circleci container building stuff [1449156] Bugzilla::Memcached should use smaller…

View On WordPress

April 05, 2018 07:24 PM

Bugzilla Update

Watch the April 4 Bugzilla Project Meeting

Below are the minutes from today’s Bugzilla meeting along with a link to the video recording of the meeting.

video of this meeting

  1. Updates from Dylan
    1. CyberShadow is a big help getting Harmony in shape to be run by itself
    2. If you’re making a large change, make lots of small pull requests if possible (they can be on the same bug) as it makes it easier to review. That’s been working very well with CyberShadow’s changes.
    3. Bugzilla (Harmony) now working with Mojolicious which is a big deal (mostly for OAuth2 compatibility)
    4. “Rake me over the coals if there’s no 5.0.{next} release by the next meeting”
  2. Updates from Dave
    1. Still waiting on hosting info for our bots and email from Mozilla, hopefully soon
    2. History section on the website has been moved to the About page. It’s been updated with the help of Asa Dotzler correcting some inaccuracies and expanding it based on old Mozilla press releases and commit records.
  3. Next meeting is on Wednesday, May 2, at 9pm BST (UTC+1), 4pm EDT (UTC-4), 1pm PDT (UTC-7)

April 05, 2018 04:49 AM

Dylan Hardison

Bugzilla ❤️ Mojolicious

The rumors are true: Bugzilla ❤️ Mojolicious

In this pull request it is possible to: Call Bugzilla’s authentication function from Mojolicious controllers Render Bugzilla’s templates (which are template toolkit) from Mojo’s render (no small thing as we do some odd things to TT2) Parts of bugzilla that need to examine the HTTP request can (mostly) do so now This patch does a lot of plumbing, but the result of this work is that you could…

View On WordPress

April 05, 2018 02:30 AM

March 20, 2018

Dylan Hardison

happy bmo push day!

Two weeks worth of changes. Seven contributors. 89 files changed, 747 insertions(+), 1249 deletions(-).

release tag the following changes have been pushed to [1443559] Remove “Urgency” (mapped to priority) field from the “form.doc” bug form for MDN content bugs [1441903] Cleanup Makefile.PL [1444088] review link for patches on the requests page no longer shows up [1444627] Display saved searches on MyDashboard as an inline list [1439993] Remove COMPILE_DIR => setting from…

View On WordPress

March 20, 2018 02:27 PM

March 18, 2018

Dylan Hardison

bugzilla harmony news

Thanks to the near-heroic efforts of CyberShadow the unstable harmony branch is getting quite close to being able to run as an independent Bugzilla install. For a number of reasons, bmo has a separate repo for managing its dependencies. Mostly this is about efficiency — dependencies change less often then the code, and we can have much faster builds and CI. In a step towards independence I have…

View On WordPress

March 18, 2018 05:06 PM

March 12, 2018

Dylan Hardison

bugzilla/harmony unstable is kinda sorta working

bugzilla/harmony unstable is kinda sorta working

So after staring at a wall for a while, I realized it was lingering code in the ‘BMO’ extension causing my non-working-ness from Saturday. So current status is that a checkout of bugzilla/harmony‘s unstable branch should be runnable now. cpanm –with-feature=bmo –installdeps . perl then edit localconfig to point db_host to a mysql db, set urlbase to http://localhost:5000/, and then…

View On WordPress

March 12, 2018 03:53 AM

March 10, 2018

Dylan Hardison

bugzilla harmony saturday update

What have I been hacking on this Saturday?

PSGI seems pretty stable, so now to make the rest actually work! First I’m moving schema changes out of extensions. This doesn’t pass tests yet — and it is not complete. I’ll have to update templates as well. Next, a lot of our UX code assumes a workflow that is different than the default values. Ideally we’d support changing the workflow, but for the moment the default workflow provided should…

View On WordPress

March 10, 2018 10:30 PM

March 07, 2018

Bugzilla Update

Watch the March 7 Bugzilla Project Meeting

There was a lot of good information in today’s project meeting.  Below are the minutes from the meeting as well as a link to the video recording of the meeting.  Be sure to check out and comment on the new project roadmaps!

video recording of this meeting

The next meeting will be on Wednesday, April 4, at 20:00 UTC (9pm BST / 4pm EDT / 1pm PDT)

March 07, 2018 10:47 PM

March 06, 2018

Dylan Hardison

happy bmo push day!

This push includes 21 updates from 6 contributors; two of those are first-time contributors.

release tag the following changes have been pushed to [1437383] Create PhabBugz class for loading of a user object from phabricator [1441329] Fix typos in the PhahBugz module [1438206] Process SES email bounces properly [1441475] BMO is vulnerable to reverse tabbnabbing [1437384] in PhabBugz extension should be extended to also query for new…

View On WordPress

March 06, 2018 03:52 PM

March 05, 2018

Dylan Hardison

Small Pull Requests, a week-ish later

I think my new name might be Dylan “Twenty Unreviewed Pull Requests” Hardison.

I think this “chains of PRs” thing is working quite well. I’ve been playing around with how I name the branches, and thinking harder about automating it. Of course it’s really ballooning the number unreviewed PRs I have. I think my new name might be Dylan “Twenty Unreviewed Pull Requests” Hardison. But some observations: Even with no automation, this isn’t much more work for me. Smaller commits…

View On WordPress

March 05, 2018 06:05 PM

March 01, 2018

Dylan Hardison

How does `cd` work?


In my last blog post, I dove into some of the code behind the sudo command. I thought this was pretty fun. sudo is one of those commands that I use quite often but haven’t had the chance to look into truly. I started thinking about other commands that I use on a daily basis but had little understanding of the internals of. The first command that came to mind is cd. cd stands for change directory. Simply put, it allows you to set your current working directory to a different directory.

$ pwd
$ cd dev
$ pwd

So, the first thing I did was figure out what exactly was invoked when I ran cd on the command line. I used the which command which displays the path of the binary associated with a particular command.

$ which cd
$ cat /usr/bin/cd
# $FreeBSD: src/usr.bin/alias/,v 1.2 2005/10/24 22:32:19 cperciva Exp $
# This file is in the public domain.
builtin `echo ${0##*/} | tr \[:upper:] \[:lower:]` ${1+"$@"}

Oh, bother! Reading shell scripts can be such a hassle sometimes. I know the tr command is used to translate characters. In this particular case, the second half of the command, the part after the pipe symbol, basically converts the command cd dev to CD dev. I have no idea why this is. In any case, this modified command is passed to the builtin command which is handled by the shell (Bourne shell) that we are using.

I decided to dive into the code for the Bourne shell to see what I might be able to figure out about these builtins. I came across the definition of the cd builtin here. I’ll admit that it was a hassle to read this file. For one, the source for Bash does not have a mirror on GitHub, so I had to browse through a hosted version with a somewhat cumbersome file browser.

Nonetheless, I read through some of the code that was defined in this file. Some of it was in functions, and other bits were in templates, but after a while, I figured that most of the code was a wrapper around a function called chdir. A lot of the functions defined in the cd.def file linked above actually just invoke chdir and handle errors and parameter cleaning.

I did some Googling on what the chdir function is. It’s a function that is a standard part of Unix. So to figure out more about how chdir worked, I had to dive into the code for the Linux kernel which can be found here.

I started, as I usually do, by searching for the term “chdir” using the GitHub search bar. I find that it is accessible and useful. After scrolling through some of the results, I realized that OMG something I learned at university was going to come handy to me.

The chdir function implements a system call. What’s a system call? A system call is a process by which a user program (like Bash) requests access from the kernel to execute a command. Usually, this is done for commands that require a certain amount of privilege. Changing into a directory is a privileged command from the operating system perspective. You’re essentially giving the user access to a new portion of the file system on the machine so it makes sense that programs would have to pass that request to the kernel for security reasons.

So all in all, here is what happens when you run cd on the command line.

  1. The cd builtin is invoked as part of the Bash shell.
  2. The Bash shell invokes the chdir function.
  3. The chdir function is part of Unix and invokes the chdir system call.
  4. The Unix kernel executes the chdir call and does its own low-level thing.

I could dive in a little bit more into how #4 works, but let’s be honest, I’ve already read too much code at this point, and my eyes are starting to hurt.

I hope reading this was illuminating for you!

UPDATE: After reading this blog post, Julia Evans (@bork), provided the following insights on why cd is implemented the way it is through an email.

So!! Why is cd a builtin function in bash and not a program? Some builtins (like time) are bash builtins but they would also work as standalone programs. Is cd one of those?

It turns out that cd HAS to be a shell builtin otherwise it wouldn’t work! Here’s why:

Every process has a set of attributes that the Linux kernel stores. These attributes are things like environment variables, signal handlers, and – the process’s current working directory!!!

Processes aren’t allowed to change each other’s attributes (if I’m a program, I can’t change the working directory or environment variables of another process).

SO!! If you had a /usr/bin/cd program that ran chdir, that would be fine, but when you started it it would change its own working directory and exit which is not very helpful. It wouldn’t change the working directory of you (the parent process)

So bash has to call the chdir syscall itself which is why it’s a builtin.

Thanks, Julia!

March 01, 2018 09:33 PM

February 27, 2018

Dylan Hardison

An experiment for making big changes with smaller commits on GitHub.

An experiment for making big changes with smaller commits on GitHub.

It is safe to say that smaller commits is a perennial topic at Mozilla. On the reviewer side, I definitely prefer smaller commits. I find it’s easier — and faster — to review them. Often not in the neighborhood of microcommits, but at least 300-400 line patches. For the thing I work on we use GitHub, and PRs seem to work best on small things. When tasked with a large task, it seems that many…

View On WordPress

February 27, 2018 12:35 AM

February 23, 2018

Dylan Hardison

happy 3rd bmo push day

changes, on a friday? Almost never, but content changes? lgtm.

release tag the following changes have been pushed to [1439693] Update bug form.mdn for product name [1440107] Allow ‘self’ frames in bug modal again (fix socorro lens) discuss these changes on

View On WordPress

February 23, 2018 04:00 PM

February 21, 2018

Dylan Hardison

happy bmo supplemental push for mdn day

bmo ❤️ mdn (, so we did a special push to fix their custom form

release tag the following changes have been pushed to [1439693] Update bug form.mdn for product name [1439797] Enable reporting-only CSP by default discuss these changes on

View On WordPress

February 21, 2018 07:57 PM

February 20, 2018

Dylan Hardison

happy bmo push day!

happy bmo push day!

release tag the following changes have been pushed to [1433993] Outdated FreeOTP link in user preferences [1433833] Add index to email_rates.message_ts [1436301] Exempt bot accounts from idle group removal [1430259] Update policy code in BMO PhabBugz extension to update custom policy if a private bugs groups have changed. [1343248] Migrate secbugstats scripts to bmo…

View On WordPress

February 20, 2018 02:38 PM

February 16, 2018

Bugzilla Update

Release of Bugzilla 5.1.2, 5.0.4, and 4.4.13

Today we have several new releases.

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 5.0.4 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.4.13 is a security update for the 4.4 branch:

Bugzilla 5.1.2 is an unstable development release. This release has not received QA testing from the Bugzilla Project, and should not be used in production environments. Development releases exist as previews of the features that the next major release of Bugzilla will contain. They also exist for testing purposes, to collect bug reports and feedback, so if you find a bug in this development release (or you don’t like how some feature works) please tell us.

February 16, 2018 08:13 PM

Dylan Hardison

Profiling as validation

Among a bunch of other things that are going on, we’re migrating to a new home in AWS. So the team (bobm and ckolos) have been very dedicated to validating the new stuff is as good, and hopefully better than the old stuff. To this end they’ve been working with another engineer (rpapa) to do load testing. Some of the load testing results have been a bit unusual, perhaps even…

View On WordPress

February 16, 2018 02:31 AM

January 25, 2018

Dylan Hardison

My Little Features: Deprecation is Magic

I’m still hammering out what features I want to see land in BMO this year but one thing I’ve come to realize is that often the way forward on an old code base is through careful deprecation. Adding new features, or even fixing existing bugs is often blocked by the immense weight of past decisions. Recently we deprecated support for IE 11. This was magic: We can use more modern javascript…

View On WordPress

January 25, 2018 08:26 PM

happy bmo push day!

release tag the following changes have been pushed to [1429600] Update type checking to treat bug id as a simple string or undefined [1426414] Send preload headers for webfonts [1431135] add bug status css class to markup [1430495] Make loading of Requests dropdown faster [1429785] In-page back navigation broken [1429688] focus Search Bugs box by default…

View On WordPress

January 25, 2018 05:06 AM

January 22, 2018

Dylan Hardison

mom's birthday

mom’s birthday

Today would be my mom’s 64th birthday. I dedicate each Dec 29th to renewing my GPG keys (it is the day my GPG keys would expire if I take no action). I celebrate mom’s birthday in this way because after one of the (many) times I’ve lost key material (lost in the sense of “the private key is gone” or “I forget the passphrase”) mom offered to keep printouts of said secrets. Unfortunately, mom…

View On WordPress

January 22, 2018 04:28 PM

The Binding of libcmark-gfm: Segfaults and Debugging

I’m continuing to make progress on providing a markdown binding for perl with that rich, full-bodied GitHub flavor.

So for various reasons — including “I want to support markdown” I have been working to get a binding to github’s fork of cmark. For the first part of this, I got some help in #native writing a Perl module in the Alien:: namespace, namely Alien::libcmark_gfm. Armed with this module, I’ve been seeking to make CommonMark work against the GitHub fork of libcmark. So far things…

View On WordPress

January 22, 2018 12:26 AM

January 21, 2018

Dylan Hardison

Generating a bunch of decent mock data on


Do you enjoy creative writing? Would you like to help Bugzilla?

Right now, only myself and dkl are allowed access to sanitized bugzilla (bmo) database dumps. The vagrant and docker dev environments come with very rudimentry data sets (one product, a handful of versions) and it means the experience of our many contributors is sub-optimal. Frequently they’re not able to effectively test their ideas.

With this in mind, I have set up This is running the mozillabteam/bmo:latest docker container on a VM that I own. It contains no data that isn’t in the git repo.

I want to give people accounts on this – literally anyone – and have them:

1) Create products, components, versions, milestones, flags, tracking flags, keywords, groups, and other misc. metadata 2) Invent a bunch of fake bugs and comments. The more the better.

After we have a sufficient number of these created, I will take the data, sanitize it, and publish the data for all to use.

To get started, DM me on twitter (@dylan_hardison), send me an email dylan [at] or hop in irc: #bugzilla on


Thematically, we can pretend this is a bug tracker for a fictional operating system written in Brainf*ck, or perhaps the bug tracker for the fictionary “sword art online” anime. Or perhaps you can throw some machine learning at this – I don’t care as long as I get a diverse dataset for testing.

January 21, 2018 12:12 AM

January 05, 2018

Dylan Hardison

Looking back at Bugzilla and BMO in 2017

Looking Back

Recently in the Bugzilla Project meeting, Gerv informed us that he would be resigning, and it was pretty clear that my lack of technical leadership was the cause. While I am sad to see Gerv go, it did make me realize I need to write more about the things I do.

As is evident in this post, all of the things I’ve accomplished have been related to the BMO codebase and not upstream Bugzilla – which is why upstream must be rebased on BMO. See Bug 1427884 for one of the blockers to this.

Accessibility Changes

In 2017, we made over a dozen a11y changes, and I’ve heard from a well-known developer that using BMO with a screen reader is far superior to other bugzillas. :-)

Infrastructure Changes

BMO is quite happy to use carton to manage its perl dependencies, and Docker handle its system-level dependencies.

We’re quite close to being able to run on Kubernetes.

While the code is currently turned off in production, we also feature a very advanced query translator that allows the use of ElasticSearch to index all bugs and comments.

Performance Changes

I sort of wanted to turn each of these into a separate blog post, but I never got time for that – and I’m even more excited about writing about future work. But rather than just let them hide away in bugs, I thought I’d at least list them and give a short summary.


Developer Experience Changes

My favorite communities optimize for fun. Frequently fun means being able to get things done. So in 2017 I did the following:


In the last year, we had almost 500 commits to the BMO repo, from 20 different people. Some people were new, and some were returning contributors (such as Sebastin Santy).

January 05, 2018 05:45 AM

June 18, 2017

Dylan Hardison

wilwheaton: scarlettohairdye: killerchickadee: buttheadhatesthetcc: lauralot89: Jesus Christ was...






Jesus Christ was a brown Jew in the Middle East, conceived out of wedlock in an arguably interracial if not interspecies (deity and human) relationship, raised by his mother and stepfather in place of his absent father.  He may not have had a Y chromosome.  He spent his early youth as a refugee in Egypt, where his family no doubt survived initially on handouts from the wealthy (You think they kept that gold, frankincense, and myrrh from the wise men?  Hell no, they sold that stuff for food and lodging).  He later returned with his parents to their occupied homeland and lived in poverty.

The religion of Jesus’s people has no concept of a permanent hell and instructed its priests on how to induce miscarriages.  Jesus explicitly rejected the concept of disability as a divine punishment.  He spoke out against religious hypocrites.  He had enough respect for women to let his mother choose the time of his first miracle.  He blessed a same sex couple.  He told a rich man that he must give up his wealth to get to heaven, and also told a parable about a rich man suffering in agony in presumably Gehinnom (basically Purgatory) just to hammer the point home.  He told people to pay their taxes.  He declared “love your neighbor” to be one of the two commandments on which all laws hang.  He commanded his followers to help the poor.  He commanded them to help the sick and the needy.  He spent time with social outcasts.  He healed the servant of a high priest during his arrest rather than fighting back.  He was put to death by the occupying government because he was a political radical.

Trump and his administration are xenophobic, misogynistic, racist, fear-mongering, warmongering, tax-dodging, anti-Semitic, anti-choice, anti-welfare, anti-equal pay, anti-LGBTQIA+, anti-immigration, support tax cuts for the rich, support Citizen’s United, want to keep refugees out of this country, want to limit our ability to speak against the government, plan to abolish the Affordable Care Act, and they wrap all of that up behind a banner of “Christian family values.”  If you support them, you have no right to call yourself a follower of Christ.

it’s so rare, yet so fulfilling, to see the J-man on my dash

One of my friends is literally the most religious Christian I have ever met. What does that mean in regards to her lifestyle and outlook? She loves everyone. EVERYONE. Unconditionally. And she supports healthcare and education and birth control and everything that’s necessary to have a healthy, stable society.

Because that’s what her homeboy JC would want.

Canon Jesus is better than Fandom Jesus.


“Canon Jesus is better than Fandom Jesus.”


June 18, 2017 02:36 PM

April 13, 2017

Dylan Hardison

BMO ❤️ Carton

Back when I started working on BMO we couldn’t add new dependencies without having someone build an RPM. For no particularly good reason, this made it so in general we didn’t add new dependencies often.

However, about a year ago I started poking at carton and came up with a process to run carton in a docker container that mirrors production, and tar up the resulting local/ directory.

For the last 6 months or so we have been able to add dependencies whenever we want. We can also track changes to the full dependency tree.

The code for this is on github as mozilla-bteam/carton-bundles and it is a little ugly, but packaging code is rarely elegant.

April 13, 2017 11:11 PM

April 06, 2017

Dylan Hardison

Optimizing BMO: Part 1, An Easy Fix

One way of squeezing performance out of apache, as noted in this blog post by Hayden James is to disable htaccess files – which are not needed when you have control over the httpd’s config files. Doing this allows the web server to spend less time calling stat() for .htaccess files that don’t even exist – for instance a request to https://something.something/foo/bar/baz is at least 4 calls to stat() .htaccess (once at /, then /foo, then /foo/bar, and finally /foo/bar/baz assuming that baz is also a directory.

As it turns out for BMO this is even easier: bugzilla already sends configuration to the apache process.

Because of this, we can search for .htaccess files at apache startup time, load them using the server->add_config() method and tell apache to not bother looking for them during subsequent requests.

This change was quite small and the performance gain in production should be noticeable (but not large). As it turns out, some of those stat() calls also hit NFS, which will be a discussion for Part 2.

April 06, 2017 12:34 PM

February 07, 2017

Bugzilla Tips


Bugzilla comments are, by default, plain text – so typing <U> will produce less-than, U, greater-than rather than underlined text. However, Bugzilla will automatically make hyperlinks out of certain sorts of text in comments. For example, the text will be turned into a link: Other strings which get linkified in the obvious manner are:

A corollary here is that if you type one or more bug numbers in a comment, you should put the word “bug” before each of them, so they gets autolinkified for the convenience of others.

But also, if you have some text you want linkified in this way, perhaps for use in a blog post, status report, wiki page or other place, you can use the handy linkifier, which will do exactly that for you.

February 07, 2017 11:39 AM

January 05, 2017

Mark Côté

BMO in 2016

Stuff that landed in 2016

Here’s a sampling of improvements to BMO that were launched in 2016.

Improvements to bug-modal

We’ve continued to refine the modal bug view, aka the “experimental UI”. The BMO team fixed 39 bugs relating to the new interface in 2016. We’ve got a couple more blockers before we make the modal view the default, which should happen in the middle of January. We know there are a still a few outstanding bugs and some missing functionality, so we will leave the standard view available for a little while, at least until all the blockers of bug 1273046 are resolved. All future improvements to the bug view will happen only on the modal UI, which is not just more usable but also more hackable.

HTML email

There was actually no real work involved here, as HTML email was added to BMO years ago. At that time, since it was a new feature, we didn’t enable it by default… and then 4 years went by. Just a couple weeks ago a new BMO user suggested that we implement HTML emails, having no idea that the option was already there (buried in many other preferences). That was the prompting we needed to finally enable it by default.

Readable bug statuses

Emma Humphries added readable statuses prominently displayed at the top of the status panel (in the modal UI only). They quickly summarize the status of a bug in a visible place, mainly for triaging and tracking purposes.

A couple examples:

This is part of Emma’s on-going efforts to improve contributors' experiences.

Time zones

One big change we made this year was hopefully completely invisible to everyone: BMO’s database moved to UTC. When BMO was originally deployed in 1998, the database, being based in California, was set to Pacific Time. 6 years later someone suggested that UTC would be a better choice. When I took over management of the BMO team about 4.5 years ago, I was pretty horrified that a major application would be running in any time zone other than UTC, not in the least because of the confusion caused by an hour being repeated every year when PDT switched back to PST, since the presence or absence of DST is not noted in the database. However, we were never able to justify the required effort to move over to UTC, that is, until last year, as we were setting up a failover system in AWS. RDS, the natural choice for a MySQL-based application, supported only UTC, thus giving us a hard requirement to migrate. A heroic effort by dkl got us smoothly switched over in May 2016.

Memory-usage & perf improvements

We’ve known for some time that Bugzilla has a persistent memory leak. It was never a huge issue because the webheads would automatically restart Apache processes when their memory usage got too high, but it is understandably something that lurks in the back of the minds of the developers working on Bugzilla. Dylan finally got frustrated enough to fix a big leak, which resulted in the webheads restarting much less frequently, which in turn led to a performance increase. He’s been investigating and fixing other such leaks when he finds the time.

Stuff we’re wrapping up in 2017

Some of the bigger projects bled over into 2017.

Content Security Policy

We’ve been working on implementing CSP in BMO, starting with the new modal bug view. It was pretty hairy due to generated HTML, inline JavaScript, and other old web-dev techniques that make security harder. After some back and forth, we’re just about there; see bug 1286290 for progress.

Note that CSP can break browser extensions. Since the modal UI is relatively new, there are probably not too many extensions designed for it; however, we’ll be spreading CSP over time. And of course, we’ll be removing the old bug view at some point, which will definitely break some things.

Elastic Quicksearch

In the spring, Dylan hacked up a prototype of a quicksearch alternative powered by an Elasticsearch index. It’s lightning fast, so we explored setting it up in production. Of course a prototype is always easier than the real thing, and we had to do some structural work to BMO to make it possible, although that in turn has had side benefits. The indexing code is just about ready to roll out, and while we’re verifying that it works correctly, we’ll be finishing up the search code. You can also follow the main tracker bug for the whole deployment.

New stuff in 2017

We’re expecting to wrap up the above features in Q1, and we’ve already developed a road map for the first half of 2017 with some fun and long-awaited features. Emma will be going over this in another post, wearing her hat as BMO product manager, a job she has recently, and graciously, taken on herself!

January 05, 2017 04:54 PM

October 12, 2016

Dylan Hardison

happy bmo push day, wednesday edition

the following changes have been pushed to

discuss these changes on

October 12, 2016 01:55 PM

October 08, 2016

Dylan Hardison

Bigger and better search box on BMO

There is a lot of power hidden in quick search and my data suggests it is under-utilized.

For instance, searching for all review flags is the literal search tag review? Similarly you can do needinfo? to find all bugs with needinfos directed at me (actually, this query performs a slightly broader search but I have code to fix that).

There are dozens more examples. The quick search help is a long read, and most people won’t bother.

A long time ago glob suggested stealing the UI from DXR, where you get a little quick-help on the operator syntax for DXR searches. It’s a pretty simple change, right?

Well, our search box is small. So the first thing it needs is to be bigger.

bigger quicksearch box

More room to work with. This required replacing the table-based layout with some flex boxes. The top-line is nearly pixel perfect to its previous table-based implementation.

We can also hide some things and begin making the UI responsive

portrait view of bmo quicksearch.

I hope to post a followup showing the quicksearch syntax helper, but this is at the moment just a side task.

(Although it ties well into the goal of implementing elasticsearch on BMO).

October 08, 2016 02:57 PM

October 04, 2016

Dylan Hardison

Sorry, I meant I changed it from 226s to 184ms

About twenty days ago it came to my attention via my colleague Ed Morley that BMO’s bzapi was very slow. It turns out he had reported the same issue the prior year as well!

Performance problems are very enjoyable to work on, I find. Especially when they are reproducible.

I lost most of the day on this, but in the end I was able to take the slowest function from executing at a leisurely 226 seconds to a very fast 184 milliseconds

October 04, 2016 01:14 AM

September 19, 2016

Dylan Hardison

Perl + Bugzilla in Outreachy Round 13!

I am very proud to be mentoring in the 13th round of Outreachy.

We have a a list of ideas and a growing list of bugs that would make for good “small contributions”.

I’m open to emails or irc discussions and I’ll try to answer any and all questions.

email: dylan [at] mozilla [dot] com or join #bugzilla, I’m in IRC from 13:00 UTC until about 22:00 UTC.

Here’s are project blurb:

Perl is a highly capable, feature-rich programming language with over 28 years of development, making it one of the longest standing FOSS projects. The Perl Foundation is funding a position working on Bugzilla, a widely used, Perl-based issue tracker. In 18 years of development, Bugzilla has grown into a complex application that is used in many different workflows by organizations including Mozilla, the GNOME Project, Red Hat, and Some of this complexity is particularly evident in the search functionality, both in implementation and in user interface. We have several proposals to simplify and improve searching, which will positively impact Bugzilla sites around the world.

September 19, 2016 07:34 PM

August 24, 2016

Dylan Hardison

happy bmo push day!

the following changes have been pushed to

discuss these changes on

August 24, 2016 12:47 PM

August 06, 2016

Emmanuel Seyman

Installing Bugzilla on RHEL/Centos 7.x

One recurring subject on the bugzilla support mailing list is the installation of the Perl modules that Bugzilla requires to function. When you are installing on a Linux distribution, the recommended course of action is to use the packages supplied by the distribution. But, on some distributions, these can be of a version lower than what is required by Bugzilla.

One such distribution is RHEL. Version 7 came out in 2014 and its long support life cycle make it popular for people to use it as a base for their bugzilla instance. Using Bugzilla 4.4 is easy because all Bugzilla's mandatory Perl packages are available by default but Bugzilla 5.0 came out after RHEL7 was released and a number of Perl modules aren't available or are outdated. Users then try to install the missing modules themselves, which is something easy to get wrong. I decided to see if this could be made simpler.

My first step was to see if the modules could be supplied in EPEL, a third-party software repository for RHEL and CentOS but some of the modules concerned are in RHEL which makes them impossible to update via EPEL. Another solution is the use of Fedora's COPR, an easy way for Fedora developers to maintain third party repositories for Fedora and/or RHEL.

It took some amount of tweaking to find the package versions that allow you to run Bugzilla 5.0.x without replacing RHEL7's entire Perl stack but the end result is here. Activating a COPR repo is pretty straightforward:

At which point, you can install bugzilla just like any other application:

The bugzilla package will be updated everytime a new version of Bugzilla is released and its support will end when the Bugzilla developers end support for the 5.0 branch.

August 06, 2016 03:13 PM

July 13, 2016

Dylan Hardison

happy bmo push day!

the following changes have been pushed to

discuss these changes on

July 13, 2016 04:47 AM

Fixed some memory leaks in

So last week I fixed Bug 1282606 which has resulted in a bit of a performance improvement for

Restarts per Hour

Apache2::SizeLimit is configured to kill processes once they use more than 700M and this happened about every 7 minutes.

About two weeks ago, while working on some performance issues relating to BMO’s new show_bug ui, I discovered that the problem could get worse: running out of memory every 60 seconds. Should everyone switch to the new UI (which is intended to put less load on the server) a lot more load would be on the server. That’s pretty bad, since we want everyone using the new UI as soon as possible. :)

This memory leak isn’t new, and I had filed an investigatory bug about it last year. Memory leaks in perl are caused by having cyclic references, and the solution is to not have cycles, use weak references, or to break the cycle when you’re done with whatever data structure it is part of.

I understand the problem, and I know how to fix it… but maybe I don’t know where the problem is?

Thankfully there is a tool for this on CPAN: Devel::MAT.

Using Devel::MAT, it is possible to dump the address space of a perl program and explore it in great detail in a GUI.

I didn’t set out to remove all the memory leaks this time, just the ones that were the biggest or grew the fastest. This meant the TrackingFlags extension Flag objects, the Bug object, and the Comment object.

The changes are on github for the curious, and the resulting charts below speak for themselves.

Average Request Time

Requests Before Restart

Age of Process Before Restart

July 13, 2016 02:10 AM

July 04, 2016

Dylan Hardison

Bugzilla on Heroku: Part 2

I’ve gotten bugzilla to run acceptably well on Heroku. Still no memcached support (I need to work on my patch for that a little bit more and then add support to bugzilla for it) but I’m pretty happy with this.

Until I take it down, here is a working Bugzilla 5.1.1+ install on heroku, fronted by Emails works, so it is possible to create new accounts.

July 04, 2016 04:36 PM

July 03, 2016

Dylan Hardison

Bugzilla on heroku? And hacking on Memcached::libmemcached

I thought I’d take this long weekend to do something fun: get Bugzilla running under Heroku.


  1. fork the perl/psgi buildpack to run bugzilla. bugzilla buildpack
  2. swear at Gerv for adding login name support and swear at for spinning in an infinite loop when it can’t prompt for a missing config Bug 1284021.
  3. start writing support for storing the “params” data in the db instead of the filesystem, as the filesystem in heroku is ephermal.
  4. realize that you’ll want to memcache this, so might as well add memcache to heroku
  5. realize MemCachier (one of the herkoku memcached providers) requires username and passwords and that the perl bindings don’t support this

After some research… I realize this feature requires support for the binary protocol and is based on SASL. Fine. I’ll learn XS (perl’s FFI) and contribute code to Memcache::libmemcached. How hard can it be?

It turned out to be not very hard but it’s a work in progress (and definitely leaks memory right now).

Oh yeah, and bugzilla does run on heroku, but it won’t be useful until the params stuff can be stored in the DB.

July 03, 2016 12:53 AM

May 19, 2016

Bugzilla Update

Release of Bugzilla 4.4.12, 5.0.3, and 5.1.1

Today we have several new releases for you!

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the
Security Advisory that was published along with these releases.

Bugzilla 5.0.3 is our latest stable release. It contains various
useful bug fixes and security improvements:

Bugzilla 4.4.12 is a security update for the 4.4 branch:

Bugzilla 5.1.1 is an unstable development release.
This release has not received QA testing from the Bugzilla Project, and should not
be used in production environments. Development releases exist as previews of the
features that the next major release of Bugzilla will contain. They also exist for
testing purposes, to collect bug reports and feedback, so if you find a bug in this
development release (or you don’t like how some feature works) please
tell us.

Note:Make sure ImageMagick is up-to-date if the BmpConvert extension is enabled.
If no updated ImageMagick version is available for your OS, we recommend to disable the BmpConvert
extension for now (bug 1269793).

May 19, 2016 07:04 PM

Dylan Hardison

happy bmo push day!

the following changes have been pushed to

discuss these changes on

May 19, 2016 06:19 AM

May 16, 2016

Mark Côté

BMO's database takes a leap forward

For historical reasons (or “hysterical raisins” as gps says) that elude me, the BMO database has been in (ughhh) Pacific Time since it was first created. This caused some weirdness on every daylight savings time switch (particularly in the fall when 2:00-3:00 am technically occurs twice), but not enough to justify the work in fixing it (it’s been this way for close to two decades, so that means lots of implicit assumptions in the code).

However, we’re planning to move BMO to AWS at some point, and their standard db solution (RDS) only supports UTC. Thus we finally had the excuse to do the work, and, after a bunch of planning, developing, and reviewing, the migration happened yesterday without issues. I am unreasonably excited by this and proud to have witnessed the correction of this egregious violation of standard db principles 18 years after BMO was originally deployed.

Thanks to the BMO team and the DBAs!

May 16, 2016 12:52 AM

May 13, 2016

Dylan Hardison

Developing Bugzilla with Plackup and cpanminus

(note: these notes will be incorporated into bugzilla documentation after some revisions) (updated: Fri, May 13 2016 at 10:24 US/Eastern with additions from dkl)

Firstly, we need some system dependencies:

System Dependencies for Debian and Ubuntu users

$ sudo apt-get update
$ sudo apt-get install git perl-modules build-essential cpanminus \
   libssl-dev libexpat1-dev

System Dependencies for CentOS users


Bugzilla Code & CPAN Modules

A checkout of bugzilla is required, and everything we do will be from that directory.

$ git clone
$ cd bugzilla

Next, we can run with its fancy new –cpanm flag to install all our dependencies into local directory. Note that we do not need to be root for any of these commands, and nothing (from this point on) needs to be installed to the system.

$ perl --cpanm

If all goes well, that should complete with a message saying you need to edit localconfig and re-run checksetup. If you’re fine just using SQLite, you do not need to edit localconfig. Just re-run, as below:

$ perl

The above will prompt you for an email, username, realname and password. After you provide those details, it will configure the database.

Now we can run a development server, with the following perl command below:

$  perl -Ilocal/lib/perl5 local/bin/plackup -R Bugzilla -R -R templates app.psgi

This will start an http on http://localhost:5000.

Navigate to it, and login using the email/password you used above.

Bugzilla will direct you to set the urlbase – which you can set to to the same “http://localhost:5000” above.

Any changes made to files in Bugzilla and templates will trigger a restart. Currently changes to .cgi files might not be picked up, but that should be solved shortly.

Note that these are draft instructions, I appreciate testing and feedback on them. I will fill the todo items in as time allows, and hopefully we get this into the documentation for the Bugzilla 6.0 release.

May 13, 2016 02:11 AM

February 08, 2016

Bugzilla Update

Bugzilla Project Meeting Wednesday, July 17th, at 14:00 UTC

Much of this post is taken from a message I posted to the developers list a few days ago, so my apologies in advance to anyone reading it again. I’ve expanded on a few things and added the information about the upcoming meeting, so it’s probably worth re-reading.

For those unaware of the context, Frédéric Buclin last week announced that he was stepping down from his Assistant Project Manager position after 9 years.

To Frédéric: Thanks again (and again!) for all your hard work over the years! As stretched as I’ve been for time myself it has been a true godsend to have you picking up my slack the last few years. You will be missed!

To everyone else: For the time being, I’ll be handling approval requests, so if you have something up for approval and it’s not getting attention, I’m the one to pester.

This is sort of the end of an era for the Bugzilla project… Both Frédéric and Max (who left to work at Google a couple years ago and stepped down from his position earlier this year for lack of time) have been with the project for much longer than most people ever stick with a single employer in IT-related jobs (of which an open source project of this magnitude has a lot of similarities). For an open source project, that’s outright amazing, as people tend to come and go a lot in most projects. It’s kind of surprising that I’ve been around longer than them, but I’m kind of a “lifer” in some ways, and in reality I’ve had a good break from the project for the last few years because Max and Frédéric have been mostly taking care of everything while I’ve been busy with other things.

So it’s time to begin a new era. Since I’ve had a good break to clear the monotony I’m going to be trying to get more involved myself again (which I’ve been saying ever since Max left, but I have a lot more incentive now). I’d also like to kickstart a new team to lead the project, and kind of re-organize if you will. We have a number of positions within the project for various functions, which we’ve never really paid attention to as people moved on. So some questions we’ll be asking at our upcoming meeting are:

We also have some other “reinventing the project” type topics while we’re at it. There’s a number of things we’ve been talking about doing for a long time that we never really moved on, and some of the big elusive dreams (the big UI overhaul!) have actually been making progress as well, lately. When we’re in the middle of big changes like this, I think it’s a good time to review where we are, get everyone on the same page, and tackle some of these things we keep talking about.

We also have a lot of new useful technology at our disposal since the last time we had a project meeting. We’re going to experiment with using Google Hangouts for the meeting this time, and using their feature to stream the Hangout via YouTube for those who want to watch without participating. We’ll also keep our usual meeting IRC channel open so people who don’t have a Google+ account and don’t want to get one can still participate and ask questions via IRC.

The preliminary agenda and participation instructions have been posted at The meeting will be held on Wednesday, July 17th, at 14:00 UTC. And before anyone complains about the time, this was the best time to avoid inconveniencing the largest number of people. The Bugzilla Project has a global pool of contributors, and we have active contributors in a wide variety of time zones. The time chosen puts the meeting in the middle of the night for the fewest number of people. Those on the west coast in the US will probably have to get up a little early, and those in eastern Australia will be up a little later.

A lot of the emails and comments I’ve gotten since Frédéric’s announcement have been really positive, so I’m encouraged by the number of people who are still committed to keeping Bugzilla vibrant! We’ll see you at the meeting on Wednesday!

February 08, 2016 12:30 AM

January 08, 2016

Mark Côté

BMO in 2015

It’s been a whole year since my last BMO update, partly because I’ve been busy with MozReview (and blogging a lot about it), and partly because the BMO team got distracted from our goals by a few sudden priority changes, which I’ll get to later in this post.

Plans from 2014

Even with some large interruptions, we fully achieved three of our five goals for the year and made good progress on a fourth.

Alternative Bug Views

Have you tried out the new modal UI? Although not completely finished (it lacks a few features that the standard UI has), it’s very usable. I don’t remember the last time I had to switch back, and I’ve been using it for at least 6 months. Bonus: gone is the intermediate page when you change a bug’s product, a gripe from time immemorial!

Even though there are still a large number of controls, the new UI is a lot more streamlined. glob gave a brief presentation at a Mozilla Project Meeting in November if you’d like to learn more.

The part we haven’t yet undertaken is building on this new framework to provide alternate views of bug data depending on what the user is trying to accomplish. We want to experiment with stripping down the presented data to only what is needed for a particular task, e.g. developing, triaging, approving, etc. The new UI is a lot more flexible than the old, so in 2016 we’ll build out at least one new task-centric view.

GitHub Authentication

If you haven’t noticed, you can log into BMO via GitHub. If you’ve never used BMO before, you’ll be prompted to set up an account after authenticating. As with Persona, only users with no special privileges (i.e. not admins nor people in security groups) can log in via GitHub.

Auth Delegation

Originally designed to smooth the process of logging into Review Board, auth delegation for API keys is actually a general-use feature that greatly improves the user experience, not to mention security, of third-party apps by allowing them to delegate authentication to BMO. There’s now no reason for apps to directly ask for your BMO credentials!

MozReview Details

There’s now a panel just above the attachments table that shows all the MozReview commits associated with the displayed bug along with a bit of other info:

We’re currently sorting out a single method to display other relevant information, notably, status of reviews, and then we’ll add that to this table.

Improved Searchability

This is the big item we haven’t made much progress on. We’ve got a plan to mirror some data to an Elasticsearch cluster and wire it into Quick Search. We’ve even started on the implementation, but it’s not going to be ready until mid-2016. It will increase search speeds, understandably one of the more common complaints about BMO.

Curve balls

We had two sets of surprises in 2015. One was work that ended up consuming more time than we had expected, and the other was important work that suddenly got a big priority boost.

BMO Backup in AWS

The first is that we moved the BMO failover out of a data center in Phoenix and into the cloud. IT did most of the work, but we had to make a series of changes to BMO to facilitate the move. We also had a lot of testing to do to. The upside is that our new failover system has had more testing than our old one had for quite some time!

Hardened Security

In August we found out that an attacker had compromised a privileged BMO account, using a combination of a weak, reused password and an exploit in another web site. In addition to a huge forensics effort from the great security folks at Mozilla, the BMO team implemented a number of security enhancements to BMO, most notably two-factor authentication. This work naturally took high priority and is the main reason for the slippage of our big 2015 goals. Here’s to a more secure 2016!

Other Stuff

As usual, the BMO team rolled out a pile of smaller fixes, enhancements, improvements, and new features. A few notable examples include

You can always find the exhaustive list of recent changes to BMO on the wiki or on the group/mailing list.

January 08, 2016 01:20 AM

December 22, 2015

Bugzilla Update

Release of Bugzilla 5.0.2, 4.4.11, and 4.2.16

Today we have several new releases for you!

All of today’s releases contain two security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 5.0.2 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.4.11 is a security update for the 4.4 branch:

Bugzilla 4.2.16 is a security update for the 4.2 branch:

December 22, 2015 10:38 PM

September 10, 2015

Bugzilla Update

Release of Bugzilla 5.0.1, 4.4.10, and 4.2.15

Today we have several new releases for you!

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 5.0.1 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.4.10 is a security and bug fix update for the 4.4 branch:

Bugzilla 4.2.15 is a security and bug fix update for the 4.2 branch:

September 10, 2015 08:47 PM

September 08, 2015

Bugzilla Tips

Better Bugzilla Documentation

The recent release of Bugzilla 5.0 was the first release sporting the new, improved Bugzilla documentation, which has been moved from DocBook XML to reStructuredText to make it easier to edit and so more likely to be up-to-date :-).

Readers of this blog may be particularly interested in the User Guide. Feedback on the new docs is welcome, particularly suggestions for where it is lacking – please use the bug filing link at the bottom of each docs page.

September 08, 2015 12:50 PM

August 07, 2015

Bugzilla Tips

New Modal UI for show_bug on BMO

For the past few months, a new experimental modal bug view has been available on This hides a lot of complexity both through being read-only initially, and also through having expandable and collapsible sections. It is also quicker and less resource-intensive to load. It requires you to click an “Edit” button, which reveals much more UI, when you want to do more than the common operations – add a comment, CC/NI people, or change the Status or Resolution. It is also more dynamic – for example, there are no more interstital pages when moving bugs between Products, because when you change the Product, the new lists of Components, Milestones etc. load into the widgets automatically.

A picture is worth a thousand words. Here’s the initial view:


And here’s the Edit view:


You can enable it in the Preferences – change “Use experimental user interface” to “On”.

Thanks to glob for doing the hard work on making this happen. This view has not yet made it upstream to Bugzilla itself.

August 07, 2015 02:35 PM

July 30, 2015

Bugzilla Update

Bugzilla 4.2 will be EOL on 2015/11/30

As discussed on the Bugzilla developers mailing list/newsgroup and confirmed by project leads, Bugzilla’s new release policy is to end-of-life the oldest supported major version four months after a new major release. Since this is the first time we’re enacting this policy, we’ve extended the date to be 4 months from today rather than from Bugzilla 5.0’s exact release date (July 7th).

Thus Bugzilla 4.2 will be end-of-lifed on 30 November 2015. This means no fixes of any kind will be issued for Bugzilla 4.2 from that date onwards. As usual, all Bugzilla admins are encouraged to upgrade to the latest version of Bugzilla as soon as possible, especially those running 4.2 or earlier.

To expand a bit further on our new EOL process, after a new major release, we will support three major releases (including the new one) for four months.  After four months, we will drop support for the oldest one.  We will continue to support the remaining two until four months after the next release.

For example, currently we support 4.2, 4.4, and 5.0.  In four months’ time we will drop support for 4.2 and support only 4.4 and 5.0.  When the next major version comes out, perhaps 5.2*, we will support 4.4, 5.0, and 5.2.  Four months later, we will drop support for 4.4 and support only 5.0 and 5.2.

Mark Côté
Assistant Project Lead, Bugzilla

* It may be 6.0 if there are major and/or breaking changes.

July 30, 2015 06:13 AM

July 07, 2015

Bugzilla Update

Release of Bugzilla 5.0

Today the Bugzilla Project is extremely proud to announce the release of Bugzilla 5.0!

It has been slightly over two years since we released Bugzilla 4.4 on May 2013. This new major release comes with many new features and improvements to WebServices and performance.

We hope that you enjoy and appreciate the results of the past two years of hard work by our entirely-volunteer community.

EOL for 4.0.x

Please note that the release of Bugzilla 5.0 also marks End Of Life for the Bugzilla 4.0 series, meaning that there will be no further updates for the 4.0.x series, even if there are serious security
issues found in that series. We recommend that all installations running the 4.0 series upgrade as soon as possible to 5.0.

July 07, 2015 09:20 PM

April 21, 2015

Bugzilla Update

VCS updates: bzr moving, cvs ending

At the Bugzilla project meeting on 2015-03-25 the project lead and assistant leads agreed on two major changes to Bugzilla’s source-code hosting:
  1. CVS support is officially dropped as of now. 4.0 is the last version that was released on CVS, and it will be EOLed when 5.0 comes out (very soon; rc3 was just released). In the event of a release on the 4.0 branch before it is EOLed, any Bugzilla installations that have not migrated to bzr or git will have to apply patches to upgrade, which will continue to be distributed as usual. Bugzilla site admins are strongly encouraged to migrate to pulling from as soon as possible.
  2. Bazaar hosting has been officially switched from to is already active and syncing changes from is no longer syncing changes and will soon be shut down. Any sites upgrading from must do one of the following to apply any future upgrades, in order of preference: will continue to mirror changes from for the 4.0, 4.2, and 4.4 branches as long as they are supported. Note that, at the moment, master/trunk is being mirrored as well, but no other branches, including and subsequent to 5.0, will be mirrored to, and trunk mirroring may cease at any time.

Note that is waiting on a proper certificate; please use plain http until this is resolved.

The Bugzilla team apologizes for any inconvenience. Please see our support options if you have trouble migrating.

Mark Côté
Assistant Project Lead, Bugzilla

April 21, 2015 09:13 PM

April 15, 2015

Bugzilla Update

Release of Bugzilla 5.0rc3, 4.4.9, 4.2.14, and 4.0.18

Today we have several new releases for you!

Bugzilla 5.0rc3 is our third Release Candidate for Bugzilla 5.0. This release has received QA testing, and should be considerably more stable than the development releases before it. It is still not considered fully stable, and so you should understand that if you use it, you use it at your own risk.

If feedback from this release candidate indicates that it is mostly stable, then Bugzilla 5.0 will be released in a few weeks. If feedback indicates that more extensive fixes are needed, there may be another release candidate after this one.

Bugzilla 4.4.9 is our latest stable release. It contains various useful bug fixes:

Bugzilla 4.2.14 is a bugfix update for the 4.2 branch:

Bugzilla 4.0.18 is a bugfix update for the 4.0 branch:

April 15, 2015 08:22 PM

February 18, 2015

Gervase Markham


Complaining about (BMO) is a Mozilla project activity as old as the hills. Back in 2009, it was realised by the Foundation that to make everyone happy was (and still is) an impossible task, and I was given a mandate to “help people solve their own problems”. So around September 2009, I released the first version of my Bugzilla API proxy software, BzAPI. This software presented a clean, well-documented RESTful interface on the front end, and did all sorts of things on the back end (XML, CSV, RPC, HTML scraping) that developers no longer had to worry about. We made a dev server VM for it so people could try it out –

It was popular. Extremely popular. People started building things, and then more things, all of which depended on this server for Bugzilla data. For various reasons, IT never got around to building a production instance, and so over the last five years, I’ve been maintaining this core piece of Mozilla project infrastructure, which was depended on by TBPL and many, many other tools which interfaced with Bugzilla. At its peak, it serviced 400,000 requests per day.

Over the intervening years, BMO itself acquired a REST API which slowly became more capable, and then a BzAPI-compatible API shim was implemented on top of it by the excellent dkl, so people could change their code to access BMO directly just by updating the endpoint URL. After a few false starts, requests to are now served directly by BMO, via that shim code. Earlier today, the api-dev VM was finally powered down.

Here’s to you, api-dev. Good job.

February 18, 2015 04:04 PM

Bugzilla Tips

Bugzilla Has New Documentation

The Bugzilla team recently finished a big project to update, rewrite, improve and centralize Bugzilla’s documentation. You can find it at In particular, there’s a User Guide which will be useful to, er, Bugzilla users.

If you have suggestions for further improvements to the documentation, please let the team know.

February 18, 2015 02:36 PM

January 27, 2015

Bugzilla Update

Release of Bugzilla 5.0rc2, 4.4.8, 4.2.13, and 4.0.17

Sorry for the new release so soon, but we found a regression in the release last week and felt it would be best to get the fix out now rather than wait.

Bugzilla 5.0rc2 is our second Release Candidate for Bugzilla 5.0. This release has receive QA testing, and should be considerably more stable than the development releases before it. It is still not considered fully stable, and so you should understand that if you use it, you use it at your own risk.

If feedback from this release candidate indicates that it is mostly stable, then Bugzilla 5.0 will be released in a few weeks. If feedback indicates that more extensive fixes are needed, there may be another release candidate after this one.

Bugzilla 4.4.8 is our latest stable release. It contains an important bug fix:

Bugzilla 4.2.13 is a bug fix update for the 4.2 branch:

Bugzilla 4.0.17 is a bug fix update for the 4.0 branch:

January 27, 2015 08:23 PM

January 21, 2015

Bugzilla Update

Release of Bugzilla 4.4.7, 4.2.12, 4.0.16, and 5.0rc1

Today we have several new releases for you!

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 5.0rc1 is our first Release Candidate for Bugzilla 5.0. This release has received QA testing, and should be considerably more stable than the development releases before it. It is still not considered fully stable, and so you should understand that if you use it, you use it at your own risk.

If feedback from this release candidate indicates that it is mostly stable, then Bugzilla 5.0 will be released in a few weeks. If feedback indicates that more extensive fixes are needed, there may be another release candidate after this one.

Bugzilla 4.4.7 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.2.12 is a security and bugfix update for the 4.2 branch:

Bugzilla 4.0.16 is a security and bugfix update for the 4.0 branch:

January 21, 2015 11:00 PM

January 19, 2015

Mark Côté

BMO 2014 Statistics

Everyone loves statistics! Right? Right? Hello?

tap tap

feedback screech

Well anyway, here are some numbers from BMO in 2014:

BMO Usage:

33 243 new users registered
45 628 users logged in
23 063 users performed an action
160 586 new bugs filed
138 127 bugs resolved
100 194 patches attached

BMO Development:

1 325 bugs filed
1 214 bugs resolved

Conclusion: there are a lot of dedicated Mozillians out there!

January 19, 2015 01:09 AM

January 07, 2015

Mark Côté

BMO 2014 update part II

The second half of 2014 was spent finishing up some performance work and shifting into usability improvements, which will continue into 2015.

More performance!

By the end of 2014, we’d managed to pick most of the low-to-medium-hanging fruit in the world of Bugzilla server-side performance. The result is approximately doubling the performance of authenticated bug views. Here are graphs from January 2014 and October 2014:

The server now also minifies and concatenates JavaScript and CSS files. This affects cold loads mostly, since these files are cached, but even on reload it saves a few round trips.

As mentioned above, we’re shifting focus away from performance work and towards usability/work-flow improvements, but there will still be perf benefits, both by reducing and delaying loading of content and by making it easier for users to accomplish common tasks.

New, better documentation

We’ve converted the upstream Bugzilla documentation to reStructuredText, massively updated and reorganized it, and, perhaps of most interest to anyone reading this, completely rewrote the API docs, which were very hard to grok.

For BMO specifically, we’ve fixed up the wiki page. We’ve also started a user guide, but it’s just a skeleton at the moment. There are lots of users out there who know the ins and outs of BMO, so feel free to contribute a section!

GMail support

To support Mozilla’s transition to GMail, we added two features. First, we now limit the number of emails sent to a user per minute and per hour, since GMail will temporarily disable accounts that receive too much mail, and some BMO users receive a lot of bugmail. Second, since GMail’s ability to filter mail by headers is limited compared to other email servers, users can now include the X-Bugzilla-* headers in the body via General Preferences.

Other things


As I’ve said a few times now, in 2015 we’re going to do a lot of work on improving general BMO-user productivity: usability, UX, work flows, whatever you want to call it. I’ll write more about this later, but here are a few things we’re looking into:

As usual, if you have questions or comments, you can leave them here, but an even better place is the mailing list, also available as a Google Group and via NNTP.

January 07, 2015 07:31 PM

December 06, 2014

Bugzilla Tips

Making Bugzilla Searches Faster

People often wonder how to make searches in Bugzilla faster on large installations. Two things will give you the most bang for the buck:

Do those two things, and your searches will be much faster.

Coincidentally enough, Bugzilla’s “Simple Search” (BMO version) allows you to specify precisely those two things.

December 06, 2014 10:14 PM

December 01, 2014

Gervase Markham

Search Bugzilla with Yahoo!

The Bugzilla team is aware that there are currently 5 different methods of searching Bugzilla (as explained in yesterday’s presentation) – Instant Search, Simple Search, Advanced Search, Google Search and QuickSearch. It has been argued that this is too many, and that we should simplify the options available – perhaps building a search which is all three of Instant, Simple and Quick, instead of just one of them. Some Bugzilla developers have sympathy with that view.

I, however, having caught the mood of the times, feel that Mozilla is all about choice, and there is still not enough choice in Bugzilla search. Therefore, I have decided to add a sixth option for those who want it. As of today, December 1st, by installing this GreaseMonkey script, you can now search Bugzilla with Yahoo! Search. (To do this, obviously, you will need a copy of GreaseMonkey.) It looks like this:

In the future, I may create a Bugzilla extension which allows users to fill the fourth tab on the search page with the search engine of their choice, perhaps leveraging the OpenSearch standard. Then, you will be able to search Bugzilla using the search engine which provides the best experience in your locale.

Viva choice!

December 01, 2014 12:01 AM

November 28, 2014

Gervase Markham

Bugzilla for Humans, II

In 2010, johnath did a very popular video introducing people to Bugzilla, called “Bugzilla for Humans“. While age has been kind to johnath, it has been less kind to his video, which now contains several screenshots and bits of dialogue which are out of date. And, being a video featuring a single presenter, it is somewhat difficult to “patch” it.

Enter Popcorn Maker, the Mozilla Foundation’s multimedia presentation creation tool. I have written a script for a replacement presentation, voiced it up, and used Popcorn Maker to put it together. It’s branded as being in the “Understanding Mozilla” series, as a sequel to “Understanding Mozilla: Communications” which I made last year.

So, I present “Understanding Mozilla: Bugzilla“, an 8.5 minute introduction to Bugzilla as we use it here in the Mozilla project:

Because it’s a Popcorn presentation, it can be remixed. So if the instructions ever change, or Bugzilla looks different, new screenshots can be patched in or erroneous sections removed. It’s not trivial to seamlessly patch my voiceover unless you get me to do it, but it’s still much more possible than patching a video. (In fact, the current version contains a voice patch.) It can also be localized – the script is available, and someone could translate it into another language, voice it up, and then remix the presentation and adjust the transitions accordingly.

Props go to the Popcorn team for making such a great tool, and the Developer Tools team for Responsive Design View and the Screenshot button, which makes it trivial to reel off a series of screenshots of a website in a particular custom size/shape format without any need for editing.

November 28, 2014 11:52 AM

November 25, 2014

Bugzilla Tips

Email Filtering on

Vanilla Bugzilla lets you decide which bugmail you receive based on what changed about a bug. But there are a couple of extensions which give you even more control, and both are installed on So the new email filtering pipeline is as follows:

Firstly, ComponentWatching, as the name implies, lets you “watch” particular products or components, so you get put on the list to receive bugmail for all changes to any bugs in those products or components. This is very useful if you have an interest in a particular area of the project. You can also watch particular users – that function is built-in.

Secondly, the normal email filters run, which exclude or include you from emails based on the particular fields which have been changed in the bug update.

Lastly, the BugmailFilter extension allows you to define “include” or “exclude” rules based on any one of:

Using these three capabilities in tandem, it should be possible to carefully control how much bugmail you receive, even if you are using a system like Gmail which does not have good client-side filtering.

November 25, 2014 05:40 PM

November 19, 2014

Gervase Markham

BMO show_bug Load Times 2x Faster Since January

The load time for viewing bugs on has got 2x faster since January. See this tweet for graphical evidence.

If you are looking for a direction in which to send your bouquets, glob is your man.

November 19, 2014 10:42 AM

October 06, 2014

Bugzilla Update

Release of Bugzilla 4.4.6, 4.2.11, 4.0.15, and 4.5.6

Today we have several new releases for you!

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 4.4.6 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.2.11 is a security update for the 4.2 branch:

Bugzilla 4.0.15 is a security update for the 4.0 branch:

Bugzilla 4.5.6 is an unstable development release. This release has not received QA testing from the Bugzilla Project, and should not be used in production environments. Development releases exist as previews of the features that the next major release of Bugzilla will contain. They also exist for testing purposes, to collect bug reports and feedback, so if you find a bug in this development release (or you don’t like how some feature works) please tell us.

October 06, 2014 07:33 PM

Gervase Markham

New Class of Vulnerability in Perl Web Applications

We did a Bugzilla security release today, to fix some holes responsibly disclosed to us by Check Point Vulnerability Research, to whom we are very grateful. The most serious of them would allow someone to create and control an account for an arbitrary email address they don’t own. If your Bugzilla gives group permissions based on someone’s email domain, as some do, this could be a privilege escalation.

(Update 2014-10-07 05:42 BST: to be clear, this pattern is most commonly used to add “all people in a particular company” to a group, using an email address regexp like .*$. It is used this way on to allow Mozilla Corporation employees access to e.g. Human Resources bugs. Membership of the Mozilla security group, which has access to unfixed vulnerabilities, is done on an individual basis and could not be obtained using this bug. The same is true of BMO admin privileges.)

These bugs are actually quite interesting, because they seem to represent a new Perl-specific security problem. (At least, as far as I’m aware it’s new, but perhaps we are about to find that everyone knows about it but us. Update 2014-10-08 09:20 BST: everything old is new again; but the level of response, including changes to, suggest that this had mostly faded from collective memory.) This is how it works. I’m using the most serious bug as my example. The somewhat less serious bugs caused by this pattern were XSS holes. (Check Point are going to be presenting on this vulnerability at the 31st Chaos Communications Congress in December in Hamburg, so check their analysis out too.)

Here’s the vulnerable code:

my $otheruser = Bugzilla::User->create({
    login_name => $login_name, 
    realname   => $cgi->param('realname'), 
    cryptpassword => $password});

This code creates a new Bugzilla user in the database when someone signs up. $cgi is an object representing the HTTP request made to the page.

The issue is a combination of two things. Firstly, the $cgi->param() call is context-sensitive – it can return a scalar or an array, depending on the context in which you call it – i.e. the type of the variable you assign the return value to. The ability for functions to do this is a Perl “do what I mean” feature.

Let’s say you called a page as follows, with 3 instances of the same parameter:


If you call param() in an array context (the @ sigil represents a variable which is an array), you get an array of values:

@values = $cgi->param('foo');
['bar', 'baz', 'quux']

If you call it in a scalar context (the $ sigil represents a variable which is a scalar), you get a single value, probably the first one:

$value = $cgi->param('foo'); 

So what context is it being called in, in the code under suspicion? Well, that’s exactly the problem. It turns out that functions called during hash value assignment are evaluated in a list context. However, when the result comes back, that value or those values are assigned to be part of uthe hash as if they were a set of individual, comma-separated scalars. I suspect this behaviour exists because of the close relationship of lists and hashes; it allows you to do stuff like:

my @array = ("foo", 3, "bar", 6);
my %hash = @array;
{ "foo" => 3, "bar" => 6 }

Therefore, when assigning the result of a function call as a hash value, if the return value is a single scalar, all goes as you would expect, but if it’s an array, the second and subsequent values end up being added as key/value pairs in the hash as well. This allows an attacker to override values already in the hash (specified earlier), which may have already been validated, with values controlled by them. In our case, real_name can be any string, so doesn’t need validation, but login_name most definitely does, and it already has been by the time this code is called.

So, in the case of the problematic code above, something like:


would end up overriding the already-validated login_name variable, giving the attacker control of the value used in the call to Bugzilla::User->create(). Oops.

We found 15 instances of this pattern in our code, four of which were exploitable to some degree. If you maintain a Perl web application, you may want to audit it for this pattern. Clearly, param() calls are the first thing to look for, but it’s possible that this pattern could occur with other modules which use the same context-sensitive return feature. The generic fix is to require the function call to be evaluated in scalar context:

my $otheruser = Bugzilla::User->create({
    login_name => $login_name, 
    realname   => scalar $cgi->param('realname'), 
    cryptpassword => $password});

I’d say it might be wise to not ever allow hash values to be assigned directly from functions without a call to scalar.

October 06, 2014 07:13 PM

August 28, 2014

Bugzilla Update Disclosure

UPDATE: We have reset all passwords on all Landfill test Bugzilla systems. All users will be required to set a new password the next time they access the test Bugzilla systems.

One of our developers discovered that, starting on about May 4th, 2014, for a period of around 3 months, during the migration of our testing server for test builds of the Bugzilla software, database dump files containing email addresses and encrypted passwords of roughly 97,000 users of the test build were posted on a publicly accessible server.  As soon as we became aware, the database dump files were removed from the server immediately, and we’ve modified the testing process to not require database dumps.

Generally, developers who use our test builds have told us they understand that these builds are insecure and may break, so they do not use passwords they would reuse elsewhere.  However, because it is possible that some users could have reused their passwords on other websites or authentication systems, we’ve sent notices to the users who were affected by this disclosure and recommended that they change any similar passwords they may be using. It’s important to note that, unless users reused the password they used on, this does not affect email addresses or passwords.

We are deeply sorry for any inconvenience or concern this incident may cause you.


Mark Côté

Assistant Project Lead, Bugzilla

August 28, 2014 08:11 PM

July 24, 2014

Bugzilla Update

Release of Bugzilla 4.0.14, 4.2.10, 4.4.5, and 4.5.5

Today we have several new releases for you!

All of today’s releases contain a security fix. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 4.4.5 is our latest stable release and contains a security fix.

Bugzilla 4.2.10 is a security update for the 4.2 branch:

Bugzilla 4.0.14 is a security update for the 4.0 branch:

Bugzilla 4.5.5 is an unstable development release. This release has not received QA testing from the Bugzilla Project, and should not be used in production environments. Development releases exist as previews of the features that the next major release of Bugzilla will contain. They also exist for testing purposes, to collect bug reports and feedback, so if you find a bug in this development release (or you don’t like how some feature works) please tell us.

July 24, 2014 09:40 PM

May 03, 2014

Gervase Markham

Bugzilla 1,000,000 Bug Sweepstake Results

Milestone bug 1,000,000 was filed on 2014-04-23 at 01:10 ZST by Archaeopteryx (although rumour has it he used a script, as he also filed the 12 previous bugs in quick succession). The title of the bug was initially “Long word suggestions can move/shift keyboard partially off screen so it overflows” (a Firefox OS Gaia::Keyboard bug, now bug 1000025), but has since been changed to “Celebrate 1000000 bugs, bring your own drinks.”

The winner of the sweepstake to guess the date and time is Gijs Kruitbosch, who guessed 2014-04-25 05:43:21 – which is 2 days, 4 hours, 33 minutes and 5 seconds out. This is a rather wider error, measured in seconds, than the previous sweepstake, but this one had a much longer time horizon – it was instituted 9 months ago. So that’s an error of about 0.95%. The 800,000 bug winner had an error of about 1.55% using the same calculation, so in those terms Gijs’ effort is actually better.

Gijs writes:

I’m Dutch, recently moved to Britain, and I’ll be celebrating my 10th “mozversary” sometime later this year (for those who are counting, bugs had 6 digits and started with “2” when I got going). Back in 2004, I got started by working on ChatZilla, later the Venkman JS debugger and a bit of Firefox, and last year I started working on Firefox as my day job. Outside of Mozilla, I play the piano every now and then, and try to adjust to living in a nation that puts phone booths in its cycle paths.

The two runners-up are Håvard Mork (2d 14h 50m 52s out) and Mark Banner (8d 8h 24m 36s out). Håvard writes:

My name is Håvard Mork. I’m a Java software developer, working with Firefox and web localization to Norwegian. I’ve been involved with localization since 2003. I think localization is rewarding, because it is a process of understanding the mindset of the users, and their perception of IT.

I’m surprised that my estimate came that close. I spent almost an hour trying to figure out how much bug numbers grow, and estimate the exponential components. Unfortunately I lost the equation, so need to start over for the 2,000,000 sweepstakes…

Mark writes:

I’m Mark Banner, also known as Standard8 on irc, I work from home in the UK. I came to Mozilla through volunteering on Thunderbird, and then working at Mozilla Messaging. I still manage Thunderbird releases. Alongside those, I am working on the Loop project (formally Talkilla), which is aiming to provide a real time communications service for Mozilla products, built on top of WebRTC.

Gijs will get a Most Splendid Package, and also a knitted thing from Sheeri as a special bonus prize! The other winners will receive something a little less splendid, but I’m sure it’ll be worth having nevertheless.

May 03, 2014 07:57 PM

April 28, 2014

Gervase Markham Stats At 1,000,000

Thanks to glob, we’ve got some interesting stats from BMO as it crosses the 1M bug mark.


NEW          103655
ASSIGNED       8826
REOPENED       3598
RESOLVED     640326
VERIFIED     220235
CLOSED         1628



DUPLICATE    119242
EXPIRED       10677
FIXED        303099
INVALID       58096
MOVED            27
WONTFIX       36179


DUPLICATE     64702
EXPIRED          27
FIXED        108935
INVALID       17099
MOVED           150
WONTFIX        6105

Bugs Filed Per Day (April)

2014-04-01    519
2014-04-02    531
2014-04-03    620
2014-04-04    373
2014-04-05    133
2014-04-06    132
2014-04-07    544
2014-04-08    622
2014-04-09    597
2014-04-10    571
2014-04-11    467
2014-04-12    156
2014-04-13    170
2014-04-14    573
2014-04-15    580
2014-04-16    574
2014-04-17    619
2014-04-18    356
2014-04-19    168
2014-04-20    118
2014-04-21    445
2014-04-22    635
2014-04-23    787
2014-04-24    562
2014-04-25    498
2014-04-26    173

Busiest Days Ever

2013-12-30    1360 (bulk import from another tracker)
2013-12-29    1081 (bulk import from another tracker)
2008-07-22    1037 (automated security scanner filing bugs)
2012-10-01    1013 (Gaia bugs import)
2014-02-11    805
2014-04-23    787
2014-02-04    678
2013-01-09    675
2013-11-19    647
2014-04-22    635

User Activity

(You can find these stats about yourself by going to your own user profile. If you are logged in, you can search for other users and see their stats.)

Top 10: Assignees         349671          16385      15056             13350        11974         10995         4768             4697            4672    4273

Top 10: Reporters          8037          6129      5032            4789             4351      4348     4038            3680            3651         3528

Top 10: Commenters          347695           148481     65552           58588            50560         48840            48704           47453           43596         42885

Top 11: Patches Attached             8080            4879            4502            4397        4079                3930    3890          3739          3659               3530               3411

Top 11: Reviews           15581            14869                9424              8352            8103        7272    6198          5983              5499        5346        5126

April 28, 2014 12:38 PM

April 19, 2014

Bugzilla Update

Release of Bugzilla 4.5.4, 4.4.4, 4.2.9, and 4.0.13

There are four new releases today. All of today’s releases contain an important bug fix discovered since the last release.

Bugzilla 4.4.4 is our latest stable release. It is a bug fix update for the 4.4 branch:

Bugzilla 4.2.9 is a bug fix update for the 4.2 branch:

Bugzilla 4.0.13 is a bug fix update for the 4.0 branch:

Bugzilla 4.5.4 is an unstable development release. This release has not received QA testing from the Bugzilla Project, and should not be used in production environments. Development releases exist as previews of the features that the next major release of Bugzilla will contain. They also exist for testing purposes, to collect bug reports and feedback, so if you find a bug in this development release (or you don’t like how some feature works) please tell us.

April 19, 2014 02:08 PM

April 18, 2014

Bugzilla Update

Release of Bugzilla 4.5.3, 4.4.3, 4.2.8, and 4.0.12

Today we have several new releases for you!

All of today’s releases contain security fixes. We recommend that all Bugzilla administrators read the Security Advisory that was published along with these releases.

Bugzilla 4.4.3 is our latest stable release. It contains various useful bug fixes and security improvements:

Bugzilla 4.2.8 is a security update for the 4.2 branch as well as contains several bug fixes:

Bugzilla 4.0.12 is a security update for the 4.0 branch:

Bugzilla 4.5.3 is an unstable development release. This release has not received QA testing from the Bugzilla Project, and should not be used in production environments. Development releases exist as previews of the features that the next major release of Bugzilla will contain. They also exist for testing purposes, to collect bug reports and feedback, so if you find a bug in this development release (or you don’t like how some feature works) please tell us.

April 18, 2014 02:34 AM

March 27, 2014

Bugzilla Update

(re)introducing Mark Côté, Bugzilla Assistant Project Lead

I’ve invited Mark Côté to step up to fill the Assistant Project Lead position vacated by Simon Green two months ago. He’ll also be taking on a role as a “Community Coordinator” to try to step up efforts to make new community members feel welcome and encourage more involvement.

You probably all know him most recently for his leadership in the project to move our source control from bzr to git. He’s a long-time developer outside of Bugzilla, and has been heavily involved with Bugzilla the last year or so via his participation in maintaining He’s mainly been in the role of a project manager for BMO, and that’s really what Bugzilla needs right now. We haven’t had a really good project manager or community coordinator in a long time, and the state of the project kinda shows it. In another first (in recent history), “approval” rights aren’t initially coming with the job. Any patches that need commit approval can continue to be directed towards Byron (glob) or myself (justdave).

I’ve had a long-standing policy of trying to avoid having the entire senior leadership team being employed by Mozilla, in order to try to keep it a real community project and not feel like it was being controlled by Mozilla, but the reality is that nobody else from outside of Mozilla has been involved enough to step into this kind of role in the recent past, and it’s better to have it filled and get things done than to leave it vacant and let the project stagnate even further. If he’s an effective community builder, that problem will probably solve itself eventually.

We’re going to try to set up another real-time project meeting soon either on IRC or Air Mozilla or in Google Hangouts again (that wasn’t too bad when we did it) so we can regroup on where we are and where we plan to go. Expect to be hearing from Mark on that soon.

For more information about Mark, see his Mozillians profile at or his LinkedIn profile at or find him in the #bugzilla channel on IRC as mcote.

March 27, 2014 03:12 AM

February 17, 2014

Bugzilla Update

Git Migration Scheduled

A migration of Bugzilla and related code from to will be perfomed on Tuesday, 11 March 2014, starting at 17:00 UTC.  At this time, all Bazaar branches on will be made read-only (aside from a few admin accounts), and the migration to git repos on will commence.  It should take around 1.5 hours to migrate everything, after which point write access will be enabled on the git repos for all users previously authorized on  A script will periodically mirror changes from git to bzr for all currently supported Bugzilla branches (4.0, 4.2, and 4.4).  Changes will not be mirrored for any other branches of Bugzilla nor any other related branches (extensions, misc, etc.).

We will start mirroring changes to read-only repos on GitHub at some point (to be determined) after the migration to will remain the repository of record, meaning the only place to which changes should be committed by developers.  All mirroring, e.g. to GitHub and, will be unidirectional.

We’ve already done one test migration; see  It was successful aside from some missed file deletions, resulting in extra files on a handful of git repos after the migration.  I manually deleted the superfluous files after migration, and I also fixed the migration script to account for this oddity in Bazaar’s fast-export output, so it won’t happen during the real migration.

I would like to open up testing to all developers, starting with another complete, fresh migration, on Tuesday, 18 February 2014, around 17:00 UTC.  To test the git-to-bzr mirroring script, we’ll create a new branch, “migration-test”, off of Bugzilla trunk and run the mirroring script on it after the migration.  We’ll leave it running until the real migration, and I invite anyone with commit access to to commit changes to the test-migration branch on git and ensure that they are mirrored properly to bzr.

The full migration and testing plan, along with other details, is at

Post reprinted from Mark  Côté’s post on

February 17, 2014 08:23 AM