Tag Archives: firefox

A Year of Open Source

This last year has been quite an interesting ride for me. A year ago, basically to the day, I first met the crazy people I now work with everyday on Popcorn and Popcorn Maker: Scott Downe, Jon Buckley, David Seifried and Chris De Cairos back at a friendly gathering to watch a UFC pay per view. I was pretty much set on taking the first course in open source taught at Seneca by David Humphrey but it was rather interesting to meet these new people and learn more about it and the kinds of things they did.

A year ago I literally didn’t know much. I had the basics skills taught to me by the previous courses at Seneca but honestly they didn’t prepare me for a lot of things with the web. Working on Popcorn introduced me to a lot of things that I didn’t know, especially with javascript. Man, I remember when I had no clue how to really use an XHR request and was so stumped on what to do for my really crappy tumblr plugin. Compared to today it’s quite amazing the differences. I can’t claim I know everything or even a lot right now but I find myself able to tackle a lot of different things but more importantly I’m able to work with others on the things I can’t figure out and discuss different approaches to finding a solution for a problem.

More importantly it’s the opportunities I get working on this project. It’s so cool to work with, collaborate with and meet all the ridiculously smart people with Mozilla. Some of those include:

Bobby Richter
Secretrobotron. Pretty much a genius. He’s sort of the software lead and an awesome guy. He also likes long walks on the beach, morning sunrises and phish.

Kate Hudson
Popcorn Makers designer. Truthfully she is also one hell of a programmer and could easily just work on any of the bugs the rest of us do. Honestly quite amazing. Don’t ever get her angry though.

What’s even better has to be some of the reactions of people when we show them the software and seeing how they use it. Yesterday and today we have been doing user testing and just watching them use the app is awesome. When you write up all of this and think about user experience for the app you feel like you have it covered pretty well but then all of the sudden someone brings up something completely different. You thought maybe the purpose of something was obvious but it isn’t at all and basically get’s ignored or some cool feature that get’s overlooked completely because it isn’t apparent at all that it even exists.

Honestly, I can’t wait for what the future brings. We have so many cool things left that we want to do and I just want to keep on working harder and harder. There’s so many cool things I want to do and working on all of these has given me many ideas for what I’d like to do in the future and the best part is with open source I can easily make that happen 🙂

Anyways, back to work!

Advertisements

Dealing with Bugs in Other Software

This is a well over due blog post. Honestly, we’ve been so busy with Popcorn Maker it’s been hard to just stop and take sometime to write one of these up 🙂 Anyway, I wanted to talk about one of the things that came along one day out of the blue.

Sometimes there actually isn’t bugs in your code!
We’ve always been striving to make our tests better and it’s an ongoing process. However one day out of the blue we quickly discovered that our tests literally would just stop running in Firefox after the update to version 14. Now our tests load a lot of video; It’s just part of the territory when your library is involved with video on the web it’s just going to happen. Now, recently in Firefox 14 there was a limit placed on the amount threads that can be used to decode media. Combine this with a bug in the code and our test runner would literally just stall in Firefox.

This is obviously a problem and we can’t just ignore the problem and let our tests just “not work” in Firefox until it gets fixed. At this point our tests were all grouped up into single files based on what they were testing. All media module tests were in one single test file, all trackevent test files were grouped together and so on. This meant that we weren’t giving the browser as much time as possible to teardown every single piece of media. In the end what we wound up doing was breaking down every single QUnit qsyncTest block we had into it’s HTML page. This not only means that it’s easier to trace problems with failing/stalling tests but it helps solve our issue of loading too much video too quickly for the browser.

Often one solution creates another problem
As the title suggests, this only opened the door for another problem with our how we load our node server locally. To replace having to use a generic makefile system we use shell.js along with various node modules like CSLINT, JSHINT etc and this allows us to avoid writing annoying makefile code with javascript instead. In particular we use their exec command which allows us to execute typical commands on the command line. For an easy way to start our server we have a shortcut command that changes into the cornfield app’s directory and starts the server.

Now the problem here was another bug that’s present with running shell.js’ exec function asynchronously. Without it our node app would eat a lot of system resources and with it our test runner would cause the server to crash by loading so many resources quickly. If you manually started the app no problems would arrise. Alas, we were blocked by another bug.

However, a clever solution was found! Our code went from:

target.server = function() {
  echo('### Serving butter');

  cd( CORNFIELD_DIR );
  exec('node app.js', { async: true });
};

To….

target.server = function() {
  echo('### Serving butter');

  cd( CORNFIELD_DIR );

  // Use child_process.spawn here for a long-running server process
  // (replaces `exec('node app.js', { async: true });`).
  var server = spawn( 'node', [ 'app.js' ] );

  // Mostly stolen from http://nodejs.org/docs/v0.3.5/api/child_processes.html#child_process.spawn
  server.stdout.on( 'data', function( data ) {
    process.stdout.write( data );
  });

  server.stderr.on( 'data', function( data ) {
    process.stderr.write( "" + data );
  });

  server.on( 'exit', function( code ) {
    console.log( 'server process exited with code ' + code );
  });
};

This allowed us to properly have the server be started asynchronously and no longer have any issues with the tests running in Firefox!

Still need that continuous integration stuff though so we can have our tests always running upon checkin, allowing us to catch any problems earlier.

Code Code Code …. Review Review Review

Code freeze for the 0.5 release of Popcorn Maker/Butter is coming up Tuesday and we have a lot to do before we close the code merging flood gates. In addition to this week being 1 day shorter for most of us because of Victoria day, we also had people venturing off to a conference downtown Thursday making things even tighter schedule wise. Needless to say we have needed to pick up the load to nth level to make up for all the lost man power (or maybe it’s finger power?)!

This week I finished off a few things. One was adding a test harness system for Butter. Now we can have one file that will run all of our QUnit based tests all at once, helping make testing much easier. It will land Monday since it’s now the weekend and whatnot.

I also finished updating the the cornfield tests. They now are use the proper updated methods and on top of that fixing them exposed a few problems with our validation for passed in data. On top of that they are just plain better than they were before. A lot of places in the tests previous used timeouts to call QUnit’s start() method to QUnit know it could run each assertion which could cause potential issues. They now follow a “cascading” style if you will that will have subsequent calls to browserid to happen withing the callback function of the previous call, allowing us to know that the previous work has finished preventing any asynchronous issues. For example:

asyncTest("Async API", 7, function() {
 var foundProject = false;

 butter.cornfield.list( function( res ){
 ok(res, "The project list response has data" );
  equal( res.error, "okay", "Project list status is \"okay\"" );
  ok( res.projects, "There is a list of projects" );

  for( i = 0, len = res.projects.length; i < len; i++ ){
   if( res.projects[ i ].name === filename ){
    foundProject = true;
    break;
   }
  }

  equal( false, foundProject, filename + " is not present in the projects list" );

  butter.cornfield.load( filename, function( res ){
   deepEqual( res, { error: "project not found" }, "The project load response is project not found" );

   butter.cornfield.save(filename, stringedData, function(res){
    equal( res.error, "okay", "The project save response is okay" );

    filename = res.project._id;

    butter.cornfield.load(filename, function(res){
     deepEqual( JSON.parse(res.project), data.data, "The project is the same" );

     start();
    });
   });
  });
 });
});

With this setup we know start won’t be called until our servers has completed our requests with browserid and our database holding project information.

On top of that I took over the manual tests that we are making. These tests contain a list of instructions for people to run and what they expected outcome should be if they follow those instructions. They can then report if the expected results were achieved or not. Eventually these will be linked for a database of some sort where we can monitor the results people will be reporting. Most of the real issues that we had were because the tests themselves were being loaded into an iframe. This caused issues with the sources provided for various plugins and other scripts because they were relative links in our config file. I also realized we were missing some vendor prefixed versions of linear-gradient on the index.html page, causing the links for the manual tests not to appear in browsers that weren’t FF, making it look like nothing loaded at all.

On top of all this there were many reviews. While reviews require that the person doing the reviews takes their time looking over it and is a very important task, it also means that someone has sort of gained enough experience in the project that they can be “trusted” so to speak which is a nice sign that I am progressing a lot within the Butter and Popcorn projects.

Other Cool Stuff
We also have basically a complete rewrite to our flash players in the works. David Humphrey started the work to basically mimic the HTMLMediaElement spec and early testing is showing it’s going to work quite awesome.

We also have the Django front end coming up for the 0.5 release, slated to be available on June 15th. This will create a new template management system and bring a lot of new cool stuff to the project.

0.5 is slated to be about 200 tickets! Yikes.

Can’t wait for the rest 🙂

I borked something!

I’m so proud. No really, it’s awesome.

Anyway, over our reading week I noticed myself getting a bunch of bugmail from a bug that I couldn’t recall ever CC’ing myself to. This was rather strange I thought, so I figure I would check it out.

The ticket starts off at first with people looking which nightly build was the cause, trying to find the exact regression point. Eventually they made their way to my simple dumb fix for Unprefix Blob.mozSlice, basically just removing the moz prefix which is apart of an ongoing process/debate about adopting better standards because of WebKit on mobile browsers. Removing these prefixes will help prevent a situation in the mobile browsing market akin to Internet explorers dominance during the 90’s.

Anyway, it turns out I broke the fast upload feature for registered accounts on Mediafire. They can no longer upload files in FF13+.

Woo!

OSD700 – Release 0.6

Truthfully a smaller release than what I wanted, at least in quantity of work accomplished. This release relied more on research along with trial and error. Although even then, still lacking in my opinion.

What I think is most important from me in this release is the fact that I am transitioning to more difficult tasks. Things that require a decent bit more thought behind them.

My work has been primarily involved in two tickets.

Bug 720768 – Add an equivalent to DispatchToSelf in nsContentUtils
Bug 516811 – “load” events for poster image loads bubble out

With Bug 720768 it seems pretty simple on the surface because of some of the work I have done in the past. All that needed to be done was remove duplicate code that appears in multiple places and add a helper method to nsContentUtils that will cover the removed codes functionality. The problem here comes with dealing with the version of the code in SmsManager.cpp. When initializing this event there is a static cast being done which is completely different than how it is done in any other place in the code. At this point I’m trying to speak with some of the people experienced with that area of the code because from what I can tell/what I know, I don’t think it can be done completely with this one method.

With Bug 516811 there is a problem with the load event for a videos poster image. This one is going to involve adding a new method that will catch the load event itself so I can suppress it and fire out a different one instead called mozposterloaded event instead. I’ve looked over some of the code plenty and it’s going to involve some work with the EventStateManager. Time to open up some of the mouselock code and talk to Steven, Diogo and Raymond some to get an idea of where I should be looking!

Again, rather short on this release. I wish I was able to do a lot more work but other things (cough PRJ666) take up too much of my time :/

See you in two weeks!

Mozilla Bug Process using Git

With OSD700 now under way and many of the students in our class working on various sections of the mozilla code base I figured it would be helpful to give everyone a few pointers and a brief explanation of how to go about this.

Typical work flow might be like this:

  1. Get a bug to work on. Seems dumb to put but hear me out. With Bugzilla you aren’t able to just grab bugs as you please as a new user. You either need to get someone with the access rights to assign people or ask around about this. David Humphrey would obviously be a good place to start.
  2. Code Code Code Code Code Co…………. You get the point. Work on your bug. When you have it to point where either you feel it solves all the problems adequately or you just want some feed back proceed to point 3.
  3. At this point you have probably made many different commits. Maybe they were small changes, maybe they weren’t. Either way it’s best to rebase your code against master before doing a diff on your code to make the patch itself.
    1. Make a new temporary branch that is literally a duplicate of the one you have been working in.

      git checkout -b branchNameHere

    2. Until you get used to the process it’s best to take this precaution incase you mess things up (trust me, it hurts when you do).

    3. Go and update your master branch. You will want to make sure you have the most recent version of mozilla-central on your machine. Then return to the branch containing your changes.
    4. git rebase -i master


      This will allow you to rebase your code and, for what we want to do, combine all of our commits into ONE so we have only one nice patch file. After doing that you will wind up getting something like this on your screen:

    5. If you read the comment’s in blue you can see that there are options we can take while rebasing. What were are concerned with here is f, or fixup. We need one commit to still have p/pick beside it but every other commit needs to have f/fixup instead. For the one commit that will have pick beside it you need to ensure that it’s commit message is an appropriate one as it won’t be accepted otherwise by your reviewers. Something like the first commit in my example.
    6. After continuing you just need to finish off the rebase process and get ready to use a super duper awesome alias!
  4. Once you have finished rebasing you are on the home stretch. All it takes is one simple little git command and your are done. The only problem is this command isn’t available by default because it helps setup your patches to be in an acceptable format for Mozilla. Add this to your .gitconfig under [ alias ]:
    hgp = show --format=\"From: %an %n%s%n%b\" -U8

    Once you have this in your .gitconfig all it takes is a simple command to make a patch that will show the differences in your code compared to your local mozilla-central (Which should be up to date if you followed the instructions!):

    git hgp >place/file/in/whatever/directory/you/want/it/to/be/makeThisAMeaningFulNameForYourBug.patch
    
    For Example:
    
    git hgp >~/Desktop/bug698384.patch
    
  5. Woot! Now you just need to upload it to your bug, set it for review? and then who is going to review it! Wait, how do you know who should/could review it? Fear not! This should help you. If it doesn’t well, ask Humph if all else fails. Or #developers.

One final note. If your work ever requires you to use git mv on any of your files (maybe it needs to move to a different spot or you need to rename it completely) you will want to add this to your .gitconfig:

[diff]
  renames = copy

This will help ensure your patches are in an acceptable format for Mozilla.

Hope this helps!

Beginning Firefox Development

Today’s class in open source development was an interesting one. Basically in addition to the other open source projects we are working on during the semester we will be attempting to fix a bug (or was it add a feature?) along with our professor David Humphrey. The bug in particular is located here and is about implementing the Mouse Lock API into Firefox.

This feature will basically allow a user freedom of movement with their mouse. If their cursor hits the sides of whatever window they are using it will allow them to continue scrolling in the desired direction rather than forcing the user to pick up their mouse, move it in the opposite direction you were, set it down and then continue moving it in the original direction to scroll further. An example can be seen here with it’s current implementation in Flash. The w3c specs that we will be following for the Mouse Lock API can be found here.

We also had Mike Shaver in today, a former VP at Mozilla. He helped us get an idea of the kinds of things we should keep in mind while developing for big projects like Firefox. When it comes to code review we need to realize that the people who do the reviews (experts) are often very busy so it can make turn around time a week or more. There are different kinds of flags as well when it comes to reviews that will allow us to notify what we are looking for in regards to response on the bug. Making progress doesn’t come from simply adding more or changing existing code but can be simply finding information about existing code through different APIs.

Firefox in itself is one massive and complicated project. It involves multiple programming languages controlling different layers of the program and has dynamically generated code and tools, making it impossible for any one person to even consider understanding it all them self. This is why ambient learning is an important concept as it allows you to focus yourself on a particular subject or area of the code and become and expert on it allowing yourself to be of aid whenever someone else needs to use or understand your code. Until then, you’re going to have to ask for help! This can be done in many different ways including but not limited to: getting an example, using mailing lists, looking at blame logs for people who worked on the issue and information in a bug tracking service such as Bugzilla.

There are also some interesting sites that can help out the aspiring Mozilla developer by helping them learn about existing code.

  • https://mxr.mozilla.org/ Allows you to search through the different branches of code and looks for instances of essentially anything. For example, if you wanted to know the common uses for some method called…. lets say testSomethingHere(), you could search for that specific string across multiple files in a branch of the Mozilla repo all at once.
  • https://tbpl.mozilla.org/ Allows you to have a quick view of the different built in test opts.

Phew, a lot to take in all at once. For now, as emphasized, we need to take it all in small steps. It starts my simply getting the latest code from mozilla central which for use will be done through a github repo of our professor found here. I’ve cloned this and have it on my machine at home and all setup and needless to say, there is A LOT of code there. The next step is to make our own build of Firefox! I completed this using the steps I found here but I have the slight feeling I didn’t necessarily complete the task as intended. I’m sure I will find that out later after seeing some more blog posts over the week from my fellow classmates 😛

For now, that is all. Time to get some sleep for the rest of the week!

Good luck hacking everyone!