Monthly Archives: April 2012

Using find with sed to replace a whole buncha stuff!

So, taking what I’ve learned through open source it’s good to document things you have had difficulty with/learned. Not only can it potentially help others but chances are you probably will forget the exact specifics and you now have an easy to find solution to your problem!

My groups PRJ666 project is winding down to the final few polishing steps. One thing that has annoyed me is that all of our links have been prefaced with /PRJ666-Implementation/pages/ to prevent previous issues we were having before, which are now gone. I wanted to remove all of this nonsense because, in the off chance someone actually uses this project (Hah!) and they decide to change one element of the deployment (a highly likely move)…. KABOOM. Site’s broken.

I know there are a lot of powerful commands available to me using command line unix. I just didn’t really know how to combine them together.

I used a combination of find, sed and a bash script to help solve this problem. Here’s what the code looked like:

for files in `find PRJ666-Implementation/web/pages/ -type f -print 2>/dev/null`
sed 's/\/PRJ666\-Implementation\/pages\//\.\.\//g' $files >$files.tmp
mv $files.tmp $files

This finds all files in a directory, including it’s subdirectories and searches for the string /PRJ666-Implementation/pages/ and removes it by replacing it with ../ .

Hope this helps someone out there like me. These things are definitely one of my many weaknesses and it’s something I’m looking to improve on.


OSD700 – 1.0 Release

Our final release of the course is ment to show not only all the work that we have accomplished but show that we can have something that we can easily show people. Something our in the world of open source that is visible to other people. While obviously all those more “minor” things we do along the way are just as important, if not more, they aren’t quite as noticeable.

For me this was implementing video Playback Jitter, one of many missing media statistics in Firefox. While I have some working at this point, it’s not in a passable state yet because I do not know how to fix one other area of the code.

Playback Jitter, as defined by the spec, is the intended duration of the current video frame spent on screen, minus the actual duration spent on screen. Some of this data (at least, what I think is the correct data) was already available to be inside VideoFrameContainer::SetCurrentFrame via the information passed in about the current frame with aTarget (the target time this frame would be presented) and mPaintTime/lastPaintTime(the time the last frame/current frame being displayed actually was presented). These two seemed like the right information I needed to calculate the second part there, with that being the time the frame actually spent on screen.

if (!lastPaintTime.IsNull() && !aTargetTime.IsNull()) {
mPlaybackJitter += (aIntendedDuration - (aTargetTime - lastPaintTime).ToMilliseconds());

The key part above was that I needed to introduce a way to this method to know the actual time the frame was supposed to spend on the screen. I looked around at what was calling this and found nsBuiltinDecoderStateMachine::RenderVideoFrame which just so happens to have access to a class called VideoData which contains information about the start and end time of the current frame to be rendered.

From there I added a new argument to the SetCurrentFrame method that would contain this information for me:

void SetCurrentFrame(const gfxIntSize& aIntrinsicSize, Image* aImage,
                     TimeStamp aTargetTime,
                     PRInt64 aIntendedDuration);

And with the appropriate linkage through the use if IDLs and making getters available I can now access this information on my own Firefox builds. I’m unsure how close this is to being right, but once someone has time to review it I’ll be fixing up my mistakes ASAP as this stuff is a lot of fun.

I’ve also taken the task of making the HTMLProgressElement no longer inherit from HTMLFormElement and instead inherits from HTMLGenericElement. This required removing an attribute, it’s getters/setters, as well as updating forward declarations of various methods and updating some tests. I liked this one because it was sort of a mini test of everything I have learned along the way. It allows me to step from the IDL level of things and make all the connections needed in the C++ land of the code.

I’ve enjoyed working on all this Firefox stuff because it really allows me to see the kind of thought process and work that goes into making something as simple as document.getElementById in javascript land actually works. I plan on continuing with the bugs I have but also work more with Core/DOM bugs as they have been very fun and a good learning process for me. This course has been a blast and I have loved every minute of it.

Pit Stop Number One in Open Source – The Start of the Second Leg

My how this has been quite the experience. I’m going to try and keep this structured but honestly I’ll probably head off on tangents every now and then.

This all started in September last year for me when I decided to take the first of two open source courses taught here at Seneca, both by David Humphrey. I took it based off the recommendation from my friend Dave Seifried as he enjoyed the course and was continuing to do work on it through the position he obtained with the Centre for Development of Open Technology, CDOT for short, at Seneca College @ York. I figured I’d give it a shot because a lot of the work he was doing was involved with the web and I was fairly confident that I wanted to push my programming career in that path when it was all said an done. Granted I had no idea about the specifics beyond just “Web Development” but I knew it was a start.

Oh yeah, I also happened to meet these buffoons one fateful night if you will at the end of August in 2011 who definitely helped give me more interest in taking the particular course: Jon Buckley, Chris DeCairos and Scott Downe.

I definitely didn’t come into the course knowing a lot about open source itself. I definitely had somewhat of an idea but it was mostly limited to “So you mean the software that’s free right?”. I had no clue what some of the truly deep philosophy behind it was before reading The Cathedral and the Bazaar by Eric S. Raymond or watching Revolution OS(Still a great watch BTW. Totally went back over it today, or is it yesterday…). Both of these really opened not only my eyes but my mind as to what an open source community was and the kinds of cool things people have done over the years.

From there I was quickly exposed to some of the basic principles of any well run open source project. I found a typo in a comment in one file I was looking over and sort of jokingly pointed it out in IRC. I was then told to file it, fix it and then submit it up for review. Next thing I knew I was being assigned to handle all the updates that were happening to this particular piece of code as it was in an area that was being focused on for that release and bam, I pretty much became the owner of that little plugin. Granted in the big scope of things it’s honestly rather small but that doesn’t change the fact that it was rather cool and exciting to say I have code that is out there in projects being used by REAL people compared to all the silly projects we do in the rest of our courses.

The important part behind all of this were the different real world practices I was learning. Some of it was simply better ways to code as up til this point I hadn’t been exposed to much javascript and that’s what I was primarily programming with for most of this course. Beyond just the raw coding skills I learned about the work that goes into making a good piece of code and getting it accepted. Part of this was learning to adapt to specific coding styles but a lot of it was learning about the review process that goes into any good project. It’s this kind of stuff that you can easily take to anything else you work on after this point and apply it to better all the code you write.

Blog! Blog! Blog!
This is definitely another thing that I have been exposed to because of the work I’ve done in these courses. Blogging is a great tool for an open source developer because it allows you to document the various work you are doing. You can write about the difficulties you have had and perhaps other people will notice it and try and help you out; or you could simply write about how you solved a problem or the steps you took to doing something and then easily reference it later for your own use or to share with others. I’ve done it many times myself when needing to figure out how to do something again, such as pushing patches for FireFox bugs up to the try server.

OSD700 – The No Failing allowed version of Open Source
What I mean by that is before it was quite alright if we didn’t actually land anything as it was all still a learning process (and really still is). The idea now was that we weren’t “open source babies” anymore and we need to put our own stamp on the web in some form. For most of us this was through working on Firefox bugs and I was not an exception here.

I could easily list off the exact bugs I have worked on, what’s landed and whatnot but that’s not the point here. The point is pushing ourselves harder in this course has allowed me to learn and do so much more than I have in any other course. I went from contributing to an open source library to an open source mega project. The sheer size of the code base and complexity of the stuff you see with FireFox is astounding. No one man/woman can ever understand it all. Such a person doesn’t exist.

The big thing here though was the amount of confidence it gives each and everyone of us. I’m no C++ guru but working with this kind of code helped me sharpen my own skills and learn about how they are able to connect the side of the browser I was used to with Javascript all the way down to the core operations with your OS. I had never even heard of IDL files before this but now I know what they are, what they do and how to work with them in your code. Without doing this kind of work I would never have the confidence to go and actually report bugs to big projects like these, especially fairly significant bugs (at least, in my opinion) like what I threw over at Google!

These courses have really shaped me in ways that are hard to imagine. They have given me the opportunity to work with real projects and the ability to say that I have contributed to these real world projects that thousands or even millions of people use! They have given me the opportunity to strengthen my theoretical knowledge in the languages that I have worked with and exposed me to real world implementations using these languages, showing me more efficient ways of doing some of the things that I have learned in my previous courses. On top of all that I have gotten the opportunity to meet some cool people who work on these various projects and work along side of them.

I still have a lot to learn. Hell, I’ve only actually been programming for about three years now and working with these projects have been real eye openers to me, but in a good way. An awesome way. This may be the first pit stop in my little adventure here but there’s a lot of big things coming and I can’t wait. If you are a student at the college and still have the opportunity to take these courses; Do it. You won’t regret it.

Till next time!

OSD700 – Release 0.9

The journey is nearly complete and more work has been done!

I figured out why I broke video playback in the original demo I tried to show the linkage of my mozPlaybackJitter statistic. I was originally trying to also subtract the frameDelay from my calculation at the time. The problem is when it’s going through the first time there literally is no value for this yet. I’m fairly certain this is what the cause of the problem was.

This then brings me to the issue of solving the second part of the equation. I am fairly confident that Ei (Desired duration of frame i spent on the screen) was being calculated correctly as this information was available to me in the VideoData argument of nsBuiltInDecoderStateMachine::RenderVideoFrame, from there simply passing that into VideoFrameContainer::SetCurrentFrame. The second part, Ai (Actual duration frame i spent on the screen) is a little bit trickier. Eventually after looking around it a little bit more the thought of using the passed in argument aTargetTime of SetCurrentFrame and subtracting mPaintTime in ImageContainer from it. In my head at least this value should represent the actual time the frame spent on screen.

The next part of the problem is converting it all down to one type. As it stands I haven’t been able to figure this one out. The information I get for Ei is of PRInt64 type. The target time and paint time of the last image/video frame are both of TimeStamp type. From everything I can tell there doesn’t appear to be a way for me to convert these into some sort of similar type (preferably TimeDuration, as this would allow me to convert it to various measurements in seconds).

The final part of the problem is the fact that most of the people who would be able to assist me with this area of the code all live in/around Australia making for a massive time difference. Oh yeah, it’s a long weekend right now to boot!

I also took on Bug 686913 as a quick addition. Also to help get back into things as I had to take some time off at various points. This one isn’t too difficult but it’s different at the same time because I have never had to deal with changing the class entirely that one inherits from. The main difference here is dealing with all those NS_FORWARD declarations. I pushed my patch to try and there’s probably more fails than what would normally be desired.

I also spent some time looking into Bug 723020 which involves adding column information to window.onerror allowing for more detailed error information for developers. The problem I’m left with here is the W3 bug linked doesn’t, from what I can tell, have any information on where this spec is now. I wanted to go read up on it as it’s definitely interesting but I’m stuck there.

For our final release I hope to have my video statistic implemented and finished. Perhaps even staged. Along with that I want to be able to finally get some work done on this new one as well as it’s very interesting to me.

Till then!

Implementing Video Playback Statistics in Firefox – Part II

So, after my demo horribly failed during my presentation last week it was clear that my initial attempt at just making some sort of data about the video being accessible through my mozPlaybackJitter attribute didn’t work at all. The video would never load and trying to play it would freeze my browser.


I hate it when mistakes aren’t at compile time…

Been going over my code and honestly, being a rather inexperienced programmer, this is a rather futile task. I haven’t been able to nor do I feel I’ll be able to fix it this way. I’m thinking it’s time to pull out the o’ll GDB.

More soon!