Paul O’Shannessy

blah. blah. blah.

What’s the Deal With V8Monkey & SpiderNode?

tl;dr Neither are being actively worked on and could use some love.


After NodeConf, development of V8Monkey & SpiderNode slowed. We all got a bit burnt out in the weeks preceding NodeConf and took a little break. Shortly after, Shawn left Mozilla to work at Facebook with Rob on [redacted] and the little break became longer. I was in the middle of planning a wedding & helping the (now-)wife with her company. Shawn & Rob were busy with work and some other side projects. Mozilla opened an office in SF in August, so I stopped riding the train down the peninsula with Shawn & Rob as often. Long story short, we lost interest.

In October I pulled current mozilla-central into the V8Monkey tree & tried to update from there. But some core things changed. APIs changed around Typed Arrays in SpiderMonkey. JSScript stopped being a JSObject which broke some of our assumptions with our implementation. I have a patch locally to try to fix those, but they’re incomplete (it immediately segfaults). It’s been 3 months and the JS engine doesn’t idle, so it’s entirely possible the world changed again.

SpiderNode is untouched. But Node has definitely changed - it’s gone from 0.4.x to 0.6.x and works on Windows! It’s now using a newer version of V8, and further, using more V8 APIs that V8Monkey hasn’t ported. It also looks like Buffers got a pretty big overhaul (I started to merge that, but it got messy). There’s surely some other work that needs to be done.

The Future

There has been some mention of the JS team here at Mozilla implementing the V8 API. Not me or somebody doing it in their free time, but a full-time employee. I haven’t seen that happen, but that’s the best possible outcome. Assuming we have 100% compatibility (which is the only target that makes sense), then the path to reviving SpiderNode from there is easy. Basically we’d just need to fix the build system again to compile SpiderMonkey & use that. Brendan has also threatened to unbitrot V8Monkey, but he’s a busy guy, so we’ll see.

If that doesn’t happen, then the future is grim. I don’t have the time or drive to do this myself. If you’re interested in helping out though, keep reading…

Want to help?

Awesome! Get in touch & I can help get you started. I’m more than happy to get that going because I’d like to see these projects live on. Keep in mind that the real work mostly happens in V8Monkey — it’s 100% C++ and you’re implementing one JS engine’s API with another.

And Just Like That I’m a Firefox Peer

Not-so-long story short, as of a couple weeks ago Firefox review policies changed and now I can review any code going into Firefox (browser/)! In actuality though, if a review request came my way in an area I wasn’t familiar with, I would just give feedback and redirect the review to somebody more familiar with that code.

Read up on the new Firefox review policy and see who else can review your patches to Firefox.

max_concurrent_tabs is Dead; Long Live restore_on_demand

Since last September, you could set browser.sessionstore.max_concurrent_tabs to 0, and you would essentially have a built-in BarTab. I slipped that in later in the Firefox 4 release cycle as a part of cascaded session restore, but it required going into about:config and changing a preference value (or installing something like BarTab Lite).

Starting with the latest nightly (and soon to be Aurora 8), browser.sessionstore.max_concurrent_tabs is no more. We’re no longer allowing you to specify a specific number of tabs to restore concurrently. Instead we now allow you to either restore on demand, or use the hard-coded 3 tabs at a time value. The new preference is called browser.sessionstore.restore_on_demand. If you had customized max_concurrent_tabs and set it to 0, then restore_on_demand will be migrated to true. Unlike the old preference, restore_on_demand is exposed in the Preferences/Options dialog to make it accessible to a larger audience.

For the details, check out bug 648683.

hg qedit

Mercurial queues are still far from my ideal workflow, but they mostly just work so I still use them. I haven’t gotten into qqueues because that seems like even more management I don’t need. If I had a 17 part patch then I would probably give those a go, but I don’t.

Instead I end up reordering the patches in my queue more often that I want. I run vim .hg/patches/series, press a j, k, d, & p a few times, then quit. But sometimes I forget that I’m not at hg root so that doesn’t work (usually tab completion is a good indicator but then I need to cd a bit). Enter hg qedit.

[alias]
qedit = !vim $(hg root)/.hg/patches/series

Just add the above to your .hgrc and you now have a new command which will pop open vim so you can reorder your patches. And of course if you want this to use your editor of choice, you can change “vim” to “$EDITOR” to make that happen.


A couple people have mentioned that there are easier ways to move things around in your queue without editing your series file…

Gavin told me about qup, and Dave said that hg qpush --move <name> achieves the same thing.

Building Firefox on OS X Lion

I know there are people here at Mozilla who have been building Firefox on Lion releases for a while now, but I couldn’t find anything that described all the problems. So here I am.

TL;DR: Reinstall XCode & apply a couple patches.

Mercurial

I had originally installed Mercurial via easy_install, so your results may be different. Regardless it was broken. Apparently you can edit the executable and it works, but I didn’t try. Attempting to install Mercurial again failed with Python headers are required to build Mercurial.

Reinstall XCode then reinstall Mercurial and you’re set.

make Not Found

Reinstalling XCode fixes that too.

Build Fails with expected initializer before ‘NS_NORETURN’

Your build will probably fail. The patch in bug 655339 works to get past this.

Build Fails with $MACOSX_DEPLOYMENT_TARGET mismatch error

distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.6" but "10.7" during configure

I first thought this was a problem with specifying the wrong target SDK, but I was wrong. This is apparently a python bug. Bug 659881 has more details (and a workaround patch). The bug talks about python via MacPorts, but I’m not using MacPorts nor any non-default python installs.

Simple Tab Stats Script

My Firefox usage has always been a bit excessive. I used to open windows to group my different tasks. Then Panorama came along and I started using groups. Then I had too many groups so I started opening new windows. Luckily I set max_concurrent_tabs to 0 so I avoid loading all of those pages at startup (it’s that BarTab-like thing).

My curiosity got the better of me so I wrote a little script to tell me some facts about my usage. There are extensions that do some or all of this, but I didn’t actually want to install anything, I just wanted a quick snapshot of what was up. (That said I might turn it into an about:something sort of extension.) You can see what it tells you - it’s not much but it told me what I wanted to know. My usage was higher earlier today, and I’m sure there are people who are much more abusive of Firefox, so no judging.

Judging me on coding style? I would be. I started writing this in the error console, so I left out whitespace and used shitty variable names. Then it grew up and got all awkward.

SpiderNode at NodeConf 2011

Two weeks ago, I traveled with Shawn and Rob to Portland for the inaugural NodeConf. To summarize: it was a great experience and a well run conference. Other people have done overviews, so I’m not going to go there, but read a few here, here, and here. I’m going to talk about SpiderNode at NodeConf instead.

A Quick Aside on the State of Things

We got SpiderNode running on the train ride shortly before getting on a plane to Portland. Under 24 hours before NodeConf. It didn’t pass all tests (still doesn’t) and we had to hack around Node’s Buffers pretty majorly. Ryan’s classic node_chat ran and that was a great state to be in going into NodeConf. As for other demos, those were written right before the presentation so they were a bit unpolished.

Back to NodeConf

As we’d suspected for a while, Brendan Eich was the “Mozilla Person Secret Talk” listed on the schedule. He talked a little bit about ES.next and SpiderNode, with a couple demos. In typical Brendan fashion, it was a lot of information to get in a short period of time. If you want to see the slides, Brendan has them on his blog.

Overall, I think the reception we got was really positive. People seemed excited. Some viewed it as a way to run Node on architectures that V8 doesn’t support (before we left NodeConf I had an email from somebody telling me he had patches to get SpiderNode building on Solaris/Sparc). Others were just excited that this could open up the doors to making Node faster (more benchmarks to compete on). Other people were excited because they just really want block scoping (let).

Other people weren’t so excited. And that’s fair. But there seemed to be some misconceptions about the project and why we were at NodeConf contributing to what felt a bit like disdain.

Selling You

There were some people who seemed convinced that we were there to sell people on using SpiderNode. I think Brendan made this pretty clear during his talk, but here it is again: we’re not trying to sell you. We don’t think you should use it right now. We’re getting closer, but it might be a little while before we suggest people use it for anything besides experimenting. We don’t pass all of the Node tests and we’re almost certainly not faster at this point.

I think Brendan put it best:

We are not out to make a maintained, competing fork of Node, just a friendly downstream that should go away as soon as possible. We aren’t selling anything to Node users.

Splitting the Community

The Node community is pretty awesome and we really don’t want to split it in anyway. I’m sorry that some of you felt that’s what we were doing. If that was you, I’d love to find out what you meant.

The only angle I can see on this one is that the JavaScript one would write using SpiderMonkey’s JS extensions wouldn’t run on normal (V8) Node. This goes back to the experiment thing. If you’re writing Node modules or code you want to share widely using the JS extensions STOP.

Contributing Back

I didn’t see any comments about this topic, but I wanted to make it clear that we’re most certainly going to be contributing back to Node. We created a repository to start that process now. We’re waiting for somebody to handle this pull request, which should make some uses of Buffer faster.

There will surely be other things that can be upstreamed outside of our use of V8Monkey (and the build system changes required to support that).


I hope this clears some things up for people. Let me know if it doesn’t or you have other concerns about SpiderNode’s existence.

About That Hybrid “V8Monkey” Engine

I’ve sort of been working on this thing…

Several weeks ago I was curious about why there was no implementation of Node.js using the SpiderMonkey JavaScript engine (the one we use in Firefox). So I tweeted about it and several people said they wanted such a thing and would help work on it.

After some poking around the source, we realized that Node was tied pretty closely to V8, and there really wouldn’t be any way to use SpiderMonkey without ripping Node apart and rebuilding it. Not only would that suck now, but it would likely suck long into the future as Node gets updated. This port would fall behind and nobody wants that.

V8Monkey

So we decided that we would implement the V8 API on top of SpiderMonkey. That way we can just plug our work into Node, and it would “just work”. We’ve been chugging along on this project and have made some great progress. Thanks to John Ford we even have an automated build system running all of our tests on every checkin.

The JS team at Mozilla is also really interested in just having this API around. It has potential for other projects like this, but also raises awareness of API differences and might help push forward changes to the SpiderMonkey API. We’ve already pushed for additions to the SpiderMonkey API (for example, there was previously no exposed API to check ==, only ===). There have also been discussions about turning the SpiderMonkey API into a C++ API (not just C).

It also turns out that we’re not the first people to go down this track. There’s a team at Yahoo! who is very interested in Node and even did a very similar project. They even put the code out there to prove it and it’s been a helpful reference for us. They stopped short to get Node mostly running, while our goal is a bit more ambitious and implement the whole V8 API.

SpiderNode

SpiderNode is what we’re calling our fork of Node. Once we have our V8 binding working, we’ll shift our work over here and focus on integrating SpiderMonkey into the Node build system. Ideally we’d love to get this upstreamed and give Node developers a choice.

We think V8 is great and the fact that Node has become so widely used is a testament to that. But we also think there’s room for competition here. Browser based competition is old-hat. Let’s move this battle to the servers :)

With some hackery from John, we now have a way to build SpiderNode. As of Tuesday node.cc compiles (OMG AWESOME) but that’s it.

Can You Help?

Absolutely. The code for both v8monkey and SpiderNode are on GitHub for now. We have a "hacking" wiki page there, as well as a general TODO page.

We also chat a lot on IRC: #spidernode on irc.mozilla.org.

It’s Not Just Me

That’s for damn sure. Rob Arnold and Shawn Wilsher have been doing most of the heavy lifting. Rob has done some work on SpiderMonkey in a past life and is really good at this sort of language stuff. Shawn is also really smart and definitely no stranger to using SpiderMonkey. This project would be nowhere without these guys. John Ford set up a buildbot for us, which has been incredibly helpful for catching all the tests I break. Having a build engineer helping out has been really great.

Brendan Eich (you may have heard of him) said he’s going to start contributing. My intern Mehdi Mulani submitted a couple patches early on and others have stepped up and are working on patches right now.

My Experience So Far

Honestly, this is the first large scale C/C++ project I’ve worked on. I took 1/2 a semester of C my freshman year in college (over 6 years ago) and I’ve written some random pieces of C++ at Mozilla. Mostly though, I write JavaScript and before that Ruby & PHP - a lot of scripting languages. It’s a drastic change from what I’m used to and it’s taken me a while to get comfortable. Luckily I’m working with friends who have helped teach me as I go. Learning C++ while making one JavaScript engine work on top of another has been a real sink or swim experience.

It’s been fun though. I really needed something entirely different from Firefox for a little while and this has fit the bill perfectly.

Expect to see more from me and others in the coming days & weeks. And hey, perhaps you’ll see one of us at NodeConf.

Multiple Assignment in JS With Objects

Note: Turns out this only works in Firefox since it is the only browser to include versions of JavaScript beyond 1.5. Perhaps something will show up in ECMAScript N/Harmony/whatever.


Alright, JavaScript is pretty cool. There are plenty of “tricks” that still boggle my mind. I just found another cool trick in Mardak code (he would do this…).

This is about multiple assignment in JS. Well, it’s actually about destructuring assignment, but potato, potato.

Hopefully by now we all know that it can be done with arrays.

What I didn’t know is that it can also be done using objects.

This mostly makes sense now that I know it works, I was just a bit surprised. The syntax is strange, but it’s oddly satisfying (and follows from the assignment from an array).

Turns out this works for nested objects too…

Just keep in mind the same rules for object references applies. If I change obj2.foo.dude, that changes e.dude

And last but not least, we can also access values directly with nested objects.

Modifying obj2.foo.dude here will have no effect on g.

Add this to the list of cool things that have been implemented in JS 1.6, 1.7, and 1.8, but yet still aren’t available in all browsers (sigh). Generators, Iterators, array comprehensions, block scoping with let… all cool things that are used regularly within the Firefox codebase.

2 Years at Mozilla

On February 23rd, 2009 I started full-time at Mozilla. So much has happened since then, yet it still feels like it was just a couple months ago.

In addition to my time as an intern, I feel like I’ve seen a lot at Mozilla. With Firefox 4 I’ll be seeing my 4th release; 3.0 came out very shortly into my internship, followed by 3.5 soon after I started. 3.6 was my first version where I was around for the whole release, and 4 is so close I can taste it. I’ve had 4 bosses (Connor, Beltzner, Dietrich, and Dolske). We’re now on our 2nd CEO; I’ve seen 1 VP Engineering leave. I’ve met awesome people, and attended 2 summits.

I’ve worked on the password manager, satchel, session restore, the Jetpack prototype, the Add-on SDK, per tab network prioritization, and a whole bunch of random front-end stuff. I ported the Sync extension into Firefox and helped make Firefox work on Windows CE. I’ve even written docshell & widget code (even if it didn’t all get checked in)! The fact that I’ve gotten to work on such varying parts of Firefox is one of my favorite things about my job.

A lot has happened in my personal life as well. I lived by myself for the first time while Amanda was back at school (actually by myself, not counting dorm life or sharing a house). I’ve cleaned up more cat puke than I ever thought I would. I’ve made new friends and grown apart from older friends. I got engaged! I helped my fiancée start a company.

I guess it’s actually been a pretty long 2 years.

Updated Firefox Extension: Always Ask 2.0pre

Around the endgame for Firefox 3.6, I wrote an extension called Always Ask to make sure I always got a prompt when quitting. I used it for a while and then forgot about it when it was no longer compatible as Firefox 3.7 (now 4) moved forward.

Now that we’re at the endgame for Firefox 4, and especially with all the changes surrounding the quit dialog, I figured it should be updated. So I’m releasing v2.0pre today which updates the extension to work with component registration changes and quit dialog changes in Firefox 4. This also includes the translations that were submitted via Babelzilla (most of them over a year ago). Thanks to all of you who took the time to translate those strings.

Get it now. Or look at the code.

Why 2.0 “pre”?

Mostly just in case there are any further changes needed to be compatible with Firefox 4 (there better not be though).

Why not restartless?

This would probably work really well as a restartless extension. But I was really lazy and didn’t feel like figuring out the l10n part of that (though it looks like Erik Vold has that figured out already).

New Firefox Extension: switch-to-tab Blacklist

Switch To Tab is a pretty cool feature that will be a part of Firefox 4. I didn’t work much on it, but I contributed a bit to it last year (see this post for one example).

For the most part, I really like this new feature. It’s saved me much tab duplication over the past several month. However there are some sites that it doesn’t play well with, namely some JavaScript “applications” and pages with frames. I’ve quietly dealt with it and worked around the issues.

But a couple weeks ago after trying to do multiple Google Maps search, I decided that I’d had enough. I never wanted Google Maps to show up in switch to tab and I was going to write an add-on to make that possible. So I spent a bit of time figuring out how I might do that and looking at this new bootstrapped extension stuff (which is pretty awesome by the way). I didn’t get far enough to make it actually work and stalled. Then I got back to work and fixed some more blockers and promptly forgot about this idea.

Yesterday I started working on it again but my current approach wasn’t working (turns out gBrowser._placesAutocomplete is readonly and it means it). So I asked Shawn about it and he came up with the answer: cheat. Don’t try to intercept the additions to the database, just revert them after the fact. Smart guy.

I wrote a quick and dirty version and iterated. I released a “good enough” version 1.0 last night. “Good enough” meant it was restartless but required restarting to re-process the blacklist preference, which was… dumb. Over lunch today I finished v1.1 to fix that and a couple other small issues.

Let me have it

switch-to-tab Blacklist on AMO

Can I see the code?

It’s on github. I need to clean it up because it’s a bit hard to look at, so don’t judge me too much.

This picture is not worth 1000 words

Previously, that autocomplete result would have been said “Switch to tab:”.

Customizing the blacklist

I’m storing a JSON array of regular expressions (stored as strings which get turned into regular expressions with new RegExp(...)). This is stored in a preference: extension.switchToTabBlacklist.blacklist. You need to edit it by hand for now.

Where’s my GUI

I don’t have visual basic on this computer so I couldn’t make one.

To be honest, I just didn’t care enough to make it. I made this extension for me and I know how to edit a preference. I would gladly take patches though if somebody wants to make one.

About That Quit Dialog…

Remember when we turned it off a couple weeks ago? We did that by just flipping the browser.warnOnQuit preference to false. I mentioned that you could get old behavior back by flipping that preference back to true. That’s not going to work anymore.

I just landed bug 629485 to change things again, so pay attention. When we turned off the quit dialog, we took away any way of stopping the last window from closing without any warning, even if you explicitly set the visible pref browser.tabs.warnOnClose which would show the window closing warning (when you have multiple tabs). That wasn’t so cool. So we played around with the logic and made that possible. But in order to do so, we had to change the default value of browser.warnOnQuit to true and create a new preference which controls the quit dialog (browser.showQuitDialog). I know this sounds a bit inane, but there really wasn’t a better way to do it and maintain browser.warnOnQuit == false as an override.

It’s all a bit confusing, so I’ve documented the conditions under which we won’t show a dialog, as well as the rules for choosing which dialog to show. Some of that comment is perfectly clear; the rest requires a little bit of understanding of the code (but it shouldn’t be too hard to figure out if you want).

Don’t expect me to write another post like this. I don’t expect any further changes to how this stuff works.

TL;DR

  • We enabled the window closing dialog when closing the last window would otherwise just quit (mostly important to Windows and Linux users).
  • If you want the quit dialog back, set browser.showQuitWarning to true and make sure browser.warnOnQuit is also true.

Update: Feb. 8, 2011: Yup. I fucked that up a little bit. We aren’t showing the quit dialog if you close the last window, even with the prefs mentioned above. That should be fixed in the next nightly.

The new Firefox default home page is seriously sexy. Seriously.

A couple weeks ago Stephen Horlander designed an updated default home page for Firefox. Over the weekend, we landed most of it. So first off, a huge thanks to him.

Mihai Sucan did an awesome job converting the HTML &amp; CSS Steven prototyped into something we could check in, which involved a lot of fun with media queries  (seriously, resize your window with it open), as well as a lot of work to make sure it worked with RTL locales.

And then finally, I hooked up the &#8220;Restore Previous Session&#8221; button. Not the HTML &amp; CSS (that was Stephen &amp; Mihai), just the parts that made it hide when appropriate and work when pressed. Now hopefully a few people realize that feature exists :)

I also can&#8217;t forget to mention Marco Bonardo, who hooked up about:home back in August so that we could even get to this point.

Keep in mind, the image I&#8217;m showing here is the unbranded &#8220;Minefield&#8221; version of the page. When loading this in Firefox it won&#8217;t look like your computer is going to explode. I think there is a little bit more tweaking that will be done, but it already looks so much better than it used to.

The new Firefox default home page is seriously sexy. Seriously.

A couple weeks ago Stephen Horlander designed an updated default home page for Firefox. Over the weekend, we landed most of it. So first off, a huge thanks to him.

Mihai Sucan did an awesome job converting the HTML & CSS Steven prototyped into something we could check in, which involved a lot of fun with media queries (seriously, resize your window with it open), as well as a lot of work to make sure it worked with RTL locales.

And then finally, I hooked up the “Restore Previous Session” button. Not the HTML & CSS (that was Stephen & Mihai), just the parts that made it hide when appropriate and work when pressed. Now hopefully a few people realize that feature exists :)

I also can’t forget to mention Marco Bonardo, who hooked up about:home back in August so that we could even get to this point.

Keep in mind, the image I’m showing here is the unbranded “Minefield” version of the page. When loading this in Firefox it won’t look like your computer is going to explode. I think there is a little bit more tweaking that will be done, but it already looks so much better than it used to.

The California DMV has Terrible Browser Recommendations

It appears as though the California DMV has not updated their website in the past 10 years…

To be able to use this online application, your computer will need to have one of the latest versions of either the Internet Explorer, Netscape Communicator/Navigator, or equivalent web browsers. For Internet Explorer you must have version 4.0 or higher. We suggest you use Netscape Communicator/Navigator version 6.0 or higher to eliminate any possible printer problems.

California DMV Browser Requirements

Internet Explorer 4 came out in 1997. Netscape 6 came out in 2000. How far we’ve come…

In related news, the next available appointment (to take the “written” test) at the DMV in San Francisco is halfway through March. I bet I could get a new passport sooner.