Node.js and Yahoo

I kicked off NodeSummit today with a short talk about where Yahoo uses Node, why we continue to use Node, and the practices that help us use Node effectively.


First talk of the day.

I spoke a bit about a project I’ve spent the last year on: using Node to handle data pushed out of Jenkins. Folks appreciated the honesty about Jenkins’ user interface.

Jenkins is the solution that we trust, but nobody loves using it. At Yahoo, we use Node to help us make Jenkins a bit better to use:

  • scripting job creation, editing, deletion
  • handling log messages
  • displaying information about builds with a web interface

Noted.

During the talk I mentioned something called RDL.

Nope, not the Romanian Deadlift. It’s something we call “Resource Description Language”, a machine-readable spec for web APIs. We started using it for some internal deployment APIs we worked on last month. This spec is transformed into hapi route configurations that contain detailed joi validators.

What’s great about this is that when you change the RDL, it changes the validators and comments which we present using the lout module. We do this by merging together existing hapi route objects using the RDL spec as the source of truth, so the hapi routes we have in code are validator-free and require the RDL as the source of truth. This has worked very well and our spec always matches the reality. This works a lot better than other systems that actually generate source code from RDL, which is not ideal because it cannot stay in sync as your code changes.

It’s been fun! I’ll also be on the “Scaling Business Critical Node.js Applications” and “v0.12 and Beyond” panels tomorrow, so check those out if you’re attending NodeSummit.

Eran Hammer’s Response to Netflix’s “Node.js in Flames”

Eran Hammer on Node.js web framework design tradeoffs and why there is no “best” framework:

If you haven’t read Netflix’s Node.js in Flames blog post you should. It is a great deep dive into debugging a node performance problem. The post includes useful tips that can help you solve similar problems.

That said…

My feedback from the perspective of a framework developer is quite different. I found the tone and attitude towards express.js to be concerning and somewhat offensive. Here was a developer blaming the framework he chose for poor architecture when they never bothered to actually learn how the framework works in the first place.

Recommended for understanding why Express works the way it does, what other frameworks do differently, and why none are superior to the other without considering your own requirements.

“Sorry, your coffee isn’t an artisanal ritual.”

Marco Arment writes about coffee lovers who look down upon Keurig K-Cups:

We’re the ones who keep creating, replacing, Kickstarting, and spending top dollar on ever-more-specialized equipment, even when it differs from established products only in arbitrary or purely decorative ways that have no discernable effect on the actual coffee (except maybe prolonging the process of making it).

We’re the ones who obsess over every little detail of brewing technique as if they matter much more than they really do, making good coffee ever more alienating and confusing to casual coffee drinkers who don’t have time to study and fuss over it as much as we do. […]

Maybe we’d get some of the Keurig fans to use our methods if we weren’t so pretentious, wasteful, expensive, and inaccessible ourselves. […]

Our obsession with gear and “rituals” is only distracting them — and us — from the real problem: old, mediocre, or badly roasted beans.

The Invention of the AeroPress

Zachary Crockett at Priceonomics brings us the story of Alen Alder’s company Aerobie and how a toy manufacturer started making coffee makers.

The AeroPress was conceived at Alan Adler’s dinner table. The company was having a team meal, when the wife of Aerobie’s sales manager posed a question: “What do you guys do when you just want one cup of coffee?”

A long-time coffee enthusiast and self-proclaimed “one cup kinda guy,” Adler had wondered this many times himself. He’d grown increasingly frustrated with his coffee maker, which yielded 6-8 cups per brew. In typical Adler fashion, he didn’t let the problem bother him long: he set out to invent a better way to brew single cup of coffee.

I love my AeroPress. It really does make a great cup.

Cross-Browser Testing at DeveloperWeek

The fine folks at Sauce Labs invited me to speak at DeveloperWeek San Francisco about browser testing at Yahoo scale. We test YUI on IE 6, 7, 8, 9, 10, & 11, iOS 6 & 7, Chrome, Firefox, and Safari, which runs over 120,000 tests on every push.

We’ve learned a lot about how to spend our time and the challenges of CI testing with multiple browsers. I began with a introduction to Yeti, the easy way to get started with web testing, which also drives over a million automated tests every week for YUI.

Check out the slides from my talk. If you were there, thanks for being a great audience! There was a great turnout with smart questions.

If you are interested in working on our unique frontend to CI testing with me at Yahoo, get in touch.

Infrastructure as force multiplier

Steve Klabnik considers “Is npm worth 26MM?”:

If you want to repeatably manufacture an open source ecosystem, you need capital to do so. And a firm that’s progressive enough to understand the indirect payoffs of investing in infrastructure is poised to have a huge advantage.

npm’s CEO:

Yeti 0.2.27 — First-class code coverage

Less than a week since the last release and as promised, Yeti 0.2.27 has shipped today with code coverage support.

Yeti 0.2.27 provides first-class code coverage reporting provided by Istanbul.

Simply use Yeti with the -c or --coverage option. By default, Yeti will instrument your code on-the-fly and show a brief summary.

Yeti has ran 1,207,780 tests for YUI since Yeti 0.2.26 shipped 6 days ago.

I’m very excited to bring code coverage to yo/tests.

Yeti 0.2.26 released

I’ve released a new version of Yeti, the test runner we use here on Yahoo’s YUI team. Since August 2013, Yeti has automated 33,661,505 tests in CI for us.

Today’s release prints useful feedback to stderr when Yeti is used in CI. It also includes a fix for issue #74 (Unable to serve error) and #68 (improve DOH support).

When using Yeti to in CI, e.g. to produce JUnit XML output, previous versions of Yeti would go silent after testing began as Yeti produced XML output on stdout. This made it difficult to determine if Yeti was doing anything while tests ran. When using Yeti with --junit, today’s release prints status after every test completion to stderr while XML prints to stdout.

Big thanks to @henryqdineen for contributing the fix for DOH support!

In addition to fixing bugs, we have made some improvements to Yeti’s own tests and to Yeti’s documentation. Yeti’s website now uses Pure for your viewing pleasure on desktop and mobile devices.

Expect more updates soon. Code coverage is next on my list.