Node.js – Now with 90% less Hyperbole

Many of us in the nodejs community sat back and watched as the theatrics of last weekend unfolded. We read the posts and comments, sometimes laughed or just shook our heads – so silly.

I’d like to present here our use of nodejs in a production environment.

I’ll provide more detail below, but here is a summary of why we use nodejs for our web application and services layer:

1) Full BDD environment
2) Common language, from server to browser
3) Vibrant and growing open source community
4) Mature package manager
5) Functional programming is fun

Behavior Driven Development

BDD is the next evolution of Test Driven Development. While it still has the test first orientation of TDD, the initial focus is on natural language functional tests that end users or product owners can understand. It is also referred to as outside-in testing since you start testing at the outer-most layer of the application and then work your way into unit testing and coding.

Cucumber is a popular functional testing tool for Ruby. Since a similar tool did not yet exist, my team mate Robert Malko developed peanut for nodejs. The rest of the team has made contributions to the project and we’ve already released a few versions of peanut.

There are a number of unit testing frameworks already available for nodejs. We use one called expresso.

Peanut supports generating output for our Continuous Integration server, jenkins. We have an LCD screen mounted on the wall that shows us the state of the builds for our various projects. Anytime code is pushed or merged into our git repository, it triggers a build which includes running all the functional tests. We immediately see the state of our projects in green or red.

We’ve established a git workflow that works well for us and includes running all the tests after a QA branch is merged into local master. If any tests fail, the changes will not be pushed up to the repository.

Common language

We have a number of other languages in our technology stack including Java and Cascalog. Our core web platform (fully MVC) as well as our services layer that exposes an API is nodejs. There is something very powerful and satisfying in writing model, view, controller, and API code in Javascript. If we need to do something in the mongo shell – Javascript. If we need to pass data between layers, it’s all JSON – from server right into browser-side jQuery.

Community

With over 5400 nodejs projects hosted on github there are no shortage of options for working with node in any context. Just for fun, here’s a laundry list of the packages we use:

testing:

expresso peanut should sinon soda tobi selenium

app:

async carrier cluster connect-assetmanager connect-auth dateformat express express-form eyes FastLegS hashlib jade mailer mkdirp mongodb mongoskin nib request stylus underscore validator wrench

Between Joyent, a hugely prolific commercial business (who just released their own cloud services for nodejs), Google (creator and maintainer of the underlying V8 Javascript engine) and the open source community, nodejs will continue to mature.

The bottom line for us is that we have found it to be very stable and fast in our production environment. While we don’t have millions of users, we do have some pretty intensive tasks which includes gathering millions of lines of reports and log files from the major search engines via API calls.

What about support, you say?

There are usually more than 500 people logged into the #Node.js IRC chatroom (656 as I write this at 11:00am EST on a Saturday). Many of the people logged into IRC are the major contributors to the node community. That combined with comments for nodejs package providers and forking on github provides better support than most commercial for-pay support contracts we have encountered.

Our team has forked, fixed and had pull requests accepted on a few of the nodejs projects. Likewise, our projects peanut and FastLegS have had contributions and feature requests from the community.

Node Package Manager

NPM is a tool to manage the publishing and installation of packages for nodejs. All of the packages enumerated above have been published to the npm repository.

Installing npm is an easy one-liner (assuming you have nodejs itself installed):

curl http://npmjs.org/install.sh | sh

Since it’s 1.0 release (and beyond), npm has made defining, building and deploying our applications very straightforward. We’ve published a number of packages to the npm repository and we make use of the repository for the dependent packages in our projects.

The simple yet powerful package requirements put into a package.json file make the last part of a deployment as simple as:

npm install

Functional Programming

Of course, this is completely subjective, but I find functional programming with Javascript fun and satisfying. “Javascript: The Good Parts” is required reading in order to not fall prey to the pitfalls baked into the Javascript language.

While I submit that all languages have their pitfalls, I will concede that Javascript has some gnarly ones:

> "3" == 3
true
> "3" === 3
false

We learned lessons early on about not grabbing too much in-process memory and relying on streaming as much as possible in nodejs, for instance.

Still, getting into the functional mindset and writing good code (with good test coverage written first) is very rewarding. Fibonacci examples aside, we have found nothing faster for making a lot (many thousands) of simultaneous HTTP requests to web services providers. One of the fun challenges at the outset was to throttle our requests so as not to bump up against API request quota limits that Google and many of the other data providers have.

As of this writing, nodejs is at a stable 0.4.12 release and an unstable 0.5.8 release. We have found the 0.4 releases to be very stable and we have two major applications in our production environment.