Node.js Speed Up Processing Using Throat Instead of Looping

Node.js Speed up Processing of Asynchronous Requests Using Throat Instead of Looping through an array of items. I’ve found this technique to work awesomely with Node.js, i’m keen to hear your thoughts on this method.

Throttle the parallelism of an asynchronous, promise returning, function / functions. This has special utility when you set the concurrency to 1. That way you get a mutually exclusive lock.

Vanilla JavaScript Array

function returnPromiseArrayValues( resultsArray ) {

  //http://jsperf.com/jquery-each-vs-for-loop/6
  var returnObj = [];
  for ( var i = 0; i < resultsArray.length; i += 1 ) {
    returnObj.push( self.createFeedObject( resultsArray[i] ) );
  }
  return returnObj;

}

Using Throat

function returnPromiseArrayValues( resultsArray ) {

  //https://www.npmjs.org/package/throat
  return Q.all( resultsArray.map( throat( 10, function ( arrayItem ) {
    return self.createFeedObject( arrayItem );
  })));

}

You can set the number of array items to process at any one time asynchronously. I’ve got this set to 10 but it could as easily be lower, say 2 to restrict requests for easier scaling.

TODO: Add to a jsperf or such to prove it’s faster over a large dataset.