http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/async/README.md ---------------------------------------------------------------------- diff --git a/node_modules/async/README.md b/node_modules/async/README.md new file mode 100644 index 0000000..951f76e --- /dev/null +++ b/node_modules/async/README.md @@ -0,0 +1,1425 @@ +# Async.js + +Async is a utility module which provides straight-forward, powerful functions +for working with asynchronous JavaScript. Although originally designed for +use with [node.js](http://nodejs.org), it can also be used directly in the +browser. Also supports [component](https://github.com/component/component). + +Async provides around 20 functions that include the usual 'functional' +suspects (map, reduce, filter, eachâ¦) as well as some common patterns +for asynchronous control flow (parallel, series, waterfallâ¦). All these +functions assume you follow the node.js convention of providing a single +callback as the last argument of your async function. + + +## Quick Examples + +```javascript +async.map(['file1','file2','file3'], fs.stat, function(err, results){ + // results is now an array of stats for each file +}); + +async.filter(['file1','file2','file3'], fs.exists, function(results){ + // results now equals an array of the existing files +}); + +async.parallel([ + function(){ ... }, + function(){ ... } +], callback); + +async.series([ + function(){ ... }, + function(){ ... } +]); +``` + +There are many more functions available so take a look at the docs below for a +full list. This module aims to be comprehensive, so if you feel anything is +missing please create a GitHub issue for it. + +## Common Pitfalls + +### Binding a context to an iterator + +This section is really about bind, not about async. If you are wondering how to +make async execute your iterators in a given context, or are confused as to why +a method of another library isn't working as an iterator, study this example: + +```js +// Here is a simple object with an (unnecessarily roundabout) squaring method +var AsyncSquaringLibrary = { + squareExponent: 2, + square: function(number, callback){ + var result = Math.pow(number, this.squareExponent); + setTimeout(function(){ + callback(null, result); + }, 200); + } +}; + +async.map([1, 2, 3], AsyncSquaringLibrary.square, function(err, result){ + // result is [NaN, NaN, NaN] + // This fails because the `this.squareExponent` expression in the square + // function is not evaluated in the context of AsyncSquaringLibrary, and is + // therefore undefined. +}); + +async.map([1, 2, 3], AsyncSquaringLibrary.square.bind(AsyncSquaringLibrary), function(err, result){ + // result is [1, 4, 9] + // With the help of bind we can attach a context to the iterator before + // passing it to async. Now the square function will be executed in its + // 'home' AsyncSquaringLibrary context and the value of `this.squareExponent` + // will be as expected. +}); +``` + +## Download + +The source is available for download from +[GitHub](http://github.com/caolan/async). +Alternatively, you can install using Node Package Manager (npm): + + npm install async + +__Development:__ [async.js](https://github.com/caolan/async/raw/master/lib/async.js) - 29.6kb Uncompressed + +## In the Browser + +So far it's been tested in IE6, IE7, IE8, FF3.6 and Chrome 5. Usage: + +```html +<script type="text/javascript" src="async.js"></script> +<script type="text/javascript"> + + async.map(data, asyncProcess, function(err, results){ + alert(results); + }); + +</script> +``` + +## Documentation + +### Collections + +* [each](#each) +* [eachSeries](#eachSeries) +* [eachLimit](#eachLimit) +* [map](#map) +* [mapSeries](#mapSeries) +* [mapLimit](#mapLimit) +* [filter](#filter) +* [filterSeries](#filterSeries) +* [reject](#reject) +* [rejectSeries](#rejectSeries) +* [reduce](#reduce) +* [reduceRight](#reduceRight) +* [detect](#detect) +* [detectSeries](#detectSeries) +* [sortBy](#sortBy) +* [some](#some) +* [every](#every) +* [concat](#concat) +* [concatSeries](#concatSeries) + +### Control Flow + +* [series](#series) +* [parallel](#parallel) +* [parallelLimit](#parallellimittasks-limit-callback) +* [whilst](#whilst) +* [doWhilst](#doWhilst) +* [until](#until) +* [doUntil](#doUntil) +* [forever](#forever) +* [waterfall](#waterfall) +* [compose](#compose) +* [applyEach](#applyEach) +* [applyEachSeries](#applyEachSeries) +* [queue](#queue) +* [cargo](#cargo) +* [auto](#auto) +* [iterator](#iterator) +* [apply](#apply) +* [nextTick](#nextTick) +* [times](#times) +* [timesSeries](#timesSeries) + +### Utils + +* [memoize](#memoize) +* [unmemoize](#unmemoize) +* [log](#log) +* [dir](#dir) +* [noConflict](#noConflict) + + +## Collections + +<a name="forEach" /> +<a name="each" /> +### each(arr, iterator, callback) + +Applies an iterator function to each item in an array, in parallel. +The iterator is called with an item from the list and a callback for when it +has finished. If the iterator passes an error to this callback, the main +callback for the each function is immediately called with the error. + +Note, that since this function applies the iterator to each item in parallel +there is no guarantee that the iterator functions will complete in order. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err) which must be called once it has + completed. If no error has occured, the callback should be run without + arguments or with an explicit null argument. +* callback(err) - A callback which is called after all the iterator functions + have finished, or an error has occurred. + +__Example__ + +```js +// assuming openFiles is an array of file names and saveFile is a function +// to save the modified contents of that file: + +async.each(openFiles, saveFile, function(err){ + // if any of the saves produced an error, err would equal that error +}); +``` + +--------------------------------------- + +<a name="forEachSeries" /> +<a name="eachSeries" /> +### eachSeries(arr, iterator, callback) + +The same as each only the iterator is applied to each item in the array in +series. The next iterator is only called once the current one has completed +processing. This means the iterator functions will complete in order. + + +--------------------------------------- + +<a name="forEachLimit" /> +<a name="eachLimit" /> +### eachLimit(arr, limit, iterator, callback) + +The same as each only no more than "limit" iterators will be simultaneously +running at any time. + +Note that the items are not processed in batches, so there is no guarantee that + the first "limit" iterator functions will complete before any others are +started. + +__Arguments__ + +* arr - An array to iterate over. +* limit - The maximum number of iterators to run at any time. +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err) which must be called once it has + completed. If no error has occured, the callback should be run without + arguments or with an explicit null argument. +* callback(err) - A callback which is called after all the iterator functions + have finished, or an error has occurred. + +__Example__ + +```js +// Assume documents is an array of JSON objects and requestApi is a +// function that interacts with a rate-limited REST api. + +async.eachLimit(documents, 20, requestApi, function(err){ + // if any of the saves produced an error, err would equal that error +}); +``` + +--------------------------------------- + +<a name="map" /> +### map(arr, iterator, callback) + +Produces a new array of values by mapping each value in the given array through +the iterator function. The iterator is called with an item from the array and a +callback for when it has finished processing. The callback takes 2 arguments, +an error and the transformed item from the array. If the iterator passes an +error to this callback, the main callback for the map function is immediately +called with the error. + +Note, that since this function applies the iterator to each item in parallel +there is no guarantee that the iterator functions will complete in order, however +the results array will be in the same order as the original array. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err, transformed) which must be called once + it has completed with an error (which can be null) and a transformed item. +* callback(err, results) - A callback which is called after all the iterator + functions have finished, or an error has occurred. Results is an array of the + transformed items from the original array. + +__Example__ + +```js +async.map(['file1','file2','file3'], fs.stat, function(err, results){ + // results is now an array of stats for each file +}); +``` + +--------------------------------------- + +<a name="mapSeries" /> +### mapSeries(arr, iterator, callback) + +The same as map only the iterator is applied to each item in the array in +series. The next iterator is only called once the current one has completed +processing. The results array will be in the same order as the original. + + +--------------------------------------- + +<a name="mapLimit" /> +### mapLimit(arr, limit, iterator, callback) + +The same as map only no more than "limit" iterators will be simultaneously +running at any time. + +Note that the items are not processed in batches, so there is no guarantee that + the first "limit" iterator functions will complete before any others are +started. + +__Arguments__ + +* arr - An array to iterate over. +* limit - The maximum number of iterators to run at any time. +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err, transformed) which must be called once + it has completed with an error (which can be null) and a transformed item. +* callback(err, results) - A callback which is called after all the iterator + functions have finished, or an error has occurred. Results is an array of the + transformed items from the original array. + +__Example__ + +```js +async.mapLimit(['file1','file2','file3'], 1, fs.stat, function(err, results){ + // results is now an array of stats for each file +}); +``` + +--------------------------------------- + +<a name="filter" /> +### filter(arr, iterator, callback) + +__Alias:__ select + +Returns a new array of all the values which pass an async truth test. +_The callback for each iterator call only accepts a single argument of true or +false, it does not accept an error argument first!_ This is in-line with the +way node libraries work with truth tests like fs.exists. This operation is +performed in parallel, but the results array will be in the same order as the +original. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A truth test to apply to each item in the array. + The iterator is passed a callback(truthValue) which must be called with a + boolean argument once it has completed. +* callback(results) - A callback which is called after all the iterator + functions have finished. + +__Example__ + +```js +async.filter(['file1','file2','file3'], fs.exists, function(results){ + // results now equals an array of the existing files +}); +``` + +--------------------------------------- + +<a name="filterSeries" /> +### filterSeries(arr, iterator, callback) + +__alias:__ selectSeries + +The same as filter only the iterator is applied to each item in the array in +series. The next iterator is only called once the current one has completed +processing. The results array will be in the same order as the original. + +--------------------------------------- + +<a name="reject" /> +### reject(arr, iterator, callback) + +The opposite of filter. Removes values that pass an async truth test. + +--------------------------------------- + +<a name="rejectSeries" /> +### rejectSeries(arr, iterator, callback) + +The same as reject, only the iterator is applied to each item in the array +in series. + + +--------------------------------------- + +<a name="reduce" /> +### reduce(arr, memo, iterator, callback) + +__aliases:__ inject, foldl + +Reduces a list of values into a single value using an async iterator to return +each successive step. Memo is the initial state of the reduction. This +function only operates in series. For performance reasons, it may make sense to +split a call to this function into a parallel map, then use the normal +Array.prototype.reduce on the results. This function is for situations where +each step in the reduction needs to be async, if you can get the data before +reducing it then it's probably a good idea to do so. + +__Arguments__ + +* arr - An array to iterate over. +* memo - The initial state of the reduction. +* iterator(memo, item, callback) - A function applied to each item in the + array to produce the next step in the reduction. The iterator is passed a + callback(err, reduction) which accepts an optional error as its first + argument, and the state of the reduction as the second. If an error is + passed to the callback, the reduction is stopped and the main callback is + immediately called with the error. +* callback(err, result) - A callback which is called after all the iterator + functions have finished. Result is the reduced value. + +__Example__ + +```js +async.reduce([1,2,3], 0, function(memo, item, callback){ + // pointless async: + process.nextTick(function(){ + callback(null, memo + item) + }); +}, function(err, result){ + // result is now equal to the last value of memo, which is 6 +}); +``` + +--------------------------------------- + +<a name="reduceRight" /> +### reduceRight(arr, memo, iterator, callback) + +__Alias:__ foldr + +Same as reduce, only operates on the items in the array in reverse order. + + +--------------------------------------- + +<a name="detect" /> +### detect(arr, iterator, callback) + +Returns the first value in a list that passes an async truth test. The +iterator is applied in parallel, meaning the first iterator to return true will +fire the detect callback with that result. That means the result might not be +the first item in the original array (in terms of order) that passes the test. + +If order within the original array is important then look at detectSeries. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A truth test to apply to each item in the array. + The iterator is passed a callback(truthValue) which must be called with a + boolean argument once it has completed. +* callback(result) - A callback which is called as soon as any iterator returns + true, or after all the iterator functions have finished. Result will be + the first item in the array that passes the truth test (iterator) or the + value undefined if none passed. + +__Example__ + +```js +async.detect(['file1','file2','file3'], fs.exists, function(result){ + // result now equals the first file in the list that exists +}); +``` + +--------------------------------------- + +<a name="detectSeries" /> +### detectSeries(arr, iterator, callback) + +The same as detect, only the iterator is applied to each item in the array +in series. This means the result is always the first in the original array (in +terms of array order) that passes the truth test. + + +--------------------------------------- + +<a name="sortBy" /> +### sortBy(arr, iterator, callback) + +Sorts a list by the results of running each value through an async iterator. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err, sortValue) which must be called once it + has completed with an error (which can be null) and a value to use as the sort + criteria. +* callback(err, results) - A callback which is called after all the iterator + functions have finished, or an error has occurred. Results is the items from + the original array sorted by the values returned by the iterator calls. + +__Example__ + +```js +async.sortBy(['file1','file2','file3'], function(file, callback){ + fs.stat(file, function(err, stats){ + callback(err, stats.mtime); + }); +}, function(err, results){ + // results is now the original array of files sorted by + // modified date +}); +``` + +--------------------------------------- + +<a name="some" /> +### some(arr, iterator, callback) + +__Alias:__ any + +Returns true if at least one element in the array satisfies an async test. +_The callback for each iterator call only accepts a single argument of true or +false, it does not accept an error argument first!_ This is in-line with the +way node libraries work with truth tests like fs.exists. Once any iterator +call returns true, the main callback is immediately called. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A truth test to apply to each item in the array. + The iterator is passed a callback(truthValue) which must be called with a + boolean argument once it has completed. +* callback(result) - A callback which is called as soon as any iterator returns + true, or after all the iterator functions have finished. Result will be + either true or false depending on the values of the async tests. + +__Example__ + +```js +async.some(['file1','file2','file3'], fs.exists, function(result){ + // if result is true then at least one of the files exists +}); +``` + +--------------------------------------- + +<a name="every" /> +### every(arr, iterator, callback) + +__Alias:__ all + +Returns true if every element in the array satisfies an async test. +_The callback for each iterator call only accepts a single argument of true or +false, it does not accept an error argument first!_ This is in-line with the +way node libraries work with truth tests like fs.exists. + +__Arguments__ + +* arr - An array to iterate over. +* iterator(item, callback) - A truth test to apply to each item in the array. + The iterator is passed a callback(truthValue) which must be called with a + boolean argument once it has completed. +* callback(result) - A callback which is called after all the iterator + functions have finished. Result will be either true or false depending on + the values of the async tests. + +__Example__ + +```js +async.every(['file1','file2','file3'], fs.exists, function(result){ + // if result is true then every file exists +}); +``` + +--------------------------------------- + +<a name="concat" /> +### concat(arr, iterator, callback) + +Applies an iterator to each item in a list, concatenating the results. Returns the +concatenated list. The iterators are called in parallel, and the results are +concatenated as they return. There is no guarantee that the results array will +be returned in the original order of the arguments passed to the iterator function. + +__Arguments__ + +* arr - An array to iterate over +* iterator(item, callback) - A function to apply to each item in the array. + The iterator is passed a callback(err, results) which must be called once it + has completed with an error (which can be null) and an array of results. +* callback(err, results) - A callback which is called after all the iterator + functions have finished, or an error has occurred. Results is an array containing + the concatenated results of the iterator function. + +__Example__ + +```js +async.concat(['dir1','dir2','dir3'], fs.readdir, function(err, files){ + // files is now a list of filenames that exist in the 3 directories +}); +``` + +--------------------------------------- + +<a name="concatSeries" /> +### concatSeries(arr, iterator, callback) + +Same as async.concat, but executes in series instead of parallel. + + +## Control Flow + +<a name="series" /> +### series(tasks, [callback]) + +Run an array of functions in series, each one running once the previous +function has completed. If any functions in the series pass an error to its +callback, no more functions are run and the callback for the series is +immediately called with the value of the error. Once the tasks have completed, +the results are passed to the final callback as an array. + +It is also possible to use an object instead of an array. Each property will be +run as a function and the results will be passed to the final callback as an object +instead of an array. This can be a more readable way of handling results from +async.series. + + +__Arguments__ + +* tasks - An array or object containing functions to run, each function is passed + a callback(err, result) it must call on completion with an error (which can + be null) and an optional result value. +* callback(err, results) - An optional callback to run once all the functions + have completed. This function gets a results array (or object) containing all + the result arguments passed to the task callbacks. + +__Example__ + +```js +async.series([ + function(callback){ + // do some stuff ... + callback(null, 'one'); + }, + function(callback){ + // do some more stuff ... + callback(null, 'two'); + } +], +// optional callback +function(err, results){ + // results is now equal to ['one', 'two'] +}); + + +// an example using an object instead of an array +async.series({ + one: function(callback){ + setTimeout(function(){ + callback(null, 1); + }, 200); + }, + two: function(callback){ + setTimeout(function(){ + callback(null, 2); + }, 100); + } +}, +function(err, results) { + // results is now equal to: {one: 1, two: 2} +}); +``` + +--------------------------------------- + +<a name="parallel" /> +### parallel(tasks, [callback]) + +Run an array of functions in parallel, without waiting until the previous +function has completed. If any of the functions pass an error to its +callback, the main callback is immediately called with the value of the error. +Once the tasks have completed, the results are passed to the final callback as an +array. + +It is also possible to use an object instead of an array. Each property will be +run as a function and the results will be passed to the final callback as an object +instead of an array. This can be a more readable way of handling results from +async.parallel. + + +__Arguments__ + +* tasks - An array or object containing functions to run, each function is passed + a callback(err, result) it must call on completion with an error (which can + be null) and an optional result value. +* callback(err, results) - An optional callback to run once all the functions + have completed. This function gets a results array (or object) containing all + the result arguments passed to the task callbacks. + +__Example__ + +```js +async.parallel([ + function(callback){ + setTimeout(function(){ + callback(null, 'one'); + }, 200); + }, + function(callback){ + setTimeout(function(){ + callback(null, 'two'); + }, 100); + } +], +// optional callback +function(err, results){ + // the results array will equal ['one','two'] even though + // the second function had a shorter timeout. +}); + + +// an example using an object instead of an array +async.parallel({ + one: function(callback){ + setTimeout(function(){ + callback(null, 1); + }, 200); + }, + two: function(callback){ + setTimeout(function(){ + callback(null, 2); + }, 100); + } +}, +function(err, results) { + // results is now equals to: {one: 1, two: 2} +}); +``` + +--------------------------------------- + +<a name="parallel" /> +### parallelLimit(tasks, limit, [callback]) + +The same as parallel only the tasks are executed in parallel with a maximum of "limit" +tasks executing at any time. + +Note that the tasks are not executed in batches, so there is no guarantee that +the first "limit" tasks will complete before any others are started. + +__Arguments__ + +* tasks - An array or object containing functions to run, each function is passed + a callback(err, result) it must call on completion with an error (which can + be null) and an optional result value. +* limit - The maximum number of tasks to run at any time. +* callback(err, results) - An optional callback to run once all the functions + have completed. This function gets a results array (or object) containing all + the result arguments passed to the task callbacks. + +--------------------------------------- + +<a name="whilst" /> +### whilst(test, fn, callback) + +Repeatedly call fn, while test returns true. Calls the callback when stopped, +or an error occurs. + +__Arguments__ + +* test() - synchronous truth test to perform before each execution of fn. +* fn(callback) - A function to call each time the test passes. The function is + passed a callback(err) which must be called once it has completed with an + optional error argument. +* callback(err) - A callback which is called after the test fails and repeated + execution of fn has stopped. + +__Example__ + +```js +var count = 0; + +async.whilst( + function () { return count < 5; }, + function (callback) { + count++; + setTimeout(callback, 1000); + }, + function (err) { + // 5 seconds have passed + } +); +``` + +--------------------------------------- + +<a name="doWhilst" /> +### doWhilst(fn, test, callback) + +The post check version of whilst. To reflect the difference in the order of operations `test` and `fn` arguments are switched. `doWhilst` is to `whilst` as `do while` is to `while` in plain JavaScript. + +--------------------------------------- + +<a name="until" /> +### until(test, fn, callback) + +Repeatedly call fn, until test returns true. Calls the callback when stopped, +or an error occurs. + +The inverse of async.whilst. + +--------------------------------------- + +<a name="doUntil" /> +### doUntil(fn, test, callback) + +Like doWhilst except the test is inverted. Note the argument ordering differs from `until`. + +--------------------------------------- + +<a name="forever" /> +### forever(fn, callback) + +Calls the asynchronous function 'fn' repeatedly, in series, indefinitely. +If an error is passed to fn's callback then 'callback' is called with the +error, otherwise it will never be called. + +--------------------------------------- + +<a name="waterfall" /> +### waterfall(tasks, [callback]) + +Runs an array of functions in series, each passing their results to the next in +the array. However, if any of the functions pass an error to the callback, the +next function is not executed and the main callback is immediately called with +the error. + +__Arguments__ + +* tasks - An array of functions to run, each function is passed a + callback(err, result1, result2, ...) it must call on completion. The first + argument is an error (which can be null) and any further arguments will be + passed as arguments in order to the next task. +* callback(err, [results]) - An optional callback to run once all the functions + have completed. This will be passed the results of the last task's callback. + + + +__Example__ + +```js +async.waterfall([ + function(callback){ + callback(null, 'one', 'two'); + }, + function(arg1, arg2, callback){ + callback(null, 'three'); + }, + function(arg1, callback){ + // arg1 now equals 'three' + callback(null, 'done'); + } +], function (err, result) { + // result now equals 'done' +}); +``` + +--------------------------------------- +<a name="compose" /> +### compose(fn1, fn2...) + +Creates a function which is a composition of the passed asynchronous +functions. Each function consumes the return value of the function that +follows. Composing functions f(), g() and h() would produce the result of +f(g(h())), only this version uses callbacks to obtain the return values. + +Each function is executed with the `this` binding of the composed function. + +__Arguments__ + +* functions... - the asynchronous functions to compose + + +__Example__ + +```js +function add1(n, callback) { + setTimeout(function () { + callback(null, n + 1); + }, 10); +} + +function mul3(n, callback) { + setTimeout(function () { + callback(null, n * 3); + }, 10); +} + +var add1mul3 = async.compose(mul3, add1); + +add1mul3(4, function (err, result) { + // result now equals 15 +}); +``` + +--------------------------------------- +<a name="applyEach" /> +### applyEach(fns, args..., callback) + +Applies the provided arguments to each function in the array, calling the +callback after all functions have completed. If you only provide the first +argument then it will return a function which lets you pass in the +arguments as if it were a single function call. + +__Arguments__ + +* fns - the asynchronous functions to all call with the same arguments +* args... - any number of separate arguments to pass to the function +* callback - the final argument should be the callback, called when all + functions have completed processing + + +__Example__ + +```js +async.applyEach([enableSearch, updateSchema], 'bucket', callback); + +// partial application example: +async.each( + buckets, + async.applyEach([enableSearch, updateSchema]), + callback +); +``` + +--------------------------------------- + +<a name="applyEachSeries" /> +### applyEachSeries(arr, iterator, callback) + +The same as applyEach only the functions are applied in series. + +--------------------------------------- + +<a name="queue" /> +### queue(worker, concurrency) + +Creates a queue object with the specified concurrency. Tasks added to the +queue will be processed in parallel (up to the concurrency limit). If all +workers are in progress, the task is queued until one is available. Once +a worker has completed a task, the task's callback is called. + +__Arguments__ + +* worker(task, callback) - An asynchronous function for processing a queued + task, which must call its callback(err) argument when finished, with an + optional error as an argument. +* concurrency - An integer for determining how many worker functions should be + run in parallel. + +__Queue objects__ + +The queue object returned by this function has the following properties and +methods: + +* length() - a function returning the number of items waiting to be processed. +* concurrency - an integer for determining how many worker functions should be + run in parallel. This property can be changed after a queue is created to + alter the concurrency on-the-fly. +* push(task, [callback]) - add a new task to the queue, the callback is called + once the worker has finished processing the task. + instead of a single task, an array of tasks can be submitted. the respective callback is used for every task in the list. +* unshift(task, [callback]) - add a new task to the front of the queue. +* saturated - a callback that is called when the queue length hits the concurrency and further tasks will be queued +* empty - a callback that is called when the last item from the queue is given to a worker +* drain - a callback that is called when the last item from the queue has returned from the worker + +__Example__ + +```js +// create a queue object with concurrency 2 + +var q = async.queue(function (task, callback) { + console.log('hello ' + task.name); + callback(); +}, 2); + + +// assign a callback +q.drain = function() { + console.log('all items have been processed'); +} + +// add some items to the queue + +q.push({name: 'foo'}, function (err) { + console.log('finished processing foo'); +}); +q.push({name: 'bar'}, function (err) { + console.log('finished processing bar'); +}); + +// add some items to the queue (batch-wise) + +q.push([{name: 'baz'},{name: 'bay'},{name: 'bax'}], function (err) { + console.log('finished processing bar'); +}); + +// add some items to the front of the queue + +q.unshift({name: 'bar'}, function (err) { + console.log('finished processing bar'); +}); +``` + +--------------------------------------- + +<a name="cargo" /> +### cargo(worker, [payload]) + +Creates a cargo object with the specified payload. Tasks added to the +cargo will be processed altogether (up to the payload limit). If the +worker is in progress, the task is queued until it is available. Once +the worker has completed some tasks, each callback of those tasks is called. + +__Arguments__ + +* worker(tasks, callback) - An asynchronous function for processing an array of + queued tasks, which must call its callback(err) argument when finished, with + an optional error as an argument. +* payload - An optional integer for determining how many tasks should be + processed per round; if omitted, the default is unlimited. + +__Cargo objects__ + +The cargo object returned by this function has the following properties and +methods: + +* length() - a function returning the number of items waiting to be processed. +* payload - an integer for determining how many tasks should be + process per round. This property can be changed after a cargo is created to + alter the payload on-the-fly. +* push(task, [callback]) - add a new task to the queue, the callback is called + once the worker has finished processing the task. + instead of a single task, an array of tasks can be submitted. the respective callback is used for every task in the list. +* saturated - a callback that is called when the queue length hits the concurrency and further tasks will be queued +* empty - a callback that is called when the last item from the queue is given to a worker +* drain - a callback that is called when the last item from the queue has returned from the worker + +__Example__ + +```js +// create a cargo object with payload 2 + +var cargo = async.cargo(function (tasks, callback) { + for(var i=0; i<tasks.length; i++){ + console.log('hello ' + tasks[i].name); + } + callback(); +}, 2); + + +// add some items + +cargo.push({name: 'foo'}, function (err) { + console.log('finished processing foo'); +}); +cargo.push({name: 'bar'}, function (err) { + console.log('finished processing bar'); +}); +cargo.push({name: 'baz'}, function (err) { + console.log('finished processing baz'); +}); +``` + +--------------------------------------- + +<a name="auto" /> +### auto(tasks, [callback]) + +Determines the best order for running functions based on their requirements. +Each function can optionally depend on other functions being completed first, +and each function is run as soon as its requirements are satisfied. If any of +the functions pass an error to their callback, that function will not complete +(so any other functions depending on it will not run) and the main callback +will be called immediately with the error. Functions also receive an object +containing the results of functions which have completed so far. + +Note, all functions are called with a results object as a second argument, +so it is unsafe to pass functions in the tasks object which cannot handle the +extra argument. For example, this snippet of code: + +```js +async.auto({ + readData: async.apply(fs.readFile, 'data.txt', 'utf-8') +}, callback); +``` + +will have the effect of calling readFile with the results object as the last +argument, which will fail: + +```js +fs.readFile('data.txt', 'utf-8', cb, {}); +``` + +Instead, wrap the call to readFile in a function which does not forward the +results object: + +```js +async.auto({ + readData: function(cb, results){ + fs.readFile('data.txt', 'utf-8', cb); + } +}, callback); +``` + +__Arguments__ + +* tasks - An object literal containing named functions or an array of + requirements, with the function itself the last item in the array. The key + used for each function or array is used when specifying requirements. The + function receives two arguments: (1) a callback(err, result) which must be + called when finished, passing an error (which can be null) and the result of + the function's execution, and (2) a results object, containing the results of + the previously executed functions. +* callback(err, results) - An optional callback which is called when all the + tasks have been completed. The callback will receive an error as an argument + if any tasks pass an error to their callback. Results will always be passed + but if an error occurred, no other tasks will be performed, and the results + object will only contain partial results. + + +__Example__ + +```js +async.auto({ + get_data: function(callback){ + // async code to get some data + }, + make_folder: function(callback){ + // async code to create a directory to store a file in + // this is run at the same time as getting the data + }, + write_file: ['get_data', 'make_folder', function(callback){ + // once there is some data and the directory exists, + // write the data to a file in the directory + callback(null, filename); + }], + email_link: ['write_file', function(callback, results){ + // once the file is written let's email a link to it... + // results.write_file contains the filename returned by write_file. + }] +}); +``` + +This is a fairly trivial example, but to do this using the basic parallel and +series functions would look like this: + +```js +async.parallel([ + function(callback){ + // async code to get some data + }, + function(callback){ + // async code to create a directory to store a file in + // this is run at the same time as getting the data + } +], +function(err, results){ + async.series([ + function(callback){ + // once there is some data and the directory exists, + // write the data to a file in the directory + }, + function(callback){ + // once the file is written let's email a link to it... + } + ]); +}); +``` + +For a complicated series of async tasks using the auto function makes adding +new tasks much easier and makes the code more readable. + + +--------------------------------------- + +<a name="iterator" /> +### iterator(tasks) + +Creates an iterator function which calls the next function in the array, +returning a continuation to call the next one after that. It's also possible to +'peek' the next iterator by doing iterator.next(). + +This function is used internally by the async module but can be useful when +you want to manually control the flow of functions in series. + +__Arguments__ + +* tasks - An array of functions to run. + +__Example__ + +```js +var iterator = async.iterator([ + function(){ sys.p('one'); }, + function(){ sys.p('two'); }, + function(){ sys.p('three'); } +]); + +node> var iterator2 = iterator(); +'one' +node> var iterator3 = iterator2(); +'two' +node> iterator3(); +'three' +node> var nextfn = iterator2.next(); +node> nextfn(); +'three' +``` + +--------------------------------------- + +<a name="apply" /> +### apply(function, arguments..) + +Creates a continuation function with some arguments already applied, a useful +shorthand when combined with other control flow functions. Any arguments +passed to the returned function are added to the arguments originally passed +to apply. + +__Arguments__ + +* function - The function you want to eventually apply all arguments to. +* arguments... - Any number of arguments to automatically apply when the + continuation is called. + +__Example__ + +```js +// using apply + +async.parallel([ + async.apply(fs.writeFile, 'testfile1', 'test1'), + async.apply(fs.writeFile, 'testfile2', 'test2'), +]); + + +// the same process without using apply + +async.parallel([ + function(callback){ + fs.writeFile('testfile1', 'test1', callback); + }, + function(callback){ + fs.writeFile('testfile2', 'test2', callback); + } +]); +``` + +It's possible to pass any number of additional arguments when calling the +continuation: + +```js +node> var fn = async.apply(sys.puts, 'one'); +node> fn('two', 'three'); +one +two +three +``` + +--------------------------------------- + +<a name="nextTick" /> +### nextTick(callback) + +Calls the callback on a later loop around the event loop. In node.js this just +calls process.nextTick, in the browser it falls back to setImmediate(callback) +if available, otherwise setTimeout(callback, 0), which means other higher priority +events may precede the execution of the callback. + +This is used internally for browser-compatibility purposes. + +__Arguments__ + +* callback - The function to call on a later loop around the event loop. + +__Example__ + +```js +var call_order = []; +async.nextTick(function(){ + call_order.push('two'); + // call_order now equals ['one','two'] +}); +call_order.push('one') +``` + +<a name="times" /> +### times(n, callback) + +Calls the callback n times and accumulates results in the same manner +you would use with async.map. + +__Arguments__ + +* n - The number of times to run the function. +* callback - The function to call n times. + +__Example__ + +```js +// Pretend this is some complicated async factory +var createUser = function(id, callback) { + callback(null, { + id: 'user' + id + }) +} +// generate 5 users +async.times(5, function(n, next){ + createUser(n, function(err, user) { + next(err, user) + }) +}, function(err, users) { + // we should now have 5 users +}); +``` + +<a name="timesSeries" /> +### timesSeries(n, callback) + +The same as times only the iterator is applied to each item in the array in +series. The next iterator is only called once the current one has completed +processing. The results array will be in the same order as the original. + + +## Utils + +<a name="memoize" /> +### memoize(fn, [hasher]) + +Caches the results of an async function. When creating a hash to store function +results against, the callback is omitted from the hash and an optional hash +function can be used. + +The cache of results is exposed as the `memo` property of the function returned +by `memoize`. + +__Arguments__ + +* fn - the function you to proxy and cache results from. +* hasher - an optional function for generating a custom hash for storing + results, it has all the arguments applied to it apart from the callback, and + must be synchronous. + +__Example__ + +```js +var slow_fn = function (name, callback) { + // do something + callback(null, result); +}; +var fn = async.memoize(slow_fn); + +// fn can now be used as if it were slow_fn +fn('some name', function () { + // callback +}); +``` + +<a name="unmemoize" /> +### unmemoize(fn) + +Undoes a memoized function, reverting it to the original, unmemoized +form. Comes handy in tests. + +__Arguments__ + +* fn - the memoized function + +<a name="log" /> +### log(function, arguments) + +Logs the result of an async function to the console. Only works in node.js or +in browsers that support console.log and console.error (such as FF and Chrome). +If multiple arguments are returned from the async function, console.log is +called on each argument in order. + +__Arguments__ + +* function - The function you want to eventually apply all arguments to. +* arguments... - Any number of arguments to apply to the function. + +__Example__ + +```js +var hello = function(name, callback){ + setTimeout(function(){ + callback(null, 'hello ' + name); + }, 1000); +}; +``` +```js +node> async.log(hello, 'world'); +'hello world' +``` + +--------------------------------------- + +<a name="dir" /> +### dir(function, arguments) + +Logs the result of an async function to the console using console.dir to +display the properties of the resulting object. Only works in node.js or +in browsers that support console.dir and console.error (such as FF and Chrome). +If multiple arguments are returned from the async function, console.dir is +called on each argument in order. + +__Arguments__ + +* function - The function you want to eventually apply all arguments to. +* arguments... - Any number of arguments to apply to the function. + +__Example__ + +```js +var hello = function(name, callback){ + setTimeout(function(){ + callback(null, {hello: name}); + }, 1000); +}; +``` +```js +node> async.dir(hello, 'world'); +{hello: 'world'} +``` + +--------------------------------------- + +<a name="noConflict" /> +### noConflict() + +Changes the value of async back to its original value, returning a reference to the +async object.
http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/async/component.json ---------------------------------------------------------------------- diff --git a/node_modules/async/component.json b/node_modules/async/component.json new file mode 100644 index 0000000..bbb0115 --- /dev/null +++ b/node_modules/async/component.json @@ -0,0 +1,11 @@ +{ + "name": "async", + "repo": "caolan/async", + "description": "Higher-order functions and common patterns for asynchronous code", + "version": "0.1.23", + "keywords": [], + "dependencies": {}, + "development": {}, + "main": "lib/async.js", + "scripts": [ "lib/async.js" ] +} http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/async/lib/async.js ---------------------------------------------------------------------- diff --git a/node_modules/async/lib/async.js b/node_modules/async/lib/async.js new file mode 100755 index 0000000..1eebb15 --- /dev/null +++ b/node_modules/async/lib/async.js @@ -0,0 +1,958 @@ +/*global setImmediate: false, setTimeout: false, console: false */ +(function () { + + var async = {}; + + // global on the server, window in the browser + var root, previous_async; + + root = this; + if (root != null) { + previous_async = root.async; + } + + async.noConflict = function () { + root.async = previous_async; + return async; + }; + + function only_once(fn) { + var called = false; + return function() { + if (called) throw new Error("Callback was already called."); + called = true; + fn.apply(root, arguments); + } + } + + //// cross-browser compatiblity functions //// + + var _each = function (arr, iterator) { + if (arr.forEach) { + return arr.forEach(iterator); + } + for (var i = 0; i < arr.length; i += 1) { + iterator(arr[i], i, arr); + } + }; + + var _map = function (arr, iterator) { + if (arr.map) { + return arr.map(iterator); + } + var results = []; + _each(arr, function (x, i, a) { + results.push(iterator(x, i, a)); + }); + return results; + }; + + var _reduce = function (arr, iterator, memo) { + if (arr.reduce) { + return arr.reduce(iterator, memo); + } + _each(arr, function (x, i, a) { + memo = iterator(memo, x, i, a); + }); + return memo; + }; + + var _keys = function (obj) { + if (Object.keys) { + return Object.keys(obj); + } + var keys = []; + for (var k in obj) { + if (obj.hasOwnProperty(k)) { + keys.push(k); + } + } + return keys; + }; + + //// exported async module functions //// + + //// nextTick implementation with browser-compatible fallback //// + if (typeof process === 'undefined' || !(process.nextTick)) { + if (typeof setImmediate === 'function') { + async.nextTick = function (fn) { + // not a direct alias for IE10 compatibility + setImmediate(fn); + }; + async.setImmediate = async.nextTick; + } + else { + async.nextTick = function (fn) { + setTimeout(fn, 0); + }; + async.setImmediate = async.nextTick; + } + } + else { + async.nextTick = process.nextTick; + if (typeof setImmediate !== 'undefined') { + async.setImmediate = function (fn) { + // not a direct alias for IE10 compatibility + setImmediate(fn); + }; + } + else { + async.setImmediate = async.nextTick; + } + } + + async.each = function (arr, iterator, callback) { + callback = callback || function () {}; + if (!arr.length) { + return callback(); + } + var completed = 0; + _each(arr, function (x) { + iterator(x, only_once(function (err) { + if (err) { + callback(err); + callback = function () {}; + } + else { + completed += 1; + if (completed >= arr.length) { + callback(null); + } + } + })); + }); + }; + async.forEach = async.each; + + async.eachSeries = function (arr, iterator, callback) { + callback = callback || function () {}; + if (!arr.length) { + return callback(); + } + var completed = 0; + var iterate = function () { + iterator(arr[completed], function (err) { + if (err) { + callback(err); + callback = function () {}; + } + else { + completed += 1; + if (completed >= arr.length) { + callback(null); + } + else { + iterate(); + } + } + }); + }; + iterate(); + }; + async.forEachSeries = async.eachSeries; + + async.eachLimit = function (arr, limit, iterator, callback) { + var fn = _eachLimit(limit); + fn.apply(null, [arr, iterator, callback]); + }; + async.forEachLimit = async.eachLimit; + + var _eachLimit = function (limit) { + + return function (arr, iterator, callback) { + callback = callback || function () {}; + if (!arr.length || limit <= 0) { + return callback(); + } + var completed = 0; + var started = 0; + var running = 0; + + (function replenish () { + if (completed >= arr.length) { + return callback(); + } + + while (running < limit && started < arr.length) { + started += 1; + running += 1; + iterator(arr[started - 1], function (err) { + if (err) { + callback(err); + callback = function () {}; + } + else { + completed += 1; + running -= 1; + if (completed >= arr.length) { + callback(); + } + else { + replenish(); + } + } + }); + } + })(); + }; + }; + + + var doParallel = function (fn) { + return function () { + var args = Array.prototype.slice.call(arguments); + return fn.apply(null, [async.each].concat(args)); + }; + }; + var doParallelLimit = function(limit, fn) { + return function () { + var args = Array.prototype.slice.call(arguments); + return fn.apply(null, [_eachLimit(limit)].concat(args)); + }; + }; + var doSeries = function (fn) { + return function () { + var args = Array.prototype.slice.call(arguments); + return fn.apply(null, [async.eachSeries].concat(args)); + }; + }; + + + var _asyncMap = function (eachfn, arr, iterator, callback) { + var results = []; + arr = _map(arr, function (x, i) { + return {index: i, value: x}; + }); + eachfn(arr, function (x, callback) { + iterator(x.value, function (err, v) { + results[x.index] = v; + callback(err); + }); + }, function (err) { + callback(err, results); + }); + }; + async.map = doParallel(_asyncMap); + async.mapSeries = doSeries(_asyncMap); + async.mapLimit = function (arr, limit, iterator, callback) { + return _mapLimit(limit)(arr, iterator, callback); + }; + + var _mapLimit = function(limit) { + return doParallelLimit(limit, _asyncMap); + }; + + // reduce only has a series version, as doing reduce in parallel won't + // work in many situations. + async.reduce = function (arr, memo, iterator, callback) { + async.eachSeries(arr, function (x, callback) { + iterator(memo, x, function (err, v) { + memo = v; + callback(err); + }); + }, function (err) { + callback(err, memo); + }); + }; + // inject alias + async.inject = async.reduce; + // foldl alias + async.foldl = async.reduce; + + async.reduceRight = function (arr, memo, iterator, callback) { + var reversed = _map(arr, function (x) { + return x; + }).reverse(); + async.reduce(reversed, memo, iterator, callback); + }; + // foldr alias + async.foldr = async.reduceRight; + + var _filter = function (eachfn, arr, iterator, callback) { + var results = []; + arr = _map(arr, function (x, i) { + return {index: i, value: x}; + }); + eachfn(arr, function (x, callback) { + iterator(x.value, function (v) { + if (v) { + results.push(x); + } + callback(); + }); + }, function (err) { + callback(_map(results.sort(function (a, b) { + return a.index - b.index; + }), function (x) { + return x.value; + })); + }); + }; + async.filter = doParallel(_filter); + async.filterSeries = doSeries(_filter); + // select alias + async.select = async.filter; + async.selectSeries = async.filterSeries; + + var _reject = function (eachfn, arr, iterator, callback) { + var results = []; + arr = _map(arr, function (x, i) { + return {index: i, value: x}; + }); + eachfn(arr, function (x, callback) { + iterator(x.value, function (v) { + if (!v) { + results.push(x); + } + callback(); + }); + }, function (err) { + callback(_map(results.sort(function (a, b) { + return a.index - b.index; + }), function (x) { + return x.value; + })); + }); + }; + async.reject = doParallel(_reject); + async.rejectSeries = doSeries(_reject); + + var _detect = function (eachfn, arr, iterator, main_callback) { + eachfn(arr, function (x, callback) { + iterator(x, function (result) { + if (result) { + main_callback(x); + main_callback = function () {}; + } + else { + callback(); + } + }); + }, function (err) { + main_callback(); + }); + }; + async.detect = doParallel(_detect); + async.detectSeries = doSeries(_detect); + + async.some = function (arr, iterator, main_callback) { + async.each(arr, function (x, callback) { + iterator(x, function (v) { + if (v) { + main_callback(true); + main_callback = function () {}; + } + callback(); + }); + }, function (err) { + main_callback(false); + }); + }; + // any alias + async.any = async.some; + + async.every = function (arr, iterator, main_callback) { + async.each(arr, function (x, callback) { + iterator(x, function (v) { + if (!v) { + main_callback(false); + main_callback = function () {}; + } + callback(); + }); + }, function (err) { + main_callback(true); + }); + }; + // all alias + async.all = async.every; + + async.sortBy = function (arr, iterator, callback) { + async.map(arr, function (x, callback) { + iterator(x, function (err, criteria) { + if (err) { + callback(err); + } + else { + callback(null, {value: x, criteria: criteria}); + } + }); + }, function (err, results) { + if (err) { + return callback(err); + } + else { + var fn = function (left, right) { + var a = left.criteria, b = right.criteria; + return a < b ? -1 : a > b ? 1 : 0; + }; + callback(null, _map(results.sort(fn), function (x) { + return x.value; + })); + } + }); + }; + + async.auto = function (tasks, callback) { + callback = callback || function () {}; + var keys = _keys(tasks); + if (!keys.length) { + return callback(null); + } + + var results = {}; + + var listeners = []; + var addListener = function (fn) { + listeners.unshift(fn); + }; + var removeListener = function (fn) { + for (var i = 0; i < listeners.length; i += 1) { + if (listeners[i] === fn) { + listeners.splice(i, 1); + return; + } + } + }; + var taskComplete = function () { + _each(listeners.slice(0), function (fn) { + fn(); + }); + }; + + addListener(function () { + if (_keys(results).length === keys.length) { + callback(null, results); + callback = function () {}; + } + }); + + _each(keys, function (k) { + var task = (tasks[k] instanceof Function) ? [tasks[k]]: tasks[k]; + var taskCallback = function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (args.length <= 1) { + args = args[0]; + } + if (err) { + var safeResults = {}; + _each(_keys(results), function(rkey) { + safeResults[rkey] = results[rkey]; + }); + safeResults[k] = args; + callback(err, safeResults); + // stop subsequent errors hitting callback multiple times + callback = function () {}; + } + else { + results[k] = args; + async.setImmediate(taskComplete); + } + }; + var requires = task.slice(0, Math.abs(task.length - 1)) || []; + var ready = function () { + return _reduce(requires, function (a, x) { + return (a && results.hasOwnProperty(x)); + }, true) && !results.hasOwnProperty(k); + }; + if (ready()) { + task[task.length - 1](taskCallback, results); + } + else { + var listener = function () { + if (ready()) { + removeListener(listener); + task[task.length - 1](taskCallback, results); + } + }; + addListener(listener); + } + }); + }; + + async.waterfall = function (tasks, callback) { + callback = callback || function () {}; + if (tasks.constructor !== Array) { + var err = new Error('First argument to waterfall must be an array of functions'); + return callback(err); + } + if (!tasks.length) { + return callback(); + } + var wrapIterator = function (iterator) { + return function (err) { + if (err) { + callback.apply(null, arguments); + callback = function () {}; + } + else { + var args = Array.prototype.slice.call(arguments, 1); + var next = iterator.next(); + if (next) { + args.push(wrapIterator(next)); + } + else { + args.push(callback); + } + async.setImmediate(function () { + iterator.apply(null, args); + }); + } + }; + }; + wrapIterator(async.iterator(tasks))(); + }; + + var _parallel = function(eachfn, tasks, callback) { + callback = callback || function () {}; + if (tasks.constructor === Array) { + eachfn.map(tasks, function (fn, callback) { + if (fn) { + fn(function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (args.length <= 1) { + args = args[0]; + } + callback.call(null, err, args); + }); + } + }, callback); + } + else { + var results = {}; + eachfn.each(_keys(tasks), function (k, callback) { + tasks[k](function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (args.length <= 1) { + args = args[0]; + } + results[k] = args; + callback(err); + }); + }, function (err) { + callback(err, results); + }); + } + }; + + async.parallel = function (tasks, callback) { + _parallel({ map: async.map, each: async.each }, tasks, callback); + }; + + async.parallelLimit = function(tasks, limit, callback) { + _parallel({ map: _mapLimit(limit), each: _eachLimit(limit) }, tasks, callback); + }; + + async.series = function (tasks, callback) { + callback = callback || function () {}; + if (tasks.constructor === Array) { + async.mapSeries(tasks, function (fn, callback) { + if (fn) { + fn(function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (args.length <= 1) { + args = args[0]; + } + callback.call(null, err, args); + }); + } + }, callback); + } + else { + var results = {}; + async.eachSeries(_keys(tasks), function (k, callback) { + tasks[k](function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (args.length <= 1) { + args = args[0]; + } + results[k] = args; + callback(err); + }); + }, function (err) { + callback(err, results); + }); + } + }; + + async.iterator = function (tasks) { + var makeCallback = function (index) { + var fn = function () { + if (tasks.length) { + tasks[index].apply(null, arguments); + } + return fn.next(); + }; + fn.next = function () { + return (index < tasks.length - 1) ? makeCallback(index + 1): null; + }; + return fn; + }; + return makeCallback(0); + }; + + async.apply = function (fn) { + var args = Array.prototype.slice.call(arguments, 1); + return function () { + return fn.apply( + null, args.concat(Array.prototype.slice.call(arguments)) + ); + }; + }; + + var _concat = function (eachfn, arr, fn, callback) { + var r = []; + eachfn(arr, function (x, cb) { + fn(x, function (err, y) { + r = r.concat(y || []); + cb(err); + }); + }, function (err) { + callback(err, r); + }); + }; + async.concat = doParallel(_concat); + async.concatSeries = doSeries(_concat); + + async.whilst = function (test, iterator, callback) { + if (test()) { + iterator(function (err) { + if (err) { + return callback(err); + } + async.whilst(test, iterator, callback); + }); + } + else { + callback(); + } + }; + + async.doWhilst = function (iterator, test, callback) { + iterator(function (err) { + if (err) { + return callback(err); + } + if (test()) { + async.doWhilst(iterator, test, callback); + } + else { + callback(); + } + }); + }; + + async.until = function (test, iterator, callback) { + if (!test()) { + iterator(function (err) { + if (err) { + return callback(err); + } + async.until(test, iterator, callback); + }); + } + else { + callback(); + } + }; + + async.doUntil = function (iterator, test, callback) { + iterator(function (err) { + if (err) { + return callback(err); + } + if (!test()) { + async.doUntil(iterator, test, callback); + } + else { + callback(); + } + }); + }; + + async.queue = function (worker, concurrency) { + if (concurrency === undefined) { + concurrency = 1; + } + function _insert(q, data, pos, callback) { + if(data.constructor !== Array) { + data = [data]; + } + _each(data, function(task) { + var item = { + data: task, + callback: typeof callback === 'function' ? callback : null + }; + + if (pos) { + q.tasks.unshift(item); + } else { + q.tasks.push(item); + } + + if (q.saturated && q.tasks.length === concurrency) { + q.saturated(); + } + async.setImmediate(q.process); + }); + } + + var workers = 0; + var q = { + tasks: [], + concurrency: concurrency, + saturated: null, + empty: null, + drain: null, + push: function (data, callback) { + _insert(q, data, false, callback); + }, + unshift: function (data, callback) { + _insert(q, data, true, callback); + }, + process: function () { + if (workers < q.concurrency && q.tasks.length) { + var task = q.tasks.shift(); + if (q.empty && q.tasks.length === 0) { + q.empty(); + } + workers += 1; + var next = function () { + workers -= 1; + if (task.callback) { + task.callback.apply(task, arguments); + } + if (q.drain && q.tasks.length + workers === 0) { + q.drain(); + } + q.process(); + }; + var cb = only_once(next); + worker(task.data, cb); + } + }, + length: function () { + return q.tasks.length; + }, + running: function () { + return workers; + } + }; + return q; + }; + + async.cargo = function (worker, payload) { + var working = false, + tasks = []; + + var cargo = { + tasks: tasks, + payload: payload, + saturated: null, + empty: null, + drain: null, + push: function (data, callback) { + if(data.constructor !== Array) { + data = [data]; + } + _each(data, function(task) { + tasks.push({ + data: task, + callback: typeof callback === 'function' ? callback : null + }); + if (cargo.saturated && tasks.length === payload) { + cargo.saturated(); + } + }); + async.setImmediate(cargo.process); + }, + process: function process() { + if (working) return; + if (tasks.length === 0) { + if(cargo.drain) cargo.drain(); + return; + } + + var ts = typeof payload === 'number' + ? tasks.splice(0, payload) + : tasks.splice(0); + + var ds = _map(ts, function (task) { + return task.data; + }); + + if(cargo.empty) cargo.empty(); + working = true; + worker(ds, function () { + working = false; + + var args = arguments; + _each(ts, function (data) { + if (data.callback) { + data.callback.apply(null, args); + } + }); + + process(); + }); + }, + length: function () { + return tasks.length; + }, + running: function () { + return working; + } + }; + return cargo; + }; + + var _console_fn = function (name) { + return function (fn) { + var args = Array.prototype.slice.call(arguments, 1); + fn.apply(null, args.concat([function (err) { + var args = Array.prototype.slice.call(arguments, 1); + if (typeof console !== 'undefined') { + if (err) { + if (console.error) { + console.error(err); + } + } + else if (console[name]) { + _each(args, function (x) { + console[name](x); + }); + } + } + }])); + }; + }; + async.log = _console_fn('log'); + async.dir = _console_fn('dir'); + /*async.info = _console_fn('info'); + async.warn = _console_fn('warn'); + async.error = _console_fn('error');*/ + + async.memoize = function (fn, hasher) { + var memo = {}; + var queues = {}; + hasher = hasher || function (x) { + return x; + }; + var memoized = function () { + var args = Array.prototype.slice.call(arguments); + var callback = args.pop(); + var key = hasher.apply(null, args); + if (key in memo) { + callback.apply(null, memo[key]); + } + else if (key in queues) { + queues[key].push(callback); + } + else { + queues[key] = [callback]; + fn.apply(null, args.concat([function () { + memo[key] = arguments; + var q = queues[key]; + delete queues[key]; + for (var i = 0, l = q.length; i < l; i++) { + q[i].apply(null, arguments); + } + }])); + } + }; + memoized.memo = memo; + memoized.unmemoized = fn; + return memoized; + }; + + async.unmemoize = function (fn) { + return function () { + return (fn.unmemoized || fn).apply(null, arguments); + }; + }; + + async.times = function (count, iterator, callback) { + var counter = []; + for (var i = 0; i < count; i++) { + counter.push(i); + } + return async.map(counter, iterator, callback); + }; + + async.timesSeries = function (count, iterator, callback) { + var counter = []; + for (var i = 0; i < count; i++) { + counter.push(i); + } + return async.mapSeries(counter, iterator, callback); + }; + + async.compose = function (/* functions... */) { + var fns = Array.prototype.reverse.call(arguments); + return function () { + var that = this; + var args = Array.prototype.slice.call(arguments); + var callback = args.pop(); + async.reduce(fns, args, function (newargs, fn, cb) { + fn.apply(that, newargs.concat([function () { + var err = arguments[0]; + var nextargs = Array.prototype.slice.call(arguments, 1); + cb(err, nextargs); + }])) + }, + function (err, results) { + callback.apply(that, [err].concat(results)); + }); + }; + }; + + var _applyEach = function (eachfn, fns /*args...*/) { + var go = function () { + var that = this; + var args = Array.prototype.slice.call(arguments); + var callback = args.pop(); + return eachfn(fns, function (fn, cb) { + fn.apply(that, args.concat([cb])); + }, + callback); + }; + if (arguments.length > 2) { + var args = Array.prototype.slice.call(arguments, 2); + return go.apply(this, args); + } + else { + return go; + } + }; + async.applyEach = doParallel(_applyEach); + async.applyEachSeries = doSeries(_applyEach); + + async.forever = function (fn, callback) { + function next(err) { + if (err) { + if (callback) { + return callback(err); + } + throw err; + } + fn(next); + } + next(); + }; + + // AMD / RequireJS + if (typeof define !== 'undefined' && define.amd) { + define([], function () { + return async; + }); + } + // Node.js + else if (typeof module !== 'undefined' && module.exports) { + module.exports = async; + } + // included directly via <script> tag + else { + root.async = async; + } + +}()); http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/async/package.json ---------------------------------------------------------------------- diff --git a/node_modules/async/package.json b/node_modules/async/package.json new file mode 100644 index 0000000..11d8c82 --- /dev/null +++ b/node_modules/async/package.json @@ -0,0 +1,95 @@ +{ + "_args": [ + [ + { + "raw": "async@~0.2.6", + "scope": null, + "escapedName": "async", + "name": "async", + "rawSpec": "~0.2.6", + "spec": ">=0.2.6 <0.3.0", + "type": "range" + }, + "/Users/yueguo/tmp/griffin-site/node_modules/uglify-js" + ] + ], + "_from": "async@>=0.2.6 <0.3.0", + "_id": "[email protected]", + "_inCache": true, + "_installable": true, + "_location": "/async", + "_npmUser": { + "name": "caolan", + "email": "[email protected]" + }, + "_npmVersion": "1.3.2", + "_phantomChildren": {}, + "_requested": { + "raw": "async@~0.2.6", + "scope": null, + "escapedName": "async", + "name": "async", + "rawSpec": "~0.2.6", + "spec": ">=0.2.6 <0.3.0", + "type": "range" + }, + "_requiredBy": [ + "/uglify-js" + ], + "_resolved": "https://registry.npmjs.org/async/-/async-0.2.10.tgz";, + "_shasum": "b6bbe0b0674b9d719708ca38de8c237cb526c3d1", + "_shrinkwrap": null, + "_spec": "async@~0.2.6", + "_where": "/Users/yueguo/tmp/griffin-site/node_modules/uglify-js", + "author": { + "name": "Caolan McMahon" + }, + "bugs": { + "url": "https://github.com/caolan/async/issues"; + }, + "dependencies": {}, + "description": "Higher-order functions and common patterns for asynchronous code", + "devDependencies": { + "nodelint": ">0.0.0", + "nodeunit": ">0.0.0", + "uglify-js": "1.2.x" + }, + "directories": {}, + "dist": { + "shasum": "b6bbe0b0674b9d719708ca38de8c237cb526c3d1", + "tarball": "https://registry.npmjs.org/async/-/async-0.2.10.tgz"; + }, + "homepage": "https://github.com/caolan/async#readme";, + "jam": { + "main": "lib/async.js", + "include": [ + "lib/async.js", + "README.md", + "LICENSE" + ] + }, + "licenses": [ + { + "type": "MIT", + "url": "https://github.com/caolan/async/raw/master/LICENSE"; + } + ], + "main": "./lib/async", + "maintainers": [ + { + "name": "caolan", + "email": "[email protected]" + } + ], + "name": "async", + "optionalDependencies": {}, + "readme": "ERROR: No README data found!", + "repository": { + "type": "git", + "url": "git+https://github.com/caolan/async.git"; + }, + "scripts": { + "test": "nodeunit test/test-async.js" + }, + "version": "0.2.10" +} http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/asynckit/LICENSE ---------------------------------------------------------------------- diff --git a/node_modules/asynckit/LICENSE b/node_modules/asynckit/LICENSE new file mode 100644 index 0000000..c9eca5d --- /dev/null +++ b/node_modules/asynckit/LICENSE @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2016 Alex Indigo + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. http://git-wip-us.apache.org/repos/asf/incubator-griffin-site/blob/4f8fa326/node_modules/asynckit/README.md ---------------------------------------------------------------------- diff --git a/node_modules/asynckit/README.md b/node_modules/asynckit/README.md new file mode 100644 index 0000000..ddcc7e6 --- /dev/null +++ b/node_modules/asynckit/README.md @@ -0,0 +1,233 @@ +# asynckit [](https://www.npmjs.com/package/asynckit) + +Minimal async jobs utility library, with streams support. + +[](https://travis-ci.org/alexindigo/asynckit) +[](https://travis-ci.org/alexindigo/asynckit) +[](https://ci.appveyor.com/project/alexindigo/asynckit) + +[](https://coveralls.io/github/alexindigo/asynckit?branch=master) +[](https://david-dm.org/alexindigo/asynckit) +[](https://www.bithound.io/github/alexindigo/asynckit) + +<!-- [](https://www.npmjs.com/package/reamde) --> + +AsyncKit provides harness for `parallel` and `serial` iterators over list of items represented by arrays or objects. +Optionally it accepts abort function (should be synchronously return by iterator for each item), and terminates left over jobs upon an error event. For specific iteration order built-in (`ascending` and `descending`) and custom sort helpers also supported, via `asynckit.serialOrdered` method. + +It ensures async operations to keep behavior more stable and prevent `Maximum call stack size exceeded` errors, from sync iterators. + +| compression | size | +| :----------------- | -------: | +| asynckit.js | 12.34 kB | +| asynckit.min.js | 4.11 kB | +| asynckit.min.js.gz | 1.47 kB | + + +## Install + +```sh +$ npm install --save asynckit +``` + +## Examples + +### Parallel Jobs + +Runs iterator over provided array in parallel. Stores output in the `result` array, +on the matching positions. In unlikely event of an error from one of the jobs, +will terminate rest of the active jobs (if abort function is provided) +and return error along with salvaged data to the main callback function. + +#### Input Array + +```javascript +var parallel = require('asynckit').parallel + , assert = require('assert') + ; + +var source = [ 1, 1, 4, 16, 64, 32, 8, 2 ] + , expectedResult = [ 2, 2, 8, 32, 128, 64, 16, 4 ] + , expectedTarget = [ 1, 1, 2, 4, 8, 16, 32, 64 ] + , target = [] + ; + +parallel(source, asyncJob, function(err, result) +{ + assert.deepEqual(result, expectedResult); + assert.deepEqual(target, expectedTarget); +}); + +// async job accepts one element from the array +// and a callback function +function asyncJob(item, cb) +{ + // different delays (in ms) per item + var delay = item * 25; + + // pretend different jobs take different time to finish + // and not in consequential order + var timeoutId = setTimeout(function() { + target.push(item); + cb(null, item * 2); + }, delay); + + // allow to cancel "leftover" jobs upon error + // return function, invoking of which will abort this job + return clearTimeout.bind(null, timeoutId); +} +``` + +More examples could be found in [test/test-parallel-array.js](test/test-parallel-array.js). + +#### Input Object + +Also it supports named jobs, listed via object. + +```javascript +var parallel = require('asynckit/parallel') + , assert = require('assert') + ; + +var source = { first: 1, one: 1, four: 4, sixteen: 16, sixtyFour: 64, thirtyTwo: 32, eight: 8, two: 2 } + , expectedResult = { first: 2, one: 2, four: 8, sixteen: 32, sixtyFour: 128, thirtyTwo: 64, eight: 16, two: 4 } + , expectedTarget = [ 1, 1, 2, 4, 8, 16, 32, 64 ] + , expectedKeys = [ 'first', 'one', 'two', 'four', 'eight', 'sixteen', 'thirtyTwo', 'sixtyFour' ] + , target = [] + , keys = [] + ; + +parallel(source, asyncJob, function(err, result) +{ + assert.deepEqual(result, expectedResult); + assert.deepEqual(target, expectedTarget); + assert.deepEqual(keys, expectedKeys); +}); + +// supports full value, key, callback (shortcut) interface +function asyncJob(item, key, cb) +{ + // different delays (in ms) per item + var delay = item * 25; + + // pretend different jobs take different time to finish + // and not in consequential order + var timeoutId = setTimeout(function() { + keys.push(key); + target.push(item); + cb(null, item * 2); + }, delay); + + // allow to cancel "leftover" jobs upon error + // return function, invoking of which will abort this job + return clearTimeout.bind(null, timeoutId); +} +``` + +More examples could be found in [test/test-parallel-object.js](test/test-parallel-object.js). + +### Serial Jobs + +Runs iterator over provided array sequentially. Stores output in the `result` array, +on the matching positions. In unlikely event of an error from one of the jobs, +will not proceed to the rest of the items in the list +and return error along with salvaged data to the main callback function. + +#### Input Array + +```javascript +var serial = require('asynckit/serial') + , assert = require('assert') + ; + +var source = [ 1, 1, 4, 16, 64, 32, 8, 2 ] + , expectedResult = [ 2, 2, 8, 32, 128, 64, 16, 4 ] + , expectedTarget = [ 0, 1, 2, 3, 4, 5, 6, 7 ] + , target = [] + ; + +serial(source, asyncJob, function(err, result) +{ + assert.deepEqual(result, expectedResult); + assert.deepEqual(target, expectedTarget); +}); + +// extended interface (item, key, callback) +// also supported for arrays +function asyncJob(item, key, cb) +{ + target.push(key); + + // it will be automatically made async + // even it iterator "returns" in the same event loop + cb(null, item * 2); +} +``` + +More examples could be found in [test/test-serial-array.js](test/test-serial-array.js). + +#### Input Object + +Also it supports named jobs, listed via object. + +```javascript +var serial = require('asynckit').serial + , assert = require('assert') + ; + +var source = [ 1, 1, 4, 16, 64, 32, 8, 2 ] + , expectedResult = [ 2, 2, 8, 32, 128, 64, 16, 4 ] + , expectedTarget = [ 0, 1, 2, 3, 4, 5, 6, 7 ] + , target = [] + ; + +var source = { first: 1, one: 1, four: 4, sixteen: 16, sixtyFour: 64, thirtyTwo: 32, eight: 8, two: 2 } + , expectedResult = { first: 2, one: 2, four: 8, sixteen: 32, sixtyFour: 128, thirtyTwo: 64, eight: 16, two: 4 } + , expectedTarget = [ 1, 1, 4, 16, 64, 32, 8, 2 ] + , target = [] + ; + + +serial(source, asyncJob, function(err, result) +{ + assert.deepEqual(result, expectedResult); + assert.deepEqual(target, expectedTarget); +}); + +// shortcut interface (item, callback) +// works for object as well as for the arrays +function asyncJob(item, cb) +{ + target.push(item); + + // it will be automatically made async + // even it iterator "returns" in the same event loop + cb(null, item * 2); +} +``` + +More examples could be found in [test/test-serial-object.js](test/test-serial-object.js). + +_Note: Since _object_ is an _unordered_ collection of properties, +it may produce unexpected results with sequential iterations. +Whenever order of the jobs' execution is important please use `serialOrdered` method._ + +### Ordered Serial Iterations + +TBD + +For example [compare-property](compare-property) package. + +### Streaming interface + +TBD + +## Want to Know More? + +More examples can be found in [test folder](test/). + +Or open an [issue](https://github.com/alexindigo/asynckit/issues) with questions and/or suggestions. + +## License + +AsyncKit is licensed under the MIT license.
