Nice article Nuno. You say you had trouble finding streaming json parsers.
I wrote mine a while ago. It's at http://github.com/Floby/node-json-streams It doesn't perform as well as I hoped, but the approach is a little different. On Thursday, 15 August 2013 22:00:39 UTC+2, [email protected] wrote: > > My actual question: Is this the fastest way I can accomplish my goal? > Thanks. > > As part of a work project I have written a converter function that takes a > JSON string or object as input and returns a key:value space separated > string. The goal is to do this transformation as quickly as possible, the > system this will be part of is supposed to handle about 20k to 40k > transactions per second. Although not all transactions will need the > transformation. > > The case is that data flow had this key:value comma separated format > originally. The upstream components and my component have changed the > format to JSON, but the downstream component has not. Therefore I have do a > conversion for them before sending it down. I dont agree with that decision > but it is what it is. > > There are a couple of quirks that I had to deal with. > > The JSON contains some values that are base64 encoded and then wrapped > like this: > "key":{"enc":"base64","value":"base64value"} > which would have to be converted to: > key:base64value > > The JSON also contains some arrays of integers: > "arr":[1,2,3,4,5,6] > which would have to be converted to: > arr:1,2,3,4,5,6 > > so given this as input: > { > "timestamp":"1364911321", > "sip":3470129951, > { > "enc":"base64", > "value":"base64value" > }, > "ar":[1,2,3,4,5,6] > } > > > I want this ouput: > > timestamp:1364911321 sip:3470129951 cliid:base64value ar:1,2,3,4,5,6 > > > > > This is my first pass at this and would appreciate your input. > > /** > * > * Class to handler converting JSON objects and Strings to > * Repper formatted strings. > * > * All quotes, braces are removed > * All commas are converted to spaces > * > * TODO handle arrays wrapped in [] > */ > var JSONToRepper = function () {} > > JSONToRepper.replacer = function(key, value) { > /* > Look for wrapped values and return only the base64 value for > the given key > */ > if('undefined' !== typeof value.enc && value.enc === 'base64'){ > return value.value; > } else if (typeof value === 'number' && !isFinite(value)) { > return String(value); > } > return value; > } > /** > * @param String|Object json > * @return String > * > * Can accept a JSON string or object and will return a string in > * Repper format. > */ > JSONToRepper.prototype.convert = function(json) { > > var _this = this; > > if('object' === typeof json){ > json = JSON.stringify(json, _this.replacer); > } > /* > * Remove all braces {} and quotes "" > * Replace all commas with except when they appear inside [] > * Replace all brackets [] > */ > return json.replace(/[\"|\{|\}]/g, > '').replace(/,(?=[^\]]*(?:\[|$))/g, ' ').replace(/\[|\]/g, ''); > } > > exports.JSONToRepper = new JSONToRepper(); > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
