Thanks for sharing. Seems like great work. Feel free to use the benchmarks with your work :) Would love to know how it compares.
Nuno On Fri, Aug 16, 2013 at 1:01 PM, Floby <[email protected]> wrote: > Nice article Nuno. > > You say you had trouble finding streaming json parsers. > > I wrote mine a while ago. It's at > http://github.com/Floby/node-json-streams > It doesn't perform as well as I hoped, but the approach is a little > different. > > > On Thursday, 15 August 2013 22:00:39 UTC+2, [email protected] wrote: >> >> My actual question: Is this the fastest way I can accomplish my goal? >> Thanks. >> >> As part of a work project I have written a converter function that takes >> a JSON string or object as input and returns a key:value space separated >> string. The goal is to do this transformation as quickly as possible, the >> system this will be part of is supposed to handle about 20k to 40k >> transactions per second. Although not all transactions will need the >> transformation. >> >> The case is that data flow had this key:value comma separated format >> originally. The upstream components and my component have changed the >> format to JSON, but the downstream component has not. Therefore I have do a >> conversion for them before sending it down. I dont agree with that decision >> but it is what it is. >> >> There are a couple of quirks that I had to deal with. >> >> The JSON contains some values that are base64 encoded and then wrapped >> like this: >> "key":{"enc":"base64","value":**"base64value"} >> which would have to be converted to: >> key:base64value >> >> The JSON also contains some arrays of integers: >> "arr":[1,2,3,4,5,6] >> which would have to be converted to: >> arr:1,2,3,4,5,6 >> >> so given this as input: >> { >> "timestamp":"1364911321", >> "sip":3470129951, >> { >> "enc":"base64", >> "value":"base64value" >> }, >> "ar":[1,2,3,4,5,6] >> } >> >> >> I want this ouput: >> >> timestamp:1364911321 sip:3470129951 cliid:base64value ar:1,2,3,4,5,6 >> >> >> >> >> This is my first pass at this and would appreciate your input. >> >> /** >> * >> * Class to handler converting JSON objects and Strings to >> * Repper formatted strings. >> * >> * All quotes, braces are removed >> * All commas are converted to spaces >> * >> * TODO handle arrays wrapped in [] >> */ >> var JSONToRepper = function () {} >> >> JSONToRepper.replacer = function(key, value) { >> /* >> Look for wrapped values and return only the base64 value for >> the given key >> */ >> if('undefined' !== typeof value.enc && value.enc === 'base64'){ >> return value.value; >> } else if (typeof value === 'number' && !isFinite(value)) { >> return String(value); >> } >> return value; >> } >> /** >> * @param String|Object json >> * @return String >> * >> * Can accept a JSON string or object and will return a string in >> * Repper format. >> */ >> JSONToRepper.prototype.convert = function(json) { >> >> var _this = this; >> >> if('object' === typeof json){ >> json = JSON.stringify(json, _this.replacer); >> } >> /* >> * Remove all braces {} and quotes "" >> * Replace all commas with except when they appear inside [] >> * Replace all brackets [] >> */ >> return json.replace(/[\"|\{|\}]/g, '').replace(/,(?=[^\]]*(?:\[|$ >> **))/g, ' ').replace(/\[|\]/g, ''); >> } >> >> exports.JSONToRepper = new JSONToRepper(); >> >> -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
