I think the tokenizer part of my lib is what is closest to yours (it emits 'begin-object', 'boolean', 'number', 'end-object' and so on) but it's regexp based so I think it doesn't perform as good. But I'll try nevertheless, to see how far off I am.
On Monday, 19 August 2013 11:39:32 UTC+2, Nuno Job wrote: > > Thanks for sharing. Seems like great work. > > Feel free to use the benchmarks with your work :) Would love to know how > it compares. > > Nuno > > > On Fri, Aug 16, 2013 at 1:01 PM, Floby <[email protected] > <javascript:>>wrote: > >> Nice article Nuno. >> >> You say you had trouble finding streaming json parsers. >> >> I wrote mine a while ago. It's at >> http://github.com/Floby/node-json-streams >> It doesn't perform as well as I hoped, but the approach is a little >> different. >> >> >> On Thursday, 15 August 2013 22:00:39 UTC+2, [email protected] wrote: >>> >>> My actual question: Is this the fastest way I can accomplish my goal? >>> Thanks. >>> >>> As part of a work project I have written a converter function that takes >>> a JSON string or object as input and returns a key:value space separated >>> string. The goal is to do this transformation as quickly as possible, the >>> system this will be part of is supposed to handle about 20k to 40k >>> transactions per second. Although not all transactions will need the >>> transformation. >>> >>> The case is that data flow had this key:value comma separated format >>> originally. The upstream components and my component have changed the >>> format to JSON, but the downstream component has not. Therefore I have do a >>> conversion for them before sending it down. I dont agree with that decision >>> but it is what it is. >>> >>> There are a couple of quirks that I had to deal with. >>> >>> The JSON contains some values that are base64 encoded and then wrapped >>> like this: >>> "key":{"enc":"base64","value":**"base64value"} >>> which would have to be converted to: >>> key:base64value >>> >>> The JSON also contains some arrays of integers: >>> "arr":[1,2,3,4,5,6] >>> which would have to be converted to: >>> arr:1,2,3,4,5,6 >>> >>> so given this as input: >>> { >>> "timestamp":"1364911321", >>> "sip":3470129951, >>> { >>> "enc":"base64", >>> "value":"base64value" >>> }, >>> "ar":[1,2,3,4,5,6] >>> } >>> >>> >>> I want this ouput: >>> >>> timestamp:1364911321 sip:3470129951 cliid:base64value ar:1,2,3,4,5,6 >>> >>> >>> >>> >>> This is my first pass at this and would appreciate your input. >>> >>> /** >>> * >>> * Class to handler converting JSON objects and Strings to >>> * Repper formatted strings. >>> * >>> * All quotes, braces are removed >>> * All commas are converted to spaces >>> * >>> * TODO handle arrays wrapped in [] >>> */ >>> var JSONToRepper = function () {} >>> >>> JSONToRepper.replacer = function(key, value) { >>> /* >>> Look for wrapped values and return only the base64 value for >>> the given key >>> */ >>> if('undefined' !== typeof value.enc && value.enc === 'base64'){ >>> return value.value; >>> } else if (typeof value === 'number' && !isFinite(value)) { >>> return String(value); >>> } >>> return value; >>> } >>> /** >>> * @param String|Object json >>> * @return String >>> * >>> * Can accept a JSON string or object and will return a string in >>> * Repper format. >>> */ >>> JSONToRepper.prototype.convert = function(json) { >>> >>> var _this = this; >>> >>> if('object' === typeof json){ >>> json = JSON.stringify(json, _this.replacer); >>> } >>> /* >>> * Remove all braces {} and quotes "" >>> * Replace all commas with except when they appear inside [] >>> * Replace all brackets [] >>> */ >>> return json.replace(/[\"|\{|\}]/g, '').replace(/,(?=[^\]]*(?:\[|$ >>> **))/g, ' ').replace(/\[|\]/g, ''); >>> } >>> >>> exports.JSONToRepper = new JSONToRepper(); >>> >>> -- >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected]<javascript:> >> To unsubscribe from this group, send email to >> [email protected] <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "nodejs" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
