Hi mscdex, I think dicer benchmarks are not exactly accurate. You are using only 2 occurrences of the boundary in a 100MB Buffer. I think that a reliable test for multipart streams (as for multiple file uploads) implies the presence of a lot of boundaries and big data to get the average results.
In general Bop and Qap parser are much more faster, when you have a lot of occurrences for boundaries. You have quite the same datarate results for Qap and Bop in your benchmarks, but the results are quite different in a real world example. Qap is a general good solution for sparse presence of boundaries and because it consumes less memory with big patterns. Bop is the better solution for small patterns and a lot of occurrences. ( I think also it's the fastest parser for multipart streams) I present some typical results for Qap and Bop (as you will see in benchmarks) with *700 MB test Buffer and 225917 occurrences of boundary:* *for Bop -> * https://github.com/rootslab/bop * * *node bench/small-pattern-data-rate.js 700 2* - building test buffer 700MB - current pattern: "---------------------------2046863043300497616870820724\r\n" - pattern copied* 225917 *times in test buffer - pattern length is *57 bytes* - current gap factor is 2 - patterns gap (distance) is 3.17 KBytes - buffer creation time: 0.842 secs - test buffer size is *700 MBytes* - check if results length is equal to 225917 ... - results matched are *225917* - total elapsed time is *0.268 secs* - parsing data rate is *20.41 Gbit/s* *for Qap ->* https://github.com/rootslab/qap *node bench/small-pattern-data-rate.js 700 2* * * - current pattern: "---------------------------2046863043300497616870820724\r\n" - pattern length is *57 bytes* - current gap factor is 2 - patterns gap (distance) is 3.17 KBytes - buffer creation time:* 0.87 secs* - test buffer size is* 700 MBytes* - check if results length is equal to* 225917* ... - results matched are *225917* - total elapsed time is *1.228 secs* - parsing data rate is* 4.45 Gbit/s* As you will see. results changes a lot. Il giorno domenica 7 aprile 2013 23:47:37 UTC+2, mscdex ha scritto: > > Announcing dicer[1], a fast, streaming multipart parser. > > FWIW here are some results I received using the simple 'bench- > multipart-parser.js' from node-formidable for various multipart > parsers currently available on npm: > > [email protected] (streaming): 2857.14 mb/sec > [email protected] (streaming): 1282.05 mb/sec > [email protected] (streaming): 980.39 mb/sec > [email protected] (streaming): 598.80 mb/sec > [email protected] (non-streaming, used in formaline and poor-form): 3125.00 > mb/sec > [email protected] (non-streaming): 3125.00 mb/sec > > > [1] https://github.com/mscdex/dicer > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
