Your results are deceptive for a couple of reasons. 1) You are still performing an operations per second test opposed to Testing against the clock. Converting results from one method to look like the results of a different method does not change how you test, which means that your test is still equally flawed. This is flawed, because it accounts for API access in addition to execution time. We already know that API access is significantly more costly than raw execution on the data. Since the API access cannot be distinguished from execution time in your results there is no way to determine which aspect of the operation you are actually performance testing.
2) These numbers are too small to be valid. There is variance in the results that occurs per test result. This variance increases as the memory footprint of the executing application increases. If the variance between results of identical tests is more than 25% of the result value than you are testing with an extraordinarily low confidence. 3) You cannot verify the presence of variance without running the identical test multiple times. The more times the identical test is executed the more certainty can be attributed to a known variance of the results. The more certainly you can identify exactly what the variance is the higher the percentage confidence of accuracy increases. > Sure I could write the loops myself. I do not misunderstand your test suite. I am simply not interested in loops. This test has nothing to do with loops. This is about measuring selector execution speed. I beginning to believe you are not interested in discovering the results of a fact finding mission, but instead seek to prove a point with the least amount of effort. If you really want numbers then you actually have to put forth some effort and create your own test platform with your own clock code. If you are either incapable or unwilling to provide that effort then you can wait until I have the time to do so. Thanks, Austin Cheney, CISSP -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Scott Sauyet Sent: Tuesday, September 13, 2011 7:14 AM To: The JSMentors JavaScript Discussion Group Subject: [JSMentors] Re: spaces in attribute values Austin Cheney wrote: > Scott Sauyet wrote: >> Let's talk real numbers here. >> >> http://jsperf.com/xpath-vs-dom > > This test is invalid. There are almost no valid examples where > operations per second account for any sort of performance associated > with web technologies. You brought up performance, saying: | I have compared XPath to CSS selectors. jQuery is not compiled byte | code, and so it is too slow to be of worthy consideration. You can | easily verify the difference yourself. Just compare a complex operation | navigating the DOM from one location to a different, and not necessarily | descendent, location versus using DOM methods while separately timing | each operation against a millisecond clock. The timing difference is | staggering, especially for results in a loop and queries against an | extremely large DOM. [1] I simply tested this. I don't believe you really need it explained, but just in case: milliseconds per operation = 1000 / (operations per second) So the tests as they stand right now for FF6.0, a single operation takes about jQuery: 0.02826935ms XPath: 0.01788768ms DOM: 0.00045303ms Clearly the DOM method is fastest, and XPath somewhat faster than jQuery, but any of them is easily fast enough for a human-initiated event. The main point is that for this sort of code (accessing the great-aunt H3 node), any of these techniques is fast enough for ordinary usage. If this were done in a tight loop, perhaps we'd have to revisit the question. > The only exception I can think of is testing > certain conditions in a loop. I have already mentioned, several times, > that there is a performance hit for accessing a foreign API. If you have some numbers on this, I'd love to see them. JSPerf is an excellent way to write quick performance tests. > This said > a regular expression operation would also fail on your test and should > this suggest that regular expressions are disgustingly slow? I'm not sure what test you think they could fail. There was no predicate here. I checked first that all the functions in question did what they were supposed to do (and this version of the tests actually do fail in IE and Android -- I'd love to see better ones) and then I simply compared speed. There was no notion here of passing or failing a test. > No. It is > my guess that the regular expression operator is the fastest way to > access anything in the form of a large string, and if this is true then > your test fails to identify with its sole intention. As I understood it the goal was to access the DOM node represented in the markup with an H3 tag and which was a sibling to the parent of the parent of a particular button node known in advance. I understand how to test the innerHTML against a regex, but how could you use that to find this particular node in the document? > Exactly like I mentioned in my prior email the only valid way to test > this sort of operation is against a millisecond clock. This means that > if you want to see a benchmark then you have to do some actual work, > opposed to merely feeding numbers into some extraneous application. Maybe you misunderstand JSPerf. It's doing live benchmarking of code you enter. Sure I could write the loops myself. Before JSPerf came along, I used JSLitmus to do these things on my own site. But it's not calculating speeds, it's measuring them. > A valid test is stated in the form of a question. > [... further description of the scientific method elided ... ] What test do you think I'm performing here? This is plain benchmarking, a form of data-gathering, not construction of a scientific hypothesis. > You did not seem to know how to use regular expressions to access the > DOM. [ ... ] No, I didn't -- and still don't -- understand how you would use a regular express to retrieve the DOM node you're trying to target. I use regexes regularly in my day-to-day JS coding, probably more than some of my coworkers would like, but I can't see how to turn a regex match on the innerHTML back into a reference to a DOM node. Do you have a way to do that? > If you need help writing a timing clock you can take the one I wrote in > the Pretty Diff tool, but it must wrap each test operation closely and > uniquely in order to prevent cross interference. If you would rather I > just produce this test then you will need to wait a week until I have > the necessary time available. I've written many of them over the years, thank you. I think the one used by JSPerf is excellent, though. It used to be based on Robert Kieffer's JSLitmus [2], but they switched to John David-Dalton's Benchmark [3], both of which I've used independent of JSPerf, and both of which seem to be excellent benchmarking tools. And I'm still curious about what sort of hypothesis you think I should be testing. -- Scott [1] http://groups.google.com/group/jsmentors/msg/63c1404f0fdbe959 [2] http://www.broofa.com/Tools/JSLitmus/JSLitmus.js [3] http://benchmarkjs.com/ -- To view archived discussions from the original JSMentors Mailman list: http://www.mail-archive.com/[email protected]/ To search via a non-Google archive, visit here: http://www.mail-archive.com/[email protected]/ To unsubscribe from this group, send email to [email protected] -- To view archived discussions from the original JSMentors Mailman list: http://www.mail-archive.com/[email protected]/ To search via a non-Google archive, visit here: http://www.mail-archive.com/[email protected]/ To unsubscribe from this group, send email to [email protected]
