On Thu, Feb 12, 2015 at 8:16 AM, Peter Haworth <p...@lcsql.com> wrote:
> > Oh yes, and doesn't matter whether you're using LC 7 or something prior to > that, although that problem appears to be a bug that is already being > fixed. > > I think that statement requires a qualifier - 'if you are not dealing with unicode text'. I did a similar test to you using a Valentina DB with and without unicode. Specifically I used a UTF8 db so the unicode test data had to be passed through LCs textDecode(dboutput,"utf8") to get the correct results; which obviously takes time. My results suggest that if Ali's fix is anything to go by, when working with utf8 data there could be a noticeable difference for big data sets and that Mark's qrevers() shines a very favourable light on LCs capabilities. I wish to again point out, NOT because I don't understand how LC counts lines NOT because I want to change the way LC works NOT because I want to open a discussion about whehter it's right or wrong, but simply for awareness, that the DB approach MAY produce a different result than the LC functions thus far offered. Lets say for example you had a list of 10,000 customers and their email addresses. Most customers do have an email, a few don't and it just so happens that your first customer Aardvark, and last customer, Zoe, don't have email. If you read just the emails into an LC variable the first line will be blank and you'll have 9999 lines. If you use the LC provided functions thus far presented to reverse the data, you'll end up with the first line not being blank, and 9998 lines. On the other hand, if you use a DB, your record count will always be 10000 and your first and last record will always be blank no matter which direction you sort it. What did surprise me was that there was NO penalty to adding an ORDER BY clause to the SELECT statement. My tests included: natural = SELECT data FROM db -- NO order by clause forward = SELECT data FROM db ORDER BY recid reverse = SELECT data FROM db ORDER BY recid DESC My 10,000 and 100,000 results below: This test was for Unicode = NO With a data set that looks like: aa bb cc !"#$% &'()* +,-./ 01234 56789 :;<=> ?@ABC DEFGH IJKLM NOPQR STUVW XYZ[\ ]^_`a bcdef ghijk lmnop qrstu vwxyz ..... repeated 9999 times for a Total of 10006 lines of data. xx yy zz Extracting the data in natural order took 44 ms Extracting the data in Forward order took 41 ms Extracting by Reverse order took 41 ms This test was for Unicode = YES With a data set that looks like: aa bb cc àâäãå çéèêë íìîïñ óòôöõ úùûü† °¢£§• ¶ß®©™ ´¨≠ÆØ ∞±≤≥¥ µ∂∑∏π ∫ªºΩæ ø¿¡¬√ ƒ≈∆«» … ÀÃÕ Œœ–—“ ”‘’÷◊ ÿŸ⁄€‹ ›fifl‡· ..... repeated 9999 times for a Total of 10006 lines of data. xx yy zz Extracting the data in natural order took 77 ms Extracting the data in Forward order took 77 ms Extracting by Reverse order took 76 ms This test was for Unicode = NO With a data set that looks like: aa bb cc !"#$% &'()* +,-./ 01234 56789 :;<=> ?@ABC DEFGH IJKLM NOPQR STUVW XYZ[\ ]^_`a bcdef ghijk lmnop qrstu vwxyz ..... repeated 99999 times for a Total of 100006 lines of data. xx yy zz Extracting the data in natural order took 545 ms Extracting the data in Forward order took 556 ms Extracting by Reverse order took 535 ms This test was for Unicode = YES With a data set that looks like: aa bb cc àâäãå çéèêë íìîïñ óòôöõ úùûü† °¢£§• ¶ß®©™ ´¨≠ÆØ ∞±≤≥¥ µ∂∑∏π ∫ªºΩæ ø¿¡¬√ ƒ≈∆«» … ÀÃÕ Œœ–—“ ”‘’÷◊ ÿŸ⁄€‹ ›fifl‡· ..... repeated 99999 times for a Total of 100006 lines of data. xx yy zz Extracting the data in natural order took 998 ms Extracting the data in Forward order took 1010 ms Extracting by Reverse order took 1005 ms _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
