Hi, Oliver, Moving to ICU3.8.1 still introduces 2% degradation on SPECjbb2005. We might stick to current version for a while, I will have a look when I have spare time.
Thanks, Aleksey. On Jan 17, 2008 1:46 PM, Oliver Deakin <[EMAIL PROTECTED]> wrote: > That's great - thanks Aleksey! > > > Regards, > Oliver > > Aleksey Shipilev wrote: > > Wow, thanks Oliver. My head is spinning these days, I forgot about new > > ICU library. > > > > Ok, last time we have measured the performance without Tony's patch > > onboard, so we might recheck migration to ICU 3.8.1 again. Since > > Tony's patch delegates much more functionality to ICU, the performance > > benefits/degradations should be more evident. Hopefully will check > > tomorrow. > > > > Thanks, > > Aleksey. > > > > On Jan 16, 2008 8:51 PM, Oliver Deakin <[EMAIL PROTECTED]> wrote: > > > >> Hi Aleksey (and anyone else interested!) > >> > >> Do you have any further comments on these changes? With the recent > >> discussion over Tony's work on removing duplicate locale data I was > >> wondering if it would be ok to step up to this stable release if icu4j > >> and deal with performance issues "in the wild"? It seems to me that > >> while we stall upgrading to a release version of this library it's > >> performance will never be closely examined, and as such will continue to > >> put the upgrade on hold indefinitely. > >> > >> I would suggest we move to icu4j 3.8.1 and begin to feed back > >> performance queries to the ICU team. Are there objections? > >> > >> Regards, > >> Oliver > >> > >> > >> Oliver Deakin wrote: > >> > >>> Hi Aleksey, > >>> > >>> Thanks for performance testing the changes. It's great that the Dacapo > >>> benchmark has not been affected by this change, but I agree that the > >>> small SPEC degradation is a concern. However, I would argue that we > >>> should still upgrade to the new version. At this point I value bug > >>> fixes and stability over performance, and IMHO moving to a proper > >>> release version of icu4j is preferable to using the current mid > >>> development cycle build we have in SVN, even if there is a slight > >>> degradation in one of the benchmarks. > >>> > >>> I would personally suggest that we still move to the new version of > >>> icu4j after M4 and address the performance issues as an ongoing task > >>> with the ICU developers. > >>> > >>> Regards, > >>> Oliver > >>> > >>> Aleksey Shipilev wrote: > >>> > >>>> Hi, Oliver! > >>>> > >>>> This change: > >>>> a. does not affect composite Dacapo performance > >>>> b. degrades SPECjbb2005 performance for 1-2%. > >>>> > >>>> Taking (b) into the account, I would like to vote for deferring > >>>> immediate moving to ICU 3.8.1. We can revisit this one more time if > >>>> there's a way to beat the degradation. > >>>> > >>>> Thanks, > >>>> Aleksey. > >>>> > >>>> On Dec 14, 2007 3:53 PM, Oliver Deakin <[EMAIL PROTECTED]> > >>>> wrote: > >>>> > >>>> > >>>>> Thanks for offering to help Aleksey, that's great! > >>>>> > >>>>> Ive opened HARMONY-5313 and attached a script and a patch to be > >>>>> applied. > >>>>> Please run the script before applying the patch, as one file needs > >>>>> to be > >>>>> moved before it is patched. Once it's applied, if you run the > >>>>> fetch-depends target you should see that the new ICU4J jars are > >>>>> downloaded. Then if you just rebuild you should be ready to run with > >>>>> the > >>>>> new version. > >>>>> > >>>>> Thanks for the help! > >>>>> Oliver > >>>>> > >>>>> > >>>>> Aleksey Shipilev wrote: > >>>>> > >>>>> > >>>>>> Thanks, Oliver! > >>>>>> > >>>>>> Performance data on microtests is looking good, however I wonder what > >>>>>> impact it has on DRLVM and large benchmarks. Haven't you filed JIRA > >>>>>> for this issue? If I'll have exact steps to build configuration with > >>>>>> new ICU, I could check it's performance before committing the changes > >>>>>> to the trunk. > >>>>>> > >>>>>> Aleksey. > >>>>>> > >>>>>> On Dec 13, 2007 8:08 PM, Oliver Deakin > >>>>>> <[EMAIL PROTECTED]> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>>> Thanks Aleksey. > >>>>>>> > >>>>>>> I have run the JUnit tests with the IBM VME and they pass without any > >>>>>>> new failures. I have also run the encoding/decoding test in > >>>>>>> HARMONY-3709 > >>>>>>> a number of times - in general, ICU4J 3.8.1 performs in that test as > >>>>>>> well or slightly better than the ICU4J 3.8 jar we are currently > >>>>>>> using. I > >>>>>>> have attached the results of the latest run I made [1]. > >>>>>>> > >>>>>>> Regards, > >>>>>>> Oliver > >>>>>>> > >>>>>>> [1] > >>>>>>> DECODING: > >>>>>>> Small Input: > >>>>>>> Decoding: GB18030 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 890.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 593.0 > >>>>>>> Small Input: > >>>>>>> Decoding: ISO-8859-1 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 328.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 328.0 > >>>>>>> Small Input: > >>>>>>> Decoding: UTF-8 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 344.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 360.0 > >>>>>>> Large Input: > >>>>>>> Decoding: GB18030 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 2110.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 1968.0 > >>>>>>> Large Input: > >>>>>>> Decoding: ISO-8859-1 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 157.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 156.0 > >>>>>>> Large Input: > >>>>>>> Decoding: UTF-8 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 735.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 719.0 > >>>>>>> > >>>>>>> ENCODING: > >>>>>>> Small Input: > >>>>>>> Encoding: GB18030 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 969.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 1063.0 > >>>>>>> Small Input: > >>>>>>> Encoding: ISO-8859-1 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 344.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 344.0 > >>>>>>> Small Input: > >>>>>>> Encoding: UTF-8 , 1000000 times > >>>>>>> Current ICU4J Milliseconds: 359.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 359.0 > >>>>>>> Large Input: > >>>>>>> Encoding: GB18030 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 7407.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 7297.0 > >>>>>>> Large Input: > >>>>>>> Encoding: ISO-8859-1 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 219.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 219.0 > >>>>>>> Large Input: > >>>>>>> Encoding: UTF-8 , 1000 times > >>>>>>> Current ICU4J Milliseconds: 625.0 > >>>>>>> ICU4J 3.8.1 Milliseconds: 610.0 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Aleksey Shipilev wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> Good news, Oliver! > >>>>>>>> > >>>>>>>> Anyway, basing on our previous experiences with ICU changes, we > >>>>>>>> might > >>>>>>>> first try how Harmony performs with new ICU onboard before making > >>>>>>>> this > >>>>>>>> change default. That's not only JUnit and other validation > >>>>>>>> suites, but > >>>>>>>> performance too (say, Dacapo and other benchmarks performance). > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Aleksey. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Dec 13, 2007 2:53 PM, Oliver Deakin > >>>>>>>> <[EMAIL PROTECTED]> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> ICU 3.8.1 has just been released, so Id like to propose that > >>>>>>>>> after M4 we > >>>>>>>>> upgrade to this release, add it to the fetch-depends target and > >>>>>>>>> remove > >>>>>>>>> the "homemade" ICU4J jar we have stored in SVN. > >>>>>>>>> > >>>>>>>>> Im running the tests with the new version now to make sure there > >>>>>>>>> are no > >>>>>>>>> regressions and will be happy to make the required changes when > >>>>>>>>> the time > >>>>>>>>> comes. > >>>>>>>>> > >>>>>>>>> Objections? > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> Oliver > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Oliver Deakin > >>>>>>>>> Unless stated otherwise above: > >>>>>>>>> IBM United Kingdom Limited - Registered in England and Wales > >>>>>>>>> with number 741598. > >>>>>>>>> Registered office: PO Box 41, North Harbour, Portsmouth, > >>>>>>>>> Hampshire PO6 3AU > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> Oliver Deakin > >>>>>>> Unless stated otherwise above: > >>>>>>> IBM United Kingdom Limited - Registered in England and Wales with > >>>>>>> number 741598. > >>>>>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > >>>>>>> PO6 3AU > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>> -- > >>>>> > >>>>> Oliver Deakin > >>>>> Unless stated otherwise above: > >>>>> IBM United Kingdom Limited - Registered in England and Wales with > >>>>> number 741598. > >>>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire > >>>>> PO6 3AU > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >> -- > >> Oliver Deakin > >> Unless stated otherwise above: > >> IBM United Kingdom Limited - Registered in England and Wales with number > >> 741598. > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > >> > >> > >> > > > > > > -- > > Oliver Deakin > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > >
