Hi, I was commenting the other day that ITK4 seems to have lower performance than ITK3. I put together a little benchmark to test this objectively. Results for a couple of systems are below. The benchmark is neither particularly sophisticated nor exact, but i have tried to make it typical of what i use ITK for. I have not tried it with Hans' patch as that got abandoned. Will follow up separately on that.
In the numbers below, you will see that performance seems to have degraded across the board. One exception is the 2D image test. In this test i was lazy, and i left the image axis aligned with unit spacing. I am suspecting a special case optimization in ITK4. At the moment, this blocks my ability to upgrade my projects to ITK4, as they are performance critical. Any hints and tips to get those milliseconds back will be much appreciated. The code is at https://github.com/rupertbrooks/itkbench, i'd be curious if others get the same results, and of course, criticism and improvements are welcome. Note that the output about what processor / cores is on the system is bogus. The windows system is a core i7 with 4 cores, the linux one is a Core2 Quad. I suspect a bug in itksys::SystemInformation, but i will follow up on that separately also. In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds, USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on. Cheers, Rupert Data follows..... itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo System: CAMD5C5PMN1 Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache Serial #: Cache: -1 Clock: 2800 Cores: 4 cpus x 1 Cores = 4 OSName: Windows Release: XP Professional Version: Service Pack 3 (Build 2600) Platform: x86 Operating System is 32 bit ITK Version: 3.20.0 Virtual Memory: Total: 2047 Available: 2021 Physical Memory: Total:3581 Available: 1457 Probe Name: Count Min Mean Stdev Max Total FilterChain_1_threads 10 0.0345688 0.0361805 0.0381523 0.0375099 0.361805 FilterChain_2_threads 10 0.0257301 0.0265144 0.0279574 0.0282478 0.265144 FilterChain_3_threads 10 0.022007 0.0232452 0.0245125 0.0240822 0.232452 FilterChain_4_threads 10 0.0199661 0.0221756 0.0235205 0.0288773 0.221756 Image2D 10 0.144909 0.14665 0.154586 0.148136 1.4665 Image3D 10 0.19125 0.193378 0.203846 0.196495 1.93378 MeanSquares_1_threads 10 0.612728 0.616159 0.649503 0.627789 6.16159 MeanSquares_2_threads 10 0.359703 0.368051 0.388106 0.392696 3.68051 MeanSquares_3_threads 10 0.268024 0.280672 0.296142 0.312557 2.80672 MeanSquares_4_threads 10 0.199249 0.213209 0.224918 0.2243 2.13209 itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo System: CAMD5C5PMN1 Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache Serial #: Cache: -1 Clock: 2800 Cores: 4 cpus x 1 Cores = 4 OSName: Windows Release: XP Professional Version: Service Pack 3 (Build 2600) Platform: x86 Operating System is 32 bit ITK Version: 4.3.0 Virtual Memory: Total: 2047 Available: 2008 Physical Memory: Total:3581 Available: 1574 Probe Name: Count Min Mean Stdev Max Total FilterChain_1_threads 10 0.0797005 0.0808155 0.0851898 0.0820313 0.808155 FilterChain_2_threads 10 0.0670509 0.0921925 0.0991914 0.109177 0.921925 FilterChain_3_threads 10 0.0648499 0.113566 0.120975 0.125893 1.13566 FilterChain_4_threads 10 0.0536919 0.104662 0.114558 0.132114 1.04662 Image2D 10 0.0953979 0.0965282 0.101753 0.0978088 0.965282 Image3D 10 0.196468 0.198288 0.209017 0.200562 1.98288 MeanSquares_1_threads 10 0.982071 0.986089 1.03944 0.996445 9.86089 MeanSquares_2_threads 10 0.686028 0.728621 0.768821 0.799675 7.28621 MeanSquares_3_threads 10 0.534885 0.554387 0.5846 0.578133 5.54387 MeanSquares_4_threads 10 0.441551 0.463463 0.488903 0.502502 4.63463 itk3 Linux 64 bit gcc 4.6.1 Build Release System: morrigan Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz Serial #: Cache: 2048 Clock: 2003 Cores: 4 cpus x 4 Cores = 16 OSName: Linux Release: 3.0.0-21-generic Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012 Platform: x86_64 Operating System is 64 bit ITK Version: 3.20.1 Virtual Memory: Total: 20795 Available: 20795 Physical Memory: Total:8001 Available: 3416 Probe Name: Count Min Mean Stdev Max Total FilterChain_1_threads 10 0.0309269 0.031097 0.0327796 0.031528 0.31097 FilterChain_2_threads 10 0.021672 0.0248219 0.0261976 0.026264 0.248219 FilterChain_3_threads 10 0.0213931 0.0217796 0.0229634 0.0228851 0.217796 FilterChain_4_threads 10 0.0198419 0.0207421 0.0218725 0.0216701 0.207421 Image2D 10 0.0263629 0.0264159 0.0278451 0.0267591 0.264159 Image3D 10 0.0263779 0.0263906 0.0278181 0.026463 0.263906 MeanSquares_1_threads 10 0.463909 0.465901 0.491113 0.473644 4.65901 MeanSquares_2_threads 10 0.349138 0.40529 0.429438 0.487785 4.0529 MeanSquares_3_threads 10 0.331908 0.357685 0.377834 0.400941 3.57685 MeanSquares_4_threads 10 0.299428 0.336914 0.356878 0.395144 3.36914 itk4 Linux 64-bit gcc 4.6.1 Build Release System: morrigan Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz Serial #: Cache: 2048 Clock: 2003 Cores: 4 cpus x 4 Cores = 16 OSName: Linux Release: 3.0.0-21-generic Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012 Platform: x86_64 Operating System is 64 bit ITK Version: 4.3.0 Virtual Memory: Total: 20795 Available: 20795 Physical Memory: Total:8001 Available: 3400 Probe Name: Count Min Mean Stdev Max Total FilterChain_1_threads 10 0.0663331 0.0668247 0.0704416 0.0681288 0.668247 FilterChain_2_threads 10 0.0353 0.0393441 0.042048 0.0534279 0.393441 FilterChain_3_threads 10 0.025275 0.0265869 0.0282408 0.0364931 0.265869 FilterChain_4_threads 10 0.0208249 0.0222288 0.0235333 0.028336 0.222288 Image2D 10 0.026361 0.026421 0.0278503 0.0266371 0.26421 Image3D 10 0.0263991 0.0266509 0.0280954 0.0274949 0.266509 MeanSquares_1_threads 10 0.69005 0.69459 0.732169 0.701676 6.9459 MeanSquares_2_threads 10 0.590396 0.699826 0.739994 0.766702 6.99826 MeanSquares_3_threads 10 0.527146 0.538844 0.568047 0.548791 5.38844 MeanSquares_4_threads 10 0.417008 0.454278 0.480139 0.533593 4.54278 -------------------------------------------------------------- Rupert Brooks [email protected]
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://www.itk.org/mailman/listinfo/insight-developers
