Hi, all: The previous experiment uses 3 huawei ecs instances as workers each with 16 cores and 32GB. Spark executor use 12 cores and 24GB. Using 74GB LineItem in 100GB TPCH.
Today I run another experiment using 1 huawei RH2288 machine with 32 cores and 128GB. Spark executor use 30 cores and 90GB. Using 7.3GB LineItem in 10GB TPCH. And the results are as below: Code Branch Prefetch Batch Size (default 100) Load1 (s) Load2 (s) Load3 (s) Compact 3 Loads (s) Time Reduced Perf Enhanced master NA 100 147.4 142.3 144.6 201.4 Baseline Baseline master NA 32000 140.8 138.7 141.6 196.2 2.6% 2.7% PR2906 enable 100 143.9 142.5 146.2 99.9 50.4% 101.6% PR2906 enable 32000 142.1 139.3 136.9 98.3 51.2% 104.9% PR2906 disable 100 146.7 137.4 139.6 200.6 0.4% 0.4% PR2906 disable 32000 145.2 145.0 139.7 195.7 2.8% 2.9% It also shows this PR will not decrease the compaction performance if disabled and will enhance the performance if enabled. -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
