[
https://issues.apache.org/jira/browse/KYLIN-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-5274:
--------------------------------
Description:
h3. Background
The following code are called for each call of *_KylinConfig#getOptional_* ,
which needs to be optimized. In some case, it will improve query performance.
{code:java}
tected StrSubstitutor getSubstitutor() {
// env > properties
final Map<String, Object> all = Maps.newHashMap(); // create a new map
every time
all.putAll((Map) properties);
all.putAll(STATIC_SYSTEM_ENV);
return new StrSubstitutor(all);
} {code}
h3. How to fix
1. Not to create a new map each time.
2. Not to use Properties because it extends _Hashtable._
h3. Micro Benchmark
Use JMH to show performance(avg time):
{code:java}
import org.apache.kylin.common.util.NLocalFileMetadataTestCase;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Threads;
import org.openjdk.jmh.annotations.Warmup;
import java.util.concurrent.TimeUnit;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Warmup(iterations = 1)
@Measurement(iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
@Threads(1)
@Fork(value = 1, jvmArgs = {"-Xms2G", "-Xmx2G"})
@State(Scope.Benchmark)
public class KylinConfigBenchmark {
@Setup
public void setUp() throws Exception {
NLocalFileMetadataTestCase case1 = new NLocalFileMetadataTestCase();
case1.createTestMetadata();
}
@Benchmark
public void getProperty() {
KylinConfig config = KylinConfig.getInstanceFromEnv();
for(int i = 0; i<= 1000_000; i++ ){
config.getJdbcDriverClass();
}
}
public static void main(String[] args) throws Exception {
org.openjdk.jmh.Main.main(args);
}
} {code}
h4. Before Applied
!image-2022-10-08-17-35-34-500.png!
h4. After Applied
!image-2022-10-08-17-50-07-929.png!
was:
h3. Background
The following code are called for each call of *_KylinConfig#getOptional_* ,
which needs to be optimized. In some case, it will improve query performance.
{code:java}
tected StrSubstitutor getSubstitutor() {
// env > properties
final Map<String, Object> all = Maps.newHashMap(); // create a new map
every time
all.putAll((Map) properties);
all.putAll(STATIC_SYSTEM_ENV);
return new StrSubstitutor(all);
} {code}
h3. How to fix
1. Not to create a new map each time.
2. Not to use Properties because it extends _Hashtable._
h3. Micro Benchmark
Use JMH to show performance(avg time):
{code:java}
import org.apache.kylin.common.util.NLocalFileMetadataTestCase;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Threads;
import org.openjdk.jmh.annotations.Warmup;
import java.util.concurrent.TimeUnit;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Warmup(iterations = 1)
@Measurement(iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
@Threads(1)
@Fork(value = 1, jvmArgs = {"-Xms2G", "-Xmx2G"})
@State(Scope.Benchmark)
public class KylinConfigBenchmark {
@Setup
public void setUp() throws Exception {
NLocalFileMetadataTestCase case1 = new NLocalFileMetadataTestCase();
case1.createTestMetadata();
}
@Benchmark
public void getProperty() {
KylinConfig config = KylinConfig.getInstanceFromEnv();
for(int i = 0; i<= 1000_000; i++ ){
config.getJdbcDriverClass();
}
}
public static void main(String[] args) throws Exception {
org.openjdk.jmh.Main.main(args);
}
} {code}
h4. Before Applied
!image-2022-10-08-17-35-34-500.png!
h4. After Applied
> Improve performance of getSubstitutor
> -------------------------------------
>
> Key: KYLIN-5274
> URL: https://issues.apache.org/jira/browse/KYLIN-5274
> Project: Kylin
> Issue Type: Improvement
> Reporter: Xiaoxiang Yu
> Assignee: Zhong Yanghong
> Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-10-08-17-35-34-500.png,
> image-2022-10-08-17-50-07-929.png
>
>
> h3. Background
> The following code are called for each call of *_KylinConfig#getOptional_* ,
> which needs to be optimized. In some case, it will improve query performance.
>
> {code:java}
> tected StrSubstitutor getSubstitutor() {
> // env > properties
> final Map<String, Object> all = Maps.newHashMap(); // create a new map
> every time
> all.putAll((Map) properties);
> all.putAll(STATIC_SYSTEM_ENV);
> return new StrSubstitutor(all);
> } {code}
>
>
> h3. How to fix
> 1. Not to create a new map each time.
> 2. Not to use Properties because it extends _Hashtable._
> h3. Micro Benchmark
> Use JMH to show performance(avg time):
>
> {code:java}
> import org.apache.kylin.common.util.NLocalFileMetadataTestCase;
> import org.openjdk.jmh.annotations.Benchmark;
> import org.openjdk.jmh.annotations.BenchmarkMode;
> import org.openjdk.jmh.annotations.Fork;
> import org.openjdk.jmh.annotations.Measurement;
> import org.openjdk.jmh.annotations.Mode;
> import org.openjdk.jmh.annotations.OutputTimeUnit;
> import org.openjdk.jmh.annotations.Scope;
> import org.openjdk.jmh.annotations.Setup;
> import org.openjdk.jmh.annotations.State;
> import org.openjdk.jmh.annotations.Threads;
> import org.openjdk.jmh.annotations.Warmup;
> import java.util.concurrent.TimeUnit;
> @BenchmarkMode(Mode.AverageTime)
> @OutputTimeUnit(TimeUnit.MILLISECONDS)
> @Warmup(iterations = 1)
> @Measurement(iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
> @Threads(1)
> @Fork(value = 1, jvmArgs = {"-Xms2G", "-Xmx2G"})
> @State(Scope.Benchmark)
> public class KylinConfigBenchmark {
> @Setup
> public void setUp() throws Exception {
> NLocalFileMetadataTestCase case1 = new NLocalFileMetadataTestCase();
> case1.createTestMetadata();
> }
> @Benchmark
> public void getProperty() {
> KylinConfig config = KylinConfig.getInstanceFromEnv();
> for(int i = 0; i<= 1000_000; i++ ){
> config.getJdbcDriverClass();
> }
> }
> public static void main(String[] args) throws Exception {
> org.openjdk.jmh.Main.main(args);
> }
> } {code}
>
>
> h4. Before Applied
> !image-2022-10-08-17-35-34-500.png!
>
> h4. After Applied
> !image-2022-10-08-17-50-07-929.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
