Active cube run set to pending and a new cube request is run without finishing
I have obeserved that when the "kylin.job.max-concurrent-jobs" properties is used for a cluster configuration and set to for e.g. lets say value 1. Then for a given cube building job cluster, during a cube build if another cube build request is submitted the currently running job is set to pending and Kylin starts running the new job on the same cluster. Then again it might pause the running job and any step and then return to the execution of the first job. This is usually observed after the first step but can be random. Is this expected behavior? The good thing is ultimately both jobs run to success. Please share any knowledge on this. thanks! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Can cube be queried while it is refreshing
Can a cube be queried while it is refreshing? If yes, how does it handle the switch over to the refreshed data? Can that involve downtime? Thanks! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
High cardinality column search very slow
I have a column with a count distinct of about 12million. This is an integer
field (for e.g. call it account with just numbers in it).
This is account drives a lot of reports where the user enters the account in
a search box on tableau and then tableau sends a query like:
SELECT account ,
SUM(earnings)
FROM sales
WHERE (POSITION('538807' IN LOWER(account)) > 0)
GROUP BY 1
;
This search is slow. It takes about 15 to 30 seconds or more.
The same when implemented on a column with defined as varchar is rather
fast.
Can you please help understand what is going on?
thanks in advance
--
Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: Re:Cube migration across hadoop clusters: What is -updateMappingPath updateTableMapping.json
Hi Zhichao, thank you for your reply. Thanks for providing the file format as it is not available anywhere. I am not sure on what hadoop jars are missing. I am doing the cube migration on Kylin 3.1.1 on AWS EMR 5.29. And I am following the Install Kylin on AWS EMR instructions. Here is a new log: Can you please check and may be tell what might be missing: Thanks in advance. [hadoop@ip-10-0-1-141 apache-kylin-3.1.1-bin-hbase1x]$ ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-10-0-1-141.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-10-0-1-231.ec2.internal:7070 -cube cube_SAR_EXEC_Cmpny_SAR_Account_3_DNB_Cmpny_Search -execute true -overwrite true -schemaOnly false Using cached dependency... skip flink_dependency OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/kylin/apache-kylin-3.1.1-bin-hbase1x/tool/kylin-tool-3.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Running org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-10-0-1-141.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-10-0-1-231.ec2.internal:7070 -cube cube_SAR_EXEC_Cmpny_SAR_Account_3_DNB_Cmpny_Search -execute true -overwrite true -schemaOnly false 2021-02-27 11:29:33,204 INFO [main] common.KylinConfig:118 : Loading kylin-defaults.properties from file:/usr/local/kylin/apache-kylin-3.1.1-bin-hbase1x/tool/kylin-tool-3.1.1.jar!/kylin-defaults.properties 2021-02-27 11:29:33,222 INFO [main] common.KylinConfig:352 : Use KYLIN_HOME=/usr/local/kylin/apache-kylin-3.1.1-bin-hbase1x 2021-02-27 11:29:33,224 INFO [main] common.KylinConfig:153 : Initialized a new KylinConfig from getInstanceFromEnv : 245765246 2021-02-27 11:29:35,139 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x2532b351 connecting to ZooKeeper ensemble=ip-10-0-1-141.ec2.internal:2181 2021-02-27 11:29:35,144 INFO [main] zookeeper.ZooKeeper:100 : Client environment:zookeeper.version=3.4.14-5f788d5e8f90539ee331702c753fa250727128f4, built on 12/14/2019 00:35 GMT 2021-02-27 11:29:35,144 INFO [main] zookeeper.ZooKeeper:100 : Client environment:host.name=ip-10-0-1-141.ec2.internal 2021-02-27 11:29:35,144 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.version=1.8.0_272 2021-02-27 11:29:35,144 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.vendor=Red Hat, Inc. 2021-02-27 11:29:35,144 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.56.amzn1.x86_64/jre 2021-02-27 11:29:35,145 INFO [main] zookeeper.ZooKeeper:100 : Client
Cube migration across hadoop clusters: What is -updateMappingPath updateTableMapping.json
Per documentation: http://kylin.apache.org/docs/tutorial/cube_migration.html In the example for "II. Migrate across two Hadoop clusters" ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:ky...@qa.env:17070 -kylinUriDst ADMIN:ky...@prod.env:1 -cube kylin_sales_cube -updateMappingPath $KYLIN_HOME/updateTableMapping.json -execute true -schemaOnly false -overwrite true What is the updateTableMapping.json file? I have not been able to get any information on what this file should be. Is this a file to be created by us? can I have a sample? I am encountering error in trying to copy a cube with cube data to another hadoop cluster. Here is the log: Command without -updateMappingPath: ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-85-18.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-94-63.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true -schemaOnly false /*removed previous log*/ 2021-02-26 16:48:30,274 INFO [Cube-kylin_sales_cube-data-migration-pool-0] migration.CubeMigrationCrossClusterCLI:695 : start to copy hdfs directory from hdfs://ip-172-31-85-18.ec2.internal:8020/kylin/kylin_metadata/kylin-8dbe2fbb-7960-c2a5-3a20-ba26958f9691 to hdfs://ip-172-31-94-63.ec2.internal:8020/kylin/kylin_metadata/kylin-8dbe2fbb-7960-c2a5-3a20-ba26958f9691 2021-02-26 16:48:30,435 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:30,435 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:30,659 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:30,659 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:30,877 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.SimpleCopyListing:589 : Paths (files+dirs) cnt = 71; dirCnt = 15 2021-02-26 16:48:30,877 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.SimpleCopyListing:384 : Build file listing completed. 2021-02-26 16:48:30,879 INFO [Cube-kylin_sales_cube-data-migration-pool-0] Configuration.deprecation:1285 : io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb 2021-02-26 16:48:30,879 INFO [Cube-kylin_sales_cube-data-migration-pool-0] Configuration.deprecation:1285 : io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor 2021-02-26 16:48:31,370 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.DistCp:94 : Number of paths in the copy list: 71 2021-02-26 16:48:31,400 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.DistCp:94 : Number of paths in the copy list: 71 2021-02-26 16:48:31,448 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:31,486 WARN [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobResourceUploader:64 : Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 2021-02-26 16:48:32,227 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.SimpleCopyListing:589 : Paths (files+dirs) cnt = 11; dirCnt = 5 2021-02-26 16:48:32,227 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.SimpleCopyListing:384 : Build file listing completed. 2021-02-26 16:48:32,243 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.DistCp:94 : Number of paths in the copy list: 11 2021-02-26 16:48:32,259 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.DistCp:94 : Number of paths in the copy list: 11 2021-02-26 16:48:32,306 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-85-18.ec2.internal/172.31.85.18:8032 2021-02-26 16:48:32,311 WARN [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobResourceUploader:64 : Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 2021-02-26 16:48:32,367 INFO [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobSubmitter:200 : number of splits:6 2021-02-26 16:48:32,398 INFO [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobSubmitter:200 : number of splits:26 2021-02-26 16:48:32,490 INFO [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobSubmitter:289 : Submitting tokens for job: job_1614247321131_0022 2021-02-26 16:48:32,491 INFO [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobSubmitter:289 : Submitting tokens for job: job_1614247321131_0023 2021-02-26 16:48:32,604 INFO
Re: Cube migration across hadoop clusters
Thanks Zhichao, I tried that and am now getting this: [hadoop@ip-172-31-81-255 apache-kylin-3.1.0-bin-hbase1x]$ [hadoop@ip-172-31-81-255 apache-kylin-3.1.0-bin-hbase1x]$ ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-81-255.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-91-198.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true -schemaOnly false Using cached dependency... OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Running org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-81-255.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-91-198.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true -schemaOnly false 2021-02-26 10:06:54,000 INFO [main] common.KylinConfig:118 : Loading kylin-defaults.properties from file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/kylin-defaults.properties 2021-02-26 10:06:54,028 DEBUG [main] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable 2021-02-26 10:06:54,029 INFO [main] common.KylinConfig:352 : Use KYLIN_HOME=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-26 10:06:54,031 INFO [main] common.KylinConfig:153 : Initialized a new KylinConfig from getInstanceFromEnv : 245765246 2021-02-26 10:06:56,326 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x545607f2 connecting to ZooKeeper ensemble=ip-172-31-81-255.ec2.internal:2181 2021-02-26 10:06:56,331 INFO [main] zookeeper.ZooKeeper:100 : Client environment:zookeeper.version=3.4.14-5f788d5e8f90539ee331702c753fa250727128f4, built on 12/14/2019 00:35 GMT 2021-02-26 10:06:56,331 INFO [main] zookeeper.ZooKeeper:100 : Client environment:host.name=ip-172-31-81-255.ec2.internal 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.version=1.8.0_272 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.vendor=Red Hat, Inc. 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.56.amzn1.x86_64/jre 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.class.path= /*removed*/ 2021-02-26 10:06:56,333 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native 2021-02-26 10:06:56,333 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.io.tmpdir=/tmp 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.compiler= 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.name=Linux 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.arch=amd64 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.version=4.14.154-99.181.amzn1.x86_64 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.name=hadoop 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.home=/home/hadoop 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.dir=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-26 10:06:56,336 INFO [main] zookeeper.ZooKeeper:442 : Initiating client connection, connectString=ip-172-31-81-255.ec2.internal:2181 sessionTimeout=9 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@7207cb51 2021-02-26 10:06:56,357 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1025 : Opening socket connection to server ip-172-31-81-255.ec2.internal/172.31.81.255:2181. Will not attempt to authenticate using SASL (unknown error) 2021-02-26 10:06:56,363 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:879 : Socket connection established to ip-172-31-81-255.ec2.internal/172.31.81.255:2181, initiating session 2021-02-26 10:06:56,369 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1299 : Session establishment complete on server
Re: Cube migration across hadoop clusters
hi Yaqian, thanks for the response. I am using this method from the documentation: "II. Migrate across two Hadoop clusters" using Kylin version 3.1.0 on EMR 5.29 on both source and destination clusters. I tried in 2 ways: (1) Migrate from one kylin instance on emr 5.29 to another kylin instance on emr 5.29 with the Hbase S3 location same on both. (2) Migrate from one kylin instance on emr 5.29 to another kylin instance on emr 5.29 with Hbase s3 location different. In both ways the cube metadata/definition was migrated but not the cube data. Here is the log for (1): [hadoop@ip-172-31-90-136 apache-kylin-3.1.0-bin-hbase1x]$ ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-90-136.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-81-180.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true Using cached dependency... OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Running org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-90-136.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-81-180.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true 2021-02-25 09:36:50,551 INFO [main] common.KylinConfig:118 : Loading kylin-defaults.properties from file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/kylin-defaults.properties 2021-02-25 09:36:50,576 DEBUG [main] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable 2021-02-25 09:36:50,579 INFO [main] common.KylinConfig:352 : Use KYLIN_HOME=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-25 09:36:50,582 INFO [main] common.KylinConfig:153 : Initialized a new KylinConfig from getInstanceFromEnv : 245765246 2021-02-25 09:36:55,215 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x5d3b58ca connecting to ZooKeeper ensemble=ip-172-31-90-136.ec2.internal:2181 2021-02-25 09:36:55,224 INFO [main] zookeeper.ZooKeeper:100 : Client environment:zookeeper.version=3.4.14-5f788d5e8f90539ee331702c753fa250727128f4, built on 12/14/2019 00:35 GMT 2021-02-25 09:36:55,224 INFO [main] zookeeper.ZooKeeper:100 : Client environment:host.name=ip-172-31-90-136.ec2.internal 2021-02-25 09:36:55,225 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.version=1.8.0_272 2021-02-25 09:36:55,225 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.vendor=Red Hat, Inc. 2021-02-25 09:36:55,225 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.56.amzn1.x86_64/jre 2021-02-25 09:36:55,225 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.class.path=/*removed*/ 2021-02-25 09:36:55,226 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native 2021-02-25 09:36:55,227 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.io.tmpdir=/tmp 2021-02-25 09:36:55,227 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.compiler= 2021-02-25 09:36:55,227 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.name=Linux 2021-02-25 09:36:55,228 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.arch=amd64 2021-02-25 09:36:55,229 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.version=4.14.154-99.181.amzn1.x86_64 2021-02-25 09:36:55,229 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.name=hadoop 2021-02-25 09:36:55,229 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.home=/home/hadoop 2021-02-25 09:36:55,229 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.dir=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-25 09:36:55,230 INFO [main] zookeeper.ZooKeeper:442 : Initiating client connection, connectString=ip-172-31-90-136.ec2.internal:2181 sessionTimeout=9 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@7e91ed74 2021-02-25 09:36:55,289 INFO [main-SendThread(ip-172-31-90-136.ec2.internal:2181)] zookeeper.ClientCnxn:1025 : Opening socket connection to server ip-172-31-90-136.ec2.internal/172.31.90.136:2181. Will not attempt to authenticate
Cube migration across hadoop clusters
Per: http://kylin.apache.org/docs/tutorial/cube_migration.html It says: "Support to migrate cube data (segments data on HBase) from QA env to PROD env." I tried the migration but it only migrates the cube metadata without the data. The source kylin instance has the cube buiilt and in ready state. After I executed the command it said it was successful. But when I checked the destination kylin instance it only got the cube definition and not the data. Can you please clarify if migrating the data is possible or not? Or is the migration of the data a separate activity which involves the hbase snapshot import export. thanks in advance -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Kylin Cluster mode. SQL works in standalone mode. Errors in Cluster mode. No dictionary found.
I installed kylin on EMR in standalone mode. Ran a bunch of SQLs and they run fine. Then I add a query cluster. So I have kylin.server.mode in one cluster set to "all" and one cluster in "query" mode. I set kylin.server.cluster-servers=jobnode1:7070,querynode1:7070 on both query and job cluster. Restart kylin on both Then I get this: No dictionary found by /dict/DEFAULT.KYLIN_ACCOUNT/ACCOUNT_COUNTRY/1196ba3d-22e6-0337-cfe8-aaceb02d81dd.dict, invalid cube state; cube segmentsales_cube1[220100_2012040100], col BUYER_ACCOUNT:DEFAULT.KYLIN_ACCOUNT.ACCOUNT_COUNTRY while executing SQL: "SELECT ACCOUNT_COUNTRY, SUM(PRICE) , COUNT(*) FROM KYLIN_SALES JOIN KYLIN_ACCOUNT ON KYLIN_SALES.BUYER_ID = KYLIN_ACCOUNT.ACCOUNT_ID GROUP BY ACCOUNT_COUNTRY LIMIT 200" is there some other setting that needs to be done which is not mentioned on http://kylin.apache.org/docs/install/kylin_cluster.html -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Kylin Cluster Mode with EMR. Is a restart of kylin required on all job and query servers when new node is added.
Kylin Cluster Mode. I am using Kylin on EMR. Is a restart of kylin required on all job and query servers when new node is added. Then there will be downtime. How do we achieve High Availibility. from the guide: " Configure the Kylin node list kylin.server.cluster-servers, including all nodes (the current node is also included). When the event changes, the node receiving the change needs to notify all other nodes (the current node is also included). " So if I have 2 job servers and 2 query servers: then kylin.server.cluster-servers=jobnode1:7070,jobnode2:7070,querynode1:7070,querynode2:7070 If let's say query node 2 on EMR fails. And I add a new node with new EMR which will have a new IP. Then for the new node: kylin.server.cluster-servers=jobnode1:7070,jobnode2:7070,querynode1:7070,querynode3:7070 And then looks like I have to change the string on all the nodes to: kylin.server.cluster-servers=jobnode1:7070,jobnode2:7070,querynode1:7070,querynode3:7070 And then restart kylin on all nodes. This will incur downtime. Can you please help me understand how High Availability can be achieved then. Thanks in advance! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: Cluster mode - High availability. Metadata storage options other than MySQL?
thanks! What about something like dynamo db? -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Cluster mode - High availability. Metadata storage options other than MySQL?
What are the recommended High availability options. >From the apache kylin website it recommends the cluster mode installation and then there is an option to use MySql for metadata storage. MySQL being an RDBMS is not best suited for an HA architecture. Any other options that work with Apache Kylin? Thanks! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Cluster mode kylin.metadata.url and kylin.server.cluster-servers examples for AWS EMR
Hi I am trying to install Kylin in cluster mode on AWS EMRs using the documentation: http://kylin.apache.org/docs/install/kylin_cluster.html Can you please help me with example values of kylin.metadata.url and kylin.server.cluster-servers so I know the format. thanks! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: error while using left join
We are facing the same issue in tableau when we give a filter on the left join. It is converting to inner join. Can you please let us know if there is any solution for this? Thanks! -- Sent from: http://apache-kylin.74782.x6.nabble.com/
jdbc connection not working.
I have attempted to connect to Kylin using jdbc driver for the following Kylin versions 2.6, 3.0.2 and 3.1 and have been unable to make connection. The driver files I have used are from the '$KYLIN_HOME/lib' directory: for version 3.1: kylin-jdbc-3.1.0.jar, jcl-over-slf4j-1.7.21.jar, slf4j-api-1.7.21.jar for version 2.6.6: kylin-jdbc-2.6.6.jar for version 3.0.2: kylin-jdbc-3.0.2.jar Connection URL I am using: jdbc:kylin://"EMR cluster hostname":7070/learn_kylin JDBC Driver class: org.apache.kylin.jdbc.Driver Will appreciate any insight into what might be going wrong. Error logs: 38f9d37464b4% javac ky.java 38f9d37464b4% java ky Connecting to database... SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. java.sql.SQLException: org.apache.kylin.jdbc.shaded.org.apache.http.conn.HttpHostConnectException: Connection to http://ec2-3-236-65-136.compute-1.amazonaws.com:7070 refused at org.apache.kylin.jdbc.KylinConnection.(KylinConnection.java:72) at org.apache.kylin.jdbc.KylinJdbcFactory.newConnection(KylinJdbcFactory.java:77) at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at ky.main(ky.java:22) Caused by: org.apache.kylin.jdbc.shaded.org.apache.http.conn.HttpHostConnectException: Connection to http://ec2-3-236-65-136.compute-1.amazonaws.com:7070 refused at org.apache.kylin.jdbc.shaded.org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.kylin.jdbc.KylinClient.connect(KylinClient.java:285) at org.apache.kylin.jdbc.KylinConnection.(KylinConnection.java:70) ... 5 more Caused by: java.net.ConnectException: Operation timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:606) at org.apache.kylin.jdbc.shaded.org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.kylin.jdbc.shaded.org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ... 13 more Goodbye! 38f9d37464b4% javac ky.java 38f9d37464b4% java ky Connecting to database... SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. java.sql.SQLException: java.net.ConnectException: Operation timed out (Connection timed out) at org.apache.kylin.jdbc.KylinConnection.(KylinConnection.java:72) at org.apache.kylin.jdbc.KylinJdbcFactory.newConnection(KylinJdbcFactory.java:77) at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at ky.main(ky.java:22) Caused by: java.net.ConnectException: Operation timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
Build Dimension Dictionary: org.apache.kylin.engine.mr.exception.HadoopShellException: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast to org.apache.hadoop.io.LongWritabl
Hi, I am getting this error during cube build, can anyone please help? Thanks!!!: #4 Step Name: Build Dimension Dictionary Duration: 0.04 mins Waiting: 0 seconds org.apache.kylin.engine.mr.exception.HadoopShellException: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast to org.apache.hadoop.io.LongWritable at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableLongObjectInspector.getPrimitiveJavaObject(WritableLongObjectInspector.java:46) at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:276) at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:197) at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53) at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97) at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204) at org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107) at org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90) at org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98) at org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:133) at org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1116) at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1032) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:90) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:49) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:71) at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) result code:2 at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: Cube build failure: org/apache/spark/api/java/function/Function
Hi Shaofeng, Thank you very much for your reply. Can I please know how I can delete the segments? Are the overlapping segments causing the error? I get the overlapping segments error when for example I submit two date ranges as follows: 2020-05-04 to 2020-05-06 and 2020-05-05 to 2020-05-06. And if I were to give new date range that I have not used in the past e.g 2020-05-08 to 2020-05-09 then I get "Oops!!! org/apache/spark/api/java/function/Function" Please let me know. thanks -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Cube build failure: org/apache/spark/api/java/function/Function
I am following instructions to build cube with spark from: http://kylin.apache.org/docs/tutorial/cube_spark.html I get the following error when building cube: Cube build failure: org/apache/spark/api/java/function/Function Can you please help? Here is the log: 2020-06-06 14:58:38,289 ERROR [http-bio-7070-exec-2] controller.CubeController:398 : Segments overlap: kylin_sales_cube[2012010100_2015010100] and kylin_sales_cube[2014050100_2020050200] java.lang.IllegalStateException: Segments overlap: kylin_sales_cube[2012010100_2015010100] and kylin_sales_cube[2014050100_2020050200] at org.apache.kylin.metadata.model.Segments.validate(Segments.java:433) at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:385) at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:298) at org.apache.kylin.cube.CubeManager$SegmentAssist.appendSegment(CubeManager.java:739) at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:659) at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:651) at org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:233) at org.apache.kylin.rest.service.JobService.submitJob(JobService.java:202) at org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:395) at org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:355) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:653) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at
Re: No tables on KYLIN WEB UI and NoSuchObjectException(message:default.kylin_intermediate_* table not found) when building cube
Thank you very much for your response Yaqian. So, regarding the question where tables are not showing, I tested this. I created a hive external table on table data in S3 location (parquet file). Then I created another table out of it using CTAS so that the new table is part of the same hive database and has storage in hdfs; however even after doing that the new tables are not showing in the DEFAULT database which is the one I installed with what came with the sample.sh file. So, how do I get table available in the Kylin UI for usage when I am adding new tables in hive. Thanks in advance. -- Sent from: http://apache-kylin.74782.x6.nabble.com/
Re: subscribe kylin
I have subscribed but none of my posts are going through. How come your post of subscribe was accepted whereas none of my posts are getting through. I am not sure if this reply is going to make it. -- Sent from: http://apache-kylin.74782.x6.nabble.com/
MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refuse
I am a beginner. I installed Apache Kylin on EMR per instructions on the kylin website and am at my first step in the tutorials which is building the sample kylin_sales cube. When I try to build I keep getting this error and the second step "Redistribute Flat Hive Table": MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:480) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:247) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:129) at org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:165) at org.apache.kylin.source.hive.CLIHiveClient.getHiveTableRows(CLIHiveClient.java:129) at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.computeRowCount(RedistributeFlatHiveTableStep.java:38) at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.doWork(RedistributeFlatHiveTableStep.java:81) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 14 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:529) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:247) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:129) at org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:165) at org.apache.kylin.source.hive.CLIHiveClient.getHiveTableRows(CLIHiveClient.java:129) at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.computeRowCount(RedistributeFlatHiveTableStep.java:38) at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.doWork(RedistributeFlatHiveTableStep.java:81) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- Sent from: http://apache-kylin.74782.x6.nabble.com/
