[
https://issues.apache.org/jira/browse/KYLIN-4427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077872#comment-17077872
]
Xiaoxiang Yu commented on KYLIN-4427:
-------------------------------------
I am testing it at HDI 3.6. Following is cluster's template.
{code:json}
{
"$schema":
"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#";,
"contentVersion": "1.0.0.0",
"parameters": {
"clusters_kylin_system_cube_test_cluster_name": {
"defaultValue": "123",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.HDInsight/clusters",
"apiVersion": "2018-06-01-preview",
"name": "[parameters('123')]",
"location": "China East",
"tags": {
"Owner": "xiaoxiang",
"Cost Center": "",
"Period": "1 day"
},
"properties": {
"clusterVersion": "3.6.1000.67",
"osType": "Linux",
"clusterDefinition": {
"blueprint":
"https://blueprints.azurehdinsight.net/hbase-3.6.1000.67.2003240620.json";,
"kind": "HBASE",
"componentVersion": {
"HBase": "1.1"
}
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "standard_d3_v2"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "workernode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "standard_d4_v2"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "standard_d1_v2"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
}
]
},
"tier": "standard",
"storageProfile": {
"storageaccounts": [
{
"name": "123.blob.core.chinacloudapi.cn",
"container": "123",
"isDefault": true
}
]
}
}
}
]
}
{code}
I am using `apache-kylin-2.6.5-bin-hbase1x.tar.gz`, and modify kylin.properties:
{code:java}
kylin.metrics.monitor-enabled=true
kylin.metrics.reporter-job-enabled=true
kylin.metrics.reporter-query-enabled=true
kylin.web.dashboard-enabled=true
{code}
After kylin instance started, I see the logs:
{code:java}
2020-04-08 05:54:48,459 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:54:51,967 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:54:52,954 DEBUG [http-bio-7070-exec-7]
badquery.BadQueryHistoryManager:65 : Loaded 0 Bad Query(s)
2020-04-08 05:54:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:55:24,787 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:55:48,459 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:55:51,967 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:55:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:56:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:56:48,459 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:56:51,968 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:56:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:57:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:57:48,460 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:57:51,968 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:57:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:58:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:58:48,461 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:58:51,969 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:58:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:59:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 05:59:48,461 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 05:59:51,969 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 05:59:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 06:00:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 06:00:48,461 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:145 : The number of records in the blocking queue is
less than 10 and the duration from last reporting is less than 600000ms. Will
delay to report!
2020-04-08 06:00:51,969 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 06:00:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 06:01:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 06:01:48,462 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:444 : Trying to connect to metastore with URI
thrift://hn0-kylin.nert0r5niyaepiqzqwkbwnagve.bx.internal.chinacloudapp.cn:9083
2020-04-08 06:01:48,463 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:540 : Connected to metastore.
2020-04-08 06:01:48,929 INFO [metrics-blocking-reservoir-scheduler-0]
DependencyResolver:1106 : ivysettings.xml file not found in HIVE_HOME or
HIVE_CONF_DIR,/etc/hive/2.6.5.3015-8/0/ivysettings.xml will be used
2020-04-08 06:01:49,170 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:444 : Trying to connect to metastore with URI
thrift://hn1-kylin.nert0r5niyaepiqzqwkbwnagve.bx.internal.chinacloudapp.cn:9083
2020-04-08 06:01:49,171 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:540 : Connected to metastore.
2020-04-08 06:01:50,168 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:701 : Created local directory:
/home/sshuser/apache-kylin-2.6.5-bin-hbase1x/tomcat/temp/sshuser
2020-04-08 06:01:50,169 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:701 : Created local directory:
/home/sshuser/apache-kylin-2.6.5-bin-hbase1x/tomcat/temp/49f0ff10-87f0-49a9-b873-bd68f81836b0_resources
2020-04-08 06:01:50,255 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:701 : Created HDFS directory:
hdfs://mycluster/tmp/hive/sshuser/49f0ff10-87f0-49a9-b873-bd68f81836b0
2020-04-08 06:01:50,648 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:701 : Created local directory:
/home/sshuser/apache-kylin-2.6.5-bin-hbase1x/tomcat/temp/sshuser/49f0ff10-87f0-49a9-b873-bd68f81836b0
2020-04-08 06:01:50,690 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:701 : Created HDFS directory:
hdfs://mycluster/tmp/hive/sshuser/49f0ff10-87f0-49a9-b873-bd68f81836b0/_tmp_space.db
2020-04-08 06:01:50,704 INFO [metrics-blocking-reservoir-scheduler-0]
tez.TezSessionState:129 : Opening the session with id
49f0ff10-87f0-49a9-b873-bd68f81836b0 for thread
metrics-blocking-reservoir-scheduler-0 log trace id - query id -
2020-04-08 06:01:50,705 INFO [metrics-blocking-reservoir-scheduler-0]
tez.TezSessionState:145 : User of session id
49f0ff10-87f0-49a9-b873-bd68f81836b0 is sshuser
2020-04-08 06:01:50,774 INFO [metrics-blocking-reservoir-scheduler-0]
tez.DagUtils:898 : Jar dir is null/directory doesn't exist. Choosing
HIVE_INSTALL_DIR - /user/sshuser/.hiveJars
2020-04-08 06:01:50,983 INFO [metrics-blocking-reservoir-scheduler-0]
tez.DagUtils:724 : Resource modification time: 1586318675000
2020-04-08 06:01:51,076 INFO [metrics-blocking-reservoir-scheduler-0]
sqlstd.SQLStdHiveAccessController:95 : Created SQLStdHiveAccessController for
session context : HiveAuthzSessionContext
[sessionString=49f0ff10-87f0-49a9-b873-bd68f81836b0, clientType=HIVECLI]
2020-04-08 06:01:51,078 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:346 : Mestastore configuration hive.metastore.filter.hook
changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
2020-04-08 06:01:51,081 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:444 : Trying to connect to metastore with URI
thrift://hn0-kylin.nert0r5niyaepiqzqwkbwnagve.bx.internal.chinacloudapp.cn:9083
2020-04-08 06:01:51,081 INFO [metrics-blocking-reservoir-scheduler-0]
hive.metastore:540 : Connected to metastore.
2020-04-08 06:01:51,113 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClient:177 : Tez Client Version: [ component=tez-api,
version=0.7.0.2.6.5.3015-8, revision=99b800b802c2c9737efd7040342c266fd1a53ec5,
SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git,
buildTime=2019-12-05T13:33:09Z ]
2020-04-08 06:01:51,114 INFO [metrics-blocking-reservoir-scheduler-0]
tez.TezSessionState:193 : Opening new Tez Session (id:
49f0ff10-87f0-49a9-b873-bd68f81836b0, scratch dir:
hdfs://mycluster/tmp/hive/sshuser/_tez_session_dir/49f0ff10-87f0-49a9-b873-bd68f81836b0)
2020-04-08 06:01:51,242 INFO [metrics-blocking-reservoir-scheduler-0]
client.AHSProxy:42 : Connecting to Application History server at
headnodehost/10.0.0.15:10200
2020-04-08 06:01:51,248 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClient:351 : Session mode. Starting session.
2020-04-08 06:01:51,251 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClientUtils:173 : Using tez.lib.uris value from configuration:
/hdp/apps/2.6.5.3015-8/tez/tez.tar.gz
2020-04-08 06:01:51,341 INFO [metrics-blocking-reservoir-scheduler-0]
client.RequestHedgingRMFailoverProxyProvider:146 : Looking for the active RM in
[rm1, rm2]...
2020-04-08 06:01:51,347 INFO [metrics-blocking-reservoir-scheduler-0]
client.RequestHedgingRMFailoverProxyProvider:170 : Found active RM [rm2]
2020-04-08 06:01:51,398 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClient:122 : Tez system stage directory
hdfs://mycluster/tmp/hive/sshuser/_tez_session_dir/49f0ff10-87f0-49a9-b873-bd68f81836b0/.tez/application_1586317575543_0033
doesn't exist and is created
2020-04-08 06:01:51,970 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2020-04-08 06:01:52,003 INFO [metrics-blocking-reservoir-scheduler-0]
impl.YarnClientImpl:274 : Submitted application application_1586317575543_0033
2020-04-08 06:01:52,006 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClient:385 : The url to track the Tez Session:
http://hn1-kylin.nert0r5niyaepiqzqwkbwnagve.bx.internal.chinacloudapp.cn:8088/proxy/application_1586317575543_0033/
2020-04-08 06:01:54,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2020-04-08 06:01:55,641 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=Driver.run
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:55,642 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=compile
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:55,731 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:436 : We are setting the hadoop caller context from to
sshuser_20200408060155_3790fa71-111b-41e2-af31-15bd4a0d98f9
2020-04-08 06:01:55,737 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=parse
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:55,876 INFO [metrics-blocking-reservoir-scheduler-0]
parse.ParseDriver:190 : Parsing command: ALTER TABLE
KYLIN.HIVE_METRICS_JOB_EXCEPTION_QA ADD IF NOT EXISTS PARTITION
(kday_date='2020-04-08')
2020-04-08 06:01:56,718 INFO [metrics-blocking-reservoir-scheduler-0]
parse.ParseDriver:218 : Parse Completed
2020-04-08 06:01:56,719 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=parse start=1586325715737
end=1586325716719 duration=982 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:56,719 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=semanticAnalyze
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,340 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:486 : Semantic Analysis Completed
2020-04-08 06:01:57,340 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=semanticAnalyze start=1586325716719
end=1586325717340 duration=621 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,351 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:253 : Returning Hive schema: Schema(fieldSchemas:null,
properties:null)
2020-04-08 06:01:57,351 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=compile start=1586325715642
end=1586325717351 duration=1709 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,351 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:568 : We are resetting the hadoop caller context to
2020-04-08 06:01:57,351 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:173 : Concurrency mode is disabled, not creating a lock manager
2020-04-08 06:01:57,351 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,352 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:1420 : Setting caller context to query id
sshuser_20200408060155_3790fa71-111b-41e2-af31-15bd4a0d98f9
2020-04-08 06:01:57,352 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:1423 : Starting
command(queryId=sshuser_20200408060155_3790fa71-111b-41e2-af31-15bd4a0d98f9):
ALTER TABLE KYLIN.HIVE_METRICS_JOB_EXCEPTION_QA ADD IF NOT EXISTS PARTITION
(kday_date='2020-04-08')
2020-04-08 06:01:57,671 INFO [metrics-blocking-reservoir-scheduler-0]
impl.TimelineClientImpl:302 : Timeline service address:
http://headnodehost:8188/ws/v1/timeline/
2020-04-08 06:01:57,836 INFO [metrics-blocking-reservoir-scheduler-0]
hooks.ATSHook:114 : Created ATS Hook
2020-04-08 06:01:57,837 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,841 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1586325717837
end=1586325717841 duration=4 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,846 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=runTasks
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:57,859 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:1763 : Starting task [Stage-0:DDL] in serial mode
2020-04-08 06:01:58,085 INFO [ATS Logger 0] hooks.ATSHook:123 : ATS domain
created:hive_49f0ff10-87f0-49a9-b873-bd68f81836b0(sshuser,sshuser)
2020-04-08 06:01:58,420 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=runTasks start=1586325717846
end=1586325718420 duration=574 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,422 INFO [metrics-blocking-reservoir-scheduler-0]
hooks.ATSHook:114 : Created ATS Hook
2020-04-08 06:01:58,422 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG
method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,425 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG
method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1586325718422
end=1586325718425 duration=3 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,425 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:1655 : Resetting the caller context to
2020-04-08 06:01:58,425 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=Driver.execute start=1586325717351
end=1586325718425 duration=1074 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,426 INFO [metrics-blocking-reservoir-scheduler-0]
ql.Driver:1106 : OK
2020-04-08 06:01:58,427 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:149 : <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,427 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=releaseLocks start=1586325718427
end=1586325718427 duration=0 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,427 INFO [metrics-blocking-reservoir-scheduler-0]
log.PerfLogger:177 : </PERFLOG method=Driver.run start=1586325715641
end=1586325718427 duration=2786 from=org.apache.hadoop.hive.ql.Driver>
2020-04-08 06:01:58,443 INFO [metrics-blocking-reservoir-scheduler-0]
tez.TezSessionPoolManager:198 : Closing tez session default? false
2020-04-08 06:01:58,443 INFO [metrics-blocking-reservoir-scheduler-0]
tez.TezSessionState:294 : Closing Tez Session
2020-04-08 06:01:58,444 INFO [metrics-blocking-reservoir-scheduler-0]
client.TezClient:518 : Shutting down Tez Session,
sessionName=HIVE-49f0ff10-87f0-49a9-b873-bd68f81836b0,
applicationId=application_1586317575543_0033
2020-04-08 06:01:58,612 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:790 : Deleted directory:
hdfs://mycluster/tmp/hive/sshuser/49f0ff10-87f0-49a9-b873-bd68f81836b0 on fs
with scheme hdfs
2020-04-08 06:01:58,613 INFO [metrics-blocking-reservoir-scheduler-0]
session.SessionState:790 : Deleted directory:
/home/sshuser/apache-kylin-2.6.5-bin-hbase1x/tomcat/temp/sshuser/49f0ff10-87f0-49a9-b873-bd68f81836b0
on fs with scheme file
2020-04-08 06:01:58,752 ERROR [metrics-blocking-reservoir-scheduler-0]
hive.HiveReservoirReporter:119 : Append Support not enabled
java.lang.UnsupportedOperationException: Append Support not enabled
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.append(NativeAzureFileSystem.java:1587)
at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1188)
at
org.apache.kylin.metrics.lib.impl.hive.HiveProducer.write(HiveProducer.java:179)
at
org.apache.kylin.metrics.lib.impl.hive.HiveProducer.send(HiveProducer.java:123)
at
org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter$HiveReservoirListener.onRecordUpdate(HiveReservoirReporter.java:117)
at
org.apache.kylin.metrics.lib.impl.BlockingReservoir.notifyListenerOfUpdatedRecord(BlockingReservoir.java:105)
at
org.apache.kylin.metrics.lib.impl.BlockingReservoir.onRecordUpdate(BlockingReservoir.java:93)
at
org.apache.kylin.metrics.lib.impl.BlockingReservoir.access$300(BlockingReservoir.java:33)
at
org.apache.kylin.metrics.lib.impl.BlockingReservoir$ReporterRunnable.run(BlockingReservoir.java:152)
at java.lang.Thread.run(Thread.java:748)
2020-04-08 06:01:58,753 WARN [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:95 : It fails to notify listener
org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter$HiveReservoirListener@4a82f1b8
of updated records [{KTIME_HOUR=5, PROJECT=LEARN_KYLIN,
JOB_ID=ef1fb713-ee7e-82e0-a433-5ebbb92c45c8,
EVENT_TYPE=METRICS_JOB_EXCEPTION_QA,
EXCEPTION=org.apache.kylin.job.exception.ExecuteException,
HOST=hn0-kylin:10.0.0.16, KDAY_TIME=05:51:31, KTIMESTAMP=1586325091451,
CUBING_TYPE=NULL, CUBE_NAME=kylin_sales_cube, KDAY_DATE=2020-04-08,
KTIME_MINUTE=51, KTIME_SECOND=31, JOB_TYPE=BUILD, KWEEK_BEGIN_DATE=2020-04-05,
KYEAR_BEGIN_DATE=2020-01-01, KMONTH_BEGIN_DATE=2020-04-01, KUSER=ADMIN},
{KTIME_HOUR=5, PROJECT=LEARN_KYLIN,
JOB_ID=ef1fb713-ee7e-82e0-a433-5ebbb92c45c8,
EVENT_TYPE=METRICS_JOB_EXCEPTION_QA,
EXCEPTION=org.apache.kylin.job.exception.ExecuteException,
HOST=hn0-kylin:10.0.0.16, KDAY_TIME=05:54:45, KTIMESTAMP=1586325285724,
CUBING_TYPE=NULL, CUBE_NAME=kylin_sales_cube, KDAY_DATE=2020-04-08,
KTIME_MINUTE=54, KTIME_SECOND=45, JOB_TYPE=BUILD, KWEEK_BEGIN_DATE=2020-04-05,
KYEAR_BEGIN_DATE=2020-01-01, KMONTH_BEGIN_DATE=2020-04-01, KUSER=ADMIN}]
2020-04-08 06:01:58,753 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:109 : The HA listener
org.apache.kylin.metrics.lib.impl.StubReservoirReporter$StubReservoirListener@6e56abc5
for updated records [{KTIME_HOUR=5, PROJECT=LEARN_KYLIN,
JOB_ID=ef1fb713-ee7e-82e0-a433-5ebbb92c45c8,
EVENT_TYPE=METRICS_JOB_EXCEPTION_QA,
EXCEPTION=org.apache.kylin.job.exception.ExecuteException,
HOST=hn0-kylin:10.0.0.16, KDAY_TIME=05:51:31, KTIMESTAMP=1586325091451,
CUBING_TYPE=NULL, CUBE_NAME=kylin_sales_cube, KDAY_DATE=2020-04-08,
KTIME_MINUTE=51, KTIME_SECOND=31, JOB_TYPE=BUILD, KWEEK_BEGIN_DATE=2020-04-05,
KYEAR_BEGIN_DATE=2020-01-01, KMONTH_BEGIN_DATE=2020-04-01, KUSER=ADMIN},
{KTIME_HOUR=5, PROJECT=LEARN_KYLIN,
JOB_ID=ef1fb713-ee7e-82e0-a433-5ebbb92c45c8,
EVENT_TYPE=METRICS_JOB_EXCEPTION_QA,
EXCEPTION=org.apache.kylin.job.exception.ExecuteException,
HOST=hn0-kylin:10.0.0.16, KDAY_TIME=05:54:45, KTIMESTAMP=1586325285724,
CUBING_TYPE=NULL, CUBE_NAME=kylin_sales_cube, KDAY_DATE=2020-04-08,
KTIME_MINUTE=54, KTIME_SECOND=45, JOB_TYPE=BUILD, KWEEK_BEGIN_DATE=2020-04-05,
KYEAR_BEGIN_DATE=2020-01-01, KMONTH_BEGIN_DATE=2020-04-01, KUSER=ADMIN}] will
be started
2020-04-08 06:01:58,753 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:140 : There's no record in the blocking queue.
2020-04-08 06:02:24,785 INFO [FetcherRunner 935914837-51]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
{code}
> Wrong FileSystem error when trying to enable system cubes and Dashboard in
> Kylin 2.6.4
> --------------------------------------------------------------------------------------
>
> Key: KYLIN-4427
> URL: https://issues.apache.org/jira/browse/KYLIN-4427
> Project: Kylin
> Issue Type: Bug
> Components: Metrics
> Affects Versions: v2.6.4
> Reporter: Preeti V
> Assignee: Xiaoxiang Yu
> Priority: Major
> Attachments: KylinMetrics.JPG, image-2020-04-03-10-45-15-290.png,
> image-2020-04-03-10-45-20-859.png, image-2020-04-03-10-45-45-894.png,
> image-2020-04-06-15-46-37-356.png, image-2020-04-06-17-14-52-443.png
>
>
> I am trying to enable system cubes for the Dashboard using Kylin version
> 2.6.4 The tables are created correctly and the cube builds successfully but
> there is no query or job data on the dashboard, it shows 0.
>
> We use Azure storage for Hive(wasb:// file system). I can see that there is
> no data being updated on the Hive_Metrics tables in Azure. In Kylin logs I
> see the below error
>
>
> 2020-03-12 20:02:41,790 ERROR [metrics-blocking-reservoir-scheduler-0]
> hive.HiveReservoirReporter:119 : Wrong FS:
> wasb://*****.blob.core.windows.net/hive/warehouse/kylin.db/hive_metrics_query_cube_qa/kday_date=2020-03-12,
> expected: hdfs://*****-prod-bn01
> java.lang.IllegalArgumentException: Wrong FS:
> wasb://*****.blob.core.windows.net/hive/warehouse/kylin.db/hive_metrics_query_cube_qa/kday_date=2020-03-12,
> expected: hdfs://*****-prod-bn01
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:666)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:214)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1442)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$26.doCall(DistributedFileSystem.java:1438)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1454)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1448)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.write(HiveProducer.java:137)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.send(HiveProducer.java:122)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter$HiveReservoirListener.onRecordUpdate(HiveReservoirReporter.java:117)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.notifyListenerOfUpdatedRecord(BlockingReservoir.java:105)
>
>
> I checked the hive configs and it has the warehouse metastore dir correctly
> pointing to azure. I found another thread with similar problem where they are
> trying to use S3 instead of hdfs.
> [http://apache-kylin.74782.x6.nabble.com/jira-Created-KYLIN-4385-KYLIN-system-cube-failing-to-update-table-when-run-on-EMR-with-S3-as-storageS-td14234.html]
>
>
> I also followed the recommendations here
> [https://www.mail-archive.com/[email protected]/msg04347.html] and
> enabled all the necessary config values.
> Is this a bug in Kylin or a configuration issue on my cluster? Any help or
> guidance is appreciated.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
