Hi,
I am not able to sync the table from hive to Kylin data source.
I have a table kylin_cpg_dg in the hive:
[cid:002da0b1-411f-4e83-b8ca-231963faaa22]
I am not able to sync this in Kylin
[cid:50741009-f1ee-46bc-be31-e07cc404b2ee]
here are my logs from three servers in HA mode.
1)
2019-07-17 21:20:16,885 DEBUG [http-bio-7070-exec-76]
security.KylinAuthenticationProvider:126 : Authenticated user
org.springframework.security.authentication.UsernamePasswordAuthenticationToken@3704b3d2:
Principal: org.springframework.security.core.userdetails.User@3b40b2f:
Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER; Credentials: [PROTECTED]; Authenticated:
true; Details:
org.springframework.security.web.authentication.WebAuthenticationDetails@ffff8868:
RemoteIpAddress: 24.30.233.234; SessionId: null; Granted Authorities:
ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER
2019-07-17 21:20:16,887 DEBUG [http-bio-7070-exec-76]
controller.UserController:52 : User login:
org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN;
Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2019-07-17 21:20:16,889 DEBUG [http-bio-7070-exec-62] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 21:20:16,890 DEBUG [http-bio-7070-exec-62] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 21:20:18,606 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:20:20,900 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:20:24,432 DEBUG [http-bio-7070-exec-34]
project.ProjectL2Cache:199 : Loading L2 project cache for learn_kylin
2019-07-17 21:20:24,433 WARN [http-bio-7070-exec-34]
realization.RealizationRegistry:91 : No provider for realization type
INVERTED_INDEX
2019-07-17 21:20:24,433 WARN [http-bio-7070-exec-34]
realization.RealizationRegistry:91 : No provider for realization type
INVERTED_INDEX
2019-07-17 21:20:24,434 WARN [http-bio-7070-exec-34]
realization.RealizationRegistry:91 : No provider for realization type
INVERTED_INDEX
2019-07-17 21:20:26,681 INFO [http-bio-7070-exec-34] hive.metastore:407 :
Trying to connect to metastore with URI thrift://ip-10-0-0-46.ec2.internal:9083
2019-07-17 21:20:26,682 INFO [http-bio-7070-exec-34] hive.metastore:481 :
Opened a connection to metastore, current connections: 4
2019-07-17 21:20:26,684 INFO [http-bio-7070-exec-34] hive.metastore:534 :
Connected to metastore.
2019-07-17 21:20:27,833 INFO [http-bio-7070-exec-34] hive.metastore:407 :
Trying to connect to metastore with URI thrift://ip-10-0-0-45.ec2.internal:9083
2019-07-17 21:20:27,834 INFO [http-bio-7070-exec-34] hive.metastore:481 :
Opened a connection to metastore, current connections: 5
2019-07-17 21:20:27,835 INFO [http-bio-7070-exec-34] hive.metastore:534 :
Connected to metastore.
2019-07-17 21:20:50,900 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:21:18,606 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:21:20,922 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:21:50,900 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:21:54,153 DEBUG [http-bio-7070-exec-34] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 21:21:54,155 DEBUG [http-bio-7070-exec-34] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 21:22:18,607 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:22:20,899 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:22:50,920 INFO [FetcherRunner 352123462-50]
threadpool.DefaultFetcherRunner:96 : Job Fetcher: 0 should running, 0 actual
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-07-17 21:23:18,607 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2)
2019-07-17 21:23:16,821 INFO [http-bio-7070-exec-75] common.KylinConfig:332 :
Loading kylin-defaults.properties from
file:/usr/local/kylin/apache-kylin-2.5.0-bin-hbase1x/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
2019-07-17 21:23:16,836 DEBUG [http-bio-7070-exec-75] common.KylinConfig:291 :
KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2019-07-17 21:23:22,976 DEBUG [http-bio-7070-exec-86]
badquery.BadQueryHistoryManager:65 : Loaded 0 Bad Query(s)
2019-07-17 21:23:27,717 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:23:30,468 INFO [http-bio-7070-exec-86] hive.metastore:407 :
Trying to connect to metastore with URI thrift://ip-10-0-0-46.ec2.internal:9083
2019-07-17 21:23:30,470 INFO [http-bio-7070-exec-86] hive.metastore:481 :
Opened a connection to metastore, current connections: 4
2019-07-17 21:23:30,471 INFO [http-bio-7070-exec-86] hive.metastore:534 :
Connected to metastore.
2019-07-17 21:23:32,486 INFO [http-bio-7070-exec-68] hive.metastore:407 :
Trying to connect to metastore with URI thrift://ip-10-0-0-46.ec2.internal:9083
2019-07-17 21:23:32,487 INFO [http-bio-7070-exec-68] hive.metastore:481 :
Opened a connection to metastore, current connections: 5
2019-07-17 21:23:32,488 INFO [http-bio-7070-exec-68] hive.metastore:534 :
Connected to metastore.
2019-07-17 21:24:27,717 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:25:27,718 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:26:25,136 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1]
ipc.AbstractRpcClient:217 : Cleanup idle connection to
ip-10-0-0-54.ec2.internal/10.0.0.54:16020
2019-07-17 21:26:27,718 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:27:27,719 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:28:27,719 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:29:27,720 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:30:27,720 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:31:27,721 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:32:27,721 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:33:27,721 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
2019-07-17 21:34:27,722 INFO [BadQueryDetector] service.BadQueryDetector:147 :
Detect bad query.
3)
2019-07-17 20:49:46,875 DEBUG [http-bio-7070-exec-4]
security.KylinAuthenticationProvider:57 : User cache [-108, 112, -63, -32, 41,
-87, -81, 81, -32, 61, -35, -111, 7, 56, -29, -59] is removed due to EXPLICIT
2019-07-17 20:49:46,973 DEBUG [http-bio-7070-exec-4]
security.KylinAuthenticationProvider:113 : User ADMIN authorities :
[ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER]
2019-07-17 20:49:46,973 DEBUG [http-bio-7070-exec-4]
cachesync.CachedCrudAssist:190 : Saving ManagedUser at /user/ADMIN
2019-07-17 20:49:46,976 DEBUG [pool-5-thread-1] cachesync.Broadcaster:113 :
Servers in the cluster: [ip-10-0-0-45.ec2.internal:7070,
ip-10-0-0-46.ec2.internal:7070, ip-10-0-0-51.ec2.internal:7070]
2019-07-17 20:49:46,977 DEBUG [pool-5-thread-1] cachesync.Broadcaster:123 :
Announcing new broadcast to all: BroadcastEvent{entity=user, event=update,
cacheKey=ADMIN}
2019-07-17 20:49:46,978 DEBUG [http-bio-7070-exec-4]
security.KylinAuthenticationProvider:126 : Authenticated user
org.springframework.security.authentication.UsernamePasswordAuthenticationToken@37047726:
Principal: org.springframework.security.core.userdetails.User@3b40b2f:
Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER; Credentials: [PROTECTED]; Authenticated:
true; Details:
org.springframework.security.web.authentication.WebAuthenticationDetails@ffff4c9c:
RemoteIpAddress: 10.0.0.59; SessionId: null; Granted Authorities: ROLE_ADMIN,
ROLE_ANALYST, ROLE_MODELER
2019-07-17 20:49:46,992 DEBUG [http-bio-7070-exec-18] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 20:49:46,994 DEBUG [http-bio-7070-exec-18] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 20:49:47,051 DEBUG [http-bio-7070-exec-20] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 20:49:47,053 DEBUG [http-bio-7070-exec-20] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:02,894 DEBUG [http-bio-7070-exec-20] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:02,897 DEBUG [http-bio-7070-exec-20] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:03,033 DEBUG [http-bio-7070-exec-9] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:03,035 DEBUG [http-bio-7070-exec-9] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:03,159 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:247
: Broadcasting UPDATE, user, ADMIN
2019-07-17 20:50:03,161 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:281
: Done broadcasting UPDATE, user, ADMIN
I am looking forward to hearing from you.
Thanks,
Krishna
