[
https://issues.apache.org/jira/browse/KYLIN-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139005#comment-17139005
]
ASF GitHub Bot commented on KYLIN-3832:
---------------------------------------
hit-lacus edited a comment on pull request #1272:
URL: https://github.com/apache/kylin/pull/1272#issuecomment-645741145
# Test Report
### Root cause
From the documentation of pg, we know that when identifier is quoted,
identifier is case sensitive, but Kylin use all upper-case, which make quoted
identifier not identifiable by pg.
At my patch, I disable all quote switch in `postgresql.xml`, so we cannot
support reserved word as identifier, which is rare.
### Test Result
After applied my patch, build from pg data source is fine, but pushdown is
still not work, because it is caused by another patch from KYLIN-4068 .
- Build Success
```sh
2020-06-18 11:41:50,986 DEBUG [http-bio-8005-exec-4]
extensible.JdbcHiveInputBase:129 : sqoop cmd: --connect
"jdbc:postgresql://10.1.3.94:5432/kylin_test_db" --driver
"org.postgresql.Driver" --username "postgres" --password "password" --query
"SELECT FILM_PLAY.AUDIENCE_ID as FILM_PLAY_AUDIENCE_ID ,FILM_PLAY.FILM_ID as
FILM_PLAY_FILM_ID ,FILM_PLAY.WATCH_TIME ,FILM_PLAY.PAYMENT as FILM_PLAY_PAYMENT
FROM SC1.FILM_PLAY as FILM_PLAY LEFT JOIN SC2.AUDIENCE as AUDIENCE ON
FILM_PLAY.AUDIENCE_ID = AUDIENCE.AUDIENCE_ID LEFT JOIN PUBLIC.FILM as FILM ON
FILM_PLAY.FILM_ID = FILM.FILM_ID WHERE 1=1 AND (FILM_PLAY.WATCH_TIME >=
'2012-01-01 00:00:00' AND FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00') AND
\$CONDITIONS" --target-dir
"hdfs://cdh-master:8020/LACUS/LACUS-tmp/kylin-ef6e4fd7-f72d-f7a0-fdca-b32bb0153493/kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666"
--split-by "FILM_PLAY.WATCH_TIME" --boundary-query "SELECT
min(FILM_PLAY.WATCH_TIME), max(FILM_PLAY.WATCH_TIME) FROM SC1.FILM_PLAY as
FILM_PLAY WHERE FILM_PLAY.WATCH_TIME >= '2012-01-01 00:00:00' AND
FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00'" --null-string ''
--fields-terminated-by '|' --num-mappers 4
2020-06-18 11:41:51,046 DEBUG [http-bio-8005-exec-4]
hbase.HBaseConnection:181 : Using the working dir FS for HBase:
hdfs://cdh-master:8020
2020-06-18 11:41:51,049 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:41:51,050 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=create, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:41:51,053 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:267 : Broadcasting CREATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:41:51,054 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:301 : Done broadcasting CREATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:41:51,066 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:41:51,066 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute, event=create,
cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:41:51,069 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:267 : Broadcasting CREATE, execute,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:41:51,070 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:301 : Done broadcasting CREATE, execute,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:05,813 INFO [FetcherRunner 1308001235-46]
threadpool.FetcherRunner:65 :
CubingJob{id=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493, name=BUILD CUBE -
FilmAnalysis - 20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50,
state=READY} prepare to schedule and its priority is 20
2020-06-18 11:42:05,813 INFO [FetcherRunner 1308001235-46]
threadpool.FetcherRunner:69 :
CubingJob{id=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493, name=BUILD CUBE -
FilmAnalysis - 20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50,
state=READY} scheduled
2020-06-18 11:42:05,813 INFO [FetcherRunner 1308001235-46]
threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 1 actual
running, 0 stopped, 1 ready, 4 already succeed, 0 error, 7 discarded, 0 others
2020-06-18 11:42:05,813 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.AbstractExecutable:161 :
Executing AbstractExecutable (BUILD CUBE - FilmAnalysis -
20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50)
2020-06-18 11:42:05,816 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493 from READY to RUNNING
2020-06-18 11:42:05,816 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:05,816 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:05,818 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:05,818 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:05,818 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.AbstractExecutable:161 :
Executing AbstractExecutable (Sqoop To Flat Hive Table)
2020-06-18 11:42:05,819 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:05,820 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-00 from READY to RUNNING
2020-06-18 11:42:05,820 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:05,820 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : exe
cmd:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/sqoop/bin/sqoop import
-Dorg.apache.sqoop.splitter.allow_text_splitter=true
-Dmapreduce.job.queuename=default --connect
"jdbc:postgresql://10.1.3.94:5432/kylin_test_db" --driver
"org.postgresql.Driver" --username "postgres" --password "password" --query
"SELECT FILM_PLAY.AUDIENCE_ID as FILM_PLAY_AUDIENCE_ID ,FILM_PLAY.FILM_ID as
FILM_PLAY_FILM_ID ,FILM_PLAY.WATCH_TIME ,FILM_PLAY.PAYMENT as FILM_PLAY_PAYMENT
FROM SC1.FILM_PLAY as FILM_PLAY LEFT JOIN SC2.AUDIENCE as AUDIENCE ON
FILM_PLAY.AUDIENCE_ID = AUDIENCE.AUDIENCE_ID LEFT JOIN PUBLIC.FILM as FILM ON
FILM_PLAY.FILM_ID = FILM.FILM_ID WHERE 1=1 AND (FILM_PLAY.WATCH_TIME >=
'2012-01-01 00:00:00' AND FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00') AND
\$CONDITIONS" --target-dir
"hdfs://cdh-master:8020/LACUS/LACUS-tmp/kylin-ef6e4fd7-f72d-f7a0-fdca-b32bb0153493/kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666"
--split-by "FILM_PLAY.WATCH_TIME" --boundary-query "SELECT
min(FILM_PLAY.WATCH_TIME), max(FILM_PLAY.WATCH_TIME) FROM SC1.FILM_PLAY as
FILM_PLAY WHERE FILM_PLAY.WATCH_TIME >= '2012-01-01 00:00:00' AND
FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00'" --null-string ''
--fields-terminated-by '|' --num-mappers 4
2020-06-18 11:42:05,823 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:05,824 DEBUG [http-bio-8005-exec-3]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:05,853 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : Warning:
/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/sqoop/bin/../../accumulo
does not exist! Accumulo imports will fail.
2020-06-18 11:42:05,853 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : Please set
$ACCUMULO_HOME to the root of your Accumulo installation.
2020-06-18 11:42:11,111 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.6
2020-06-18 11:42:11,124 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is
insecure. Consider using -P instead.
2020-06-18 11:42:11,181 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit
driver however appropriate connection manager is not being set (via
--connection-manager). Sqoop is going to fall back to
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which
connection manager should be used next time.
2020-06-18 11:42:11,192 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO manager.SqlManager: Using default fetchSize of 1000
2020-06-18 11:42:11,192 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO tool.CodeGenTool: Beginning code generation
2020-06-18 11:42:11,480 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO manager.SqlManager: Executing SQL statement: SELECT
FILM_PLAY.AUDIENCE_ID as FILM_PLAY_AUDIENCE_ID ,FILM_PLAY.FILM_ID as
FILM_PLAY_FILM_ID ,FILM_PLAY.WATCH_TIME ,FILM_PLAY.PAYMENT as FILM_PLAY_PAYMENT
FROM SC1.FILM_PLAY as FILM_PLAY LEFT JOIN SC2.AUDIENCE as AUDIENCE ON
FILM_PLAY.AUDIENCE_ID = AUDIENCE.AUDIENCE_ID LEFT JOIN PUBLIC.FILM as FILM ON
FILM_PLAY.FILM_ID = FILM.FILM_ID WHERE 1=1 AND (FILM_PLAY.WATCH_TIME >=
'2012-01-01 00:00:00' AND FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00') AND
(1 = 0)
2020-06-18 11:42:11,494 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO manager.SqlManager: Executing SQL statement: SELECT
FILM_PLAY.AUDIENCE_ID as FILM_PLAY_AUDIENCE_ID ,FILM_PLAY.FILM_ID as
FILM_PLAY_FILM_ID ,FILM_PLAY.WATCH_TIME ,FILM_PLAY.PAYMENT as FILM_PLAY_PAYMENT
FROM SC1.FILM_PLAY as FILM_PLAY LEFT JOIN SC2.AUDIENCE as AUDIENCE ON
FILM_PLAY.AUDIENCE_ID = AUDIENCE.AUDIENCE_ID LEFT JOIN PUBLIC.FILM as FILM ON
FILM_PLAY.FILM_ID = FILM.FILM_ID WHERE 1=1 AND (FILM_PLAY.WATCH_TIME >=
'2012-01-01 00:00:00' AND FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00') AND
(1 = 0)
2020-06-18 11:42:11,516 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
2020-06-18 11:42:12,833 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 注:
/tmp/sqoop-root/compile/dfa8a452da439c529a7a93624d04417f/QueryResult.java使用或覆盖了已过时的
API。
2020-06-18 11:42:12,833 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 注: 有关详细信息,
请使用 -Xlint:deprecation 重新编译。
2020-06-18 11:42:12,851 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:12 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-root/compile/dfa8a452da439c529a7a93624d04417f/QueryResult.jar
2020-06-18 11:42:12,868 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:12 INFO mapreduce.ImportJobBase: Beginning query import.
2020-06-18 11:42:13,052 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:13 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use
mapreduce.job.jar
2020-06-18 11:42:13,698 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:13 INFO Configuration.deprecation: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
2020-06-18 11:42:13,758 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:13 INFO client.RMProxy: Connecting to ResourceManager at
cdh-master/10.1.3.90:8032
2020-06-18 11:42:16,102 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO db.DBInputFormat: Using read commited transaction isolation
2020-06-18 11:42:16,103 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT
min(FILM_PLAY.WATCH_TIME), max(FILM_PLAY.WATCH_TIME) FROM SC1.FILM_PLAY as
FILM_PLAY WHERE FILM_PLAY.WATCH_TIME >= '2012-01-01 00:00:00' AND
FILM_PLAY.WATCH_TIME < '2012-12-01 00:00:00'
2020-06-18 11:42:16,108 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 WARN db.DateSplitter: Encountered a NULL date in the split column.
Splits may be poorly balanced.
2020-06-18 11:42:16,109 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 WARN db.DateSplitter: Encountered a NULL date in the split column.
Splits may be poorly balanced.
2020-06-18 11:42:16,189 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO mapreduce.JobSubmitter: number of splits:1
2020-06-18 11:42:16,381 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1589169585068_6108
2020-06-18 11:42:16,590 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO impl.YarnClientImpl: Submitted application
application_1589169585068_6108
2020-06-18 11:42:16,623 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO mapreduce.Job: The url to track the job:
http://cdh-master:8088/proxy/application_1589169585068_6108/
2020-06-18 11:42:16,623 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:16 INFO mapreduce.Job: Running job: job_1589169585068_6108
2020-06-18 11:42:21,799 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:21 INFO mapreduce.Job: Job job_1589169585068_6108 running in uber mode :
false
2020-06-18 11:42:21,800 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:21 INFO mapreduce.Job: map 0% reduce 0%
2020-06-18 11:42:26,846 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:26 INFO mapreduce.Job: map 100% reduce 0%
2020-06-18 11:42:27,858 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:27 INFO mapreduce.Job: Job job_1589169585068_6108 completed successfully
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:27 INFO mapreduce.Job: Counters: 30
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : File
System Counters
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
FILE: Number of bytes read=0
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
FILE: Number of bytes written=145645
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
FILE: Number of read operations=0
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
FILE: Number of large read operations=0
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
FILE: Number of write operations=0
2020-06-18 11:42:27,963 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
HDFS: Number of bytes read=137
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
HDFS: Number of bytes written=0
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
HDFS: Number of read operations=4
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
HDFS: Number of large read operations=0
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
HDFS: Number of write operations=2
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : Job
Counters
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Launched map tasks=1
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Other local map tasks=1
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total time spent by all maps in occupied slots (ms)=8910
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total time spent by all reduces in occupied slots (ms)=0
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total time spent by all map tasks (ms)=2970
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total vcore-seconds taken by all map tasks=2970
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total megabyte-seconds taken by all map tasks=9123840
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : Map-Reduce
Framework
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Map input records=0
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Map output records=0
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Input split bytes=137
2020-06-18 11:42:27,964 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Spilled Records=0
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Failed Shuffles=0
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Merged Map outputs=0
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : GC
time elapsed (ms)=60
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
CPU time spent (ms)=770
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Physical memory (bytes) snapshot=314957824
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Virtual memory (bytes) snapshot=4686204928
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Total committed heap usage (bytes)=808452096
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : File Input
Format Counters
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Bytes Read=0
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : File
Output Format Counters
2020-06-18 11:42:27,965 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 :
Bytes Written=0
2020-06-18 11:42:27,970 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:27 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 14.2625 seconds
(0 bytes/sec)
2020-06-18 11:42:27,973 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] sqoop.SqoopCmdStep:41 : 20/06/18
11:42:27 INFO mapreduce.ImportJobBase: Retrieved 0 records.
2020-06-18 11:42:28,318 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-00 from RUNNING to SUCCEED
2020-06-18 11:42:28,320 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:28,320 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:28,323 DEBUG [http-bio-8005-exec-5]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,324 DEBUG [http-bio-8005-exec-5]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,329 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:28,329 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:28,329 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493 from RUNNING to READY
2020-06-18 11:42:28,331 INFO [FetcherRunner 1308001235-46]
threadpool.FetcherRunner:65 :
CubingJob{id=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493, name=BUILD CUBE -
FilmAnalysis - 20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50,
state=READY} prepare to schedule and its priority is 20
2020-06-18 11:42:28,331 INFO [FetcherRunner 1308001235-46]
threadpool.FetcherRunner:69 :
CubingJob{id=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493, name=BUILD CUBE -
FilmAnalysis - 20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50,
state=READY} scheduled
2020-06-18 11:42:28,331 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.AbstractExecutable:161 :
Executing AbstractExecutable (BUILD CUBE - FilmAnalysis -
20120101000000_20121201000000 - GMT+08:00 2020-06-18 11:41:50)
2020-06-18 11:42:28,331 INFO [FetcherRunner 1308001235-46]
threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 1 actual
running, 0 stopped, 1 ready, 4 already succeed, 0 error, 7 discarded, 0 others
2020-06-18 11:42:28,332 DEBUG [http-bio-8005-exec-5]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,332 DEBUG [http-bio-8005-exec-5]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,334 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:28,334 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:28,334 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493 from READY to RUNNING
2020-06-18 11:42:28,336 DEBUG [pool-6-thread-1] cachesync.Broadcaster:119 :
Servers in the cluster: [localhost:8005]
2020-06-18 11:42:28,336 DEBUG [pool-6-thread-1] cachesync.Broadcaster:129 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=ef6e4fd7-f72d-f7a0-fdca-b32bb0153493}
2020-06-18 11:42:28,336 DEBUG [http-bio-8005-exec-4]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,336 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.AbstractExecutable:161 :
Executing AbstractExecutable (Create Intermediate Flat Hive Table)
2020-06-18 11:42:28,337 DEBUG [http-bio-8005-exec-4]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,339 DEBUG [http-bio-8005-exec-4]
cachesync.Broadcaster:267 : Broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,340 DEBUG [http-bio-8005-exec-4]
cachesync.Broadcaster:301 : Done broadcasting UPDATE, execute_output,
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493
2020-06-18 11:42:28,340 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] execution.ExecutableManager:479 : job
id:ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-01 from READY to RUNNING
2020-06-18 11:42:28,343 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : cmd:
2020-06-18 11:42:28,343 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : hive -e "USE
LACUS;
DROP TABLE IF EXISTS
kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666;
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666
(
\`FILM_PLAY_AUDIENCE_ID\` int
,\`FILM_PLAY_FILM_ID\` int
,\`FILM_PLAY_WATCH_TIME\` timestamp
,\`FILM_PLAY_PAYMENT\` int
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
LOCATION
'hdfs://cdh-master:8020/LACUS/LACUS-tmp/kylin-ef6e4fd7-f72d-f7a0-fdca-b32bb0153493/kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666';
ALTER TABLE
kylin_intermediate_filmanalysis_5b0b0b6a_0ad3_14fd_30ed_27cf74504666 SET
TBLPROPERTIES('auto.purge'='true');
" --hiveconf hive.merge.mapredfiles=false --hiveconf
hive.auto.convert.join=true --hiveconf dfs.replication=2 --hiveconf
hive.exec.compress.output=true --hiveconf
hive.auto.convert.join.noconditionaltask=true --hiveconf
mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false
--hiveconf hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf
hive.stats.autogather=true
2020-06-18 11:42:28,509 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : Java
HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support
was removed in 8.0
2020-06-18 11:42:31,097 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : Java
HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and
will likely be removed in a future release
2020-06-18 11:42:32,403 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : Java
HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support
was removed in 8.0
2020-06-18 11:42:33,169 INFO [BadQueryDetector]
service.BadQueryDetector:148 : Detect bad query.
2020-06-18 11:42:33,208 INFO [metrics-blocking-reservoir-scheduler-0]
impl.BlockingReservoir:176 : There's no record in the blocking queue.
2020-06-18 11:42:33,518 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 :
2020-06-18 11:42:33,518 INFO [Scheduler 1125220712 Job
ef6e4fd7-f72d-f7a0-fdca-b32bb0153493-276] jdbc.HiveCmdStep:41 : Logging
initialized using configuration in
jar:file:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/jars/hive-common-1.1.0-cdh5.7.6.jar!/hive-log4j.properties
2020-06-18 11:42:35,811 INFO [FetcherRunner 1308001235-46]
threadpool.DefaultFetcherRunner:115 : Job Fetcher: 1 should running, 1 actual
running, 0 stopped, 0 ready, 4 already succeed, 0 error, 7 discarded, 0 others
```
- Pushdown error
```sh
2020-06-18 10:53:12,241 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] service.QueryService:679 : The
corrected query: select * from (select film_id, count(distinct payment)
from sc1.film_play
group by film_id) limit 50000
2020-06-18 10:53:12,301 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] routing.QueryRouter:63 : Find
candidates by table SC1.FILM_PLAY and project=PgTestPrj :
CUBE[name=FilmCube_clone_clone]
2020-06-18 10:53:12,302 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] routing.QueryRouter:51 : Applying
rule: class
org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
realizations before: [CUBE[name=FilmCube_clone_clone]], realizations after:
[CUBE[name=FilmCube_clone_clone]]
2020-06-18 10:53:12,302 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] cube.CubeCapabilityChecker:119 :
Exclude cube FilmCube_clone_clone because unmatched aggregations: [FunctionDesc
[expression=COUNT_DISTINCT, parameter=SC1.FILM_PLAY.PAYMENT, returnType=null]]
2020-06-18 10:53:12,303 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] routing.QueryRouter:51 : Applying
rule: class
org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
realizations before: [CUBE[name=FilmCube_clone_clone]], realizations after: []
2020-06-18 10:53:12,303 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] rules.RealizationSortRule:40 :
2020-06-18 10:53:12,303 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] routing.QueryRouter:51 : Applying
rule: class org.apache.kylin.query.routing.rules.RealizationSortRule,
realizations before: [], realizations after: []
2020-06-18 10:53:12,304 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] routing.RealizationChooser:92 : Give
up on model DataModelDesc [name=FilmModel] because no suitable realization is
found
2020-06-18 10:53:12,304 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] util.PushDownExecutor:71 : Query
failed to utilize pre-calculation, routing to other engines
java.sql.SQLException: Error while executing SQL "select * from (select
film_id, count(distinct payment)
from sc1.film_play
group by film_id) limit 50000": No realization found for OLAPContext,
CUBE_UNMATCHED_AGGREGATION[FunctionDesc [expression=COUNT_DISTINCT,
parameter=1_6906aea2:SC1.FILM_PLAY.PAYMENT, returnType=null]], CUBE_NOT_READY,
CUBE_NOT_READY, rel#83:OLAPTableScan.OLAP.[](table=[SC1,
FILM_PLAY],ctx=,fields=[0, 1, 2, 3, 4, 5])
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at
org.apache.kylin.rest.service.QueryService.executeRequest(QueryService.java:1003)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:699)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:221)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:507)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:467)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:398)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1195)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:317)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by:
org.apache.kylin.metadata.realization.NoRealizationFoundException: No
realization found for OLAPContext, CUBE_UNMATCHED_AGGREGATION[FunctionDesc
[expression=COUNT_DISTINCT, parameter=1_6906aea2:SC1.FILM_PLAY.PAYMENT,
returnType=null]], CUBE_NOT_READY, CUBE_NOT_READY,
rel#83:OLAPTableScan.OLAP.[](table=[SC1, FILM_PLAY],ctx=,fields=[0, 1, 2, 3, 4,
5])
at
org.apache.kylin.query.routing.RealizationChooser.attemptSelectRealization(RealizationChooser.java:102)
at
org.apache.kylin.query.routing.RealizationChooser.selectRealization(RealizationChooser.java:59)
at
org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:96)
at
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:103)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1278)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:796)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:655)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 84 more
2020-06-18 10:53:12,307 DEBUG [Query
72145593-a090-2676-9574-747a5df549db-64] util.PushDownExecutor:123 : Query
Pushdown runner org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl@16265819
2020-06-18 10:53:12,332 ERROR [Query
72145593-a090-2676-9574-747a5df549db-64] service.QueryService:1055 : pushdown
engine failed current query too
java.lang.RuntimeException: executeQuery failed
at
org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl.executeQuery(PushdownRunnerSDKImpl.java:71)
at
org.apache.kylin.query.util.PushDownExecutor.queryBySingleRunner(PushDownExecutor.java:148)
at
org.apache.kylin.query.util.PushDownExecutor.pushDownQuery(PushDownExecutor.java:88)
at
org.apache.kylin.query.util.PushDownUtil.tryPushDownSelectQuery(PushDownUtil.java:63)
at
org.apache.kylin.rest.service.QueryService.pushDownQuery(QueryService.java:1052)
at
org.apache.kylin.rest.service.QueryService.executeRequest(QueryService.java:1008)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:699)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:221)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:507)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:467)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:398)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1195)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:317)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: subquery in FROM must
have an alias
建议：For example, FROM (SELECT ...) [AS] foo.
位置：15
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:310)
at
org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:296)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:273)
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:226)
at
org.apache.kylin.job.shaded.org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.kylin.job.shaded.org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl.executeQuery(PushdownRunnerSDKImpl.java:54)
... 87 more
2020-06-18 10:53:12,334 ERROR [Query
72145593-a090-2676-9574-747a5df549db-64] service.QueryService:569 : Exception
while executing query
java.lang.RuntimeException: executeQuery failed
at
org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl.executeQuery(PushdownRunnerSDKImpl.java:71)
at
org.apache.kylin.query.util.PushDownExecutor.queryBySingleRunner(PushDownExecutor.java:148)
at
org.apache.kylin.query.util.PushDownExecutor.pushDownQuery(PushDownExecutor.java:88)
at
org.apache.kylin.query.util.PushDownUtil.tryPushDownSelectQuery(PushDownUtil.java:63)
at
org.apache.kylin.rest.service.QueryService.pushDownQuery(QueryService.java:1052)
at
org.apache.kylin.rest.service.QueryService.executeRequest(QueryService.java:1008)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:699)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:221)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:507)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:467)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:398)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
at
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1195)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:317)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: subquery in FROM must
have an alias
建议：For example, FROM (SELECT ...) [AS] foo.
位置：15
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:310)
at
org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:296)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:273)
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:226)
at
org.apache.kylin.job.shaded.org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.kylin.job.shaded.org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
at
org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl.executeQuery(PushdownRunnerSDKImpl.java:54)
... 87 more
2020-06-18 10:53:12,335 WARN [Query
72145593-a090-2676-9574-747a5df549db-64] common.QueryContext:234 : Cannot find
CubeSegmentStatisticsResult for context 0
2020-06-18 10:53:12,335 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] service.QueryService:1187 : Processed
rows for each storageContext:
2020-06-18 10:53:12,336 INFO [Query
72145593-a090-2676-9574-747a5df549db-64] service.QueryService:384 :
==========================[QUERY]===============================
Query Id: 72145593-a090-2676-9574-747a5df549db
SQL: select film_id, count(distinct payment)
from sc1.film_play
group by film_id
User: ADMIN
Success: false
Duration: 0.111
Project: PgTestPrj
Realization Names: []
Cuboid Ids: []
Total scan count: 0
Total scan bytes: 0
Result row count: 0
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Is Query Push-Down: false
Is Prepare: false
Trace URL: null
Message: executeQuery failed
==========================[QUERY]===============================
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Kylin pushdown to support postgresql
> ------------------------------------
>
> Key: KYLIN-3832
> URL: https://issues.apache.org/jira/browse/KYLIN-3832
> Project: Kylin
> Issue Type: New Feature
> Components: Query Engine
> Affects Versions: v2.5.2
> Reporter: hailin.huang
> Assignee: weibin0516
> Priority: Major
> Fix For: v3.1.0
>
>
> when I run pushdown to postgresql in my env, I encount the below exception.
> it seems that kylin need support more JDBC Driver,
> PushDownRunnerJdbcImpl.class should be more general.
> 2019-02-26 16:12:53,168 ERROR [Query 207dcf77-7c14-8078-ea8b-79644a0c576d-48]
> service.QueryService:989 : pushdown engine failed current query too
> java.sql.SQLException: Unrecognized column type: int8
> at
> org.apache.kylin.query.adhoc.PushDownRunnerJdbcImpl.toSqlType(PushDownRunnerJdbcImpl.java:260)
> at
> org.apache.kylin.query.adhoc.PushDownRunnerJdbcImpl.extractColumnMeta(PushDownRunnerJdbcImpl.java:192)
> at
> org.apache.kylin.query.adhoc.PushDownRunnerJdbcImpl.executeQuery(PushDownRunnerJdbcImpl.java:68)
> at
> org.apache.kylin.query.util.PushDownUtil.tryPushDownQuery(PushDownUtil.java:122)
> at
> org.apache.kylin.query.util.PushDownUtil.tryPushDownSelectQuery(PushDownUtil.java:69)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
