himper commented on issue #11134:
URL:
https://github.com/apache/dolphinscheduler/issues/11134#issuecomment-1663369939
3.1.7 The problem still exists
23/08/03 14:17:12 INFO Executor: Running task 0.0 in stage 2.0 (TID 2)
[INFO] 2023-08-03 14:17:14.660 +0800 - -> 23/08/03 14:17:14 INFO JDBCRDD:
closed connection
23/08/03 14:17:14 INFO Executor: Finished task 0.0 in stage 2.0 (TID
2). 1669 bytes result sent to driver
23/08/03 14:17:14 INFO TaskSetManager: Finished task 0.0 in stage 2.0
(TID 2) in 1604 ms on localhost (executor driver) (1/1)
23/08/03 14:17:14 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose
tasks have all completed, from pool
23/08/03 14:17:14 INFO DAGScheduler: ShuffleMapStage 2 (save at
JdbcWriter.java:86) finished in 1.690 s
23/08/03 14:17:14 INFO DAGScheduler: looking for newly runnable stages
23/08/03 14:17:14 INFO DAGScheduler: running: Set()
23/08/03 14:17:14 INFO DAGScheduler: waiting: Set(ResultStage 3)
23/08/03 14:17:14 INFO DAGScheduler: failed: Set()
23/08/03 14:17:14 INFO DAGScheduler: Submitting ResultStage 3
(MapPartitionsRDD[16] at save at JdbcWriter.java:86), which has no missing
parents
23/08/03 14:17:14 INFO MemoryStore: Block broadcast_4 stored as values
in memory (estimated size 36.6 KB, free 116.9 MB)
23/08/03 14:17:14 INFO MemoryStore: Block broadcast_4_piece0 stored as
bytes in memory (estimated size 15.8 KB, free 116.9 MB)
23/08/03 14:17:14 INFO BlockManagerInfo: Added broadcast_4_piece0 in
memory on
dolphinscheduler-worker-1.dolphinscheduler-worker-headless.dolphinscheduler.svc.cluster.local:36171
(size: 15.8 KB, free: 116.9 MB)
23/08/03 14:17:14 INFO SparkContext: Created broadcast 4 from broadcast
at DAGScheduler.scala:1184
23/08/03 14:17:14 INFO DAGScheduler: Submitting 1 missing tasks from
ResultStage 3 (MapPartitionsRDD[16] at save at JdbcWriter.java:86) (first 15
tasks are for partitions Vector(0))
23/08/03 14:17:14 INFO TaskSchedulerImpl: Adding task set 3.0 with 1
tasks
23/08/03 14:17:14 INFO TaskSetManager: Starting task 0.0 in stage 3.0
(TID 3, localhost, executor driver, partition 0, ANY, 7767 bytes)
23/08/03 14:17:14 INFO Executor: Running task 0.0 in stage 3.0 (TID 3)
[INFO] 2023-08-03 14:17:15.737 +0800 - -> 23/08/03 14:17:14 INFO
ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local
blocks and 0 remote blocks
23/08/03 14:17:14 INFO ShuffleBlockFetcherIterator: Started 0 remote
fetches in 0 ms
23/08/03 14:17:14 INFO CodeGenerator: Code generated in 98.387902 ms
23/08/03 14:17:15 INFO CodeGenerator: Code generated in 201.736892 ms
23/08/03 14:17:15 ERROR Executor: Exception in task 0.0 in stage 3.0
(TID 3)
java.sql.BatchUpdateException: Batch entry 0 INSERT INTO
t_ds_dq_execute_result
("rule_type","rule_name","process_definition_id","process_instance_id","task_instance_id","statistics_value","comparison_value","comparison_type","check_type","threshold","operator","failure_strategy","error_output_path","create_time","update_time")
VALUES
(3,'(multi_table_value_comparison)',0,22,24,1691033463,1691033476,0,0,0,0,0,'s3a://dolphinscheduler/user/root/data_quality_error_data/0_22_tag_quality','2023-08-03
14:16:03','2023-08-03 14:16:03') was aborted: ERROR: column "create_time" is
of type timestamp without time zone but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
Position: 337 Call getNextException to see other errors in the batch.
at
org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:165)
at
org.postgresql.core.ResultHandlerDelegate.handleError(ResultHandlerDelegate.java:52)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2367)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:560)
at
org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:887)
at
org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:910)
at
org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1663)
at
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:676)
at
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:838)
at
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:838)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:980)
at
org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:980)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2107)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2107)
at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:123)
at
org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:411)
at
org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:417)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.postgresql.util.PSQLException: ERROR: column
"create_time" is of type timestamp without time zone but expression is of type
character varying
Hint: You will need to rewrite or cast the expression.
Position: 337
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366)
... 19 more
23/08/03 14:17:15 WARN TaskSetManager: Lost task 0.0 in stage 3.0 (TID
3, localhost, executor driver): java.sql.BatchUpdateException: Batch entry 0
INSERT INTO t_ds_dq_execute_result
("rule_type","rule_name","process_definition_id","process_instance_id","task_instance_id","statistics_value","comparison_value","comparison_type","check_type","threshold","operator","failure_strategy","error_output_path","create_time","update_time")
VALUES
(3,'(multi_table_value_comparison)',0,22,24,1691033463,1691033476,0,0,0,0,0,'s3a://dolphinscheduler/user/root/data_quality_error_data/0_22_tag_quality','2023-08-03
14:16:03','2023-08-03 14:16:03') was aborted: ERROR: column "create_time" is
of type timestamp without time zone but expression is of type character varying
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
