nlnjnj opened a new issue, #15704: URL: https://github.com/apache/dolphinscheduler/issues/15704
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened ```log [WI-0][TI-14832] - [ERROR] 2024-03-13 10:01:36.988 +0800 o.a.d.s.m.r.e.MasterTaskExecutor:[60] - Get a exception when execute the task, will try to cancel the task -- Wed, Mar 13 2024 10:01:36 am | org.springframework.jdbc.BadSqlGrammarException: Wed, Mar 13 2024 10:01:36 am | ### Error querying database. Cause: org.postgresql.util.PSQLException: ERROR: invalid reference to FROM-clause entry for table "instance" Wed, Mar 13 2024 10:01:36 am | Hint: There is an entry for table "instance", but it cannot be referenced from this part of the query. Wed, Mar 13 2024 10:01:36 am | Position: 981 Wed, Mar 13 2024 10:01:36 am | ### The error may exist in class path resource [org/apache/dolphinscheduler/dao/mapper/TaskInstanceMapper.xml] Wed, Mar 13 2024 10:01:36 am | ### The error may involve defaultParameterMap Wed, Mar 13 2024 10:01:36 am | ### The error occurred while setting parameters Wed, Mar 13 2024 10:01:36 am | ### SQL: select instance.id, instance.name, instance.task_type, instance.task_code, instance.task_definition_version, instance.process_instance_id, instance.state, instance.submit_time, instance.start_time, instance.end_time, instance.host, instance.execute_path, instance.log_path, instance.alert_flag, instance.retry_times, instance.pid, instance.app_link, instance.flag, instance.is_cache, instance.cache_key, instance.retry_interval, instance.max_retry_times, instance.task_instance_priority, instance.worker_group,instance.environment_code , instance.executor_id, instance.first_submit_time, instance.delay_time, instance.task_params, instance.var_pool, instance.dry_run, instance.test_flag, instance.task_group_id, instance.task_execute_type from t_ds_task_instance instance join ( select task_code, max(end_time) as max_end_time from t_ds_task_instance where 1=1 and test_flag = ? and instance.process_instance_id = ? and task_code in ( ? ) group by task_co de ) t_max on instance.process_instance_id = t_max.process_instance_id and instance.task_code = t_max.task_code and instance.end_time = t_max.max_end_time Wed, Mar 13 2024 10:01:36 am | ### Cause: org.postgresql.util.PSQLException: ERROR: invalid reference to FROM-clause entry for table "instance" Wed, Mar 13 2024 10:01:36 am | Hint: There is an entry for table "instance", but it cannot be referenced from this part of the query. Wed, Mar 13 2024 10:01:36 am | Position: 981 Wed, Mar 13 2024 10:01:36 am | ; bad SQL grammar []; nested exception is org.postgresql.util.PSQLException: ERROR: invalid reference to FROM-clause entry for table "instance" Wed, Mar 13 2024 10:01:36 am | Hint: There is an entry for table "instance", but it cannot be referenced from this part of the query. Wed, Mar 13 2024 10:01:36 am | Position: 981 Wed, Mar 13 2024 10:01:36 am | at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:239) Wed, Mar 13 2024 10:01:36 am | at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70) Wed, Mar 13 2024 10:01:36 am | at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:91) Wed, Mar 13 2024 10:01:36 am | at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441) Wed, Mar 13 2024 10:01:36 am | at com.sun.proxy.$Proxy141.selectList(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:224) Wed, Mar 13 2024 10:01:36 am | at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.executeForMany(MybatisMapperMethod.java:166) Wed, Mar 13 2024 10:01:36 am | at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:77) Wed, Mar 13 2024 10:01:36 am | at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148) Wed, Mar 13 2024 10:01:36 am | at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89) Wed, Mar 13 2024 10:01:36 am | at com.sun.proxy.$Proxy152.findLastTaskInstances(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.dao.repository.impl.TaskInstanceDaoImpl.queryLastTaskInstanceListIntervalInProcessInstance(TaskInstanceDaoImpl.java:179) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.dao.repository.impl.TaskInstanceDaoImpl$$FastClassBySpringCGLIB$$5c344c57.invoke(<generated>) Wed, Mar 13 2024 10:01:36 am | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) Wed, Mar 13 2024 10:01:36 am | at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) Wed, Mar 13 2024 10:01:36 am | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.dao.repository.impl.TaskInstanceDaoImpl$$EnhancerBySpringCGLIB$$a3433e12.queryLastTaskInstanceListIntervalInProcessInstance(<generated>) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.dependResultByAllTaskOfProcessInstance(DependentExecute.java:213) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.calculateResultForTasks(DependentExecute.java:161) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.getDependentResultForItem(DependentExecute.java:136) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.getDependResultForItem(DependentExecute.java:422) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.getModelDependResult(DependentExecute.java:395) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.utils.DependentExecute.finish(DependentExecute.java:364) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.runner.task.dependent.DependentAsyncTaskExecuteFunction.isAllDependentTaskFinished(DependentAsyncTaskExecuteFunction.java:220) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.runner.task.dependent.DependentAsyncTaskExecuteFunction.getAsyncTaskExecutionStatus(DependentAsyncTaskExecuteFunction.java:100) Wed, Mar 13 2024 10:01:36 am | at org.apache.dolphinscheduler.server.master.runner.execute.AsyncMasterTaskDelayQueueLooper.lambda$run$0(AsyncMasterTaskDelayQueueLooper.java:90) Wed, Mar 13 2024 10:01:36 am | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Wed, Mar 13 2024 10:01:36 am | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Wed, Mar 13 2024 10:01:36 am | at java.lang.Thread.run(Thread.java:750) Wed, Mar 13 2024 10:01:36 am | Caused by: org.postgresql.util.PSQLException: ERROR: invalid reference to FROM-clause entry for table "instance" Wed, Mar 13 2024 10:01:36 am | Hint: There is an entry for table "instance", but it cannot be referenced from this part of the query. Wed, Mar 13 2024 10:01:36 am | Position: 981 Wed, Mar 13 2024 10:01:36 am | at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:356) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:181) Wed, Mar 13 2024 10:01:36 am | at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:170) Wed, Mar 13 2024 10:01:36 am | at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44) Wed, Mar 13 2024 10:01:36 am | at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:64) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79) Wed, Mar 13 2024 10:01:36 am | at sun.reflect.GeneratedMethodAccessor72.invoke(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Wed, Mar 13 2024 10:01:36 am | at java.lang.reflect.Method.invoke(Method.java:498) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:64) Wed, Mar 13 2024 10:01:36 am | at com.sun.proxy.$Proxy185.query(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) Wed, Mar 13 2024 10:01:36 am | at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:81) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62) Wed, Mar 13 2024 10:01:36 am | at com.sun.proxy.$Proxy184.query(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:151) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:145) Wed, Mar 13 2024 10:01:36 am | at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140) Wed, Mar 13 2024 10:01:36 am | at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) Wed, Mar 13 2024 10:01:36 am | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Wed, Mar 13 2024 10:01:36 am | at java.lang.reflect.Method.invoke(Method.java:498) Wed, Mar 13 2024 10:01:36 am | at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427) Wed, Mar 13 2024 10:01:36 am | ... 30 common frames omitted ``` ### What you expected to happen Task dependent on task works normally. ### How to reproduce Upgrade from 3.1.8 to 3.2.1 Use k8s deploy with helm chart ```zsh noglob helm3 upgrade --install dolphinscheduler ./ \ --set image.registry=apache \ --set postgresql.persistence.enabled=true,postgresql.persistence.size=20Gi \ --set master.replicas=1,worker.replicas=2,nodeSelector.nodepool=worker-cluster \ --set ingress.enabled=true,ingress.host=dolphin.xxx.com \ --set ingress.tls.enabled=true,ingress.tls.secretName=tls-wildcard \ --set common.sharedStoragePersistence.enabled=true,common.sharedStoragePersistence.storage=1Gi \ --set minio.enabled=false \ --set conf.common."resource\.storage\.type"="HDFS" \ --set conf.common."resource\.hdfs\.fs\.defaultFS"="file:///" \ --set common.fsFileResourcePersistence.enabled=true,common.fsFileResourcePersistence.storage=1Gi \ --namespace dolphinscheduler --create-namespace ``` ### Anything else The default registry is incorrect ```yaml image: # -- Docker image repository for the DolphinScheduler registry: apache/dolphinscheduler ``` ```yaml image: # -- Docker image repository for the DolphinScheduler registry: apache ``` The build-in sql template has a syntax error: ```sql select instance.id, instance.name, instance.task_type, instance.task_code, instance.task_definition_version, instance.process_instance_id, instance.state, instance.submit_time, instance.start_time, instance.end_time, instance.host, instance.execute_path, instance.log_path, instance.alert_flag, instance.retry_times, instance.pid, instance.app_link, instance.flag, instance.is_cache, instance.cache_key, instance.retry_interval, instance.max_retry_times, instance.task_instance_priority, instance.worker_group, instance.environment_code, instance.executor_id, instance.first_submit_time, instance.delay_time, instance.task_params, instance.var_pool, instance.dry_run, instance.test_flag, instance.task_group_id, instance.task_execute_type from t_ds_task_instance instance join (select task_code, max(end_time) as max_end_time from t_ds_task_instance where 1 = 1 and test_flag = ? and **instance**.process_instance_id = ? and task_code in (?) group by task_code) t_max on instance.process_instance_id = t_max.process_instance_id and instance.task_code = t_max.task_code and instance.end_time = t_max.max_end_time ``` ### Version 3.2.x ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
