wangmingzhiJohn commented on issue #5733: URL: https://github.com/apache/dolphinscheduler/issues/5733#issuecomment-873326110
@ruanwenjun  It seems that something went wrong when I want to check the log This is api log: ``` [INFO] 2021-07-03 09:41:31.974 org.apache.dolphinscheduler.api.controller.LoginController:[84] - login user name: admin [INFO] 2021-07-03 09:41:31.988 org.apache.dolphinscheduler.api.security.PasswordAuthenticator:[59] - sessionId : fb779b5c-4c06-4ea2-bb5c-74e446c0ce84 [INFO] 2021-07-03 09:41:33.227 org.apache.dolphinscheduler.api.controller.UsersController:[299] - login user admin,get user info [INFO] 2021-07-03 09:41:33.629 org.apache.dolphinscheduler.api.controller.TenantController:[112] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:35.001 org.apache.dolphinscheduler.api.controller.UsersController:[116] - login user admin, list user paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:36.053 org.apache.dolphinscheduler.api.controller.AlertGroupController:[124] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:37.948 org.apache.dolphinscheduler.api.controller.TenantController:[112] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:41.786 org.apache.dolphinscheduler.api.controller.AlertGroupController:[124] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:46.122 org.apache.dolphinscheduler.api.controller.UsersController:[116] - login user admin, list user paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:47.187 org.apache.dolphinscheduler.api.controller.TenantController:[112] - login user admin, list paging, pageNo: 1, searchVal: , pageSize: 10 [INFO] 2021-07-03 09:41:47.974 org.apache.dolphinscheduler.api.controller.ProjectController:[157] - login user admin, query project list paging [INFO] 2021-07-03 09:41:50.098 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[76] - count task state, user:admin, start date: 2021-07-03 00:00:00, end date:2021-07-03 09:41:50, project id 2 [INFO] 2021-07-03 09:41:50.098 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[126] - count process definition , user:admin, project id:2 [INFO] 2021-07-03 09:41:50.098 org.apache.dolphinscheduler.api.controller.DataAnalysisController:[104] - count process instance state, user:admin, start date: 2021-07-03 00:00:00, end date:2021-07-03 09:41:50, project id:2 [INFO] 2021-07-03 09:41:52.247 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[290] - query process definition list paging, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[237] - query detail of process definition, login user:admin, project name:wmz-project1, process definition id:5 [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[257] - query process definition list, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ResourcesController:[177] - query resource list, login user:admin, resource type:FILE [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ProjectController:[264] - login user admin, query all project list [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:, program type:{} [INFO] 2021-07-03 09:41:54.278 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:PYTHON, program type:{} [INFO] 2021-07-03 09:41:54.373 org.apache.dolphinscheduler.api.controller.TenantController:[135] - login user admin, query tenant list [INFO] 2021-07-03 09:41:54.373 org.apache.dolphinscheduler.api.controller.WorkerGroupController:[135] - query all worker group: login user admin [INFO] 2021-07-03 09:42:00.328 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[290] - query process definition list paging, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:42:02.568 org.apache.dolphinscheduler.api.controller.WorkerGroupController:[135] - query all worker group: login user admin [INFO] 2021-07-03 09:42:02.573 org.apache.dolphinscheduler.api.controller.ExecutorController:[173] - login user admin, check process definition 5 [INFO] 2021-07-03 09:42:02.692 org.apache.dolphinscheduler.api.controller.AlertGroupController:[96] - login user admin, query all alertGroup [INFO] 2021-07-03 09:42:02.692 org.apache.dolphinscheduler.api.controller.ExecutorController:[199] - login user admin, get process definition receiver and cc [INFO] 2021-07-03 09:42:02.693 org.apache.dolphinscheduler.api.service.ExecutorService:[436] - processInstanceId null [INFO] 2021-07-03 09:42:03.727 org.apache.dolphinscheduler.api.controller.ExecutorController:[111] - login user admin, start process instance, project name: wmz-project1, process definition id: 5, schedule time: , failure policy: CONTINUE, node name: , node dep: TASK_POST, notify type: NONE, notify group id: default,receivers:,receiversCc:, run mode: RUN_MODE_SERIAL,process instance priority:MEDIUM, workerGroup: default, timeout: null [INFO] 2021-07-03 09:42:03.939 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[290] - query process definition list paging, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:42:05.493 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[101] - query all process instance list, login user:admin,project name:wmz-project1, define id:0,search value:,executor name:,state type:null,host:,start time:, end time:,page number:1, page size:10 [INFO] 2021-07-03 09:42:06.731 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[201] - query process instance detail by id, login user:admin,project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:42:06.731 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[257] - query process definition list, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:42:06.731 org.apache.dolphinscheduler.api.controller.ProjectController:[264] - login user admin, query all project list [INFO] 2021-07-03 09:42:06.732 org.apache.dolphinscheduler.api.controller.ResourcesController:[177] - query resource list, login user:admin, resource type:FILE [INFO] 2021-07-03 09:42:06.732 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:, program type:{} [INFO] 2021-07-03 09:42:06.732 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:PYTHON, program type:{} [INFO] 2021-07-03 09:42:06.835 org.apache.dolphinscheduler.api.controller.WorkerGroupController:[135] - query all worker group: login user admin [INFO] 2021-07-03 09:42:06.836 org.apache.dolphinscheduler.api.controller.TenantController:[135] - login user admin, query tenant list [INFO] 2021-07-03 09:42:06.941 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[130] - query task instance list by process instance id, login user:admin, project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:42:08.815 org.apache.dolphinscheduler.api.controller.LoggerController:[77] - login user admin, view 33 task instance log ,skipLineNum 0 , limit 1000 [INFO] 2021-07-03 09:42:08.817 org.apache.dolphinscheduler.api.service.LoggerService:[76] - log host : 10.146.40.92 , logPath : /data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/logs/5/28/33.log , logServer port : 50051 [INFO] 2021-07-03 09:42:08.817 org.apache.dolphinscheduler.service.log.LogClientService:[72] - roll view log, host : 10.146.40.92, port : 50051, path /data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/logs/5/28/33.log, skipLineNum 0 ,limit 1000 [ERROR] 2021-07-03 09:42:18.819 org.apache.dolphinscheduler.service.log.LogClientService:[85] - roll view log error org.apache.dolphinscheduler.remote.exceptions.RemotingTimeoutException: wait response on the channel Host{address='10.146.40.92:50051'} timeout 10000 at org.apache.dolphinscheduler.remote.NettyRemotingClient.sendSync(NettyRemotingClient.java:287) at org.apache.dolphinscheduler.service.log.LogClientService.rollViewLog(LogClientService.java:78) at org.apache.dolphinscheduler.api.service.LoggerService.queryLog(LoggerService.java:78) at org.apache.dolphinscheduler.api.controller.LoggerController.queryLog(LoggerController.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1633) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:97) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:94) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:561) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:766) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) at java.lang.Thread.run(Thread.java:745) [WARN] 2021-07-03 09:42:20.301 org.apache.dolphinscheduler.remote.future.ResponseFuture:[185] - remove timeout request : ResponseFuture{opaque=6, timeoutMillis=10000, invokeCallback=null, releaseSemaphore=null, latch=java.util.concurrent.CountDownLatch@7892de39[Count = 1], beginTimestamp=1625276528819, responseCommand=null, sendOk=true, cause=null} [INFO] 2021-07-03 09:42:22.437 org.apache.dolphinscheduler.api.controller.UsersController:[299] - login user admin,get user info [INFO] 2021-07-03 09:42:22.626 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[257] - query process definition list, login user:admin, project name:wmz-project1 [INFO] 2021-07-03 09:42:22.627 org.apache.dolphinscheduler.api.controller.ProjectController:[264] - login user admin, query all project list [INFO] 2021-07-03 09:42:22.627 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[201] - query process instance detail by id, login user:admin,project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:42:22.627 org.apache.dolphinscheduler.api.controller.ResourcesController:[177] - query resource list, login user:admin, resource type:FILE [INFO] 2021-07-03 09:42:22.628 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:, program type:{} [INFO] 2021-07-03 09:42:22.628 org.apache.dolphinscheduler.api.controller.ResourcesController:[293] - query resource list, login user:admin, resource type:PYTHON, program type:{} [INFO] 2021-07-03 09:42:22.719 org.apache.dolphinscheduler.api.controller.WorkerGroupController:[135] - query all worker group: login user admin [INFO] 2021-07-03 09:42:22.720 org.apache.dolphinscheduler.api.controller.TenantController:[135] - login user admin, query tenant list [INFO] 2021-07-03 09:42:22.721 org.apache.dolphinscheduler.api.controller.AlertGroupController:[96] - login user admin, query all alertGroup [INFO] 2021-07-03 09:42:22.721 org.apache.dolphinscheduler.api.controller.WorkerGroupController:[135] - query all worker group: login user admin [INFO] 2021-07-03 09:42:22.837 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[130] - query task instance list by process instance id, login user:admin, project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:42:41.142 org.apache.dolphinscheduler.api.controller.LoggerController:[77] - login user admin, view 33 task instance log ,skipLineNum 0 , limit 1000 [INFO] 2021-07-03 09:42:41.144 org.apache.dolphinscheduler.api.service.LoggerService:[76] - log host : 10.146.40.92 , logPath : /data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/logs/5/28/33.log , logServer port : 50051 [INFO] 2021-07-03 09:42:41.144 org.apache.dolphinscheduler.service.log.LogClientService:[72] - roll view log, host : 10.146.40.92, port : 50051, path /data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/logs/5/28/33.log, skipLineNum 0 ,limit 1000 [ERROR] 2021-07-03 09:42:51.147 org.apache.dolphinscheduler.service.log.LogClientService:[85] - roll view log error org.apache.dolphinscheduler.remote.exceptions.RemotingTimeoutException: wait response on the channel Host{address='10.146.40.92:50051'} timeout 10000 at org.apache.dolphinscheduler.remote.NettyRemotingClient.sendSync(NettyRemotingClient.java:287) at org.apache.dolphinscheduler.service.log.LogClientService.rollViewLog(LogClientService.java:78) at org.apache.dolphinscheduler.api.service.LoggerService.queryLog(LoggerService.java:78) at org.apache.dolphinscheduler.api.controller.LoggerController.queryLog(LoggerController.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1633) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:97) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:94) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:561) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:766) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) at java.lang.Thread.run(Thread.java:745) [WARN] 2021-07-03 09:42:52.301 org.apache.dolphinscheduler.remote.future.ResponseFuture:[185] - remove timeout request : ResponseFuture{opaque=7, timeoutMillis=10000, invokeCallback=null, releaseSemaphore=null, latch=java.util.concurrent.CountDownLatch@6283ae1c[Count = 1], beginTimestamp=1625276561146, responseCommand=null, sendOk=true, cause=null} [INFO] 2021-07-03 09:43:53.372 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[130] - query task instance list by process instance id, login user:admin, project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:43:53.477 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[201] - query process instance detail by id, login user:admin,project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:45:23.371 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[130] - query task instance list by process instance id, login user:admin, project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:45:23.488 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[201] - query process instance detail by id, login user:admin,project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:46:53.382 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[130] - query task instance list by process instance id, login user:admin, project name:wmz-project1, process instance id:28 [INFO] 2021-07-03 09:46:53.481 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[201] - query process instance detail by id, login user:admin,project name:wmz-project1, process instance id:28 ``` And this is worker log: ``` [INFO] 2021-07-03 09:42:05.385 org.apache.dolphinscheduler.server.worker.processor.TaskExecuteProcessor:[110] - received command : TaskExecuteRequestCommand{taskExecutionContext='{"cmdTypeIfComplement":0,"dataxTaskExecutionContext":{"dataSourceId":0,"dataTargetId":0,"sourcetype":0,"targetType":0},"executorId":1,"procedureTaskExecutionContext":{},"processDefineId":5,"processId":0,"processInstanceId":28,"projectId":2,"queue":"default","resources":{},"sqlTaskExecutionContext":{"warningGroupId":0},"sqoopTaskExecutionContext":{"dataSourceId":0,"dataTargetId":0,"sourcetype":0,"targetType":0},"taskInstanceId":33,"taskJson":"{\"conditionResult\":\"{\\\"successNode\\\":[\\\"\\\"],\\\"failedNode\\\":[\\\"\\\"]}\",\"conditionsTask\":false,\"depList\":[],\"dependence\":\"{}\",\"forbidden\":false,\"id\":\"tasks-98253\",\"maxRetryTimes\":0,\"name\":\"shell-node1\",\"params\":\"{\\\"rawScript\\\":\\\"echo \\\\\\\"here is shell\\\\\\\"\\\",\\\"localParams\\\":[],\\\"resourceList\\\":[]}\",\"preTa sks\":\"[]\",\"retryInterval\":1,\"runFlag\":\"NORMAL\",\"taskInstancePriority\":\"MEDIUM\",\"taskTimeoutParameter\":{\"enable\":false,\"interval\":0},\"timeout\":\"{\\\"enable\\\":false,\\\"strategy\\\":\\\"\\\"}\",\"type\":\"SHELL\",\"workerGroup\":\"default\"}","taskName":"shell-node1","taskTimeout":0,"taskTimeoutStrategy":0,"taskType":"SHELL","tenantCode":"root","workerGroup":"default"}'} [INFO] 2021-07-03 09:42:05.387 org.apache.dolphinscheduler.server.worker.processor.TaskExecuteProcessor:[137] - task instance local execute path : /tmp/dolphinscheduler/exec/process/2/5/28/33 [INFO] 2021-07-03 09:42:05.387 org.apache.dolphinscheduler.common.utils.FileUtils:[115] - create dir success /tmp/dolphinscheduler/exec/process/2/5/28/33 [INFO] 2021-07-03 09:42:05.387 - [taskAppId=TASK-5-28-33]:[115] - create dir success /tmp/dolphinscheduler/exec/process/2/5/28/33 [INFO] 2021-07-03 09:42:05.390 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[106] - script path : /tmp/dolphinscheduler/exec/process/2/5/28/33 [INFO] 2021-07-03 09:42:05.391 - [taskAppId=TASK-5-28-33]:[85] - shell task params {"rawScript":"echo \"here is shell\"","localParams":[],"resourceList":[]} [INFO] 2021-07-03 09:42:05.392 - [taskAppId=TASK-5-28-33]:[160] - raw script : echo "here is shell" [INFO] 2021-07-03 09:42:05.392 - [taskAppId=TASK-5-28-33]:[161] - task execute path : /tmp/dolphinscheduler/exec/process/2/5/28/33 [INFO] 2021-07-03 09:42:05.393 - [taskAppId=TASK-5-28-33]:[87] - tenantCode user:root, task dir:5_28_33 [INFO] 2021-07-03 09:42:05.393 - [taskAppId=TASK-5-28-33]:[92] - create command file:/tmp/dolphinscheduler/exec/process/2/5/28/33/5_28_33.command [INFO] 2021-07-03 09:42:05.393 - [taskAppId=TASK-5-28-33]:[111] - command : #!/bin/sh BASEDIR=$(cd `dirname $0`; pwd) cd $BASEDIR source /data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/conf/env/dolphinscheduler_env.sh /tmp/dolphinscheduler/exec/process/2/5/28/33/5_28_33_node.sh [INFO] 2021-07-03 09:42:05.395 - [taskAppId=TASK-5-28-33]:[327] - task run command: sudo -u root sh /tmp/dolphinscheduler/exec/process/2/5/28/33/5_28_33.command [INFO] 2021-07-03 09:42:05.397 - [taskAppId=TASK-5-28-33]:[208] - process start, process id is: 1459 [INFO] 2021-07-03 09:42:05.397 - [taskAppId=TASK-5-28-33]:[217] - process has exited, execute path:/tmp/dolphinscheduler/exec/process/2/5/28/33, processId:1459 ,exitStatusCode:0 [INFO] 2021-07-03 09:42:05.398 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[141] - task instance id : 33,task final status : FAILURE [INFO] 2021-07-03 09:42:05.399 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[161] - develop mode is: false [INFO] 2021-07-03 09:42:05.399 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[179] - exec local path: /tmp/dolphinscheduler/exec/process/2/5/28/33 cleared. [INFO] 2021-07-03 09:42:06.398 - [taskAppId=TASK-5-28-33]:[129] - -> /usr/bin/nohup: invalid option -- 'u' Try '/usr/bin/nohup --help' for more information. [INFO] 2021-07-03 09:42:06.398 - [taskAppId=TASK-5-28-33]:[127] - FINALIZE_SESSION ``` This is master log: ``` [INFO] 2021-07-03 09:42:04.450 org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[145] - find one command: id: 28, type: START_PROCESS [INFO] 2021-07-03 09:42:04.460 org.apache.dolphinscheduler.server.master.runner.MasterSchedulerService:[153] - start master exec thread , split DAG ... [INFO] 2021-07-03 09:42:04.462 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[315] - prepare process :28 end [INFO] 2021-07-03 09:42:04.464 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[792] - add task to stand by list: shell-node1 [INFO] 2021-07-03 09:42:04.465 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[805] - remove task from stand by list: shell-node1 [INFO] 2021-07-03 09:42:04.468 org.apache.dolphinscheduler.service.process.ProcessService:[845] - start submit task : shell-node1, instance id:28, state: RUNNING_EXECUTION [INFO] 2021-07-03 09:42:04.470 org.apache.dolphinscheduler.service.process.ProcessService:[858] - end submit task to db successfully:shell-node1 state:SUBMITTED_SUCCESS complete, instance id:28 state: RUNNING_EXECUTION [INFO] 2021-07-03 09:42:04.471 org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[216] - task ready to submit: TaskInstance{id=33, name='shell-node1', taskType='SHELL', processDefinitionId=5, processInstanceId=28, processInstanceName='null', taskJson='{"conditionResult":"{\"successNode\":[\"\"],\"failedNode\":[\"\"]}","conditionsTask":false,"depList":[],"dependence":"{}","forbidden":false,"id":"tasks-98253","maxRetryTimes":0,"name":"shell-node1","params":"{\"rawScript\":\"echo \\\"here is shell\\\"\",\"localParams\":[],\"resourceList\":[]}","preTasks":"[]","retryInterval":1,"runFlag":"NORMAL","taskInstancePriority":"MEDIUM","taskTimeoutParameter":{"enable":false,"interval":0},"timeout":"{\"enable\":false,\"strategy\":\"\"}","type":"SHELL","workerGroup":"default"}', state=SUBMITTED_SUCCESS, submitTime=Sat Jul 03 09:42:04 HKT 2021, startTime=null, endTime=null, host='null', executePath='null', logPath='null', retryTimes=0, alertFlag=NO, processInstance=null, proc essDefine=null, pid=0, appLink='null', flag=YES, dependency='null', duration=null, maxRetryTimes=0, retryInterval=1, taskInstancePriority=MEDIUM, processInstancePriority=MEDIUM, dependentResult='null', workerGroup='default', executorId=1, executorName='null'} [INFO] 2021-07-03 09:42:04.471 org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[227] - master submit success, task : shell-node1 [INFO] 2021-07-03 09:42:04.472 org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[116] - wait task: process id: 28, task id:33, task name:shell-node1 complete [INFO] 2021-07-03 09:42:05.391 org.apache.dolphinscheduler.server.master.processor.TaskAckProcessor:[69] - taskAckCommand : TaskExecuteAckCommand{taskInstanceId=33, startTime=Sat Jul 03 09:42:05 HKT 2021, host='10.146.40.92:1234', status=1, logPath='/data1/wangmingzhi/apache-dolphinscheduler-1.3.6-bin/logs/5/28/33.log', executePath='/tmp/dolphinscheduler/exec/process/2/5/28/33'} [INFO] 2021-07-03 09:42:05.400 org.apache.dolphinscheduler.server.master.processor.TaskResponseProcessor:[71] - received command : TaskExecuteResponseCommand{taskInstanceId=33, status=6, endTime=Sat Jul 03 09:42:05 HKT 2021, processId=1459, appIds=''} [INFO] 2021-07-03 09:42:06.478 org.apache.dolphinscheduler.server.master.runner.MasterTaskExecThread:[102] - task :shell-node1 id:33, process id:28, exec thread completed [INFO] 2021-07-03 09:42:07.472 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[864] - task :shell-node1, id:33 complete, state is FAILURE [INFO] 2021-07-03 09:42:08.474 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[764] - work flow process instance [id: 28, name:test3-0-1625276524454], state change from RUNNING_EXECUTION to FAILURE, cmd type: START_PROCESS [INFO] 2021-07-03 09:42:08.477 org.apache.dolphinscheduler.server.master.runner.MasterExecThread:[925] - process:28 end, state :FAILURE ``` Maybe you want the full worker log, I think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
