github-actions[bot] commented on issue #14583: URL: https://github.com/apache/dolphinscheduler/issues/14583#issuecomment-1641462975
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement. ### Description When I use helm to install dolphin scheduler(v3.1.7), I installed datax plugin as document describes, I found the loss of python2.7 but I can't found python2.7 instruction in datax plugin page. I think it should be described detailed. worker datax error logs below: ``` [INFO] 2023-07-19 13:09:25.054 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskDispatchProcessor:[100] - [WorkflowInstance-0][TaskInstance-0] - Receive task dispatch request, command: TaskDispatchCommand(super=BaseCommand(messageSenderAddress=dolphinscheduler-master-0.dolphinscheduler-master-headless:5678, messageReceiverAddress=null, messageSendTime=1689743365049), taskExecutionContext=TaskExecutionContext(taskInstanceId=15, taskName=同步 hrdb.meal_info 表至 216, firstSubmitTime=Wed Jul 19 13:09:25 CST 2023, startTime=null, taskType=DATAX, host=null, executePath=null, logPath=null, taskJson=null, processId=0, processDefineCode=10270021725504, processDefineVersion=2, appIds=null, processInstanceId=15, scheduleTime=null, globalParams=null, executorId=1, cmdTypeIfComplement=0, tenantCode=root, queue=default, processDefineId=0, projectId=0, projectCode=10267113244896, taskParams={"localParams":[],"resourceList":[],"customConfig":0,"dsType":"MYSQL","dataSource":3,"dtType":"MYSQL","dataTarget":1,"sql":"SELECT id,\r\n claimant_account,\r\n claimant_accname,\r\n using_user_account,\r\n using_user_accname,\r\n amount,\r\n oa_flowno,\r\n status,\r\n restaurant_code,\r\n restaurant_name,\r\n effective_time,\r\n expiration_time,\r\n allocation_time,\r\n writeoff_time,\r\n assignment_time,\r\n verification_time,\r\n creater,\r\n creater_name,\r\n create_time,\r\n updater,\r\n update_name,\r\n update_time,\r\n source,\r\n meal_number,\r\n meal_total,\r\n parent_id\r\nFROM aiko_hr.meal_info;","targetTable":"hrdb.meal_info","jobSpeedByte":0,"jobSpeedRecord":1000,"preStatements":[],"postStatements":[],"xms":1,"xmx":1}, envFile=null, environmentConfig=null, definedParams=null, prepareParamsMap=null, taskAppId=null, taskTimeoutStrategy=null, taskTimeout=2147483647, workerGroup=default, delayTime=0, currentExecutionStatus=TaskExecutionS desc='submit success'}, taskLogName=null, resourceParametersHelper=org.apache.dolphinscheduler.plugin.task.api.parameters.resource.ResourceParametersHelper@2f80a2a0, endTime=null, sqlTaskExecutionContext=null, k8sTaskExecutionContext=K8sTaskExecutionContext{configYaml='null'}, resources={}, varPool=null, dryRun=0, paramsMap={}, dataQualityTaskExecutionContext=org.apache.dolphinscheduler.plugin.task.api.DataQualityTaskExecutionContext@21474920, cpuQuota=-1, memoryMax=-1)) [INFO] 2023-07-19 13:09:25.055 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[100] - [WorkflowInstance-15][TaskInstance-15] - Set task logger name: taskAppId=TASK-20230719-10270021725504_2-15-15 [INFO] 2023-07-19 13:09:25.055 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskDispatchProcessor:[143] - [WorkflowInstance-15][TaskInstance-15] - Submit task to wait queue success, current queue size is 1 [INFO] 2023-07-19 13:09:25.056 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[155] - [WorkflowInstance-15][TaskInstance-15] - Begin to pulling task [INFO] 2023-07-19 13:09:25.058 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[187] - [WorkflowInstance-15][TaskInstance-15] - Begin to initialize task [INFO] 2023-07-19 13:09:25.058 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[191] - [WorkflowInstance-15][TaskInstance-15] - Set task startTime: Wed Jul 19 13:09:25 CST 2023 [INFO] 2023-07-19 13:09:25.058 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[195] - [WorkflowInstance-15][TaskInstance-15] - Set task envFile: /opt/dolphinscheduler/conf/dolphinscheduler_env.sh [INFO] 2023-07-19 13:09:25.058 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[200] - [WorkflowInstance-15][TaskInstance-15] - Set task appId: 15_15 [INFO] 2023-07-19 13:09:25.058 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[202] - [WorkflowInstance-15][TaskInstance-15] - End initialize task [INFO] 2023-07-19 13:09:25.059 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[208] - [WorkflowInstance-15][TaskInstance-15] - Set task status to TaskExecutionStatus{code=1, desc='running'} [INFO] 2023-07-19 13:09:25.059 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[211] - [WorkflowInstance-15][TaskInstance-15] - TenantCode:root check success [INFO] 2023-07-19 13:09:25.059 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[214] - [WorkflowInstance-15][TaskInstance-15] - ProcessExecDir:/tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15 check success [INFO] 2023-07-19 13:09:25.059 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[217] - [WorkflowInstance-15][TaskInstance-15] - Resources:{} check success [INFO] 2023-07-19 13:09:25.060 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[229] - [WorkflowInstance-15][TaskInstance-15] - Task plugin: DATAX create success [INFO] 2023-07-19 13:09:25.060 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[137] - [WorkflowInstance-15][TaskInstance-15] - datax task params {"localParams":[],"resourceList":[],"customConfig":0,"dsType":"MYSQL","dataSource":3,"dtType":"MYSQL","dataTarget":1,"sql":"SELECT id,\r\n claimant_account,\r\n claimant_accname,\r\n using_user_account,\r\n using_user_accname,\r\n amount,\r\n oa_flowno,\r\n status,\r\n restaurant_code,\r\n restaurant_name,\r\n effective_time,\r\n expiration_time,\r\n allocation_time,\r\n writeoff_time,\r\n assignment_time,\r\n verification_time,\r\n creater,\r\n creater_name,\r\n create_time,\r\n updater,\r\n update_name,\r\n update_time,\r\n source,\r\n meal_number,\r\n meal_total,\r\n parent_id\r\nFROM aiko_hr.meal_info;","targetTable":"hrdb.meal_info","jobSpeedByte":0,"jobSpeedRecord":1000,"preStatements":[],"postStatements":[],"xms":1,"xmx":1} [INFO] 2023-07-19 13:09:25.060 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[232] - [WorkflowInstance-15][TaskInstance-15] - Success initialized task plugin instance success [INFO] 2023-07-19 13:09:25.060 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[235] - [WorkflowInstance-15][TaskInstance-15] - Success set taskVarPool: null [INFO] 2023-07-19 13:09:25.061 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[86] - [WorkflowInstance-15][TaskInstance-15] - Begin to create command file:/tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15/15_15.command [INFO] 2023-07-19 13:09:25.061 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[123] - [WorkflowInstance-15][TaskInstance-15] - Success create command file, command: #!/bin/bash BASEDIR=$(cd `dirname $0`; pwd) cd $BASEDIR source /opt/dolphinscheduler/conf/dolphinscheduler_env.sh /tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15/15_15_node.sh [INFO] 2023-07-19 13:09:25.065 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[322] - [WorkflowInstance-15][TaskInstance-15] - task run command: sudo -u root -E bash /tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15/15_15.command [INFO] 2023-07-19 13:09:25.065 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[219] - [WorkflowInstance-15][TaskInstance-15] - process start, process id is: 6444 [INFO] 2023-07-19 13:09:25.074 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[241] - [WorkflowInstance-15][TaskInstance-15] - process has exited. execute path:/tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15, processId:6444 ,exitStatusCode:127 ,processWaitForStatus:true ,processExitValue:127 [INFO] 2023-07-19 13:09:25.075 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[261] - [WorkflowInstance-15][TaskInstance-15] - Send task execute result to master, the current task status: TaskExecutionStatus{code=6, desc='failure'} [INFO] 2023-07-19 13:09:25.075 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[114] - [WorkflowInstance-15][TaskInstance-15] - Remove the current task execute context from worker cache [INFO] 2023-07-19 13:09:25.075 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[269] - [WorkflowInstance-15][TaskInstance-15] - The current execute mode isn't develop mode, will clear the task execute file: /tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15 [INFO] 2023-07-19 13:09:25.075 +0800 TaskLogLogger-class org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable:[284] - [WorkflowInstance-15][TaskInstance-15] - Success clear the task execute file: /tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15 [INFO] 2023-07-19 13:09:25.898 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskExecuteRunningAckProcessor:[60] - [WorkflowInstance-0][TaskInstance-15] - task execute running ack command : TaskExecuteRunningAckMessage(success=true, taskInstanceId=15) [INFO] 2023-07-19 13:09:25.902 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskExecuteResultAckProcessor:[60] - [WorkflowInstance-0][TaskInstance-15] - Receive task execute response ack command : TaskExecuteAckCommand(super=BaseCommand(messageSenderAddress=dolphinscheduler-master-0.dolphinscheduler-master-headless:5678, messageReceiverAddress=dolphinscheduler-worker-0.dolphinscheduler-worker-headless:1234, messageSendTime=1689743365900), taskInstanceId=15, success=true) [INFO] 2023-07-19 13:09:26.067 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[208] - [WorkflowInstance-0][TaskInstance-0] - -> /tmp/dolphinscheduler/exec/process/root/10267113244896/10270021725504_2/15/15/15_15_node.sh: line 1: /usr/bin/python2.7: No such file or directory [INFO] 2023-07-19 13:09:26.067 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[202] - [WorkflowInstance-0][TaskInstance-0] - FINALIZE_SESSION [INFO] 2023-07-19 13:09:34.199 +0800 org.apache.dolphinscheduler.server.worker.task.WorkerHeartBeatTask:[89] - [WorkflowInstance-0][TaskInstance-0] - Success write worker group heartBeatInfo into registry, workerRegistryPath: /nodes/worker/dolphinscheduler-worker-0.dolphinscheduler-worker-headless:1234 workerHeartBeatInfo: {"startupTime":1689728609561,"reportTime":1689743374197,"cpuUsage":0.0,"memoryUsage":0.03,"loadAverage":0.03,"availablePhysicalMemorySize":485.98,"maxCpuloadAvg":160.0,"reservedMemory":0.3,"diskAvailable":1792.04,"serverStatus":0,"processId":8,"workerHostWeight":100,"workerWaitingTaskCount":0,"workerExecThreadCount":100} ``` ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
