EngonVHKxZ opened a new issue, #5797: URL: https://github.com/apache/seatunnel/issues/5797
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened When I synchronize data to FTP, may I create a temporary file at a certain step? But the folder creation failed, so the temporary files also failed. When recursively creating a folder, if the folder does not exist, it should have been created continuously from the root directory until all directories are created. In fact, after creating the first non-existent directory, it stops ### SeaTunnel Version 2.3.3 ### SeaTunnel Config ```conf env { job.mode="BATCH" execution.parallelism=5 }, source { FtpFile { host = "xxx.xxx.xxx.xxx" port = 7111 user = "LIS" username = "LIS" password = "LIS341" file_format_type = "csv" path = "/LIS/seatunnel/color.csv" schema = { fields { ID = string NAME = string } } sheet_name = "red_20231025135549" } }, transform {}, sink { FtpFile { host = "xxx.xxx.xxx.xxx" port = 7111 username = "LIS" user = "LIS" password = "LIS341" path = "/LIS/seatunnel/color_copy.csv" custom_filename = "seatunnel" file_format_type = "csv" sink_columns = ["ID","NAME"] } } ``` ### Running Command ```shell run in SeaTunnelEngineExample.class ``` ### Error Exception ```log 2023-11-07 10:46:46,017 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== 2023-11-07 10:46:46,017 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 2023-11-07 10:46:46,017 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues 2023-11-07 10:46:46,017 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 2023-11-07 10:46:46,019 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:45) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Write this data [SeaTunnelRow{tableId=, kind=+I, fields=[ID, NAME]}] to file failed at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:232) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:61) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:76) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:51) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:52) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Write this data [SeaTunnelRow{tableId=, kind=+I, fields=[ID, NAME]}] to file failed at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:129) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:43) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:227) ... 15 more Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Open file output stream [/tmp/seatunnel\seatunnel\774102414088208385\27e338d164\T_774102414088208385_27e338d164_0_1\NON_PARTITION\T_774102414088208385_27e338d164_0_1_0.csv] failed at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.getOrCreateOutputStream(TextWriteStrategy.java:152) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.write(TextWriteStrategy.java:81) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:126) ... 17 more Caused by: java.io.IOException: create(): Mkdirs failed to create: /tmp/seatunnel/seatunnel/774102414088208385/27e338d164/T_774102414088208385_27e338d164_0_1/NON_PARTITION at org.apache.seatunnel.connectors.seatunnel.file.ftp.system.SeaTunnelFTPFileSystem.create(SeaTunnelFTPFileSystem.java:260) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1118) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1098) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1057) at org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils.getOutputStream(FileSystemUtils.java:108) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.getOrCreateOutputStream(TextWriteStrategy.java:138) ... 19 more at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184) ... 2 more 2023-11-07 10:46:46,019 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:45) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Write this data [SeaTunnelRow{tableId=, kind=+I, fields=[ID, NAME]}] to file failed at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:232) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:61) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:76) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:51) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:52) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Write this data [SeaTunnelRow{tableId=, kind=+I, fields=[ID, NAME]}] to file failed at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:129) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:43) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:227) ... 15 more Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Open file output stream [/tmp/seatunnel\seatunnel\774102414088208385\27e338d164\T_774102414088208385_27e338d164_0_1\NON_PARTITION\T_774102414088208385_27e338d164_0_1_0.csv] failed at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.getOrCreateOutputStream(TextWriteStrategy.java:152) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.write(TextWriteStrategy.java:81) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:126) ... 17 more Caused by: java.io.IOException: create(): Mkdirs failed to create: /tmp/seatunnel/seatunnel/774102414088208385/27e338d164/T_774102414088208385_27e338d164_0_1/NON_PARTITION at org.apache.seatunnel.connectors.seatunnel.file.ftp.system.SeaTunnelFTPFileSystem.create(SeaTunnelFTPFileSystem.java:260) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1118) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1098) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1057) at org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils.getOutputStream(FileSystemUtils.java:108) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.getOrCreateOutputStream(TextWriteStrategy.java:138) ... 19 more at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184) ... 2 more ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
