ruanwenjun commented on issue #5318:
URL: https://github.com/apache/seatunnel/issues/5318#issuecomment-1728722557
> I also encountered a similar problem.
>
> ##### version
> * `2.3.2`
>
> ##### Engine
> * `zeta`
>
> ##### Config
> ```
> env {
> # You can set flink configuration here
> execution.parallelism = 1
> job.mode = "BATCH"
>
> # You can set spark configuration here
> spark.app.name = "SeaTunnel"
> spark.executor.instances = 1
> spark.executor.cores = 1
> spark.executor.memory = "1g"
> spark.master = local
> }
>
> source {
> FakeSource {
> result_table_name = "sftp"
> schema = {
> fields {
> c_map = "map<string, string>"
> c_array = "array<int>"
> c_string = string
> c_boolean = boolean
> c_tinyint = tinyint
> c_smallint = smallint
> c_int = int
> c_bigint = bigint
> c_float = float
> c_double = double
> c_bytes = bytes
> c_date = date
> c_decimal = "decimal(38, 18)"
> c_timestamp = timestamp
> c_row = {
> c_map = "map<string, string>"
> c_array = "array<int>"
> c_string = string
> c_boolean = boolean
> c_tinyint = tinyint
> c_smallint = smallint
> c_int = int
> c_bigint = bigint
> c_float = float
> c_double = double
> c_bytes = bytes
> c_date = date
> c_decimal = "decimal(38, 18)"
> c_timestamp = timestamp
> }
> }
> }
> }
> }
>
> sink {
> SftpFile {
> host = "localhost"
> port = 22
> user = cheney
> password = xxxxxx
> path = "/home/cheney/tmp/seatunnel/json"
> source_table_name = "sftp"
> row_delimiter = "\n"
> partition_dir_expression = "${k0}=${v0}"
> is_partition_field_write_in_file = true
> file_name_expression = "${transactionId}_${now}"
> file_format_type = "json"
> filename_time_format = "yyyy.MM.dd"
> is_enable_transaction = true
> }
> }
> ```
>
> ##### Command
> ```shell
> ./bin/seatunnel.sh --config ./config/json/fake_to_sftp_file_json.conf -e
local
> ```
>
> ##### Error Exception
> ```shell
> 2023-08-18 16:04:46,600 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (744829028610867201), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-FakeSource-sftp]-SourceTask (1/1)] end with state FINISHED
> 2023-08-18 16:04:47,282 ERROR
org.apache.seatunnel.connectors.seatunnel.file.sink.commit.FileSinkAggregatedCommitter
- commit aggregatedCommitInfo error, aggregatedCommitInfo =
FileAggregatedCommitInfo(transactionMap={/tmp/seatunnel/seatunnel/744829028610867201/db1540c7c1/T_744829028610867201_db1540c7c1_0_1={/tmp/seatunnel/seatunnel/744829028610867201/db1540c7c1/T_744829028610867201_db1540c7c1_0_1/NON_PARTITION/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json=/home/cheney/tmp/seatunnel/json/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json}},
partitionDirAndValuesMap={})
>
org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException:
ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as
(read,list,write,move,copy,sync) etc...] - rename file
:[/tmp/seatunnel/seatunnel/744829028610867201/db1540c7c1/T_744829028610867201_db1540c7c1_0_1/NON_PARTITION/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json]
to
[/home/cheney/tmp/seatunnel/json/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json]
error
> at
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils.renameFile(FileSystemUtils.java:170)
~[connector-file-sftp-2.3.2.jar:2.3.2]
> at
org.apache.seatunnel.connectors.seatunnel.file.sink.commit.FileSinkAggregatedCommitter.lambda$commit$0(FileSinkAggregatedCommitter.java:52)
~[connector-file-sftp-2.3.2.jar:2.3.2]
> at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
> at
org.apache.seatunnel.connectors.seatunnel.file.sink.commit.FileSinkAggregatedCommitter.commit(FileSinkAggregatedCommitter.java:44)
~[connector-file-sftp-2.3.2.jar:2.3.2]
> at
org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask.notifyCheckpointComplete(SinkAggregatedCommitterTask.java:276)
~[seatunnel-starter.jar:2.3.2]
> at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointFinishedOperation.lambda$run$0(CheckpointFinishedOperation.java:91)
~[seatunnel-starter.jar:2.3.2]
> at
org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
[seatunnel-starter.jar:2.3.2]
> ```
>
> I am sure I have permission to rename directory.
>
> ```shell
> ❯ ll /home/cheney/tmp/
> total 20K
> 40937109 4.0K drwxr-xr-x 3 cheney cheney 4.0K 8月17日 14:33 .
> 37748738 12K drwx------ 116 cheney cheney 12K 8月18日 15:59 ..
> 40937110 4.0K drwxr-xr-x 3 cheney cheney 4.0K 8月17日 14:33 seatunnel
> ```
>
> ```shell
> ❯ ssh cheney@localhost -C mv
/tmp/seatunnel/seatunnel/744829028610867201/db1540c7c1/T_744829028610867201_db1540c7c1_0_1/NON_PARTITION/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json
/home/cheney/tmp/seatunnel/json/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json
> cheney@localhost's password:
> ❯ ls
/home/cheney/tmp/seatunnel/json/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json
>
/home/cheney/tmp/seatunnel/json/T_744829028610867201_db1540c7c1_0_1_2023.08.18_0.json
> ```
It seems you use `mv` command rather than rename
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
