sekikn commented on issue #19174:
URL: https://github.com/apache/pulsar/issues/19174#issuecomment-1384674332
Thank you for the advice @tisonkun! I finally figured out the root cause.
Simply put, moving the `optionsBuilder` field to a local variable in the
`createTmpFile` method should fix the problem. I'll submit a PR later.
Following your advice, I set Alluxio's log level to DEBUG and got the
following error.
```
alluxio.exception.FileDoesNotExistException: Path
"/pulsar/tmp/7836e23f-78bc-4d3e-922f-004940749a5f_tmp.txt" does not exist.
at
alluxio.master.file.DefaultFileSystemMaster.checkLoadMetadataOptions(DefaultFileSystemMaster.java:1196)
at
alluxio.master.file.DefaultFileSystemMaster.getFileInfo(DefaultFileSystemMaster.java:870)
at
alluxio.master.file.DefaultFileSystemMaster.createFile(DefaultFileSystemMaster.java:1613)
at
alluxio.master.file.FileSystemMasterClientServiceHandler.lambda$createFile$4(FileSystemMasterClientServiceHandler.java:180)
```
The code in question is
https://github.com/Alluxio/alluxio/blob/v2.7.3/core/server/master/src/main/java/alluxio/master/file/DefaultFileSystemMaster.java#L1613.
Entering this route means `isOperationComplete` at L1610 returned `true`, and
it indicates the same operation (`createFile` here) is retried again.
Every Alluxio's filesystem operation is tracked by an unique ID if
`alluxio.master.filesystem.operation.retry.cache.enabled` is true (enabled by
default). But in our `AlluxioSink`, [the builder for CreateFilePOptions, in
which the operation ID is stored, is created once in the `open`
method](https://github.com/apache/pulsar/blob/master/pulsar-io/alluxio/src/main/java/org/apache/pulsar/io/alluxio/sink/AlluxioSink.java#L117)
and reused.
Because the operation ID is generated when the builder is instantiated, the
first `createFile` operation is successful,
but the subsequent ones are regerded as the same operation as the first one,
since the operation ID is not changed.
So we have to use a new builder object for each `createFile` operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
