jfrazee commented on a change in pull request #5098:
URL: https://github.com/apache/nifi/pull/5098#discussion_r675229270
##########
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/PutAzureDataLakeStorage.java
##########
@@ -114,30 +119,23 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
final DataLakeServiceClient storageClient =
getStorageClient(context, flowFile);
final DataLakeFileSystemClient fileSystemClient =
storageClient.getFileSystemClient(fileSystem);
final DataLakeDirectoryClient directoryClient =
fileSystemClient.getDirectoryClient(directory);
- final DataLakeFileClient fileClient;
+ DataLakeFileClient fileClient;
final String conflictResolution =
context.getProperty(CONFLICT_RESOLUTION).getValue();
boolean overwrite = conflictResolution.equals(REPLACE_RESOLUTION);
+ final String tempFilePrefix =
defaultIfBlank(System.getProperty("tempFilePrefix"), TEMP_FILE_PREFIX);
try {
- fileClient = directoryClient.createFile(fileName, overwrite);
-
- final long length = flowFile.getSize();
- if (length > 0) {
- try (final InputStream rawIn = session.read(flowFile);
final BufferedInputStream bufferedIn = new BufferedInputStream(rawIn)) {
- uploadContent(fileClient, bufferedIn, length);
- } catch (Exception e) {
- removeTempFile(fileClient);
- throw e;
- }
- }
+ fileClient = directoryClient.createFile(tempFilePrefix +
fileName, true);
Review comment:
Since tempFilePrefix is more or less static, this opens up the
possibility for some weird behavior if two processors, nodes, threads, clusters
try to write a file with the same name.
Two solutions come to mind:
1. Add a random suffix
2. Make temp file prefix a processor property and allow for EL so the user
can account for that if needed
Also, because of this, I think it'd be good to open a follow-on issue to add
some lease handling. I wouldn't want that to hold back this change though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
