balazsgerner commented on PR #8544:
URL: https://github.com/apache/nifi/pull/8544#issuecomment-2039019933
> In order for the append to work I had to set `Writing Strategy` to `Simple
write`, if I leave the default `Write and rename`, it actually deletes the
file. Is this intended? If not we should add another dependent property so if
AVRO is chosen, the Writing Strategy must be Simple Write.
To be honest, during the whole dev testing I left the 'Writing Strategy'
property on default value which is 'Write and rename', and it worked completely
fine for me. I double checked it now and it still does. As I see it, my ticket
does not affect the connection between writing strategy and conflict resolution
strategy so it should work just as before.
I also checked 'Writing strategy'='Simple write', and that also works for me
without any problems.
If you experience file deletion I assume your flow might need a tweak. I do
not know the processors you use exactly, but I had file deletion problems when
I left the 'GetHDFS' or the 'GetFile' processors 'Keep source file' on default
which is 'false'. In order to make my flow reexecutable I changed that value to
'true'.
For testing I used the following flow:
1. GetFile('Keep source file'='true') -> PutHDFS(append,AVRO,'Write and
rename'), but it also works with 'Simple write'
2. GetHDFS('Keep source file'='true') -> ConvertAvroToJSON (I just used this
step to validate the content of the avro file, not sure if the convert
processor exists upstream, I used a custom nifi instance for this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
