loukey-lj commented on PR #6602:
URL: https://github.com/apache/hudi/pull/6602#issuecomment-1240324396
> Hi loukey-lj, could you share which kind of filesystem you use? HDFS or S3
or any other type. The problem you mentioned should be covered in UT
TestHoodieLogFormat#testRollover. I just have a try on master code and go over
HDFS implementation code. It has correct behaviour.
@guanziyue Thanks for review, the OutputStream used by the test class and
the actual task run is different, testRollover is HdfsDataOutputStream and the
other is SizeAwareFSDataOutputStream, SizeAwareFSDataOutputStream startPosition
always 0.
Maybe the right way
to fix this issue is change the constructor of SizeAwareFSDataOutputStream
`super(out, null) ` to `super(out, null, out.getPos())` , but it's used in
so many places that I can't assess the impact of this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
