[
https://issues.apache.org/jira/browse/HUDI-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451521#comment-17451521
]
Danny Chen commented on HUDI-2891:
----------------------------------
Fixed via master branch: ea009b55a385564f76beb99468f363edd3c4828a
> Kafka-connect sink still uses timeline-server-based markers as default
> ----------------------------------------------------------------------
>
> Key: HUDI-2891
> URL: https://issues.apache.org/jira/browse/HUDI-2891
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> By default, Kafka-connect sink using Java client should use direct markers by
> default. Errors are thrown if timeline-server-based markers are used. This
> could be because each sink task worker starts its own embedded timeline
> server, causing concurrent writes to the same marker file, leading to
> undefined behavior.
>
> {code:java}
> Found checksum error: b[0,
> 91]=706172746974696f6e5f342f31463946373736444442363643353038373631333144443446443036303332365f302d302d305f32303231313132393137343031393530362e706172717565742e6d61726b65722e415050454e440a
>
> (org.apache.hadoop.fs.FSInputChecker:309)org.apache.hadoop.fs.ChecksumException:
> Checksum error:
> file:/tmp/hoodie/hudi-test-topic/.hoodie/.temp/20211129174217738/MARKERS6 at
> 0 exp: -509813218 got: -1454124197 {code}
>
>
> [https://gist.github.com/yihua/12d02aec4174b657b2a8ac3cd7972a5a]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
