[
https://issues.apache.org/jira/browse/HUDI-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Mahindra resolved HUDI-2745.
-----------------------------------
> Record count does not match input after compaction is scheduled when running
> Hudi Kafka Connect sink
> ----------------------------------------------------------------------------------------------------
>
> Key: HUDI-2745
> URL: https://issues.apache.org/jira/browse/HUDI-2745
> Project: Apache Hudi
> Issue Type: Bug
> Components: Compaction
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.10.0
>
>
> Spark Shell command to do snapshot query:
> {code:java}
> val basePath = "/tmp/hoodie/hudi-test-topic"
> val df = spark.read.format("hudi").load(basePath)
> df.createOrReplaceTempView("hudi_test_table")
> spark.sql("select count(*) from hudi_test_table").show() {code}
> Two cases of count mismatch:
> (1) Compaction scheduled, more deltacommits later on: the count does not
> match input size. After compaction is executed. The count becomes correct.
> (2) Clustering scheduled, more deltacommits later on: the count is correct,
> equal to the input size. After clustering is executed, the count drops and
> becomes incorrect.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
