ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710023639
@bvaradar Isnt the ``` --source-ordering-field _ts_ms ``` Then
precombine should be looking in for _ts_ms right for deletion ?
Delete worked fine for me as I replace ```--hoodie-conf
hoodie.compaction.payload.class=org.apache.hudi.common.model.DebeziumAvroPayload
``` with ``` --payload-class org.apache.hudi.common.model.DebeziumAvroPayload
```
I checked the same scenario using the spark submit you had provided with
AvroDFSSource and Im getting the same results but when I execute for
AvroKafkaSource the earlier error comes up
```
+-------------------+--------------------+------------------+----------------------+--------------------+-------------+---+------+----+--------------------+--------------+------------------+----+----------------+
|_hoodie_commit_time|_hoodie_commit_seqno|_hoodie_record_key|_hoodie_partition_path|
_hoodie_file_name| _ts_ms|_op|inc_id|year|
violation_desc|violation_code|case_individual_id|flag|last_modified_ts|
+-------------------+--------------------+------------------+----------------------+--------------------+-------------+---+------+----+--------------------+--------------+------------------+----+----------------+
| 20201016125655| 20201016125655_0_1| 12|
|b2699714-2916-4c4...|1602791105044| c| 12|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601385000000000|
| 20201016125655| 20201016125655_0_2| 36|
|b2699714-2916-4c4...|1602791105045| c| 36|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601398800000000|
| 20201016125655| 20201016125655_0_3| 11|
|b2699714-2916-4c4...|1602791105044| c| 11|2020|UNREASONABLE SPEE...|
1180D| 17475574| I|1601381400000000|
| 20201016125655| 20201016125655_0_4| 37|
|b2699714-2916-4c4...|1602791105045| c| 37|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601399400000000|
| 20201016125655| 20201016125655_0_5| 13|
|b2699714-2916-4c4...|1602791105044| c| 13|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601388600000000|
| 20201016125655| 20201016125655_0_6| 4|
|b2699714-2916-4c4...|1602791105043| c| 4|2019|AGGRAVATED UNLIC ...|
5112A2| 17475368| I|1569337200000000|
| 20201016125655| 20201016125655_0_7| 10|
|b2699714-2916-4c4...|1602791105043| c| 10|2020|UNREASONABLE SPEE...|
1180D| 17475574| I|1601377800000000|
| 20201016125655| 20201016125655_0_8| 34|
|b2699714-2916-4c4...|1602791105044| c| 34|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601392200000000|
| 20201016125655| 20201016125655_0_9| 35|
|b2699714-2916-4c4...|1602791105045| c| 35|2020|UNREASONABLE SPEE...|
1180E| 17475574| I|1601395800000000|
| 20201016125655| 20201016125655_0_10| 38|
|b2699714-2916-4c4...|1602791105045| c| 38|2020|UNREASONABLE SPEE...|
1180D| 17475574| I|1601402400000000|
| 20201016125655| 20201016125655_0_11| 1|
|b2699714-2916-4c4...|1602791105042| c| 1|2016|DRIVING WHILE INT...|
11923| 17475366| I|1600945380000000|
| 20201016125655| 20201016125655_0_12| 2|
|b2699714-2916-4c4...|1602791105043| c| 2|2020|UNREASONABLE SPEE...|
2180F| 17475569| U|1601377200000000|
| 20201016125655| 20201016125655_0_13| 9|
|b2699714-2916-4c4...|1602791105043| c| 9|2020|UNREASONABLE SPEE...|
1180E| 17475573| I|1601377200000000|
+-------------------+--------------------+------------------+----------------------+--------------------+-------------+---+------+----+--------------------+--------------+------------------+----+----------------+
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
