ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-712404711
Not sure if this is gonna be of any help but attaching the latest logs. I
can see this messages towards the end
```
at
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-712404711
Not sure if this is gonna be of any help but attaching the latest logs. I
can see this messages towards the end
```
at
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710034993
I followed these steps :
```
- Took fresh clone of release-0.6.0 branch
- applied the patch provided
- build and used the jar to run below commands
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710034993
**AvroKafkaSource** :
```
spark-submit --packages org.apache.spark:spark-avro_2.11:2.4.4 --class
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710023639
@bvaradar Isnt the ``` --source-ordering-field _ts_ms ``` Then
precombine should be looking in for _ts_ms right for deletion ?
Delete worked fine for me as
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710023639
@bvaradar Isnt the ``` --source-ordering-field _ts_ms ``` Then
precombine should be looking in for _ts_ms right for deletion ?
Delete worked fine for me as
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710023639
@bvaradar Isnt the ``` --source-ordering-field _ts_ms ``` Then
precombine should be looking in for _ts_ms right for deletion ?
I checked the same scenario
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-710023639
@bvaradar Isnt the ``` --source-ordering-field _ts_ms ``` Then
precombine should be looking in for _ts_ms right for deletion ?
I checked the same scenario
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-709557478
@bvaradar I thought that at first. To confirm the same I retried the
scenario multiple times. Im getting the same error everytime. Only during
Deletes
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-708433619
@bvaradar The patch worked successfully for Insert and upserts except for
Delete.
I think due to the way debezium loads the delete changes into kafka this
issue is
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-705990567
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-706110206
@bvaradar Thanks for noticing it. I think that solved the previous error but
producing following error now :
```
20/10/09 10:32:09 INFO AppInfoParser: Kafka
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-706057675
@bvaradar Please find the details :
```
spark-submit --class
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer --jars
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-705990567
@bvaradar Yes Im using the above mentioned url for schema
```
{
"connect.name": "airflow.public.motor_crash_violation_incidents.Envelope",
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-705990567
Yes Im using the above mentioned url for schema
```
{
"connect.name": "airflow.public.motor_crash_violation_incidents.Envelope",
"fields": [
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-70517
I found this Error message in the logs :
```
INFO DAGScheduler: Job 11 finished: sum at DeltaSync.java:406, took
0.108824 s
20/10/07 20:27:07 ERROR
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-704918685
@bvaradar : Thanks for the code . I followed your instructions but tried to
add _is_hoodie_deleted column to the dataset using following code for testing
Im
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-704569933
@bvaradar I implemented the transformer class as
```public class DebeziumCustomTransformer implements Transformer {
private static final Logger LOG =
ashishmgofficial edited a comment on issue #2149:
URL: https://github.com/apache/hudi/issues/2149#issuecomment-704569933
@bvaradar I implemented the transformer class as
```public class DebeziumCustomTransformer implements Transformer {
private static final Logger LOG =
19 matches
Mail list logo
