bvaradar commented on issue #915: [HUDI-268] Shade and relocate Avro dependency in hadoop-mr-bundle URL: https://github.com/apache/incubator-hudi/pull/915#issuecomment-534552878 @umehrot2 : Unfortunately, the examples that I mentioned is not in the open source world. If you want to reproduce it, a high level direction would be 1. Implement HoodieRecordPayload class in a separate jar. (For testing : simply extend OverwriteWithLatestAvroPayload.java) 2. If you want to understand, how this is used in reader : See here : https://github.com/apache/incubator-hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordScanner.java#L109 3. This is a quick and dirty way to repro this : You can simply use the docker based demo steps as described in https://hudi.incubator.apache.org/docker_demo.html but have the following changes: (a) Pass --payload-class for all delta-streamer invocations with the class that you created in (1) (b) After you ingest the data once using deltastreamer, you can simply add an entry to hoodie.properties file in the dataset .hoodie folder -> Add "hoodie.compaction.payload.class = <Fully-Qualified-Class-Name-of-Payload-that-you-created-in-step-1> (c) Continue the steps. (d) In step (6a) where you query Realtime table, you should see the class getting instantiated
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
