[GitHub] [incubator-hudi] dengziming commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module
dengziming commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module URL: https://github.com/apache/incubator-hudi/pull/1151#issuecomment-613764273 Thanks for your reply @vinothchandar . Do you mean we can just delete the 3 XxxDeltaStreamerExample and replace them with a run_examples.sh and some config files? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] dengziming commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module
dengziming commented on issue #1151: [WIP][HUDI-476] Add hudi-examples module URL: https://github.com/apache/incubator-hudi/pull/1151#issuecomment-612881257 Hello, @vinothchandar ,Sorry for the late reply. I want to address some of your comment and here are issues: 1. I tried make data prep part of the deltastreamer themselves and then also provide same defaults for input/output paths, but at last I found my code the same as `org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer#main`, it seems that we don't have to add deltastreamer examples because the HoodieDeltaStreamer class is already a complete deltastreamer example. Maybe we just need to add some simple examples rather than a very complete and unified example. 2. I wrote a `run_hoodie_examples.sh` to run the example and reuse spark-bundle/utilities-bundle instead of building a fat jar, but the build process of hudi-utilities-bundle will relocate `com.beust.jcommander.` to `org.apache.hudi.com.beust.jcommander.`, and my example have a dependency on `com.beust.jcommander.` and the spark-shell failed, so should I also add a relocation to pom.xml of hudi-examples. How do you think about these 2 problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] dengziming commented on issue #1151: [WIP] [HUDI-476] Add hudi-examples module
dengziming commented on issue #1151: [WIP] [HUDI-476] Add hudi-examples module URL: https://github.com/apache/incubator-hudi/pull/1151#issuecomment-581785306 > @dengziming are you still working on this? I tried to reuse more code but I can't improve it anymore, I think it's ready for review, thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] dengziming commented on issue #1151: [WIP] [HUDI-476] Add hudi-examples module
dengziming commented on issue #1151: [WIP] [HUDI-476] Add hudi-examples module URL: https://github.com/apache/incubator-hudi/pull/1151#issuecomment-573487458 > Could we also provide delta streamer examples? @vinothchandar hi, I am working on it, `HoodieDeltaStreamer` seems more complex so I need some time to debug and review the code, and it will be completed soon. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
