cdmikechen edited a comment on issue #1073: [HUDI-377] Adding Delete() support to DeltaStreamer URL: https://github.com/apache/incubator-hudi/pull/1073#issuecomment-568942977 @vinothchandar > Are you asking for incremental pull to provide the both before and after images of a record like how > Oracle ogg CDC Stream is? if so, this is a much larger feature.. we can discuss on a separate JIRA. I mean if hudi can get increment data by spark datasource api like ```java Dataset<Row> hoodieIncViewDF = spark.read() .format("org.apache.hudi") .option(DataSourceReadOptions.VIEW_TYPE_OPT_KEY(), DataSourceReadOptions.VIEW_TYPE_INCREMENTAL_OPT_VAL()) .option(DataSourceReadOptions.BEGIN_INSTANTTIME_OPT_KEY(), <beginInstantTime>) .load(tablePath); ``` Or Hive query like ```sql set hoodie.lims_method.consume.mode=INCREMENTAL; set hoodie.lims_method.consume.start.timestamp=xxxx; set hoodie.lims_method.consume.max.commits=1; select `_hoodie_commit_time`, xxxx from table_name where `_hoodie_commit_time` >= 'xxxx'; ``` Should we also support some incremental view or api or method to get delete rows after delete action. I think this should be considered at the same time after this issue and other related issues are submitted.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
