taherk77 commented on issue #917: [HUDI-251] JDBC incremental load to HUDI with DeltaStreamer URL: https://github.com/apache/incubator-hudi/pull/917#issuecomment-533891750 https://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_incremental_imports mentions --last_value. I think we should be computing that within the source. My vision is once the user specifies the incremental column and the interval we should be pulling the data every interval. so in short deltastreamer will never get shut down unless it is manually shut and will keep pulling new data from the JDBC source. The other option is that when we set is_incremental as false then we pull all the data at once and then write it.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
