[
https://issues.apache.org/jira/browse/HUDI-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pratyaksh Sharma updated HUDI-485:
----------------------------------
Description:
HiveIncrementalPuller checks the clause in incrementalSqlFile like this ->
if (!incrementalSQL.contains("`_hoodie_commit_time` > '%targetBasePath'"))
{ LOG.info("Incremental SQL : " + incrementalSQL + " does not contain
`_hoodie_commit_time` > %targetBasePath. Please add " + "this clause for
incremental to work properly."); throw new HoodieIncrementalPullSQLException(
"Incremental SQL does not have clause `_hoodie_commit_time` >
'%targetBasePath', which " + "means its not pulling incrementally"); }
Basically we are trying to add a placeholder here which is later replaced with
config.fromCommitTime here -
incrementalPullSQLtemplate.add("incrementalSQL", String.format(incrementalSQL,
config.fromCommitTime));
Hence, the above check needs to replaced with `_hoodie_commit_time` > %s
was:
HiveIncrementalPuller checks the clause in incrementalSqlFile like this ->
if (!incrementalSQL.contains("`_hoodie_commit_time` > '%targetBasePath'")) {
LOG.info("Incremental SQL : " + incrementalSQL
+ " does not contain `_hoodie_commit_time` > %targetBasePath. Please add "
+ "this clause for incremental to work properly.");
throw new HoodieIncrementalPullSQLException(
"Incremental SQL does not have clause `_hoodie_commit_time` >
'%targetBasePath', which "
+ "means its not pulling incrementally");
}
Basically we are trying to add a placeholder here which is later replaced with
config.fromCommitTime here -
incrementalPullSQLtemplate.add("incrementalSQL", String.format(incrementalSQL,
config.fromCommitTime));
Hence, the above check needs to replaced with `_hoodie_commit_time` >
%targetBasePath
> Check for where clause is wrong in HiveIncrementalPuller
> --------------------------------------------------------
>
> Key: HUDI-485
> URL: https://issues.apache.org/jira/browse/HUDI-485
> Project: Apache Hudi (incubating)
> Issue Type: Sub-task
> Components: Incremental Pull, newbie
> Reporter: Pratyaksh Sharma
> Assignee: Pratyaksh Sharma
> Priority: Major
>
> HiveIncrementalPuller checks the clause in incrementalSqlFile like this ->
> if (!incrementalSQL.contains("`_hoodie_commit_time` > '%targetBasePath'"))
> { LOG.info("Incremental SQL : " + incrementalSQL + " does not contain
> `_hoodie_commit_time` > %targetBasePath. Please add " + "this clause for
> incremental to work properly."); throw new HoodieIncrementalPullSQLException(
> "Incremental SQL does not have clause `_hoodie_commit_time` >
> '%targetBasePath', which " + "means its not pulling incrementally"); }
> Basically we are trying to add a placeholder here which is later replaced
> with config.fromCommitTime here -
> incrementalPullSQLtemplate.add("incrementalSQL",
> String.format(incrementalSQL, config.fromCommitTime));
> Hence, the above check needs to replaced with `_hoodie_commit_time` > %s
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
