danny0405 commented on code in PR #8993:
URL: https://github.com/apache/hudi/pull/8993#discussion_r1233168816
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/SqlQueryEqualityPreCommitValidator.java:
##########
@@ -54,13 +54,11 @@ protected String getQueryConfigName() {
@Override
protected void validateUsingQuery(String query, String prevTableSnapshot,
String newTableSnapshot, SQLContext sqlContext) {
- String queryWithPrevSnapshot =
query.replaceAll(HoodiePreCommitValidatorConfig.VALIDATOR_TABLE_VARIABLE,
prevTableSnapshot);
- String queryWithNewSnapshot =
query.replaceAll(HoodiePreCommitValidatorConfig.VALIDATOR_TABLE_VARIABLE,
newTableSnapshot);
- LOG.info("Running query on previous state: " + queryWithPrevSnapshot);
- Dataset<Row> prevRows = sqlContext.sql(queryWithPrevSnapshot).cache();
+ LOG.info("Running query on previous state: " + prevTableSnapshot);
+ Dataset<Row> prevRows = executeSqlQuery(sqlContext, query,
prevTableSnapshot).cache();
LOG.info("Total rows in prevRows " + prevRows.count());
- LOG.info("Running query on new state: " + queryWithNewSnapshot);
- Dataset<Row> newRows = sqlContext.sql(queryWithNewSnapshot).cache();
+ LOG.info("Running query on new state: " + newTableSnapshot);
Review Comment:
which line is the fix for the schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
