the-other-tim-brown commented on code in PR #8638:
URL: https://github.com/apache/hudi/pull/8638#discussion_r1194396609
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/transform/SqlQueryBasedTransformer.java:
##########
@@ -47,17 +49,21 @@ public Dataset<Row> apply(JavaSparkContext jsc,
SparkSession sparkSession, Datas
TypedProperties properties) {
String transformerSQL =
properties.getString(SqlTransformerConfig.TRANSFORMER_SQL.key());
if (null == transformerSQL) {
- throw new IllegalArgumentException("Missing configuration : (" +
SqlTransformerConfig.TRANSFORMER_SQL.key() + ")");
+ throw new HoodieDeltaStreamerTransformPlanException("Missing
configuration : (" + SqlTransformerConfig.TRANSFORMER_SQL.key() + ")");
}
- // tmp table name doesn't like dashes
- String tmpTable =
TMP_TABLE.concat(UUID.randomUUID().toString().replace("-", "_"));
- LOG.info("Registering tmp table : " + tmpTable);
- rowDataset.createOrReplaceTempView(tmpTable);
- String sqlStr = transformerSQL.replaceAll(SRC_PATTERN, tmpTable);
- LOG.debug("SQL Query for transformation : (" + sqlStr + ")");
- Dataset<Row> transformed = sparkSession.sql(sqlStr);
- sparkSession.catalog().dropTempView(tmpTable);
- return transformed;
+ try {
+ // tmp table name doesn't like dashes
+ String tmpTable =
TMP_TABLE.concat(UUID.randomUUID().toString().replace("-", "_"));
+ LOG.info("Registering tmp table : " + tmpTable);
+ rowDataset.createOrReplaceTempView(tmpTable);
+ String sqlStr = transformerSQL.replaceAll(SRC_PATTERN, tmpTable);
+ LOG.debug("SQL Query for transformation : (" + sqlStr + ")");
+ Dataset<Row> transformed = sparkSession.sql(sqlStr);
+ sparkSession.catalog().dropTempView(tmpTable);
+ return transformed;
+ } catch (Exception e) {
+ throw new HoodieDeltaStreamerTransformExecutionException("Failed to
apply sql query based transformer", e);
Review Comment:
Will this catch execution exceptions or just exceptions related to the spark
plan that is generated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
