SourabhBadhya commented on code in PR #4549:
URL: https://github.com/apache/hive/pull/4549#discussion_r1284238847
##########
ql/src/java/org/apache/hadoop/hive/ql/HiveQueryLifeTimeHook.java:
##########
@@ -71,20 +71,23 @@ private void checkAndRollbackCTAS(QueryLifeTimeHookContext
ctx) {
QueryPlan queryPlan = ctx.getHookContext().getQueryPlan();
boolean isCTAS = Optional.ofNullable(queryPlan.getQueryProperties())
.map(queryProps -> queryProps.isCTAS()).orElse(false);
+ // return early if the query is not CATS type.
+ if (!isCTAS)
+ return;
PrivateHookContext pCtx = (PrivateHookContext) ctx.getHookContext();
Path tblPath = pCtx.getContext().getLocation();
- try {
- FileSystem fs = tblPath.getFileSystem(conf);
- if (!fs.exists(tblPath)) {
- return;
+ if (tblPath != null) {
+ try {
+ FileSystem fs = tblPath.getFileSystem(conf);
+ if (!fs.exists(tblPath)) {
+ return;
+ }
+ } catch (Exception e) {
+ throw new RuntimeException("Not able to check whether the CTAS table
directory exists due to: ", e);
}
- } catch (Exception e) {
- throw new RuntimeException("Not able to check whether the CTAS table
directory exists due to: ", e);
- }
- if (isCTAS && tblPath != null) {
Review Comment:
Please include this. The idea is to clean up any directories which are
created by CTAS but are not committed. Hence cleanup must be scheduled only
when there is a directory and the query is a CTAS.
##########
ql/src/java/org/apache/hadoop/hive/ql/HiveQueryLifeTimeHook.java:
##########
@@ -71,20 +71,23 @@ private void checkAndRollbackCTAS(QueryLifeTimeHookContext
ctx) {
QueryPlan queryPlan = ctx.getHookContext().getQueryPlan();
boolean isCTAS = Optional.ofNullable(queryPlan.getQueryProperties())
.map(queryProps -> queryProps.isCTAS()).orElse(false);
+ // return early if the query is not CATS type.
+ if (!isCTAS)
Review Comment:
I dont think we must add this since the check is already done below.
##########
ql/src/java/org/apache/hadoop/hive/ql/HiveQueryLifeTimeHook.java:
##########
@@ -71,20 +71,23 @@ private void checkAndRollbackCTAS(QueryLifeTimeHookContext
ctx) {
QueryPlan queryPlan = ctx.getHookContext().getQueryPlan();
boolean isCTAS = Optional.ofNullable(queryPlan.getQueryProperties())
.map(queryProps -> queryProps.isCTAS()).orElse(false);
+ // return early if the query is not CATS type.
+ if (!isCTAS)
+ return;
PrivateHookContext pCtx = (PrivateHookContext) ctx.getHookContext();
Path tblPath = pCtx.getContext().getLocation();
- try {
- FileSystem fs = tblPath.getFileSystem(conf);
- if (!fs.exists(tblPath)) {
- return;
+ if (tblPath != null) {
Review Comment:
I feel its better to include entire logic below -
```
if (isCTAS && tblPath != null) {
try {
FileSystem fs = tblPath.getFileSystem(conf);
if (!fs.exists(tblPath)) {
return;
}
} catch (Exception e) {
throw new RuntimeException("Not able to check whether the CTAS table
directory exists due to: ", e);
}
// Remaining logic
....
}
```
The CTAS check and the non-nullability of the path will be checked then in a
single if clause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
