Venkatesh created GOBBLIN-1579:
----------------------------------
Summary: Propagate HiveTableLocationNotMatchException to fail the
job execution
Key: GOBBLIN-1579
URL: https://issues.apache.org/jira/browse/GOBBLIN-1579
Project: Apache Gobblin
Issue Type: Bug
Reporter: Venkatesh
When existing target table path is different from desired path for an hive data
movement the below mentioned code in UnpartitionedTableFileSet class gets
executed.
default:
log.error("Source and target table are not compatible. Aborting copy of table
" + this.helper.getTargetTable(),
new
HiveTableLocationNotMatchException(this.helper.getTargetTable().getDataLocation(),
existingTargetTable.get().getDataLocation()));
multiTimer.close();
return Lists.newArrayList();
HiveTableLocationNotMatchException is just logged as an Error and empty list is
returned. Though the workunits computation failed, because of the empty
arraylist, the created workunits is zero and the job status is set as COMMITTED.
Error log:
2021-11-16 11:37:04 PST ERROR [request-allocator-0]
org.apache.gobblin.data.management.copy.hive.UnpartitionedTableFileSet -
Source and target table are not compatible. Aborting copy of table
employeeorg.apache.gobblin.data.management.copy.hive.HiveTableLocationNotMatchException:
Desired target location [hdfs://localhost:8020/tmp/hive/tc_db_1_copy/employee]
and already registered target location
file:/tmp/hive/tc_db_copy/employee/data/employee do not agree. at
org.apache.gobblin.data.management.copy.hive.UnpartitionedTableFileSet.generateCopyEntities(UnpartitionedTableFileSet.java:86)
Solution:
Throw HiveTableLocationNotMatchException so that the exception will be
propogated to the caller and the job run will result in failure.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
