miklosgergely commented on a change in pull request #969: HIVE-23123 Disable
export/import of views and materialized views
URL: https://github.com/apache/hive/pull/969#discussion_r408995668
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##########
@@ -163,10 +170,34 @@ a database ( directory )
the dbTracker / tableTracker are setup correctly always.
*/
TableContext tableContext = new TableContext(dbTracker,
work.dbNameToLoadIn);
- TableEvent tableEvent = (TableEvent) next;
- LoadTable loadTable = new LoadTable(tableEvent, loadContext,
iterator.replLogger(),
- tableContext, loadTaskTracker);
- tableTracker = loadTable.tasks(work.isIncrementalLoad());
+ FSTableEvent tableEvent = (FSTableEvent) next;
+ if
(TableType.VIRTUAL_VIEW.name().equals(tableEvent.getMetaData().getTable().getTableType()))
{
+ tableTracker = new TaskTracker(1);
+ tableTracker.addTask(createViewTask(tableEvent.getMetaData(),
work.dbNameToLoadIn, conf));
+ } else {
+ LoadTable loadTable = new LoadTable(tableEvent, loadContext,
iterator.replLogger(), tableContext,
+ loadTaskTracker);
+ tableTracker = loadTable.tasks(work.isIncrementalLoad());
+
+ /*
+ for table replication if we reach the max number of tasks then
for the next run we will
+ try to reload the same table again, this is mainly for ease of
understanding the code
+ as then we can avoid handling == > loading partitions for the
table given that
+ the creation of table lead to reaching max tasks vs, loading
next table since current
+ one does not have partitions.
+ */
+
+ // for a table we explicitly try to load partitions as there is no
separate partitions events.
+ LoadPartitions loadPartitions =
+ new LoadPartitions(loadContext, iterator.replLogger(),
loadTaskTracker, tableEvent,
+ work.dbNameToLoadIn, tableContext);
+ TaskTracker partitionsTracker = loadPartitions.tasks();
+ partitionsPostProcessing(iterator, scope, loadTaskTracker,
tableTracker,
Review comment:
Modified it, please confirm that like this it will be ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
