[
https://issues.apache.org/jira/browse/HIVE-26960?focusedWorklogId=842731&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-842731
]
ASF GitHub Bot logged work on HIVE-26960:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Feb/23 01:19
Start Date: 01/Feb/23 01:19
Worklog Time Spent: 10m
Work Description: pudidic commented on code in PR #3995:
URL: https://github.com/apache/hive/pull/3995#discussion_r1092651154
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadWork.java:
##########
@@ -159,20 +159,25 @@ public ReplLoadWork(HiveConf hiveConf, String
dumpDirectory,
* for the same.
*/
Path incBootstrapDir = new Path(dumpDirectory,
ReplUtils.INC_BOOTSTRAP_ROOT_DIR_NAME);
- if (fs.exists(incBootstrapDir)) {
- if (isSecondFailover) {
- String[] bootstrappedTables = getBootstrapTableList(new
Path(dumpDirectory).getParent(), hiveConf);
- LOG.info("Optimised bootstrap load for database {} with initial
bootstrapped table list as {}",
- dbNameToLoadIn, tablesToBootstrap);
- // Get list of tables bootstrapped.
+ if (isSecondFailover) {
+ String[] bootstrappedTables = getBootstrapTableList(new
Path(dumpDirectory).getParent(), hiveConf);
+ LOG.info("Optimised bootstrap load for database {} with initial
bootstrapped table list as {}",
+ dbNameToLoadIn, tablesToBootstrap);
+ // Get list of tables bootstrapped.
+ if (fs.exists(incBootstrapDir)) {
Path tableMetaPath = new Path(incBootstrapDir,
EximUtil.METADATA_PATH_NAME + "/" + sourceDbName);
tablesToBootstrap =
- Stream.of(fs.listStatus(tableMetaPath)).map(st ->
st.getPath().getName()).collect(Collectors.toList());
- List<String> tableList = Arrays.asList(bootstrappedTables);
- tablesToDrop = ListUtils.subtract(tableList, tablesToBootstrap);
- LOG.info("Optimised bootstrap for database {} with drop table list
as {} and bootstrap table list as {}",
- dbNameToLoadIn, tablesToDrop, tablesToBootstrap);
+ Stream.of(fs.listStatus(tableMetaPath)).map(st ->
st.getPath().getName()).collect(Collectors.toList());
+ }
+ else {
+ tablesToBootstrap = Collections.emptyList();
Review Comment:
Elegant code without for loop, nulls. :)
Issue Time Tracking
-------------------
Worklog Id: (was: 842731)
Time Spent: 1h 10m (was: 1h)
> Optimized bootstrap does not drop newly added tables at source.
> ---------------------------------------------------------------
>
> Key: HIVE-26960
> URL: https://issues.apache.org/jira/browse/HIVE-26960
> Project: Hive
> Issue Type: Bug
> Reporter: Rakshith C
> Assignee: Rakshith C
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Scenario:
> Replication is setup from DR to PROD after failover from PROD to DR and no
> existing tables are modified at PROD but a new table is added at PROD.
> Observations:
> * _bootstrap directory won't be created during second cycle of optimized
> bootstrap because existing tables were not modified.
> * Based on this, it will not initialize list of tables to drop at PROD.
> * This leads to the new table created at PROD not being dropped.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
