[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27698: Fix Version/s: (was: 2.5.8) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.7.0 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault updated HBASE-27698: -- Fix Version/s: (was: 2.6.0) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.7.0, 2.5.8 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27698: Fix Version/s: 2.7.0 2.5.8 (was: 2.5.7) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.6.0, 2.7.0, 2.5.8 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27698: Fix Version/s: 2.5.7 (was: 2.5.6) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.6.0, 2.5.7 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27698: Fix Version/s: 2.5.6 (was: 2.5.5) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.6.0, 2.5.6 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Fix Version/s: (was: 2.4.18) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.6.0, 2.5.5 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Fix Version/s: 2.6.0 2.5.5 2.4.18 > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 2.6.0, 2.5.5, 2.4.18 > > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can get meta location from wal directories > with .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String metaReplicaNode : metaReplicaNodes) { > int replicaId = > zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); > RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, > replicaId); > info.append(" ").append(state); > put.setTimestamp(state.getStamp()); > MetaTableAccessor.addRegionInfo(put, state.getRegion()); > if (state.getServerName() != null) { > MetaTableAccessor.addLocation(put, state.getServerName(), > HConstants.NO_SEQNUM, replicaId); > } > > put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) > .setFamily(HConstants.CATALOG_FAMILY) > > .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) > > .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); > } > if (!put.isEmpty()) { > LOG.info(info.toString()); > masterRegion.update(r -> r.put(put)); > } else { > LOG.info("No meta location available on zookeeper, skip migrating..."); > } > } > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Description: In HBase 1.x versions meta server location from zookeeper will be removed when the server stopped. In such cases migrating to 2.5.x branches may not create any meta entries in master data. So in case if we could not find the meta location from zookeeper we can get meta location from wal directories with .meta extension and add to master data. {noformat} private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, KeeperException { // try migrate data from zookeeper try (ResultScanner scanner = masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) { if (scanner.next() != null) { // notice that all replicas for a region are in the same row, so the migration can be // done with in a one row put, which means if we have data in catalog family then we can // make sure that the migration is done. LOG.info("The {} family in master local region already has data in it, skip migrating...", HConstants.CATALOG_FAMILY_STR); return; } } // start migrating byte[] row = CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); Put put = new Put(row); List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); StringBuilder info = new StringBuilder("Migrating meta locations:"); for (String metaReplicaNode : metaReplicaNodes) { int replicaId = zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, replicaId); info.append(" ").append(state); put.setTimestamp(state.getStamp()); MetaTableAccessor.addRegionInfo(put, state.getRegion()); if (state.getServerName() != null) { MetaTableAccessor.addLocation(put, state.getServerName(), HConstants.NO_SEQNUM, replicaId); } put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) .setFamily(HConstants.CATALOG_FAMILY) .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); } if (!put.isEmpty()) { LOG.info(info.toString()); masterRegion.update(r -> r.put(put)); } else { LOG.info("No meta location available on zookeeper, skip migrating..."); } } {noformat} was: In HBase 1.x versions meta server location from zookeeper will be removed when the server stopped. In such cases migrating to 2.5.x branches may not create any meta entries in master data. So in case if we could not find the meta location from zookeeper we can meta location from wal directories with .meta extension and add to master data. {noformat} private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, KeeperException { // try migrate data from zookeeper try (ResultScanner scanner = masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) { if (scanner.next() != null) { // notice that all replicas for a region are in the same row, so the migration can be // done with in a one row put, which means if we have data in catalog family then we can // make sure that the migration is done. LOG.info("The {} family in master local region already has data in it, skip migrating...", HConstants.CATALOG_FAMILY_STR); return; } } // start migrating byte[] row = CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); Put put = new Put(row); List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); StringBuilder info = new StringBuilder("Migrating meta locations:"); for (String metaReplicaNode : metaReplicaNodes) { int replicaId = zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, replicaId); info.append(" ").append(state); put.setTimestamp(state.getStamp()); MetaTableAccessor.addRegionInfo(put, state.getRegion()); if (state.getServerName() != null) { MetaTableAccessor.addLocation(put, state.getServerName(), HConstants.NO_SEQNUM, replicaId); } put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) .setFamily(HConstants.CATALOG_FAMILY) .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); } if (!put.isEmpty()) { LOG.info(info.toString()); masterRegion.update(r -> r.put(put)); } else { LOG.
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Summary: Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase (was: Migrate meta locations from zookeeper may not always possible if we migrate from 1.x HBase) > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can meta location from wal directories with > .meta extension and add to master data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Affects Version/s: 2.5.0 > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can meta location from wal directories with > .meta extension and add to master data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27698) Migrate meta locations from zookeeper to master data may not always possible if we migrate from 1.x HBase
[ https://issues.apache.org/jira/browse/HBASE-27698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-27698: Description: In HBase 1.x versions meta server location from zookeeper will be removed when the server stopped. In such cases migrating to 2.5.x branches may not create any meta entries in master data. So in case if we could not find the meta location from zookeeper we can meta location from wal directories with .meta extension and add to master data. {noformat} private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, KeeperException { // try migrate data from zookeeper try (ResultScanner scanner = masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) { if (scanner.next() != null) { // notice that all replicas for a region are in the same row, so the migration can be // done with in a one row put, which means if we have data in catalog family then we can // make sure that the migration is done. LOG.info("The {} family in master local region already has data in it, skip migrating...", HConstants.CATALOG_FAMILY_STR); return; } } // start migrating byte[] row = CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); Put put = new Put(row); List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); StringBuilder info = new StringBuilder("Migrating meta locations:"); for (String metaReplicaNode : metaReplicaNodes) { int replicaId = zooKeeper.getZNodePaths().getMetaReplicaIdFromZNode(metaReplicaNode); RegionState state = MetaTableLocator.getMetaRegionState(zooKeeper, replicaId); info.append(" ").append(state); put.setTimestamp(state.getStamp()); MetaTableAccessor.addRegionInfo(put, state.getRegion()); if (state.getServerName() != null) { MetaTableAccessor.addLocation(put, state.getServerName(), HConstants.NO_SEQNUM, replicaId); } put.add(CellBuilderFactory.create(CellBuilderType.SHALLOW_COPY).setRow(put.getRow()) .setFamily(HConstants.CATALOG_FAMILY) .setQualifier(RegionStateStore.getStateColumn(replicaId)).setTimestamp(put.getTimestamp()) .setType(Cell.Type.Put).setValue(Bytes.toBytes(state.getState().name())).build()); } if (!put.isEmpty()) { LOG.info(info.toString()); masterRegion.update(r -> r.put(put)); } else { LOG.info("No meta location available on zookeeper, skip migrating..."); } } {noformat} was:In HBase 1.x versions meta server location from zookeeper will be removed when the server stopped. In such cases migrating to 2.5.x branches may not create any meta entries in master data. So in case if we could not find the meta location from zookeeper we can meta location from wal directories with .meta extension and add to master data. > Migrate meta locations from zookeeper to master data may not always possible > if we migrate from 1.x HBase > - > > Key: HBASE-27698 > URL: https://issues.apache.org/jira/browse/HBASE-27698 > Project: HBase > Issue Type: Bug > Components: migration >Affects Versions: 2.5.0 >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > > In HBase 1.x versions meta server location from zookeeper will be removed > when the server stopped. In such cases migrating to 2.5.x branches may not > create any meta entries in master data. So in case if we could not find the > meta location from zookeeper we can meta location from wal directories with > .meta extension and add to master data. > {noformat} > private void tryMigrateMetaLocationsFromZooKeeper() throws IOException, > KeeperException { > // try migrate data from zookeeper > try (ResultScanner scanner = > masterRegion.getScanner(new > Scan().addFamily(HConstants.CATALOG_FAMILY))) { > if (scanner.next() != null) { > // notice that all replicas for a region are in the same row, so the > migration can be > // done with in a one row put, which means if we have data in catalog > family then we can > // make sure that the migration is done. > LOG.info("The {} family in master local region already has data in > it, skip migrating...", > HConstants.CATALOG_FAMILY_STR); > return; > } > } > // start migrating > byte[] row = > CatalogFamilyFormat.getMetaKeyForRegion(RegionInfoBuilder.FIRST_META_REGIONINFO); > Put put = new Put(row); > List metaReplicaNodes = zooKeeper.getMetaReplicaNodes(); > StringBuilder info = new StringBuilder("Migrating meta locations:"); > for (String