[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Skrabak updated PHOENIX-4010:
---
Attachment: PHOENIX-4010.addendum.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Csaba Skrabak
>Priority: Major
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.addendum.patch, PHOENIX-4010.patch,
> PHOENIX-4010_v1.patch, PHOENIX-4010_v2.patch, PHOENIX-4010_v2_rebased.patch,
> PHOENIX-4010_v2_rebased_1.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010_v2_rebased_1.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch,
> PHOENIX-4010_v2.patch, PHOENIX-4010_v2_rebased_1.patch,
> PHOENIX-4010_v2_rebased.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010_v2_rebased.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch,
> PHOENIX-4010_v2.patch, PHOENIX-4010_v2_rebased.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: (was: PHOENIX-4010_v2_rebased.patch)
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch,
> PHOENIX-4010_v2.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010_v2_rebased.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch,
> PHOENIX-4010_v2.patch, PHOENIX-4010_v2_rebased.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010_v2.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch,
> PHOENIX-4010_v2.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010_v1.patch
Extended last patch with :-
* Limit memory for hash join cache on the client(40% default of client memory
though configurable)
* HashJoinIT is modified to have all test pass when hash join cache is not
present on server
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: (was: PHOENIX-4010.patch)
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: (was: PHOENIX-4010.patch)
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: (was: PHOENIX-4010.patch)
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache
[
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal updated PHOENIX-4010:
---
Attachment: PHOENIX-4010.patch
> Hash Join cache may not be send to all regionservers when we have stale HBase
> meta cache
>
>
> Key: PHOENIX-4010
> URL: https://issues.apache.org/jira/browse/PHOENIX-4010
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4010.patch
>
>
> If the region locations changed and our HBase meta cache is not updated then
> we might not be sending hash join cache to all region servers hosting the
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
> while (true) {
> try {
> // We could surface the package projected
> HConnectionImplementation.getNumberOfCachedRegionLocations
> // to get the sizing info we need, but this would require a
> new class in the same package and a cast
> // to this implementation class, so it's probably not worth
> it.
> List locations = Lists.newArrayList();
> byte[] currentKey = HConstants.EMPTY_START_ROW;
> do {
> HRegionLocation regionLocation =
> connection.getRegionLocation(
> TableName.valueOf(tableName), currentKey, reload);
> locations.add(regionLocation);
> currentKey = regionLocation.getRegionInfo().getEndKey();
> } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
> return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List locations =
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
> int nRegions = locations.size();
>
> .
> if ( ! servers.contains(entry) &&
> keyRanges.intersectRegion(regionStartKey,
> regionEndKey,
> cacheUsingTable.getIndexType() ==
> IndexType.LOCAL)) {
> // Call RPC once per server
> servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on
> regionserver RS1.
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,
> but stale meta cache will still give old region locations i.e R1 and R2 on
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they
> are hosted on same regionserver. so, the query on a table will fail as it
> will unable to find hash table cache on RS2 for processing regions R2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
