[jira] [Commented] (PHOENIX-4010) Hash Join cache may not be send to all regionservers when we have stale HBase meta cache

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/PHOENIX-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099302#comment-16099302
 ] 

ASF GitHub Bot commented on PHOENIX-4010:
-----------------------------------------

Github user JamesRTaylor commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/268#discussion_r129185736
  
    --- Diff: 
phoenix-core/src/main/java/org/apache/phoenix/iterate/TableResultIterator.java 
---
    @@ -163,8 +185,38 @@ public Tuple next() throws SQLException {
                                 }
                             }
                             
plan.getContext().getConnection().getQueryServices().clearTableRegionCache(htable.getTableName());
    -                        this.scanIterator =
    -                                plan.iterator(scanGrouper, newScan);
    +                        if (e1 instanceof HashJoinCacheNotFoundException) {
    +                            logger.debug(
    +                                    "Retrying when Hash Join cache is not 
found on the server ,by sending the cache again");
    +                            if (retry <= 0) { throw e1; }
    +                            retry--;
    +                            try {
    +                                Long cacheId = 
((HashJoinCacheNotFoundException)e1).getCacheId();
    +                                if 
(!hashCacheClient.addHashCacheToServer(newScan.getStartRow(),
    +                                        
ServerCacheClient.getCacheForId(caches, cacheId), 
plan.getTableRef().getTable())) { throw e1; }
    +                                
    +                                try {
    +                                    
if(ScanUtil.isClientSideUpsertSelect(newScan)) {
    --- End diff --
    
    Why is this necessary? Can we generalize this is a better way?


> Hash Join cache may not be send to all regionservers when we have stale HBase 
> meta cache
> ----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4010
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4010
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Ankit Singhal
>            Assignee: Ankit Singhal
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-4010.patch, PHOENIX-4010_v1.patch, 
> PHOENIX-4010_v2.patch, PHOENIX-4010_v2_rebased_1.patch, 
> PHOENIX-4010_v2_rebased.patch
>
>
>  If the region locations changed and our HBase meta cache is not updated then 
> we might not be sending hash join cache to all region servers hosting the 
> regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =false;
>         while (true) {
>             try {
>                 // We could surface the package projected 
> HConnectionImplementation.getNumberOfCachedRegionLocations
>                 // to get the sizing info we need, but this would require a 
> new class in the same package and a cast
>                 // to this implementation class, so it's probably not worth 
> it.
>                 List<HRegionLocation> locations = Lists.newArrayList();
>                 byte[] currentKey = HConstants.EMPTY_START_ROW;
>                 do {
>                     HRegionLocation regionLocation = 
> connection.getRegionLocation(
>                             TableName.valueOf(tableName), currentKey, reload);
>                     locations.add(regionLocation);
>                     currentKey = regionLocation.getRegionInfo().getEndKey();
>                 } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_ROW));
>                 return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List<HRegionLocation> locations = 
> services.getAllTableRegions(cacheUsingTable.getPhysicalName().getBytes());
>             int nRegions = locations.size();
>             
> .....
>  if ( ! servers.contains(entry) && 
>                         keyRanges.intersectRegion(regionStartKey, 
> regionEndKey,
>                                 cacheUsingTable.getIndexType() == 
> IndexType.LOCAL)) {  
>                     // Call RPC once per server
>                     servers.add(entry);
> {code}
> For eg:- Table ’T’ has two regions R1 and R2 originally hosted on 
> regionserver RS1. 
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2 ,  
> but stale meta cache will still give old region locations i.e R1 and R2 on 
> RS1 and when we start copying hash table, we copy for R1 and skip R2 as they 
> are hosted on same regionserver. so, the query on a table will fail as it 
> will unable to find hash table cache on RS2 for processing regions R2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)