[jira] [Updated] (HBASE-9270) [0.94] FSTableDescriptors caching is racy

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-9270:
--

Status: Open  (was: Patch Available)

Cancelling stale patch. Is this still a problem?

> [0.94] FSTableDescriptors caching is racy
> -
>
> Key: HBASE-9270
> URL: https://issues.apache.org/jira/browse/HBASE-9270
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.11
>Reporter: Andrew Purtell
>Assignee: Rekha Joshi
>Priority: Minor
> Attachments: HBASE-9270.1.patch
>
>
> An occasionally failing test in 0.92 branch that concurrently executes master 
> operations on a single table found this problem in FSTableDescriptors:
> {code}
> diff --git src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 
> src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index e882621..b0042cd 100644
> --- src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++ src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -221,8 +221,15 @@ public class FSTableDescriptors implements 
> TableDescriptors {
>  if 
> (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) {
>throw new NotImplementedException();
>  }
> -if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> -long modtime = getTableInfoModtime(this.fs, this.rootdir, 
> htd.getNameAsString());
> +if (fsreadonly) {
> +  // Cannot cache here.
> +  // We can't know if a modtime from the most recent file found in a
> +  // directory listing at some arbitrary point in time still corresponds
> +  // to the latest, nor that our htd is the latest.
> +  return;
> +}
> +// Cache with the modtime of the descriptor we wrote
> +long modtime = updateHTableDescriptor(this.fs, this.rootdir, 
> htd).getModificationTime();
>  this.cache.put(htd.getNameAsString(), new 
> TableDescriptorModtime(modtime, htd));
>}
> {code}
> After HBASE-7305 master operations are serialized by a write lock on the 
> table.
> However, 0.94 has code with the same issue:
> {code}
>   @Override
>   public void add(HTableDescriptor htd) throws IOException {
> if (Bytes.equals(HConstants.ROOT_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (Bytes.equals(HConstants.META_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) 
> {
>   throw new NotImplementedException();
> }
> if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> String tableName = htd.getNameAsString();
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tableName);
> long dirmodtime = getTableDirModtime(this.fs, this.rootdir, tableName);
> this.cache.put(tableName, new TableDescriptorModtime(modtime, dirmodtime, 
> htd));
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HBASE-9270) [0.94] FSTableDescriptors caching is racy

[Rekha Joshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rekha Joshi updated HBASE-9270:
---

Status: Patch Available  (was: Open)

Patch has only forced return, as 0.94 has refined getTableInfoModtime(), 
getTableDirModtime(), and overloaded getTableInfoPath() methods.

> [0.94] FSTableDescriptors caching is racy
> -
>
> Key: HBASE-9270
> URL: https://issues.apache.org/jira/browse/HBASE-9270
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.11
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-9270.1.patch
>
>
> An occasionally failing test in 0.92 branch that concurrently executes master 
> operations on a single table found this problem in FSTableDescriptors:
> {code}
> diff --git src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 
> src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index e882621..b0042cd 100644
> --- src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++ src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -221,8 +221,15 @@ public class FSTableDescriptors implements 
> TableDescriptors {
>  if 
> (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) {
>throw new NotImplementedException();
>  }
> -if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> -long modtime = getTableInfoModtime(this.fs, this.rootdir, 
> htd.getNameAsString());
> +if (fsreadonly) {
> +  // Cannot cache here.
> +  // We can't know if a modtime from the most recent file found in a
> +  // directory listing at some arbitrary point in time still corresponds
> +  // to the latest, nor that our htd is the latest.
> +  return;
> +}
> +// Cache with the modtime of the descriptor we wrote
> +long modtime = updateHTableDescriptor(this.fs, this.rootdir, 
> htd).getModificationTime();
>  this.cache.put(htd.getNameAsString(), new 
> TableDescriptorModtime(modtime, htd));
>}
> {code}
> After HBASE-7305 master operations are serialized by a write lock on the 
> table.
> However, 0.94 has code with the same issue:
> {code}
>   @Override
>   public void add(HTableDescriptor htd) throws IOException {
> if (Bytes.equals(HConstants.ROOT_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (Bytes.equals(HConstants.META_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) 
> {
>   throw new NotImplementedException();
> }
> if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> String tableName = htd.getNameAsString();
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tableName);
> long dirmodtime = getTableDirModtime(this.fs, this.rootdir, tableName);
> this.cache.put(tableName, new TableDescriptorModtime(modtime, dirmodtime, 
> htd));
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HBASE-9270) [0.94] FSTableDescriptors caching is racy

[Rekha Joshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rekha Joshi updated HBASE-9270:
---

Attachment: HBASE-9270.1.patch

Patch has only forced return, as 0.94 has refined getTableInfoModtime(), 
getTableDirModtime(), and overloaded getTableInfoPath() methods.

> [0.94] FSTableDescriptors caching is racy
> -
>
> Key: HBASE-9270
> URL: https://issues.apache.org/jira/browse/HBASE-9270
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.11
>Reporter: Andrew Purtell
>Priority: Minor
> Attachments: HBASE-9270.1.patch
>
>
> An occasionally failing test in 0.92 branch that concurrently executes master 
> operations on a single table found this problem in FSTableDescriptors:
> {code}
> diff --git src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 
> src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index e882621..b0042cd 100644
> --- src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++ src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -221,8 +221,15 @@ public class FSTableDescriptors implements 
> TableDescriptors {
>  if 
> (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) {
>throw new NotImplementedException();
>  }
> -if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> -long modtime = getTableInfoModtime(this.fs, this.rootdir, 
> htd.getNameAsString());
> +if (fsreadonly) {
> +  // Cannot cache here.
> +  // We can't know if a modtime from the most recent file found in a
> +  // directory listing at some arbitrary point in time still corresponds
> +  // to the latest, nor that our htd is the latest.
> +  return;
> +}
> +// Cache with the modtime of the descriptor we wrote
> +long modtime = updateHTableDescriptor(this.fs, this.rootdir, 
> htd).getModificationTime();
>  this.cache.put(htd.getNameAsString(), new 
> TableDescriptorModtime(modtime, htd));
>}
> {code}
> After HBASE-7305 master operations are serialized by a write lock on the 
> table.
> However, 0.94 has code with the same issue:
> {code}
>   @Override
>   public void add(HTableDescriptor htd) throws IOException {
> if (Bytes.equals(HConstants.ROOT_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (Bytes.equals(HConstants.META_TABLE_NAME, htd.getName())) {
>   throw new NotImplementedException();
> }
> if (HConstants.HBASE_NON_USER_TABLE_DIRS.contains(htd.getNameAsString())) 
> {
>   throw new NotImplementedException();
> }
> if (!this.fsreadonly) updateHTableDescriptor(this.fs, this.rootdir, htd);
> String tableName = htd.getNameAsString();
> long modtime = getTableInfoModtime(this.fs, this.rootdir, tableName);
> long dirmodtime = getTableDirModtime(this.fs, this.rootdir, tableName);
> this.cache.put(tableName, new TableDescriptorModtime(modtime, dirmodtime, 
> htd));
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)