[jira] [Work started] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15281 started by Rajesh Nishtala. --- > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Status: In Progress (was: Patch Available) > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Status: Patch Available (was: In Progress) > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work stopped] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15281 stopped by Rajesh Nishtala. --- > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Attachment: HBASE-15281-v1.patch > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Status: Patch Available (was: Open) > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > Attachments: HBASE-15281-v1.patch > > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Status: Open (was: Patch Available) > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
[ https://issues.apache.org/jira/browse/HBASE-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-15281: Status: Patch Available (was: Open) > Allow the FileSystem inside HFileSystem to be wrapped > - > > Key: HBASE-15281 > URL: https://issues.apache.org/jira/browse/HBASE-15281 > Project: HBase > Issue Type: New Feature > Components: Filesystem Integration, hbase >Reporter: Rajesh Nishtala >Assignee: Rajesh Nishtala >Priority: Minor > > It would be very useful for us to be able to wrap the filesystems > encapsulated by HFileSystem with other FilterFileSystems. This allows for > more detailed logging of the operations to the DFS. Internally, the data > logged from this method has allowed us to show application engineers where > there schemas are inefficient and inducing too much IO. This patch will just > allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15281) Allow the FileSystem inside HFileSystem to be wrapped
Rajesh Nishtala created HBASE-15281: --- Summary: Allow the FileSystem inside HFileSystem to be wrapped Key: HBASE-15281 URL: https://issues.apache.org/jira/browse/HBASE-15281 Project: HBase Issue Type: New Feature Components: Filesystem Integration, hbase Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor It would be very useful for us to be able to wrap the filesystems encapsulated by HFileSystem with other FilterFileSystems. This allows for more detailed logging of the operations to the DFS. Internally, the data logged from this method has allowed us to show application engineers where there schemas are inefficient and inducing too much IO. This patch will just allow the filesystem to be pluggable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13471) Deadlock closing a region
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505541#comment-14505541
]
Rajesh Nishtala commented on HBASE-13471:
-
I just updated the patch based on [~eclark]'s comments in the diff
Deadlock closing a region
-
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 0.98.13
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13471) Deadlock closing a region
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13471:
Attachment: HBASE-13471-v1.patch
Deadlock closing a region
-
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 0.98.13
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13471:
Summary: Fix a possible infinite loop in doMiniBatchMutation (was:
Deadlock closing a region)
Fix a possible infinite loop in doMiniBatchMutation
---
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 0.98.13
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506406#comment-14506406
]
Rajesh Nishtala commented on HBASE-13471:
-
[~lhofhansl] all the callers were already doing this check before making the
call which made the call inside getRowLockInterbal redundant.
Fix a possible infinite loop in doMiniBatchMutation
---
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13471) Deadlock closing a region
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13471:
Attachment: HBASE-13471.patch
Deadlock closing a region
-
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13471) Deadlock closing a region
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504047#comment-14504047
]
Rajesh Nishtala commented on HBASE-13471:
-
In fairness I think there are two bugs here. (1) the client has a row / region
mismatch under some circumstances that are yet TBD and (2) when that occurs
there's a possible infinite loop. This addresses the later by propagating up
the wrong region information to the client. With this fix in we can hopefully
find the cause of (1) with the extra debugging information that results from
the fix for (2).
Deadlock closing a region
-
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13471) Deadlock closing a region
[
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504036#comment-14504036
]
Rajesh Nishtala commented on HBASE-13471:
-
The fix is up at https://reviews.facebook.net/D37437. Looks like there's a
possible infinite loop that can occur in doMiniBatchMutation with the readLock
held causing the doClose() to never be able to grab its lock.
Deadlock closing a region
-
Key: HBASE-13471
URL: https://issues.apache.org/jira/browse/HBASE-13471
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Attachments: HBASE-13471.patch
{code}
Thread 4139
(regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
State: WAITING
Blocked count: 131
Waited count: 228
Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13416) Recreating a deleted table causes replication of old WALS
[ https://issues.apache.org/jira/browse/HBASE-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483905#comment-14483905 ] Rajesh Nishtala commented on HBASE-13416: - [~esteban] after disabling and deleting the table I also remove any peers and readd the peers. The unit test that can replicate this behavior just landed. The commit id is d5e81523f719d54e44d24cf6ef8112355aa6281a. In the setupTablesAndReplication there's a section that rolls wal edits before we delete the old tables. If this is commented out, and the test is run twice, the second time we can reliably repro the bug. Recreating a deleted table causes replication of old WALS - Key: HBASE-13416 URL: https://issues.apache.org/jira/browse/HBASE-13416 Project: HBase Issue Type: Bug Components: Replication Reporter: Rajesh Nishtala Priority: Critical 1) Create a table and setup replication to another cluster 2) Write some data into the source table 3) Disable and delete the table from the source cluster and the sink cluster 4) Recreate the table with the same schema in the source and sink clusters 5) The source cluster is empty but the sink cluster has a copy of the old data that is not in the source cluster. To work around: 1) disable the table in the source cluster 2) Roll the WALs across all region servers 3) Delete the table in the source cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13413) Create an integration test for Replication
Rajesh Nishtala created HBASE-13413: --- Summary: Create an integration test for Replication Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481529#comment-14481529 ] Rajesh Nishtala commented on HBASE-13413: - I've put up an initial version of the test here: https://reviews.facebook.net/D36423 Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-13413: Attachment: HBASE-13413.patch Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor Attachments: HBASE-13413.patch We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481531#comment-14481531 ] Rajesh Nishtala commented on HBASE-13413: - Hi Dima, Thanks for the support! I'd love to get your comments on the diff thats up. Right now its a simple extension of the IntegrationTestBigLinkedList. Thanks! Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13416) Deleted/Recreated tables causes replication of old WALS to get replicated incorrectly
Rajesh Nishtala created HBASE-13416: --- Summary: Deleted/Recreated tables causes replication of old WALS to get replicated incorrectly Key: HBASE-13416 URL: https://issues.apache.org/jira/browse/HBASE-13416 Project: HBase Issue Type: Bug Components: Replication Reporter: Rajesh Nishtala 1) Create a table and setup replication to another cluster 2) Write some data into the source table 3) Disable and delete the table from the source cluster and the sink cluster 4) Recreate the table with the same schema in the source and sink clusters 5) The source cluster is empty but the sink cluster has a copy of the old data that is not in the source cluster. To work around: 1) disable the table in the source cluster 2) Roll the WALs across all region servers 3) Delete the table in the source cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-13413: Attachment: HBASE-13413-v1.patch next rev Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Affects Versions: 1.0.0, 2.0.0 Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor Attachments: HBASE-13413-v1.patch, HBASE-13413.patch We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13416) Recreating a deleted table causes replication of old WALS
[ https://issues.apache.org/jira/browse/HBASE-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-13416: Summary: Recreating a deleted table causes replication of old WALS (was: Recreating a deleted table causes replication of old WALS to get replicated incorrectly) Recreating a deleted table causes replication of old WALS - Key: HBASE-13416 URL: https://issues.apache.org/jira/browse/HBASE-13416 Project: HBase Issue Type: Bug Components: Replication Reporter: Rajesh Nishtala 1) Create a table and setup replication to another cluster 2) Write some data into the source table 3) Disable and delete the table from the source cluster and the sink cluster 4) Recreate the table with the same schema in the source and sink clusters 5) The source cluster is empty but the sink cluster has a copy of the old data that is not in the source cluster. To work around: 1) disable the table in the source cluster 2) Roll the WALs across all region servers 3) Delete the table in the source cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13416) Recreating a deleted table causes replication of old WALS to get replicated incorrectly
[ https://issues.apache.org/jira/browse/HBASE-13416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-13416: Summary: Recreating a deleted table causes replication of old WALS to get replicated incorrectly (was: Deleted/Recreated tables causes replication of old WALS to get replicated incorrectly) Recreating a deleted table causes replication of old WALS to get replicated incorrectly --- Key: HBASE-13416 URL: https://issues.apache.org/jira/browse/HBASE-13416 Project: HBase Issue Type: Bug Components: Replication Reporter: Rajesh Nishtala 1) Create a table and setup replication to another cluster 2) Write some data into the source table 3) Disable and delete the table from the source cluster and the sink cluster 4) Recreate the table with the same schema in the source and sink clusters 5) The source cluster is empty but the sink cluster has a copy of the old data that is not in the source cluster. To work around: 1) disable the table in the source cluster 2) Roll the WALs across all region servers 3) Delete the table in the source cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Nishtala updated HBASE-13413: Attachment: HBASE-13413-v2.patch Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Affects Versions: 1.0.0, 2.0.0 Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor Attachments: HBASE-13413-v1.patch, HBASE-13413-v2.patch, HBASE-13413.patch We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344214#comment-14344214
]
Rajesh Nishtala commented on HBASE-13087:
-
I just uploaded a git revert of HBASE-13017 on branch-1. I had to make a few
minor changes primarily with imports to get it to work. It passed all the unit
tests and I verified that the steps above work and the bug is not reproduced.
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Fix For: 1.1.0
Attachments: HBASE-13087-v1.txt
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344383#comment-14344383
]
Rajesh Nishtala commented on HBASE-13087:
-
BTW this patch is not intended to be applied against master so that Hadoop QA
message is probably expected.
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Fix For: 1.1.0
Attachments: HBASE-13087-v1.txt
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13087:
Status: Open (was: Patch Available)
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Labels: newbie, patch
Fix For: 1.1.0
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13087:
Labels: (was: newbie patch)
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Fix For: 1.1.0
Attachments: HBASE-13087-v1.txt
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13087:
Attachment: HBASE-13087-v1.txt
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Fix For: 1.1.0
Attachments: HBASE-13087-v1.txt
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Nishtala updated HBASE-13087:
Fix Version/s: (was: 2.0.0)
Labels: newbie patch (was: )
Affects Version/s: 1.1.0
Status: Patch Available (was: Open)
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Labels: patch, newbie
Fix For: 1.1.0
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98
[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341131#comment-14341131
]
Rajesh Nishtala commented on HBASE-13087:
-
So Elliot and I spent some time digging into this over the past few days and we
found a couple of interesting things. The META table descriptions are correct
in Master and in HDFS. So we thought forcing the RegionServer to reload META
would fix it. However, when we tried this we found that it was still reading
the old META table descriptor. After diving deeper into the code Elliot found
that FSTableDescriptors::get(final TableName tablename) used a hard coded
value for the HTableDescriptor of META. As a result no amount of reloading and
forced reloading would fix this.
So there are two not so great paths forward:
(1) Patch 0.98 so that 0.98 can be hot upgradable to branch-1
(2) Change the upgrade process to tell users to upgrade the region servers
first and then master.
Neither of these options are particularly pleasing but I wanted to open this up
to the broader group to get people's thoughts.
branch-1 isn't rolling upgradable from 0.98
---
Key: HBASE-13087
URL: https://issues.apache.org/jira/browse/HBASE-13087
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
Fix For: 2.0.0, 1.1.0
{code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family table does not exist in region hbase:meta,,1.1588230740 in table
'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 =
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
{NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY =
'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL =
'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true',
BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
: 1 time,
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
at
org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
at
org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
at
org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
at
org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
at
org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
at
org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
at
org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
at
org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
