[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291031#comment-17291031 ] Josh Elser commented on HBASE-12751: Updated the release notes of this change as a result of HBASE-25601 > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 1.3.0, 2.0.0 >Reporter: Elliott Neil Clark >Assignee: Michael Stack >Priority: Major > Fix For: 2.0.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138355#comment-15138355 ] stack commented on HBASE-12751: --- [~eclark] Nice side effect Regards HBASE-15047, as said before... radical. HBASE-15082 is in now (smile). > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: stack > Fix For: 2.0.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137443#comment-15137443 ] stack commented on HBASE-12751: --- [~eclark] Who did the mvcc fixup that was in original patch. You see how HBASE-15213 fingers it as fixing increment performance regression? > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: stack > Fix For: 2.0.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137925#comment-15137925 ] Elliott Clark commented on HBASE-12751: --- Yeah that was something I did to try and really only have one sequence id. HBASE-15047 is my thought on how to take it one step farther. No context switches for anyone. > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: stack > Fix For: 2.0.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011962#comment-15011962 ] Enis Soztutar commented on HBASE-12751: --- Resolve this issue? > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: stack > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903923#comment-14903923 ] Hudson commented on HBASE-12751: FAILURE: Integrated in HBase-TRUNK #6831 (See [https://builds.apache.org/job/HBase-TRUNK/6831/]) HBASE-12751 Allow RowLock to be reader writer (stack: rev 2c83d8a263ee7c6751448bcc72868da86757f7f2) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiVersionConcurrencyControlBasic.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALReaderOnSecureWAL.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/WALPerformanceEvaluation.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkLoad.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicaFailover.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestDefaultWALProviderWithHLogKey.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/FaultyFSLog.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSecureWAL.java * hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesModel.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileRefresherChore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDistributedLogReplay.java * hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestDefaultWALProvider.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestServerCrashProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/HashedBytes.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ReplayHLogKey.java * hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollingNoCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestIOFencing.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiVersionConcurrencyControl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java * hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java * hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java *
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903667#comment-14903667
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761716/12751.v41.txt
against master branch at commit f697d2d312bc04072ccd64c231319e7e571d1aff.
ATTACHMENT ID: 12761716
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 110
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning(TestHRegion.java:3756)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15678//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15678//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15678//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15678//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903708#comment-14903708
]
stack commented on HBASE-12751:
---
Says we are hung up here:
{code}
"main" prio=10 tid=0x7f6f94009000 nid=0x1669 in Object.wait()
[0x7f6f9d7d3000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007c7452b08> (a java.lang.Object)
at
org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.waitForRead(MultiVersionConcurrencyControl.java:207)
- locked <0x0007c7452b08> (a java.lang.Object)
at
org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.completeAndWait(MultiVersionConcurrencyControl.java:143)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2257)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2061)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2026)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2016)
at
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1423)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1344)
- locked <0x0007c7452b38> (a java.lang.Object)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1295)
at
org.apache.hadoop.hbase.HBaseTestingUtility.closeRegionAndWAL(HBaseTestingUtility.java:352)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning(TestHRegion.java:3756)
{code}
All other threads are hanging out doing nothing. There may be something to this
but it hasn't shown up during runs of this patch up to this.
I ran this patch on rig. 5 out of 6 runs completed. One failed because of
aTestHCM issue I've been watching.
Going to commit.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751.v39.txt, 12751.v40.txt, 12751.v41.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902858#comment-14902858
]
stack commented on HBASE-12751:
---
Has these as hanging:
kalashnikov:hbase.git stack$ python ./dev-support/findHangingTests.py
https://builds.apache.org/job/PreCommit-HBASE-Build/15664//consoleText
Fetching the console output from the URL
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
Hanging test : org.apache.hadoop.hbase.client.TestReplicasClient
Hanging test : org.apache.hadoop.hbase.regionserver.TestHRegion
Hanging test : org.apache.hadoop.hbase.namespace.TestNamespaceAuditor
Hanging test : org.apache.hadoop.hbase.TestAcidGuarantees
Hanging test : org.apache.hadoop.hbase.TestZooKeeper
Printing Failing tests
Trying all tests locally and they pass. Looking at failures up in jenkins.
For TestZooKeeper... it is an interesting one... seems unrelated though:
{code}
2015-09-22 08:54:42,962 INFO [RS:0;asf906:46174]
regionserver.HRegionServer(1099): stopping server
asf906.gq1.ygridcore.net,46174,1442912078095; zookeeper connection closed.
2015-09-22 08:54:42,962 INFO [RS:0;asf906:46174]
regionserver.HRegionServer(1102): RS:0;asf906:46174 exiting
2015-09-22 08:54:42,962 ERROR [SyncThread:0] server.NIOServerCnxn(178):
Unexpected Exception:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
at
org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
at
org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
at
org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
at
org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
at org.apache.zookeeper.server.DataTree.deleteNode(DataTree.java:591)
at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:802)
at
org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
at
org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
at
org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
at
org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
at
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
{code}
I went through all the others and nothing untoward. Let me retry.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751.v39.txt, 12751.v40.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902291#comment-14902291
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761569/12751.v40.txt
against master branch at commit 45d67435adc9195325bfccc78b7e1a0202a446e5.
ATTACHMENT ID: 12761569
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 110
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15664//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15664//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15664//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15664//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751.v39.txt, 12751.v40.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901978#comment-14901978 ] Elliott Clark commented on HBASE-12751: --- +1 commit it. > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751.v40.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, > HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, > HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, > HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, > HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, > HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, > HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, > HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, > HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901956#comment-14901956
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12761563/12751.v39.txt
against master branch at commit 45d67435adc9195325bfccc78b7e1a0202a446e5.
ATTACHMENT ID: 12761563
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 110
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15660//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15660//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15660//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15660//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15660//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751.v39.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901693#comment-14901693
]
stack commented on HBASE-12751:
---
The TestIOFencing failure is because test is parameterized and the DLR run was
hanging. Fixed.
I removed TestVisibilityLabelsWithDistributedLogReplay because this patch
breaks DLR and DLR is deprecated as-is.
The TestDistributedLogSplitting and TestServerCrashProcedure are both hung up
in MVCC waiting on write number to progress. The accounting must be wrong. Let
me see If I do the below hack, tests pass...
{code}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
index a8ffa8d..e70e7bd 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
@@ -2099,7 +2099,7 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
writeFlushRequestMarkerToWAL(wal, writeFlushWalMarker));
// TODO: Lets see if we hang here, if there is a scenario where an
outstanding reader
// with a read point is in advance of this write point.
-mvcc.completeAndWait(writeEntry);
+mvcc.complete(writeEntry);
writeEntry = null;
return new PrepareFlushResult(flushResult, myseqid);
} else {
@@ -2254,7 +2254,7 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
// uncommitted transactions from being written into HFiles.
// We have to block before we start the flush, otherwise keys that
// were removed via a rollbackMemstore could be written to Hfiles.
- mvcc.completeAndWait(writeEntry);
+ mvcc.complete(writeEntry);
// set writeEntry to null to prevent mvcc.complete from being called
again inside finally
// block
writeEntry = null;
@@ -3158,7 +3158,7 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
// STEP 8. Advance mvcc. This will make this put visible to scanners and
getters.
// --
if (writeEntry != null) {
-mvcc.completeAndWait(writeEntry);
+mvcc.complete(writeEntry);
writeEntry = null;
} else if (isInReplay) {
// ensure that the sequence id of the region is at least as big as
orig log seq id
@@ -3203,7 +3203,7 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
}
if (writeEntry != null) mvcc.complete(writeEntry);
} else if (writeEntry != null) {
-mvcc.completeAndWait(writeEntry);
+mvcc.complete(writeEntry);
}
if (locked) {
@@ -6856,7 +6856,7 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
}
// 13. Roll mvcc forward
if (writeEntry != null) {
- mvcc.completeAndWait(writeEntry);
+ mvcc.complete(writeEntry);
}
{code}
TestReplicasClient seems broke in a different way altogether. Digging.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901960#comment-14901960 ] stack commented on HBASE-12751: --- No hanging tests. All tests passed: [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 36 licence. [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache HBase .. SUCCESS [2.014s] [INFO] Apache HBase - Checkstyle . SUCCESS [0.050s] [INFO] Apache HBase - Resource Bundle SUCCESS [0.052s] [INFO] Apache HBase - Annotations SUCCESS [0.091s] [INFO] Apache HBase - Protocol ... SUCCESS [0.102s] [INFO] Apache HBase - Common . SUCCESS [0.228s] [INFO] Apache HBase - Procedure .. SUCCESS [0.064s] [INFO] Apache HBase - Client . SUCCESS [0.285s] [INFO] Apache HBase - Hadoop Compatibility ... SUCCESS [0.108s] [INFO] Apache HBase - Hadoop Two Compatibility ... SUCCESS [0.098s] [INFO] Apache HBase - Prefix Tree SUCCESS [0.099s] [INFO] Apache HBase - Server . SUCCESS [1.057s] [INFO] Apache HBase - Testing Util ... SUCCESS [0.036s] [INFO] Apache HBase - Thrift . SUCCESS [0.096s] [INFO] Apache HBase - Shell .. SUCCESS [0.193s] [INFO] Apache HBase - Integration Tests .. SUCCESS [0.102s] [INFO] Apache HBase - Examples ... SUCCESS [0.100s] [INFO] Apache HBase - Rest ... SUCCESS [0.107s] [INFO] Apache HBase - External Block Cache ... SUCCESS [0.037s] [INFO] Apache HBase - Assembly ... SUCCESS [0.064s] [INFO] Apache HBase - Shaded . SUCCESS [0.036s] [INFO] Apache HBase - Shaded - Client SUCCESS [0.032s] [INFO] Apache HBase - Shaded - Server SUCCESS [0.069s] [INFO] Apache HBase - Spark .. SUCCESS [0.082s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 8.680s [INFO] Finished at: Tue Sep 22 05:10:06 UTC 2015 [INFO] Final Memory: 39M/1290M [INFO] Looks like its the javadoc issue and no tests included? [~eclark] commit? > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751.v39.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, > 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, > HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, > HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, > HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, > HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, > HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, > HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, > HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, > HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14747145#comment-14747145
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12756170/12751.v38.txt
against master branch at commit d2e338181800ae3cef55ddca491901b65259dc7f.
ATTACHMENT ID: 12756170
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15616//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15616//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15616//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15616//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790723#comment-14790723 ] stack commented on HBASE-12751: --- kalashnikov:hbase.git stack$ python ./dev-support/findHangingTests.py https://builds.apache.org/job/PreCommit-HBASE-Build/15616//consoleText Fetching the console output from the URL Printing hanging tests Hanging test : org.apache.hadoop.hbase.TestIOFencing Hanging test : org.apache.hadoop.hbase.master.TestDistributedLogSplitting Hanging test : org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure Hanging test : org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay Printing Failing tests Failing test : org.apache.hadoop.hbase.client.TestReplicasClient Let me look into these. > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, > 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, > HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, > HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, > HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, > HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, > HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, > HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, > HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, > HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791603#comment-14791603 ] stack commented on HBASE-12751: --- Dang. The hangs are legit and reproducible. Will be back after try and figure the why. > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt, > 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, > 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, > HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, > HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, > HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, > HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, > HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, > HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, > HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, > HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746902#comment-14746902
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12756168/12751.v37.txt
against master branch at commit fe2c4f630d3b5f3346c9ee9f95c256186c9e6907.
ATTACHMENT ID: 12756168
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java:[82,17]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java:[82,45]
cannot find symbol
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:testCompile
(default-testCompile) on project hbase-server: Compilation failure: Compilation
failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java:[82,17]
cannot find symbol
[ERROR] symbol: class AtomicLong
[ERROR] location: class
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationWALReaderManager.java:[82,45]
cannot find symbol
[ERROR] symbol: class AtomicLong
[ERROR] location: class
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15615//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751.v37.txt, 12751.v38.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> 12751v36.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742193#comment-14742193
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755563/12751v36.txt
against master branch at commit 36675c81fb2803418191e44c7eb9560a92a23411.
ATTACHMENT ID: 12755563
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 5 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:110)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3890)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15574//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15574//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15574//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15574//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v36.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741287#comment-14741287
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755418/12751.rebased.v35.txt
against master branch at commit c438052cc2280121727d4ae0883f0b76cad5816a.
ATTACHMENT ID: 12755418
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 5 zombie test(s):
at
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint.testRegionReplicaReplicationIgnoresDisabledTables(TestRegionReplicaReplicationEndpoint.java:350)
at
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint.testRegionReplicaReplicationIgnoresDroppedTables(TestRegionReplicaReplicationEndpoint.java:336)
at
org.apache.hadoop.hbase.replication.TestMasterReplication.testCyclicReplication3(TestMasterReplication.java:220)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.replication.TestPerTableCFReplication.testPerTableCFReplication(TestPerTableCFReplication.java:334)
at
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager.test(TestReplicationWALReaderManager.java:184)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15564//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15564//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15564//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15564//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740426#comment-14740426
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755323/12751.rebased.v35.txt
against master branch at commit fda317cebb5d306cabf1899e05cedb0225b2b62b.
ATTACHMENT ID: 12755323
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.mapreduce.TestImportExport
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testListLabelsWithRegEx(TestVisibilityLabelsWithDefaultVisLabelService.java:220)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15552//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15552//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15552//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15552//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
> HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
> HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
> HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
> HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
> HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
> HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
> HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
> HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741504#comment-14741504 ] stack commented on HBASE-12751: --- All this fails: 2015-09-11 13:08:01.896 Vim[4556:17136812] Can't allocate a new block for a pasteboard. Creation of a new Pasteboard will fail. kalashnikov:hbase.git stack$ python dev-support/findHangingTests.py https://builds.apache.org/job/PreCommit-HBASE-Build/15564//consoleText Fetching the console output from the URL Printing hanging tests Hanging test : org.apache.hadoop.hbase.replication.TestMasterReplication Hanging test : org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Hanging test : org.apache.hadoop.hbase.master.handler.TestCreateTableHandler Hanging test : org.apache.hadoop.hbase.TestIOFencing Hanging test : org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint Hanging test : org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient Hanging test : org.apache.hadoop.hbase.master.TestDistributedLogSplitting Hanging test : org.apache.hadoop.hbase.io.hfile.TestHFileBlock Hanging test : org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer Hanging test : org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures Hanging test : org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingTTL Hanging test : org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure Hanging test : org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster Hanging test : org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache Hanging test : org.apache.hadoop.hbase.master.TestMasterFailover Hanging test : org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner Hanging test : org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover Hanging test : org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager Hanging test : org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 Hanging test : org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite Hanging test : org.apache.hadoop.hbase.regionserver.compactions.TestCompactionWithThroughputController Hanging test : org.apache.hadoop.hbase.master.TestMasterShutdown Hanging test : org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay Hanging test : org.apache.hadoop.hbase.snapshot.TestExportSnapshot Hanging test : org.apache.hadoop.hbase.regionserver.TestRegionReplicas Printing Failing tests Failing test : org.apache.hadoop.hbase.client.TestFastFail Failing test : org.apache.hadoop.hbase.master.procedure.TestWALProcedureStoreOnHDFS > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt, > 12751.rebased.v35.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, > HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, > HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, > HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, > HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, > HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, > HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, > HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, > HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741913#comment-14741913
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755538/12751.rebased.v35.txt
against master branch at commit 36675c81fb2803418191e44c7eb9560a92a23411.
ATTACHMENT ID: 12755538
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15571//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751.rebased.v35.txt,
> 12751.rebased.v35.txt, 12751.rebased.v35.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740036#comment-14740036
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755243/12751.rebased.v34.txt
against master branch at commit bf26088d7be4386864148516b151dfb0a858c416.
ATTACHMENT ID: 12755243
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 99 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 8 zombie test(s):
at
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testVerifyRepJob(TestReplicationSmallTests.java:461)
at
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:148)
at
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager.test(TestReplicationWALReaderManager.java:184)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint.testRegionReplicaReplicationPeerIsCreatedForModifyTable(TestRegionReplicaReplicationEndpoint.java:159)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15539//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15539//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15539//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15539//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751.rebased.v34.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737931#comment-14737931
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755006/12751.rebased.v33.txt
against master branch at commit a11bb2a933ad799546e7179fdf6ce75e3e22d44b.
ATTACHMENT ID: 12755006
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 96 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15511//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15511//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15511//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15511//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751.rebased.v33.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737412#comment-14737412 ] Elliott Clark commented on HBASE-12751: --- not sure how TestTags and TestPerColumnFamilyFlush got broken but everything else seems related. > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, > HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, > HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, > HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, > HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, > HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch, > HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch, > HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, > HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, > HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737531#comment-14737531 ] stack commented on HBASE-12751: --- TestTags is simple fix (I broke tag handling). All rest are likely related, yeah. Thanks for that (smile). We used to block on SyncFuture and if sync failed, we'd clean them all up -- we had a list. Now we are waiting on latch in WALKey#getWriteEntry too... to get WriteEntry. Let me see... > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt, > 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, > HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, > HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, > HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, > HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, > HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch, > HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch, > HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, > HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, > HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737230#comment-14737230
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754911/12751.rebased.v32.txt
against master branch at commit 27d3ab43efeabe2a0e1173858b6994b17b5c355b.
ATTACHMENT ID: 12754911
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 122
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.regionserver.TestTags
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.TestFailedAppendAndSync
{color:red}-1 core zombie tests{color}. There are 8 zombie test(s):
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:228)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:110)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3890)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15500//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15500//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15500//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15500//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751.rebased.v32.txt, 12751.rebased.v32.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
> HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
> HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
> HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
> HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
> HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
> HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
> HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
> HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735503#comment-14735503
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754714/12751.rebased.v26.txt
against master branch at commit e95358a7fc3f554dcbb351c8b7295cafc01e8c23.
ATTACHMENT ID: 12754714
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 2.0.3) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn post-site goal
to fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15471//testReport/
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15471//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15471//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
> HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
> HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
> HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
> HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
> HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
> HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
> HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
> HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735842#comment-14735842
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754745/12751.rebased.v26.txt
against master branch at commit e95358a7fc3f554dcbb351c8b7295cafc01e8c23.
ATTACHMENT ID: 12754745
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.coprocessor.TestWALObserver
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15475//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15475//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15475//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15475//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751v22.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch,
> HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch,
> HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch,
> HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch,
> HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch,
> HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch,
> HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
> HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
> HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735990#comment-14735990
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754767/12751.rebased.v27.txt
against master branch at commit e95358a7fc3f554dcbb351c8b7295cafc01e8c23.
ATTACHMENT ID: 12754767
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ txid = this.wal.append(this.htableDescriptor,
getRegionInfo(), walKey, walEdits, true);
+if (i < (edits.size() - 1) &&
!CellUtil.matchingQualifier(cell, edits.get(i + 1))) idx++;
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.regionserver.TestHMobStore
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.regionserver.TestTags
org.apache.hadoop.hbase.regionserver.TestStore
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.regionserver.TestFailedAppendAndSync
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.regionserver.TestRecoveredEdits
{color:red}-1 core zombie tests{color}. There are 12 zombie test(s):
at
org.apache.hadoop.hbase.client.TestScannerTimeout.test3686b(TestScannerTimeout.java:212)
at
org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testScanWithCompactionInternals(TestBlockEvictionFromClient.java:854)
at
org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testScanWithCompaction(TestBlockEvictionFromClient.java:799)
at
org.apache.hadoop.hbase.client.TestMultiParallel.testBatchWithPut(TestMultiParallel.java:342)
at
org.apache.hadoop.hbase.client.TestReplicasClient.testSmallScanWithReplicas(TestReplicasClient.java:606)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning(TestHRegion.java:3756)
at
org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:141)
at
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient.testCloneLinksAfterDelete(TestMobCloneSnapshotFromClient.java:215)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:228)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15478//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15478//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15478//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15478//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
>
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736028#comment-14736028
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754775/12751.rebased.v29.txt
against master branch at commit e95358a7fc3f554dcbb351c8b7295cafc01e8c23.
ATTACHMENT ID: 12754775
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ txid = this.wal.append(this.htableDescriptor,
getRegionInfo(), walKey, walEdits, true);
+if (i < (edits.size() - 1) &&
!CellUtil.matchingQualifier(cell, edits.get(i + 1))) idx++;
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.coprocessor.TestWALObserver
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15479//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15479//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15479//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15479//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
> HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
> HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
> HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
> HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
> HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
> HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
> HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
> HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
> HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736227#comment-14736227
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754809/12751.rebased.v31.txt
against master branch at commit e95358a7fc3f554dcbb351c8b7295cafc01e8c23.
ATTACHMENT ID: 12754809
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 93 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestStore
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15484//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15484//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15484//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15484//console
This message is automatically generated.
> Allow RowLock to be reader writer
> -
>
> Key: HBASE-12751
> URL: https://issues.apache.org/jira/browse/HBASE-12751
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt,
> 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt,
> 12751.rebased.v31.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt,
> 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
> HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
> HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
> HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
> HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
> HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
> HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
> HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
> HBASE-12751-v9.patch, HBASE-12751.patch
>
>
> Right now every write operation grabs a row lock. This is to prevent values
> from changing during a read modify write operation (increment or check and
> put). However it limits parallelism in several different scenarios.
> If there are several puts to the same row but different columns or stores
> then this is very limiting.
> If there are puts to the same column then mvcc number should ensure a
> consistent ordering. So locking is not needed.
> However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736175#comment-14736175 ] stack commented on HBASE-12751: --- I put it up on rb: https://reviews.apache.org/r/38205/ [~eclark] FYI > Allow RowLock to be reader writer > - > > Key: HBASE-12751 > URL: https://issues.apache.org/jira/browse/HBASE-12751 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > Attachments: 12751.rebased.v25.txt, 12751.rebased.v26.txt, > 12751.rebased.v26.txt, 12751.rebased.v27.txt, 12751.rebased.v29.txt, > 12751.rebased.v31.txt, 12751v22.txt, 12751v23.txt, 12751v23.txt, > 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, > HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, > HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, > HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, > HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, > HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, > HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, > HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, > HBASE-12751-v9.patch, HBASE-12751.patch > > > Right now every write operation grabs a row lock. This is to prevent values > from changing during a read modify write operation (increment or check and > put). However it limits parallelism in several different scenarios. > If there are several puts to the same row but different columns or stores > then this is very limiting. > If there are puts to the same column then mvcc number should ensure a > consistent ordering. So locking is not needed. > However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731555#comment-14731555
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754273/12751.rebased.v25.txt
against master branch at commit 2969093b5b39cb950d8710cfffa7e55484d40259.
ATTACHMENT ID: 12754273
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 87 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMap());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1840 checkstyle errors (more than the master's current 1834 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, List clusterIds, long nonceGroup, long nonce,
MultiVersionConcurrencyControl mvcc) {
+ long logSeqNum, final long now, List clusterIds, long nonceGroup,
long nonce, MultiVersionConcurrencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
org.apache.hadoop.hbase.filter.TestFilter
org.apache.hadoop.hbase.filter.TestInvocationRecordFilter
org.apache.hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan
org.apache.hadoop.hbase.regionserver.TestKeepDeletes
org.apache.hadoop.hbase.filter.TestDependentColumnFilter
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange
org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.regionserver.TestResettingCounters
org.apache.hadoop.hbase.regionserver.TestMinVersions
org.apache.hadoop.hbase.regionserver.TestScanner
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
org.apache.hadoop.hbase.io.encoding.TestPrefixTree
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.regionserver.TestWideScanner
org.apache.hadoop.hbase.master.TestCatalogJanitor
org.apache.hadoop.hbase.client.TestIntraRowPagination
org.apache.hadoop.hbase.regionserver.TestColumnSeeking
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
org.apache.hadoop.hbase.regionserver.TestBlocksScanned
org.apache.hadoop.hbase.mob.TestMobFileCache
org.apache.hadoop.hbase.regionserver.TestHRegionInfo
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction
org.apache.hadoop.hbase.regionserver.TestScanWithBloomError
org.apache.hadoop.hbase.regionserver.TestBulkLoad
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.camel.component.yammer.YammerMultipleUsersRouteTest.testConsumeAllUsers(YammerMultipleUsersRouteTest.java:36)
Test results:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717019#comment-14717019
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752796/12751v23.txt
against master branch at commit 8f95318f6252c1c0b7a073619525eae6d991f47b.
ATTACHMENT ID: 12752796
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 84 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] Error invoking method 'get(java.lang.Integer)' in
java.util.ArrayList at META-INF/LICENSE.vm[line 1619, column 22]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on
project hbase-assembly: Error rendering velocity resource. Error invoking
method 'get(java.lang.Integer)' in java.util.ArrayList at
META-INF/LICENSE.vm[line 1619, column 22]: InvocationTargetException: Index: 0,
Size: 0 - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn goals -rf :hbase-assembly
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15297//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717884#comment-14717884
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752867/12751v23.txt
against master branch at commit 8f95318f6252c1c0b7a073619525eae6d991f47b.
ATTACHMENT ID: 12752867
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 84 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConcurrencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConcurrencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.master.procedure.TestWALProcedureStoreOnHDFS
org.apache.hadoop.hbase.security.access.TestNamespaceCommands
{color:red}-1 core zombie tests{color}. There are 10 zombie test(s):
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testListLabelsWithRegEx(TestVisibilityLabelsWithDefaultVisLabelService.java:220)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15307//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15307//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15307//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15307//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: 12751v22.txt, 12751v23.txt, 12751v23.txt, 12751v23.txt,
12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch,
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715295#comment-14715295 ] stack commented on HBASE-12751: --- v23 is building now over on hadoopqa. It failed TestPerColumnFamilyFlush and this fails local for me too. Looking. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: 12751v22.txt, 12751v23.txt, 12751v23.txt, HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715350#comment-14715350
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752522/12751v23.txt
against master branch at commit aca8c3b74b09646c72c4e0fe26a4b2103da0d288.
ATTACHMENT ID: 12752522
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 84 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConcurrencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConcurrencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
{color:red}-1 core zombie tests{color}. There are 5 zombie test(s):
at
org.apache.camel.component.jetty.JettySuspendWhileInProgressTest.testJettySuspendWhileInProgress(JettySuspendWhileInProgressTest.java:55)
at
org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd.testEndToEnd(TestFuzzyRowFilterEndToEnd.java:143)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15276//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15276//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15276//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15276//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: 12751v22.txt, 12751v23.txt, 12751v23.txt,
HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch,
HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch,
HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch,
HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch,
HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch,
HBASE-12751-v20.patch, HBASE-12751-v21.patch, HBASE-12751-v3.patch,
HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715473#comment-14715473
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752522/12751v23.txt
against master branch at commit aca8c3b74b09646c72c4e0fe26a4b2103da0d288.
ATTACHMENT ID: 12752522
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 84 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConcurrencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConcurrencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mob.TestDefaultMobStoreFlusher
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
org.apache.hadoop.hbase.mob.compactions.TestPartitionedMobCompactor
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
org.apache.hadoop.hbase.TestServerSideScanMetricsFromClientSide
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
{color:red}-1 core zombie tests{color}. There are 28 zombie test(s):
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithACL.testLabelsTableOpsWithDifferentUsers(TestVisibilityLabelsWithACL.java:233)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testListLabelsWithRegEx(TestVisibilityLabelsWithDefaultVisLabelService.java:220)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15275//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15275//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712255#comment-14712255
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752357/12751v22.txt
against master branch at commit 506726ed2832b069602c6b7e2ccd5ec9a81013a6.
ATTACHMENT ID: 12752357
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 81 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1302,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1398,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1610,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:testCompile
(default-testCompile) on project hbase-server: Compilation failure: Compilation
failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1302,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR] required:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.wal.WALKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,boolean
[ERROR] found:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.regionserver.wal.HLogKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,java.util.concurrent.atomic.AtomicLong,boolean,nulltype
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1398,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR] required:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.wal.WALKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,boolean
[ERROR] found:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.regionserver.wal.HLogKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,java.util.concurrent.atomic.AtomicLong,boolean,nulltype
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java:[1610,12]
method append in interface org.apache.hadoop.hbase.wal.WAL cannot be applied
to given types;
[ERROR] required:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.wal.WALKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,boolean
[ERROR] found:
org.apache.hadoop.hbase.HTableDescriptor,org.apache.hadoop.hbase.HRegionInfo,org.apache.hadoop.hbase.regionserver.wal.HLogKey,org.apache.hadoop.hbase.regionserver.wal.WALEdit,java.util.concurrent.atomic.AtomicLong,boolean,nulltype
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn goals -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15259//console
This message is automatically
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710173#comment-14710173
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12752051/HBASE-12751-v21.patch
against master branch at commit 9334a47d4570f8adfc003f0fb2c5969a88c3bba0.
ATTACHMENT ID: 12752051
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 106
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:red}-1 javac{color}. The applied patch generated 27 javac compiler
warnings (more than the master's current 26 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1856 checkstyle errors (more than the master's current 1849 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConcurrencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConcurrencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3799)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15224//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15224//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15224//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15224//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15224//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v21.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703746#comment-14703746
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751285/HBASE-12751-v20.patch
against master branch at commit 16f8d277088987dc8d4bb1614d05ce712bfc245d.
ATTACHMENT ID: 12751285
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 81 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationEndpoint
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.TestGlobalMemStoreSize
org.apache.hadoop.hbase.wal.TestWALFiltering
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
org.apache.hadoop.hbase.replication.TestReplicationWithTags
org.apache.hadoop.hbase.master.procedure.TestTruncateTableProcedure
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.procedure.TestAddColumnFamilyProcedure
org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure
org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
{color:red}-1 core zombie tests{color}. There are 16 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsWithComplexLabels(TestVisibilityLabels.java:216)
at
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702789#comment-14702789
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751213/HBASE-12751-v20.patch
against master branch at commit 1bb9e3ae966853db8c4138b4abbe14636d7592db.
ATTACHMENT ID: 12751213
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 81 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
{color:red}-1 core zombie tests{color}. There are 8 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testListLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:198)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes.testDeleteColumnWithSpecificTimeStampUsingMultipleVersionsUnMatchingVisExpression(TestVisibilityLabelsWithDeletes.java:992)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3799)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15162//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15162//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15162//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15162//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch,
HBASE-12751-v20.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702101#comment-14702101
]
stack commented on HBASE-12751:
---
Reviewing latest version:
{code}
Seems weird that the region doesn't own its sequenceid. Notion is that now the
mvcc instance owns the sequenceid and that there is an mvcc instance per
region. Thats fine. Just need to draw it out some. For example here in mvcc
class javadoc it say:
Manages the read/write consistency. This provides...
Should be 'for the region'?
I did not realize that the append-to-WAL was in more than just the WAL append
method but it is also up here in processRowsWithLocks and elsewhere...That is a
little crazy. Any chance at unification in a follow-on in your opinion? They
differ by a little only and the steps are pretty involved
Lots of good clean up in here. Better method namings in mvcc class.
We run through all edits twice, once to set OperationStatus.SUCCESS and then
later to do postPut/postDelete on CP. Could these be combined? CP only looks at
current batch so should be ok.
Need to work on case where edits may get into the WAL then we fail memstore
update or sync. They are in WAL... the edits might get replayed or replicated
though client may have gotten message they were rejected. Not an issue
introduced here...
When would sequenceid be non-null in below:
if (cell.getSequenceId() == 0) {
CellUtil.setSequenceId(cell, mvccNum);
}
Maybe this should be an exception so we can find cases where this is set
already? Perhaps in DLR when replay? Later we set sequenceid for these.
How is this addressed?
// TODO: If we are using FlushAllStoresPolicy, then this can make edits
visible from other
// stores while they are still in flight because the flush commit
marker will not contain
// flushes from ALL stores.
The comment is removed without comment. It looks like still a prob?
Why this long currentSeqId = mvcc.readPoint.get(); and not mvcc.getReadPoint?
Ain't this done already for you higher up when you come in for a region
operation?
// Make sure the row is inside of this region before getting the lock for
it.
checkRow(row, row lock);
Would be good to avoid another set of compares.
// TODO(eclark): Should this be an array of fixed size to
// reduce the number of allocations on the write path?
// This could be equal to the number of handlers + a small number.
Sounds right. If more than outstanding handlers, something is wrong.
So, mvcc write number can be incremented in a few places for various reasons...
but looking in FSWALEntry#stampRegionSequenceId, order in WAL and into memstore
should be good.
writeCompactionMarker and writeRegionEventMarker in WALUtil are hard to follow
-- the intrinsic sequence manipulation. Makes sense if you dig but this stuff
is complicated (not you... just is)
{code}
Let me see why all the test failures.
I think we should get this in. Good cleanup, some optimization, and a couple
of good bug fixes including a dataloss.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch,
HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch,
HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch,
HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702403#comment-14702403
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751157/HBASE-12751-v19%20%281%29.patch
against master branch at commit 1bb9e3ae966853db8c4138b4abbe14636d7592db.
ATTACHMENT ID: 12751157
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 106
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:red}-1 javac{color}. The applied patch generated 27 javac compiler
warnings (more than the master's current 26 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1860 checkstyle errors (more than the master's current 1852 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapred.TestTableInputFormat
[7]
org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.snapshot.TestExportSnapshot
org.apache.hadoop.hbase.util.TestHBaseFsckEncryption
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot
org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.TestNamespace
org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite
org.apache.hadoop.hbase.util.TestCoprocessorScanPolicy
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.replication.TestMasterReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15155//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15155//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15155//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15155//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15155//console
This message is automatically generated.
Allow RowLock to be reader writer
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702413#comment-14702413 ] stack commented on HBASE-12751: --- java.lang.OutOfMemoryError: unable to create new native thread Running this patch locally, near all above pass... TestImportExport and TestRowCounter fail... Let me put up new patch that addresses feedback and the above red lines back from hadoopqa. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702198#comment-14702198 ] Elliott Clark commented on HBASE-12751: --- bq.Should be 'for the region'? Yep. bq.Any chance at unification in a follow-on in your opinion? Totally. There are still some weird things in the write path that need to be polished up. bq.We run through all edits twice, once to set OperationStatus.SUCCESS and then later to do postPut/postDelete on CP. Could these be combined? I would think that it's ok. Though I don't have much experience with CP use cases. bq.How is this addressed? It's not and I shouldn't have removed it. bq.Why this long currentSeqId = mvcc.readPoint.get(); and not mvcc.getReadPoint? I should make it getReadPoint. bq.Ain't this done already for you higher up when you come in for a region operation? That was there before. At FB we're still seeing an error every once in a while about row out of range for replication traffic. It would be great to remove it or make it happen less frequently. bq.writeCompactionMarker and writeRegionEventMarker in WALUtil are hard to follow Yeah they haven't had enough attention from me on cleaning that code up. It was mostly done by a quick pass where the rest of the mvcc stuff got some more love. bq.Let me see why all the test failures. What I've been seeing is a deadlock on taking down a regionserver. Running only one method in the class that errors out means that everything passes. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697979#comment-14697979 ] Elliott Clark commented on HBASE-12751: --- Rebased and update https://reviews.facebook.net/D32079 Everything should be good. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697978#comment-14697978 ] Elliott Clark commented on HBASE-12751: --- Yeah I totally want to get this in. I have just been a little more focused on 1.2 lately. Any help would be greatly appreciated. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697974#comment-14697974 ] stack commented on HBASE-12751: --- We want to get this in [~eclark]? I can take it from here if you want? (You've done the hard part) Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698133#comment-14698133
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12750630/HBASE-12751-v19.patch
against master branch at commit ecba08829da149aca11929b5b88a6df880e0da35.
ATTACHMENT ID: 12750630
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 106
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:red}-1 javac{color}. The applied patch generated 27 javac compiler
warnings (more than the master's current 26 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1865 checkstyle errors (more than the master's current 1857 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd
org.apache.hadoop.hbase.mapreduce.TestImportTSVWithTTLs
org.apache.hadoop.hbase.mapreduce.TestTableInputFormat
org.apache.hadoop.hbase.TestMetaTableAccessor
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.mapreduce.TestHashTable
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{color:red}-1 core zombie tests{color}. There are 22 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestRowCounter.testRowCounterColumnWithColonInQualifier(TestRowCounter.java:125)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:240)
at
org.apache.hadoop.hbase.mapreduce.TestCopyTable.testCopyTable(TestCopyTable.java:123)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:372)
at
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatTestBase.testWithMapReduce(TableSnapshotInputFormatTestBase.java:106)
at
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatTestBase.testWithMapReduceAndOfflineHBaseMultiRegion(TableSnapshotInputFormatTestBase.java:101)
at
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed.testTimeRangeMapRed(TestTimeRangeMapRed.java:163)
at
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646354#comment-14646354 ] Elliott Clark commented on HBASE-12751: --- I'm still working on the last two test failures. Everything seems fine except for the regionservers won't go down. That causes the cluster to stay up and the test to timeout. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646636#comment-14646636
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12747804/HBASE-12751-v18.patch
against master branch at commit 05de2ec5801fbba4577fb363f858a6e6f282c104.
ATTACHMENT ID: 12747804
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 106
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:red}-1 javac{color}. The applied patch generated 27 javac compiler
warnings (more than the master's current 26 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1872 checkstyle errors (more than the master's current 1864 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:110)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14925//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14925//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14925//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14925//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14925//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch,
HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14641897#comment-14641897 ] stack commented on HBASE-12751: --- bq. ...could we just add to MemStore in the WAL appender/ringbuffer consuming thread... Thinking on it more, the above won't do much for us. We'd be adding edits to memstore making them available for reads before the sync completes. Add the edits to an intermediate collection that gets added to memstore after syncs complete (would only make sense if the collection were uncontended)? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642109#comment-14642109
]
stack commented on HBASE-12751:
---
Studying the patch more, I have better idea what is going on. It seems mvcc is
carrying what was sequenceid in HRegion now so my comment above on two counters
is off. Also deleted a comment that wasn't helping.
Would it be better if mvcc didn't know anything about memstore? Should we be
asking mvcc to advance memstore...
{code}
mvcc.advanceMemstore(key.getWriteEntry());
{code}
... though I see that mvcc used to know about memstore so this is nit (I just
saw we used to have completeMemstoreInsertWithSeqNum)
We dropped an optimization when we let go of the below and instead just say
true for getRowLock.. Or was it something that just didn't work? (Or looks like
the semantic changed from waitFor to whether a read or write lock?)
{code}
boolean shouldBlock = numReadyToWrite == 0;
{code}
For sure memstoreCells (removed) is same as...
{code}
for (int j=0; jfamilyMaps.length;j++) {
for(ListCell cells:familyMaps[j].values()) {
...
{code}
No chance that what is passed to Store to add gets filtered or ends up being
subset for any reason?
The comment below no longer applies? It is dropped in the patch.
{code}
// there may be some in-flight transactions, but they won't be made
visible since they are
// either greater than flush seq number or they were already dropped
via flush.
// TODO: If we are using FlushAllStoresPolicy, then this can make edits
visible from other
// stores while they are still in flight because the flush commit
marker will not contain
// flushes from ALL stores.
getMVCC().advanceMemstoreReadPointIfNeeded(flush.getFlushSequenceNumber());
{code}
In mvcc it is memstoreRead for the read point. The name seems to say it applies
to memstore only. It does not apply to hfiles? HFiles carry sequenceids on
edits and in metadata.
Will be back to do final 1/3rd of patch.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642106#comment-14642106 ] Elliott Clark commented on HBASE-12751: --- bq.So mvcc has its own running sequenceid-like thing now and it runs independent of sequenceid All sequence id numbers are the mvcc number of the edit. Anything that doesn't apply a wal edit get's a sequence id by creating a mvcc write number and waiting for it to be completed. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642275#comment-14642275 ] Elliott Clark commented on HBASE-12751: --- bq.Would it be better if mvcc didn't know anything about memstore? Should we be asking mvcc to advance memstore... Yeah that's pretty bad wording. Something like advanceReadPoint? Some other wording for sure. bq.We dropped an optimization when we let go of the below and instead just say true for getRowLock.. Or was it something that just didn't work? (Or looks like the semantic changed from waitFor to whether a read or write lock?) The hope is that since we won't be blocking for any normal puts there is no need for the added complexity. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14641799#comment-14641799
]
stack commented on HBASE-12751:
---
Should we read at with the same mvcc in these two locations below (could edit
we write the WAL disagree with the file sequenceid we write out)?
{code}
97 RegionEventDescriptor regionEventDesc =
ProtobufUtil.toRegionEventDescriptor(
98 - RegionEventDescriptor.EventType.REGION_CLOSE, getRegionInfo(),
getSequenceId().get(),
99 + RegionEventDescriptor.EventType.REGION_CLOSE, getRegionInfo(),
mvcc.memstoreRead.get(),
100getRegionServerServices().getServerName(), storeFiles);
101 -WALUtil.writeRegionEventMarker(wal, getTableDesc(), getRegionInfo(),
regionEventDesc,
102 - getSequenceId());
103 +WALUtil.writeRegionEventMarker(wal, getTableDesc(), getRegionInfo(),
regionEventDesc);
104 •
105 // Store SeqId in HDFS when a region closes
106 // checking region folder exists is due to many tests which delete
the table folder while a
107 // table is still online
108 if (this.fs.getFileSystem().exists(this.fs.getRegionDir())) {
109WALSplitter.writeRegionSequenceIdFile(this.fs.getFileSystem(),
this.fs.getRegionDir(),
110 -getSequenceId().get(), 0);
111 +mvcc.memstoreReadPoint(), 0);
{code}
... and later on so log matches the condition check.
bq. That allows a creating the sequence id/mvcc number for real.
So mvcc has its own running sequenceid-like thing now and it runs independent
of sequenceid. Previous we were just using WAL sequenceid everywhere (though we
had that +1B hackery going on). It took some work to get us down to one
sequence only. The patch adds back another. If just fixing the ordering problem
(ordering in memstore vs WAL), could we just add to MemStore in the WAL
appender/ringbuffer consuming thread? Would that be too slow? You'd have to
rollback still if failure but less moving parts?
bq. mvcc.completeMemstoreInsert(mvcc.beginMemstoreInsert());
The above seems odd. Make a method on mvcc to do the above? Could return the
current read point (going by what is needed later in the patch)
And the mvcc initialize semantic has changed? It used to be
advanceMemstoreReadPointIfNeeded but now we override initialize and call it
often. It is a bit hard to follow what initialize does now given it is called
frequently.
Why does WALKey take an mvcc? And then, how comes you can get mvcc from WALKey?
I'd think WALKey would be made once on construction and would not be
concerned with stuff like mvccs and writeEntries. Or if it was, that it'd be
internal to WALKey implementation (You'd not be able to get mvcc from WALKey).
WALKey has an mvcc but then you can call setWriteEntry? Would be cool if WALKey
knew nought of mvccs.
In the below:
{code}
115 we = mvcc.beginMemstoreInsert();
116 regionSequenceId = we.getWriteNumber();
{code}
... is this the region sequenceid? Or is it the mvcc number?
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14641350#comment-14641350
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12747127/HBASE-12751-v17.patch
against master branch at commit dad4cad30e5b0c69694ee90908ad8e74c592d821.
ATTACHMENT ID: 12747127
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 106
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:red}-1 javac{color}. The applied patch generated 25 javac compiler
warnings (more than the master's current 24 warnings).
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1885 checkstyle errors (more than the master's current 1866 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ public HLogKey(final byte[] encodedRegionName, final TableName tablename,
final long now, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ public WALKey(final byte[] encodedRegionName, final TableName tablename,
final long now, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long txid = log.append(htd, hri, new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc),
+new WALKey(info.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+new WALKey(hri.getEncodedNameAsBytes(), htd.getTableName(),
System.currentTimeMillis(), mvcc),
+final WALKey logkey = new WALKey(hri.getEncodedNameAsBytes(),
hri.getTable(), now, mvcc);
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
{color:red}-1 core zombie tests{color}. There are 7 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:110)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2.testWritingPEData(TestHFileOutputFormat2.java:335)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:378)
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScanBase.testScan(TestTableInputFormatScanBase.java:243)
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.testScanEmptyToEmpty(TestTableInputFormatScan1.java:45)
at
org.apache.hadoop.hbase.mapreduce.TestCellCounter.testCellCounteEndTimeRange(TestCellCounter.java:179)
at
org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatTestBase.testScan(MultiTableInputFormatTestBase.java:255)
at
org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatTestBase.testScanEmptyToEmpty(MultiTableInputFormatTestBase.java:196)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14889//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14889//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14889//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639243#comment-14639243 ] Elliott Clark commented on HBASE-12751: --- bq.Let me look at the patch again. The above description sounds like how it mostly worked previous. Let me see Thanks E. Yeah it's pretty similar. The patch is just adding to the wal append/sync queue earlier. That allows a creating the sequence id/mvcc number for real. Because the edits are already in the queue for the wal we know for sure that their ordering is the same as the sequence number. The bonus nice thing from this re-ordering is that adding to the memstore and the append are running in parallel. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638576#comment-14638576 ] stack commented on HBASE-12751: --- bq. People were really digging into sequence id a lot on tests. Crew on your side [~eclark]? Some notes: https://docs.google.com/document/d/16beczDie-KU1uSpJvd0GoUlQbPtQBL93rOOPqnE5Ma0/edit# bq. Do we want to make the locking fair? I think so as a means of evening out the latencies. bq. All edits are queued in the wal before they are put in the memstore... Our comparitor on the map does that. Let me look at the patch again. The above description sounds like how it mostly worked previous. Let me see Thanks E. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637314#comment-14637314 ] Elliott Clark commented on HBASE-12751: --- Targeting 1.3 but we can always re-schedule if this takes longer than anticipated. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637309#comment-14637309 ] Elliott Clark commented on HBASE-12751: --- So here's a new patch with a cleaner row locking. It uses the java provided re-entrant lock for reader writer. It passes the few unit tests that I've been using for spot checking. Lets see how this looks on hadoop qa. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637558#comment-14637558 ] Elliott Clark commented on HBASE-12751: --- Do we want to make the locking fair? or should normal puts be able to starve check and mutates? Fairness will increase the size of the read write lock. However it will keep things from starving. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638087#comment-14638087
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746677/HBASE-12751-v16.patch
against master branch at commit 493f36c899bc990de0400fbf777a2bf29c5c60e3.
ATTACHMENT ID: 12746677
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 102
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1887 checkstyle errors (more than the master's current 1867 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ public HLogKey(final byte[] encodedRegionName, final TableName tablename,
final long now, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ public WALKey(final byte[] encodedRegionName, final TableName tablename,
final long now, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.snapshot.TestExportSnapshot
org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass
org.apache.hadoop.hbase.TestMovedRegionsCleaner
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorEndpoint
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.TestPartialResultsFromClientSide
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.quotas.TestQuotaAdmin
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testLabelsWithAppend(TestVisibilityLabels.java:620)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.quotas.TestQuotaThrottle.testUserTableReadAndWriteThrottle(TestQuotaThrottle.java:206)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:378)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637636#comment-14637636
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746596/HBASE-12751-v14.patch
against master branch at commit 5ec5552be0534dbf4b07ef6607741ae6f9ab0495.
ATTACHMENT ID: 12746596
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1889 checkstyle errors (more than the master's current 1870 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.wal.TestDefaultWALProvider
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.wal.TestWALFactory
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface.testHBase3758(TestRegionObserverInterface.java:424)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddLabels(TestVisibilityLabelsWithDefaultVisLabelService.java:110)
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:427)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.coprocessor.TestClassLoading.testClassLoadingFromRelativeLibDirInJar(TestClassLoading.java:379)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14861//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14861//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14861//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637933#comment-14637933
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746627/HBASE-12751-v15.patch
against master branch at commit 493f36c899bc990de0400fbf777a2bf29c5c60e3.
ATTACHMENT ID: 12746627
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1886 checkstyle errors (more than the master's current 1867 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.wal.TestDefaultWALProvider
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testListLabelsWithRegEx(TestVisibilityLabelsWithDefaultVisLabelService.java:220)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//artifact/patchprocess/patchReleaseAuditWarnings.txt
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14865//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635386#comment-14635386 ] Elliott Clark commented on HBASE-12751: --- I'm going to re-write the row lock method. It's gotten a little messy, might as well clean it up while I'm messing around. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634150#comment-14634150 ] Lars Hofhansl commented on HBASE-12751: --- What if we write without WAL? Does that matter? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634153#comment-14634153 ] Elliott Clark commented on HBASE-12751: --- An edit that's skipping the wal will still queue an empty edit to the wal, just not wait on it to append or sync. ( that's the same as what's there now just in a different order ) Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634072#comment-14634072 ] Elliott Clark commented on HBASE-12751: --- Attaching the rebased patch. bq.How is order in memstore for sure matching the append order to WAL now? All edits are queued in the wal before they are put in the memstore. After they are queued the first thing that's done is to assign a sequence number. The insert to memstore waits on that number to be added to the wal edit. Once that number is found it's used by the memstore for ordering. Our comparitor on the map does that. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625855#comment-14625855 ] stack commented on HBASE-12751: --- Took another look at patch. Nice cleanup. How is order in memstore for sure matching the append order to WAL now? What we need to get this in? You need a bit of testing on this end? Just say and I can try some stuff... Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14581249#comment-14581249
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12738930/HBASE-12751-v12.patch
against master branch at commit 0f93986015122a517e1a0c949e159bf8fb218092.
ATTACHMENT ID: 12738930
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1931 checkstyle errors (more than the master's current 1912 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestChoreService.testForceTrigger(TestChoreService.java:398)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14374//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14374//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14374//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14374//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553554#comment-14553554
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12734282/HBASE-12751-v11.patch
against master branch at commit 3a1e101dcd4cf0e7149c0d8b1c13f96adf04278e.
ATTACHMENT ID: 12734282
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1915 checkstyle errors (more than the master's current 1894 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.wal.TestDefaultWALProvider
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
{color:red}-1 core zombie tests{color}. There are 7 zombie test(s):
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.regionserver.TestCorruptedRegionStoreFile.testLosingFileDuringScan(TestCorruptedRegionStoreFile.java:141)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14133//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14133//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14133//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14133//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538532#comment-14538532
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731955/HBASE-12751-v9.patch
against master branch at commit 671ae8f150ceeb24fcc99743c865fccfb772b130.
ATTACHMENT ID: 12731955
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 102
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1918 checkstyle errors (more than the master's current 1896 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.wal.TestDefaultWALProvider
org.apache.hadoop.hbase.regionserver.wal.TestFSHLog
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.coprocessor.TestWALObserver
{color:red}-1 core zombie tests{color}. There are 8 zombie test(s):
at
org.apache.hadoop.hbase.client.TestScannerTimeout.test3686a(TestScannerTimeout.java:193)
at
org.apache.hadoop.hbase.client.TestMultiParallel.testBatchWithPut(TestMultiParallel.java:340)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
at
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient.testCloneSnapshot(TestCloneSnapshotFromClient.java:171)
at
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538904#comment-14538904
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12732000/HBASE-12751-v10.patch
against master branch at commit fa6dc9c44e28aa608d5204db814e8442b95eb125.
ATTACHMENT ID: 12732000
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1918 checkstyle errors (more than the master's current 1896 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.regionserver.wal.TestFSHLog
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.wal.TestDefaultWALProvider
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.wal.TestWALFactory
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsOnRSRestart(TestVisibilityLabels.java:365)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddVisibilityLabelsOnRSRestart(TestVisibilityLabelsWithDefaultVisLabelService.java:122)
at
org.apache.hadoop.hbase.regionserver.TestAtomicOperation.testPutAndCheckAndPutInParallel(TestAtomicOperation.java:558)
at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:229)
Test results:
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533203#comment-14533203
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731208/HBASE-12751-v8.patch
against master branch at commit 11b76732c0ec80a2cccbe7937440bd107e577c8b.
ATTACHMENT ID: 12731208
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 102
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1918 checkstyle errors (more than the master's current 1896 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.regionserver.wal.TestFSHLog
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testRegionReplicaSecondaryCannotFlush(TestHRegionReplayEvents.java:216)
at
org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders.testSeekingOnSample(TestDataBlockEncoders.java:206)
at
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding.testChangingEncodingWithCompaction(TestChangingEncoding.java:215)
at
org.apache.camel.test.spring.CamelSpringTestContextLoader.loadBeanDefinitions(CamelSpringTestContextLoader.java:185)
at
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523470#comment-14523470 ] stack commented on HBASE-12751: --- The patch looks really nice; a radical simplification. Let me study more. Am also working on DLR tests at moment so we'll be able to see if this breaks it. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522951#comment-14522951
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12729726/HBASE-12751-v7.patch
against master branch at commit 1d26d2640734b919d9bf266326e211be0478c711.
ATTACHMENT ID: 12729726
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 102
new or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1920 checkstyle errors (more than the master's current 1898 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestStoreFileRefresherChore
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13910//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13910//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13910//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13910//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13910//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522899#comment-14522899
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12729721/HBASE-12751-v6.patch
against master branch at commit 1d26d2640734b919d9bf266326e211be0478c711.
ATTACHMENT ID: 12729721
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 98 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1920 checkstyle errors (more than the master's current 1898 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+final
WALProtos.BulkLoadDescriptor descriptor) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestStoreFileRefresherChore
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13908//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13908//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13908//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13908//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13908//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494440#comment-14494440
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725267/HBASE-12751-v5.patch
against master branch at commit 4f151444b58ae85b93f76254961358932e0ffb9b.
ATTACHMENT ID: 12725267
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 94 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1928 checkstyle errors (more than the master's current 1910 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+this.htableDescriptor.getTableName(),
WALKey.NO_SEQUENCE_ID, nonceGroup, nonce, mvcc);
+ final WALEdit edits, final AtomicLong sequenceId, final boolean
inMemstore) throws IOException {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
final MultiVersionConsistencyControl mvcc) {
+ final long now, ListUUID clusterIds, long nonceGroup, long nonce,
MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+ long logSeqNum, final long now, ListUUID clusterIds, long nonceGroup,
long nonce, MultiVersionConsistencyControl mvcc) {
+long txid = log.append(htd, info, getWalKey(System.currentTimeMillis()),
edit, sequenceId, true);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestScanner
org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster
org.apache.hadoop.hbase.regionserver.wal.TestWALActionsListener
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13700//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13700//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13700//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13700//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13700//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482961#comment-14482961
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12723578/HBASE-12751-v2.patch
against master branch at commit 57f9c821287a3780384bcb37d764f13300c93a8d.
ATTACHMENT ID: 12723578
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 82 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1928 checkstyle errors (more than the master's current 1921 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final WALEdit edits, final AtomicLong sequenceId, final boolean
inMemstore) throws IOException {
+if (!this.getEdit().isReplay()
!CollectionUtils.isEmpty(getEdit().getCells()) inMemstore) {
+long txid = log.append(htd, info, getWalKey(System.currentTimeMillis()),
edit, sequenceId, true);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestResettingCounters
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13596//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13596//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13596//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13596//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13596//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483007#comment-14483007
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12723583/HBASE-12751-v3.patch
against master branch at commit 57f9c821287a3780384bcb37d764f13300c93a8d.
ATTACHMENT ID: 12723583
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1928 checkstyle errors (more than the master's current 1921 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final WALEdit edits, final AtomicLong sequenceId, final boolean
inMemstore) throws IOException {
+if (!this.getEdit().isReplay()
!CollectionUtils.isEmpty(getEdit().getCells()) inMemstore) {
+long txid = log.append(htd, info, getWalKey(System.currentTimeMillis()),
edit, sequenceId, true);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestResettingCounters
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13597//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13597//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13597//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13597//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13597//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483381#comment-14483381 ] stack commented on HBASE-12751: --- [~jleach] FYI Skimmed. When we come back from an append, the WALEdit has its seqid? Good. bq. 3092doRollBackMemstore = true; // If we have a failure, we need to clean what we wrote Dang, I suppose we still need above pity couldn't push the insert to memstore out past the sync.. so could do without the rollback stuff. Fix step numbering order in processRowsWithLocks This looks like nice ironing out of some of the chicanery we have going on in our write path. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483469#comment-14483469 ] Elliott Clark commented on HBASE-12751: --- bq.When we come back from an append, the WALEdit has its seqid? Good. Not always but normally the append happens fast enough that it's instant. However walkey.getSequenceId has a count down latch. So getting the mvccNum waits to make sure that the cells are at least appended. bq.Dang, I suppose we still need above pity couldn't push the insert to memstore out past the sync.. so could do without the rollback stuff. That would be pretty radical. Let me think on that for a little bit. bq.This looks like nice ironing out of some of the chicanery we have going on in our write path. I think there is still some more to iron out around the Mvcc write entry. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483700#comment-14483700
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12723666/HBASE-12751-v4.patch
against master branch at commit 57f9c821287a3780384bcb37d764f13300c93a8d.
ATTACHMENT ID: 12723666
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1930 checkstyle errors (more than the master's current 1921 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ final WALEdit edits, final AtomicLong sequenceId, final boolean
inMemstore) throws IOException {
+if (!this.getEdit().isReplay()
!CollectionUtils.isEmpty(getEdit().getCells()) inMemstore) {
+long txid = log.append(htd, info, getWalKey(System.currentTimeMillis()),
edit, sequenceId, true);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestKeepDeletes
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.filter.TestFilter
org.apache.hadoop.hbase.regionserver.TestScanner
org.apache.hadoop.hbase.regionserver.TestStoreFileRefresherChore
org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
org.apache.hadoop.hbase.regionserver.TestColumnSeeking
org.apache.hadoop.hbase.regionserver.TestWideScanner
org.apache.hadoop.hbase.regionserver.TestMinVersions
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.filter.TestDependentColumnFilter
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction
org.apache.hadoop.hbase.client.TestIntraRowPagination
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13598//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13598//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13598//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13598//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13598//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378104#comment-14378104 ] Elliott Clark commented on HBASE-12751: --- Sure so all the fun happens in HRegion#doMiniBatchMutation. I'll talk about line numbers and current implementation though they might change so I've linked what I currently see. h2. Normal Flow * on line [3039|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3039] row locks for all the mutations possible are obtained. In the current code this means that only one set of mutations for a row can continue at a time. However if we change that to reduce contention on rows things break farther down. * on line [3109|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3109] a pre-assigned read point is assigned 1M + some sequence This sequence is independent of the actual sequence id. It is entirely possible for this sequence to be assigned to edits in a different order than they will actually be applied to the wal log. * on line [3141|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3141] the mutations are added to the memstore and the memstore adds them to CellSkipListSet which uses KVComparator.COMPARATOR to sort mutations. They sort in the pretty normal order of rk, family, qual, timestamp, mvcc number. Since the sequence Id here is 1M + some these edits sort before anything that's committed. * Each edit that's applied to the memstore is also added to the memstoreCells ArrayList. * the list of edits applied is packaged into a wal edit. * [3182|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3181] this waledit is then appended to the wal * inside of FSHlog [1191|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L1191] an FSWalEntry is created and added to the ring buffer. * at some point the ring buffer calls [1920|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L1920] the event handler's FSHlog.RingBufferEventHandler#onEvent method. * this event handler notices that the event is an append and so it calls [2026|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L2026] FSHlog.RingBufferEventHandler#append. * inside that function at [2037|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L2037] we call FSWalEntry#stampRegionSequenceId * FSWalEntry#stampRegionSequenceId goes through every mutation that is in the memstore arraylist and changes their sequence id. [127|https://github.com/apache/hbase/blob/a75a2ace4f52daa1eb415f085dfad920b30c3349/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java#L127] * The mutations in the memstore's CellSkipListSet get a new sequence id. However the comparator is only run on insert so any change in the sequence number that would change order isn't reflected in the order that CellSkipListSet will return key values. * HRegion#doMiniBatchMutation releases the row locks and continues on. * the WAL is synced and everything is returned as success. h2. The Issue The sequence id from MultiVersionConsistencyControl#getPreAssignedWriteNumber is completely independent of the sequence id that the ring buffer hands out. Right now we luck out and never have to deal with this because all edits to the same row are serialized by the row locks. However if we change it so that more than one edit for a row can be in the memstore's map with a pre-assigned sequence id then this all breaks. Consider the following event for two puts coming in at the same time for the same row, cf, qual, and timestamp but different values (assuming that row locks are reader/writer and normal puts only need readers): * Put #1 comes in gets reader lock * Put #2 comes in gets reader lock * Put #1 gets initial sequence id of 1M * Put #2 gets initial sequence id of 1M+1 * Put #1 inserts into memstore. * Put #2 inserts into memstore. * Memstore is now ordered like this [ Put#2, Put#1 ] because 1M+1 sorts before 1M * Put #2 appends to the FSWal. With this Put #2 gets real sequence id of 1 * Put #1 appends to the FSWal. With this Put #1 gets real sequence id
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378131#comment-14378131 ] stack commented on HBASE-12751: --- Excellent, excellent stuff (this whole issue and this finding) So, first response is same as yours: Add to memstore after WAL; the edits can't be 'seen' yet anyways because they are beyond the read point But our current ordering of memstore first was about perf so would need to look at this. Other things are going on here though FYI, our [~busbey] is looking at WAL perf in the layer below DFS client and is having some radical thoughts... so this is right time to raise reorder on the append pipeline; longer time between syncs means more grouping... higher throughput. You looking at the rerdering then [~eclark]? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377358#comment-14377358 ] stack commented on HBASE-12751: --- Say more [~eclark]. Sort in the memstore? But the mvcc that is +1B shouldn't be seen because it beyond read point? (Sounds like a 'fun' debug session -- nice one) Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378539#comment-14378539 ] Enis Soztutar commented on HBASE-12751: --- bq. You looking at the rerdering then Elliott Clark? My first POC patch at HBASE-8763 does the reorder. I believe the approach over there is still viable. However, I was not able to do a proper perf analysis of the approach though. Jeff took over the patch to not change the ordering (which is a bigger change definitely). After the disruptor, we had to do the mock-append as a workaround. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378517#comment-14378517
]
Enis Soztutar commented on HBASE-12751:
---
Makes sense. From my notes from 1 year ago:
{quote}
Different strategies for transactions depending on the type:
Read-only(get, scan)
Write-only(put)
Read-Write (increment, append, row processor)
Current transaction models
Current model for write only transactions
1. lock the rows
2. update latest timestamps
3. get mvcc number
3.1. sync writeQueue
3.2. increment write number
3.3. add trx to the writeQueue
4. Insert to memstore (with mvcc number)
5. AppendNoSync() to WAL
5.1. obtain seq_num
5.2. append entry to pendingWrites (our own linked list buffer)
6. Release row locks
7. sync WAL
7.1. hlogflush (append pending writes to hdfs buffer)
7.2. increase sync point
8. Complete memstore insert
8.1. sync writeQueue
8.2. mark write completed.
8.3. advanceMvcc number as much as possible (from the writeQueue). Notify all
waiters
8.4. wait for mvcc read number to advance past our mvcc write number
Current model for read only transactions
1. Obtain mvcc read number
2. Perform scans with that mvcc number
Current model for read-write transactions (increment)
1. lock the row
2. get an mvcc number, wait for all transaction to finish before this number
3. get mvcc number
4. Do the reads for the transaction, compute writes for the transaction. Always
use current time
5. Insert to a temporary memstore
6. AppendNoSync() to WAL
7. Insert to memstore
8. Release row locks
9. sync WAL
10. Complete memstore insert
Current model for read-write transactions (processRowsWithLocks)
1. lock the rows
2. Do the reads for the transaction, compute writes for the transaction
(RowProcessor)
3. get mvcc write number
4. Insert to memstore
5. AppendNoSync() to WAL
6. Release row locks
7. sync WAL
8. Complete memstore insert
Note, there is no waiting for all trx to finish after acquiring row locks (bug?)
Observations:
- Write-only trxs do not need the row lock. Pure writes can be serialized
with mvcc or seq_num’s.
- Read-only trxs should be kept as is for performance (use mvcc)
- Read-write trxs can conflict with other read-write trx / or write
trasactions
{quote}
We do not need row locks for conflict resolution for write-only transactions,
but we still need locking for conflicts between read-write and read-write/write
only transactions. Doing shared locks should help with this. We are optimizing
for write-only transactions at the expense of read-write transactions (which
today have to wait for ALL transactions that started before its start time to
finish).
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378258#comment-14378258 ] Elliott Clark commented on HBASE-12751: --- Yeah I'll look at the re-ordering now that [~nkedel] has done the hard work of creating the initial patch. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376453#comment-14376453 ] Nate Edel commented on HBASE-12751: --- org.apache.hadoop.hbase.TestAcidGuarantees appears to have non-deterministic freezing behaviors when running locally. Could this be related to HBASE-13295 ? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376459#comment-14376459 ] Elliott Clark commented on HBASE-12751: --- Doubtful. That one has been pretty stable. How is it on master without your patch? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375472#comment-14375472
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706449/HBASE-12751-v1.patch
against master branch at commit 845f5de121e92a99b41b30dc86cb3f2898e0254f.
ATTACHMENT ID: 12706449
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestAcidGuarantees.testMixedAtomicity(TestAcidGuarantees.java:362)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13363//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Nate Edel
Attachments: HBASE-12751-v1.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377157#comment-14377157 ] Elliott Clark commented on HBASE-12751: --- So [~nkedel] and I have been debugging this today. The test that's failing is a real test failure. However it's do to the late binding sequence id work that went in a while ago. The issue is that if we allow two WALEdits for the same row to be outstanding then it's possible that the pre-created sequence number (1M + a incrementing long) will not sort in the same way as the sequence id inside of the waledit. This late binding is currently pretty intwined with the FSHLog. [~stack] or [~busbey] or [~jeffreyz] any thoughts on how to fix this cleanly? My only thought was to move memstore insertion after hlog.append. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377226#comment-14377226 ] Lars Hofhansl commented on HBASE-12751: --- Is this from the disruptor code? Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751-v1.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368305#comment-14368305
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705479/HBASE-12751-v1.patch
against master branch at commit f9a17edc252a88c5a1a2c7764e3f9f65623e0ced.
ATTACHMENT ID: 12705479
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerIsAlive(TestHttpServerLifecycle.java:71)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13304//console
This message is automatically generated.
Allow RowLock to be reader writer
-
Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Nate Edel
Attachments: HBASE-12751-v1.patch, HBASE-12751.patch
Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368062#comment-14368062 ] Lars Hofhansl commented on HBASE-12751: --- The patch does not actually use readlock anywhere, right? (maybe you forgot to add true to the call to getRowLockInternal) Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Nate Edel Attachments: HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
