[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323918#comment-15323918
]
Hudson commented on HBASE-15946:
FAILURE: Integrated in HBase-1.4 #209 (See
[https://builds.apache.org/job/HBase-1.4/209/])
HBASE-15946 Eliminate possible security concerns in RS web UI's store (antonov:
rev 37807b38a380ec32950aec934b4b8328fdf1cdbf)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
* hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15991) CompactingMemstore#InMemoryFlushRunnable should implement Comparable/Comparator
[
https://issues.apache.org/jira/browse/HBASE-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323891#comment-15323891
]
ramkrishna.s.vasudevan commented on HBASE-15991:
Yes. Configure a table with a CF that has INMEMORY flush. Memstore allocated is
70% and pure write scenario.
Heap size tested wtih both 32/40G.
> CompactingMemstore#InMemoryFlushRunnable should implement
> Comparable/Comparator
> ---
>
> Key: HBASE-15991
> URL: https://issues.apache.org/jira/browse/HBASE-15991
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15991.patch
>
>
> Configuring CompactingMemstore for a table fails due to the following error
> {code}
> 2016-06-08 23:27:03,761 ERROR [B.defaultRpcServer.handler...
> 2016-06-08 23:27:03,761 ERROR
> [B.defaultRpcServer.handler=38,queue=8,port=16041] ipc.RpcServer: Unexpected
> throwable object
> java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable
> cannot be cast to java.lang.Comparable
> at
> java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:357)
> at
> java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:489)
> at
> org.apache.hadoop.hbase.util.StealJobQueue$1.offer(StealJobQueue.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1361)
> at
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.checkActiveSize(CompactingMemStore.java:258)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd(AbstractMemStore.java:403)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:113)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:630)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemstore(HRegion.java:3769)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3740)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:3222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2954)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:868)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:830)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2307)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34826)
>
> {code}
> It is a straight forward fix. But If we implement the Comparable the
> compareTo() should be based on what attribute? Should be based on the time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15991) CompactingMemstore#InMemoryFlushRunnable should implement Comparable/Comparator
[
https://issues.apache.org/jira/browse/HBASE-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15991:
---
Issue Type: Sub-task (was: Bug)
Parent: HBASE-16003
> CompactingMemstore#InMemoryFlushRunnable should implement
> Comparable/Comparator
> ---
>
> Key: HBASE-15991
> URL: https://issues.apache.org/jira/browse/HBASE-15991
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15991.patch
>
>
> Configuring CompactingMemstore for a table fails due to the following error
> {code}
> 2016-06-08 23:27:03,761 ERROR [B.defaultRpcServer.handler...
> 2016-06-08 23:27:03,761 ERROR
> [B.defaultRpcServer.handler=38,queue=8,port=16041] ipc.RpcServer: Unexpected
> throwable object
> java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable
> cannot be cast to java.lang.Comparable
> at
> java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:357)
> at
> java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:489)
> at
> org.apache.hadoop.hbase.util.StealJobQueue$1.offer(StealJobQueue.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1361)
> at
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.checkActiveSize(CompactingMemStore.java:258)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd(AbstractMemStore.java:403)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:113)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:630)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemstore(HRegion.java:3769)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3740)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:3222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2954)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:868)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:830)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2307)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34826)
>
> {code}
> It is a straight forward fix. But If we implement the Comparable the
> compareTo() should be based on what attribute? Should be based on the time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15991) CompactingMemstore#InMemoryFlushRunnable should implement Comparable/Comparator
[
https://issues.apache.org/jira/browse/HBASE-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323885#comment-15323885
]
Anoop Sam John commented on HBASE-15991:
When use the CompactingMemstore and an in memory flush happens, this is coming
in right Ram?
> CompactingMemstore#InMemoryFlushRunnable should implement
> Comparable/Comparator
> ---
>
> Key: HBASE-15991
> URL: https://issues.apache.org/jira/browse/HBASE-15991
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15991.patch
>
>
> Configuring CompactingMemstore for a table fails due to the following error
> {code}
> 2016-06-08 23:27:03,761 ERROR [B.defaultRpcServer.handler...
> 2016-06-08 23:27:03,761 ERROR
> [B.defaultRpcServer.handler=38,queue=8,port=16041] ipc.RpcServer: Unexpected
> throwable object
> java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable
> cannot be cast to java.lang.Comparable
> at
> java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:357)
> at
> java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:489)
> at
> org.apache.hadoop.hbase.util.StealJobQueue$1.offer(StealJobQueue.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1361)
> at
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.checkActiveSize(CompactingMemStore.java:258)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd(AbstractMemStore.java:403)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:113)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:630)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemstore(HRegion.java:3769)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3740)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:3222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2954)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:868)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:830)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2307)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34826)
>
> {code}
> It is a straight forward fix. But If we implement the Comparable the
> compareTo() should be based on what attribute? Should be based on the time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15999) Do not append the current thread name for the threads in the INMEMORY_COMPACTION_POOL
[
https://issues.apache.org/jira/browse/HBASE-15999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15999:
---
Issue Type: Sub-task (was: Bug)
Parent: HBASE-16003
> Do not append the current thread name for the threads in the
> INMEMORY_COMPACTION_POOL
> -
>
> Key: HBASE-15999
> URL: https://issues.apache.org/jira/browse/HBASE-15999
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15999.patch
>
>
> In the INMEMORY_COMPACTION_POOL ThreadPoolExecutor for every thread the
> current thread name is also appended. Since we are using a pool the name gets
> appended and we end up in names like this
> {code}
> "B.defaultRpcServer.handler=89,queue=9,port=16041-inmemoryCompactions-1465492533442-inmemoryCompactions-1465492548754-inmemoryCompactions-1465492548913-inmemoryCompactions-1465492549625-inmemoryCompactions-1465492549956-inmemoryCompactions-1465492567040-inmemoryCompactions-1465492567160-inmemoryCompactions-1465492578465-inmemoryCompactions-1465492578707-inmemoryCompactions-1465492579292-inmemoryCompactions-1465492579357-inmemoryCompactions-1465492579786-inmemoryCompactions-1465492580059-inmemoryCompactions-1465492589975-inmemoryCompactions-1465492590192-inmemoryCompactions-1465492590484-inmemoryCompactions-1465492591144-inmemoryCompactions-1465492592603-inmemoryCompactions-1465492592799-inmemoryCompactions-1465492597106-inmemoryCompactions-1465492602925-inmemoryCompactions-1465492606620-inmemoryCompactions-1465492651478-inmemoryCompactions-1465492653460-inmemoryCompactions-1465492677020-inmemoryCompactions-1465492680857-inmemoryCompactions-1465492681989-inmemoryCompactions-1465492721818-inmemoryCompactions-1465492723562-inmemoryCompactions-1465492724801-inmemoryCompactions-1465492726665-inmemoryCompactions-1465492745750-inmemoryCompactions-1465492745964-inmemoryCompactions-1465492746578-inmemoryCompactions-1465492756867-inmemoryCompactions-1465492764727-inmemoryCompactions-1465492766944-inmemoryCompactions-1465492767098-inmemoryCompactions-1465492785298-inmemoryCompactions-1465492788334-inmemoryCompactions-1465492795954-inmemoryCompactions-1465493047265-inmemoryCompactions-1465493091530-inmemoryCompactions-1465493185684"
> #6006 daemon prio=5 os_prio=0 tid=0x0daa6800 nid=0x454a runnable
> [0x7f50fd0b9000]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16003) Compacting memstore related fixes
ramkrishna.s.vasudevan created HBASE-16003: -- Summary: Compacting memstore related fixes Key: HBASE-16003 URL: https://issues.apache.org/jira/browse/HBASE-16003 Project: HBase Issue Type: Task Affects Versions: 2.0.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Will group all the compacting memstore related sub jiras under this parent task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15991) CompactingMemstore#InMemoryFlushRunnable should implement Comparable/Comparator
[
https://issues.apache.org/jira/browse/HBASE-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323876#comment-15323876
]
ramkrishna.s.vasudevan commented on HBASE-15991:
bq.can you give some background on this issue:
We are trying to load batchMutates using the PerformanceEvaluation tool. Since
we have lot of parallel mutations we tend to get log of Inmemory flushes. Since
the InMemoryflushRunnable gets added to a queue and that is Priority based we
are not able to do the queue based operations for retrieval as
InMemoryflushRunnable is not Comparable.
Just changing to FIFO based queue will not have this problem at all and yes it
solves the problem. But do let us know what you think of this and without
fixing this we are not able to perform any InMemoryFlushes.
> CompactingMemstore#InMemoryFlushRunnable should implement
> Comparable/Comparator
> ---
>
> Key: HBASE-15991
> URL: https://issues.apache.org/jira/browse/HBASE-15991
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15991.patch
>
>
> Configuring CompactingMemstore for a table fails due to the following error
> {code}
> 2016-06-08 23:27:03,761 ERROR [B.defaultRpcServer.handler...
> 2016-06-08 23:27:03,761 ERROR
> [B.defaultRpcServer.handler=38,queue=8,port=16041] ipc.RpcServer: Unexpected
> throwable object
> java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable
> cannot be cast to java.lang.Comparable
> at
> java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:357)
> at
> java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:489)
> at
> org.apache.hadoop.hbase.util.StealJobQueue$1.offer(StealJobQueue.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1361)
> at
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.checkActiveSize(CompactingMemStore.java:258)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd(AbstractMemStore.java:403)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:113)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:630)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemstore(HRegion.java:3769)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3740)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:3222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2954)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:868)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:830)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2307)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34826)
>
> {code}
> It is a straight forward fix. But If we implement the Comparable the
> compareTo() should be based on what attribute? Should be based on the time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323841#comment-15323841
]
Hudson commented on HBASE-15946:
FAILURE: Integrated in HBase-Trunk_matrix #1020 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1020/])
HBASE-15946. Eliminate possible security concerns in Store File metrics.
(antonov: rev 6da6babe4faa7b2b16775d3cd5c861e71ef4cf31)
* hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323840#comment-15323840 ] Hudson commented on HBASE-15958: FAILURE: Integrated in HBase-Trunk_matrix #1020 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1020/]) HBASE-15958 Implement ClaimQueues on top of HBase (eclark: rev babdedc1b0f0159eb526fb5c9ee08525de7ce404) * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesHBaseImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesArguments.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateHBaseImpl.java > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323842#comment-15323842
]
Hudson commented on HBASE-15925:
FAILURE: Integrated in HBase-Trunk_matrix #1020 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1020/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev 108d39a7277bdcbc8b4bc9c50bab457a86a71971)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323815#comment-15323815
]
Mikhail Antonov commented on HBASE-15946:
-
Sorry, somehow I overlooked that.
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov reassigned HBASE-15946:
---
Assignee: Sean Mackrory (was: Mikhail Antonov)
Yeah, totally. I was sure it was assigned to him already..
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323776#comment-15323776
]
Hudson commented on HBASE-15946:
SUCCESS: Integrated in HBase-1.3-IT #700 (See
[https://builds.apache.org/job/HBase-1.3-IT/700/])
HBASE-15946 Eliminate possible security concerns in RS web UI's store (antonov:
rev d8d63d67152af8eed48f8863a0e13d3e71fc097c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
* hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323762#comment-15323762
]
Hudson commented on HBASE-15925:
SUCCESS: Integrated in HBase-1.4 #208 (See
[https://builds.apache.org/job/HBase-1.4/208/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev e6a46bbb8dc12e85381803325cad3c25b21b6ba9)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323759#comment-15323759
]
Hudson commented on HBASE-15925:
FAILURE: Integrated in HBase-1.2 #646 (See
[https://builds.apache.org/job/HBase-1.2/646/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev f245fe106e80d3eb08c172903cbc70900a5b8a51)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323751#comment-15323751
]
Ted Yu commented on HBASE-15946:
This should be assigned to Sean Mackrory, right ?
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323743#comment-15323743
]
Hadoop QA commented on HBASE-15946:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-15946 does not apply to branch-1.3. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809342/HBASE-15946-branch-1.3-mantonov.diff
|
| JIRA Issue | HBASE-15946 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2170/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323744#comment-15323744
]
Hudson commented on HBASE-15925:
FAILURE: Integrated in HBase-1.3 #731 (See
[https://builds.apache.org/job/HBase-1.3/731/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev c9d4ee8b4f39b01fe38c5d44e9338645432f200d)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323740#comment-15323740
]
Mikhail Antonov commented on HBASE-15946:
-
Pushed to master, branch-1 and branch-1.3. Thanks [~mackrorysd]!
Attached the diff for branch-1 and 1.3 as it didn't cherry-pick cleanly, for
reference.
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15946:
Attachment: HBASE-15946-branch-1.3-mantonov.diff
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-branch-1.3-mantonov.diff,
> HBASE-15946-v1.patch, HBASE-15946-v2.patch, HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15905) Makefile build env incorrectly links in tests
[ https://issues.apache.org/jira/browse/HBASE-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyadharshini karthikeyan updated HBASE-15905: --- Attachment: HBASE-15905.HBASE-14850.v1.patch This patch has the solution for make fail on a clean checkout. @Elliot - It will be helpful to resolve the issue if you can give a detail on other two points. Thank you > Makefile build env incorrectly links in tests > - > > Key: HBASE-15905 > URL: https://issues.apache.org/jira/browse/HBASE-15905 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Priyadharshini karthikeyan > Attachments: HBASE-15905.HBASE-14850.v1.patch > > > Right now the makefile build system doesn't seem to do so well. > * Tests are included in the lib > * Documentation includes the protobuf dir > * just running make on a clean check out fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15905) Makefile build env incorrectly links in tests
[ https://issues.apache.org/jira/browse/HBASE-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyadharshini karthikeyan reassigned HBASE-15905: -- Assignee: Priyadharshini karthikeyan > Makefile build env incorrectly links in tests > - > > Key: HBASE-15905 > URL: https://issues.apache.org/jira/browse/HBASE-15905 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Priyadharshini karthikeyan > > Right now the makefile build system doesn't seem to do so well. > * Tests are included in the lib > * Documentation includes the protobuf dir > * just running make on a clean check out fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15971: -- Attachment: handlers.fp.png Handlers are erratic still but the peaks are higher. When the q backs up, the peaks fall as though as though we are 'off' the fast path. Seems sensitive to amount of Readers. If 6 as is in this case, then we go fast. If 12, we slow down. Seems like the Readers also act as a bit of bottleneck which means the Handlers get to go the fast path more often. If I drive even more load, ops go up to 320k. 15% idle so there is more to be had here. Work still to do. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, HBASE-15971.branch-1.001.patch, > branch-1.hits.png, branch-1.png, handlers.fp.png, hits.fp.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[
https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323679#comment-15323679
]
Hadoop QA commented on HBASE-15974:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 8s {color}
| {color:red} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 4 new + 5
unchanged - 0 fixed = 9 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 0s {color}
| {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 4 new + 5
unchanged - 0 fixed = 9 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 28s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 20s
{color} | {color:red} hbase-client generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 4m 0s
{color} | {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 1
new + 14 unchanged - 0 fixed = 15 total (was 14) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 55s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color}
[jira] [Updated] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15971: -- Attachment: hits.fp.png Here is the hits I was getting w/ this patch... This is 6 Readers and 48 Handlers (I have 48 cpus). The extreme right is when I had 48 Readers and 48 Handlers. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, HBASE-15971.branch-1.001.patch, > branch-1.hits.png, branch-1.png, hits.fp.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15594) [YCSB] Improvements
[
https://issues.apache.org/jira/browse/HBASE-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15594:
--
Attachment: (was: HBASE-15594.branch-1.001.patch)
> [YCSB] Improvements
> ---
>
> Key: HBASE-15594
> URL: https://issues.apache.org/jira/browse/HBASE-15594
> Project: HBase
> Issue Type: Umbrella
>Reporter: stack
>Priority: Critical
> Attachments: fast.patch, fifo.hits.png, fifo.readers.png
>
>
> Running YCSB and getting good results is an arcane art. For example, in my
> testing, a few handlers (100) with as many readers as I had CPUs (48), and
> upping connections on clients to same as #cpus made for 2-3x the throughput.
> The above config changes came of lore; which configurations need tweaking is
> not obvious going by their names, there were no indications from the app on
> where/why we were blocked or on which metrics are important to consider. Nor
> was any of this stuff written down in docs.
> Even still, I am stuck trying to make use of all of the machine. I am unable
> to overrun a server though 8 client nodes trying to beat up a single node
> (workloadc, all random-read, with no data returned -p readallfields=false).
> There is also a strange phenomenon where if I add a few machines, rather than
> 3x the YCSB throughput when 3 nodes in cluster, each machine instead is doing
> about 1/3rd.
> This umbrella issue is to host items that improve our defaults and noting how
> to get good numbers running YCSB. In particular, I want to be able to
> saturate a machine.
> Here are the configs I'm currently working with. I've not done the work to
> figure client-side if they are optimal (weird is how big a difference
> client-side changes can make -- need to fix this). On my 48 cpu machine, I
> can do about 370k random reads a second from data totally cached in
> bucketcache. If I short-circuit the user gets so they don't do any work but
> return immediately, I can do 600k ops a second but the CPUs are at 60-70%
> only. I cannot get them to go above this. Working on it.
> {code}
>
>
> hbase.ipc.server.read.threadpool.size
>
> 48
>
>
>
> hbase.regionserver.handler.count
>
> 100
>
>
>
> hbase.client.ipc.pool.size
>
> 100
>
>
>
> hbase.htable.threads.max
>
> 48
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323661#comment-15323661 ] stack commented on HBASE-15971: --- Here is a hack copying a trick from Kudu that [~tlipcon] pointed me at. Needs work (hack is in the 'wrong' place). Gets us almost 2x. Will work on a better version... Now to go find why big diff between 0.98 and 1.0. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, HBASE-15971.branch-1.001.patch, > branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15971: -- Attachment: HBASE-15971.branch-1.001.patch > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, HBASE-15971.branch-1.001.patch, > branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15594) [YCSB] Improvements
[
https://issues.apache.org/jira/browse/HBASE-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15594:
--
Attachment: HBASE-15594.branch-1.001.patch
> [YCSB] Improvements
> ---
>
> Key: HBASE-15594
> URL: https://issues.apache.org/jira/browse/HBASE-15594
> Project: HBase
> Issue Type: Umbrella
>Reporter: stack
>Priority: Critical
> Attachments: HBASE-15594.branch-1.001.patch, fast.patch,
> fifo.hits.png, fifo.readers.png
>
>
> Running YCSB and getting good results is an arcane art. For example, in my
> testing, a few handlers (100) with as many readers as I had CPUs (48), and
> upping connections on clients to same as #cpus made for 2-3x the throughput.
> The above config changes came of lore; which configurations need tweaking is
> not obvious going by their names, there were no indications from the app on
> where/why we were blocked or on which metrics are important to consider. Nor
> was any of this stuff written down in docs.
> Even still, I am stuck trying to make use of all of the machine. I am unable
> to overrun a server though 8 client nodes trying to beat up a single node
> (workloadc, all random-read, with no data returned -p readallfields=false).
> There is also a strange phenomenon where if I add a few machines, rather than
> 3x the YCSB throughput when 3 nodes in cluster, each machine instead is doing
> about 1/3rd.
> This umbrella issue is to host items that improve our defaults and noting how
> to get good numbers running YCSB. In particular, I want to be able to
> saturate a machine.
> Here are the configs I'm currently working with. I've not done the work to
> figure client-side if they are optimal (weird is how big a difference
> client-side changes can make -- need to fix this). On my 48 cpu machine, I
> can do about 370k random reads a second from data totally cached in
> bucketcache. If I short-circuit the user gets so they don't do any work but
> return immediately, I can do 600k ops a second but the CPUs are at 60-70%
> only. I cannot get them to go above this. Working on it.
> {code}
>
>
> hbase.ipc.server.read.threadpool.size
>
> 48
>
>
>
> hbase.regionserver.handler.count
>
> 100
>
>
>
> hbase.client.ipc.pool.size
>
> 100
>
>
>
> hbase.htable.threads.max
>
> 48
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323630#comment-15323630
]
Hudson commented on HBASE-15925:
SUCCESS: Integrated in HBase-1.1-JDK7 #1730 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1730/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev 737b1d6def1084c731a64ce9c1e14c7f1d8cbf8a)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15974: --- Status: Patch Available (was: Open) > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15974: --- Description: Currently ReplicationQueuesClient utilizes a ZooKeeper implementation ReplicationQueuesClientZkImpl that attempts to read from the ZNode where ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase implementation for ReplicationQueuesClient. The review is posted at https://reviews.apache.org/r/48521/ was:Currently ReplicationQueuesClient utilizes a ZooKeeper implementation ReplicationQueuesClientZkImpl that attempts to read from the ZNode where ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase implementation for ReplicationQueuesClient. > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15974: --- Attachment: HBASE-15974.patch > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323617#comment-15323617
]
Hudson commented on HBASE-15925:
SUCCESS: Integrated in HBase-1.2-IT #528 (See
[https://builds.apache.org/job/HBase-1.2-IT/528/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev f245fe106e80d3eb08c172903cbc70900a5b8a51)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323615#comment-15323615
]
Mikhail Antonov commented on HBASE-15946:
-
[~busbey] this fix applies also to 1.2, so I think it should go to branch-1.2?
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch,
> HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323606#comment-15323606
]
Hudson commented on HBASE-15990:
FAILURE: Integrated in HBase-Trunk_matrix #1019 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1019/])
HBASE-15990 The priority value of subsequent coprocessors in the (yuzhihong:
rev 55a04b78102a3a919c6e2e86fcdf98dd1d9a24e4)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorHost.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15950) Fix memstore size estimates to be more tighter
[ https://issues.apache.org/jira/browse/HBASE-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323594#comment-15323594 ] Enis Soztutar commented on HBASE-15950: --- bq. So how the object size estimates looks like now? (For a KV added into CSLM) In the test that I was doing, the JFR profiler shows 2.8G as used heap for memstore objects, and we are estimating 2.9 (previously 4.5). Pretty close. bq. Use UnsafeAvailChecker#isAvailable() instead. Done. bq. These changes are also based on JOL Yes. That particular change is because, we were not counting the object header size from OBJECT, but by REFERENCE*2, and we were not aligning the array objects. bq. This is good to do, but I think the warning in the release note likely needs to be louder and that it should also go in upgrade notes for 2.0. Let me update the release notes with your suggestion. bq. Should we consider pairing this change with a reduction in the default memstore usage percent? I think it is fine to leave it as it is. > Fix memstore size estimates to be more tighter > -- > > Key: HBASE-15950 > URL: https://issues.apache.org/jira/browse/HBASE-15950 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: Screen Shot 2016-06-02 at 8.48.27 PM.png, > hbase-15950-v0.patch > > > While testing something else, I was loading a region with a lot of data. > Writing 30M cells in 1M rows, with 1 byte values. > The memstore size turned out to be estimated as 4.5GB, while with the JFR > profiling I can see that we are using 2.8GB for all the objects in the > memstore (KV + KV byte[] + CSLM.Node + CSLM.Index). > This obviously means that there is room in the write cache that we are not > effectively using. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323591#comment-15323591 ] Vladimir Rodionov commented on HBASE-15862: --- To clarify further: Until we get physical layout change support, both #1 and #2 are bad in case if table/column family descriptors are changed. Restore will fail in both cases. #3 requires some substantial work and I do not want to block this JIRA until we get HBASE-15448(9) resolved. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323573#comment-15323573 ] Vladimir Rodionov commented on HBASE-15862: --- Schema evolution and its support is on the roadmap - HBASE-15449. We have 3 options: # Restore to a current schema (truncate approach) # Restore to a schema of a last full backup (delete approach) # Restore to a schema of a backup being restored (PIT schema- HBASE-15448). No. 3 is right one, but its in Phase 3. For now we can use either #1 or #2. I prefer #1, because, in most cases, the most recent schema (region splits) will be much closer to our PIT, than schema (region splits) of a last full backup. I vote for #1 (current patch), just we have to clarify that until we get HBASE-15448(9) *we do not support physical layout change of a table*. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323558#comment-15323558
]
Hudson commented on HBASE-15925:
SUCCESS: Integrated in HBase-1.3-IT #699 (See
[https://builds.apache.org/job/HBase-1.3-IT/699/])
HBASE-15925 provide default values for hadoop compat module related (busbey:
rev c9d4ee8b4f39b01fe38c5d44e9338645432f200d)
* pom.xml
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15584) Revisit handling of BackupState#CANCELLED
[
https://issues.apache.org/jira/browse/HBASE-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323520#comment-15323520
]
Ted Yu commented on HBASE-15584:
[~mbertozzi]:
What do you think of Romil's comment above ?
> Revisit handling of BackupState#CANCELLED
> -
>
> Key: HBASE-15584
> URL: https://issues.apache.org/jira/browse/HBASE-15584
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Minor
>
> During review of HBASE-15411, Enis made the following point:
> {code}
> nobody puts the backup in cancelled state. setCancelled() is not used. So if
> I abort a backup, who writes to the system table the new state?
> Not sure whether this is a phase 1 patch issue or due to this patch. We can
> open a new jira and address it there if you do not want to do it in this
> patch.
> Also maybe this should be named ABORTED rather than CANCELLED.
> {code}
> This issue is to decide whether this state should be kept (e.g. through
> notification from procedure V2 framework in response to abortion).
> If it is to be kept, the state should be renamed ABORTED.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15860) Improvements for HBASE-14280 - Fixing Bulkload for HDFS HA Clusters
[
https://issues.apache.org/jira/browse/HBASE-15860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323518#comment-15323518
]
Esteban Gutierrez commented on HBASE-15860:
---
Thanks for the patch [~wchevreuil]! since
DFSUtil.getRpcAddressesForNameserviceId has been around for a while it should
be ok just to call directly instead of using a class loader.
> Improvements for HBASE-14280 - Fixing Bulkload for HDFS HA Clusters
> ---
>
> Key: HBASE-15860
> URL: https://issues.apache.org/jira/browse/HBASE-15860
> Project: HBase
> Issue Type: Improvement
> Components: util
>Affects Versions: 1.0.0
>Reporter: Wellington Chevreuil
>Priority: Minor
> Attachments: HBASE-15860.patch
>
>
> HBASE-14280 introduced fix for bulkload failures when referring a remote
> cluster name service id if "bulkloading" from a HA cluster.
> HBASE-14280 solution on *FSHDFSUtils.getNNAddresses* was to invoke
> *DFSUtil.getNNServiceRpcAddressesForCluster* instead of
> *DFSUtil.getNNServiceRpcAddresses*. This works for hadoop 2.6 and above.
> Proposed change here is to use "*DFSUtil.getRpcAddressesForNameserviceId*"
> instead, which already returns only addresses for specific nameservice
> informed. This is available since hadoop 2.4.
> Sample proposal on FSHDFSUtils.getNNAddresses:
> ...
> {noformat}
> String nameServiceId = serviceName.split(":")[1];
> if (dfsUtilClazz == null) {
> dfsUtilClazz = Class.forName("org.apache.hadoop.hdfs.DFSUtil");
> }
> if (getNNAddressesMethod == null) {
> getNNAddressesMethod =
> dfsUtilClazz.getMethod("getRpcAddressesForNameserviceId",
> Configuration.class,
> String.class, String.class);
> }
> Map nnMap =
> (Map) getNNAddressesMethod
> .invoke(null, conf, nameServiceId, null);
> for (Map.Entry e2 : nnMap.entrySet()) {
> InetSocketAddress addr = e2.getValue();
> addresses.add(addr);
> }
> ...
> {noformat}
> Will also add test conditions for *FSHDFSUtils.isSameHdfs* to verify scenario
> when multiple name service ids are defined.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15499) Add multiple data type support for increment
[ https://issues.apache.org/jira/browse/HBASE-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323497#comment-15323497 ] stack commented on HBASE-15499: --- Sounds good to me. Which branches to commit to [~heliangliang]? > Add multiple data type support for increment > > > Key: HBASE-15499 > URL: https://issues.apache.org/jira/browse/HBASE-15499 > Project: HBase > Issue Type: New Feature > Components: API >Reporter: He Liangliang >Assignee: He Liangliang > Attachments: HBASE-15499-V2.diff, HBASE-15499-V3.diff, > HBASE-15499-V4.diff, HBASE-15499-V5.diff, HBASE-15499-V5.patch, > HBASE-15499.diff > > > Currently the increment assumes long with byte-wise serialization. It's > useful to support flexible data type/serializations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323489#comment-15323489
]
Hudson commented on HBASE-15990:
FAILURE: Integrated in HBase-1.4 #207 (See
[https://builds.apache.org/job/HBase-1.4/207/])
HBASE-15990 The priority value of subsequent coprocessors in the (yuzhihong:
rev a4e2b3856ea2db7f221330a6f6493f18ebfbc14f)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorHost.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323472#comment-15323472 ] Jerry He commented on HBASE-15862: -- We can call alterTable with the past PIT table descriptor after truncate? IIRC. Incremental restore does that to handle possible schema change that happened between the backups. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15925:
Resolution: Fixed
Release Note: Downstream users of HBase dependencies that do not properly
activate Maven profiles should now see a correct transitive dependency on the
default hadoop-compatibility-module.
Status: Resolved (was: Patch Available)
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15958) Implement ClaimQueues on top of HBase
[
https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323452#comment-15323452
]
Hadoop QA commented on HBASE-15958:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 24s
{color} | {color:green} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 0 new +
5 unchanged - 2 fixed = 5 total (was 7) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 23s
{color} | {color:green} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 0
new + 5 unchanged - 2 fixed = 5 total (was 7) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 0s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323451#comment-15323451 ] Jerry He commented on HBASE-15862: -- An issue with the 'truncate' is that it preserves the schema of the table. It is possible the current table schema is different (e.g. with a new column family or a column family has been dropped). The restore (bulk load) can fail due to mismatch column families. (This can be fixed with some kind of 'ignore'). But again the table is not in a PIT state in the past. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15584) Revisit handling of BackupState#CANCELLED
[
https://issues.apache.org/jira/browse/HBASE-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323450#comment-15323450
]
Romil Choksi commented on HBASE-15584:
--
[~tedyu]
I tried the list_procedures command while I had a couple of full backups in
progress.
{code}
hbase(main):008:0> list_procedures
Id Name State Start_Time Last_Update
1 CreateTableProcedure (table=hbase:backup) ROLLEDBACK Wed Jun 08 23:42:00 UTC
2016 Wed Jun 08 23:42:00 UTC 2016
36 FullTableBackupProcedure
(targetRootDir=hdfs://hbase-test-secure-rc-7:8020/user/hbase) RUNNABLE Thu Jun
09 21:20:44 UTC 2016 Thu Jun 09 21:20:46 UTC 2016
37 FullTableBackupProcedure
(targetRootDir=hdfs://hbase-test-secure-rc-7:8020/user/hbase) RUNNABLE Thu Jun
09 21:20:46 UTC 2016 Thu Jun 09 21:20:46 UTC 2016
3 row(s) in 0.0240 seconds
{code}
Now just looking at the list_procedures output, I am not sure which procId
belongs to which backup process. I can probably take a look at the hbase backup
history, and compare the timestamps. Or go through the logs to get the
necessary details. In such cases, I think it will be better for the user to
have cancel option that takes in backupId and doesn't have to deal with
figuring out the procId.
> Revisit handling of BackupState#CANCELLED
> -
>
> Key: HBASE-15584
> URL: https://issues.apache.org/jira/browse/HBASE-15584
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Minor
>
> During review of HBASE-15411, Enis made the following point:
> {code}
> nobody puts the backup in cancelled state. setCancelled() is not used. So if
> I abort a backup, who writes to the system table the new state?
> Not sure whether this is a phase 1 patch issue or due to this patch. We can
> open a new jira and address it there if you do not want to do it in this
> patch.
> Also maybe this should be named ABORTED rather than CANCELLED.
> {code}
> This issue is to decide whether this state should be kept (e.g. through
> notification from procedure V2 framework in response to abortion).
> If it is to be kept, the state should be renamed ABORTED.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323410#comment-15323410 ] Gary Helmling commented on HBASE-15921: --- Yeah, I understand and agree with the motivation there. We definitely want to be sure that all existing tests pass with AsyncTableImpl. My point is just that changing the default implementation returned by ConnectionImplementation.getTable() shouldn't be done yet. A configuration option changing the implementation used seems like the way to go. After AsyncTableImpl has seen some real world usage we can look at dropping HTable. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323370#comment-15323370
]
Hadoop QA commented on HBASE-15925:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 34s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 23s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 24s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 19s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 28s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 23s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 102m 47s
{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 156m 33s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809257/HBASE-15925.1.patch |
| JIRA Issue | HBASE-15925 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 55a04b7 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2164/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2164/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323369#comment-15323369 ] Jurriaan Mous commented on HBASE-15921: --- The primary reason I made the new TableImpl because I wanted to run all existing Tests on table to guarantee AyncTable is working correctly with all exotic conditions being tested. It would be too much work to duplicate all table test code and convert it to async. I wanted to also see how TableImpl behaves in performance tests before deciding definite strategy. That's why HTable is not yet removed in current latest patch. We could indeed introduce a way to keep both in with a configuration and also test both while TableImpl stabilizes. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15335) Add composite key support in row key
[
https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323365#comment-15323365
]
Hadoop QA commented on HBASE-15335:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15335 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809289/HBASE-15335-5.patch |
| JIRA Issue | HBASE-15335 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2168/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Add composite key support in row key
>
>
> Key: HBASE-15335
> URL: https://issues.apache.org/jira/browse/HBASE-15335
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zhan Zhang
>Assignee: Zhan Zhang
> Attachments: HBASE-15335-1.patch, HBASE-15335-2.patch,
> HBASE-15335-3.patch, HBASE-15335-4.patch, HBASE-15335-5.patch
>
>
> Add composite key filter support in the connector.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16002) Many DataType constructors are not public
Nick Dimiduk created HBASE-16002:
Summary: Many DataType constructors are not public
Key: HBASE-16002
URL: https://issues.apache.org/jira/browse/HBASE-16002
Project: HBase
Issue Type: Bug
Reporter: Nick Dimiduk
Using the {{Struct}} class to build a rowkey, I noticed than many of the
provided {{DataType}} implementations' constructors are protected. They should
be public in most (all?) cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323360#comment-15323360
]
Hadoop QA commented on HBASE-15946:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 51s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m 48s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 129m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808703/HBASE-15946-v3.patch |
| JIRA Issue | HBASE-15946 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 55a04b7 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs
[jira] [Commented] (HBASE-5291) Add Kerberos HTTP SPNEGO authentication support to HBase web consoles
[
https://issues.apache.org/jira/browse/HBASE-5291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323359#comment-15323359
]
Josh Elser commented on HBASE-5291:
---
Not sure why HadoopQA, reran .003, but whatever. The 2nd to most recent run was
fine -- the last one looks like there were host-level issues on the machine.
[~busbey], [~devaraj], [~apurtell], any of you fine gentlemen have a moment to
give the .003 patch a glance for me?
> Add Kerberos HTTP SPNEGO authentication support to HBase web consoles
> -
>
> Key: HBASE-5291
> URL: https://issues.apache.org/jira/browse/HBASE-5291
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver, security
>Reporter: Andrew Purtell
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-5291.001.patch, HBASE-5291.002.patch,
> HBASE-5291.003.patch
>
>
> Like HADOOP-7119, the same motivations:
> {quote}
> Hadoop RPC already supports Kerberos authentication.
> {quote}
> As does the HBase secure RPC engine.
> {quote}
> Kerberos enables single sign-on.
> Popular browsers (Firefox and Internet Explorer) have support for Kerberos
> HTTP SPNEGO.
> Adding support for Kerberos HTTP SPNEGO to [HBase] web consoles would provide
> a unified authentication mechanism and single sign-on for web UI and RPC.
> {quote}
> Also like HADOOP-7119, the same solution:
> A servlet filter is configured in front of all Hadoop web consoles for
> authentication.
> This filter verifies if the incoming request is already authenticated by the
> presence of a signed HTTP cookie. If the cookie is present, its signature is
> valid and its value didn't expire; then the request continues its way to the
> page invoked by the request. If the cookie is not present, it is invalid or
> it expired; then the request is delegated to an authenticator handler. The
> authenticator handler then is responsible for requesting/validating the
> user-agent for the user credentials. This may require one or more additional
> interactions between the authenticator handler and the user-agent (which will
> be multiple HTTP requests). Once the authenticator handler verifies the
> credentials and generates an authentication token, a signed cookie is
> returned to the user-agent for all subsequent invocations.
> The authenticator handler is pluggable and 2 implementations are provided out
> of the box: pseudo/simple and kerberos.
> 1. The pseudo/simple authenticator handler is equivalent to the Hadoop
> pseudo/simple authentication. It trusts the value of the user.name query
> string parameter. The pseudo/simple authenticator handler supports an
> anonymous mode which accepts any request without requiring the user.name
> query string parameter to create the token. This is the default behavior,
> preserving the behavior of the HBase web consoles before this patch.
> 2. The kerberos authenticator handler implements the Kerberos HTTP SPNEGO
> implementation. This authenticator handler will generate a token only if a
> successful Kerberos HTTP SPNEGO interaction is performed between the
> user-agent and the authenticator. Browsers like Firefox and Internet Explorer
> support Kerberos HTTP SPNEGO.
> We can build on the support added to Hadoop via HADOOP-7119. Should just be a
> matter of wiring up the filter to our infoservers in a similar manner.
> And from
> https://issues.apache.org/jira/browse/HBASE-5050?focusedCommentId=13171086=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13171086
> {quote}
> Hadoop 0.23 onwards has a hadoop-auth artifact that provides SPNEGO/Kerberos
> authentication for webapps via a filter. You should consider using it. You
> don't have to move Hbase to 0.23 for that, just consume the hadoop-auth
> artifact, which has no dependencies on the rest of Hadoop 0.23 artifacts.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15978) Netty API leaked into public API
[
https://issues.apache.org/jira/browse/HBASE-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323353#comment-15323353
]
Jurriaan Mous commented on HBASE-15978:
---
I will take a look this weekend.
> Netty API leaked into public API
>
>
> Key: HBASE-15978
> URL: https://issues.apache.org/jira/browse/HBASE-15978
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Priority: Blocker
>
> Noticed out public
> {{[client.Future|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Future.html]}}
> interface extends Netty, which means our public API is bound to a specific
> Netty API and release. IIRC we were minimizing our public-facing surface area
> and asserting ownership over the whole of it so as to control our
> compatibility. Ie, we've done this with Protobuf as well. Not sure if this
> has made it back to other branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323335#comment-15323335 ] Elliott Clark commented on HBASE-15958: --- I am +1 assuming the jenkins run is clean. > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Attachment: HBASE-15335-5.patch > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-15335-1.patch, HBASE-15335-2.patch, > HBASE-15335-3.patch, HBASE-15335-4.patch, HBASE-15335-5.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15335) Add composite key support in row key
[
https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323287#comment-15323287
]
Hadoop QA commented on HBASE-15335:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 1m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 31s
{color} | {color:red} hbase-spark-jdk1.8.0 with JDK v1.8.0 generated 1 new + 18
unchanged - 0 fixed = 19 total (was 18) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 46s
{color} | {color:red} hbase-spark-jdk1.7.0_79 with JDK v1.7.0_79 generated 1
new + 18 unchanged - 0 fixed = 19 total (was 18) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323282#comment-15323282
]
Mikhail Antonov commented on HBASE-15925:
-
totally, +1
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15991) CompactingMemstore#InMemoryFlushRunnable should implement Comparable/Comparator
[
https://issues.apache.org/jira/browse/HBASE-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323275#comment-15323275
]
Eshcar Hillel commented on HBASE-15991:
---
[~anoop.hbase], [~ram_krish], can you give some background on this issue:
Does changing the queue solves the comparable problem?
Do you see this issue also when running other operations or just batch mutate?
We ran quit a lot of experiments with the code in HBASE-14920 (but not batch
operations) and never had this issue
> CompactingMemstore#InMemoryFlushRunnable should implement
> Comparable/Comparator
> ---
>
> Key: HBASE-15991
> URL: https://issues.apache.org/jira/browse/HBASE-15991
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-15991.patch
>
>
> Configuring CompactingMemstore for a table fails due to the following error
> {code}
> 2016-06-08 23:27:03,761 ERROR [B.defaultRpcServer.handler...
> 2016-06-08 23:27:03,761 ERROR
> [B.defaultRpcServer.handler=38,queue=8,port=16041] ipc.RpcServer: Unexpected
> throwable object
> java.lang.ClassCastException:
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable
> cannot be cast to java.lang.Comparable
> at
> java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:357)
> at
> java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:489)
> at
> org.apache.hadoop.hbase.util.StealJobQueue$1.offer(StealJobQueue.java:56)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1361)
> at
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.checkActiveSize(CompactingMemStore.java:258)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd(AbstractMemStore.java:403)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:113)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:630)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemstore(HRegion.java:3769)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3740)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:3222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2954)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2896)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:868)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:830)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2307)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34826)
>
> {code}
> It is a straight forward fix. But If we implement the Comparable the
> compareTo() should be based on what attribute? Should be based on the time?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15978) Netty API leaked into public API
[
https://issues.apache.org/jira/browse/HBASE-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323204#comment-15323204
]
stack commented on HBASE-15978:
---
[~ram_krish] There is more than just netty's Future showing through in this
manner. Was going to consult w/ [~jurmous] on how to address (I could do it in
my club-footed way but I'm pretty sure he'll have a better idea than I). Netty
showing through in our API also came up over in HBASE-15978 in review. Will be
back.
> Netty API leaked into public API
>
>
> Key: HBASE-15978
> URL: https://issues.apache.org/jira/browse/HBASE-15978
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Priority: Blocker
>
> Noticed out public
> {{[client.Future|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Future.html]}}
> interface extends Netty, which means our public API is bound to a specific
> Netty API and release. IIRC we were minimizing our public-facing surface area
> and asserting ownership over the whole of it so as to control our
> compatibility. Ie, we've done this with Protobuf as well. Not sure if this
> has made it back to other branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323201#comment-15323201 ] stack commented on HBASE-15921: --- bq. Why we let out EventExecutor? Especially given it a netty class. Can we contain the fact that the implementation is netty-based? [~jurmous] A version of this has come up over in HBASE-15978 FYI. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323196#comment-15323196 ] Ted Yu commented on HBASE-15862: [~jerryhe]: What do you think of the latest patch ? > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15862: -- Attachment: HBASE-15862-v3.patch v3 with updated javadoc > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch, > HBASE-15862-v3.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15335) Add composite key support in row key
[ https://issues.apache.org/jira/browse/HBASE-15335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15335: --- Attachment: HBASE-15335-4.patch > Add composite key support in row key > > > Key: HBASE-15335 > URL: https://issues.apache.org/jira/browse/HBASE-15335 > Project: HBase > Issue Type: Sub-task >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Attachments: HBASE-15335-1.patch, HBASE-15335-2.patch, > HBASE-15335-3.patch, HBASE-15335-4.patch > > > Add composite key filter support in the connector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323186#comment-15323186
]
Sean Busbey commented on HBASE-15925:
-
you fine with fixing that nit on commit, provided precommit doesn't explode in
an unexpected way?
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323180#comment-15323180
]
Gary Helmling commented on HBASE-15921:
---
{quote}
The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
{quote}
So this patch completely removes access to HTable from ConnectionImplementation?
I haven't had time to look through the patch in detail, yet, but I don't think
it's appropriate to remove the ability to use the current HTable implementation
in this patch. TableImpl needs to wait or be exposed in a different way than
changing ConnectionImplementation.getTable(). AsyncTableImpl and HTable need
to exist side by side for a while, giving a chance for issues to be worked out.
Performance is only one consideration. For all of their problems, the current
HTable and RpcClient code paths have had a lot of work invested in squashing
bugs and bad behavior. AsyncTableImpl needs to demonstrate a similar level of
behavior before it becomes the only implementation.
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15982) Interface ReplicationEndpoint extends Guava's Service
[
https://issues.apache.org/jira/browse/HBASE-15982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323168#comment-15323168
]
Andrew Purtell commented on HBASE-15982:
bq. maybe we can wrap it so that RE is not exposed to it directly.
This would work
> Interface ReplicationEndpoint extends Guava's Service
> -
>
> Key: HBASE-15982
> URL: https://issues.apache.org/jira/browse/HBASE-15982
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
>
> We have Guava's Service leaking into the LimitedPrivate interface
> ReplicationEndpoint:
> {code}
> public interface ReplicationEndpoint extends Service,
> ReplicationPeerConfigListener
> {code}
> This required a private patch when I updated Guava for our internal
> deployments. This is going to be a problem for us for long term maintenance
> and implenters of pluggable replication endpoints. LP is only less than
> public by a degree. We shouldn't leak types from third part code into either
> Public or LP APIs in my opinion. Let's fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323155#comment-15323155
]
Mikhail Antonov commented on HBASE-15925:
-
Skimmed, looks good to me, except typo on the last line (nit).
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Status: Patch Available (was: Open) > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Status: Open (was: Patch Available) > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Attachment: HBASE-15958.patch > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Attachment: (was: HBASE-15958.patch) > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-14925) Develop HBase shell command/tool to list table's region info through command line
[ https://issues.apache.org/jira/browse/HBASE-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-14925 started by huaxiang sun. > Develop HBase shell command/tool to list table's region info through command > line > - > > Key: HBASE-14925 > URL: https://issues.apache.org/jira/browse/HBASE-14925 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Romil Choksi >Assignee: huaxiang sun > > I am going through the hbase shell commands to see if there is anything I can > use to get all the regions info just for a particular table. I don’t see any > such command that provides me that information. > It would be better to have a command that provides region info, start key, > end key etc taking a table name as the input parameter. This is available > through HBase UI on clicking on a particular table's link > A tool/shell command to get a list of regions for a table or all tables in a > tabular structured output (that is machine readable) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323124#comment-15323124
]
Mikhail Antonov commented on HBASE-15946:
-
Seems we never kicked in hadoop-qa, I'll commit it today pending QA bot run.
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch,
> HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15946:
Status: Patch Available (was: Open)
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.1, 1.3.0
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch,
> HBASE-15946-v3.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15925:
Status: Patch Available (was: Open)
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.1.5, 1.0.3, 1.2.1, 1.2.0, 1.1.0, 1.0.0
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-15925:
---
Assignee: Sean Busbey
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-15925:
Attachment: HBASE-15925.1.patch
-01
- add properties to the top level pom that match those in the hadoop-2.0
profile
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15982) Interface ReplicationEndpoint extends Guava's Service
[
https://issues.apache.org/jira/browse/HBASE-15982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323048#comment-15323048
]
Enis Soztutar commented on HBASE-15982:
---
We've done the RE as a guava service, because our {{Service}} class is
half-finished:
{code}
// This is a WIP. We have Services throughout hbase. Either have all implement
what is here or
// just remove this as an experiment that did not work out.
// TODO: Move on to guava Service after we update our guava version; later
guava has nicer
// Service implmentation.
// TODO: Move all Services on to this one Interface.
@InterfaceAudience.Private
public interface Service {
{code}
I still want to use Guava as our long term Service interface, because the
interface it pretty well defined. However, maybe we can wrap it so that RE is
not exposed to it directly.
> Interface ReplicationEndpoint extends Guava's Service
> -
>
> Key: HBASE-15982
> URL: https://issues.apache.org/jira/browse/HBASE-15982
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
>
> We have Guava's Service leaking into the LimitedPrivate interface
> ReplicationEndpoint:
> {code}
> public interface ReplicationEndpoint extends Service,
> ReplicationPeerConfigListener
> {code}
> This required a private patch when I updated Guava for our internal
> deployments. This is going to be a problem for us for long term maintenance
> and implenters of pluggable replication endpoints. LP is only less than
> public by a degree. We shouldn't leak types from third part code into either
> Public or LP APIs in my opinion. Let's fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15990:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the patch, ChiaPing.
Thanks for the review, Gary.
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15952) Bulk load data replication is not working when RS user does not have permission on hfile-refs node
[ https://issues.apache.org/jira/browse/HBASE-15952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323039#comment-15323039 ] Hudson commented on HBASE-15952: FAILURE: Integrated in HBase-1.3 #730 (See [https://builds.apache.org/job/HBase-1.3/730/]) HBASE-15952 Bulk load data replication is not working when RS user does (ashishsinghi: rev d320ac28ee04e0e7cbac86281f04876ca3a905e5) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java > Bulk load data replication is not working when RS user does not have > permission on hfile-refs node > -- > > Key: HBASE-15952 > URL: https://issues.apache.org/jira/browse/HBASE-15952 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15952.branch-1.v1.patch, HBASE-15952.patch, > HBASE-15952.v1.patch > > > In our recent testing in secure cluster we found that when a RS user does not > have permission on hfile-refs znode then RS was failing to replicate the bulk > loaded data as the hfile-refs znode is created by hbase client and RS user > may not have permission to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[
https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323026#comment-15323026
]
Ted Yu commented on HBASE-15862:
Looks good overall.
{code}
+public class TestBackupDeleteRestore extends TestBackupBase {
{code}
Please javadoc for what the scenario the test covers.
> Backup - Delete- Restore does not restore deleted data
> --
>
> Key: HBASE-15862
> URL: https://issues.apache.org/jira/browse/HBASE-15862
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch
>
>
> This was discovered during testing. If we delete row after full backup and
> perform immediately restore, the deleted row still remains deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16001) Support batching of CAS operations
Nick Dimiduk created HBASE-16001:
Summary: Support batching of CAS operations
Key: HBASE-16001
URL: https://issues.apache.org/jira/browse/HBASE-16001
Project: HBase
Issue Type: Improvement
Reporter: Nick Dimiduk
CAS operations are made available only via direct calls to
{{Table#checkAndXXX}}. There's no Mutation type for {{checkAndPut}} for
example. That means they cannot be used with {{BufferedMutator}}'s batching
capabilities, they must be sent one-for-one with an RPC. Extend the
{{Mutation}} type hierarchy to support sending these CAS operations more
efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15862: -- Attachment: HBASE-15862-v2.patch v2 patch with truncate. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch, HBASE-15862-v2.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15952) Bulk load data replication is not working when RS user does not have permission on hfile-refs node
[ https://issues.apache.org/jira/browse/HBASE-15952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322980#comment-15322980 ] Hudson commented on HBASE-15952: FAILURE: Integrated in HBase-1.4 #206 (See [https://builds.apache.org/job/HBase-1.4/206/]) HBASE-15952 Bulk load data replication is not working when RS user does (ashishsinghi: rev a40ec70da9d9891f5af074535717fe20658a00cc) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java > Bulk load data replication is not working when RS user does not have > permission on hfile-refs node > -- > > Key: HBASE-15952 > URL: https://issues.apache.org/jira/browse/HBASE-15952 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15952.branch-1.v1.patch, HBASE-15952.patch, > HBASE-15952.v1.patch > > > In our recent testing in secure cluster we found that when a RS user does not > have permission on hfile-refs znode then RS was failing to replicate the bulk > loaded data as the hfile-refs znode is created by hbase client and RS user > may not have permission to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322979#comment-15322979
]
Hudson commented on HBASE-15975:
ABORTED: Integrated in HBase-0.98-matrix #354 (See
[https://builds.apache.org/job/HBase-0.98-matrix/354/])
HBASE-15975 logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr
(matteo.bertozzi: rev c8208994b3d796f70ed05d43187994ed98e61ac3)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-15975-v001.patch, HBASE-15975-v002.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322965#comment-15322965
]
Gary Helmling commented on HBASE-15990:
---
+1 on latest
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16000) Table#checkAndPut() docs are too vague
Nick Dimiduk created HBASE-16000:
Summary: Table#checkAndPut() docs are too vague
Key: HBASE-16000
URL: https://issues.apache.org/jira/browse/HBASE-16000
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Nick Dimiduk
Priority: Minor
The docs for our CAS operations that accept a {{CompareOp}} are too vague.
They're copied from the part that implies {{CompareOp.EQUAL}}, which means they
don't tell you how your op, the provided {{value}}, and any present value are
compared.
Only by digging into HRegion.java can I see that the operation is, for example
with CompareOp.GREATER, we get {{value.compareTo(existingKeyValue) > 0}}.
Javadocs should be explicit about the order of operands.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15999) Do not append the current thread name for the threads in the INMEMORY_COMPACTION_POOL
[
https://issues.apache.org/jira/browse/HBASE-15999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322941#comment-15322941
]
stack commented on HBASE-15999:
---
+1
> Do not append the current thread name for the threads in the
> INMEMORY_COMPACTION_POOL
> -
>
> Key: HBASE-15999
> URL: https://issues.apache.org/jira/browse/HBASE-15999
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15999.patch
>
>
> In the INMEMORY_COMPACTION_POOL ThreadPoolExecutor for every thread the
> current thread name is also appended. Since we are using a pool the name gets
> appended and we end up in names like this
> {code}
> "B.defaultRpcServer.handler=89,queue=9,port=16041-inmemoryCompactions-1465492533442-inmemoryCompactions-1465492548754-inmemoryCompactions-1465492548913-inmemoryCompactions-1465492549625-inmemoryCompactions-1465492549956-inmemoryCompactions-1465492567040-inmemoryCompactions-1465492567160-inmemoryCompactions-1465492578465-inmemoryCompactions-1465492578707-inmemoryCompactions-1465492579292-inmemoryCompactions-1465492579357-inmemoryCompactions-1465492579786-inmemoryCompactions-1465492580059-inmemoryCompactions-1465492589975-inmemoryCompactions-1465492590192-inmemoryCompactions-1465492590484-inmemoryCompactions-1465492591144-inmemoryCompactions-1465492592603-inmemoryCompactions-1465492592799-inmemoryCompactions-1465492597106-inmemoryCompactions-1465492602925-inmemoryCompactions-1465492606620-inmemoryCompactions-1465492651478-inmemoryCompactions-1465492653460-inmemoryCompactions-1465492677020-inmemoryCompactions-1465492680857-inmemoryCompactions-1465492681989-inmemoryCompactions-1465492721818-inmemoryCompactions-1465492723562-inmemoryCompactions-1465492724801-inmemoryCompactions-1465492726665-inmemoryCompactions-1465492745750-inmemoryCompactions-1465492745964-inmemoryCompactions-1465492746578-inmemoryCompactions-1465492756867-inmemoryCompactions-1465492764727-inmemoryCompactions-1465492766944-inmemoryCompactions-1465492767098-inmemoryCompactions-1465492785298-inmemoryCompactions-1465492788334-inmemoryCompactions-1465492795954-inmemoryCompactions-1465493047265-inmemoryCompactions-1465493091530-inmemoryCompactions-1465493185684"
> #6006 daemon prio=5 os_prio=0 tid=0x0daa6800 nid=0x454a runnable
> [0x7f50fd0b9000]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322898#comment-15322898
]
Hudson commented on HBASE-15975:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1226 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1226/])
HBASE-15975 logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr
(matteo.bertozzi: rev c8208994b3d796f70ed05d43187994ed98e61ac3)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-15975-v001.patch, HBASE-15975-v002.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15990) The priority value of subsequent coprocessors in the Coprocessor.Priority.SYSTEM list are not incremented by one
[
https://issues.apache.org/jira/browse/HBASE-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15990:
---
Summary: The priority value of subsequent coprocessors in the
Coprocessor.Priority.SYSTEM list are not incremented by one (was: The priority
value of subsequent coprocessor in the Coprocessor.Priority.SYSTEM list are not
incremented by one)
> The priority value of subsequent coprocessors in the
> Coprocessor.Priority.SYSTEM list are not incremented by one
>
>
> Key: HBASE-15990
> URL: https://issues.apache.org/jira/browse/HBASE-15990
> Project: HBase
> Issue Type: Bug
>Affects Versions: master
>Reporter: ChiaPing Tsai
>Assignee: ChiaPing Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15990.patch
>
>
> {code:title=CoprocessorHost.java|borderStyle=solid}
> // The Coprocessor.PRIORITY_SYSTEM should be replaced by priority
> this.coprocessors.add(loadInstance(implClass,
> Coprocessor.PRIORITY_SYSTEM, conf));
> LOG.info("System coprocessor " + className + " was loaded " +
> "successfully with priority (" + priority++ + ").");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager
[
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322768#comment-15322768
]
Hadoop QA commented on HBASE-14743:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 14s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 0s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 54s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestMetricsHeapMemoryManager |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12809195/HBASE-14743.006.patch
|
| JIRA Issue | HBASE-14743 |
| Optional Tests |
[jira] [Commented] (HBASE-15952) Bulk load data replication is not working when RS user does not have permission on hfile-refs node
[ https://issues.apache.org/jira/browse/HBASE-15952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322723#comment-15322723 ] Hudson commented on HBASE-15952: FAILURE: Integrated in HBase-Trunk_matrix #1017 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1017/]) HBASE-15952 Bulk load data replication is not working when RS user does (ashishsinghi: rev 9012a0b123b3eea8b08c8687cef812e83e9b491d) * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesHBaseImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java > Bulk load data replication is not working when RS user does not have > permission on hfile-refs node > -- > > Key: HBASE-15952 > URL: https://issues.apache.org/jira/browse/HBASE-15952 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15952.branch-1.v1.patch, HBASE-15952.patch, > HBASE-15952.v1.patch > > > In our recent testing in secure cluster we found that when a RS user does not > have permission on hfile-refs znode then RS was failing to replicate the bulk > loaded data as the hfile-refs znode is created by hbase client and RS user > may not have permission to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322722#comment-15322722 ] Ted Yu commented on HBASE-15862: I am fine with calling truncate for -overwrite option. The patch needs to be modified - it should call hbaseadmin#truncateTable(, true) > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322702#comment-15322702
]
Hudson commented on HBASE-15975:
FAILURE: Integrated in HBase-1.1-JDK8 #1816 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1816/])
HBASE-15975 logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr
(matteo.bertozzi: rev bccedc6072281760816261735773475fb91ca7b9)
* hbase-client/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-15975-v001.patch, HBASE-15975-v002.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322676#comment-15322676 ] Carter Shanklin commented on HBASE-15862: - Wanted to give my thoughts, the question here is specifically around whether the -overwrite should or should not truncate the tables in the table set before restoring. To me as a user I would expect it to do the truncation automatically because I said -overwrite, this is a destructive operation and I accept this as a user. As I look at some of the comments I'm not sure if Jerry agrees or not, the DB2 behavior seems consistent with automatically truncating. Having the user manually delete the tables: 1. Makes me switch to a different tool (the HBase Ruby shell as opposed the backup commands) 2. When we get to backup/restore for Phoenix, automatic truncation will have several advantages, such as not breaking or forcing re-creation of secondary indexes. So why force users to take that extra step? > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
