[jira] [Created] (HBASE-9996) move the log io operation out of updateLock scope
Liang Xie created HBASE-9996:
Summary: move the log io operation out of updateLock scope
Key: HBASE-9996
URL: https://issues.apache.org/jira/browse/HBASE-9996
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.94.13
Reporter: Liang Xie
Assignee: Liang Xie
I am working at another write outlier issue, and just saw this code pattern:
{code}
synchronized(updateLock) {
...
LOG.info(xxx)
...
}
{code}
let's move the log statement out of this sync block. seems it's ok in trunk
code, just a 0.94 issue
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9996) move the log io operation out of updateLock scope
[
https://issues.apache.org/jira/browse/HBASE-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Xie updated HBASE-9996:
-
Status: Patch Available (was: Open)
move the log io operation out of updateLock scope
-
Key: HBASE-9996
URL: https://issues.apache.org/jira/browse/HBASE-9996
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.94.13
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HBase-9996-0.94.txt
I am working at another write outlier issue, and just saw this code pattern:
{code}
synchronized(updateLock) {
...
LOG.info(xxx)
...
}
{code}
let's move the log statement out of this sync block. seems it's ok in trunk
code, just a 0.94 issue
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9996) move the log io operation out of updateLock scope
[
https://issues.apache.org/jira/browse/HBASE-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Xie updated HBASE-9996:
-
Attachment: HBase-9996-0.94.txt
move the log io operation out of updateLock scope
-
Key: HBASE-9996
URL: https://issues.apache.org/jira/browse/HBASE-9996
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.94.13
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HBase-9996-0.94.txt
I am working at another write outlier issue, and just saw this code pattern:
{code}
synchronized(updateLock) {
...
LOG.info(xxx)
...
}
{code}
let's move the log statement out of this sync block. seems it's ok in trunk
code, just a 0.94 issue
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9987) Remove some synchronisation points in HConnectionManager
[ https://issues.apache.org/jira/browse/HBASE-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826320#comment-13826320 ] Hudson commented on HBASE-9987: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-9987 Remove some synchronisation points in HConnectionManager (nkeywal: rev 1543051) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java Remove some synchronisation points in HConnectionManager Key: HBASE-9987 URL: https://issues.apache.org/jira/browse/HBASE-9987 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9987.v1.patch, 9987.v2.patch Change a Map to a concurrentMap Removed the cachedServer (introduced in HBASE-4785). I suspect that this function is not needed anymore as we also have a list of dead servers, and accessing the list is not blocking. I will dig into this more however. The patch gives a 10% improvement with the NoClusterClient. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-7663) [Per-KV security] Visibility labels
[ https://issues.apache.org/jira/browse/HBASE-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826327#comment-13826327 ] Hudson commented on HBASE-7663: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-7663 [Per-KV security] Visibility labels (anoopsamjohn: rev 1543314) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/InvalidLabelException.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/LabelAlreadyExistsException.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityConstants.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/StreamUtils.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/VisibilityLabelsProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/VisibilityLabels.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionExpander.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ParseException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/SimpleScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ZKVisibilityLabelWatcher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/ExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/LeafExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/NonLeafExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/Operator.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionExpander.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsOpWithDifferentUsersNoACL.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithACL.java * /hbase/trunk/hbase-shell/src/main/ruby/hbase.rb * /hbase/trunk/hbase-shell/src/main/ruby/hbase/hbase.rb * /hbase/trunk/hbase-shell/src/main/ruby/hbase/visibility_labels.rb * /hbase/trunk/hbase-shell/src/main/ruby/shell.rb *
[jira] [Commented] (HBASE-9831) 'hbasefsck.numthreads' property isn't passed to hbck via cmdline -D option
[
https://issues.apache.org/jira/browse/HBASE-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826325#comment-13826325
]
Hudson commented on HBASE-9831:
---
SUCCESS: Integrated in HBase-TRUNK #4687 (See
[https://builds.apache.org/job/HBase-TRUNK/4687/])
HBASE-9831 'hbasefsck.numthreads' property isn't passed to hbck via cmdline -D
option (Takeshi Miao) (jmhsieh: rev 1543137)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
'hbasefsck.numthreads' property isn't passed to hbck via cmdline -D option
--
Key: HBASE-9831
URL: https://issues.apache.org/jira/browse/HBASE-9831
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.94.12
Reporter: takeshi.miao
Assignee: takeshi.miao
Priority: Minor
Labels: hbck
Fix For: 0.98.0, 0.96.1, 0.94.14
Attachments: HBASE-9831-0.94-v02.patch, HBASE-9831-0.94-v03.patch,
HBASE-9831-trunk-v01.patch, HBASE-9831-trunk-v02.patch,
HBASE-9831-trunk-v03.patch, HBASE-9831.v01.patch
We use generic option way to pass _'hbasefsck.numthreads'_ property to
_'hbase hbck'_, but it does not accept our new setting value
{code}
hbase hbck -D hbasefsck.numthreads=5
{code}
We can still find there are threads over than 5 we already set via generic
opttion
{code}
[2013-10-24
09:25:02,561][pool-2-thread-6][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,562][pool-2-thread-10][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,565][pool-2-thread-13][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,566][pool-2-thread-11][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,567][pool-2-thread-9][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,568][pool-2-thread-12][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,570][pool-2-thread-7][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
[2013-10-24
09:25:02,571][pool-2-thread-14][DEBUG][org.apache.hadoop.security.UserGroupInformation]:
PrivilegedAction as:hbase/spn-d-hdn1.s...@ispn.trendmicro.com
(auth:KERBEROS) from:sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) (UserGroupInformation.java:1430)
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9994) ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom()
[ https://issues.apache.org/jira/browse/HBASE-9994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826324#comment-13826324 ] Hudson commented on HBASE-9994: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-9994 ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom() (tedyu: rev 1543309) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/codec/MessageCodec.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom() - Key: HBASE-9994 URL: https://issues.apache.org/jira/browse/HBASE-9994 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0 Attachments: 9994-v1.txt The following classes use ByteString.copyFrom() which should be replaced with ZeroCopyLiteralByteString.wrap() : hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java hbase-server/src/main/java/org/apache/hadoop/hbase/codec/MessageCodec.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9949) Fix the race condition between Compaction and StoreScanner.init
[ https://issues.apache.org/jira/browse/HBASE-9949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826326#comment-13826326 ] Hudson commented on HBASE-9949: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-9949 Addendum takes out test infrastructure and new test (tedyu: rev 1543305) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/InjectionEvent.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/InjectionHandler.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java Fix the race condition between Compaction and StoreScanner.init --- Key: HBASE-9949 URL: https://issues.apache.org/jira/browse/HBASE-9949 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.89-fb Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Priority: Minor Fix For: 0.89-fb, 0.98.0 Attachments: 9949-0.96.addendum, 9949-trunk-v1.txt, 9949-trunk-v2.txt, 9949-trunk-v3.txt, 9949.addendum Original Estimate: 48h Remaining Estimate: 48h The StoreScanner constructor has multiple stages and there can be a race betwwen an ongoing compaction and the StoreScanner constructor where we might get the list of scanners before a compaction and seek on those scanners after the compaction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9973) [ACL]: Users with 'Admin' ACL permission will lose permissions after upgrade to 0.96.x from 0.94.x or 0.92.x
[
https://issues.apache.org/jira/browse/HBASE-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826322#comment-13826322
]
Hudson commented on HBASE-9973:
---
SUCCESS: Integrated in HBase-TRUNK #4687 (See
[https://builds.apache.org/job/HBase-TRUNK/4687/])
HBASE-9973 Users with 'Admin' ACL permission will lose permissions after
upgrade to 0.96.x from 0.94.x or 0.92.x (Himanshu Vashishtha) (mbertozzi: rev
1543179)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/migration/NamespaceUpgrade.java
[ACL]: Users with 'Admin' ACL permission will lose permissions after upgrade
to 0.96.x from 0.94.x or 0.92.x
Key: HBASE-9973
URL: https://issues.apache.org/jira/browse/HBASE-9973
Project: HBase
Issue Type: Bug
Components: migration, security
Affects Versions: 0.96.0, 0.96.1
Reporter: Aleksandr Shulman
Assignee: Himanshu Vashishtha
Labels: acl
Fix For: 0.98.0, 0.96.1
Attachments: 9973-v2.patch, 9973-v2.patch, 9973.patch
In our testing, we have uncovered that the ACL permissions for users with the
'A' credential do not hold after the upgrade to 0.96.x.
This is because in the ACL table, the entry for the admin user is a
permission on the '_acl_' table with permission 'A'. However, because of the
namespace transition, there is no longer an '_acl_' table. Therefore, that
entry in the hbase:acl table is no longer valid.
Example:
{code}hbase(main):002:0 scan 'hbase:acl'
ROW COLUMN+CELL
TestTablecolumn=l:hdfs, timestamp=1384454830701, value=RW
TestTablecolumn=l:root, timestamp=1384455875586, value=RWCA
_acl_column=l:root, timestamp=1384454767568, value=C
_acl_column=l:tableAdmin, timestamp=1384454788035, value=A
hbase:aclcolumn=l:root, timestamp=1384455875786, value=C
{code}
In this case, the following entry becomes meaningless:
{code} _acl_column=l:tableAdmin, timestamp=1384454788035,
value=A {code}
As a result,
Proposed fix:
I see the fix being relatively straightforward. As part of the migration,
change any entries in the '_acl_' table with key '_acl_' into a new row with
key 'hbase:acl', all else being the same. And the old entry would be deleted.
This can go into the standard migration script that we expect users to run.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9924) Avoid potential filename conflict in region_mover.rb
[ https://issues.apache.org/jira/browse/HBASE-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826321#comment-13826321 ] Hudson commented on HBASE-9924: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-9924 Avoid potential filename conflict in region_mover.rb (tedyu: rev 1543225) * /hbase/trunk/bin/region_mover.rb Avoid potential filename conflict in region_mover.rb Key: HBASE-9924 URL: https://issues.apache.org/jira/browse/HBASE-9924 Project: HBase Issue Type: Improvement Components: shell Affects Versions: 0.96.0, 0.94.13 Reporter: Liang Xie Assignee: Liang Xie Fix For: 0.98.0, 0.96.1 Attachments: HBase-9924.txt when i worked at a shared/common box with my colleague, found this error while moving region: NativeException: java.io.FileNotFoundException: /tmp/hh-hadoop-srv-st01.bj (Permission denied) writeFile at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:283 unloadRegions at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:354 (root) at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:480 2013-11-07 15:08:12 Unload host hh-hadoop-srv-st01.bj failed. The root cause is currently getFilename in region move script will get the same output with diff users. One possible quick fix is just add the username to the filename. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8369) MapReduce over snapshot files
[ https://issues.apache.org/jira/browse/HBASE-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826323#comment-13826323 ] Hudson commented on HBASE-8369: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-8369 MapReduce over snapshot files (enis: rev 1543195) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractClientScanner.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/TestCellUtil.java * /hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestTableSnapshotInputFormat.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/MapReduce.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HDFSBlocksDistribution.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/ClientSideRegionScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/TableSnapshotScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ScanPerformanceEvaluation.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestTableSnapshotScanner.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement.java MapReduce over snapshot files - Key: HBASE-8369 URL: https://issues.apache.org/jira/browse/HBASE-8369 Project: HBase Issue Type: New Feature Components: mapreduce, snapshots Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.0 Attachments: HBASE-8369-0.94.patch, HBASE-8369-0.94_v2.patch, HBASE-8369-0.94_v3.patch, HBASE-8369-0.94_v4.patch, HBASE-8369-0.94_v5.patch, HBASE-8369-trunk_v1.patch, HBASE-8369-trunk_v2.patch, HBASE-8369-trunk_v3.patch, hbase-8369_v0.patch, hbase-8369_v11.patch, hbase-8369_v5.patch, hbase-8369_v6.patch, hbase-8369_v7.patch, hbase-8369_v8.patch, hbase-8369_v9.patch The idea is to add an InputFormat, which can run the mapreduce job over snapshot files directly bypassing hbase server layer. The IF is similar in usage to TableInputFormat, taking a Scan object from the user, but instead of running from an online table, it runs from a table snapshot. We do one split per region in the snapshot, and open an HRegion inside the RecordReader. A RegionScanner is used internally for doing the scan without any HRegionServer bits. Users have been asking and searching for ways to run MR jobs by reading directly from hfiles, so this allows new use cases if reading from stale data is ok: - Take snapshots periodically, and run MR jobs only on snapshots. - Export snapshots to remote hdfs cluster, run the MR jobs at that cluster without HBase cluster. - (Future use case) Combine snapshot data with online hbase data: Scan from yesterday's snapshot, but read today's data from online hbase cluster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9982) TestClientNoCluster should use random numbers
[ https://issues.apache.org/jira/browse/HBASE-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826328#comment-13826328 ] Hudson commented on HBASE-9982: --- SUCCESS: Integrated in HBase-TRUNK #4687 (See [https://builds.apache.org/job/HBase-TRUNK/4687/]) HBASE-9982 TestClientNoCluster should use random numbers (nkeywal: rev 1543053) * /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java TestClientNoCluster should use random numbers - Key: HBASE-9982 URL: https://issues.apache.org/jira/browse/HBASE-9982 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.0, 0.96.1 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.96.1 Attachments: 9982.v1.patch, 9982.v2.patch Using random number increases the number of calls to the meta scanner. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9997) Add per KV security details to HBase book
Anoop Sam John created HBASE-9997: - Summary: Add per KV security details to HBase book Key: HBASE-9997 URL: https://issues.apache.org/jira/browse/HBASE-9997 Project: HBase Issue Type: Sub-task Reporter: Anoop Sam John Per KV visibility labels Per KV ACLs -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9992: -- Fix Version/s: 0.94.14 0.96.1 0.98.0 [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9992: -- Affects Version/s: 0.96.0 0.94.13 [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9992: -- Status: Patch Available (was: Open) [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-9992: - Assignee: Jonathan Hsieh [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9992: -- Component/s: hbck [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9992: -- Attachment: hbase-9992.patch Patch is a revert of HBASE-9831 and then the wrapping of the tool mentioned description. Passes unit TestHBaseFsck. Will test with -D params before committing. [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9969) Improve KeyValueHeap using loser tree
[
https://issues.apache.org/jira/browse/HBASE-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826366#comment-13826366
]
Hadoop QA commented on HBASE-9969:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614566/KeyValueHeapBenchmark_v1.ods
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7927//console
This message is automatically generated.
Improve KeyValueHeap using loser tree
-
Key: HBASE-9969
URL: https://issues.apache.org/jira/browse/HBASE-9969
Project: HBase
Issue Type: Improvement
Components: Performance, regionserver
Reporter: Chao Shi
Assignee: Chao Shi
Fix For: 0.98.0, 0.96.1
Attachments: 9969-0.94.txt, KeyValueHeapBenchmark_v1.ods,
hbase-9969-pq-v1.patch, hbase-9969-v2.patch, hbase-9969-v3.patch,
hbase-9969.patch, hbase-9969.patch, kvheap-benchmark.png, kvheap-benchmark.txt
LoserTree is the better data structure than binary heap. It saves half of the
comparisons on each next(), though the time complexity is on O(logN).
Currently A scan or get will go through two KeyValueHeaps, one is merging KVs
read from multiple HFiles in a single store, the other is merging results
from multiple stores. This patch should improve the both cases whenever CPU
is the bottleneck (e.g. scan with filter over cached blocks, HBASE-9811).
All of the optimization work is done in KeyValueHeap and does not change its
public interfaces. The new code looks more cleaner and simpler to understand.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stoping ReplicationSink when using custom implementation for the ReplicationSink.
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9995: -- Fix Version/s: 0.94.15 0.96.1 0.98.0 Status: Patch Available (was: Open) Not stoping ReplicationSink when using custom implementation for the ReplicationSink. - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: HBASE-9995.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stoping ReplicationSink when using custom implementation for the ReplicationSink.
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9995: -- Status: Open (was: Patch Available) Not stoping ReplicationSink when using custom implementation for the ReplicationSink. - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Attachments: HBASE-9995.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9969) Improve KeyValueHeap using loser tree
[
https://issues.apache.org/jira/browse/HBASE-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826388#comment-13826388
]
Matt Corgan commented on HBASE-9969:
Not sure how i didn't see it before, but here is the optimization that avoids
sending the scanner through the heap when the next KV is from the same scanner:
KeyValueHeap.pollRealKV() : line 350
{code}
// Compare the current scanner to the next scanner. We try to avoid
// putting the current one back into the heap if possible.
KeyValue nextKV = nextEarliestScanner.peek();
if (nextKV == null || comparator.compare(curKV, nextKV) 0) {
// We already have the scanner with the earliest KV, so return it.
return kvScanner;
}
{code}
Does the LoserTreeKeyValueHeap do a check like that?
Improve KeyValueHeap using loser tree
-
Key: HBASE-9969
URL: https://issues.apache.org/jira/browse/HBASE-9969
Project: HBase
Issue Type: Improvement
Components: Performance, regionserver
Reporter: Chao Shi
Assignee: Chao Shi
Fix For: 0.98.0, 0.96.1
Attachments: 9969-0.94.txt, KeyValueHeapBenchmark_v1.ods,
hbase-9969-pq-v1.patch, hbase-9969-v2.patch, hbase-9969-v3.patch,
hbase-9969.patch, hbase-9969.patch, kvheap-benchmark.png, kvheap-benchmark.txt
LoserTree is the better data structure than binary heap. It saves half of the
comparisons on each next(), though the time complexity is on O(logN).
Currently A scan or get will go through two KeyValueHeaps, one is merging KVs
read from multiple HFiles in a single store, the other is merging results
from multiple stores. This patch should improve the both cases whenever CPU
is the bottleneck (e.g. scan with filter over cached blocks, HBASE-9811).
All of the optimization work is done in KeyValueHeap and does not change its
public interfaces. The new code looks more cleaner and simpler to understand.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9995) Not stoping ReplicationSink when using custom implementation for the ReplicationSink.
[
https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826415#comment-13826415
]
Hadoop QA commented on HBASE-9995:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614565/HBASE-9995.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 9
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7928//console
This message is automatically generated.
Not stoping ReplicationSink when using custom implementation for the
ReplicationSink.
-
Key: HBASE-9995
URL: https://issues.apache.org/jira/browse/HBASE-9995
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.13
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Priority: Minor
Fix For: 0.98.0, 0.96.1, 0.94.15
Attachments: HBASE-9995.patch
Missed this in HBASE-9975.
Also solving a new javadoc warning induced by HBASE-9975
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9996) move the log io operation out of updateLock scope
[
https://issues.apache.org/jira/browse/HBASE-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826448#comment-13826448
]
Jean-Marc Spaggiari commented on HBASE-9996:
sgtm
Any risks that this.fs.getFileStatus(oldFile).getLen() changes between the sync
and when it's called just after?
+1
move the log io operation out of updateLock scope
-
Key: HBASE-9996
URL: https://issues.apache.org/jira/browse/HBASE-9996
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.94.13
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HBase-9996-0.94.txt
I am working at another write outlier issue, and just saw this code pattern:
{code}
synchronized(updateLock) {
...
LOG.info(xxx)
...
}
{code}
let's move the log statement out of this sync block. seems it's ok in trunk
code, just a 0.94 issue
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9996) move the log io operation out of updateLock scope
[
https://issues.apache.org/jira/browse/HBASE-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826474#comment-13826474
]
Hadoop QA commented on HBASE-9996:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614578/HBase-9996-0.94.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7930//console
This message is automatically generated.
move the log io operation out of updateLock scope
-
Key: HBASE-9996
URL: https://issues.apache.org/jira/browse/HBASE-9996
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.94.13
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HBase-9996-0.94.txt
I am working at another write outlier issue, and just saw this code pattern:
{code}
synchronized(updateLock) {
...
LOG.info(xxx)
...
}
{code}
let's move the log statement out of this sync block. seems it's ok in trunk
code, just a 0.94 issue
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9998) Fix javadoc warnings induced by commits
[ https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9998: -- Fix Version/s: 0.96.1 0.98.0 Status: Patch Available (was: Open) Fix javadoc warnings induced by commits --- Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9998) Fix javadoc warnings induced by commits
Anoop Sam John created HBASE-9998: - Summary: Fix javadoc warnings induced by commits Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9998) Fix javadoc warnings induced by commits
[ https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9998: -- Attachment: HBASE-9998.patch Fix javadoc warnings induced by commits --- Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9998) Fix javadoc warnings induced by commits
[ https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826486#comment-13826486 ] Nicolas Liochon commented on HBASE-9998: +1 Fix javadoc warnings induced by commits --- Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9988) DOn't use HRI#getEncodedName in the client
[ https://issues.apache.org/jira/browse/HBASE-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9988: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed, thanks for the review, Nick. DOn't use HRI#getEncodedName in the client -- Key: HBASE-9988 URL: https://issues.apache.org/jira/browse/HBASE-9988 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9988.v1.patch, 9988.v2.patch This functions does a lazy initialisation. It cost memory and it creates a synchronisation point. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9959) Remove some array copy - server side
[ https://issues.apache.org/jira/browse/HBASE-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9959: --- Attachment: 9959.v5-0.96.patch Remove some array copy - server side Key: HBASE-9959 URL: https://issues.apache.org/jira/browse/HBASE-9959 Project: HBase Issue Type: Bug Components: Protobufs, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9959-trunk.v1.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959.v1.patch, 9959.v3.patch, 9959.v4.patch, 9959.v5-0.96.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9959) Remove some array copy - server side
[ https://issues.apache.org/jira/browse/HBASE-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9959: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.96 trunk, thanks for the review, Ted Stack. Remove some array copy - server side Key: HBASE-9959 URL: https://issues.apache.org/jira/browse/HBASE-9959 Project: HBase Issue Type: Bug Components: Protobufs, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9959-trunk.v1.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959.v1.patch, 9959.v3.patch, 9959.v4.patch, 9959.v5-0.96.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[
https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826534#comment-13826534
]
Hadoop QA commented on HBASE-9992:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614585/hbase-9992.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
10 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7929//console
This message is automatically generated.
[hbck] Refactor so that arbitrary -D cmdline options are included
--
Key: HBASE-9992
URL: https://issues.apache.org/jira/browse/HBASE-9992
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.96.0, 0.94.13
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.96.1, 0.94.14
Attachments: hbase-9992.patch
A review of HBASE-9831 pointed out the fact that -D options aren't being
passed into the configuration object used by hbck. This means overriding -D
options will not work unless special hooks are for specific options. A first
attempt to fix this was in HBASE-9831 but it affected many other files.
The right approach would be to create a new HbckTool class that had the
configured interface and change to existing HBaseFsck main to instantiate
that to have it parse args, and then create the HBaseFsck object inside run.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9989) Add a test on get in TestClientNoCluster
[ https://issues.apache.org/jira/browse/HBASE-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9989: --- Status: Patch Available (was: Open) Add a test on get in TestClientNoCluster Key: HBASE-9989 URL: https://issues.apache.org/jira/browse/HBASE-9989 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.96.1 Attachments: 9989.v1.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9999) Add support for small reverse scan
[ https://issues.apache.org/jira/browse/HBASE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-: -- Description: HBASE-4811 adds feature of reverse scan. This JIRA adds the support for small reverse scan. This is activated when both 'reversed' and 'small' attributes are true in Scan Object was: HBASE-4811 added feature of reverse scan. This JIRA adds the support for small reverse scan. This is activated when both 'reversed' and 'small' attributes are true in Scan Object Add support for small reverse scan -- Key: HBASE- URL: https://issues.apache.org/jira/browse/HBASE- Project: HBase Issue Type: Improvement Reporter: Ted Yu HBASE-4811 adds feature of reverse scan. This JIRA adds the support for small reverse scan. This is activated when both 'reversed' and 'small' attributes are true in Scan Object -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-10000) Initiate lease recovery for outstanding WAL files at the very beginning of recovery
Ted Yu created HBASE-1: -- Summary: Initiate lease recovery for outstanding WAL files at the very beginning of recovery Key: HBASE-1 URL: https://issues.apache.org/jira/browse/HBASE-1 Project: HBase Issue Type: Improvement Reporter: Ted Yu At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9999) Add support for small reverse scan
Ted Yu created HBASE-: - Summary: Add support for small reverse scan Key: HBASE- URL: https://issues.apache.org/jira/browse/HBASE- Project: HBase Issue Type: Improvement Reporter: Ted Yu HBASE-4811 added feature of reverse scan. This JIRA adds the support for small reverse scan. This is activated when both 'reversed' and 'small' attributes are true in Scan Object -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Status: Open (was: Patch Available) Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Attachment: 9976.v7.patch Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Status: Patch Available (was: Open) Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826551#comment-13826551 ] Ted Yu commented on HBASE-9995: --- +1 Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: HBASE-9995.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9995: -- Summary: Not stopping ReplicationSink when using custom implementation for the ReplicationSink (was: Not stoping ReplicationSink when using custom implementation for the ReplicationSink.) Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: HBASE-9995.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10000) Initiate lease recovery for outstanding WAL files at the very beginning of recovery
[ https://issues.apache.org/jira/browse/HBASE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-1: --- Description: At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Thanks to Nicolas Liochon and Jeffery whose discussion gave rise to this idea. was: At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Initiate lease recovery for outstanding WAL files at the very beginning of recovery --- Key: HBASE-1 URL: https://issues.apache.org/jira/browse/HBASE-1 Project: HBase Issue Type: Improvement Reporter: Ted Yu At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Thanks to Nicolas Liochon and Jeffery whose discussion gave rise to this idea. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10000) Initiate lease recovery for outstanding WAL files at the very beginning of recovery
[ https://issues.apache.org/jira/browse/HBASE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-1: --- Description: At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Thanks to Nicolas Liochon and Jeffery discussion with whom gave rise to this idea. was: At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Thanks to Nicolas Liochon and Jeffery whose discussion gave rise to this idea. Initiate lease recovery for outstanding WAL files at the very beginning of recovery --- Key: HBASE-1 URL: https://issues.apache.org/jira/browse/HBASE-1 Project: HBase Issue Type: Improvement Reporter: Ted Yu At the beginning of recovery, master can send lease recovery requests concurrently for outstanding WAL files using a thread pool. Each split worker would first check whether the WAL file it processes is closed. Thanks to Nicolas Liochon and Jeffery discussion with whom gave rise to this idea. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
Nicolas Liochon created HBASE-10001: --- Summary: Add a coprocessor to help testing the performances without taking into account the i/o Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.13, 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-10001: Status: Patch Available (was: Open) Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.13, 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-10001: Attachment: 10001.v1.patch Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[
https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826602#comment-13826602
]
Anoop Sam John commented on HBASE-10001:
bq.private static final Log LOG = LogFactory.getLog(BaseRegionObserver.class);
You want LogFactory.getLog(WriteSinkCoprocessor.class); ?
{quote}
if (coprocessorHost.prePut((Put) m, walEdit, m.getDurability())) {
// pre hook says skip this Put
// mark as success and skip in doMiniBatchMutation
batchOp.retCodeDetails[i] = OperationStatus.SUCCESS;
}
{quote}
You can use prePut() hook and do a bypass from there so that even WAL write
also can be avoided?
Add a coprocessor to help testing the performances without taking into
account the i/o
--
Key: HBASE-10001
URL: https://issues.apache.org/jira/browse/HBASE-10001
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.98.0, 0.96.0, 0.94.13
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.98.0, 0.96.1, 0.94.15
Attachments: 10001.v1.patch
We have a mockup to test only the client. If we want to include the network,
without beeing limited by the i/o, we don't have much tools.
This coprocessor helps to test this.
I put it in the main code as to make it usable without adding a jar...
I don't think it's possible avoid the WAL writes in the coprocessors. It
would be great to have it to simplify the test with any kind of client (i.e.
w/o changing the durability).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826609#comment-13826609 ] Nicolas Liochon commented on HBASE-10001: - bq. You want LogFactory.getLog(WriteSinkCoprocessor.class); ? Yes :-) bq. You can use prePut() hook and do a bypass from there so that even WAL write also can be avoided? Great, let me update the patch. Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-10001: Status: Open (was: Patch Available) Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.13, 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-7662) [Per-KV security] Store and apply per cell ACLs into/from KeyValue tags
[ https://issues.apache.org/jira/browse/HBASE-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826612#comment-13826612 ] Andrew Purtell commented on HBASE-7662: --- Now that HBASE-7663 is committed I'm going to do a small refactor to move the setACL methods to the new Query base class for Get and Scan, and will add javadoc per above review comments. Will also bring over a large unit test based on TestMiniClusterLoadParallel from earlier work done for 0.94. Assuming the +1 above is still valid after these changes. Will commit this week and post final patch unless objection. [Per-KV security] Store and apply per cell ACLs into/from KeyValue tags --- Key: HBASE-7662 URL: https://issues.apache.org/jira/browse/HBASE-7662 Project: HBase Issue Type: Sub-task Components: Coprocessors, security Affects Versions: 0.98.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: 7662.patch, 7662.patch, 7662.patch, 7662.patch We can improve the performance of per-cell authorization if the read of the cell ACL, if any, is combined with the sequential read of the cell data already in progress. When tags are inlined with KVs in block encoding (see HBASE-7448, and more generally HBASE-7233), we can use them to carry cell ACLs instead of using out-of-line storage (HBASE-7661) for that purpose. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9976) Don't create duplicated TableName objects
[
https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826613#comment-13826613
]
Hadoop QA commented on HBASE-9976:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614626/9976.v7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
10 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.master.balancer.TestFavoredNodeAssignmentHelper.testSecondaryAndTertiaryPlacementWithMultipleRacks(TestFavoredNodeAssignmentHelper.java:187)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7931//console
This message is automatically generated.
Don't create duplicated TableName objects
-
Key: HBASE-9976
URL: https://issues.apache.org/jira/browse/HBASE-9976
Project: HBase
Issue Type: Bug
Components: Client, regionserver
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Fix For: 0.98.0, 0.96.1
Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch,
9976.v7.patch, 9976.v7.patch
A profiling show that the table name is reponsible for 25% of the memory
needed to keep the region locations. As well, comparisons will be faster if
two identical table names are a single java object.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826618#comment-13826618 ] Andrew Purtell commented on HBASE-10001: IIRC you can also modify the put in the pre hook: put.writeToWAL(false) Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-10002) Improve buffer copies in write path continued
stack created HBASE-10002: - Summary: Improve buffer copies in write path continued Key: HBASE-10002 URL: https://issues.apache.org/jira/browse/HBASE-10002 Project: HBase Issue Type: Umbrella Components: IPC/RPC Reporter: stack Assignee: stack HBASE-5945 Reduce buffer copies in IPC server response path added gathering of buffers before writing the channel. There are more improvements to be made yet. The composition of the cellblock is currently done in heap first. Todd over in original issues suggests further improvements to be had: https://issues.apache.org/jira/browse/HBASE-5945?focusedCommentId=13269213page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13269213 Let this issue be an umbrella issue for addressing these improvements. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10002) Improve buffer copies in write path continued
[ https://issues.apache.org/jira/browse/HBASE-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826628#comment-13826628 ] stack commented on HBASE-10002: --- This issue is about the read path as opposed to write. Improve buffer copies in write path continued - Key: HBASE-10002 URL: https://issues.apache.org/jira/browse/HBASE-10002 Project: HBase Issue Type: Umbrella Components: IPC/RPC Reporter: stack Assignee: stack HBASE-5945 Reduce buffer copies in IPC server response path added gathering of buffers before writing the channel. There are more improvements to be made yet. The composition of the cellblock is currently done in heap first. Todd over in original issues suggests further improvements to be had: https://issues.apache.org/jira/browse/HBASE-5945?focusedCommentId=13269213page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13269213 Let this issue be an umbrella issue for addressing these improvements. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[
https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826643#comment-13826643
]
Nicolas Liochon commented on HBASE-10001:
-
Actually, if I read correctly the code in HRegion, just marking the operations
as success as I do in the v1 is enough:
{code}
//
// STEP 4. Build WAL edit
// --
boolean hasWalAppends = false;
Durability durability = Durability.USE_DEFAULT;
for (int i = firstIndex; i lastIndexExclusive; i++) {
// Skip puts that were determined to be invalid during preprocessing
if (batchOp.retCodeDetails[i].getOperationStatusCode()
!= OperationStatusCode.NOT_RUN) { //
= if the copro marked it as SUCCESS we don't write it to
the WAL.
continue;
}
{code}
Add a coprocessor to help testing the performances without taking into
account the i/o
--
Key: HBASE-10001
URL: https://issues.apache.org/jira/browse/HBASE-10001
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.98.0, 0.96.0, 0.94.13
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.98.0, 0.96.1, 0.94.15
Attachments: 10001.v1.patch
We have a mockup to test only the client. If we want to include the network,
without beeing limited by the i/o, we don't have much tools.
This coprocessor helps to test this.
I put it in the main code as to make it usable without adding a jar...
I don't think it's possible avoid the WAL writes in the coprocessors. It
would be great to have it to simplify the test with any kind of client (i.e.
w/o changing the durability).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826652#comment-13826652 ] Anoop Sam John commented on HBASE-10001: Oh yes.. preBatchMutation is called before WAL write. So this is just fine. Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826654#comment-13826654 ] Nicolas Liochon commented on HBASE-10001: - Thanks for the confirmation, I'm going to update the comments upload the v2 then. Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-10001: Status: Patch Available (was: Open) Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.13, 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch, 10001.v2.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10001) Add a coprocessor to help testing the performances without taking into account the i/o
[ https://issues.apache.org/jira/browse/HBASE-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-10001: Attachment: 10001.v2.patch Add a coprocessor to help testing the performances without taking into account the i/o -- Key: HBASE-10001 URL: https://issues.apache.org/jira/browse/HBASE-10001 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.98.0, 0.96.0, 0.94.13 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: 10001.v1.patch, 10001.v2.patch We have a mockup to test only the client. If we want to include the network, without beeing limited by the i/o, we don't have much tools. This coprocessor helps to test this. I put it in the main code as to make it usable without adding a jar... I don't think it's possible avoid the WAL writes in the coprocessors. It would be great to have it to simplify the test with any kind of client (i.e. w/o changing the durability). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9998) Fix javadoc warnings induced by commits
[ https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9998: -- Status: Open (was: Patch Available) Fix javadoc warnings induced by commits --- Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9998) Fix javadoc warnings induced by commits
[ https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9998: -- Status: Patch Available (was: Open) Fix javadoc warnings induced by commits --- Key: HBASE-9998 URL: https://issues.apache.org/jira/browse/HBASE-9998 Project: HBase Issue Type: Improvement Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9998.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-10003) OnlineMerge should be extended to allow bulk merging
Clint Heath created HBASE-10003: --- Summary: OnlineMerge should be extended to allow bulk merging Key: HBASE-10003 URL: https://issues.apache.org/jira/browse/HBASE-10003 Project: HBase Issue Type: Improvement Components: Admin Affects Versions: 0.94.6, 0.98.0 Reporter: Clint Heath Now that we have Online Merge capabilities, the function of that tool should be extended to make it much easier for HBase operations folks to use. Currently it is a very manual process (one fraught with confusion) to hand pick two regions that are contiguous to each other in the META table such that the admin can manually request those two regions to be merged. In the real world, when admins find themselves wanting to merge regions, it's usually because they've greatly increased their hbase.hregion.max.filesize property and they have way too many regions on a table and want to reduce the region count for that entire table quickly and easily. Why can't the OnlineMerge command just take a -max argument along with a table name which tells it to go ahead and merge all regions of said table until the resulting regions are all of max size? This takes the voodoo out of the process and quickly gets the admin what they're looking for. As part of this improvement, I also suggest a -regioncount argument for OnlineMerge, which will attempt to reduce the table's region count down to the specified #. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9995: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to Trunk, 0.96 and 0.94 branches. Thanks for the review Ted. Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9995: -- Attachment: (was: HBASE-9995.patch) Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9995: -- Attachment: HBASE-9995_0.94.patch HBASE-9995.patch Patches what I committed Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: HBASE-9995.patch, HBASE-9995_0.94.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9949) Fix the race condition between Compaction and StoreScanner.init
[
https://issues.apache.org/jira/browse/HBASE-9949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826704#comment-13826704
]
Ted Yu commented on HBASE-9949:
---
@Jon:
I was thinking about the following approach for testing :
1. Introduce config parameter for StoreScanner implementation which would be
used by HStore for creating scanner
2. In TestStoreScanner, add StoreScanner implementation which extends
StoreScanner and set the above config parameter to this class.
3. Register custom ChangedReadersObserver through the following API of HStore :
{code}
public void addChangedReaderObserver(ChangedReadersObserver o) {
{code}
The BEFORE_SEEK hook would be activated before Store.getScanner() is called.
The AFTER_SEEK hook can be activated at the end of ctor of StoreScanner wrapper
created in #2
The custom ChangedReadersObserver would activate the COMPACT_COMPLETE hook.
What do you think ?
Fix the race condition between Compaction and StoreScanner.init
---
Key: HBASE-9949
URL: https://issues.apache.org/jira/browse/HBASE-9949
Project: HBase
Issue Type: Bug
Components: Scanners
Affects Versions: 0.89-fb
Reporter: Manukranth Kolloju
Assignee: Manukranth Kolloju
Priority: Minor
Fix For: 0.89-fb, 0.98.0
Attachments: 9949-0.96.addendum, 9949-trunk-v1.txt,
9949-trunk-v2.txt, 9949-trunk-v3.txt, 9949.addendum
Original Estimate: 48h
Remaining Estimate: 48h
The StoreScanner constructor has multiple stages and there can be a race
betwwen an ongoing compaction and the StoreScanner constructor where we might
get the list of scanners before a compaction and seek on those scanners after
the compaction.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9989) Add a test on get in TestClientNoCluster
[
https://issues.apache.org/jira/browse/HBASE-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826705#comment-13826705
]
Hadoop QA commented on HBASE-9989:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614414/9989.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
10 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7932//console
This message is automatically generated.
Add a test on get in TestClientNoCluster
Key: HBASE-9989
URL: https://issues.apache.org/jira/browse/HBASE-9989
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Trivial
Fix For: 0.98.0, 0.96.1
Attachments: 9989.v1.patch
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Status: Open (was: Patch Available) Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch, 9976.v8.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826710#comment-13826710 ] Nicolas Liochon commented on HBASE-9976: TestFavoredNodeAssignmentHelper was creating 6 TableName. As we do a copy on write, we're slower than before on such a scenario. Locally the test passed, but it's likely slower on Jenkins. I fixed this by having a single table for this test. Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch, 9976.v8.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Status: Patch Available (was: Open) Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.96.0, 0.98.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch, 9976.v8.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9976) Don't create duplicated TableName objects
[ https://issues.apache.org/jira/browse/HBASE-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9976: --- Attachment: 9976.v8.patch Don't create duplicated TableName objects - Key: HBASE-9976 URL: https://issues.apache.org/jira/browse/HBASE-9976 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9976.v1.patch, 9976.v4.patch, 9976.v6.patch, 9976.v7.patch, 9976.v7.patch, 9976.v8.patch A profiling show that the table name is reponsible for 25% of the memory needed to keep the region locations. As well, comparisons will be faster if two identical table names are a single java object. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826715#comment-13826715 ] Nick Dimiduk commented on HBASE-9992: - So the wrapper class is to avoid changing tests? Is that a significant amount of cleanup? This approach seems hackish to me and threading the conf object through the layer looks error-prone. [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9989) Add a test on get in TestClientNoCluster
[ https://issues.apache.org/jira/browse/HBASE-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826722#comment-13826722 ] Nicolas Liochon commented on HBASE-9989: Committed, thanks for the review Nick. Add a test on get in TestClientNoCluster Key: HBASE-9989 URL: https://issues.apache.org/jira/browse/HBASE-9989 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.96.1 Attachments: 9989.v1.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9989) Add a test on get in TestClientNoCluster
[ https://issues.apache.org/jira/browse/HBASE-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9989: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Add a test on get in TestClientNoCluster Key: HBASE-9989 URL: https://issues.apache.org/jira/browse/HBASE-9989 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.96.1 Attachments: 9989.v1.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9893) Incorrect assert condition in OrderedBytes decoding
[
https://issues.apache.org/jira/browse/HBASE-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9893:
Attachment: HBASE-9893.00-1.patch
Reattaching patch for buildbot. Javadoc warnings aren't this patch.
Incorrect assert condition in OrderedBytes decoding
---
Key: HBASE-9893
URL: https://issues.apache.org/jira/browse/HBASE-9893
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.96.0
Reporter: He Liangliang
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9893.00-1.patch, HBASE-9893.00.patch,
HBASE-9893.patch
The following assert condition is incorrect when decoding blob var byte array.
{code}
assert t == 0 : Unexpected bits remaining after decoding blob.;
{code}
When the number of bytes to decode is multiples of 8 (i.e the original number
of bytes is multiples of 7), this assert may fail.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9893) Incorrect assert condition in OrderedBytes decoding
[
https://issues.apache.org/jira/browse/HBASE-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9893:
Status: Open (was: Patch Available)
Incorrect assert condition in OrderedBytes decoding
---
Key: HBASE-9893
URL: https://issues.apache.org/jira/browse/HBASE-9893
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.96.0
Reporter: He Liangliang
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9893.00.patch, HBASE-9893.patch
The following assert condition is incorrect when decoding blob var byte array.
{code}
assert t == 0 : Unexpected bits remaining after decoding blob.;
{code}
When the number of bytes to decode is multiples of 8 (i.e the original number
of bytes is multiples of 7), this assert may fail.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-9893) Incorrect assert condition in OrderedBytes decoding
[
https://issues.apache.org/jira/browse/HBASE-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9893:
Fix Version/s: 0.96.1
0.98.0
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
Incorrect assert condition in OrderedBytes decoding
---
Key: HBASE-9893
URL: https://issues.apache.org/jira/browse/HBASE-9893
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.96.0
Reporter: He Liangliang
Assignee: Nick Dimiduk
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: HBASE-9893.00-1.patch, HBASE-9893.00.patch,
HBASE-9893.patch
The following assert condition is incorrect when decoding blob var byte array.
{code}
assert t == 0 : Unexpected bits remaining after decoding blob.;
{code}
When the number of bytes to decode is multiples of 8 (i.e the original number
of bytes is multiples of 7), this assert may fail.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HBASE-10004) Some fixes for scoping sequence Ids to region level
Himanshu Vashishtha created HBASE-10004: --- Summary: Some fixes for scoping sequence Ids to region level Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-10004: Status: Patch Available (was: Open) Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-10004: Attachment: 10004.patch Attached is a patch to fix them. Ran TestHLog multiple times; Ran HLogPE with 5 threads for 1 iterations multiple times. They all pass. Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[
https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826766#comment-13826766
]
Jonathan Hsieh commented on HBASE-9992:
---
I don't see how this is hacky -- can you explain why. This approach now
captures *all* cases where configuration params from a -D at the command line
get put into the conf. If you look at the v3 version of the patch at
HBASE-9831. What I committed for HBASE-9831 only caught the specific case
mentioned in the HBASE-9831 case.
Excluding the revert, here's the patch:
{code}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util
index 09e3acd..02c3b7a 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
@@ -164,7 +164,7 @@ import com.google.protobuf.ServiceException;
*/
@InterfaceAudience.Public
@InterfaceStability.Evolving
-public class HBaseFsck extends Configured implements Tool {
+public class HBaseFsck extends Configured {
public static final long DEFAULT_TIME_LAG = 6; // default value of 1
minute
public static final long DEFAULT_SLEEP_BEFORE_RERUN = 1;
private static final int MAX_NUM_THREADS = 50; // #threads to contact regions
@@ -3590,15 +3590,23 @@ public class HBaseFsck extends Configured implements
Tool {
URI defaultFs = hbasedir.getFileSystem(conf).getUri();
FSUtils.setFsDefault(conf, new Path(defaultFs));
-int ret = ToolRunner.run(new HBaseFsck(conf), args);
+int ret = ToolRunner.run(new HBaseFsckTool(conf), args);
System.exit(ret);
}
- @Override
- public int run(String[] args) throws Exception {
-exec(executor, args);
-return getRetCode();
- }
+ /**
+ * This is a Tool wrapper that gathers -Dxxx=yyy configuration settings from
the command line.
+ */
+ static class HBaseFsckTool extends Configured implements Tool {
+HBaseFsckTool(Configuration conf) { super(conf); }
+@Override
+public int run(String[] args) throws Exception {
+ HBaseFsck hbck = new HBaseFsck(getConf());
+ hbck.exec(hbck.executor, args);
+ return hbck.getRetCode();
+}
+ };
+
public HBaseFsck exec(ExecutorService exec, String[] args) throws
KeeperException, IOException,
ServiceException, InterruptedException {
{code}
[hbck] Refactor so that arbitrary -D cmdline options are included
--
Key: HBASE-9992
URL: https://issues.apache.org/jira/browse/HBASE-9992
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.96.0, 0.94.13
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.96.1, 0.94.14
Attachments: hbase-9992.patch
A review of HBASE-9831 pointed out the fact that -D options aren't being
passed into the configuration object used by hbck. This means overriding -D
options will not work unless special hooks are for specific options. A first
attempt to fix this was in HBASE-9831 but it affected many other files.
The right approach would be to create a new HbckTool class that had the
configured interface and change to existing HBaseFsck main to instantiate
that to have it parse args, and then create the HBaseFsck object inside run.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-10004: Attachment: 10004.patch Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-10004: Attachment: (was: 10004.patch) Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-10004: Attachment: 10004.patch Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch, 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826790#comment-13826790 ] Himanshu Vashishtha commented on HBASE-10004: - With correct formatting. Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch, 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826795#comment-13826795 ] Matteo Bertozzi commented on HBASE-10004: - +1 looks good to me Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch, 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-10004) Some fixes for scoping sequence Ids to region level
[ https://issues.apache.org/jira/browse/HBASE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826808#comment-13826808 ] Himanshu Vashishtha commented on HBASE-10004: - Thanks for the review Matteo. Some fixes for scoping sequence Ids to region level --- Key: HBASE-10004 URL: https://issues.apache.org/jira/browse/HBASE-10004 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.98.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.98.0 Attachments: 10004.patch, 10004.patch While looking at trunk, I figured out some issues related to fix provided in HBASE-8741. This jira is to fix them: 1) Not so helpful log message in FSHLog#getFileNumFromFileName. 2) HLogPE verify method, the region:sequenceId map was not getting updated. 3) reverse use of assertEquals arguments in TestHLog#testFindMemStoresEligibleForFlush. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9992) [hbck] Refactor so that arbitrary -D cmdline options are included
[ https://issues.apache.org/jira/browse/HBASE-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826818#comment-13826818 ] Nick Dimiduk commented on HBASE-9992: - Hacky in that it does not follow the pattern of directly extending Tool. I agree that it's an improvement over the reverted special casing logic. [hbck] Refactor so that arbitrary -D cmdline options are included -- Key: HBASE-9992 URL: https://issues.apache.org/jira/browse/HBASE-9992 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.96.0, 0.94.13 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: hbase-9992.patch A review of HBASE-9831 pointed out the fact that -D options aren't being passed into the configuration object used by hbck. This means overriding -D options will not work unless special hooks are for specific options. A first attempt to fix this was in HBASE-9831 but it affected many other files. The right approach would be to create a new HbckTool class that had the configured interface and change to existing HBaseFsck main to instantiate that to have it parse args, and then create the HBaseFsck object inside run. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9998) Fix javadoc warnings induced by commits
[
https://issues.apache.org/jira/browse/HBASE-9998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826822#comment-13826822
]
Hadoop QA commented on HBASE-9998:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614618/HBASE-9998.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7933//console
This message is automatically generated.
Fix javadoc warnings induced by commits
---
Key: HBASE-9998
URL: https://issues.apache.org/jira/browse/HBASE-9998
Project: HBase
Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: HBASE-9998.patch
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9875) NamespaceJanitor chore is not used
[ https://issues.apache.org/jira/browse/HBASE-9875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826824#comment-13826824 ] Ted Yu commented on HBASE-9875: --- [~jxiang],[~toffer]: Your opinion would be appreciated. NamespaceJanitor chore is not used -- Key: HBASE-9875 URL: https://issues.apache.org/jira/browse/HBASE-9875 Project: HBase Issue Type: Task Reporter: Ted Yu Currently NamespaceJanitor chore is not started by HMaster. Jimmy identified some race conditions around the janitor. Francis said the janitor can do the cleanup for some failure scenarios. This JIRA is to solicit discussion on how the cleanup should be properly done. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9994) ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom()
[ https://issues.apache.org/jira/browse/HBASE-9994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826846#comment-13826846 ] Hudson commented on HBASE-9994: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9994 ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom() (tedyu: rev 1543309) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/codec/MessageCodec.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java ZeroCopyLiteralByteString.wrap() should be used in place of ByteString.copyFrom() - Key: HBASE-9994 URL: https://issues.apache.org/jira/browse/HBASE-9994 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0 Attachments: 9994-v1.txt The following classes use ByteString.copyFrom() which should be replaced with ZeroCopyLiteralByteString.wrap() : hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.java hbase-server/src/main/java/org/apache/hadoop/hbase/codec/MessageCodec.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9989) Add a test on get in TestClientNoCluster
[ https://issues.apache.org/jira/browse/HBASE-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826845#comment-13826845 ] Hudson commented on HBASE-9989: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9989 Add a test on get in TestClientNoCluster (nkeywal: rev 1543514) * /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java Add a test on get in TestClientNoCluster Key: HBASE-9989 URL: https://issues.apache.org/jira/browse/HBASE-9989 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.96.1 Attachments: 9989.v1.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-7663) [Per-KV security] Visibility labels
[ https://issues.apache.org/jira/browse/HBASE-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826849#comment-13826849 ] Hudson commented on HBASE-7663: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-7663 [Per-KV security] Visibility labels (anoopsamjohn: rev 1543314) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/Authorizations.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/CellVisibility.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/InvalidLabelException.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/LabelAlreadyExistsException.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityConstants.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsValidator.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/StreamUtils.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/VisibilityLabelsProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto * /hbase/trunk/hbase-protocol/src/main/protobuf/VisibilityLabels.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/DefaultScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionExpander.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ExpressionParser.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ParseException.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/SimpleScanLabelGenerator.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelsManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/ZKVisibilityLabelWatcher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/ExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/LeafExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/NonLeafExpressionNode.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/expression/Operator.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionExpander.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestExpressionParser.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsOpWithDifferentUsersNoACL.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithACL.java * /hbase/trunk/hbase-shell/src/main/ruby/hbase.rb * /hbase/trunk/hbase-shell/src/main/ruby/hbase/hbase.rb * /hbase/trunk/hbase-shell/src/main/ruby/hbase/visibility_labels.rb *
[jira] [Commented] (HBASE-9959) Remove some array copy - server side
[ https://issues.apache.org/jira/browse/HBASE-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826847#comment-13826847 ] Hudson commented on HBASE-9959: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9959 Remove some array copy - server side (nkeywal: rev 1543434) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Put.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestOperation.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java Remove some array copy - server side Key: HBASE-9959 URL: https://issues.apache.org/jira/browse/HBASE-9959 Project: HBase Issue Type: Bug Components: Protobufs, regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9959-trunk.v1.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959-trunk.v2.patch, 9959.v1.patch, 9959.v3.patch, 9959.v4.patch, 9959.v5-0.96.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch, 9959.v5.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9995) Not stopping ReplicationSink when using custom implementation for the ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826842#comment-13826842 ] Hudson commented on HBASE-9995: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9995 Not stopping ReplicationSink when using custom implementation for the ReplicationSink (anoopsamjohn: rev 1543501) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java Not stopping ReplicationSink when using custom implementation for the ReplicationSink - Key: HBASE-9995 URL: https://issues.apache.org/jira/browse/HBASE-9995 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.13 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0, 0.96.1, 0.94.15 Attachments: HBASE-9995.patch, HBASE-9995_0.94.patch Missed this in HBASE-9975. Also solving a new javadoc warning induced by HBASE-9975 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server response path
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826844#comment-13826844 ] Hudson commented on HBASE-5945: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-5945 Reduce buffer copies in IPC server response path (stack: rev 1543458) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BufferChain.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestBufferChain.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java Reduce buffer copies in IPC server response path Key: HBASE-5945 URL: https://issues.apache.org/jira/browse/HBASE-5945 Project: HBase Issue Type: Improvement Components: IPC/RPC Affects Versions: 0.95.2 Reporter: Todd Lipcon Assignee: stack Fix For: 0.96.1 Attachments: 5945-in-progress.2.1.patch, 5945-in-progress.2.patch, 5945-in-progress.patch, 5945v2.txt, 5945v4.txt, 5945v4.txt, 5945v5.txt, buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt, with_patch.png, without_patch.png The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9949) Fix the race condition between Compaction and StoreScanner.init
[ https://issues.apache.org/jira/browse/HBASE-9949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826848#comment-13826848 ] Hudson commented on HBASE-9949: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9949 Addendum takes out test infrastructure and new test (tedyu: rev 1543305) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/InjectionEvent.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/InjectionHandler.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java Fix the race condition between Compaction and StoreScanner.init --- Key: HBASE-9949 URL: https://issues.apache.org/jira/browse/HBASE-9949 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.89-fb Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Priority: Minor Fix For: 0.89-fb, 0.98.0 Attachments: 9949-0.96.addendum, 9949-trunk-v1.txt, 9949-trunk-v2.txt, 9949-trunk-v3.txt, 9949.addendum Original Estimate: 48h Remaining Estimate: 48h The StoreScanner constructor has multiple stages and there can be a race betwwen an ongoing compaction and the StoreScanner constructor where we might get the list of scanners before a compaction and seek on those scanners after the compaction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9988) DOn't use HRI#getEncodedName in the client
[ https://issues.apache.org/jira/browse/HBASE-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826843#comment-13826843 ] Hudson commented on HBASE-9988: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #844 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/844/]) HBASE-9988 DOn't use HRI#getEncodedName in the client (nkeywal: rev 1543427) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java DOn't use HRI#getEncodedName in the client -- Key: HBASE-9988 URL: https://issues.apache.org/jira/browse/HBASE-9988 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.96.1 Attachments: 9988.v1.patch, 9988.v2.patch This functions does a lazy initialisation. It cost memory and it creates a synchronisation point. -- This message was sent by Atlassian JIRA (v6.1#6144)
