[jira] [Commented] (HBASE-13599) The Example Provided in Section 69: Examples of the Documentation Does Not Compile
[
https://issues.apache.org/jira/browse/HBASE-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537627#comment-14537627
]
Lars Francke commented on HBASE-13599:
--
[~busbey] while you're at it - I'm afraid this one was committed without JIRA
reference as well
https://github.com/apache/hbase/commit/0dfb3647237523204765e95cc1c161751e8a8987
The Example Provided in Section 69: Examples of the Documentation Does Not
Compile
--
Key: HBASE-13599
URL: https://issues.apache.org/jira/browse/HBASE-13599
Project: HBase
Issue Type: Bug
Components: documentation
Affects Versions: 1.0.0
Reporter: David Newcomer
Assignee: Lars Francke
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-13599-v1.patch
I'm trying to build and run the example java code I found in the HBase
Documentation, and I'm running into several issues.
1. I don't have the code/library used in the following import:
import static com.example.hbase.Constants.*;
I don't believe it is included in any of the HBase libraries or documentation.
2. All of the methods in createOrOverwrite() that use table.getName() should
instead be using table.getTableName()
3. The interface org.apache.hadoop.hbase.client.Admin is abstract, and can't
be instantiated with a Configuration. Constructing an
org.apache.hadoop.hbase.client.HBaseAdmin would allow the code to compile,
but that constructor is deprecated.
4. I have no references to the field TABLE_NAME or CF_DEFAULT. I'm
assuming they are Strings in com.example.hbase.Constants. Perhaps those
variables should simply be copied into the the Example?
Link to the documentation section:
http://hbase.apache.org/book.html#_examples
code
package com.example.hbase.admin;
import java.io.IOException;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.io.compress.Compression.Algorithm;
import org.apache.hadoop.conf.Configuration;
import static com.example.hbase.Constants.*;
public class CreateSchema {
public static void createOrOverwrite(Admin admin, HTableDescriptor table)
throws IOException {
if (admin.tableExists(table.getName())) {
admin.disableTable(table.getName());
admin.deleteTable(table.getName());
}
admin.createTable(table);
}
public static void createSchemaTables (Configuration config) {
try {
final Admin admin = new Admin(config);
HTableDescriptor table = new
HTableDescriptor(TableName.valueOf(TABLE_NAME));
table.addFamily(new
HColumnDescriptor(CF_DEFAULT).setCompressionType(Algorithm.SNAPPY));
System.out.print(Creating table. );
createOrOverwrite(admin, table);
System.out.println( Done.);
admin.close();
} catch (Exception e) {
e.printStackTrace();
System.exit(-1);
}
}
}
/code
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13655) Deprecate duplicate getCompression methods in HColumnDescriptor
[
https://issues.apache.org/jira/browse/HBASE-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537629#comment-14537629
]
Lars Francke commented on HBASE-13655:
--
Thanks everyone for taking a look and committing!
Deprecate duplicate getCompression methods in HColumnDescriptor
---
Key: HBASE-13655
URL: https://issues.apache.org/jira/browse/HBASE-13655
Project: HBase
Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-13655-v1.patch, HBASE-13655.patch
{{HColumnDescriptor}} has {{getCompression}} and {{getCompressionType}}
methods returning the same thing. The same goes for
{{getCompactionCompression}} and {{getCompactionCompressionType}}.
This deprecates the methods without {{Type}} in 2.0.0 so they can be removed
in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13448) New Cell implementation with cached component offsets/lengths
[
https://issues.apache.org/jira/browse/HBASE-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537639#comment-14537639
]
Anoop Sam John commented on HBASE-13448:
This is my test
{code}
public class MyTest {
public static final HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
public final static byte[] row1 = Bytes.toBytes(row1);
public final static byte[] row2 = Bytes.toBytes(row2);
public final static byte[] fam = Bytes.toBytes(info);
public final static byte[] qual = Bytes.toBytes(qual);
public final static byte[] val = Bytes.toBytes(value1);
@Rule
public final TestName TEST_NAME = new TestName();
@BeforeClass
public static void setupBeforeClass() throws Exception {
TEST_UTIL.startMiniCluster(2);
}
@Test
public void testSimpleScan() throws Throwable {
TableName tableName = TableName.valueOf(TEST_NAME.getMethodName());
Table table = TEST_UTIL.createTable(tableName, fam);
Put put = new Put(row1);
put.addColumn(fam, qual, val);
table.put(put);
/*put = new Put(row2);
put.addColumn(fam, qual, val);
table.put(put);*/
Scan s = new Scan();
//s.addColumn(fam, qual);
s.setCaching(1);
ResultScanner scanner = table.getScanner(s);
Result res = scanner.next();
/*Get get = new Get(row1);
get.addColumn(fam, qual);
res = table.get(get);
System.out.println(res);*/
}
}
{code}'
Above is the Scan test.
Yes, this is a new clean table with only one cell in it.
Scan with out ExplicitColumnTracker
{code}
public short getRowLength() {
System.out.println(getRowLength + this);
new Exception().printStackTrace();
return Bytes.toShort(this.bytes, getKeyOffset());
}
{code}
Just using the above in getRowLength() method of KeyValue, am tracking the
usage.
getRowLength - 6
RegionScannerImpl.nextInternal(HRegion.java:5509)
StoreScanner.next(StoreScanner.java:509)
ScanQueryMatcher.match(ScanQueryMatcher.java:281)
Above 3 are direct calls to getRowLength
SQM.match (Line 308 and 309) calls getQualOffset() and getQualLength which in
turn calls getRowLength() on same KV 3 times!
New Cell implementation with cached component offsets/lengths
-
Key: HBASE-13448
URL: https://issues.apache.org/jira/browse/HBASE-13448
Project: HBase
Issue Type: Sub-task
Components: Scanners
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0
Attachments: HBASE-13448.patch, HBASE-13448_V2.patch, gc.png, hits.png
This can be extension to KeyValue and can be instantiated and used in read
path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13448) New Cell implementation with cached component offsets/lengths
[
https://issues.apache.org/jira/browse/HBASE-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537639#comment-14537639
]
Anoop Sam John edited comment on HBASE-13448 at 5/11/15 7:44 AM:
-
This is my test
{code}
public class MyTest {
public static final HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
public final static byte[] row1 = Bytes.toBytes(row1);
public final static byte[] row2 = Bytes.toBytes(row2);
public final static byte[] fam = Bytes.toBytes(info);
public final static byte[] qual = Bytes.toBytes(qual);
public final static byte[] val = Bytes.toBytes(value1);
@Rule
public final TestName TEST_NAME = new TestName();
@BeforeClass
public static void setupBeforeClass() throws Exception {
TEST_UTIL.startMiniCluster(2);
}
@Test
public void testSimpleScan() throws Throwable {
TableName tableName = TableName.valueOf(TEST_NAME.getMethodName());
Table table = TEST_UTIL.createTable(tableName, fam);
Put put = new Put(row1);
put.addColumn(fam, qual, val);
table.put(put);
/*put = new Put(row2);
put.addColumn(fam, qual, val);
table.put(put);*/
Scan s = new Scan();
//s.addColumn(fam, qual);
s.setCaching(1);
ResultScanner scanner = table.getScanner(s);
Result res = scanner.next();
/*Get get = new Get(row1);
get.addColumn(fam, qual);
res = table.get(get);
System.out.println(res);*/
}
}
{code}'
Above is the Scan test.
Yes, this is a new clean table with only one cell in it.
Scan with out ExplicitColumnTracker
{code}
public short getRowLength() {
new Exception(getRowLength + this).printStackTrace();
return Bytes.toShort(this.bytes, getKeyOffset());
}
{code}
Just using the above in getRowLength() method of KeyValue, am tracking the
usage.
getRowLength - 6
RegionScannerImpl.nextInternal(HRegion.java:5509)
StoreScanner.next(StoreScanner.java:509)
ScanQueryMatcher.match(ScanQueryMatcher.java:281)
Above 3 are direct calls to getRowLength
SQM.match (Line 308 and 309) calls getQualOffset() and getQualLength which in
turn calls getRowLength() on same KV 3 times!
was (Author: anoop.hbase):
This is my test
{code}
public class MyTest {
public static final HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
public final static byte[] row1 = Bytes.toBytes(row1);
public final static byte[] row2 = Bytes.toBytes(row2);
public final static byte[] fam = Bytes.toBytes(info);
public final static byte[] qual = Bytes.toBytes(qual);
public final static byte[] val = Bytes.toBytes(value1);
@Rule
public final TestName TEST_NAME = new TestName();
@BeforeClass
public static void setupBeforeClass() throws Exception {
TEST_UTIL.startMiniCluster(2);
}
@Test
public void testSimpleScan() throws Throwable {
TableName tableName = TableName.valueOf(TEST_NAME.getMethodName());
Table table = TEST_UTIL.createTable(tableName, fam);
Put put = new Put(row1);
put.addColumn(fam, qual, val);
table.put(put);
/*put = new Put(row2);
put.addColumn(fam, qual, val);
table.put(put);*/
Scan s = new Scan();
//s.addColumn(fam, qual);
s.setCaching(1);
ResultScanner scanner = table.getScanner(s);
Result res = scanner.next();
/*Get get = new Get(row1);
get.addColumn(fam, qual);
res = table.get(get);
System.out.println(res);*/
}
}
{code}'
Above is the Scan test.
Yes, this is a new clean table with only one cell in it.
Scan with out ExplicitColumnTracker
{code}
public short getRowLength() {
System.out.println(getRowLength + this);
new Exception().printStackTrace();
return Bytes.toShort(this.bytes, getKeyOffset());
}
{code}
Just using the above in getRowLength() method of KeyValue, am tracking the
usage.
getRowLength - 6
RegionScannerImpl.nextInternal(HRegion.java:5509)
StoreScanner.next(StoreScanner.java:509)
ScanQueryMatcher.match(ScanQueryMatcher.java:281)
Above 3 are direct calls to getRowLength
SQM.match (Line 308 and 309) calls getQualOffset() and getQualLength which in
turn calls getRowLength() on same KV 3 times!
New Cell implementation with cached component offsets/lengths
-
Key: HBASE-13448
URL: https://issues.apache.org/jira/browse/HBASE-13448
Project: HBase
Issue Type: Sub-task
Components: Scanners
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0
Attachments: HBASE-13448.patch, HBASE-13448_V2.patch, gc.png, hits.png
This can be extension to KeyValue and can be instantiated and used in read
path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13448) New Cell implementation with cached component offsets/lengths
[ https://issues.apache.org/jira/browse/HBASE-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537685#comment-14537685 ] Anoop Sam John commented on HBASE-13448: Doing more tests around Scan (with out ExplicitColumnTracker), it depends on the number of rows/cols also On the 1st cell in 1st row we have 6 calls to getRowLength and on subsequent cols it is 4 On the 2nd row onwards instead of 6, it will be 9 calls. Again in all these tests, the cells come from memstore alone (one scanner). When we have 1 memstore scanner + N StoreFiles, it will be more calls. Am checking the ECT case, there the calls to getRowLength() is many more because it is being indirectly used by almost all other getXXXLength/Offset methods New Cell implementation with cached component offsets/lengths - Key: HBASE-13448 URL: https://issues.apache.org/jira/browse/HBASE-13448 Project: HBase Issue Type: Sub-task Components: Scanners Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0 Attachments: HBASE-13448.patch, HBASE-13448_V2.patch, gc.png, hits.png This can be extension to KeyValue and can be instantiated and used in read path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[
https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537684#comment-14537684
]
Ayappan commented on HBASE-13338:
-
This is about the hbase shell issue.
I configured the shell to be invoked in ruby 1.8 mode
HBASE_OPTS=$HBASE_OPTS $HBASE_SHELL_OPTS
- CLASS=org.jruby.Main -X+O ${JRUBY_OPTS} ${HBASE_HOME}/bin/hirb.rb
+ CLASS=org.jruby.Main --1.8 -X+O ${JRUBY_OPTS} ${HBASE_HOME}/bin/hirb.rb
elif [ $COMMAND = hbck ] ; then
and got this error
[ayappan@soe07-vm3 bin]$ ./hbase shell
include_class is deprecated. Use java_import.
include_class is deprecated. Use java_import.
include_class is deprecated. Use java_import.
NoMethodError: undefined method `getTerminal' for Java::Jline::Terminal:Module
refresh_width at
/home/ayappan/hbase/bin/../hbase-shell/src/main/ruby/shell/formatter.rb:33
initialize at
/home/ayappan/hbase/bin/../hbase-shell/src/main/ruby/shell/formatter.rb:46
(root) at /home/ayappan/hbase/bin/../bin/hirb.rb:128
It seems like the getTerminal method doesn't exist with the jline that ships
with this version. The jline version is upated to 2.11.
And also the jruby community suggest to go with 1.9 ruby mode since 1.8 has
been EOL in the jruby world for years
https://github.com/jruby/jruby/issues/2912
Hbase to use PowerPC supported Jruby version 1.7.20
---
Key: HBASE-13338
URL: https://issues.apache.org/jira/browse/HBASE-13338
Project: HBase
Issue Type: Task
Components: dependencies, shell
Affects Versions: 1.0.0, 0.98.12
Environment: PowerPC64, PowerPC64LE
Reporter: Ayappan
Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE
libraries. The latest released 1.2.8 version has PowerPC libraries and the
jruby development version(1.7.20--SNAPSHOT) has been updated to make use of
this version. Hbase still uses much older jruby 1.6.8 version which don't
have the native libraries and this affects the Hbase shell in PowerPCs. So
Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to
support PowerPC.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-1989) Admin (et al.) not accurate with Column vs. Column-Family usage
[ https://issues.apache.org/jira/browse/HBASE-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537623#comment-14537623 ] Lars Francke commented on HBASE-1989: - Thanks stack! [~busbey] I do not object, thank you for catching this. Admin (et al.) not accurate with Column vs. Column-Family usage --- Key: HBASE-1989 URL: https://issues.apache.org/jira/browse/HBASE-1989 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.20.1, 0.90.1 Reporter: Doug Meil Assignee: Lars Francke Priority: Minor Fix For: 2.0.0 Attachments: HBASE-1989-v1.patch, HBASE-1989.patch, hbase1989.patch Consider the classes Admin and HColumnDescriptor. HColumnDescriptor is really referring to a column family and not a column (i.e., family:qualifer). Likewise, in Admin there is a method called addColumn that takes an HColumnDescriptor instance. I labeled this a bug in the sense that it produces conceptual confusion because there is a big difference between a column and column-family in HBase and these terms should be used consistently. The code works, though. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13611) update clover to work for current versions
[ https://issues.apache.org/jira/browse/HBASE-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13611: Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 1.0.2 2.0.0 Status: Resolved (was: Patch Available) update clover to work for current versions -- Key: HBASE-13611 URL: https://issues.apache.org/jira/browse/HBASE-13611 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13611.1.patch.txt, HBASE-13611.2.patch.txt our clover profile uses a super old version of the clover plugin, sufficiently old that it requires maven 2. update to use ~4.x clover version and make sure things work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13510) Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL
[ https://issues.apache.org/jira/browse/HBASE-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537655#comment-14537655 ] Anoop Sam John commented on HBASE-13510: We have 2 impls for BloomFilter interface. ie. CompoundBloomFilter and ByteBloomFilter. We are not at all using ByteBloomFilter.(Which is having diff way of dealing with ROW_COL bloom key). This was there in use for HFileV1 I believe. So we can do those cleanup also. We no loner need ByteBloomFilter as is a BloomFilter. This class was having some util methods which now moved to a Util class (static methods)... Still ByteBloomFilter is in use and used to store some state(in this patch) .Can we avoid it fully ? Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL - Key: HBASE-13510 URL: https://issues.apache.org/jira/browse/HBASE-13510 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-13510_1.patch, HBASE-13510_2.patch In order to address the comments over in HBASE-10800 related to comparing Cell with a serialized KV's key we had some need for that in Bloom filters. After discussing with Anoop, we found that it may be possible to remove/modify some of the APIs in the BloomFilter interfaces and for doing that we can purge ByteBloomFilter. I read the code and found that ByteBloomFilter was getting used in V1 version only. Now as it is obsolete we can remove this code and move some of the static APIs in ByteBloomFilter to some other util class or bloom related classes which will help us in refactoring the code too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13650) Tests failing for 0.94.x against hadoop 2.x
[
https://issues.apache.org/jira/browse/HBASE-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537688#comment-14537688
]
Dima Spivak commented on HBASE-13650:
-
Looks like the same issue discussed by [~lhofhansl] and [~tedyu] a while back
[on the mailing
list|http://mail-archives.us.apache.org/mod_mbox/hbase-user/201401.mbox/%3ccalte62x4us_fl24uzwdv-ah0bmgn8awfzr8szbsbqkd0h3s...@mail.gmail.com%3E].
I can reproduce the same failures with freshly-generated .proto files and
confirmed that
{{com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar}} is the only
protobuf JAR on the classpath.
Tests failing for 0.94.x against hadoop 2.x
---
Key: HBASE-13650
URL: https://issues.apache.org/jira/browse/HBASE-13650
Project: HBase
Issue Type: Test
Affects Versions: 0.94.18
Reporter: Kuldeep
After changing hadoop.version in hadoop-2.0 profile and regenrating protobuf
files as mentioned here
https://hbase.apache.org/book.html#hadoop2.hbase_0.94
tests agains hadoop 2.4.0 fail with following expceptions
Results :
Tests in error:
testFailingCreateTable(org.apache.hadoop.hbase.rest.client.TestRemoteAdminRetries):
This is supposed to be overridden by subclasses.
testGetScanner(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries):
This is supposed to be overridden by subclasses.
testSingleRowPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries):
This is supposed to be overridden by subclasses.
testMultiRowPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries):
This is supposed to be overridden by subclasses.
testCheckAndPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries):
This is supposed to be overridden by subclasses.
testCheckAndDelete(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries):
This is supposed to be overridden by subclasses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13510) Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL
[ https://issues.apache.org/jira/browse/HBASE-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537649#comment-14537649 ] Anoop Sam John commented on HBASE-13510: Yes Stack. When the bloom is ROW_COL, we make the bloom key (bytes) by creating a KeyValue, key kind of bytes representation and use that. So every time we have to make this bloom key bytes, we will create a KV object out of this row, col and take that key portion bytes. So already the obj creation was there and the API used to return the key bytes. Now this is changed to return KV type. (We use that and pass to CellComparator) As I asked above, can we just use this util way within the used area only? Just avoid this API from interface/class? Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL - Key: HBASE-13510 URL: https://issues.apache.org/jira/browse/HBASE-13510 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-13510_1.patch, HBASE-13510_2.patch In order to address the comments over in HBASE-10800 related to comparing Cell with a serialized KV's key we had some need for that in Bloom filters. After discussing with Anoop, we found that it may be possible to remove/modify some of the APIs in the BloomFilter interfaces and for doing that we can purge ByteBloomFilter. I read the code and found that ByteBloomFilter was getting used in V1 version only. Now as it is obsolete we can remove this code and move some of the static APIs in ByteBloomFilter to some other util class or bloom related classes which will help us in refactoring the code too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13657) Improve test run time
Ashish Singhi created HBASE-13657:
-
Summary: Improve test run time
Key: HBASE-13657
URL: https://issues.apache.org/jira/browse/HBASE-13657
Project: HBase
Issue Type: Umbrella
Components: test
Reporter: Ashish Singhi
In some tests we are doing some operations in {{@Before}} and {{@After}}
annotated methods which are not really required to be done before and after
every test run, instead we can move them in {{@BeforeClass}} and
{{@AfterClass}} annotated methods respectively and hence improve the test run
time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13658: -- Attachment: HBASE-13658.patch Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13510) Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL
[ https://issues.apache.org/jira/browse/HBASE-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537704#comment-14537704 ] ramkrishna.s.vasudevan commented on HBASE-13510: bq.As I asked above, can we just use this util way within the used area only? Just avoid this API from interface/class? Okie we can move this to a local area. bq.till ByteBloomFilter is in use and used to store some state(in this patch) .Can we avoid it fully ? May be we can rename Bytebloomfilter to something else? The CompoundBloomFilter needs to have a bloom chunk to actually do this bloom filter related things. [~stack] bq.Sounds good. What we going to use instead? We are using in this patch compare(Cell, cell). bq.We can't have Blooms use Cells? There'd be a copy of the row bytes if Cell or something? Yes we will always have copy of byte[] for the ROW_COL case of blooms. For the ROW bloom only the row part of the cell is enough. So for such bloom we could use the Bytes.BYTES_RAWCOMPARATOR. Only for ROW_COL we would need this Cell based comparison - the bloom being formed from the row and col of the Keyvalue. bq.We have to do this? We only use the row and column portions? Or we hash whole key part in bloom? Only row and col portion but we form that as a keyvalue and use the key part formed from such a keyvalue. bq.Why is that when bloom is for row/column? We put whole key in there? See the above reply. bq.Can you redo the above We were using BytebloomFilter in V1, since V2 everything is a CompoundBloom. So we are removing the usage of ByteBloomfilter here (as a BloomFilter type). Both the blooms were dealing with byte[] only. Now this patch tries to deal only with CompoundBloomFilter with ROW and ROW_COL as the type and the ROW_COL depends on the CellComparator whereas the ROW depends on Bytes.Byte_RAWCOMPARATOR. Refactor Bloom filters to make use of Cell Comparators in case of ROW_COL - Key: HBASE-13510 URL: https://issues.apache.org/jira/browse/HBASE-13510 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-13510_1.patch, HBASE-13510_2.patch In order to address the comments over in HBASE-10800 related to comparing Cell with a serialized KV's key we had some need for that in Bloom filters. After discussing with Anoop, we found that it may be possible to remove/modify some of the APIs in the BloomFilter interfaces and for doing that we can purge ByteBloomFilter. I read the code and found that ByteBloomFilter was getting used in V1 version only. Now as it is obsolete we can remove this code and move some of the static APIs in ByteBloomFilter to some other util class or bloom related classes which will help us in refactoring the code too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13611) update clover to work for current versions
[ https://issues.apache.org/jira/browse/HBASE-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537723#comment-14537723 ] Hudson commented on HBASE-13611: SUCCESS: Integrated in HBase-1.2 #70 (See [https://builds.apache.org/job/HBase-1.2/70/]) HBASE-13611 update clover profile to work with clover 4.x and maven 3. (busbey: rev 6213fa2fce452dace06c0ec0396cb5fe420afe4d) * hbase-server/src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java * pom.xml * hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java update clover to work for current versions -- Key: HBASE-13611 URL: https://issues.apache.org/jira/browse/HBASE-13611 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13611.1.patch.txt, HBASE-13611.2.patch.txt our clover profile uses a super old version of the clover plugin, sufficiently old that it requires maven 2. update to use ~4.x clover version and make sure things work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13655) Deprecate duplicate getCompression methods in HColumnDescriptor
[
https://issues.apache.org/jira/browse/HBASE-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537745#comment-14537745
]
Hudson commented on HBASE-13655:
SUCCESS: Integrated in HBase-TRUNK #6471 (See
[https://builds.apache.org/job/HBase-TRUNK/6471/])
HBASE-13655 Deprecate duplicate getCompression methods in HColumnDescriptor
(Lars Francke lars.fran...@gmail.com) (stack: rev
8e5a1832562aa9b5c12ddcc1949cbcea33ae85cf)
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java
Deprecate duplicate getCompression methods in HColumnDescriptor
---
Key: HBASE-13655
URL: https://issues.apache.org/jira/browse/HBASE-13655
Project: HBase
Issue Type: Improvement
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-13655-v1.patch, HBASE-13655.patch
{{HColumnDescriptor}} has {{getCompression}} and {{getCompressionType}}
methods returning the same thing. The same goes for
{{getCompactionCompression}} and {{getCompactionCompressionType}}.
This deprecates the methods without {{Type}} in 2.0.0 so they can be removed
in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13658) Improve the test run time for TestAccessController* classes
Ashish Singhi created HBASE-13658: - Summary: Improve the test run time for TestAccessController* classes Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[
https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537818#comment-14537818
]
Hadoop QA commented on HBASE-13580:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12730770/HBASE-13580-v4.patch
against master branch at commit 9aeafe30b7d932e562f803fd071812cd27aebaf8.
ATTACHMENT ID: 12730770
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ $LOG.warn(Unable to scan region= + r.getRegionNameAsString() +
start key is empty. + e.message)
+ return admin.getClusterStatus().getMaster().getHostname(),
admin.getClusterStatus().getMaster().getPort()
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders.testSeekingOnSample(TestDataBlockEncoders.java:206)
at
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWriteInternals(TestCacheOnWrite.java:271)
at
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWrite(TestCacheOnWrite.java:472)
at
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding.testCrazyRandomChanges(TestChangingEncoding.java:248)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14003//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14003//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14003//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14003//console
This message is automatically generated.
region_mover.rb broken with TypeError: no public constructors for
Java::OrgApacheHadoopHbaseClient::HTable
--
Key: HBASE-13580
URL: https://issues.apache.org/jira/browse/HBASE-13580
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 2.0.0
Environment: x86_64 GNU/Linux
Reporter: Samir Ahmic
Assignee: Samir Ahmic
Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch,
HBASE-13580-v4.patch, HBASE-13580.patch
I was testing region_mover.rb on master branch in distributed cluster and
hit this error. I have fixed this by using Connection#getTable instead of
HTable but look like this script needs some additional work:
1. Remove master server from region move targets list
2. --exclude=FILE option is not working for me
I will try to get this script in functional state if there is no objections ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13611) update clover to work for current versions
[ https://issues.apache.org/jira/browse/HBASE-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537744#comment-14537744 ] Hudson commented on HBASE-13611: SUCCESS: Integrated in HBase-TRUNK #6471 (See [https://builds.apache.org/job/HBase-TRUNK/6471/]) HBASE-13611 update clover profile to work with clover 4.x and maven 3. (busbey: rev 9aeafe30b7d932e562f803fd071812cd27aebaf8) * hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java * hbase-server/src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java * pom.xml update clover to work for current versions -- Key: HBASE-13611 URL: https://issues.apache.org/jira/browse/HBASE-13611 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13611.1.patch.txt, HBASE-13611.2.patch.txt our clover profile uses a super old version of the clover plugin, sufficiently old that it requires maven 2. update to use ~4.x clover version and make sure things work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13611) update clover to work for current versions
[ https://issues.apache.org/jira/browse/HBASE-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537801#comment-14537801 ] Hudson commented on HBASE-13611: SUCCESS: Integrated in HBase-1.1 #477 (See [https://builds.apache.org/job/HBase-1.1/477/]) HBASE-13611 update clover profile to work with clover 4.x and maven 3. (busbey: rev 44fdd02d80135930b750dd3a870d13625c756629) * hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java * pom.xml * hbase-server/src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java update clover to work for current versions -- Key: HBASE-13611 URL: https://issues.apache.org/jira/browse/HBASE-13611 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13611.1.patch.txt, HBASE-13611.2.patch.txt our clover profile uses a super old version of the clover plugin, sufficiently old that it requires maven 2. update to use ~4.x clover version and make sure things work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[ https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samir Ahmic updated HBASE-13580: Status: Patch Available (was: Open) Just tested on master branch All works fine. region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable -- Key: HBASE-13580 URL: https://issues.apache.org/jira/browse/HBASE-13580 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Environment: x86_64 GNU/Linux Reporter: Samir Ahmic Assignee: Samir Ahmic Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch, HBASE-13580-v4.patch, HBASE-13580.patch I was testing region_mover.rb on master branch in distributed cluster and hit this error. I have fixed this by using Connection#getTable instead of HTable but look like this script needs some additional work: 1. Remove master server from region move targets list 2. --exclude=FILE option is not working for me I will try to get this script in functional state if there is no objections ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13549) metric for failed lease recovery
[ https://issues.apache.org/jira/browse/HBASE-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Singh Chouhan updated HBASE-13549: --- Attachment: HBASE-13549-0.98.patch Thought i'd give this a go. [~busbey] is this patch ok? metric for failed lease recovery Key: HBASE-13549 URL: https://issues.apache.org/jira/browse/HBASE-13549 Project: HBase Issue Type: Sub-task Components: Usability Reporter: Sean Busbey Attachments: HBASE-13549-0.98.patch we should publish metrics for * lease recovery failures * lease recovery timeout right now you have to examine the logs of individual RS to get this information and aggregating it is painful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13611) update clover to work for current versions
[ https://issues.apache.org/jira/browse/HBASE-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537789#comment-14537789 ] Hudson commented on HBASE-13611: SUCCESS: Integrated in HBase-1.0 #907 (See [https://builds.apache.org/job/HBase-1.0/907/]) HBASE-13611 update clover profile to work with clover 4.x and maven 3. (busbey: rev 549319a4aa57b9732a62c4275d6f00a677e2ce81) * hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java * pom.xml * hbase-server/src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java update clover to work for current versions -- Key: HBASE-13611 URL: https://issues.apache.org/jira/browse/HBASE-13611 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13611.1.patch.txt, HBASE-13611.2.patch.txt our clover profile uses a super old version of the clover plugin, sufficiently old that it requires maven 2. update to use ~4.x clover version and make sure things work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[ https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samir Ahmic updated HBASE-13580: Status: Open (was: Patch Available) I need to fix this lines length. region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable -- Key: HBASE-13580 URL: https://issues.apache.org/jira/browse/HBASE-13580 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Environment: x86_64 GNU/Linux Reporter: Samir Ahmic Assignee: Samir Ahmic Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch, HBASE-13580-v4.patch, HBASE-13580.patch I was testing region_mover.rb on master branch in distributed cluster and hit this error. I have fixed this by using Connection#getTable instead of HTable but look like this script needs some additional work: 1. Remove master server from region move targets list 2. --exclude=FILE option is not working for me I will try to get this script in functional state if there is no objections ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13217) Procedure fails due to ZK issue
[
https://issues.apache.org/jira/browse/HBASE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537852#comment-14537852
]
Stephen Yuan Jiang commented on HBASE-13217:
sounds good. thanks for the explanation.
+1
Procedure fails due to ZK issue
---
Key: HBASE-13217
URL: https://issues.apache.org/jira/browse/HBASE-13217
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Reporter: ramkrishna.s.vasudevan
Assignee: Stephen Yuan Jiang
Attachments: HBASE-13217-v2.patch, HBASE-13217.patch
When ever I try to flush explicitly in the trunk code the flush procedure
fails due to ZK issue
{code}
ERROR: org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable
via
stobdtserver3,16040,1426172670959:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.procedure.Procedure.isCompleted(Procedure.java:368)
at
org.apache.hadoop.hbase.procedure.flush.MasterFlushTableProcedureManager.isProcedureDone(MasterFlushTableProcedureManager.java:196)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isProcedureDone(MasterRpcServices.java:905)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:47019)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2073)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.procedure.Subprocedure.cancel(Subprocedure.java:273)
at
org.apache.hadoop.hbase.procedure.ProcedureMember.controllerConnectionFailure(ProcedureMember.java:225)
at
org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.sendMemberAcquired(ZKProcedureMemberRpcs.java:254)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:166)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
{code}
Once this occurs, even on restart of the RS the RS becomes unusable. I have
verified that the ZK remains intact and there is no problem with it. a bit
older version of trunk ( 3months) does not have this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13217) Procedure fails due to ZK issue
[
https://issues.apache.org/jira/browse/HBASE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537851#comment-14537851
]
Stephen Yuan Jiang commented on HBASE-13217:
sounds good. thanks for the explanation.
+1
Procedure fails due to ZK issue
---
Key: HBASE-13217
URL: https://issues.apache.org/jira/browse/HBASE-13217
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Reporter: ramkrishna.s.vasudevan
Assignee: Stephen Yuan Jiang
Attachments: HBASE-13217-v2.patch, HBASE-13217.patch
When ever I try to flush explicitly in the trunk code the flush procedure
fails due to ZK issue
{code}
ERROR: org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable
via
stobdtserver3,16040,1426172670959:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.procedure.Procedure.isCompleted(Procedure.java:368)
at
org.apache.hadoop.hbase.procedure.flush.MasterFlushTableProcedureManager.isProcedureDone(MasterFlushTableProcedureManager.java:196)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isProcedureDone(MasterRpcServices.java:905)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:47019)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2073)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.procedure.Subprocedure.cancel(Subprocedure.java:273)
at
org.apache.hadoop.hbase.procedure.ProcedureMember.controllerConnectionFailure(ProcedureMember.java:225)
at
org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.sendMemberAcquired(ZKProcedureMemberRpcs.java:254)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:166)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
{code}
Once this occurs, even on restart of the RS the RS becomes unusable. I have
verified that the ZK remains intact and there is no problem with it. a bit
older version of trunk ( 3months) does not have this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[ https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samir Ahmic updated HBASE-13580: Status: Patch Available (was: Open) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable -- Key: HBASE-13580 URL: https://issues.apache.org/jira/browse/HBASE-13580 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Environment: x86_64 GNU/Linux Reporter: Samir Ahmic Assignee: Samir Ahmic Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch, HBASE-13580-v4.patch, HBASE-13580-v5.patch, HBASE-13580.patch I was testing region_mover.rb on master branch in distributed cluster and hit this error. I have fixed this by using Connection#getTable instead of HTable but look like this script needs some additional work: 1. Remove master server from region move targets list 2. --exclude=FILE option is not working for me I will try to get this script in functional state if there is no objections ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13658: -- Status: Patch Available (was: Open) Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11830) TestReplicationThrottler.testThrottling failed on virtual boxes
[ https://issues.apache.org/jira/browse/HBASE-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-11830: --- Affects Version/s: 1.2.0 1.1.0 2.0.0 Status: Patch Available (was: Open) TestReplicationThrottler.testThrottling failed on virtual boxes --- Key: HBASE-11830 URL: https://issues.apache.org/jira/browse/HBASE-11830 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 1.1.0, 1.2.0 Environment: kvm with Centos 6.5, openjdk1.7 Reporter: Sergey Soldatov Assignee: Stephen Yuan Jiang Priority: Minor Attachments: HBASE-11830.patch during test runs TestReplicationThrottler.testThrottling sometimes fails with assertion testThrottling(org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler) Time elapsed: 0.229 sec FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler.testThrottling(TestReplicationThrottler.java:69) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11830) TestReplicationThrottler.testThrottling failed on virtual boxes
[ https://issues.apache.org/jira/browse/HBASE-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537857#comment-14537857 ] Stephen Yuan Jiang commented on HBASE-11830: Attach a patch to relax the strict restriction that we expect certain operation run under 1ms. I use 75% of expected value - i think as long as throttling happens, a little short sleep time does not matter much. TestReplicationThrottler.testThrottling failed on virtual boxes --- Key: HBASE-11830 URL: https://issues.apache.org/jira/browse/HBASE-11830 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 1.1.0, 1.2.0 Environment: kvm with Centos 6.5, openjdk1.7 Reporter: Sergey Soldatov Assignee: Stephen Yuan Jiang Priority: Minor Attachments: HBASE-11830.patch during test runs TestReplicationThrottler.testThrottling sometimes fails with assertion testThrottling(org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler) Time elapsed: 0.229 sec FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler.testThrottling(TestReplicationThrottler.java:69) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[ https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samir Ahmic updated HBASE-13580: Attachment: HBASE-13580-v5.patch New patch with lines length fix. region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable -- Key: HBASE-13580 URL: https://issues.apache.org/jira/browse/HBASE-13580 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Environment: x86_64 GNU/Linux Reporter: Samir Ahmic Assignee: Samir Ahmic Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch, HBASE-13580-v4.patch, HBASE-13580-v5.patch, HBASE-13580.patch I was testing region_mover.rb on master branch in distributed cluster and hit this error. I have fixed this by using Connection#getTable instead of HTable but look like this script needs some additional work: 1. Remove master server from region move targets list 2. --exclude=FILE option is not working for me I will try to get this script in functional state if there is no objections ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11830) TestReplicationThrottler.testThrottling failed on virtual boxes
[ https://issues.apache.org/jira/browse/HBASE-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-11830: --- Attachment: HBASE-11830.patch TestReplicationThrottler.testThrottling failed on virtual boxes --- Key: HBASE-11830 URL: https://issues.apache.org/jira/browse/HBASE-11830 Project: HBase Issue Type: Bug Components: test Environment: kvm with Centos 6.5, openjdk1.7 Reporter: Sergey Soldatov Assignee: Stephen Yuan Jiang Priority: Minor Attachments: HBASE-11830.patch during test runs TestReplicationThrottler.testThrottling sometimes fails with assertion testThrottling(org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler) Time elapsed: 0.229 sec FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler.testThrottling(TestReplicationThrottler.java:69) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13217) Procedure fails due to ZK issue
[
https://issues.apache.org/jira/browse/HBASE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-13217:
---
Assignee: Jerry He (was: Stephen Yuan Jiang)
Procedure fails due to ZK issue
---
Key: HBASE-13217
URL: https://issues.apache.org/jira/browse/HBASE-13217
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Reporter: ramkrishna.s.vasudevan
Assignee: Jerry He
Attachments: HBASE-13217-v2.patch, HBASE-13217.patch
When ever I try to flush explicitly in the trunk code the flush procedure
fails due to ZK issue
{code}
ERROR: org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable
via
stobdtserver3,16040,1426172670959:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.procedure.Procedure.isCompleted(Procedure.java:368)
at
org.apache.hadoop.hbase.procedure.flush.MasterFlushTableProcedureManager.isProcedureDone(MasterFlushTableProcedureManager.java:196)
at
org.apache.hadoop.hbase.master.MasterRpcServices.isProcedureDone(MasterRpcServices.java:905)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:47019)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2073)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/hbase/flush-table-proc/acquired/TestTable/stobdtserver3,16040,1426172670959
at
org.apache.hadoop.hbase.procedure.Subprocedure.cancel(Subprocedure.java:273)
at
org.apache.hadoop.hbase.procedure.ProcedureMember.controllerConnectionFailure(ProcedureMember.java:225)
at
org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.sendMemberAcquired(ZKProcedureMemberRpcs.java:254)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:166)
at
org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
... 1 more
{code}
Once this occurs, even on restart of the RS the RS becomes unusable. I have
verified that the ZK remains intact and there is no problem with it. a bit
older version of trunk ( 3months) does not have this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11830) TestReplicationThrottler.testThrottling failed on virtual boxes
[
https://issues.apache.org/jira/browse/HBASE-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537983#comment-14537983
]
Hadoop QA commented on HBASE-11830:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731911/HBASE-11830.patch
against master branch at commit 9aeafe30b7d932e562f803fd071812cd27aebaf8.
ATTACHMENT ID: 12731911
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14005//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14005//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14005//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14005//console
This message is automatically generated.
TestReplicationThrottler.testThrottling failed on virtual boxes
---
Key: HBASE-11830
URL: https://issues.apache.org/jira/browse/HBASE-11830
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: kvm with Centos 6.5, openjdk1.7
Reporter: Sergey Soldatov
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-11830.patch
during test runs TestReplicationThrottler.testThrottling sometimes fails with
assertion
testThrottling(org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler)
Time elapsed: 0.229 sec FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler.testThrottling(TestReplicationThrottler.java:69)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13659) Improve test run time for TestMetaWithReplicas class
[ https://issues.apache.org/jira/browse/HBASE-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13659: -- Attachment: HBASE-13659.patch Improve test run time for TestMetaWithReplicas class Key: HBASE-13659 URL: https://issues.apache.org/jira/browse/HBASE-13659 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: HBASE-13659.patch In TestMetaWithReplicas, start and shutdown of mini cluster is done at start and end of every test in that class respectively, which makes the test class to take more time to complete. Instead we can start and stop the mini cluster only once per the class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537985#comment-14537985 ] Ted Yu commented on HBASE-13658: How much improvement in test runtime does the patch achieve ? Thanks Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13659) Improve test run time for TestMetaWithReplicas class
Ashish Singhi created HBASE-13659: - Summary: Improve test run time for TestMetaWithReplicas class Key: HBASE-13659 URL: https://issues.apache.org/jira/browse/HBASE-13659 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi In TestMetaWithReplicas, start and shutdown of mini cluster is done at start and end of every test in that class respectively, which makes the test class to take more time to complete. Instead we can start and stop the mini cluster only once per the class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13657) Improve test run time
[
https://issues.apache.org/jira/browse/HBASE-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HBASE-13657:
--
Labels: beginner beginners (was: )
Improve test run time
-
Key: HBASE-13657
URL: https://issues.apache.org/jira/browse/HBASE-13657
Project: HBase
Issue Type: Umbrella
Components: test
Reporter: Ashish Singhi
Labels: beginner, beginners
In some tests we are doing some operations in {{@Before}} and {{@After}}
annotated methods which are not really required to be done before and after
every test run, instead we can move them in {{@BeforeClass}} and
{{@AfterClass}} annotated methods respectively and hence improve the test run
time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[
https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537999#comment-14537999
]
Hadoop QA commented on HBASE-13580:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731910/HBASE-13580-v5.patch
against master branch at commit 9aeafe30b7d932e562f803fd071812cd27aebaf8.
ATTACHMENT ID: 12731910
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14004//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14004//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14004//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14004//console
This message is automatically generated.
region_mover.rb broken with TypeError: no public constructors for
Java::OrgApacheHadoopHbaseClient::HTable
--
Key: HBASE-13580
URL: https://issues.apache.org/jira/browse/HBASE-13580
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 2.0.0
Environment: x86_64 GNU/Linux
Reporter: Samir Ahmic
Assignee: Samir Ahmic
Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch,
HBASE-13580-v4.patch, HBASE-13580-v5.patch, HBASE-13580.patch
I was testing region_mover.rb on master branch in distributed cluster and
hit this error. I have fixed this by using Connection#getTable instead of
HTable but look like this script needs some additional work:
1. Remove master server from region move targets list
2. --exclude=FILE option is not working for me
I will try to get this script in functional state if there is no objections ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[
https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537980#comment-14537980
]
Ted Yu commented on HBASE-13580:
Looks good.
{code}
123 rescue java.lang.NullPointerException,
java.lang.NullPointerException = e
{code}
One occurrence of NPE should be enough, right ?
region_mover.rb broken with TypeError: no public constructors for
Java::OrgApacheHadoopHbaseClient::HTable
--
Key: HBASE-13580
URL: https://issues.apache.org/jira/browse/HBASE-13580
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 2.0.0
Environment: x86_64 GNU/Linux
Reporter: Samir Ahmic
Assignee: Samir Ahmic
Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch,
HBASE-13580-v4.patch, HBASE-13580-v5.patch, HBASE-13580.patch
I was testing region_mover.rb on master branch in distributed cluster and
hit this error. I have fixed this by using Connection#getTable instead of
HTable but look like this script needs some additional work:
1. Remove master server from region move targets list
2. --exclude=FILE option is not working for me
I will try to get this script in functional state if there is no objections ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13658) Improve the test run time for TestAccessController* classes
[
https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537987#comment-14537987
]
Ashish Singhi commented on HBASE-13658:
---
Locally I was able to see 3x times. But would like to see what Hadoop QA has
got to say.
But Hadoop QA run,
https://builds.apache.org/job/PreCommit-HBASE-Build/14006/console got
terminated with the following error.
{noformat}
/home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests
-Pcompile-protobuf -X -DHBasePatchProcess
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/patchprocess/patchProtocErrors.txt
21
==
There are Protoc compilation errors.
==
[ERROR] Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:2.5.1:protoc (compile-protoc) on project
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: protoc version
is 'libprotoc 2.4.1', expected version is '2.5.0' - [Help 1]
{noformat}
something weird!
Improve the test run time for TestAccessController* classes
---
Key: HBASE-13658
URL: https://issues.apache.org/jira/browse/HBASE-13658
Project: HBase
Issue Type: Sub-task
Components: test
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Attachments: HBASE-13658.patch
Improve the test run time for TestAccessController* classes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13659) Improve test run time for TestMetaWithReplicas class
[ https://issues.apache.org/jira/browse/HBASE-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13659: -- Status: Patch Available (was: Open) Improve test run time for TestMetaWithReplicas class Key: HBASE-13659 URL: https://issues.apache.org/jira/browse/HBASE-13659 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: HBASE-13659.patch In TestMetaWithReplicas, start and shutdown of mini cluster is done at start and end of every test in that class respectively, which makes the test class to take more time to complete. Instead we can start and stop the mini cluster only once per the class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13658: -- Attachment: 13658.patch Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: 13658.patch, HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11830) TestReplicationThrottler.testThrottling failed on virtual boxes
[ https://issues.apache.org/jira/browse/HBASE-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538076#comment-14538076 ] Stephen Yuan Jiang commented on HBASE-11830: The org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2 failed; it does NOT related to this patch. TestReplicationThrottler.testThrottling failed on virtual boxes --- Key: HBASE-11830 URL: https://issues.apache.org/jira/browse/HBASE-11830 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 1.1.0, 1.2.0 Environment: kvm with Centos 6.5, openjdk1.7 Reporter: Sergey Soldatov Assignee: Stephen Yuan Jiang Priority: Minor Attachments: HBASE-11830.patch during test runs TestReplicationThrottler.testThrottling sometimes fails with assertion testThrottling(org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler) Time elapsed: 0.229 sec FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.hbase.replication.regionserver.TestReplicationThrottler.testThrottling(TestReplicationThrottler.java:69) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13662) RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a retriable failure
[
https://issues.apache.org/jira/browse/HBASE-13662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13662:
Attachment: HBASE-13662-v0.patch
RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a
retriable failure
---
Key: HBASE-13662
URL: https://issues.apache.org/jira/browse/HBASE-13662
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.0.1, 0.94.27, 0.98.10.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13662-v0.patch
while fixing HBASE-13651 I noticed that if we have a failure inside the
RSRpcService.scan(), when the request has a hasNextCallSeq() the nextCallSeq
is incremented and not rolledback, which means that the client retry will
send a request with a nextCallSeq not up to date, which result in an
OutOfOrderScannerNextException.
{code}
if (rows 0) {
if (request.hasNextCallSeq()) {
if (request.getNextCallSeq() != rsh.nextCallSeq) {
throw new OutOfOrderScannerNextException(...)
}
// Increment the nextCallSeq value which is the next expected from client.
rsh.nextCallSeq++;
}
}
try {
...scan code...
}
{code}
after the scanner heartbeat patches HBASE-13090, we seems to be able to
recover from that OutOfOrder exception, but the error show up anyway.
After a discussion with [~saint@gmail.com] we ended up saying that
decrementing the callSeq on exception seems to be fine. but we had the open
question about having that nextCallSeq to be atomic, if that was supposed to
prevent concurrent requests with the same id. any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13661) Correct @Public annotation marking discovered in 1.1.0RC0
[ https://issues.apache.org/jira/browse/HBASE-13661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538320#comment-14538320 ] Enis Soztutar commented on HBASE-13661: --- see https://people.apache.org/~enis/1.0.1_1.1.0RC0_compat_report.html#Binary_Removed Correct @Public annotation marking discovered in 1.1.0RC0 - Key: HBASE-13661 URL: https://issues.apache.org/jira/browse/HBASE-13661 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0, 1.2.0 Over on the 1.1.0RC0 VOTE thread, Enis discovered some errors in InterfaceAudience annotations. Let's fix them. Filed by [~ndimiduk] on [~enis]'s behalf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
[
https://issues.apache.org/jira/browse/HBASE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13663:
---
Attachment: 13663-v1.txt
HMaster fails to restart 'HMaster: Failed to become active master'
--
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
Assignee: Ted Yu
Attachments: 13663-v1.txt
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13338) Hbase to use PowerPC supported Jruby version 1.7.20
[ https://issues.apache.org/jira/browse/HBASE-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538498#comment-14538498 ] Sean Busbey commented on HBASE-13338: - unfortunately, we're stuck with Ruby 1.8 until HBase 2.0. Can we override the version of jline? Hbase to use PowerPC supported Jruby version 1.7.20 --- Key: HBASE-13338 URL: https://issues.apache.org/jira/browse/HBASE-13338 Project: HBase Issue Type: Task Components: dependencies, shell Affects Versions: 1.0.0, 0.98.12 Environment: PowerPC64, PowerPC64LE Reporter: Ayappan Older versions of jffi (till 1.2.7) don't have native PPC64 PPC64LE libraries. The latest released 1.2.8 version has PowerPC libraries and the jruby development version(1.7.20--SNAPSHOT) has been updated to make use of this version. Hbase still uses much older jruby 1.6.8 version which don't have the native libraries and this affects the Hbase shell in PowerPCs. So Hbase needs to be updated to make use of the upcoming Jruby release 1.7.20 to support PowerPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13637) branch-1.1 does not build against hadoop-2.2.
[
https://issues.apache.org/jira/browse/HBASE-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13637:
---
Assignee: zhangduo
branch-1.1 does not build against hadoop-2.2.
-
Key: HBASE-13637
URL: https://issues.apache.org/jira/browse/HBASE-13637
Project: HBase
Issue Type: Bug
Reporter: Nick Dimiduk
Assignee: zhangduo
Fix For: 1.1.0
Attachments: HBASE-13637-branch-1.1.patch
From RC0 VOTE thread,
{quote}
The build is broken with Hadoop-2.2 because mini-kdc is not found:
\[ERROR\] Failed to execute goal on project hbase-server: Could not resolve
dependencies for project org.apache.hbase:hbase-server:jar:1.1.0: Could not
find artifact org.apache.hadoop:hadoop-minikdc:jar:2.2
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13651) Handle StoreFileScanner FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538096#comment-14538096 ] Matteo Bertozzi commented on HBASE-13651: - This patch was posted before our discussion. I haven't found any prefetch or similar thing in the scanner. so decrementing that in the finally sounds good to me. I'll open a new jira for this thing since it is a separate issue, and I'll add some testing around failures inside the scanner. https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L212 Handle StoreFileScanner FileNotFoundException - Key: HBASE-13651 URL: https://issues.apache.org/jira/browse/HBASE-13651 Project: HBase Issue Type: Bug Affects Versions: 0.94.27, 0.98.10.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13651-0.94-draft.patch, HBASE-13651-draft.patch Example: * Machine-1 is serving Region-X and start compaction * Machine-1 goes in GC pause * Region-X gets reassigned to Machine-2 * Machine-1 exit from the GC pause * Machine-1 (re)moves the compacted files * Machine-1 get the lease expired and shutdown Machine-2 has now tons of FileNotFoundException on scan. If we reassign the region everything is ok, because we pickup the files compacted by Machine-1. This problem doesn't happen in the new code 1.0+ (i think but I haven't checked, it may be 1.1) where we write on the WAL the compaction event before (re)moving the files. A workaround is handling FileNotFoundException and refresh the store files, or shutdown the region and reassign. the first one is easy in 1.0+ the second one requires more work because at the moment we don't have the code to notify the master that the RS is closing the region, alternatively we can shutdown the entire RS (it is not a good solution but the case is rare enough) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13533) section on configuring ~/.m2/settings.xml has no anchor
[ https://issues.apache.org/jira/browse/HBASE-13533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538136#comment-14538136 ] Misty Stanley-Jones commented on HBASE-13533: - Sorry, the latest patch does not apply. I think you made it from the wrong part of the directory or used `diff` instead of `git diff`. Please use `git format-patch` to generate your patch if possible, and re-attach it. section on configuring ~/.m2/settings.xml has no anchor --- Key: HBASE-13533 URL: https://issues.apache.org/jira/browse/HBASE-13533 Project: HBase Issue Type: Bug Components: documentation Reporter: Nick Dimiduk Assignee: Gabor Liptak Priority: Trivial Attachments: HBASE-13533.1.patch, HBASE-13533.2.patch, HBASE-13533.patch From http://hbase.apache.org/book.html#maven.snapshot there's a link to mvn.settings.file that is invalid. I think the intended destination is the example listing at the beginning of 131.1, above. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12751: -- Attachment: HBASE-12751-v9.patch Passes TestHRegion. Still not passing on part of test atomic operation. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12751-v1.patch, HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13580) region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable
[ https://issues.apache.org/jira/browse/HBASE-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538363#comment-14538363 ] Hudson commented on HBASE-13580: FAILURE: Integrated in HBase-TRUNK #6472 (See [https://builds.apache.org/job/HBase-TRUNK/6472/]) HBASE-13580 region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable (Samir Ahmic) (tedyu: rev 211786e00ce194dfa3efb4b35eefd4aa3236984c) * bin/region_mover.rb region_mover.rb broken with TypeError: no public constructors for Java::OrgApacheHadoopHbaseClient::HTable -- Key: HBASE-13580 URL: https://issues.apache.org/jira/browse/HBASE-13580 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Environment: x86_64 GNU/Linux Reporter: Samir Ahmic Assignee: Samir Ahmic Fix For: 2.0.0 Attachments: HBASE-13580-v2.patch, HBASE-13580-v3.patch, HBASE-13580-v4.patch, HBASE-13580-v5.patch, HBASE-13580.patch I was testing region_mover.rb on master branch in distributed cluster and hit this error. I have fixed this by using Connection#getTable instead of HTable but look like this script needs some additional work: 1. Remove master server from region move targets list 2. --exclude=FILE option is not working for me I will try to get this script in functional state if there is no objections ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13662) RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a retriable failure
Matteo Bertozzi created HBASE-13662:
---
Summary: RSRpcService.scan() throws an OutOfOrderScannerNext if
the scan has a retriable failure
Key: HBASE-13662
URL: https://issues.apache.org/jira/browse/HBASE-13662
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.10.1, 0.94.27, 1.0.1, 2.0.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
while fixing HBASE-13651 I noticed that if we have a failure inside the
RSRpcService.scan(), when the request has a hasNextCallSeq() the nextCallSeq
is incremented and not rolledback, which means that the client retry will send
a request with a nextCallSeq not up to date, which result in an
OutOfOrderScannerNextException.
{code}
if (rows 0) {
if (request.hasNextCallSeq()) {
if (request.getNextCallSeq() != rsh.nextCallSeq) {
throw new OutOfOrderScannerNextException(...)
}
// Increment the nextCallSeq value which is the next expected from client.
rsh.nextCallSeq++;
}
}
try {
...scan code...
}
{code}
after the scanner heartbeat patches HBASE-13090, we seems to be able to recover
from that OutOfOrder exception, but the error show up anyway.
After a discussion with [~saint@gmail.com] we ended up saying that
decrementing the callSeq on exception seems to be fine. but we had the open
question about having that nextCallSeq to be atomic, if that was supposed to
prevent concurrent requests with the same id. any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13637) branch-1.1 does not build against hadoop-2.2.
[
https://issues.apache.org/jira/browse/HBASE-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538504#comment-14538504
]
Andrew Purtell commented on HBASE-13637:
bq. Then we can still build with hadoop-2.2 by specifying
hadoop-two-minikdc.version explicitly. The common case is that we want those
versions to be the same unless there is a reason.
Sounds reasonable to me. +1 after this change
branch-1.1 does not build against hadoop-2.2.
-
Key: HBASE-13637
URL: https://issues.apache.org/jira/browse/HBASE-13637
Project: HBase
Issue Type: Bug
Reporter: Nick Dimiduk
Assignee: zhangduo
Fix For: 1.1.0
Attachments: HBASE-13637-branch-1.1.patch
From RC0 VOTE thread,
{quote}
The build is broken with Hadoop-2.2 because mini-kdc is not found:
\[ERROR\] Failed to execute goal on project hbase-server: Could not resolve
dependencies for project org.apache.hbase:hbase-server:jar:1.1.0: Could not
find artifact org.apache.hadoop:hadoop-minikdc:jar:2.2
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13637) branch-1.1 does not build against hadoop-2.2.
[
https://issues.apache.org/jira/browse/HBASE-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538311#comment-14538311
]
Enis Soztutar commented on HBASE-13637:
---
I think this is the right approach, but I would prefer to always use the same
version of Hadoop unless specified otherwise. Can we do something like this:
{code}
hadoop-two-minikdc.version${hadoop-two.version}/hadoop-two-minikdc.version
{code}
Then we can still build with hadoop-2.2 by specifying
{{hadoop-two-minikdc.version}} explicitly. The common case is that we want
those versions to be the same unless there is a reason.
branch-1.1 does not build against hadoop-2.2.
-
Key: HBASE-13637
URL: https://issues.apache.org/jira/browse/HBASE-13637
Project: HBase
Issue Type: Bug
Reporter: Nick Dimiduk
Fix For: 1.1.0
Attachments: HBASE-13637-branch-1.1.patch
From RC0 VOTE thread,
{quote}
The build is broken with Hadoop-2.2 because mini-kdc is not found:
\[ERROR\] Failed to execute goal on project hbase-server: Could not resolve
dependencies for project org.apache.hbase:hbase-server:jar:1.1.0: Could not
find artifact org.apache.hadoop:hadoop-minikdc:jar:2.2
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-13661) Correct @Public annotation marking discovered in 1.1.0RC0
[ https://issues.apache.org/jira/browse/HBASE-13661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar reassigned HBASE-13661: - Assignee: Enis Soztutar (was: Nick Dimiduk) Correct @Public annotation marking discovered in 1.1.0RC0 - Key: HBASE-13661 URL: https://issues.apache.org/jira/browse/HBASE-13661 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0, 1.2.0 Over on the 1.1.0RC0 VOTE thread, Enis discovered some errors in InterfaceAudience annotations. Let's fix them. Filed by [~ndimiduk] on [~enis]'s behalf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13413: --- Fix Version/s: 0.98.13 Ported back to 0.98. Tested with two small clusters. Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Affects Versions: 1.0.0, 2.0.0 Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor Fix For: 2.0.0, 1.1.0, 0.98.13 Attachments: HBASE-13413-0.98.patch, HBASE-13413-v1.patch, HBASE-13413-v2.patch, HBASE-13413.patch We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13413) Create an integration test for Replication
[ https://issues.apache.org/jira/browse/HBASE-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13413: --- Attachment: HBASE-13413-0.98.patch Create an integration test for Replication -- Key: HBASE-13413 URL: https://issues.apache.org/jira/browse/HBASE-13413 Project: HBase Issue Type: Test Components: integration tests Affects Versions: 1.0.0, 2.0.0 Reporter: Rajesh Nishtala Assignee: Rajesh Nishtala Priority: Minor Fix For: 2.0.0, 1.1.0, 0.98.13 Attachments: HBASE-13413-0.98.patch, HBASE-13413-v1.patch, HBASE-13413-v2.patch, HBASE-13413.patch We want to have an end-to-end test for replication. it can write data into one cluster (with replication setup) and then read data from the other. The test should be capable of running for a long time and be reliant even under chaos monkey testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13651) Handle StoreFileScanner FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538421#comment-14538421 ] Andrew Purtell commented on HBASE-13651: I see fix versions for 0.94 and 0.98 and a patch for 0.94 and master. We don't have RSRpcServices in 0.98. You'll find that code in HRegionServer. Handle StoreFileScanner FileNotFoundException - Key: HBASE-13651 URL: https://issues.apache.org/jira/browse/HBASE-13651 Project: HBase Issue Type: Bug Affects Versions: 0.94.27, 0.98.10.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13651-0.94-draft.patch, HBASE-13651-draft.patch Example: * Machine-1 is serving Region-X and start compaction * Machine-1 goes in GC pause * Region-X gets reassigned to Machine-2 * Machine-1 exit from the GC pause * Machine-1 (re)moves the compacted files * Machine-1 get the lease expired and shutdown Machine-2 has now tons of FileNotFoundException on scan. If we reassign the region everything is ok, because we pickup the files compacted by Machine-1. This problem doesn't happen in the new code 1.0+ (i think but I haven't checked, it may be 1.1) where we write on the WAL the compaction event before (re)moving the files. A workaround is handling FileNotFoundException and refresh the store files, or shutdown the region and reassign. the first one is easy in 1.0+ the second one requires more work because at the moment we don't have the code to notify the master that the RS is closing the region, alternatively we can shutdown the entire RS (it is not a good solution but the case is rare enough) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
[ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538419#comment-14538419 ] Eshcar Hillel commented on HBASE-13071: --- 2 check styles error added in this patch: (1) forgot to remove redundant import in ClientSimpleScanner, (2) added a line to the method loadCache() in ClientScanner which caused it to overflow (151 lines). Hbase Streaming Scan Feature Key: HBASE-13071 URL: https://issues.apache.org/jira/browse/HBASE-13071 Project: HBase Issue Type: New Feature Reporter: Eshcar Hillel Attachments: 99.eshcar.png, HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch, HBASE-13071_trunk_10.patch, HBASE-13071_trunk_2.patch, HBASE-13071_trunk_3.patch, HBASE-13071_trunk_4.patch, HBASE-13071_trunk_5.patch, HBASE-13071_trunk_6.patch, HBASE-13071_trunk_7.patch, HBASE-13071_trunk_8.patch, HBASE-13071_trunk_9.patch, HBASE-13071_trunk_rebase_1.0.patch, HBASE-13071_trunk_rebase_2.0.patch, HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf, HbaseStreamingScanEvaluationwithMultipleClients.pdf, gc.delay.png, gc.eshcar.png, gc.png, hits.delay.png, hits.eshcar.png, hits.png, latency.delay.png, latency.png, network.png A scan operation iterates over all rows of a table or a subrange of the table. The synchronous nature in which the data is served at the client side hinders the speed the application traverses the data: it increases the overall processing time, and may cause a great variance in the times the application waits for the next piece of data. The scanner next() method at the client side invokes an RPC to the regionserver and then stores the results in a cache. The application can specify how many rows will be transmitted per RPC; by default this is set to 100 rows. The cache can be considered as a producer-consumer queue, where the hbase client pushes the data to the queue and the application consumes it. Currently this queue is synchronous, i.e., blocking. More specifically, when the application consumed all the data from the cache --- so the cache is empty --- the hbase client retrieves additional data from the server and re-fills the cache with new data. During this time the application is blocked. Under the assumption that the application processing time can be balanced by the time it takes to retrieve the data, an asynchronous approach can reduce the time the application is waiting for data. We attach a design document. We also have a patch that is based on a private branch, and some evaluation results of this code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
[ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538418#comment-14538418 ] Eshcar Hillel commented on HBASE-13071: --- 2 check styles error added in this patch: (1) forgot to remove redundant import in ClientSimpleScanner, (2) added a line to the method loadCache() in ClientScanner which caused it to overflow (151 lines). Hbase Streaming Scan Feature Key: HBASE-13071 URL: https://issues.apache.org/jira/browse/HBASE-13071 Project: HBase Issue Type: New Feature Reporter: Eshcar Hillel Attachments: 99.eshcar.png, HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch, HBASE-13071_trunk_10.patch, HBASE-13071_trunk_2.patch, HBASE-13071_trunk_3.patch, HBASE-13071_trunk_4.patch, HBASE-13071_trunk_5.patch, HBASE-13071_trunk_6.patch, HBASE-13071_trunk_7.patch, HBASE-13071_trunk_8.patch, HBASE-13071_trunk_9.patch, HBASE-13071_trunk_rebase_1.0.patch, HBASE-13071_trunk_rebase_2.0.patch, HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf, HbaseStreamingScanEvaluationwithMultipleClients.pdf, gc.delay.png, gc.eshcar.png, gc.png, hits.delay.png, hits.eshcar.png, hits.png, latency.delay.png, latency.png, network.png A scan operation iterates over all rows of a table or a subrange of the table. The synchronous nature in which the data is served at the client side hinders the speed the application traverses the data: it increases the overall processing time, and may cause a great variance in the times the application waits for the next piece of data. The scanner next() method at the client side invokes an RPC to the regionserver and then stores the results in a cache. The application can specify how many rows will be transmitted per RPC; by default this is set to 100 rows. The cache can be considered as a producer-consumer queue, where the hbase client pushes the data to the queue and the application consumes it. Currently this queue is synchronous, i.e., blocking. More specifically, when the application consumed all the data from the cache --- so the cache is empty --- the hbase client retrieves additional data from the server and re-fills the cache with new data. During this time the application is blocked. Under the assumption that the application processing time can be balanced by the time it takes to retrieve the data, an asynchronous approach can reduce the time the application is waiting for data. We attach a design document. We also have a patch that is based on a private branch, and some evaluation results of this code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
[
https://issues.apache.org/jira/browse/HBASE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538436#comment-14538436
]
Ted Yu commented on HBASE-13663:
Here is related code where NPE happened:
{code}
for (HRegionLocation hrl : locations) {
HRegionInfo regionInfo = hrl.getRegionInfo();
{code}
In this case hrl was null.
From MetaTableAccessor#getRegionLocations():
{code}
// In case the region replica is newly created, it's location might be
null. We usually do not
// have HRL's in RegionLocations object with null ServerName. They are
handled as null HRLs.
if (location == null || location.getServerName() == null) {
locations.add(null);
{code}
This means that it was possible for null HRegionLocation to appear in the
return value.
Attached is patch which handles this situation.
HMaster fails to restart 'HMaster: Failed to become active master'
--
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
Assignee: Ted Yu
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13651) Handle StoreFileScanner FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538446#comment-14538446 ] Matteo Bertozzi commented on HBASE-13651: - the fix in 98 should be the same as 94. as soon everyone agrees on the approach here and HBASE-13662 gets in. I'll update the patch for all branches Handle StoreFileScanner FileNotFoundException - Key: HBASE-13651 URL: https://issues.apache.org/jira/browse/HBASE-13651 Project: HBase Issue Type: Bug Affects Versions: 0.94.27, 0.98.10.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13651-0.94-draft.patch, HBASE-13651-draft.patch Example: * Machine-1 is serving Region-X and start compaction * Machine-1 goes in GC pause * Region-X gets reassigned to Machine-2 * Machine-1 exit from the GC pause * Machine-1 (re)moves the compacted files * Machine-1 get the lease expired and shutdown Machine-2 has now tons of FileNotFoundException on scan. If we reassign the region everything is ok, because we pickup the files compacted by Machine-1. This problem doesn't happen in the new code 1.0+ (i think but I haven't checked, it may be 1.1) where we write on the WAL the compaction event before (re)moving the files. A workaround is handling FileNotFoundException and refresh the store files, or shutdown the region and reassign. the first one is easy in 1.0+ the second one requires more work because at the moment we don't have the code to notify the master that the RS is closing the region, alternatively we can shutdown the entire RS (it is not a good solution but the case is rare enough) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538447#comment-14538447 ] Andrew Purtell commented on HBASE-13658: -1 in current form If you're going to use the same table for every unit, each unit must now clean up after itself. Not every unit creates users and adds grants, but some do. Add cleanup code for when that is the case and I'll +1 Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: 13658.patch, HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13665: - Attachment: HBASE-13665.00.branch-1.patch Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13664) Use HBase 1.0 interfaces in ConnectionCache
[
https://issues.apache.org/jira/browse/HBASE-13664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538687#comment-14538687
]
Hadoop QA commented on HBASE-13664:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12732016/HBASE-13664.patch
against master branch at commit 174632111c1f844151eea682331d9b5b157e4dd4.
ATTACHMENT ID: 12732016
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14015//console
This message is automatically generated.
Use HBase 1.0 interfaces in ConnectionCache
---
Key: HBASE-13664
URL: https://issues.apache.org/jira/browse/HBASE-13664
Project: HBase
Issue Type: Bug
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Fix For: 1.0.2, 1.1.1
Attachments: HBASE-13664.patch
ConnectionCache uses the new HBase interfaces in master. Update branch-1*
to use them as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538689#comment-14538689
]
Hadoop QA commented on HBASE-13665:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12732013/HBASE-13665.00.branch-1.patch
against branch-1 branch at commit 174632111c1f844151eea682331d9b5b157e4dd4.
ATTACHMENT ID: 12732013
{color:red}-1 @author{color}. The patch appears to contain 1 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14016//console
This message is automatically generated.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch,
HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538706#comment-14538706 ] Enis Soztutar commented on HBASE-13665: --- The site and book you have looks good. Let me check the branch-1.0 commit for this. Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13664) Use HBase 1.0 interfaces in ConnectionCache
[ https://issues.apache.org/jira/browse/HBASE-13664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538707#comment-14538707 ] Ted Yu commented on HBASE-13664: Please name your patch with 'branch-1' in its name so that QA bot applies it on correct branch. Use HBase 1.0 interfaces in ConnectionCache --- Key: HBASE-13664 URL: https://issues.apache.org/jira/browse/HBASE-13664 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 1.0.2, 1.1.1 Attachments: HBASE-13664.patch ConnectionCache uses the new HBase interfaces in master. Update branch-1* to use them as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13599) The Example Provided in Section 69: Examples of the Documentation Does Not Compile
[
https://issues.apache.org/jira/browse/HBASE-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538718#comment-14538718
]
Lars Francke commented on HBASE-13599:
--
Thanks again.
I used the make_patch script. Is there enything I can do to provide better
patches? Put JIRA and title in the commit message?
The Example Provided in Section 69: Examples of the Documentation Does Not
Compile
--
Key: HBASE-13599
URL: https://issues.apache.org/jira/browse/HBASE-13599
Project: HBase
Issue Type: Bug
Components: documentation
Affects Versions: 1.0.0
Reporter: David Newcomer
Assignee: Lars Francke
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-13599-v1.patch
I'm trying to build and run the example java code I found in the HBase
Documentation, and I'm running into several issues.
1. I don't have the code/library used in the following import:
import static com.example.hbase.Constants.*;
I don't believe it is included in any of the HBase libraries or documentation.
2. All of the methods in createOrOverwrite() that use table.getName() should
instead be using table.getTableName()
3. The interface org.apache.hadoop.hbase.client.Admin is abstract, and can't
be instantiated with a Configuration. Constructing an
org.apache.hadoop.hbase.client.HBaseAdmin would allow the code to compile,
but that constructor is deprecated.
4. I have no references to the field TABLE_NAME or CF_DEFAULT. I'm
assuming they are Strings in com.example.hbase.Constants. Perhaps those
variables should simply be copied into the the Example?
Link to the documentation section:
http://hbase.apache.org/book.html#_examples
code
package com.example.hbase.admin;
import java.io.IOException;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.io.compress.Compression.Algorithm;
import org.apache.hadoop.conf.Configuration;
import static com.example.hbase.Constants.*;
public class CreateSchema {
public static void createOrOverwrite(Admin admin, HTableDescriptor table)
throws IOException {
if (admin.tableExists(table.getName())) {
admin.disableTable(table.getName());
admin.deleteTable(table.getName());
}
admin.createTable(table);
}
public static void createSchemaTables (Configuration config) {
try {
final Admin admin = new Admin(config);
HTableDescriptor table = new
HTableDescriptor(TableName.valueOf(TABLE_NAME));
table.addFamily(new
HColumnDescriptor(CF_DEFAULT).setCompressionType(Algorithm.SNAPPY));
System.out.print(Creating table. );
createOrOverwrite(admin, table);
System.out.println( Done.);
admin.close();
} catch (Exception e) {
e.printStackTrace();
System.exit(-1);
}
}
}
/code
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13412) Region split decisions should have jitter
[ https://issues.apache.org/jira/browse/HBASE-13412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13412: --- Fix Version/s: 0.98.13 Both the original commit and addendum picked back to 0.98 cleanly and the new test passes repeatedly (ran it 10 times) This change introduces a new configuration parameter and a behavioral change for ConstantSizeRegionSplitPolicy and policies that inherit from it. The change is desirable (it's the raison d'etre of this issue) but to alleviate any compatibility concerns I committed an addendum to 0.98 only that will not apply jitter unless the new configuration parameter is explicitly set. Region split decisions should have jitter - Key: HBASE-13412 URL: https://issues.apache.org/jira/browse/HBASE-13412 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 1.0.0, 2.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.1.0, 0.98.13 Attachments: HBASE-13412-v1.patch, HBASE-13412-v2.patch, HBASE-13412-v3.patch, HBASE-13412.addendum.0.98.patch, HBASE-13412.patch, hbase-13412.addendum.patch Whenever a region splits it causes lots of IO (compactions are queued for a while). Because of this it's important to make sure that well distributed tables don't have all of their regions split at exactly the same time. This is basically the same as our compaction jitter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13666) book.pdf is not renamed during site build
Nick Dimiduk created HBASE-13666:
Summary: book.pdf is not renamed during site build
Key: HBASE-13666
URL: https://issues.apache.org/jira/browse/HBASE-13666
Project: HBase
Issue Type: Task
Components: site
Reporter: Nick Dimiduk
Noticed this while testing HBASE-13665. Looks like the post-site hook is broken
or not executed, so the file {{book.pdf}} is not copied over to the expected
name {{apache_hbase_reference_guide.pdf}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538671#comment-14538671
]
Nick Dimiduk commented on HBASE-13665:
--
Not sure why the patch is failing to apply; it's generated from my local commit
with {{git format-patch HEAD^}}.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch,
HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
[
https://issues.apache.org/jira/browse/HBASE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538704#comment-14538704
]
Ted Yu commented on HBASE-13663:
TestSmallBlock seems to have come from hdfs Jenkins build - not related to the
patch.
HMaster fails to restart 'HMaster: Failed to become active master'
--
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
Assignee: Ted Yu
Attachments: 13663-v1.txt
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13665: - Attachment: HBASE-13665.00.branch-1.patch Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch, HBASE-13665.00.branch-1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13533) section on configuring ~/.m2/settings.xml has no anchor
[ https://issues.apache.org/jira/browse/HBASE-13533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538721#comment-14538721 ] Gabor Liptak commented on HBASE-13533: -- [~misty] The previous patch got stale. I uploaded a new one based on current master. section on configuring ~/.m2/settings.xml has no anchor --- Key: HBASE-13533 URL: https://issues.apache.org/jira/browse/HBASE-13533 Project: HBase Issue Type: Bug Components: documentation Reporter: Nick Dimiduk Assignee: Gabor Liptak Priority: Trivial Attachments: HBASE-13533.1.patch, HBASE-13533.2.patch, HBASE-13533.3.patch, HBASE-13533.patch From http://hbase.apache.org/book.html#maven.snapshot there's a link to mvn.settings.file that is invalid. I think the intended destination is the example listing at the beginning of 131.1, above. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538724#comment-14538724
]
Enis Soztutar commented on HBASE-13665:
---
These are the commits in branch-1.0:
{code}
commit 34e538c711c58b2f5393701ac9dc165305ad2646
Author: Enis Soztutar e...@apache.org
Date: Tue Jan 27 19:13:06 2015 -0800
HBASE-12926 Backport HBASE-12688 (Update site with a bootstrap-based UI)
for HBASE-12918 (Andrew Purtell)
commit cb77a925e5988e60533e9439e8b8694997c3478b
Author: Enis Soztutar e...@apache.org
Date: Tue Jan 27 19:09:06 2015 -0800
HBASE-12918 Backport asciidoc changes (apurtell and enis)
{code}
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch,
HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch,
HBASE-13665.00.branch-1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538736#comment-14538736 ] Enis Soztutar commented on HBASE-13665: --- I think we've missed HBASE-12918 in branch-1 and branch-1.1 (commit 34e538c7 in branch-1.0). If the patch is similar to 34e538c7 in pom.xml changes and just a copy of docs from master branch, then +1. Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch, HBASE-13665.00.branch-1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538743#comment-14538743
]
Hadoop QA commented on HBASE-13665:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12732029/HBASE-13665.00.branch-1.patch
against branch-1 branch at commit 174632111c1f844151eea682331d9b5b157e4dd4.
ATTACHMENT ID: 12732029
{color:red}-1 @author{color}. The patch appears to contain 1 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14017//console
This message is automatically generated.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch,
HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.patch,
HBASE-13665.00.branch-1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13658) Improve the test run time for TestAccessController* classes
[ https://issues.apache.org/jira/browse/HBASE-13658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538308#comment-14538308 ] Ted Yu commented on HBASE-13658: +1 Improve the test run time for TestAccessController* classes --- Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Attachments: 13658.patch, HBASE-13658.patch Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13662) RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a retriable failure
[
https://issues.apache.org/jira/browse/HBASE-13662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538413#comment-14538413
]
Andrew Purtell commented on HBASE-13662:
+1
I like the test.
bq. we had the open question about having that nextCallSeq to be atomic, if
that was supposed to prevent concurrent requests with the same id. any thoughts?
The first thought that comes to mind is better safe than sorry.
RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a
retriable failure
---
Key: HBASE-13662
URL: https://issues.apache.org/jira/browse/HBASE-13662
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.0.1, 0.94.27, 0.98.10.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13662-v0.patch
while fixing HBASE-13651 I noticed that if we have a failure inside the
RSRpcService.scan(), when the request has a hasNextCallSeq() the nextCallSeq
is incremented and not rolledback, which means that the client retry will
send a request with a nextCallSeq not up to date, which result in an
OutOfOrderScannerNextException.
{code}
if (rows 0) {
if (request.hasNextCallSeq()) {
if (request.getNextCallSeq() != rsh.nextCallSeq) {
throw new OutOfOrderScannerNextException(...)
}
// Increment the nextCallSeq value which is the next expected from client.
rsh.nextCallSeq++;
}
}
try {
...scan code...
}
{code}
after the scanner heartbeat patches HBASE-13090, we seems to be able to
recover from that OutOfOrder exception, but the error show up anyway.
After a discussion with [~saint@gmail.com] we ended up saying that
decrementing the callSeq on exception seems to be fine. but we had the open
question about having that nextCallSeq to be atomic, if that was supposed to
prevent concurrent requests with the same id. any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
[
https://issues.apache.org/jira/browse/HBASE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-13663:
--
Assignee: Ted Yu
HMaster fails to restart 'HMaster: Failed to become active master'
--
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
Assignee: Ted Yu
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13651) Handle StoreFileScanner FileNotFoundException
[ https://issues.apache.org/jira/browse/HBASE-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538421#comment-14538421 ] Andrew Purtell edited comment on HBASE-13651 at 5/11/15 6:54 PM: - I see affects versions for 0.94 and 0.98 and a patch for 0.94 and master. We don't have RSRpcServices in 0.98. You'll find that code in HRegionServer. was (Author: apurtell): I see fix versions for 0.94 and 0.98 and a patch for 0.94 and master. We don't have RSRpcServices in 0.98. You'll find that code in HRegionServer. Handle StoreFileScanner FileNotFoundException - Key: HBASE-13651 URL: https://issues.apache.org/jira/browse/HBASE-13651 Project: HBase Issue Type: Bug Affects Versions: 0.94.27, 0.98.10.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13651-0.94-draft.patch, HBASE-13651-draft.patch Example: * Machine-1 is serving Region-X and start compaction * Machine-1 goes in GC pause * Region-X gets reassigned to Machine-2 * Machine-1 exit from the GC pause * Machine-1 (re)moves the compacted files * Machine-1 get the lease expired and shutdown Machine-2 has now tons of FileNotFoundException on scan. If we reassign the region everything is ok, because we pickup the files compacted by Machine-1. This problem doesn't happen in the new code 1.0+ (i think but I haven't checked, it may be 1.1) where we write on the WAL the compaction event before (re)moving the files. A workaround is handling FileNotFoundException and refresh the store files, or shutdown the region and reassign. the first one is easy in 1.0+ the second one requires more work because at the moment we don't have the code to notify the master that the RS is closing the region, alternatively we can shutdown the entire RS (it is not a good solution but the case is rare enough) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
Romil Choksi created HBASE-13663:
Summary: HMaster fails to restart 'HMaster: Failed to become
active master'
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13663) HMaster fails to restart 'HMaster: Failed to become active master'
[
https://issues.apache.org/jira/browse/HBASE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13663:
---
Status: Patch Available (was: Open)
HMaster fails to restart 'HMaster: Failed to become active master'
--
Key: HBASE-13663
URL: https://issues.apache.org/jira/browse/HBASE-13663
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 1.1.0
Reporter: Romil Choksi
Assignee: Ted Yu
Attachments: 13663-v1.txt
HMaster fails to restart 'HMaster: Failed to become active master'
from Master log:
{code}
2015-05-08 11:25:14,020 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Failed to become active master
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-05-08 11:25:14,023 FATAL [MasterNOde:16000.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2885)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:483)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:763)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1632)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13664) Use HBase 1.0 in ConnectionCache
Solomon Duskis created HBASE-13664: -- Summary: Use HBase 1.0 in ConnectionCache Key: HBASE-13664 URL: https://issues.apache.org/jira/browse/HBASE-13664 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 1.0.2, 1.1.1 ConnectionCache uses the new HBase interfaces in master. Update branch-1* to use them as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13549) metric for failed lease recovery
[
https://issues.apache.org/jira/browse/HBASE-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538461#comment-14538461
]
Andrew Purtell commented on HBASE-13549:
Changes to any branch need to go through master. Please patch the master branch
[~abhishek.chouhan].
{code}
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
@@ -530,4 +531,14 @@ class MetricsRegionServerWrapperImpl
public long getBlockedRequestsCount() {
return blockedRequestsCount;
}
+
+ @Override
+ public long getLeaseRecoveryFailures() {
+return FSUtils.getLeaseRecoveryFailures();
+ }
+
+ @Override
+ public long getLeaseRecoveryTimeouts() {
+return FSUtils.getLeaseRecoveryTimeouts();
+ }
{code}
It would be better if these counts are kept in the metrics classes, using
metrics counters, like all the others.
{code}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
index 57eeb7c..b321f15 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
@@ -99,6 +100,10 @@ public abstract class FSUtils {
/** Set to true on Windows platforms */
public static final boolean WINDOWS =
System.getProperty(os.name).startsWith(Windows);
+ /** To keep count of lease recovery failures and timeouts */
+ private static final AtomicLong leaseRecoveryFailures = new AtomicLong();
+ private static final AtomicLong leaseRecoveryTimeouts = new AtomicLong();
+
protected FSUtils() {
super();
}
{code}
Ditto
metric for failed lease recovery
Key: HBASE-13549
URL: https://issues.apache.org/jira/browse/HBASE-13549
Project: HBase
Issue Type: Sub-task
Components: Usability
Reporter: Sean Busbey
Attachments: HBASE-13549-0.98.patch
we should publish metrics for
* lease recovery failures
* lease recovery timeout
right now you have to examine the logs of individual RS to get this
information and aggregating it is painful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13661) Correct @Public annotation marking discovered in 1.1.0RC0
[ https://issues.apache.org/jira/browse/HBASE-13661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-13661: -- Attachment: hbase-13661_v1-branch-1.1.patch Simple patch which fixes a couple of things: - Bring back RegionLocator to HTable. Let's keep it in branch-1 since 1.0 is like this. - Bring back Scan.HINT_LOOKAHEAD, not used anymore. - AuthUtil and proc v2 classes and NamespaceAuditor from Public to Private - ServerLoad had the change of int - long for getReadRequestCount. It is not strictly compatible, but it is a required bug fix. I think we need the same fix for total request count as well which the patch contains. This is intended for branch-1. The master patch should not contain the HTable and Scan changes in here. Correct @Public annotation marking discovered in 1.1.0RC0 - Key: HBASE-13661 URL: https://issues.apache.org/jira/browse/HBASE-13661 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0, 1.2.0 Attachments: hbase-13661_v1-branch-1.1.patch Over on the 1.1.0RC0 VOTE thread, Enis discovered some errors in InterfaceAudience annotations. Let's fix them. Filed by [~ndimiduk] on [~enis]'s behalf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13661) Correct @Public annotation marking discovered in 1.1.0RC0
[ https://issues.apache.org/jira/browse/HBASE-13661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-13661: -- Status: Patch Available (was: Open) Correct @Public annotation marking discovered in 1.1.0RC0 - Key: HBASE-13661 URL: https://issues.apache.org/jira/browse/HBASE-13661 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0, 1.2.0 Attachments: hbase-13661_v1-branch-1.1.patch Over on the 1.1.0RC0 VOTE thread, Enis discovered some errors in InterfaceAudience annotations. Let's fix them. Filed by [~ndimiduk] on [~enis]'s behalf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538553#comment-14538553
]
Hadoop QA commented on HBASE-13665:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12731989/HBASE-13665.00.branch-1.1.patch
against branch-1.1 branch at commit fa6dc9c44e28aa608d5204db814e8442b95eb125.
ATTACHMENT ID: 12731989
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14012//console
This message is automatically generated.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13665: - Attachment: HBASE-13665.00.branch-1.1.patch Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch, HBASE-13665.00.branch-1.1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-13665:
-
Attachment: HBASE-13665.00.branch-1.1.patch
This patch is a selective application of {{git diff branch-1.1 master pom.xml}}
plus {{git checkout master src/main}}. I've pushed the build results to
http://people.apache.org/~ndimiduk/1.1.0/site/ for easier inspection.
[~misty] mind giving my pom changes a once-over? Is there anything else I need
to include here?
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments: HBASE-13665.00.branch-1.1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13665: - Status: Patch Available (was: Open) Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-13665.00.branch-1.1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538545#comment-14538545
]
Nick Dimiduk commented on HBASE-13665:
--
Looks like the rename of {{book.pdf}} to {{apache_hbase_reference_guide.pdf}}
doesn't fire either.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538623#comment-14538623 ] Nick Dimiduk commented on HBASE-13665: -- Sorry, the above diffs are backwards. Looks like the rename of book.pdf isn't working on master either :) Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13665) Fix docs and site building on branch-1
[
https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538640#comment-14538640
]
Hadoop QA commented on HBASE-13665:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12732006/HBASE-13665.00.branch-1.1.patch
against branch-1.1 branch at commit fa6dc9c44e28aa608d5204db814e8442b95eb125.
ATTACHMENT ID: 12732006
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14014//console
This message is automatically generated.
Fix docs and site building on branch-1
--
Key: HBASE-13665
URL: https://issues.apache.org/jira/browse/HBASE-13665
Project: HBase
Issue Type: Task
Components: documentation, site
Affects Versions: 1.1.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 1.1.0, 1.2.0
Attachments: HBASE-13665.00.branch-1.1.patch,
HBASE-13665.00.branch-1.1.patch
It was noticed during 1.1.0RC0 that the docs are built with the old docbook
stuff. This should be fixed so we package the correct bits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13412) Region split decisions should have jitter
[ https://issues.apache.org/jira/browse/HBASE-13412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13412: --- Attachment: HBASE-13412.addendum.0.98.patch Region split decisions should have jitter - Key: HBASE-13412 URL: https://issues.apache.org/jira/browse/HBASE-13412 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 1.0.0, 2.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.1.0, 0.98.13 Attachments: HBASE-13412-v1.patch, HBASE-13412-v2.patch, HBASE-13412-v3.patch, HBASE-13412.addendum.0.98.patch, HBASE-13412.patch, hbase-13412.addendum.patch Whenever a region splits it causes lots of IO (compactions are queued for a while). Because of this it's important to make sure that well distributed tables don't have all of their regions split at exactly the same time. This is basically the same as our compaction jitter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13665) Fix docs and site building on branch-1
Nick Dimiduk created HBASE-13665: Summary: Fix docs and site building on branch-1 Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13665) Fix docs and site building on branch-1
[ https://issues.apache.org/jira/browse/HBASE-13665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13665: - Fix Version/s: 1.2.0 1.1.0 Fix docs and site building on branch-1 -- Key: HBASE-13665 URL: https://issues.apache.org/jira/browse/HBASE-13665 Project: HBase Issue Type: Task Components: documentation, site Affects Versions: 1.1.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 1.1.0, 1.2.0 Attachments: HBASE-13665.00.branch-1.1.patch It was noticed during 1.1.0RC0 that the docs are built with the old docbook stuff. This should be fixed so we package the correct bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
