[jira] [Resolved] (HBASE-12710) TestAssignmentManager may fail due to illegal character in namespace
[
https://issues.apache.org/jira/browse/HBASE-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-12710.
Resolution: Cannot Reproduce
TestAssignmentManager may fail due to illegal character in namespace
Key: HBASE-12710
URL: https://issues.apache.org/jira/browse/HBASE-12710
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
Attachments: 12710-1.0.txt,
org.apache.hadoop.hbase.master.TestAssignmentManager-output.txt
I saw TestAssignmentManager#testMasterRestartWhenTableInEnabling fail when
running test suite on branch-1:
{code}
testMasterRestartWhenTableInEnabling(org.apache.hadoop.hbase.master.TestAssignmentManager)
Time elapsed: 0.714 sec FAILURE!
java.lang.AssertionError: Table should be enabled.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hbase.master.TestAssignmentManager.testMasterRestartWhenTableInEnabling(TestAssignmentManager.java:1023)
{code}
In test output, I saw:
{code}
2014-12-18 00:45:18,230 FATAL [kiyo:38816.activeMasterManager]
master.HMaster(1802): Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: Illegal character 10 at 0. Namespaces
can only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9]:
^Ehbase^R^Dmeta
at
org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:215)
at
org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:204)
at org.apache.hadoop.hbase.TableName.init(TableName.java:307)
at
org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:344)
at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:465)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2686)
at
org.apache.hadoop.hbase.HTableDescriptor.convert(HTableDescriptor.java:1525)
at
org.apache.hadoop.hbase.HTableDescriptor.parseFrom(HTableDescriptor.java:1487)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:526)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptorForTableDirectory(FSTableDescriptors.java:730)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptor(FSTableDescriptors.java:706)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptor(FSTableDescriptors.java:693)
at
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:466)
at
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:145)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:125)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:592)
{code}
But running TestAssignmentManager standalone, I don't see the above exception.
Since TestAssignmentManager is a medium test, the above exception was likely
caused by concurrent tests.
TestAssignmentManager should be categorized as large test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13897) OOM occurs when Import importing a row that including too much KeyValue
[ https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HBASE-13897: Attachment: HBASE-13897-master.patch Attach the patch for master branch. OOM occurs when Import importing a row that including too much KeyValue Key: HBASE-13897 URL: https://issues.apache.org/jira/browse/HBASE-13897 Project: HBase Issue Type: Bug Affects Versions: 0.98.13 Reporter: Liu Junhong Assignee: Liu Junhong Fix For: 0.98.14 Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch When importing a row with too many KeyValues (may have too many columns or versions)，KeyValueReducer will incur OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13925) Use zookeeper multi to clear znodes in ZKProcedureUtil
[ https://issues.apache.org/jira/browse/HBASE-13925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604739#comment-14604739 ] Ashish Singhi commented on HBASE-13925: --- [~andrew.purt...@gmail.com], it looks like HBASE-7847 was not committed to 0.98 and 1.3. To apply this patch cleanly to those branches we need to have that and IMO that should be good to commit in 0.98 and 1.3, what you say ? If yes, should I combine that jira and this jira patch code and attach the patch here or open a backport jira for it ? Use zookeeper multi to clear znodes in ZKProcedureUtil -- Key: HBASE-13925 URL: https://issues.apache.org/jira/browse/HBASE-13925 Project: HBase Issue Type: Improvement Affects Versions: 0.98.13 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0, 0.98.14, 1.3.0 Attachments: HBASE-13925-v1-again.patch, HBASE-13925-v1.patch, HBASE-13925.patch Address the TODO in ZKProcedureUtil clearChildZNodes() and clearZNodes methods -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13897) OOM occurs when Import importing a row that including too much KeyValue
[ https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604732#comment-14604732 ] Liu Junhong commented on HBASE-13897: - HI, I attach the patch for master branch.This patch runs ok in 0.98.6. But i have no cluster that version is 2.0.0-SNAPSHOT.Can you test the patch, please?Or, is there anything that i can do? Thank you. OOM occurs when Import importing a row that including too much KeyValue Key: HBASE-13897 URL: https://issues.apache.org/jira/browse/HBASE-13897 Project: HBase Issue Type: Bug Affects Versions: 0.98.13 Reporter: Liu Junhong Assignee: Liu Junhong Fix For: 0.98.14 Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch When importing a row with too many KeyValues (may have too many columns or versions)，KeyValueReducer will incur OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13980) Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
[
https://issues.apache.org/jira/browse/HBASE-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604751#comment-14604751
]
Anoop Sam John commented on HBASE-13980:
blockedFlushCount should get more weight. +1 for the idea.
Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
Key: HBASE-13980
URL: https://issues.apache.org/jira/browse/HBASE-13980
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Assignee: Abhilash
Priority: Minor
Currently DefaultHeapMemoryTuner doesn't distinguish blockedFlushCount vs
unblockedFlushCount.
In its tune() method:
{code}
long totalFlushCount = blockedFlushCount+unblockedFlushCount;
rollingStatsForCacheMisses.insertDataValue(cacheMissCount);
rollingStatsForFlushes.insertDataValue(totalFlushCount);
{code}
Occurrence of blocked flush indicates that upper limit for memstore is not
sufficient.
We should either give blockedFlushCount more weight or, take tuning action
based on blockedFlushCount directly.
See discussion from tail of HBASE-13876.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13981) Fix ImportTsv spelling and usage issues
Lars George created HBASE-13981:
---
Summary: Fix ImportTsv spelling and usage issues
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Fix For: 2.0.0, 1.2.0
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13982) Add info for visibility labels/cell TTLs to ImportTsv
Lars George created HBASE-13982: --- Summary: Add info for visibility labels/cell TTLs to ImportTsv Key: HBASE-13982 URL: https://issues.apache.org/jira/browse/HBASE-13982 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 1.1.0.1 Reporter: Lars George Fix For: 2.0.0, 1.2.0 HBASE-9832 added support for two more optional, special TSV columns, but no usage info was added. Add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Liptak updated HBASE-13981:
-
Release Note: HBASE-13981 Fix ImportTsv spelling and usage issues
Status: Patch Available (was: Open)
Fix ImportTsv spelling and usage issues
---
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-13981.1.patch
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in
code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13897) OOM may occur when Import imports a row with too many KeyValues
[ https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604871#comment-14604871 ] Ted Yu commented on HBASE-13897: You can use dev-support/checkstyle_report.py to find checkstyle warnings and correct them. OOM may occur when Import imports a row with too many KeyValues --- Key: HBASE-13897 URL: https://issues.apache.org/jira/browse/HBASE-13897 Project: HBase Issue Type: Bug Affects Versions: 0.98.13 Reporter: Liu Junhong Assignee: Liu Junhong Fix For: 0.98.14 Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch When importing a row with too many KeyValues (may have too many columns or versions)，KeyValueReducer will incur OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13897) OOM may occur when Import imports a row with too many KeyValues
[ https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13897: --- Summary: OOM may occur when Import imports a row with too many KeyValues (was: OOM occurs when Import importing a row that including too much KeyValue) OOM may occur when Import imports a row with too many KeyValues --- Key: HBASE-13897 URL: https://issues.apache.org/jira/browse/HBASE-13897 Project: HBase Issue Type: Bug Affects Versions: 0.98.13 Reporter: Liu Junhong Assignee: Liu Junhong Fix For: 0.98.14 Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch When importing a row with too many KeyValues (may have too many columns or versions)，KeyValueReducer will incur OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Liptak updated HBASE-13981:
-
Attachment: HBASE-13981.1.patch
Fix ImportTsv spelling and usage issues
---
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-13981.1.patch
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in
code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels
[
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-13861:
---
Assignee: Matt Warhaftig
Status: Patch Available (was: Open)
Please see attached 'hbase-13861-v1.patch'. Only two lines updated but a
chance to play with BucketCache (staying positive :-) ).
BucketCacheTmpl.jamon has wrong bucket free and used labels
---
Key: HBASE-13861
URL: https://issues.apache.org/jira/browse/HBASE-13861
Project: HBase
Issue Type: Bug
Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
Labels: beginner
Fix For: 2.0.0, 1.2.0
See this from the template, and note the label and actual values for the last
two columns.
{noformat}
table class=table table-striped
tr
thBucket Offset/th
thAllocation Size/th
thFree Count/th
thUsed Count/th
/tr
%for Bucket bucket: buckets %
tr
td% bucket.getBaseOffset() %/td
td% bucket.getItemAllocationSize() %/td
td% bucket.getFreeBytes() %/td
td% bucket.getUsedBytes() %/td
/tr
{noformat}
They are labeled counts but are bytes, duh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels
[
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-13861:
---
Attachment: hbase-13861-v1.patch
BucketCacheTmpl.jamon has wrong bucket free and used labels
---
Key: HBASE-13861
URL: https://issues.apache.org/jira/browse/HBASE-13861
Project: HBase
Issue Type: Bug
Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: hbase-13861-v1.patch
See this from the template, and note the label and actual values for the last
two columns.
{noformat}
table class=table table-striped
tr
thBucket Offset/th
thAllocation Size/th
thFree Count/th
thUsed Count/th
/tr
%for Bucket bucket: buckets %
tr
td% bucket.getBaseOffset() %/td
td% bucket.getItemAllocationSize() %/td
td% bucket.getFreeBytes() %/td
td% bucket.getUsedBytes() %/td
/tr
{noformat}
They are labeled counts but are bytes, duh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13968: -- Assignee: Ashish Singhi Status: Patch Available (was: Open) Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13968: -- Attachment: HBASE-13968.patch Removed BufferedMutatorImpl#getWriteBuffer. Please review. Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13897) OOM occurs when Import importing a row that including too much KeyValue
[
https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604811#comment-14604811
]
Hadoop QA commented on HBASE-13897:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12742397/HBASE-13897-master.patch
against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06.
ATTACHMENT ID: 12742397
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1906 checkstyle errors (more than the master's current 1900 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ReducerKeyValueWritableComparable, KeyValue,
ImmutableBytesWritable, KeyValue.Context context)
+ public static class KeyValueSortImporter extends
TableMapperKeyValueWritableComparable, KeyValue {
+KeyValueWritableComparable[] startKeyWraps = new
KeyValueWritableComparable[startKeys.length - 1];
+ startKeyWraps[i - 1] = new
KeyValueWritableComparable(KeyValueUtil.createFirstOnRow(startKeys[i]));
+Path partitionsPath = new
Path(TotalOrderPartitioner.getPartitionFile(job.getConfiguration()));
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14594//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14594//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14594//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14594//console
This message is automatically generated.
OOM occurs when Import importing a row that including too much KeyValue
Key: HBASE-13897
URL: https://issues.apache.org/jira/browse/HBASE-13897
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.13
Reporter: Liu Junhong
Assignee: Liu Junhong
Fix For: 0.98.14
Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch
When importing a row with too many KeyValues (may have too many columns or
versions)，KeyValueReducer will incur OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[
https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604855#comment-14604855
]
Hadoop QA commented on HBASE-13968:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12742402/HBASE-13968.patch
against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06.
ATTACHMENT ID: 12742402
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14595//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14595//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14595//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14595//console
This message is automatically generated.
Remove deprecated methods from BufferedMutator class
Key: HBASE-13968
URL: https://issues.apache.org/jira/browse/HBASE-13968
Project: HBase
Issue Type: Sub-task
Components: API
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 2.0.0
Attachments: HBASE-13968.patch
As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the
review board to remove BufferedMutator#getWriteBuffer method as it deprecated
since 1.0.0
So as part of this jira we can remove all the deprecated and unused methods
from BufferedMutator class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604792#comment-14604792 ] Anoop Sam John commented on HBASE-13968: +1 Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Liptak reassigned HBASE-13981:
Assignee: Gabor Liptak
Fix ImportTsv spelling and usage issues
---
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
Labels: beginner
Fix For: 2.0.0, 1.2.0
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in
code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13861) BucketCacheTmpl.jamon has wrong bucket free and used labels
[
https://issues.apache.org/jira/browse/HBASE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604997#comment-14604997
]
Hadoop QA commented on HBASE-13861:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12742426/hbase-13861-v1.patch
against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06.
ATTACHMENT ID: 12742426
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14597//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14597//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14597//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14597//console
This message is automatically generated.
BucketCacheTmpl.jamon has wrong bucket free and used labels
---
Key: HBASE-13861
URL: https://issues.apache.org/jira/browse/HBASE-13861
Project: HBase
Issue Type: Bug
Components: regionserver, UI
Affects Versions: 1.1.0
Reporter: Lars George
Assignee: Matt Warhaftig
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: hbase-13861-v1.patch
See this from the template, and note the label and actual values for the last
two columns.
{noformat}
table class=table table-striped
tr
thBucket Offset/th
thAllocation Size/th
thFree Count/th
thUsed Count/th
/tr
%for Bucket bucket: buckets %
tr
td% bucket.getBaseOffset() %/td
td% bucket.getItemAllocationSize() %/td
td% bucket.getFreeBytes() %/td
td% bucket.getUsedBytes() %/td
/tr
{noformat}
They are labeled counts but are bytes, duh.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604951#comment-14604951
]
Hadoop QA commented on HBASE-13981:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12742417/HBASE-13981.1.patch
against master branch at commit 7dbb2e69776bae8c2f2781f36528c0e784f93a06.
ATTACHMENT ID: 12742417
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14596//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14596//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14596//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14596//console
This message is automatically generated.
Fix ImportTsv spelling and usage issues
---
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-13981.1.patch
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in
code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13897) OOM may occur when Import imports a row with too many KeyValues
[
https://issues.apache.org/jira/browse/HBASE-13897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604963#comment-14604963
]
Ted Yu commented on HBASE-13897:
{code}
179 context.setStatus(Wrote + index);
{code}
Add more to the above log. e.g. Wrote + index + KeyValues
{code}
232 throw new IOException(Region splitted after job
initialization);
{code}
The past tense of split is split.
{code}
674 System.err.println(If there is no large results that including too
much KeyValue
675 + whitch can use the memery sort in reducer, pass the
option:);
{code}
Syntax error. Should be:
{code}
674 System.err.println(If there is no large result that includes too
many KeyValues
675 + which can use the memory sort in reducer, pass the option:);
{code}
{code}
676 System.err.println( -D + HAS_LARGE_RESULT + =false);
{code}
Should the option be turned off by default ?
OOM may occur when Import imports a row with too many KeyValues
---
Key: HBASE-13897
URL: https://issues.apache.org/jira/browse/HBASE-13897
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.13
Reporter: Liu Junhong
Assignee: Liu Junhong
Fix For: 0.98.14
Attachments: HBASE-13897-0.98.patch, HBASE-13897-master.patch
When importing a row with too many KeyValues (may have too many columns or
versions)，KeyValueReducer will incur OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13957) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty
[ https://issues.apache.org/jira/browse/HBASE-13957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apekshit Sharma updated HBASE-13957: Description: This sub-task is for adding core functionality for configuration improvements. Few configurations will be added/moved to new class ConfigurationProperty to get a better sense of use patterns. (was: This sub-task is for adding core functionality for configuration improvements. Few configurations will be added/moved to new enum HConfig to get a better sense of use patterns.) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty Key: HBASE-13957 URL: https://issues.apache.org/jira/browse/HBASE-13957 Project: HBase Issue Type: Sub-task Reporter: Apekshit Sharma Attachments: HBASE-13957.patch This sub-task is for adding core functionality for configuration improvements. Few configurations will be added/moved to new class ConfigurationProperty to get a better sense of use patterns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13957) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty
[ https://issues.apache.org/jira/browse/HBASE-13957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apekshit Sharma updated HBASE-13957: Summary: Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty (was: Add enum HConfig. Add new functions to ConfigurationManager which use HConfig.) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty Key: HBASE-13957 URL: https://issues.apache.org/jira/browse/HBASE-13957 Project: HBase Issue Type: Sub-task Reporter: Apekshit Sharma Attachments: HBASE-13957.patch This sub-task is for adding core functionality for configuration improvements. Few configurations will be added/moved to new enum HConfig to get a better sense of use patterns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13957) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty
[ https://issues.apache.org/jira/browse/HBASE-13957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605027#comment-14605027 ] Apekshit Sharma commented on HBASE-13957: - new changes are up on review board now. (discarded old board because of major changes in design) Add ConfigurationProperty. Add new functions to ConfigurationManager which use the new ConfigurationProperty Key: HBASE-13957 URL: https://issues.apache.org/jira/browse/HBASE-13957 Project: HBase Issue Type: Sub-task Reporter: Apekshit Sharma Attachments: HBASE-13957.patch This sub-task is for adding core functionality for configuration improvements. Few configurations will be added/moved to new class ConfigurationProperty to get a better sense of use patterns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13960) HConnection stuck with UnknownHostException
[
https://issues.apache.org/jira/browse/HBASE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kurt Young updated HBASE-13960:
---
Description:
when put/get from hbase, if we meet a temporary dns failure causes resolve RS's
host, the error will never recovered. put/get will failed with
UnknownHostException forever.
I checked the code, and the reason maybe:
1. when RegionServerCallable or MultiServerCallable prepare(), it gets a
ClientService.BlockingInterface stub from Hconnection
2. In HConnectionImplementation::getClient, it caches the stub with a
BlockingRpcChannelImplementation
3. In BlockingRpcChannelImplementation(),
this.isa = new InetSocketAddress(sn.getHostname(), sn.getPort()); If we
meet a temporary dns failure then the address in isa will be null.
4. then we launch the real rpc call, the following stack is:
Caused by: java.net.UnknownHostException: unknown host: xxx.host2
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.init(RpcClient.java:385)
at
org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:351)
at
org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1523)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1435)
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
Besides, i noticed there is a protection in RpcClient:
if (remoteId.getAddress().isUnresolved()) {
throw new UnknownHostException(unknown host: +
remoteId.getAddress().getHostName());
}
shouldn't we do something when this situation occurred?
was:
when put/get from hbase, if we meet a temporary dns failure causes resolve RS's
host, the error will never recovered. put/get will failed with
UnknownHostException forever.
I checked the code, and the reason maybe:
1. when RegionServerCallable or MultiServerCallable prepare(), it gets a
ClientService.BlockingInterface stub from Hconnection
2. In HConnectionImplementation::getClient, it caches the stub with a
BlockingRpcChannelImplementation
3. In BlockingRpcChannelImplementation(),
this.isa = new InetSocketAddress(sn.getHostname(), sn.getPort()); If we
meet a temporary dns failure then the address in isa will be null.
4. then we launch the real rpc call, the following stack is:
Caused by: java.net.UnknownHostException: unknown host:
r101072047.sqa.zmf.tbsite.net
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.init(RpcClient.java:385)
at
org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:351)
at
org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1523)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1435)
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
Besides, i noticed there is a protection in RpcClient:
if (remoteId.getAddress().isUnresolved()) {
throw new UnknownHostException(unknown host: +
remoteId.getAddress().getHostName());
}
shouldn't we do something when this situation occurred?
HConnection stuck with UnknownHostException
Key: HBASE-13960
URL: https://issues.apache.org/jira/browse/HBASE-13960
Project: HBase
Issue Type: Bug
Components: hbase
Affects Versions: 0.98.8
Reporter: Kurt Young
when put/get from hbase, if we meet a temporary dns failure causes resolve
RS's host, the error will never recovered. put/get will failed with
UnknownHostException forever.
I checked the code, and the reason maybe:
1. when RegionServerCallable or MultiServerCallable prepare(), it gets a
ClientService.BlockingInterface stub from Hconnection
2. In HConnectionImplementation::getClient, it caches the stub with a
BlockingRpcChannelImplementation
3. In BlockingRpcChannelImplementation(),
this.isa = new InetSocketAddress(sn.getHostname(), sn.getPort()); If we
meet a temporary dns failure then the address in isa will be null.
4. then we launch the real rpc call, the following stack is:
Caused by: java.net.UnknownHostException: unknown host: xxx.host2
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.init(RpcClient.java:385)
at
org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:351)
at
org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1523)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1435)
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
at
[jira] [Updated] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13983: -- Assignee: stack Priority: Minor (was: Major) Component/s: documentation Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
stack created HBASE-13983: - Summary: Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Reporter: stack See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605048#comment-14605048 ] stack commented on HBASE-13983: --- The methods are getStartKeys, getEndKeys, and getStartEndKeys. Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13983: -- Attachment: 13983.branch-1.txt Patch for all on branch-1. Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Attachments: 13983.branch-1.txt See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13983: -- Attachment: 13983.txt Patch for master branch that just removes these oddball methods from HTable. Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Attachments: 13983.branch-1.txt, 13983.txt See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13214) Remove deprecated and unused methods from HTable class
[ https://issues.apache.org/jira/browse/HBASE-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605062#comment-14605062 ] stack commented on HBASE-13214: --- HBASE-13983 is resolution of the discussion above around the oddball methods getStartKeys, etc. (after a trip to the dev list for discussion under heading Semantic Versioning Worksheet. Remove deprecated and unused methods from HTable class -- Key: HBASE-13214 URL: https://issues.apache.org/jira/browse/HBASE-13214 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13214-v1.patch, HBASE-13214-v2-again-v1.patch, HBASE-13214-v2-again.patch, HBASE-13214-v2.patch, HBASE-13214-v3.patch, HBASE-13214-v3.patch, HBASE-13214-v4.patch, HBASE-13214-v5.patch, HBASE-13214.patch Methods like #getRegionLocation(), #isTableEnabled() etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13984) Add option to allow caller to know the heartbeat and scanner position when scanner timeout
He Liangliang created HBASE-13984: - Summary: Add option to allow caller to know the heartbeat and scanner position when scanner timeout Key: HBASE-13984 URL: https://issues.apache.org/jira/browse/HBASE-13984 Project: HBase Issue Type: Improvement Reporter: He Liangliang Assignee: He Liangliang HBASE-13090 introduced scanner heartbeat. However, there are still some limitations (see HBASE-13215). In some application, for example, an operation access hbase to scan table data, and there is strict limit that this call must return in a fixed interval. At the same time, this call is stateless, so the call must return the next position to continue the scan. This is typical use case for online applications. Based on this requirement, some improvements are proposed: 1. Allow client set a flag whether pass the heartbeat (a fake row) to the caller (via ResultScanner next) 2. Allow the client pass a timeout to the server, which can override the server side default value 3. When requested by the client, the server peek the next cell and return to the client in the heartbeat message -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
[ https://issues.apache.org/jira/browse/HBASE-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Xu updated HBASE-13985: -- Description: When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. Of course, the default value of this option should be true. was:When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. Add configuration to skip validating HFile format when bulk loading millions of HFiles -- Key: HBASE-13985 URL: https://issues.apache.org/jira/browse/HBASE-13985 Project: HBase Issue Type: Improvement Affects Versions: 0.98.13 Reporter: Victor Xu Assignee: Victor Xu Priority: Minor Fix For: 0.98.14 Attachments: HBASE-13985.patch When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. Of course, the default value of this option should be true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
[ https://issues.apache.org/jira/browse/HBASE-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Xu updated HBASE-13985: -- Attachment: HBASE-13985.patch Add patch for this issue. Both master and 0.98 branches can apply it. Add configuration to skip validating HFile format when bulk loading millions of HFiles -- Key: HBASE-13985 URL: https://issues.apache.org/jira/browse/HBASE-13985 Project: HBase Issue Type: Improvement Affects Versions: 0.98.13 Reporter: Victor Xu Assignee: Victor Xu Priority: Minor Fix For: 0.98.14 Attachments: HBASE-13985.patch When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
Victor Xu created HBASE-13985: - Summary: Add configuration to skip validating HFile format when bulk loading millions of HFiles Key: HBASE-13985 URL: https://issues.apache.org/jira/browse/HBASE-13985 Project: HBase Issue Type: Improvement Affects Versions: 0.98.13 Reporter: Victor Xu Assignee: Victor Xu Priority: Minor Fix For: 0.98.14 When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
[ https://issues.apache.org/jira/browse/HBASE-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Xu updated HBASE-13985: -- Labels: regionserver (was: ) Add configuration to skip validating HFile format when bulk loading millions of HFiles -- Key: HBASE-13985 URL: https://issues.apache.org/jira/browse/HBASE-13985 Project: HBase Issue Type: Improvement Affects Versions: 0.98.13 Reporter: Victor Xu Assignee: Victor Xu Priority: Minor Labels: regionserver Fix For: 0.98.14 Attachments: HBASE-13985.patch When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all. Of course, the default value of this option should be true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605145#comment-14605145 ] Anoop Sam John commented on HBASE-13968: bq.TEST-org.apache.hadoop.hbase.master.TestDistributedLogSplitting.xml.init Not related to this patch.. The test is already flakey test. Will commit now Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13214) Remove deprecated and unused methods from HTable class
[ https://issues.apache.org/jira/browse/HBASE-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605128#comment-14605128 ] Ashish Singhi commented on HBASE-13214: --- Thanks Stack for all the support. Remove deprecated and unused methods from HTable class -- Key: HBASE-13214 URL: https://issues.apache.org/jira/browse/HBASE-13214 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13214-v1.patch, HBASE-13214-v2-again-v1.patch, HBASE-13214-v2-again.patch, HBASE-13214-v2.patch, HBASE-13214-v3.patch, HBASE-13214-v3.patch, HBASE-13214-v4.patch, HBASE-13214-v5.patch, HBASE-13214.patch Methods like #getRegionLocation(), #isTableEnabled() etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605156#comment-14605156 ] Ashish Singhi commented on HBASE-13968: --- Thanks Anoop for the quick review. Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13968) Remove deprecated methods from BufferedMutator class
[ https://issues.apache.org/jira/browse/HBASE-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-13968: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks Ashish Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Components: API Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Attachments: HBASE-13968.patch As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13981) Fix ImportTsv spelling and usage issues
[
https://issues.apache.org/jira/browse/HBASE-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605170#comment-14605170
]
Ashish Singhi commented on HBASE-13981:
---
The patch changes configuration property name, should we add a new one by
deprecating the old one and warn the user on using the old one ?
{code}
public final static String ATTRIBUTE_SEPARATOR_CONF_KEY =
96attributes.separator;
{code}
Fix ImportTsv spelling and usage issues
---
Key: HBASE-13981
URL: https://issues.apache.org/jira/browse/HBASE-13981
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 1.1.0.1
Reporter: Lars George
Assignee: Gabor Liptak
Labels: beginner
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-13981.1.patch
The {{ImportTsv}} tool has various spelling and formatting issues. Fix those.
In code:
{noformat}
public final static String ATTRIBUTE_SEPERATOR_CONF_KEY =
attributes.seperator;
{noformat}
It is separator.
In usage text:
{noformat}
input data. Another special columnHBASE_TS_KEY designates that this column
should be
{noformat}
Space missing.
{noformat}
Record with invalid timestamps (blank, non-numeric) will be treated as bad
record.
{noformat}
Records ... as bad records - plural missing twice.
{noformat}
HBASE_ATTRIBUTES_KEY can be used to specify Operation Attributes per record.
Should be specified as key=value where -1 is used
as the seperator. Note that more than one OperationAttributes can be
specified.
{noformat}
- Remove line wraps and indentation.
- Fix separator.
- Fix wrong separator being output, it is not -1 (wrong constant use in
code)
- General wording/style could be better (eg. last sentence now uses
OperationAttributes without a space).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
[
https://issues.apache.org/jira/browse/HBASE-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605101#comment-14605101
]
Ted Yu commented on HBASE-13985:
{code}
179* Skip reference, HFileLink, files starting with _ and non-valid
hfiles.
{code}
The last part of the above comment should be revised to take new parameter into
account.
Do you know how much percentage hfile validation took for your workload ?
Thanks
Add configuration to skip validating HFile format when bulk loading millions
of HFiles
--
Key: HBASE-13985
URL: https://issues.apache.org/jira/browse/HBASE-13985
Project: HBase
Issue Type: Improvement
Affects Versions: 0.98.13
Reporter: Victor Xu
Assignee: Victor Xu
Priority: Minor
Labels: regionserver
Fix For: 0.98.14
Attachments: HBASE-13985.patch
When bulk loading millions of HFile into one HTable, checking HFile format is
the most time-consuming phase. Maybe we could use a parallel mechanism to
increase the speed, but when it comes to millions of HFiles, it may still
cost dozens of minutes. So I think it's necessary to add an option for
advanced user to bulkload without checking HFile format at all.
Of course, the default value of this option should be true.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605137#comment-14605137 ] Ashish Singhi commented on HBASE-13983: --- +1 (non-binding) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Attachments: 13983.branch-1.txt, 13983.txt See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605110#comment-14605110 ] Anoop Sam John commented on HBASE-13983: bq.they will be removed in hbase 1.0.0. they will be removed in hbase 2.0.0 right? Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Attachments: 13983.branch-1.txt, 13983.txt See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13983) Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605111#comment-14605111 ] Anoop Sam John commented on HBASE-13983: +1 Doc how the oddball HTable methods getStartKey, getEndKey, etc. will be removed in 2.0.0 Key: HBASE-13983 URL: https://issues.apache.org/jira/browse/HBASE-13983 Project: HBase Issue Type: Task Components: documentation Reporter: stack Assignee: stack Priority: Minor Attachments: 13983.branch-1.txt, 13983.txt See the mail thread on dev Semantic Versioning Worksheet for background discussion. A few methods in HTable were NOT deprecated when we cut 1.0.0: getStartKeys, getEndKeys This issue is about doc'ing that even though they were not deprecated, they will be removed in hbase 1.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
