[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670087#comment-13670087
]
Matteo Bertozzi commented on HBASE-8642:
{quote}1) for the logic inside deleteSnapshotsByTable(), I was just copying
from #deleteSnapshots(final Pattern pattern), is it ok?{quote}
No is not ok, your new code should be better than the existent one :)
It's not wrong, but it's just an extra copy... should be fixed also in
deleteSnapshots()
{quote}
3) for shell support, I just studied the 6353. Seems there are 2 options:
a) add new list_snapshots_by_table.rb and delete_snapshot_by_table.rb to
separate the case from regex based operation;
b) still in list_snapshots.rb and delete_snapshot.rb, but add -table
option to branch out from regex based operation path.
{quote}
Having a -table option don't seems to be inline with the current shell, I think
that stuff like {{TABLE = 'name*' }} is used elsewhere (see create).. but I
will pick the easy way with a new .rb
{quote}seems that delete_snapshot.rb has no logic for regex based operation
currently, will we also support it in this stream?{quote}
yeah the delete snapshots is not in the shell at the moment.. I think there's a
jira or a mention to open a new jira for the shell support.
[~jmhsieh] was against the regex delete. not sure what he thinks about this
one... but at least this doesn't contain a regex for the table name.
[Snapshot] List and delete snapshot by table
Key: HBASE-8642
URL: https://issues.apache.org/jira/browse/HBASE-8642
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Julian Zhou
Priority: Minor
Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Attachments: 8642-trunk-0.95-v0.patch
Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670093#comment-13670093
]
Hadoop QA commented on HBASE-8645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585359/8645.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.thrift.TestThriftServer
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
org.apache.hadoop.hbase.rest.TestRowResource
org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
org.apache.hadoop.hbase.mapreduce.TestCopyTable
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler
org.apache.hadoop.hbase.coprocessor.TestClassLoading
org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.client.TestFromClientSide3
org.apache.hadoop.hbase.master.TestMaster
org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
org.apache.hadoop.hbase.client.TestSnapshotFromClient
org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.master.TestMasterMetricsWrapper
org.apache.hadoop.hbase.regionserver.TestJoinedScanners
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.constraint.TestConstraint
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.replication.TestReplicationQueueFailover
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
org.apache.hadoop.hbase.TestIOFencing
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
org.apache.hadoop.hbase.TestHBaseTestingUtility
org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
org.apache.hadoop.hbase.fs.TestBlockReorder
org.apache.hadoop.hbase.client.TestClientTimeouts
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.rest.TestMultiRowResource
org.apache.hadoop.hbase.client.TestSnapshotMetadata
[jira] [Commented] (HBASE-8654) src assembly does not include hbase-hadoop2-compat module
[ https://issues.apache.org/jira/browse/HBASE-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670098#comment-13670098 ] Hudson commented on HBASE-8654: --- Integrated in HBase-TRUNK #4150 (See [https://builds.apache.org/job/HBase-TRUNK/4150/]) HBASE-8654 src assembly does not include hbase-hadoop2-compat module (Revision 1487713) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-assembly/src/main/assembly/src.xml src assembly does not include hbase-hadoop2-compat module - Key: HBASE-8654 URL: https://issues.apache.org/jira/browse/HBASE-8654 Project: HBase Issue Type: Bug Components: build Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.0 Attachments: 8654.txt The src.xml assembly under hbase-assembly does not include the hbase-hadoop2-compat (more joys of maven -- uses default profile which is hadoop1 when making module set so we have to do perverse include of the hbase-hadoop2-compat). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8344) Improve the assignment when node failures happen to choose the secondary RS as the new primary RS
[ https://issues.apache.org/jira/browse/HBASE-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670104#comment-13670104 ] Devaraj Das commented on HBASE-8344: bq. if favoredNodes.size 3, you will get an out of bounds exception. In the method implementation where this code appears, there is a check just above the block of code that makes sure favoredNodes.size is equal to 3 before proceeding. bq. FavoredNodes.favoredNodesMap is a ConcurrentHashMap, but updateFavoredNodesMap() and getFavoredNodes() are also synchronized. Is this intended? Yeah methods need not be synchronized as of the current code since the underlying datastructure is synchronized but in spirit I intended for the methods to be synchronized. If you feel strongly about it, I can remove the synchronized on the methods. bq. FavoredNodeAssignmentHelper.FAVORED_NODES_NUM - should this be read from configuration default replication count for dfs? What if we are running with replication 2 or 4, we would still have 3 favored nodes? This is a TODO overall. Currently, yeah, there is an assumption that the replication is 3. If the replication is 2 or 4, things should work (as opposed to crashing) and will be status quo (for example, the master will assume there is a tertiary regionserver for a region even with a replication of 2 but in reality the tertiary server is as good as any random regionserver w.r.t hosting the region's blocks). In practice, we will mostly have a replication factor of 3 to have the failure zones taken care of and so it should be okay.. bq. FavoredNodeAssignmentManager.initialize() looks expensive. Do you need to call this for every roundRobin / random assignment. It seems that that info can only change if RS are going down / up. I assume you meant FavoredNodeAssignmentHelper.initialize.. Hmm.. That's how it is in the current codebase, and there are some datastructures that are valid per assignment (like uniqueRackList). Given all the other work we do in assignments (including ZK), this is probably going to be noise. But sure, I can look at this issue in a follow up.. Improve the assignment when node failures happen to choose the secondary RS as the new primary RS - Key: HBASE-8344 URL: https://issues.apache.org/jira/browse/HBASE-8344 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.95.2 Attachments: hbase-8344-1.txt, hbase-8344-2.1.txt, hbase-8344-2.2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8655) Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true)
[ https://issues.apache.org/jira/browse/HBASE-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-8655: -- Attachment: HBASE-8655.patch Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true) --- Key: HBASE-8655 URL: https://issues.apache.org/jira/browse/HBASE-8655 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.0 Reporter: Anoop Sam John Fix For: 0.94.9 Attachments: HBASE-8655.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8654) src assembly does not include hbase-hadoop2-compat module
[ https://issues.apache.org/jira/browse/HBASE-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670108#comment-13670108 ] Hudson commented on HBASE-8654: --- Integrated in hbase-0.95 #221 (See [https://builds.apache.org/job/hbase-0.95/221/]) HBASE-8654 src assembly does not include hbase-hadoop2-compat module (Revision 1487714) Result = SUCCESS stack : Files : * /hbase/branches/0.95/hbase-assembly/src/main/assembly/src.xml src assembly does not include hbase-hadoop2-compat module - Key: HBASE-8654 URL: https://issues.apache.org/jira/browse/HBASE-8654 Project: HBase Issue Type: Bug Components: build Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.0 Attachments: 8654.txt The src.xml assembly under hbase-assembly does not include the hbase-hadoop2-compat (more joys of maven -- uses default profile which is hadoop1 when making module set so we have to do perverse include of the hbase-hadoop2-compat). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8631: - Attachment: (was: hbase-8631-v4.patch) Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8631: - Attachment: hbase-8631-v4.patch Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8641) IndexBuilder example : CF name of the src table is hard coded
[ https://issues.apache.org/jira/browse/HBASE-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670114#comment-13670114 ] Anoop Sam John commented on HBASE-8641: --- Thanks Ram. Will commit tonight IST IndexBuilder example : CF name of the src table is hard coded - Key: HBASE-8641 URL: https://issues.apache.org/jira/browse/HBASE-8641 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Attachments: HBASE-8641.patch When running the IndexBuilder example we can pass the tablename, family name and qualifier name for indexing that data. But in the code the family name is hard coded to be only attributes. So this example will work only when family name of the src table is attributes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8655) Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true)
[ https://issues.apache.org/jira/browse/HBASE-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670113#comment-13670113 ] Anoop Sam John commented on HBASE-8655: --- Simple port. Will commit tonight if there is no objection Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true) --- Key: HBASE-8655 URL: https://issues.apache.org/jira/browse/HBASE-8655 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.0 Reporter: Anoop Sam John Fix For: 0.94.9 Attachments: HBASE-8655.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8631) Meta Region First Recovery
[
https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670126#comment-13670126
]
Hadoop QA commented on HBASE-8631:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585366/hbase-8631-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5877//console
This message is automatically generated.
Meta Region First Recovery
--
Key: HBASE-8631
URL: https://issues.apache.org/jira/browse/HBASE-8631
Project: HBase
Issue Type: Bug
Components: MTTR
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Attachments: hbase-8631.patch, hbase-8631-v2.patch,
hbase-8631-v3.patch, hbase-8631-v4.patch
We have a separate wal for meta region. While log splitting logic haven't
taken the advantage of this and splitlogworker still picks a wal file
randomly. Imaging if we have multiple region servers including meta RS fails
about the same time while meta wal is recovered last, all failed regions have
to wait meta recovered and then can be online again.
The open JIRA is to let splitlogworker to pick a meta wal file firstly and
then others.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8631) Meta Region First Recovery
[
https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670162#comment-13670162
]
Hadoop QA commented on HBASE-8631:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585374/hbase-8631-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5878//console
This message is automatically generated.
Meta Region First Recovery
--
Key: HBASE-8631
URL: https://issues.apache.org/jira/browse/HBASE-8631
Project: HBase
Issue Type: Bug
Components: MTTR
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Attachments: hbase-8631.patch, hbase-8631-v2.patch,
hbase-8631-v3.patch, hbase-8631-v4.patch
We have a separate wal for meta region. While log splitting logic haven't
taken the advantage of this and splitlogworker still picks a wal file
randomly. Imaging if we have multiple region servers including meta RS fails
about the same time while meta wal is recovered last, all failed regions have
to wait meta recovered and then can be online again.
The open JIRA is to let splitlogworker to pick a meta wal file firstly and
then others.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5433) [REST] Add metrics to keep track of success/failure count
[
https://issues.apache.org/jira/browse/HBASE-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Huang updated HBASE-5433:
--
Description:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);的
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
was:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
[REST] Add metrics to keep track of success/failure count
-
Key: HBASE-5433
URL: https://issues.apache.org/jira/browse/HBASE-5433
Project: HBase
Issue Type: Improvement
Components: metrics, REST
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Mubarak Seyed
Labels: noob
Fix For: 0.94.0, 0.95.0
Attachments: HBASE-5433.trunk.v1.patch
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of
the method implementation and it is very hard to monitor requests (unless
turn on debug, find the row_key and validate it in get/scan using hbase
shell), it will be very useful to ops to keep an eye as requests from cross
data-centers are trying to write data to one cluster using REST gateway
through load balancer (and there is no visibility of which REST-server/RS
failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);的
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
--
This message is
[jira] [Updated] (HBASE-5433) [REST] Add metrics to keep track of success/failure count
[
https://issues.apache.org/jira/browse/HBASE-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Huang updated HBASE-5433:
--
Description:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
was:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);的
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
[REST] Add metrics to keep track of success/failure count
-
Key: HBASE-5433
URL: https://issues.apache.org/jira/browse/HBASE-5433
Project: HBase
Issue Type: Improvement
Components: metrics, REST
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Mubarak Seyed
Labels: noob
Fix For: 0.94.0, 0.95.0
Attachments: HBASE-5433.trunk.v1.patch
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of
the method implementation and it is very hard to monitor requests (unless
turn on debug, find the row_key and validate it in get/scan using hbase
shell), it will be very useful to ops to keep an eye as requests from cross
data-centers are trying to write data to one cluster using REST gateway
through load balancer (and there is no visibility of which REST-server/RS
failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
--
This message is
[jira] [Updated] (HBASE-5433) [REST] Add metrics to keep track of success/failure count
[
https://issues.apache.org/jira/browse/HBASE-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Huang updated HBASE-5433:
--
Description:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
was:
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of the
method implementation and it is very hard to monitor requests (unless turn on
debug, find the row_key and validate it in get/scan using hbase shell), it will
be very useful to ops to keep an eye as requests from cross data-centers are
trying to write data to one cluster using REST gateway through load balancer
(and there is no visibility of which REST-server/RS failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
[REST] Add metrics to keep track of success/failure count
-
Key: HBASE-5433
URL: https://issues.apache.org/jira/browse/HBASE-5433
Project: HBase
Issue Type: Improvement
Components: metrics, REST
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Mubarak Seyed
Labels: noob
Fix For: 0.94.0, 0.95.0
Attachments: HBASE-5433.trunk.v1.patch
In a production environment, the visibility of successful REST request(s) are
not getting exposed to metric system as we have only one metric (requests)
today.
Proposing to add more metrics such as successful_get_count, failed_get_count,
successful_put_count, failed_put_count
The current implementation increases the request count at the beginning of
the method implementation and it is very hard to monitor requests (unless
turn on debug, find the row_key and validate it in get/scan using hbase
shell), it will be very useful to ops to keep an eye as requests from cross
data-centers are trying to write data to one cluster using REST gateway
through load balancer (and there is no visibility of which REST-server/RS
failed to write data)
{code}
Response update(final CellSetModel model, final boolean replace) {
// for requests
servlet.getMetrics().incrementRequests(1);
..
..
table.put(puts);
table.flushCommits();
ResponseBuilder response = Response.ok();
// for successful_get_count
servlet.getMetrics().incrementSuccessfulGetRequests(1);
return response.build();
} catch (IOException e) {
// for failed_get_count
servlet.getMetrics().incrementFailedGetRequests(1);
throw new WebApplicationException(e,
Response.Status.SERVICE_UNAVAILABLE);
} finally {
}
}
{code}
--
This message is
[jira] [Commented] (HBASE-6364) Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table
[ https://issues.apache.org/jira/browse/HBASE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670176#comment-13670176 ] stack commented on HBASE-6364: -- @nkeywal nevermind. The issue I am seeing is not particular to this issue. Please ignore. Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table - Key: HBASE-6364 URL: https://issues.apache.org/jira/browse/HBASE-6364 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.90.6, 0.92.1, 0.94.0 Reporter: Suraj Varma Assignee: Nicolas Liochon Labels: client Fix For: 0.94.2 Attachments: 6364.94.v2.nolargetest.patch, 6364.94.v2.nolargetest.security-addendum.patch, 6364-host-serving-META.v1.patch, 6364.v11.nolargetest.patch, 6364.v1.patch, 6364.v1.patch, 6364.v2.patch, 6364.v3.patch, 6364.v3.patch, 6364.v5.patch, 6364.v5.withtests.patch, 6364.v6.patch, 6364.v6.withtests.patch, 6364.v7.withtests.patch, 6364.v8.withtests.patch, 6364.v9.patch, stacktrace.txt When a server host with a Region Server holding the .META. table is powered down on a live cluster, while the HBase cluster itself detects and reassigns the .META. table, connected HBase Client's take an excessively long time to detect this and re-discover the reassigned .META. Workaround: Decrease the ipc.socket.timeout on HBase Client side to a low value (default is 20s leading to 35 minute recovery time; we were able to get acceptable results with 100ms getting a 3 minute recovery) This was found during some hardware failure testing scenarios. Test Case: 1) Apply load via client app on HBase cluster for several minutes 2) Power down the region server holding the .META. server (i.e. power off ... and keep it off) 3) Measure how long it takes for cluster to reassign META table and for client threads to re-lookup and re-orient to the lesser cluster (minus the RS and DN on that host). Observation: 1) Client threads spike up to maxThreads size ... and take over 35 mins to recover (i.e. for the thread count to go back to normal) - no client calls are serviced - they just back up on a synchronized method (see #2 below) 2) All the client app threads queue up behind the oahh.ipc.HBaseClient#setupIOStreams method http://tinyurl.com/7js53dj After taking several thread dumps we found that the thread within this synchronized method was blocked on NetUtils.connect(this.socket, remoteId.getAddress(), getSocketTimeout(conf)); The client thread that gets the synchronized lock would try to connect to the dead RS (till socket times out after 20s), retries, and then the next thread gets in and so forth in a serial manner. Workaround: --- Default ipc.socket.timeout is set to 20s. We dropped this to a low number (1000 ms, 100 ms, etc) on the client side hbase-site.xml. With this setting, the client threads recovered in a couple of minutes by failing fast and re-discovering the .META. table on a reassigned RS. Assumption: This ipc.socket.timeout is only ever used during the initial HConnection setup via the NetUtils.connect and should only ever be used when connectivity to a region server is lost and needs to be re-established. i.e it does not affect the normal RPC actiivity as this is just the connect timeout. During RS GC periods, any _new_ clients trying to connect will fail and will require .META. table re-lookups. This above timeout workaround is only for the HBase client side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670178#comment-13670178 ] Ted Yu commented on HBASE-8631: --- +1 Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8645: - Attachment: 8645v2.txt I missed a conversion (in the ServerName#toString). Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8656) Rpc call may not be notified in SecureClient
cuijianwei created HBASE-8656:
-
Summary: Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new RemoteException(WritableUtils.readString(in),
WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call map
by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map of
this connection will be notified to receive an exception. However, the current
rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670204#comment-13670204
]
Hadoop QA commented on HBASE-8645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585388/8645v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestServerName
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5879//console
This message is automatically generated.
Change ServerName so it uses hostname only, not FQDN hostnames
---
Key: HBASE-8645
URL: https://issues.apache.org/jira/browse/HBASE-8645
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 8645.txt, 8645.txt, 8645v2.txt
No need of dragging around domain part of a hostname when we make
ServerNames. Rather than do a.example.org,6,12345 as we currently do,
just output: 1,6,12345. This will make names displayed in UI and or shell
smaller. Will also tighten up our logs a little, especially where ServerName
is part of a thrad name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670206#comment-13670206
]
Hadoop QA commented on HBASE-8645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585388/8645v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestServerName
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5880//console
This message is automatically generated.
Change ServerName so it uses hostname only, not FQDN hostnames
---
Key: HBASE-8645
URL: https://issues.apache.org/jira/browse/HBASE-8645
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 8645.txt, 8645.txt, 8645v2.txt
No need of dragging around domain part of a hostname when we make
ServerNames. Rather than do a.example.org,6,12345 as we currently do,
just output: 1,6,12345. This will make names displayed in UI and or shell
smaller. Will also tighten up our logs a little, especially where ServerName
is part of a thrad name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8645: - Attachment: 8645v3.txt Here we go again (I love unit tests!) Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670247#comment-13670247
]
Hadoop QA commented on HBASE-8645:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585391/8645v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestServerName
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5881//console
This message is automatically generated.
Change ServerName so it uses hostname only, not FQDN hostnames
---
Key: HBASE-8645
URL: https://issues.apache.org/jira/browse/HBASE-8645
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt
No need of dragging around domain part of a hostname when we make
ServerNames. Rather than do a.example.org,6,12345 as we currently do,
just output: 1,6,12345. This will make names displayed in UI and or shell
smaller. Will also tighten up our logs a little, especially where ServerName
is part of a thrad name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8645: - Attachment: 8645v4.txt Try again Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8654) src assembly does not include hbase-hadoop2-compat module
[ https://issues.apache.org/jira/browse/HBASE-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670285#comment-13670285 ] Hudson commented on HBASE-8654: --- Integrated in hbase-0.95-on-hadoop2 #117 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/117/]) HBASE-8654 src assembly does not include hbase-hadoop2-compat module (Revision 1487714) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-assembly/src/main/assembly/src.xml src assembly does not include hbase-hadoop2-compat module - Key: HBASE-8654 URL: https://issues.apache.org/jira/browse/HBASE-8654 Project: HBase Issue Type: Bug Components: build Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.0 Attachments: 8654.txt The src.xml assembly under hbase-assembly does not include the hbase-hadoop2-compat (more joys of maven -- uses default profile which is hadoop1 when making module set so we have to do perverse include of the hbase-hadoop2-compat). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3787) Increment is non-idempotent but client retries RPC
[ https://issues.apache.org/jira/browse/HBASE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670286#comment-13670286 ] stack commented on HBASE-3787: -- HDFS trying to solve similar issue HDFS-4849. Sergey, you are not first to have this issue it seems (smile). It even has a 'name' http://www.freesoft.org/CIE/RFC/1813/47.htm (via our Todd). Some ok suggested improvements in here: https://www.kernel.org/doc/ols/2009/ols2009-pages-95-100.pdf Increment is non-idempotent but client retries RPC -- Key: HBASE-3787 URL: https://issues.apache.org/jira/browse/HBASE-3787 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.4, 0.95.2 Reporter: dhruba borthakur Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.95.1 Attachments: HBASE-3787-partial.patch, HBASE-3787-v0.patch, HBASE-3787-v1.patch, HBASE-3787-v2.patch, HBASE-3787-v3.patch, HBASE-3787-v4.patch, HBASE-3787-v5.patch, HBASE-3787-v5.patch The HTable.increment() operation is non-idempotent. The client retries the increment RPC a few times (as specified by configuration) before throwing an error to the application. This makes it possible that the same increment call be applied twice at the server. For increment operations, is it better to use HConnectionManager.getRegionServerWithoutRetries()? Another option would be to enhance the IPC module to make the RPC server correctly identify if the RPC is a retry attempt and handle accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670290#comment-13670290
]
Hadoop QA commented on HBASE-8645:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585397/8645v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5882//console
This message is automatically generated.
Change ServerName so it uses hostname only, not FQDN hostnames
---
Key: HBASE-8645
URL: https://issues.apache.org/jira/browse/HBASE-8645
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt
No need of dragging around domain part of a hostname when we make
ServerNames. Rather than do a.example.org,6,12345 as we currently do,
just output: 1,6,12345. This will make names displayed in UI and or shell
smaller. Will also tighten up our logs a little, especially where ServerName
is part of a thrad name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8649) Private method HStore#createWriterInTmp(long) is never called
[ https://issues.apache.org/jira/browse/HBASE-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670299#comment-13670299 ] Hudson commented on HBASE-8649: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #548 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/548/]) HBASE-8649 Private method HStore#createWriterInTmp(long) is never called (Ted Yu) (Revision 1487702) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java Private method HStore#createWriterInTmp(long) is never called - Key: HBASE-8649 URL: https://issues.apache.org/jira/browse/HBASE-8649 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Priority: Minor Fix For: 0.98.0 Attachments: 8649.txt HStore#createWriterInTmp(long) is never called. We can drop the method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8654) src assembly does not include hbase-hadoop2-compat module
[ https://issues.apache.org/jira/browse/HBASE-8654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670300#comment-13670300 ] Hudson commented on HBASE-8654: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #548 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/548/]) HBASE-8654 src assembly does not include hbase-hadoop2-compat module (Revision 1487713) Result = FAILURE stack : Files : * /hbase/trunk/hbase-assembly/src/main/assembly/src.xml src assembly does not include hbase-hadoop2-compat module - Key: HBASE-8654 URL: https://issues.apache.org/jira/browse/HBASE-8654 Project: HBase Issue Type: Bug Components: build Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.0 Attachments: 8654.txt The src.xml assembly under hbase-assembly does not include the hbase-hadoop2-compat (more joys of maven -- uses default profile which is hadoop1 when making module set so we have to do perverse include of the hbase-hadoop2-compat). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[ https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8653: -- Attachment: (was: 8653-v2.txt) master seems to be deleting region tmp directory from under compaction -- Key: HBASE-8653 URL: https://issues.apache.org/jira/browse/HBASE-8653 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Blocker Fix For: 0.95.1 Attachments: 8653-v2.txt, HBASE-8653-v0.patch Putting it in .1, feel free to move to .2. We have observed some compaction errors where the code was creating a new HDFS block, and the file would not exist. Upon investigation, we found the .tmp directory delete request on namenode from master IP shortly before that. There are no specific logs on master, but one thing running at that time was CatalogJanitor. CatalogJanitor calls HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact, everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp directory. We didn't go thru details on how it arrived there (or if there may have been other culprit), but it appears that deleting stuff if (readOnly) is not the intended behavior and it should be if (!readOnly). Given that readOnly is not really used (or rather is always true except some inconsequential usage in test) perhaps entire cleanup should be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[ https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8653: -- Attachment: 8653-v2.txt master seems to be deleting region tmp directory from under compaction -- Key: HBASE-8653 URL: https://issues.apache.org/jira/browse/HBASE-8653 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Blocker Fix For: 0.95.1 Attachments: 8653-v2.txt, HBASE-8653-v0.patch Putting it in .1, feel free to move to .2. We have observed some compaction errors where the code was creating a new HDFS block, and the file would not exist. Upon investigation, we found the .tmp directory delete request on namenode from master IP shortly before that. There are no specific logs on master, but one thing running at that time was CatalogJanitor. CatalogJanitor calls HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact, everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp directory. We didn't go thru details on how it arrived there (or if there may have been other culprit), but it appears that deleting stuff if (readOnly) is not the intended behavior and it should be if (!readOnly). Given that readOnly is not really used (or rather is always true except some inconsequential usage in test) perhaps entire cleanup should be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8534: Attachment: HBASE-8534-trunk-f.patch HBASE-8534-0.94-f.patch fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670368#comment-13670368
]
stack commented on HBASE-8645:
--
Here is how it changes lot output:
Before this patch:
{code}
2013-05-30 03:54:21,542 INFO
[RegionServer:0;asf001.sp2.ygridcore.net,58265,1369886061348]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:58265
connecting to ZooKeeper ensemble=localhost:60188
2013-05-30 03:54:21,544 INFO
[RegionServer:1;asf001.sp2.ygridcore.net,56020,1369886061408]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:56020
connecting to ZooKeeper ensemble=localhost:60188
2013-05-30 03:54:21,545 DEBUG
[RegionServer:0;asf001.sp2.ygridcore.net,58265,1369886061348-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:58265 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-30 03:54:21,546 INFO
[RegionServer:2;asf001.sp2.ygridcore.net,59675,1369886061430]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:59675
connecting to ZooKeeper ensemble=localhost:60188
2013-05-30 03:54:21,547 DEBUG
[RegionServer:1;asf001.sp2.ygridcore.net,56020,1369886061408-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:56020 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-30 03:54:21,547 DEBUG
[RegionServer:0;asf001.sp2.ygridcore.net,58265,1369886061348-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:58265-0x13ef3926fe10001 connected
2013-05-30 03:54:21,549 DEBUG
[RegionServer:1;asf001.sp2.ygridcore.net,56020,1369886061408-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:56020-0x13ef3926fe10002 connected
2013-05-30 03:54:21,550 DEBUG
[RegionServer:1;asf001.sp2.ygridcore.net,56020,1369886061408]
zookeeper.ZKUtil(431): regionserver:56020-0x13ef3926fe10002 Set watcher on
existing znode=/hbase/master
2013-05-30 03:54:21,550 DEBUG
[RegionServer:0;asf001.sp2.ygridcore.net,58265,1369886061348]
zookeeper.ZKUtil(431): regionserver:58265-0x13ef3926fe10001 Set watcher on
existing znode=/hbase/master
2013-05-30 03:54:21,551 DEBUG
[RegionServer:2;asf001.sp2.ygridcore.net,59675,1369886061430-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:59675 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
{code}
After this patch:
{code}
...
2013-05-30 11:53:17,685 INFO [RS:1;asf000,49894,1369914797554]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:49894
connecting to ZooKeeper ensemble=localhost:53298
2013-05-30 11:53:17,685 INFO [RS:2;asf000,54417,1369914797577]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:54417
connecting to ZooKeeper ensemble=localhost:53298
2013-05-30 11:53:17,686 INFO [RS:0;asf000,52780,1369914797476]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:52780
connecting to ZooKeeper ensemble=localhost:53298
2013-05-30 11:53:17,689 DEBUG [RS:1;asf000,49894,1369914797554-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:49894 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-30 11:53:17,689 DEBUG [RS:2;asf000,54417,1369914797577-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:54417 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-30 11:53:17,690 DEBUG [RS:0;asf000,52780,1369914797476-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:52780 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-30 11:53:17,690 DEBUG [RS:1;asf000,49894,1369914797554-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:49894-0x13ef548eb090001 connected
2013-05-30 11:53:17,691 DEBUG [RS:2;asf000,54417,1369914797577-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:54417-0x13ef548eb090002 connected
2013-05-30 11:53:17,691 DEBUG [RS:0;asf000,52780,1369914797476-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:52780-0x13ef548eb090003 connected
2013-05-30 11:53:17,692 DEBUG [RS:2;asf000,54417,1369914797577]
zookeeper.ZKUtil(431): regionserver:54417-0x13ef548eb090002 Set watcher on
existing znode=/hbase/master
2013-05-30 11:53:17,692 DEBUG [RS:0;asf000,52780,1369914797476]
zookeeper.ZKUtil(431): regionserver:52780-0x13ef548eb090003 Set watcher on
existing znode=/hbase/master
2013-05-30 11:53:17,693 DEBUG [RS:1;asf000,49894,1369914797554]
zookeeper.ZKUtil(431): regionserver:49894-0x13ef548eb090001 Set watcher on
existing znode=/hbase/master
2013-05-30 11:53:17,699 DEBUG [RS:2;asf000,54417,1369914797577]
zookeeper.ZKUtil(433): regionserver:54417-0x13ef548eb090002 Set watcher on
znode that does not yet exist, /hbase/running
2013-05-30 11:53:17,699 DEBUG [RS:1;asf000,49894,1369914797554]
zookeeper.ZKUtil(433): regionserver:49894-0x13ef548eb090001 Set watcher on
znode that does not yet exist, /hbase/running
2013-05-30 11:53:17,699 DEBUG
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670370#comment-13670370 ] Aleksey Gorshkov commented on HBASE-8534: - Nick, I suppose that there is no good solution with change System.getSecurityManager() in case if multiple tests are run in parallel in one JVM. I propose solution with ExitUtil. This solution id good in case if multiple tests are run in parallel in one JVM. About TestTableMapReduceUtil. I hope that this is junit tests and here will test only default configuration. Test markers was changed. new patches. patch HBASE-8534-0.94-f.patch for branch-0.94 patch HBASE-8534-trunk-f.patch for branch-0.95 and trunk fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670385#comment-13670385 ] Julian Zhou commented on HBASE-8642: Hi [~mbertozzi], I was just adding shell support just now. Curious about rename_snapshot could not work? looks like HBaseAdmin has no #renameSnapshot? [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670397#comment-13670397 ] Julian Zhou commented on HBASE-8642: Oh, thanks [~mbertozzi] for pointing. I should do a search first.. ok, may not include into this stream? Another question is that why not define more functions in admin module pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex).. then shell/command/*.rb module could invoke them directly, currently, list_snapshots module has its own regex processing logic implemented? xxx.to_java_bytes is mandatory for ruby-java method call? [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[
https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670396#comment-13670396
]
Hadoop QA commented on HBASE-8653:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585412/8653-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5883//console
This message is automatically generated.
master seems to be deleting region tmp directory from under compaction
--
Key: HBASE-8653
URL: https://issues.apache.org/jira/browse/HBASE-8653
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Blocker
Fix For: 0.95.1
Attachments: 8653-v2.txt, HBASE-8653-v0.patch
Putting it in .1, feel free to move to .2.
We have observed some compaction errors where the code was creating a new
HDFS block, and the file would not exist. Upon investigation, we found the
.tmp directory delete request on namenode from master IP shortly before that.
There are no specific logs on master, but one thing running at that time was
CatalogJanitor. CatalogJanitor calls
HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact,
everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp
directory.
We didn't go thru details on how it arrived there (or if there may have been
other culprit), but it appears that deleting stuff if (readOnly) is not the
intended behavior and it should be if (!readOnly). Given that readOnly is not
really used (or rather is always true except some inconsequential usage in
test) perhaps entire cleanup should be removed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670389#comment-13670389 ] Matteo Bertozzi commented on HBASE-8642: [~julian.zhou] yeah rename_snapshot doesn't work, take a look at the last comment on HBASE-7218. [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8534: Status: Open (was: Patch Available) fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8534: Status: Patch Available (was: Open) fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670411#comment-13670411 ] Nick Dimiduk commented on HBASE-8534: - Regarding the junit tests being run in an isolated environment with default configs, I think you're right, that should be the common case. I'm thinking about a user who is running the test suite as a post-install smoke-test (as I believe we are advocating for 0.95+). In that scenario, their site-config can be picked up. I expect they would be running integration tests only, so this is probably fine. Regarding ExitUtil, managing shared state in ExitUtil is equivalent to overriding the shared state that is a SecurityManager. I'd rather see the SecurityManager (with my previous comment about the constructor logic being fixed) over this ExitUtil implementation; the SecurityManager is less intrusive to non-test code. [~nkeywal], [~apurtell] do either of you have an opinion on ExitUtil vs the custom SecurityManager in patch e? fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670415#comment-13670415 ] Nick Dimiduk commented on HBASE-8534: - Also, can a JIRA admin tweak [~aleksgor]'s account so issues can be assigned to him? He's made quite a few contributions now. fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8551) Run IntegrationTestBigLinkedList and Bakunin concurrently against 0.95 trunk now hbase-7006 is in
[ https://issues.apache.org/jira/browse/HBASE-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-8551. -- Resolution: Duplicate Fix Version/s: (was: 0.95.2) 0.95.1 Marking duplicate of HBASE-8583 (though latter is subtask of this one) Run IntegrationTestBigLinkedList and Bakunin concurrently against 0.95 trunk now hbase-7006 is in - Key: HBASE-8551 URL: https://issues.apache.org/jira/browse/HBASE-8551 Project: HBase Issue Type: Task Reporter: stack Priority: Critical Fix For: 0.95.1 New critical task; run IntegrationTestBigLinkedList on cluster to verify no dataloss now we have hbase-7006 in place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes)
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670421#comment-13670421 ] stack commented on HBASE-7055: -- [~ndimiduk] If according to Sergey, its hard to find a use for it, yeah, I get cold feet on commit. [~sershe] You should mark what you want to get into 0.95 w/ 0.95.2 so gets reviewed... (I'm only looking at 0.95 stuff these times). Lets move this issue out of 0.95 if we want stripes? port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes) -- Key: HBASE-7055 URL: https://issues.apache.org/jira/browse/HBASE-7055 Project: HBase Issue Type: Task Components: Compaction Affects Versions: 0.95.2 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.95.1 Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, HBASE-6371-v3-refactor-only-squashed.patch, HBASE-6371-v4-refactor-only-squashed.patch, HBASE-6371-v5-refactor-only-squashed.patch, HBASE-7055-v0.patch, HBASE-7055-v1.patch, HBASE-7055-v2.patch, HBASE-7055-v3.patch, HBASE-7055-v4.patch, HBASE-7055-v5.patch, HBASE-7055-v6.patch, HBASE-7055-v7.patch, HBASE-7055-v7.patch See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[
https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670423#comment-13670423
]
Hadoop QA commented on HBASE-8534:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585414/HBASE-8534-trunk-f.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 25 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
org.apache.hadoop.hbase.master.TestTableLockManager
org.apache.hadoop.hbase.security.access.TestAccessController
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.rest.TestTableResource.testTableListXML(TestTableResource.java:188)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5884//console
This message is automatically generated.
fix coverage org.apache.hadoop.hbase.mapreduce
--
Key: HBASE-8534
URL: https://issues.apache.org/jira/browse/HBASE-8534
Project: HBase
Issue Type: Test
Affects Versions: 0.94.8, 0.95.2
Reporter: Aleksey Gorshkov
Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch,
HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch,
HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch,
HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch
fix coverage org.apache.hadoop.hbase.mapreduce
patch HBASE-8534-0.94.patch for branch-0.94
patch HBASE-8534-trunk.patch for branch-0.95 and trunk
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670426#comment-13670426
]
Matteo Bertozzi commented on HBASE-8642:
{quote}Another question is that why not define more functions in admin module
pointing to e.g. HBaseAdmin#listSnapshots(regex), #deleteSnapshots(regex)..
then shell/command/*.rb module could invoke them directly, currently,
list_snapshots module has its own regex processing logic implemented?
xxx.to_java_bytes is mandatory for ruby-java method call?{quote}
No particular reason, the admin.listSnapshots(regex) co were added later, and
the shell was not updated to use them. and since the regex stuff is just a loop
with an if there's no real advantage of having it in the admin, aside saving
you from write 3 extra lines of code.
Jon was suggesting to have extra logic in the shell for stuff like delete with
regex, where you may want to add an are you sure? check in front.
[Snapshot] List and delete snapshot by table
Key: HBASE-8642
URL: https://issues.apache.org/jira/browse/HBASE-8642
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Julian Zhou
Priority: Minor
Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Attachments: 8642-trunk-0.95-v0.patch
Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8344) Improve the assignment when node failures happen to choose the secondary RS as the new primary RS
[
https://issues.apache.org/jira/browse/HBASE-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670436#comment-13670436
]
Nick Dimiduk commented on HBASE-8344:
-
From {{segregateRegions}},
{noformat}
+ if (favoredNodes != null) {
{noformat}
When will the globalFavoredNodesAssignmentPlan not contain a list of favored
nodes for this region? Perhaps in the case of an empty Region?
Improve the assignment when node failures happen to choose the secondary RS
as the new primary RS
-
Key: HBASE-8344
URL: https://issues.apache.org/jira/browse/HBASE-8344
Project: HBase
Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
Priority: Critical
Fix For: 0.95.2
Attachments: hbase-8344-1.txt, hbase-8344-2.1.txt, hbase-8344-2.2.txt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
stack created HBASE-8657: Summary: Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8657: - Attachment: fixups.txt Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Attachments: fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8657: - Assignee: stack Status: Patch Available (was: Open) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[
https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670440#comment-13670440
]
stack commented on HBASE-8657:
--
Here is what is in the patch:
{code}
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
Minor tighter LOG formatting
M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
Made it so you need to turn on trace to see location refreshes (too much
logging
of duplicate actions)
Slight tightening of logs
Comment clean up. Add count of retries to log.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
Make meta scan setup trace level (client scanner logs at debug when it goes
to scan.
We were logging the scan setup twice)
M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java
Minor fixup.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
Fix NPE seen in hbase-it logs when all servers killed... we were NPE'ing
trying to
get .META. location.
M hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
Set client retries same as hbase-default says they are.
M hbase-it/src/test/java/org/apache/hadoop/hbase/util/ChaosMonkey.java
Include my little chaos monkey NPE fix.
{code}
Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
-
Key: HBASE-8657
URL: https://issues.apache.org/jira/browse/HBASE-8657
Project: HBase
Issue Type: Sub-task
Reporter: stack
Attachments: fixups.txt
This is a miscellaneous set of fixups that come of my staring at hbase-it
logs trying to follow what is going on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[
https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670443#comment-13670443
]
Matteo Bertozzi commented on HBASE-8657:
{code}
- LOG.debug(Finished scanning region + this.currentRegion);
+ LOG.debug(Finished region= + this.currentRegion);
{code}
do you think that scanning is not useful?
(other than that looks good to me)
Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
-
Key: HBASE-8657
URL: https://issues.apache.org/jira/browse/HBASE-8657
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: fixups.txt
This is a miscellaneous set of fixups that come of my staring at hbase-it
logs trying to follow what is going on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[ https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670438#comment-13670438 ] Matteo Bertozzi commented on HBASE-8653: +1 on v2: if (!readOnly) master seems to be deleting region tmp directory from under compaction -- Key: HBASE-8653 URL: https://issues.apache.org/jira/browse/HBASE-8653 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Blocker Fix For: 0.95.1 Attachments: 8653-v2.txt, HBASE-8653-v0.patch Putting it in .1, feel free to move to .2. We have observed some compaction errors where the code was creating a new HDFS block, and the file would not exist. Upon investigation, we found the .tmp directory delete request on namenode from master IP shortly before that. There are no specific logs on master, but one thing running at that time was CatalogJanitor. CatalogJanitor calls HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact, everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp directory. We didn't go thru details on how it arrived there (or if there may have been other culprit), but it appears that deleting stuff if (readOnly) is not the intended behavior and it should be if (!readOnly). Given that readOnly is not really used (or rather is always true except some inconsequential usage in test) perhaps entire cleanup should be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8641) IndexBuilder example : CF name of the src table is hard coded
[ https://issues.apache.org/jira/browse/HBASE-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-8641: -- Status: Patch Available (was: Open) IndexBuilder example : CF name of the src table is hard coded - Key: HBASE-8641 URL: https://issues.apache.org/jira/browse/HBASE-8641 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Attachments: HBASE-8641.patch When running the IndexBuilder example we can pass the tablename, family name and qualifier name for indexing that data. But in the code the family name is hard coded to be only attributes. So this example will work only when family name of the src table is attributes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8641) IndexBuilder example : CF name of the src table is hard coded
[ https://issues.apache.org/jira/browse/HBASE-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-8641: -- Resolution: Fixed Fix Version/s: 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to Trunk. IndexBuilder example : CF name of the src table is hard coded - Key: HBASE-8641 URL: https://issues.apache.org/jira/browse/HBASE-8641 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Minor Fix For: 0.98.0 Attachments: HBASE-8641.patch When running the IndexBuilder example we can pass the tablename, family name and qualifier name for indexing that data. But in the code the family name is hard coded to be only attributes. So this example will work only when family name of the src table is attributes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670478#comment-13670478 ] stack commented on HBASE-8657: -- I think it redundant since the logger is the ClientScanner; that should be scanning context enough. Thanks Matteo. (In general this seems nitpicky but I'm trying to follow what is going on in hbase-it tests and I want to get it so I can look at a log file on my laptop and see all the important stuff w/o having to scroll right...). Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8655) Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true)
[ https://issues.apache.org/jira/browse/HBASE-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670483#comment-13670483 ] Himanshu Vashishtha commented on HBASE-8655: +1 Backport to 94 - HBASE-8346(Prefetching .META. rows in case only when useCache is set to true) --- Key: HBASE-8655 URL: https://issues.apache.org/jira/browse/HBASE-8655 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.0 Reporter: Anoop Sam John Fix For: 0.94.9 Attachments: HBASE-8655.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670485#comment-13670485 ] Jeffrey Zhong commented on HBASE-8631: -- [~saint@gmail.com] Could this patch be check in 0.95 which fixes the issues found in my integration tests of hbase-7006? Thanks. Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8631: - Fix Version/s: 0.95.1 0.98.0 Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.98.0, 0.95.1 Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Zhou updated HBASE-8642: --- Attachment: 8642-trunk-0.95-v1.patch Hi, [~mbertozzi], applied the changes. Could you help review? Thanks~ [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670504#comment-13670504 ] Matteo Bertozzi commented on HBASE-8642: v1 looks good to me [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[
https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670507#comment-13670507
]
Hadoop QA commented on HBASE-8657:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585427/fixups.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5885//console
This message is automatically generated.
Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
-
Key: HBASE-8657
URL: https://issues.apache.org/jira/browse/HBASE-8657
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: fixups.txt
This is a miscellaneous set of fixups that come of my staring at hbase-it
logs trying to follow what is going on.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8631) Meta Region First Recovery
[ https://issues.apache.org/jira/browse/HBASE-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8631: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.95. Thanks for the patch J and for the reviews lads. Meta Region First Recovery -- Key: HBASE-8631 URL: https://issues.apache.org/jira/browse/HBASE-8631 Project: HBase Issue Type: Bug Components: MTTR Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.98.0, 0.95.1 Attachments: hbase-8631.patch, hbase-8631-v2.patch, hbase-8631-v3.patch, hbase-8631-v4.patch We have a separate wal for meta region. While log splitting logic haven't taken the advantage of this and splitlogworker still picks a wal file randomly. Imaging if we have multiple region servers including meta RS fails about the same time while meta wal is recovered last, all failed regions have to wait meta recovered and then can be online again. The open JIRA is to let splitlogworker to pick a meta wal file firstly and then others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8657: - Attachment: fixup2.txt Here is what I applied. Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: fixup2.txt, fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[ https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670520#comment-13670520 ] Sergey Shelukhin commented on HBASE-8653: - I will commit v2 later today if there are no objections master seems to be deleting region tmp directory from under compaction -- Key: HBASE-8653 URL: https://issues.apache.org/jira/browse/HBASE-8653 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Blocker Fix For: 0.95.1 Attachments: 8653-v2.txt, HBASE-8653-v0.patch Putting it in .1, feel free to move to .2. We have observed some compaction errors where the code was creating a new HDFS block, and the file would not exist. Upon investigation, we found the .tmp directory delete request on namenode from master IP shortly before that. There are no specific logs on master, but one thing running at that time was CatalogJanitor. CatalogJanitor calls HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact, everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp directory. We didn't go thru details on how it arrived there (or if there may have been other culprit), but it appears that deleting stuff if (readOnly) is not the intended behavior and it should be if (!readOnly). Given that readOnly is not really used (or rather is always true except some inconsequential usage in test) perhaps entire cleanup should be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8657: - Resolution: Fixed Fix Version/s: 0.95.1 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Thanks for review Matteo. v2 adds printing out duration spent retrying failed write. Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.1 Attachments: fixup2.txt, fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8647) ChaosMonkey NPE
[
https://issues.apache.org/jira/browse/HBASE-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8647.
--
Resolution: Invalid
Fixed by the commit against HBASE-8657
ChaosMonkey NPE
---
Key: HBASE-8647
URL: https://issues.apache.org/jira/browse/HBASE-8647
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Attachments: 8647.txt
Running tests I see this from time to time:
{code}
19 2013-05-29 13:42:37,868 INFO [Thread-476]
util.ChaosMonkey$RestartRandomRs(274): Performing action: Restart random
region server
20 2013-05-29 13:42:37,869 WARN [Thread-476]
util.ChaosMonkey$PeriodicRandomActionPolicy(578): Exception occured during
performing action: java.lang.NullPointerException
21 ,...at
org.apache.hadoop.hbase.util.ChaosMonkey$Action.getCurrentServers(ChaosMonkey.java:160)
22 ,...at
org.apache.hadoop.hbase.util.ChaosMonkey$RestartRandomRs.perform(ChaosMonkey.java:275)
23 ,...at
org.apache.hadoop.hbase.util.ChaosMonkey$PeriodicRandomActionPolicy.runOneIteration(ChaosMonkey.java:576)
24 ,...at
org.apache.hadoop.hbase.util.ChaosMonkey$PeriodicPolicy.run(ChaosMonkey.java:488)
25 ,...at
org.apache.hadoop.hbase.util.ChaosMonkey$CompositeSequentialPolicy.run(ChaosMonkey.java:458)
26 ,...at java.lang.Thread.run(Thread.java:680)
{code}
Our monkey has killed everything.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8652) Number of compacting KVs is not reset at the end of compaction
[
https://issues.apache.org/jira/browse/HBASE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670539#comment-13670539
]
Sergey Shelukhin commented on HBASE-8652:
-
compaction uses current progress until the next compaction, at which point it
is reset, so totalCompactingKVs as well as current are reset implicitly. What I
wonder about is how did this ever work with parallel compactions, which are
possible. Perhaps progress should be moved to CompactionContext object.
Number of compacting KVs is not reset at the end of compaction
--
Key: HBASE-8652
URL: https://issues.apache.org/jira/browse/HBASE-8652
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Looking at master:60010/master-status#compactStas , I noticed that 'Num.
Compacting KVs' column stays unchanged at non-zero value(s).
In DefaultCompactor#compact(), we have this at the beginning:
{code}
this.progress = new CompactionProgress(fd.maxKeyCount);
{code}
But progress.totalCompactingKVs is not reset at the end of compact().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8652) Number of compacting KVs is not reset at the end of compaction
[
https://issues.apache.org/jira/browse/HBASE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670540#comment-13670540
]
Sergey Shelukhin commented on HBASE-8652:
-
(what I meant to say by reset implicitly is that it seems to be by design to
show last compaction until the next one, if everything works correctly current
and total should be the same so it'd show 100% compaction progress).
Number of compacting KVs is not reset at the end of compaction
--
Key: HBASE-8652
URL: https://issues.apache.org/jira/browse/HBASE-8652
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Looking at master:60010/master-status#compactStas , I noticed that 'Num.
Compacting KVs' column stays unchanged at non-zero value(s).
In DefaultCompactor#compact(), we have this at the beginning:
{code}
this.progress = new CompactionProgress(fd.maxKeyCount);
{code}
But progress.totalCompactingKVs is not reset at the end of compact().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670543#comment-13670543 ] Sergey Shelukhin commented on HBASE-8657: - Not that it matters for this patch, just wondering Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.1 Attachments: fixup2.txt, fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8657) Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs
[ https://issues.apache.org/jira/browse/HBASE-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670542#comment-13670542 ] Sergey Shelukhin commented on HBASE-8657: - Why remove []-s around toStringBinary, it makes it harder to read (or sed) from the log Miscellaneous log fixups for hbase-it; tidier logging, fix a few NPEs - Key: HBASE-8657 URL: https://issues.apache.org/jira/browse/HBASE-8657 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 0.98.0, 0.95.1 Attachments: fixup2.txt, fixups.txt This is a miscellaneous set of fixups that come of my staring at hbase-it logs trying to follow what is going on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8658) hbase clean is deaf to the --config DIR option
stack created HBASE-8658: Summary: hbase clean is deaf to the --config DIR option Key: HBASE-8658 URL: https://issues.apache.org/jira/browse/HBASE-8658 Project: HBase Issue Type: Bug Reporter: stack We need this doing migrations. I'd think lots of folks will have their configs other than default location (I need it testing migration) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8658) hbase clean is deaf to the --config DIR option
[ https://issues.apache.org/jira/browse/HBASE-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8658: - Fix Version/s: 0.95.1 Assignee: stack hbase clean is deaf to the --config DIR option -- Key: HBASE-8658 URL: https://issues.apache.org/jira/browse/HBASE-8658 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 We need this doing migrations. I'd think lots of folks will have their configs other than default location (I need it testing migration) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670557#comment-13670557
]
Hadoop QA commented on HBASE-8642:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585434/8642-trunk-0.95-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5886//console
This message is automatically generated.
[Snapshot] List and delete snapshot by table
Key: HBASE-8642
URL: https://issues.apache.org/jira/browse/HBASE-8642
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Julian Zhou
Priority: Minor
Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch
Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8658) hbase clean is deaf to the --config DIR option
[ https://issues.apache.org/jira/browse/HBASE-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8658: - Status: Patch Available (was: Open) hbase clean is deaf to the --config DIR option -- Key: HBASE-8658 URL: https://issues.apache.org/jira/browse/HBASE-8658 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 We need this doing migrations. I'd think lots of folks will have their configs other than default location (I need it testing migration) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-8534: --- Assignee: Aleksey Gorshkov fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Assignee: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670625#comment-13670625 ] Nicolas Liochon commented on HBASE-8534: bq. Also, can a JIRA admin tweak Aleksey Gorshkov's account so issues can be assigned to him? He's made quite a few contributions now. Done. bq. Ddo either of you have an opinion on ExitUtil vs the custom SecurityManager in patch e? No strong opinion. With the SecurityManager, do we have a risk that it conflict with the test driver itself? I don't understand what http://jira.codehaus.org/browse/SUREFIRE-767 says for example. On the other hand, the activeTest() is not really elegant. Lastly, using System.exit is often an anti pattern, could it be the root cause of the problem? fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Assignee: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8653) master seems to be deleting region tmp directory from under compaction
[
https://issues.apache.org/jira/browse/HBASE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670631#comment-13670631
]
Hadoop QA commented on HBASE-8653:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585412/8653-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5887//console
This message is automatically generated.
master seems to be deleting region tmp directory from under compaction
--
Key: HBASE-8653
URL: https://issues.apache.org/jira/browse/HBASE-8653
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Blocker
Fix For: 0.95.1
Attachments: 8653-v2.txt, HBASE-8653-v0.patch
Putting it in .1, feel free to move to .2.
We have observed some compaction errors where the code was creating a new
HDFS block, and the file would not exist. Upon investigation, we found the
.tmp directory delete request on namenode from master IP shortly before that.
There are no specific logs on master, but one thing running at that time was
CatalogJanitor. CatalogJanitor calls
HRegionFileSystem::openRegionFromFileSystem with readOnly == true (in fact,
everyone does); if readOnly is true, HRegionFileSystem nukes the .tmp
directory.
We didn't go thru details on how it arrived there (or if there may have been
other culprit), but it appears that deleting stuff if (readOnly) is not the
intended behavior and it should be if (!readOnly). Given that readOnly is not
really used (or rather is always true except some inconsequential usage in
test) perhaps entire cleanup should be removed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) fix coverage org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670638#comment-13670638 ] Nick Dimiduk commented on HBASE-8534: - bq. Lastly, using System.exit is often an anti pattern, could it be the root cause of the problem? That's really the root of the issue. IIRC, this is the same thing that complicates HBASE-8367. I'm not sure what the implications of SUREFIRE-767 mean. @Aleksey's original SecurityManager patch appeared to run correctly. I guess I'd prefer to see the SecurityManager implementation, minus the stateful constructor as the least-invasive. We can followup with a general purge of System.exit if that remains a pressing issue. Sorry for the churn, Aleksey. Would you mind also updating HBASE-8611 so as to use same implementation? fix coverage org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Assignee: Aleksey Gorshkov Attachments: HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8344) Improve the assignment when node failures happen to choose the secondary RS as the new primary RS
[ https://issues.apache.org/jira/browse/HBASE-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670662#comment-13670662 ] Devaraj Das commented on HBASE-8344: bq. When will the globalFavoredNodesAssignmentPlan not contain a list of favored nodes for this region? Actually, not all regions might have favored-nodes when they enter the method (FavoredNodeLoadBalancer.roundRobinAssignment). For example if the table is getting created with pre-split regions; the regions will have favored-nodes when the FavoredNodeLoadBalancer.roundRobinAssignment returns. OTOH a region could have favored-nodes when it enters the method - when we are trying to handle failure of its currently assigned regionserver. Improve the assignment when node failures happen to choose the secondary RS as the new primary RS - Key: HBASE-8344 URL: https://issues.apache.org/jira/browse/HBASE-8344 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.95.2 Attachments: hbase-8344-1.txt, hbase-8344-2.1.txt, hbase-8344-2.2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8637) IntegrationTestBigLinkedListWithChaosMonkey uses the wrong table name
[
https://issues.apache.org/jira/browse/HBASE-8637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670677#comment-13670677
]
Elliott Clark commented on HBASE-8637:
--
so on a real cluster running:
{code}hbase org.apache.hadoop.hbase.IntegrationTestsDriver -r
IntegrationTestBigLinkedListWithChaosMonkey{code}
results in the table IntegrationTestBigLinkedListWithChaosMonkey being created.
However here's what's in the table:
{code}
hbase(main):001:0 describe 'IntegrationTestBigLinkedListWithChaosMonkey'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hbase/jenkins-hbase-095-99/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hadoop/hadoop-2.0.4-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
DESCRIPTION
ENABLED
'IntegrationTestBigLinkedListWithChaosMonkey', {NAME = 'meta',
DATA_BLOCK_ENCODING = 'NONE', BLOOMFILTER = 'ROW', REPLICATION_ true
SCOPE = '0', VERSIONS = '1', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL
= '2147483647', KEEP_DELETED_CELLS = 'false', BL
OCKSIZE = '65536', IN_MEMORY = 'false', ENCODE_ON_DISK = 'true', BLOCKCACHE
= 'true'}
1 row(s) in 1.5600 seconds
hbase(main):002:0 scan 'IntegrationTestBigLinkedListWithChaosMonkey'
ROW COLUMN+CELL
0 row(s) in 0.0300 seconds
{code}
I would bet that it's another config issue with yarn not picking up configs if
it's out of process.
IntegrationTestBigLinkedListWithChaosMonkey uses the wrong table name
-
Key: HBASE-8637
URL: https://issues.apache.org/jira/browse/HBASE-8637
Project: HBase
Issue Type: Bug
Components: test
Reporter: Elliott Clark
IntegrationTestBigLinkedListWithChaosMonkey creates a table named
IntegrationTestBigLinkedListWithChaosMonkey but when inserting data it
doesn't insert any data into it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670696#comment-13670696 ] Elliott Clark commented on HBASE-8645: -- I like the logging change, but will this break anyone with a cluster spread across domains ? Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8658) hbase clean is deaf to the --config DIR option
[ https://issues.apache.org/jira/browse/HBASE-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8658: - Attachment: 8658.txt A few tweaks to make it so can pass --config down to clean script. hbase clean is deaf to the --config DIR option -- Key: HBASE-8658 URL: https://issues.apache.org/jira/browse/HBASE-8658 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 Attachments: 8658.txt We need this doing migrations. I'd think lots of folks will have their configs other than default location (I need it testing migration) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8635) Define prefetcher.resultsize.max as percentage
[ https://issues.apache.org/jira/browse/HBASE-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reassigned HBASE-8635: -- Assignee: Jimmy Xiang Define prefetcher.resultsize.max as percentage -- Key: HBASE-8635 URL: https://issues.apache.org/jira/browse/HBASE-8635 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-8635.patch Currently hbase.hregionserver.prefetcher.resultsize.max defines global limit for prefetching. The default value is 256MB. It would be more flexible to define this measure as a percentage of the heap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8635) Define prefetcher.resultsize.max as percentage
[ https://issues.apache.org/jira/browse/HBASE-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8635: --- Attachment: trunk-8635.patch Define prefetcher.resultsize.max as percentage -- Key: HBASE-8635 URL: https://issues.apache.org/jira/browse/HBASE-8635 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-8635.patch Currently hbase.hregionserver.prefetcher.resultsize.max defines global limit for prefetching. The default value is 256MB. It would be more flexible to define this measure as a percentage of the heap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8659) LruBlockCache logs too much
Sergey Shelukhin created HBASE-8659:
---
Summary: LruBlockCache logs too much
Key: HBASE-8659
URL: https://issues.apache.org/jira/browse/HBASE-8659
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
LruBlockCache logs too much.
{code}
grep -c . hbase-hbase-regionserver-.log
77539
grep -c LruBlockCache hbase-hbase-regionserver-..log
64459
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8635) Define prefetcher.resultsize.max as percentage
[ https://issues.apache.org/jira/browse/HBASE-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8635: --- Status: Patch Available (was: Open) Yes, it is more flexible to use percentage of max heap size. Define prefetcher.resultsize.max as percentage -- Key: HBASE-8635 URL: https://issues.apache.org/jira/browse/HBASE-8635 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-8635.patch Currently hbase.hregionserver.prefetcher.resultsize.max defines global limit for prefetching. The default value is 256MB. It would be more flexible to define this measure as a percentage of the heap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670716#comment-13670716 ] stack commented on HBASE-8645: -- bq. ...but will this break anyone with a cluster spread across domains ? It would. Do folks do that? If they do, then this change is out I'd say. You know of cluster done that way? Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8635) Define prefetcher.resultsize.max as percentage
[ https://issues.apache.org/jira/browse/HBASE-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670717#comment-13670717 ] Jimmy Xiang commented on HBASE-8635: Attached a patch. It changed the meaning of hbase.hregionserver.prefetcher.resultsize.max from the actual bytes to the percentage. Didn't change the parameter name. Added a check to make sure the value is between 0 and 1. The default is 0.1. The max heap size is calculated only once in HRegionServer to save some time, although it could change according to the java doc. Define prefetcher.resultsize.max as percentage -- Key: HBASE-8635 URL: https://issues.apache.org/jira/browse/HBASE-8635 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-8635.patch Currently hbase.hregionserver.prefetcher.resultsize.max defines global limit for prefetching. The default value is 256MB. It would be more flexible to define this measure as a percentage of the heap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8659) LruBlockCache logs too much
[
https://issues.apache.org/jira/browse/HBASE-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-8659:
Status: Patch Available (was: Open)
changing to trace level
LruBlockCache logs too much
---
Key: HBASE-8659
URL: https://issues.apache.org/jira/browse/HBASE-8659
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8659-v0.patch
LruBlockCache logs too much.
{code}
grep -c . hbase-hbase-regionserver-.log
77539
grep -c LruBlockCache hbase-hbase-regionserver-..log
64459
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8659) LruBlockCache logs too much
[
https://issues.apache.org/jira/browse/HBASE-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-8659:
Attachment: HBASE-8659-v0.patch
LruBlockCache logs too much
---
Key: HBASE-8659
URL: https://issues.apache.org/jira/browse/HBASE-8659
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8659-v0.patch
LruBlockCache logs too much.
{code}
grep -c . hbase-hbase-regionserver-.log
77539
grep -c LruBlockCache hbase-hbase-regionserver-..log
64459
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670720#comment-13670720 ] Jean-Daniel Cryans commented on HBASE-8645: --- Replication is cross-domain when talking RS to RS. Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670726#comment-13670726 ] stack commented on HBASE-8645: -- [~jdcryans] Yeah, but where do members of the other cluster show in ours? Their ServerNames in particular? Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670729#comment-13670729 ] Elliott Clark commented on HBASE-8645: -- bq.Do folks do that? I have heard of people that have a cluster that spans multiple ec2 zones. I *think* those can be on different domains. ip-10-1-0-1.compute-1.amazonaws.com vs ip-10-2-0-1.compute-2.amazonaws.com Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670731#comment-13670731 ] stack commented on HBASE-8645: -- Let me take a look and see if can get away w/ a short name for use as part of a thread name, no where critical. Will be back. Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8660) [rest] support secure REST access
Jimmy Xiang created HBASE-8660: -- Summary: [rest] support secure REST access Key: HBASE-8660 URL: https://issues.apache.org/jira/browse/HBASE-8660 Project: HBase Issue Type: Improvement Components: REST, security Reporter: Jimmy Xiang REST interface is accessed over http, which is most likely done from outside of a HBase cluster, perhaps over internet. It will be great if we can make it secure. To be secure, we need to: 1. authenticate the caller, 2. check authorization if needed, 3. make the connection secure (https) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670735#comment-13670735 ] stack commented on HBASE-8645: -- [~eclark] yeah... I think I can only use short names in thread names... everywhere ServerName should be as is (in master you'd want to see full name if only the domain differed). Thanks lads. Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5050) [rest] SPNEGO-based authentication
[ https://issues.apache.org/jira/browse/HBASE-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5050: --- Issue Type: Sub-task (was: Improvement) Parent: HBASE-8660 [rest] SPNEGO-based authentication -- Key: HBASE-5050 URL: https://issues.apache.org/jira/browse/HBASE-5050 Project: HBase Issue Type: Sub-task Components: REST, security Reporter: Andrew Purtell Assignee: Jimmy Xiang Currently the REST gateway can authenticate to a HBase cluster using a preconfigured principal. This provides a limited form of secure operation where one or more gateways can be deployed with distinct principals granting appropriate levels of privilege, but the service ports must be protected through network ACLs. This is at best a stopgap. SPNEGO is the standard mechanism for Kerberos authentication over HTTP. Enhance the REST gateway such that it provides this option, and issues requests to the HBase cluster with the established context. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8661) [rest] support REST over https
Jimmy Xiang created HBASE-8661: -- Summary: [rest] support REST over https Key: HBASE-8661 URL: https://issues.apache.org/jira/browse/HBASE-8661 Project: HBase Issue Type: Sub-task Reporter: Jimmy Xiang Currently, REST server works with http only. We can make it work with https too so that the data on the wire is secure. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8659) LruBlockCache logs too much
[
https://issues.apache.org/jira/browse/HBASE-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670748#comment-13670748
]
stack commented on HBASE-8659:
--
Funny. It can be useful figuring our hit rate, especially if no monitoring
going on. I suppose they can get it from the /jmx or via metrics emissions or
look at the UI (is it in the UI... oh, it has its own tab even:
http://sss-6.ent.cloudera.com:60030/rs-status#blockCacheStats).
+1 on commit.
LruBlockCache logs too much
---
Key: HBASE-8659
URL: https://issues.apache.org/jira/browse/HBASE-8659
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8659-v0.patch
LruBlockCache logs too much.
{code}
grep -c . hbase-hbase-regionserver-.log
77539
grep -c LruBlockCache hbase-hbase-regionserver-..log
64459
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
