[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624740#comment-13624740 ] chunhui shen commented on HBASE-7824: - bq.Since ZK session timeout take a while, HMaster#splitLogAndExpireIfOnline will kick in so there won't be any issue. 1.ZK seession will timeout once java process exit. 2.I think in a complex network we shouldn't assert that ZK session timeout happen after HMaster#splitLogAndExpireIfOnline. e.g. the return of getMetaLocationOrReadLocationFromRoot is hanged for a little time. bq.are you fine with this adjustment? Sorry, why do this adjustment? In addition, is it only for 0.94, no trunk? If trunk only, I think there is no above trouble since ROOT has dropped in trunk Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8286) Move site back up out of hbase-assembly; bad idea
[ https://issues.apache.org/jira/browse/HBASE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8286: - Resolution: Fixed Fix Version/s: 0.98.0 0.95.0 Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Updated doc. on how to build site and deploy it. Its easy enough I'd say that anyone should be able to do it .. bug me if not the case. Move site back up out of hbase-assembly; bad idea - Key: HBASE-8286 URL: https://issues.apache.org/jira/browse/HBASE-8286 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.95.0, 0.98.0 Attachments: 8286.txt Redoing the packaging, I thought it smart putting it all under the new hbase-assembly module. A few weeks later, it is not looking like a good idea (Enis suggested moving the site was maybe 'odd'). Here are a few issues: + The generated reports will have mention of hbase-assembly because that is where they are being generated (Elliott pointed out the the scm links mention hbase-assembly instead of trunk). + Doug Meil wants to build and deploy the site but currently deploy is a bit awkward to explain because site presumes it is top level so staging goes to the wrong place and you have to come along and do fixup afterward; it shouldn't be so hard. A possible downside is that we will break Nicks site check added to hadoopqa because site is an aggregating build plugin... but lets see. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624750#comment-13624750 ] Hudson commented on HBASE-6286: --- Integrated in HBase-TRUNK #4023 (See [https://builds.apache.org/job/HBase-TRUNK/4023/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465323) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465322) Result = FAILURE stack : Files : * /hbase/trunk/src/main/site/resources/images/hbase_logo.png stack : Files : * /hbase/trunk/hbase-assembly/pom.xml * /hbase/trunk/hbase-assembly/src/assembly * /hbase/trunk/hbase-assembly/src/docbkx * /hbase/trunk/hbase-assembly/src/main * /hbase/trunk/hbase-assembly/src/main/assembly * /hbase/trunk/hbase-assembly/src/main/assembly/components.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-one-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-two-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/src.xml * /hbase/trunk/hbase-assembly/src/site * /hbase/trunk/hbase-assembly/src/xslt * /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml * /hbase/trunk/pom.xml * /hbase/trunk/src * /hbase/trunk/src/main * /hbase/trunk/src/main/docbkx * /hbase/trunk/src/main/docbkx/book.xml * /hbase/trunk/src/main/docbkx/case_studies.xml * /hbase/trunk/src/main/docbkx/community.xml * /hbase/trunk/src/main/docbkx/configuration.xml * /hbase/trunk/src/main/docbkx/customization.xsl * /hbase/trunk/src/main/docbkx/developer.xml * /hbase/trunk/src/main/docbkx/external_apis.xml * /hbase/trunk/src/main/docbkx/getting_started.xml * /hbase/trunk/src/main/docbkx/ops_mgt.xml * /hbase/trunk/src/main/docbkx/performance.xml * /hbase/trunk/src/main/docbkx/preface.xml * /hbase/trunk/src/main/docbkx/rpc.xml * /hbase/trunk/src/main/docbkx/schema_design.xml * /hbase/trunk/src/main/docbkx/security.xml * /hbase/trunk/src/main/docbkx/shell.xml * /hbase/trunk/src/main/docbkx/troubleshooting.xml * /hbase/trunk/src/main/docbkx/upgrading.xml * /hbase/trunk/src/main/docbkx/zookeeper.xml * /hbase/trunk/src/main/site * /hbase/trunk/src/main/site/resources * /hbase/trunk/src/main/site/resources/css * /hbase/trunk/src/main/site/resources/css/freebsd_docbook.css * /hbase/trunk/src/main/site/resources/css/site.css * /hbase/trunk/src/main/site/resources/doap_Hbase.rdf * /hbase/trunk/src/main/site/resources/images * /hbase/trunk/src/main/site/resources/images/big_h_logo.svg * /hbase/trunk/src/main/site/resources/images/hbase_logo.svg * /hbase/trunk/src/main/site/site.vm * /hbase/trunk/src/main/site/site.xml * /hbase/trunk/src/main/site/xdoc * /hbase/trunk/src/main/site/xdoc/acid-semantics.xml * /hbase/trunk/src/main/site/xdoc/bulk-loads.xml * /hbase/trunk/src/main/site/xdoc/cygwin.xml * /hbase/trunk/src/main/site/xdoc/index.xml * /hbase/trunk/src/main/site/xdoc/metrics.xml * /hbase/trunk/src/main/site/xdoc/old_news.xml * /hbase/trunk/src/main/site/xdoc/pseudo-distributed.xml * /hbase/trunk/src/main/site/xdoc/replication.xml * /hbase/trunk/src/main/site/xdoc/resources.xml * /hbase/trunk/src/main/site/xdoc/sponsors.xml * /hbase/trunk/src/main/xslt * /hbase/trunk/src/main/xslt/configuration_to_docbook_section.xsl Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624751#comment-13624751
]
Jeffrey Zhong commented on HBASE-7824:
--
It's only for 0.94 and the adjustment is guaranteed that splitLog will happen
right before {code}assignmentManager.assignMeta();{code} just like before to
deal with the possible data loss issue you mentioned.
Improve master start up time when there is log splitting work
-
Key: HBASE-7824
URL: https://issues.apache.org/jira/browse/HBASE-7824
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.8
Attachments: hbase-7824.patch, hbase-7824_v2.patch,
hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
When there is log split work going on, master start up waits till all log
split work completes even though the log split has nothing to do with meta
region servers.
It's a bad behavior considering a master node can run when log split is
happening while its start up is blocking by log split work.
Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7937) Retry log rolling to support HA NN scenario
[
https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624753#comment-13624753
]
Hadoop QA commented on HBASE-7937:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577433/HBase-7937-0.94.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5169//console
This message is automatically generated.
Retry log rolling to support HA NN scenario
---
Key: HBASE-7937
URL: https://issues.apache.org/jira/browse/HBASE-7937
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.94.5
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.95.1
Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch,
HBASE-7937-v1.patch
A failure in log rolling causes regionserver abort. In case of HA NN, it will
be good if there is a retry mechanism to roll the logs.
A corresponding jira for MemStore retries is HBASE-7507.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7937) Retry log rolling to support HA NN scenario
[ https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7937: - Attachment: HBase-7937-0.94.txt Attached is a patch for 0.94 branch Retry log rolling to support HA NN scenario --- Key: HBASE-7937 URL: https://issues.apache.org/jira/browse/HBASE-7937 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.95.1 Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch, HBASE-7937-v1.patch A failure in log rolling causes regionserver abort. In case of HA NN, it will be good if there is a retry mechanism to roll the logs. A corresponding jira for MemStore retries is HBASE-7507. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624754#comment-13624754
]
stack commented on HBASE-8287:
--
+1 on commit. Makes sense. The TestTableLockManager is flakey but you should
check the TableHFileOutputFormat fail. If you cannot reproduce local, go ahead
w/ commit.
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.95.2, 0.98.0
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624757#comment-13624757
]
ramkrishna.s.vasudevan commented on HBASE-7824:
---
{code}
ServerName currentMetaServer =
this.catalogTracker.getMetaLocationOrReadLocationFromRoot();
{code}
This is read in two places. One in finishInitialization() and the other inside
assignMEta where the META RS is checked with prev ROOT server.
Can we check this only once and then make the pseudo code change as above
mentioned by Jeffrey?
Improve master start up time when there is log splitting work
-
Key: HBASE-7824
URL: https://issues.apache.org/jira/browse/HBASE-7824
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.8
Attachments: hbase-7824.patch, hbase-7824_v2.patch,
hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
When there is log split work going on, master start up waits till all log
split work completes even though the log split has nothing to do with meta
region servers.
It's a bad behavior considering a master node can run when log split is
happening while its start up is blocking by log split work.
Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8278) Log message after Memstore flush is always with sequence id -1
[
https://issues.apache.org/jira/browse/HBASE-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-8278:
--
Attachment: HBASE-8278_1_addendum.patch
Addendum as said by Ted.
Log message after Memstore flush is always with sequence id -1
--
Key: HBASE-8278
URL: https://issues.apache.org/jira/browse/HBASE-8278
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 0.95.0, 0.98.0
Attachments: HBASE-8278_1_addendum.patch, HBASE-8278.patch
{code}
2013-04-03 16:12:03,404 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=167, memsize=130.0 M, into tmp file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,415 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
as
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b,
entries=114390, sequenceid=167, filesize=115.3 M
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~130.0 M/136352880, currentsize=46.0 M/48222360
for region TestTable,,1365019914410.d424e2c515e1239c99867ffdaf525ed3. in
825ms, sequenceid=-1, compaction requested=false
{code}
Check the last line the sequenceid is always -1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8278) Log message after Memstore flush is always with sequence id -1
[
https://issues.apache.org/jira/browse/HBASE-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624765#comment-13624765
]
Hudson commented on HBASE-8278:
---
Integrated in HBase-TRUNK #4029 (See
[https://builds.apache.org/job/HBase-TRUNK/4029/])
HBASE-8278 - Log message after Memstore flush is always with sequence id -1
(addendum to remove sequenceId) (Ram) (Revision 1465331)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Log message after Memstore flush is always with sequence id -1
--
Key: HBASE-8278
URL: https://issues.apache.org/jira/browse/HBASE-8278
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 0.95.0, 0.98.0
Attachments: HBASE-8278_1_addendum.patch, HBASE-8278.patch
{code}
2013-04-03 16:12:03,404 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=167, memsize=130.0 M, into tmp file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,415 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
as
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b,
entries=114390, sequenceid=167, filesize=115.3 M
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~130.0 M/136352880, currentsize=46.0 M/48222360
for region TestTable,,1365019914410.d424e2c515e1239c99867ffdaf525ed3. in
825ms, sequenceid=-1, compaction requested=false
{code}
Check the last line the sequenceid is always -1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624769#comment-13624769
]
Jeffrey Zhong commented on HBASE-7824:
--
I pasted the whole related code snippet as below. Ram's suggestion is possible
because no one re-assign META till assignmentManager.assignMeta(). The modified
logic is same as before the patch.
{code}
...
ServerName currentMetaServer =
this.catalogTracker.getMetaLocationOrReadLocationFromRoot();
if (currentMetaServer != null
!currentMetaServer.equals(previousRootServer)) {
fileSystemManager.splitAllLogs(currentMetaServer);
if (this.serverManager.isServerOnline(currentMetaServer)) {
this.serverManager.expireServer(currentMetaServer);
}
}
assignmentManager.assignMeta();
enableSSHandWaitForMeta();
...
{code}
Improve master start up time when there is log splitting work
-
Key: HBASE-7824
URL: https://issues.apache.org/jira/browse/HBASE-7824
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.8
Attachments: hbase-7824.patch, hbase-7824_v2.patch,
hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
When there is log split work going on, master start up waits till all log
split work completes even though the log split has nothing to do with meta
region servers.
It's a bad behavior considering a master node can run when log split is
happening while its start up is blocking by log split work.
Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7937) Retry log rolling to support HA NN scenario
[ https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7937: - Attachment: HBase-7937-trunk.txt Uploaded patch for trunk Retry log rolling to support HA NN scenario --- Key: HBASE-7937 URL: https://issues.apache.org/jira/browse/HBASE-7937 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.95.1 Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch, HBase-7937-trunk.txt, HBASE-7937-v1.patch A failure in log rolling causes regionserver abort. In case of HA NN, it will be good if there is a retry mechanism to roll the logs. A corresponding jira for MemStore retries is HBASE-7507. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624782#comment-13624782
]
Hadoop QA commented on HBASE-8287:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577425/hbase-trunk-8287.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5170//console
This message is automatically generated.
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.95.2, 0.98.0
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624786#comment-13624786
]
Hadoop QA commented on HBASE-8287:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577425/hbase-trunk-8287.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5171//console
This message is automatically generated.
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.95.2, 0.98.0
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624788#comment-13624788 ] chunhui shen commented on HBASE-7824: - bq.fileSystemManager.splitAllLogs(currentMetaServer); Should we take care of log-split concurrency between master initialization thread and SSH？ Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624789#comment-13624789 ] Jeffrey Zhong commented on HBASE-7824: -- Yep, I've taken care of that. Basically, synchronized log split tasks before SSH is enabled. Once you agree the approach in general, I'll submit the modified patch for review. A question through: Should we do expireServer firstly and then do log splitting after we have the log splitting synchronization mechanism before SSH is enabled. Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7937) Retry log rolling to support HA NN scenario
[
https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624791#comment-13624791
]
Hadoop QA commented on HBASE-7937:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577439/HBase-7937-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5172//console
This message is automatically generated.
Retry log rolling to support HA NN scenario
---
Key: HBASE-7937
URL: https://issues.apache.org/jira/browse/HBASE-7937
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.94.5
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.95.1
Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch,
HBase-7937-trunk.txt, HBASE-7937-v1.patch
A failure in log rolling causes regionserver abort. In case of HA NN, it will
be good if there is a retry mechanism to roll the logs.
A corresponding jira for MemStore retries is HBASE-7507.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624792#comment-13624792 ] chunhui shen commented on HBASE-7824: - bq.log splitting synchronization mechanism before SSH is enabled Yes, it's a solution. What's the difference if do expireServer firstly? None? Go as your thought Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8251) enable SSH before assign META on Master startup
[ https://issues.apache.org/jira/browse/HBASE-8251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624848#comment-13624848 ] Jieshan Bean commented on HBASE-8251: - Yes, one corner case is META RS is killed just before new assignment happening(Killing happens either during calling processRegionInTransitionAndBlockUntilAssigned or verifyMetaRegionLocation), so rit is false and metaRegionLocation is false(Only under this scenario may trigger a new assignment). It may cause data-loss and double assignment. Thanks, Chunhui, Rama and Jeffrey. I'm thinking about adding a check(check for whether currentMetaServer is a online server or a processing dead server, need to change something in ServerManager) before calling assignMeta. enable SSH before assign META on Master startup --- Key: HBASE-8251 URL: https://issues.apache.org/jira/browse/HBASE-8251 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 0.94.6 Reporter: Jieshan Bean Assignee: Jieshan Bean Attachments: HBASE-8251-94.patch I think HBASE-5918 could not fix this issue. In HMaster#assignRootAndMeta: 1. Assign ROOT. 2. Block until ROOT be opened. 3. Assign META. 4. Block until META be opened. SSH is enabled after step 4. So if the RS who host ROOT dies before step 4, master will be blocked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624854#comment-13624854 ] Hudson commented on HBASE-6286: --- Integrated in hbase-0.95 #131 (See [https://builds.apache.org/job/hbase-0.95/131/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465325) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-assembly/pom.xml * /hbase/branches/0.95/hbase-assembly/src/assembly * /hbase/branches/0.95/hbase-assembly/src/docbkx * /hbase/branches/0.95/hbase-assembly/src/main * /hbase/branches/0.95/hbase-assembly/src/main/assembly * /hbase/branches/0.95/hbase-assembly/src/main/assembly/components.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/hadoop-one-compat.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/hadoop-two-compat.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/src.xml * /hbase/branches/0.95/hbase-assembly/src/site * /hbase/branches/0.95/hbase-assembly/src/xslt * /hbase/branches/0.95/hbase-common/src/main/resources/hbase-default.xml * /hbase/branches/0.95/pom.xml * /hbase/branches/0.95/src * /hbase/branches/0.95/src/main * /hbase/branches/0.95/src/main/docbkx * /hbase/branches/0.95/src/main/docbkx/book.xml * /hbase/branches/0.95/src/main/docbkx/case_studies.xml * /hbase/branches/0.95/src/main/docbkx/community.xml * /hbase/branches/0.95/src/main/docbkx/configuration.xml * /hbase/branches/0.95/src/main/docbkx/customization.xsl * /hbase/branches/0.95/src/main/docbkx/developer.xml * /hbase/branches/0.95/src/main/docbkx/external_apis.xml * /hbase/branches/0.95/src/main/docbkx/getting_started.xml * /hbase/branches/0.95/src/main/docbkx/ops_mgt.xml * /hbase/branches/0.95/src/main/docbkx/performance.xml * /hbase/branches/0.95/src/main/docbkx/preface.xml * /hbase/branches/0.95/src/main/docbkx/rpc.xml * /hbase/branches/0.95/src/main/docbkx/schema_design.xml * /hbase/branches/0.95/src/main/docbkx/security.xml * /hbase/branches/0.95/src/main/docbkx/shell.xml * /hbase/branches/0.95/src/main/docbkx/troubleshooting.xml * /hbase/branches/0.95/src/main/docbkx/upgrading.xml * /hbase/branches/0.95/src/main/docbkx/zookeeper.xml * /hbase/branches/0.95/src/main/site * /hbase/branches/0.95/src/main/site/resources * /hbase/branches/0.95/src/main/site/resources/css * /hbase/branches/0.95/src/main/site/resources/css/freebsd_docbook.css * /hbase/branches/0.95/src/main/site/resources/css/site.css * /hbase/branches/0.95/src/main/site/resources/doap_Hbase.rdf * /hbase/branches/0.95/src/main/site/resources/images * /hbase/branches/0.95/src/main/site/resources/images/big_h_logo.svg * /hbase/branches/0.95/src/main/site/resources/images/hbase_logo.png * /hbase/branches/0.95/src/main/site/resources/images/hbase_logo.svg * /hbase/branches/0.95/src/main/site/site.vm * /hbase/branches/0.95/src/main/site/site.xml * /hbase/branches/0.95/src/main/site/xdoc * /hbase/branches/0.95/src/main/site/xdoc/acid-semantics.xml * /hbase/branches/0.95/src/main/site/xdoc/bulk-loads.xml * /hbase/branches/0.95/src/main/site/xdoc/cygwin.xml * /hbase/branches/0.95/src/main/site/xdoc/index.xml * /hbase/branches/0.95/src/main/site/xdoc/metrics.xml * /hbase/branches/0.95/src/main/site/xdoc/old_news.xml * /hbase/branches/0.95/src/main/site/xdoc/pseudo-distributed.xml * /hbase/branches/0.95/src/main/site/xdoc/replication.xml * /hbase/branches/0.95/src/main/site/xdoc/resources.xml * /hbase/branches/0.95/src/main/site/xdoc/sponsors.xml * /hbase/branches/0.95/src/main/xslt * /hbase/branches/0.95/src/main/xslt/configuration_to_docbook_section.xsl Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8278) Log message after Memstore flush is always with sequence id -1
[
https://issues.apache.org/jira/browse/HBASE-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624855#comment-13624855
]
Hudson commented on HBASE-8278:
---
Integrated in hbase-0.95 #131 (See
[https://builds.apache.org/job/hbase-0.95/131/])
HBASE-8278 - Log message after Memstore flush is always with sequence id -1
(Addeundum to remove sequenceId) (Ram) (Revision 1465332)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Log message after Memstore flush is always with sequence id -1
--
Key: HBASE-8278
URL: https://issues.apache.org/jira/browse/HBASE-8278
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 0.95.0, 0.98.0
Attachments: HBASE-8278_1_addendum.patch, HBASE-8278.patch
{code}
2013-04-03 16:12:03,404 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=167, memsize=130.0 M, into tmp file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,415 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
as
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b,
entries=114390, sequenceid=167, filesize=115.3 M
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~130.0 M/136352880, currentsize=46.0 M/48222360
for region TestTable,,1365019914410.d424e2c515e1239c99867ffdaf525ed3. in
825ms, sequenceid=-1, compaction requested=false
{code}
Check the last line the sequenceid is always -1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624862#comment-13624862
]
Hadoop QA commented on HBASE-8287:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577425/hbase-trunk-8287.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5173//console
This message is automatically generated.
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.95.2, 0.98.0
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8045) Fix .META. migration after HBASE-3171
[ https://issues.apache.org/jira/browse/HBASE-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8045: -- Attachment: HBASE-8045.patch Patch for 95. Please review. Fix .META. migration after HBASE-3171 - Key: HBASE-8045 URL: https://issues.apache.org/jira/browse/HBASE-8045 Project: HBase Issue Type: Task Reporter: Jean-Daniel Cryans Priority: Blocker Fix For: 0.95.1 Attachments: HBASE-8045.patch HBASE-3171 doesn't manage the migration correctly, see MetaMigrationConvertingToPB and its unit test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8045) Fix .META. migration after HBASE-3171
[ https://issues.apache.org/jira/browse/HBASE-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8045: -- Status: Patch Available (was: Open) Fix .META. migration after HBASE-3171 - Key: HBASE-8045 URL: https://issues.apache.org/jira/browse/HBASE-8045 Project: HBase Issue Type: Task Reporter: Jean-Daniel Cryans Priority: Blocker Fix For: 0.95.1 Attachments: HBASE-8045.patch HBASE-3171 doesn't manage the migration correctly, see MetaMigrationConvertingToPB and its unit test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8278) Log message after Memstore flush is always with sequence id -1
[
https://issues.apache.org/jira/browse/HBASE-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624876#comment-13624876
]
Hudson commented on HBASE-8278:
---
Integrated in hbase-0.95-on-hadoop2 #59 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/59/])
HBASE-8278 - Log message after Memstore flush is always with sequence id -1
(Addeundum to remove sequenceId) (Ram) (Revision 1465332)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Log message after Memstore flush is always with sequence id -1
--
Key: HBASE-8278
URL: https://issues.apache.org/jira/browse/HBASE-8278
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 0.95.0, 0.98.0
Attachments: HBASE-8278_1_addendum.patch, HBASE-8278.patch
{code}
2013-04-03 16:12:03,404 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=167, memsize=130.0 M, into tmp file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,415 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
as
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b,
entries=114390, sequenceid=167, filesize=115.3 M
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~130.0 M/136352880, currentsize=46.0 M/48222360
for region TestTable,,1365019914410.d424e2c515e1239c99867ffdaf525ed3. in
825ms, sequenceid=-1, compaction requested=false
{code}
Check the last line the sequenceid is always -1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624875#comment-13624875 ] Hudson commented on HBASE-6286: --- Integrated in hbase-0.95-on-hadoop2 #59 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/59/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465325) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-assembly/pom.xml * /hbase/branches/0.95/hbase-assembly/src/assembly * /hbase/branches/0.95/hbase-assembly/src/docbkx * /hbase/branches/0.95/hbase-assembly/src/main * /hbase/branches/0.95/hbase-assembly/src/main/assembly * /hbase/branches/0.95/hbase-assembly/src/main/assembly/components.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/hadoop-one-compat.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/hadoop-two-compat.xml * /hbase/branches/0.95/hbase-assembly/src/main/assembly/src.xml * /hbase/branches/0.95/hbase-assembly/src/site * /hbase/branches/0.95/hbase-assembly/src/xslt * /hbase/branches/0.95/hbase-common/src/main/resources/hbase-default.xml * /hbase/branches/0.95/pom.xml * /hbase/branches/0.95/src * /hbase/branches/0.95/src/main * /hbase/branches/0.95/src/main/docbkx * /hbase/branches/0.95/src/main/docbkx/book.xml * /hbase/branches/0.95/src/main/docbkx/case_studies.xml * /hbase/branches/0.95/src/main/docbkx/community.xml * /hbase/branches/0.95/src/main/docbkx/configuration.xml * /hbase/branches/0.95/src/main/docbkx/customization.xsl * /hbase/branches/0.95/src/main/docbkx/developer.xml * /hbase/branches/0.95/src/main/docbkx/external_apis.xml * /hbase/branches/0.95/src/main/docbkx/getting_started.xml * /hbase/branches/0.95/src/main/docbkx/ops_mgt.xml * /hbase/branches/0.95/src/main/docbkx/performance.xml * /hbase/branches/0.95/src/main/docbkx/preface.xml * /hbase/branches/0.95/src/main/docbkx/rpc.xml * /hbase/branches/0.95/src/main/docbkx/schema_design.xml * /hbase/branches/0.95/src/main/docbkx/security.xml * /hbase/branches/0.95/src/main/docbkx/shell.xml * /hbase/branches/0.95/src/main/docbkx/troubleshooting.xml * /hbase/branches/0.95/src/main/docbkx/upgrading.xml * /hbase/branches/0.95/src/main/docbkx/zookeeper.xml * /hbase/branches/0.95/src/main/site * /hbase/branches/0.95/src/main/site/resources * /hbase/branches/0.95/src/main/site/resources/css * /hbase/branches/0.95/src/main/site/resources/css/freebsd_docbook.css * /hbase/branches/0.95/src/main/site/resources/css/site.css * /hbase/branches/0.95/src/main/site/resources/doap_Hbase.rdf * /hbase/branches/0.95/src/main/site/resources/images * /hbase/branches/0.95/src/main/site/resources/images/big_h_logo.svg * /hbase/branches/0.95/src/main/site/resources/images/hbase_logo.png * /hbase/branches/0.95/src/main/site/resources/images/hbase_logo.svg * /hbase/branches/0.95/src/main/site/site.vm * /hbase/branches/0.95/src/main/site/site.xml * /hbase/branches/0.95/src/main/site/xdoc * /hbase/branches/0.95/src/main/site/xdoc/acid-semantics.xml * /hbase/branches/0.95/src/main/site/xdoc/bulk-loads.xml * /hbase/branches/0.95/src/main/site/xdoc/cygwin.xml * /hbase/branches/0.95/src/main/site/xdoc/index.xml * /hbase/branches/0.95/src/main/site/xdoc/metrics.xml * /hbase/branches/0.95/src/main/site/xdoc/old_news.xml * /hbase/branches/0.95/src/main/site/xdoc/pseudo-distributed.xml * /hbase/branches/0.95/src/main/site/xdoc/replication.xml * /hbase/branches/0.95/src/main/site/xdoc/resources.xml * /hbase/branches/0.95/src/main/site/xdoc/sponsors.xml * /hbase/branches/0.95/src/main/xslt * /hbase/branches/0.95/src/main/xslt/configuration_to_docbook_section.xsl Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8278) Log message after Memstore flush is always with sequence id -1
[
https://issues.apache.org/jira/browse/HBASE-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624878#comment-13624878
]
Hudson commented on HBASE-8278:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #481 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/481/])
HBASE-8278 - Log message after Memstore flush is always with sequence id -1
(addendum to remove sequenceId) (Ram) (Revision 1465331)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Log message after Memstore flush is always with sequence id -1
--
Key: HBASE-8278
URL: https://issues.apache.org/jira/browse/HBASE-8278
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Trivial
Fix For: 0.95.0, 0.98.0
Attachments: HBASE-8278_1_addendum.patch, HBASE-8278.patch
{code}
2013-04-03 16:12:03,404 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=167, memsize=130.0 M, into tmp file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,415 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/.tmp/4875565167ad4ce08c1b4cd935501e5b
as
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/TestTable/d424e2c515e1239c99867ffdaf525ed3/info/4875565167ad4ce08c1b4cd935501e5b,
entries=114390, sequenceid=167, filesize=115.3 M
2013-04-03 16:12:03,424 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~130.0 M/136352880, currentsize=46.0 M/48222360
for region TestTable,,1365019914410.d424e2c515e1239c99867ffdaf525ed3. in
825ms, sequenceid=-1, compaction requested=false
{code}
Check the last line the sequenceid is always -1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624877#comment-13624877 ] Hudson commented on HBASE-6286: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #481 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/481/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465323) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465322) Result = FAILURE stack : Files : * /hbase/trunk/src/main/site/resources/images/hbase_logo.png stack : Files : * /hbase/trunk/hbase-assembly/pom.xml * /hbase/trunk/hbase-assembly/src/assembly * /hbase/trunk/hbase-assembly/src/docbkx * /hbase/trunk/hbase-assembly/src/main * /hbase/trunk/hbase-assembly/src/main/assembly * /hbase/trunk/hbase-assembly/src/main/assembly/components.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-one-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-two-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/src.xml * /hbase/trunk/hbase-assembly/src/site * /hbase/trunk/hbase-assembly/src/xslt * /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml * /hbase/trunk/pom.xml * /hbase/trunk/src * /hbase/trunk/src/main * /hbase/trunk/src/main/docbkx * /hbase/trunk/src/main/docbkx/book.xml * /hbase/trunk/src/main/docbkx/case_studies.xml * /hbase/trunk/src/main/docbkx/community.xml * /hbase/trunk/src/main/docbkx/configuration.xml * /hbase/trunk/src/main/docbkx/customization.xsl * /hbase/trunk/src/main/docbkx/developer.xml * /hbase/trunk/src/main/docbkx/external_apis.xml * /hbase/trunk/src/main/docbkx/getting_started.xml * /hbase/trunk/src/main/docbkx/ops_mgt.xml * /hbase/trunk/src/main/docbkx/performance.xml * /hbase/trunk/src/main/docbkx/preface.xml * /hbase/trunk/src/main/docbkx/rpc.xml * /hbase/trunk/src/main/docbkx/schema_design.xml * /hbase/trunk/src/main/docbkx/security.xml * /hbase/trunk/src/main/docbkx/shell.xml * /hbase/trunk/src/main/docbkx/troubleshooting.xml * /hbase/trunk/src/main/docbkx/upgrading.xml * /hbase/trunk/src/main/docbkx/zookeeper.xml * /hbase/trunk/src/main/site * /hbase/trunk/src/main/site/resources * /hbase/trunk/src/main/site/resources/css * /hbase/trunk/src/main/site/resources/css/freebsd_docbook.css * /hbase/trunk/src/main/site/resources/css/site.css * /hbase/trunk/src/main/site/resources/doap_Hbase.rdf * /hbase/trunk/src/main/site/resources/images * /hbase/trunk/src/main/site/resources/images/big_h_logo.svg * /hbase/trunk/src/main/site/resources/images/hbase_logo.svg * /hbase/trunk/src/main/site/site.vm * /hbase/trunk/src/main/site/site.xml * /hbase/trunk/src/main/site/xdoc * /hbase/trunk/src/main/site/xdoc/acid-semantics.xml * /hbase/trunk/src/main/site/xdoc/bulk-loads.xml * /hbase/trunk/src/main/site/xdoc/cygwin.xml * /hbase/trunk/src/main/site/xdoc/index.xml * /hbase/trunk/src/main/site/xdoc/metrics.xml * /hbase/trunk/src/main/site/xdoc/old_news.xml * /hbase/trunk/src/main/site/xdoc/pseudo-distributed.xml * /hbase/trunk/src/main/site/xdoc/replication.xml * /hbase/trunk/src/main/site/xdoc/resources.xml * /hbase/trunk/src/main/site/xdoc/sponsors.xml * /hbase/trunk/src/main/xslt * /hbase/trunk/src/main/xslt/configuration_to_docbook_section.xsl Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8045) Fix .META. migration after HBASE-3171
[
https://issues.apache.org/jira/browse/HBASE-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624891#comment-13624891
]
Hadoop QA commented on HBASE-8045:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577442/HBASE-8045.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5176//console
This message is automatically generated.
Fix .META. migration after HBASE-3171
-
Key: HBASE-8045
URL: https://issues.apache.org/jira/browse/HBASE-8045
Project: HBase
Issue Type: Task
Reporter: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.95.1
Attachments: HBASE-8045.patch
HBASE-3171 doesn't manage the migration correctly, see
MetaMigrationConvertingToPB and its unit test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624898#comment-13624898
]
Nicolas Liochon commented on HBASE-8287:
I've used this jira to test some precommit env settings. It confirms the patch
is ok.
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.95.2, 0.98.0
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8251) enable SSH before assign META on Master startup
[ https://issues.apache.org/jira/browse/HBASE-8251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jieshan Bean updated HBASE-8251: Attachment: HBASE-8251-94-v2.patch New version of patch for 94, addressed all the comments. enable SSH before assign META on Master startup --- Key: HBASE-8251 URL: https://issues.apache.org/jira/browse/HBASE-8251 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 0.94.6 Reporter: Jieshan Bean Assignee: Jieshan Bean Attachments: HBASE-8251-94.patch, HBASE-8251-94-v2.patch I think HBASE-5918 could not fix this issue. In HMaster#assignRootAndMeta: 1. Assign ROOT. 2. Block until ROOT be opened. 3. Assign META. 4. Block until META be opened. SSH is enabled after step 4. So if the RS who host ROOT dies before step 4, master will be blocked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624931#comment-13624931 ] Jimmy Xiang commented on HBASE-8256: [~nkeywal], could you please feel free to modify the patch as needed and post a new one? I tried to monitor the JVM/threads used by surefire, and didn't see the number changes much. I am sure I have missed some surefire settings. [~sershe], what's an r? Add category Flaky for tests which are flaky Key: HBASE-8256 URL: https://issues.apache.org/jira/browse/HBASE-8256 Project: HBase Issue Type: Bug Components: build, test Affects Versions: 0.95.1, 0.98.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: trunk-8256_v0.patch To make the Jenkin build more useful, it is good to keep it blue/green. We can mark those flaky tests flaky, and don't run them by default. However, people can still run them. We can also set up a Jekin build just for those flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624958#comment-13624958 ] Nicolas Liochon commented on HBASE-8256: I've documented the settings in the reference guide. Unfortunately, the surefire settings changed in 2.14, and it's not perfectly documented (see HBASE-4955 for how to use them), hence the risk of doing something that will have to be redone very soon... Add category Flaky for tests which are flaky Key: HBASE-8256 URL: https://issues.apache.org/jira/browse/HBASE-8256 Project: HBase Issue Type: Bug Components: build, test Affects Versions: 0.95.1, 0.98.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: trunk-8256_v0.patch To make the Jenkin build more useful, it is good to keep it blue/green. We can mark those flaky tests flaky, and don't run them by default. However, people can still run them. We can also set up a Jekin build just for those flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624959#comment-13624959 ] Andrew Purtell commented on HBASE-8256: --- I think this is ok because we went the flakey test category to be an empty set ideally. So lets get the builds green by segregating them and then fix them before we have to worry about new versions of surefire. Easy. :-) Add category Flaky for tests which are flaky Key: HBASE-8256 URL: https://issues.apache.org/jira/browse/HBASE-8256 Project: HBase Issue Type: Bug Components: build, test Affects Versions: 0.95.1, 0.98.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: trunk-8256_v0.patch To make the Jenkin build more useful, it is good to keep it blue/green. We can mark those flaky tests flaky, and don't run them by default. However, people can still run them. We can also set up a Jekin build just for those flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7937) Retry log rolling to support HA NN scenario
[ https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624960#comment-13624960 ] Himanshu Vashishtha commented on HBASE-7937: Hey Liang, looking at the 0.94 patch, it looks it is redoing what was done by HBaseFileSystem and HFlogFileSystem. They already have the retrying logic. If we this route, then we will be defining properties for retry number/wait time, etc for all such processes. I am working on something similar to HBASE-8211 for 0.95, and fold the hlog rolling retry in that effort. Retry log rolling to support HA NN scenario --- Key: HBASE-7937 URL: https://issues.apache.org/jira/browse/HBASE-7937 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.95.1 Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch, HBase-7937-trunk.txt, HBASE-7937-v1.patch A failure in log rolling causes regionserver abort. In case of HA NN, it will be good if there is a retry mechanism to roll the logs. A corresponding jira for MemStore retries is HBASE-7507. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-7824: - Attachment: hbase-7824-v9.patch Make sure splitLog happen once before we assign Meta to address data loss concern from [~zjushch] Thanks [~zjushch] for the reviewing! Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch, hbase-7824-v9.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8251) enable SSH before assign META on Master startup
[
https://issues.apache.org/jira/browse/HBASE-8251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624971#comment-13624971
]
Jeffrey Zhong commented on HBASE-8251:
--
[~jeason] The changes still not guarantee the possible double meta assignment.
For example, right after {code}boolean needToAssign = !(currentMetaServer !=
null
this.serverManager.isOnlineOrProcessingDeadServer(currentMetaServer));{code},
currentMetaServer is marked dead by master. Then we still have two possible
places to assign meta.
In addition, I thought about enableSSH globally idea before. A concern is that
once SSH is enabled and meta assignment could be delayed by other dead servers
log splitting work from SSHs so master starts up will slower than before. It's
not ideal otherwise the existing code could have skipped
assignmentManager.assignMeta(); right after expireServer.
The above are just IMHO. Thanks.
enable SSH before assign META on Master startup
---
Key: HBASE-8251
URL: https://issues.apache.org/jira/browse/HBASE-8251
Project: HBase
Issue Type: Bug
Components: master, Region Assignment
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8251-94.patch, HBASE-8251-94-v2.patch
I think HBASE-5918 could not fix this issue. In HMaster#assignRootAndMeta:
1. Assign ROOT.
2. Block until ROOT be opened.
3. Assign META.
4. Block until META be opened.
SSH is enabled after step 4. So if the RS who host ROOT dies before step 4,
master will be blocked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625026#comment-13625026
]
Jeffrey Zhong commented on HBASE-7824:
--
Test result for v9 patch:
{code}
Test Suite Results :
Tests run: 1339, Failures: 0, Errors: 0, Skipped: 13
Integration: IntegrationTestDataIngestWithChaosMonkey
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 353.049 sec
{code}
Improve master start up time when there is log splitting work
-
Key: HBASE-7824
URL: https://issues.apache.org/jira/browse/HBASE-7824
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.8
Attachments: hbase-7824.patch, hbase-7824_v2.patch,
hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch,
hbase-7824-v9.patch
When there is log split work going on, master start up waits till all log
split work completes even though the log split has nothing to do with meta
region servers.
It's a bad behavior considering a master node can run when log split is
happening while its start up is blocking by log split work.
Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5416) Improve performance of scans with some kind of filters.
[ https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5416: -- Attachment: 5416-TestJoinedScanners-0.94.txt Improve performance of scans with some kind of filters. --- Key: HBASE-5416 URL: https://issues.apache.org/jira/browse/HBASE-5416 Project: HBase Issue Type: Improvement Components: Filters, Performance, regionserver Affects Versions: 0.90.4 Reporter: Max Lapan Assignee: Sergey Shelukhin Fix For: 0.94.5, 0.95.0 Attachments: 5416-0.94-v1.txt, 5416-0.94-v2.txt, 5416-0.94-v3.txt, 5416-drop-new-method-from-filter.txt, 5416-Filtered_scans_v6.patch, 5416-TestJoinedScanners-0.94.txt, 5416-v13.patch, 5416-v14.patch, 5416-v15.patch, 5416-v16.patch, 5416-v5.txt, 5416-v6.txt, Filtered_scans.patch, Filtered_scans_v2.patch, Filtered_scans_v3.patch, Filtered_scans_v4.patch, Filtered_scans_v5.1.patch, Filtered_scans_v5.patch, Filtered_scans_v7.patch, HBASE-5416-v10.patch, HBASE-5416-v11.patch, HBASE-5416-v12.patch, HBASE-5416-v12.patch, HBASE-5416-v7-rebased.patch, HBASE-5416-v8.patch, HBASE-5416-v9.patch, org.apache.hadoop.hbase.regionserver.TestHRegion-output.txt When the scan is performed, whole row is loaded into result list, after that filter (if exists) is applied to detect that row is needed. But when scan is performed on several CFs and filter checks only data from the subset of these CFs, data from CFs, not checked by a filter is not needed on a filter stage. Only when we decided to include current row. And in such case we can significantly reduce amount of IO performed by a scan, by loading only values, actually checked by a filter. For example, we have two CFs: flags and snap. Flags is quite small (bunch of megabytes) and is used to filter large entries from snap. Snap is very large (10s of GB) and it is quite costly to scan it. If we needed only rows with some flag specified, we use SingleColumnValueFilter to limit result to only small subset of region. But current implementation is loading both CFs to perform scan, when only small subset is needed. Attached patch adds one routine to Filter interface to allow filter to specify which CF is needed to it's operation. In HRegion, we separate all scanners into two groups: needed for filter and the rest (joined). When new row is considered, only needed data is loaded, filter applied, and only if filter accepts the row, rest of data is loaded. At our data, this speeds up such kind of scans 30-50 times. Also, this gives us the way to better normalize the data into separate columns by optimizing the scans performed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625058#comment-13625058 ] Hudson commented on HBASE-6286: --- Integrated in HBase-TRUNK #4037 (See [https://builds.apache.org/job/HBase-TRUNK/4037/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea; ADDENDUM -- MANUAL I COPIED WAS STALE (Revision 1465460) Result = FAILURE stack : Files : * /hbase/trunk/src/main/docbkx/book.xml * /hbase/trunk/src/main/docbkx/case_studies.xml * /hbase/trunk/src/main/docbkx/community.xml * /hbase/trunk/src/main/docbkx/configuration.xml * /hbase/trunk/src/main/docbkx/developer.xml * /hbase/trunk/src/main/docbkx/upgrading.xml Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8251) enable SSH before assign META on Master startup
[
https://issues.apache.org/jira/browse/HBASE-8251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625064#comment-13625064
]
Jieshan Bean commented on HBASE-8251:
-
bq.*, currentMetaServer is marked dead by master. Then we still have two
possible places to assign meta.
This RS should belong to the set of ServerManager#onlineServers just before
marking it as dead(See the related code of ServerManager#onlineServers and
DeadServer#processingDeadServers.
). So the below check returns true:
{code}
this.serverManager
.isOnlineOrProcessingDeadServer(currentMetaServer).
{code}
So needToAssign is false. No new assign would happen under this scenario.
Correct me if I misunderstood you:). Thank you.
enable SSH before assign META on Master startup
---
Key: HBASE-8251
URL: https://issues.apache.org/jira/browse/HBASE-8251
Project: HBase
Issue Type: Bug
Components: master, Region Assignment
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8251-94.patch, HBASE-8251-94-v2.patch
I think HBASE-5918 could not fix this issue. In HMaster#assignRootAndMeta:
1. Assign ROOT.
2. Block until ROOT be opened.
3. Assign META.
4. Block until META be opened.
SSH is enabled after step 4. So if the RS who host ROOT dies before step 4,
master will be blocked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625076#comment-13625076 ] Hudson commented on HBASE-6286: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #485 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/485/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea; ADDENDUM -- MANUAL I COPIED WAS STALE (Revision 1465460) Result = FAILURE stack : Files : * /hbase/trunk/src/main/docbkx/book.xml * /hbase/trunk/src/main/docbkx/case_studies.xml * /hbase/trunk/src/main/docbkx/community.xml * /hbase/trunk/src/main/docbkx/configuration.xml * /hbase/trunk/src/main/docbkx/developer.xml * /hbase/trunk/src/main/docbkx/upgrading.xml Upgrade maven-compiler-plugin to 2.5.1 -- Key: HBASE-6286 URL: https://issues.apache.org/jira/browse/HBASE-6286 Project: HBase Issue Type: Improvement Components: build Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.94.2 Attachments: HBASE-6286.patch time mvn -PlocalTests clean install -DskipTests With 2.5.1: |user|1m35.634s|1m31.178s|1m31.366s| |sys|0m06.540s|0m05.376s|0m05.488s| With 2.0.2 (current): |user|2m01.168s|1m54.027s|1m57.799s| |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8045) Fix .META. migration after HBASE-3171
[ https://issues.apache.org/jira/browse/HBASE-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625085#comment-13625085 ] rajeshbabu commented on HBASE-8045: --- checking failed tests. Fix .META. migration after HBASE-3171 - Key: HBASE-8045 URL: https://issues.apache.org/jira/browse/HBASE-8045 Project: HBase Issue Type: Task Reporter: Jean-Daniel Cryans Priority: Blocker Fix For: 0.95.1 Attachments: HBASE-8045.patch HBASE-3171 doesn't manage the migration correctly, see MetaMigrationConvertingToPB and its unit test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625088#comment-13625088
]
chunhui shen commented on HBASE-7824:
-
Minor comments:
{code}+ * @param previousRootServer ServerName of previous root region server
before current start up
+ * @return
+ * @throws InterruptedException{code}
remove @return
{code}
+} catch (Exception ex) {
+ LOG.warn(Retry setClusterDown failed, ex);
+}
{code}
LOG.error seems more reasonable since using error before
Some doubt:
{code}
+ this.fileSystemManager.splitAllLogs(preRootServer);
+ this.fileSystemManager.splitAllLogs(preMetaServer);
+fileSystemManager.splitAllLogs(currentMetaServer);
{code}
Should use the flag 'shouldSplitMetaSeparately' like other log-split?
In master#finishInitialization, after handling other dead servers in SSH, we
will call assignmentManager.joinCluster(), it seems have some problems, e.g.
1.in AssignmentManager#processDeadServersAndRegionsInTransition, how about if
we mark it as a clean cluster startup?
2.if we mark it as a failover, is there any conflict between SSH and
AssignmentManager#processDeadServersAndRecoverLostRegions
An important attention:
From DeadServer#cleanPreviousInstance, a deadserver will be removed if the
same HostnamePort servername is online.
It means a server will not belong to deadservers even if it is processed in SSH.
Improve master start up time when there is log splitting work
-
Key: HBASE-7824
URL: https://issues.apache.org/jira/browse/HBASE-7824
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.8
Attachments: hbase-7824.patch, hbase-7824_v2.patch,
hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch,
hbase-7824-v9.patch
When there is log split work going on, master start up waits till all log
split work completes even though the log split has nothing to do with meta
region servers.
It's a bad behavior considering a master node can run when log split is
happening while its start up is blocking by log split work.
Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-8287:
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and 0.95
Thanks for the review
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8218) pass HTable as a parameter to method of AggregationClient
[
https://issues.apache.org/jira/browse/HBASE-8218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625093#comment-13625093
]
cuijianwei commented on HBASE-8218:
---
Maybe, we can add HConnection and ExecutorService as members of
AggregationClient. Then, we can invoke constructor: {code} public
HTable(final byte[] tableName, final HConnection connection, final
ExecutorService pool) throws IOException {code} to construct a HTable in each
method. The current signatures won't change, and this constructor of HTable is
lighter because we don't need to construct new HConnection and ExecutorService.
However, we need a close() method for user to close HConnection and
ExecutorService.
pass HTable as a parameter to method of AggregationClient
-
Key: HBASE-8218
URL: https://issues.apache.org/jira/browse/HBASE-8218
Project: HBase
Issue Type: Improvement
Components: Client, Coprocessors
Affects Versions: 0.94.3
Reporter: cuijianwei
In AggregationClient, methods such as max(...), min(...) pass 'tableName' as
a parameter, then a HTable will be created in the method, before the method
return, the created HTable will be closed.
The process above may be heavy because each call must create and close a
HTable. The situation becomes worse when there is only one thread access
HBase using AggregationClient. The underly HConnection of created HTable will
also be created and then closed each time when we invoke these method because
no other HTables using the HConnection. This operation is heavy. Therefore,
can we add another group of methods which pass HTable or HTablePool as a
parameter to methods defined in AggregationClient?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8251) enable SSH before assign META on Master startup
[
https://issues.apache.org/jira/browse/HBASE-8251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625095#comment-13625095
]
Jeffrey Zhong commented on HBASE-8251:
--
Oh, I c. You changed the existing logic. For example, the above check is within
the {code}if (!rit !metaRegionLocation) {{code}
Old logic will offline the meta server if it's online and re-assign meta while
the patch skips meta assignment. I'm not sure if it's a good modification for
scenario that a Meta location is pointing to an offline server or a wrong
server.
enable SSH before assign META on Master startup
---
Key: HBASE-8251
URL: https://issues.apache.org/jira/browse/HBASE-8251
Project: HBase
Issue Type: Bug
Components: master, Region Assignment
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8251-94.patch, HBASE-8251-94-v2.patch
I think HBASE-5918 could not fix this issue. In HMaster#assignRootAndMeta:
1. Assign ROOT.
2. Block until ROOT be opened.
3. Assign META.
4. Block until META be opened.
SSH is enabled after step 4. So if the RS who host ROOT dies before step 4,
master will be blocked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7937) Retry log rolling to support HA NN scenario
[ https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625106#comment-13625106 ] Liang Xie commented on HBASE-7937: -- O,i c, seems my attached code is only suitable for older release, e.g. 0.94.3 :) got it, thanks for explaining, [~himan...@cloudera.com]. Retry log rolling to support HA NN scenario --- Key: HBASE-7937 URL: https://issues.apache.org/jira/browse/HBASE-7937 Project: HBase Issue Type: Bug Components: wal Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.95.1 Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch, HBase-7937-trunk.txt, HBASE-7937-v1.patch A failure in log rolling causes regionserver abort. In case of HA NN, it will be good if there is a retry mechanism to roll the logs. A corresponding jira for MemStore retries is HBASE-7507. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8282) User triggered flushes does not allow compaction to get triggered even if compaction criteria is met
[
https://issues.apache.org/jira/browse/HBASE-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625110#comment-13625110
]
Anoop Sam John commented on HBASE-8282:
---
bq.I think javadoc in flushcache() should be consistent with javadoc in
internalFlushcache().
Yes Ted, I also observer this inconsistency while going through the code after
Ram's mail. We can correct this also.
User triggered flushes does not allow compaction to get triggered even if
compaction criteria is met
Key: HBASE-8282
URL: https://issues.apache.org/jira/browse/HBASE-8282
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.98.0, 0.94.7, 0.95.1
Observe the below logs
{code}
2013-04-04 17:43:45,825 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.3 M
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44388936
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:43:45,831 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
with permission=rwxrwxrwx
2013-04-04 17:43:45,834 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2013-04-04 17:43:45,835 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e:
CompoundBloomFilterWriter
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
NO General Bloom and NO DeleteFamily was added to HFile
(hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e)
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=1051817, memsize=42.3 M, into tmp file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,093 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
as
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e,
entries=54911, sequenceid=1051817, filesize=35.1 M
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~42.3 M/44388936, currentsize=0/0 for region
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9. in 278ms,
sequenceid=-1, compaction requested=true
2013-04-04 17:43:56,335 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Stats: total=394.09 MB, free=3.61 GB, max=4.00 GB, blocks=5263,
accesses=244882, hits=42988, hitRatio=17.55%, , cachingAccesses=49869,
cachingHits=24251, cachingHitsRatio=48.63%, , evictions=0, evicted=20349,
evictedPerRun=Infinity
2013-04-04 17:44:31,119 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.7 M
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44764248
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:44:31,136 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/c5c2cd6aaf1644daa9c858149da39081
[jira] [Comment Edited] (HBASE-8282) User triggered flushes does not allow compaction to get triggered even if compaction criteria is met
[
https://issues.apache.org/jira/browse/HBASE-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625110#comment-13625110
]
Anoop Sam John edited comment on HBASE-8282 at 4/8/13 3:51 AM:
---
bq.I think javadoc in flushcache() should be consistent with javadoc in
internalFlushcache().
Yes Ted, I also observed this inconsistency while going through the code after
Ram's mail. We can correct this also.
was (Author: anoop.hbase):
bq.I think javadoc in flushcache() should be consistent with javadoc in
internalFlushcache().
Yes Ted, I also observer this inconsistency while going through the code after
Ram's mail. We can correct this also.
User triggered flushes does not allow compaction to get triggered even if
compaction criteria is met
Key: HBASE-8282
URL: https://issues.apache.org/jira/browse/HBASE-8282
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.98.0, 0.94.7, 0.95.1
Observe the below logs
{code}
2013-04-04 17:43:45,825 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.3 M
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44388936
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:43:45,831 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
with permission=rwxrwxrwx
2013-04-04 17:43:45,834 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2013-04-04 17:43:45,835 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e:
CompoundBloomFilterWriter
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
NO General Bloom and NO DeleteFamily was added to HFile
(hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e)
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=1051817, memsize=42.3 M, into tmp file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,093 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
as
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e,
entries=54911, sequenceid=1051817, filesize=35.1 M
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~42.3 M/44388936, currentsize=0/0 for region
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9. in 278ms,
sequenceid=-1, compaction requested=true
2013-04-04 17:43:56,335 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Stats: total=394.09 MB, free=3.61 GB, max=4.00 GB, blocks=5263,
accesses=244882, hits=42988, hitRatio=17.55%, , cachingAccesses=49869,
cachingHits=24251, cachingHitsRatio=48.63%, , evictions=0, evicted=20349,
evictedPerRun=Infinity
2013-04-04 17:44:31,119 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.7 M
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44764248
[jira] [Commented] (HBASE-8282) User triggered flushes does not allow compaction to get triggered even if compaction criteria is met
[
https://issues.apache.org/jira/browse/HBASE-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625113#comment-13625113
]
ramkrishna.s.vasudevan commented on HBASE-8282:
---
I will work on this JIRA today and make the necessary changes.
User triggered flushes does not allow compaction to get triggered even if
compaction criteria is met
Key: HBASE-8282
URL: https://issues.apache.org/jira/browse/HBASE-8282
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.98.0, 0.94.7, 0.95.1
Observe the below logs
{code}
2013-04-04 17:43:45,825 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.3 M
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44388936
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:43:45,831 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
with permission=rwxrwxrwx
2013-04-04 17:43:45,834 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2013-04-04 17:43:45,835 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e:
CompoundBloomFilterWriter
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
NO General Bloom and NO DeleteFamily was added to HFile
(hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e)
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=1051817, memsize=42.3 M, into tmp file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,093 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
as
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e,
entries=54911, sequenceid=1051817, filesize=35.1 M
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~42.3 M/44388936, currentsize=0/0 for region
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9. in 278ms,
sequenceid=-1, compaction requested=true
2013-04-04 17:43:56,335 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Stats: total=394.09 MB, free=3.61 GB, max=4.00 GB, blocks=5263,
accesses=244882, hits=42988, hitRatio=17.55%, , cachingAccesses=49869,
cachingHits=24251, cachingHitsRatio=48.63%, , evictions=0, evicted=20349,
evictedPerRun=Infinity
2013-04-04 17:44:31,119 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.7 M
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44764248
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:44:31,136 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/c5c2cd6aaf1644daa9c858149da39081
with permission=rwxrwxrwx
2013-04-04 17:44:31,139 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625122#comment-13625122
]
Hudson commented on HBASE-8287:
---
Integrated in hbase-0.95 #132 (See
[https://builds.apache.org/job/hbase-0.95/132/])
HBASE-8287 TestRegionMergeTransactionOnCluster failed in trunk build #4010
(Revision 1465530)
Result = FAILURE
zjushch :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DispatchMergingRegionHandler.java
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625133#comment-13625133 ] Lars Hofhansl commented on HBASE-7801: -- Any comments on the latest patch? It should give us all the functionality with maximum future flexibility. Allow a deferred sync option per Mutation. -- Key: HBASE-7801 URL: https://issues.apache.org/jira/browse/HBASE-7801 Project: HBase Issue Type: Sub-task Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt Won't have time for parent. But a deferred sync option on a per operation basis comes up quite frequently. In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a special mutation attribute. For batch operation we'd take the safest sync option of any of the mutations. I.e. if there is at least one that wants to be flushed we'd sync the batch, if there's none of those but at least one that wants deferred flush we defer flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625134#comment-13625134 ] Anoop Sam John commented on HBASE-7801: --- [~lhofhansl] I will see the patch today. Allow a deferred sync option per Mutation. -- Key: HBASE-7801 URL: https://issues.apache.org/jira/browse/HBASE-7801 Project: HBase Issue Type: Sub-task Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt Won't have time for parent. But a deferred sync option on a per operation basis comes up quite frequently. In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a special mutation attribute. For batch operation we'd take the safest sync option of any of the mutations. I.e. if there is at least one that wants to be flushed we'd sync the batch, if there's none of those but at least one that wants deferred flush we defer flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625137#comment-13625137
]
Hudson commented on HBASE-8287:
---
Integrated in HBase-TRUNK #4039 (See
[https://builds.apache.org/job/HBase-TRUNK/4039/])
HBASE-8287 TestRegionMergeTransactionOnCluster failed in trunk build #4010
(Revision 1465528)
Result = FAILURE
zjushch :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DispatchMergingRegionHandler.java
TestRegionMergeTransactionOnCluster failed in trunk build #4010
---
Key: HBASE-8287
URL: https://issues.apache.org/jira/browse/HBASE-8287
Project: HBase
Issue Type: Bug
Components: master
Reporter: chunhui shen
Assignee: chunhui shen
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: hbase-trunk-8287.patch
From the log of trunk build #4010:
{code}
2013-04-04 05:45:59,396 INFO
[MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
handler.DispatchMergingRegionHandler(157):
Cancel merging regions
testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
because can't move them together after 842ms
2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
master.AssignmentManager$4(1164):
The master has opened the region
testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
that was onlin
e on quirinus.apache.org,45718,1365054345790
{code}
There's a small probability that fail to move merging regions together to
same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1774) HTable$ClientScanner modifies its input parameters
[
https://issues.apache.org/jira/browse/HBASE-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625147#comment-13625147
]
Anoop Sam John commented on HBASE-1774:
---
So now we can this note in the javadoc for HTableInterface#getScanner(Scan scan)
{code}
* Note that the passed {@link Scan}'s start row and caching properties
* maybe changed.
{code}
[~lhofhansl] we can close this issue as WontFix?
HTable$ClientScanner modifies its input parameters
--
Key: HBASE-1774
URL: https://issues.apache.org/jira/browse/HBASE-1774
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.20.0
Reporter: Jim Kellerman
Assignee: Jim Kellerman
Priority: Critical
HTable$ClientScanner modifies the Scan that is passed to it on construction.
I would consider this to be bad programming practice because if I wanted to
use the same Scan object to scan multiple tables, I would not expect one
table scan to effect the other, but it does.
If input parameters are going to be modified either now or later it should be
called out *loudly* in the javadoc. The only way I found this behavior was by
creating an application that did scan multiple tables using the same Scan
object and having 'wierd stuff' happen.
In my opinion, if you want to modify a field in an input parameter, you
should:
- make a copy of the original object
- optionally return a reference to the copy.
There is no javadoc about this behavior. The only thing I found was a comment
in HTable$ClientScanner:
{code}
// HEADSUP: The scan internal start row can change as we move through
table.
{code}
Is there a use case that requires this behavior? If so, I would recommend
that ResultScanner (and the classes that implement it) provide an accessor
to the mutable copy of the input Scan and leave the input argument alone.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8288) HBaseFileSystem: Refactoring and correct semantics for createPath methods
Himanshu Vashishtha created HBASE-8288: -- Summary: HBaseFileSystem: Refactoring and correct semantics for createPath methods Key: HBASE-8288 URL: https://issues.apache.org/jira/browse/HBASE-8288 Project: HBase Issue Type: Bug Components: Filesystem Integration Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 This jira is for two issues I see in the HBaseFileSystem class: 1) Load testing on a 7 node cluster using ycsb insert workload shows that static initialization of conf properties results in a slightly better throughput. Though the initialization uses HBaseConfiguration.create() call which is expensive (and I tried to avoid that in its first version), this class is used for most of the filesystem class, and had to invoke an additional checkAndSetXX call before making the fs call because it is not certain whether the retry properties are set or not. Having initialize them in static block removes that limitation. 2) Correct semantics for CreatePathXXX method. In case the overwrite flag is false and file already exists, underlying fs throws an exception. It should be re-thrown to the caller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8288) HBaseFileSystem: Refactoring and correct semantics for createPath methods
[ https://issues.apache.org/jira/browse/HBASE-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8288: --- Attachment: HBase-8288-v1.patch Added a test case for createPath with a false overwrite flag. HBaseFileSystem: Refactoring and correct semantics for createPath methods - Key: HBASE-8288 URL: https://issues.apache.org/jira/browse/HBASE-8288 Project: HBase Issue Type: Bug Components: Filesystem Integration Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8288-v1.patch This jira is for two issues I see in the HBaseFileSystem class: 1) Load testing on a 7 node cluster using ycsb insert workload shows that static initialization of conf properties results in a slightly better throughput. Though the initialization uses HBaseConfiguration.create() call which is expensive (and I tried to avoid that in its first version), this class is used for most of the filesystem class, and had to invoke an additional checkAndSetXX call before making the fs call because it is not certain whether the retry properties are set or not. Having initialize them in static block removes that limitation. 2) Correct semantics for CreatePathXXX method. In case the overwrite flag is false and file already exists, underlying fs throws an exception. It should be re-thrown to the caller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8288) HBaseFileSystem: Refactoring and correct semantics for createPath methods
[
https://issues.apache.org/jira/browse/HBASE-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625168#comment-13625168
]
Himanshu Vashishtha commented on HBASE-8288:
Some numbers.
With out patch:
{code}
Target Throughput Threads Workload Operation Number of
Operations Average Throughput Average Latency Average MinLatency
Average MaxLatency Average 95thPercentileLatencyAverage
99thPercentileLatency
12 240 randomWrite INSERT 3600102970 18757
282605400 0
14 280 randomWrite INSERT 420097722 23767
380977460 0
18 360 randomWrite INSERT 540085490 35417
490785510 0
{code}
With patch:
{code}
Target Throughput Threads Workload Operation Number of
Operations Average Throughput Average Latency Average MinLatency
Average MaxLatency Average 95thPercentileLatencyAverage
99thPercentileLatency
12 240 randomWrite INSERT 3600104425 18357
274003340 0
14 280 randomWrite INSERT 4200100603 22867
316764180 0
18 360 randomWrite INSERT 540088809 33387
476365000 0
{code}
HBaseFileSystem: Refactoring and correct semantics for createPath methods
-
Key: HBASE-8288
URL: https://issues.apache.org/jira/browse/HBASE-8288
Project: HBase
Issue Type: Bug
Components: Filesystem Integration
Affects Versions: 0.94.6
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Fix For: 0.94.7
Attachments: HBase-8288-v1.patch
This jira is for two issues I see in the HBaseFileSystem class:
1) Load testing on a 7 node cluster using ycsb insert workload shows that
static initialization of conf properties results in a slightly better
throughput. Though the initialization uses HBaseConfiguration.create() call
which is expensive (and I tried to avoid that in its first version), this
class is used for most of the filesystem class, and had to invoke an
additional checkAndSetXX call before making the fs call because it is not
certain whether the retry properties are set or not. Having initialize them
in static block removes that limitation.
2) Correct semantics for CreatePathXXX method. In case the overwrite flag is
false and file already exists, underlying fs throws an exception. It should
be re-thrown to the caller.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13625173#comment-13625173 ] ramkrishna.s.vasudevan commented on HBASE-8285: --- In ServerCallable() once on receiving NSRE, when we retry don't we relocate the region inside connect(tries != 0);? HBaseClient never recovers for single HTable.get() calls when regions move -- Key: HBASE-8285 URL: https://issues.apache.org/jira/browse/HBASE-8285 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.6.1 Reporter: Varun Sharma Assignee: Varun Sharma Priority: Critical Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8285-0.94.txt, 8285-trunk.txt Steps to reproduce this bug: 1) Gracefull restart a region server causing regions to get redistributed. 2) Client call to this region keeps failing since Meta Cache is never purged on the client for the region that moved. Reason behind the bug: 1) Client continues to hit the old region server. 2) The old region server throws NotServingRegionException which is not handled correctly and the META cache entries are never purged for that server causing the client to keep hitting the old server. The reason lies in ServerCallable code since we only purge META cache entries when there is a RetriesExhaustedException, SocketTimeoutException or ConnectException. However, there is no case check for NotServingRegionException(s). Why is this not a problem for Scan(s) and Put(s) ? a) If a region server is not hosting a region/scanner, then an UnknownScannerException is thrown which causes a relocateRegion() call causing a refresh of the META cache for that particular region. b) For put(s), the processBatchCallback() interface in HConnectionManager is used which clears out META cache entries for all kinds of exceptions except DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
