subject:"\[jira\] \[Commented\] \(HBASE\-10855\) Enable hfilev3 by default"

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-03 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959298#comment-13959298
 ] 

stack commented on HBASE-10855:
---

FYI, here is what happens if you start a hbase-0.96.x over a directory written 
w/ hfile v3s:

{code}
Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem 
reading HFile Trailer from file 
file:/var/folders/cj/jgfy62h13vz019xgz681df_rgp/T/hbase-stack/hbase/data/hbase/meta/1588230740/info/38f517aa7abf40ffbc4f8fa7b170f4bc
at 
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:581)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:607)
at 
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1083)
at 
org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:188)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:351)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:446)
at 
org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:478)
at 
org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:109)
at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:431)
at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:428)
... 6 more
Caused by: java.lang.IllegalArgumentException: Invalid HFile version: 3 
(expected to be between 2 and 2)
at 
org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:912)
at 
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:429)
at 
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:567)
... 15 more
...

2014-04-03 14:53:28,114 INFO  [RS_OPEN_META-172.21.8.204:50714-0] 
handler.OpenRegionHandler: Opening of region {ENCODED = 1588230740, NAME = 
'hbase:meta,,1', STARTKEY = '', ENDKEY = ''} failed, transitioning from 
OPENING to FAILED_OPEN in ZK, expecting version 28
2014-04-03 14:53:28,114 DEBUG [RS_OPEN_META-172.21.8.204:50714-0] 
zookeeper.ZKAssign: regionserver:50714-0x1452994d1c50002, 
quorum=localhost:2181, baseZNode=/hbase Transitioning 1588230740 from 
RS_ZK_REGION_OPENING to RS_ZK_REGION_FAILED_OPEN
2014-04-03 14:53:28,116 DEBUG [RS_OPEN_META-172.21.8.204:50714-0] 
zookeeper.ZKAssign: regionserver:50714-0x1452994d1c50002, 
quorum=localhost:2181, baseZNode=/hbase Transitioned node 1588230740 from 
RS_ZK_REGION_OPENING to RS_ZK_REGION_FAILED_OPEN
2014-04-03 14:53:28,116 DEBUG [AM.ZK.Worker-pool2-t9] master.AssignmentManager: 
Handling RS_ZK_REGION_FAILED_OPEN, server=172.21.8.204,50714,1396561996130, 
region=1588230740, current_state={1588230740 state=OPENING, ts=1396562008105, 
server=172.21.8.204,50714,1396561996130}
2014-04-03 14:53:28,116 WARN  [AM.ZK.Worker-pool2-t9] master.RegionStates: 
Failed to open/close 1588230740 on 172.21.8.204,50714,1396561996130, set to 
FAILED_OPEN
2014-04-03 14:53:28,116 INFO  [AM.ZK.Worker-pool2-t9] master.RegionStates: 
Transitioned {1588230740 state=OPENING, ts=1396562008105, 
server=172.21.8.204,50714,1396561996130} to {1588230740 state=FAILED_OPEN, 
ts=1396562008116, server=172.21.8.204,50714,1396561996130}
{code}

We fail to open the region which is the right thing to do.

 Enable hfilev3 by default
 -

 Key: HBASE-10855
 URL: https://issues.apache.org/jira/browse/HBASE-10855
 Project: HBase
  Issue Type: Sub-task
  Components: HFile
Reporter: stack
Assignee: stack
 Fix For: 0.99.0

 Attachments: 10855.txt, 10855.txt, 10855.txt, 10855.txt, 10855v2.txt, 
 10855v2.txt, 10855v2.txt


 Distributed log replay needs this.  Should be on by default in 1.0/0.99.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-01 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956147#comment-13956147
 ] 

ramkrishna.s.vasudevan commented on HBASE-10855:


For me too it passes locally changing the version to V3.
But just seeing the test case one guess would be that because we use V3 once we 
flush we would atleast write the tag length (of type short).
Here we only make two puts for two diff families.  The size of the HFile may be 
a bigger by 2 bytes now. Will that be a reason here?


 Enable hfilev3 by default
 -

 Key: HBASE-10855
 URL: https://issues.apache.org/jira/browse/HBASE-10855
 Project: HBase
  Issue Type: Sub-task
  Components: HFile
Reporter: stack
Assignee: stack
 Fix For: 0.99.0

 Attachments: 10855.txt, 10855.txt, 10855.txt, 10855.txt


 Distributed log replay needs this.  Should be on by default in 1.0/0.99.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-01 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956150#comment-13956150
 ] 

stack commented on HBASE-10855:
---

Let me look at it [~ram_krish]  I have a little rig here so can dig.  Will bug 
you fellows if can't figure it.  It seems like a good test but a little fragile 
anyways 

 Enable hfilev3 by default
 -

 Key: HBASE-10855
 URL: https://issues.apache.org/jira/browse/HBASE-10855
 Project: HBase
  Issue Type: Sub-task
  Components: HFile
Reporter: stack
Assignee: stack
 Fix For: 0.99.0

 Attachments: 10855.txt, 10855.txt, 10855.txt, 10855.txt


 Distributed log replay needs this.  Should be on by default in 1.0/0.99.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-01 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956813#comment-13956813
 ] 

Hadoop QA commented on HBASE-10855:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12638081/10855v2.txt
  against trunk revision .
  ATTACHMENT ID: 12638081

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.procedure.TestZKProcedure
  org.apache.hadoop.hbase.master.TestMasterNoCluster

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9158//console

This message is automatically generated.

 Enable hfilev3 by default
 -

 Key: HBASE-10855
 URL: https://issues.apache.org/jira/browse/HBASE-10855
 Project: HBase
  Issue Type: Sub-task
  Components: HFile
Reporter: stack
Assignee: stack
 Fix For: 0.99.0

 Attachments: 10855.txt, 10855.txt, 10855.txt, 10855.txt, 10855v2.txt


 Distributed log replay needs this.  Should be on by default in 1.0/0.99.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-01 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13956985#comment-13956985
]

Hadoop QA commented on HBASE-10855:
---

{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638106/10855v2.txt
against trunk revision .
ATTACHMENT ID: 12638106

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9161//console

This message is automatically generated.

Enable hfilev3 by default
-

Key: HBASE-10855
URL: https://issues.apache.org/jira/browse/HBASE-10855
Project: HBase
Issue Type: Sub-task
Components: HFile
Reporter: stack
Assignee: stack
Fix For: 0.99.0

Attachments: 10855.txt, 10855.txt, 10855.txt, 10855.txt, 10855v2.txt,
10855v2.txt

Distributed log replay needs this. Should be on by default in 1.0/0.99.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HBASE-10855) Enable hfilev3 by default

2014-04-01 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13957109#comment-13957109
]

Hadoop QA commented on HBASE-10855:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638128/10855v2.txt
against trunk revision .
ATTACHMENT ID: 12638128