[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-29 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078881#comment-14078881
 ] 

Hudson commented on HBASE-11316:


FAILURE: Integrated in HBase-TRUNK #5353 (See 
[https://builds.apache.org/job/HBase-TRUNK/5353/])
HBASE-11316 Expand info about compactions beyond HBASE-11120 ADDENDUM TO FIX 
BROKE TESTS (stack: rev 85853f69c88c08db61af7fb6ef3b5432f926e531)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java


> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316-9.patch, HBASE-11316.patch, addendum.txt, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-29 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078779#comment-14078779
 ] 

stack commented on HBASE-11316:
---

Pushed addendum.  [~jdcryans] now I only owe you $2.1455M dollars.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316-9.patch, HBASE-11316.patch, addendum.txt, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-29 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078104#comment-14078104
 ] 

Jean-Daniel Cryans commented on HBASE-11316:


+1, thanks for looking at this Stack and sorry I overlooked those failing tests.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316-9.patch, HBASE-11316.patch, addendum.txt, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-29 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078013#comment-14078013
 ] 

stack commented on HBASE-11316:
---

Interesting.  The test failures are legit:

org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

The hbase-default additions seem to break above tests.  Let me isolate which 
changes are responsible for the break.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316-9.patch, HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-28 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077336#comment-14077336
 ] 

Hudson commented on HBASE-11316:


FAILURE: Integrated in HBase-TRUNK #5349 (See 
[https://builds.apache.org/job/HBase-TRUNK/5349/])
HBASE-11316 Expand info about compactions beyond HBASE-11120 (Misty 
Stanley-Jones) (jdcryans: rev afae1e2583b8b32d828efbadce363247c784a93e)
* src/main/docbkx/book.xml
* hbase-common/src/main/resources/hbase-default.xml


> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316-9.patch, HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-28 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077225#comment-14077225
 ] 

Jean-Daniel Cryans commented on HBASE-11316:


Comments:

{noformat}
-Name of the failsafe snapshot taken by the restore operation.
+Name of the failsafe snapshot taken by the restore opecompn.
{noformat}

Typo slipped in there.

{noformat}
+  Being Stuck
{noformat}

Looking at the output, this line (and the other titles at that level) is 
comically small. There's probably we can do short of changing the style sheet.

And I'm good with the rest.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-27 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075773#comment-14075773
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12658042/HBASE-11316-8.patch
  against trunk revision .
  ATTACHMENT ID: 12658042

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+HBase since version 1.0 and ask that a column family have both its 
meta and DATA blocks hosted onheap in the L1 tier by
+A StoreFile is a facade of 
HFile. In terms of compaction, use of
+  
xlink:href="https://issues.apache.org/jira/browse/HBASE-11316";>HBASE-11316.
+drop (filter out) deletes or expired versions, because of 
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  will compact larger StoreFiles. However, during 
reads, HBase will need to seek
+  through fewer StpreFo;es to accomplish the read. 
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10197//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316-7.patch, HBASE-11316-8.patch, 
> HBASE-1

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-17 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065941#comment-14065941
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12656417/HBASE-11316-7.patch
  against trunk revision .
  ATTACHMENT ID: 12656417

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+A StoreFile is a facade of 
HFile. In terms of compaction, use of
+  
xlink:href="https://issues.apache.org/jira/browse/HBASE-11316";>HBASE-11316.
+drop (filter out) deletes or expired versions, because of 
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  will compact larger StoreFiles. However, during 
reads, HBase will need to seek
+  through fewer StpreFo;es to accomplish the read. 
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}}

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10115//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-17 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065900#comment-14065900
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

The last patch broke everything, don't use it. Stand by for a fix.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased.patch, 
> HBASE-11316-6.patch, HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-17 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065864#comment-14065864
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12656404/HBASE-11316-6-rebased-v2.patch
  against trunk revision .
  ATTACHMENT ID: 12656404

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+A StoreFile is a facade of 
HFile. In terms of compaction, use of
+  
xlink:href="https://issues.apache.org/jira/browse/HBASE-11316";>HBASE-11316.
+drop (filter out) deletes or expired versions, because of 
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  will compact larger StoreFiles. However, during 
reads, HBase will need to seek
+  through fewer StpreFo;es to accomplish the read. 
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}}

{color:red}-1 site{color}.  The patch appears to cause mvn site goal to 
fail.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
  
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10113//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased-v2.patch, HBASE-11316-6-rebased

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-09 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057026#comment-14057026
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12654913/HBASE-11316-6-rebased.patch
  against trunk revision .
  ATTACHMENT ID: 12654913

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/10008//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6-rebased.patch, HBASE-11316-6.patch, HBASE-11316.patch, 
> ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049661#comment-14049661
 ] 

stack commented on HBASE-11316:
---

bq. Well, it seems like you and JD are in disagreement on this so I am unsure 
how to proceed.

Sorry for confusion caused. Go storefile and store.  JD and Esteban have a 
point especially given that is how we talk up files in UI (metrics also talk 
storefile). Suggest talking up how relates to hfile and column family since 
these are notions that are all over the hbase shop too. Thanks Misty.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6.patch, HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049623#comment-14049623
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12653533/HBASE-11316-6.patch
  against trunk revision .
  ATTACHMENT ID: 12653533

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 findbugs{color}.  The patch appears to introduce 13 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+A StoreFile is a facade of 
HFile. In terms of compaction, use of
+  
xlink:href="https://issues.apache.org/jira/browse/HBASE-11316";>HBASE-11316.
+drop (filter out) deletes or expired versions, because of 
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  will compact larger StoreFiles. However, during 
reads, HBase will need to seek
+  through fewer StpreFo;es to accomplish the read. 
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}}

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9936//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316-6.patch, HBASE-113

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049588#comment-14049588
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

Well, it seems like you and JD are in disagreement on this so I am unsure how 
to proceed. :)

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049574#comment-14049574
 ] 

stack commented on HBASE-11316:
---

Misty you have enough to go on? Just ask if not.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049476#comment-14049476
 ] 

Jean-Daniel Cryans commented on HBASE-11316:


IMO, unless we specifically refer to something related to the HFile 
implementation, we should always use StoreFile. That's also how we present in 
the web UI when list the number of Stores and StoreFiles in a region.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049410#comment-14049410
 ] 

stack commented on HBASE-11316:
---

In the code, StoreFile is a facade over our file implementation HFile.  Store 
is how we implement the modeling concept column family.  We should have named 
Stores 'ColumnFamily' rather than Store.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049405#comment-14049405
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

OK, then how do they relate? Especially Store / Column Family. It seems really 
strange to me to talk about an HFile being "in" a Column Family. Can you help 
me out?

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049402#comment-14049402
 ] 

stack commented on HBASE-11316:
---

Folks seem to update hfile rather than put in place a new version (it is on 
version 3 at the moment).  hfile seems to be known outside hbase (we bulk 
import hfiles, not storefiles).

hfile is used more often in our doc currently:

kalashnikov:docbkx stack$ grep hfile *|wc
  97 980   11524
kalashnikov:docbkx stack$ grep -i hfile * |wc
 2524413   35218
kalashnikov:docbkx stack$ grep -i storefile * |wc
 1882386   22833

If you just use storefile, folks will wonder where hfile fits.

You are the boss. Pick one.  Just explain how the two relate on first use.  
Good on you misty.  Ditto with column family/Store.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049354#comment-14049354
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

So, [~jdcryans], [~esteban], and I talked about this yesterday and he seemed to 
have the opinion that we should be using StoreFile and Store, since we could 
support other types of StoreFiles besides HFile later on. Based on that, I 
changed it all back. Now I will hold off on further changes until I get more  
consistent advice about how I Should be referring to these things. :)

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-07-01 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049114#comment-14049114
 ] 

stack commented on HBASE-11316:
---

bq. Anyway I played around with this and tried to be consistent in using HFile 
instead of StoreFile, Column Family instead of Store.

This is the way to go IMO.

Looking at PDF, you have this in spite of the above: "it flushes its contents 
to a StoreFile. The number of StoreFiles in a Store increases over time. "  
What you thinking given the above?

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch, ch9_compactions.pdf
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048555#comment-14048555
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12653316/HBASE-11316-5.patch
  against trunk revision .
  ATTACHMENT ID: 12653316

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 findbugs{color}.  The patch appears to introduce 13 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+drop (filter out) deletes or expired versions, because of 
potential side effects. See hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  will compact larger StoreFiles. However, during 
reads, HBase will need to seek
+  through fewer StpreFo;es to accomplish the read. 
Consider this approach if you
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}}
+create 'orders_table', 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}  

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9912//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-30 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048503#comment-14048503
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

Ready for review (again).

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, 
> HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048496#comment-14048496
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12653289/HBASE-11316-4.patch
  against trunk revision .
  ATTACHMENT ID: 12653289

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 findbugs{color}.  The patch appears to introduce 13 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+ The time for which a region will block updates after 
reaching the StoreFile (HFile)
+  can try raising this value. Default: the value of LONG.MAX_VALUE, 
expressed in bytes.
+  
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+compaction. This may be because not all HFiles in the 
Column Family are available to
+less than the sum of the sizes of the other 
HFiles in the set.
+HFile. Its function is to remove deletes and expired 
versions, and reset locality
+HFile to be compacted, a major compaction is run, even if 
it would otherwise have
+  compaction. Its effect is to limit compaction of 
large HFiles. The value of
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836";>Store
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9908//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> ta

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-30 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048483#comment-14048483
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

Got clarification from JD that it is most appropriate to talk about StoreFiles 
and Stores, not HFiles and ColumnFamilies. I'll update that patch to reflect 
that. Sorry about the churn.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-30 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048371#comment-14048371
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

Working on this today. Do you prefer HFile or StoreFile? I didn't actually 
realize they were the same thing when I was originally writing this.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-26 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044939#comment-14044939
 ] 

stack commented on HBASE-11316:
---

I see 'agiven' in first description section in the hbase-default.xml.   I 
started reviewing and it looks really good -- thanks in particular for working 
on these hbase-default.xml descriptions -- but I kept tripping over your use of 
HStore vs Store vs StoreFile vs HStoreFile.  I started to edit and was adding 
in after StoreFile (HFile) in commas because that is how we usually refer to 
these files  and Column Family for Store since they equate but I gave up 
thinking I should be punting this to you since you are probably trying to have 
one voice across the doc and one set of terminology you'd like to apply.  How 
would you like to deal with HStoreFile vs StoreFile vs HFile?  And Store vs 
HStore vs Column Family?  Good on you Misty.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-22 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040307#comment-14040307
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

BTW, noticed and fixed a problem (empty parentheses in the first line of the 
section) but won't regenerate the PDF for that.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-22 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040303#comment-14040303
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

I can't find either of your typos and am a bit worried you might be looking at 
an old output. Let me generate a PDF and attach. I don't know about Review 
Board -- who should I ask?

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-20 Thread Lars George (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038796#comment-14038796
 ] 

Lars George commented on HBASE-11316:
-

Typo in "agiven" (missing space) and "milliseconds.." (two dots). I need to 
read the rest still, but that will take a little longer. Maybe this is larger 
enough to stick it on review board - or else it is getting messy feedback wise.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-15 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032068#comment-14032068
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12650499/HBASE-11316-3.patch
  against trunk revision .
  ATTACHMENT ID: 12650499

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 findbugs{color}.  The patch appears to introduce 1 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+is a minor or major compaction) is called the 
compaction policy.
+  
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  major compactions. This approach alleviates some of the 
challenges of larger regions.
+Stripe compaction is fully compatible with  and works in
+  conjunction with either the ExploringCompactionPolicy or 
RatioBasedCompactionPolicy. It can be enabled for
+  You can enable stripe compaction for a table or a column 
family, by setting its hbase.hstore.engine.class to
+alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' 
=> 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}
+alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}}
+create 'orders_table', 'blobs_cf', CONFIGURATION => 
{'hbase.hstore.engine.class' => 
'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 
'hbase.hstore.blockingStoreFiles' => '100'}  
+hbase.store.stripe.compaction.flushToL0 
to true. This will

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
  
org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9776//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316-3.patch, HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe com

[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-13 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031321#comment-14031321
 ] 

stack commented on HBASE-11316:
---

Ugh.  This no longer applies.  Please rebase [~misty]  I took a read through.  
It is very good.  Minor is that a minor compaction could end up with one file 
when done (you say it can't) and you never edit hbase-default.xml.  You add 
your configs to hbase-site.xml.

The writeups you have on each of the compaction configs, should these go into 
the actual hbase-default.xml too?

Excellent Misty.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-12 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028915#comment-14028915
 ] 

Hadoop QA commented on HBASE-11316:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12650004/HBASE-11316-2.patch
  against trunk revision .
  ATTACHMENT ID: 12650004

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+which resides within a Store. The number of StoreFiles 
increases over time. Compaction is an operation which 
reduces the number of
+StoreFile. Minor compactions do not drop 
(filter out) deletes or expired
+versions, because of potential side effects. See  and compaction policy.
+  
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java.
+  The maximum number of files which will be 
selected for a single minor
+  There are two different thread pools for 
compactions, one for large
+  major compactions. This approach alleviates some of the 
challenges of larger regions.
+Stripe compaction is fully compatible with  and works in
+  conjunction with either the ExploringCompactionPolicy or 
RatioBasedCompactionPolicy. It can be enabled for

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9753//console

This message is automatically generated.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120

2014-06-11 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028853#comment-14028853
 ] 

Misty Stanley-Jones commented on HBASE-11316:
-

File size shrunk because I rebased -- sorry.

> Expand info about compactions beyond HBASE-11120
> 
>
> Key: HBASE-11316
> URL: https://issues.apache.org/jira/browse/HBASE-11316
> Project: HBase
>  Issue Type: Bug
>  Components: Compaction, documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, 
> HBASE-11316.patch
>
>
> Round 2 - expand info about the algorithms, talk about stripe compaction, and 
> talk more about configuration. Hopefully find some rules of thumb.



--
This message was sent by Atlassian JIRA
(v6.2#6252)