[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-07-29 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078882#comment-14078882
 ] 

Hudson commented on HBASE-11120:


FAILURE: Integrated in HBase-TRUNK #5353 (See 
[https://builds.apache.org/job/HBase-TRUNK/5353/])
HBASE-11316 Expand info about compactions beyond HBASE-11120 ADDENDUM TO FIX 
BROKE TESTS (stack: rev 85853f69c88c08db61af7fb6ef3b5432f926e531)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java


 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch, 
 hbase-11120.addendum.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-07-28 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077338#comment-14077338
 ] 

Hudson commented on HBASE-11120:


FAILURE: Integrated in HBase-TRUNK #5349 (See 
[https://builds.apache.org/job/HBase-TRUNK/5349/])
HBASE-11316 Expand info about compactions beyond HBASE-11120 (Misty 
Stanley-Jones) (jdcryans: rev afae1e2583b8b32d828efbadce363247c784a93e)
* hbase-common/src/main/resources/hbase-default.xml
* src/main/docbkx/book.xml


 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch, 
 hbase-11120.addendum.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-10 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026649#comment-14026649
 ] 

Jonathan Hsieh commented on HBASE-11120:


Patch seems to have broken trunk.  Posting and committing trivial addendum to 
fix shortly.




 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-10 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026661#comment-14026661
 ] 

Jonathan Hsieh commented on HBASE-11120:


jimmy seems to have beaten me to the commit. :)  thanks!

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Jonathan Hsieh
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch, 
 hbase-11120.addendum.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-10 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026681#comment-14026681
 ] 

stack commented on HBASE-11120:
---

Thanks for fix up [~jxiang] and [~jmhsieh].

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch, 
 hbase-11120.addendum.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-10 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026876#comment-14026876
 ] 

Hudson commented on HBASE-11120:


SUCCESS: Integrated in HBase-TRUNK #5191 (See 
[https://builds.apache.org/job/HBase-TRUNK/5191/])
HBASE-11120 Addendum for fixing compilation error (jxiang: rev 
ee762257835a060e9e01244fb8b0dc97682af098)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreConfig.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java


 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch, 
 hbase-11120.addendum.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025292#comment-14025292
 ] 

Jean-Daniel Cryans commented on HBASE-11120:


bq.  I don't see hbase.hstore.compaction.min.size in code base any more

Yeah I was confused by this too, [~stack], until I found this thread that you 
started for this reason a while back: 
http://osdir.com/ml/general/2013-06/msg41068.html

Basically, {{CompactionConfiguration}} was written with a configuration 
prefix so doing a grep on that config (or hbase.hstore.compaction.max.size) 
won't return anything. I think we should undo the refactoring in that class 
since it obviously breaks a pattern we're used to, which is to completely spell 
out the configuration names so we can easily search for them.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, 
 HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025525#comment-14025525
 ] 

stack commented on HBASE-11120:
---

[~jdcryans] Thanks for the digging. Agree. Let me change it.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, 
 HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025561#comment-14025561
 ] 

stack commented on HBASE-11120:
---

So now [~misty], why did the file size shrink so much? (I can commit my 
addendum along w/ your doc patch when done).

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025815#comment-14025815
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

I don't know why it shrunk. At the time I was very confused and it seemed as 
though maybe somehow the HBASE-11120-3-rebased.patch got applied to master. Let 
me try to replay it and make sure that everything is really there so I don't 
lose any work. I will let you know.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025890#comment-14025890
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

I am figuring that out by doing 'git annotate' and searching for the changes.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025888#comment-14025888
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

OK it shrunk because somehow a lot of these changes got applied to master by 
your commit on HBASE-11199, [~stack]. Can you please confirm?

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-09 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026117#comment-14026117
 ] 

Hudson commented on HBASE-11120:


FAILURE: Integrated in HBase-TRUNK #5190 (See 
[https://builds.apache.org/job/HBase-TRUNK/5190/])
HBASE-11120 Update documentation about major compaction algorithm (Misty 
Stanley-Jones) + addendum fixing CompactionConfiguration (stack: rev 
948b22f082e1c598fc553a34772226de31b8b0b7)
* src/main/docbkx/book.xml
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java


 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 0.99.0

 Attachments: 11120_addendum.txt, HBASE-11120-2.patch, 
 HBASE-11120-3-rebased.patch, HBASE-11120-3.patch, 
 HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-07 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021081#comment-14021081
 ] 

stack commented on HBASE-11120:
---

Why did the patch go from 21k to 6k [~misty]?  Did you get less ambitious?

Looking the below:

{code}
 entryhbase.hstore.compaction.min.size/entry
 entryA StoreFile smaller than this size (in bytes) will 
always be eligible for
   minor compaction./entry
-entry10/entry
+entry128 MB/entry
   /row
   row
 entryhbase.hstore.compaction.max.size/entry
 entryA StoreFile larger than this size (in bytes) will 
be excluded from minor
   compaction./entry
-entry1000/entry
+entryLong.MAX_VALUE/entry
{code}

... I don't see hbase.hstore.compaction.min.size in code base any more and you 
can't write Long.MAX_VALUE or 128 MB as values... our configuration system 
can't digest anything but numbers.

Let me know why big difference in size.  Maybe this patch should have had more 
in it?  Thanks.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, 
 HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-06-05 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14019516#comment-14019516
 ] 

Hadoop QA commented on HBASE-11120:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12648582/HBASE-11120-4-rebased.patch
  against trunk revision .
  ATTACHMENT ID: 12648582

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+
xlink:href=https://issues.apache.org/jira/browse/HBASE-7842;ExploringCompaction
 Policy/link/title
+termIf this compaction was user-requested, perform the 
requested type of compaction./term
+compaction is requested manually, HBase always runs that 
type of compaction. If the
+user requests a major compaction, the major compaction 
still runs even if the are

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9700//console

This message is automatically generated.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120-4-rebased.patch, HBASE-11120-4.patch, 
 HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-29 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012875#comment-14012875
 ] 

Jean-Daniel Cryans commented on HBASE-11120:


Spent some more time reading through and I saw that I had missed a few things. 
BTW, thanks again Misty for working on this, it is much needed.

I'm not sure about this part:

{quote}
However, if the normal compaction
+algorithm do not find any normally-eligible StoreFiles, a 
major compaction is
+the only way to get out of this situation, and is forced. 
This is also called a
+size-based or size-triggered major compaction.
{quote}

The exploration compaction will select the smallest set of files to compact and 
it won't trigger a major compaction. Also, I looked for size-based and 
size-triggered in the code but couldn't find anything so I'm not sure what 
this is referring to.

This is wrong:

{quote}
+termIf this compaction was user-requested, do a major 
compaction./term
+listitem
+  paraCompactions can run on a schedule or can be initiated 
manually. If a
+compaction is requested manually, it is always a major 
compaction. If the
+compaction is user-requested, the major compaction still 
happens even if the are
+more than varnamehbase.hstore.compaction.max/varname 
files that need
+compaction./para
+/listitem
{quote}

It will do a minor compaction if the user asks for that, or a major compaction 
if it's specifically asked for.

It might be good to mention that hbase.hstore.compactionThreshold was the old 
name for hbase.hstore.compaction.min in this section:

{quote}
+  row
+entryhbase.hstore.compaction.min/entry
+entryThe minimum number of files which must be eligible 
for compaction before
+  compaction can run./entry
+entry3/entry
+  /row
{quote}

Those two are wrong (if 1000 bytes was the max we'd probably never compact :P):

{quote}
+  row
+entryhbase.hstore.compaction.min.size/entry
+entryA StoreFile smaller than this size (in bytes) will 
always be eligible for
+  minor compaction./entry
+entry10/entry
+  /row
+  row
+entryhbase.hstore.compaction.max.size/entry
+entryA StoreFile larger than this size (in bytes) will 
be excluded from minor
+  compaction./entry
+entry1000/entry
+  /row
{quote}

It should be 128MB and Long.MAX_VALUE as per: 
https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionConfiguration.java#L71

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-29 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013249#comment-14013249
 ] 

Hadoop QA commented on HBASE-11120:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12647508/HBASE-11120-4.patch
  against trunk revision .
  ATTACHMENT ID: 12647508

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+
xlink:href=https://issues.apache.org/jira/browse/HBASE-7842;ExploringCompaction
 Policy/link/title
+termIf this compaction was user-requested, perform the 
requested type of compaction./term
+compaction is requested manually, HBase always runs that 
type of compaction. If the
+user requests a major compaction, the major compaction 
still runs even if the are

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9642//console

This message is automatically generated.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120-4.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-27 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010754#comment-14010754
 ] 

Hadoop QA commented on HBASE-11120:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12647031/HBASE-11120-3-rebased.patch
  against trunk revision .
  ATTACHMENT ID: 12647031

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:red}-1 release audit{color}.  The applied patch generated 32 release 
audit warnings (more than the trunk's current 0 warnings).

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+Obviously, once a major compaction has run, such behavior will 
not be the case anymore...footnote
+linkend=major.compactions.change.query.results / for a more 
in-depth explanation. This
+
xlink:href=https://issues.apache.org/jira/browse/HBASE-7842;Exploring 
Compaction Policy/link/title
+
filenamehbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java/filename./para

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/9613//console

This message is automatically generated.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120-2.patch, HBASE-11120-3-rebased.patch, 
 HBASE-11120-3.patch, HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-22 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006215#comment-14006215
 ] 

Sergey Shelukhin commented on HBASE-11120:
--

This may need correcting?
{noformat}
+entry /
+entry /
+entry1.2F/entry
+  /row
{noformat}

{noformat}
+entryhbase.hstore.compaction.max.size/entry
+entryThe maximum size for a file to be eligible for 
compaction, expressed in
+  bytes./entry
+entry1000/entry
{noformat}
This is not a very good description imho...

It would be good to insert link to effect of major compaction on read results 
into the section about deletes. There's description somewhere in the book 
already.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-22 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006480#comment-14006480
 ] 

Jean-Daniel Cryans commented on HBASE-11120:


On top of Sergey's comments:

This could also use a sentence regarding TTLs, they are simply filtered out and 
tombstones aren't created:

{noformat}
bq. titleCompaction and Deletions/title
{noformat}

Same kind of comment for this para, the old versions are filtered more than 
deleted:

{noformat}
+titleCompaction and Versions/title
{noformat}

One thing bothers me about this para:

{noformat}
+paraThe compaction algorithms used by HBase have evolved over 
time. HBase 0.96
+  introduced a new algorithm for compaction file selection. To 
find out about the old
+  algorithm, see xref
+linkend=compaction /. The rest of this section describes 
the new algorithm, which
+  was implemented in link
+
xlink:href=https://issues.apache.org/jira/browse/HBASE-7842;HBASE-7842/link.
{noformat}

What's written next is much more about how to pick the files that will then be 
considered for compaction than the actual new policy which is in 
ExploringCompactionPolicy.java. In other words, the text that follows what I 
just quoted is ok but not really new in 0.96 via HBASE-7842, and it seems to me 
that we should explain what HBASE-7842 does.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-22 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006714#comment-14006714
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

bq. It would be good to insert link to effect of major compaction on read 
results into the section about deletes. There's description somewhere in the 
book already.

Do you mean 5.9.2.2. Major compactions change query results?

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-21 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005462#comment-14005462
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

Integrating review comments from [~jdcryans], cancelling patch review in the 
meantime.

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-21 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005472#comment-14005472
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

 Check to see if we absolutely have to compact in case we have too many store 
files
then the next para:
+termIf this compaction was user-requested, try to do a 
major
+  compaction./term
see the link I posted above, we only major compact when the user asks for it
I think the code you saw was more like forcing the major compaction if the user 
was indeed requesting it

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (HBASE-11120) Update documentation about major compaction algorithm

2014-05-21 Thread Misty Stanley-Jones (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005471#comment-14005471
 ] 

Misty Stanley-Jones commented on HBASE-11120:
-

a store can only have so many files, and that's called the blocking store file 
count
so you cannot flush data if you have too many files, you have to wait for a 
compaciton
and you cannot insert if the memstore is over some threshold
but then the compaction selection might not select a single file if none of 
them are good
so if you get there, you're stuck
that's the accounting here:
boolean mayBeStuck = (candidateFiles.size() - filesCompacting.size() + 
futureFiles)
= storeConfigInfo.getBlockingFileCount();
Jean-Daniel Cryans
10:29 am
if our files we're supposed to compact, minus the ones we're already 
compacting, plus 1 since those files will end up being a new store file, is 
greater or equal to the blocking file count
then we force a compacting
compaction

 Update documentation about major compaction algorithm
 -

 Key: HBASE-11120
 URL: https://issues.apache.org/jira/browse/HBASE-11120
 Project: HBase
  Issue Type: Bug
  Components: Compaction, documentation
Affects Versions: 0.98.2
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-11120.patch


 [14:20:38]  jdcryans seems that there's 
 http://hbase.apache.org/book.html#compaction and 
 http://hbase.apache.org/book.html#managed.compactions
 [14:20:56]  jdcryans the latter doesn't say much, except that you 
 should manage them
 [14:21:44]  jdcryans the former gives a good description of the 
 _old_ selection algo
 [14:45:25]  jdcryans this is the new selection algo since C5 / 
 0.96.0: https://issues.apache.org/jira/browse/HBASE-7842



--
This message was sent by Atlassian JIRA
(v6.2#6252)