date:20130709

[jira] [Commented] (HBASE-8817) Enhance The Apache HBase Reference Guide

2013-07-09 Thread Thomas Pan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702960#comment-13702960
]

Thomas Pan commented on HBASE-8817:
---

Let me start with the simpler one, regarding flushes. Please verify that my
understanding is correct.

2.3.1.1. HBase Default Configuration

hbase.regionserver.global.memstore.upperLimit

Maximum size of all memstores in a region server before new updates are blocked
and flushes are forced. Defaults to 40% of heap. Updates are blocked and region
level flushes are forced until size of all memstores in a region server hits
hbase.regionserver.global.memstore.lowerLimit.

hbase.hregion.memstore.flush.size

Memstore will be flushed to disk if size of the memstore exceeds this number of
bytes. When that happens, memstores that belong to the same region will all be
flushed. Value is checked by a thread that runs every
hbase.server.thread.wakefrequency.

9.7.6.1. MemStore
The MemStore holds in-memory modifications to the Store. Modifications are
KeyValues. When asked to flush, current memstore is moved to snapshot and is
cleared. HBase continues to serve edits out of new memstore and backing
snapshot until flusher reports in that the flush succeeded. At this point the
snapshot is let go. Note that when flush happens, memstores that belong to the
same region will all be flushed out.

Enhance The Apache HBase Reference Guide

Key: HBASE-8817
URL: https://issues.apache.org/jira/browse/HBASE-8817
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Thomas Pan

I am reading the guide here: http://hbase.apache.org/book.html#regions.arch
I have noticed the following:
- There is lack of coverage on scan. Note that Lars has a very good
presentation from last year's HBaseCon:
http://www.slideshare.net/cloudera/3-learning-h-base-internals-lars-hofhansl-salesforce-final
- Under section 9.7.6.1, there is no mentioning of flush per region if a
memstore reached the size threshold (and a possible flush of all memstores if
over all memory usage of all the memstores reached memstore.upperlimit). Note
that people need to combine the information provided in section 9 with
section 2.3.1.1 to fully understand the situation.
- Compaction tuning strategies. Both Nicolas from Facebook and Sergey from
Hortonworks have presented various compaction algorithms. Sometime, once all
the necessary code is in master, we need to have the coverage.
Let me know how I can help on the matter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-8897) Add Suspend and Resume to Chaos Monkey

2013-07-09 Thread Elliott Clark (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-8897:
-

Status: Patch Available  (was: Open)

 Add Suspend and Resume to Chaos Monkey
 --

 Key: HBASE-8897
 URL: https://issues.apache.org/jira/browse/HBASE-8897
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.95.1, 0.98.0
Reporter: Elliott Clark
Assignee: Elliott Clark
 Attachments: HBASE-8897-0.patch


 We should add suspend and resume to chaos monkey.  They are already exposed 
 through HBaseClusterManager.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8015) Support for Namespaces

2013-07-09 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702969#comment-13702969
 ] 

stack commented on HBASE-8015:
--

I took a quick look over at 
https://github.com/francisliu/hbase_namespace/commit/18973af443e8c6424c978364e2d0db2a5abe1286#L5R705

Patch is massive but seems to be in the right direction.  I'm thinking weeks -- 
three to four weeks -- for this to stabilize and get committed?

 Support for Namespaces
 --

 Key: HBASE-8015
 URL: https://issues.apache.org/jira/browse/HBASE-8015
 Project: HBase
  Issue Type: New Feature
Reporter: Francis Liu
Assignee: Francis Liu
 Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8884) Pluggable RpcScheduler

2013-07-09 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702975#comment-13702975
]

Hadoop QA commented on HBASE-8884:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591369/hbase-8884-v3.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).

{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6264//console

This message is automatically generated.

Pluggable RpcScheduler
--

Key: HBASE-8884
URL: https://issues.apache.org/jira/browse/HBASE-8884
Project: HBase
Issue Type: Improvement
Components: IPC/RPC
Reporter: Chao Shi
Assignee: Chao Shi
Attachments: hbase-8884.patch, hbase-8884-v2.patch,
hbase-8884-v3.patch

Today, the RPC scheduling mechanism is pretty simple: it execute requests in
isolated thread-pools based on their priority. In the current implementation,
all normal get/put requests are using the same pool. We'd like to add some
per-user or per-region level isolation, so that a misbehaved user/region will
not saturate the thread-pool and cause DoS to others easily. The idea is
similar to FairScheduler in MR. The current scheduling code is not standalone
and is mixed with others (Connection#processRequest). The issue is the first
step to extract it to an interface, so that people are free to write and test
their own implementations.
This patch doesn't make it completely pluggable yet, as some parameters are
pass from constructor. This is because HMaster and HRegionServer both use
RpcServer and they have different thread-pool size config. Let me know if you
have a solution to this.

[jira] [Commented] (HBASE-8817) Enhance The Apache HBase Reference Guide

2013-07-09 Thread Thomas Pan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702994#comment-13702994
]

Thomas Pan commented on HBASE-8817:
---

Here is the draft about scan. I just jot it down for my better understanding.

When client issues a scan against a table, Hbase generates RegionScanner
objects, one per region, to serve the scan request. RegionScanner object
contains a list of StoreScanner objects, one per column family. Each
StoreScanner object further contains a list of StoreFileScanner objects,
corresponding to each store file and HFile of the corresponding column family,
and a list of KeyValueScanner objects for memstore. The two lists are merge
into one, sorted in ascending order with scan object for memstore at the end of
the list. When a StoreFileScanner object is constructed, it is associated with
a MultiVersionConsistencyControl read point, which is the current memstoreTS at
the moment, filtering out any new updates beyond the read point.

Enhance The Apache HBase Reference Guide

Key: HBASE-8817
URL: https://issues.apache.org/jira/browse/HBASE-8817
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Thomas Pan

1 2 3 >

1 - 100 of 242 matches

Mail list logo