[
https://issues.apache.org/jira/browse/HBASE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085442#comment-13085442
]
[email protected] commented on HBASE-4192:
------------------------------------------------------
bq. On 2011-08-13 01:02:17, Todd Lipcon wrote:
bq. > src/main/java/org/apache/hadoop/hbase/master/HMaster.java, line 997
bq. > <https://reviews.apache.org/r/1463/diff/2/?file=32490#file32490line997>
bq. >
bq. > I really don't like that we're threading the RPCServer through to
the HRegion and HLog objects. There's something really messy about it: imagine,
for example, if we had two separate RPC servers on each daemon, listening on
different ports (this is done in the Hadoop namenode in order to achieve some
easy QoS). Then we'd have to thread two RPCServers through.
bq. >
bq. > Instead, I think the individual operations should use ThreadLocals
to grab the current Call object, perhaps? If the ThreadLocal lookups are too
expensive, then perhaps a new interface like "CallContext" which we thread
through the processing of individual operations?
I don't understand how ThreadLocals would solve this problem. We would still
have to pass the ThreadLocal all the way from the region server to the HLog.
Is there something I am missing?
- Vlad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1463/#review1436
-----------------------------------------------------------
On 2011-08-12 23:44:23, Vlad Dogaru wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1463/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-12 23:44:23)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Changes:
bq.
bq. 1. Add hbase.region.wal.batchentries configuration parameter. If this is
bq. enabled, batch entries to the HLog in a queue.
bq. 2. Use delayed RPCs for sync requests when aggresive batching is enabled.
bq. This frees up RPC handler threads for the duration of the sync.
bq. 3. Pass the RPC server instance all the way to down to HLog. This is
needed
bq. to find out the current remote call, mark it as delayed, and finally
complete
bq. it when the sync is done.
bq. 4. Use the region read-write consistency control to avoid exposing to
bq. RegionScanners edits which have not yet been synced.
bq. 5. Change a few tests which directly create HRegions or HLogs. The
bq. rpcServers passed in are null, HLog falls back to classic RPCs when it has
no
bq. knowledge of the RPC server.
bq. 6. Add TestBatchEntriesLogRolling, which is identical to TestLogRolling,
bq. except that it uses aggressive batching. I'm not sure how to add tests
that
bq. verify the same functionality but don't duplicate code, suggestion are
bq. welcome.
bq.
bq. The new parameter is disabled by default.
bq.
bq.
bq. This addresses bug HBASE-4192.
bq. https://issues.apache.org/jira/browse/HBASE-4192
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java 7117bce
bq. src/main/java/org/apache/hadoop/hbase/master/HMaster.java a00b93d
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 83ff7b2
bq. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
7a917da
bq.
src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java
8ec53d3
bq. src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 887f736
bq. src/main/resources/hbase-default.xml 66548ca
bq.
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
dc43eb2
bq.
src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
381ac90
bq.
bq. Diff: https://reviews.apache.org/r/1463/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. All unit tests run with aggressive batching turned on and off.
bq.
bq.
bq. Thanks,
bq.
bq. Vlad
bq.
bq.
> Optimize HLog for Throughput Using Delayed RPCs
> -----------------------------------------------
>
> Key: HBASE-4192
> URL: https://issues.apache.org/jira/browse/HBASE-4192
> Project: HBase
> Issue Type: New Feature
> Components: wal
> Affects Versions: 0.92.0
> Reporter: Vlad Dogaru
> Priority: Minor
>
> Introduce a new HLog configuration parameter (batchEntries) for more
> aggressive batching of appends. If this is enabled, HLog appends are not
> written to the HLog writer immediately, but batched and written either
> periodically or when a sync is requested. Because sync times become larger,
> they use delayed RPCs to free up RPC handler threads.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
