[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
runzhiwang updated HDDS-3240: ----------------------------- Summary: Improve write efficiency by creating container in parallel. (was: Improve write efficiency by creating container parallel.) > Improve write efficiency by creating container in parallel. > ----------------------------------------------------------- > > Key: HDDS-3240 > URL: https://issues.apache.org/jira/browse/HDDS-3240 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Reporter: runzhiwang > Assignee: runzhiwang > Priority: Major > Attachments: screenshot-1.png > > > 1. As the image shows, when stage is > DispatcherContext.WriteChunkStage.WRITE_DATA, the leader datanode execute two > steps: KeyValueHandleHandler.handleCreateContainer, > keyValueHandler.handleWriteChunk, after leader finish the two steps, the two > follower datanodes execute two steps: > KeyValueHandleHandler.handleCreateContainer, > keyValueHandler.handleWriteChunk. > 2. The problem is KeyValueHandleHandler.handleCreateContainer cost about > 300ms, so the leader and two followers cost 600ms to createContainer. The > total cost of the whole write is about 1000ms, so it's a waste for leader and > follower createContainer in sequential. > 3. Besides, when createContainer, RocksDB.open cost about 200ms. > So I will try to: > A. leader datanode and 2 follower datanodes create container in parallel, not > in sequential. > B. optimize RocksDB.open. > !screenshot-1.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org