[
https://issues.apache.org/jira/browse/HDFS-16519?focusedWorklogId=760683&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760683
]
ASF GitHub Bot logged work on HDFS-16519:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Apr/22 08:13
Start Date: 22/Apr/22 08:13
Worklog Time Spent: 10m
Work Description: cndaimin commented on PR #4101:
URL: https://github.com/apache/hadoop/pull/4101#issuecomment-1106154139
@tasanuma Thanks for your review and comments! Yes, we have used the feature
in our production cluster for nearly 1 year and it looks good.
- `dfs.datanode.data.write.bandwidthPerSec`: Currently we have no limitation
on user writes, and when user write bandwidth exceeds the bottleneck of
hardware (NIC generally), the write will automatically slow down.
- `dfs.datanode.ec.reconstruction.xmits.weight`: We keep it to default value
of 0.5
- `dfs.datanode.data.transfer.bandwidthPerSec`: Same as the new added
`dfs.datanode.ec.reconstruct.read.bandwidthPerSec`, we set it to half of
available network traffic to be sure it will not affect user read/write.
Issue Time Tracking
-------------------
Worklog Id: (was: 760683)
Time Spent: 1h 50m (was: 1h 40m)
> Add throttler to EC reconstruction
> ----------------------------------
>
> Key: HDFS-16519
> URL: https://issues.apache.org/jira/browse/HDFS-16519
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, ec
> Affects Versions: 3.3.1, 3.3.2
> Reporter: daimin
> Assignee: daimin
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> HDFS already have throttlers for data transfer(replication) and balancer, the
> throttlers reduce the impact of these background procedures to user
> read/write.
> We should add a throttler to EC background reconstruction too.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
