[
https://issues.apache.org/jira/browse/HDFS-16531?focusedWorklogId=754070&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-754070
]
ASF GitHub Bot logged work on HDFS-16531:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Apr/22 13:34
Start Date: 07/Apr/22 13:34
Worklog Time Spent: 10m
Work Description: sodonnel opened a new pull request, #4148:
URL: https://github.com/apache/hadoop/pull/4148
### Description of PR
I recently came across a NN log where about 800k setRep calls were made,
setting the replication from 3 to 3 - ie leaving it unchanged. Obviously the
application should be fixed, but we could have an optimisation for this.
When the replication is unchanged in a case like this, we log an edit
record, an audit log, and perform some quota checks etc. I believe we should
still log an audit in these sort of cases, but we can skip all the checks and
avoid writing an edit.
### How was this patch tested?
Added a new test and validated the code paths were correct around writing
and syncing edits or not with some log messages I then removed.
Issue Time Tracking
-------------------
Worklog Id: (was: 754070)
Remaining Estimate: 0h
Time Spent: 10m
> Avoid setReplication logging an edit record if old replication equals the new
> value
> -----------------------------------------------------------------------------------
>
> Key: HDFS-16531
> URL: https://issues.apache.org/jira/browse/HDFS-16531
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I recently came across a NN log where about 800k setRep calls were made,
> setting the replication from 3 to 3 - ie leaving it unchanged.
> Even in a case like this, we log an edit record, an audit log, and perform
> some quota checks etc.
> I believe it should be possible to avoid some of the work if we check for
> oldRep == newRep and jump out of the method early.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
