[
https://issues.apache.org/jira/browse/HDFS-15887?focusedWorklogId=574162&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-574162
]
ASF GitHub Bot logged work on HDFS-15887:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Mar/21 13:39
Start Date: 30/Mar/21 13:39
Worklog Time Spent: 10m
Work Description: jianghuazhu opened a new pull request #2839:
URL: https://github.com/apache/hadoop/pull/2839
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 574162)
Remaining Estimate: 0h
Time Spent: 10m
> Make LogRoll and TailEdits execute in parallel
> ----------------------------------------------
>
> Key: HDFS-15887
> URL: https://issues.apache.org/jira/browse/HDFS-15887
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Attachments: edit_files.jpg
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the EditLogTailer class, LogRoll and TailEdits are executed in a thread,
> and when a checkpoint occurs, it will compete with TailEdits for lock
> (FSNamesystem#cpLock).
> Usually, it takes a long time to execute checkpoint, which will cause the
> size of the generated edit log file to be relatively large.
> For example, here is an actual effect:
> The StandbyCheckpointer log is triggered as follows : edit_files.jpg
> 2021-03-11 09:18:42,513 [769071096]-INFO [Standby State
> Checkpointer:StandbyCheckpointer$CheckpointerThread@335]-Triggering
> checkpoint because there have been 5142154 txns since the last checkpoint,
> which exceeds the configured threshold 1000000
> When loading an edit log with a large amount of data, the processing time
> will be longer. We should make the edit log size as even as possible, which
> is good for the operation of the system.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
