[
https://issues.apache.org/jira/browse/HDFS-11291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16917447#comment-16917447
]
Surendra Singh Lilhore commented on HDFS-11291:
-----------------------------------------------
This issue should be fixed... I saw some log where unnecessary edit logs are
getting log which is not required....
{code:java}
2019-08-27 10:50:07,281 INFO
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream:
Fast-forwarding stream
'http://smc-nn02.jq:8480/getJournal?jid=nameservice1&segmentTxId=4358634905&storageInfo=-64%3A1209372736%3A1547031657597%3Acluster7&inProgressOk=true,
http://smc-nn03.jq:8480/getJournal?jid=nameservice1&segmentTxId=4358634905&storageInfo=-64%3A1209372736%3A1547031657597%3Acluster7&inProgressOk=true,
http://smc-nn01.jq:8480/getJournal?jid=nameservice1&segmentTxId=4358634905&storageInfo=-64%3A1209372736%3A1547031657597%3Acluster7&inProgressOk=true'
to transaction ID 4358634905
2019-08-27 10:50:07,281 INFO
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream:
Fast-forwarding stream
'http://smc-nn02.jq:8480/getJournal?jid=nameservice1&segmentTxId=4358634905&storageInfo=-64%3A1209372736%3A1547031657597%3Acluster7&inProgressOk=true'
to transaction ID 4358634905
2019-08-27 10:50:07,651 INFO
org.apache.hadoop.hdfs.server.namenode.FSDirectory: Increasing replication from
2 to 2 for
/user/smctest/.sparkStaging/application_1561429828507_20410/__spark_libs__4604495262435387108.zip
2019-08-27 10:50:07,665 INFO
org.apache.hadoop.hdfs.server.namenode.FSDirectory: Increasing replication from
2 to 2 for
/user/smctest/.sparkStaging/application_1561429828507_20410/__spark_conf__.zip
2019-08-27 10:50:07,816 INFO
org.apache.hadoop.hdfs.server.namenode.FSDirectory: Increasing replication from
2 to 2 for
/user/smctest/.sparkStaging/application_1561429828507_20411/__spark_libs__4607202820795793784.zip
{code}
Here file replication is already 2 and it is again setting 2. This type of
audit log is not required...
> Avoid unnecessary edit log for setStoragePolicy() and setReplication()
> ----------------------------------------------------------------------
>
> Key: HDFS-11291
> URL: https://issues.apache.org/jira/browse/HDFS-11291
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
> Priority: Major
> Attachments: HDFS-11291.001.patch, HDFS-11291.002.patch
>
>
> We are setting the storage policy for file without checking the current
> policy of file for avoiding extra getStoragePolicy() rpc call. Currently
> namenode is not checking the current storage policy before setting new one
> and adding edit logs. I think if the old and new storage policy is same we
> can avoid set operation.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
