[
https://issues.apache.org/jira/browse/HDFS-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880082#comment-16880082
]
Hudson commented on HDFS-12862:
-------------------------------
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #16868 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16868/])
HDFS-12862. CacheDirective becomes invalid when NN restart or failover.
(weichiu: rev ec851e4db24fad68c1d70a981b4253c0207abc45)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCacheDirectives.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
> CacheDirective becomes invalid when NN restart or failover
> ----------------------------------------------------------
>
> Key: HDFS-12862
> URL: https://issues.apache.org/jira/browse/HDFS-12862
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: caching, hdfs
> Affects Versions: 2.7.1
> Environment:
> Reporter: Wang XL
> Assignee: Wang XL
> Priority: Major
> Labels: patch
> Attachments: HDFS-12862-branch-2.7.1.001.patch,
> HDFS-12862-trunk.002.patch, HDFS-12862-trunk.003.patch,
> HDFS-12862-trunk.004.patch, HDFS-12862.005.patch, HDFS-12862.006.patch,
> HDFS-12862.007.patch
>
>
> The logic in FSNDNCacheOp#modifyCacheDirective is not correct. when modify
> cacheDirective,the expiration in directive may be a relative expiryTime, and
> EditLog will serial a relative expiry time.
> {code:java}
> // Some comments here
> static void modifyCacheDirective(
> FSNamesystem fsn, CacheManager cacheManager, CacheDirectiveInfo
> directive,
> EnumSet<CacheFlag> flags, boolean logRetryCache) throws IOException {
> final FSPermissionChecker pc = getFsPermissionChecker(fsn);
> cacheManager.modifyDirective(directive, pc, flags);
> fsn.getEditLog().logModifyCacheDirectiveInfo(directive, logRetryCache);
> }
> {code}
> But when SBN replay the log ,it will invoke
> FSImageSerialization#readCacheDirectiveInfo as a absolute expiryTime.It will
> result in the inconsistency .
> {code:java}
> public static CacheDirectiveInfo readCacheDirectiveInfo(DataInput in)
> throws IOException {
> CacheDirectiveInfo.Builder builder =
> new CacheDirectiveInfo.Builder();
> builder.setId(readLong(in));
> int flags = in.readInt();
> if ((flags & 0x1) != 0) {
> builder.setPath(new Path(readString(in)));
> }
> if ((flags & 0x2) != 0) {
> builder.setReplication(readShort(in));
> }
> if ((flags & 0x4) != 0) {
> builder.setPool(readString(in));
> }
> if ((flags & 0x8) != 0) {
> builder.setExpiration(
> CacheDirectiveInfo.Expiration.newAbsolute(readLong(in)));
> }
> if ((flags & ~0xF) != 0) {
> throw new IOException("unknown flags set in " +
> "ModifyCacheDirectiveInfoOp: " + flags);
> }
> return builder.build();
> }
> {code}
> In other words， fsn.getEditLog().logModifyCacheDirectiveInfo(directive,
> logRetryCache) may serial a relative expiry time,But
> builder.setExpiration(CacheDirectiveInfo.Expiration.newAbsolute(readLong(in)))
> read it as a absolute expiryTime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
