[
https://issues.apache.org/jira/browse/HDDS-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141784#comment-17141784
]
maobaolong commented on HDDS-3545:
----------------------------------
Thanks [~elek] and [~arp] for the discussion in the apache slack channel for
this tickets. Until now, we all think HDDS-2939 can make it possible, But not
sure when HDDS-2939 would be available in master branch.
The following is the reply from [~elek]
{quote}
HDDS-2939 is the umbrella of the new approach It should be possible to add some
support with the current approach. IMHO the only tricky question is that how to
make it optional. This is an additional update for each file update (update
modification time). It might not be required for all the use cases.
With current approach you can:
updating /a/b/c/d
create /a/b/c/ (directory entry) of doesn't exist
update the modification time in the same batch
It makes everything slower (if it's required all the time) but can be possible.
One way to make it optional to make it part of CreateFile and not CommitKey.
CommitKey can be the pure ObjectStore method but CreateFile should provide the
file system sematics
{quote}
So, i think until now, we cannot support MR jobhistory well, because MR
jobhistory can display the new jobs when the modification time of the parent
directory change, but now we cannot support the modification time of directory.
> MR Jobhistory cannot work well with o3fs hadoop compatible file system
> ----------------------------------------------------------------------
>
> Key: HDDS-3545
> URL: https://issues.apache.org/jira/browse/HDDS-3545
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Filesystem
> Reporter: maobaolong
> Priority: Major
> Labels: TriagePending
>
> After take a look at the code of JobHistory, i see jobhistory use
> `fs.getModificationTime()` to get the directory modification time, and use it
> as a condition of starting scan the job directories.
> But, for ozone, wile insert a child to a `directory`, the modification time
> of the `directory` don't update now.
> So we should update the modification time of `directory`, otherwise, MR
> Jobhistory and some other component which use the modification time of
> `directory` cannot work as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
