Hi Steve, I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751> for documenting msync API. Would appreciate your suggestions.
Stay safe, --Konstantin On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <ste...@cloudera.com> wrote: > > > On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.had...@gmail.com> > wrote: > >> Hey Steve, >> >> Thanks for the references. I was reading but still need to understand how >> exactly this applies to msync. >> > > mainly: pull it up and it becomes part of the broader API, so needs to be > specified in a way which can be understood by users and for implementors of > others stores: to give their own stores the same semantics. > > What does the HDFS one do? > > > >> Will come up with a plan and post it on a new jira. >> Will make sure to create it under HADOOP and ping Hadoop Common list for >> visibility. >> >> > thanks > > >> You are right about ViewFS. The impl should make sure it calls msync() on >> all mount points that enabled observer reads. >> >> > That's the kind of issue this process aims to resolve. Another is to > identify where we have HDFS-layer "quirks" and at least document them (e.g. > how hdfs streams are thread safe, rename isn't Posix, ...) and list what we > know breaks if you don't re-implement >
