Uma Maheswara Rao G created HDFS-6346:
-----------------------------------------
Summary: Optimize OP_SET_XATTRS by persisting single Xattr entry
per setXattr api call
Key: HDFS-6346
URL: https://issues.apache.org/jira/browse/HDFS-6346
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Uma Maheswara Rao G
When a new xattrs set on an Inode, it may add this with OP_SET_XATTRS and let's
say [USER.name1:value1]
On a next call if we set another xattrs, then it may store along with older
existing xattrs again. It may be like [USER.name1:value1, USER.name2:value2]
So, on adding more xattrs on same Inode, that list may grow and we keep store
the entries of already existing name, value fairs.
Right now we defaulted the max Xattrs on an Inode to 32 and configured. If user
modified it to much larger value and start setting xattrs, then edits loading
may create issue like my above example.
But I didn't refer any usecase of having large number of xattrs, this is just
the assumption to consider a case. My biggest doubt is whether we will have
such real usecases to have huge xattrs on a single INode.
So, here is a thought on having OP_SET_XATTR for each setXAttr operation to be
logged, When we replay them we need to consolidate. This is some initial
thought we can think more if others also feel we need to consider this case to
handle.
Otherwise we endup storing Xattrs entries in editlog file as n(n+1)/2 where n
is number xattrs for a file/dir. This may be issue only when we have large
number configured max xattrs for inode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
