Re: [Gluster-devel] A tool to check the integrity of the .glusterfs dir on a brick?

2014-05-26 Thread Dan Mons
I've not heard of such a tool, but speaking purely from an end-user
point of view it would be invaluable.

We don't have the option to blow our storage away and start again to
fix such problems.  Similarly we are told with newer versions of
GlusterFS to stay out of the back-end brick storage and not fiddle
(which is what we used to do to fix high level logical problems).

An "fsck" style tool for GlusterFS that fixes up back end structures
would be incredibly useful.

-Dan


Dan Mons
Unbreaker of broken things
Cutting Edge
http://cuttingedge.com.au


On 26 May 2014 20:23, Niels de Vos  wrote:
> Hi,
>
> Pranith and Xavi were discussing a bug that caused a directory symlink
> under /.glusterfs/<1st-part-gfid>/<2nd-part-gfid>/ to
> be missing. For bugs like this, it might be useful to have a tool that
> checks the integrity of the .glusterfs directory structure on a brick.
>
> Some simple things that it could check:
> - a brick has a .glusterfs directory
> - the root of the brick has a 000...001 GFID
> - a .glusterfs/<1st-part-gfid>/<2nd-part-gfid>/ should either
>   be a symlink (for a directory) or a hardlink (link-count >= 2)
> - the trusted.gfid xattr should match the 
> - any file not under .glusterfs should have a trusted.gfid and that GFID
>   is available under the .glusterfs directory
>
> By default, such a tool should only report issues, and not fix them. Not
> all issues might be fixable by checking one brick anyway.
>
> Is anyone aware of such a tool? Should we provide this?
>
> Thanks,
> Niels
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


[Gluster-devel] A tool to check the integrity of the .glusterfs dir on a brick?

2014-05-26 Thread Niels de Vos
Hi,

Pranith and Xavi were discussing a bug that caused a directory symlink 
under /.glusterfs/<1st-part-gfid>/<2nd-part-gfid>/ to 
be missing. For bugs like this, it might be useful to have a tool that 
checks the integrity of the .glusterfs directory structure on a brick.

Some simple things that it could check:
- a brick has a .glusterfs directory
- the root of the brick has a 000...001 GFID
- a .glusterfs/<1st-part-gfid>/<2nd-part-gfid>/ should either 
  be a symlink (for a directory) or a hardlink (link-count >= 2)
- the trusted.gfid xattr should match the 
- any file not under .glusterfs should have a trusted.gfid and that GFID 
  is available under the .glusterfs directory

By default, such a tool should only report issues, and not fix them. Not 
all issues might be fixable by checking one brick anyway.

Is anyone aware of such a tool? Should we provide this?

Thanks,
Niels
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


Re: [Gluster-devel] Log level readability in gluster logfile

2014-05-26 Thread Vijay Bellur

On 05/19/2014 11:08 AM, Atin Mukherjee wrote:

Hi List,

I would appreciate if you can go through the patch [1] and let me know
whether this makes sense or not.

[1] http://review.gluster.org/#/c/7790/



This change has a very good potential to break log parsers/analyzers of 
glusterfs that are already out there. For example, I think the fluentd 
plugin for glusterfs [1] might get affected if this change gets merged. 
It would be good to think more on how we can bring about this change in 
a non-disruptive manner.


-Vijay

[1] 
https://github.com/keithseahus/fluent-plugin-glusterfs/blob/master/lib/fluent/plugin/in_glusterfs_log.rb


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


Re: [Gluster-devel] regarding special treatment of ENOTSUP for setxattr

2014-05-26 Thread Pranith Kumar Karampuri
Please review http://review.gluster.com/7788 submitted to remove the filtering 
of that error.

Pranith
- Original Message -
> From: "Harshavardhana" 
> To: "Pranith Kumar Karampuri" 
> Cc: "Kaleb KEITHLEY" , "Gluster Devel" 
> 
> Sent: Friday, May 23, 2014 2:12:02 AM
> Subject: Re: [Gluster-devel] regarding special treatment of ENOTSUP for 
> setxattr
> 
> http://review.gluster.com/#/c/7823/ - the fix here
> 
> On Thu, May 22, 2014 at 1:41 PM, Harshavardhana
>  wrote:
> > Here are the important locations in the XFS tree coming from 2.6.32 branch
> >
> > STATIC int
> > xfs_set_acl(struct inode *inode, int type, struct posix_acl *acl)
> > {
> > struct xfs_inode *ip = XFS_I(inode);
> > unsigned char *ea_name;
> > int error;
> >
> > if (S_ISLNK(inode->i_mode)) > I would
> > generally think this is the issue.
> > return -EOPNOTSUPP;
> >
> > STATIC long
> > xfs_vn_fallocate(
> > struct inode*inode,
> > int mode,
> > loff_t  offset,
> > loff_t  len)
> > {
> > longerror;
> > loff_t  new_size = 0;
> > xfs_flock64_t   bf;
> > xfs_inode_t *ip = XFS_I(inode);
> > int cmd = XFS_IOC_RESVSP;
> > int attr_flags = XFS_ATTR_NOLOCK;
> >
> > if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE))
> > return -EOPNOTSUPP;
> >
> > STATIC int
> > xfs_ioc_setxflags(
> > xfs_inode_t *ip,
> > struct file *filp,
> > void__user *arg)
> > {
> > struct fsxattr  fa;
> > unsigned intflags;
> > unsigned intmask;
> > int error;
> >
> > if (copy_from_user(&flags, arg, sizeof(flags)))
> > return -EFAULT;
> >
> > if (flags & ~(FS_IMMUTABLE_FL | FS_APPEND_FL | \
> >   FS_NOATIME_FL | FS_NODUMP_FL | \
> >   FS_SYNC_FL))
> > return -EOPNOTSUPP;
> >
> > Perhaps some sort of system level acl's are being propagated by us
> > over symlinks() ? - perhaps this is the related to the same issue of
> > following symlinks?
> >
> > On Sun, May 18, 2014 at 10:48 AM, Pranith Kumar Karampuri
> >  wrote:
> >> Sent the following patch to remove the special treatment of ENOTSUP here:
> >> http://review.gluster.org/7788
> >>
> >> Pranith
> >> - Original Message -
> >>> From: "Kaleb KEITHLEY" 
> >>> To: gluster-devel@gluster.org
> >>> Sent: Tuesday, May 13, 2014 8:01:53 PM
> >>> Subject: Re: [Gluster-devel] regarding special treatment of ENOTSUP for
> >>> setxattr
> >>>
> >>> On 05/13/2014 08:00 AM, Nagaprasad Sathyanarayana wrote:
> >>> > On 05/07/2014 03:44 PM, Pranith Kumar Karampuri wrote:
> >>> >>
> >>> >> - Original Message -
> >>> >>> From: "Raghavendra Gowdappa" 
> >>> >>> To: "Pranith Kumar Karampuri" 
> >>> >>> Cc: "Vijay Bellur" , gluster-devel@gluster.org,
> >>> >>> "Anand Avati" 
> >>> >>> Sent: Wednesday, May 7, 2014 3:42:16 PM
> >>> >>> Subject: Re: [Gluster-devel] regarding special treatment of ENOTSUP
> >>> >>> for setxattr
> >>> >>>
> >>> >>> I think with "repetitive log message suppression" patch being merged,
> >>> >>> we
> >>> >>> don't really need gf_log_occasionally (except if they are logged in
> >>> >>> DEBUG or
> >>> >>> TRACE levels).
> >>> >> That definitely helps. But still, setxattr calls are not supposed to
> >>> >> fail with ENOTSUP on FS where we support gluster. If there are special
> >>> >> keys which fail with ENOTSUPP, we can conditionally log setxattr
> >>> >> failures only when the key is something new?
> >>>
> >>> I know this is about EOPNOTSUPP (a.k.a. ENOTSUPP) returned by
> >>> setxattr(2) for legitimate attrs.
> >>>
> >>> But I can't help but wondering if this isn't related to other bugs we've
> >>> had with, e.g., lgetxattr(2) called on invalid xattrs?
> >>>
> >>> E.g. see https://bugzilla.redhat.com/show_bug.cgi?id=765202. We have a
> >>> hack where xlators communicate with each other by getting (and setting?)
> >>> invalid xattrs; the posix xlator has logic to filter out  invalid
> >>> xattrs, but due to bugs this hasn't always worked perfectly.
> >>>
> >>> It would be interesting to know which xattrs are getting errors and on
> >>> which fs types.
> >>>
> >>> FWIW, in a quick perusal of a fairly recent (3.14.3) kernel, in xfs
> >>> there are only six places where EOPNOTSUPP is returned, none of them
> >>> related to xattrs. In ext[34] EOPNOTSUPP can be returned if the
> >>> user_xattr option is not enabled (enabled by default in ext4.) And in
> >>> the higher level vfs xattr code there are many places where EOPNOTSUPP
> >>> _might_ be returned, primarily only if subordinate function calls aren't
> >>> invoked which would clear the default or return a different error.
> >>>
> >>> --
> >>>
> >>> Kaleb
> >>>
> >>>
> >>>
> >>>
> >>>
> >