Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Mon, May 14, 2018 at 02:24:38PM +0100, Andrew Price wrote:
> We can still keep this minimal if that's better for the Debian package.
> 
> The issue is due to the log header checksum (lh_hash) being calculated using
> sizeof(struct gfs2_log_header) and that size has increased in the latest
> kernel gfs2_ondisk.h.
> 
> For the purposes of the package we can fix that by replacing those sizeof's
> with the correct value, as below.

Thanks again, mount is working now.  I will test some more and release
a new package if all is good.

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 13/05/18 10:13, Valentin Vidic wrote:

On Wed, May 09, 2018 at 05:53:19PM +0100, Andrew Price wrote:

You should be able to fix this by using the configure.ac and
gfs2/libgfs2/meta.c files from master. The diff should be very small. It'll
need an ./autogen.sh to rebuild the configure script.

The downside of that approach is that the new fields will be recognised but
not used in the utils. The next release of gfs2-utils will add that
functionality.


As I feared, the gfs2 kernel module does not want to mount the
filesystem created with patched gfs2-utils. I guess that I will
need to take the current master branch to get it working again?


We can still keep this minimal if that's better for the Debian package.

The issue is due to the log header checksum (lh_hash) being calculated 
using sizeof(struct gfs2_log_header) and that size has increased in the 
latest kernel gfs2_ondisk.h.


For the purposes of the package we can fix that by replacing those 
sizeof's with the correct value, as below.


Andy

diff --git a/gfs2/libgfs2/recovery.c b/gfs2/libgfs2/recovery.c
index cad723f0..7326574f 100644
--- a/gfs2/libgfs2/recovery.c
+++ b/gfs2/libgfs2/recovery.c
@@ -66,7 +66,7 @@ int get_log_header(struct gfs2_inode *ip, unsigned int 
blk,

tmp = (struct gfs2_log_header *)bh->b_data;
saved_hash = tmp->lh_hash;
tmp->lh_hash = 0;
-   hash = gfs2_disk_hash(bh->b_data, sizeof(struct gfs2_log_header));
+   hash = gfs2_disk_hash(bh->b_data, 48);
tmp->lh_hash = saved_hash;
gfs2_log_header_in(, bh);
brelse(bh);
@@ -234,7 +234,7 @@ int clean_journal(struct gfs2_inode *ip, struct 
gfs2_log_header *head)

lh->lh_sequence = cpu_to_be64(head->lh_sequence + 1);
lh->lh_flags = cpu_to_be32(GFS2_LOG_HEAD_UNMOUNT);
lh->lh_blkno = cpu_to_be32(lblock);
-   hash = gfs2_disk_hash((const char *)lh, sizeof(struct 
gfs2_log_header));

+   hash = gfs2_disk_hash((const char *)lh, 48);
lh->lh_hash = cpu_to_be32(hash);
bmodified(bh);
brelse(bh);
diff --git a/gfs2/libgfs2/structures.c b/gfs2/libgfs2/structures.c
index 1cc88b50..a8da4a0c 100644
--- a/gfs2/libgfs2/structures.c
+++ b/gfs2/libgfs2/structures.c
@@ -133,7 +133,7 @@ int lgfs2_write_journal_data(struct gfs2_inode *ip)
lh.lh_sequence = seq;
lh.lh_blkno = bh->b_blocknr - jext0;
gfs2_log_header_out(, bh);
-   hash = gfs2_disk_hash(bh->b_data, sizeof(struct 
gfs2_log_header));

+   hash = gfs2_disk_hash(bh->b_data, 48);
((struct gfs2_log_header *)bh->b_data)->lh_hash = 
cpu_to_be32(hash);


if (bwrite(bh)) {
@@ -185,7 +185,7 @@ int write_journal(struct gfs2_inode *jnl, unsigned 
bsize, unsigned int blocks)

lh.lh_sequence = seq;
lh.lh_blkno = x;
gfs2_log_header_out(, bh);
-   hash = gfs2_disk_hash(bh->b_data, sizeof(struct 
gfs2_log_header));

+   hash = gfs2_disk_hash(bh->b_data, 48);
((struct gfs2_log_header *)bh->b_data)->lh_hash = 
cpu_to_be32(hash);


bmodified(bh);

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Wed, May 09, 2018 at 05:53:19PM +0100, Andrew Price wrote:
> You should be able to fix this by using the configure.ac and
> gfs2/libgfs2/meta.c files from master. The diff should be very small. It'll
> need an ./autogen.sh to rebuild the configure script.
> 
> The downside of that approach is that the new fields will be recognised but
> not used in the utils. The next release of gfs2-utils will add that
> functionality.

As I feared, the gfs2 kernel module does not want to mount the
filesystem created with patched gfs2-utils. I guess that I will
need to take the current master branch to get it working again?

[   51.435041] gfs2: GFS2 installed
[   51.435534] gfs2: fsid=debian:test: Trying to join cluster "lock_dlm", 
"debian:test"
[   51.438526] dlm: Using TCP for communications
[   51.438945] dlm: test: joining the lockspace group...
[   51.443191] dlm: test: group event done 0 0
[   51.443555] dlm: test: dlm_recover 1
[   51.443859] dlm: test: add member 2130706433
[   51.84] dlm: test: dlm_recover_members 1 nodes
[   51.444882] dlm: test: join complete
[   51.445204] dlm: test: generation 1 slots 1 1:2130706433
[   51.445630] dlm: test: dlm_recover_directory
[   51.445975] dlm: test: dlm_recover_directory 0 in 0 new
[   51.446395] dlm: test: dlm_recover_directory 0 out 0 messages
[   51.447376] dlm: test: dlm_recover 1 generation 1 done: 0 ms
[   51.968133] gfs2: fsid=debian:test: first mounter control generation 0
[   51.968882] gfs2: fsid=debian:test: Joined cluster. Now mounting FS...
[   51.969802] gfs2: fsid=debian:test.0: journal 0 mapped with 1 extents
[   51.970365] gfs2: fsid=debian:test.0: jid=0, already locked for use
[   51.970851] gfs2: fsid=debian:test.0: jid=0: Looking at journal...
[   52.025589] gfs2: fsid=debian:test.0: fatal: filesystem consistency error
 inode = 1 18
 function = find_good_lh, file = 
/build/linux-hny3SU/linux-4.16.5/fs/gfs2/recovery.c, line = 200
[   52.027113] gfs2: fsid=debian:test.0: about to withdraw this file system
[   52.030045] gfs2: fsid=debian:test.0: telling LM to unmount
[   52.030496] dlm: test: leaving the lockspace group...
[   52.032696] dlm: test: group event done 0 0
[   52.033093] dlm: test: release_lockspace final free
[   52.033595] gfs2: fsid=debian:test.0: withdrawn
[   52.033978] CPU: 0 PID: 109 Comm: kworker/0:2 Not tainted 4.16.0-1-amd64 #1 
Debian 4.16.5-1
[   52.034619] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
1.11.1-1 04/01/2014
[   52.035270] Workqueue: gfs_recovery gfs2_recover_func [gfs2]
[   52.035710] Call Trace:
[   52.036326]  dump_stack+0x5c/0x85
[   52.037073]  gfs2_lm_withdraw+0x15f/0x170 [gfs2]
[   52.037841]  gfs2_consist_inode_i+0x35/0x40 [gfs2]
[   52.038605]  find_good_lh+0x6d/0x80 [gfs2]
[   52.039307]  gfs2_find_jhead+0x8a/0x170 [gfs2]
[   52.040036]  gfs2_recover_func+0x426/0x910 [gfs2]
[   52.040781]  ? sysfs_slab_add+0x257/0x270
[   52.041494]  ? update_load_avg+0x563/0x6c0
[   52.042182]  ? update_load_avg+0x563/0x6c0
[   52.042857]  ? set_next_entity+0x96/0x1b0
[   52.043570]  ? pick_next_task_fair+0x2fc/0x5f0
[   52.044322]  ? __switch_to+0xa2/0x450
[   52.044988]  ? process_one_work+0x17b/0x360
[   52.045675]  ? gfs2_find_jhead+0x170/0x170 [gfs2]
[   52.046388]  process_one_work+0x17b/0x360
[   52.047068]  worker_thread+0x2e/0x390
[   52.047704]  ? process_one_work+0x360/0x360
[   52.048369]  kthread+0x113/0x130
[   52.048958]  ? kthread_create_worker_on_cpu+0x70/0x70
[   52.049708]  ? kthread_create_worker_on_cpu+0x70/0x70
[   52.050428]  ret_from_fork+0x35/0x40
[   52.051059] gfs2: fsid=debian:test.0: jid=0: Failed
[   52.055033] gfs2: fsid=debian:test.0: error recovering journal 0: -5
[   52.421441] dlm: closing connection to node 2130706433

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Wed, May 09, 2018 at 05:53:19PM +0100, Andrew Price wrote:
> You should be able to fix this by using the configure.ac and
> gfs2/libgfs2/meta.c files from master. The diff should be very small. It'll
> need an ./autogen.sh to rebuild the configure script.
> 
> The downside of that approach is that the new fields will be recognised but
> not used in the utils. The next release of gfs2-utils will add that
> functionality.

Thanks for the help, the testsuite is working now with a small patch
like you suggested.  I still want to check if there are any issues
mounting a filesystem created with the tools and hopefully this can
be used until the release :)

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Fri, May 04, 2018 at 02:16:24PM +0100, Andrew Price wrote:
> Try this.

Thanks, only meta test fails now, probably size check in some other place:

## - ##
## Test results. ##
## - ##

ERROR: All 30 tests were run,
1 failed unexpectedly.

##  ##
## Summary of the failures. ##
##  ##
Failed tests:
gfs2-utils master test suite test groups:

 NUM: FILE-NAME:LINE TEST-GROUP-NAME
  KEYWORDS

  29: libgfs2.at:3   meta.c
  libgfs2

## -- ##
## Detailed failed tests. ##
## -- ##

# -*- compilation -*-
29. libgfs2.at:3: testing meta.c ...
./libgfs2.at:3: test x"$ENABLE_UNIT_TESTS" = "xyes" || exit 77
./libgfs2.at:4: check_meta
stderr:
gfs2_rgrp: rg_reserved: offset is 68, expected 48
gfs2_rgrp: size mismatch between struct 128 and fields 108
gfs2_log_header: size mismatch between struct 120 and fields 48
stdout:
Running suite(s): libgfs2
0%: Checks: 1, Failures: 1, Errors: 0
check_meta.c:9:F:Meta:test_lgfs2_meta:0: Assertion 'lgfs2_selfcheck() == 0' 
failed
./libgfs2.at:4: exit code was 1, expected 0
29. libgfs2.at:3: 29. meta.c (libgfs2.at:3): FAILED (libgfs2.at:4)

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 04/05/18 14:01, Andrew Price wrote:

On 04/05/18 13:01, Valentin Vidic wrote:

On Fri, May 04, 2018 at 12:07:01PM +0100, Andrew Price wrote:
Hm curious. I *can* reproduce the same segfaults but only if I build 
with

the 4.15 header and then rebuild with the 4.16 header *without* first
running ./configure. If I run ./configure the problem resolves itself.


Thanks, I removed all traces of 4.15 from the system, did a clean
checkout of the repo and master does not segfault anymore :)


Ok good, we're on the same page :)


I will try to fix 3.1.10 now...


I suspect you'll find the problem in gfs2/libgfs2/ondisk.h where 
gfs2_rgrp_{in,out} copies the reserved data fields. As the field is 
smaller in the new gfs2_ondisk.h, the copies could write past the end of 
the struct into subsequent fields of the containing structure (the 
rgrp_tree). We should probably use sizeof() there. Same with 
gfs2_rindex_{in,out}.


Try this.

Andy

diff --git a/gfs2/libgfs2/ondisk.c b/gfs2/libgfs2/ondisk.c
index 66de2234..bf3c6638 100644
--- a/gfs2/libgfs2/ondisk.c
+++ b/gfs2/libgfs2/ondisk.c
@@ -189,7 +189,7 @@ void gfs2_rindex_in(struct gfs2_rindex *ri, char *buf)
CPIN_64(ri, str, ri_data0);
CPIN_32(ri, str, ri_data);
CPIN_32(ri, str, ri_bitbytes);
-   CPIN_08(ri, str, ri_reserved, 64);
+   CPIN_08(ri, str, ri_reserved, sizeof(ri->ri_reserved));
 }

 void gfs2_rindex_out(const struct gfs2_rindex *ri, char *buf)
@@ -205,7 +205,7 @@ void gfs2_rindex_out(const struct gfs2_rindex *ri, 
char *buf)


CPOUT_32(ri, str, ri_bitbytes);

-   CPOUT_08(ri, str, ri_reserved, 64);
+   CPOUT_08(ri, str, ri_reserved, sizeof(ri->ri_reserved));
 }

 void gfs2_rindex_print(const struct gfs2_rindex *ri)
@@ -228,7 +228,7 @@ void gfs2_rgrp_in(struct gfs2_rgrp *rg, struct 
gfs2_buffer_head *bh)

CPIN_32(rg, str, rg_free);
CPIN_32(rg, str, rg_dinodes);

-   CPIN_08(rg, str, rg_reserved, 80);
+   CPIN_08(rg, str, rg_reserved, sizeof(rg->rg_reserved));
 }

 void gfs2_rgrp_out(const struct gfs2_rgrp *rg, char *buf)
@@ -240,7 +240,7 @@ void gfs2_rgrp_out(const struct gfs2_rgrp *rg, char 
*buf)

CPOUT_32(rg, str, rg_free);
CPOUT_32(rg, str, rg_dinodes);

-   CPOUT_08(rg, str, rg_reserved, 80);
+   CPOUT_08(rg, str, rg_reserved, sizeof(rg->rg_reserved));
 }

 void gfs2_rgrp_out_bh(const struct gfs2_rgrp *rg, struct 
gfs2_buffer_head *bh)

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 04/05/18 13:01, Valentin Vidic wrote:

On Fri, May 04, 2018 at 12:07:01PM +0100, Andrew Price wrote:

Hm curious. I *can* reproduce the same segfaults but only if I build with
the 4.15 header and then rebuild with the 4.16 header *without* first
running ./configure. If I run ./configure the problem resolves itself.


Thanks, I removed all traces of 4.15 from the system, did a clean
checkout of the repo and master does not segfault anymore :)


Ok good, we're on the same page :)


I will try to fix 3.1.10 now...


I suspect you'll find the problem in gfs2/libgfs2/ondisk.h where 
gfs2_rgrp_{in,out} copies the reserved data fields. As the field is 
smaller in the new gfs2_ondisk.h, the copies could write past the end of 
the struct into subsequent fields of the containing structure (the 
rgrp_tree). We should probably use sizeof() there. Same with 
gfs2_rindex_{in,out}.


Andy

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Fri, May 04, 2018 at 12:07:01PM +0100, Andrew Price wrote:
> Hm curious. I *can* reproduce the same segfaults but only if I build with
> the 4.15 header and then rebuild with the 4.16 header *without* first
> running ./configure. If I run ./configure the problem resolves itself.

Thanks, I removed all traces of 4.15 from the system, did a clean
checkout of the repo and master does not segfault anymore :)

I will try to fix 3.1.10 now...

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 04/05/18 11:54, Valentin Vidic wrote:

On Fri, May 04, 2018 at 11:46:08AM +0100, Andrew Price wrote:

Ok, so just to be clear, after installing the 4.16 gfs2_ondisk.h and then
doing "./configure && make clean && make" the bug still occurs?


Hm curious. I *can* reproduce the same segfaults but only if I build 
with the 4.15 header and then rebuild with the 4.16 header *without* 
first running ./configure. If I run ./configure the problem resolves itself.



That is correct, attaching 4.16 gfs2_ondisk.h so you can test.


Thanks, it matches the mainline version.

Andy

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Fri, May 04, 2018 at 11:46:08AM +0100, Andrew Price wrote:
> Ok, so just to be clear, after installing the 4.16 gfs2_ondisk.h and then
> doing "./configure && make clean && make" the bug still occurs?

That is correct, attaching 4.16 gfs2_ondisk.h so you can test.

-- 
Valentin
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
/*
 * Copyright (C) Sistina Software, Inc.  1997-2003 All rights reserved.
 * Copyright (C) 2004-2006 Red Hat, Inc.  All rights reserved.
 *
 * This copyrighted material is made available to anyone wishing to use,
 * modify, copy, or redistribute it subject to the terms and conditions
 * of the GNU General Public License v.2.
 */

#ifndef __GFS2_ONDISK_DOT_H__
#define __GFS2_ONDISK_DOT_H__

#include 

#define GFS2_MAGIC		0x01161970
#define GFS2_BASIC_BLOCK	512
#define GFS2_BASIC_BLOCK_SHIFT	9

/* Lock numbers of the LM_TYPE_NONDISK type */

#define GFS2_MOUNT_LOCK		0
#define GFS2_LIVE_LOCK		1
#define GFS2_FREEZE_LOCK	2
#define GFS2_RENAME_LOCK	3
#define GFS2_CONTROL_LOCK	4
#define GFS2_MOUNTED_LOCK	5

/* Format numbers for various metadata types */

#define GFS2_FORMAT_NONE	0
#define GFS2_FORMAT_SB		100
#define GFS2_FORMAT_RG		200
#define GFS2_FORMAT_RB		300
#define GFS2_FORMAT_DI		400
#define GFS2_FORMAT_IN		500
#define GFS2_FORMAT_LF		600
#define GFS2_FORMAT_JD		700
#define GFS2_FORMAT_LH		800
#define GFS2_FORMAT_LD		900
#define GFS2_FORMAT_LB		1000
#define GFS2_FORMAT_EA		1600
#define GFS2_FORMAT_ED		1700
#define GFS2_FORMAT_QC		1400
/* These are format numbers for entities contained in files */
#define GFS2_FORMAT_RI		1100
#define GFS2_FORMAT_DE		1200
#define GFS2_FORMAT_QU		1500
/* These are part of the superblock */
#define GFS2_FORMAT_FS		1801
#define GFS2_FORMAT_MULTI	1900

/*
 * An on-disk inode number
 */

struct gfs2_inum {
	__be64 no_formal_ino;
	__be64 no_addr;
};

/*
 * Generic metadata head structure
 * Every inplace buffer logged in the journal must start with this.
 */

#define GFS2_METATYPE_NONE	0
#define GFS2_METATYPE_SB	1
#define GFS2_METATYPE_RG	2
#define GFS2_METATYPE_RB	3
#define GFS2_METATYPE_DI	4
#define GFS2_METATYPE_IN	5
#define GFS2_METATYPE_LF	6
#define GFS2_METATYPE_JD	7
#define GFS2_METATYPE_LH	8
#define GFS2_METATYPE_LD	9
#define GFS2_METATYPE_LB	12
#define GFS2_METATYPE_EA	10
#define GFS2_METATYPE_ED	11
#define GFS2_METATYPE_QC	14

struct gfs2_meta_header {
	__be32 mh_magic;
	__be32 mh_type;
	__be64 __pad0;		/* Was generation number in gfs1 */
	__be32 mh_format;
	/* This union is to keep userspace happy */
	union {
		__be32 mh_jid;		/* Was incarnation number in gfs1 */
		__be32 __pad1;
	};
};

/*
 * super-block structure
 *
 * It's probably good if SIZEOF_SB <= GFS2_BASIC_BLOCK (512 bytes)
 *
 * Order is important, need to be able to read old superblocks to do on-disk
 * version upgrades.
 */

/* Address of superblock in GFS2 basic blocks */
#define GFS2_SB_ADDR		128

/* The lock number for the superblock (must be zero) */
#define GFS2_SB_LOCK		0

/* Requirement:  GFS2_LOCKNAME_LEN % 8 == 0
   Includes: the fencing zero at the end */
#define GFS2_LOCKNAME_LEN	64

struct gfs2_sb {
	struct gfs2_meta_header sb_header;

	__be32 sb_fs_format;
	__be32 sb_multihost_format;
	__u32  __pad0;	/* Was superblock flags in gfs1 */

	__be32 sb_bsize;
	__be32 sb_bsize_shift;
	__u32 __pad1;	/* Was journal segment size in gfs1 */

	struct gfs2_inum sb_master_dir; /* Was jindex dinode in gfs1 */
	struct gfs2_inum __pad2; /* Was rindex dinode in gfs1 */
	struct gfs2_inum sb_root_dir;

	char sb_lockproto[GFS2_LOCKNAME_LEN];
	char sb_locktable[GFS2_LOCKNAME_LEN];

	struct gfs2_inum __pad3; /* Was quota inode in gfs1 */
	struct gfs2_inum __pad4; /* Was licence inode in gfs1 */
#define GFS2_HAS_UUID 1
	__u8 sb_uuid[16]; /* The UUID, maybe 0 for backwards compat */
};

/*
 * resource index structure
 */

struct gfs2_rindex {
	__be64 ri_addr;	/* grp block disk address */
	__be32 ri_length;	/* length of rgrp header in fs blocks */
	__u32 __pad;

	__be64 ri_data0;	/* first data location */
	__be32 ri_data;	/* num of data blocks in rgrp */

	__be32 ri_bitbytes;	/* number of bytes in data bitmaps */

	__u8 ri_reserved[64];
};

/*
 * resource group header structure
 */

/* Number of blocks per byte in rgrp */
#define GFS2_NBBY		4
#define GFS2_BIT_SIZE		2
#define GFS2_BIT_MASK		0x0003

#define GFS2_BLKST_FREE		0
#define GFS2_BLKST_USED		1
#define GFS2_BLKST_UNLINKED	2
#define GFS2_BLKST_DINODE	3

#define GFS2_RGF_JOURNAL	0x0001
#define GFS2_RGF_METAONLY	0x0002
#define GFS2_RGF_DATAONLY	0x0004
#define GFS2_RGF_NOALLOC	0x0008
#define GFS2_RGF_TRIMMED	0x0010

struct gfs2_rgrp_lvb {
	__be32 rl_magic;
	__be32 rl_flags;
	__be32 rl_free;
	__be32 rl_dinodes;
	__be64 rl_igeneration;
	__be32 rl_unlinked;
	__be32 __pad;
};

struct gfs2_rgrp {
	struct gfs2_meta_header rg_header;

	__be32 rg_flags;
	__be32 rg_free;
	__be32 rg_dinodes;
	union {
		__be32 __pad;
		__be32 rg_skip; /* Distance to the next rgrp in fs blocks */
	};
	__be64 rg_igeneration;
	/* The 

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 04/05/18 11:42, Valentin Vidic wrote:

On Fri, May 04, 2018 at 11:27:35AM +0100, Andrew Price wrote:

One other thing to check is whether ./configure was re-run after
gfs2_ondisk.h was upgraded.


Yes, just restoring the previous version (4.15) of

   /usr/include/linux/gfs2_ondisk.h

fixes the tests.



Ok, so just to be clear, after installing the 4.16 gfs2_ondisk.h and 
then doing "./configure && make clean && make" the bug still occurs?


Andy

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Fri, May 04, 2018 at 11:27:35AM +0100, Andrew Price wrote:
> One other thing to check is whether ./configure was re-run after
> gfs2_ondisk.h was upgraded.

Yes, just restoring the previous version (4.15) of

  /usr/include/linux/gfs2_ondisk.h

fixes the tests.

-- 
Valentin

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

On 04/05/18 11:21, Valentin Vidic wrote:

On Fri, May 04, 2018 at 11:06:59AM +0100, Andrew Price wrote:

Can you reproduce it with the master branch? Also, what is the size of your
test file? And what architecture is this on?


Master branch seems to fail too.  Test file was 200 MB on amd64.
Also attaching testsuite.log now in case it helps.


Ok thanks, I'll look into it.

One other thing to check is whether ./configure was re-run after 
gfs2_ondisk.h was upgraded.


Andy



## - ##
## gfs2-utils master test suite. ##
## - ##

mkfs.gfs2 tests

   1: Locking protocol validation ok
   2: Resource group size validation  ok
   3: Journal size validation ok
   4: Block count validation  ok
   5: Quota change file size validation   ok
   6: Locking protocols   FAILED (mkfs.at:36)
   7: Valid block sizes 512-4096  FAILED (mkfs.at:42)
   8: Max. blocks, min. block sizeFAILED (mkfs.at:50)
   9: Max. blocks, max. block sizeFAILED (mkfs.at:55)
  10: Max. resource group sizeFAILED (mkfs.at:60)
  11: Min. resource group sizeFAILED (mkfs.at:65)
  12: Max. resource group size, min. block size   FAILED (mkfs.at:70)
  13: Max. journal size   FAILED (mkfs.at:75)
  14: Min. journal size   FAILED (mkfs.at:80)
  15: Max. quota change file size FAILED (mkfs.at:85)
  16: Min. quota change file size FAILED (mkfs.at:90)
  17: Lock table validation   FAILED (mkfs.at:98)
  18: Device i/o limits handling  FAILED (mkfs.at:114)
  19: Resource group alignmentFAILED (mkfs.at:122)
  20: Values of rg_skip   FAILED (mkfs.at:129)
  21: Values of rg_data0, rg_data, rg_bitbytesFAILED (mkfs.at:145)
  22: Small filesystems   FAILED (mkfs.at:162)

fsck.gfs2 tests

  23: Conflicting options ok
  24: Fix invalid block sizes FAILED (fsck.at:17)
  25: Fix invalid goal blocks FAILED (fsck.at:26)
  26: Fix bad resource group #0   FAILED (fsck.at:31)
  27: Fix bad resource group #1   FAILED (fsck.at:36)
  28: Fix bad rindex entry #1 FAILED (fsck.at:46)
  29: Rebuild bad journal FAILED (fsck.at:52)

gfs2_edit tests

  30: Save/restoremeta, defaults  FAILED (edit.at:7)
  31: Save/restoremeta, no compressionFAILED (edit.at:20)
  32: Save/restoremeta, min. block size   FAILED (edit.at:30)
  33: Save/restoremeta, 4 journalsFAILED (edit.at:40)
  34: Save/restoremeta, min. block size, 4 journals   FAILED (edit.at:50)

libgfs2 unit tests

  35: meta.c  FAILED (libgfs2.at:4)
  36: rgrp.c  ok

## - ##
## Test results. ##
## - ##

ERROR: All 36 tests were run,
29 failed unexpectedly.
## -- ##
## testsuite.log was created. ##
## -- ##

Please send `tests/testsuite.log' and all information you think might
help:

To: 
Subject: [gfs2-utils master] testsuite: 6 7 8 9 10 11 12 13 14 15 16 17 18 
19 20 21 22 24 25 26 27 28 29 30 31 32 33 34 35 failed

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Andrew Price]

Hi Valentin,

On 04/05/18 10:50, Valentin Vidic wrote:

On Fri, May 04, 2018 at 11:42:31AM +0200, Valentin Vidic wrote:

We are starting to see mkfs segfaults after updating
linux-libc-dev from 4.15.17-1 to 4.16.5-1 due to a
change in gfs2_ondisk.h.  Hope there is enough info
below for a fix? :)


Can you reproduce it with the master branch? Also, what is the size of 
your test file? And what architecture is this on?


Andy


Here goes another trace:

(gdb) run -p lock_nolock -b 512 test
Starting program: 
/build/gfs2-utils-7xEZHP/gfs2-utils-3.1.10/gfs2/mkfs/mkfs.gfs2 -p lock_nolock 
-b 512 test
This will destroy any data on test
Are you sure you want to proceed? [y/n] y
Adding journals: Done
Building resource groups: Done

Program received signal SIGSEGV, Segmentation fault.
0xbdc0 in brelse (bh=0x55773730) at buf.c:108
108 buf.c: No such file or directory.
(gdb) bt
#0  0xbdc0 in brelse (bh=0x55773730) at buf.c:108
#1  0x99f5 in gfs2_rgrp_relse (rgd=rgd@entry=0x557723c0) at 
rgrp.c:206
#2  0xd936 in block_alloc (sdp=sdp@entry=0x7fffe560, 
blksreq=blksreq@entry=1, state=state@entry=3, blkno=blkno@entry=0x7fffe160, 
dinode=dinode@entry=1) at fs_ops.c:197
#3  0xdf77 in lgfs2_dinode_alloc (sdp=sdp@entry=0x7fffe560, 
blksreq=blksreq@entry=1, blkno=blkno@entry=0x7fffe160) at fs_ops.c:204
#4  0x5556128f in build_master (sdp=0x7fffe560) at structures.c:28
#5  0x74fc in main (argc=, argv=) at 
main_mkfs.c:996

Re: [Cluster-devel] gfs2-utils: mkfs segfault with 4.16 kernel

[Valentin Vidic]

On Fri, May 04, 2018 at 11:42:31AM +0200, Valentin Vidic wrote:
> We are starting to see mkfs segfaults after updating
> linux-libc-dev from 4.15.17-1 to 4.16.5-1 due to a
> change in gfs2_ondisk.h.  Hope there is enough info
> below for a fix? :)

Here goes another trace:

(gdb) run -p lock_nolock -b 512 test
Starting program: 
/build/gfs2-utils-7xEZHP/gfs2-utils-3.1.10/gfs2/mkfs/mkfs.gfs2 -p lock_nolock 
-b 512 test
This will destroy any data on test
Are you sure you want to proceed? [y/n] y
Adding journals: Done 
Building resource groups: Done 

Program received signal SIGSEGV, Segmentation fault.
0xbdc0 in brelse (bh=0x55773730) at buf.c:108
108 buf.c: No such file or directory.
(gdb) bt
#0  0xbdc0 in brelse (bh=0x55773730) at buf.c:108
#1  0x99f5 in gfs2_rgrp_relse (rgd=rgd@entry=0x557723c0) at 
rgrp.c:206
#2  0xd936 in block_alloc (sdp=sdp@entry=0x7fffe560, 
blksreq=blksreq@entry=1, state=state@entry=3, blkno=blkno@entry=0x7fffe160, 
dinode=dinode@entry=1) at fs_ops.c:197
#3  0xdf77 in lgfs2_dinode_alloc (sdp=sdp@entry=0x7fffe560, 
blksreq=blksreq@entry=1, blkno=blkno@entry=0x7fffe160) at fs_ops.c:204
#4  0x5556128f in build_master (sdp=0x7fffe560) at structures.c:28
#5  0x74fc in main (argc=, argv=) at 
main_mkfs.c:996

-- 
Valentin