Re: [Cluster-devel] [GFS2 PATCH v3 00/19] gfs2: misc recovery patch collection

2019-04-30 Thread Steven Whitehouse
Hi, On 01/05/2019 00:03, Bob Peterson wrote: Here is version 3 of the patch set I posted on 23 April. It is revised based on bugs I found testing with xfstests. This is a collection of patches I've been using to address the myriad of recovery problems I've found. I'm still finding them, so the

Re: [Cluster-devel] [GFS2 PATCH v3 09/19] gfs2: Ignore recovery attempts if gfs2 has io error or is withdrawn

2019-04-30 Thread Steven Whitehouse
Hi, On 01/05/2019 00:03, Bob Peterson wrote: This patch addresses various problems with gfs2/dlm recovery. For example, suppose a node with a bunch of gfs2 mounts suddenly reboots due to kernel panic, and dlm determines it should perform recovery. DLM does so from a pseudo-state machine

Re: [Cluster-devel] [PATCH] libgfs2: Import gfs2_ondisk.h

2019-04-09 Thread Steven Whitehouse
Hi, On 09/04/2019 13:48, Andrew Price wrote: On 09/04/2019 13:21, Steven Whitehouse wrote: Hi, On 09/04/2019 13:18, Andrew Price wrote: On 09/04/2019 13:03, Christoph Hellwig wrote: On Tue, Apr 09, 2019 at 10:41:53AM +0100, Andrew Price wrote: Give gfs2-utils its own copy

Re: [Cluster-devel] [PATCH] libgfs2: Import gfs2_ondisk.h

2019-04-09 Thread Steven Whitehouse
Hi, On 09/04/2019 13:18, Andrew Price wrote: On 09/04/2019 13:03, Christoph Hellwig wrote: On Tue, Apr 09, 2019 at 10:41:53AM +0100, Andrew Price wrote: Give gfs2-utils its own copy of gfs2_ondisk.h which uses userspace types. This allows us to always support the latest ondisk structures and

Re: [Cluster-devel] [GFS2 PATCH v3] gfs2: clean_journal improperly set sd_log_flush_head

2019-04-02 Thread Steven Whitehouse
Hi, On 28/03/2019 17:10, Bob Peterson wrote: Hi, Andreas found some problems with the previous version. Here is version 3. Ross: Can you please test this one with your scenario? Thanks. Bob Peterson --- This patch fixes regressions in 588bff95c94efc05f9e1a0b19015c9408ed7c0ef. Due to that

Re: [Cluster-devel] [PATCH 0/2] gfs2: Switch to the new mount API

2019-03-18 Thread Steven Whitehouse
Thanks for sorting this out so quickly, Steve. On 17/03/2019 17:40, Andrew Price wrote: These patches convert gfs2 and gfs2meta to use fs_context. In both cases we still use sget() instead of sget_fc() as there doesn't seem to be a clear idiomatic way to propagate the bdev currently.

Re: [Cluster-devel] [GFS2 PATCH 8/9] gfs2: Do log_flush in gfs2_ail_empty_gl even if ail list is empty

2019-02-15 Thread Steven Whitehouse
On 13/02/2019 15:21, Bob Peterson wrote: Before this patch, if gfs2_ail_empty_gl saw there was nothing on the ail list, it would return and not flush the log. The problem is that there could still be a revoke for the rgrp sitting on the sd_log_le_revoke list that's been recently taken off the

Re: [Cluster-devel] [GFS2 PATCH 5/9] gfs2: Keep transactions on ail1 list until after issuing revokes

2019-02-15 Thread Steven Whitehouse
Hi, On 13/02/2019 15:21, Bob Peterson wrote: Before this patch, function gfs2_write_revokes would call function gfs2_ail1_empty, then run the ail1 list, issuing revokes. But gfs2_ail1_empty can move transactions to the ail2 list, and thus, their revokes were never issued. This patch adds a new

Re: [Cluster-devel] [GFS2 PATCH 4/9] gfs2: Force withdraw to replay journals and wait for it to finish

2019-02-15 Thread Steven Whitehouse
Hi, On 13/02/2019 15:21, Bob Peterson wrote: When a node withdraws from a file system, it often leaves its journal in an incomplete state. This is especially true when the withdraw is caused by io errors writing to the journal. Before this patch, a withdraw would try to write a "shutdown"

Re: [Cluster-devel] [GFS2 PATCH 3/9] gfs2: Empty the ail for the glock when rgrps are invalidated

2019-02-15 Thread Steven Whitehouse
Hi, On 13/02/2019 15:21, Bob Peterson wrote: Before this patch, function rgrp_go_inval would not invalidate the ail list, which meant that there might still be buffers outstanding on the ail that had revokes still pending. If the revokes had still not been written when the glock was given to

Re: [Cluster-devel] [GFS2 PATCH 1/9] gfs2: Introduce concept of a pending withdraw

2019-02-15 Thread Steven Whitehouse
On 13/02/2019 15:21, Bob Peterson wrote: File system withdraws can be delayed when inconsistencies are discovered when we cannot withdraw immediately, for example, when critical spin_locks are held. But delaying the withdraw can cause gfs2 to ignore the error and keep running for a short

Re: [Cluster-devel] [PATCH 1/2] gfs2: Fix occasional glock use-after-free

2019-01-31 Thread Steven Whitehouse
Hi, On 31/01/2019 10:55, Ross Lagerwall wrote: Each gfs2_bufdata stores a reference to a glock but the reference count isn't incremented. This causes an occasional use-after-free of the glock. Fix by taking a reference on the glock during allocation and dropping it when freeing. Another good

Re: [Cluster-devel] [PATCH 2/2] gfs2: Fix lru_count going negative

2019-01-31 Thread Steven Whitehouse
Hi, On 31/01/2019 10:55, Ross Lagerwall wrote: Under certain conditions, lru_count may drop below zero resulting in a large amount of log spam like this: vmscan: shrink_slab: gfs2_dump_glock+0x3b0/0x630 [gfs2] \ negative objects to delete nr=-1 This happens as follows: 1) A glock is

Re: [Cluster-devel] kernel BUG at fs/gfs2/inode.h:64

2019-01-09 Thread Steven Whitehouse
Hi, On 09/01/2019 17:14, Tim Smith wrote: On Wednesday, 9 January 2019 15:35:05 GMT Andreas Gruenbacher wrote: On Wed, 9 Jan 2019 at 14:43, Mark Syms wrote: We don't yet know how the assert got triggered as we've only seen it once and in the original form it looks like it would be very hard

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Implement special writepage for ail start operations

2019-01-03 Thread Steven Whitehouse
Hi, On 02/01/2019 21:12, Bob Peterson wrote: Hi, This patch is a working prototype to fix the hangs that result from doing certain jdata I/O, primarily xfstests/269. My earlier prototypes used a special "fs_specific" wbc flag that I suspect the upstream community wouldn't like. So this

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Panic when an io error occurs writing

2018-12-19 Thread Steven Whitehouse
Hi, On 18/12/2018 16:09, Mark Syms wrote: Thanks Bob, We believe we have seen these issues from time to time in our automated testing but I suspect that they're indicating a configuration problem with the backing storage. For flexibility a proportion of our purely functional testing will use

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Panic when an io error occurs writing to the journal

2018-12-17 Thread Steven Whitehouse
Hi, On 17/12/2018 09:04, Edwin Török wrote: On 17/12/2018 13:54, Bob Peterson wrote: Hi, Before this patch, gfs2 would try to withdraw when it encountered io errors writing to its journal. That's incorrect behavior because if it can't write to the journal, it cannot write revokes for the

Re: [Cluster-devel] [PATCH 0/2] gfs2: improvements to recovery and withdraw process (v2)

2018-11-22 Thread Steven Whitehouse
Hi, On 21/11/18 18:52, Bob Peterson wrote: Hi, This is a second draft of a two-patch set to fix some of the nasty journal recovery problems I've found lately. The original post from 08 November had horribly bad and inaccurate comments, and Steve Whitehouse and Andreas Gruenbacher pointed

Re: [Cluster-devel] [PATCH 0/2] gfs2: improvements to recovery and withdraw process

2018-11-20 Thread Steven Whitehouse
Hi, On 08/11/18 20:25, Bob Peterson wrote: Hi, This is a first draft of a two-patch set to fix some of the nasty journal recovery problems I've found lately. The problems have to do with file system corruption caused when recovery replays a journal after the resource group blocks have been

Re: [Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-20 Thread Steven Whitehouse
Hi, On 19/11/18 21:26, Bob Peterson wrote: Hi, - Original Message - On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. This patch eliminates a goto in function finish_xmote so we can begin unraveling the cryptic logic with later

Re: [Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-20 Thread Steven Whitehouse
Hi, On 19/11/18 21:06, Bob Peterson wrote: Hi Steve, - Original Message - On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. Before this patch, do_xmote was called with a gh parameter, but only for promotes, not demotes. This patch

Re: [Cluster-devel] gfs2: Remove vestigial bd_ops (version 2)

2018-11-20 Thread Steven Whitehouse
supporting it. Signed-off-by: Bob Peterson Acked-by: Steven Whitehouse Steve. --- fs/gfs2/incore.h | 1 - fs/gfs2/log.c| 1 - fs/gfs2/trans.c | 8 +++- 3 files changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index 888b62cfd6d1..663759abe60d

Re: [Cluster-devel] [PATCH 02/13] GFS2: Make do_xmote determine its own gh parameter

2018-11-19 Thread Steven Whitehouse
On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. Before this patch, do_xmote was called with a gh parameter, but only for promotes, not demotes. This patch allows do_xmote to determine the gh autonomously. Signed-off-by: Bob Peterson ---

Re: [Cluster-devel] [PATCH 03/13] GFS2: Eliminate a goto in finish_xmote

2018-11-19 Thread Steven Whitehouse
On 19/11/18 13:29, Bob Peterson wrote: This is another baby step toward a better glock state machine. This patch eliminates a goto in function finish_xmote so we can begin unraveling the cryptic logic with later patches. Signed-off-by: Bob Peterson --- fs/gfs2/glock.c | 11 +-- 1

Re: [Cluster-devel] [DLM PATCH] dlm: Don't swamp the CPU with callbacks queued during recovery

2018-11-09 Thread Steven Whitehouse
Hi, On 08/11/18 19:04, Bob Peterson wrote: Hi, Before this patch, recovery would cause all callbacks to be delayed, put on a queue, and afterward they were all queued to the callback work queue. This patch does the same thing, but occasionally takes a break after 25 of them so it won't swamp

Re: [Cluster-devel] [PATCH] gfs2: Put bitmap buffers in put_super

2018-11-06 Thread Steven Whitehouse
Hi, While that looks like a good fix for now, we should look at this again in due course. Why do we have a ref to the rgrp buffers held here in the first place? Unless the buffers are pinned in the journal there should not be a ref held, otherwise they cannot respond to memory pressure,

Re: [Cluster-devel] [GFS2 PATCH 0/4] jhead lookup using bios

2018-10-17 Thread Steven Whitehouse
Hi, This all looks good to me, so now we just need lots of testing. Are you still seeing good speed ups vs the current code? Steve. On 16/10/18 05:07, Abhi Das wrote: This is my latest version of this patchset based on inputs from Andreas and Steve. We readahead the journal sequentially in

Re: [Cluster-devel] [PATCH 7/9] gfs2: Fix marking bitmaps non-full

2018-10-12 Thread Steven Whitehouse
Hi, On 12/10/18 13:06, Bob Peterson wrote: - Original Message - Hi, The series looks good I think. This one though looks like a bug fix and should probably go to -stable too? Steve. I concur. So can I add your reviewed-by before I push? Bob Peterson Yes, please do, Steve.

Re: [Cluster-devel] [PATCH 7/9] gfs2: Fix marking bitmaps non-full

2018-10-12 Thread Steven Whitehouse
Hi, On 11/10/18 20:20, Andreas Gruenbacher wrote: Reservations in gfs can span multiple gfs2_bitmaps (but they won't span multiple resource groups). When removing a reservation, we want to clear the GBF_FULL flags of all involved gfs2_bitmaps, not just that of the first bitmap.

Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock

2018-10-09 Thread Steven Whitehouse
somewhere, Steve. -Original Message- From: Tim Smith Sent: 08 October 2018 14:27 To: Steven Whitehouse Cc: Mark Syms ; cluster-devel@redhat.com; Ross Lagerwall Subject: Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock On Monday, 8 October 2018 14:13

Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock

2018-10-08 Thread Steven Whitehouse
Hi, On 08/10/18 14:10, Tim Smith wrote: On Monday, 8 October 2018 14:03:24 BST Steven Whitehouse wrote: On 08/10/18 13:59, Mark Syms wrote: That sounds entirely reasonable so long as you are absolutely sure that nothing is ever going to mess with that glock, we erred on the side of more

Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock

2018-10-08 Thread Steven Whitehouse
Original Message- From: Steven Whitehouse Sent: 08 October 2018 13:56 To: Mark Syms ; cluster-devel@redhat.com Cc: Ross Lagerwall ; Tim Smith Subject: Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock Hi, On 08/10/18 13:36, Mark Syms wrote: During

Re: [Cluster-devel] [PATCH 2/2] GFS2: Flush the GFS2 delete workqueue before stopping the kernel threads

2018-10-08 Thread Steven Whitehouse
Hi, On 08/10/18 13:36, Mark Syms wrote: From: Tim Smith Flushing the workqueue can cause operations to happen which might call gfs2_log_reserve(), or get stuck waiting for locks taken by such operations. gfs2_log_reserve() can io_schedule(). If this happens, it will never wake because the

Re: [Cluster-devel] [PATCH 1/2] GFS2: use schedule timeout in find insert glock

2018-10-08 Thread Steven Whitehouse
Hi, On 08/10/18 13:36, Mark Syms wrote: During a VM stress test we encountered a system lockup and kern.log contained kernel: [21389.462707] INFO: task python:15480 blocked for more than 120 seconds. kernel: [21389.462749] Tainted: G O 4.4.0+10 #1 kernel: [21389.462763] "echo 0 >

Re: [Cluster-devel] [PATCH 06/11] gfs2: Only use struct gfs2_rbm for bitmap manipulations

2018-10-08 Thread Steven Whitehouse
Hi, On 05/10/18 20:18, Andreas Gruenbacher wrote: GFS2 uses struct gfs2_rbm to represent a filesystem block number as a bit position within a resource group. This representation is used in the bitmap manipulation code to prevent excessive conversions between block numbers and bit positions,

Re: [Cluster-devel] [PATCH 02/11] gfs2: Move rs_{sizehint, rgd_gh} fields into the inode

2018-10-08 Thread Steven Whitehouse
Hi, On 05/10/18 20:18, Andreas Gruenbacher wrote: Move the rs_sizehint and rs_rgd_gh fields from struct gfs2_blkreserv into the inode: they are more closely related to the inode than to a particular reservation. Yes, that makes sense I think - these fields have moved around a bit during the

Re: [Cluster-devel] [PATCH 11/11] gfs2: Add local resource group locking

2018-10-08 Thread Steven Whitehouse
Hi, On 05/10/18 20:18, Andreas Gruenbacher wrote: From: Bob Peterson Prepare for treating resource group glocks as exclusive among nodes but shared among all tasks running on a node: introduce another layer of node-specific locking that the local tasks can use to coordinate their accesses.

Re: [Cluster-devel] [PATCH 10/11] gfs2: Pass resource group to rgblk_free

2018-10-08 Thread Steven Whitehouse
On 05/10/18 20:18, Andreas Gruenbacher wrote: Function rgblk_free can only deal with one resource group at a time, so pass that resource group is as a parameter. Several of the callers already have the resource group at hand, so we only need additional lookup code in a few places.

Re: [Cluster-devel] [PATCH 07/11] gfs2: Fix marking bitmaps non-full

2018-10-08 Thread Steven Whitehouse
On 05/10/18 20:18, Andreas Gruenbacher wrote: Reservations in gfs can span multiple gfs2_bitmaps (but they won't span multiple resource groups). When removing a reservation, we want to clear the GBF_FULL flags of all involved gfs2_bitmaps, not just that of the first bitmap. This looks like

Re: [Cluster-devel] [PATCH 01/11] gfs2: Always check the result of gfs2_rbm_from_block

2018-10-08 Thread Steven Whitehouse
Hi, On 05/10/18 20:18, Andreas Gruenbacher wrote: When gfs2_rbm_from_block fails, the rbm it returns is undefined, so we always want to make sure gfs2_rbm_from_block has succeeded before looking at the rbm. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/rgrp.c | 7 --- 1 file changed,

Re: [Cluster-devel] [GFS2 PATCH] gfs2: print fsid when rgrp errors are found

2018-10-04 Thread Steven Whitehouse
Hi, Presumably the other calls to pr_warn have the same issue? Perhaps we should just get rid of pr_warn completely unless there are cases where a super block is not available to pass to fs_warn, Steve. On 04/10/18 16:18, Bob Peterson wrote: Hi, This patch allows gfs2_setbit to provide

Re: [Cluster-devel] [PATCH 0/2] GFS2: inplace_reserve performance improvements

2018-09-28 Thread Steven Whitehouse
Hi, On 28/09/18 14:43, Tim Smith wrote: On Friday, 28 September 2018 14:18:59 BST Steven Whitehouse wrote: Hi, On 28/09/18 13:50, Mark Syms wrote: Hi Bon, The patches look quite good and would seem to help in the intra-node congestion case, which our first patch was trying to do. We

Re: [Cluster-devel] [PATCH 0/2] GFS2: inplace_reserve performance improvements

2018-09-28 Thread Steven Whitehouse
Hi, On 28/09/18 13:50, Mark Syms wrote: Hi Bon, The patches look quite good and would seem to help in the intra-node congestion case, which our first patch was trying to do. We haven't tried them yet but I'll pull a build together and try to run it over the weekend. We don't however, see

Re: [Cluster-devel] [GFS2 RFC PATCH 3/3] gfs2: introduce bio_pool to readahead journal to find jhead

2018-09-25 Thread Steven Whitehouse
Hi, On 25/09/18 06:38, Abhi Das wrote: This patch adds a new data structure called bio_pool. This is basically a dynamically allocated array of struct bio* and associated variables to manage this data structure. The array is used in a circular fashion until the entire array has bios that are

Re: [Cluster-devel] [PATCH 0/2] GFS2: inplace_reserve performance improvements

2018-09-20 Thread Steven Whitehouse
Hi, On 20/09/18 18:47, Mark Syms wrote: Thanks for that Bob, we've been watching with interest the changes going in upstream but at the moment we're not really in a position to take advantage of them. Due to hardware vendor support certification requirements XenServer can only very

Re: [Cluster-devel] Sending patches for GFS2

2018-09-20 Thread Steven Whitehouse
Hi, On 20/09/18 14:53, Mark Syms wrote: We have a couple of patches for GFS2 which address some performance issues we’ve observed in our testing. What branch would you like the patches to be based on top off (we have a custom patched build so they’ll need to be realigned first). Thanks,

Re: [Cluster-devel] [GFS2 PATCH 0/4] Speed up journal head lookup

2018-09-07 Thread Steven Whitehouse
Hi, On 06/09/18 18:02, Abhi Das wrote: This is the upstream version of the rhel7 patchset I'd posted earlier for review. It is slightly different in parts owing to some bits already being present and the hash/crc computation code being different due to the updated log header structure.

Re: [Cluster-devel] [PATCH 0/8] Get rid of gfs2_write_calc_reserv in gfs2_iomap_begin_write

2018-08-29 Thread Steven Whitehouse
Hi, On 28/08/18 22:47, Andreas Gruenbacher wrote: Following several cleanups, this is a first step towards being more accurate in the number of blocks we reserve in the journal. I hope this part of the code is easier to understand now as well. Thanks, Andreas Looks good to me. Lets make

Re: [Cluster-devel] [GFS2 PATCH] gfs2: don't hold the sd_jindex_spin during recovery

2018-08-20 Thread Steven Whitehouse
Hi, On 17/08/18 16:53, Bob Peterson wrote: Hi, The sd_jindex_spin is used to serialize access to the sd_jindex_list. Before this patch function gfs2_recover_set would hold the spin_lock while recovery is running. Since recovery may take a very long time, other processes needing to use the

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Don't set GFS2_RDF_UPTODATE when the lvb is updated

2018-08-20 Thread Steven Whitehouse
Acked-by: Steven Whitehouse On 17/08/18 16:51, Bob Peterson wrote: Hi, The GFS2_RDF_UPTODATE flag in the rgrp is used to determine when a rgrp buffer is valid. It's cleared when the glock is invalidated, signifying that the buffer data is now invalid. But before this patch, function

Re: [Cluster-devel] RFC: growing files more than 4k at the time while writing

2018-08-16 Thread Steven Whitehouse
if it is faster, Steve. Thanks a lot for your help? Stefano From: Steven Whitehouse Sent: Thursday, August 16, 2018 3:03 PM To: Stefano Panella; cluster-devel@redhat.com; rpete...@redhat.com; agrue...@redhat.com Cc: Edvin Torok; Tim Smith; Mark Syms; Ross

Re: [Cluster-devel] [GFS2 PATCH] gfs2: improve debug information when lvb mismatches are found

2018-08-16 Thread Steven Whitehouse
dump code dump any lvbs as well as the rgrp itself. Yes, that is a good plan. Acked-by: Steven Whitehouse Steve. Signed-off-by: Bob Peterson --- fs/gfs2/rgrp.c | 41 - fs/gfs2/util.c | 3 +++ 2 files changed, 39 insertions(+), 5 deletions(-) diff

Re: [Cluster-devel] RFC: growing files more than 4k at the time while writing

2018-08-16 Thread Steven Whitehouse
Hi, On 16/08/18 13:56, Stefano Panella wrote: Hi, I am looking at gfs2 performance when there is Virtual Machine workload and have some questions: If we have a system where every host is writing sequentially, o_direct, 512k at the time, 20 files or more each host, we can see very high CPU

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Always update the in-core rgrp when the buffer_head is read

2018-08-16 Thread Steven Whitehouse
Hi, On 15/08/18 18:04, Bob Peterson wrote: Hi, Before this patch, function gfs2_rgrp_bh_get would only update the rgrp info from the buffer if the GFS2_RDF_UPTODATE flag was clear. This went back to the days when gfs2 kept rgrp version numbers, and re-read the buffer_heads constantly, not

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Simplify iterative add loop in foreach_descriptor

2018-08-10 Thread Steven Whitehouse
Hi, On 10/08/18 13:13, Andreas Gruenbacher wrote: On 9 August 2018 at 11:35, Steven Whitehouse wrote: Hi, On 08/08/18 19:52, Bob Peterson wrote: Hi, Before this patch, function foreach_descriptor repeatedly called function gfs2_replay_incr_blk which just incremented the value while

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Simplify iterative add loop in foreach_descriptor

2018-08-09 Thread Steven Whitehouse
Hi, On 08/08/18 19:52, Bob Peterson wrote: Hi, Before this patch, function foreach_descriptor repeatedly called function gfs2_replay_incr_blk which just incremented the value while decrementing another, and checked for wrap. This is a waste of time. This patch just adds the value and adjusts

Re: [Cluster-devel] [GFS2 PATCH] gfs2: cleanup: call gfs2_rgrp_ondisk2lvb from gfs2_rgrp_out

2018-07-27 Thread Steven Whitehouse
Hi, On 26/07/18 19:15, Bob Peterson wrote: Hi, Before this patch gfs2_rgrp_ondisk2lvb was called after every call to gfs2_rgrp_out. This patch just calls it directly from within gfs2_rgrp_out, and moves the function to be before it so we don't need a function prototype. Yes, this looks like

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Return all reservations when rgrp_brelse is called

2018-07-13 Thread Steven Whitehouse
On 13/07/18 21:26, Bob Peterson wrote: Hi, Before this patch, function gfs2_rgrp_brelse would release its buffer_heads for the rgrp bitmaps, but it did not release its reservations. The problem is: When we need to call brelse, we're basically letting go of the bitmaps, which means our

Re: [Cluster-devel] [bug report] [GFS2] The core of GFS2

2018-07-11 Thread Steven Whitehouse
Hi, On 11/07/18 13:18, Dan Carpenter wrote: Hello David Teigland, The patch b3b94faa5fe5: "[GFS2] The core of GFS2" from Jan 16, 2006, leads to the following static checker warning: fs/gfs2/dir.c:1104 dir_split_leaf() warn: 'new' can also be NULL fs/gfs2/dir.c 1083

Re: [Cluster-devel] [GFS2 PATCH 00/12] Radical Reform of glock state machine

2018-07-10 Thread Steven Whitehouse
Hi, On 09/07/18 18:50, Bob Peterson wrote: GFS2 glocks go through a very complex and confusing series of steps to transition from one locking mode to another. For example, to transition from unlocked (UN) mode to Shared (SH) mode we need to "promote" the lock, and to do so requires a series of

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Eliminate bitmap clones

2018-07-03 Thread Steven Whitehouse
Hi, On 03/07/18 14:28, Bob Peterson wrote: Hi Steve, - Original Message - Do we really still need "clone bitmaps" in gfs2? If so, why? I think maybe we can get rid of them. Can someone (Steve Whitehouse perhaps?) think of a scenario in which they're still needed? If so, please

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Eliminate bitmap clones

2018-07-03 Thread Steven Whitehouse
Hi, On 02/07/18 18:58, Bob Peterson wrote: Hi, Do we really still need "clone bitmaps" in gfs2? If so, why? I think maybe we can get rid of them. Can someone (Steve Whitehouse perhaps?) think of a scenario in which they're still needed? If so, please elaborate and give an example. Regards,

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Skip taking journal recovery locks for spectators

2018-06-25 Thread Steven Whitehouse
Hi, On 24/06/18 13:49, Bob Peterson wrote: - Original Message - Bob, On 22 June 2018 at 00:05, Bob Peterson wrote: Hi, Before this patch, spectator mounts would try to acquire the dlm's control lock and mounted lock as part of its normal recovery sequence. It's not necessary

Re: [Cluster-devel] iomap preparations for GFS2 v2

2018-06-15 Thread Steven Whitehouse
Hi, On 15/06/18 09:03, Christoph Hellwig wrote: On Thu, Jun 14, 2018 at 03:04:38PM +0200, Andreas Gruenbacher wrote: I saw that you've pushed this onto the gfs2-iomap branch in your xfs repository. I've rebased the gfs2 iomap-write branch onto that; there's a trivial patch for adding a

Re: [Cluster-devel] [GFS2 PATCH] gfs2: eliminate rs_inum and reduce the size of gfs2 inodes

2018-06-14 Thread Steven Whitehouse
Hi, Yes, that seems like a good plan Acked-by: Steven Whitehouse Steve. On 13/06/18 14:58, Bob Peterson wrote: Hi, Before this patch, block reservations kept track of the inode number. At one point, that was a valid thing to do. However, since we made the reservation a part of the inode

Re: [Cluster-devel] [PATCH] gfs2: add error handling for kmem_cache_zalloc

2018-06-12 Thread Steven Whitehouse
Hi, On 12/06/18 08:24, cgxu...@gmx.com wrote: 在 2018年6月12日,上午11:38,Zhouyang Jia 写道: When kmem_cache_zalloc fails, the lack of error-handling code may cause unexpected results. This patch adds error-handling code after calling kmem_cache_zalloc. Should not fail on this case. Thanks,

Re: [Cluster-devel] [PATCH v2] MAINTAINERS: Add Andreas Gruenbacher as a maintainer for gfs2

2018-06-04 Thread Steven Whitehouse
b60179d948bb..73745884a595 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5939,8 +5939,8 @@ S:Maintained F:scripts/get_maintainer.pl GFS2 FILE SYSTEM -M: Steven Whitehouse M:Bob Peterson +M: Andreas Gruenbacher L:cluster-devel@redhat.com W:http

Re: [Cluster-devel] [PATCH] MAINTAINERS: Add Andreas Gruenbacher as a maintainer for gfs2

2018-06-04 Thread Steven Whitehouse
+++ b/MAINTAINERS @@ -5941,6 +5941,7 @@ F:scripts/get_maintainer.pl GFS2 FILE SYSTEM M:Steven Whitehouse M:Bob Peterson +M: Andreas Gruenbacher L:cluster-devel@redhat.com W:http://sources.redhat.com/cluster/ T:git git://git.kernel.org/pub/scm/linux

Re: [Cluster-devel] [GFS2 PATCH] GFS2: gfs2_free_extlen can return an extent that is too long

2018-06-04 Thread Steven Whitehouse
uns 5824 blocks past the end of the bitmap. This patch fixes the calculation so it won't exceed the last bitmap. It also adds a BUG_ON to guard against overflows in the future. Signed-off-by: Bob Peterson Acked-by: Steven Whitehouse --- fs/gfs2/incore.h | 1 + fs/gfs2/rgrp.c | 2 +- 2 fi

Re: [Cluster-devel] [GFS2 PATCH] gfs2: revert af21ca8ed50f01c5278c5ded6dad6f05e8a5d2e4

2018-06-01 Thread Steven Whitehouse
Hi, On 30/05/18 20:18, Bob Peterson wrote: Hi, Patch af21ca8ed5 is defective and needs to be reverted. The idea was to only reserve single blocks for directories, with the thought that they should be packed together. That's all well and good for creating new directories, but directories have

Re: [Cluster-devel] [GFS2 PATCH] GFS2: Add to tail, not head, of transaction

2018-06-01 Thread Steven Whitehouse
Hi, On 30/05/18 20:24, Bob Peterson wrote: Hi, Before this patch, frunction gfs2_trans_add_meta called list_add to add a buffer to a transaction, tr_buf. Later, in the before_commit functions, it traversed the list in sequential order, which meant that they were processed in a sub-optimal

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 11:10, Christoph Hellwig wrote: On Wed, May 30, 2018 at 11:02:08AM +0100, Steven Whitehouse wrote: In that case,  maybe it would be simpler to drop it for GFS2. Unless we are getting a lot of benefit from it, then we should probably just follow the generic pattern here

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 10:59, Christoph Hellwig wrote: On Wed, May 30, 2018 at 10:30:32AM +0100, Steven Whitehouse wrote: I may have missed the context here, but I thought that the boundary was a generic thing meaning "there will have to be a metadata read before more blocks can be mapped&qu

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 06:50, Darrick J. Wong wrote: On Wed, May 23, 2018 at 04:43:34PM +0200, Christoph Hellwig wrote: Just define a range of fs specific flags and use that in gfs2 instead of exposing this internal flag flobally. Signed-off-by: Christoph Hellwig Looks ok to me, but better if the

Re: [Cluster-devel] [PATCH] dlm: remove O_NONBLOCK flag in sctp_connect_to_sock

2018-05-30 Thread Steven Whitehouse
Hi, On 29/05/18 04:09, Gang He wrote: We should remove O_NONBLOCK flag when calling sock->ops->connect() in sctp_connect_to_sock() function. Why? 1. up to now, sctp socket connect() function ignores the flag argument, that means O_NONBLOCK flag does not take effect, then we should remove it to

Re: [Cluster-devel] [RFC PATCH 3/3] gfs2: read journal in large chunks to locate the head

2018-05-29 Thread Steven Whitehouse
Hi, On 29/05/18 09:05, Abhi Das wrote: Use bio(s) to read in the journal sequentially in large chunks and locate the head of the journal. This is faster in most cases when compared to the existing bisect method which operates one block at a time. Overall this looks very good. One thing that

Re: [Cluster-devel] [RFC PATCH 2/3] gfs2: add timing info for various stages of journal recovery

2018-05-29 Thread Steven Whitehouse
Hi, Looks good. Acked-by: Steven Whitehouse Steve. On 29/05/18 09:05, Abhi Das wrote: Tells you how many milliseconds each stage of journal recovery takes. Signed-off-by: Abhi Das --- fs/gfs2/ops_fstype.c | 5 + fs/gfs2/recovery.c | 20 ++-- 2 files changed, 19

Re: [Cluster-devel] [RFC PATCH 1/3] gfs2: allow map_journal_extents() to take a journal descriptor as argument

2018-05-29 Thread Steven Whitehouse
Hi, Looks good. Acked-by: Steven Whitehouse Steve. On 29/05/18 09:05, Abhi Das wrote: This function now maps the extents for the journal whose descriptor is passed in as argument. Signed-off-by: Abhi Das --- fs/gfs2/log.h| 1 + fs/gfs2/ops_fstype.c | 5 ++--- 2 files changed

Re: [Cluster-devel] [RFC PATCH 0/3] Speed up journal head lookup

2018-05-29 Thread Steven Whitehouse
Hi, On 29/05/18 09:05, Abhi Das wrote: This patchset uses bio(s) to read in the journal in large chunks to lookup the journal head. This will replace the existing bisect method of reading the journal block-by-block to find the head. These are RHEL7 patches because it was simpler for me to

Re: [Cluster-devel] [GFS2 PATCH 2/2] GFS2: Introduce rgrp sharing

2018-05-21 Thread Steven Whitehouse
Hi, On 15/05/18 19:29, Bob Peterson wrote: This patch takes advantage of the new glock holder sharing feature for rgrps. New functions rgrp_lock and rgrp_unlock use a new semaphore to gain temporary exclusive access to the rgrps. This is needed whenever a multi-block reservation is reserved,

Re: [Cluster-devel] [PATCH 0/2] Improve throughput through rgrp sharing (v2)

2018-05-09 Thread Steven Whitehouse
Hi, On 08/05/18 21:04, Bob Peterson wrote: Hi, On 18 April, I posted v1 of this patch set. The idea is to allow multiple processes on a node to share a glock that's held exclusively in order to improve performance. Sharing rgrps allows for better throughput by reducing contention. Version 1

Re: [Cluster-devel] [PATCH] dlm: fix a clerical error when set SCTP_NODELAY

2018-05-02 Thread Steven Whitehouse
On 02/05/18 03:37, Gang He wrote: There is a clerical error when turn off Nagle's algorithm in sctp_connect_to_sock() function, this results in turn off Nagle's algorithm failure. After this correction, DLM performance will be improved obviously when using SCTP procotol. Signed-off-by: Gang

Re: [Cluster-devel] vmalloc with GFP_NOFS

2018-04-24 Thread Steven Whitehouse
Hi, On 24/04/18 17:27, Michal Hocko wrote: Hi, it seems that we still have few vmalloc users who perform GFP_NOFS allocation: drivers/mtd/ubi/io.c fs/ext4/xattr.c fs/gfs2/dir.c fs/gfs2/quota.c fs/nfs/blocklayout/extent_tree.c fs/ubifs/debug.c fs/ubifs/lprops.c fs/ubifs/lpt_commit.c

Re: [Cluster-devel] About locking granularity of gfs2

2018-04-24 Thread Steven Whitehouse
Hi, On 24/04/18 04:54, Guoqing Jiang wrote: Hi Steve, Thanks for your reply. On 04/24/2018 11:03 AM, Steven Whitehouse wrote: Hi, On 24/04/18 03:52, Guoqing Jiang wrote: Hi, Since gfs2 can "allow parallel allocation from different nodes simultaneously as the locking granularity i

Re: [Cluster-devel] About locking granularity of gfs2

2018-04-23 Thread Steven Whitehouse
Hi, On 24/04/18 03:52, Guoqing Jiang wrote: Hi, Since gfs2 can "allow parallel allocation from different nodes simultaneously as the locking granularity is one lock per resource group" per section 3.2 of [1]. Could it possible to make the locking granularity also applies to R/W IO? Then,

Re: [Cluster-devel] [GFS2 PATCH 2/2] GFS2: Take advantage of new EXSH glock mode for rgrps

2018-04-19 Thread Steven Whitehouse
Hi, On 18/04/18 20:39, Bob Peterson wrote: - Original Message - Hi, On 18/04/18 17:58, Bob Peterson wrote: This patch switches rgrp locking to EXSH mode so that it's only taken when the rgrp is added to an active transaction, or when blocks are being reserved. As soon as the

Re: [Cluster-devel] [GFS2 PATCH 1/2] GFS2: Introduce EXSH (exclusively shared on one node)

2018-04-19 Thread Steven Whitehouse
Hi, On 18/04/18 20:32, Bob Peterson wrote: - Original Message - Hi, On 18/04/18 17:58, Bob Peterson wrote: This patch is a first step in rgrp sharing. It allows for a new type of glock mode called EXSH, which stands for a lock that is Exclusive to one node, but shared amongst

Re: [Cluster-devel] [GFS2 PATCH 2/2] GFS2: Take advantage of new EXSH glock mode for rgrps

2018-04-18 Thread Steven Whitehouse
Hi, On 18/04/18 17:58, Bob Peterson wrote: This patch switches rgrp locking to EXSH mode so that it's only taken when the rgrp is added to an active transaction, or when blocks are being reserved. As soon as the transaction is ended, the rgrp exclusivity is released. This allows for rgrp

Re: [Cluster-devel] [GFS2 PATCH 1/2] GFS2: Introduce EXSH (exclusively shared on one node)

2018-04-18 Thread Steven Whitehouse
Hi, On 18/04/18 17:58, Bob Peterson wrote: This patch is a first step in rgrp sharing. It allows for a new type of glock mode called EXSH, which stands for a lock that is Exclusive to one node, but shared amongst processes on that node. Like a Shared glock, multiple processes may acquire the

Re: [Cluster-devel] [PATCH] dlm: prompt the user SCTP is experimental

2018-04-09 Thread Steven Whitehouse
Hi, On 09/04/18 06:02, Gang He wrote: Hello David, If the user sets "protocol=tcp" in the configuration file /etc/dlm/dlm.conf under two-rings cluster environment, DLM kernel module will not work with the below error message, [ 43.696924] DLM installed [ 149.552039] ocfs2: Registered

Re: [Cluster-devel] WARNING: kobject bug in sysfs_warn_dup

2018-04-05 Thread Steven Whitehouse
Hi, On 05/04/18 14:34, Greg KH wrote: On Thu, Apr 05, 2018 at 10:00:04AM +0100, Steven Whitehouse wrote: Hi, On 05/04/18 09:52, Dmitry Vyukov wrote: On Thu, Apr 5, 2018 at 10:36 AM, Steven Whitehouse <swhit...@redhat.com> wrote: Hi, On 05/04/18 09:19, Dmitry Vyukov wrote: On Th

Re: [Cluster-devel] WARNING: kobject bug in sysfs_warn_dup

2018-04-05 Thread Steven Whitehouse
Hi, On 05/04/18 09:52, Dmitry Vyukov wrote: On Thu, Apr 5, 2018 at 10:36 AM, Steven Whitehouse <swhit...@redhat.com> wrote: Hi, On 05/04/18 09:19, Dmitry Vyukov wrote: On Thu, Apr 5, 2018 at 8:34 AM, Greg KH <gre...@linuxfoundation.org> wrote: On Wed, Apr 04, 2018 at 07:0

Re: [Cluster-devel] WARNING: kobject bug in sysfs_warn_dup

2018-04-05 Thread Steven Whitehouse
Hi, On 05/04/18 09:19, Dmitry Vyukov wrote: On Thu, Apr 5, 2018 at 8:34 AM, Greg KH wrote: On Wed, Apr 04, 2018 at 07:02:01PM -0700, syzbot wrote: Hello, syzbot hit the following crash on upstream commit 3e968c9f1401088abc9a19ae6ff571644d37a355 (Wed Apr 4

Re: [Cluster-devel] WARNING in account_page_dirtied

2018-04-04 Thread Steven Whitehouse
Hi, On 04/04/18 13:36, Jan Kara wrote: Hi, On Wed 04-04-18 10:24:48, Steven Whitehouse wrote: On 03/04/18 13:05, Jan Kara wrote: Hello, On Sun 01-04-18 10:01:02, syzbot wrote: syzbot hit the following crash on upstream commit 10b84daddbec72c6b440216a69de9a9605127f7a (Sat Mar 31 17:59:00

Re: [Cluster-devel] WARNING in account_page_dirtied

2018-04-04 Thread Steven Whitehouse
Hi, On 03/04/18 13:05, Jan Kara wrote: Hello, On Sun 01-04-18 10:01:02, syzbot wrote: syzbot hit the following crash on upstream commit 10b84daddbec72c6b440216a69de9a9605127f7a (Sat Mar 31 17:59:00 2018 +) Merge branch 'perf-urgent-for-linus' of

Re: [Cluster-devel] [PATCH v2 0/2] gfs2: Stop using rhashtable_walk_peek

2018-03-29 Thread Steven Whitehouse
Hi, Can we solve the problem another way, by not taking refs on the glocks when we are iterating over them for the debugfs files? I assume that is the main issue here. We didn't used to take refs since the rcu locking was enough during the walk itself. We used to only keep track of the hash

Re: [Cluster-devel] [PATCH] gfs2: Dirty source inode during rename

2018-03-05 Thread Steven Whitehouse
Acked-by: Steven Whitehouse <swhit...@redhat.com> Looks good. Thanks, Steve. On 27/02/18 11:50, Andreas Gruenbacher wrote: Mark the source inode dirty during a rename instead of just updating the underlying buffer head. Otherwise, fsync may find the inode clean and will then skip fl

Re: [Cluster-devel] a few I_DIRTY* related cleanups

2018-02-21 Thread Steven Whitehouse
On 21/02/18 15:54, Christoph Hellwig wrote: Move I_DIRTY_INODE to fs.h, and use the I_DIRTY* defines more consistently. Thanks - looks good to me: Acked-by: Steven Whitehouse <swhit...@redhat.com> Steve.

Re: [Cluster-devel] [PATCH v2 2/6] gfs2: Remove ordered write mode handling from gfs2_trans_add_data

2018-01-30 Thread Steven Whitehouse
Hi, On 29/01/18 22:18, Andreas Gruenbacher wrote: In journaled data mode, we need to add each buffer head to the current transaction. In ordered write mode, we only need to add the inode to the ordered inode list. So far, both cases are handled in gfs2_trans_add_data. This makes the code

Re: [Cluster-devel] [GFS2 PATCH] gfs2: Remove inode from ordered write list in gfs2_write_inode()

2018-01-29 Thread Steven Whitehouse
Hi, Looks good. Do you have any figures for how big a reduction in inodes on the order list this gives? Steve. On 29/01/18 05:02, Abhi Das wrote: The vfs clears the I_DIRTY inode flag before calling gfs2_write_inode() having queued any data that needed to be written to disk. This is a good

  1   2   3   4   5   6   7   8   9   10   >