Re: [PATCH] Btrfs: throttle delayed refs better

2015-10-14 Thread Alex Lyakas
Hi Josef, Looking at the latest Linus tree, I still see: if (actual_count > 0) { u64 runtime = ktime_to_ns(ktime_sub(ktime_get(), start)); ... avg = fs_info->avg_delayed_ref_runtime * 3 + runtime; avg = div64_u64(avg, 4); So we need to divide "runtime" by "actual_count" before acco

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-27 Thread Josef Bacik
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/27/2014 10:38 AM, 钱凯 wrote: > I'm a little confused of what "avg_delayed_ref_runtime" means. > > In __btrfs_run_delayed_refs(), "avg_delayed_ref_runtime" is set to > the runtime of all delayed refs processed in current transaction > commit. Howe

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-27 Thread 钱凯
I'm a little confused of what "avg_delayed_ref_runtime" means. In __btrfs_run_delayed_refs(), "avg_delayed_ref_runtime" is set to the runtime of all delayed refs processed in current transaction commit. However, in btrfs_should_throttle_delayed_refs(), we based on the following condition to decide

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-15 Thread Johannes Hirte
On Fri, 14 Feb 2014 14:29:35 -0500 Josef Bacik wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > > On 02/14/2014 02:25 PM, Johannes Hirte wrote: > > On Thu, 6 Feb 2014 16:19:46 -0500 Josef Bacik > > wrote: > > > >> Ok so I thought I reproduced the problem but I just reproduced a

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-14 Thread Josef Bacik
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/14/2014 02:25 PM, Johannes Hirte wrote: > On Thu, 6 Feb 2014 16:19:46 -0500 Josef Bacik > wrote: > >> Ok so I thought I reproduced the problem but I just reproduced a >> different problem. Please undo any changes you've made and >> apply th

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-14 Thread Johannes Hirte
On Thu, 6 Feb 2014 16:19:46 -0500 Josef Bacik wrote: > Ok so I thought I reproduced the problem but I just reproduced a > different problem. Please undo any changes you've made and apply > this patch and reproduce and then provide me with any debug output > that gets spit out. I'm sending this

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-06 Thread Josef Bacik
On 02/05/2014 05:57 PM, Johannes Hirte wrote: > On Wed, 5 Feb 2014 16:46:57 -0500 > Josef Bacik wrote: > >> >> On 02/05/2014 04:42 PM, Johannes Hirte wrote: >>> On Wed, 5 Feb 2014 14:36:39 -0500 >>> Josef Bacik wrote: >>> On 02/05/2014 02:30 PM, Johannes Hirte wrote: > On Wed, 5 Feb 201

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-06 Thread Josef Bacik
On 02/05/2014 05:57 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 16:46:57 -0500 Josef Bacik wrote: On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Baci

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 16:46:57 -0500 Josef Bacik wrote: > > On 02/05/2014 04:42 PM, Johannes Hirte wrote: > > On Wed, 5 Feb 2014 14:36:39 -0500 > > Josef Bacik wrote: > > > >> On 02/05/2014 02:30 PM, Johannes Hirte wrote: > >>> On Wed, 5 Feb 2014 14:00:57 -0500 > >>> Josef Bacik wrote: > >>> > >>

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Baci

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Baci

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik wrote: > > On 02/05/2014 02:30 PM, Johannes Hirte wrote: > > On Wed, 5 Feb 2014 14:00:57 -0500 > > Josef Bacik wrote: > > > >> On 02/05/2014 12:34 PM, Johannes Hirte wrote: > >>> On Wed, 5 Feb 2014 10:49:15 -0500 > >>> Josef Bacik wrote: > >>> > >>

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:00:57 -0500 Josef Bacik wrote: On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik wrote: Ok none of those make sense which makes me think it may be the ktime bits, instead of un-ap

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 12:34 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik wrote: Ok none of those make sense which makes me think it may be the ktime bits, instead of un-applying the whole patch could you just comment out the parts ktime_t start = ktime_get(); an

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Wed, 5 Feb 2014 10:49:15 -0500 Josef Bacik wrote: > Ok none of those make sense which makes me think it may be the ktime > bits, instead of un-applying the whole patch could you just comment > out the parts > > ktime_t start = ktime_get(); > > and > > if (actual_count > 0

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Josef Bacik
On 02/05/2014 03:14 AM, Johannes Hirte wrote: On Tue, 4 Feb 2014 09:12:54 -0500 Josef Bacik wrote: Hrm I was hoping that was going to be more helpful. Can you get perf record -ag and then perf report while it's at full cpu and get the first 3 or 4 things with their traces? Here it comes: #

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-05 Thread Johannes Hirte
On Tue, 4 Feb 2014 09:12:54 -0500 Josef Bacik wrote: > Hrm I was hoping that was going to be more helpful. Can you get perf > record -ag and then perf report while it's at full cpu and get the > first 3 or 4 things with their traces? Here it comes: # # captured on: Wed Feb 5 00:11:4

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-04 Thread Josef Bacik
On 02/03/2014 05:53 PM, Johannes Hirte wrote: On Mon, 3 Feb 2014 16:08:08 -0500 Josef Bacik wrote: On 02/03/2014 01:28 PM, Johannes Hirte wrote: On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Johannes Hirte
On Mon, 3 Feb 2014 16:08:08 -0500 Josef Bacik wrote: > > On 02/03/2014 01:28 PM, Johannes Hirte wrote: > > On Thu, 23 Jan 2014 13:07:52 -0500 > > Josef Bacik wrote: > > > >> On one of our gluster clusters we noticed some pretty big lag > >> spikes. This turned out to be because our transaction

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Josef Bacik
On 02/03/2014 01:28 PM, Johannes Hirte wrote: On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Johannes Hirte
On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik wrote: > On one of our gluster clusters we noticed some pretty big lag > spikes. This turned out to be because our transaction commit was > taking like 3 minutes to complete. This is because we have like 30 > gigs of metadata, so our global reserve

Re: [PATCH] Btrfs: throttle delayed refs better

2014-01-24 Thread Josef Bacik
On 01/24/2014 02:34 AM, Liu Bo wrote: On Thu, Jan 23, 2014 at 01:07:52PM -0500, Josef Bacik wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like 30

Re: [PATCH] Btrfs: throttle delayed refs better

2014-01-23 Thread Liu Bo
On Thu, Jan 23, 2014 at 01:07:52PM -0500, Josef Bacik wrote: > On one of our gluster clusters we noticed some pretty big lag spikes. This > turned out to be because our transaction commit was taking like 3 minutes to > complete. This is because we have like 30 gigs of metadata, so our global > re

[PATCH] Btrfs: throttle delayed refs better

2014-01-23 Thread Josef Bacik
On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like 30 gigs of metadata, so our global reserve would end up being the max which is like 512 mb. So our thrott