Re: [Crash-utility] [PATCH 0/4] speed up handling of dumps with many tasks

[Greg Thelen]

On Sun, Apr 15, 2018 at 1:27 AM Dave Anderson  wrote:

> - Original Message -
> > On Tue, Apr 10, 2018 at 8:20 AM Dave Anderson 
wrote:
> >
> > > - Original Message -
> > > > This series decreases crash startup and 'ps' processing time when
handling dumps
> > > > with many tasks.  Prior to the series a 1M task dump took 45m to
load and 45m
> > > > more to run ps.  Once patched, startup+ps time drops below 40
seconds.
> >
> > > Thanks Greg -- the patch is queued for crash-7.2.2:
> >
> >
> >
https://github.com/crash-utility/crash/commit/c1a8d0c968a256fc4617524aedb59155977907ec
> >
> > Thanks Dave.  I tested the c1a8d0 github patch and the performance is
good.
> >
> > It might be too late to fix the commit log, but the patch contains all
4 of
> > my original patches, so it reduces startup+ps time 90m=>40s (rather than
> > 45m=>40s).   Not a big deal though.

> Yeah, since it looks like it would require a "force push" with an amended
> message (which github strongly discourages because it screws up people
who've
> cloned the repo), I'm not going to do that.

> But for what it's worth, I will change it in the changelog that gets
> mailed out in the crash-7.2.2 ANNOUNCE message on this mailing list,
> and it will also will be seen in the updated version of
> http://people.redhat.com/anderson/crash.changelog.html.

Makes sense.  Thanks.

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH 0/4] speed up handling of dumps with many tasks

[Dave Anderson]

- Original Message -
> On Tue, Apr 10, 2018 at 8:20 AM Dave Anderson  wrote:
> 
> > - Original Message -
> > > This series decreases crash startup and 'ps' processing time when 
> > > handling dumps
> > > with many tasks.  Prior to the series a 1M task dump took 45m to load and 
> > > 45m
> > > more to run ps.  Once patched, startup+ps time drops below 40 seconds.
> 
> > Thanks Greg -- the patch is queued for crash-7.2.2:
> 
> 
> https://github.com/crash-utility/crash/commit/c1a8d0c968a256fc4617524aedb59155977907ec
> 
> Thanks Dave.  I tested the c1a8d0 github patch and the performance is good.
> 
> It might be too late to fix the commit log, but the patch contains all 4 of
> my original patches, so it reduces startup+ps time 90m=>40s (rather than
> 45m=>40s).   Not a big deal though.

Yeah, since it looks like it would require a "force push" with an amended
message (which github strongly discourages because it screws up people who've
cloned the repo), I'm not going to do that.  

But for what it's worth, I will change it in the changelog that gets
mailed out in the crash-7.2.2 ANNOUNCE message on this mailing list,
and it will also will be seen in the updated version of 
http://people.redhat.com/anderson/crash.changelog.html.

Dave

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH 0/4] speed up handling of dumps with many tasks

[Greg Thelen]

On Tue, Apr 10, 2018 at 8:20 AM Dave Anderson  wrote:

> - Original Message -
> > This series decreases crash startup and 'ps' processing time when
handling dumps
> > with many tasks.  Prior to the series a 1M task dump took 45m to load
and 45m
> > more to run ps.  Once patched, startup+ps time drops below 40 seconds.

> Thanks Greg -- the patch is queued for crash-7.2.2:


https://github.com/crash-utility/crash/commit/c1a8d0c968a256fc4617524aedb59155977907ec

Thanks Dave.  I tested the c1a8d0 github patch and the performance is good.

It might be too late to fix the commit log, but the patch contains all 4 of
my original patches, so it reduces startup+ps time 90m=>40s (rather than
45m=>40s).   Not a big deal though.

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH 0/4] speed up handling of dumps with many tasks

[Dave Anderson]

- Original Message -
> This series decreases crash startup and 'ps' processing time when handling 
> dumps
> with many tasks.  Prior to the series a 1M task dump took 45m to load and 45m
> more to run ps.  Once patched, startup+ps time drops below 40 seconds.

Thanks Greg -- the patch is queued for crash-7.2.2:

  
https://github.com/crash-utility/crash/commit/c1a8d0c968a256fc4617524aedb59155977907ec

Dave


> 
> Greg Thelen (4):
>   refactor store_context => add_context
>   refactor task_to_pid
>   remove unreachable (and slow) code
>   index task_context by task
> 
>  defs.h |   2 +
>  task.c | 191 +++--
>  2 files changed, 92 insertions(+), 101 deletions(-)
> 
> --
> 2.17.0.484.g0c8726318c-goog
> 
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
> 

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Re: [Crash-utility] [PATCH 0/4] speed up handling of dumps with many tasks

[Dave Anderson]

- Original Message -
> This series decreases crash startup and 'ps' processing time when handling 
> dumps
> with many tasks.  Prior to the series a 1M task dump took 45m to load and 45m
> more to run ps.  Once patched, startup+ps time drops below 40 seconds.

Hi Greg,

This sounds really good, although I have to say I've never encountered a 
dumpfile
that had such a large task count.  I will review and test the patch-set next 
week.

BTW, did you test this on a live system that has such a large task count? 
Like for example, as I recall, the task_exists() checks that you removed are
checking for tasks that were found when scanning the lists, but gone by the
time it got around to storing them. 

Thanks,
  Dave

> 
> Greg Thelen (4):
>   refactor store_context => add_context
>   refactor task_to_pid
>   remove unreachable (and slow) code
>   index task_context by task
> 
>  defs.h |   2 +
>  task.c | 191 +++--
>  2 files changed, 92 insertions(+), 101 deletions(-)
> 
> --
> 2.17.0.484.g0c8726318c-goog
> 
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
> 

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility