Re: glibc build process slowness

2007-02-26 Thread Jakub Jelinek 
On Sun, Feb 25, 2007 at 09:18:27PM -0500, Paul Smith wrote:
 On Wed, 2007-02-21 at 20:15 +, Mark Seaborn wrote:
 
  I profiled make.  It's spending around 60% of the time in
  new_pattern_rule(), which does a linear search through the list of
  pattern rules to check for duplicate rules.  glibc generates ~2500
  rules (in sysd-rules).
 
 Holy moly!  How in the world do you get that many pattern rules?!?!  The
 point of pattern rules is that they represent an entire class of
 targets, which means you typically would have orders of magnitude fewer
 pattern rules than you have targets.

Several pattern rules for each directory in which sources should be looked up.
There are usually 30-40 directories in which files should be searched
and glibc needs a well defined order how to pick up the desired source
location for particular object file.  E.g. pick .S in preference of .s,
then .c, repeated by several different prefixes (none, rtld-, ptw-, m_),
times the number of different object file types (.o, .os, .og, .op, .oS).
So that is 5 * 4 * 3 * 30 at minimum.
...
$(objpfx)%.o: $(..)sysdeps/pthread/%.S $(before-compile); 
$(compile-command.S)
$(objpfx)%.o: $(..)sysdeps/pthread/%.s $(before-compile); 
$(compile-command.s)
$(objpfx)rtld-%.o: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)rtld-%.o: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
$(objpfx)ptw-%.o: $(..)sysdeps/pthread/%.S $(before-compile); 
$(compile-command.S)
$(objpfx)ptw-%.o: $(..)sysdeps/pthread/%.s $(before-compile); 
$(compile-command.s)
$(objpfx)m_%.o: $(..)sysdeps/pthread/s_%.S $(before-compile); 
$(compile-command.S)
$(objpfx)m_%.o: $(..)sysdeps/pthread/s_%.s $(before-compile); 
$(compile-command.s)
$(objpfx)%.o: $(..)sysdeps/pthread/%.c $(before-compile); 
$(compile-command.c)
$(objpfx)rtld-%.o: $(..)sysdeps/pthread/%.c $(before-compile);
$(compile-command.c)
$(objpfx)ptw-%.o: $(..)sysdeps/pthread/%.c $(before-compile); 
$(compile-command.c)
$(objpfx)m_%.o: $(..)sysdeps/pthread/s_%.c $(before-compile); 
$(compile-command.c)
$(objpfx)%.os: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)%.os: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
$(objpfx)rtld-%.os: $(..)sysdeps/pthread/%.S $(before-compile);   
$(compile-command.S)
$(objpfx)rtld-%.os: $(..)sysdeps/pthread/%.s $(before-compile);   
$(compile-command.s)
$(objpfx)ptw-%.os: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)ptw-%.os: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
$(objpfx)m_%.os: $(..)sysdeps/pthread/s_%.S $(before-compile);
$(compile-command.S)
$(objpfx)m_%.os: $(..)sysdeps/pthread/s_%.s $(before-compile);
$(compile-command.s)
$(objpfx)%.os: $(..)sysdeps/pthread/%.c $(before-compile);
$(compile-command.c)
$(objpfx)rtld-%.os: $(..)sysdeps/pthread/%.c $(before-compile);   
$(compile-command.c)
$(objpfx)ptw-%.os: $(..)sysdeps/pthread/%.c $(before-compile);
$(compile-command.c)
$(objpfx)m_%.os: $(..)sysdeps/pthread/s_%.c $(before-compile);
$(compile-command.c)
$(objpfx)%.op: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)%.op: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
$(objpfx)rtld-%.op: $(..)sysdeps/pthread/%.S $(before-compile);   
$(compile-command.S)
$(objpfx)rtld-%.op: $(..)sysdeps/pthread/%.s $(before-compile);   
$(compile-command.s)
$(objpfx)ptw-%.op: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)ptw-%.op: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
$(objpfx)m_%.op: $(..)sysdeps/pthread/s_%.S $(before-compile);
$(compile-command.S)
$(objpfx)m_%.op: $(..)sysdeps/pthread/s_%.s $(before-compile);
$(compile-command.s)
$(objpfx)%.op: $(..)sysdeps/pthread/%.c $(before-compile);
$(compile-command.c)
$(objpfx)rtld-%.op: $(..)sysdeps/pthread/%.c $(before-compile);   
$(compile-command.c)
$(objpfx)ptw-%.op: $(..)sysdeps/pthread/%.c $(before-compile);
$(compile-command.c)
$(objpfx)m_%.op: $(..)sysdeps/pthread/s_%.c $(before-compile);
$(compile-command.c)
$(objpfx)%.og: $(..)sysdeps/pthread/%.S $(before-compile);
$(compile-command.S)
$(objpfx)%.og: $(..)sysdeps/pthread/%.s $(before-compile);
$(compile-command.s)
...
and so on.

Jakub


___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

Re: glibc build process slowness

2007-02-25 Thread Paul Smith 
On Wed, 2007-02-21 at 20:15 +, Mark Seaborn wrote:

 I profiled make.  It's spending around 60% of the time in
 new_pattern_rule(), which does a linear search through the list of
 pattern rules to check for duplicate rules.  glibc generates ~2500
 rules (in sysd-rules).

Holy moly!  How in the world do you get that many pattern rules?!?!  The
point of pattern rules is that they represent an entire class of
targets, which means you typically would have orders of magnitude fewer
pattern rules than you have targets.

I don't have anything against making this more efficient, I'm just...
surprised.

 I was considering refactoring this properly, but the current
 new_pattern_rule() doesn't look quite right.  The comment says it
 looks for an identical rule.  The actual test is
 
   the old rule has 1 target (or multiple identical targets)
   and
   there exists a target in the new rule the same as the old rule's target

I agree that this doesn't seem correct.

I got your set of patches, and I'll take a look.  Thanks!

-- 
---
 Paul D. Smith [EMAIL PROTECTED]  Find some GNU make tips at:
 http://www.gnu.org  http://make.paulandlesley.org
 Please remain calm...I may be mad, but I am a professional. --Mad Scientist


___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make