Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-20 Thread Masahiro Yamada
Hello Tom,


  Anyway, evaluating cc-option multiple times isn't nice and its
  optimization is on my TODO list.
 
 Can we move this up on the TODO list please?  Some quick testing
 around here says that this is the big culprit on build times.  I get
 pre-Kbuild series build time to match post-Kbuild series by building
 with CONFIG_CC_OPT_CACHE_DISABLE=y.

Sure.
I will deal with this task as the first priority.

Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-20 Thread Tom Rini
On Thu, Feb 20, 2014 at 05:25:27PM +0900, Masahiro Yamada wrote:

 Hello Tom,
 
   Anyway, evaluating cc-option multiple times isn't nice and its
   optimization is on my TODO list.
  
  Can we move this up on the TODO list please?  Some quick testing
  around here says that this is the big culprit on build times.  I get
  pre-Kbuild series build time to match post-Kbuild series by building
  with CONFIG_CC_OPT_CACHE_DISABLE=y.
 
 Sure.
 I will deal with this task as the first priority.

Thanks!  This is something we can push up to the kernel community too
and I hope they'll (a) agree with and (b) be happy about.

-- 
Tom


signature.asc
Description: Digital signature
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-19 Thread Masahiro Yamada
Hello Simon,


  Thanks for your feedback.
 
  Hmm, Kbuild is 9 times slower on your computer.
  This is a big difference. (It was about 1.6 x slower on my box.)
 
  I don't know where such a difference came from.
 
 Note this is a 32-core machine.

You are passing -j1 to the command line in here,

  time 
  CROSS_COMPILE=/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm
  -linux-gnueabihf- ARCH=arm make -j1

So you are using only 1 core, right?


 
  There seem to be noticeable pauses between things happening. I'm not
  quite sure how to dig into it more. Is it possible that cc-option is
  no-longer caching the various compiler options?
 
  I guess it's possible.
 
  Is sandbox build slow as well?
  Sandbox has no cc-option.
  If cc-option is the cause of pauses, sandbox build
  should be faster.
 
 Yes actually sandbox is pretty fast.

It's interesting and I want to know how fast.
Could you post the result of Sandbox build?


Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-19 Thread Tom Rini
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/18/2014 08:57 PM, Masahiro Yamada wrote:
 Hello Tom,
 
 I notice Kbuild fails to build venice2 board because commit 
 52ef43b0529  added an empty Makefile, 
 arch/arm/cpu/armv7/tegra124/Makefile.
 
 What shall we do with this?
 
 Do you need v10 or shall I fix it in a follow-up patch?

Lets go with follow up patch.

- -- 
Tom
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTBKluAAoJENk4IS6UOR1WxNAP/0ITe8+c1IjxQjdNL2Otyh4K
y23jEXo3wypC1t1NO11b0wuhr/3eXMgmKIE3S3vuo/zgNatCiPuuer+XDaHyYW00
4IZcD/CFulMyWOFAzoZvdvInuY60QPvBZi2LEjmtlsm35CC/gR7jtPWzOTOvdPxl
Fnk0tUNa1p5s7B+lRWnVbp6Mg6UEBP55p3nO0jKwYO6dQkGXj0qvqr21eMz+QcDL
3T+pV/llPieh+gwkHgz6Ze6sOOhQpAWmUr3GmYEUiXTJFTyTzxplWNEcluAzwGU/
RjboXErzNPdr3nRUjQ6qGwUe0plR4DU39uxjd87ipxK+4kvRpT2yiDQIxQp+0Aqi
6OthFlR0UJzEg2yFYwz1oQsRLoGzPJeYhfNtBXhBU0Jv1iBEEZb83+narnhgkyQg
kXHUeoRmfctlNPqAxMVv5k6UpHTB8cchZ1btNaaxLNv+hcQ7sfecfEPflmUyf/aD
sttN43PvT26GvcnH79x8dgR/r/ShJPlTVkvW32Z8hRostrGHyL7IefEMLe9zGYC5
+BG65pz0U0d9/taYwdL+UfukYzbjiOQufGlfnoQ1JJjsN7L30rrTcMSgOc0PzJXN
oDIEpcNwyUvbM4e8MnIexVtLmdm44mOWqvSfOvVrPTMzlkDYsWFOYmsgeLKhtW2j
Q6efuCYcEvJWP1zcdP+X
=KiYY
-END PGP SIGNATURE-
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-19 Thread Masahiro Yamada
Hello Tom,


  I notice Kbuild fails to build venice2 board because commit 
  52ef43b0529  added an empty Makefile, 
  arch/arm/cpu/armv7/tegra124/Makefile.
  
  What shall we do with this?
  
  Do you need v10 or shall I fix it in a follow-up patch?
 
 Lets go with follow up patch.

I have posted it.
http://patchwork.ozlabs.org/patch/321899/



Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-19 Thread Tom Rini
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/18/2014 04:02 AM, Masahiro Yamada wrote:
 Hello Simon,
 
 
 
 I'm not sure whether to start a new thread or not, but here in
 one observations which might be useful.
 
 With current master, a 'null' build (with nothing changed) takes
 about 5s for me.
 
 time
 CROSS_COMPILE=/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm

 
- -linux-gnueabihf- ARCH=arm make -j1
 real 0m4.978s user 0m1.144s sys 0m0.360s
 
 
 With the kbuild series, it takes a lot longer:
 
 real 0m46.600s user 0m17.628s sys 0m8.664s
 
 Thanks for your feedback.
 
 Hmm, Kbuild is 9 times slower on your computer. This is a big
 difference. (It was about 1.6 x slower on my box.)
 
 I don't know where such a difference came from.
 
 There seem to be noticeable pauses between things happening. I'm
 not quite sure how to dig into it more. Is it possible that
 cc-option is no-longer caching the various compiler options?
 
 I guess it's possible.
 
 Is sandbox build slow as well? Sandbox has no cc-option. If
 cc-option is the cause of pauses, sandbox build should be faster.
 
 Anyway, evaluating cc-option multiple times isn't nice and its
 optimization is on my TODO list.

Can we move this up on the TODO list please?  Some quick testing
around here says that this is the big culprit on build times.  I get
pre-Kbuild series build time to match post-Kbuild series by building
with CONFIG_CC_OPT_CACHE_DISABLE=y.

- -- 
Tom
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTBOnSAAoJENk4IS6UOR1WlKsP/ArbW+W4MkrKoAM6WCyWrPRh
6uDCK9Hhf3HanF49ejwGFUhvI2dHNAKuuvQ2PF523OVfxzZRUPH+D8FJQ5eA7ezO
N+yIcTDpvGI4y6aR4X/H5QCwvqeL0GsUhqP+xDQ4exAkaI/Ij92i/BYuZSwyoU8w
yfJU13RRjwstXcVheERJzQRdsC7lY2O+c1Z0+o+S9L0aJ25iuo57huUPD+6V0zsg
VEgzfFUGjPWtjtfkU9jzCIMqedhQbz4Hoewav1Bn8SNq6aG6mZtysUJitWanjJrl
fCb9a1vPbkhgJLACewkpbtUDLTys+sofSl5mbULT03bGNKDK+QG443nvSvPkfh/I
6QDdk7bH/o38ZP1kvLuzatFUXK7Z66LPwOIMVlgheIbZnyVkryOfpA8stwWRhpGk
rueQjJBuhbrcfdwRswfAX8iDRYMcuXsMyk4/Kpg2KcwEC+H540pEeZZKJYL3Enf6
Mpe7koYtnhN0AFdgIK97qQ7QYSJ9DmrsDVYAro1oL4jTqDQS8y02zdW2rhlIpIFW
vqlQ861AiXdmzUPkJS5t9qAEU9JqfK7SKS53yNBPD0tocEeJOWp9veEjvoUgm3rH
LmwCGmdNzQLlyVj0w+gTeFb90L4fb/MAfwGcb7cQak6Rc7FbgX/3MkMB6cDOpYKu
AIQTrHf3stEVnjPSAHAm
=gu7X
-END PGP SIGNATURE-
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-18 Thread Masahiro Yamada
Hello Simon,


 
 I'm not sure whether to start a new thread or not, but here in one
 observations which might be useful.
 
 With current master, a 'null' build (with nothing changed) takes about
 5s for me.
 
 time 
 CROSS_COMPILE=/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm
 -linux-gnueabihf- ARCH=arm make -j1
 real 0m4.978s
 user 0m1.144s
 sys 0m0.360s
 
 
 With the kbuild series, it takes a lot longer:
 
 real 0m46.600s
 user 0m17.628s
 sys 0m8.664s

Thanks for your feedback.

Hmm, Kbuild is 9 times slower on your computer.
This is a big difference. (It was about 1.6 x slower on my box.)

I don't know where such a difference came from.

 There seem to be noticeable pauses between things happening. I'm not
 quite sure how to dig into it more. Is it possible that cc-option is
 no-longer caching the various compiler options?

I guess it's possible.

Is sandbox build slow as well?
Sandbox has no cc-option.
If cc-option is the cause of pauses, sandbox build
should be faster.

Anyway, evaluating cc-option multiple times isn't nice
and its optimization is on my TODO list.

Historically, U-Boot has included all
config.mk (arch/*/config.mk and board/*/config.mk)
every time descending into subdirectories.
That means cc-options are evaluated over and over again.
Caching cc-option is one of work arounds, but we should not
revive it.

What we should do is to include arch/*/config.mk and board/*/config.mk
only once at the top Makefile and export options.


Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-18 Thread Tom Rini
On Sat, Feb 15, 2014 at 06:30:59PM -0700, Simon Glass wrote:
 Hi Masahiro,
 
 On 6 February 2014 14:10, Tom Rini tr...@ti.com wrote:
  On Mon, Feb 03, 2014 at 12:46:30PM +0900, Masahiro Yamada wrote:
 
  Hello Simon and Tom,
 
 
  This is my analysis of Kbuild performance.
  [snip]
  Conclusion:
  The main reasons of the slow down with Kbuild are fixdep and arg-check.
  Both of them are really important features for Kbuild and Kconfig.
  - fixdep is mandatory for our better life with Kconfig.
  - arg-check is for perfect dependency tracking.
 
 
 Thanks for your detailed analysis of this. The slower build time is
 unfortunate but I think it is worth it. The fixdep thing is similar to
 the penalty from the autoconf series I did a while ago. Maybe the
 scripts can be sped up, I'm not sure.
 
 
 
What's your plan about this series?
Are we ready to switch to Kbuild, or need more review?
  
   Lets get the performance problem Simon found figured out, but then
   otherwise, yes, I think we're about ready to merge.
 
  Tom, are you satisfied with my analisys?
 
  I am, yes.  I guess if we're unhappy with the times, if we can improve
  them we can push that back to the kernel and that's a good thing.
 
 Well IMO the sooner this is merged the better so that the Makefiles
 become stable again.

Agreed.  Weather ate most of last week for me when I had hoped to clear
out some things and then grab this series.  Back at it all now..

-- 
Tom


signature.asc
Description: Digital signature
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-18 Thread Masahiro Yamada
Hello Tom,

I notice Kbuild fails to build venice2 board
because commit 52ef43b0529  added an empty Makefile,
arch/arm/cpu/armv7/tegra124/Makefile.

What shall we do with this?

Do you need v10 or shall I fix it in a follow-up patch?


Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-18 Thread Simon Glass
Hi Masahiro,

On 18 February 2014 01:02, Masahiro Yamada yamad...@jp.panasonic.com wrote:
 Hello Simon,



 I'm not sure whether to start a new thread or not, but here in one
 observations which might be useful.

 With current master, a 'null' build (with nothing changed) takes about
 5s for me.

 time 
 CROSS_COMPILE=/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm
 -linux-gnueabihf- ARCH=arm make -j1
 real 0m4.978s
 user 0m1.144s
 sys 0m0.360s


 With the kbuild series, it takes a lot longer:

 real 0m46.600s
 user 0m17.628s
 sys 0m8.664s

 Thanks for your feedback.

 Hmm, Kbuild is 9 times slower on your computer.
 This is a big difference. (It was about 1.6 x slower on my box.)

 I don't know where such a difference came from.

Note this is a 32-core machine.


 There seem to be noticeable pauses between things happening. I'm not
 quite sure how to dig into it more. Is it possible that cc-option is
 no-longer caching the various compiler options?

 I guess it's possible.

 Is sandbox build slow as well?
 Sandbox has no cc-option.
 If cc-option is the cause of pauses, sandbox build
 should be faster.

Yes actually sandbox is pretty fast.


 Anyway, evaluating cc-option multiple times isn't nice
 and its optimization is on my TODO list.

 Historically, U-Boot has included all
 config.mk (arch/*/config.mk and board/*/config.mk)
 every time descending into subdirectories.
 That means cc-options are evaluated over and over again.
 Caching cc-option is one of work arounds, but we should not
 revive it.

 What we should do is to include arch/*/config.mk and board/*/config.mk
 only once at the top Makefile and export options.

That sounds good to me. Thanks for looking at it.

Regards,
Simon
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-15 Thread Simon Glass
Hi Masahiro,

On 6 February 2014 14:10, Tom Rini tr...@ti.com wrote:
 On Mon, Feb 03, 2014 at 12:46:30PM +0900, Masahiro Yamada wrote:

 Hello Simon and Tom,


 This is my analysis of Kbuild performance.
 [snip]
 Conclusion:
 The main reasons of the slow down with Kbuild are fixdep and arg-check.
 Both of them are really important features for Kbuild and Kconfig.
 - fixdep is mandatory for our better life with Kconfig.
 - arg-check is for perfect dependency tracking.


Thanks for your detailed analysis of this. The slower build time is
unfortunate but I think it is worth it. The fixdep thing is similar to
the penalty from the autoconf series I did a while ago. Maybe the
scripts can be sped up, I'm not sure.



   What's your plan about this series?
   Are we ready to switch to Kbuild, or need more review?
 
  Lets get the performance problem Simon found figured out, but then
  otherwise, yes, I think we're about ready to merge.

 Tom, are you satisfied with my analisys?

 I am, yes.  I guess if we're unhappy with the times, if we can improve
 them we can push that back to the kernel and that's a good thing.

Well IMO the sooner this is merged the better so that the Makefiles
become stable again.

Regards,
Simon
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-15 Thread Simon Glass
Hi Masahiro,

On 15 February 2014 18:30, Simon Glass s...@chromium.org wrote:
 Hi Masahiro,

 On 6 February 2014 14:10, Tom Rini tr...@ti.com wrote:
 On Mon, Feb 03, 2014 at 12:46:30PM +0900, Masahiro Yamada wrote:

 Hello Simon and Tom,


 This is my analysis of Kbuild performance.
 [snip]
 Conclusion:
 The main reasons of the slow down with Kbuild are fixdep and arg-check.
 Both of them are really important features for Kbuild and Kconfig.
 - fixdep is mandatory for our better life with Kconfig.
 - arg-check is for perfect dependency tracking.


 Thanks for your detailed analysis of this. The slower build time is
 unfortunate but I think it is worth it. The fixdep thing is similar to
 the penalty from the autoconf series I did a while ago. Maybe the
 scripts can be sped up, I'm not sure.

I'm not sure whether to start a new thread or not, but here in one
observations which might be useful.

With current master, a 'null' build (with nothing changed) takes about
5s for me.

time 
CROSS_COMPILE=/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm
-linux-gnueabihf- ARCH=arm make -j1
real 0m4.978s
user 0m1.144s
sys 0m0.360s


With the kbuild series, it takes a lot longer:

real 0m46.600s
user 0m17.628s
sys 0m8.664s

There seem to be noticeable pauses between things happening. I'm not
quite sure how to dig into it more. Is it possible that cc-option is
no-longer caching the various compiler options?

Regards,
Simon
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-06 Thread Tom Rini
On Mon, Feb 03, 2014 at 12:46:30PM +0900, Masahiro Yamada wrote:

 Hello Simon and Tom,
 
 
 This is my analysis of Kbuild performance.
[snip]
 Conclusion:
 The main reasons of the slow down with Kbuild are fixdep and arg-check.
 Both of them are really important features for Kbuild and Kconfig.
 - fixdep is mandatory for our better life with Kconfig.
 - arg-check is for perfect dependency tracking.
 
 
 
   What's your plan about this series?
   Are we ready to switch to Kbuild, or need more review?
  
  Lets get the performance problem Simon found figured out, but then
  otherwise, yes, I think we're about ready to merge.
 
 Tom, are you satisfied with my analisys?

I am, yes.  I guess if we're unhappy with the times, if we can improve
them we can push that back to the kernel and that's a good thing.

-- 
Tom


signature.asc
Description: Digital signature
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-02 Thread Masahiro Yamada
Hello Simon and Tom,


This is my analysis of Kbuild performance.



[1] -j option is working

What I must say first is -j option is working correctly with Kbuild.

You can double-check by following the steps below.

Apply Kbuild series v8 on
commit 07e2822d158940a0e8ba45b6ab0344ffa1011a07.


First, build without -j option.

$ make mrproper
$ time make CROSS_COMPILE=arm-linux-gnueabi-  snow_config all
Configuring for snow board...
  GEN include/autoconf.mk.dep
  [snip]

real1m15.089s
user0m44.092s
sys 0m32.513s


Next, build with -j8 option.
(Run make mrproper every time because we want to be sure
that there are no generated files before build.)

$ make mrproper
$ time make -j8 CROSS_COMPILE=arm-linux-gnueabi-  snow_config all 
  [snip]

real0m17.223s
user0m50.010s
sys 0m29.038s


It is much faster with -j8 option than without -j option.
(4.3 times faster on my box.)



You will easily notice another proof that -j option is working.


If you do not add -j option, the shorten log will be displayed
in the alphabetical order of output file name:

  LD  arch/arm/cpu/built-in.o
  CC  arch/arm/cpu/armv7/cache_v7.o
  CC  arch/arm/cpu/armv7/cpu.o
  CC  arch/arm/cpu/armv7/syslib.o
  CC  arch/arm/cpu/armv7/s5p-common/cpu_info.o
  CC  arch/arm/cpu/armv7/s5p-common/timer.o
  CC  arch/arm/cpu/armv7/s5p-common/sromc.o
  CC  arch/arm/cpu/armv7/s5p-common/pwm.o
  LD  arch/arm/cpu/armv7/s5p-common/built-in.o
  LD  arch/arm/cpu/armv7/built-in.o
  AS  arch/arm/cpu/armv7/start.o
  CC  arch/arm/cpu/armv7/exynos/clock.o
  CC  arch/arm/cpu/armv7/exynos/power.o
 

On the other hand, if you add -j option, the log will be shown
in a different order:

  LD  arch/arm/cpu/built-in.o
  CC  arch/arm/cpu/armv7/cache_v7.o
  CC  arch/arm/cpu/armv7/exynos/clock.o
  CC  disk/part.o
  AS  arch/arm/lib/crt0.o
  CC  board/samsung/common/board.o
  CC  board/samsung/smdk5250/smdk5250_spl.o
  CC  common/main.o
  LD  drivers/block/built-in.o
  LD  board/samsung/common/built-in.o
  CC  arch/arm/cpu/armv7/cpu.o
  AS  arch/arm/lib/relocate.o
  CC  disk/part_dos.o
  CC  arch/arm/cpu/armv7/exynos/power.o



[2] Is Kbuild slower than the old U-Boot build system?

Yes, Kbuild is definitely slower.
(But, as far as I tested,  I don't think it is 3 times slower.)

Let's compare the build time with the conventional build system.

Checkout master branch.
(commit 07e2822d158940a0e8ba45b6ab0344ffa1011a07)

$ make mrproper
$ time make CROSS_COMPILE=arm-linux-gnueabi-  snow

real0m45.612s
user0m28.367s
sys 0m18.969s


So, Kbuild(=real 1m15.089s) is 1.6 times slower
than the old U-Boot build system.


[3] Why is Kbuild slower?

One reason is fixdep.

The helper program, fixdep, parses the source file
and all headers included from it to search all CONFIG
macro used there. This is a rather heavy task.

If you don't know the reason why fixdep is necessary,
please read the comment block of scripts/basic/fixdep.c
It is true that fixdep is meaningless for now,
but it will be a great help when switching to Kconfig.
We will get more return than we pay.
(And Kconfig series is almost ready.
I will test more and post version 1 within a couple of weeks.)


Let' check how big the impact of fixdep is.

I prepared a patch for you to disable fixdep:
http://patchwork.ozlabs.org/patch/316057/

Apply it on
commit 07e2822d158940a0e8ba45b6ab0344ffa1011a07 + Kbuild v8

And then, build.

$ make mrproper
$ time make CROSS_COMPILE=arm-linux-gnueabi-  snow_config all 

real1m2.436s
user0m38.281s
sys 0m25.871s

It is faster by 1.2 times faster without fixdep than it is with fixdep.


Another big factor is arg-check.
This excellent routine is defined in scripts/Kbuild.include.

The dependency tracking of U-Boot old build system is
absolutely unreliable.
It compares the timestamp between object files and source files,
but never checks the arguments given to the compiler.

Kbuild checks both of them to precisely detect which objects must
be re-built.

To see how heavy arg-check task is,
build with KBUILD_NOCMDDEP=1 option to disable arg-check.

$ make mrproper
$ time make CROSS_COMPILE=arm-linux-gnueabi- KBUILD_NOCMDDEP=1 snow_config all

real0m41.882s
user0m28.432s
sys 0m14.971s

See?
It is as fast as the old U-boot system(=0m45.612) .


Conclusion:
The main reasons of the slow down with Kbuild are fixdep and arg-check.
Both of them are really important features for Kbuild and Kconfig.
- fixdep is mandatory for our better life with Kconfig.
- arg-check is for perfect dependency tracking.



  What's your plan about this series?
  Are we ready to switch to Kbuild, or need more review?
 
 Lets get the performance problem Simon found figured out, but then
 otherwise, yes, I think we're about ready to merge.

Tom, are you satisfied with my analisys?

But, please hold merging Kbuild series.
I will post version 9 with a 

Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-02-01 Thread Tom Rini
On Fri, Jan 31, 2014 at 06:12:49PM +0900, Masahiro Yamada wrote:

 Hello Tom,
  
 Now, this kbuild series cleanly applies on the current u-boot/master.
 (commit 07e2822d158940a0e8ba45b6ab0344ffa1011a07)
 
 What's your plan about this series?
 Are we ready to switch to Kbuild, or need more review?

Lets get the performance problem Simon found figured out, but then
otherwise, yes, I think we're about ready to merge.

-- 
Tom


signature.asc
Description: Digital signature
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-01-31 Thread Masahiro Yamada
Hello Tom,
 
Now, this kbuild series cleanly applies on the current u-boot/master.
(commit 07e2822d158940a0e8ba45b6ab0344ffa1011a07)

What's your plan about this series?
Are we ready to switch to Kbuild, or need more review?

Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-01-31 Thread Simon Glass
Hi Masahiro,

On 29 January 2014 05:25, Masahiro Yamada yamad...@jp.panasonic.com wrote:

 We switched to Kbuild style makefiles at v2014.01-rc1 release.
 With that modification, we can write makefiles simpler.
 But it is NOT real Kbuild. We need more progress.

 As the next step, this series imports (+ adjusts) build scripts
 from Linux Kernel under scripts/ directory.
 By applying this series, we can get more advantages:
   - short log
   - perfect dependency tracking
   - preparation to the next step, Kconfig
   - other things...

Everything looks good except for me I see a significant performance
regression on this. I suspect it is just that -j is not working
properly on make.

HEAD is now at 05ed548 dts: re-write dts/Makefile more simply with Kbuild
(=05ed54) u time crosfw -b snow -w
Configuring for snow board...

real 0m27.723s
user 0m47.940s
sys 0m10.320s
(=05ed54) u co upstream/master
Note: checking out 'upstream/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 07e2822... board: nios2: Check if flash is configured
before calling early_flash_cmd_reset()
(detached:remotes/upstream/master=07e282) u time crosfw -b snow -w
Configuring for snow board...

real 0m8.818s
user 0m36.290s
sys 0m5.040s


So it is 3x slower with Kbuild for some reason. Do you see this? For
reference my commit stack is:

05ed548 (HEAD, try-masa8) dts: re-write dts/Makefile more simply with Kbuild
f23f43b kernel-doc: fix some errors
ddb3799 kernel-doc: update kernel-doc related files to Linux v3.13
13101c1 kernel-doc: move kernel-doc tools to scripts/
f1e1317 tools/env: cross-compile fw_printenv without setting HOSTCC
7da0fca kbuild: Do not generate .*.su files at the top directory
7b7bc27 board: sandburst: delete FORCEBUILD
a34156a kbuild: check clean source and generate Makefile for out-of-tree build
e575e1d kbuild: support simultaneous board configuration and make all
08fe20b kbuild: use scripts/Makefile.clean
fc90fc8 Makefile: refactor tools-all targets
1f8bd74 Makefile: Do not pass MTD_VERSION from the top Makefile
59f28d7 kbuild: refactor Makefile and spl/Makefile more
bfdce2c examples: move api/ and standalone/ entry to examples/Makefile
4e7826b kbuild: change the top Makefile to more Kbuild-ish structure
0b1d2bf Makefile: remove a cleaning target tidy
ae96eae kbuild: generate {spl, tpl}-autoconf.mk only when it is necessary
7583bbb kbuild: move include directives of board configuration files
1917011 kbuild: convert some make rules to Kbuild style
b848d52 kbuild: move some lines to more suitable place
d5503c3 kbuild: delete temporary build scripts
6353f2b kbuild: use Linux Kernel build scripts
e0a95fa kbuild: import more build scripts from Linux v3.13 tag
d4911b8 Makefile: rename scripts/Makefile.build to scripts/Makefile.build.tmp
cee0118 kbuild: add dummy obj-y to create built-in.o
2799542 kbuild: change out-of-tree build
4591455 Makefile: move some flags to examples makefiles
9ef671c Makefile: move some flags to spl/Makefile
f0d005c Makefile: move more stuff to top Makefile
9ff6912 Makefile: refactor include path settings
cf2c342 Makefile: move more flags to the top Makefile
78d5a5e kbuild: Use Kbuild.include
846ef21 kbuild: import Kbuild.include from linux v3.13 tag
a022d38 Makefile: move BFD_ROOT_DIR to tools/gdb/Makefile
4295829 Makefile: move some variable definitions to the top Makefile
63f247e Makfile: move suffix rules to Makefile.build
a2d8588 nand-spl: Use scripts/Makefile.build
2188969 examples: Use scripts/Makefile.build
a72f52f board: samsung: refactor host programs
296248d tools: convert makefiles to kbuild style
cf5e39b Makefile.host.tmp: add a new script to refactor tools
1bb87d8 .gitignore: ingore files generated by Kbuild
07e2822 (upstream/master) board: nios2: Check if flash is configured
before calling early_flash_cmd_reset()


Regards,
Simon


  Kbuild without Kconfig
  --

 First of all, to make things clearer, let me explain
 the difference between Kbuild and Kconfig.
 They are, I think, sometimes confusing.

  Kbuild - build system used for Linux Kernel.
 Some features of Kbuild are:

(a) We can describe makefiles simply.
   Just add objects to obj-y like this:
   obj-$(CONFIG_FOO) += foo.o

(b) We can describe directory descending nicely.
   Add directories with a slash to obj-y like this:
   obj-$(CONFIG_BAR) += bar/

(c) Short log like follows:
   CC  common/foo.o
   CC  common/bar.o
   LD  common/built-in.o

(d) Perfect dependency tracking
   I think this is the biggest advantage.
   To 

Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-01-30 Thread Andreas Bießmann
Dear Masahiro Yamada,

On 29.01.14 13:25, Masahiro Yamada wrote:

snip

  How to Build ?
  --
 
 We can build the same as before.
 Do board configuraton first and then run make.
 
   $ make  omap4_panda_config
   Configuring for omap4_panda board...
   $ make  CROSS_COMPILE=arm-linux-gnueabi-
   GEN include/autoconf.mk.dep
   GEN include/autoconf.mk
   CC  lib/asm-offsets.s
   GEN include/generated/generic-asm-offsets.h
   CC  arch/arm/cpu/armv7/omap4/asm-offsets.s
   GEN include/generated/asm-offsets.h
   HOSTCC  scripts/basic/fixdep
...
 
 You will find a difference at a glance: short log.
 If you need detail log message, please add V=1.
 (You can also use V=2)
 
 Please note we can no longer use
   $ make omap4_panda CROSS_COMPILE=arm-linux-gnueabi-
 to do board configuration and make at the same time.
 
 Instead, we can use Kbuild-ish way for that purpose:
   $ make omap4_panda_config all CROSS_COMPILE=arm-linux-gnuabi-

compile test for this setup and a few others run successfully on OS X
Lion as build-host.
Runtime test on at least one avr32 and one at91 board will follow this
weekend.

I saw a few times full compiler output instead of the shortened 'CC
...'. Especially generating the linker script, reformatting binaries
with objcopy and doing the final linkage of u-boot ELF. Maybe you want
to sort that out?
Sorry if that was discussed before, haven't followed the whole discussion.

Nevertheless, many thanks for your great work!

Best regards

Andreas Bießmann
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot


Re: [U-Boot] [PATCH v8 0/38] Switch over to real Kbuild

2014-01-30 Thread Masahiro Yamada
Hello Andreas,

Thanks for your test.

 I saw a few times full compiler output instead of the shortened 'CC
 ...'. Especially generating the linker script, reformatting binaries
 with objcopy and doing the final linkage of u-boot ELF. Maybe you want
 to sort that out?

Yes, I will do this in a follow-up series.


Best Regards
Masahiro Yamada

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot