Re: WPA stream_out form memory consumption

2014-04-15 Thread Jan Hubicka
On Mon, Apr 7, 2014 at 8:20 PM, Jan Hubicka hubi...@ucw.cz wrote: AFAIK we settled on a simpler one dropping columns at stream-out time that also helped. As for the correct way to do the optimization we agreed(?) that streaming the locations elsewhere and using references to them is

Re: WPA stream_out form memory consumption

2014-04-08 Thread Martin Liška
On 04/07/2014 07:10 PM, Jan Hubicka wrote: I added new graph for 'xloc.column = 0' hack, just applied this single patch to trunk. Link: https://drive.google.com/file/d/0B0pisUJ80pO1MW11WHdjMk9KQnc/edit?usp=sharing Good, does these two patches combine together well? (they are rater orthogonal,

Re: WPA stream_out form memory consumption

2014-04-08 Thread Richard Biener
On Mon, Apr 7, 2014 at 8:20 PM, Jan Hubicka hubi...@ucw.cz wrote: AFAIK we settled on a simpler one dropping columns at stream-out time that also helped. As for the correct way to do the optimization we agreed(?) that streaming the locations elsewhere and using references to them is more

Re: WPA stream_out form memory consumption

2014-04-07 Thread Martin Liška
On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_allocate_node) 0: 0.0% 0: 0.0% 182236800: 9.7% 0:

Re: WPA stream_out form memory consumption

2014-04-07 Thread Richard Biener
On Mon, Apr 7, 2014 at 1:25 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_allocate_node)

Re: WPA stream_out form memory consumption

2014-04-07 Thread Martin Liška
On 04/04/2014 05:10 PM, Martin Liška wrote: On 04/03/2014 03:07 PM, Richard Biener wrote: On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška mli...@suse.cz wrote: On 04/02/2014 04:13

Re: WPA stream_out form memory consumption

2014-04-07 Thread Martin Liška
On 04/07/2014 01:49 PM, Richard Biener wrote: On Mon, Apr 7, 2014 at 1:25 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444

Re: WPA stream_out form memory consumption

2014-04-07 Thread Jan Hubicka
I added new graph for 'xloc.column = 0' hack, just applied this single patch to trunk. Link: https://drive.google.com/file/d/0B0pisUJ80pO1MW11WHdjMk9KQnc/edit?usp=sharing Good, does these two patches combine together well? (they are rater orthogonal, but perhaps with columns disabled

Re: WPA stream_out form memory consumption

2014-04-07 Thread Jan Hubicka
AFAIK we settled on a simpler one dropping columns at stream-out time that also helped. As for the correct way to do the optimization we agreed(?) that streaming the locations elsewhere and using references to them is more appropriate. At stream-in (or before stream-out) we can then read

Re: WPA stream_out form memory consumption

2014-04-04 Thread Martin Liška
On 04/03/2014 03:07 PM, Richard Biener wrote: On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška mli...@suse.cz wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48

Re: WPA stream_out form memory consumption

2014-04-03 Thread Richard Biener
On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška mli...@suse.cz wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48 AM, Martin Liška wrote: Previous patch is wrong, I did a mistake in name ;) Martin On 03/27/2014 09:52 AM, Martin Liška wrote: On 03/25/2014 09:50 PM,

Re: WPA stream_out form memory consumption

2014-04-03 Thread Richard Biener
On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška mli...@suse.cz wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48 AM, Martin Liška wrote: Previous patch is

Re: WPA stream_out form memory consumption

2014-04-03 Thread Jan Hubicka
On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška mli...@suse.cz wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška mli...@suse.cz wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48 AM, Martin Liška wrote:

Re: WPA stream_out form memory consumption

2014-04-03 Thread Jan Hubicka
Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_allocate_node) 0: 0.0% 0: 0.0% 182236800: 9.7% 0: 0.0% 555600 toplev.c:960

Re: WPA stream_out form memory consumption

2014-04-02 Thread Martin Liška
On 03/27/2014 10:48 AM, Martin Liška wrote: Previous patch is wrong, I did a mistake in name ;) Martin On 03/27/2014 09:52 AM, Martin Liška wrote: On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do

Re: WPA stream_out form memory consumption

2014-04-02 Thread Jan Hubicka
Hello, taking latest trunk gcc, I built Firefox and Chromium. Both projects compiled without debugging symbols and -O2 on an 8-core machine. Firefox: -flto=9, peak memory usage (in LTRANS): 11GB Chromium: -flto=6, peak memory usage (in parallel WPA phase ): 16.5GB I see, the

Re: WPA stream_out form memory consumption

2014-04-02 Thread Jan Hubicka
Previous email presents a bit misleading graphs (influenced by --enable-gather-detailed-mem-stats). Firefox: -flto=9, WPA peak: 8GB, LTRANS peak: 8GB -flto=4, WPA peak: 5GB, LTRANS peak: 3.5GB -flto=1, WPA peak: 3.5GB, LTRANS peak: ~1GB These data shows that parallel WPA streaming

Re: WPA stream_out form memory consumption

2014-03-27 Thread Martin Liška
On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB. When WPA start

Re: WPA stream_out form memory consumption

2014-03-27 Thread Martin Liška
Previous patch is wrong, I did a mistake in name ;) Martin On 03/27/2014 09:52 AM, Martin Liška wrote: On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel:

Re: WPA stream_out form memory consumption

2014-03-25 Thread Jan Hubicka
Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB. When WPA start to fork, memory consumption increases so

WPA stream_out form memory consumption

2014-03-18 Thread Martin Liška
Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB. When WPA start to fork, memory consumption increases so

Re: WPA stream_out form memory consumption

2014-03-18 Thread Richard Biener
On Tue, Mar 18, 2014 at 4:09 PM, Martin Liška mli...@suse.cz wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1

Re: WPA stream_out form memory consumption

2014-03-18 Thread Richard Biener
On Tue, Mar 18, 2014 at 4:13 PM, Richard Biener richard.guent...@gmail.com wrote: On Tue, Mar 18, 2014 at 4:09 PM, Martin Liška mli...@suse.cz wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: