On Mon, Oct 8, 2012 at 11:29 PM, David Malcolm <dmalc...@redhat.com> wrote:
> On Mon, 2012-10-08 at 18:21 +0200, Richard Guenther wrote:
>> On Mon, Oct 8, 2012 at 5:17 PM, David Malcolm <dmalc...@redhat.com> wrote:
>> > I'm working on a static analysis extension to GCC via my
>> > gcc-python-plugin [1]
>> >
>> > The analysis is interprocedural (memory leak detection, as it happens).
>> > I have it working on one translation unit at a time, and I'm attempting
>> > to get it to work within Link Time Optimization so that it can see the
>> > interactions of all of the functions within a program or library, but
>> > I'm running what might be a bug (or I could be misunderstanding LTO).
>> >
>> > I have my plugin working within lto1, using "-flto" and "-fplugin"; my
>> > command line looks like:
>> > $ gcc -fPIC -shared -flto -g
>> > -fplugin=/home/david/coding/gcc-python/gcc-python/sm/python.so
>> > -fplugin-arg-python-script=examples/show-lto-supergraph.py
>> > tests/sm/lto/input-*.c --save-temps
>> >
>> > lto1 appears to be invoked twice by gcc. The first time, there are two
>> > passes:
>> > IPA_PASS named "whole-program"
>> > IPA_PASS named "inline"
>> > and "in_lto_p" is true.
>> >
>> > The second invocation has these passes, called per-function:
>> > GIMPLE_PASS "*free_cfg_annotations"
>> > GIMPLE_PASS "cplxlower0"
>> > GIMPLE_PASS "optimized"
>> > RTL_PASS "expand"
>> > GIMPLE_PASS "*rest_of_compilation"
>> > RTL_PASS "*init_function"
>> > ...etc.., normal rest of compilation from RTL stage onwards
>> > and "in_lto_p" is false.
>> >
>> > I've attempted to wire in my interprocedural analysis as a new IPA pass
>> > before or after "whole-program" and "inline" within the first invocation
>> > of lto1. In each case it is able to walk the callgraph, and the
>> > callgraph it sees appears to be correct: walking the linked list of
>> > cgraph_nodes the various cgraph_edge and cgraph_node are as expected,
>> > and for each cgraph_node, cgraph_node->decl is a non-NULL tree instance
>> > of type FUNCTION_DECL (as expected); I'm able to generate a graphviz
>> > rendering of the whole-program callgraph within lto1 from my plugin.
>> >
>> > However, for every cgraph_node, the
>> > DECL_STRUCT_FUNCTION(cgraph_node->decl) is (struct function*)NULL, which
>> > thwarts my code's attempt to look up the CFG of each underlying function
>> > and work on the underlying gimple.
>> >
>> > Looking with eu-readelf at the .o files shows these lto sections are
>> > present ("f" is the name of one of the functions that I'd like to access
>> > the gimple CFG of):
>> > $ eu-readelf -a input-f.o|grep lto
>> > [ 5] .gnu.lto_.inline.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 0000006c 00000026 0 E 0 0 1
>> > [ 6] .gnu.lto_f.c8904cb9a96e7417 PROGBITS 0000000000000000 00000092
>> > 00000164 0 E 0 0 1
>> > [ 7] .gnu.lto_.cgraph.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 000001f6 00000032 0 E 0 0 1
>> > [ 8] .gnu.lto_.vars.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 00000228 00000012 0 E 0 0 1
>> > [ 9] .gnu.lto_.refs.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 0000023a 00000013 0 E 0 0 1
>> > [10] .gnu.lto_.statics.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 0000024d 00000014 0 E 0 0 1
>> > [11] .gnu.lto_.decls.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 00000261 000001f3 0 E 0 0 1
>> > [12] .gnu.lto_.symtab.c8904cb9a96e7417 PROGBITS 0000000000000000
>> > 00000454 00000011 0 E 0 0 1
>> > [13] .gnu.lto_.opts PROGBITS 0000000000000000 00000465 000000e9
>> > 0 E 0 0 1
>> > 25: 0000000000000001 1 OBJECT GLOBAL DEFAULT COMMON __gnu_lto_v1
>> > [ 2a] __gnu_lto_v1
>> >
>> > Are the (struct function*) and CFG meant to exist at this stage, and be
>> > in gimple form, and is DECL_STRUCT_FUNCTION meant to be non-NULL for
>> > functions in translation units that were compiled with -flto? (or have
>> > I misunderstood LTO?)
>>
>> It depends if you are in the WPA stage (lto1 was invoked with -fwpa)
>> in which case
>> no function bodies and thus no CFG is available at all, or if you are in
>> LTRANS
>> stage (lto1 was invoked with -fltrans) which see _part_ of the whole programs
>> callgraph and function bodies (and thus CFGs). As a workaround you probably
>> can make your example work by using -flto-partition=none which merges WPA
>> and LTRANS stages and pull in the wole program into a single link-time TU.
>
> Many thanks: "-fwpa" was indeed being passed by "gcc" to "lto1", and on
> adding "-flto-partition=none" to the gcc command line -fwpa goes away,
> and my plugin is able to see all of the gimple CFG from all of the .o
> files (and thus generate pretty graphviz renderings of the supergraph of
> all CFGs etc).
>
> FWIW the -fwpa behavior was surprising to me, and felt like something of
> a "gotcha". Based on my reading of
> http://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html#LTO-Overview
> I had assumed that "LTO mode" was the default when setting "-flto", and
> that "WHOPR mode" was something that I had to add an extra option to
> enable (and thus I merely skimmed those parts of the docs), whereas it
> seems to be the other way around. In my defense, the sentence "WHOPR
> can be seen as an extension of the usual LTO mode of compilation" on
> that page does give the impression (to me, at least), incorrectly, that
> unpartitioned LTO is the default - should that be reworded? (That page
> could really use some diagrams; does the docs toolchain support this?)
Maybe the documentation can be improved, but from gccint I cannot
derive what mode is the default.
The -flto documentation in the user manual probably needs an overhaul
though.
Richard.
> BTW, if I'm reading the code correctly there's a minor docs error
> within:
> http://gcc.gnu.org/onlinedocs/gccint/LTO-object-file-layout.html#LTO-object-file-layout
> Looking at 4.7.0, gcc/lto-streamer.c:lto_get_section_name() sets the
> name of sections of type "LTO_section_function_body" to contain the name
> of their function *without* adding the corresponding entry from the
> lto_section_name table (aka "function_body").
>
> So the relevant part of that page should (I believe) read:
> "Function bodies (.gnu.lto_.<function_name>)"
> rather than:
> "Function bodies (.gnu.lto_.function_body.<name>)"
>
> Thanks again; hope the above is helpful.
> Dave
>
