Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Stephen Crane]

Thank you so much. I just added documentation for the new hook to
https://gcc.gnu.org/wiki/whopr/driver so that should finish off this
feature.

I'm happy to fix any bugs that might crop up related to this hook, of
course. I don't follow the binutils list closely, so I apologize in
advance if I miss any issues related to this feature. Just CC me
explicitly and I'll jump on it.

Thanks,
- stephen

On Mon, Dec 11, 2017 at 3:03 PM, Sriraman Tallam  wrote:
> On Mon, Dec 11, 2017 at 2:16 PM, Stephen Crane  wrote:
>> Thanks for committing the GCC portion and following up on this. I had
>> been meaning to write and ask. I don't have commit privs for binutils,
>> so either you or Cary will have to commit the binutils patch as well,
>> if it's not too much trouble. I think much has changed to need a
>> rebase?
>
> I just committed your patch.  I had to make one very minor change to
> plugin_new_section_layout.c to compile, move the loop initialization
> declaration outside as that is not allowed on C.  I tested the patch.
>
> Thanks
> Sri
>
>>
>> Thanks,
>> Stephen
>>
>> On Mon, Dec 11, 2017 at 2:10 PM, Sriraman Tallam  wrote:
>>> On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:
> include/ChangeLog:
> 2017-11-09  Stephen Crane  
>
> * plugin-api.h: Add new plugin hook to allow processing of input
> files added by a plugin.
> (ld_plugin_new_input_handler): New funcion hook type.
> (ld_plugin_register_new_input): New interface.
> (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
> (tv_register_new_input): New member.
>
>
> gold/ChangeLog:
> 2017-11-09  Stephen Crane  
>
> * plugin.cc (Plugin::load): Include hooks for register_new_input
> in transfer vector.
> (Plugin::new_input): New function.
> (register_new_input): New function.
> (Plugin_manager::claim_file): Call Plugin::new_input if in
> replacement phase.
> * plugin.h (Plugin::set_new_input_handler): New function.
> * testsuite/plugin_new_section_layout.c: New plugin to test
> new_input plugin API.
> * testsuite/plugin_final_layout.sh: Add new input test.
> * testsuite/Makefile.am (plugin_layout_new_file): New test case.
> * testsuite/Makefile.in: Regenerate.

 These are OK. Thanks!

 Sri, I'm out of town through 11/18, and won't be able to commit the
 include/ patch to GCC before Stage 1 ends. Can you take care of it?
 (If not, I'll take care of it when I get back -- it was approved
 during Stage 1, so I think it's OK to commit early in Stage 3,
 especially since it's nothing but new declarations.)
>>>
>>> Stephen, I was looking at binutils and realized this patch has not
>>> been committed yet.  I only committed the GCC portion, plugin-api.h.
>>>
>>> Thanks
>>> Sri
>>>

 -cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Sriraman Tallam via gcc-patches]

On Mon, Dec 11, 2017 at 2:16 PM, Stephen Crane  wrote:
> Thanks for committing the GCC portion and following up on this. I had
> been meaning to write and ask. I don't have commit privs for binutils,
> so either you or Cary will have to commit the binutils patch as well,
> if it's not too much trouble. I think much has changed to need a
> rebase?

I just committed your patch.  I had to make one very minor change to
plugin_new_section_layout.c to compile, move the loop initialization
declaration outside as that is not allowed on C.  I tested the patch.

Thanks
Sri

>
> Thanks,
> Stephen
>
> On Mon, Dec 11, 2017 at 2:10 PM, Sriraman Tallam  wrote:
>> On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:
 include/ChangeLog:
 2017-11-09  Stephen Crane  

 * plugin-api.h: Add new plugin hook to allow processing of input
 files added by a plugin.
 (ld_plugin_new_input_handler): New funcion hook type.
 (ld_plugin_register_new_input): New interface.
 (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
 (tv_register_new_input): New member.


 gold/ChangeLog:
 2017-11-09  Stephen Crane  

 * plugin.cc (Plugin::load): Include hooks for register_new_input
 in transfer vector.
 (Plugin::new_input): New function.
 (register_new_input): New function.
 (Plugin_manager::claim_file): Call Plugin::new_input if in
 replacement phase.
 * plugin.h (Plugin::set_new_input_handler): New function.
 * testsuite/plugin_new_section_layout.c: New plugin to test
 new_input plugin API.
 * testsuite/plugin_final_layout.sh: Add new input test.
 * testsuite/Makefile.am (plugin_layout_new_file): New test case.
 * testsuite/Makefile.in: Regenerate.
>>>
>>> These are OK. Thanks!
>>>
>>> Sri, I'm out of town through 11/18, and won't be able to commit the
>>> include/ patch to GCC before Stage 1 ends. Can you take care of it?
>>> (If not, I'll take care of it when I get back -- it was approved
>>> during Stage 1, so I think it's OK to commit early in Stage 3,
>>> especially since it's nothing but new declarations.)
>>
>> Stephen, I was looking at binutils and realized this patch has not
>> been committed yet.  I only committed the GCC portion, plugin-api.h.
>>
>> Thanks
>> Sri
>>
>>>
>>> -cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Sriraman Tallam via gcc-patches]

On Mon, Dec 11, 2017 at 2:16 PM, Stephen Crane  wrote:
> Thanks for committing the GCC portion and following up on this. I had
> been meaning to write and ask. I don't have commit privs for binutils,
> so either you or Cary will have to commit the binutils patch as well,
> if it's not too much trouble. I think much has changed to need a
> rebase?

Ok, let me apply your patch.  I will get back if there are inconsistencies.

Thanks
Sri

>
> Thanks,
> Stephen
>
> On Mon, Dec 11, 2017 at 2:10 PM, Sriraman Tallam  wrote:
>> On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:
 include/ChangeLog:
 2017-11-09  Stephen Crane  

 * plugin-api.h: Add new plugin hook to allow processing of input
 files added by a plugin.
 (ld_plugin_new_input_handler): New funcion hook type.
 (ld_plugin_register_new_input): New interface.
 (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
 (tv_register_new_input): New member.


 gold/ChangeLog:
 2017-11-09  Stephen Crane  

 * plugin.cc (Plugin::load): Include hooks for register_new_input
 in transfer vector.
 (Plugin::new_input): New function.
 (register_new_input): New function.
 (Plugin_manager::claim_file): Call Plugin::new_input if in
 replacement phase.
 * plugin.h (Plugin::set_new_input_handler): New function.
 * testsuite/plugin_new_section_layout.c: New plugin to test
 new_input plugin API.
 * testsuite/plugin_final_layout.sh: Add new input test.
 * testsuite/Makefile.am (plugin_layout_new_file): New test case.
 * testsuite/Makefile.in: Regenerate.
>>>
>>> These are OK. Thanks!
>>>
>>> Sri, I'm out of town through 11/18, and won't be able to commit the
>>> include/ patch to GCC before Stage 1 ends. Can you take care of it?
>>> (If not, I'll take care of it when I get back -- it was approved
>>> during Stage 1, so I think it's OK to commit early in Stage 3,
>>> especially since it's nothing but new declarations.)
>>
>> Stephen, I was looking at binutils and realized this patch has not
>> been committed yet.  I only committed the GCC portion, plugin-api.h.
>>
>> Thanks
>> Sri
>>
>>>
>>> -cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Stephen Crane]

Thanks for committing the GCC portion and following up on this. I had
been meaning to write and ask. I don't have commit privs for binutils,
so either you or Cary will have to commit the binutils patch as well,
if it's not too much trouble. I think much has changed to need a
rebase?

Thanks,
Stephen

On Mon, Dec 11, 2017 at 2:10 PM, Sriraman Tallam  wrote:
> On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:
>>> include/ChangeLog:
>>> 2017-11-09  Stephen Crane  
>>>
>>> * plugin-api.h: Add new plugin hook to allow processing of input
>>> files added by a plugin.
>>> (ld_plugin_new_input_handler): New funcion hook type.
>>> (ld_plugin_register_new_input): New interface.
>>> (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
>>> (tv_register_new_input): New member.
>>>
>>>
>>> gold/ChangeLog:
>>> 2017-11-09  Stephen Crane  
>>>
>>> * plugin.cc (Plugin::load): Include hooks for register_new_input
>>> in transfer vector.
>>> (Plugin::new_input): New function.
>>> (register_new_input): New function.
>>> (Plugin_manager::claim_file): Call Plugin::new_input if in
>>> replacement phase.
>>> * plugin.h (Plugin::set_new_input_handler): New function.
>>> * testsuite/plugin_new_section_layout.c: New plugin to test
>>> new_input plugin API.
>>> * testsuite/plugin_final_layout.sh: Add new input test.
>>> * testsuite/Makefile.am (plugin_layout_new_file): New test case.
>>> * testsuite/Makefile.in: Regenerate.
>>
>> These are OK. Thanks!
>>
>> Sri, I'm out of town through 11/18, and won't be able to commit the
>> include/ patch to GCC before Stage 1 ends. Can you take care of it?
>> (If not, I'll take care of it when I get back -- it was approved
>> during Stage 1, so I think it's OK to commit early in Stage 3,
>> especially since it's nothing but new declarations.)
>
> Stephen, I was looking at binutils and realized this patch has not
> been committed yet.  I only committed the GCC portion, plugin-api.h.
>
> Thanks
> Sri
>
>>
>> -cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Sriraman Tallam via gcc-patches]

On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:
>> include/ChangeLog:
>> 2017-11-09  Stephen Crane  
>>
>> * plugin-api.h: Add new plugin hook to allow processing of input
>> files added by a plugin.
>> (ld_plugin_new_input_handler): New funcion hook type.
>> (ld_plugin_register_new_input): New interface.
>> (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
>> (tv_register_new_input): New member.
>>
>>
>> gold/ChangeLog:
>> 2017-11-09  Stephen Crane  
>>
>> * plugin.cc (Plugin::load): Include hooks for register_new_input
>> in transfer vector.
>> (Plugin::new_input): New function.
>> (register_new_input): New function.
>> (Plugin_manager::claim_file): Call Plugin::new_input if in
>> replacement phase.
>> * plugin.h (Plugin::set_new_input_handler): New function.
>> * testsuite/plugin_new_section_layout.c: New plugin to test
>> new_input plugin API.
>> * testsuite/plugin_final_layout.sh: Add new input test.
>> * testsuite/Makefile.am (plugin_layout_new_file): New test case.
>> * testsuite/Makefile.in: Regenerate.
>
> These are OK. Thanks!
>
> Sri, I'm out of town through 11/18, and won't be able to commit the
> include/ patch to GCC before Stage 1 ends. Can you take care of it?
> (If not, I'll take care of it when I get back -- it was approved
> during Stage 1, so I think it's OK to commit early in Stage 3,
> especially since it's nothing but new declarations.)

Stephen, I was looking at binutils and realized this patch has not
been committed yet.  I only committed the GCC portion, plugin-api.h.

Thanks
Sri

>
> -cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Sriraman Tallam via gcc-patches]

On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:

> > include/ChangeLog:
> > 2017-11-09  Stephen Crane  
> >
> > * plugin-api.h: Add new plugin hook to allow processing of input
> > files added by a plugin.
> > (ld_plugin_new_input_handler): New funcion hook type.
> > (ld_plugin_register_new_input): New interface.
> > (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
> > (tv_register_new_input): New member.
> >
> >
> > gold/ChangeLog:
> > 2017-11-09  Stephen Crane  
> >
> > * plugin.cc (Plugin::load): Include hooks for register_new_input
> > in transfer vector.
> > (Plugin::new_input): New function.
> > (register_new_input): New function.
> > (Plugin_manager::claim_file): Call Plugin::new_input if in
> > replacement phase.
> > * plugin.h (Plugin::set_new_input_handler): New function.
> > * testsuite/plugin_new_section_layout.c: New plugin to test
> > new_input plugin API.
> > * testsuite/plugin_final_layout.sh: Add new input test.
> > * testsuite/Makefile.am (plugin_layout_new_file): New test case.
> > * testsuite/Makefile.in: Regenerate.
>
> These are OK. Thanks!
>
> Sri, I'm out of town through 11/18, and won't be able to commit the
> include/ patch to GCC before Stage 1 ends. Can you take care of it?
> (If not, I'll take care of it when I get back -- it was approved
> during Stage 1, so I think it's OK to commit early in Stage 3,
> especially since it's nothing but new declarations.)
>

Committed plugin-api.h patch to GCC trunk.

Thanks
Sri


>
> -cary
>

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Sriraman Tallam via gcc-patches]

On Thu, Nov 9, 2017 at 9:04 PM, Cary Coutant  wrote:

> > include/ChangeLog:
> > 2017-11-09  Stephen Crane  
> >
> > * plugin-api.h: Add new plugin hook to allow processing of input
> > files added by a plugin.
> > (ld_plugin_new_input_handler): New funcion hook type.
> > (ld_plugin_register_new_input): New interface.
> > (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
> > (tv_register_new_input): New member.
> >
> >
> > gold/ChangeLog:
> > 2017-11-09  Stephen Crane  
> >
> > * plugin.cc (Plugin::load): Include hooks for register_new_input
> > in transfer vector.
> > (Plugin::new_input): New function.
> > (register_new_input): New function.
> > (Plugin_manager::claim_file): Call Plugin::new_input if in
> > replacement phase.
> > * plugin.h (Plugin::set_new_input_handler): New function.
> > * testsuite/plugin_new_section_layout.c: New plugin to test
> > new_input plugin API.
> > * testsuite/plugin_final_layout.sh: Add new input test.
> > * testsuite/Makefile.am (plugin_layout_new_file): New test case.
> > * testsuite/Makefile.in: Regenerate.
>
> These are OK. Thanks!
>
> Sri, I'm out of town through 11/18, and won't be able to commit the
> include/ patch to GCC before Stage 1 ends. Can you take care of it?
> (If not, I'll take care of it when I get back -- it was approved
> during Stage 1, so I think it's OK to commit early in Stage 3,
> especially since it's nothing but new declarations.)
>

Sure, I will do this today.

Thanks
Sri


>
> -cary
>

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Cary Coutant]

> include/ChangeLog:
> 2017-11-09  Stephen Crane  
>
> * plugin-api.h: Add new plugin hook to allow processing of input
> files added by a plugin.
> (ld_plugin_new_input_handler): New funcion hook type.
> (ld_plugin_register_new_input): New interface.
> (LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
> (tv_register_new_input): New member.
>
>
> gold/ChangeLog:
> 2017-11-09  Stephen Crane  
>
> * plugin.cc (Plugin::load): Include hooks for register_new_input
> in transfer vector.
> (Plugin::new_input): New function.
> (register_new_input): New function.
> (Plugin_manager::claim_file): Call Plugin::new_input if in
> replacement phase.
> * plugin.h (Plugin::set_new_input_handler): New function.
> * testsuite/plugin_new_section_layout.c: New plugin to test
> new_input plugin API.
> * testsuite/plugin_final_layout.sh: Add new input test.
> * testsuite/Makefile.am (plugin_layout_new_file): New test case.
> * testsuite/Makefile.in: Regenerate.

These are OK. Thanks!

Sri, I'm out of town through 11/18, and won't be able to commit the
include/ patch to GCC before Stage 1 ends. Can you take care of it?
(If not, I'll take care of it when I get back -- it was approved
during Stage 1, so I think it's OK to commit early in Stage 3,
especially since it's nothing but new declarations.)

-cary

Re: [PATCH] [GOLD] Add plugin API for processing plugin-added input files

[Stephen Crane]

Cary Coutant  writes:

> The patch for include/plugin-api.h needs to be separated out (with a
> ChangeLog entry of its own), and applied first to the GCC tree, then
> synced to binutils. After that, the rest of this patch is OK to apply.

Thanks. Attached is the patch for plugin-api.h which needs to be applied
to GCC and binutils (plugin_api_new_input.patch). Also attached is the
gold patch to implement the new functionality
(gold_new_input_handler.patch). ChangeLogs for both patches are below. I
don't have write access to either repo, so would you be willing to
commit on my behalf?

> Can you also update the GCC wiki whopr/driver page?

I'd be happy to if I can be added to the EditorGroup (my username is
StephenCrane).


include/ChangeLog:
2017-11-09  Stephen Crane  

* plugin-api.h: Add new plugin hook to allow processing of input
files added by a plugin.
(ld_plugin_new_input_handler): New funcion hook type.
(ld_plugin_register_new_input): New interface.
(LDPT_REGISTER_NEW_INPUT_HOOK): New enum val.
(tv_register_new_input): New member.


gold/ChangeLog:
2017-11-09  Stephen Crane  

* plugin.cc (Plugin::load): Include hooks for register_new_input
in transfer vector.
(Plugin::new_input): New function.
(register_new_input): New function.
(Plugin_manager::claim_file): Call Plugin::new_input if in
replacement phase.
* plugin.h (Plugin::set_new_input_handler): New function.
* testsuite/plugin_new_section_layout.c: New plugin to test
new_input plugin API.
* testsuite/plugin_final_layout.sh: Add new input test.
* testsuite/Makefile.am (plugin_layout_new_file): New test case.
* testsuite/Makefile.in: Regenerate.

commit f45d97f0ee4c202602c9951ade622caf2e27ffe2
Author: Stephen Crane 
Date:   Thu Sep 21 14:22:37 2017 -0700

Add plugin API for processing plugin-added input files

Gold plugins may wish to further process an input file added by a plugin. For
example, the plugin may need to assign a unique segment for sections in a
plugin-generated input file. This patch adds a plugin callback that the linker
will call when reading symbols from a new input file added after the
all_symbols_read event (i.e. an input file added by a plugin).

diff --git a/include/plugin-api.h b/include/plugin-api.h
index 3a3e8b456db..f081f85dfaf 100644
--- a/include/plugin-api.h
+++ b/include/plugin-api.h
@@ -365,6 +365,20 @@ enum ld_plugin_status
 (*ld_plugin_get_input_section_size) (const struct ld_plugin_section section,
  uint64_t *secsize);
 
+typedef
+enum ld_plugin_status
+(*ld_plugin_new_input_handler) (const struct ld_plugin_input_file *file);
+
+/* The linker's interface for registering the "new_input" handler. This handler
+   will be notified when a new input file has been added after the
+   all_symbols_read event, allowing the plugin to, for example, set a unique
+   segment for sections in plugin-generated input files. */
+
+typedef
+enum ld_plugin_status
+(*ld_plugin_register_new_input) (ld_plugin_new_input_handler handler);
+
+
 enum ld_plugin_level
 {
   LDPL_INFO,
@@ -407,7 +421,8 @@ enum ld_plugin_tag
   LDPT_UNIQUE_SEGMENT_FOR_SECTIONS = 27,
   LDPT_GET_SYMBOLS_V3 = 28,
   LDPT_GET_INPUT_SECTION_ALIGNMENT = 29,
-  LDPT_GET_INPUT_SECTION_SIZE = 30
+  LDPT_GET_INPUT_SECTION_SIZE = 30,
+  LDPT_REGISTER_NEW_INPUT_HOOK = 31
 };
 
 /* The plugin transfer vector.  */
@@ -441,6 +456,7 @@ struct ld_plugin_tv
 ld_plugin_unique_segment_for_sections tv_unique_segment_for_sections;
 ld_plugin_get_input_section_alignment tv_get_input_section_alignment;
 ld_plugin_get_input_section_size tv_get_input_section_size;
+ld_plugin_register_new_input tv_register_new_input;
   } tv_u;
 };
 
diff --git a/gold/plugin.cc b/gold/plugin.cc
index 5ea23b5bdd..49212b4309 100644
--- a/gold/plugin.cc
+++ b/gold/plugin.cc
@@ -167,6 +167,9 @@ static enum ld_plugin_status
 get_input_section_size(const struct ld_plugin_section section,
uint64_t* secsize);
 
+static enum ld_plugin_status
+register_new_input(ld_plugin_new_input_handler handler);
+
 };
 
 #endif // ENABLE_PLUGINS
@@ -211,7 +214,7 @@ Plugin::load()
   sscanf(ver, "%d.%d", , );
 
   // Allocate and populate a transfer vector.
-  const int tv_fixed_size = 29;
+  const int tv_fixed_size = 30;
 
   int tv_size = this->args_.size() + tv_fixed_size;
   ld_plugin_tv* tv = new ld_plugin_tv[tv_size];
@@ -345,6 +348,10 @@ Plugin::load()
   tv[i].tv_tag = LDPT_GET_INPUT_SECTION_SIZE;
   tv[i].tv_u.tv_get_input_section_size = get_input_section_size;
 
+  ++i;
+  tv[i].tv_tag = LDPT_REGISTER_NEW_INPUT_HOOK;
+  tv[i].tv_u.tv_register_new_input = register_new_input;
+
   ++i;
   tv[i].tv_tag = LDPT_NULL;
   tv[i].tv_u.tv_val = 0;
@@ -383,6 +390,15 @@ Plugin::all_symbols_read()