[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[pinskia at gcc dot gnu.org via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

Andrew Pinski  changed:

   What|Removed |Added

 CC||joerg at netbsd dot org

--- Comment #19 from Andrew Pinski  ---
*** Bug 47047 has been marked as a duplicate of this bug. ***

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

Richard Biener  changed:

   What|Removed |Added

 Status|UNCONFIRMED |RESOLVED
 Resolution|--- |FIXED
   Target Milestone|--- |8.0

--- Comment #18 from Richard Biener  ---
Fixed in GCC 8.

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[alex_y_xu at yahoo dot ca]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #17 from alex_y_xu at yahoo dot ca ---
seems like this should be RESOLVED FIXED?

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[boris at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #16 from boris at gcc dot gnu.org ---
Author: boris
Date: Thu Jan 18 13:17:37 2018
New Revision: 256847

URL: https://gcc.gnu.org/viewcvs?rev=256847=gcc=rev
Log:
Add ability to remap file names in __FILE__, etc (PR other/70268)

This commit adds the -fmacro-prefix-map option that allows remapping of file
names in __FILE__, __BASE_FILE__, and __builtin_FILE(), similar to how
-fdebug-prefix-map allows to do the same for debug information.

Additionally, it adds -ffile-prefix-map which can be used to specify both
mappings with a single option (and, should we need to add more -f*-prefix-map
options in the future, those as well).

libcpp/ChangeLog:

2018-01-18  Boris Kolpackov  

PR other/70268
* include/cpplib.h (cpp_callbacks::remap_filename): New callback.
* libcpp/macro.c (_cpp_builtin_macro_text): Call remap_filename for
__FILE__ and __BASE_FILE__.


gcc/ChangeLog:

2018-01-18  Boris Kolpackov  

PR other/70268
* common.opt: (-ffile-prefix-map): New option.
* opts.c (common_handle_option): Defer it.
* opts-global.c (handle_common_deferred_options): Handle it.
* debug.h (remap_debug_filename, add_debug_prefix_map): Move to...
* file-prefix-map.h: New file.
(remap_debug_filename, add_debug_prefix_map): ...here.
(add_macro_prefix_map, add_file_prefix_map, remap_macro_filename): New.
* final.c (debug_prefix_map, add_debug_prefix_map
remap_debug_filename): Move to...
* file-prefix-map.c: New file.
(file_prefix_map, add_prefix_map, remap_filename) ...here and rename,
generalize, get rid of alloca(), use strrchr() instead of strchr().
(add_macro_prefix_map, add_debug_prefix_map, add_file_prefix_map):
Implement in terms of add_prefix_map().
(remap_macro_filename, remap_debug_filename): Implement in term of
remap_filename().
* Makefile.in (OBJS, PLUGIN_HEADERS): Add new files.
* builtins.c (fold_builtin_FILE): Call remap_macro_filename().
* dbxout.c: Include file-prefix-map.h.
* varasm.c: Likewise.
* vmsdbgout.c: Likewise.
* xcoffout.c: Likewise.
* dwarf2out.c: Likewise plus omit new options from DW_AT_producer.
* doc/cppopts.texi (-fmacro-prefix-map): Document.
* doc/invoke.texi (-ffile-prefix-map): Document.
(-fdebug-prefix-map): Update description.


gcc/c-family/ChangeLog:

2018-01-18  Boris Kolpackov  

PR other/70268
* c-family/c.opt (-fmacro-prefix-map): New option.
* c-family/c-opts.c (c_common_handle_option): Handle it.
* c-family/c-lex.c (init_c_lex): Set remap_filename cpp callback.
* c-family/c-ppoutput.c (init_pp_output): Likewise.


gcc/testsuite/ChangeLog:

2018-01-18  Boris Kolpackov  

PR other/70268
* c-c++-common/ffile-prefix-map.c: New test.
* c-c++-common/fmacro-prefix-map.c: New test.
* c-c++-common/cpp/ffile-prefix-map.c: New test.
* c-c++-common/cpp/fmacro-prefix-map.c: New test.


Added:
trunk/gcc/file-prefix-map.c
trunk/gcc/file-prefix-map.h   (with props)
trunk/gcc/testsuite/c-c++-common/cpp/ffile-prefix-map.c
trunk/gcc/testsuite/c-c++-common/cpp/fmacro-prefix-map.c
trunk/gcc/testsuite/c-c++-common/ffile-prefix-map.c
trunk/gcc/testsuite/c-c++-common/fmacro-prefix-map.c
Modified:
trunk/gcc/ChangeLog
trunk/gcc/Makefile.in
trunk/gcc/builtins.c
trunk/gcc/c-family/ChangeLog
trunk/gcc/c-family/c-lex.c
trunk/gcc/c-family/c-opts.c
trunk/gcc/c-family/c-ppoutput.c
trunk/gcc/c-family/c.opt
trunk/gcc/common.opt
trunk/gcc/dbxout.c
trunk/gcc/debug.h
trunk/gcc/doc/cppopts.texi
trunk/gcc/doc/invoke.texi
trunk/gcc/dwarf2out.c
trunk/gcc/final.c
trunk/gcc/opts-global.c
trunk/gcc/opts.c
trunk/gcc/testsuite/ChangeLog
trunk/gcc/varasm.c
trunk/gcc/vmsdbgout.c
trunk/gcc/xcoffout.c
trunk/libcpp/ChangeLog
trunk/libcpp/include/cpplib.h
trunk/libcpp/macro.c

Propchange: trunk/gcc/file-prefix-map.h
('svn:eol-style' added)

Propchange: trunk/gcc/file-prefix-map.h
('svn:keywords' added)

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[infinity0 at pwned dot gg]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #15 from infinity0 at pwned dot gg ---
(In reply to infinity0 from comment #14)
> > it has a unified option (-ffile-prefix-map) [..]
> 
> Oh, nice. That might save me some work, then.
> 
> Could you bounce me the thread? Or failing that, tell me the Message-ID of
> one of the messages, so I can reply to it.
> 
> > [..] that I believe does exactly what you want
> 

Thanks for forwarding the mail. I've posted a follow-up patch to be applied on
top of your patch, that implements the two things I mentioned.

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[infinity0 at pwned dot gg]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #14 from infinity0 at pwned dot gg ---
> it has a unified option (-ffile-prefix-map) [..]

Oh, nice. That might save me some work, then.

Could you bounce me the thread? Or failing that, tell me the Message-ID of one
of the messages, so I can reply to it.

> [..] that I believe does exactly what you want

Basically yes. However one minor extra thing my patch enables, is the remapping
of arbitrary file paths - whereas the current form can't remap paths with "="
in it. This is not a major issue, but I'd recommend that you change:

-  p = strchr (arg, '=');
+  p = strrchr (arg, '=');

This would be a bit more flexible.

> the environment variable is a bad idea.

The problem with doing this on the command-line is that the command-line
arguments then contain the build-path. Sometimes, builds like to save the
command-line arguments somewhere.

Using an environment variable avoids this issue. I agree envvars are dirty in
general, but in this case one would be *removing* information from what GCC is
already doing (taking in absolute-path information via the filesystem), as
opposed to *adding* information which is what other envvars do.

A more refined way to avoid this issue, is to allow $-based substitution, like
`-ffile-prefix-map=$BUILD_PATH=/usr/my/path` where BUILD_PATH is read from the
environment. Then we avoid adding $BUILD_PATH to the command-line args, but
/usr/my/path is still in there. Daniel Kahn Gillmor implemented this here [1]
though it was rejected in favour of a simpler approach for the bug mentioned
#69821 [2], however in retrospect this mechanism is actually very useful and
flexible. I found out recently that NetBSD still carries this patch to this
day, for reproducible builds. [3]

[1] https://gcc.gnu.org/ml/gcc-patches/2015-12/msg01168.html
[2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69821
[3]
http://cvsweb.netbsd.org/bsdweb.cgi/src/external/gpl3/gcc/dist/gcc/final.c?rev=1.2=text/x-cvsweb-markup

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #13 from Boris Kolpackov  ---
No, I was not aware, thanks for the pointer. I skimmed through it and I agree,
the environment variable is a bad idea. In fact, if you look at the patch that
I've proposed, it has a unified option (-ffile-prefix-map) that I believe does
exactly what you want (remap any references to matching paths in the result of
the compilation) while still allowing you to do more fine-grained mapping
(e.g., only for __FILE__). And it has a provision for future extensions.

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[infinity0 at pwned dot gg]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

infinity0 at pwned dot gg changed:

   What|Removed |Added

 CC||infinity0 at pwned dot gg

--- Comment #12 from infinity0 at pwned dot gg ---
Are you aware of this patch?
https://gcc.gnu.org/ml/gcc-patches/2017-07/msg01315.html

If the overall motivation is to support reproducible builds, then you might be
interested in that, which is a unified mechanism that allows future extensions
to cover things *other* than __FILE__ and debug paths. (For example,
reproducible .o files which, I am told, contain STT_FILE entries that contain
the build path.)

The patch is not accepted yet; I had some review comments back in August and
I'm supposed to amend it to use a command-line flag instead of an environment
variable instead. I was going through my TODO backlog (non-GCC) since that
time, but I'll be finally working on this over the next week or so.

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #11 from Boris Kolpackov  ---
Third revision of the patch:

https://gcc.gnu.org/ml/gcc-patches/2017-12/msg00544.html

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #10 from Boris Kolpackov  ---
Second revision of the patch:

https://gcc.gnu.org/ml/gcc-patches/2017-12/msg00379.html

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #9 from Boris Kolpackov  ---
I've proposed a different implementation (and a bit different options names)
here:

https://gcc.gnu.org/ml/gcc-patches/2017-11/msg01451.html

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[tom.rini at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

Tom Rini  changed:

   What|Removed |Added

 CC||tom.rini at gmail dot com

--- Comment #8 from Tom Rini  ---
This would also be useful in cases where we care about the size of our outputs
but may have cases where file paths are included (critical debug type messages
for example).

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[satta at debian dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #7 from Sascha Steinbiss  ---
Is there any progress on this? Actually such a functionality as provided by
-ffile-prefix-map would also be highly desirable in the context of the
Reproducible Builds effort [1]. Build-specific source paths included via
__FILE__ in gcc-built artefacts are quite common and -- at least from my
experience -- a problematic source of binary variation.

Cheers
Sascha

[1] https://reproducible-builds.org

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[hongxu.jia at windriver dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #6 from hongxu jia  ---
(In reply to rguent...@suse.de from comment #5)
> On Thu, 17 Mar 2016, hongxu.jia at windriver dot com wrote:
> 
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268
> > 
> > --- Comment #3 from hongxu jia  ---
> > (In reply to Richard Biener from comment #2)
> > > IMHO it doesn't make sense to prune system header paths here and having 
> > > them 
> > > is desired.
> > 
> > Hi Richard,
> > 
> > It is just a simple example to explain the requirements.
> > 
> > In our OpenEmbedded project, we do many cross complication,
> > and have very complex build path.
> > 
> > Here is the actual use of option '-fdebug-prefix-map' in our project:
> > 
> > DEBUG_FLAGS="-g -feliminate-unused-debug-types
> > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
> > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
> > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/x86_64-linux=
> > -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/qemux86-64=
> > "
> 
> I see.  Care to propose a patch then?

Yes, also sent to gcc-patc...@gcc.gnu.org for review

//Hongxu

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[hongxu.jia at windriver dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #1 from hongxu jia  ---
So I suggest gcc could provide a option '-ffile-prefix-map=='
to map  directory in __FILE__ and replace it with  directory.

It is similar to '-fdebug-prefix-map==', but the latter is used
for DWARF, and the newly added option only works on '__FILE__', '__BASE_FILE__'
and '__builtin_FILE()'

---
5. Compile C source file with '-ffile-prefix-map'
$ gcc ./test.c -ffile-prefix-map=/usr/include= -o test

4. Execute and the header file is not pull path.
$ ./test 
__FILE__ ./test.c
__FILE__ /test.h
---

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[hongxu.jia at windriver dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #4 from hongxu jia  ---
Created attachment 37995
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=37995=edit
Implement patch

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #2 from Richard Biener  ---
IMHO it doesn't make sense to prune system header paths here and having them 
is desired.

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[rguenther at suse dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #5 from rguenther at suse dot de  ---
On Thu, 17 Mar 2016, hongxu.jia at windriver dot com wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268
> 
> --- Comment #3 from hongxu jia  ---
> (In reply to Richard Biener from comment #2)
> > IMHO it doesn't make sense to prune system header paths here and having 
> > them 
> > is desired.
> 
> Hi Richard,
> 
> It is just a simple example to explain the requirements.
> 
> In our OpenEmbedded project, we do many cross complication,
> and have very complex build path.
> 
> Here is the actual use of option '-fdebug-prefix-map' in our project:
> 
> DEBUG_FLAGS="-g -feliminate-unused-debug-types
> -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
> -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
> -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/x86_64-linux=
> -fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/qemux86-64=
> "

I see.  Care to propose a patch then?

[Bug other/70268] add option -ffile-prefix-map to map one directory name (old) to another (new) in __FILE__, __BASE_FILE__and __builtin_FILE()

[hongxu.jia at windriver dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70268

--- Comment #3 from hongxu jia  ---
(In reply to Richard Biener from comment #2)
> IMHO it doesn't make sense to prune system header paths here and having them 
> is desired.

Hi Richard,

It is just a simple example to explain the requirements.

In our OpenEmbedded project, we do many cross complication,
and have very complex build path.

Here is the actual use of option '-fdebug-prefix-map' in our project:

DEBUG_FLAGS="-g -feliminate-unused-debug-types
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/work/core2-64-poky-linux/service/0.1-r0/service-0.1=/usr/src/service
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/x86_64-linux=
-fdebug-prefix-map=/buildarea/raid0/hjia/build-20160316-yocto-buildpath-2/tmp/sysroots/qemux86-64=
"

//Hongxu