Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-11-01 Thread Daniel Kahn Gillmor
On Tue 2016-11-01 11:56:00 -0400, Ximin Luo wrote: > But the TIMESTAMP patch is for __TIMESTAMP__ whereas our > currently-existing Debian patches are only for __DATE__ and __TIME__. ah, right. sorry i totally missed that, and of course it makes sense. Perhaps you could call that out explicitly in

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-11-01 Thread Ximin Luo
Hey, thanks for the review. Daniel Kahn Gillmor: > > Looking at what we're doing in debian, i see: > > > https://sources.debian.net/src/gcc-6/6.2.0-9/debian/patches/gcc-SOURCE_DATE_EPOCH.diff/ > > https://sources.debian.net/src/gcc-6/6.2.0-9/debian/patches/gcc-SOURCE_DATE_EPOCH-doc.dif

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-11-01 Thread Daniel Kahn Gillmor
Hi Ximin-- On Tue 2016-11-01 09:52:00 -0400, Ximin Luo wrote: > Updated patches here: > > https://gist.github.com/infinity0/72aba22411a2e8baaae4649329f96643 > > Everyone please feel free to comment / review before I soon send them off to > GCC. This looks great, thank you! I have not tried runn

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-11-01 Thread Ximin Luo
Ximin Luo: > Daniel Kahn Gillmor: >> On Thu 2016-10-27 11:18:00 -0400, Ximin Luo wrote: >>> So, I can implement option B fairly easily, but there is one ugly fact >>> about the syntax that we should think about. The way that GCC does it, >>> it means you can't strip a prefix that itself contains th

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-27 Thread Ximin Luo
Daniel Kahn Gillmor: > On Thu 2016-10-27 11:18:00 -0400, Ximin Luo wrote: >> So, I can implement option B fairly easily, but there is one ugly fact >> about the syntax that we should think about. The way that GCC does it, >> it means you can't strip a prefix that itself contains the "=" >> characte

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-27 Thread Daniel Kahn Gillmor
On Thu 2016-10-27 11:18:00 -0400, Ximin Luo wrote: > So, I can implement option B fairly easily, but there is one ugly fact > about the syntax that we should think about. The way that GCC does it, > it means you can't strip a prefix that itself contains the "=" > character. > > Now this is not goin

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-27 Thread Ximin Luo
HW42: > Daniel Kahn Gillmor: >> On Tue 2016-10-25 08:29:00 -0400, Ximin Luo wrote: >>> Option A >>> >>> oldprefix = getenv("SOURCE_ROOT_DIR") >>> newprefix = getenv("SOURCE_ROOT_PREFIX") or "." >>> >>> Pros: Simple, easy to understand. Works almost exactly as >>> debug-prefix-map which has already

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-26 Thread HW42
Daniel Kahn Gillmor: > On Tue 2016-10-25 08:29:00 -0400, Ximin Luo wrote: >> Option A >> >> oldprefix = getenv("SOURCE_ROOT_DIR") >> newprefix = getenv("SOURCE_ROOT_PREFIX") or "." >> >> Pros: Simple, easy to understand. Works almost exactly as >> debug-prefix-map which has already existed for age

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-25 Thread Daniel Kahn Gillmor
On Tue 2016-10-25 08:29:00 -0400, Ximin Luo wrote: > Option A > > oldprefix = getenv("SOURCE_ROOT_DIR") > newprefix = getenv("SOURCE_ROOT_PREFIX") or "." > > Pros: Simple, easy to understand. Works almost exactly as debug-prefix-map > which has already existed for ages in GCC. > Cons: Uses two en

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-25 Thread Ximin Luo
Ximin Luo: > [..] I was trying to come to agreement on the exact form the proposal should > take. > > The behaviour is already settled: > > if path.startswith(R): > newprefix = P or "." > path = newprefix + path[len(R):] > > [..] What I'm asking is suggestions on how we should get R and P i

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-24 Thread Ximin Luo
Daniel Kahn Gillmor: > On Mon 2016-10-24 16:18:00 -0400, Ximin Luo wrote: >> Almost, with two minor corrections: >> >> I picked SOURCE_ROOT_DIR; SOURCE_ROOT has 6395 pages of results in >> sources.debian.net and I didn't want to check that none of these are >> environment variables. > > fair enoug

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-24 Thread Daniel Kahn Gillmor
On Mon 2016-10-24 16:18:00 -0400, Ximin Luo wrote: > Almost, with two minor corrections: > > I picked SOURCE_ROOT_DIR; SOURCE_ROOT has 6395 pages of results in > sources.debian.net and I didn't want to check that none of these are > environment variables. fair enough. > SOURCE_ROOT_PREFIX would o

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-24 Thread Ximin Luo
Daniel Kahn Gillmor: > On Mon 2016-10-24 15:33:00 -0400, Ximin Luo wrote: >> HW42: >>> Btw: Do you know why currently debug-prefix-map maps the source dir to >>> '.'? (My guess is because that's the easiest in dpkg-buildflags) I think >>> for debugging between package boundaries '${srcpkg}-${srcver

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-24 Thread Daniel Kahn Gillmor
On Mon 2016-10-24 15:33:00 -0400, Ximin Luo wrote: > HW42: >> Btw: Do you know why currently debug-prefix-map maps the source dir to >> '.'? (My guess is because that's the easiest in dpkg-buildflags) I think >> for debugging between package boundaries '${srcpkg}-${srcver}' would be >> much better.

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-10-24 Thread Ximin Luo
HW42: > Daniel Kahn Gillmor: >> On Tue 2016-09-06 16:02:00 -0400, Ximin Luo wrote: >>> Thanks, I did see this a while ago and forgot about it. However it >>> does differ from the current proposal in an important way. >>> >>> Current proposal (2): GCC should, if SOURCE_ROOT is set and >>> debug-pref

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-19 Thread Ximin Luo
HW42: > Ximin Luo: >> Ximin Luo: >>> I'm going to suggest SOURCE_ROOT_DIR [..] >> >> OTOH, taking into account dkg's earlier point about overloading >> "root", perhaps SOURCE_BASE_DIR would be better. >> >> It is used quite heavily: >> >> https://codesearch.debian.net/search?q=\bSOURCE_BASE_DIR\b >

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-18 Thread HW42
Ximin Luo: > Ximin Luo: >> I'm going to suggest SOURCE_ROOT_DIR [..] > > OTOH, taking into account dkg's earlier point about overloading > "root", perhaps SOURCE_BASE_DIR would be better. > > It is used quite heavily: > > https://codesearch.debian.net/search?q=\bSOURCE_BASE_DIR\b > > but only b

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-18 Thread HW42
Ximin Luo: > HW42: >>> I preferred the ${x}dir style instead of dir_${x} or ${x}_dir because >>> of some existing conventions like >>> >>> https://www.gnu.org/prep/standards/html_node/Directory-Variables.html >> >> Well, OTOH, freedesktop.org uses _DIR: >> >> https://specifications.freedesktop.org/

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-17 Thread Ximin Luo
Ximin Luo: > I'm going to suggest SOURCE_ROOT_DIR [..] OTOH, taking into account dkg's earlier point about overloading "root", perhaps SOURCE_BASE_DIR would be better. It is used quite heavily: https://codesearch.debian.net/search?q=\bSOURCE_BASE_DIR\b but only by cmake files: https://codesea

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-17 Thread Ximin Luo
HW42: >> I preferred the ${x}dir style instead of dir_${x} or ${x}_dir because >> of some existing conventions like >> >> https://www.gnu.org/prep/standards/html_node/Directory-Variables.html > > Well, OTOH, freedesktop.org uses _DIR: > > https://specifications.freedesktop.org/basedir-spec/basedi

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-12 Thread HW42
Ximin Luo: > Daniel Kahn Gillmor: >> On Fri 2016-09-09 01:33:00 +0200, HW42 wrote: >>> According to codesearch.d.n it seems only SOURCE_DIR_ROOT isn't taken >>> yet. (Google finds at least one case). >> >> thanks for looking, that's what i should have done in the first place. >> >>> also bikesheddi

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-12 Thread Ximin Luo
Daniel Kahn Gillmor: > On Fri 2016-09-09 01:33:00 +0200, HW42 wrote: >> According to codesearch.d.n it seems only SOURCE_DIR_ROOT isn't taken >> yet. (Google finds at least one case). > > thanks for looking, that's what i should have done in the first place. > >> also bikeshedding: I dislike SOUR

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-09 Thread Daniel Kahn Gillmor
On Fri 2016-09-09 01:33:00 +0200, HW42 wrote: > According to codesearch.d.n it seems only SOURCE_DIR_ROOT isn't taken > yet. (Google finds at least one case). thanks for looking, that's what i should have done in the first place. > also bikeshedding: I dislike SOURCE_DIR_ROOT because the name is

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-08 Thread HW42
Daniel Kahn Gillmor: > On Tue 2016-09-06 16:02:00 -0400, Ximin Luo wrote: >> Thanks, I did see this a while ago and forgot about it. However it >> does differ from the current proposal in an important way. >> >> Current proposal (2): GCC should, if SOURCE_ROOT is set and >> debug-prefix-map is not

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-07 Thread Daniel Kahn Gillmor
On Tue 2016-09-06 16:02:00 -0400, Ximin Luo wrote: > Thanks, I did see this a while ago and forgot about it. However it > does differ from the current proposal in an important way. > > Current proposal (2): GCC should, if SOURCE_ROOT is set and > debug-prefix-map is not given, *automatically* use t

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-06 Thread Ximin Luo
Daniel Kahn Gillmor: > On Wed 2016-08-24 07:20:00 -0400, Ximin Luo wrote: >> 2. Define another variable SOURCE_ROOT to be set to the top-level >> source dir, and patch GCC to use this as the default value for >> debug-prefix-map (and the analogue for other languages / tools). >> >> This would have

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-06 Thread Daniel Kahn Gillmor
On Wed 2016-08-24 07:20:00 -0400, Ximin Luo wrote: > 2. Define another variable SOURCE_ROOT to be set to the top-level > source dir, and patch GCC to use this as the default value for > debug-prefix-map (and the analogue for other languages / tools). > > This would have the same concrete behaviour

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-06 Thread Holger Levsen
On Mon, Sep 05, 2016 at 11:26:00PM +, HW42 wrote: > Sorry I have currently no link handy. But the process for C is not very > complex: [great example] very nice, thanks for this writeup, HW42! -- cheers, Holger signature.asc Description: Digital signature _

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-06 Thread Christoph Berg
Re: Ximin Luo 2016-09-06 <1f0f5aa5-449b-5bca-7e0b-e8dfab756...@debian.org> > Well, we could have dpkg-buildflags also set SOURCE_ROOT somehow. A hacky > strawman solution, is to patch: > > 1. dpkg-buildflags --export to also define SOURCE_ROOT > 2. /usr/share/dpkg/buildflags.mk > > Debhelper and

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-06 Thread Ximin Luo
HW42: > Ximin Luo: >> However the question is, do we want to do this every time an upstream >> saves CFLAGS somewhere? > [...] >> 2. Define another variable SOURCE_ROOT to be set to the top-level >> source dir, and patch GCC to use this as the default value for >> debug-prefix-map (and the analogue

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-05 Thread HW42
Ximin Luo: > HW42: >> Ximin Luo: >>> [..] >>> >>> 1. Patch GCC to set debug-prefix-map to "$pwd=." by default (and the >>> analogue for other languages / tools). >>> >>> This behaviour is concretely different from the current situation: >>> during recursive-make scenarios, the gcc in subdirectories

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-05 Thread Ximin Luo
HW42: > Ximin Luo: >> [..] >> >> 1. Patch GCC to set debug-prefix-map to "$pwd=." by default (and the >> analogue for other languages / tools). >> >> This behaviour is concretely different from the current situation: >> during recursive-make scenarios, the gcc in subdirectories will have >> differe

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-05 Thread HW42
Ximin Luo: > However the question is, do we want to do this every time an upstream > saves CFLAGS somewhere? [...] > 2. Define another variable SOURCE_ROOT to be set to the top-level > source dir, and patch GCC to use this as the default value for > debug-prefix-map (and the analogue for other lang

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-09-04 Thread Christoph Berg
Re: Ximin Luo 2016-08-24 > However, I am not sure if this is the best approach. At present, the output > of dpkg-buildflags is itself dependent on the build-path, and some packages > *might* save this value somewhere in their output. (This is the case for > perl, although in perl's specific cas

Re: [Reproducible-builds] Minimising work needed for this build-path issue

2016-08-24 Thread HW42
Ximin Luo: > Hi all, > > I'm thinking of ways to minimise the work needed to fix the build path > issues that we're having. > > The present method is that dpkg-buildflags sets > "-fdebug-prefix-map=$PWD=." into various *FLAGS envvars, and Make (or > another build tool) will pass this to gcc. This