Re: [gem5-dev] annoying rebuild of swig wrappers during NULL build

[Andreas Hansson]

Note that Swig is about to be removed and replaced with Pybind11. Perhaps
the problem will still be there, but don’t invest time if that’s not the
case.

Andreas

On 22/04/2017, 00:55, "gem5-dev on behalf of Steve Reinhardt"
 wrote:

>Yup, I remember getting frustrated by this and looking into it and
>realizing that the problem is that the dependencies get reordered. Seemed
>like the right thing would be to sort the dependencies at the right point
>so that the insertion order doesn't matter, but I never figured out when
>or
>how to do that, so I gave up. Best wishes on finding a solution!
>
>Steve
>
>
>On Fri, Apr 21, 2017 at 4:36 PM, Gabe Black  wrote:
>
>> Hi folks. I've been looking into why the swig wrappers are all rebuilt
>>even
>> after a successful build when nothing has changed. I think I know why,
>>but
>> I'm not quite sure how to fix it.
>>
>> Basically, when scons scans the files looking for dependencies using the
>> ClassicCPP scanner class (which has been used to build a swig scanner),
>>it
>> will recursively look through dependencies which are identified in an
>> earlier pass. This makes sense for regular-ole-C, where that might be
>>the
>> only way to identify dependencies from nested includes. If it gets to a
>> point where there's an include and there isn't a file for it yet
>>(because
>> it's a generated file probably), then it will stop recursing. Later on,
>> when that file is actually generated, its dependencies will be scanned
>>for,
>> and will transitively become dependencies for the original file (I
>>think).
>>
>> This leads to a problem where the files involved don't exist in the
>>first
>> build, but older versions of them do exist in subsequent builds. The
>>first
>> build will stop recursing when it hits those files and their
>>dependencies
>> won't be added. The later builds will be able to recurse into those
>>files,
>> however, and they'll add those dependencies right away. The list of
>> dependencies are correct in either case (at least as far as I can tell),
>> but because they're added in a different way the order is different.
>> Because scons is really picky, the dependency ordering change causes
>>those
>> files to be "rebuilt", resulting in an identical tree.
>>
>> Subsequent NULL builds will have the same dependency ordering as the
>>second
>> build, assuming nothing else has changed, and so the annoying swig
>>rebuild
>> won't happen.
>>
>> At this point I'm sort of at a loss as to what to do, because this
>>seems to
>> just be something broken in how scons handles scanning for dependencies
>>in
>> generated files. If anybody has any ideas, please let me know. This
>>isn't
>> anything that urgently needs fixing, but it would be nice to knock off
>>that
>> rough edge.
>>
>> Gabe
>> ___
>> gem5-dev mailing list
>> gem5-dev@gem5.org
>> http://m5sim.org/mailman/listinfo/gem5-dev
>___
>gem5-dev mailing list
>gem5-dev@gem5.org
>http://m5sim.org/mailman/listinfo/gem5-dev

IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] annoying rebuild of swig wrappers during NULL build

[Steve Reinhardt]

Yup, I remember getting frustrated by this and looking into it and
realizing that the problem is that the dependencies get reordered. Seemed
like the right thing would be to sort the dependencies at the right point
so that the insertion order doesn't matter, but I never figured out when or
how to do that, so I gave up. Best wishes on finding a solution!

Steve


On Fri, Apr 21, 2017 at 4:36 PM, Gabe Black  wrote:

> Hi folks. I've been looking into why the swig wrappers are all rebuilt even
> after a successful build when nothing has changed. I think I know why, but
> I'm not quite sure how to fix it.
>
> Basically, when scons scans the files looking for dependencies using the
> ClassicCPP scanner class (which has been used to build a swig scanner), it
> will recursively look through dependencies which are identified in an
> earlier pass. This makes sense for regular-ole-C, where that might be the
> only way to identify dependencies from nested includes. If it gets to a
> point where there's an include and there isn't a file for it yet (because
> it's a generated file probably), then it will stop recursing. Later on,
> when that file is actually generated, its dependencies will be scanned for,
> and will transitively become dependencies for the original file (I think).
>
> This leads to a problem where the files involved don't exist in the first
> build, but older versions of them do exist in subsequent builds. The first
> build will stop recursing when it hits those files and their dependencies
> won't be added. The later builds will be able to recurse into those files,
> however, and they'll add those dependencies right away. The list of
> dependencies are correct in either case (at least as far as I can tell),
> but because they're added in a different way the order is different.
> Because scons is really picky, the dependency ordering change causes those
> files to be "rebuilt", resulting in an identical tree.
>
> Subsequent NULL builds will have the same dependency ordering as the second
> build, assuming nothing else has changed, and so the annoying swig rebuild
> won't happen.
>
> At this point I'm sort of at a loss as to what to do, because this seems to
> just be something broken in how scons handles scanning for dependencies in
> generated files. If anybody has any ideas, please let me know. This isn't
> anything that urgently needs fixing, but it would be nice to knock off that
> rough edge.
>
> Gabe
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] annoying rebuild of swig wrappers during NULL build

[Gabe Black]

Hi folks. I've been looking into why the swig wrappers are all rebuilt even
after a successful build when nothing has changed. I think I know why, but
I'm not quite sure how to fix it.

Basically, when scons scans the files looking for dependencies using the
ClassicCPP scanner class (which has been used to build a swig scanner), it
will recursively look through dependencies which are identified in an
earlier pass. This makes sense for regular-ole-C, where that might be the
only way to identify dependencies from nested includes. If it gets to a
point where there's an include and there isn't a file for it yet (because
it's a generated file probably), then it will stop recursing. Later on,
when that file is actually generated, its dependencies will be scanned for,
and will transitively become dependencies for the original file (I think).

This leads to a problem where the files involved don't exist in the first
build, but older versions of them do exist in subsequent builds. The first
build will stop recursing when it hits those files and their dependencies
won't be added. The later builds will be able to recurse into those files,
however, and they'll add those dependencies right away. The list of
dependencies are correct in either case (at least as far as I can tell),
but because they're added in a different way the order is different.
Because scons is really picky, the dependency ordering change causes those
files to be "rebuilt", resulting in an identical tree.

Subsequent NULL builds will have the same dependency ordering as the second
build, assuming nothing else has changed, and so the annoying swig rebuild
won't happen.

At this point I'm sort of at a loss as to what to do, because this seems to
just be something broken in how scons handles scanning for dependencies in
generated files. If anybody has any ideas, please let me know. This isn't
anything that urgently needs fixing, but it would be nice to knock off that
rough edge.

Gabe
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev