Re: bazel 4.2 and 3.7 no longer building with native clang

[Chris Jones]

On 14/07/2022 12:23 pm, Steven Smith wrote:

I've conceded defeat to bazel. I've lost interest in the battle but I wish you 
luck if you wish to take it up...



My interest in bazel is the minimum necessary to get tensorflow-related 
projects working.


That was also my starting point. Unfortunately the rabbit hole proved to 
be very very deep..




FWIW, bazel 5 builds and works on macOS, but previous versions do not, and I 
strongly doubt whether any magic selection of compiler will get them to work: 
https://github.com/bazelbuild/bazel/issues/15876


The bazel port indeed provides a number of builds. The primary one which 
I used to try and keep up with the latest release, and a bunch of 
sub-ports that only exist because tensorflow usually has a maximum 
version it will support using. So the versions there are literally only 
there to support the versions previously needed. Please feel free to 
remove them as they are no longer used by any other ports, although note 
this is not currently the case, e.g.


Oberon ~ > port list depends:bazel-3.7
py37-dm-tree   @0.1.5  python/py-dm-tree
py37-tensorflow@2.6.0  python/py-tensorflow
py37-tensorflow-data-validation @0.28.0 
python/py-tensorflow-data-validation

py37-tensorflow-metadata   @0.29.0 python/py-tensorflow-metadata
py37-tensorflow-probability@0.12.1 
python/py-tensorflow-probability

py37-tfx-bsl   @0.28.1 python/py-tfx-bsl
py38-dm-tree   @0.1.5  python/py-dm-tree
py38-tensorflow@2.6.0  python/py-tensorflow
py38-tensorflow-data-validation @0.28.0 
python/py-tensorflow-data-validation

py38-tensorflow-metadata   @0.29.0 python/py-tensorflow-metadata
py38-tensorflow-probability@0.12.1 
python/py-tensorflow-probability

py38-tfx-bsl   @0.28.1 python/py-tfx-bsl
py39-dm-tree   @0.1.5  python/py-dm-tree
py39-tensorflow@2.6.0  python/py-tensorflow
py39-tensorflow-data-validation @0.28.0 
python/py-tensorflow-data-validation

py39-tensorflow-metadata   @0.29.0 python/py-tensorflow-metadata
py39-tensorflow-probability@0.12.1 
python/py-tensorflow-probability

py39-tfx-bsl   @0.28.1 python/py-tfx-bsl

you can try, but it might be those ports cannot just be moved to the 
latest bazel... The maximum allowed is usually a bit behind the most recent.


Chris



My takeaway is to avoid any version prior to version 5 (implying that the 
existing MacPorts bazel state is both broken and obsolete), and that only if 
absolutely necessary previous versions can be obtained from binary downloads 
from github.

Personally, I have also adopted the stance of avoiding bazel entirely unless 
absolutely necessary.


On Jul 14, 2022, at 05:54, Chris Jones  wrote:

Hi


On 13/07/2022 10:50 pm, Ryan Schmidt wrote:

On Jul 13, 2022, at 11:46, Steven Smith wrote:
How does one reconfigure the bazel Portfile to use a previous version of clang 
that has a chance of working to build previous versions of bazel, which are in 
turn necessary to build the tensor flow packages.

If certain compilers that MacPorts might choose will not build a port, add 
their names to compiler.blacklist and MacPorts will choose another compiler.
If you only want to exclude certain versions of a compiler (such as {clang < 700 
>= 900}), include the compiler_blacklist_versions portgroup.
If you've excluded all compilers MacPorts might choose, add a suitable compiler 
to compiler.fallback.


This is all correct from the MacPorts side, but bazel is a different beast. It 
goes out of its way to not use the 'standard' ways of selecting compilers etc. 
That is why the bazel PG and portfile has all the convoluted logic it does to 
try and enforce the MP choice. I know this logic is fragile, it always has been.

Personally, I've conceded defeat to bazel. I've lost interest in the battle but 
I wish you luck if you wish to take it up...

Chris

Re: bazel 4.2 and 3.7 no longer building with native clang

[Steven Smith]

> I've conceded defeat to bazel. I've lost interest in the battle but I wish 
> you luck if you wish to take it up...


My interest in bazel is the minimum necessary to get tensorflow-related 
projects working.

FWIW, bazel 5 builds and works on macOS, but previous versions do not, and I 
strongly doubt whether any magic selection of compiler will get them to work: 
https://github.com/bazelbuild/bazel/issues/15876

My takeaway is to avoid any version prior to version 5 (implying that the 
existing MacPorts bazel state is both broken and obsolete), and that only if 
absolutely necessary previous versions can be obtained from binary downloads 
from github.

Personally, I have also adopted the stance of avoiding bazel entirely unless 
absolutely necessary.

> On Jul 14, 2022, at 05:54, Chris Jones  wrote:
> 
> Hi
> 
>> On 13/07/2022 10:50 pm, Ryan Schmidt wrote:
>>> On Jul 13, 2022, at 11:46, Steven Smith wrote:
>>> How does one reconfigure the bazel Portfile to use a previous version of 
>>> clang that has a chance of working to build previous versions of bazel, 
>>> which are in turn necessary to build the tensor flow packages.
>> If certain compilers that MacPorts might choose will not build a port, add 
>> their names to compiler.blacklist and MacPorts will choose another compiler.
>> If you only want to exclude certain versions of a compiler (such as {clang < 
>> 700 >= 900}), include the compiler_blacklist_versions portgroup.
>> If you've excluded all compilers MacPorts might choose, add a suitable 
>> compiler to compiler.fallback.
> 
> This is all correct from the MacPorts side, but bazel is a different beast. 
> It goes out of its way to not use the 'standard' ways of selecting compilers 
> etc. That is why the bazel PG and portfile has all the convoluted logic it 
> does to try and enforce the MP choice. I know this logic is fragile, it 
> always has been.
> 
> Personally, I've conceded defeat to bazel. I've lost interest in the battle 
> but I wish you luck if you wish to take it up...
> 
> Chris

Re: bazel 4.2 and 3.7 no longer building with native clang

[Chris Jones]

Hi

On 13/07/2022 10:50 pm, Ryan Schmidt wrote:

On Jul 13, 2022, at 11:46, Steven Smith wrote:


How does one reconfigure the bazel Portfile to use a previous version of clang 
that has a chance of working to build previous versions of bazel, which are in 
turn necessary to build the tensor flow packages.


If certain compilers that MacPorts might choose will not build a port, add 
their names to compiler.blacklist and MacPorts will choose another compiler.

If you only want to exclude certain versions of a compiler (such as {clang < 700 
>= 900}), include the compiler_blacklist_versions portgroup.

If you've excluded all compilers MacPorts might choose, add a suitable compiler 
to compiler.fallback.



This is all correct from the MacPorts side, but bazel is a different 
beast. It goes out of its way to not use the 'standard' ways of 
selecting compilers etc. That is why the bazel PG and portfile has all 
the convoluted logic it does to try and enforce the MP choice. I know 
this logic is fragile, it always has been.


Personally, I've conceded defeat to bazel. I've lost interest in the 
battle but I wish you luck if you wish to take it up...


Chris

Re: bazel 4.2 and 3.7 no longer building with native clang

[Ryan Schmidt]

On Jul 13, 2022, at 11:46, Steven Smith wrote:

> How does one reconfigure the bazel Portfile to use a previous version of 
> clang that has a chance of working to build previous versions of bazel, which 
> are in turn necessary to build the tensor flow packages.

If certain compilers that MacPorts might choose will not build a port, add 
their names to compiler.blacklist and MacPorts will choose another compiler.

If you only want to exclude certain versions of a compiler (such as {clang < 
700 >= 900}), include the compiler_blacklist_versions portgroup.

If you've excluded all compilers MacPorts might choose, add a suitable compiler 
to compiler.fallback.

bazel 4.2 and 3.7 no longer building with native clang

[Steven Smith]

I’m trying to update a bunch of tensorflow packages, but am getting hung up on 
bazel issues (see https://trac.macports.org/ticket/64442 
).

I have a fix for this issue, but am unable to build either bazel 4.2 or 3.7 
with the native /usr/bin/clang compiler, chosen by the current Portfile 
settings. FWIW, the latest bazel version 5.2 (not yet updated into MacPorts) 
builds with this compiler.

How does one reconfigure the bazel Portfile to use a previous version of clang 
that has a chance of working to build previous versions of bazel, which are in 
turn necessary to build the tensor flow packages.

Right now I’m just hacking at the settings in the current Portfile, but each 
random hack just produces a different build failure.

E.g.
> compiler.blacklist-append {clang >= 900}  
>   
> compiler.blacklist-append {clang < 710}   
>   
> compiler.blacklist-append {macports-clang-[7-8].0}
>   
> set min_darwin16  
>   


> :info:build ../usr/bin/xcrun --sdk macosx /opt/local/bin/clang-mp-13 
> -mmacosx-version-min=10.9 -fobjc-arc -framework CoreServices -framework 
> Foundation -o 
> /opt/local/var/macports/build/_opt_local_ports_devel_bazel/bazel-4.2/work/.tmp/bazel_1HyYSmmI/archive/xcode-locator
>  tools/osx/xcode_locator.m
> :info:build ld: file not found: 
> /opt/local/libexec/llvm-13/lib/arc/libarclite_macosx.a
> :info:build clang: error: linker command failed with exit code 1 (use -v to 
> see invocation)

These binaries used to build. How does one determine the Xcode/llvm/compiler 
settings that will get them to build again?

smime.p7s
Description: S/MIME cryptographic signature