[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Adriaan de Groot  changed:

   What|Removed |Added

 Status|New |Closed
 Resolution|--- |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #23 from commit-h...@freebsd.org ---
A commit references this bug:

Author: adridg
Date: Fri Apr  5 11:46:46 UTC 2019
New revision: 497948
URL: https://svnweb.freebsd.org/changeset/ports/497948

Log:
  Update CMake to latest release, 3.14.1

  Changelog: https://cmake.org/cmake/help/v3.14/release/3.14.html
  Local patches: fixes for Boost, Python, and OpenMP

  Affected ports:
   - opencpn, hugin needed help in finding wx
   - kadu is over-enthusiastic in finding non-existent X11 components
   - xlife++ does weird things in parsing help output from cmake

  Thanks antoine@ for multiple exp-runs.
  Thanks tcberner@ for much prep-work.

  PR:   236534 223678 227428

Changes:
  head/astro/opencpn/files/patch-CMakeLists.txt
  head/devel/cmake/Makefile
  head/devel/cmake/distinfo
  head/devel/cmake/files/patch-Modules_FindBoost.cmake
  head/devel/cmake/files/patch-Modules_FindLua.cmake
  head/devel/cmake/files/patch-Modules_FindOpenMP.cmake
  head/devel/cmake/files/patch-git_ff06b35
  head/devel/cmake/pkg-plist
  head/devel/cmake-doc/Makefile
  head/devel/cmake-doc/pkg-plist
  head/devel/cmake-gui/Makefile
  head/graphics/hugin/files/patch-sf_f6f56eb9
  head/math/xlife++/files/patch-cmake_xlifepp__user__management.cmake
  head/polish/kadu/files/patch-kadu-core_CMakeLists.txt
  head/polish/kadu/files/patch-plugins_idle_CMakeLists.txt
  head/polish/kadu/files/patch-plugins_pcspeaker_CMakeLists.txt
  head/polish/kadu/files/patch-plugins_screenshot_CMakeLists.txt

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

 Blocks||227427


Referenced Bugs:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227427
[Bug 227427] devel/boost-all: Update to 1.67.0
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   See Also||https://github.com/rdkit/rd
   ||kit/pull/1864

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   See Also||https://github.com/jrl-umi3
   ||218/jrl-cmakemodules/pull/1
   ||30

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   See Also||https://github.com/mantidpr
   ||oject/mantid/pull/22305

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   See Also||https://github.com/esa/pagm
   ||o2/pull/169

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   See Also||https://github.com/FreeCAD/
   ||FreeCAD/pull/1415

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #22 from Willem Jan Withagen  ---
(In reply to Willem Jan Withagen from comment #21)

Must have been a pilot error, since I reloaded ports into the jail.
And now poudriere just builds fine. And I'll submit an upgrade.

Note that I can compile ceph/HEAD without your patch, so I'll test your
boost-patch to see what the compiler complains about, and use that to submit a
PR to Ceph if needed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #21 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #20)

I update the ports tree with

poudriere ports -p local12 -u

So where would I find the tree/database to run pkg against?

Or does that always run against the pkg.conf in /usr/local/etc?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #20 from Jan Beich  ---
(In reply to Willem Jan Withagen from comment #19)
> Error: /usr/local/bin/ceph-mgr is linked to 
> /usr/local/lib/libboost_python.so.1.66.0 from devel/boost-python-libs but it 
> is not declared as
 a dependency
> Warning: you need LIB_DEPENDS+=libboost_python.so:devel/boost-python-libs

Did you use an old ports/ tree?
https://lists.freebsd.org/pipermail/svn-ports-head/2018-March/170962.html
https://svnweb.freebsd.org/changeset/ports/463782

> Which also shows the "main problem". it is still linked against Boost 1.66.

HEAD has Boost 1.67.0
2018Q2 has Boost 1.66.0
2017Q4 and 2018Q1 has Boost 1.65.1
2017Q3 has Boost 1.64.0
...

If "pkg -vv | fgrep url" returns /quarterly at the end then you're on 2018Q2.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #19 from Willem Jan Withagen  ---
(In reply to Willem Jan Withagen from comment #17)

Hi Jan,

There is an update to the luminous tree: 12.2.5.
So I ran the current net/ceph port through poudriere after fixing the version
and distinfo. 

That makes it compile without much problems.
 - virtenv is indeed not needed.
 - boost_pythons complains with an error:
---
Error: /usr/local/bin/ceph-mgr is linked to
/usr/local/lib/libboost_python.so.1.66.0 from devel/boost-python-libs but it is
not declared as
 a dependency
Warning: you need LIB_DEPENDS+=libboost_python.so:devel/boost-python-libs
---

Which also shows the "main problem". it is still linked against Boost 1.66.
That even though I portsnapped to update the complete ports tree for the
poudriere  env.

Is this expected?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #18 from Willem Jan Withagen  ---
(In reply to Willem Jan Withagen from comment #17)

Yesterday I had some time to work a bit on this, and with boost 1.67 and CMake
3.11.0_2 I was able to build and test correctly.
This was using a default environment based on python 2.7, even though Ceph
starts to move to python3. (I have both installed)

So a challenge might be to have a poudriere env that will be able to have both
python 2.7 AND python3? As I have the feeling that this is not really what the
ports team is aiming for?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #17 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #16)

extracts from the Cpeh-python discussion:

Mimic - after a "best effort" to migrate, some Python code still runs on Python
2, but Shaman has a Python 3 build flavor and teuthology is capable of testing
in Python 3-only environment.

Nautilus - all remaining packaged Python code has been migrated and explicitly
runs (and is tested) on Python 3. Python 2 versions of our code do not need to
be provided or packaged, because all target platforms support Python 3. There
might still be some Python 2 scripts in the git repo, but these are either
special cases or for developer use only, and are not packaged. 

(These are the 2 upcoming releases)

And also:
> I'm not objecting to making the leap (it's better than trying to
> support 2 and 3 at the same time!), but let's make sure we understand
> consequences for users.

Meaning that Ceph never intended to have both a 2.7 and 3.x python version.
And it is currently 2.7 for most of the consumer facing items.
Some of the test parts are actually already 3.x compatible, and or running on
3.x

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #16 from Jan Beich  ---
Boost 1.67 has landed. Try commenting out -DBOOST_PYTHON_SUFFIX in
Mk/Uses/python.mk then build devel/libarea in order to better understand CMake
3.11.0 shortcomings.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #15 from commit-h...@freebsd.org ---
A commit references this bug:

Author: jbeich
Date: Wed Apr 18 13:58:16 UTC 2018
New revision: 467712
URL: https://svnweb.freebsd.org/changeset/ports/467712

Log:
  devel/cmake: allow consumers to qualify boost_python version

  PR:   227428
  Exp-run by:   antoine

Changes:
  head/Mk/Uses/python.mk
  head/devel/cmake/Makefile
  head/devel/cmake/files/patch-Modules_FindBoost.cmake

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #14 from Jan Beich  ---
(In reply to Jan Beich from comment #13)
> If "pythonXY" is used then Boost < 1.67 users are out of luck.

Err, mainly FreeBSD Ports users. According to FindBoost.cmake some
distributions (Debian/Ubuntu, Gentoo, RPMs) implemented their own versioning
*before* Boost 1.67.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #13 from Jan Beich  ---
(In reply to Adriaan de Groot from comment #11)
> how much of that would just go away if we fix the CMake module?

None. My ${PY_BOOST_LIB} fixes are for non-CMake ports or CMake ports with
their own way to find Boost.Python.

> how much goes away by itself with CMake 3.11.0?

None. CMake added "pythonXY" (versioned) support as Boost component. If a
project doesn't switch from "python" to "pythonXY" CMake won't find
libboost_pythonXY installed by Boost 1.67. If "pythonXY" is used then Boost <
1.67 users are out of luck. Obviously, downstream (e.g., FreeBSD Ports) maybe
unhappy with whatever Python version was requested, often preferring a more
recent one.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #12 from Willem Jan Withagen  ---
(In reply to Adriaan de Groot from comment #11)

If the fix in the Cmake stuff (either FreeBSD specific or not) allows for
this call
find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED)
to do "the right thing" (tm), then No changes to the ceph-source are needed.

As things go now I'll have to add a selector on FreeBSD to append the
python-version. No biggie, but again specific code for a specific problem.

That said, to has started with a discussion on ceph-dev to determine what to do
with 2.7 and how fast to fase it out, and make all 3.x code. 

That could/would leave FreeBSD users that are mainly on 2.7 in the cold.
Which should not be a major problem from opperational point, since you do not
want to do much other on the ceph storage nodes. It could however conflict on
compute nodes where scripting depends on 2.7 and 2.7 compatible classes.

So I'm driven here by what Ceph does in the code, and by FreeBSD packages in
what is possible and desirable. And that in the negative time I have available
ATM.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #11 from Adriaan de Groot  ---
I see jbeich@ committing lots of workarounds for Boost::Python 1.67 .. how much
of that would just go away if we fix the CMake module? Also, how much goes away
by itself with CMake 3.11.0? There's a specific commit in there

FindBoost: Add support for Boost 1.67 with Python version suffixes

Which, if it doesn't work, needs fixing on our (my, CMake) end.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

   Keywords|needs-patch |patch

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #10 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #9)

Oke, I understand the solution, but the Ceph Cmake code is:
  list(APPEND BOOST_COMPONENTS python)
  ..
  find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED)

Which is there because of all kinds of reasons, mainly having to do with
different distro and versions of distros holding boost libs with a too low
version.

So from you remark I conclude that this code WILL require handholding.
Even after things are patched?

I do expect to see some python version in Ceph in the near future, because they
are running into 2.7 <> 3.x issues as well.

The other compilation errors are a "problem" as I have very little time at the
moment. Trying to get $WORK projects finished before I have a little bit of
R

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #9 from Jan Beich  ---
If -DBOOST_PYTHON_SUFFIX=27 is passed "find_package(Boost COMPONENTS python)"
will look not only for libboost_python.so but also libboost_python27.so.
Ideally, CMake should try Python_ADDITIONAL_VERSIONS for Boost then fall back
to common suffixes. Requiring ugly Boost.Python ifdefs in consumers is
non-starter unless a specifc Boost version is explicitly requested i.e.,
"find_package(Boost 1.67 ...)". And consumers cannot jump on the latest Boost
version without alienating downstream[1].

[1] https://repology.org/metapackage/boost/versions

(In reply to Willem Jan Withagen from comment #8)
> In which case I'll fix the Cmake build to differentiate on this.

No need. With the patch net/ceph makes past "configure" stage until it hits C++
bustage (a la ports r467364).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #8 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #7)

So to make sure:
`CMAKE_ARGS+=   -DBOOST_PYTHON_SUFFIX:STRING=${PYTHON_SUFFIX}`
gets automagically added to the Cmake args if USES=python is set.

In which case I'll fix the Cmake build to differentiate on this.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

maintainer-approval requested: [Bug 227428] devel/cmake: fails to find suffixed libboost_python : [Attachment 192545] workaround

[bugzilla-noreply]

Jan Beich <jbe...@freebsd.org> has asked k...@freebsd.org for
maintainer-approval:
Bug 227428: devel/cmake: fails to find suffixed libboost_python
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Attachment 192545: workaround
https://bugs.freebsd.org/bugzilla/attachment.cgi?id=192545=edit



--- Comment #7 from Jan Beich <jbe...@freebsd.org> ---
Created attachment 192545
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=192545=edit
workaround

(In reply to Willem Jan Withagen from comment #6)
> Is there a ENV during CMake-ing that will tell the wanted Python version?

Like this? ;)

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #6 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #5)

Is there a ENV during CMake-ing that will tell the wanted Python version?

That will allow me to request either 2.7 or 3.x during CMake
Which will atleast work when libboost_python27.so is requested.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #5 from Jan Beich  ---
CMake 3.11.0 doesn't help.
net/ceph: https://ptpb.pw/dFRs
math/kig: https://ptpb.pw/UUwO

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Adriaan de Groot  changed:

   What|Removed |Added

 CC||adr...@freebsd.org

--- Comment #4 from Adriaan de Groot  ---
There is a CMake 3.11 update in exp-run, and I hear from tcberner@ that there
is a hefty rewrite in FindBoost (again) that specifically tackles Python. So
I'll check if that rewrite is in 3.11.0 release. If it is, this PR may go away
automagically.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #3 from Willem Jan Withagen  ---
(In reply to Willem Jan Withagen from comment #2)

The orignal code in Ceph is like:
  list(APPEND BOOST_COMPONENTS python)
  ..
  find_package(Boost 1.66 COMPONENTS ${BOOST_COMPONENTS} REQUIRED)

And then the CMAKE modules do their stuff.

Asking for a different python version there fixes the issue when there is that
version library.
So it is fixable from Ceph, but I strongly think it needs to be fixed in the
actual CMake code that tries to find the libboost_python{PY_VER}.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

--- Comment #2 from Willem Jan Withagen  ---
(In reply to Jan Beich from comment #1)

Hi Jan,

I think that the fallout stems from Cmake not understandig that the library is
not called libboost_python any longer, so perhaps the fix should go into the
Cmake module that tries to find and there account for the python-versioning.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug 227428] devel/cmake: fails to find suffixed libboost_python

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227428

Jan Beich  changed:

   What|Removed |Added

Summary|[cmake] net/ceph: fails to  |devel/cmake: fails to find
   |build with boost 1.67   |suffixed libboost_python

--- Comment #1 from Jan Beich  ---
More ports are affected this time.

$ make -C/usr/ports/math/kig
[...]
-- Could NOT find BoostPython (missing: BoostPython_LIBRARIES
BoostPython_INCLUDE_DIRS) 
[...]
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: share/katepart5/syntax/python-kig.xml
[...]
https://ptpb.pw/RMyh

-- 
You are receiving this mail because:
You are on the CC list for the bug.