Re: llvm: -python: adjust RDEP

2020-03-18 Thread Klemens Nanni 
On Wed, Mar 18, 2020 at 03:40:47PM +0100, Jeremie Courreges-Anglas wrote:
> I got curious about which problems could result from the current deps
> layout.  So here's an attempt to switch all the llvm subpackages to
> python3.
Thanks a lot!

> While here:
> - also byte-compile the .py files in the lldb subpackage.  compileall.py
>   tripped up over symbolication.py so process this file with 2to3
> - drop gtest from PLIST and rdeps.  To be committed seperately, but
>   might as well ride this change.
OK for gtest removal now, the rest I'd like to build and run through
fake/update-plist/etc. myself first (build takes long!)

> make update works fine with a py-llvm->py3-llvm quirks entry.
> import lldb, lldb.formatters, lldb.runtime, lldb.utils, clang and
> clang.cindex don't explode in flight, scan-view also works
> fine, hmaptool and git-clang-formatter *seem* to be happy.
Sounds very good, I'll try some runtime tests as well.
  
  
>  post-build:
> - cd ${WRKSRC}/docs && make -f Makefile.sphinx man
> - cd ${WRKSRC}/tools/clang/docs && make -f Makefile.sphinx man
> + cd ${WRKSRC}/docs && \
> + make -f Makefile.sphinx man 
> SPHINXBUILD="sphinx-build${MODPY_BIN_SUFFIX}"
> + cd ${WRKSRC}/tools/clang/docs && \
> + make -f Makefile.sphinx man 
> SPHINXBUILD="sphinx-build${MODPY_BIN_SUFFIX}"
s/make/${MAKE_PROGRAM}/ I guess, also there's -C instead of cd.

Re: llvm: -python: adjust RDEP

2020-03-18 Thread Jeremie Courreges-Anglas 
On Mon, Mar 16 2020, Jeremie Courreges-Anglas  wrote:

[...]

> I took a look some days/weeks ago, sadly I didn't note down
> why the switch wasn't straightforward.  Maybe I was just busy with
> something else.  Looks like a bunch of python files import __future__ so
> this looks promising.

I got curious about which problems could result from the current deps
layout.  So here's an attempt to switch all the llvm subpackages to
python3.

While here:
- also byte-compile the .py files in the lldb subpackage.  compileall.py
  tripped up over symbolication.py so process this file with 2to3
- drop gtest from PLIST and rdeps.  To be committed seperately, but
  might as well ride this change.

make update works fine with a py-llvm->py3-llvm quirks entry.
import lldb, lldb.formatters, lldb.runtime, lldb.utils, clang and
clang.cindex don't explode in flight, scan-view also works
fine, hmaptool and git-clang-formatter *seem* to be happy.

Feedback / oks welcome.


Index: Makefile
===
RCS file: /cvs/ports/devel/llvm/Makefile,v
retrieving revision 1.245
diff -u -p -r1.245 Makefile
--- Makefile15 Mar 2020 19:33:32 -  1.245
+++ Makefile18 Mar 2020 13:28:06 -
@@ -16,11 +16,11 @@ DISTNAME =  llvm-${LLVM_V}.src
 PKGNAME =  llvm-${LLVM_V}
 PKGSPEC-main = llvm-=${LLVM_V}
 PKGNAME-main = llvm-${LLVM_V}
-PKGNAME-python =   py-llvm-${LLVM_V}
+PKGNAME-python =   py3-llvm-${LLVM_V}
 PKGNAME-lldb = lldb-${LLVM_V}
-REVISION-main =8
-REVISION-python = 0
-REVISION-lldb =1
+REVISION-main =9
+REVISION-python = 1
+REVISION-lldb =2
 
 CATEGORIES =   devel
 
@@ -49,13 +49,13 @@ MULTI_PACKAGES = -main -python -lldb
 COMPILER = base-clang ports-gcc
 MODULES =  devel/cmake \
lang/python
+MODPY_VERSION =${MODPY_DEFAULT_VERSION_3}
 
 TEST_DEPENDS = devel/dejagnu \
shells/bash
 BUILD_DEPENDS +=   devel/swig \
-   textproc/py-sphinx \
-   textproc/py-recommonmark
-RUN_DEPENDS += devel/gtest
+   textproc/py-sphinx${MODPY_FLAVOR} \
+   textproc/py-recommonmark${MODPY_FLAVOR}
 
 .include 
 
@@ -79,7 +79,7 @@ WANTLIB-lldb +=   curses edit form panel
 # XXX linked against libLLVM-x.so, should be LIB_DEPENDS
 RUN_DEPENDS-lldb = ${BUILD_PKGPATH},-main
 RUN_DEPENDS-lldb +=${MODPY_RUN_DEPENDS} \
-   devel/py-six
+   devel/py-six${MODPY_FLAVOR}
 LIB_DEPENDS-lldb = ${LIB_DEPENDS}
 
 CONFIGURE_STYLE =  cmake
@@ -127,6 +127,7 @@ post-extract:
mv ${WRKDIR}/cfe-${LLVM_V}.src ${WRKSRC}/tools/clang
mv ${WRKDIR}/lld-${LLVM_V}.src ${WRKSRC}/tools/lld
mv ${WRKDIR}/lldb-${LLVM_V}.src ${WRKSRC}/tools/lldb
+   ${LOCALBASE}/bin/2to3 -w 
${WRKSRC}/tools/lldb/examples/python/symbolication.py
 
 pre-configure:
@${SUBST_CMD} ${WRKSRC}/tools/clang/lib/Driver/ToolChains/OpenBSD.cpp \
@@ -134,8 +135,10 @@ pre-configure:
-@ln -s ${MODPY_BIN} ${WRKDIR}/bin/python
 
 post-build:
-   cd ${WRKSRC}/docs && make -f Makefile.sphinx man
-   cd ${WRKSRC}/tools/clang/docs && make -f Makefile.sphinx man
+   cd ${WRKSRC}/docs && \
+   make -f Makefile.sphinx man 
SPHINXBUILD="sphinx-build${MODPY_BIN_SUFFIX}"
+   cd ${WRKSRC}/tools/clang/docs && \
+   make -f Makefile.sphinx man 
SPHINXBUILD="sphinx-build${MODPY_BIN_SUFFIX}"
 
 post-install:
${INSTALL_DATA_DIR} \
@@ -143,7 +146,7 @@ post-install:
${INSTALL_DATA} ${WRKSRC}/tools/clang/bindings/python/clang/* \
${PREFIX}/lib/python${MODPY_VERSION}/site-packages/clang
${MODPY_BIN} ${MODPY_LIBDIR}/compileall.py \
-   ${PREFIX}/lib/python${MODPY_VERSION}/site-packages/clang
+   ${PREFIX}/lib/python${MODPY_VERSION}/site-packages/
${INSTALL_DATA} ${WRKSRC}/docs/_build/man/* \
${PREFIX}/man/man1
${INSTALL_DATA} ${WRKSRC}/tools/clang/docs/_build/man/clang.1 \
Index: pkg/PLIST-lldb
===
RCS file: /cvs/ports/devel/llvm/pkg/PLIST-lldb,v
retrieving revision 1.4
diff -u -p -r1.4 PLIST-lldb
--- pkg/PLIST-lldb  6 Mar 2020 14:39:57 -   1.4
+++ pkg/PLIST-lldb  18 Mar 2020 13:28:06 -
@@ -608,15 +608,29 @@ include/lldb/lldb-versioning.h
 @static-lib lib/liblldbUtilityHelpers.a
 lib/python${MODPY_VERSION}/site-packages/lldb/
 lib/python${MODPY_VERSION}/site-packages/lldb/__init__.py
+${MODPY_COMMENT}lib/python${MODPY_VERSION}/site-packages/lldb/${MODPY_PYCACHE}/
+lib/python${MODPY_VERSION}/site-packages/lldb/${MODPY_PYCACHE}__init__.${MODPY_PYC_MAGIC_TAG}pyc
+lib/python${MODPY_VERSION}/site-packages/lldb/${MODPY_PYCACHE}embedded_interpreter.${MODPY_PYC_MAGIC_TAG}pyc
 @so lib/python${MODPY_VERSION}/site-packages/lldb/_lldb.so

Re: llvm: -python: adjust RDEP

2020-03-16 Thread Jeremie Courreges-Anglas 
On Mon, Mar 16 2020, Klemens Nanni  wrote:
> On Mon, Mar 16, 2020 at 02:57:16PM +0100, Jeremie Courreges-Anglas wrote:
>> On Sun, Mar 15 2020, Klemens Nanni  wrote:
>> > All subpackages ave a RDEP on python, so explicitness for -python is
>> > redundant.
>> 
>> -python ships a true python module, IMO it should have an rdep on python
>> like other python modules in the tree.  Same goes for -lldb.
> Note that i'm certainly not removing the RDEP, it is already registered
> through the module.  I don't want to specify it twice because it is a)
> redundant and b) brings other problems.

In my PoV RUN_DEPENDS-python is explicit and doesn't bring other
problems.  Which problems do you have in mind?

>> -main ships a bunch of python scripts, but right now those scripts
>> aren't properly MODPY_ADJ'd.  I think it makes more sense to discuss
>> removing the python rdep in this subpackage.  scan-view for example is
>> a python script, thankfully scan-build alone is enough.
>> 
>> > More importantly, this uncovers how the devel/gtest RDEP has been
>> > clobbered by it;
>> 
>> The deps of the -python subpackage make sense to me as-is, without an
>> explicit dependency on devel/gtest.
> gtest already is an explicit RDEP for every subpackage through global
> RUN_DEPENDS, but that is another story of its own as you elaborated.

I don't understand this.  The -python and -lldb subpackage have explicit
rdeps that aren't affected by the global RUN_DEPENDS, only -main
inherits the global RUN_DEPENDS.

>> Not ok with this diff, but I'm happy with moving llvm to python3 if
>> possible.  I took a look some days/weeks ago, sadly I didn't note down
>> why the switch wasn't straightforward.  Maybe I was just busy with
>> something else.  Looks like a bunch of python files import __future__ so
>> this looks promising.
> For clarity, here are all RDEPs before
>
>   RUN_DEPENDS-main=devel/gtest lang/python/2.7
>   RUN_DEPENDS-python=lang/python/2.7
>   RUN_DEPENDS-lldb=devel/llvm,,-main lang/python/2.7 devel/py-six
>
> and after my diff:
>
>   RUN_DEPENDS-main=devel/gtest lang/python/2.7
>   RUN_DEPENDS-python=devel/gtest lang/python/2.7
>   RUN_DEPENDS-lldb=devel/llvm,,-main lang/python/2.7 devel/py-six
>
> Afterall, only gtest is added (and can be removed with a separate diff)
> once we have more insight.

So far I've tried to minimize churn in this port, that means try to
avoid changes if they don't bring a visible improvement.  Adding gtest
to RUN_DEPENDS-python is not an improvement per se.

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

Re: llvm: -python: adjust RDEP

2020-03-16 Thread Klemens Nanni 
On Mon, Mar 16, 2020 at 02:57:16PM +0100, Jeremie Courreges-Anglas wrote:
> On Sun, Mar 15 2020, Klemens Nanni  wrote:
> > All subpackages ave a RDEP on python, so explicitness for -python is
> > redundant.
> 
> -python ships a true python module, IMO it should have an rdep on python
> like other python modules in the tree.  Same goes for -lldb.
Note that i'm certainly not removing the RDEP, it is already registered
through the module.  I don't want to specify it twice because it is a)
redundant and b) brings other problems.

> -main ships a bunch of python scripts, but right now those scripts
> aren't properly MODPY_ADJ'd.  I think it makes more sense to discuss
> removing the python rdep in this subpackage.  scan-view for example is
> a python script, thankfully scan-build alone is enough.
> 
> > More importantly, this uncovers how the devel/gtest RDEP has been
> > clobbered by it;
> 
> The deps of the -python subpackage make sense to me as-is, without an
> explicit dependency on devel/gtest.
gtest already is an explicit RDEP for every subpackage through global
RUN_DEPENDS, but that is another story of its own as you elaborated.

> Not ok with this diff, but I'm happy with moving llvm to python3 if
> possible.  I took a look some days/weeks ago, sadly I didn't note down
> why the switch wasn't straightforward.  Maybe I was just busy with
> something else.  Looks like a bunch of python files import __future__ so
> this looks promising.
For clarity, here are all RDEPs before

RUN_DEPENDS-main=devel/gtest lang/python/2.7
RUN_DEPENDS-python=lang/python/2.7
RUN_DEPENDS-lldb=devel/llvm,,-main lang/python/2.7 devel/py-six

and after my diff:

RUN_DEPENDS-main=devel/gtest lang/python/2.7
RUN_DEPENDS-python=devel/gtest lang/python/2.7
RUN_DEPENDS-lldb=devel/llvm,,-main lang/python/2.7 devel/py-six

Afterall, only gtest is added (and can be removed with a separate diff)
once we have more insight.

Re: llvm: -python: adjust RDEP

2020-03-16 Thread Jeremie Courreges-Anglas 
On Sun, Mar 15 2020, Klemens Nanni  wrote:
> All subpackages ave a RDEP on python, so explicitness for -python is
> redundant.

-python ships a true python module, IMO it should have an rdep on python
like other python modules in the tree.  Same goes for -lldb.

-main ships a bunch of python scripts, but right now those scripts
aren't properly MODPY_ADJ'd.  I think it makes more sense to discuss
removing the python rdep in this subpackage.  scan-view for example is
a python script, thankfully scan-build alone is enough.

> More importantly, this uncovers how the devel/gtest RDEP has been
> clobbered by it;

The deps of the -python subpackage make sense to me as-is, without an
explicit dependency on devel/gtest.

> I cannot yet say what inside LLVM actually requires
> gtest(1) at runtime and the dependency was added in 2017 for the 4.0.1
> update, but we can fix this in a different commit.

Actually gtest was added in Makefile rev 1.71 for an llvm 3.5 snapshot,
the two lines below were added to pkg/PLIST for llvm 3.0 and are now
outdated.  I'll do a few more tests but I'm about to delete the gtest
rdep, the installed files only reference gtest in three places which
don't seem relevant.  Brad, do you know if gtest is still useful in
RUN_DEPENDS?


Index: pkg/PLIST-main
===
RCS file: /cvs/ports/devel/llvm/pkg/PLIST-main,v
retrieving revision 1.16
diff -u -p -r1.16 PLIST-main
--- pkg/PLIST-main  6 Mar 2020 14:39:57 -   1.16
+++ pkg/PLIST-main  16 Mar 2020 00:50:52 -
@@ -2455,8 +2455,6 @@ lib/cmake/llvm/VersionFromVCS.cmake
 @static-lib lib/libclangStaticAnalyzerCore.a
 @static-lib lib/libclangStaticAnalyzerFrontend.a
 @static-lib lib/libclangTooling.a
-@comment lib/libgtest.a
-@comment lib/libgtest_main.a
 @static-lib lib/libclangToolingASTDiff.a
 @static-lib lib/libclangToolingCore.a
 @static-lib lib/libclangToolingInclusions.a


> Eventually, I want to use Python 3 inside LLVM - this diff is a first
> step, but it already showed side effects, so let's go one by one.
>
>
> Feedback? OK?

Not ok with this diff, but I'm happy with moving llvm to python3 if
possible.  I took a look some days/weeks ago, sadly I didn't note down
why the switch wasn't straightforward.  Maybe I was just busy with
something else.  Looks like a bunch of python files import __future__ so
this looks promising.

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

llvm: -python: adjust RDEP

2020-03-15 Thread Klemens Nanni 
All subpackages ave a RDEP on python, so explicitness for -python is
redundant.

More importantly, this uncovers how the devel/gtest RDEP has been
clobbered by it;  I cannot yet say what inside LLVM actually requires
gtest(1) at runtime and the dependency was added in 2017 for the 4.0.1
update, but we can fix this in a different commit.

Eventually, I want to use Python 3 inside LLVM - this diff is a first
step, but it already showed side effects, so let's go one by one.


Feedback? OK?


Index: Makefile
===
RCS file: /cvs/ports/devel/llvm/Makefile,v
retrieving revision 1.244
diff -u -p -r1.244 Makefile
--- Makefile6 Mar 2020 22:22:03 -   1.244
+++ Makefile15 Mar 2020 18:47:04 -
@@ -19,7 +19,7 @@ PKGNAME-main =llvm-${LLVM_V}
 PKGNAME-python =   py-llvm-${LLVM_V}
 PKGNAME-lldb = lldb-${LLVM_V}
 REVISION-main =8
-REVISION-python = 0
+REVISION-python = 1
 REVISION-lldb =1
 
 CATEGORIES =   devel
@@ -71,7 +71,6 @@ LIB_DEPENDS-main =${LIB_DEPENDS}
 
 # clang python module loads libclang.so dynamically with cdll.LoadLibrary()
 WANTLIB-python =   clang
-RUN_DEPENDS-python =   ${MODPY_RUN_DEPENDS}
 LIB_DEPENDS-python =   ${BUILD_PKGPATH},-main
 
 WANTLIB-lldb = ${WANTLIB} ${MODPY_WANTLIB}