On 2019/10/20 21:58, Branko Čibej wrote:
On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
On 2019/10/20 8:37, Branko Čibej wrote:
On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
On 2019/10/20 6:59, Branko Čibej wrote:
On 19.10.2019 23:06, Branko Čibej wrote:
On 19.10.2019 19:55, Branko Čibej wrote:
On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
On 2019/10/18 8:39, Branko Čibej wrote:
On 17.10.2019 23:46, Branko Čibej wrote:
On 17.10.2019 23:01, Daniel Shahaf wrote:
Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
On 17.10.2019 22:37, Daniel Shahaf wrote:
Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
On 16.10.2019 10:01, Daniel Shahaf wrote:
Which reminds me: it'd be nice to set up a buildbot to
run the
build and
test process under Python 3. Any committer can edit the
buildbot
scripts[1], but the question is which of the buildbot slaves
has Python 3
installed?
https://ci.apache.org/builders/svn-x64-macosx-local-python3
Been there, done that, bought the DVD.
Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
on that
bot? They (currently) fail under python3.
I don't know. It's possible, I suppose, that the activation of
the
python3 virtual environment has no effect on the test
driver. It
might
not be a bad idea to have the tests print the Python version
in the
summary and possibly in the log of every test case.
What's the value of $(PYTHON) in Makefile? That's what the
«check»
target uses.
Yep, apparently that's the bug ... I'm testing script changes
now,
along
with r1868566 for good measure.
I've set up this:
https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
It will build and test the core libraries and swig-py bindings
with
Python 3 on the swig-py3 branch.
I hope ... build #0 running as we speak.
Unfortunately build #2, which ran after upgrading SWIG and Python,
failed
to build SWIG Python bindings because of SWIG 4.0, as reported on
SVN-4818.
This also affects on trunk with SWIG 4.0.
(e.g.
https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
produced,
but the modules don't work correctly.
It seems they were caused by incompatibility of Python code for
proxy
object generated by SWIG, and it can not be resolved so simple....
(importlib module vs simply use 'import', absense of
_swig_setattr()
and _swig_getattr(), etc.)
My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
Hm, that did not really help. On the swig-py3 branch:
/bin/sh: line 1: 62481 Segmentation fault: 11
/srv/virtualenv-3/bin/python
/srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
make: *** [check-swig-py] Error 139
So, the Python bindings seem to have crashed the Python 3
interpreter on
macOS?
Yes, they did:
Process 28278 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason =
EXC_BAD_ACCESS (code=1, address=0x48)
frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
Python`PyErr_Restore:
-> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
Target 0: (Python) stopped.
frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
40156 #endif
40157
-> 40158 md = d = PyModule_GetDict(m);
40159 (void)md;
40160
40161 SWIG_InitializeModule(0);
(lldb) print m
(PyObject *) $0 = 0x000000010279ad70
(lldb) print *m
(PyObject) $1 = {
ob_refcnt = 785
ob_type = 0x000000010026ea30
}
I guess the _core module linked incompatible Python library to execute.
on
https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
:
[[[
checking for compiling Python extensions... clang
checking for linking Python extensions... clang -bundle -undefined
dynamic_lookup -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
-F/usr/local/opt/python/Frameworks -framework Python
checking for linking Python libraries... -bundle -undefined
dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
]]]
Good catch. This certainly looks correct, it's where Homebrew installs
python. However ...
$ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
/opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
/System/Library/Frameworks/Python.framework/Versions/2.7/Python
(compatibility version 2.7.0, current version 2.7.10)
...
That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
finds the Python 3 installation.
(Actually that would be DYLD_FRAMEWORK_PATH.)
... or have to use Run-Path Dependent Libraries feature on link time,
if DYLD_LIBRARY_PATH cannot work well due to SIP.
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
The link option in log above comes from result of
'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
or add option to configure to pass Python dynamic library path?
Using RPATH for the Python framework on macOS would be the right
solution, but it doesn't solve the problem. It looks like we need some
magic incantation on macOS to explicitly pick the Python 3 framework;
using just the -F option doesn't work as expected, the path to the
system default Python 2.7 framework is hard-coded in to the extension
libraries.
Umm.... it seems extension libraries actually don't need to link
"Python" dynamic library.
e.g.
$ otool -L `python -c 'import _socket;print(_socket.__file__)'`
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_socket.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
_core.so built on FreeBSD:
$ ldd subversion/bindings/swig/python/.libs/_core.so
subversion/bindings/swig/python/.libs/_core.so:
libsvn_swig_py-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py/.libs/libsvn_swig_py-1.so.0
(0x801298000)
libsvn_client-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_client/.libs/libsvn_client-1.so.0
(0x8014ac000)
libsvn_wc-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_wc/.libs/libsvn_wc-1.so.0
(0x801730000)
libsvn_diff-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_diff/.libs/libsvn_diff-1.so.0
(0x8019d5000)
libsvn_ra-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra/.libs/libsvn_ra-1.so.0
(0x801bec000)
libsvn_ra_local-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_local/.libs/libsvn_ra_local-1.so.0
(0x801df8000)
libsvn_repos-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_repos/.libs/libsvn_repos-1.so.0
(0x802001000)
libsvn_fs-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs/.libs/libsvn_fs-1.so.0
(0x80223c000)
libsvn_fs_fs-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_fs/.libs/libsvn_fs_fs-1.so.0
(0x802449000)
libsvn_fs_x-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_x/.libs/libsvn_fs_x-1.so.0
(0x802698000)
libsvn_fs_util-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_util/.libs/libsvn_fs_util-1.so.0
(0x8028e7000)
libsvn_ra_svn-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_svn/.libs/libsvn_ra_svn-1.so.0
(0x802ae9000)
libsvn_ra_serf-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_serf/.libs/libsvn_ra_serf-1.so.0
(0x802d0d000)
libserf-1.so.1 => /usr/local/lib/libserf-1.so.1 (0x802f3e000)
libsvn_delta-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_delta/.libs/libsvn_delta-1.so.0
(0x803159000)
libsvn_subr-1.so.0 =>
/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_subr/.libs/libsvn_subr-1.so.0
(0x803377000)
libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x803608000)
libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x803835000)
libz.so.6 => /lib/libz.so.6 (0x803a5f000)
libsqlite3.so.0 => /usr/local/lib/libsqlite3.so.0 (0x803c77000)
libmagic.so.4 => /usr/lib/libmagic.so.4 (0x803fd9000)
liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x8041fc000)
libutf8proc.so.2 => /usr/local/lib/libutf8proc.so.2 (0x80442b000)
libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x804675000)
libintl.so.8 => /usr/local/lib/libintl.so.8 (0x8048af000)
libthr.so.3 => /lib/libthr.so.3 (0x804aba000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libssl.so.8 => /usr/lib/libssl.so.8 (0x804ce2000)
libcrypto.so.8 => /lib/libcrypto.so.8 (0x805000000)
libcrypt.so.5 => /lib/libcrypt.so.5 (0x80546f000)
libdb-5.3.so.0 => /usr/local/lib/libdb-5.3.so.0 (0x80568e000)
libgdbm.so.6 => /usr/local/lib/libgdbm.so.6 (0x805a33000)
--
Yasuhito FUTATSUKI <futat...@poem.co.jp>
