Re: Python3 work [was: The run up to Subversion 1.13.0]
On Fri, Oct 11, 2019 at 2:38 PM Yasuhito FUTATSUKI wrote: > > On 2019-09-30 06:32, Johan Corveleyn wrote: > ... > > At this point, I think the only thing blocking reintegration of this > > branch into trunk seems to be the C4115 warning / error. As I said, I > > have no informed opinion either way (and Brane was okay with turning > > it into a warning). So if Bert doesn't reply within a couple of days, > > I'd say we are good to go. > > ... and more than a week has past. So I've commited the patch address > to it on swig-py3 branch, as r1868290. Great. I've retested it all on Windows (once with Python 2.7.16, and once with Python 3.7.4) and it looks good to me. +1 to reintegrate the branch into trunk. -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019-09-30 06:32, Johan Corveleyn wrote: ... At this point, I think the only thing blocking reintegration of this branch into trunk seems to be the C4115 warning / error. As I said, I have no informed opinion either way (and Brane was okay with turning it into a warning). So if Bert doesn't reply within a couple of days, I'd say we are good to go. ... and more than a week has past. So I've commited the patch address to it on swig-py3 branch, as r1868290. Thanks, -- Yasuhito FUTATSUKI
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 29.09.2019 23:32, Johan Corveleyn wrote: > BTW, I have one more question: is it expected that "swig-python > bindings compiled with python 3" cannot be used by python 2? Because > if I run the testsuite with Python 2.7.16, after I have compiled them > with Python 3.7.4, I get the following error: > > [[[ > C:\research\svn\dev\swig-py3>python --version > Python 2.7.16 > > C:\research\svn\dev\swig-py3>runtest.bat --swig=python > Testing Release configuration on local repository. > -- Running Swig Python tests -- > Traceback (most recent call last): > File > "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py", > line 23, in > import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc, > repository, \ > File > "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\mergeinfo.py", > line 22, in > from svn import core, repos, fs > File > "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests/..\svn\core.py", > line 26, in > from libsvn.core import * > File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line 137 > def apr_initialize() -> "apr_status_t": > ^ > SyntaxError: invalid syntax > [Test runner reported failure] > ]]] > > So distributors / packagers that want to support both py2 and py3 > users, will have to create two sets of bindings. Yes, they will. Python 2 and 3 libraries have to be separate anyway, and the native bits of our bindings use different runtime libs. Interesting and also very nice that Swig uses PEP 484 type annotations. The Python 2 parser doesn't recognise those. -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Sun, Sep 29, 2019 at 8:42 PM Yasuhito FUTATSUKI wrote: ... > Hmm.. It seems the diffrence comes from difference of implementation > of class subprocess.Popen. > > I tweaked again clean up code for class svn.fs.FileDiff. Only if > Python >= 3.3, it wait for maximum 10 sec to terminate subprocess. > 'timeout' parameter for subprocess.Popen.wait() is added on Python 3.3. > and not back ported to Python 2, so if we want to use wait() on > earlier Python, it may wait forever (It can be possible to implement > timeout by using poll() and sleep loop, but is it needed here?) Okay, with that latest patch the testsuite runs fine now. Perfect. [[[ C:\research\svn\dev\swig-py3>runtest.bat --swig=python 'ruby' is not recognized as an internal or external command, operable program or batch file. Testing Release configuration on local repository. -- Running Swig Python tests -- . -- Ran 153 tests in 63.938s OK ]]] > > When I treat ResourceWarning as an error, I get a lot more output, > > especially before the test runs start. I suppose those are unrelated > > to that one RW above. > > Yes, ... and ResourceWarnings are usually hided by default warning filter. > > https://docs.python.org/3/library/warnings.html#default-warning-filter > > Only while executing unittest.main(), warning filter is overrided by > its optional parameter `warning'. Its default is specified filter on > Python start up by environment variable PYTHONWARNINGS or -W option > if it is specified, otherwise "default" action for all warning > categories (i.e. show all warnings). > > > [[[ > > C:\research\svn\dev\swig-py3>set PYTHONWARNINGS=error::ResourceWarning:: > > > > C:\research\svn\dev\swig-py3>runtest.bat --swig=python > > Exception ignored in: <_io.FileIO name='build.conf' mode='rb' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper name='build.conf' > > mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include\\apr_version.h' > > mode='rb' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include > > \\apr_version.h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\include\\apu_versi > > on.h' mode='rb' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\in > > clude\\apu_version.h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xml/expat/lib\\exp > > at.h' mode='rb' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xm > > l/expat/lib\\expat.h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='rb' > > closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='r' enco > > ding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqlite3.h' > > mode='rb' > > closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqli > > te3.h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='rb' > > closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='r' encoding=' > > cp1252'> > > Exception ignored in: <_io.FileIO > > name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' > > mode='rb' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' mode='r' > > encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.h' > > mode='rb' clo > > sefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release. > > h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > > name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/opensslv.h' > > mode='rb > > ' closefd=True> > > ResourceWarning: unclosed file <_io.TextIOWrapper > > name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/open > > sslv.h' mode='r' encoding='cp1252'> > > Exception ignored in: <_io.FileIO > >
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/29 23:36, Johan Corveleyn wrote: On Sun, Sep 29, 2019 at 2:00 PM Yasuhito FUTATSUKI wrote: On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote: On 2019/09/28 5:51, Johan Corveleyn wrote: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..C:\Python37\lib\subprocess.py:858: ResourceWarning: subprocess 8548 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name=3> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 66.182s OK ]]] On FreeBSD those warning are also produced. [[[ futatuki@retina-alpha[15] make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py ../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 10.773s OK ]]] I think these warnings comes from difference of warning between Python 2 and Python 3, and not affect quality of swig Python binding itself but affect quality of its test code. This was partly wrong. The warning caused by subprocess comes from class svn.fs.FileDiff.get_pipe(), not in test code but in support class. ResourceWarning itself appeared in Python 3.2 https://docs.python.org/3/whatsnew/3.2.html (section "Other Language Changes") If we can allow this warning and want to hide it, we can use environment variable PYTHONWARNINGS='ignore::ResourceWarning::', as the document above says. [[[ futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py . -- Ran 153 tests in 10.757s OK ]]] Of course, we can also resolve those warnings by fixing clean up code on tests. The patch attached is expected to fix these warnings. I already tested on FreeBSD with Python 3.7 (treat ResourceWarning as an error). [[[ futatuki@retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Sun, Sep 29, 2019 at 2:00 PM Yasuhito FUTATSUKI wrote: > > On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote: > > On 2019/09/28 5:51, Johan Corveleyn wrote: > > >> [[[ > >> Testing Release configuration on local repository. > >> -- Running Swig Python tests -- > >> ..C:\Python37\lib\subprocess.py:858: > >> ResourceWarning: subprocess > >> 8548 is still running > >>ResourceWarning, source=self) > >> ResourceWarning: Enable tracemalloc to get the object allocation traceback > >> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file > >> <_io.BufferedReader name=3> > >>testMethod() > >> ResourceWarning: Enable tracemalloc to get the object allocation traceback > >> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file > >> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'> > >>testMethod() > >> ResourceWarning: Enable tracemalloc to get the object allocation traceback > >> .. > >> -- > >> Ran 153 tests in 66.182s > >> > >> OK > >> ]]] > > > > On FreeBSD those warning are also produced. > > > > [[[ > > futatuki@retina-alpha[15] make check-swig-py > > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py > > > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; > > do if [ -n "$DYLD_LIBRARY_PATH" ]; then > > LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else > > LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; > > /usr/local/bin/python3.7 > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py > > ../usr/local/lib/python3.7/subprocess.py:858: > > ResourceWarning: subprocess 81136 is still running > >ResourceWarning, source=self) > > ResourceWarning: Enable tracemalloc to get the object allocation traceback > > /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed > > file <_io.BufferedReader name=4> > >testMethod() > > ResourceWarning: Enable tracemalloc to get the object allocation traceback > > ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed > > file <_io.BufferedReader name='/tmp/tmporh5038d'> > >testMethod() > > ResourceWarning: Enable tracemalloc to get the object allocation traceback > > .. > > -- > > Ran 153 tests in 10.773s > > > > OK > > ]]] > > > > I think these warnings comes from difference of warning between Python 2 > > and Python 3, and not affect quality of swig Python binding itself but > > affect quality of its test code. > > This was partly wrong. The warning caused by subprocess comes from > class svn.fs.FileDiff.get_pipe(), not in test code but in support class. > > > > > ResourceWarning itself appeared in Python 3.2 > > > > https://docs.python.org/3/whatsnew/3.2.html > > (section "Other Language Changes") > > > > If we can allow this warning and want to hide it, we can use > > environment variable PYTHONWARNINGS='ignore::ResourceWarning::', > > as the document above says. > > > > [[[ > > futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' > > make check-swig-py > > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py > > > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; > > do if [ -n "$DYLD_LIBRARY_PATH" ]; then > > LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else > > LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; > > /usr/local/bin/python3.7 > > /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py > > . > > -- > > Ran 153 tests in 10.757s > > > > OK > > ]]] > > > > Of course, we can also resolve those warnings by fixing clean up code > > on tests. > > The patch attached is expected to fix these warnings. I already tested > on FreeBSD with Python 3.7 (treat ResourceWarning as an error). > > [[[ > futatuki@retina-alpha[113] env
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote: On 2019/09/28 5:51, Johan Corveleyn wrote: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..C:\Python37\lib\subprocess.py:858: ResourceWarning: subprocess 8548 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name=3> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 66.182s OK ]]] On FreeBSD those warning are also produced. [[[ futatuki@retina-alpha[15] make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py ../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 10.773s OK ]]] I think these warnings comes from difference of warning between Python 2 and Python 3, and not affect quality of swig Python binding itself but affect quality of its test code. This was partly wrong. The warning caused by subprocess comes from class svn.fs.FileDiff.get_pipe(), not in test code but in support class. ResourceWarning itself appeared in Python 3.2 https://docs.python.org/3/whatsnew/3.2.html (section "Other Language Changes") If we can allow this warning and want to hide it, we can use environment variable PYTHONWARNINGS='ignore::ResourceWarning::', as the document above says. [[[ futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py . -- Ran 153 tests in 10.757s OK ]]] Of course, we can also resolve those warnings by fixing clean up code on tests. The patch attached is expected to fix these warnings. I already tested on FreeBSD with Python 3.7 (treat ResourceWarning as an error). [[[ futatuki@retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done;
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/28 22:36, Branko Čibej wrote: On 28.09.2019 11:20, Yasuhito FUTATSUKI wrote: On 2019/09/24 16:34, Branko Čibej wrote: On 23.09.2019 22:48, Johan Corveleyn wrote: On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI wrote: On 2019/09/23 6:16, Johan Corveleyn wrote: Building with Python 3.7.4 still fails with the same error though (no problem, I know your patch wasn't addressing that, just mentioning it here for completeness). Consequently I haven't been able to run the swig-python tests with python 3.7 yet. [[[ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj ]]] Here is a patch not to treat C4115 as error, globally. If this makes it possible to build with Python 3.7 on Windows, then next step can be to limit to apply this relaxation of compile option to files which contains "#include " (most of them are generated by swig...). Yes, that makes the build succeed, thanks. I can't comment on whether or not it's good to change this error into a warning overall, or just for a limited set of files. I'll leave that discussion to others :-). It should not be an error because the source is valid C, regardless of what Microsoft's compiler thinks about it. :) However it is ourselves to decide treat it as an error, on r876281. https://svn.apache.org/viewvc?view=revision=876281 Unfortunately I can't find any reason or discussion about this decision, yet. That's why I try to treat it carefully. (Of course there might be no serious reason, though) The most likely reason is that /our/ code should not contain a use of 'struct foo*' without at least a forward declaration or a typedef. That's a coding style decision that we shouldn't enforce on generated code (but it should be a warning, not an error). If you can turn off that compiler option only for the files generated by Swig, that would be great, but just turning it into a warning is good enough, IMO. I agree your opinion. I also think a warning is enough for this purpose. And, I found that C4055 is also once treated as an error on r876281, but not to be treated as error again on 876286 for generated swig python code, just same reason :) (Though C4055 seems obsolete on Visual Studio 2017 and later) https://svn.apache.org/viewvc?view=revision=876286 (Daniel, thank you for your suggestion about revision number offset) Cheers, -- Yasuhito FUTATSUKI
Re: Python3 work [was: The run up to Subversion 1.13.0]
Yasuhito FUTATSUKI wrote on Sat, 28 Sep 2019 09:20 +00:00: > https://svn.apache.org/viewvc?view=revision=876281 > > Unfortunately I can't find any reason or discussion about this decision, yet. Are you aware of the "+ 840074" revision number offset that needs to be applied to old revision numbers (those < 5) in our history? See https://svn.apache.org/repos/asf/subversion/README (sic) for details. In this case, it means that where r876281 (= r36207) says "r36160", "r876234" should be substituted: https://svn.apache.org/r876234 However, that patch doesn't seem related to the compiler diagnostic at hand. Bert, do you remember the rationale behind including C4115 in r876281? Cheers, Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 28.09.2019 11:20, Yasuhito FUTATSUKI wrote: > On 2019/09/24 16:34, Branko Čibej wrote: >> On 23.09.2019 22:48, Johan Corveleyn wrote: >>> On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI >>> wrote: On 2019/09/23 6:16, Johan Corveleyn wrote: > > Building with Python 3.7.4 still fails with the same error though (no > problem, I know your patch wasn't addressing that, just mentioning it > here for completeness). Consequently I haven't been able to run the > swig-python tests with python 3.7 yet. > > [[[ > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type > definition in parentheses > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj > > ]]] Here is a patch not to treat C4115 as error, globally. If this makes it possible to build with Python 3.7 on Windows, then next step can be to limit to apply this relaxation of compile option to files which contains "#include " (most of them are generated by swig...). >>> Yes, that makes the build succeed, thanks. I can't comment on whether >>> or not it's good to change this error into a warning overall, or just >>> for a limited set of files. I'll leave that discussion to others :-). >> >> It should not be an error because the source is valid C, regardless of >> what Microsoft's compiler thinks about it. :) > > However it is ourselves to decide treat it as an error, on r876281. > > https://svn.apache.org/viewvc?view=revision=876281 > > Unfortunately I can't find any reason or discussion about this > decision, yet. > That's why I try to treat it carefully. (Of course there might be no > serious reason, though) The most likely reason is that /our/ code should not contain a use of 'struct foo*' without at least a forward declaration or a typedef. That's a coding style decision that we shouldn't enforce on generated code (but it should be a warning, not an error). If you can turn off that compiler option only for the files generated by Swig, that would be great, but just turning it into a warning is good enough, IMO. -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/24 16:34, Branko Čibej wrote: On 23.09.2019 22:48, Johan Corveleyn wrote: On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI wrote: On 2019/09/23 6:16, Johan Corveleyn wrote: Building with Python 3.7.4 still fails with the same error though (no problem, I know your patch wasn't addressing that, just mentioning it here for completeness). Consequently I haven't been able to run the swig-python tests with python 3.7 yet. [[[ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj ]]] Here is a patch not to treat C4115 as error, globally. If this makes it possible to build with Python 3.7 on Windows, then next step can be to limit to apply this relaxation of compile option to files which contains "#include " (most of them are generated by swig...). Yes, that makes the build succeed, thanks. I can't comment on whether or not it's good to change this error into a warning overall, or just for a limited set of files. I'll leave that discussion to others :-). It should not be an error because the source is valid C, regardless of what Microsoft's compiler thinks about it. :) However it is ourselves to decide treat it as an error, on r876281. https://svn.apache.org/viewvc?view=revision=876281 Unfortunately I can't find any reason or discussion about this decision, yet. That's why I try to treat it carefully. (Of course there might be no serious reason, though) -- Yasuhito FUTATSUKI
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/28 5:51, Johan Corveleyn wrote: Great! That fixes this particular problem, and the test suite now runs successfully ... almost. There seems to be one more problem: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..C:\Python37\lib\subprocess.py:858: ResourceWarning: subprocess 8548 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name=3> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 66.182s OK ]]] On FreeBSD those warning are also produced. [[[ futatuki@retina-alpha[15] make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py ../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 10.773s OK ]]] I think these warnings comes from difference of warning between Python 2 and Python 3, and not affect quality of swig Python binding itself but affect quality of its test code. ResourceWarning itself appeared in Python 3.2 https://docs.python.org/3/whatsnew/3.2.html (section "Other Language Changes") If we can allow this warning and want to hide it, we can use environment variable PYTHONWARNINGS='ignore::ResourceWarning::', as the document above says. [[[ futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py . -- Ran 153 tests in 10.757s OK ]]] Of course, we can also resolve those warnings by fixing clean up code on tests. (The warning about 'ruby' is not a big deal I suppose, but it's also something I saw when running gen-make.py with python 3.7 -- not when I'm running it with python 2.7) I think that message comes from build/generator/gen_win_dependencies.py, GenDependenciesBase._find_ruby() line 967 (message from shell program on Windows). https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754=markup#l967 This may be also caused with Python 2, if 'ruby' is not in command search path. Thanks. I'll try to take a closer look, but it's not a priority. It's still a bit strange to me that Python 2 doesn't show this warning while Python 3 does
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Fri, Sep 27, 2019 at 8:14 PM Yasuhito FUTATSUKI wrote: > Thank you for testing. And thank you for hanging in there for the py3 work :-). > It seems that urllib.request.pathname2url() on Windows doesn't accept > bytes on Python 3, while on Linux/Unix it accepts both of bytes and str. > > The patch attached may fix it. Great! That fixes this particular problem, and the test suite now runs successfully ... almost. There seems to be one more problem: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..C:\Python37\lib\subprocess.py:858: ResourceWarning: subprocess 8548 is still running ResourceWarning, source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name=3> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'> testMethod() ResourceWarning: Enable tracemalloc to get the object allocation traceback .. -- Ran 153 tests in 66.182s OK ]]] > > (The warning about 'ruby' is not a big deal I suppose, but it's also > > something I saw when running gen-make.py with python 3.7 -- not when > > I'm running it with python 2.7) > > I think that message comes from build/generator/gen_win_dependencies.py, > GenDependenciesBase._find_ruby() line 967 (message from shell program > on Windows). > > https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754=markup#l967 > > This may be also caused with Python 2, if 'ruby' is not in command > search path. Thanks. I'll try to take a closer look, but it's not a priority. It's still a bit strange to me that Python 2 doesn't show this warning while Python 3 does (environment is the same, PATH is the same, except for Python itself). -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Thank you for testing.
On 2019/09/24 5:48, Johan Corveleyn wrote:
So, on to the next problem. When testing with Python 3.7.4 on Windows,
I get the following error:
[[[
C:\research\svn\dev\swig-py3>python win-tests.py -c --log-level=DEBUG
--release --swig=python R:\test_py
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
line 23, in
import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
repository, \
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\__init__.py",
line 23, in
from trac.versioncontrol.tests import svn_fs
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\svn_fs.py",
line 73, in
REPOS_URL = pathname2url(temp_path).encode('UTF-8')
File "C:\Python37\lib\nturl2path.py", line 53, in pathname2url
if not ':' in p:
TypeError: a bytes-like object is required, not 'str'
[Test runner reported failure]
]]]
It seems that urllib.request.pathname2url() on Windows doesn't accept
bytes on Python 3, while on Linux/Unix it accepts both of bytes and str.
The patch attached may fix it.
(The warning about 'ruby' is not a big deal I suppose, but it's also
something I saw when running gen-make.py with python 3.7 -- not when
I'm running it with python 2.7)
I think that message comes from build/generator/gen_win_dependencies.py,
GenDependenciesBase._find_ruby() line 967 (message from shell program
on Windows).
https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754=markup#l967
This may be also caused with Python 2, if 'ruby' is not in command
search path.
Cheers,
--
Yasuhito FUTATSUKI
On branch swig-py3: fix test for swig-py on Python 3 on Windows
[ in subversion/bindings/swig/python/tests/]
* trac/versioncontrol/tests/svn_fs.py (REPOS_PATH, REPOS_URL),
On Python 3, pass a str object as argument to urllib.request.pathname2url()
instead of a bytes.
* util.py (file_uri_for_path):
On Python 3, pass a str object as argument to urllib.request.pathname2url()
instead of a bytes even if the argment `path' is a bytes object.
Reported by: jcorvel
Index: subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py
===
--- subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py
(revision 1867212)
+++ subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py
(working copy)
@@ -68,8 +68,8 @@
from trac.versioncontrol import Changeset, Node
from trac.versioncontrol.svn_fs import SubversionRepository
-temp_path = tempfile.mktemp("-trac-svnrepos").encode('UTF-8')
-REPOS_PATH = core.svn_dirent_internal_style(temp_path)
+temp_path = tempfile.mktemp("-trac-svnrepos")
+REPOS_PATH = core.svn_dirent_internal_style(temp_path.encode('UTF-8'))
REPOS_URL = pathname2url(temp_path).encode('UTF-8')
del temp_path
Index: subversion/bindings/swig/python/tests/utils.py
===
--- subversion/bindings/swig/python/tests/utils.py (revision 1867212)
+++ subversion/bindings/swig/python/tests/utils.py (working copy)
@@ -79,7 +79,10 @@
def file_uri_for_path(path):
"""Return the file: URI corresponding to the given path."""
- uri_path = pathname2url(path).encode('UTF-8')
+ if isinstance(path, str):
+uri_path = pathname2url(path).encode('UTF-8')
+ else:
+uri_path = pathname2url(path.decode('UTF-8')).encode('UTF-8')
# pathname2url claims to return the path part of the URI, but on Windows
# it returns both the authority and path parts for no reason, which
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 23.09.2019 22:48, Johan Corveleyn wrote: > On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI > wrote: >> On 2019/09/23 6:16, Johan Corveleyn wrote: >>> On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI >>> wrote: On 2019/09/20 0:44, Daniel Shahaf wrote: > Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00: >> The FAIL on test_merge_peg3 is a newline style issue on reading file >> in raw mode, which is introduced by my patch. >> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438) >> >> Rest of those errors and failures are in the tests that were added only >> to swig-py, and it seems most of the causes of them are confusion of >> paths in platform specific format and Subversion's canoical format, >> as the comment in SubversionClientTestCase.test_update4() says. >> >> (I'll try to fix those issues on check-swig-py later, hopefully >> next weekend.) > Thanks! No rush. Now I re-sync swig-py3 branch with trunk and make a patch addressing issues on testing on other than Unix/Linux platform. I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6. Could anyone please test it on Windows ? >>> Tested on Windows 7, and yes, this fixes the previous test failures >>> with Python 2.7.16. Thanks! >>> >>> Test output: >>> [[[ >>> Testing Release configuration on local repository. >>> -- Running Swig Python tests -- >>> ..s. >>> . >>> -- >>> Ran 153 tests in 63.214s >>> >>> OK (skipped=1) >>> ]]] >>> >> I've commited it, as r1866362. Thanks! >> >>> Building with Python 3.7.4 still fails with the same error though (no >>> problem, I know your patch wasn't addressing that, just mentioning it >>> here for completeness). Consequently I haven't been able to run the >>> swig-python tests with python 3.7 yet. >>> >>> [[[ >>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type >>> definition in parentheses >>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj >>> ]]] >> Here is a patch not to treat C4115 as error, globally. If this makes >> it possible to build with Python 3.7 on Windows, then next step can be >> to limit to apply this relaxation of compile option to files which >> contains "#include " (most of them are generated by swig...). > Yes, that makes the build succeed, thanks. I can't comment on whether > or not it's good to change this error into a warning overall, or just > for a limited set of files. I'll leave that discussion to others :-). It should not be an error because the source is valid C, regardless of what Microsoft's compiler thinks about it. :) -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI wrote:
>
> On 2019/09/23 6:16, Johan Corveleyn wrote:
> > On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI
> > wrote:
> >>
> >> On 2019/09/20 0:44, Daniel Shahaf wrote:
> >>> Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
> >>
> The FAIL on test_merge_peg3 is a newline style issue on reading file
> in raw mode, which is introduced by my patch.
> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
>
> Rest of those errors and failures are in the tests that were added only
> to swig-py, and it seems most of the causes of them are confusion of
> paths in platform specific format and Subversion's canoical format,
> as the comment in SubversionClientTestCase.test_update4() says.
>
> (I'll try to fix those issues on check-swig-py later, hopefully
> next weekend.)
> >>>
> >>> Thanks! No rush.
> >>
> >> Now I re-sync swig-py3 branch with trunk and make a patch addressing
> >> issues on testing on other than Unix/Linux platform.
> >>
> >> I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
> >>
> >> Could anyone please test it on Windows ?
> >
> > Tested on Windows 7, and yes, this fixes the previous test failures
> > with Python 2.7.16. Thanks!
> >
> > Test output:
> > [[[
> > Testing Release configuration on local repository.
> > -- Running Swig Python tests --
> > ..s.
> > .
> > --
> > Ran 153 tests in 63.214s
> >
> > OK (skipped=1)
> > ]]]
> >
>
> I've commited it, as r1866362. Thanks!
>
> >
> > Building with Python 3.7.4 still fails with the same error though (no
> > problem, I know your patch wasn't addressing that, just mentioning it
> > here for completeness). Consequently I haven't been able to run the
> > swig-python tests with python 3.7 yet.
> >
> > [[[
> > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> > definition in parentheses
> > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
> > ]]]
>
> Here is a patch not to treat C4115 as error, globally. If this makes
> it possible to build with Python 3.7 on Windows, then next step can be
> to limit to apply this relaxation of compile option to files which
> contains "#include " (most of them are generated by swig...).
Yes, that makes the build succeed, thanks. I can't comment on whether
or not it's good to change this error into a warning overall, or just
for a limited set of files. I'll leave that discussion to others :-).
So, on to the next problem. When testing with Python 3.7.4 on Windows,
I get the following error:
[[[
C:\research\svn\dev\swig-py3>python win-tests.py -c --log-level=DEBUG
--release --swig=python R:\test_py
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
line 23, in
import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
repository, \
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\__init__.py",
line 23, in
from trac.versioncontrol.tests import svn_fs
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\svn_fs.py",
line 73, in
REPOS_URL = pathname2url(temp_path).encode('UTF-8')
File "C:\Python37\lib\nturl2path.py", line 53, in pathname2url
if not ':' in p:
TypeError: a bytes-like object is required, not 'str'
[Test runner reported failure]
]]]
(The warning about 'ruby' is not a big deal I suppose, but it's also
something I saw when running gen-make.py with python 3.7 -- not when
I'm running it with python 2.7)
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/23 6:16, Johan Corveleyn wrote: On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI wrote: On 2019/09/20 0:44, Daniel Shahaf wrote: Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00: The FAIL on test_merge_peg3 is a newline style issue on reading file in raw mode, which is introduced by my patch. (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438) Rest of those errors and failures are in the tests that were added only to swig-py, and it seems most of the causes of them are confusion of paths in platform specific format and Subversion's canoical format, as the comment in SubversionClientTestCase.test_update4() says. (I'll try to fix those issues on check-swig-py later, hopefully next weekend.) Thanks! No rush. Now I re-sync swig-py3 branch with trunk and make a patch addressing issues on testing on other than Unix/Linux platform. I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6. Could anyone please test it on Windows ? Tested on Windows 7, and yes, this fixes the previous test failures with Python 2.7.16. Thanks! Test output: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..s. . -- Ran 153 tests in 63.214s OK (skipped=1) ]]] I've commited it, as r1866362. Thanks! Building with Python 3.7.4 still fails with the same error though (no problem, I know your patch wasn't addressing that, just mentioning it here for completeness). Consequently I haven't been able to run the swig-python tests with python 3.7 yet. [[[ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj ]]] Here is a patch not to treat C4115 as error, globally. If this makes it possible to build with Python 3.7 on Windows, then next step can be to limit to apply this relaxation of compile option to files which contains "#include " (most of them are generated by swig...). -- Yasuhito FUTATSUKI Index: build/generator/templates/vcnet_vcproj.ezt === --- build/generator/templates/vcnet_vcproj.ezt (revision 1867212) +++ build/generator/templates/vcnet_vcproj.ezt (working copy) @@ -60,7 +60,7 @@ DisableSpecificWarnings="4100;4127;4206;4512;4701;4706;4800" Detect64BitPortabilityProblems="FALSE" AdditionalOptions=" - /we4002 /we4003 /we4013 /we4020 /we4022 /we4024 /we4028 /we4029 /we4030 /we4031 /we4033 /we4047 /we4089 /we4113 /we4115 /we4204 /we4715" + /we4002 /we4003 /we4013 /we4020 /we4022 /we4024 /we4028 /we4029 /we4030 /we4031 /we4033 /we4047 /we4089 /we4113 /we4204 /we4715" DebugInformationFormat="3" ProgramDataBaseFileName="$(IntDir)\[target.output_pdb]" [if-any configs.forced_include_files]ForcedIncludeFiles="[for configs.forced_include_files][configs.forced_include_files][if-index configs.forced_include_files last][else];[end][end]" Index: build/generator/templates/vcnet_vcxproj.ezt === --- build/generator/templates/vcnet_vcxproj.ezt (revision 1867212) +++ build/generator/templates/vcnet_vcxproj.ezt (working copy) @@ -65,7 +65,7 @@ Level4 ProgramDatabase 4100;4127;4206;4512;4701;4706;4800;%(DisableSpecificWarnings) - 4002;4003;4013;4020;4022;4024;4028;4029;4030;4031;4033;4047;4089;4113;4115;4133;4204;4700;4715;4789;%(TreatSpecificWarningsAsErrors) + 4002;4003;4013;4020;4022;4024;4028;4029;4030;4031;4033;4047;4089;4113;4133;4204;4700;4715;4789;%(TreatSpecificWarningsAsErrors) [if-any configs.forced_include_files] [for configs.forced_include_files][configs.forced_include_files];[end]%(ForcedIncludeFiles) [end]
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI wrote: > > On 2019/09/20 0:44, Daniel Shahaf wrote: > > Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00: > > >> The FAIL on test_merge_peg3 is a newline style issue on reading file > >> in raw mode, which is introduced by my patch. > >> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438) > >> > >> Rest of those errors and failures are in the tests that were added only > >> to swig-py, and it seems most of the causes of them are confusion of > >> paths in platform specific format and Subversion's canoical format, > >> as the comment in SubversionClientTestCase.test_update4() says. > >> > >> (I'll try to fix those issues on check-swig-py later, hopefully > >> next weekend.) > > > > Thanks! No rush. > > Now I re-sync swig-py3 branch with trunk and make a patch addressing > issues on testing on other than Unix/Linux platform. > > I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6. > > Could anyone please test it on Windows ? Tested on Windows 7, and yes, this fixes the previous test failures with Python 2.7.16. Thanks! Test output: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..s. . -- Ran 153 tests in 63.214s OK (skipped=1) ]]] Building with Python 3.7.4 still fails with the same error though (no problem, I know your patch wasn't addressing that, just mentioning it here for completeness). Consequently I haven't been able to run the swig-python tests with python 3.7 yet. [[[ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj ]]] -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/20 0:44, Daniel Shahaf wrote:
Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
The FAIL on test_merge_peg3 is a newline style issue on reading file
in raw mode, which is introduced by my patch.
(https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
Rest of those errors and failures are in the tests that were added only
to swig-py, and it seems most of the causes of them are confusion of
paths in platform specific format and Subversion's canoical format,
as the comment in SubversionClientTestCase.test_update4() says.
(I'll try to fix those issues on check-swig-py later, hopefully
next weekend.)
Thanks! No rush.
Now I re-sync swig-py3 branch with trunk and make a patch addressing
issues on testing on other than Unix/Linux platform.
I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
Could anyone please test it on Windows ?
Cheers,
--
Yasuhito FUTATSUKI
On branch swig-py3: Fix test for swig-py on Windows
* subversion/bindings/swig/python/tests/client.py:
(SubversionClientTestCase.test_merge_peg3): Concern platform specific
new line style.
(SubversionClientTestCase.test_conflict): Discriminate between local path
and canonical path
(SubversionClientTestCase.test_shelf): Discriminate between local path
and canonical path
Index: subversion/bindings/swig/python/tests/client.py
===
--- subversion/bindings/swig/python/tests/client.py (revision 1867212)
+++ subversion/bindings/swig/python/tests/client.py (working copy)
@@ -439,7 +439,8 @@
readme_text = readme.read()
readme.close()
-self.assertEqual(readme_text, b'This is a test.\n')
+self.assertEqual(readme_text,
+ b'This is a test.' + os.linesep.encode('UTF-8'))
def test_platform_providers(self):
providers = core.svn_auth_get_platform_specific_client_providers(None,
None)
@@ -579,10 +580,10 @@
client.checkout2(self.repos_uri, path, rev, rev, True, True,
self.client_ctx)
-trunk_path = os.path.join(path, b'trunk')
+trunk_path = core.svn_dirent_join(path, b'trunk')
# Create a conflicting path
-os.mkdir(trunk_path)
+os.mkdir(core.svn_dirent_local_style(trunk_path))
rev.value.number = 2
@@ -619,10 +620,10 @@
self.assertTrue(isinstance(shelf, client.svn_client__shelf_t))
-new_subpath = os.path.join(b'trunk', b'new-shelf-test.txt')
-new_path = os.path.join(path, new_subpath)
+new_subpath = core.svn_relpath_join(b'trunk', b'new-shelf-test.txt')
+new_path = core.svn_dirent_join(path, new_subpath)
-with open(new_path, "wb") as fp:
+with open(core.svn_dirent_local_style(new_path), "wb") as fp:
fp.write("A new text file\n".encode('utf8'))
client.add5(new_path, core.svn_depth_unknown, False, False, False, True,
self.client_ctx, pool)
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 20.09.2019 11:53, Paul Hammant wrote: > One more thing re Python3 - there's Svn Homebrew fu for Mac folks - > https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb > - that work admirably for Python 2.7.x but has no code for Python 3. > I don't think this brew formula is maintained in the svn dev team. Of > course, pull-requests are processed as you'd expected for a project > hosted on GitHub. It's not maintained by us. I've send a number of pull requests for new versions, but whoever the maintainers are on the Homebrew team have some very specific ideas about how packages should be produced -- obviously. :) I think, if we get the Python 3 support in tell HB about it (via a PR of course), they won't mind switching the dependency to Python 3, or maybe even both. (Homebrew Py3 is more reliable in my experience than their Py2, which has strange interactions with the system default.) -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
One more thing re Python3 - there's Svn Homebrew fu for Mac folks - https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb - that work admirably for Python 2.7.x but has no code for Python 3. I don't think this brew formula is maintained in the svn dev team. Of course, pull-requests are processed as you'd expected for a project hosted on GitHub.
Re: Python3 work [was: The run up to Subversion 1.13.0]
Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00: > On 2019/09/19 8:21, Johan Corveleyn wrote: > > On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej wrote: > >> You do *not* need a definition of 'struct foo' in order to use a 'struct > >> foo*' pointer. So, no, that's not the problem. > > Actually C4115 is only a warning. > > https://docs.microsoft.com/cpp/error-messages/compiler-warnings/compiler-warning-levels-1-and-4-c4115?view=vs-2019 > > However, it seems we are treating it as an error by specifyinging build > option in build/generator/templates/vcnet_vcxproj.ezt . > > (Is it related to https://bugs.python.org/issue25878, especially > msg256496 ?) > That certainly sounds similar. For comparison, on a toy example, I get a warning if I use a pointer to an undeclared struct type: [[[ % cat foo.c void f(struct foo *); % cc -c -Wall foo.c foo.c:1:15: warning: declaration of 'struct foo' will not be visible outside of this function [-Wvisibility] void f(struct foo *); ^ 1 warning generated. % ]]] That doesn't happen if the struct type has been declared: [[[ % cat bar.c struct bar; void g(struct bar *); % cc -c -Wall bar.c % ]]] I think that -Wvisibility warning is basically what that C4115 is about? pytime.h doesn't declare 'struct timeval', neither directly nor via an #include. I haven't checked whether my build warns about -Wvisibility there. > The FAIL on test_merge_peg3 is a newline style issue on reading file > in raw mode, which is introduced by my patch. > (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438) > > Rest of those errors and failures are in the tests that were added only > to swig-py, and it seems most of the causes of them are confusion of > paths in platform specific format and Subversion's canoical format, > as the comment in SubversionClientTestCase.test_update4() says. > > (I'll try to fix those issues on check-swig-py later, hopefully > next weekend.) Thanks! No rush. Cheers, Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 2019/09/19 8:21, Johan Corveleyn wrote:
On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej wrote:
On 18.09.2019 09:20, Johan Corveleyn wrote:
On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej wrote:
On 18.09.2019 02:48, Johan Corveleyn wrote:
[[[
c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
definition in parentheses
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
]]]
That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
this on line 123:
PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
Looks like perfectly valid C to me. So we'll need a bit more context to
see where the actual error is coming from.
Not at my pc right now (I'll check again tonight), but from memory: I
think my copy of pytime.h looks the same. Line 123 seems to be the
first usage of 'timeval' though ... Is it possible that some include
is missing, so there is no declaration of the timeval struct type?
You do *not* need a definition of 'struct foo' in order to use a 'struct
foo*' pointer. So, no, that's not the problem.
Actually C4115 is only a warning.
https://docs.microsoft.com/cpp/error-messages/compiler-warnings/compiler-warning-levels-1-and-4-c4115?view=vs-2019
However, it seems we are treating it as an error by specifyinging build
option in build/generator/templates/vcnet_vcxproj.ezt .
(Is it related to https://bugs.python.org/issue25878, especially
msg256496 ?)
In other news, building the swig-py3 branch (after sync-merge from
trunk) with Python 2.7.16 (and swig 3.0.12) was successful.
Running the testsuite gives some failures though:
[[[
Testing Release configuration on local repository.
-- Running Swig Python tests --
..s..EEFF...
..
==
ERROR: test_conflict (client.SubversionClientTestCase)
Test conflict api.
--
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 593, in test_conflict
conflict = client.conflict_get(trunk_path, self.client_ctx, pool)
File "R:\test_release--swigpython\swig\pylib\libsvn\client.py", line
1642, in svn_client_conflict_get
return _client.svn_client_conflict_get(*args)
SubversionException: 155007 - 'R:\temp\tmpyeizeh-conflict\trunk' is
not a working copy
==
ERROR: test_conflict (client.SubversionClientTestCase)
Test conflict api.
--
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 94, in tearDown
self.temper.cleanup()
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\utils.py",
line 44, in cleanup
clean_func(target)
File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line
8453, in svn_io_remove_dir
return _core.svn_io_remove_dir(*args)
SubversionException: 720032 - Can't remove
'R:\temp\tmpyeizeh-conflict\.svn\wc.db'
720032 - Can't remove file 'R:\temp\tmpyeizeh-conflict\.svn\wc.db':
The process cannot access the file because it is being used by another
process.
==
FAIL: test_merge_peg3 (client.SubversionClientTestCase)
Test svn_client_merge_peg3.
--
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 442, in test_merge_peg3
self.assertEqual(readme_text, b'This is a test.\n')
AssertionError: 'This is a test.\r\n' != 'This is a test.\n'
==
FAIL: test_shelf (client.SubversionClientTestCase)
Test shelf api.
--
Traceback (most recent call last):
File
"C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 644, in test_shelf
self.assertIn(new_subpath, statused_paths)
AssertionError: 'trunk\\new-shelf-test.txt' not found in
['trunk/new-shelf-test.txt', 'trunk/new-shelf-test.txt', None]
--
Ran 153 tests in 38.984s
FAILED (failures=2, errors=2, skipped=1)
Exception svn.core.SubversionException: SubversionException("Can't
open file 'R:\\temp\\tmp9ubdal-client\\db\\fs-type':
The system cannot find the path specified. ", 720003) in > ignored
[Test runner reported failure]
]]]
(Running the swig-python tests on trunk, with Python 2.7.16, was fully
successful)
The FAIL on test_merge_peg3 is a newline style issue on
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej wrote: > > On 18.09.2019 09:20, Johan Corveleyn wrote: > > On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej wrote: > >> On 18.09.2019 02:48, Johan Corveleyn wrote: > >>> On Mon, Sep 2, 2019 at 11:45 AM Julian Foad wrote: > > Troy Curtis Jr wrote: > >> James McCoy wrote: > >>> Finishing(?) and merging the Python3 support would be ideal. That > >>> would > >>> give one release for broader feedback before being in an LTS release. > >> It needs to get reintegrated with trunk again with the latest changes, > >> but the linux side of the house was looking good. It was in trying to > >> get my Windows Dev environment back up and running again that got me > >> frustrated and gave me an excuse to wander off doing other things... > >> IIRC, there is a build issue with the Py2 bindings in that branch on > >> Windows, but I never got far enough to check it out. > >> If there was someone who could help out on the Windows side, I can > >> jump in and get the branch up to trunk and retested in order to get > >> this thing over the finish line. > Johan Corveleyn wrote: > > I might be able to devote some time to this in the coming week(s), if > > you can tell me what I need to do / test / validate / ... :-). > Until Troy says something more specific, if you could switch to the > branch, catch-up merge from trunk, and see how much works on Py3 and on > Py2, that would be helpful. > >>> I'm sorry I missed the 1.13.x-branch deadline, but I finally got > >>> around to playing with the swig-py3 branch on Windows. > >>> Downloaded the latest Python release: 3.7.4. And using swig 3.0.12. > >>> > >>> Of course I didn't reread INSTALL, so I first ran into: > >>> > >>> [[[ > >>> WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found > >>> Use "--with-py3c" to configure py3c location. > >>> ]]] > >>> > >>> No problem, after downloading py3c from Github, and adding > >>> --with-py3c, I can start building it. > >>> > >>> However, I ran into the following error: > >>> > >>> [[[ > >>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type > >>> definition in parentheses > >>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] > >>> ]]] > >> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has > >> this on line 123: > >> > >> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); > >> > >> > >> Looks like perfectly valid C to me. So we'll need a bit more context to > >> see where the actual error is coming from. > > Not at my pc right now (I'll check again tonight), but from memory: I > > think my copy of pytime.h looks the same. Line 123 seems to be the > > first usage of 'timeval' though ... Is it possible that some include > > is missing, so there is no declaration of the timeval struct type? > > > You do *not* need a definition of 'struct foo' in order to use a 'struct > foo*' pointer. So, no, that's not the problem. Hmm, okay, just tried again ... same problem. Here is some more output: [[[ C:\research\svn\dev\swig-py3> msbuild subversion_vcnet.sln /nologo /v:q /p:Configuration=release /t:__SWIG_PYTHON__ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] C:\research\svn\dev\swig-py3\subversion\bindings\swig\proxy\swig_python_external_runtime.swg(2459): warning C4459: declaration of 'swig_this' hides global declaration [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] C:\research\svn\dev\swig-py3\subversion\bindings\swig\proxy\swig_python_external_runtime.swg(2534): warning C4459: declaration of 'swig_this' hides global declaration [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] ..\..\..\subversion\bindings\swig\python\libsvn_swig_py\swigutil_py.c(2363): warning C4101: 'tmp': unreferenced local variable [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] ..\..\..\subversion\bindings\swig\python\libsvn_swig_py\swigutil_py.c(2772): warning C4018: '>': signed/unsigned mismatch [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] ]]] I tried with both the x86-64 and the x86 builds of Python 3.7.4 ... same result. Weird. I'm continuing to investigate. In other news, building the swig-py3 branch (after sync-merge from trunk) with Python 2.7.16 (and swig 3.0.12) was successful. Running the testsuite gives some failures though: [[[ Testing Release configuration on local repository. -- Running Swig Python tests -- ..s..EEFF... .. == ERROR: test_conflict
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 18.09.2019 09:20, Johan Corveleyn wrote: > On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej wrote: >> On 18.09.2019 02:48, Johan Corveleyn wrote: >>> On Mon, Sep 2, 2019 at 11:45 AM Julian Foad wrote: > Troy Curtis Jr wrote: >> James McCoy wrote: >>> Finishing(?) and merging the Python3 support would be ideal. That would >>> give one release for broader feedback before being in an LTS release. >> It needs to get reintegrated with trunk again with the latest changes, >> but the linux side of the house was looking good. It was in trying to >> get my Windows Dev environment back up and running again that got me >> frustrated and gave me an excuse to wander off doing other things... >> IIRC, there is a build issue with the Py2 bindings in that branch on >> Windows, but I never got far enough to check it out. >> If there was someone who could help out on the Windows side, I can jump >> in and get the branch up to trunk and retested in order to get this >> thing over the finish line. Johan Corveleyn wrote: > I might be able to devote some time to this in the coming week(s), if > you can tell me what I need to do / test / validate / ... :-). Until Troy says something more specific, if you could switch to the branch, catch-up merge from trunk, and see how much works on Py3 and on Py2, that would be helpful. >>> I'm sorry I missed the 1.13.x-branch deadline, but I finally got >>> around to playing with the swig-py3 branch on Windows. >>> Downloaded the latest Python release: 3.7.4. And using swig 3.0.12. >>> >>> Of course I didn't reread INSTALL, so I first ran into: >>> >>> [[[ >>> WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found >>> Use "--with-py3c" to configure py3c location. >>> ]]] >>> >>> No problem, after downloading py3c from Github, and adding >>> --with-py3c, I can start building it. >>> >>> However, I ran into the following error: >>> >>> [[[ >>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type >>> definition in parentheses >>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] >>> ]]] >> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has >> this on line 123: >> >> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); >> >> >> Looks like perfectly valid C to me. So we'll need a bit more context to >> see where the actual error is coming from. > Not at my pc right now (I'll check again tonight), but from memory: I > think my copy of pytime.h looks the same. Line 123 seems to be the > first usage of 'timeval' though ... Is it possible that some include > is missing, so there is no declaration of the timeval struct type? You do *not* need a definition of 'struct foo' in order to use a 'struct foo*' pointer. So, no, that's not the problem. -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Johan Corveleyn wrote on Wed, 18 Sep 2019 07:44 +00:00: > On Wed, Sep 18, 2019 at 9:31 AM Daniel Shahaf wrote: > > > > Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00: > > > No problem, after downloading py3c from Github, and adding > > > --with-py3c, I can start building it. > > > > What version of py3c did you download? HEAD or a release? > > Hm, I downloaded from master at https://github.com/encukou/py3c. Maybe > not such a good choice? > Should I go for 0.9 from https://github.com/encukou/py3c/releases instead? I didn't intend to imply that; I merely wanted to know which version to use for investigations and replication attempts. Note that on the /releases page there's a v1.0 dated three months after the v0.9, if you click "Show newer tags" at the top. I'm not sure why github doesn't offer that one. Cheers, Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Wed, Sep 18, 2019 at 9:31 AM Daniel Shahaf wrote: > > Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00: > > No problem, after downloading py3c from Github, and adding > > --with-py3c, I can start building it. > > What version of py3c did you download? HEAD or a release? Hm, I downloaded from master at https://github.com/encukou/py3c. Maybe not such a good choice? Should I go for 0.9 from https://github.com/encukou/py3c/releases instead? -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00: > No problem, after downloading py3c from Github, and adding > --with-py3c, I can start building it. What version of py3c did you download? HEAD or a release?
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej wrote: > > On 18.09.2019 02:48, Johan Corveleyn wrote: > > On Mon, Sep 2, 2019 at 11:45 AM Julian Foad wrote: > >>> Troy Curtis Jr wrote: > James McCoy wrote: > > Finishing(?) and merging the Python3 support would be ideal. That would > > give one release for broader feedback before being in an LTS release. > It needs to get reintegrated with trunk again with the latest changes, > but the linux side of the house was looking good. It was in trying to > get my Windows Dev environment back up and running again that got me > frustrated and gave me an excuse to wander off doing other things... > IIRC, there is a build issue with the Py2 bindings in that branch on > Windows, but I never got far enough to check it out. > If there was someone who could help out on the Windows side, I can jump > in and get the branch up to trunk and retested in order to get this > thing over the finish line. > >> Johan Corveleyn wrote: > >>> I might be able to devote some time to this in the coming week(s), if > >>> you can tell me what I need to do / test / validate / ... :-). > >> Until Troy says something more specific, if you could switch to the > >> branch, catch-up merge from trunk, and see how much works on Py3 and on > >> Py2, that would be helpful. > > I'm sorry I missed the 1.13.x-branch deadline, but I finally got > > around to playing with the swig-py3 branch on Windows. > > Downloaded the latest Python release: 3.7.4. And using swig 3.0.12. > > > > Of course I didn't reread INSTALL, so I first ran into: > > > > [[[ > > WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found > > Use "--with-py3c" to configure py3c location. > > ]]] > > > > No problem, after downloading py3c from Github, and adding > > --with-py3c, I can start building it. > > > > However, I ran into the following error: > > > > [[[ > > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type > > definition in parentheses > > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] > > ]]] > > That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has > this on line 123: > > PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); > > > Looks like perfectly valid C to me. So we'll need a bit more context to > see where the actual error is coming from. Not at my pc right now (I'll check again tonight), but from memory: I think my copy of pytime.h looks the same. Line 123 seems to be the first usage of 'timeval' though ... Is it possible that some include is missing, so there is no declaration of the timeval struct type? -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 18.09.2019 02:48, Johan Corveleyn wrote: > On Mon, Sep 2, 2019 at 11:45 AM Julian Foad wrote: >>> Troy Curtis Jr wrote: James McCoy wrote: > Finishing(?) and merging the Python3 support would be ideal. That would > give one release for broader feedback before being in an LTS release. It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out. If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line. >> Johan Corveleyn wrote: >>> I might be able to devote some time to this in the coming week(s), if >>> you can tell me what I need to do / test / validate / ... :-). >> Until Troy says something more specific, if you could switch to the >> branch, catch-up merge from trunk, and see how much works on Py3 and on >> Py2, that would be helpful. > I'm sorry I missed the 1.13.x-branch deadline, but I finally got > around to playing with the swig-py3 branch on Windows. > Downloaded the latest Python release: 3.7.4. And using swig 3.0.12. > > Of course I didn't reread INSTALL, so I first ran into: > > [[[ > WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found > Use "--with-py3c" to configure py3c location. > ]]] > > No problem, after downloading py3c from Github, and adding > --with-py3c, I can start building it. > > However, I ran into the following error: > > [[[ > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type > definition in parentheses > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] > ]]] That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has this on line 123: PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv); Looks like perfectly valid C to me. So we'll need a bit more context to see where the actual error is coming from. -- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Mon, Sep 2, 2019 at 11:45 AM Julian Foad wrote: > > Troy Curtis Jr wrote: > >> James McCoy wrote: > >>> Finishing(?) and merging the Python3 support would be ideal. That would > >>> give one release for broader feedback before being in an LTS release. > >> > >> It needs to get reintegrated with trunk again with the latest changes, but > >> the linux side of the house was looking good. It was in trying to get my > >> Windows Dev environment back up and running again that got me frustrated > >> and gave me an excuse to wander off doing other things... IIRC, there is a > >> build issue with the Py2 bindings in that branch on Windows, but I never > >> got far enough to check it out. > > >> If there was someone who could help out on the Windows side, I can jump in > >> and get the branch up to trunk and retested in order to get this thing > >> over the finish line. > > Johan Corveleyn wrote: > > I might be able to devote some time to this in the coming week(s), if > > you can tell me what I need to do / test / validate / ... :-). > > Until Troy says something more specific, if you could switch to the > branch, catch-up merge from trunk, and see how much works on Py3 and on > Py2, that would be helpful. I'm sorry I missed the 1.13.x-branch deadline, but I finally got around to playing with the swig-py3 branch on Windows. Downloaded the latest Python release: 3.7.4. And using swig 3.0.12. Of course I didn't reread INSTALL, so I first ran into: [[[ WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found Use "--with-py3c" to configure py3c location. ]]] No problem, after downloading py3c from Github, and adding --with-py3c, I can start building it. However, I ran into the following error: [[[ c:\python37\include\pytime.h(123): error C4115: 'timeval': named type definition in parentheses [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj] ]]] (PS: before firing up the swig-py3 branch, I double-checked that I could successfully build the swig-python bindings on trunk with Python 2.7.16 and swig 3.0.12, and ran the swig-python tests successfully) -- Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
On Mon, Sep 2, 2019 at 8:23 AM Julian Foad wrote: > Thanks. I see no reason to make a special effort to have swig-py > bindings still support Python 2 for Subversion 1.13 and 1.14-LTS. In > fact, it's probably best if we explicitly remove Py2 support for 1.14, > to avoid getting trapped in maintenance difficulties. Py2 is EOL 1/1/2020. Keeping it in 1.14 which implies support until 2024 would be a mistake because support will only get harder. Imagine what happens when your OS vendor removes Py2 packages. Even if Py2 still technically "works" with 1.14 (because that makes it easier to support Py3 for the remainder of 1.10), I suggest to document that Py2 is explicitly NOT supported in the 1.14 release notes. The internal policy should be that once 1.10 support ends, changes are allowed to break Py2 support and/or not be tested with Py2. I have (among other things) a Windows 10 machine available to me with Python 2 and 3. I'll be glad to help, with a little guidance. At the very least I can help with testing.
Re: Python3 work [was: The run up to Subversion 1.13.0]
Branko Čibej wrote: What Py3 support are we talking about anyway? svn build? svn test suite? swig-py bindings? This is about the work that's happening on the swig-py3 branch. So: that would be support for building the swig-py bindings for both Python3 and Python2. Trunk only support Python2 which has reached end of life. Our build and test scripts have already been modified to work with both variants of Python, or rather, with Python 2.7.x and 3.something, for some unknown value of "something." Thanks. I see no reason to make a special effort to have swig-py bindings still support Python 2 for Subversion 1.13 and 1.14-LTS. In fact, it's probably best if we explicitly remove Py2 support for 1.14, to avoid getting trapped in maintenance difficulties. Py2 support is needed if we want to backport Py3 support to 1.10.x, which would be a good thing to aim for; and given that dual-Py-version capability is mostly in place but unfinished, that sounds like we are on the right track to be confident that if we put it on trunk now we would be able to fix up the Py2 support for backport to 1.10.x later, when it's ready. It may be tempting to "just" try to fix the remaining Py2 issues right now before merge to trunk, but I need to prepare a 1.13 branch within the next 10 days to stay on schedule, so please consider prioritising merge to trunk with Py3 working, leaving Py2 to be fixed up after that. - Julian
Re: Python3 work [was: The run up to Subversion 1.13.0]
On 02.09.2019 11:45, Julian Foad wrote: >> Troy Curtis Jr wrote: >>> James McCoy wrote: Finishing(?) and merging the Python3 support would be ideal. That would give one release for broader feedback before being in an LTS release. >>> >>> It needs to get reintegrated with trunk again with the latest >>> changes, but the linux side of the house was looking good. It was in >>> trying to get my Windows Dev environment back up and running again >>> that got me frustrated and gave me an excuse to wander off doing >>> other things... IIRC, there is a build issue with the Py2 bindings >>> in that branch on Windows, but I never got far enough to check it out. > > Stefan Sperling wrote: >> If it helps, we could require Py3-only on Windows, going forward. >> That particular ecosystem has been dragging its feet for long enough ;) > > What Py3 support are we talking about anyway? svn build? svn test > suite? swig-py bindings? This is about the work that's happening on the swig-py3 branch. So: that would be support for building the swig-py bindings for both Python3 and Python2. Trunk only support Python2 which has reached end of life. Our build and test scripts have already been modified to work with both variants of Python, or rather, with Python 2.7.x and 3.something, for some unknown value of "something." -- Brane
Python3 work [was: The run up to Subversion 1.13.0]
Troy Curtis Jr wrote: James McCoy wrote: Finishing(?) and merging the Python3 support would be ideal. That would give one release for broader feedback before being in an LTS release. It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out. Stefan Sperling wrote: If it helps, we could require Py3-only on Windows, going forward. That particular ecosystem has been dragging its feet for long enough ;) What Py3 support are we talking about anyway? svn build? svn test suite? swig-py bindings? Stefan Kueng, how would that be for you and TortoiseSVN? If it's fine for you, I would be very happy to release Subversion 1.13.0 with Py3 the only supported version on Windows, if we can get it integrated to trunk in the next few days. If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line. Johan Corveleyn wrote: I might be able to devote some time to this in the coming week(s), if you can tell me what I need to do / test / validate / ... :-). Until Troy says something more specific, if you could switch to the branch, catch-up merge from trunk, and see how much works on Py3 and on Py2, that would be helpful. Can someone else check the same on *nix please? 1.13 is a "regular release" with emphasis on new features etc. rather than stability. Of course we always like to have as much stability as possible as well, but please let's take this opportunity to get something of this important work integrated right now rather than waiting until it's absolutely finished. - Julian
