Hi Asheesh, The manylinux targets look pretty interesting, docker isn't definitely the most straight-forward way of handling glibc versions. Publishing to PyPi with GitHub Actions does sound quite useful as well. Let's sync up around December 23rd to see where things are at.
-Jacob On Friday, December 6, 2019 at 7:59:31 PM UTC-8, asheesh wrote: > > Hi Jacob and all, > > From what I've seen, there are some Docker-based build environments that > are designed to overcome the glibc version binary compatibility issues you > mentioned. > > I'm semi offline right now, so I can't find references, but the thing I'd > search for is [manylinux1 docker build]. > > Docker isn't essential, but it serves as a convenient way to distribute > enough of an operating system to do these builds. > > I'm enthusiastic about helping you figure this out, so I'll watch this > email thread. Also, if you want to remote pair program on it, I'm game the > week of December 23, or any week in January. > > There's also a newer manylinux2010 build target if you need it. > > If you want to avoid a manual build step for you, I'd also be happy to > help you set up GitHub Actions, Travis CI, or some other automated system > for publishing those wheels (binary packages) to PyPI. > > Cheers, > > Asheesh. > > > On Sat, Dec 7, 2019, 06:31 Jacob Alexander <tripl...@gmail.com > <javascript:>> wrote: > >> Cool! >> >> Yeah, I agree with bumping the version. >> >> One of the things I want to figure out is how to properly push binary >> linux pypi packages to pypi.org. Basically glibc compatibility for >> Python is a nightmare so there's a need to use a glibc as old as possible >> (but also not run into issues with distros actually requiring a minimum >> version of glibc for some things; I've been having issues with glibc 2.12.2 >> and some newer distros for example). I vaguely recall there are some >> Fortunately macOS and Windows are easy in this regard. >> >> -Jacob >> >> On Friday, December 6, 2019 at 3:13:17 PM UTC-8, Kenton Varda wrote: >>> >>> Hi Jacob, >>> >>> Sorry for the delay in replying. >>> >>> This is awesome work! >>> >>> I'd love to let you take over the official pycapnp repository and merge >>> this back in. I think it's totally fine if you've made breaking API changes >>> in the name of cleaning things up. Depending on the extent of the changes >>> it might make sense to bump the major version number before the next >>> release. The Go implementation is on version 3 now so there's plenty of >>> precedent for that. >>> >>> I've filed an issue on pycapnp to propose making you a maintainer, to >>> give the current owners a chance to weigh in: >>> https://github.com/capnproto/pycapnp/issues/194 >>> >>> -Kenton >>> >>> On Mon, Dec 2, 2019 at 3:40 PM <ja...@datrium.com> wrote: >>> >>>> I don't mind helping maintain pycapnp. It will make it easier for me if >>>> I can pull over most/all of my current changes. >>>> >>>> Some things that I've done >>>> >>>> - Removed most of the references to deprecated capnproto functions >>>> (there are a few left that are a bit tricky that have to do with the >>>> .capnp >>>> file parser imports) >>>> - Added Windows support (there seems to be a bug related to the >>>> capnproto timer that's erroring out the last few tests) >>>> - asyncio support for Python (this enables TLS support for both >>>> client and servers) >>>> - Python 3.7+ support (earlier versions don't work well with >>>> asyncio and will take a bunch of work to make work) >>>> - Updated the minimum version to capnproto 0.7.0 >>>> - Moved from Travis to Github Actions >>>> - No more git flow >>>> - Updated to C++14 (fixes lots of issues, including those that used >>>> to be there with macOS) >>>> >>>> This is my github https://github.com/haata >>>> >>>> I don't have many more changes planned. Well, beyond fixing new issues >>>> that pop-up, adding support for new Python/OS options and the last few >>>> Windows test errors. I also want to cleanup the documentation a bit and >>>> add >>>> a proper changelog. >>>> >>>> I'm also not sure how the package uploads work for >>>> https://pypi.org/project/pycapnp/ and user accounts there. I have my >>>> forked version here: https://pypi.org/project/pycapnp-async/ >>>> >>>> On Monday, December 2, 2019 at 1:29:20 PM UTC-8, Kenton Varda wrote: >>>>> >>>>> On Fri, Nov 29, 2019 at 1:28 AM Jacob Alexander <tripl...@gmail.com> >>>>> wrote: >>>>> >>>>>> I'm happy to upstream (though I've taken some more extreme decisions >>>>>> in some areas in regards to prior compatibility). >>>>>> Unfortunately, I don't believe there's an active maintainer for the >>>>>> pycapnp github repo so it's currently a bit futile. :( >>>>>> >>>>> >>>>> A few months ago, Colin Jermain (https://github.com/cjermain; I don't >>>>> seem to have his e-mail) offered to help with pycapnp maintenance, so I >>>>> gave him commit rights. It looks like he did some work but hasn't touched >>>>> it in a few months, so I'm not sure if he intends to keep working on it. >>>>> If >>>>> someone else wants to take over maintainership I'm happy to help get the >>>>> right permissions set up. >>>>> >>>>> -Kenton >>>>> >>>>> >>>>>> >>>>>> On Friday, November 29, 2019 at 12:48:46 AM UTC-8, pepijn de vos >>>>>> wrote: >>>>>>> >>>>>>> Thanks for the link. >>>>>>> While I'm not actually interested in RPC or async at all, this fork >>>>>>> actually works. >>>>>>> Any chance this will be upstreamed? >>>>>>> >>>>>>> Pepijn >>>>>>> >>>>>>> On Fri, Nov 29, 2019 at 7:27 AM Jacob Alexander <tripl...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> While I haven't gotten all the tests working yet (there are some >>>>>>>> issues with some of the timer functions on Windows it seems), I spent >>>>>>>> a >>>>>>>> bunch of time getting asyncio working with pycapnp (it was the most >>>>>>>> reasonable way to get native Python TLS support working). >>>>>>>> I've fixed a lot of bugs, removed a lot of the deprecated C++ >>>>>>>> functions (the warnings were hiding a lot of serious issues). >>>>>>>> >>>>>>>> https://github.com/haata/pycapnp-async >>>>>>>> >>>>>>>> I'll be using this for a cross-platform tool I've been working so >>>>>>>> I'll at least be maintaining it for basic client functionality (server >>>>>>>> stuff works as well, minus a few Windows tests). My main use case is a >>>>>>>> TLS >>>>>>>> connection between a Rust server and Python clients (this is currently >>>>>>>> working using tokio-rustls). >>>>>>>> >>>>>>>> It does require Python 3.7 and higher (3.8 also works). I >>>>>>>> discovered some bugs in asyncio that make porting difficult to 3.5 and >>>>>>>> 3.6 >>>>>>>> (though it's likely possible with a bunch of effort). >>>>>>>> >>>>>>>> -HaaTa >>>>>>>> >>>>>>>> On Thursday, November 28, 2019 at 6:24:08 AM UTC-8, Pepijn de Vos >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Actually, it appears that the library contains a pointer, whereas >>>>>>>>> the Python lib tries to use a reference. >>>>>>>>> I'm not sure where this difference came from, but it's clearly >>>>>>>>> incorrect. >>>>>>>>> >>>>>>>>> (env) [apicula]$ nm -gD /usr/lib/libkj-async-0.7.0.so | grep >>>>>>>>> TransformPromiseNodeBase >>>>>>>>> 000000000002b1e0 T >>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyEPNS0_5EventE >>>>>>>>> (env) [apicula]$ c++filt >>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyEPNS0_5EventE >>>>>>>>> kj::_::TransformPromiseNodeBase::onReady(kj::_::Event*) >>>>>>>>> (env) [apicula]$ c++filt >>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyERNS0_5EventE >>>>>>>>> kj::_::TransformPromiseNodeBase::onReady(kj::_::Event&) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thursday, 28 November 2019 15:17:40 UTC+1, Pepijn de Vos wrote: >>>>>>>>>> >>>>>>>>>> Hey all, >>>>>>>>>> >>>>>>>>>> I'm exploring serialization libraries, so I installed the Python >>>>>>>>>> lib and got the following error >>>>>>>>>> >>>>>>>>>> ImportError: [...]capnp.cpython-38-x86_64-linux-gnu.so: >>>>>>>>>> undefined symbol: >>>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyERNS0_5EventE >>>>>>>>>> >>>>>>>>>> Which is definitely a thing, and has been for two years: >>>>>>>>>> https://github.com/capnproto/capnproto/blame/master/c%2B%2B/src/kj/async-inl.h#L392 >>>>>>>>>> >>>>>>>>>> It also seems to be linked correctly >>>>>>>>>> >>>>>>>>>> $ ldd [...]capnp.cpython-38-x86_64-linux-gnu.so >>>>>>>>>> linux-vdso.so.1 (0x00007ffe675cf000) >>>>>>>>>> libcapnpc-0.7.0.so => /usr/lib/libcapnpc-0.7.0.so ( >>>>>>>>>> 0x00007f7dc5173000) >>>>>>>>>> libcapnp-rpc-0.7.0.so => /usr/lib/libcapnp-rpc-0.7.0.so ( >>>>>>>>>> 0x00007f7dc5090000) >>>>>>>>>> libcapnp-0.7.0.so => /usr/lib/libcapnp-0.7.0.so ( >>>>>>>>>> 0x00007f7dc4ff4000) >>>>>>>>>> libkj-async-0.7.0.so => /usr/lib/libkj-async-0.7.0.so ( >>>>>>>>>> 0x00007f7dc4f60000) >>>>>>>>>> libkj-0.7.0.so => /usr/lib/libkj-0.7.0.so (0x00007f7dc4eda000 >>>>>>>>>> ) >>>>>>>>>> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f7dc4cf0000 >>>>>>>>>> ) >>>>>>>>>> libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f7dc4cd4000) >>>>>>>>>> libc.so.6 => /usr/lib/libc.so.6 (0x00007f7dc4b0d000) >>>>>>>>>> libpthread.so.0 => /usr/lib/libpthread.so.0 ( >>>>>>>>>> 0x00007f7dc4aeb000) >>>>>>>>>> libm.so.6 => /usr/lib/libm.so.6 (0x00007f7dc49a5000) >>>>>>>>>> /usr/lib64/ld-linux-x86-64.so.2 (0x00007f7dc540e000) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> So the system library version is 0.7, and the latest Python >>>>>>>>>> library is 0.6.4, not sure if the versions are just mismatched >>>>>>>>>> somehow. >>>>>>>>>> I tried installing from git with the same result, is the Python >>>>>>>>>> library just outdated? >>>>>>>>>> Or maybe the Arch package is just broken, because it doesn't >>>>>>>>>> contain the required symbol? >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Pepijn >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "Cap'n Proto" group. >>>>>>>> To unsubscribe from this topic, visit >>>>>>>> https://groups.google.com/d/topic/capnproto/1x8_5_RC9wU/unsubscribe >>>>>>>> . >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> capn...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/capnproto/f8decb16-f304-4d80-9e2c-7df4946df458%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/capnproto/f8decb16-f304-4d80-9e2c-7df4946df458%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Cap'n Proto" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to capn...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/capnproto/812fa7a9-0bf3-4eb4-b483-e20f8ebb33c0%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/capnproto/812fa7a9-0bf3-4eb4-b483-e20f8ebb33c0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Cap'n Proto" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to capn...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/capnproto/49839775-0604-4653-bda9-cb2c5d5f9d41%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/capnproto/49839775-0604-4653-bda9-cb2c5d5f9d41%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to capn...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/capnproto/8ef1d626-d327-49aa-a666-f6d56e795884%40googlegroups.com >> >> <https://groups.google.com/d/msgid/capnproto/8ef1d626-d327-49aa-a666-f6d56e795884%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/5959f7be-2864-4716-86d7-0113b63cf035%40googlegroups.com.
