On Tue, 5 Feb 2019 at 01:30, soumith <soum...@gmail.com> wrote: > > Unfortunately I'll be on a long flight, and cannot make it to the SIGBuild > meeting. > I'm definitely interested in the meeting notes and any follow-up meeting. > > > I think we should leave CUDA out of the > discussion initially and see if we can get the cpu-only wheel working > correctly. Hopefully cpu-only is viable on manylinux2014 then we can > tackle CUDA afterwards. > > 50% of the complexity is in the CUDA packaging. > The other 50% is in shipping a more modern libstdc++.so > I believe we'll make progress if we ignore CUDA, but we'll not address half > of the issue.
Yeah, we'll definitely need both to solve it fully. My thinking is that all packages need at least C++11 but only some need CUDA. Or might we end up where the libstcc++.so is incompatible with CUDA if we don't work on everything together? -- Jason > > -- > S > > On Mon, Feb 4, 2019 at 12:21 PM Antoine Pitrou <anto...@python.org> wrote: >> >> >> Le 04/02/2019 à 17:36, Uwe L. Korn a écrit : >> > I think that problem is whether this would get merged. Conda was created >> > after a meeting with Guido van Rossum and other folks at a PyCon quite >> > some years ago where the final call was that this is not a problem of >> > the core Python ecosystem and that the scientific Python community has >> > to roll their own solution. >> > >> > @Wes McKinney <mailto:wesmck...@gmail.com> or someone else: Were you at >> > this meeting and can outline why it was declined back then? >> >> I'm not sure anyone in this CC list was at that meeting (I wasn't). If >> it's important to have the precise answer, I can try to CC someone. >> >> But I think the general answer is that it's a complex and difficult >> endeavour, and the contribution structures inside the Python packaging >> ecosystem, where most people are volunteers, didn't allow for it. >> There's already enough lag maintaining the current software stack (pip >> et al.). >> >> Anaconda then came up and became history, so to speak. >> >> Regards >> >> Antoine. >> >> >> > >> > Uwe >> > >> > Am Mo., 4. Feb. 2019 um 17:33 Uhr schrieb Manuel Klimek >> > <kli...@google.com <mailto:kli...@google.com>>: >> > >> > On Mon, Feb 4, 2019 at 5:32 PM Uwe L. Korn <xho...@gmail.com >> > <mailto:xho...@gmail.com>> wrote: >> > >> > Just as a heads-up: I would like to also join the meeting but am >> > also located in Europe. >> > >> > I have spent quite some time with the packaging of wheels for >> > pyarrow and turbodbc thus I would like to also give input on >> > this. For Apache Arrow, I see newer manylinux2014 standard as a >> > possible way to go. I'm not so fond of rolloing lib(std)c++ >> > packages inside of pip. It's sadly the case that the features of >> > pip don't allow a good dependency resolution, also with taking >> > CUDA into account, a dependency resolution that differs between >> > source and binary builds of a package. For this case, exactly >> > conda was developed because it was considered out-of-scope for >> > the core Python packaging system. I'm not sure whether we >> > actually can fit all the requirements of the packages that take >> > part in this mail thread into pip without simply reimplementing >> > conda inside of pip. >> > >> > >> > One question is probably: what would that entail, and why would it >> > be bad? :) >> > >> > >> > >> > Uwe >> > >> > Am Mo., 4. Feb. 2019 um 16:34 Uhr schrieb Jason Zaman >> > <ja...@perfinion.com <mailto:ja...@perfinion.com>>: >> > >> > yeah that's expected. The timing is complicated with people >> > spread all >> > over. We will post notes after the meeting on the SIG-Build >> > mailing >> > list and I'd also be up for organizing a separate call with >> > europe >> > folks if that would be of interest. >> > >> > On Mon, 4 Feb 2019 at 19:29, 'Manuel Klimek' via SIG Build >> > <bu...@tensorflow.org <mailto:bu...@tensorflow.org>> wrote: >> > > >> > > +Dmitri Gribenko >> > > >> > > Dmitri has experience with EasyBuild, which seems to be >> > used by the HPC community to solve the bootstrap problem and >> > could be used to build a toolchain image & pip package. >> > > >> > > Unfortunately we'll not be able to join the meeting as >> > it's at midnight CEST - looking forward to the conclusions >> > from the meeting! >> > > >> > > On Mon, Feb 4, 2019 at 6:00 AM Jason Zaman >> > <ja...@perfinion.com <mailto:ja...@perfinion.com>> wrote: >> > >> >> > >> Hey all, >> > >> >> > >> We're having the TensorFlow SIG-Build meeting on 5th Feb >> > 3pm PST >> > >> >> > >> > (https://www.timeanddate.com/worldclock/fixedtime.html?iso=20190205T15&p1=224). >> > >> Agenda is linked from: >> > >> >> > >> > https://groups.google.com/a/tensorflow.org/forum/#!topic/build/akyPcGoBIy4 >> > >> >> > >> I'd like to invite everyone from this thread to join the >> > call if at >> > >> all possible. The agenda for this meeting will spend most >> > of the time >> > >> focusing on the manylinux issue and hopefully we can get >> > together to >> > >> flesh out a decent plan on how to tackle this. >> > >> >> > >> Thanks, >> > >> Jason >> > >> >> > >> On Wed, 30 Jan 2019 at 23:34, 'Manuel Klimek' via SIG Build >> > >> <bu...@tensorflow.org <mailto:bu...@tensorflow.org>> wrote: >> > >> > >> > >> > On Wed, Jan 30, 2019 at 4:21 PM Antoine Pitrou >> > <anto...@python.org <mailto:anto...@python.org>> wrote: >> > >> >> >> > >> >> >> > >> >> Le 30/01/2019 à 16:09, Manuel Klimek a écrit : >> > >> >> > >> > >> >> > On Wed, Jan 30, 2019 at 3:51 PM Antoine Pitrou >> > <anto...@python.org <mailto:anto...@python.org> >> > >> >> > <mailto:anto...@python.org >> > <mailto:anto...@python.org>>> wrote: >> > >> >> > >> > >> >> > >> > >> >> > Le 30/01/2019 à 15:35, Manuel Klimek a écrit : >> > >> >> > > >> > >> >> > > Am I reading you wrong, or are you >> > actually proposing to >> > >> >> > package another >> > >> >> > > libstdc++ version as a Python wheel? >> > >> >> > > >> > >> >> > > >> > >> >> > > That would be the idea. >> > >> >> > > >> > >> >> > > >> > >> >> > > If so, are you going to claim that the >> > given wheel is >> > >> >> > > manylinux-compatible? >> > >> >> > > >> > >> >> > > >> > >> >> > > That is my question :) Why wouldn't it be? >> > (I'd link it against >> > >> >> > > manylinux libc and other C-only system libs) >> > >> >> > >> > >> >> > The problem is when you are loading two modules >> > that link against >> > >> >> > different libstdc++ versions in the same >> > process. Incidentally, it's >> > >> >> > the problem which prompted this discussion. >> > >> >> > >> > >> >> > >> > >> >> > Sure, I'm aware :) I think as long as the >> > requirement that all libraries >> > >> >> > that want to exchange runtime-ABI-compatible >> > versions are compiled with >> > >> >> > the same toolchain, we can provide a way to mangle >> > the symbols >> > >> >> > differently. >> > >> >> >> > >> >> Ah, I see... Indeed, mangling the symbols may work for >> > this. >> > >> >> >> > >> >> That said, if you're looking to create a de facto >> > standard, why can't it >> > >> >> be proposed as a manylinux iteration? >> > >> > >> > >> > >> > >> > I'd have thought because it doesn't change the system >> > requirements, while manylinux seems to be all about system >> > requirements. >> > >> > The idea is that that toolchain would still work on any >> > manylinux compatible machine. >> > >> > >> > >> > >> > >> > >> > >> >> >> > >> >> >> > >> >> Regards >> > >> >> >> > >> >> Antoine. >> > >> > >> > >> > -- >> > >> > You received this message because you are subscribed to >> > the Google Groups "SIG Build" group. >> > >> > To unsubscribe from this group and stop receiving >> > emails from it, send an email to >> > build+unsubscr...@tensorflow.org >> > <mailto:build%2bunsubscr...@tensorflow.org>. >> > >> > Visit this group at >> > https://groups.google.com/a/tensorflow.org/group/build/. >> > > >> > > -- >> > > You received this message because you are subscribed to >> > the Google Groups "SIG Build" group. >> > > To unsubscribe from this group and stop receiving emails >> > from it, send an email to build+unsubscr...@tensorflow.org >> > <mailto:build%2bunsubscr...@tensorflow.org>. >> > > Visit this group at >> > https://groups.google.com/a/tensorflow.org/group/build/. >> >
