Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Sat, Dec 1, 2018, 06:56 Steve Holden We* should probably do more collectively to point people at > production-quality third-party modules, as I believe we currently do with > pipenv which, while not a part of the standard library, is still > recommended in the documentation as the preferred

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

We* should probably do more collectively to point people at production-quality third-party modules, as I believe we currently do with pipenv which, while not a part of the standard library, is still recommended in the documentation as the preferred method of dependency management. We should also

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Fri, 30 Nov 2018 13:00:37 -0800 Glenn Linderman wrote: > > So it would be nice if http.server and http.client could get some basic > improvements to be complete, or if the docs could point to a replacement > that is a complete server, but without a philosophy or framework > (bloatware) to

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On 11/29/2018 2:10 PM, Andrew Svetlov wrote: Neither http.client nor http.server doesn't support compression (gzip/compress/deflate) at all. I doubt if we want to add this feature: for client better to use requests or, well, aiohttp. The same for servers: almost any production ready web server

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Fri, Nov 30, 2018 at 2:12 AM Brett Cannon wrote: > > > On Thu, 29 Nov 2018 at 14:12, Andrew Svetlov > wrote: > >> Neither http.client nor http.server doesn't support compression >> (gzip/compress/deflate) at all. >> I doubt if we want to add this feature: for client better to use requests >>

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 14:12, Andrew Svetlov wrote: > Neither http.client nor http.server doesn't support compression > (gzip/compress/deflate) at all. > I doubt if we want to add this feature: for client better to use requests > or, well, aiohttp. > The same for servers: almost any production

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 11:22 PM Steve Dower wrote: > FWIW, Brotli has been supported in Microsoft Edge since early last year: > > > https://blogs.windows.com/msedgedev/2016/12/20/introducing-brotli-compression/ > > Thanks, good to know. -- Thanks, Andrew Svetlov

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

Neither http.client nor http.server doesn't support compression (gzip/compress/deflate) at all. I doubt if we want to add this feature: for client better to use requests or, well, aiohttp. The same for servers: almost any production ready web server from PyPI supports compression. I don't insist

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On 29Nov2018 1230, Gregory P. Smith wrote: On Thu, Nov 29, 2018 at 2:58 AM Andrew Svetlov > wrote: 5 cents about lz4 alternatives: Broli (mentioned above) is widely supported by web.

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 2:58 AM Andrew Svetlov wrote: > 5 cents about lz4 alternatives: Broli (mentioned above) is widely > supported by web. > > https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding > mentions it along with gzip and deflate methods. > I don't recall lz4 or

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, 2:55 PM Paul Moore ... and some users need a single, unambiguous choice for the > "official, complete" distribution. Which need the current stdlib > serves extremely well. > Except it doesn't. At least not for a large swatch of users. 10 years ago, what I wanted in Python

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 17:52, Nathaniel Smith wrote: > > On Thu, Nov 29, 2018, 08:34 Antoine Pitrou > >> >> Le 29/11/2018 à 17:25, Steve Dower a écrit : >> > >> > My experience is that the first group would benefit from a larger >> > _standard distribution_, which is not necessarily the same

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 19:08, Nathaniel Smith wrote: > > On Thu, Nov 29, 2018, 10:32 Antoine Pitrou > >> On Thu, 29 Nov 2018 09:49:32 -0800 >> Nathaniel Smith wrote: >> > >> > There are a lot of challenges to switching to a "standard distribution" >> > model. I'm not certain it's the best

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

Le 29/11/2018 à 20:05, Nathaniel Smith a écrit : > On Thu, Nov 29, 2018, 10:32 Antoine Pitrou wrote: > > On Thu, 29 Nov 2018 09:49:32 -0800 > Nathaniel Smith mailto:n...@pobox.com>> wrote: > > > > There are a lot of challenges to switching to a

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, 10:32 Antoine Pitrou On Thu, 29 Nov 2018 09:49:32 -0800 > Nathaniel Smith wrote: > > > > There are a lot of challenges to switching to a "standard distribution" > > model. I'm not certain it's the best option. But what I like about it is > > that it could potentially reduce

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 16:28, Steve Dower wrote: > My experience is that the first group would benefit from a larger > _standard distribution_, which is not necessarily the same thing as a > larger stdlib. > > I'm firmly on the "smaller core, larger distribution" side of things, > where we as the

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 09:49:32 -0800 Nathaniel Smith wrote: > > There are a lot of challenges to switching to a "standard distribution" > model. I'm not certain it's the best option. But what I like about it is > that it could potentially reduce the conflict between what our different > user

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, 08:34 Antoine Pitrou > Le 29/11/2018 à 17:25, Steve Dower a écrit : > > > > My experience is that the first group would benefit from a larger > > _standard distribution_, which is not necessarily the same thing as a > > larger stdlib. > > > > I'm firmly on the "smaller core,

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On 29/11/2018 17.32, Antoine Pitrou wrote: > We may ask ourselves if there is really a large difference between a > "standard distribution" and a "standard library". The primary > difference seems to be that the distribution is modular, while the > stdlib is not. Yes, there is a huge difference

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

Le 29/11/2018 à 17:25, Steve Dower a écrit : > > My experience is that the first group would benefit from a larger > _standard distribution_, which is not necessarily the same thing as a > larger stdlib. > > I'm firmly on the "smaller core, larger distribution" side of things, > where we as

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On 29Nov2018 0254, Antoine Pitrou wrote: I'd like to point the discussion is asymmetric here. On the one hand, people who don't have access to PyPI would _really_ benefit from a larger stdlib with more batteries included. On the other hand, people who have access to PyPI _don't_ benefit from

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 15:52, Oleg Broytman wrote: > > On Thu, Nov 29, 2018 at 09:36:51AM -0500, Benjamin Peterson > wrote: > > - stdlib modules become a permanent maintenance burden to CPython core > > developers. > >Add ditributions maintainers here. Well, given that "you shouldn't use

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 09:36:51AM -0500, Benjamin Peterson wrote: > - stdlib modules become a permanent maintenance burden to CPython core > developers. Add ditributions maintainers here. Oleg. -- Oleg Broytmanhttps://phdru.name/p...@phdru.name

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 14:56, Benjamin Peterson wrote: > While I'm sympathetic to users in such situations, I'm not sure how much we > can really help them. These are the sorts of users who are likely to still be > stuck using Python 2.6. Any stdlib improvements we discuss and implement >

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 09:53:30AM -0500, Benjamin Peterson wrote: [...] > > This is not an argument either for or against adding LZ4, I have no > > opinion either way. But it is a reminder that "just get it from PyPI" > > represents an extremely privileged position that not all Python users >

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 10:05:47 -0500 "Benjamin Peterson" wrote: > On Thu, Nov 29, 2018, at 08:45, Antoine Pitrou wrote: > > > > Le 29/11/2018 à 15:36, Benjamin Peterson a écrit : > > >> > > >> I'd like to point the discussion is asymmetric here. > > >> > > >> On the one hand, people who don't

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, 6:56 AM Benjamin Peterson > > On Wed, Nov 28, 2018, at 15:27, Steven D'Aprano wrote: > > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > > > PyPI makes getting more algorithms easy. > > > > Can we please stop over-generalising like this? PyPI makes

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 09:36:51AM -0500, Benjamin Peterson wrote: > I don't think it's asymmetric. People have raised several practical > problems with a large stdlib in this thread. These include: > > - The evelopment of stdlib modules slows to the rate of the Python > release schedule.

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, at 08:45, Antoine Pitrou wrote: > > Le 29/11/2018 à 15:36, Benjamin Peterson a écrit : > >> > >> I'd like to point the discussion is asymmetric here. > >> > >> On the one hand, people who don't have access to PyPI would _really_ > >> benefit from a larger stdlib with more

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018, at 15:27, Steven D'Aprano wrote: > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > PyPI makes getting more algorithms easy. > > Can we please stop over-generalising like this? PyPI makes getting > more algorithms easy for *SOME* people. (Sorry

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

Le 29/11/2018 à 15:36, Benjamin Peterson a écrit : >> >> I'd like to point the discussion is asymmetric here. >> >> On the one hand, people who don't have access to PyPI would _really_ >> benefit from a larger stdlib with more batteries included. >> >> On the other hand, people who have access to

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018, at 04:54, Antoine Pitrou wrote: > On Thu, 29 Nov 2018 09:52:29 + > Paul Moore wrote: > > [This is getting off-topic, so I'll limit my comments to this one email] > > > > On Thu, 29 Nov 2018 at 03:17, Brett Cannon wrote: > > > We have never really had a discussion

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 11:00, Andrew Svetlov wrote: > I worked with lz4 python binding a year ago. > It sometimes crashed to core dump when used in multithreaded environment (we > used to run compressor/decompresson with asyncio by loop.run_in_executor() > call). > I hope the bug is fixed now,

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 16:02:29 +1100 Steven D'Aprano wrote: > On Wed, Nov 28, 2018 at 07:14:03PM -0800, Brett Cannon wrote: > > On Wed, 28 Nov 2018 at 13:29, Steven D'Aprano wrote: > > > > > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > > > > > PyPI makes getting

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

5 cents about lz4 alternatives: Broli (mentioned above) is widely supported by web. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding mentions it along with gzip and deflate methods. I don't recall lz4 or Zstd metioning in this context. Both Chrome/Chromium and Firefox

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 09:52:29 + Paul Moore wrote: > [This is getting off-topic, so I'll limit my comments to this one email] > > On Thu, 29 Nov 2018 at 03:17, Brett Cannon wrote: > > We have never really had a discussion about how we want to guide the stdlib > > going forward (e.g. how much

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, Nov 29, 2018 at 6:27 AM Steven D'Aprano wrote: > > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > PyPI makes getting more algorithms easy. > > Can we please stop over-generalising like this? PyPI makes getting > more algorithms easy for *SOME* people. (Sorry for

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

[This is getting off-topic, so I'll limit my comments to this one email] On Thu, 29 Nov 2018 at 03:17, Brett Cannon wrote: > We have never really had a discussion about how we want to guide the stdlib > going forward (e.g. how much does PyPI influence things, focus/theme, etc.). > Maybe we

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Thu, 29 Nov 2018 at 09:13, Gregory P. Smith wrote: > Q: Are there other popular alternatives to fill that niche that we should > strongly consider instead or as well? > > 5 years ago the answer would've been Snappy. 15 years ago the answer > would've been LZO. > Today LZ4 hits a sweet spot

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018 at 10:43 AM Gregory P. Smith wrote: > > On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon wrote: > >> Are we getting to the point that we want a compresslib like hashlib if we >> are going to be adding more compression algorithms? >> > > Lets avoid the lib suffix when

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018 at 07:14:03PM -0800, Brett Cannon wrote: > On Wed, 28 Nov 2018 at 13:29, Steven D'Aprano wrote: > > > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > > > PyPI makes getting more algorithms easy. > > > > Can we please stop over-generalising like this?

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 at 13:29, Steven D'Aprano wrote: > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > > > PyPI makes getting more algorithms easy. > > Can we please stop over-generalising like this? PyPI makes getting > more algorithms easy for *SOME* people. (Sorry for

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018, 12:08 Antoine Pitrou On Wed, 28 Nov 2018 19:35:31 + > Jonathan Underwood wrote: > > On Wed, 28 Nov 2018 at 18:57, Antoine Pitrou > wrote: > > > > > > On Wed, 28 Nov 2018 10:43:04 -0800 > > > "Gregory P. Smith" wrote: > > [snip] > > > > I don't think adding lz4 to the

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: > PyPI makes getting more algorithms easy. Can we please stop over-generalising like this? PyPI makes getting more algorithms easy for *SOME* people. (Sorry for shouting, but you just pressed one of my buttons.) PyPI might as

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 19:35:31 + Jonathan Underwood wrote: > On Wed, 28 Nov 2018 at 18:57, Antoine Pitrou wrote: > > > > On Wed, 28 Nov 2018 10:43:04 -0800 > > "Gregory P. Smith" wrote: > [snip] > > > I don't think adding lz4 to the stdlib is worthwhile. It isn't required > > > for core

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 at 18:57, Antoine Pitrou wrote: > > On Wed, 28 Nov 2018 10:43:04 -0800 > "Gregory P. Smith" wrote: [snip] > > I don't think adding lz4 to the stdlib is worthwhile. It isn't required > > for core functionality as zlib is (lowest common denominator zip support). > > Actually,

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 10:43:04 -0800 "Gregory P. Smith" wrote: > On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon wrote: > > > Are we getting to the point that we want a compresslib like hashlib if we > > are going to be adding more compression algorithms? > > > > Lets avoid the lib suffix when

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon wrote: > Are we getting to the point that we want a compresslib like hashlib if we > are going to be adding more compression algorithms? > Lets avoid the lib suffix when unnecessary. I used the name hashlib because the name hash was already taken by

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 09:51:57 -0800 Brett Cannon wrote: > Are we getting to the point that we want a compresslib like hashlib if we > are going to be adding more compression algorithms? It may be useful as a generic abstraction wrapper for simple usage but some compression libraries have custom

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

+1 to Brett's idea. It's hard to have a good mental model already of which compression algorithms are and are not in stdlib. A package to contain them all would help a lot. On Wed, Nov 28, 2018, 12:56 PM Brett Cannon Are we getting to the point that we want a compresslib like hashlib if we > are

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

Are we getting to the point that we want a compresslib like hashlib if we are going to be adding more compression algorithms? On Wed, 28 Nov 2018 at 08:44, Antoine Pitrou wrote: > On Wed, 28 Nov 2018 10:28:19 + > Jonathan Underwood wrote: > > Hi, > > > > I have for sometime maintained the

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

On Wed, 28 Nov 2018 10:28:19 + Jonathan Underwood wrote: > Hi, > > I have for sometime maintained the Python bindings to the LZ4 > compression library[0, 1]: > > I am wondering if there is interest in having these bindings move to > the standard library to sit alongside the gzip, lzma etc