[issue30193] Allow to load buffer objects with json.loads()

Changes by Nikolay Kim : -- pull_requests: +1443 ___ Python tracker ___ ___

[issue30193] Allow to load buffer objects with json.loads()

New submission from Nikolay Kim: It is not possible to use buffer objects in json.loads() -- messages: 292487 nosy: fafhrd91 priority: normal severity: normal status: open title: Allow to load buffer objects with json.loads() versions: Python 3.6, Python 3.7

[issue8323] buffer objects are picklable but result is not unpicklable

Serhiy Storchaka added the comment: While buffer is only used in Python 2.x, there is a similar issue in 3.x with memoryview. The patch for issue22995 fixes this issue. -- nosy: +serhiy.storchaka ___ Python tracker

[issue25326] Improve error message for "character buffer objects"

Roundup Robot added the comment: New changeset 4d1bd86d3bbd by Raymond Hettinger in branch '2.7': Issue #25326: Improve an obscure error message. https://hg.python.org/cpython/rev/4d1bd86d3bbd -- nosy: +python-dev ___ Python tracker

[issue25326] Improve error message for "character buffer objects"

Changes by Raymond Hettinger : -- resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue25326] Improve error message for "character buffer objects"

Raymond Hettinger added the comment: Another alternative --- TypeError: expected a string or other character buffer object -- ___ Python tracker

[issue25326] Improve error message for "character buffer objects"

New submission from Raymond Hettinger: Python 2.7.10+ (2.7:fc6d62db8d42+, Oct 6 2015, 11:55:10) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> s = 'the tale of two cities' >>> s.count(('a', 'z'))

[issue25326] Improve error message for "character buffer objects"

Changes by Raymond Hettinger <raymond.hettin...@gmail.com>: -- components: Interpreter Core files: better_error_message.diff keywords: patch nosy: rhettinger priority: normal severity: normal stage: patch review status: open title: Improve error message for "character buf

[issue15958] bytes.join() should allow arbitrary buffer objects

Changes by Glyph Lefkowitz gl...@twistedmatrix.com: -- nosy: +glyph ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___ ___ Python-bugs-list

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: Yet one issue. You forgot to add join.h to BYTESTR_DEPS in Makefile.pre.in. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Roundup Robot added the comment: New changeset 388e43bb519d by Antoine Pitrou in branch 'default': Followup to issue #15958: add join.h to Makefile dependencies for byte strings http://hg.python.org/cpython/rev/388e43bb519d -- ___ Python tracker

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Here is a new patch checking that the sequence size didn't change. I also refactored the join() implementation into a shared function in stringlib. -- Added file: http://bugs.python.org/file27594/bytes_join_buffers3.patch

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: I added new comments. :-( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___ ___ Python-bugs-list

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: I added new comments. :-( Thanks. I think I will commit after adding the missing #undef :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Roundup Robot added the comment: New changeset 16285c1b4dda by Antoine Pitrou in branch 'default': Issue #15958: bytes.join and bytearray.join now accept arbitrary buffer objects. http://hg.python.org/cpython/rev/16285c1b4dda -- nosy: +python-dev

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Done now. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___ ___ Python-bugs-list mailing list

[issue15958] bytes.join() should allow arbitrary buffer objects

Changes by Antoine Pitrou pit...@free.fr: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: Well done. However check at top of Objects/stringlib/join.h does not protect from using the file with asciilib or ucs1lib. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: However check at top of Objects/stringlib/join.h does not protect from using the file with asciilib or ucs1lib. I'm not sure that's a problem. Someone would have to go out of their way to use join.h with only UCS1 unicode strings. Also tests would probably

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Here is an updated patch using PySequence_Fast_GET_SIZE to avoid problems when the sequence is resized during iteration. -- Added file: http://bugs.python.org/file27585/bytes_join_buffers2.patch ___ Python tracker

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Here is a patch with tests. -- Added file: http://bugs.python.org/file27554/bytes_join_buffers.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: Patch LGTM, however... $ ./python -m timeit -s a=[b'a']*10 b','.join(a) Vanilla: 3.69 msec per loop Patched: 11.6 msec per loop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Patch LGTM, however... $ ./python -m timeit -s a=[b'a']*10 b','.join(a) Vanilla: 3.69 msec per loop Patched: 11.6 msec per loop True. It is a bit of a pathological case, though. -- ___ Python tracker

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: Here is a patch with restored performance. Is not it too complicated? -- Added file: http://bugs.python.org/file27557/bytes_join_buffers_2.patch ___ Python tracker rep...@bugs.python.org

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: The problem with your approach is that the sequence could be mutated while another thread is running (_getbuffer() may release the GIL). Then the pre-computed size gets wrong. -- ___ Python tracker

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: The problem with your approach is that the sequence could be mutated while another thread is running (_getbuffer() may release the GIL). Then the pre-computed size gets wrong. Well, then I withdraw my patch. But what if the sequence will be mutated and

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: But what if the sequence will be mutated and PySequence_Size(seq) will become less seqlen? Then using PySequence_Fast_GET_ITEM() will be incorrect. Perhaps we should detect that case and raise, then. -- ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Changes by Serhiy Storchaka storch...@gmail.com: Removed file: http://bugs.python.org/file27557/bytes_join_buffers_2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Ezio Melotti added the comment: Attached patch adds support for memoryviews to bytes.join: b''.join([memoryview(b'foo'), b'bar']) b'foobar' The implementation currently has some duplication, because it does a first pass to calculate the total size to allocate, and another pass to create the

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: I think memoryview here is example only, and Antoine had in mind arbitrary buffer objects. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958

[issue15958] bytes.join() should allow arbitrary buffer objects

Ezio Melotti added the comment: Indeed. Attached new patch. Tests still need to be improved; bytearrays are still not changed. -- Added file: http://bugs.python.org/file27258/issue15958-2.diff ___ Python tracker rep...@bugs.python.org

[issue15958] bytes.join() should allow arbitrary buffer objects

Stefan Krah added the comment: We would need to release the buffers and also check for format 'B'. With issue15958-2.diff this is possible: import array a = array.array('d', [1.2345]) b''.join([b'ABC', a]) b'ABC\x8d\x97n\x12\x83\xc0\xf3?' It is unfortunate that a PyBUF_SIMPLE request does

[issue15958] bytes.join() should allow arbitrary buffer objects

Stefan Krah added the comment: Also, perhaps we can keep a fast path for bytes and bytearray, but I didn't time the difference. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Well, given the following works: import array a = array.array('d', [1.2345]) b'' + a b'\x8d\x97n\x12\x83\xc0\xf3?' It should also work for bytes.join(). I guess that means I'm against the strict-typedness of memoryviews. As the name suggests, it provides

[issue15958] bytes.join() should allow arbitrary buffer objects

Ezio Melotti added the comment: Attached new refleakless patch. -- Added file: http://bugs.python.org/file27259/issue15958-3.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue15958] bytes.join() should allow arbitrary buffer objects

Antoine Pitrou added the comment: Attached new refleakless patch. Your approach is dangerous, because the buffers may change size between two calls to PyObject_GetBuffer(). I think you should keep the Py_buffers alive in an array, and only release them at the end (it may also be slightly

[issue15958] bytes.join() should allow arbitrary buffer objects

Serhiy Storchaka added the comment: I think you should keep the Py_buffers alive in an array, and only release them at the end (it may also be slightly faster to do so). However allocation of this array may considerably slow down the function. We may need the special-case for bytes and

[issue15958] bytes.join() should allow arbitrary buffer objects

: Interpreter Core messages: 170618 nosy: pitrou priority: normal severity: normal stage: needs patch status: open title: bytes.join() should allow arbitrary buffer objects type: enhancement versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http

[issue15958] bytes.join() should allow arbitrary buffer objects

Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___ ___ Python-bugs-list

[issue15958] bytes.join() should allow arbitrary buffer objects

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15958 ___

[issue8323] buffer objects are picklable but result is not unpicklable

Changes by Florent Xicluna florent.xicl...@gmail.com: -- nosy: +flox ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8323 ___ ___ Python-bugs-list

[issue8323] buffer objects are picklable but result is not unpicklable

Richard Oudkerk shibt...@gmail.com added the comment: It seems the real issue here is that buffer objects are picklable (depending on protocol) but the resulting string is not unpicklable. There are probably lots of other examples where this happens: for instance Exception subclasses which do

[issue8323] buffer objects are picklable but result is not unpicklable

sbt shibt...@gmail.com added the comment: Buffer objects *are* picklable with protocol 2 (but not with earlier protocols). Unfortunately, the result is not unpicklable. This is not a problem with multiprocessing. (buffer seems to inherit __reduce__ and __reduce_ex__ from object.) Python

buffer objects (C-API)

Hi at all, is it possible that a buffer object deallocates the memory when the object is destroyed? I want to wrap the buffer object around some memory. Or is there any chance that the buffer object copies the memory so it will be responsible when it will be destroyed? Thanks in advance, bye.

Re: buffer objects (C-API)

On May 12, 7:33 pm, moerchendiser2k3 googler. 1.webmas...@spamgourmet.com wrote: Hi at all, is it possible that a buffer object deallocates the memory when the object is destroyed? I want to wrap the buffer object around some memory. Or is there any chance that the buffer object copies the

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

posix methods don't like bytes / buffer objects. versions: Python 3.1, Python 3.2 Added file: http://bugs.python.org/file15659/posix_fn_bytes_01.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7560

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

Changes by Sebastian Hagen sh_pyb...@memespace.net: -- type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7560 ___ ___

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

Changes by Sebastian Hagen sh_pyb...@memespace.net: Removed file: http://bugs.python.org/file15659/posix_fn_bytes_01.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7560 ___

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

Sebastian Hagen sh_pyb...@memespace.net added the comment: I'm taking that patch back. More testing would have been in order before posting; sorry for that, will repost once I've got the obvious problems worked out. -- ___ Python tracker

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

Sebastian Hagen sh_pyb...@memespace.net added the comment: And further testing reveals that all of this has in fact already been fixed in trunk. I assumed it hadn't been, because the code for at least some of the relevant functions in Modules/_posixmodule.c is the same as in 3.1.1; I didn't know

[issue7560] Various filename-taking posix methods don't like bytes / buffer objects.

Changes by Sebastian Hagen sh_pyb...@memespace.net: -- status: open - closed versions: -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7560 ___

Buffer objects

Greetings, I need a little help with buffer objects. Many Python objects export the buffer interface, or can be persuaded to create a buffer object with a buffer() call. First question, the buffer() function appears very quick. Does it just wrap the internal pointer and then exit? Second

Re: Buffer objects

Tom Harris wrote: Greetings, I need a little help with buffer objects. Many Python objects export the buffer interface, or can be persuaded to create a buffer object with a buffer() call. ... It must be me but I have found the whole buffer thing difficult to understand from the docs

[issue1617702] extended slicing for buffer objects

Thomas Wouters added the comment: Committed revision 57619. -- assignee: - twouters resolution: - fixed status: open - closed _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1617702 _