[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-28 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

Thanks, Nick! I'll move the function declaration back to abstract.h.

Waiting for Georg's input. -- It seems to me that #14330 is a blocker
that will only be fixed on Monday.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-28 Thread Georg Brandl

Georg Brandl ge...@python.org added the comment:

With the beta delayed as you say, I'm okay with this going in now.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-28 Thread Roundup Robot

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 8fbbc7c8748e by Stefan Krah in branch 'default':
Issue #12834: Fix PyBuffer_ToContiguous() for non-contiguous arrays.
http://hg.python.org/cpython/rev/8fbbc7c8748e

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-28 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

All right, 3.3 is fixed. Re-targeting for 3.2 and 2.7.

--
priority: release blocker - normal
versions: +Python 3.1 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-27 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

Any objections to committing this before beta2? What about the
len  view-len change: Does that look reasonable?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-27 Thread Nick Coghlan

Nick Coghlan ncogh...@gmail.com added the comment:

Georg, need a call on how close you are to cutting beta 2 and whether you want 
this to wait until rc1.

--
nosy: +benjamin.peterson, georg.brandl
priority: normal - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-27 Thread Nick Coghlan

Nick Coghlan ncogh...@gmail.com added the comment:

Summary of my review for Georg's benefit: I had one minor formatting nit with 
the patch (which Stefan can easily fix before committing), but it otherwise 
looked fine to me.

I also agree that the old implicit truncation was unusable in practice, as the 
*actual* length is not returned from the function, and in fact cannot be 
returned because the return type is int rather than Py_ssize_t. A length 
mismatch therefore needs to be an error.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-25 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

Here's a patch for 3.3, which consists mostly of tests. A couple of points:

  o I removed the len  view-len check in PyBuffer_ToContiguous(), since
the function is not documented and silently accepting output buffers
that are too large seems unusual to me.

  o Removed the comment in bytesobject.c Better way to get to internal 
buffer?.
IMO the answer is no: There isn't any better way that works in full 
generality.

  o Removed the need to check for overflow comment: ndim is now officially
limited to 64.


I think this can go into 3.3: It's easy to see that for contiguous buffers
PyBuffer_ToContiguous() behaves in the same manner as before (except for
the len issue).

For memoryview, buffer_to_contiguous(x, y 'C') is literally the same
code as buffer_to_c_contiguous().

--
keywords: +patch
Added file: http://bugs.python.org/file26512/issue12834.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-21 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

There is an additional problem with PyBuffer_ToContiguous():

Suppose 'view' is multi-dimensional, C-contiguous and initialized
according to PyBUF_ND, i.e. view-shape != NULL but view-strides == NULL.

Now if PyBuffer_ToContiguous() is called with 'F', PyBuffer_IsContiguous()
returns false and view-strides will be accessed.


This means that incomplete buffer information will have to be
reconstructed like it is done in the 3.3 memoryview.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-20 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

The fix would require all of these functions from memoryview.c (3.3):

last_dim_is_contiguous
cmp_structure
copy_base
copy_rec
copy_buffer


How to avoid code duplication? I could move them into abstract.c,
but conceptually they're really just low level buffer interface
functions. Also, they make a lot of assumptions (ndim = 1,
PyBUF_FULL) that are a little dangerous for a general interface.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-20 Thread Christian Heimes

Christian Heimes li...@cheimes.de added the comment:

You could move PyBuffer_ToContiguous() from abstract.c to memoryview.c.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-20 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

 You could move PyBuffer_ToContiguous() from abstract.c to memoryview.c.

For 3.3 that would be ideal, yes. I asked a while ago on python-dev
whether to backport the memoryview rewrite. The general mood was
against it.

So, for 2.7/3.2 I could add all these functions to abstract.c.
But an additional problem is that the whole test infrastructure of
Lib/test/test_buffer.py and Modules/_testbuffer.c would be missing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-19 Thread Christian Heimes

Christian Heimes li...@cheimes.de added the comment:

It looks like Stefan has fixed the issue in Python 3.3 a while ago. tobytes() 
returns the correct values with a fresh build of Python 3.3. 

$ PYTHONPATH=. /home/heimes/dev/python/py3k/python 
smc/freeimage/tests/test_image.py
test_newbuffer (__main__.TestImageNewBuffer) ... 

  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255
 80  80  80 112 112 112 160 160 160 192 192 192 240 240 240
  0   0 255   0 255   0 255   0   0   0   0 255   0 255   0
255   0   0   0   0 255   0 255   0 255   0   0   0   0 255
  0 255   0 255   0   0   0   0 255   0 255   0 255   0   0
255 255   0 255   0 255   0 255 255 255 255   0 255   0 255

However it's still broken in 3.2 (also up to date hg pull).

$ PYTHONPATH=. /home/heimes/dev/python/3.2/python 
smc/freeimage/tests/test_image.py
test_newbuffer (__main__.TestImageNewBuffer) ... 

  0   0   0   0   0   0   0   0   0   0   0   0   0   0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255  80
 80  80 112 112 112 160 160 160 192 192 192 240 240 240   0
  0 255   0 255   0 255   0   0   0   0 255   0 255   0 255
  0   0   0   0 255   0 255   0 255   0   0   0   0 255   0
255   0 255   0   0   0   0 255   0 255   0 255   0   0 255
255   0 255   0 255   0 255 255 255 255   0 255   0 255   0


Stefan, could you please port your fix to Python 3.2 and 3.3? Thanks!

--
versions: +Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-19 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

In Python 3.3 memoryobject.c is a complete rewrite. Porting the fix
separately would be quite a bit of work.

PyBuffer_ToContiguous(), which causes the problem in 2.7/3.2 and is
still broken in 3.3, could be fixed by using the recursive copy_buffer() 
function from the new memoryobject.c.


I don't know if I can fix it before the 3.3 release. When are the
next 2.7/3.2 releases?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-18 Thread Christian Heimes

Christian Heimes li...@cheimes.de added the comment:

I think that I run into the same bug today. I've developing a PEP 3118 buffer 
interface for my wrapper of FreeImage. It returns the data as non-contiguous 3d 
array with the dimension height, width, color.

I've created a small test image with 5x7 RGB pixels. The first line black, 
second white, third grey values and the rest are red, green blue and cyan, 
magenta yellow in little endian (BGR order).

NumPy handles the buffer correctly:
 arr = numpy.asarray(img)
 print(arr)

[[[  0   0   0]
  [  0   0   0]
  [  0   0   0]
  [  0   0   0]
  [  0   0   0]]

 [[255 255 255]
  [255 255 255]
  [255 255 255]
  [255 255 255]
  [255 255 255]]

 [[ 80  80  80]
  [112 112 112]
  [160 160 160]
  [192 192 192]
  [240 240 240]]

 [[  0   0 255]
  [  0 255   0]
  [255   0   0]
  [  0   0 255]
  [  0 255   0]]

 [[255   0   0]
  [  0   0 255]
  [  0 255   0]
  [255   0   0]
  [  0   0 255]]

 [[  0 255   0]
  [255   0   0]
  [  0   0 255]
  [  0 255   0]
  [255   0   0]]

 [[255 255   0]
  [255   0 255]
  [  0 255 255]
  [255 255   0]
  [255   0 255]]]

but memoryview.tobytes() seems to have an off-by-one error:

 m = memoryview(img)
 data = m.tobytes()
 len(data) ==  5 * 7 * 3
True
 for i in range(7):
... print( .join(%3i % ord(v) for v in data[i * 5 * 3:(i + 1) * 5 * 3]))
  0   0   0   0   0   0   0   0   0   0   0   0   0   0 255
255 255 255 255 255 255 255 255 255 255 255 255 255 255  80
 80  80 112 112 112 160 160 160 192 192 192 240 240 240   0
  0 255   0 255   0 255   0   0   0   0 255   0 255   0 255
  0   0   0   0 255   0 255   0 255   0   0   0   0 255   0
255   0 255   0   0   0   0 255   0 255   0 255   0   0 255
255   0 255   0 255   0 255 255 255 255   0 255   0 255   0

As you can see the first byte is missing.

Python 2.7.3 and Python 3.2.3 with numpy 1.6.1 and 
https://bitbucket.org/tiran/smc.freeimage/changeset/3134ecee2984 on 64bit 
Ubuntu.

--
nosy: +christian.heimes
versions: +Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-07-18 Thread Meador Inge

Changes by Meador Inge mead...@gmail.com:


--
nosy: +meador.inge

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-02-25 Thread Roundup Robot

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 3f9b3b6f7ff0 by Stefan Krah in branch 'default':
- Issue #10181: New memoryview implementation fixes multiple ownership
http://hg.python.org/cpython/rev/3f9b3b6f7ff0

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue12834] PyBuffer_ToContiguous() incorrect for non-contiguous arrays

2012-01-25 Thread Stefan Krah

Stefan Krah stefan-use...@bytereef.org added the comment:

I removed the dependency since PyBuffer_ToContiguous() still needs to
be fixed in abstract.c while memoryview.tobytes() now works in the
PEP-3118 repo.

--
dependencies:  -Problems with Py_buffer management in memoryobject.c (and 
elsewhere?)
title: memoryview.tobytes() incorrect for non-contiguous arrays - 
PyBuffer_ToContiguous() incorrect for non-contiguous arrays

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12834
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com