Hi David,
Yes, this is mostly a Python issue in 32-bit platforms. Look at this:
>>> import array # The native array Python module
>>> b = array.array('i')
>>> b.append(1)
>>> b.append(1L)
>>> b.append(2**32-999)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to int
and NumPy behaves exactly the same:
>>> import numpy as np
>>> a = np.array([1], dtype='i4')
>>> a[0] = 2
>>> a[0] = 2L
>>> a[0] = 2**32-999
Traceback (most recent call last):
File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to int
PyTables inherits this behaviour from NumPy, although the error raised
is a `TypeError` instead of an `OverflowError`. This is due to the
fact that error handling is not very sophisticated in the `Row`
accessor (speed is favoured instead).
So, as you can see, performing this operation (i.e. interpreting
signed 32-bit integers as unsigned ones) is not supported by Python in
32-bit platforms. So I suggest you to find another way to do what you
are after (perhaps you can use an UInt32Col and do conversions
afterwards).
Hope that helps,
Francesc
2009/12/29, David Fokkema <dfokk...@ileos.nl>:
> Hi list,
>
> PyTables 2.1.2 with HDF5 1.6.6 and both NumPy 1.3.0 and 1.4.0 on 32-bit
> vs
> PyTables 2.1.2 with HDF5 1.8.3 and NumPy 1.4.0rc1 on 64-bit
>
> I have lots of clients interpreting detector data and sending pickled
> objects over http to a server, which uses PyTables to store it (yes, I'm
> very happy now that everything's running; it works very well!). I have a
> bug on the client which I didn't notice until I had a crash on my dev
> machine. I ultimately found an inconsistency in PyTables.
>
> On the client, I'm interpreting signed 32-bit integers as unsigned ones
> (ouch). So, a value of -999 (error flag) is interpreted as 2**32 - 999 =
> 4294966297L. This value is received by the server, which tries to store
> it in a Int32Col.
>
> On 64-bit: no problem! The stored value is actually -999, so some
> conversion takes place. This is why I didn't notice.
>
> On 32-bit: in tables.tableExtension.Row.__setitem__(): invalid type
> (<type 'long'>) for column ``col``
>
> On 32-bit, I tested with the distributions NumPy (1.3.0) and PyPI's
> NumPy (1.4.0), both with the same result.
>
> This shouldn't be, right? I'm not sure if it is NumPy or PyTables...
>
> Thanks,
>
> David
>
>
>>>> import tables
>>>> class MyTable(tables.IsDescription):
> ... col = tables.Int32Col()
> ...
>>>> data = tables.openFile('/tmp/test.h5', 'w')
>>>> data.createTable('/', 'test', MyTable)
> /test (Table(0,)) ''
> description := {
> "col": Int32Col(shape=(), dflt=0, pos=0)}
> byteorder := 'little'
> chunkshape := (2048,)
>>>> row = data.root.test.row
>>>> row['col'] = 2**32 - 999
> ------------------------------------------------------------
> Traceback (most recent call last):
> File "<ipython console>", line 1, in <module>
> File "tableExtension.pyx", line 1309, in
> tables.tableExtension.Row.__setitem__
> TypeError: invalid type (<type 'long'>) for column ``col``
>
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
--
Francesc Alted
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
