2011/11/24 PyTables Org <pytab...@googlemail.com>
> Forwarding to the list. ~Josh
>
> Begin forwarded message:
>
> *From: *pytables-users-boun...@lists.sourceforge.net
> *Date: *November 24, 2011 3:33:32 AM GMT+01:00
> *To: *pytables-users-ow...@lists.sourceforge.net
> *Subject: **Auto-discard notification*
>
> The attached message has been automatically discarded.
> *From: *"Edward C. Jones" <edcjo...@comcast.net>
> *Date: *November 24, 2011 3:31:05 AM GMT+01:00
> *To: *pytables-users@lists.sourceforge.net
> *Subject: **Rules for appending a tuple to a table*
>
>
> I have two tables, sortlist and temptable, each with three columns:
> Float64Col, StringCol(16) and StringCol(16).
>
> alist = list(sortlist[-1])
>
>
Note that alist is a *list*
>
> type(alist[0]) is <type 'numpy.float64'>
> type(alist[1]) is <type 'numpy.string_'>
> type(alist[2]) is <type 'numpy.string_'>
>
> alist prints as [0.40000000000000002, 'abcdefghabcdefgh',
> '0123456701234567']
>
> with numpy types. I append the list to another table with the same columns:
>
> temptable.append([alist])
>
> list(temptable[-1]) prints as [0.40000000000000002, '', ''] (empty
> strings)
>
>
...hence, you are appending a list of a list (and not a list of a tuple, as
desired). Change the `alist` variable to be a tuple, and you will be fine.
>
> Where in the PyTables code does this behavior come from? Should an
> Exception
> have been raised? Is there a general procedure for diagnosing mysterious
> problems in PyTables?
>
> The documentation for Table.append says:
>
> The rows argument may be any object which can be converted to a record
> array compliant with the table structure (otherwise, a ValueError is
> raised). This includes NumPy record arrays, RecArray (depracated) or
> NestedRecArray (deprecated) objects if numarray is available, lists of
> tuples or array records, and a string or Python buffer.
>
>
Please notice the *list of tuples* here.
Anyway, the ultimate reason for this is that PyTables uses NumPy behind the
scenes to convert `Table.append` parameters into I/O buffers. And it
happens that NumPy has the next (funny) behaviour:
In [1]: import numpy as np
In [2]: np.array([(0.4, 'abcdefghabcdefgh', '0123456701234567')],
dtype="f8,S16,S16")
Out[2]:
array([(0.40000000000000002, 'abcdefghabcdefgh', '0123456701234567')],
dtype=[('f0', '<f8'), ('f1', '|S16'), ('f2', '|S16')])
In [3]: np.array([[np.float64(0.4), np.string_('abcdefghabcdefgh'),
np.string_('0123456701234567')]], dtype="f8,S16,S16")
Out[3]:
array([[(0.40000000000000002, '', ''),
(8.5408832230361244e+194, 'abcdefgh', ''),
(9.9583343788967447e-43, '01234567', '')]],
dtype=[('f0', '<f8'), ('f1', '|S16'), ('f2', '|S16')])
In [4]: np.__version__
Out[4]: '1.6.1'
This probably needs to be fixed in the NumPy side (feel free to file a
ticket on this).
>
> I think more details are needed here.
>
> Now I do things a little differently:
>
> alist = list(sortlist[-1])
> alist[0] = float(alist[0]) # Convert to a Python type.
> temptable.append([alist])
>
> Now list(temptable[-1]) prints as
>
> [0.40000000000000002, 'abcdefghabcdefgh', '0123456701234567']
>
> There is apparently no need to convert the numpy strings into Python
> strings.
>
> If I try the natural thing:
>
> temptable.append([(sortlist[-1]])
>
> I get an useless IndexError: tuple index out of range.
>
> This is again an artifact introduced by NumPy internally. Force the use
of a tuple:
temptable.append([tuple(sortlist[-1])])
and that should work.
--
Francesc Alted
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
