Hi all,
I like pytables a lot this morning it is driving me up the walls. I have
a file with particles and I want to modify it. I basically want to split
_some_ of the particles into smaller subparticles. The HDF5 file is
generated by some other code (written in C++), but I don't want to use
the terrible C++ interface to for this simple task.
The definition of the table containing the particles looks like this:
/table1 (Table(1049256L,), zlib(6)) 'table1'
description := {
"particle_id": Float64Col(shape=(), dflt=0.0, pos=0),
"x_ort": Float64Col(shape=(), dflt=0.0, pos=1),
"y_ort": Float64Col(shape=(), dflt=0.0, pos=2),
"z_ort": Float64Col(shape=(), dflt=0.0, pos=3),
"x_geschwindigkeit": Float64Col(shape=(), dflt=0.0, pos=4),
"y_geschwindigkeit": Float64Col(shape=(), dflt=0.0, pos=5),
"z_geschwindigkeit": Float64Col(shape=(), dflt=0.0, pos=6),
"Mue": Float64Col(shape=(), dflt=0.0, pos=7),
"Masse": Float64Col(shape=(), dflt=0.0, pos=8),
"Ladung": Float64Col(shape=(), dflt=0.0, pos=9)}
byteorder := 'little'
chunkshape := (10L,)
So my first idea was to use "for p in f.root.table:" to iterate over the
table, check if the condition is true (basically |v| > 0.1 c) and if
that is the case delete the particle using removeRows and appending new
particles.
This turned out to be a bad idea for two (or three) reasons. I would
have to avoid splitting the particles further and further. A bit ugly
but manageable.
Second problem: p contains particles, but removeRows wants row number.
How do I find that out? Well. The documentation doesn't say. Turns out
that this doesn't matter because:
Third problem: removing and appending in a table you are iterating over
doesn't work. Ok, fair enough.
So. second idea. Create a second file, copy the particles which are slow
enough, and add subparticles insted of the fast particles. Sounds good
right? Well. Problem. I can't find any way to say "create a table just
like this one over there, but without the couple of millions of rows".
And no, I don't want to write boilerplate code, defining the table
format. And rewriting it every time a new column is added to the output
of the C++ program.
Ok. So I though "If I can't create a empty table, I can copy the file to
a new name, and drop all the rows in the table. That gets me a nice and
empty table I can fill." Turn out I can't:
"NotImplementedError: You are trying to delete all the rows in table
"/table1". This is not supported right now due to limitations on the
underlying HDF5 library. Sorry!"
Great. Of course I could delete all but the first particle, append what
I want to the mostly empty table and after I'm done remove that first
particle. Messy, but at this point I don't care.
Except ... I can't. If I have the input file i and the output file o.
Both with identical tables defined. And I do:
for p in i.root.table1:
o.root.table1.append(p)
it breaks with:
File "/usr/lib/python2.5/site-packages/tables/table.py", line 1758,
in append
"rows parameter cannot be converted into a recarray object
compliant with table '%s'. The error was: <%s>" % (str(self), exc)
ValueError: rows parameter cannot be converted into a recarray object
compliant with table '/table1 (Table(1L,), zlib(6)) 'table1''. The error
was: <objects of type ``Row`` are not supported in this context, sorry;
supported objects are: NumPy array, record or scalar; homogeneous list
or tuple, integer, float, complex or string>
At that point I decided that pytables is fine for reading but just
doesn't cut it for modifying tables. Which is a shame given its name.
Patrick "Using pytables ro for now" Kilian
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
