Re: [Pytables-users] Clear chunks from CArray
hi Anthony and Antonio,
thanks for the explanations. I was hoping I could do it programmatically, but
if
it is an inherent limitation of HDF5 there is little I can do. Compression plus
a sensible chunk size should help though.
Best,
Giovanni
On 08/06/2013 04:11 AM, pytables-users-requ...@lists.sourceforge.net wrote:
Hi Anthony, hi Giovanni,
Il giorno 06/ago/2013, alle ore 00:45, Anthony Scopatzscop...@gmail.com ha
scritto:
On Mon, Aug 5, 2013 at 3:14 PM, Giovanni Luca
Ciampagliaglciamp...@gmail.com wrote:
Hi Anthony,
what do you mean precisely? I tried
del ca[:,:]
but CArray does not support __delitem__. Looking in the documentation I could
only find a method called remove_rows, but it's in Table, not CArray. Maybe
I am
missing something?
Huh, it should... This is definitely an oversight on our part. If you
could please open an issue for this -- or better yet -- write a pull request
that implements delitem, that'd be great!
So I think you are right that there is no current way to delete rows from a
CArray. Oops! (Of course, I may still be missing something as well).
It looks like EArray also has this problem too, otherwise I would just tell
you to use that.
I'm not sure to understand the problem.
The truncate method of arrays can be used to remove rows from an extendable
array.
It seems to me that it is not documented but we should add it to the UG.
CArrays cannot be resized.
Be Well
Anthony
Thank,
Giovanni
On Mon 05 Aug 2013 03:43:42 PM
EDT,pytables-users-requ...@lists.sourceforge.net
wrote:
Hello Giovanni, I think you may need to del that slice and then possibly
repack. Hope this helps. Be Well Anthony On Mon, Aug 5, 2013 at 2:09 PM,
Giovanni Luca Ciampaglia glciamp...@gmail.com wrote:
Hello all,
is there a way to clear out a chunk from a CArray? I noticed that setting
the
data to zero actually takes disk space, i.e.
***
from tables import open_file, BoolAtom
h5f = open_file('test.h5', 'w')
ca = h5f.create_carray(h5f.root, 'carray', BoolAtom(), shape=(1000,1000),
chunkshape=(1,1000))
ca[:,:] = False
h5f.close()
***
The resulting file takes 249K ...
Best,
--
Giovanni Luca Ciampaglia
HDF5 handles efficiently chunks that have never been written saving some disk
space but I doubt that chunks can be de-initializad.
If my understanding is correct, once one write some value in a chunk (even if
it is the default value) the chunk is allocated at HDF5 level and written to
disk.
At they point one can only change item values.
Also I doubt that a repack can help in this case (not tested).
The only solution IMO is compression.
cheers
--
Antonio Valentino
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
☞ http://cnets.indiana.edu/
✉ gciam...@indiana.edu
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] Clear chunks from CArray
Hello Giovanni,
I think you may need to del that slice and then possibly repack. Hope this
helps.
Be Well
Anthony
On Mon, Aug 5, 2013 at 2:09 PM, Giovanni Luca Ciampaglia
glciamp...@gmail.com wrote:
Hello all,
is there a way to clear out a chunk from a CArray? I noticed that setting
the
data to zero actually takes disk space, i.e.
***
from tables import open_file, BoolAtom
h5f = open_file('test.h5', 'w')
ca = h5f.create_carray(h5f.root, 'carray', BoolAtom(), shape=(1000,1000),
chunkshape=(1,1000))
ca[:,:] = False
h5f.close()
***
The resulting file takes 249K ...
Best,
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
☞ http://cnets.indiana.edu/
✉ gciam...@indiana.edu
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] Clear chunks from CArray
Hi Anthony,
what do you mean precisely? I tried
del ca[:,:]
but CArray does not support __delitem__. Looking in the documentation I could
only find a method called remove_rows, but it's in Table, not CArray. Maybe I
am
missing something?
Thank,
Giovanni
On Mon 05 Aug 2013 03:43:42 PM EDT,
pytables-users-requ...@lists.sourceforge.net
wrote:
Hello Giovanni, I think you may need to del that slice and then possibly
repack. Hope this helps. Be Well Anthony On Mon, Aug 5, 2013 at 2:09 PM,
Giovanni Luca Ciampaglia glciamp...@gmail.com wrote:
Hello all,
is there a way to clear out a chunk from a CArray? I noticed that setting
the
data to zero actually takes disk space, i.e.
***
from tables import open_file, BoolAtom
h5f = open_file('test.h5', 'w')
ca = h5f.create_carray(h5f.root, 'carray', BoolAtom(), shape=(1000,1000),
chunkshape=(1,1000))
ca[:,:] = False
h5f.close()
***
The resulting file takes 249K ...
Best,
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
? 910 E 10th St ? Bloomington ? IN 47408
?http://cnets.indiana.edu/
?gciam...@indiana.edu
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
☞ http://cnets.indiana.edu/
✉ gciam...@indiana.edu
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] Clear chunks from CArray
On Mon, Aug 5, 2013 at 3:14 PM, Giovanni Luca Ciampaglia
glciamp...@gmail.com wrote:
Hi Anthony,
what do you mean precisely? I tried
del ca[:,:]
but CArray does not support __delitem__. Looking in the documentation I
could
only find a method called remove_rows, but it's in Table, not CArray.
Maybe I am
missing something?
Huh, it should... This is definitely an oversight on our part. If you
could please open an issue for this -- or better yet -- write a pull
request that implements delitem, that'd be great!
So I think you are right that there is no current way to delete rows from a
CArray. Oops! (Of course, I may still be missing something as well).
It looks like EArray also has this problem too, otherwise I would just tell
you to use that.
Be Well
Anthony
Thank,
Giovanni
On Mon 05 Aug 2013 03:43:42 PM EDT,
pytables-users-requ...@lists.sourceforge.net
wrote:
Hello Giovanni, I think you may need to del that slice and then possibly
repack. Hope this helps. Be Well Anthony On Mon, Aug 5, 2013 at 2:09 PM,
Giovanni Luca Ciampaglia glciamp...@gmail.com wrote:
Hello all,
is there a way to clear out a chunk from a CArray? I noticed that
setting
the
data to zero actually takes disk space, i.e.
***
from tables import open_file, BoolAtom
h5f = open_file('test.h5', 'w')
ca = h5f.create_carray(h5f.root, 'carray', BoolAtom(),
shape=(1000,1000),
chunkshape=(1,1000))
ca[:,:] = False
h5f.close()
***
The resulting file takes 249K ...
Best,
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
? 910 E 10th St ? Bloomington ? IN 47408
?http://cnets.indiana.edu/
?gciam...@indiana.edu
--
Giovanni Luca Ciampaglia
Postdoctoral fellow
Center for Complex Networks and Systems Research
Indiana University
✎ 910 E 10th St ∙ Bloomington ∙ IN 47408
☞ http://cnets.indiana.edu/
✉ gciam...@indiana.edu
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk
___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
