Re: [Pytables-users] using vlarray

2009-12-15 Thread Francesc Alted
A Tuesday 15 December 2009 17:31:46 Ernesto escrigué: > thank you for your explanation and for your time. I'll try to use a > combined approach. > A very stupid question is the following: > > Normally I have to update each string of the vlarray (and in the table > as well) adding a character at th

Re: [Pytables-users] using vlarray

2009-12-15 Thread Ernesto
>I've been doing some benchmarks based on your requirements, and my conclusion >is that the implementation of variable length types in HDF5 is not very >efficient, specially with HDF5 1.8.x series (see [1]). So, you should avoid >using VLArrays for saving small arrays: they fit better in

Re: [Pytables-users] using vlarray

2009-12-14 Thread Francesc Alted
A Monday 14 December 2009 14:25:48 Francesc Alted escrigué: > A Saturday 12 December 2009 18:45:33 Ernesto escrigué: > > Dear Francesc, > > > > thank you for your reply. I'll try to better explain my problem using > > real examples of data and code. > > [clip] > > I've been doing some benchmarks

Re: [Pytables-users] using vlarray

2009-12-14 Thread Francesc Alted
A Saturday 12 December 2009 18:45:33 Ernesto escrigué: > Dear Francesc, > > thank you for your reply. I'll try to better explain my problem using > real examples of data and code. [clip] I've been doing some benchmarks based on your requirements, and my conclusion is that the implementation of va

Re: [Pytables-users] using vlarray

2009-12-13 Thread llewelr
Ernesto. I agree, I think pytables would be a good solution for his full (with gaps) multiple sequence alignment. Regarding your problem, hopefully others with more experience can help with optimization. Personally I've had troubles in similar situations where I had large amounts of varia

Re: [Pytables-users] using vlarray

2009-12-13 Thread Ernesto
Hi Richard, >This looks, in part, a way to store a huge multiple sequence alignment with >a reference sequence (the first character in ( ) being the DNA base in a >reference DNA molecule, but due to the inequal lengths in each VLA, it would >seem that gaps are not stored, or stored else

Re: [Pytables-users] using vlarray

2009-12-13 Thread Ernesto
I'm going to stab at understanding your problem. Correct me where I'm wrong. On Sat, Dec 12, 2009 at 12:45 PM, Ernesto wrote: As I wrote I start with an input file. It contains a string of variable length (10e7-10e8). This string consists of four different characters (A,C,G,T), the bases of a

Re: [Pytables-users] using vlarray

2009-12-12 Thread Richard Llewellyn
I'll join in the guessing game. This looks, in part, a way to store a huge multiple sequence alignment with a reference sequence (the first character in ( ) being the DNA base in a reference DNA molecule, but due to the inequal lengths in each VLA, it would seem that gaps are not stored, or stored

Re: [Pytables-users] using vlarray

2009-12-12 Thread Faisal Moledina
I'm going to stab at understanding your problem. Correct me where I'm wrong. On Sat, Dec 12, 2009 at 12:45 PM, Ernesto wrote: > As I wrote I start with an input file. It contains a string of > variable length (10e7-10e8). This string consists of four different > characters (A,C,G,T), the bases of

Re: [Pytables-users] using vlarray

2009-12-12 Thread Brent Pedersen
On Sat, Dec 12, 2009 at 9:45 AM, Ernesto wrote: > Dear Francesc, > > thank you for your reply. I'll try to better explain my problem using > real examples of data and code. > > As I wrote I start with an input file. It contains a string of > variable length (10e7-10e8). This string consists of fou

Re: [Pytables-users] using vlarray

2009-12-12 Thread Ernesto
Dear Francesc, thank you for your reply. I'll try to better explain my problem using real examples of data and code. As I wrote I start with an input file. It contains a string of variable length (10e7-10e8). This string consists of four different characters (A,C,G,T), the bases of a DNA mo

Re: [Pytables-users] using vlarray

2009-12-11 Thread Francesc Alted
Ciao Ernesto, A Thursday 10 December 2009 21:19:00 Ernesto escrigué: > Dear all, > > I'm new to pytables. I would use this package to store biological > data. In practice I have 2 input files. > The first consists in a long string of characters. The length is in > the range 10e7-10e8. I have to r

[Pytables-users] using vlarray

2009-12-10 Thread Ernesto
Dear all, I'm new to pytables. I would use this package to store biological data. In practice I have 2 input files. The first consists in a long string of characters. The length is in the range 10e7-10e8. I have to read such file character by character and store it in a table according to it