Re: dynamic allocation file buffer

2008-09-15 Thread Francesc
On 12 Set, 14:39, Aaron \Castironpi\ Brady [EMAIL PROTECTED] wrote: A consideration of other storage formats such as HDF5 might be appropriate: http://hdf.ncsa.uiuc.edu/HDF5/whatishdf5.html There are, of course, HDF5 tools available for Python. PyTablescame up within the past few weeks

Re: dynamic allocation file buffer

2008-09-12 Thread Steven D'Aprano
On Thu, 11 Sep 2008 22:40:01 -0700, Dennis Lee Bieber wrote: On 12 Sep 2008 03:37:51 GMT, Steven D'Aprano [EMAIL PROTECTED] declaimed the following in comp.lang.python: I'm pretty sure you're wrong. XML can be used for serialization, but that doesn't mean it is only sequential data. XML

Re: dynamic allocation file buffer

2008-09-12 Thread Aaron Castironpi Brady
On Sep 12, 1:30 am, Steven D'Aprano [EMAIL PROTECTED] wrote: On Thu, 11 Sep 2008 22:40:01 -0700, Dennis Lee Bieber wrote: On 12 Sep 2008 03:37:51 GMT, Steven D'Aprano [EMAIL PROTECTED] declaimed the following in comp.lang.python: I'm pretty sure you're wrong. XML can be used for

Re: dynamic allocation file buffer

2008-09-12 Thread Paul Boddie
On 12 Sep, 08:30, Steven D'Aprano [EMAIL PROTECTED] wrote: Which is why I previously said that XML was not well suited for random access. Maybe not. A consideration of other storage formats such as HDF5 might be appropriate: http://hdf.ncsa.uiuc.edu/HDF5/whatishdf5.html There are, of course,

Re: dynamic allocation file buffer

2008-09-12 Thread Aaron Castironpi Brady
On Sep 12, 4:34 am, Paul Boddie [EMAIL PROTECTED] wrote: On 12 Sep, 08:30, Steven D'Aprano [EMAIL PROTECTED] wrote: Which is why I previously said that XML was not well suited for random access. Maybe not. No, it's not. Element trees are, which if I just would have said originally...

Re: dynamic allocation file buffer

2008-09-11 Thread Steven D'Aprano
On Wed, 10 Sep 2008 11:59:35 -0700, Aaron \Castironpi\ Brady wrote: On Sep 10, 5:24 am, Steven D'Aprano [EMAIL PROTECTED] wrote: On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: Steven D'Aprano wrote: You've created a solution to a problem which (probably) only affects a very

Re: dynamic allocation file buffer

2008-09-11 Thread Fredrik Lundh
Steven D'Aprano wrote: I'm no longer *claiming* anything, I'm *asking* whether random access to a 4GB XML file is something that is credible or useful. It is my understanding that XML is particularly ill-suited to random access once the amount of data is too large to fit in RAM. An XML file

Re: dynamic allocation file buffer

2008-09-11 Thread Paul Boddie
On 11 Sep, 10:34, Fredrik Lundh [EMAIL PROTECTED] wrote: And as I said before, the only use case for *huge* XML files I've ever seen used in practice is to store large streams of record-style data; I can imagine that the manipulation of the persistent form of large graph structures might be

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron Castironpi Brady
On Sep 11, 2:40 am, Steven D'Aprano [EMAIL PROTECTED] wrote: On Wed, 10 Sep 2008 11:59:35 -0700, Aaron \Castironpi\ Brady wrote: On Sep 10, 5:24 am, Steven D'Aprano [EMAIL PROTECTED] wrote: On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: Steven D'Aprano wrote: You've created

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron Castironpi Brady
On Sep 11, 5:35 am, Paul Boddie [EMAIL PROTECTED] wrote: On 11 Sep, 10:34, Fredrik Lundh [EMAIL PROTECTED] wrote: And as I said before, the only use case for *huge* XML files I've ever seen used in practice is to store large streams of record-style data; I can imagine that the

Re: dynamic allocation file buffer

2008-09-11 Thread Paul Boddie
On 11 Sep, 19:31, Aaron \Castironpi\ Brady [EMAIL PROTECTED] wrote: An acquaintance suggests that defragmentation would be a useful service to provide along with memory management too, which also requires an index. I presume that you mean efficient access to large amounts of data in the sense

Re: dynamic allocation file buffer

2008-09-11 Thread Steven D'Aprano
On Thu, 11 Sep 2008 10:20:41 -0700, Aaron \Castironpi\ Brady wrote: XML is the wrong word for the example I was thinking of (as was already pointed out in another thread). XML is by definition sequential. I'm pretty sure you're wrong. XML can be used for serialization, but that doesn't mean

Re: dynamic allocation file buffer

2008-09-11 Thread Aaron Castironpi Brady
On Sep 11, 10:37 pm, Steven D'Aprano [EMAIL PROTECTED] wrote: On Thu, 11 Sep 2008 10:20:41 -0700, Aaron \Castironpi\ Brady wrote: XML is the wrong word for the example I was thinking of (as was already pointed out in another thread).  XML is by definition sequential. I'm pretty sure you're

Re: dynamic allocation file buffer

2008-09-10 Thread Fredrik Lundh
Steven D'Aprano wrote: You've created a solution to a problem which (probably) only affects a very small number of people, at least judging by your use-cases. Who has a 4GB XML file Getting 4GB XML files from, say, logging processes or databases that can render their output as XML is not

Re: dynamic allocation file buffer

2008-09-10 Thread Steven D'Aprano
On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: Steven D'Aprano wrote: You've created a solution to a problem which (probably) only affects a very small number of people, at least judging by your use-cases. Who has a 4GB XML file Getting 4GB XML files from, say, logging

Re: dynamic allocation file buffer

2008-09-10 Thread Aaron Castironpi Brady
On Sep 10, 5:24 am, Steven D'Aprano [EMAIL PROTECTED] wrote: On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote: Steven D'Aprano wrote: You've created a solution to a problem which (probably) only affects a very small number of people, at least judging by your use-cases. Who has a

Re: dynamic allocation file buffer

2008-09-10 Thread Aaron Castironpi Brady
On Sep 9, 10:03 pm, George Sakkis [EMAIL PROTECTED] wrote: On Sep 9, 5:59 pm, castironpi [EMAIL PROTECTED] wrote: I will try my idea again.  I want to talk to people about a module I want to write and I will take the time to explain it.  I think it's a cool idea that a lot of people,

Re: dynamic allocation file buffer

2008-09-10 Thread Paul Boddie
On Sep 10, 5:03 am, George Sakkis [EMAIL PROTECTED] wrote: So at best (i.e. if it actually makes any sense; I didn't read it), this is an ANNouncement of a pre-alpha piece of code. ANN posts rarely attract replies, even when they are about production/stable software. To be fair, at least some

dynamic allocation file buffer

2008-09-09 Thread castironpi
I will try my idea again. I want to talk to people about a module I want to write and I will take the time to explain it. I think it's a cool idea that a lot of people, forgiving the slang, could benefit from. What are its flaws? A user has a file he is using either 1/ to persist binary data

Re: dynamic allocation file buffer

2008-09-09 Thread Larry Bates
castironpi wrote: I will try my idea again. I want to talk to people about a module I want to write and I will take the time to explain it. I think it's a cool idea that a lot of people, forgiving the slang, could benefit from. What are its flaws? A user has a file he is using either 1/ to

Re: dynamic allocation file buffer

2008-09-09 Thread Steven D'Aprano
On Tue, 09 Sep 2008 14:59:19 -0700, castironpi wrote: I will try my idea again. I want to talk to people about a module I want to write and I will take the time to explain it. I think it's a cool idea that a lot of people, forgiving the slang, could benefit from. What are its flaws? [snip

Re: dynamic allocation file buffer

2008-09-09 Thread castironpi
On Sep 9, 5:44 pm, Larry Bates [EMAIL PROTECTED] wrote: castironpi wrote: I will try my idea again.  I want to talk to people about a module I want to write and I will take the time to explain it.  I think it's a cool idea that a lot of people, forgiving the slang, could benefit from.  

Re: dynamic allocation file buffer

2008-09-09 Thread castironpi
On Sep 9, 5:58 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: On Tue, 09 Sep 2008 14:59:19 -0700, castironpi wrote: I will try my idea again.  I want to talk to people about a module I want to write and I will take the time to explain it.  I think it's a cool idea that a

Re: dynamic allocation file buffer

2008-09-09 Thread George Sakkis
On Sep 9, 5:59 pm, castironpi [EMAIL PROTECTED] wrote: I will try my idea again.  I want to talk to people about a module I want to write and I will take the time to explain it.  I think it's a cool idea that a lot of people, forgiving the slang, could benefit from.   (snipped) A pure