Re: [PHP] stream_read function for registered wrapper class.

2012-09-19 Thread Adam Richardson
On Wed, Sep 19, 2012 at 12:43 PM, Rob rob_ad...@hotmail.com wrote:
 I have a very large XML file that I have to process.  It's about 7 GB.
 Some of the individual elements that I need are larger than 8192 bytes.
 I'm trying to write a Stream wrapper class to give me a specific element
 at a time, but I keep running into issues with the stream wrapper and
 fread, stream_get_content functions.

You could just use the XML Parser (SAX) as it doesn't require loading
the entire document into memory:
http://php.net/manual/en/book.xml.php

Adam

-- 
Nephtali:  A simple, flexible, fast, and security-focused PHP framework
http://nephtaliproject.com

-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



Re: [PHP] stream_read function for registered wrapper class.

2012-09-19 Thread Matijn Woudt
On Wed, Sep 19, 2012 at 6:43 PM, Rob rob_ad...@hotmail.com wrote:
 I have a very large XML file that I have to process.  It's about 7 GB.
 Some of the individual elements that I need are larger than 8192 bytes.
 I'm trying to write a Stream wrapper class to give me a specific element
 at a time, but I keep running into issues with the stream wrapper and
 fread, stream_get_content functions.

 fread will never return more than 8192 bytes (is that a bug?), so I can't
 use that at all.  When I use stream_get_content, and pass in 16384, weird
 things happen in my stream_read method.  If I return 16384 bytes, it gets
 truncated to 8192, and the stream_read method is called again and again,
 until the total returned is 16384.  Normally, stream_read is called twice
 in this case (since it returns 8192 both times).

 This really screws up what I'm trying to do with this stream wrapper.
 Admittedly, I don't know much about stream wrappers.  This is my first
 one, and I can't find much relevant documentation on them.  But it
 doesn't seem like it's working correctly to me.  If I ask for 16384 bytes
 with fread, shouldn't it give me 16384 bytes?  And if I return 16384
 bytes when I use stream_get_contents, shouldn't it return 16384 bytes the
 first time?

 Thanks for any help.

   -- Rob

I'm not sure if I'm missing something here, but PHP manual clearly
states that when using fread on streams you need to call fread in a
loop to get the amount of bytes required, as it will only return 1
packet at a time (TCP/IP are usually 8192 bytes).

(Though Adam is probably right that the builtin XML parser is the best way)

- Matijn

-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



Re: [PHP] stream_read function for registered wrapper class.

2012-09-19 Thread Rob
Sorry, I just realized my replies were going to email instead of the news 
group.

I'm going to try upgrading to php 5.4 so I have access to this:

http://php.net/manual/en/function.stream-set-chunk-size.php

And see if that fixes some of my problems.  Thanks for the suggestions.


-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php