One solution is read line by line process it write to a temporary file on
you hard drive . This way you need say a StringBuffer as big as a line long
500 should do it although it does not matter. Every time you start a new
line use setLength( 0 ); and start appending .
When you are done do what ever you have to with the content of you temporary
file and delete the file.
The question here is if you are reading over the network and sending it back
over a network, then just read over the network in to StringBuffer process
your data and send it back over the network. This way your machine acts as a
filter and at any time it is using memory as long as a line.
Use BufferedReader and OutputStreamWriter also read the comment on readLine
( ) carefully, there is a catch there !!!!!!!!!!!!!!!
I suggest you read Java Tutorial for garbage collection
Quanyu Zhu wrote:
> Hi:
> I just have one mission to read a huge file(more than 500M) using
> java. Is there anyone having such kind of experiences on that?
> My system is Java1.1.7a and running on Windows95/NT or Linux with
> X-windows. And my runtime configuration is setting up 100M to 500M
> memory size ,but my real Hdisk just left 500M free space.
> In my program, Firstly, I used the DataInputStream to read one byte by
> byte, but it seems too slow.
> Then I used the ReadFully to read the whole file into one large btye
> array. The performance imporved a lot.
> But what I will deal with is the huge file, more than 500M, how can I
> allocate this huge array?
> Does anyone have such experience on that?
> Is there any way to free the array I allocated like in C/C++? It seems
> the only way to free the unused memory is done by garbage collection
> thread, do I have any way to destory the memory which I allocated just
> using one method ???
> Any hint will be appreciated !!!
>
> Best Regard
> Kevin Zhu
>
>
> ______________________________________________________
> Get Your Private, Free Email at http://www.hotmail.com
