Since you've not closed any of the files after processing, I suspect you
are running out of file handles, not memory.
Rick
On Sat, Sep 24, 2016 at 7:31 AM, Bertram Moshier <bertrammosh...@gmail.com>
wrote:
> Hello,
>
> I'm writing a program to remove duplicate lines from multiple files.
> (About 3600 files (971 or so have duplicate lines) and all files total
> about 512MB of space.)
>
> After reading in all the files, I've discovered ooRexx will no longer
> write to any hard drive on the system. I suspect this is a memory issue,
> as usage is above 3GB and can be above 4GB. YES, I AM using the 64 bit
> version.
>
> The version number: REXX-ooRexx_4.2.0(MT)_64-bit 6.04 22 Feb 2014
>
> The program runs fine even when memory usage exceed 8GB, except for I/O
> (specifically output).
>
> Below you'll find some of the code. PLEASE note: the !!Say_Directed
> subroutine is like the UNIX tee command. It can output to both the console
> and file. It was my first indication of a problem, as any output would go
> to the console BUT not the file!
>
> The !!EOJ subroutine generates a stop and timing information message.
> This is the FAILURE as the line does not get written to disk (RC = 1 for
> lineout). The line does get written to disk if the !!EOJ occurs BEFORE the
> last do loop (e.g. after the SysFileTree). The memory usage of the last DO
> LOOP is what takes ooRexx to 3-4GB and higher.
>
> NOTE: The system is running Windows 10 Professional AND has 48GB of
> physical RAM.
>
>
> Here is a piece of the code:
>
> files_with_duplicate_lines. = ''
> files_with_duplicate_lines.0 = 0
>
> GOODMARK = 'GOOD: '
>
> OFN = 0
> output_files. = ''
> input_files. = ''
>
>
> rc = SysFileTree('C:\Program Files (x86)\Windower4\logs\*.log','
> files','O')
> if rc <> 0 then do
> call !!EOJ 1000 + rc
> end
>
> do filenum = 1 to files.0
> do lines = 1 while lines(files.filenum) > 0
> files.filenum.lines = linein(files.filenum)
> end
> files.filenum.0 = lines - 1
> end
>
> call !!EOJ 0
>
>
>
> ------------------------------------------------------------
> ------------------
>
> _______________________________________________
> Oorexx-users mailing list
> Oorexx-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-users
>
>
------------------------------------------------------------------------------
_______________________________________________
Oorexx-users mailing list
Oorexx-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-users
