Hi P.O.,
This will be my last feedback for some time, promise.
>
there's absolutely no need to apologize, or stop posting.
Reporting bugs or (suspected) issues is always welcome!
ooRexx5.0 need 7.6 seconds
>
Don't use the lines() method repeatedly in a loop (please also see my other
reply)
The fastest way to read the lines of a file, is stream~arrayIn
Instead of appending each line into a MutableBuffer, just read it with charIn(,
size). This is of course even faster than arrayIn() and if you like, may
even squeeze out another 10% if you use stream~open("read nobuffer")
I haven't looked into the reported memory leak yet.
I have tried to do a minimalistic test case
>
Any Unix-style .rex program will also run on Windows .. no need to provide
a separate version for Windows
On Wed, Jul 5, 2017 at 6:44 PM, P.O. Jonsson <[email protected]> wrote:
> Dear developers,
>
> This will be my last feedback for some time, promise.
>
> For reading files I have implemented mutable buffers that I convert only
> once the loading is finished. This takes me around the problems I get
> trying to read data directly into Stems or Arrays.
>
> Unfortunately there is still a high memory load for this. When I run my
> read routine to read 1 million lines input say 10 times the memory
> allocated to the rexx process increases for every iteration. This is the
> same if I do it in a routine or directly in the main program. Also DROPing
> everything between iterations does not help, for every iteration there is
> some 100 MB memory added to the rexx process. Since I had planned to open
> and read 1000 files or more in one go this will likely exhaust my memory :-(
>
> I have checked on different machines, this behavior is the same for WIN10,
> WIN7 (32 and 64 bit ooRexx) and for MAX OS (3 different machines tested).
>
> In ooRexx 4.2 the behavior is similar but not as grave, the increase in
> memory/iteration is only about half of what I see with ooRexx 5.0
>
> In addition to this problem I have noticed that reading files in the way I
> do it is around 3-4 times slower in ooRexx 5.0 compared to ooRexx 4.2. I
> only saw this on MAC OS, not on Windows, but I installed 4.2 and 5.0 on the
> same hardware several times after each other to confirm that I heard
> reindeers, not horses ;-)
>
> To make a „baseline“ test I wrote a C program that is doing roughly the
> same as the oorexx test. Reading in 1 Million lines in C byte by byte takes
> 1.3 seconds, ooRexx 4.2 clocks in at 2.7 seconds (which is GREAT) and
> ooRexx5.0 need 7.6 seconds. All on the same hardware. This can´t be right,
> can it?
>
> As before I have tried to do a minimalistic test case that should run on
> any platform
>
> https://www.dropbox.com/sh/f8hircvoz0ph6uh/AACQeoi8lyAB3aoVQV3ojRkPa?dl=0
>
> If you are not interested in this kind of feedback just say so and I will
> remove myself from the developers list and go back to using 4.2.
>
> Hälsningar/Regards/Grüsse,
> P.O. Jonsson
> [email protected]
>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Oorexx-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
