>
> maybe it is just bad programming
Hi P.O.,
I had a look at Split_data and as far as I can see there are a lot of
things which can be improved.
1)
You may want to re-read how to work with a MutableBuffer. E. g.
tempMB = .mutablebuffer~new('')
do while ..
tempMB = qfileIn~linein
Initializing a variable with a MutableBuffer instance, and afterwards
assigning it a String (linein() resturns a String) doesn't make sense.
I can see quite a few instances of this, e. g.
TranslatedMB = .mutablebuffer~new('')
do while ..
DO i=1 TO i_End
DO j=1 TO j_End
TranslatedMB = TranslatedMB TranslateWordMB
Again, the final TranslatedMB assignment is not what the ..MB ending of the
variables suggest.
2)
You might move invariant stuff (here: LeftWordsMB~Word(i) || '-') in an
inner loop outside the loop, e.g.
DO j=1 TO j_End
TranslateWordMB = LeftWordsMB~Word(i) || '-' || RightWordsMB~Word(j)
3)
Consider using use a single startsWith() instead of the code between lines
448 and 485
4)
IF TranslatedMB~WordPos(TranslateWordMB) > 0 THEN
..
ELSE
DO
TranslatedMB = TranslatedMB TranslateWordMB
Instead of building a long string of all things seen before, and checking
with wordPos(), you might instead put all things seen into a Set and check
with hasIndex()
5)
Generally, using Arrays may be more efficient if you can save the Stem.0
handling
But then, using the proper type of Collection and appropriate algorithm may
help much more
To give suggestions for that, I'd need more detail would on what exactly
you would like to achieve
On Tue, Jun 27, 2017 at 7:55 AM, P.O. Jonsson <oor...@jonases.se> wrote:
> Dear developers,
>
> I have had the memory bloating problem again, this time I reached 48 GB
> (the maximum for one CPU in my machine) and the process only ended after
> some 13 CPU hours with 100% CPU the whole time.
>
>
>
>
> From the logging info I could confirm that the program was stuck somewhere
> here most of the time, here are the rough steps
>
> Language pairs detected in C routine -> External call, no memory bloating
> Data processing finished after 2107 Seconds 00:58:12
> Splitting finished after 49487 Seconds 14:42:59 *-> Routine Split_data*
> Sorting finished after 16527 Seconds 19:18:27 *-> Routine Sort_data*
> Processing of Data file finished after 68123 Seconds
> Writing the Logfile TR_DE-EN-eu_logfile.txt 26 Jun 2017 19:18:28
>
> I have enclosed the Routines in question.
>
> In my dropbox I have stored the complete program with some test data to
> replicate the processing, the problem is reproducible. Just put the folder
> somewhere, move there and perform the command indicated.
>
> https://www.dropbox.com/sh/vettlcb4f8ae3cw/AACWIQivo_F2KhhytJ6izkbFa?dl=0
>
> I run Open Object Rexx Version 5.0.0, Build date: May 20 2017, Addressing
> mode: 64
> Hardware Mac Pro with dual-CPU Xeon Processors running Mac OS Sierra
> 10.12.5
>
> PS as I was making the screenshot the process finished nicely, no crash or
> anything and the memory was released. So maybe it is just bad programming,
> but at least you can confirm that then :-)
>
>
>
>
> Hälsningar/Regards/Grüsse,
> P.O. Jonsson
> oor...@jonases.se
>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
