Dani,
Thanks for taking the time to test this and post your results. I looked at some
document generation routines where I was caching some text before writing it
out with SEND PACKET (to avoid lots of calls to SEND PACKET with small amounts
of text). I changed to a blob cache and it seemed to be a little slower. I also
tried pre-sizing the blob cache to avoid dynamic resizing with TEXT TO BLOB.
Still a bit slower than just using text variable and writing it out when it is
over 4K in size.
I compared your method 6 with just writing to a file, including opening and
closing the file. The time was almost the same with writing to a file being
slightly faster.
I'm sure there are cases where you can't use files, but if that is not a
constraint it seems you can avoid the memory question and get good performance.
I suspect SSD drives disk cache on modern systems are a big help these days.
John DeSoi, Ph.D.
$targetVarSize:=$maxSize
$innerLoopMax:=Int($targetVarSize/Length($textToAdd))
$vs6:=Milliseconds
For ($i;1;$outerLoopMax)
$result:=""
$tmpTxt:=""
SET BLOB SIZE($blob;0)
For ($j;1;$innerLoopMax)
$tmpTxt:=$tmpTxt+$textToAdd
If (Length($tmpTxt)>4096)
TEXT TO BLOB($tmpTxt;$blob;UTF8 text without length;*)
$tmpTxt:=""
End if
End for
TEXT TO BLOB($tmpTxt;$blob;UTF8 text without length;*)
$result:=BLOB to text($blob;UTF8 text without length)
End for
$ve6:=Milliseconds-$vs6
C_LONGINT($ms)
C_TIME($doc)
$ms:=Milliseconds
$doc:=Create document("test1.txt")
For ($i;1;$outerLoopMax)
$result:=""
$tmpTxt:=""
SET BLOB SIZE($blob;0)
For ($j;1;$innerLoopMax)
$tmpTxt:=$tmpTxt+$textToAdd
If (Length($tmpTxt)>4096)
//TEXT TO BLOB($tmpTxt;$blob;UTF8 text without
length;*)
SEND PACKET($doc;$tmpTxt)
$tmpTxt:=""
End if
End for
//TEXT TO BLOB($tmpTxt;$blob;UTF8 text without length;*)
//$result:=BLOB to text($blob;UTF8 text without length)
End for
CLOSE DOCUMENT($doc)
$ms:=Milliseconds-$ms
ALERT(String($ve6)+" - "+String($ms))
> On Sep 18, 2018, at 11:50 AM, Dani Beaubien via 4D_Tech
> <[email protected]> wrote:
>
> Based on this test, technique #5/#6 are the fastest by far.
> Techniques 4-6 seem to have linear performance.
>
> For 10 MB text var, the #4 came in at 1613ms and #5 came in at 290.6ms
>
> Very interesting. I am currently using techniques #3 when I am creating
> exports and reports to disk. Going to switch to the #5.
>
> Downsize of the blob techniques is that you need double the amount of memory
> if you need to convert the blob back to a text variable like I did in this
> experiment.
>
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive: http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:[email protected]
**********************************************************************
