Th e overhead for writing 1 k is the same as writing 1 meg to disk. Writing one element at a time will take much longer
Regards Chuck On Mon, Jan 6, 2020 at 4:18 AM Peter Bozek via 4D_Tech <[email protected]> wrote: > Mitch, > > If the speed is the issue, you may try to write/read array elements to disk > using SEND PACKET / RECEIVE PACKET. If you want to speed up reading, insert > number of elements into first line, like > > Create document > SEND PACKET(file;Size of array(arrayElement)+"\n") > loop for all elements in array > SEND PACKET(file;arrayElement{}+"\n") > end loop > Close Document > > and vice versa > > Open document > RECEIVE PACKET(file;$t;"\n") > arraySize:=Num($t) > array text(arrayElement;arraySize) > for 1..arraySize > RECEIVE PACKET(file;$t;"\n") > arrayElement{}:=$t > end for > Close Document > > The above should avoid a part, where you (or 4D) is resizing a big memory > block to insert additional array elements. > > -- > Peter Bozek > > > On Mon, Jan 6, 2020 at 5:09 AM Mitchell Shiller via 4D_Tech < > [email protected]> wrote: > > > Hi, > > > > I have large string arrays (about 200 k elements). > > I need to write and read them to disk. > > Speed is the most important criteria. > > > > Options > > 1) Create a text variable (loop with CR delimiter) and then TEXT TO > > DOCUMENT. > > 2) VARIABLE TO BLOB, COMPRESS BLOB, BLOB TO DOCUMENT > > 3) OB SET ARRAY, JSON STRINGIFY, TEXT TO DOCUMENT > > > > Obviously using the reverse commands to read from disk. > > Option one seems by far the slowest. > > > > Anyone know if 2 or 3 is faster? > > Does the COMPRESS BLOB help? The resultant file is about 2 MB in size > > without compression. > > If yes to COMPRESS BLOB , Any différence from native 4D vs. GZIP > > compression? > > > > Thanks as always. > > > > Mitch > > > > > > > > Sent from my iPad > > ********************************************************************** > > 4D Internet Users Group (4D iNUG) > > Archive: http://lists.4d.com/archives.html > > Options: https://lists.4d.com/mailman/options/4d_tech > > Unsub: mailto:[email protected] > > ********************************************************************** > > > > -- > -- > > Peter Bozek > ********************************************************************** > 4D Internet Users Group (4D iNUG) > Archive: http://lists.4d.com/archives.html > Options: https://lists.4d.com/mailman/options/4d_tech > Unsub: mailto:[email protected] > ********************************************************************** -- ----------------------------------------------------------------------------------------- Chuck Miller Voice: (617) 739-0306 Fax: (617) 232-1064 Informed Solutions, Inc. Brookline, MA 02446 USA Registered 4D Developer Providers of 4D, Sybase & SQL Server connectivity https://www.informed-solutions.com ----------------------------------------------------------------------------------------- This message and any attached documents contain information which may be confidential, subject to privilege or exempt from disclosure under applicable law. These materials are intended only for the use of the intended recipient. If you are not the intended recipient of this transmission, you are hereby notified that any distribution, disclosure, printing, copying, storage, modification or the taking of any action in reliance upon this transmission is strictly prohibited. Delivery of this message to any person other than the intended recipient shall not compromise or waive such confidentiality, privilege or exemption from disclosure as to this communication. ********************************************************************** 4D Internet Users Group (4D iNUG) Archive: http://lists.4d.com/archives.html Options: https://lists.4d.com/mailman/options/4d_tech Unsub: mailto:[email protected] **********************************************************************
