If the /: and I. family do not have this limitation, it seems like it would be straightforward to include a fallback for large arrays. As in: literally: check the length of the array and if it's too large evaluate a J explicit definition which implements indexof.
But we'd need a version without the limit error (and some patience) to test the code. (So... j807 for that??) Thanks, -- Raul On Tue, Dec 17, 2019 at 11:04 AM Henry Rich <[email protected]> wrote: > > The implementation of the i.-family does not allow more than 2e9 items > in an array. I put a limit of 2^32 atoms as creating a limit error. > This is what you are hitting. > > If you were allowed to create a 7G character array, you would not be > allowed to perform all operations on it. > > Henry Rich > > On 12/17/2019 9:27 AM, Ric Sherlock wrote: > > 1!:1 <'mybigfile.csv' > > returns a limit error on J901 but completes successfully on j64_807. > > > > On Tue, Dec 17, 2019 at 9:56 PM bill lam <[email protected]> wrote: > > > >> does 1!:1 work? > >> > >> On Tue, Dec 17, 2019, 1:09 PM Ric Sherlock <[email protected]> wrote: > >> > >>> I have run into a difference between J901 and J807 that I hadn't come > >>> across before when dealing with large (~7GB) files. > >>> > >>> In both versions I can see the file > >>> fexist 'mybigfile.csv' > >>> 1 > >>> fsize 'mybigfile.csv' > >>> 7516003225 > >>> > >>> I can also read the beginning the file in both versions > >>> $freads 'mybigfile.csv';0 2000 > >>> 2001 > >>> > >>> However if I try to read the whole file, J807 works, where J901 returns > >>> immediately with 256{a. > >>> $freads 'mybigfile.csv' NB. J807 > >>> 7516003225 > >>> $freads 'mybigfile.csv' NB. J901 > >>> > >>> This feels like a 64 bit issue or similar but I wouldn't have thought > >> that > >>> should be an issue? Any ideas? > >>> > >>> JVERSION > >>> Engine: j901/j64avx2/linux > >>> Release-a: commercial/2019-12-12T12:44:48 > >>> Library: 9.01.18 > >>> Platform: Linux 64 > >>> Installer: J901 install > >>> InstallPath: /opt/j/j901 > >>> Contact: www.jsoftware.com > >>> > >>> JVERSION > >>> Engine: j807/j64/linux > >>> Release-d: commercial/2019-03-18T12:11:49 > >>> Library: 8.07.26 > >>> Platform: Linux 64 > >>> Installer: J807 install > >>> InstallPath: /opt/j/j64_807 > >>> Contact: www.jsoftware.com > >>> ---------------------------------------------------------------------- > >>> For information about J forums see http://www.jsoftware.com/forums.htm > >>> > >> ---------------------------------------------------------------------- > >> For information about J forums see http://www.jsoftware.com/forums.htm > >> > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
