The only memory settings that I am aware of are the 9!:20 and 9!:21 foreigns https://www.jsoftware.com/help/dictionary/dx009.htm
Cheers, bob > On Feb 3, 2023, at 23:47, Ak O <akin...@gmail.com> wrote: > > On Fri., Feb. 3, 2023, 05:17 Jan-Pieter Jacobs, <janpieter.jac...@gmail.com> > wrote: > >> 3!:2 just gives a representation of the noun. JMF memory-maps a file via >> the OS, so you can access the file's content as if it were just normal >> memory. It will stay only on disk as long as only in-place methods are used >> to change it (doing something like othervar=: +: mapped var will copy it >> entirely to memory anyhow). This page >> https://code.jsoftware.com/wiki/Mapped_Files and this page (basically the >> same as the jmf Lab): https://code.jsoftware.com/wiki/Studio/Mapped_Files >> explain how to use JMF, and how it works. >> >> I think the two mechanisms are entirely different, and both have their >> pro's and con's: >> >> reading data from file and applying 3!:2 is far simpler than JMF, and >> allows working with the data faster once read, as the data is copied to >> memory. JMF mapped nouns remain on disk, and incur the corresponding >> transfer speed penalty. > > Oh right, size is one factor. > How do I determine where the memory limit is (is this part of the 4gb > allocated to the j instance by the operating system)? > Can it be increased (or is this only for the jmf domain)? > > >> >> On the other hand JMF allows using data that doesn't fit in memory, but >> requires care to only apply in-place modification and care not to read more >> than fits your memory. >> > > Thx > > Ak. > > >> On Fri, 3 Feb 2023, 11:04 Ak O, <akin...@gmail.com> wrote: >> >>> I guess one of my questions directed at how the function works under the >>> hood. >>> >>> Is it that 3!:2 is the mechanism jmf uses to treat the map? >>> How do these forms differ? >>> >>> Ak >>> >>> On Thu., Feb. 2, 2023, 22:31 Raul Miller, <rauldmil...@gmail.com> wrote: >>> >>>> You can measure overhead with timespacex >>>> >>>> Maybe you had already been doing that? >>>> >>>> -- >>>> Raul >>>> >>>> On Thu, Feb 2, 2023 at 11:29 PM Ak O <akin...@gmail.com> wrote: >>>>> >>>>> Is this less overhead than the jmf form? >>>>> >>>>> In your example, every case where I want to operate on a file or an >>>> object >>>>> within the file requires translation through the 3!:2 operator. >>>>> >>>>> Maybe it is functionally the same as the map_jmf_ function, or am I >>>>> thinking about this incorrectly? >>>>> Or can you please explain the difference. >>>>> >>>>> Thx >>>>> >>>>> >>>>> Ak >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu., Feb. 2, 2023, 15:43 Don Guinn, <dongu...@gmail.com> wrote: >>>>> >>>>>> 'testfile.txt' fwrite~3!:1 'Hello World!';(i.3 4);<2 1$'Text >> Here';1 >>> 2 >>>> 3 4 >>>>>> 456 >>>>>> >>>>>> 3!:2 fread 'testfile.txt' >>>>>> >>>>>> ┌────────────┬─────────┬───────────┐ >>>>>> >>>>>> │Hello World!│0 1 2 3│┌─────────┐│ >>>>>> >>>>>> │ │4 5 6 7││Text Here││ >>>>>> >>>>>> │ │8 9 10 11│├─────────┤│ >>>>>> >>>>>> │ │ ││1 2 3 4 ││ >>>>>> >>>>>> │ │ │└─────────┘│ >>>>>> >>>>>> └────────────┴─────────┴───────────┘ >>>>>> >>>>>> On Thu, Feb 2, 2023 at 2:19 PM Ak O <akin...@gmail.com> wrote: >>>>>> >>>>>>> In this case the data is an array of boxes. >>>>>>> The datatype is 'boxe'd. >>>>>>> The data is of some shape. >>>>>>> >>>>>>> These things have to be preserved in order to correctly restore >>> them >>>>>>> later. (at some cost, up-front to specify them, afterwards to >>>> recall >>>>>> the >>>>>>> specifics, or something else) It might be easy to write as >> literal, >>>> but >>>>>>> reverting back can present some challenges. >>>>>>> >>>>>>> For example: >>>>>>> ary=: 'abcd';4 5 6 7; 2 2 $ 8.9 7.6 .6.5 5.4 >>>>>>> >>>>>>> datatype ary >>>>>>> boxed >>>>>>> >>>>>>> datatype ":ary >>>>>>> literal >>>>>>> ***Danger*** >>>>>>> $ary NB. An array of structures that preserves >> operational >>>>>>> intention. >>>>>>> 3 >>>>>>> $":ary NB. An array whose structure is different than its >>>>>>> operational intention. >>>>>>> 4 22 >>>>>>> >>>>>>> These are different and need to be accounted for. >>>>>>> >>>>>>> datatype (>0{ary) >>>>>>> literal >>>>>>> >>>>>>> datatype (>1{ary) >>>>>>> Integer >>>>>>> >>>>>>> datatype (>2{ary) >>>>>>> floating >>>>>>> >>>>>>> >>>>>>> Each unit has a shape and a type to preserve. >>>>>>> >>>>>>> If I want to operate on this object with the meaning intended for >>> it >>>> when >>>>>>> it was created, many errors can be avoided when unnecessary >>>> intermediate >>>>>>> conversions are avoided. >>>>>>> >>>>>>> Jmf allows you to operate directly without having the need to >>>>>>> revert from literal to your operation/operand, datatype (with the >>>>>>> associated overhead and tracking).I am not saying that there is >> no >>>>>> overhead >>>>>>> with jmf, just that the cognitive load is offset or preset >> because >>>> jmf >>>>>>> treats it for the user. >>>>>>> >>>>>>> >>>>>>> (1!2 & 1!:/, fwrite , fread) forms. >>>>>>> 1. Convert to literal. >>>>>>> 2. Write literal to file. >>>>>>> 3. Read literal from file. >>>>>>> 4. Convert literal type to operation type >>>>>>> 5. Operate with type. >>>>>>> End >>>>>>> ___ >>>>>>> >>>>>>> jmf form. >>>>>>> 1. Create jmf file. >>>>>>> 2. Map jmf file. >>>>>>> 3. Operate with file. >>>>>>> End >>>>>>> >>>>>>> >>>>>>> I am interested in understanding better how the best performance >> is >>>>>>> achieved comparing these forms. >>>>>>> >>>>>>> Thoughts. >>>>>>> >>>>>>> >>>>>>> Ak >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu., Feb. 2, 2023, 07:51 bill lam, <bbill....@gmail.com> >>> wrote: >>>>>>> >>>>>>>> Using Jmf is overkill. Also what are the advantages of jmf over >>>>>> 3!:1/3!:2 >>>>>>>> in this case? >>>>>>>> >>>>>>>> On Thu, 2 Feb 2023 at 7:17 PM Ak O <akin...@gmail.com> wrote: >>>>>>>> >>>>>>>>> load 'jmf' >>>>>>>>> NB. Loads jmf facilities. >>>>>>>>> >>>>>>>>> testfile =: {2,\?50#75 >>>>>>>>> >>>>>>>>> fn =: jpath >>>> 'C:\Users\skip\J904-user\temp\foo\testfile_name.jmf' >>>>>>>>> NB. Links a covername to the file path. >>>>>>>>> createjmf_jmf_ fn;(([:(*&8)#);testfile) >>>>>>>>> NB. Create the container for your file. >>>>>>>>> NB. -fn is your reference name >>>>>>>>> NB. -(([:(*&8)#);testfile) allocates the size of yor fike in >>>> bytes >>>>>>>>> >>>>>>>>> map_jmf_ 'testfile_disk';fn >>>>>>>>> NB. Maps the file to a noun 'testfile_disk' >>>>>>>>> ] testfile_disk_jmf_ =: testfile >>>>>>>>> >>>>>>>>> unmap_jmf 'testfile_disk_jmf_' >>>>>>>>> NB. Release mappings >>>>>>>>> Or >>>>>>>>> Exit 0 >>>>>>>>> Close session. >>>>>>>>> >>>>>>>>> >>>>>>>>> New session. >>>>>>>>> >>>>>>>>> load 'jmf' >>>>>>>>> fn =: jpath >>>>>> 'C:\Users\skip\J904-user\temp\foo\testfile_name.jmf' >>>>>>>>> map_jmf_ 'testfile1';fn >>>>>>>>> NB. Map file >>>>>>>>> ] testfile1_jmf_ >>>>>>>>> NB. File loaded as noun 'testfile1_jmf_' >>>>>>>>> >>>>>>>>> Some potential benefits of the .jmf datatype: >>>>>>>>> -Preserves header and shape information. >>>>>>>>> -If you want to map you file as read only, use the following >>>> syntax >>>>>>>>> map_jmf_ 'testfile1';fn;'';1 >>>>>>>>> -If you would like it to be copy-on-write use the following >>>> syntax >>>>>>>>> map_jmf_ 'testfile1';fn;'';2 >>>>>>>>> >>>>>>>>> >>>>>>>>> Ak >>>>>>>>> >>>>>>>>> On Wed., Feb. 1, 2023, 21:48 'Skip Cave' via Programming, < >>>>>>>>> programm...@jsoftware.com> wrote: >>>>>>>>> >>>>>>>>>> I have a boxed noun: >>>>>>>>>> >>>>>>>>>> ] testfile =: {2,\?15#50 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> >> ┌────┬─────┬─────┬─────┬─────┬─────┬────┬────┬────┬────┬─────┬────┬───┬────┐ >>>>>>>>>> >>>>>>>>>> │9 21│21 47│47 37│37 13│13 33│33 20│20 4│4 49│49 6│6 25│25 >>>> 33│33 >>>>>> 9│9 >>>>>>>> 6│6 >>>>>>>>>> 43│ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >>> >> └────┴─────┴─────┴─────┴─────┴─────┴────┴────┴────┴────┴─────┴────┴───┴────┘ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I want to store it in the following location on my machine: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ]fn =. < 'C:\Users\skip\J904-user\temp' >>>>>>>>>> >>>>>>>>>> ┌────────────────────────────┐ >>>>>>>>>> >>>>>>>>>> │C:\Users\skip\J904-user\temp│ >>>>>>>>>> >>>>>>>>>> └────────────────────────────┘ >>>>>>>>>> >>>>>>>>>> Then I will close that J session. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Later, I will start a new J session, and I want to read >> that >>>> saved >>>>>>> file >>>>>>>>>> into a noun called 'test1' >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> What is the J code for writing the noun into a file in the >>>> first >>>>>>>> session? >>>>>>>>>> >>>>>>>>>> What is the J code to read the file into a noun in the >> second >>>>>>> session? >>>>>>>>>> >>>>>>>>>> Should I use 1!2 & 1!:1, or fwrite & fread, or something >>> else? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Where in the J doc are these file operations and their >>>> tradeoffs >>>>>>>>> described? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Skip >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Skip Cave >>>>>>>>>> Cave Consulting LLC >>>>>>>>>> >>>>>>> >>>> ---------------------------------------------------------------------- >>>>>>>>>> For information about J forums see >>>>>>> http://www.jsoftware.com/forums.htm >>>>>>>>>> >>>>>>>>> >>>>>> >>> ---------------------------------------------------------------------- >>>>>>>>> For information about J forums see >>>>>> http://www.jsoftware.com/forums.htm >>>>>>>>> >>>>>>>> >>>> ---------------------------------------------------------------------- >>>>>>>> For information about J forums see >>>> http://www.jsoftware.com/forums.htm >>>>>>>> >>>>>>> >>>> ---------------------------------------------------------------------- >>>>>>> For information about J forums see >>>> http://www.jsoftware.com/forums.htm >>>>>>> >>>>>> >>> ---------------------------------------------------------------------- >>>>>> For information about J forums see >>> http://www.jsoftware.com/forums.htm >>>>>> >>>>> >> ---------------------------------------------------------------------- >>>>> For information about J forums see >> http://www.jsoftware.com/forums.htm >>>> ---------------------------------------------------------------------- >>>> For information about J forums see http://www.jsoftware.com/forums.htm >>>> >>> ---------------------------------------------------------------------- >>> For information about J forums see http://www.jsoftware.com/forums.htm >>> >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
