Antoine, thanks for the quick reply. You can actually grow memorymapped files with a mremap call ( and I think a seek/write on the file ), I do this in my applications and it works fine. I want the efficiency of writing via memory maps, so would prefer to avoid FileOutputStream
-----Original Message----- From: Antoine Pitrou [mailto:anto...@python.org] Sent: Wednesday, May 09, 2018 10:37 AM To: dev@arrow.apache.org Subject: Re: Question about streaming to memorymapped files Hi, If you don't know the output size upfront then should probably use a FileOutputStream instead. By definition, memory mapped files must have a fixed size (since they are mapped to a fixed area in virtual memory). Regards Antoine. Le 09/05/2018 à 16:31, Ambalu, Robert a écrit : > Hey, I'm looking into streaming table updates into a memory mapped file ( C++ > ) > I think I have everything I need ( MemoryMappedFile output streamer, > RecordBatchStreamWriter ) but I don't understand how to properly create the > memmap file. It looks like it requires you to preset a size to the file when > you create it, but since ill be streaming I don't actually know how big a > file im going to need... > Am I missing some other API point here? Any reason why size is required up > front and the memmap doesn't auto-grow as needed? > > Thanks in advance > - Rob > > > > > > DISCLAIMER: This e-mail message and any attachments are intended solely for > the use of the individual or entity to which it is addressed and may contain > information that is confidential or legally privileged. If you are not the > intended recipient, you are hereby notified that any dissemination, > distribution, copying or other use of this message or its attachments is > strictly prohibited. If you have received this message in error, please > notify the sender immediately and permanently delete this message and any > attachments. > > > >
