Thanks for that thorough explanation! We're going to try using mmap(). It will 
be interesting to do this from Swift.


> On May 4, 2017, at 10:04 , David Duncan <[email protected]> wrote:
> 
> 
>> On May 3, 2017, at 11:51 PM, Rick Mann <[email protected]> wrote:
>> 
>>> 
>>> On May 3, 2017, at 23:27 , Doug Hill <[email protected]> wrote:
>>> 
>>> 
>>>> On May 3, 2017, at 6:21 PM, Rick Mann <[email protected]> wrote:
>>>> 
>>>> Our iOS app works with very large data buffers (hundreds of MB). As you 
>>>> can imagine, we run into issues at times.
>>>> 
>>>> I saw some sample code that used this technique, and it got me wondering 
>>>> if this actually works around the 600 MB limitation of some iOS devices 
>>>> (which begs another question: doss the large iPad Pro with 3GB of RAM also 
>>>> limit apps to 600 MB?).
>>>> 
>>>> -- 
>>>> Rick Mann
>>>> [email protected]
>>> 
>>> The limits of mmap should be the limits of the file system. Well, 
>>> technically it’s limited by the size_t parameter which I guess is OS 
>>> specific. But these days it should be what the VFS can handle.
>>> 
>>> I just tried out this code snippet:
>>> 
>>> http://www.linuxquestions.org/questions/programming-9/mmap-tutorial-c-c-511265/
>>>  
>>> <http://www.linuxquestions.org/questions/programming-9/mmap-tutorial-c-c-511265/>
>>> 
>>> and set the size of the mapped file to 32GB Works fine on a Mac with 16GB 
>>> of RAM.
>>> 
>>> It is SLOOOOOOWWWWWW however. You have to move all that data across a bus 
>>> after all. If you need to load 3GB of data from your file, I recommend 
>>> making it happen on a background thread. And make some lights blink and 
>>> play a happy tune while the user waits.
>>> 
>>> Not sure what the limitation on some iOS devices you’re referring to. Is it 
>>> dependent on the installed RAM? Are there other artificial limits?
>> 
>> Yeah, at least on 1 GB iOS devices, apps are limited to 600 MB (even if 
>> there's more available). I think there are subtleties to that (e.g. maybe 
>> texture memory isn't counted), I'm not sure.
> 
> iOS tracks memory per-process primarily in 3 buckets – Dirty, Resident, and 
> Virtual. The hierarchy is also in that order, all Dirty memory is Resident, 
> and all Resident memory is Virtual. Virtual just means that address space is 
> allocated – there is a hard limit on this that is below the theoretical 
> address space (significantly so in the case of iOS), in the 2-4GB range (may 
> vary by device installed memory, but I don’t recall). Resident memory is 
> memory that has physical RAM allocated to it and so the CPU can manipulate 
> it. This is the 600MB limit that you mentioned (at least on 1GB devices, 
> again I don’t recall if it is different or what it is on larger devices). 
> Dirty memory is memory that has been written to. Because iOS does not have a 
> demand paging memory system, this memory typically has no where to go if you 
> hit the Resident memory limits.
> 
> This is where mmap() comes into play – it allows you to create a Virtual 
> allocation, whose Resident memory is backed by a file, just like in a demand 
> paged system. So you can write to that memory, and if your app uses too much 
> memory, the OS can attempt to recover by writing it to disk on your behalf 
> (turning Resident-Dirty memory into Virtual). Sweet right?
> 
> But remember that hard Virtual limit I mentioned earlier – that means that if 
> you try to mmap() too much you end up running out of Virtual memory, and 
> you’ll likely get abort()’d by some subsystem attempting to allocate memory.
> 
> And so even though iOS is a 64-bit OS, due to other performance 
> considerations, the virtual address space is more akin to a 32-bit OS. This 
> is of course an implementation detail that can be expanded in the future, but 
> something to keep in mind if you plan to use mmap() a lot.
> 
> From the sounds of what our app does, keeping around a single mmap()’s 
> scratch space may be reasonable. You probably still want to keep it at a 
> modest size, but it should help you process those large data buffers without 
> getting your app killed.
> --
> David Duncan
> 


-- 
Rick Mann
[email protected]


_______________________________________________

Cocoa-dev mailing list ([email protected])

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to [email protected]

Reply via email to