On Mon, May 11, 2009 at 8:42 PM, beekeeper <[email protected]> wrote:
> My application is what I consider to be a pretty common use case. I > have a large data file, which is too large to hold within the > Android's limited memory. (For this test it was 2.8MB, but it could > just as easily be 50MB, which probably qualifies as "large" from a > G1's standpoint.) I need to make a single very fast scan of the file > to build an index, and then to be able to rapidly read any single > record within the file based upon that index. The latter is, of > course, straightforward if I can count on batch operations to be > moderately efficient for some variety of random access. Memory > mapping has always been my preferred technique, but this benchmark > shows that I'd be sacrificing a great deal of performance. Luckily, > it appears that FileChannels will do the job reasonably well. For what it's worth, most of the code running on the device I believe just uses FileInputStream/FileOutputStream/RandomAccessFile, and as long as you are reading and writing byte arrays those are pretty direct calls into the appropriate system calls. For the pre-scan pass, I'd again prefer to use memory mapping, since > it avoids boundary detection issues involved with buffered I/O. Since > I can't afford 36 seconds for a file scan, however, I'm left once > again resorting to FileChannels as a clumsy but reliable alternative. To be honest, I don't see the value of using mmap for any of this stuff in Java -- ultimately the data will need to be copied into a Java byte array (because you don't want to be doing a function call to get each byte), so I don't see how that will give you much benefit than just doing a call on RandomAccessFile to read the data into your own array. Also, for a 50MB file, I would strongly strongly suggest you stay away from mmapping the whole thing. It may work, but... given that we barely have that amount of memory available for all running applications, I would be concerned about the system getting into a paging state and causing it to noticibly slow down during that time. (Also, for some perspective if this is useful, we don't run with any swap, so you never ever want to touch any of the pages that get mmapped or they can no longer be paged out.) > It sounds as if I should harken to everyone's advice and start > contributing wholesale patches to improve the performance of the more > traditional I/O functions -- a single character read shouldn't be > blindingly fast, but it can certainly do better than allocating a one- > byte array and then doing a batch read. I'm used to the more heavy- > weight world of Sun's JDKs where one doesn't really expect patches to > be well received. (I'd still imagine quite a large lag time between > contributing an efficiency improvement and having it deployed "in the > wild" where my software would benefit from it. However, making the > world a better place is it's own reward.) I would expect good optimizations to be taken in fairly quickly, so probably included within a release or two (look at the times of the releases so far for very very rough idea of what kind of time-frame that might be). You can look at the platform groups and gerrit to see what kind of patches have already been accepted. Android is definitely not modeled after Sun's process (for better or worse depending on how you want to look at it). In particular we are much more concerned about having a single maintained code base that everyone delivers products out of, rather than defining specifications so that people can implement their own compatible versions... so Android is more the code base than a specification, and any improvements we take to that are implicitly a part of what Android is. -- Dianne Hackborn Android framework engineer [email protected] Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Android Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -~----------~----~----~----~------~----~------~--~---
