Hi,
Wolfgang Maier <wolfgang.maier <at> biologie.uni-freiburg.de> writes:
>
> Dear all,
> I was just experimenting for the first time with os.posix_fadvise(), which
> is new in Python3.3 . I'm reading from a really huge file (several GB) and I
> want to use the data only once, so I don't want OS-level page caching. I
> tried os.posix_fadvise with the os.POSIX_FADV_NOREUSE and with the
> os.POSIX_FADV_DONTNEED flags, but neither seemed to have any effect on the
> caching behaviour of Ubuntu (still uses all available memory to page cache
> my I/O).
> Specifically, I was trying this:
>
> import os
> fd = os.open('myfile', os.O_RDONLY)
> # wasn't sure about the len parameter in fadvise,
> # so thought I just use it on the first 4GB
> os.posix_fadvise(fd, 0, 4000000000, os.POSIX_FADV_NOREUSE) # or DONTNEED
The Linux version of "man posix_fadvise" probably holds the answer:
"In kernels before 2.6.18, POSIX_FADV_NOREUSE had the same semantics
as POSIX_FADV_WILLNEED. This was probably a bug; since kernel
2.6.18, this flag is a no-op."
"POSIX_FADV_DONTNEED attempts to free cached pages associated with the
specified region. This is useful, for example, while streaming large
files. A program may periodically request the kernel to free cached
data that has already been used, so that more useful cached pages are
not discarded instead."
So, in summary:
- POSIX_FADV_NOREUSE doesn't do anything on (modern) Linux kernels
- POSIX_FADV_DONTNEED must be called *after* you are done with a range of
data, not before you read it (note that I haven't tested to confirm it :-))
Regards
Antoine.
--
http://mail.python.org/mailman/listinfo/python-list
