Thanks Marten. I tried memap for a few things but it seemed to create an other OS level buffer in specific situations. I think the `seek` operation in the `memmap` also caused some performance bottlenecks. Maybe I'll have time to summarize my findings an other day.
The particular usecase of `ffmpeg` is tricky since it is grabbing a lot of data from `stdout` which isn't a typical file buffer. Specifically, it is often `buffered` but `unseekable`. On Mon, Nov 5, 2018 at 9:38 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Mark, > > Having an `out` might make sense. With present numpy, if you are really > dealing with a file or file-like object, you might consider using > `np.memmap` to access the data more directly. If it is something that looks > more like a buffer, `np.frombuffer` may be useful (that doesn't copy data, > but points the array at the memory that holds the buffer). > > All the best, > > Marten > > > On Sun, Nov 4, 2018 at 10:35 PM Mark Harfouche <mark.harfou...@gmail.com> > wrote: > >> I was wondering what would your thoughts be on adding an output parameter >> to np.fromfile? >> >> The advantage would be when interfacing with executables like ffmpeg >> which are arguably easier to use by calling them as a subprocess compared >> to a shared library in python. >> >> Having the output parameter in np.fromfile would enable pre-allocation >> of large arrays that are reused during the computation of new image frames >> when decoding large video files. >> >> Thoughts are appreciated! >> >> Mark >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
