On Mon, Mar 16, 2015 at 8:15 PM Marcus Ottosson <[email protected]>
wrote:
> Thanks for trying this out Justin. I’ll address this one first in case it
> affects your interest in the other answers.
>
> All this academic stuff aside, are you pretty confident that your
> bottleneck is disk IO on the scene file?
>
> No, it certainly isn’t the bottleneck, I though that much was clear from
> the first tests I posted in which exporting was many times faster, even
> *with* writing to disk. This is out of plain curiosity only.
>
Ya no worries. It is exactly the same reason I am interesting in this
conversation as well. Just had to ask.
> So what Maya does is it moves the given path to a temp name if it
> already exists, then writes the new scene to your given path, and if all
> succeeds, it removes the previously existing one.
>
> Would it be possible that it does things differently, when writing to a
> non-existing file? In my case, it’s safe to assume the path will always be
> different and non-existing. Or were you referring to the fact that an mmap
> file must be written to disk before being manipulated?
>
> Here’s my limited understanding.
>
> import mmapimport contextlib
>
> cmds.file(rename="temp.ma") # Does not exist
> path = cmds.file(sceneName=True, q=True)
> # This has to happen, which makes it exist, which may cause the behaviour you
> described.with open(path, "wb") as f:
> f.write("empty")
> with open(path, "r+b") as f:
> with contextlib.closing(mmap.mmap(f.fileno(), 0)) as mm:
> cmds.file(exportAll=True, force=True)
>
> In which case, cmds.file is attempting to open a *new* file handle via
> the path given; but it won’t work (obviously), and I’m getting a “Can’t
> write to …” error.
>
That previous strace was on my linux workstation, at work. A dtruss on my
OSX laptop shows the exact same behavior. In your example, I think you are
overlooking the fact that you still create a file ahead of time, when you
establish the file and then memory map it. What Maya does is the same,
where it sees the existing file and renames it to a temporary one, and then
proceeds to open the original name freshly, and writes the scene to it. So
the memory mapped file and the one Maya writes are independent.
The only thing that is different when the named file absolutely doesn't
exist and you issue a save from Maya is that there is no rename of a
previously existing file.
If you are curious, you can watch the dtruss output:
$ sudo dtruss -p <maya pid>
or on linux:
$ strace -p <maya pid> -e trace=file
> I also had a look at an example file translator, and it looks like this is
> it’s given arguments.
>
> def writer(self, fileObject, optionString, accessMode):
>
> Reference
> <http://stackoverflow.com/questions/26585667/api-exporting-issue>
>
> In which case fileObject is a MFileObject from which a fullName() is
> taken and given to a Python open() command. If that’s the case, then you
> couldn’t give it a file-handle directly which makes mmap possibly
> impossible (?) without writing a translator yourself and making one in
> there.
>
Since I haven't written a file translator, I would hate to speak falsely.
But my impression of the interface is that you are handed a target path and
then it is up to you to perform the complete write. At that point, you are
basically free to do it however you want, be it a memory mapped file, a
pipe, tempfs, or writing nothing at all and updating your Facebook status.
I suppose really it is just a hook into the standard IO mechanisms for file
handling in Maya, and kind of feels similar to if you just had a function
called do_stuff() that did your same custom export into memory. It seemed
like the real thing you were after was to perform the native mayaAscii
scene export into memory instead of a disk-backed file, but apparently all
of these public routes are limited to just giving your expected file path.
Whereas what you probably want is to be able to provide an open file
descriptor.
It is completely possible that I am overlooking another way, but this is
all I can seem to see so far.
> On my linux box, by default, I have “/dev/shm” which is the shared
> memory tmpfs device.
>
> That could work; the solution can be studio-specific and doesn’t need to
> be cross-platform. The question then is whether it’s simpler to write your
> own translator, or setup and work with a tempfs device.
>
Well if it is indeed Linux, then /dev/shm probably already exists depending
on your distro. So that would be really easy since it is just like writing
a normal file. You have to explicitly mount and unmount one on OSX, and I
don't know how to do it on Windows. But if what I assumed about the
translator option is true, and your goal is to use the internal mayaAscii
export, then it doesn't sound like you have much of a choice.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Python Programming for Autodesk Maya" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODoaYPfPBWdpuS_weNnD3cPy-rCtac0K28scAJN4YDivA%40mail.gmail.com
> <https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODoaYPfPBWdpuS_weNnD3cPy-rCtac0K28scAJN4YDivA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA2QBqLkcnk2LLjfX8uCwnaG_rVu69EfkwbX3LJrHR2T3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
