Yes, that would make sense. It’d be like how mayaAscii and mayaBinary are
plain-text and binary respectively, but still equally compatible with Maya,
regardless of platform.
Perhaps it’s then safe to surmise that because pickle is spelled out as
being cross-platform compatible, and cPickle is compatible with anything
coming out of pickle, then anything coming out of cPickle is also
cross-platform compatible. Especially when considering this line.
The data streams the two modules produce are guaranteed to be
interchangeable.
To the original question, I had another idea that might prove useful.
The goal is to embed a dictionary - i.e. nested - datastructure in Maya,
and one way of doing so is to take a Python dictionary, convert it to a
string and store it in a Maya string plug.
However, what if you were to consider Maya’s plugs as key/value pairs,
where the attribute name is the key, and its value is.. well, the value? In
that case, you could either:
1. Use regular attributes, of varying types, and query them like normal,
e.g. cmds.getAttr("myNode.key")
2. Use a compound attribute, to mimic a Python dictionary, such that the
value returned from cmds.getAttr("myNode.myCompound") is similar to what
you would normally get out of a dictionary.
data = {
"project": "Avengers",
"shot": "0050",
"task": "animation",
"started": True
}
cmds.getAttr("myNode.myCompound.project") == u"Avengers"
cmds.getAttr("myNode.myCompound.shot") == u"0050"
cmds.getAttr("myNode.myCompound.started") == True
It’d even get you all attributes at once if you ask nicely.
cmds.getAttr("myNode.myCompound") == (u"Avengers", u"0050", u"animation", True)
However probably not in the order of your original dictionary, and without
the actual keys. That said, the order is reliable, as it would be in the
order that the attributes were added, so you may be able to determine the
key separate from your query. Note that this would also work with nested
compound attributes, and more datatypes than would otherwise be supported
by JSON, e.g. long, double3, mesh and nurbsCurve data.
So in this case, then Yes, you can embed a dictionary. :)
On 3 March 2018 at 19:38, Justin Israel <justinisr...@gmail.com> wrote:
>
>
> On Sat, Mar 3, 2018, 10:59 PM Marcus Ottosson <konstrukt...@gmail.com>
> wrote:
>
>> then it should be able to be dumped on Maya 2013 in Linux and read in
>> Maya 2018 on Windows.
>>
>> TLDR; compiler differences and unavailable internal datastructures.
>>
>> For starters, one thing that could be confusing is how there is pickle
>> and then there is cPickle. pickle is a pure Python library which I
>> expect have no trouble with cross-platform compatibility. cPickle on the
>> other hand is a compiled library and, like anything compiled, I’d expect
>> the same incompatibility as with it as with e.g. PyQt unless it’s compiled
>> for the particular version of Maya’s Python interpreter, and the same for
>> NumPy and friends. (Can this be *not* true?)
>>
>> However aside from that, with both pickle and cPickle you are still
>> serialising internal data structures and not all datastructures on Linux
>> are available on Windows, not all in Maya 2013 are available in Maya 2018.
>> In the simplest case of incompatibility, you could pickle a PyMEL object in
>> Maya 2018 that hasn’t yet been invented in 2013, or a MASH object that
>> didn’t exist in 2013. Or pickle some instance of one of your own plug-ins,
>> that you’ve got a different version of on another computer.
>>
>> I would think that the only way to be safe in a multi-OS, multi-Maya
>> environment, is to both stick with pickle (not cPickle) *and* ensure
>> your data is JSON-serialisable to begin with (no complex data). But in that
>> case, you’ve run out of reasons to use pickle over json anyway.
>>
>> Taking a closer look at the documentation, I also spotted..
>>
>> The pickle serialization format is guaranteed to be backwards compatible
>> across Python releases.
>>
>> Which I would read “data pickled in Maya 2013 is guaranteed to be
>> compatible with Maya 2018”, but not the other way around (that would be
>> *forwards* compatibility). This, I think, would invalidate even the
>> “safe” option I listed just now.
>>
>> With all this said, I did expect to find larger warning signs explicitly
>> written out in both the Python docs and community, but can’t seem to find a
>> conclusive answer. So with that in mind, I’ll change my position from “Odds
>> are you will run into trouble” to “Odds are you *may* run into trouble”.
>> Would you agree?
>>
>
> The only part of all those points that I agree with is that you would have
> trouble if you serialised a custom object that was not available on a
> subsequent load of that data. It could happen for reasons beyond just two
> versions of Maya. One install could be missing a plugin or library that the
> original used. But that can be avoided if one just makes sure to serialise
> only basic types, or types that are part of the same program that is doing
> the loading and saving. I would like to believe that this data is part of
> some program, which has some logic around reading and writing the data.
> This program would use the data consistently regardless of Maya 2013 or
> 2018. You couldnt expect a program to always work correctly if some
> external code was modifying the custom data outside the spec of the
> program.
>
> On your other points... Pickle vs cPickle is just a compiled
> implementation vs pure python, for speed ups. The protocol is the same. And
> no one is manually building and installing cPickle because it is part of
> the CPython stdlib. If you are worried about Maya 2020 having some new
> python implementation that no longer ships with cPickle, then the program
> can just do
>
> try:
> import cPickle as pickle
> except ImportError:
> import pickle
>
> https://pymotw.com/2/pickle/
>
> And regarding the serialised format compatibility, if you specify the
> protocol number then you use the same format regardless of future versions.
> Use protocol 2 now, and continue using it in your program even when Maya
> 2020 ships with python 3 and cPickle has a protocol 3 available (not real).
>
> It would not be expected that random programs would be reading and
> changing this serialised data if it is in a binary format. It would need to
> be specific to a python program that that needs it for future runs. If this
> data is meant to be used by anyone, such as a metadata system, then use a
> plaintext human readable format like json. But it depends on the purpose
> for this serialisation.
>
>
>>
>> On 2 March 2018 at 20:54, Justin Israel <justinisr...@gmail.com> wrote:
>>
>>>
>>>
>>> On Sat, Mar 3, 2018, 5:52 AM Marcus Ottosson <konstrukt...@gmail.com>
>>> wrote:
>>>
>>>> No, there isn’t a datatype for dictionaries.
>>>>
>>>> Converting to string via json.dumps() is one way, you can also convert
>>>> to a more compact string via something like cPickle.dumps(my_dict), or
>>>> compress it to e.g. zip and then dumps it, but as far as Maya is
>>>> concerned they are all of type string. Odds are you can convert it into any
>>>> of the other available types, like a huge number, but string isn’t a bad
>>>> option; it’s got no size limit (other than your total amount of RAM and
>>>> disk space) and probably isn’t posing any relevant performance overhead on
>>>> its own in terms of reading and writing to the attribute. The conversion
>>>> between Maya and Python can however be costly, and pickle is likely the
>>>> faster option in this case (depending on your data). The only thing to look
>>>> out for with that is that it will limit the result to machine-readable
>>>> output (e.g. Xz\939150\0klk5) and furthermore only be readable by the
>>>> particular version of Python you are running at the time of performing the
>>>> dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are
>>>> you can’t read it in 2018 on Windows (or even the same version of Maya but
>>>> on different OSes) etc.
>>>>
>>> Why would that be true? If you specify the protocol when you read and
>>> write the cPickle data, then it should be able to be dumped on Maya 2013 in
>>> Linux and read in Maya 2018 on Windows.
>>>
>>> https://docs.python.org/2/library/pickle.html#data-stream-format
>>>
>>> One could stick with binary protocol 2 the whole way through, I would
>>> think. Am I wrong?
>>>
>>>
>>> On the other hand, you can pickle more than just dictionaries. Whole
>>>> classes and instanced PyMel nodes and what else have you.
>>>>
>>>> You can see what Maya types are available as extra (i.e. “dynamic”)
>>>> attributes here
>>>> <http://help.autodesk.com/cloudhelp/2018/ENU/Maya-Tech-Docs/CommandsPython/addAttr.html>
>>>> for example.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Python Programming for Autodesk Maya" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to python_inside_maya+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>> msgid/python_inside_maya/CAFRtmOA3GN9tH%3Dz13BwwVQo35fzkv63-
>>>> c4iM3xuuSsmZxJ5f%3Dw%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOA3GN9tH%3Dz13BwwVQo35fzkv63-c4iM3xuuSsmZxJ5f%3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Python Programming for Autodesk Maya" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to python_inside_maya+unsubscr...@googlegroups.com.
>>>
>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/python_inside_maya/CAPGFgA38%2BW2k7W1bpMYsnnincG9S%
>>> 2BgS2uW9Fknh5rCpnYHuFtA%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA38%2BW2k7W1bpMYsnnincG9S%2BgS2uW9Fknh5rCpnYHuFtA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>
>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Python Programming for Autodesk Maya" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to python_inside_maya+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/python_inside_maya/CAFRtmODAvy8To4jCAk-n%
>> 3DvffzPFKGLiQu%2B19RCgapnLG%2BF%3DBKw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODAvy8To4jCAk-n%3DvffzPFKGLiQu%2B19RCgapnLG%2BF%3DBKw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Python Programming for Autodesk Maya" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to python_inside_maya+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/python_inside_maya/CAPGFgA1hnJpqDSN5AwhRX9jbvgefi
> 5_GnJrf19DQGxW2QgnHDQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA1hnJpqDSN5AwhRX9jbvgefi5_GnJrf19DQGxW2QgnHDQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to python_inside_maya+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOBYf8sFgk_sPJH4niCC3MtHe7T73qYYhJw56UZ28h4t8Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
