On Mon, Jul 13, 2020 at 12:03 PM Edwin Zimmerman <ed...@211mainstreet.net> wrote:
> I would have interest in it. > OK -- I'll see what I can do about pulling it out and putting it on gitHub. Not sure I'll have the time to clean it up and make a nice package out of it, but maybe there's some ideas in there worth sharing. -CHB > > > --Edwin > > > > > > I'm no security expert, but we've got a big pile of serialization code > that is kind of like JSON-pickly, but it will only deserialize known > objects. it's a bit of pain to declare what you want to work with, but it > seems safer. > > > > I also have a newer system (built on top of dataclasses) that serializes > to JSON, but it's "pure" JSON -- it does not store any info about object > types or anything in the JSON. Rather, the deserializer knows what it > expects, and can only unpack that. e.g., you set a class attribute to the > type "List_of_Object_A" -- and then it will use Object_A's deserializer to > try to unpack the JSON. > > > > It does mean you can't "just save" an arbitrary object, but you can use a > decorator on a datacalsss to make any object savable. > > > > I did it this way because I want the JSON to be plain old JSON, but I also > think it's substantially more secure. > > > > part of a non-released code base, but iof folks think it would be handy, I > can pull it out into its own package. > > > > -CHB > > > > > > > > On Mon, Jul 13, 2020 at 9:54 AM Wes Turner <wes.tur...@gmail.com> wrote: > > Looks like pyro4 (python remote objects) has moved to the serpent library > (as.literal_eval) [1] > > > > > defaults to a safe serializer (serpent > https://pypi.python.org/pypi/serpent ) that supports many Python data > types. > > > supports different serializers (serpent, json, marshal, msgpack, pickle, > cloudpickle, dill) > > > > And pyro5 has removed support for unsafe serializers [5]: > > > > > no support for unsafe serializers AT ALL (pickle, dill, cloudpickle) - > only safe serializers (serpent, marshal, json, msgpack) > > > > > > for now, requires msgpack to be installed as well as serpent. > > > > TBH, I'm not sure how much any of the serialization protocols have been > fuzzed for safety purposes if at all. > > > > [4] https://github.com/irmen/Pyro4#feature-overview > > > > [5] > https://pyro5.readthedocs.io/en/latest/intro.html#what-has-been-changed-since-pyro4 > > > > (FWIW, PyArrow has very fast support for "Streaming, Serialization, and > IPC" > > https://arrow.apache.org/docs/python/ipc.html and support for "Arbitrary > Object Serialization" > https://arrow.apache.org/docs/python/ipc.html#arbitrary-object-serialization > w/ pyarrow.serialize() and pyarrow.deserialize()) > > > > On Mon, Jul 13, 2020, 7:57 AM Chris Angelico <ros...@gmail.com> wrote: > > On Mon, Jul 13, 2020 at 8:58 PM Edwin Zimmerman <ed...@211mainstreet.net> > wrote: > > > > On 7/11/2020 11:17 PM, Greg Ewing wrote: > > > > On 12/07/20 1:01 pm, Edwin Zimmerman wrote: > > > > As I see it, the unsafe callables (eval, exec, os.system, etc) are > generally functions, and safe ones(int, list, dict) are generally classes, > though there certainly would be exceptions. > > > > > > Where security is concerned, "there certainly would be exceptions" > > are not words you want to hear. > > > > Agreed, that is why pickle should almost never be used. In the past, I > have looked long and hard at using pickle in my own projects, but was > always turned away because of its potential for security issues. I've > thought for years that pickle is a major security foot gun, and I think > that not allowing this by default: > > >>> pickle.loads(b"cos\nsystem\n(S'echo hello world'\ntR.") > > would be a step in the right direction. > > > > A pickle file (or equivalent blob in a database, or whatever) should > be considered equally as trusted as your source code. If you're > writing out a file that has the exact same access permissions as your > own source code, and then reading it back, you shouldn't have to worry > about pickle's safety any more than you worry about your code's safety > - anyone who could maliciously craft something for you to unpickle > could equally just edit the source code directly. > > ChrisA > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/HRJVLRL4FDF4MQ6EPEHC36P6CHZSQBZ3/ > Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/7CLM57BK3YRFOOZBSEIWV65EKPQLZXPJ/ > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > > Christopher Barker, PhD > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/Q5MTTTW35ZGIIMBH6JG72H4A5P2Q755L/ Code of Conduct: http://python.org/psf/codeofconduct/
