Okay so changing pickle.loads(p) to pickle.loads(p, encoding='bytes') seems like a partial fix if there's a .sconsign.dblite file saved in python 2 with pickle protocol 2 python 3 won't crash now, instead it'll just rebuild the sources / ignore the current state
going the opposite way from python 3 to python 2 gives me the error scons: *** [mkdocs.yml] ValueError : unsupported pickle protocol: 4 I suppose you could force it to always use pickle protocol 2 (not sure how to do that) the end result would be that it would always rebuild when going from py2 -> py3 or py3 -> py2 so it wouldn't crash but ignore the build state (perhaps the file hashes are calculated differently?) pickle protocol 4 might be a lot quicker than pickle protocol 2 btw I'm not sure maybe there should be a way of setting it as an option within SConstruct Many Thanks Richard On 10 September 2017 at 13:49, RW <[email protected]> wrote: > Hi Bill, > the ascii related error (the one you get when generating using py2 first > then py3 second) > seems to be caused by > line 119, dblite.py, p = f.read() > > python 2 reads then decodes a file to a string by default > but python 3 reads the file as binary without decoding. > it then feeds it into pickle.loads() > line 122, dblite.py, self._dict = pickle.loads(p) > > I spotted some comments about something similar here > https://stackoverflow.com/questions/28218466/unpickling- > a-python-2-object-with-python-3 > https://stackoverflow.com/questions/30469575/how-to- > pickle-and-unpickle-to-portable-string-in-python-3 > I tried some simple decodes of the f.read to get it to work but I think it > might be a case of changing from pickle.loads to pickle.load with an > encoding parameter > > For a future version of scons (past 3.0) it might be worth investigating > performance of different storage types. > Based on some of these links I think pickle 4 might be the fastest, then > json then pickle 2 > https://stackoverflow.com/questions/2259270/pickle-or-json > https://konstantin.blog/2010/pickle-vs-json-which-is-faster/ > https://stackoverflow.com/questions/23582489/python-pickle-protocol-choice > > Many Thanks > Richard > > On 10 September 2017 at 03:13, Bill Deegan <[email protected]> > wrote: > >> Greetings, >> >> This is (I hope) the last issue gating 3.0 release. >> >> *If we set the default pickle version to 2. (Currently set to >> HIGHEST_AVAILABLE, which means 4 for py3.5+ and 2 for py2.7)* >> >> If you run scons with py2.7 and then run again with py3.x, you'll get the >> following: >> ... >> scons: *** [$DISTDIR/scons-local-3.0.0.alpha.20170821.tar.gz] >> UnicodeDecodeError : 'ascii' codec can't decode byte 0x80 in position 0: >> ordinal not in range(128) >> >> If you run scons with py3 and then rerun with py2.7, you'll get the >> following: >> * It will rebuild everything you just built with py3 >> >> *If we set the default pickle version to HIGHEST_AVAILABLE, which means >> 4 for py3.5+ and 2 for py2.7* >> >> >> Here's the options I see to deal with this: >> 1. Always append .py3 to PY3 scons runs' sconsign files >> 2. Always append .py# to sconsign files based on what version of python >> was used, ignore pre-existing .sconsigns from older versions >> 3. Do something complicated: >> a. Leave pickle version as HIGHEST_AVAILABLE, detect version of pickle >> by reading first two bytes. If incompatible version detected, change file >> name to .py# and try reading that one. Always write to .sconsign? Or write >> to file read. >> b. ? >> >> Thoughts? >> >> -Bill >> >> >> _______________________________________________ >> Scons-dev mailing list >> [email protected] >> https://pairlist2.pair.net/mailman/listinfo/scons-dev >> >> >
_______________________________________________ Scons-dev mailing list [email protected] https://pairlist2.pair.net/mailman/listinfo/scons-dev
