Justin Ting added the comment: Ah, should have picked that up, coding at 3:30am doesn't do wonders for keeping a clear head.
Thanks Tim, I'll keep that in mind! *Justin Ting* *E* justingl...@gmail.com | *M* +61 424 751 665 | *L* *https://au.linkedin.com/in/justinyting <https://au.linkedin.com/in/justinyting>* | *G *https://github.com/jyting On Sun, Oct 23, 2016 at 3:48 AM, Tim Peters <rep...@bugs.python.org> wrote: > > Tim Peters added the comment: > > This has nothing to do with the _values_ you're passing - it has to do > with the length of the pickle string: > > def _send_bytes(self, buf): > n = len(buf) > # For wire compatibility with 3.2 and lower > header = struct.pack("!i", n) IT'S BLOWING UP HERE > if n > 16384: > ... > self._send(header) > self._send(buf) > > where the traceback shows it's called here: > > self._send_bytes(ForkingPickler.dumps(obj)) > > Of course the less data you're passing, the smaller the pickle, and that's > why it doesn't blow up if you pass subsets of the data. > > I'd suggest rethinking how you're sharing data, as pushing two-gigabyte > pickle strings around is bound to be the least efficient way possible even > if it didn't blow up ;-) > > ---------- > nosy: +tim.peters > > _______________________________________ > Python tracker <rep...@bugs.python.org> > <http://bugs.python.org/issue28506> > _______________________________________ > ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28506> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
