Gregory Ewing wrote: > Mok-Kong Shen wrote: >> I have yet a question out of curiosity: Why is my 2nd list structure, >> that apparently is too complex for handling by eval and json, seemingly >> not a problem for pickle? > > Pickle is intended for arbitrary data structures, so it > is designed to be able to handle deeply-nested and/or > recursive data. Eval only has to handle nesting to depths > likely to be encountered in source code. Apparently the > json parser also assumes you're not going to be using > very deep nesting.
But pickle does have the same limitation: >>> def check(load, dump): ... items = [] ... try: ... for i in range(10**6): ... assert load(dump(items)) == items ... items = [items] ... except RuntimeError: ... return i ... >>> check(json.loads, json.dumps) 994 >>> check(pickle.loads, pickle.dumps) 499 Mok-Kong Shen, for pickle and json you can increase the limit a bit: >>> import sys >>> sys.setrecursionlimit(2000) >>> check(json.loads, json.dumps) 1994 >>> check(pickle.loads, pickle.dumps) 999 But be careful, if you choose the limit too high you'll see Python react like any other C program: >>> sys.setrecursionlimit(100000) >>> items = [] >>> for i in range(100000): ... items = [items] ... >>> s = pickle.dumps(items) Segmentation fault For literal_eval() the limit is unfortunately hard-coded in the C source. Mok-Kong Shen wrote: > What I need is to have my (complicated) list to be put into > a bytearray, do some proceesing, transfer it to the recipient. > The recipient reverses the processing, obtains a bytearray > that is the same as my original one and gets from it the same > list as mine. Using pickle I can manage to do it (in fact I > did try and succeed with my 2nd list), but that's IMHO a > rather unnatural/awkward work-around. (It means that I have > to pickle out the list to a file and read in the content of > the file in order to have it as a bytearray etc. etc.) The limit has been increased before, see http://bugs.python.org/issue1881 and maybe you can get the core developers to increase it again, but generally speaking the existence of a recursion limit is the price you pay for easy interfacing with C. literal_eval() could allocate its stack dynamically, but that would probably complicate the code so that return on investment is questionable. -- https://mail.python.org/mailman/listinfo/python-list