Re: cPickle fails on manually compiled and executed Python function

[Jan Gosmann]

On 07/18/2017 01:07 AM, dieter wrote:

"Jan Gosmann"  writes:


[...]
fn = load_pyfile('fn.py')['fn']
[...]

"pickle" (and "cpickle") are serializing functions as so called
"global"s, i.e. as a module reference together with a name.
This means, they cannot handle functions computed in a module
(as in your case).
Note that I'm assigning the computed function to a global/module level 
variable. As far as I understand the documentation 
 
that should be all that matters because only the function name will be 
serialized.

I am quite convinced that "pickle" will not be able to deserialize (i.e. load)
your function (even though it appears to perform the serialization
(i.e. dump).
Actually the deserialization works fine with either module. That is both 
pickle.loads(pickle.dumps(fn)) and cPickle.loads(pickle.dumps(fn)) give 
me back the function.


By now I realized that a pretty simple workaround works. Instead of 
doing `fn = load_pyfile('fn.py')['fn']` the following function 
definition works with both pickle modules:


_fn = load_pyfile('fn.py')['fn']
def fn(*args, **kwargs):
return _fn(*args, **kwargs)

--
https://mail.python.org/mailman/listinfo/python-list

Re: cPickle fails on manually compiled and executed Python function

[dieter]

"Jan Gosmann"  writes:

> today I came across some weird behaviour (a bug?) in Python 2.7.13 (on
> Linux) with the cPickle module. The pickle module works and so does
> the pickle module in Python 3.
>
> I have a file fn.py with a minimal function definition:
>
> ```
> def fn():
> pass
> ```
>
> The actual code that I run is in a separate file (test.py):
>
> ```
> import cPickle
> import pickle
>
> def load_pyfile(filename):
> source = ''
> with open(filename, 'r') as f:
> source += f.read()
> code = compile(source, filename, 'exec')
> loaded = {'__file__': filename}
> exec(code, loaded)
> return loaded
>
> fn = load_pyfile('fn.py')['fn']
>
> print(pickle.dumps(fn))
> print('')
> print(cPickle.dumps(fn))
> ```
>
> The first print works fine, but the one with cPickle leads to an
> exception. Here is the output:
>
> ```
> c__main__
> fn
> p0
> .
> 
> Traceback (most recent call last):
>   File "test.py", line 17, in 
> print(cPickle.dumps(fn))
> TypeError: expected string or Unicode object, NoneType found

"pickle" (and "cpickle") are serializing functions as so called
"global"s, i.e. as a module reference together with a name.
This means, they cannot handle functions computed in a module
(as in your case).

I am quite convinced that "pickle" will not be able to deserialize (i.e. load)
your function (even though it appears to perform the serialization
(i.e. dump).

You are using "pickle/cPickle" for a case not anticipated (computed
functions). This means that this case is not as tested as other
cases. As a consequence, you can see different behavior between
"picke" and "cPickle".



-- 
https://mail.python.org/mailman/listinfo/python-list

cPickle fails on manually compiled and executed Python function

[Jan Gosmann]

Hi,

today I came across some weird behaviour (a bug?) in Python 2.7.13 (on 
Linux) with the cPickle module. The pickle module works and so does the 
pickle module in Python 3.


I have a file fn.py with a minimal function definition:

```
def fn():
pass
```

The actual code that I run is in a separate file (test.py):

```
import cPickle
import pickle

def load_pyfile(filename):
source = ''
with open(filename, 'r') as f:
source += f.read()
code = compile(source, filename, 'exec')
loaded = {'__file__': filename}
exec(code, loaded)
return loaded

fn = load_pyfile('fn.py')['fn']

print(pickle.dumps(fn))
print('')
print(cPickle.dumps(fn))
```

The first print works fine, but the one with cPickle leads to an 
exception. Here is the output:


```
c__main__
fn
p0
.

Traceback (most recent call last):
  File "test.py", line 17, in 
print(cPickle.dumps(fn))
TypeError: expected string or Unicode object, NoneType found
```

I don't understand why the cPickle module behaves differently in this 
case. Is this expected? And if so, how do I fix it? Or can this be 
considered a bug? (In that case I could open an issue in the bug 
tracker.)


Cheers, Jan
--
https://mail.python.org/mailman/listinfo/python-list