[issue26773] Shelve works inconsistently when carried over to child processes

[Mark Kellogg]

Mark Kellogg added the comment:

The gentoo user's version of gdbm is: gdbm-1.11

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26773] Shelve works inconsistently when carried over to child processes

[Mark Kellogg]

Mark Kellogg added the comment:

I have also been running into this issue. 

I am using Debian GNU/Linux 8, it was also reproduced on ubuntu by a coworker. 

It however was not reproducible on Gentoo in our testing.

--
nosy: +mkellogg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26773] Shelve works inconsistently when carried over to child processes

[Paul Ellenbogen]

Paul Ellenbogen added the comment:

I think this behavior is due to the underlying behavior of the dbm. The same 
code using dbm, rather than shelve, also throws KeyErrors:

from multiprocessing import Process
import dbm

db = dbm.open("example.dbm", "c")
for i in range(100):
db[str(i)] = str(i ** 2)


def parallel():
for i in range(100):
print(db[str(i)])

a = Process(target = parallel)
b = Process(target = parallel)
a.start()
b.start()
a.join()
b.join()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26773] Shelve works inconsistently when carried over to child processes

[SilentGhost]

Changes by SilentGhost :


--
components: +Extension Modules
nosy: +jnoller, sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26773] Shelve works inconsistently when carried over to child processes

[Paul Ellenbogen]

New submission from Paul Ellenbogen:

If a shelve is opened, then the processed forked, sometime the shelve will 
appear to work in the child, and other times it will throw a KeyError. I 
suspect the order of element access may trigger the issue. I have included a 
python script that will exhibit the error. It may need to be run a few times.

If shelve is not meant to be inherited by the child process in this way, it 
should consistently throw an error (probably not a KeyError) on any use, 
including the first. This way it can be caught in the child, and the shelve can 
potentially be reopened in the child.

A current workaround is to find all places where a process may fork, and reopen 
any shelves in the child process after the fork. This may work for most smaller 
scripts. This could become tedious in more complex applications that fork in 
multiple places and open shelves in multiple places.

---

Running

#!/usr/bin/env python3

import multiprocessing
import platform
import sys

print(sys.version)
print(multiprocessing.cpu_count())
print(platform.platform())


outputs:
3.4.3+ (default, Oct 14 2015, 16:03:50) 
[GCC 5.2.1 20151010]
8
Linux-4.2.0-34-generic-x86_64-with-Ubuntu-15.10-wily

--
components: Interpreter Core
files: shelve_process.py
messages: 263522
nosy: Paul Ellenbogen
priority: normal
severity: normal
status: open
title: Shelve works inconsistently when carried over to child processes
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file42475/shelve_process.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com