[issue30388] ndbm can't iterate through values on OS X

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I've attached a C program that demonstrates the problem, and have filed an 
issue with Apple about this: FB8919203

It is not clear to me what we can do about this, other than not building the 
ndbm binding on macOS.  The manpages for the ndbm APIs do not mention 
limitations on item sizes, and the dbopen manpage mention that there a no 
limitations on item sizes).

--
Added file: https://bugs.python.org/file49627/ndbm-repro.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Ronald Oussoren]

Ronald Oussoren  added the comment:

See also #33074

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Forest Gregg added the comment:

A different user:

otool -L $(python3.6 -c 'import _dbm;print(_dbm.file)')
/usr/local/var/pyenv/versions/3.6.1/lib/python3.6/lib-dynload/_dbm.cpython-36m-darwin.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 
1238.50.2)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Forest Gregg added the comment:

>From one user who had problems under both 3.5 and 3.6

otool -L $(python3.5 -c 'import _dbm;print(_dbm.__file__)')

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_dbm.cpython-35m-darwin.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
current version 125.2.0)

otool -L $(python3.6 -c 'import _dbm;print(_dbm.__file__)')

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload/_
dbm.cpython-36m-darwin.so:

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
125.2.0)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Ned Deily]

Ned Deily added the comment:

It would be helpful if you could provide the output of this command:

otool -L $(python3.6 -c 'import _dbm;print(_dbm.__file__)')

if necessary, substituting whatever command name(s) you are using to invoke 
python when failing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Forest Gregg added the comment:

I have been trying to make a small reproducible example, but haven't been able 
to isolate it. 

Running this script [1] csv_example on mac os x under either py27 or py3 does 
seem *often* cause this problem [2], [3]:

[1] 
https://github.com/dedupeio/dedupe-examples/blob/master/csv_example/csv_example.py
[2] https://github.com/dedupeio/dedupe/issues/571
[3] https://github.com/dedupeio/dedupe-examples/issues/54

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Emily Morehouse]

Changes by Emily Morehouse :


--
nosy: +emilyemorehouse

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Ned Deily]

Ned Deily added the comment:

Can you provide a complete example of how to reproduce the behavior you are 
seeing?  Ideally, it would include producing the shelve object.  Also, please 
provide the output of this command:

otool -L $(python3.6 -c 'import _dbm;print(_dbm.__file__)')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Forest Gregg added the comment:

Very sorry for the ambiguity. 

The bug appears for both python 3.5 and python 3.6.1 on OS X. I have not tried 
other versions of python on OS X.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Terry J. Reedy]

Terry J. Reedy added the comment:

Do you mean that on OSX, you code works with 3.6.1 and fails with 3.5.3?

--
components: +macOS
nosy: +ned.deily, ronaldoussoren, terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Forest Gregg added the comment:

The ndbm db's two files (in the attachment) have the following size

tmp___otctx 0 bytes
tmp___otctx.db 12857344 bytes

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

Changes by Forest Gregg :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30388] ndbm can't iterate through values on OS X

[Forest Gregg]

New submission from Forest Gregg:

On Mac OS 10.12.4, a large shelve, backed by ndbm, can be created. But when I 
attempt to iterate through the values of the shelve it raises this exception:


  File 
"/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/dedupe/api.py",
 line 281, in _blockData
for block in viewvalues(blocks):
  File 
"/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/_collections_abc.py",
 line 693, in __iter__
for key in self._mapping:
  File 
"/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/shelve.py", 
line 95, in __iter__
for k in self.dict.keys():
SystemError: Negative size passed to PyBytes_FromStringAndSize

I've confirmed that this works with Python 3.6.1. All the Python versions were 
installed from homebrew.

I cannot reproduce on linux on windows machines. I've attached a zip file of 
the ndbm files that python created.

--
components: Library (Lib)
files: ndbm.zip
messages: 293858
nosy: Forest Gregg
priority: normal
severity: normal
status: open
title: ndbm can't iterate through values on OS X
versions: Python 3.5
Added file: http://bugs.python.org/file46872/ndbm.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com