[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

PR was merged and backported to 3.7. So I am closing this as fixed. Thanks Rémi 
for the patch.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Chris Withers]

Chris Withers  added the comment:


New changeset a6516f89aa0f416c7514ac364bb48ac7d1455487 by Chris Withers (Miss 
Islington (bot)) in branch '3.7':
bpo-31855: unittest.mock.mock_open() results now respects the argument of 
read([size]) (GH-11521) (#13152)
https://github.com/python/cpython/commit/a6516f89aa0f416c7514ac364bb48ac7d1455487


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[miss-islington]

Change by miss-islington :


--
pull_requests: +13068

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Chris Withers]

Chris Withers  added the comment:


New changeset 11a8832c98b3db78727312154dd1d3ba76d639ec by Chris Withers (Rémi 
Lapeyre) in branch 'master':
bpo-31855: unittest.mock.mock_open() results now respects the argument of 
read([size]) (GH-11521)
https://github.com/python/cpython/commit/11a8832c98b3db78727312154dd1d3ba76d639ec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch, patch
pull_requests: +11098, 11099, 11100
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch
pull_requests: +11098, 11099
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +11098
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

The off-by-one error in a test added for an unrelated issue (#17467) makes me 
think @michael.foord just made a mistake in the test.

> mock_open docs mentions about using a customized mock for complex cases
I think it's more for complex things like fetching data using a callback for 
example, this seems like a normal use case for mock_open(). Even more so, 
leaving the behavior as it is now may lead someone to think there is a bug in 
its code.

I opened a new PR to fix this issue: 
https://github.com/python/cpython/pull/11521

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Internally mock_open implementation uses line based iteration [0] to keep track 
of the state change between read calls. So readline too ignores the argument. 
There is issue25690 for an alternate mock_open implementation but the code 
change is large and adds a lot of features. A simpler approach would be to use 
StringIO or BytesIO to keep track of state changes and they provide read, 
readline and readlines API. mock_open docs mentions about using a customized 
mock for complex cases but I don't know if worthy enough to make this 
enhancement given the internal implementation change to support the API or to 
document this behavior. 

Looking further there also seems to be a test case for it [1] which will fail 
if this is fixed since this returns all characters instead of first 10 like 
using open().read(10).


def test_mock_open_read_with_argument(self):
# At one point calling read with an argument was broken
# for mocks returned by mock_open
some_data = 'foo\nbar\nbaz'
mock = mock_open(read_data=some_data)
self.assertEqual(mock().read(10), some_data)


$ echo -n 'foo\nbar\nbaz' > /tmp/a.txt
$ ./python.exe
Python 3.8.0a0 (heads/master:f5107dfd42, Dec 16 2018, 13:41:57)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('/tmp/a.txt') as f:
... actual = f.read(10)
... actual, len(actual)
...
('foo\nbar\nba', 10)
>>> with open('/tmp/a.txt') as f:
... from unittest.mock import mock_open
... mock = mock_open(read_data=f.read())
... mock_data = mock().read(10)
... mock_data, len(mock_data)
...
('foo\nbar\nbaz', 11)


[0] 
https://github.com/python/cpython/blob/f5107dfd42121ef40b13eb678705802f0ff02cf9/Lib/unittest/mock.py#L2349
[1] 
https://github.com/python/cpython/blob/f5107dfd42121ef40b13eb678705802f0ff02cf9/Lib/unittest/test/testmock/testwith.py#L284

--
nosy: +cjw296, mariocj89, xtreak
versions: +Python 3.7, Python 3.8 -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> michael.foord
nosy: +michael.foord

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Ron Rothman]

Ron Rothman  added the comment:

Confirmed that the behavior exists in Python 3.6 as well.

--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31855] mock_open is not compatible with read(n) (and pickle.load)

[Ron Rothman]

New submission from Ron Rothman :

mock.mock_open works as expected when reading the entire file (read()) or when 
reading a single line (readline()), but it seems to not support reading a 
number of bytes (read(n)).

These work as expected:

from mock import mock_open, patch

# works: consume entire "file"
with patch('__main__.open', mock_open(read_data='bibble')) as m:
with open('foo') as h:
result = h.read()

assert result == 'bibble'  # ok

# works: consume one line
with patch('__main__.open', mock_open(read_data='bibble\nbobble')) as m:
with open('foo') as h:
result = h.readline()

assert result == 'bibble\n'  # ok

But trying to read only a few bytes fails--mock_open returns the entire 
read_data instead:

# consume first 3 bytes of the "file"
with patch('__main__.open', mock_open(read_data='bibble')) as m:
with open('foo') as h:
result = h.read(3)

assert result == 'bib', 'result of read: {}'.format(result)  # fails

Output:

Traceback (most recent call last):
  File "/tmp/t.py", line 25, in 
assert result == 'bib', 'result of read: {}'.format(result)
AssertionError: result of read: bibble

The unfortunate effect of this is that mock_open cannot be used with 
pickle.load.

with open('/path/to/file.pkl', 'rb') as f:
x = pickle.load(f)  # this requires f.read(1) to work

--
components: Library (Lib)
messages: 304841
nosy: ron.rothman
priority: normal
severity: normal
status: open
title: mock_open is not compatible with read(n) (and pickle.load)
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com