nikie commented on a change in pull request #15775:
URL: https://github.com/apache/beam/pull/15775#discussion_r736694095
##########
File path: sdks/python/apache_beam/io/textio_test.py
##########
@@ -1093,12 +1093,13 @@ def
test_read_with_customer_delimiter_over_buffer_size(self):
READ_BUFFER_SIZE = 10
delimiter = b'\r\n'
- with tempfile.NamedTemporaryFile() as temp_file:
+ with tempfile.NamedTemporaryFile(delete=False) as temp_file:
Review comment:
The file will be left after the test, because `delete=False`.
The source reading could be moved under the `with` block, so that it can see
the file until it is removed.
What is interesting, `write_data` helper does the same, so most tests in
this module leave temp files behind...
This test can be simplified:
```
def test_read_with_custom_delimiter_over_buffer_size(self):
"""
Corner case: delimiter is on border of size of buffer
"""
file_name, expected_data = write_data(
3, eol=EOL.CRLF, line_value=b'\rline')
assert len(expected_data) == 3
self._run_read_test(
file_name, expected_data, buffer_size=7, delimiter=b'\r\n')
```
And for this to work we need to extend `TextSourceTest._run_read_test`
method above:
* add `delimiter=None` param
* then (here we don't do simply `delimiter=None`, just to show that that
param is not required for the source):
```
kwargs = {}
if delimiter:
kwargs['delimiter'] = delimiter
source = TextSource(
file_or_pattern,
0,
compression,
True,
coders.StrUtf8Coder(),
buffer_size,
**kwargs)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
