[
https://issues.apache.org/jira/browse/BEAM-13189?focusedWorklogId=676598&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-676598
]
ASF GitHub Bot logged work on BEAM-13189:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Nov/21 17:28
Start Date: 04/Nov/21 17:28
Worklog Time Spent: 10m
Work Description: codecov[bot] edited a comment on pull request #15901:
URL: https://github.com/apache/beam/pull/15901#issuecomment-961241158
#
[Codecov](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
Report
> Merging
[#15901](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(bbbd158) into
[master](https://codecov.io/gh/apache/beam/commit/963cf92c9f4572b8ac714ab07fd539a324ee3c57?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(963cf92) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15901 +/- ##
=======================================
Coverage 83.52% 83.52%
=======================================
Files 445 445
Lines 61371 61391 +20
=======================================
+ Hits 51258 51278 +20
Misses 10113 10113
```
| [Impacted
Files](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
| Coverage Δ | |
|---|---|---|
|
[...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15901/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==)
| `94.02% <0.00%> (-2.24%)` | :arrow_down: |
|
[sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15901/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5)
| `96.35% <0.00%> (-1.01%)` | :arrow_down: |
|
[...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15901/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==)
| `93.64% <0.00%> (+0.37%)` | :arrow_up: |
|
[...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15901/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=)
| `86.36% <0.00%> (+0.50%)` | :arrow_up: |
|
[...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15901/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5)
| `76.58% <0.00%> (+0.97%)` | :arrow_up: |
------
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
Last update
[0b47f1b...bbbd158](https://codecov.io/gh/apache/beam/pull/15901?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
Read the [comment
docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 676598)
Time Spent: 1h (was: 50m)
> Add escapechar to Python TextIO reads
> -------------------------------------
>
> Key: BEAM-13189
> URL: https://issues.apache.org/jira/browse/BEAM-13189
> Project: Beam
> Issue Type: New Feature
> Components: io-py-common, io-py-files
> Reporter: Eugene Nikolaiev
> Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Existing TextIO connector can be used for splitting lines of CSV or
> tab-delimited files for its ability to read large files in parallel and
> rebalance the work. Each line then can be parsed with {{csv}} library
> separately. This works, if there are no line delimiters inside the lines.
> Otherwise the lines are split incorrectly.
> One of tab-delimited dialects uses escape characters to escape the line and
> column delimiters (usually backslash) instead of quoting the columns. This
> can be parsed with Python {{csv}} library using
> [escapechar|https://docs.python.org/3/library/csv.html#csv.Dialect.escapechar]
> dialect parameter.
> The escapechar itself can also be escaped to allow having such character
> before the line delimiters.
> Example of such file format usage: [Adobe Analytics Data
> Feed|https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/datafeeds-spec-chars.html?lang=en]
> It would be nice if TextIO transforms {{ReadFromText}} and
> {{ReadAllFromText}} had support for {{escapechar}} as follows:
>
> {code:java}
> import csv
> import tempfile
> import apache_beam as beam
> with tempfile.NamedTemporaryFile('w') as temp_file:
> # Write CSV lines with escaped line terminator
> temp_file.write('a\\\na\taa\n')
> temp_file.write('bb\tbb\n')
> temp_file.flush()
> # Read and print lines
> with beam.Pipeline() as pipeline:
> (
> pipeline
> | beam.io.ReadFromText(file_pattern=temp_file.name, escapechar=b'\\')
> | beam.Map(lambda x: print(repr(x)))
> )
> # Read lines, parse and print TSV rows
> with beam.Pipeline() as pipeline:
> (
> pipeline
> | beam.io.ReadFromText(file_pattern=temp_file.name, escapechar=b'\\')
> | beam.Map(lambda x: next(csv.reader([x], escapechar='\\',
> delimiter='\t')))
> | beam.Map(lambda x: print(repr(x)))
> )
> {code}
> This would print:
> {code:java}
> 'a\\\na\taa'
> 'bb\tbb'
> ['a\na', 'aa']
> ['bb', 'bb']
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
