So as it turns out, it was an STDOUT issue for my logging and not a data read in. Beam operated just fine but the way I was debugging was causing the glitches.
Beam is operating as expected now. On Thu, Jul 11, 2019 at 10:28 PM Kenneth Knowles <[email protected]> wrote: > Doesn't sound good. TextIO has been around a long time so I'm surprised. > Would you mind creating a ticket in Jira ( > https://issues.apache.org/jira/projects/BEAM/) and posting some technical > details, like input/output/code snippets? > > Kenn > > On Thu, Jul 11, 2019 at 9:45 AM Shannon Duncan <[email protected]> > wrote: > >> I have a file where every line is a record separated by a tab. So a tab >> delimited file. >> >> However as I read this file in using TextIO.read().from(filename) and >> pass the results to a pardo, the elements are random chunks of the records. >> I expected the element to be the entire line of text which then I'll do >> parsing on from there. >> >> This file is processed in a python pipeline with ReadFromText perfectly >> fine. Just curious what would cause this on the Java side? >> >> Thanks, >> Shannon >> >
