razvanculea commented on PR #34777: URL: https://github.com/apache/beam/pull/34777#issuecomment-2852032959
> > > is this PR tested with DataflowRunner? > > > > > > @liferoad i've added the iobase_it_test from PR 34814 and it runs on Dataflow (it's actually using TextIO for write), but i don't have an IT for TextIO/parquetIO/AvroIO/TFRecordIO, I can try to add them, but i can't use TestStream in Dataflow, so i'll need to use PubSub public topics , right ? > > > > is this PR tested with DataflowRunner? > > > > > > @liferoad i've added the iobase_it_test from PR 34814 and it runs on Dataflow (it's actually using TextIO for write), but i don't have an IT for TextIO/parquetIO/AvroIO/TFRecordIO, I can try to add them, but i can't use TestStream in Dataflow, so i'll need to use PubSub public topics , right ? > > I forgot whether periodic impulse can trigger #25598. You could try that. And better add some checks about the output results. @liferoad, I added an IT in parquetio_it_test.py using PeriodicImpulse and it's working fine on Dataflow Did not add IT for AvroIO/TFRecordIO because there is no IT at all. I could do like for parquetIO later, if we deem this really a plus. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
