pabloem commented on pull request #13154: URL: https://github.com/apache/beam/pull/13154#issuecomment-718942527
> > > I'm curious do we have a plan to build actual SDF for BQ instead of still relying on BoundedSource implementation? > > > > > > In this case, we will have a simple DoFn that _starts_ the read from BQ, but it eventually returns multiple Avro file sources that can be read individually. This is different from what we had before, where all of the BQ reading logic was part of a BoundedSource. In fact, the _CustomBigQuerySource will be removed eventually. > > I see. It seems like you will use `SDFBoundedSourceReader` in your BQ readAll transform. I think it would be nice to not build anything new directly on top of `BoundedSource` since overall we want deprecate `BoundedSource` in the feature. The bounded sdf wrapper is for helping us to do the migration smoothly. What do you think? I think that's reasonable. If any improvements are made to ReadAllFromBQ, we can make sure that they are done without relying on BoundedSource then. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
