psud commented on issue #38053: URL: https://github.com/apache/beam/issues/38053#issuecomment-4203569927
**Status Update:** After talking with a BigTable PM at Google I learned that a prerequisite for this to work is not in place yet on the Bigtable side. Bigtable Materialized Views use a structured row key format that isn't yet publicly decodable. Therefore reading data in Beam pipelines where row key parsing is required is not possible. Development therefore is blocked until this prerequisite is solved. **Workaround** The recommendation for anyone looking to read data from BigTable MVs in beam pipelines is to use the Execute SQL API from a specific beam step. So, instead of Beam processing the data, you just write SQL and use Bigtable to return what you need from the MV. Then you fan that out in `n` PCollection elements -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
