Hi everyone, I'm working on the WI mentioned in the subject: "Add Proto support to Pubsub table provider" and I have a few questions . Sorry for the long mail!
- The only method in KafkaTableProvider that performs some logic is buildBeamSqlTable. However, when taking a look at the tests in KafkaTableProviderIT, the only one that is calling that method appears to be testFake2. But if I’m not mistaken, that test doesn’t perform any test for the proto case. The only one that tests the proto case is testFake test, but that is only creating the KafkaTableProvider and that’s it. I wanted to base the PubSubTableProvider on the KafkaTableProvider since that one supports Proto and looks like it accomplishes that support by using PayloadSerializer. Is that correct? Should I follow that path? - I wanted to base the new code on the commits by Piotr, but a lot of the code he submitted appears to have been removed, but I can’t grasp the reason. There are several commits referencing one another and the Proto support. Can you shed some light on that? - Back to the PubsubTableProvider: PubSubIO, which contains reader/writer methods for protos (although via ProtoCoder), is being used by the PubsubSchemaIOProvider class for other read/write purposes. Why are the protos reader/writer not used? Because of the ProtoCoder? Should we instead implement new readers/writers by formatting the payload directly, in a similar fashion to readAvrosWithBeamSchema? Thanks a lot for the help! - Fernando Morales -- *This email and its contents (including any attachments) are being sent to you on the condition of confidentiality and may be protected by legal privilege. Access to this email by anyone other than the intended recipient is unauthorized. If you are not the intended recipient, please immediately notify the sender by replying to this message and delete the material immediately from your system. Any further use, dissemination, distribution or reproduction of this email is strictly prohibited. Further, no representation is made with respect to any content contained in this email.*
