damccorm opened a new issue, #21673: URL: https://github.com/apache/beam/issues/21673
### Context: CDAP plugins that support streaming sources include Receiver classes (ex. [HubSpotReceiver](https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotReceiver.java)) that extend `org.apache.spark.streaming.receiver.Receiver` abstract class. Receiver classes are used by plugin Streaming Utils classes (ex. [HubSpotStreamingUtils](https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingUtil.java)) to provide `getStream()` method to Streaming Source classes (ex. [HubSpotStreamingSource](https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingSource.java)) and usually placed in `cdap/plugin/\{<plugin name>}/plugin/source/streaming` folder (ex.[ HubSpot plugin repo](https://github.com/data-integrations/hubspot/tree/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming)). Reference information: * [Streaming plug-ins integration](https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.o88i6p9b13o9) (Apache CDAP Connection Design Doc) * [Plugin integration process description](https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.1h6udb1b52xc) (Apache CDAP Connection Design Doc) * [Streaming wrapper design](https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.fcafz0ydsso1) (Apache CDAP Connection Design Doc) ### Task Description: Required to design custom Spark receivers builder class so that the custom receivers can be used in Apache Beam connector via SparkReceiverIO interface (used in CDAP IO as a dependency). ### Acceptance criteria: Design of builder class(es) that will create custom Spark receivers in Apache Beam connectors (`{}SparkReceiverIO{`}). #### Note: It is necessary that this builder class be independent of CDAP receivers and can be used by any other custom Spark receiver like a part of SparkReceiverIO. Imported from Jira [BEAM-14101](https://issues.apache.org/jira/browse/BEAM-14101). Original Jira may contain additional context. Reported by: elizaveta.lomteva. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
