sijie commented on issue #4585: [spark streaming] support spark structured streaming URL: https://github.com/apache/pulsar/issues/4585#issuecomment-512911758 for people who are interested in using spark structure streaming with Pulsar, @yjshen has implemented a decent spark connector here: https://github.com/streamnative/pulsar-spark The spark connector supports both streaming and batch jobs and write data back to pulsar as well. the implementation is fully integrated with Pulsar schema. You can also use Spark SQL to query the data in pulsar. He also wrote a blog post about it : https://medium.com/streamnative/apache-pulsar-as-one-storage-455222c59017 We are looking forward to contributing this back to upstream pulsar or spark. thoughts and feedbacks are welcome.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
