Blue-Green Deployment of Structured Streaming

Hi all, We are attempting to come up with a blue-green deployment strategy for our structured streaming job to minimize down time. The general flow would be: 1. Job A is currently streaming 2. Job B comes up and starts loading Job A state without starting its query. 3. Job B completes

Offsets out of order - Spark Datasource V2

Hello Spark community, Please let me know if this is the appropriate place to ask this question – will happily move it. I haven’t been able to find the answer going to the usual outlets. I am currently implementing two custom readers for our projects (JMS / SQS) and am experiencing a problem w