Hi:
I am using spark structured streaming (v 2.2.0) to read data from files. I have 
configured checkpoint location. On stopping and restarting the application, it 
looks like it is reading the previously ingested files.  Is that expected 
behavior ?  
Is there anyway to prevent reading files that have already been ingested ? If a 
file is partially ingested, on restart - can we start reading the file from 
previously checkpointed offset ?
Thanks

Reply via email to