SreeramGarlapati commented on pull request #4473: URL: https://github.com/apache/iceberg/pull/4473#issuecomment-1086892995
@RussellSpitzer - sparkSession is holding on to the state of the stream - internally. I am hitting failures when trying to reproduce the problem locally - as it needs multiple sparkSessions in the same unittest. Is there any precedence in the codebase for this pattern that you could kindly point me to pl. putting the problem being explained above aside - the code itself has coverage in this unittest - `testResumingStreamReadFromCheckpoint` - using which I was able to debug thru & fix the code - where I need to use `File.createOrReplace()` instead of the existing `File.create()`. cc: @rajarshisarkar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
