[ https://issues.apache.org/jira/browse/SPARK-46359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
gagan taneja updated SPARK-46359: --------------------------------- Summary: add optional batch id _spark_metadata sink file status (was: add optional batch id to sink file status ) > add optional batch id _spark_metadata sink file status > ------------------------------------------------------- > > Key: SPARK-46359 > URL: https://issues.apache.org/jira/browse/SPARK-46359 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Affects Versions: 3.5.0 > Reporter: gagan taneja > Priority: Major > > Currently FileStreamSinkLog use SinkFileStatus class for serialization > purpose but it does preserve batch id which can be used effectively for > snapshots > Change the definition to > {code} > case class SinkFileStatus (path : String, size : Long, isDir : Boolean, > modificationTime : Long, blockReplication: Int, blockSize: Long, action : > String, bactchId : Option[Long]) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org