tkaymak commented on code in PR #40516: URL: https://github.com/apache/beam/pull/40516#discussion_r4294194531
########## runners/spark/4/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/StatefulParDoStreamingTranslator.java: ########## @@ -17,6 +17,7 @@ */ package org.apache.beam.runners.spark.structuredstreaming.translation.streaming; +import org.apache.beam.runners.spark.coders.CoderHelpers; Review Comment: Switched both stateful files to the structured streaming `CoderHelpers` in 5608646ba06, so that package no longer imports the legacy one. I did not merge the classes here: the legacy class encodes in nested context (`coder.encode(value, stream)`) and the structured streaming one in outer context (`CoderUtils`), and the latter encodes the unbounded source checkpoint marks that shipped in 2.77.0, so folding one into the other changes persisted bytes. Happy to do the dedup in a follow up that keeps the outer encoding for the structured streaming callers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
