gengliangwang commented on code in PR #54230: URL: https://github.com/apache/spark/pull/54230#discussion_r2785767995
########## sql/core/src/main/scala/org/apache/spark/sql/scripting/CursorState.scala: ########## @@ -32,25 +32,29 @@ sealed trait CursorState case object CursorDeclared extends CursorState /** - * Cursor has been opened and query has been parsed/analyzed. - * For parameterized cursors, the query here has parameters substituted. + * Cursor has been opened and result iterator has been created. * - * @param analyzedQuery The analyzed logical plan with parameters bound + * CRITICAL: The iterator is created at OPEN time (not first FETCH) to ensure snapshot + * semantics. When executeToIterator() is called, Spark performs file discovery and + * captures Delta snapshots. This is the ONLY way to lock in the data snapshot at OPEN time. + * + * The iterator is lazy/incremental - it doesn't materialize all results, but it does + * lock in which files/versions will be read. + * + * @param resultIterator Iterator created at OPEN time (snapshot captured) + * @param outputSchema The output attributes (needed for type checking in FETCH) */ case class CursorOpened( - analyzedQuery: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan) extends CursorState + resultIterator: Iterator[org.apache.spark.sql.catalyst.InternalRow], Review Comment: let's avoid use full qualified class name here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
