cloud-fan commented on code in PR #55145:
URL: https://github.com/apache/spark/pull/55145#discussion_r3409962876
##########
sql/pipelines/src/test/scala/org/apache/spark/sql/pipelines/graph/SystemMetadataSuite.scala:
##########
@@ -251,7 +251,7 @@ class SystemMetadataSuite
val stSystemMetadata = FlowSystemMetadata(updateContext, stFlow, graph)
val schema2StSystemMetadata = FlowSystemMetadata(updateContext,
schema2StFlow, graph)
assert(
- stSystemMetadata.flowCheckpointsDirOpt() !=
schema2StSystemMetadata.flowCheckpointsDirOpt()
+ stSystemMetadata.flowCheckpointsDir() !=
schema2StSystemMetadata.flowCheckpointsDir()
Review Comment:
Following up on the prior round: now that `flowCheckpointsDir()` documents
`@throws IllegalArgumentException`, this is API surface worth pinning, and a
test is cheap (~12 lines, no manual flow construction). A `LocalTempView`
registers an implicit flow whose destination is the *view* -- which is in
`graph.view`, not `graph.table`/`graph.sink` -- so it hits the throw directly:
```scala
test("flowCheckpointsDir throws when the destination is neither a table nor
a sink") {
val session = spark
import session.implicits._
val graph = new TestGraphRegistrationContext(spark) {
val mem: MemoryStream[Int] = MemoryStream[Int]
mem.addData(1, 2, 3)
registerView("v", query = dfFlowFunc(mem.toDF()))
}.toDataflowGraph
val updateContext = TestPipelineUpdateContext(
unresolvedGraph = graph, spark = spark, storageRoot = storageRoot,
failOnErrorEvent = true)
val viewFlow = graph.flowsTo(TableIdentifier("v")).head
val systemMetadata = FlowSystemMetadata(updateContext, viewFlow, graph)
intercept[IllegalArgumentException] {
systemMetadata.flowCheckpointsDir()
}
}
```
`flowsTo` (keyed by destination identifier) avoids guessing the flow id;
temp-view ids are single-part/unqualified, so `TableIdentifier("v")` is the
destination key. Non-blocking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
