amaliujia commented on code in PR #38347:
URL: https://github.com/apache/spark/pull/38347#discussion_r1009835612
##########
connector/connect/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -207,3 +208,23 @@ message Sample {
int64 seed = 1;
}
}
+
+// Relation of type [[Range]] that generates a sequence of integers.
+message Range {
+ // Optional. Default value = 0
+ int32 start = 1;
+ int32 end = 2;
+ // Optional. Default value = 1
+ Step step = 3;
+ // Optional. Default value is assigned by 1) SQL conf
"spark.sql.leafNodeDefaultParallelism" if
+ // it is set, or 2) spark default parallelism.
+ NumPartitions num_partitions = 4;
Review Comment:
There are two dimensions of things in this area:
1. Required versus Optional.
A field is required, meaning it must be set. A field can be optional.
Meaning it could be set or not.
2. Field has default value or not.
A field can have a default value if not set.
The second point is an addition for the first point. If there is a field
which is not set, there could be a default value to be used.
There are special cases that the default value for proto, is the same as the
default value that Spark uses. In that case we don't need to differentiate the
optionally. Otherwise we need this way to differentiate `set versus not set`,
to adopt default values of Spark (unless we don't care the default values in
Spark).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
