amaliujia commented on code in PR #38605:
URL: https://github.com/apache/spark/pull/38605#discussion_r1019647668
##########
connector/connect/docs/adding-proto-messages.md:
##########
@@ -0,0 +1,87 @@
+# Required, Optional and default values
+
+Connect adopts proto3, which does not support `required` constraint anymore.
+Every field is optional. For non-message proto fields, there is also no
`hasXXX`
+functions to easy tell if a filed is set or not-set. For those non-message
proto,
+they also have default values. E.g. int has 0 as the default value.
+
+
+### Required field
+
+Even proto3 does not offer `required` constraint, there is still some fields
that
+are semantically required. For such case, we share add comment on the field to
mark
+it as required. The expectation for client implementation (or any submitted
proto plan)
+is that such fields should be always set, and server will always assume such
fields
+are set and use whatever values of the fields directly. It is the client
side's fault
+to not offer meaningful value for `required` field and in that case, the
behavior on
+the server side is not defined.
+
+
+### Optional field and default value
+
+Optional fields could have default values when the field is not set, and we
are seeing
+two cases that depends on whether the default value for the proto field is
consistent
+with the corresponding Spark parameter default value.
+
+If both default values of the proto field and Spark parameter are the same, we
keep the
+proto field as is.
+
+For example:
+```Scala
+// Spark Plan.
+class FooPlan(size: Long = 0)
+```
+
+```protobuf
+// Connect proto.
+message Foo {
+ int64 size;
Review Comment:
I intentionally do not add the proto field number. This is to avoid people
mis-understand it as a default value thus causing confusions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
