rangadi commented on code in PR #41075:
URL: https://github.com/apache/spark/pull/41075#discussion_r1186786447
##########
connector/protobuf/src/main/scala/org/apache/spark/sql/protobuf/utils/ProtobufOptions.scala:
##########
@@ -141,6 +141,30 @@ private[sql] class ProtobufOptions(
// what information is available in a serialized proto.
val emitDefaultValues: Boolean =
parameters.getOrElse("emit.default.values", false.toString).toBoolean
+
+ // Whether to render enum fields as their integer values.
+ //
+ // As an example, consider the following message type:
+ // ```
+ // syntax = "proto3";
+ // message Person {
+ // enum Job {
+ // NONE = 0;
+ // ENGINEER = 1;
+ // DOCTOR = 2;
+ // }
+ // Job job = 1;
+ // }
+ // ```
+ //
+ // If we have an instance of this message like `Person(job = ENGINEER)`,
then the
+ // default deserialization will be:
+ // `{"job": "ENGINEER"}`
+ //
+ // But with this option set the deserialization will be:
Review Comment:
Should we add a warning about users to be careful while changing it? It will
change the schema and might be incompatible with downstream consumes.
##########
connector/protobuf/src/main/scala/org/apache/spark/sql/protobuf/utils/ProtobufOptions.scala:
##########
@@ -141,6 +141,30 @@ private[sql] class ProtobufOptions(
// what information is available in a serialized proto.
val emitDefaultValues: Boolean =
parameters.getOrElse("emit.default.values", false.toString).toBoolean
+
+ // Whether to render enum fields as their integer values.
+ //
+ // As an example, consider the following message type:
+ // ```
+ // syntax = "proto3";
+ // message Person {
+ // enum Job {
+ // NONE = 0;
+ // ENGINEER = 1;
+ // DOCTOR = 2;
+ // }
+ // Job job = 1;
+ // }
+ // ```
+ //
+ // If we have an instance of this message like `Person(job = ENGINEER)`,
then the
+ // default deserialization will be:
+ // `{"job": "ENGINEER"}`
+ //
+ // But with this option set the deserialization will be:
+ // `{"job": 1}`
+ val enumsAsInts: Boolean =
Review Comment:
Better to set this option for `jsonPrinter` in `ProtobufDeserializer` so
that `Any` values deserialized as json strings will have ints too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
