mynameborat commented on PR #26276: URL: https://github.com/apache/beam/pull/26276#issuecomment-1515128208
> * Configs needs to be readable for debugging, if we serialize this with protobuf, the serialised string will not be readable via configs while debugging - it would need a deserializer This is an internal configuration not for external consumption for user. I'd rather keep it hidden than have it readable. If you want it for debug purpose, you can use the debug logs to infer as opposed to exposing this configuration externally. >PipelineJsonRenderer already uses JSON It is external facing so that we can render the DAG for observability. It is not necessarily used internally. >IMO using protobuf might over-complicate things for a simple MapEntry<String, String> serializer `Map<string, string>` is not an evolvable data model and if you need additional metadata evolving the string to have record delimitters and so-on complicates evolution and compatibility handling. All said, If we can't leverage the protobuf representation of the pipeline, can we use `BEAM_JSON_GRAPH` which also has information about the transforms. Can that be leveraged? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
