ahmedabu98 opened a new issue, #38251: URL: https://github.com/apache/beam/issues/38251
### What would you like to happen? PCollections of Beam Rows have fixed schemas, making it hard to process records with different logical schemas. We need a semi-structured type that can contain a nested level of variable columns. Transforms can choose to unwrap this type to reconstruct the variable columns accordingly. The Variant type is a becoming a good standard across different projects (see Parquet](https://parquet.apache.org/docs/file-format/types/variantencoding/), [Spark](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.types.VariantType.html), [Flink](https://nightlies.apache.org/flink/flink-docs-master/docs/sql/reference/data-types/#variant)). It could also be a good candidate for Beam. Bonus if such a type is made portable so that other SDKs can use it too ### Issue Priority Priority: 2 (default / most feature requests should be filed as P2) ### Issue Components - [x] Component: Python SDK - [x] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam YAML - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Infrastructure - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
