[
https://issues.apache.org/jira/browse/BEAM-11571?focusedWorklogId=533253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-533253
]
ASF GitHub Bot logged work on BEAM-11571:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jan/21 21:54
Start Date: 08/Jan/21 21:54
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on a change in pull request
#13671:
URL: https://github.com/apache/beam/pull/13671#discussion_r554212074
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/ConvertHelpers.java
##########
@@ -78,12 +83,15 @@ public ConvertedSchemaInformation(
public static <T> ConvertedSchemaInformation<T>
getConvertedSchemaInformation(
Schema inputSchema, TypeDescriptor<T> outputType, SchemaRegistry
schemaRegistry) {
ConvertedSchemaInformation<T> convertedSchema = null;
- boolean toRow = outputType.equals(TypeDescriptor.of(Row.class));
- if (toRow) {
+ if (outputType.equals(TypeDescriptor.of(Row.class))) {
// If the output is of type Row, then just forward the schema of the
input type to the
// output.
convertedSchema =
new ConvertedSchemaInformation<>((SchemaCoder<T>)
SchemaCoder.of(inputSchema), null);
+ } else if (outputType.equals(TypeDescriptor.of(GenericRecord.class))) {
+ convertedSchema =
+ new ConvertedSchemaInformation<T>(
+ (SchemaCoder<T>)
AvroUtils.schemaCoder(AvroUtils.toAvroSchema(inputSchema)), null);
Review comment:
Hm it seems like this should be generalized somehow so we don't need to
have if blocks for converting to every "dynamic" class. For example we might
want something similar for protobuf's DynamicMessage. Something pluggable like
SchemaProvider, but for making conversions to/from other dynamic row types.
This is fine for now, but maybe drop a TODO and jira about generalizing?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 533253)
Time Spent: 1h (was: 50m)
> Support Conversion to GenericRecords in Convert.to transform
> ------------------------------------------------------------
>
> Key: BEAM-11571
> URL: https://issues.apache.org/jira/browse/BEAM-11571
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Beam's Convert transform allows to 'automatically' convert from a Schema
> based PCollection into a target type.
> The current implementation fails if the target PCollection is backed by
> GenericRecord. Supporting this type of conversion is important because many
> IOs e.g. AvroIO, ParquetIO have write/sink transforms that depend on the
> input PCollection in GenericRecord form.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
