[
https://issues.apache.org/jira/browse/BEAM-7886?focusedWorklogId=324784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-324784
]
ASF GitHub Bot logged work on BEAM-7886:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Oct/19 00:33
Start Date: 08/Oct/19 00:33
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #9188:
[BEAM-7886] Make row coder a standard coder and implement in Python
URL: https://github.com/apache/beam/pull/9188#discussion_r332293659
##########
File path:
runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/CommonCoderTest.java
##########
@@ -278,41 +290,90 @@ private static Object convertValue(Object value,
CommonCoder coderSpec, Coder co
return WindowedValue.of(windowValue, timestamp, windows, paneInfo);
} else if (s.equals(getUrn(StandardCoders.Enum.DOUBLE))) {
return Double.parseDouble((String) value);
+ } else if (s.equals(getUrn(StandardCoders.Enum.ROW))) {
+ Schema schema;
+ try {
+ schema =
SchemaTranslation.fromProto(SchemaApi.Schema.parseFrom(coderSpec.getPayload()));
+ } catch (InvalidProtocolBufferException e) {
+ throw new RuntimeException("Failed to parse schema payload for row
coder", e);
+ }
+
+ return parseField(value, Schema.FieldType.row(schema));
} else {
throw new IllegalStateException("Unknown coder URN: " +
coderSpec.getUrn());
}
}
+ private static Object parseField(Object value, Schema.FieldType fieldType) {
+ switch (fieldType.getTypeName()) {
+ case BYTE:
+ return ((Number) value).byteValue();
+ case INT16:
+ return ((Number) value).shortValue();
+ case INT32:
+ return ((Number) value).intValue();
+ case INT64:
+ return ((Number) value).longValue();
+ case FLOAT:
+ return Float.parseFloat((String) value);
+ case DOUBLE:
+ return Double.parseDouble((String) value);
Review comment:
I tried to be consistent with the YAML representations used for other coders
in `convertValues`. We use strings for `DoubleCoder`
[there](https://github.com/apache/beam/blob/master/runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/CommonCoderTest.java#L280),
presumably to avoid the possibility of running into precision errors? It looks
like yaml does [support](https://yaml.org/type/float.html) floating point but
explicitly states it doesn't specify a required accuracy for implementations.
@lukecwik added the `parseDouble` line I linked in
https://github.com/apache/beam/pull/8205, maybe he can clarify?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 324784)
Time Spent: 11h 40m (was: 11.5h)
> Make row coder a standard coder and implement in python
> -------------------------------------------------------
>
> Key: BEAM-7886
> URL: https://issues.apache.org/jira/browse/BEAM-7886
> Project: Beam
> Issue Type: Improvement
> Components: beam-model, sdk-java-core, sdk-py-core
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: Major
> Time Spent: 11h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
