[
https://issues.apache.org/jira/browse/BEAM-4461?focusedWorklogId=159949&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-159949
]
ASF GitHub Bot logged work on BEAM-4461:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Oct/18 10:59
Start Date: 29/Oct/18 10:59
Worklog Time Spent: 10m
Work Description: lgajowy edited a comment on issue #6856: [BEAM-4461]
Switch SQL over to use the new Beam Group transform.
URL: https://github.com/apache/beam/pull/6856#issuecomment-433868590
It seems that this commit causes failures in SQL Nexmark queries on Dataflow
runner. The failure was invisible in the weekend for us because other problems
with nexmark prevented this error to appear.
```
Caused by: java.lang.IllegalArgumentException: Provided coders for type
arguments of class
org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn contain
incompatibilities: Cannot encode elements of type class java.lang.Object with
coder org.apache.beam.sdk.coders.RowCoder@7fa2f06
at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:438)
at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:368)
at
org.apache.beam.sdk.coders.CoderRegistry.getCoder(CoderRegistry.java:318)
at
org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn.getAccumulatorCoder(CombineFnBase.java:119)
at
org.apache.beam.sdk.transforms.Combine$CombineFn.getAccumulatorCoder(Combine.java:330)
at
org.apache.beam.sdk.transforms.CombineFns$ComposedCombineFn.getAccumulatorCoder(CombineFns.java:426)
at
org.apache.beam.sdk.schemas.transforms.SchemaAggregateFn$Inner.getAccumulatorCoder(SchemaAggregateFn.java:296)
at
org.apache.beam.sdk.util.AppliedCombineFn.withInputCoder(AppliedCombineFn.java:106)
at
org.apache.beam.sdk.transforms.Combine$GroupedValues.getAppliedFn(Combine.java:2129)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translateHelper(DataflowPipelineTranslator.java:772)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translate(DataflowPipelineTranslator.java:755)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translate(DataflowPipelineTranslator.java:752)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:440)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:379)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:169)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:733)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:178)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at
org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1155)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:108)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:96)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by:
org.apache.beam.sdk.coders.CoderRegistry$IncompatibleCoderException: Cannot
encode elements of type class java.lang.Object with coder
org.apache.beam.sdk.coders.RowCoder@7fa2f06 because the coded type class
org.apache.beam.sdk.values.Row is not assignable from class java.lang.Object
at
org.apache.beam.sdk.coders.CoderRegistry.verifyCompatible(CoderRegistry.java:529)
at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:435)
... 37 more
```
See here: [example
logs](https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Dataflow/854/consoleText)
@reuvenlax I proposed a revert here:
https://github.com/apache/beam/pull/6866
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 159949)
Time Spent: 13h 50m (was: 13h 40m)
> Create a library of useful transforms that use schemas
> ------------------------------------------------------
>
> Key: BEAM-4461
> URL: https://issues.apache.org/jira/browse/BEAM-4461
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
> Time Spent: 13h 50m
> Remaining Estimate: 0h
>
> e.g. JoinBy(fields). Project, Filter, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
