[
https://issues.apache.org/jira/browse/BEAM-13202?focusedWorklogId=728270&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-728270
]
ASF GitHub Bot logged work on BEAM-13202:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Feb/22 15:30
Start Date: 16/Feb/22 15:30
Worklog Time Spent: 10m
Work Description: iemejia commented on a change in pull request #16856:
URL: https://github.com/apache/beam/pull/16856#discussion_r808121567
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/agg/CountIf.java
##########
@@ -27,49 +30,41 @@
public class CountIf {
private CountIf() {}
- public static CountIfFn combineFn() {
- return new CountIf.CountIfFn();
+ public static Combine.CombineFn<Boolean, ?, Long> combineFn() {
Review comment:
Yes, I hesitated but this is internal and this class is only
instantiated by SQL when it registers the built-in aggregators so we should be
good. sql/impl is not explicitly marked as Internal but I agree that it should.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 728270)
Time Spent: 1.5h (was: 1h 20m)
> Add Coder to CountIfFn.Accum
> ----------------------------
>
> Key: BEAM-13202
> URL: https://issues.apache.org/jira/browse/BEAM-13202
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Affects Versions: 2.33.0
> Environment: GCP Dataflow
> Reporter: Patricio Navarro
> Assignee: Ismaël Mejía
> Priority: P3
> Fix For: 2.38.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When using
> {code:java}
> org.apache.beam.sdk.extensions.sql.impl.transform.agg.CountIf
> {code}
> with dataflow-runner I got the following issue:
> {noformat}
> [WARNING]
> java.lang.RuntimeException: java.io.IOException: Could not obtain a Coder for
> the accumulator
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:78)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> Caused by: java.io.IOException: Could not obtain a Coder for the accumulator
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.extractAccumulatorCoder
> (CombineTranslation.java:207)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.translate
> (CombineTranslation.java:179)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation$KnownTransformPayloadTranslator.translate
> (PTransformTranslation.java:438)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation.toProto
> (PTransformTranslation.java:248)
> at
> org.apache.beam.runners.core.construction.SdkComponents.registerPTransform
> (SdkComponents.java:175)
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:75)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> Caused by: org.apache.beam.sdk.coders.CannotProvideCoderException: Cannot
> infer coder for type parameter AccumT
> at org.apache.beam.sdk.coders.CoderRegistry.getCoder
> (CoderRegistry.java:328)
> at
> org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn.getAccumulatorCoder
> (CombineFnBase.java:119)
> at org.apache.beam.sdk.transforms.Combine$CombineFn.getAccumulatorCoder
> (Combine.java:391)
> at
> org.apache.beam.sdk.extensions.sql.impl.transform.agg.AggregationCombineFnAdapter$WrappedCombinerBase.getAccumulatorCoder
> (AggregationCombineFnAdapter.java:75)
> at
> org.apache.beam.sdk.transforms.CombineFns$ComposedCombineFn.getAccumulatorCoder
> (CombineFns.java:430)
> at
> org.apache.beam.sdk.schemas.transforms.SchemaAggregateFn$Inner.getAccumulatorCoder
> (SchemaAggregateFn.java:335)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.extractAccumulatorCoder
> (CombineTranslation.java:204)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.translate
> (CombineTranslation.java:179)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation$KnownTransformPayloadTranslator.translate
> (PTransformTranslation.java:438)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation.toProto
> (PTransformTranslation.java:248)
> at
> org.apache.beam.runners.core.construction.SdkComponents.registerPTransform
> (SdkComponents.java:175)
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:75)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 57.006 s
> [INFO] Finished at: 2021-10-25T13:54:23Z
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
> (default-cli) on project connected-stories-analytics-logger-store: An
> exception occured while executing the Java class. java.io.IOException: Could
> not obtain a Coder for the accumulator: Cannot infer coder for type parameter
> AccumT -> [Help 1]
> {noformat}
> So I found out that
> {code:java}
> CountIfFn.Accum
> {code}
> on Beam seems not to have an associated Coder.
> FYI - [~iemejia]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
