[
https://issues.apache.org/jira/browse/BEAM-13202?focusedWorklogId=728109&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-728109
]
ASF GitHub Bot logged work on BEAM-13202:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Feb/22 09:58
Start Date: 16/Feb/22 09:58
Worklog Time Spent: 10m
Work Description: RyanSkraba commented on a change in pull request #16856:
URL: https://github.com/apache/beam/pull/16856#discussion_r807730968
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/agg/CountIf.java
##########
@@ -31,62 +30,41 @@
public class CountIf {
private CountIf() {}
- public static CountIfFn combineFn() {
- return new CountIf.CountIfFn();
+ public static Combine.CombineFn<Boolean, ?, Long> combineFn() {
+ return new CountIfFn();
}
- public static class CountIfFn extends Combine.CombineFn<Boolean,
CountIfFn.Accum, Long> {
-
- @AutoValue
- public abstract static class Accum implements Serializable {
- abstract boolean isExpressionFalse();
-
- abstract long countIfResult();
-
- static Accum empty() {
- return of(true, 0L);
- }
-
- static Accum of(boolean isExpressionFalse, long countIfResult) {
- return new AutoValue_CountIf_CountIfFn_Accum(isExpressionFalse,
countIfResult);
- }
- }
+ public static class CountIfFn extends Combine.CombineFn<Boolean, long[],
Long> {
Review comment:
Nice cleanup! At this point, would it be worthwhile to let
Count.CountFn be public, so we could just inherit it? I don't really have a
strong opinion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 728109)
Time Spent: 1h 10m (was: 1h)
> Add Coder to CountIfFn.Accum
> ----------------------------
>
> Key: BEAM-13202
> URL: https://issues.apache.org/jira/browse/BEAM-13202
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Affects Versions: 2.33.0
> Environment: GCP Dataflow
> Reporter: Patricio Navarro
> Assignee: Ismaël Mejía
> Priority: P3
> Fix For: 2.38.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When using
> {code:java}
> org.apache.beam.sdk.extensions.sql.impl.transform.agg.CountIf
> {code}
> with dataflow-runner I got the following issue:
> {noformat}
> [WARNING]
> java.lang.RuntimeException: java.io.IOException: Could not obtain a Coder for
> the accumulator
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:78)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> Caused by: java.io.IOException: Could not obtain a Coder for the accumulator
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.extractAccumulatorCoder
> (CombineTranslation.java:207)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.translate
> (CombineTranslation.java:179)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation$KnownTransformPayloadTranslator.translate
> (PTransformTranslation.java:438)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation.toProto
> (PTransformTranslation.java:248)
> at
> org.apache.beam.runners.core.construction.SdkComponents.registerPTransform
> (SdkComponents.java:175)
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:75)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> Caused by: org.apache.beam.sdk.coders.CannotProvideCoderException: Cannot
> infer coder for type parameter AccumT
> at org.apache.beam.sdk.coders.CoderRegistry.getCoder
> (CoderRegistry.java:328)
> at
> org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn.getAccumulatorCoder
> (CombineFnBase.java:119)
> at org.apache.beam.sdk.transforms.Combine$CombineFn.getAccumulatorCoder
> (Combine.java:391)
> at
> org.apache.beam.sdk.extensions.sql.impl.transform.agg.AggregationCombineFnAdapter$WrappedCombinerBase.getAccumulatorCoder
> (AggregationCombineFnAdapter.java:75)
> at
> org.apache.beam.sdk.transforms.CombineFns$ComposedCombineFn.getAccumulatorCoder
> (CombineFns.java:430)
> at
> org.apache.beam.sdk.schemas.transforms.SchemaAggregateFn$Inner.getAccumulatorCoder
> (SchemaAggregateFn.java:335)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.extractAccumulatorCoder
> (CombineTranslation.java:204)
> at
> org.apache.beam.runners.core.construction.CombineTranslation$CombineGroupedValuesPayloadTranslator.translate
> (CombineTranslation.java:179)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation$KnownTransformPayloadTranslator.translate
> (PTransformTranslation.java:438)
> at
> org.apache.beam.runners.core.construction.PTransformTranslation.toProto
> (PTransformTranslation.java:248)
> at
> org.apache.beam.runners.core.construction.SdkComponents.registerPTransform
> (SdkComponents.java:175)
> at
> org.apache.beam.runners.core.construction.PipelineTranslation$1.leaveCompositeTransform
> (PipelineTranslation.java:75)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:584)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit
> (TransformHierarchy.java:579)
> at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500
> (TransformHierarchy.java:239)
> at org.apache.beam.sdk.runners.TransformHierarchy.visit
> (TransformHierarchy.java:213)
> at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:468)
> at org.apache.beam.runners.core.construction.PipelineTranslation.toProto
> (PipelineTranslation.java:60)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:996)
> at org.apache.beam.runners.dataflow.DataflowRunner.run
> (DataflowRunner.java:203)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:322)
> at org.apache.beam.sdk.Pipeline.run (Pipeline.java:308)
> at
> com.hypertv.AnalyticsLoggerStore.PubsubToStorage.runPubsubToStoragePipeline
> (PubsubToStorage.java:115)
> at com.hypertv.AnalyticsLoggerStore.PubsubToStorage.main
> (PubsubToStorage.java:122)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:566)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
> at java.lang.Thread.run (Thread.java:829)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 57.006 s
> [INFO] Finished at: 2021-10-25T13:54:23Z
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
> (default-cli) on project connected-stories-analytics-logger-store: An
> exception occured while executing the Java class. java.io.IOException: Could
> not obtain a Coder for the accumulator: Cannot infer coder for type parameter
> AccumT -> [Help 1]
> {noformat}
> So I found out that
> {code:java}
> CountIfFn.Accum
> {code}
> on Beam seems not to have an associated Coder.
> FYI - [~iemejia]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
