Hi Alexey, I am using Beam 2.46.0, and this issue happens in both direct-runner and flink-runner (haven't tried other runners). Here's the gist to reproduce this issue. (I can not share the full piece of code, but the gist should contain enough information). This issue only happens when I join this kafka source with one side input source (created from memory). No such issue when I only query the kafka source. And also no such issue when I join another table with this side input source. So I suspect it is highly related to my kafka topic data, but I am sorry I can not share too much about our data. What I can share is that the topic schema is a nested structure, 2 sub rows inside another row.
https://gist.github.com/zjffdu/76bf394b4eae9b29eb2352dd068a1feb On Tue, Apr 18, 2023 at 11:20 PM Alexey Romanenko <aromanenko....@gmail.com> wrote: > Hi Jeff, > > Sorry for delay with an answer. > > Could you give more details (e.g. your pipeline and sql query code > snippets) about how to reproduce this issue? > Which Beam version do you use? > > — > Alexey > > On 30 Mar 2023, at 03:21, Jeff Zhang <zjf...@gmail.com> wrote: > > > Hi, folks, > > I asked this question in the beam slack channel, but there seems to be no > comments, so I would like to ask it in the user mail list again. Here's the > problem I hit, I use SqlTransform to process my data, this following > ClassCastException happens when my data has bytes type. It seems like a bug > to me. Can anyone help confirm that? Thanks > > > Caused by: java.lang.ClassCastException: org.apache.beam.vendor.calcite. > v1_28_0.org.apache.calcite.avatica.util.ByteString cannot be cast to [B > at SC.eval0(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:798) > at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:790) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn.processElement(BeamCalcRel.java:316) > at > org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn$DoFnInvoker.invokeProcessElement(Unknown > Source) > at > org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > at > org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > at > org.apache.beam.repackaged.direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:79) > at > org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:244) > at > org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:54) > at > org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:165) > at > org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:129) > (edited) > > > > -- > Best Regards > > Jeff Zhang > > > -- Best Regards Jeff Zhang
