Re: Null checking in Beam

2021-03-02 Thread Kyle Weaver
Can you try adding the generated classes to generatedClassPatterns in the JavaNatureConfiguration? https://github.com/apache/beam/blob/03b883b415d27244ddabb17a0fb5bab147b86f89/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L92 On Tue, Mar 2, 2021 at 12:05 AM Reuven Lax

Re: Random outputs for ARRAY_CONCAT_AGG fn zetasql

2021-03-02 Thread Kyle Weaver
As you can see from existing tests, Beam doesn't materialize the output array directly. Instead you must use the PAssert API. I agree with Tyson's suggestion to use `satisfies`, which lets you do arbitrary assertions on the output data. On Tue, Mar 2, 2021 at 3:57 AM Sonam Ramchand <

Re: DoFn @Setup with PipelineOptions

2021-03-02 Thread Xinyu Liu
I created a ticket to track this: https://issues.apache.org/jira/browse/BEAM-11914. Thanks everyone for the comments! Thanks, Xinyu On Mon, Mar 1, 2021 at 4:45 PM Xinyu Liu wrote: > The reason for not passing it in directly is that we have a large amount > of configs here at LinkedIn so we use

Re: Random outputs for ARRAY_CONCAT_AGG fn zetasql

2021-03-02 Thread Sonam Ramchand
Is there any way I can access the output array resulting from the sql query? Then maybe I can sort and compare both *output array* and *expected output array *for the test to pass. On Tue, Mar 2, 2021 at 12:24 AM Kenneth Knowles wrote: > Yea, the reason is that SQL relations are not ordered. So

Re: Null checking in Beam

2021-03-02 Thread Reuven Lax
I'm running into a problem with this check. I added a protocol-buffer file to a module (google-cloud-platform) that previously did have any protobuf files in it. The generated files contain lines that violate this null checker, so they won't compile. I can't annotate the files, because they are