Re: [Apace BEAM Go improvements] Lazy map side inputs

2021-03-30 Thread Robert Burke 
Buckle up, it's time to dive into Side Inputs along with Go SDK code links!

First and foremost, all the protos that cover side inputs (namely
beam_runner_api.proto [1], and beam_fn_api.proto [2], which nominally
handle construction time, and execution time concerns respectively) refer
to the design doc at
https://s.apache.org/beam-fn-state-api-and-bundle-processing [3] when it
comes to details around side inputs and the state API. I'm going to cover
side inputs exclusively, from User side, to construction, to execution.

Side Inputs are window scoped data outside of the parallel input that can
be supplied through the runner. From the user side, this allows a pipeline
to use any PCollection as additional (side) input to a DoFn, while also
handling whether that side input is in the same window as the parallel
input. It's better described in the programming guide section on Side
Inputs [4], so I won't repeat their utility here. On the execution side,
the data gets to the SDK side worker using the State Channel and API
(described in [3])

At pipeline construction time, an SDK will convert it's own internal
constructs into the Beam Pipeline protos, and in the Go SDK, that happens
in graphx/translate.go. The SDK translates DoFns into ParDoPayloads,
populating a map of SideInput protos [5][6]. But before any of that, the
user needs to be able to specify side inputs are to be used at all. So how
does a user do that in Go?

*# Part 1: Decomposition*

In the Go SDK, because at time of design, Generics didn't exist (not till
Go 1.18 at soonest), we heavily use the original generic concept: Functions
with typed parameters. If you squint, funcs can be viewed as generic
(apologies to the PL purists in the crowd). See the SDKs RFC [7] for more.
Throughout this explanation I'll be using loose Go syntax to represent some
concepts. This is not real Go code, and these symbols are arbitrary types,
rather than concrete types. After the User hands us a DoFn, we decompose
it's lifecycle methods (like ProcessElement) to see if it's well formed,
and what inputs it accepts and outputs it emits.

 So a DoFn with it's ProcessElement  method can specify a wide variety of
inputs. The designers ended up decreeing that we'll just specify an order
to the parameters (instead of explicit tags, like Java does), and that will
determine a parameter's role in a ParDo's graph representation. Roughly, as
a pseudo-regular expression, for a ProcessElement it's defined as follows :

func(FnValue, SideInput*, FnEmit*)

Where FnValue means any user element in a PCollection, SideInput can be one
of FnValue, FnIter, FnReIter, and FnEmit represents outputs from this DoFn.
FnIter, and FnReIter represent Beam Iterables, used either for SideInputs,
or for GBK values (mentioned in my last email). This is a dramatic
simplification of the more complete expression for any Lifecycle method
(seen at [8]) which has a more complete ordering. Order is important, as we
don't have much else to go on, with the first FnValue is always the
parallel input. Those Fn* terms are also defined in that same file. This is
how the SDK understands functions and parameters, though decomposition.

In the Go SDK, Side Inputs can be an iterable (represented by a `func(*V)
bool` type), a ReIterable (represented by a `func() func(*V) bool` type),
or a Singleton (represented by the type, V). We use functions to handle
iterables, because of the lack of generics. Similarly, we can and should
handle functional representations of Map access to that data (the subject
of this current discussion, BEAM-3293
).

To see what a given user type parameter represents, we have helper
functions [9] to make that determination, called in a state machine [10].
To handle map functions for BEAM-3293, new functions, representation kinds,
and cases would need to be added there.

A wrinkle in this is that Side inputs can be a singleton value (a
PCollection with 1 element), so the SDK can't statically know what a DoFn's
parameters mean without the context of it's input PCollections.  Depending
on the input PCollection, a func(K,V) that is decomposed to func(FnValue,
FnValue) could either be a DoFn that processes a PCollection with a
singleton side input PCollection, or a DoFn that processes a
PCollection>. For that we need to bind the inputs to the DoFn
parameters.

*# Part 2: Binding*

Now that the SDK understands the shape of the user DoFn, it needs to
understand how it connects to the graph. This is done through binding the
input PCollections to the DoFns parameters.

Pertinent to this discussion is [11] where side inputs kinds are bound to
specific types. This would need to handle and produce the new map kinds.
SideInput Maps are simpler, since we know they *must* be associated with a
PCollection>. If the Nth SideInput PCollection doesn't match with
the Nth side input parameter, then an error should occur at binding time.

Binding errors are tricky since the

Flaky test issue report

2021-03-30 Thread Beam Jira Bot 
This is your daily summary of Beam's current flaky tests. These are P1 issues 
because they have a major negative impact on the community and make it hard to 
determine the quality of the software.

BEAM-12061: beam_PostCommit_SQL failing on 
KafkaTableProviderIT.testFakeNested 
(https://issues.apache.org/jira/browse/BEAM-12061)
BEAM-12020: :sdks:java:container:java8:docker failing missing licenses 
(https://issues.apache.org/jira/browse/BEAM-12020)
BEAM-12019: 
apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics
 is flaky (https://issues.apache.org/jira/browse/BEAM-12019)
BEAM-11792: Python precommit failed (flaked?) installing package  
(https://issues.apache.org/jira/browse/BEAM-11792)
BEAM-11733: [beam_PostCommit_Java] [testFhirIO_Import|export] flaky 
(https://issues.apache.org/jira/browse/BEAM-11733)
BEAM-11666: 
apache_beam.runners.interactive.recording_manager_test.RecordingManagerTest.test_basic_execution
 is flaky (https://issues.apache.org/jira/browse/BEAM-11666)
BEAM-11662: elasticsearch tests failing 
(https://issues.apache.org/jira/browse/BEAM-11662)
BEAM-11661: hdfsIntegrationTest flake: network not found (py38 postcommit) 
(https://issues.apache.org/jira/browse/BEAM-11661)
BEAM-11646: beam_PostCommit_XVR_Spark failing 
(https://issues.apache.org/jira/browse/BEAM-11646)
BEAM-11645: beam_PostCommit_XVR_Flink failing 
(https://issues.apache.org/jira/browse/BEAM-11645)
BEAM-11541: testTeardownCalledAfterExceptionInProcessElement flakes on 
direct runner. (https://issues.apache.org/jira/browse/BEAM-11541)
BEAM-11540: Linter sometimes flakes on apache_beam.dataframe.frames_test 
(https://issues.apache.org/jira/browse/BEAM-11540)
BEAM-11493: Spark test failure: 
org.apache.beam.sdk.transforms.GroupByKeyTest$WindowTests.testGroupByKeyAndWindows
 (https://issues.apache.org/jira/browse/BEAM-11493)
BEAM-11492: Spark test failure: 
org.apache.beam.sdk.transforms.GroupByKeyTest$WindowTests.testGroupByKeyMergingWindows
 (https://issues.apache.org/jira/browse/BEAM-11492)
BEAM-11491: Spark test failure: 
org.apache.beam.sdk.transforms.GroupByKeyTest$WindowTests.testGroupByKeyMultipleWindows
 (https://issues.apache.org/jira/browse/BEAM-11491)
BEAM-11490: Spark test failure: 
org.apache.beam.sdk.transforms.ReifyTimestampsTest.inValuesSucceeds 
(https://issues.apache.org/jira/browse/BEAM-11490)
BEAM-11489: Spark test failure: 
org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAttemptedDistributionMetrics
 (https://issues.apache.org/jira/browse/BEAM-11489)
BEAM-11488: Spark test failure: 
org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAttemptedCounterMetrics
 (https://issues.apache.org/jira/browse/BEAM-11488)
BEAM-11487: Spark test failure: 
org.apache.beam.sdk.transforms.WithTimestampsTest.withTimestampsShouldApplyTimestamps
 (https://issues.apache.org/jira/browse/BEAM-11487)
BEAM-11486: Spark test failure: 
org.apache.beam.sdk.testing.PAssertTest.testSerializablePredicate 
(https://issues.apache.org/jira/browse/BEAM-11486)
BEAM-11485: Spark test failure: 
org.apache.beam.sdk.transforms.CombineFnsTest.testComposedCombineNullValues 
(https://issues.apache.org/jira/browse/BEAM-11485)
BEAM-11484: Spark test failure: 
org.apache.beam.runners.core.metrics.MetricsPusherTest.pushesUserMetrics 
(https://issues.apache.org/jira/browse/BEAM-11484)
BEAM-11483: Spark PostCommit Test Improvements 
(https://issues.apache.org/jira/browse/BEAM-11483)
BEAM-10995: Java + Universal Local Runner: 
WindowingTest.testWindowPreservation fails 
(https://issues.apache.org/jira/browse/BEAM-10995)
BEAM-10987: stager_test.py::StagerTest::test_with_main_session flaky on 
windows py3.6,3.7 (https://issues.apache.org/jira/browse/BEAM-10987)
BEAM-10968: flaky test: 
org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAttemptedDistributionMetrics
 (https://issues.apache.org/jira/browse/BEAM-10968)
BEAM-10955: Flink Java Runner test flake: Could not find Flink job  
(https://issues.apache.org/jira/browse/BEAM-10955)
BEAM-10923: Python requirements installation in docker container is flaky 
(https://issues.apache.org/jira/browse/BEAM-10923)
BEAM-10901: Flaky test: 
PipelineInstrumentTest.test_able_to_cache_intermediate_unbounded_source_pcollection
 (https://issues.apache.org/jira/browse/BEAM-10901)
BEAM-10899: test_FhirIO_exportFhirResourcesGcs flake with OOM 
(https://issues.apache.org/jira/browse/BEAM-10899)
BEAM-10866: PortableRunnerTestWithSubprocesses.test_register_finalizations 
flaky on macOS (https://issues.apache.org/jira/browse/BEAM-10866)
BEAM-10763: Spotless flake (NullPointerException) 
(https://issues.apache.org/jira/browse/BEAM-10763)
BEAM-10590: BigQueryQueryToTableIT flaky: test_big_query_new_types 
(https://issues.apache.org/jira/browse/BEAM-10590)
BEAM-10589: Samza ValidatesRunner failure:

P1 issues report

2021-03-30 Thread Beam Jira Bot 
This is your daily summary of Beam's current P1 issues, not including flaky 
tests.

See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the 
meaning and expectations around P1 issues.

BEAM-12071: DataFrame IO sinks do not correctly partition by window 
(https://issues.apache.org/jira/browse/BEAM-12071)
BEAM-12060: beam_PostCommit_Go_VR_Flink and beam_PostCommit_Go_VR_Spark 
failing since Mar 23, 2021 6:00:00 AM 
(https://issues.apache.org/jira/browse/BEAM-12060)
BEAM-12050: ParDoTest TimerTests that use TestStream failing for portable 
FlinkRunner (https://issues.apache.org/jira/browse/BEAM-12050)
BEAM-11965: testSplitQueryFnWithLargeDataset timeout failures 
(https://issues.apache.org/jira/browse/BEAM-11965)
BEAM-11961: InfluxDBIOIT failing with unauthorized error 
(https://issues.apache.org/jira/browse/BEAM-11961)
BEAM-11959: Python Beam SDK Harness hangs when installing pip packages 
(https://issues.apache.org/jira/browse/BEAM-11959)
BEAM-11922: 
org.apache.beam.examples.cookbook.MapClassIntegrationIT.testDataflowMapState 
has been failing in master (https://issues.apache.org/jira/browse/BEAM-11922)
BEAM-11886: MapState and SetState failing tests on Dataflow streaming 
(https://issues.apache.org/jira/browse/BEAM-11886)
BEAM-11828: JmsIO is not acknowledging messages correctly 
(https://issues.apache.org/jira/browse/BEAM-11828)
BEAM-11772: GCP BigQuery sink (file loads) uses runner determined sharding 
for unbounded data (https://issues.apache.org/jira/browse/BEAM-11772)
BEAM-11755: Cross-language consistency (RequiresStableInputs) is quietly 
broken (at least on portable flink runner) 
(https://issues.apache.org/jira/browse/BEAM-11755)
BEAM-11578: `dataflow_metrics` (python) fails with TypeError (when int 
overflowing?) (https://issues.apache.org/jira/browse/BEAM-11578)
BEAM-11576: Go ValidatesRunner failure: TestFlattenDup on Dataflow Runner 
(https://issues.apache.org/jira/browse/BEAM-11576)
BEAM-11434: Expose Spanner admin/batch clients in Spanner Accessor 
(https://issues.apache.org/jira/browse/BEAM-11434)
BEAM-11227: Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216 
(https://issues.apache.org/jira/browse/BEAM-11227)
BEAM-11148: Kafka commitOffsetsInFinalize OOM on Flink 
(https://issues.apache.org/jira/browse/BEAM-11148)
BEAM-11017: Timer with dataflow runner can be set multiple times (dataflow 
runner) (https://issues.apache.org/jira/browse/BEAM-11017)
BEAM-10883: XmlIO parsing of multibyte characters 
(https://issues.apache.org/jira/browse/BEAM-10883)
BEAM-10861: Adds URNs and payloads to PubSub transforms 
(https://issues.apache.org/jira/browse/BEAM-10861)
BEAM-10663: CrossLanguageKafkaIOTest broken on Flink Runner 
(https://issues.apache.org/jira/browse/BEAM-10663)
BEAM-10617: python CombineGlobally().with_fanout() cause duplicate combine 
results for sliding windows (https://issues.apache.org/jira/browse/BEAM-10617)
BEAM-10573: CSV files are loaded several times if they are too large 
(https://issues.apache.org/jira/browse/BEAM-10573)
BEAM-10569: SpannerIO tests don't actually assert anything. 
(https://issues.apache.org/jira/browse/BEAM-10569)
BEAM-10288: Quickstart documents are out of date 
(https://issues.apache.org/jira/browse/BEAM-10288)
BEAM-10244: Populate requirements cache fails on poetry-based packages 
(https://issues.apache.org/jira/browse/BEAM-10244)
BEAM-10100: FileIO writeDynamic with AvroIO.sink not writing all data 
(https://issues.apache.org/jira/browse/BEAM-10100)
BEAM-9917: BigQueryBatchFileLoads dynamic destination 
(https://issues.apache.org/jira/browse/BEAM-9917)
BEAM-9564: Remove insecure ssl options from MongoDBIO 
(https://issues.apache.org/jira/browse/BEAM-9564)
BEAM-9455: Environment-sensitive provisioning for Dataflow 
(https://issues.apache.org/jira/browse/BEAM-9455)
BEAM-9154: Move Chicago Taxi Example to Python 3 
(https://issues.apache.org/jira/browse/BEAM-9154)
BEAM-8407: [SQL] Some Hive tests throw NullPointerException, but get marked 
as passing (Direct Runner) (https://issues.apache.org/jira/browse/BEAM-8407)
BEAM-7717: PubsubIO watermark tracking hovers near start of epoch 
(https://issues.apache.org/jira/browse/BEAM-7717)
BEAM-7716: PubsubIO returns empty message bodies for all messages read 
(https://issues.apache.org/jira/browse/BEAM-7716)
BEAM-7195: BigQuery - 404 errors for 'table not found' when using dynamic 
destinations - sometimes, new table fails to get created 
(https://issues.apache.org/jira/browse/BEAM-7195)
BEAM-6839: User reports protobuf ClassChangeError running against 2.6.0 or 
above (https://issues.apache.org/jira/browse/BEAM-6839)
BEAM-6466: KafkaIO doesn't commit offsets while being used as bounded 
source (https://issues.apache.org/jira/browse/BEAM-6466)
BEAM-5997: EVENT_TIME timer throws exception when side input used

Re: [PROPOSAL] Preparing for Beam 2.29.0 release

2021-03-30 Thread Brian Hulette 
On Tue, Mar 30, 2021 at 12:16 PM Kenneth Knowles  wrote:

> Update:
>
> Based on a number of issues raised, I have re-cut the branch.
>
> All Jiras that were resolved to 2.30.0 I have adjusted to 2.29.0.
>
> The two remaining issues are:
>
> BEAM-11227 "Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216":
> the vendored gRPC has been released and is being debugged. We will
> cherrypick the upgrade.
>
> BEAM-12071 "DataFrame IO sinks do not correctly partition by window": this
> is not a regression but is a severe problem that makes dataframes unusable;
> a PR is already up
>

Note this makes DataFrame sinks unusable when used with windowing, not the
entire DataFrame API :)
PR is here: https://github.com/apache/beam/pull/14374


>
> If you are tracking any issue not mentioned here, be sure to let me know.
>
> Kenn
>
> On Tue, Mar 16, 2021 at 12:08 PM Kenneth Knowles  wrote:
>
>> Update:
>>
>> There are 5 issues tagged to 2.29.0 release:
>> https://issues.apache.org/jira/issues/?jql=statusCategory%20!%3D%20done%20AND%20project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>>
>> There are 4 issues in Needs Triage state with priority P0 or P1:
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20%22Triage%20Needed%22%20AND%20priority%20in%20(P0%2C%20P1)%20ORDER%20BY%20updated%20DESC
>>
>> Kenn
>>
>> On Mon, Mar 15, 2021 at 5:01 PM Kenneth Knowles  wrote:
>>
>>> Update:
>>>
>>> There are 8 issues tagged to 2.29.0 release:
>>> https://issues.apache.org/jira/issues/?jql=statusCategory%20!%3D%20done%20AND%20project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>>>
>>> There are 22 issues in Needs Triage state with priority P0 or P1:
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20%22Triage%20Needed%22%20AND%20priority%20in%20(P0%2C%20P1)%20ORDER%20BY%20updated%20DESC
>>>
>>> Kenn
>>>
>>> On Thu, Mar 11, 2021 at 8:43 PM Kenneth Knowles  wrote:
>>>

 https://issues.apache.org/jira/issues/?jql=project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20AND%20statusCategory%20%3D%20%22To%20Do%22%20%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC

 On Thu, Mar 11, 2021 at 8:43 PM Kenneth Knowles 
 wrote:

> Correction: that search did not limit to open issues. There are 17
> open.
>
> Kenn
>
> On Thu, Mar 11, 2021 at 8:42 PM Kenneth Knowles 
> wrote:
>
>> Update: I cut the branch today.
>>
>> There are 49 (!) issues with Fix Version = 2.29.0
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>>
>> If you are familiar with any of these, please help me by determining
>> if it is a critical release-blocking bug or if it is something else.
>>
>> Kenn
>>
>> On Wed, Mar 3, 2021 at 11:53 AM Ahmet Altay  wrote:
>>
>>> +1 and thank you!
>>>
>>> On Wed, Mar 3, 2021 at 11:51 AM Kenneth Knowles 
>>> wrote:
>>>
 Hi All,

 Beam 2.29.0 release is scheduled to be cut in a week, on March 10
 according to the release calendar [1].

 I'd like to volunteer myself to be the release manager for this
 release. I plan on cutting the release branch on the scheduled
 date.

 Any comments or objections?

 Kenn

 [1]
 https://calendar.google.com/calendar/u/0/embed?src=0p73sl034k80oob7seouani...@group.calendar.google.com=America/Los_Angeles

>>>

Re: [PROPOSAL] Preparing for Beam 2.29.0 release

2021-03-30 Thread Kenneth Knowles 
Update:

Based on a number of issues raised, I have re-cut the branch.

All Jiras that were resolved to 2.30.0 I have adjusted to 2.29.0.

The two remaining issues are:

BEAM-11227 "Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216": the
vendored gRPC has been released and is being debugged. We will cherrypick
the upgrade.

BEAM-12071 "DataFrame IO sinks do not correctly partition by window": this
is not a regression but is a severe problem that makes dataframes unusable;
a PR is already up

If you are tracking any issue not mentioned here, be sure to let me know.

Kenn

On Tue, Mar 16, 2021 at 12:08 PM Kenneth Knowles  wrote:

> Update:
>
> There are 5 issues tagged to 2.29.0 release:
> https://issues.apache.org/jira/issues/?jql=statusCategory%20!%3D%20done%20AND%20project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>
> There are 4 issues in Needs Triage state with priority P0 or P1:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20%22Triage%20Needed%22%20AND%20priority%20in%20(P0%2C%20P1)%20ORDER%20BY%20updated%20DESC
>
> Kenn
>
> On Mon, Mar 15, 2021 at 5:01 PM Kenneth Knowles  wrote:
>
>> Update:
>>
>> There are 8 issues tagged to 2.29.0 release:
>> https://issues.apache.org/jira/issues/?jql=statusCategory%20!%3D%20done%20AND%20project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>>
>> There are 22 issues in Needs Triage state with priority P0 or P1:
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20status%20%3D%20%22Triage%20Needed%22%20AND%20priority%20in%20(P0%2C%20P1)%20ORDER%20BY%20updated%20DESC
>>
>> Kenn
>>
>> On Thu, Mar 11, 2021 at 8:43 PM Kenneth Knowles  wrote:
>>
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20AND%20statusCategory%20%3D%20%22To%20Do%22%20%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>>>
>>> On Thu, Mar 11, 2021 at 8:43 PM Kenneth Knowles  wrote:
>>>
 Correction: that search did not limit to open issues. There are 17 open.

 Kenn

 On Thu, Mar 11, 2021 at 8:42 PM Kenneth Knowles 
 wrote:

> Update: I cut the branch today.
>
> There are 49 (!) issues with Fix Version = 2.29.0
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
>
> If you are familiar with any of these, please help me by determining
> if it is a critical release-blocking bug or if it is something else.
>
> Kenn
>
> On Wed, Mar 3, 2021 at 11:53 AM Ahmet Altay  wrote:
>
>> +1 and thank you!
>>
>> On Wed, Mar 3, 2021 at 11:51 AM Kenneth Knowles 
>> wrote:
>>
>>> Hi All,
>>>
>>> Beam 2.29.0 release is scheduled to be cut in a week, on March 10
>>> according to the release calendar [1].
>>>
>>> I'd like to volunteer myself to be the release manager for this
>>> release. I plan on cutting the release branch on the scheduled date.
>>>
>>> Any comments or objections?
>>>
>>> Kenn
>>>
>>> [1]
>>> https://calendar.google.com/calendar/u/0/embed?src=0p73sl034k80oob7seouani...@group.calendar.google.com=America/Los_Angeles
>>>
>>

Re: [VOTE] Release vendor-calcite-1_26_0 version 0.1, release candidate #1

2021-03-30 Thread Andrew Pilloud 
Sorry for the delay, the vote has passed, with +5 approving votes, and +4
binding PMC votes:

 - Kai Jiang
 - Pablo Estrada
 -  Ahmet Altay
 - Jean-Baptiste Onofre
 - Kenneth Knowles

I will finalize the release today.

Andrew

On Wed, Mar 10, 2021 at 1:35 AM Ismaël Mejía  wrote:

> Thanks Brian I see that there are a lot of reasons to do the upgrade,
> maybe in the future it would be a good idea to explain the main
> motivations of the upgrade for vendored dependencies and refer
> relevant JIRAs
>
> On Tue, Mar 9, 2021 at 6:23 PM Brian Hulette  wrote:
> >
> > There are several jiras blocked by a Calcite upgrade. See
> https://issues.apache.org/jira/browse/BEAM-9379
> >
> > On Tue, Mar 9, 2021 at 5:17 AM Ismaël Mejía  wrote:
> >>
> >> Just out of curiosity is there some feature we are expecting from
> >> Calcite that pushes this upgrade or is this just catching up for the
> >> sake of security improvements + not having old dependencies?
> >>
> >>
> >> On Tue, Mar 9, 2021 at 12:23 AM Ahmet Altay  wrote:
> >> >
> >> > +1 (binding)
> >> >
> >> > On Mon, Mar 8, 2021 at 3:21 PM Pablo Estrada 
> wrote:
> >> >>
> >> >> +1 (binding) verified signature
> >> >>
> >> >> On Mon, Mar 8, 2021 at 12:05 PM Kai Jiang 
> wrote:
> >> >>>
> >> >>> +1 (non-binding)
> >> >>>
> >> >>> On Mon, Mar 8, 2021 at 11:37 AM Andrew Pilloud 
> wrote:
> >> 
> >>  Hi everyone,
> >>  Please review and vote on the release candidate #1 for
> beam-vendor-calcite-1_26_0 version 0.1, as follows:
> >>  [ ] +1, Approve the release
> >>  [ ] -1, Do not approve the release (please provide specific
> comments)
> >> 
> >> 
> >>  Reviewers are encouraged to test their own use cases with the
> release candidate, and vote +1 if no issues are found.
> >> 
> >>  The complete staging area is available for your review, which
> includes:
> >>  * the official Apache source release to be deployed to
> dist.apache.org [1], which is signed with the key with fingerprint
> 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [2],
> >>  * all artifacts to be deployed to the Maven Central Repository [3],
> >>  * source code commit "0f52187e344dad9bca4c183fe51151b733b24e35"
> [4].
> >> 
> >>  The vote will be open for at least 72 hours. It is adopted by
> majority approval, with at least 3 PMC affirmative votes.
> >> 
> >>  Thanks,
> >>  Andrew
> >> 
> >>  [1]
> https://dist.apache.org/repos/dist/dev/beam/vendor/beam-vendor-calcite-1_26_0/0.1/
> >>  [2] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>  [3]
> https://repository.apache.org/content/repositories/orgapachebeam-1163/
> >>  [4]
> https://github.com/apache/beam/tree/0f52187e344dad9bca4c183fe51151b733b24e35
>