This is an automated email from the ASF dual-hosted git repository.
mwalenia pushed a change to branch BEAM-9646-cloud-vision
in repository https://gitbox.apache.org/repos/asf/beam.git.
discard 6beafe0 Added change to CHANGES.md
discard d5f911b Fix spotbugs errors
discard 73390fd [BEAM-9646] Add Google Cloud vision integration transform
add 6e7e09f Merge pull request #11226: [BEAM-9557] Fix timer window
boundary checking
add 59ea9c2 Merge pull request #11244 from [BEAM-3097] _ReadFromBigQuery
supports valueprovider for table
add a271dc2 Name the pipeline_v1 proto import
add 02f328e Update materialize_test.go
add c1212a6 Merge pull request #11334 [Go SDK] Name the pipeline_v1 proto
import
add 442ad98 [BEAM-9715] Ensuring annotations_test passes in all
environments(Win/*nix)
add ea54794 Merge pull request #11329 from [BEAM-9715] Ensuring
annotations_test passes in all environments
add d333b2c Fix flaky interactive_runner_test
add 02f6f05 Merge pull request #11337 from Fix flaky
interactive_runner_test
add 0c1ac2c Merge pull request #11205 [BEAM-9578] Defer expensive
artifact enumeration in Java.
add 076dccd Fix some Go SDK linter/vet warnings. (#11330)
add 1a217db [BEAM-9707] Hardcode Unified harness image for fixing
dataflow VR 2 tests
add 1f48ac6 Merge pull request #11326 from angoenka/uw_image
add 4f4e9b3 [BEAM-9691] Ensuring BQSource is avoided on FnApi
add c00df44 Merge pull request #11309 from [BEAM-9691] Ensuring BQ Native
Sink is avoided on FnApi pipelines
add f1fac7c Update go protos.
add 9841f38 [BEAM-9618] Pull bundle descriptors for Go.
add 197d2dd Merge pull request #11324 [BEAM-9618] Pull bundle descriptors
for Go.
add ba24517 [BEAM-9577] Plumb resources through Python job service and
runner.
add 8239ffa Merge pull request #11312 from [BEAM-9577] Plumb resources
through Python job service and runner.
add a35afef [BEAM-9529] Remove datastore.v1, googledatastore (#11175)
add c70d30b [BEAM-5422] Document DynamicDestinations.getTable uniqueness
requirement
add a2766fb Merge pull request #11241: [BEAM-5422] Document
DynamicDestinations.getTable uniqueness requirement
add 825363b [BEAM-9147] Add a VideoIntelligence transform to Java SDK
(#11261)
add e325044 [BEAM-9580] Allow Flink 1.10 processing timers to finish on
pipeline shutdown
add d4790fd Revert "[BEAM-9580] Downgrade Flink version to 1.9 for
Nexmark and ValidatesRunner"
add b6dbf2a [BEAM-9557] Fix strings used to verify test output
add e7f6ccb Merge pull request #11346: [BEAM-9580] Allow Flink 1.10
processing timers to finish on pipeline shutdown
add c367536 [BEAM-9714] [Go SDK] Require --region flag in Dataflow runner.
add 76d7732 Merge pull request #11348 from ibzib/beam-9199-go
add 3f92f32 Update session.go
add f449dca Update stage.go
add 632535c Update server_test.go
add a4dcf29 Update materialize.go
add ca70ab8 Update materialize_test.go
add 4a378e9 Update stage_test.go
add b6e0584 Update artifact.go
add 209508c Update provision.go
add de68243 Update retrieval.go
add cf19153 Update staging.go
add 483a9c8 Update translate.go
add 11f086a Update datamgr.go
add 3c77761 Update datamgr_test.go
add 1894be7 Update logging.go
add 1378b31 Update logging_test.go
add e229c74 Update monitoring.go
add d9772ee Update session.go
add e1a2730 Update statemgr.go
add ca30a64 Update statemgr_test.go
add 1f0eb7c Update replace.go
add 46afdfd Update replace_test.go
add 68e369c Update provision.go
add c3b3c77 Update execute.go
add 532ea91 Update job.go
add a4d5648 Update translate.go
add 84ebc24 Update translate.go
add 54014d7 Update job.go
add eab9a0f Update materialize.go
add 2548664 Update translate.go
add 9da745e Update session.go
add ecb852b Update materialize_test.go
add ba6feec Merge pull request #11343 from apache/lostluck-protosuffix
add 826a68a [BEAM-9596] Ensure metrics are available in PipelineResult
when the pipeline fails
add c04ac9f Merge pull request #11349: [BEAM-9596] Ensure metrics are
available in PipelineResult when the pipeline fails
add 964c307 Updates documentation for WINDOWED_VALUE coder.
add 3038900 Uses iterable coder for windows and copies all of timestamp
encoding specification.
add 338168d Merge pull request #11336 from acrites/documentation
add 8fb7f0f Update session.go (#11352)
add b497bf4 [BEAM-9322] [BEAM-1833] Better naming for composite transform
output tags.
add cebe628 Merge pull request #11283 [BEAM-9322] [BEAM-1833] Better
naming for composite transform output tags.
add 568db83 [BEAM-9618] Java SDK worker support for pulling bundle
descriptors. (#11328)
add 82689f1 [BEAM-9716] Alias zone to worker_zone and warn user.
add b56740f Merge pull request #11332 from ibzib/beam-9716
add 6f6809a [Beam-9063]update documentation (#10952)
add af0acea Adds nose back under packages needed for testing.
add e7663b8 Merge pull request #11354: [BEAM-9719] Adds nose back to the
base image
add cc5cf30 [BEAM-9618] Mark push registration as deprecated.
add bf4cee8 Merge pull request #11356 from robertwb/proto-notes
add 84d5402 Ensure that empty messages are not flushed to handler.
add 9280e68 Merge pull request #11351 from scwhittle/fix_log
add 71e9d3c [BEAM-9721] Add --region to Dataflow-based load tests
add bb0c0ec [BEAM-9721] LoadTestConfig: handle --region parameter and put
default value if absent
add ca4e031 Merge pull request #11345: [BEAM-9721] Add --region to
Dataflow-based load tests
add 237a99f [BEAM-9550] Increase JVM Metaspace size for the
TaskExecutors. (#11193)
add 24108f9 [BEAM-9721]Conditionally add Dataflow region to
Dataflow-based performance tests (#11344)
add 00f4f2f [BEAM-9647] fixes MQTT clientId to long
add 2b8f4d9 Merge pull request #11361: [BEAM-9647] Fix MQTT clientId too
long
add ac003b6 [BEAM-8280] Document Python 3 annotations support (#11232)
add 5f432a2 [BEAM-9731] Include more detail in passert.Equals errors.
(#11359)
add 79b2d87 [BEAM-9085] Fix performance regression in SyntheticSource on
Python 3 (#11092)
add 7c46990 [BEAM-9726] [py] Make region optional for non-service
Dataflow.
add a32613f [BEAM-9726] [java] Make region optional for non-service
runner.
add c245288 Merge pull request #11353 from ibzib/beam-9726
add 80ebb70 [BEAM-4374] Fix missing deletion of metrics.
add 95a5944 Merge pull request #11363 from lukecwik/beam4374
add d36f873 [BEAM-9651] Prevent StreamPool and stream initialization
livelock
add 28b081f Merge pull request #11364: [BEAM-9651] Prevent StreamPool and
stream initialization livelock
add f251133 Revert "[BEAM-9651] Prevent StreamPool and stream
initialization livelock"
add e8fc585 Merge pull request #11367 from
scwhittle/revert-11364-contention
add cc42ab8 [BEAM-9727] Automatically set required experiment flags for
dataflow runner v2. (#11355)
add 742a8ce Allow unset write threshold for state backed iterable coder.
add 3d0cde3 Merge pull request #11366 State backed iterable fixes.
add 8db19a4 [BEAM-9562, BEAM-6274] Fix-up timers to use Elements.Timer
proto in data channel in Beam Java and Python
add 1de50c3 Merge pull request #11314 from boyuanzz/data
add 2498596 Update environments.py to add a method to specify container
image (#11370)
add eeb0715 add missing bracket
add 0695414 Merge pull request #11247 from chie8842/patch-1
add 5f574b8 Moving to 2.22.0-SNAPSHOT on master branch.
add 247090f [BEAM-9734] Revert #11122
add 6a4ef33 Merge pull request #11376: [BEAM-9734] Revert #11122
add 1b8ac22 [BEAM-9651] Prevent StreamPool and stream initialization
livelock
add 57ca62d Merge pull request #11368: [BEAM-9651] Prevent StreamPool and
stream initialization livelock
add 82abec7 Add --region to changelog
add 7fb0abf Merge pull request #11379 from apache/region-changelog
add 5208c3d Upgrades gcsio to 2.1.2
add 75433f0 Merge pull request #11315: [BEAM-8889] Upgrades gcsio to 2.1.2
add c26af84 Attempt to stage resources via new API in portable runner.
add d5929ba Merge pull request #11311 [BEAM-9577] Attempt to stage
resources via new API in portable runner.
add 81092b7 Fix from_container_image call
add c17e7c9 Merge pull request #11383 from [BEAM-9736] Fix
from_container_image call
add d4c283e [BEAM-9562] Update Element.timer, Element.Timer to
Element.timers and Element.Timers.
add 99fa8ac Merge pull request #11373 from boyuanzz/timers
add f0ba6bf [BEAM-9735] Adding Always trigger and using it in Reshuffle
add 7082c77 Merge pull request #11365 [BEAM-9735] Adding Always trigger
and using it in Reshuffle
add 2e0cf8f [BEAM-9443] support direct_num_workers=0 (#11372)
add d1c04cb TOIL: Update Unified worker image
add d1e706e Merge pull request #11385 from angoenka/update_um_image
add 8458f47 google-api-client 1.30.9
add 3f99b33 Merge remote-tracking branch 'origin/master' into BEAM-8751
add cf4657b Merge pull request #11208 from suztomo/BEAM-8751
add 9b2dc82 Updates Dataflow stateful DoFn setup to support external
transforms
add d566479 Merge pull request #11400: [BEAM-8019] Updates Dataflow
stateful DoFn setup to support external transforms
add 537c641 [BEAM-9738] Update dataflow to setup correct docker
environment options. (#11371)
add 296f5a7 [BEAM-9136]Add licenses for dependencies for Java (#11243)
add ad804f1 [BEAM-9496] Evaluation of deferred dataframes via Beam
operations.
add 2197ba1 Fix and test tuple inputs and outputs.
add b557c12 Comments and clarification.
add b6e5082 Merge pull request #10760 [BEAM-9545] Dataframe transforms
add 009578e [BEAM-9562] Fix output timestamp to be inferred from
scheduled time when in the event time domain.
add a7f444f Merge pull request #11402 from lukecwik/timers
add 7b454e9 [BEAM-9744] Add missing region option to py perf tests.
add 6eeed69 Merge pull request #11401 from ibzib/performance-region
add 7e07607 [BEAM-9744] Remove --region option from SQL tests.
add 40903f2 Merge pull request #11408 from ibzib/performance-region
add ec0e157 Merge pull request #11389 from Refactor the BCJ and capture
controls to be more testable
add 9336272 [BEAM-i9751] upgrade zetasql to 2020.04.1 (#11410)
add 0595f79 [BEAM-9650] Add PeriodicImpulse Transform and slowly changing
side input documentation (#11182)
add a57f3a0 Documentation bug fix for FlatMapElements#via()
SimpleFunction in the javadoc should return List<String> instead of Integer
add 55dec55 Merge pull request #11412: Documentation minor fix for
FlatMapElements#via()
add 1754d03 [BEAM-7923] Screendiff Integration Tests (#11338)
add f7647ad [BEAM-9642] Add SDF execution units. (#11327)
add f950b71 [BEAM-9136] reduce third_party_dependencies size (#11416)
add acb37bb [BEAM-9746] check for 0 length copies from state (#11413)
add 7f2ab69 [BEAM-9642] Fix infinite recursion.
add d279f4b Merge pull request #11423: [BEAM-9642] Fix infinite recursion.
add a39daa6 [BEAM-2939] Update unbounded source as SDF wrapper to resume
successfully.
add 27c2c75 fixup! Fix spotbugs warning
add 1774530 Merge pull request #11409 from lukecwik/splittabledofn
add 34326a2 [BEAM-9577] Fix test to create urls from paths which are
compatible with Windows.
add ef6df74 Merge pull request #11420 from lukecwik/beam9577
add 42d3d43 Removing underscore from _ReadFromBigQuery to make it
external. It remains experimental.
add b85cdbe Merge pull request #11422 from [BEAM-9763] Removing
underscore from _ReadFromBigQuery to make it external. It re…
add d310f98 [BEAM-9756] Nexmark: only use --region in Dataflow.
add 303b371 Merge pull request #11417 from ibzib/nexmark-region
add 744119c [BEAM-8889] add gRPC suport in GCS connector (behind an
experimental-flag) (#11381)
add d7e0d5d [BEAM-8466] Make strip_iterable more strict
add 14424cc Merge pull request #11240: [BEAM-8466] Make strip_iterable
more strict
add 5f5ae6f [BEAM-9119] Disable flaky test
add 26ee594 Merge pull request #11425: [BEAM-9119] Disable flaky
test_large_elements
add 5c35755 Fixing type names for BQ Avro Tools
add 1db7186 Merge pull request #11404 from Fixing type names for BQ Avro
Tools
add a562f1e Fix py37-lint
add 7438265 Maven compatible publish repository authentication via
settings.xml
add 1658d90 Merge pull request #11399: Interpolate publish credentials as
expected for Maven settings.xml
add da4a48d [BEAM-9729, BEAM-8486] Runner-side bundle registration
cleanup. (#11358)
add 0a5cb43 Add new release 2.20.0 to beam website (#11285)
add f451643 add 2.20.0 blog post
add 4ebcc59 add a known issue
add 3162ed2 fixup! update 2.20.0 date
add 1d7f640 add 2.20.0 blog post (#11298)
add 0b025e3 Merge pull request #11151 from [BEAM-9468] Hl7v2 io
add 4d7ed73 Fix typo
add c1da629 Merge pull request #11434: Fix typo in WriteToKafka doc string
add a3a9c2d [BEAM-8671] Migrate Load Tests to Python 3.7
add aab7afc Merge pull request #11375: [BEAM-8671] Migrate Load Tests to
Python 3.7
add b9de201 Remove outdated doc for ReadFromBigQuery transform
add 209180f Merge pull request #11424: Remove outdated doc for
ReadFromBigQuery transform
add 289cb66 Add GBK load tests script for spark structured streaming
runner
add 210658a Add CoGBK load tests script for spark structured streaming
runner
add 1709a6b Add Combine load tests script for spark structured streaming
runner
add 8a14cf0 Add Pardo load tests script for spark structured streaming
runner
add 6e8e566 Merge pull request #11135 from
echauchot/add_spark-ss-runner-to-GBK-load-tests
add c9d5e93 [BEAM-9765] Exclude module-info.class from vendored Calcite.
add 65c32af Merge pull request #11429 from ibzib/module-info
add 19fe4b5 [BEAM-9678] Create Go SDK introduction kata (#11340)
add 38df510 [BEAM-9769] Ensuring JSON is the default export format for BQ
sink
add c21a2a1 Merge pull request #11433 from [BEAM-9769] Ensuring JSON is
the default export format for BQ sink
add fc9b349 Merge pull request #11436: Document Beam Schemas (forked PR)
add c3bd485 [BEAM-9648]: DirectRunner should return null on timeout
(#11275)
add 7869455 [BEAM-5379] Ignore go.sum files in RAT checks
add 0c86530 Merge pull request #11441 from apache/lostluck-gosum-rat
add 5b41b1f Include user distritribution into metric-dedicated validate
runner test.
add 656869f update inline comment.
add 34031ad To allow only assert on committed metrics, bypassing the
attempted ones.
add 8dbdce4 Use distribution matcher for testing.
add c7cd259 Merge pull request #11319 from HuangLED/master
add 99444c6 [BEAM-5605, BEAM-2939] Add support for FnApiDoFnRunner to
handle split calls. (#11414)
add a1b79fd [BEAM-9674] Don't specify selected fields when fetching
BigQuery table size (#11292)
add c38999e [BEAM-9737] Don't use docker create option -u
add 481b1d3 Merge pull request #11386: [BEAM-9737] Fix website postcommit
add 892a0a4 [BEAM-9650] Cleanup documentation on side inputs patterns
(#11415)
add cb72032 [BEAM-9768] Gradle command for Python ULR.
add a69d8ec Merge pull request #11430 [BEAM-9768] Gradle command for
Python ULR.
add 1b4bf6a [BEAM-8646] Fix external environment on OS X as well. (#11431)
add b6acaef [BEAM-9743] Fix TFRecordCodec to try harder to read/write
(#11397)
add cae80ab Correct a typo in documentation
add c60b338 Merge pull request #11398: Correct a typo in documentation
add be57a61 Updates and clarifications on type safety. (#11449)
add 16d23bc [BEAM-9577] Small fixes to portable runner staging.
add d935b5f Merge pull request #11432 [BEAM-9577] Small fixes to portable
runner staging.
add 12c66a0 [BEAM-3836] Enable dynamic splitting/checkpointing within the
Java SDK harness. (#11448)
add fa4f418 [BEAM-8872] Support split at fraction for OffsetRangeTracker
(#11418)
add 494482f [BEAM-9147] Add @Experimental annotation to VideoIntelligence
related classes
add b8323c8 Add @Experimental to the package-info as well
add d2089bc Merge pull request #11463 from
apache/BEAM-9147-add-experimental-annotation
add 37bb50d [BEAM-9692]: Make CombineValues portable (#11335)
add dee5a0a Add release date for 2.20.0
add d55cab7 Merge pull request #11465 from apache/aaltay-patch-2
add 1e7d2ff Fix user distribution metric test by using less restricting
value.
add 1bae13b Apply yapf fix.
add a4e317a Replace all explicit validation.
add 5acd866 Merge pull request #11455 from HuangLED/testFix
add 30fa01f [BEAM-9773]: Update Dataflow Debug Capture to use Google API
client Jackson 2
add 1da7b98 Remove all Dataflow google-api-client-jackson(1) dependencies
add b99a3dd Missed cleanup
add 2d98f80 Merge pull request #11442 from stevekoonce/master
add 95db26d [BEAM-6860] Fix iobase non-global windows bug
add a7169f9 Merge pull request #11439: [BEAM-6860] Fix iobase non-global
windows bug
add b106e21 [BEAM-9788] Add RetainDockerContainers flag support to Go
SDK. (#11467)
add fbf1775 [BEAM-9775] Add Go support for SDF StandardRequirements.
(#11443)
add 6060d99 [BEAM-9496] Add to_dataframe and to_pcollection APIs. (#11264)
add 2a931a5 add display:block to datatable so that the columns can have
various lengths based on the contents in them.
add fc8d18a Merge pull request #11462 from KevinGG/master
add 37838f2 [BEAM-9569] Fix BeamSqlPojoExample logRecords method to not
require Row coder inference
add c7da1b5 Merge pull request #11460: [BEAM-9569] Fix BeamSqlPojoExample
logRecords method to not require Row coder inference
add 0988049 Renames run_test -> run_scenario so that nose test runner
does not think it's a test method.
add 033c1eb Merge pull request #11457 Rename test-framework-confusing
run_test.
add 1b47b2e [BEAM-6960] More sophisticated job server starting and
stopping.
add 5878373 Abort pipeline if docker containers die unexpectedly.
add 7181427 [BEAM-6960] Go ULR validates runner tests.
add 488232d lint
add de38f6d Merge pull request #11451 [BEAM-6960] Go ULR validates runner
tests.
add 46c8d2d [BEAM-9285] Add Postcommit ValidatesRunner CI Job for Flink
on Java 11
add b605c1c Merge pull request #11113: [BEAM-9285] Add Postcommit
ValidatesRunner CI Job for Flink on Java 11
add 44638c5 Revert "[BEAM-8131] Provide Kubernetes setup for Prometheus
(#9482)" (#11347)
add 1572d57 Merge pull request #11466 from [BEAM-9787] Clear error
message on UW + BQSource
add 1905dbd [BEAM-9791] Add precommit for dataflow runner v2
add 5c27e8b Merge pull request #11470 from angoenka/uw_precommit
add 561752b [BEAM-9662][BEAM-9663] Reject Null Limit and Offset
add 8fff6d0 Merge pull request #11471: [BEAM-9662][BEAM-9663] Reject Null
Limit and Offset
add 4a7f04c [BEAM-9764] multi threading & optional pulling (#11428)
add 8de324f [BEAM-9608] BundleManagers use BundleContextManager for
configuration
add abb7bcb [BEAM-9639] Saving side inputs after producer execution, not
before consumption.
add f7ae7f6 [BEAM-9639] Separate Stage and Bundle execution. Improve
typing annotations.
add cf821e5 [BEAM-9639][BEAM-9608] Addressing review comments.
add 1fe543e Merge pull request #11270 from [BEAM-9639][BEAM-9608]
Improvements for FnApiRunner
add af71782 Fix SplittableParDoNaiveBounded DoFnInvoker.
add c00dc4a Merge pull request #11475 from boyuanzz/fix
add 6e6ff61 [BEAM-9789] Fix lock error. Add test. (#11468)
add 6663da8 Revert "[BEAM-9014] CachingShuffleBatchReader use bytes to
limit cache size."
add 623c5ed Merge pull request #11483 from
apache/revert-10418-cachesizebytes
add fc37a04 Add a timeout to the TestStream GRPC and fix the Streaming
cache timeout
add 7acbff4 Merge pull request #11440 from [BEAM-9767] Add a timeout to
the TestStream GRPC and fix the Streaming cache timeout
add 880b062 [BEAM-7981] Correct ParDo(callable) test (#11446)
add d2f91a7 update release 2.20.0 blog post and added contributors name
add e13c1d6 Merge pull request #11476 from
mszb/updating_release_blogpost_2_20_0
add 781ff5b Log job id similar to Java and Go
add 75151e3 Merge pull request #11458 from apache/aaltay-patch-1
add 084ac14 [BEAM-6597, BEAM-6467] Add support for reporting monitoring
infos to the Java SDK harness. (#11487)
add b091db6 [BEAM-8831] Do not copy generated files for Python source copy
add 7ff0f5a Merge pull request #10230 [BEAM-8831] Exclude generated files
for Python source copy
add 6cc6360 Merge pull request #11489 [BEAM-9577] Ensure required
directories exist when staging.
add fe3241f [BEAM-9642] Enabling SDF exec runners and fixing small bug.
(#11474)
add 66e9040 [BEAM-9755] changed MAKE_ADD stack pop order according to
Python version
add 84a914b Merge pull request #11493: [BEAM-9755] changed MAKE_ADD stack
pop order according to Python version
add 48db942 [BEAM-9717] update future version to 0.18.2 for Python 3.8
(#11494)
add 76387d9 [BEAM-9577] Use new artifact API to stage go artifacts.
(#11490)
add 2f3edaf Force output names to be unicode-compatible on Python 2.
(#11485)
add f35cf08 Make license_script.sh tolerate directory already existing
(#11501)
add 8cfc8a8 [BEAM-9778] remove temp directory & add docker-pull-licenses
to release scripts (#11484)
add b3596b8 [BEAM-8603] Add Python SqlTransform (#10055)
add 5b4ca7b [BEAM-9794] Reduce state cells needed for BufferingDoFnRunner
add 7869d49 Merge pull request #11478: [BEAM-9794] Reduce state cells
needed for BufferingDoFnRunner
add ef267d9 [BEAM-9701] Increments fastavro version range upper bound to
0.24. (#11506)
add 93b3a0f [BEAM-2939] Expose HasProgress interface for restriction
trackers and use the progress value during splitting
add 7568b06 [BEAM-2939] Add the ability for SDK harness runners to
provide additional monitoring infos.
add 7ac97ea [BEAM-2939] Integrate progress reporting for splittable dofns.
add a02eb60 [BEAM-2939] Fold Sizes sub-interfaces into RestrictionTracker
add cfa6692 [BEAM-2939] Drop HasSize in favor of using
HasProgress#getProgress#getWorkRemaining as the default if the DoFn doesn't
override GetSize.
add 0291976 Merge pull request #11472 from lukecwik/splittabledofn
add f813a90 [BEAM-9577] New artifact staging and retrieval service for
Java.
add 9c8718e [BEAM-9577] Multi-threaded artifact staging service backend.
add 01af3aa Use futures, better error handling.
add ec67a93 Merge pull request #11342 [BEAM-9577] New artifact staging
and retrieval service for Java.
add 9867aab [BEAM-9812] Fixing bug causing pipelines requiring temp
tables to not work
add decccbe Merge pull request #11509 from [BEAM-9812] Fixing bug causing
pipelines requiring temp tables to not work
add 55b82fc [BEAM-9514] Ensure nullability passes through sum
add a905f69 Merge pull request #11488: [BEAM-9514] Ensure nullability
passes through aggregate
add 71ccd7a [BEAM-7405] Workaround for bad Docker config
add 5761e33 Merge pull request #11512: [BEAM-7405] Workaround for bad
Docker config on Jenkins nodes.
add 89958d8 [BEAM-8871] Support trySplit for ByteKeyRangeTracker
add 28fd597 Merge pull request #11454 from boyuanzz/restriction_tracker
add d8519dd [BEAM-9811] Nightly snapshot publish error
add e55a7b3 [BEAM-9775] Adding Go SDF example, adjusting GetProgress
signature. (#11499)
add d416fd1 [BEAM-9733] Always let ImpulseSourceFunction emit a final
watermark
add 643945a [BEAM-9733] Make up for timers set while processing the bundle
add 4797f31 Merge pull request #11362: [BEAM-9733] Improve watermark and
timer handling
add fd74d4d [BEAM-9813] Replace outputs with respect to the correct tag.
add 5262732 Merge pull request #11515: [BEAM-9813] Replace outputs with
respect to the correct tag.
add 5d350cb [BEAM-9761] Disable auto builds for tests that are currently
failing.
add 022fe5e Merge pull request #11519: [BEAM-9761] Disable auto builds
for tests that are currently failing.
add 6d29fb8 [BEAM-9811] Copy pom for all publishMavenJavaPublication tasks
add 1321104 add licenses and license types for zetasql packages (#11516)
add 24361d1 Added a batch example with covid tracking data for
interactive notebook. (#11469)
add 4a52fb4 [BEAM-9488] Ensure we pass through PCollection ids instead of
attempting to fix them up. (#11514)
add 3777b79 Fix invalid formatting specifier in verify
add 7fd4008 Merge pull request #11525: Fix invalid formatting specifier
in verify
add 5fab78a [BEAM-9820] Upgrade Flink 1.9.x to 1.9.3
add b8085ad Merge pull request #11526: [BEAM-9820] Upgrade Flink 1.9.x to
1.9.3
add 9ed5a92 [BEAM-9819] Relaxes the version range for httplib2 to
<0.16.0. (#11522)
add ae6ef78 [BEAM-9646] Add Google Cloud vision integration transform
add 1a33eef Fix spotbugs errors
add 86175c8 Add @Experimental annotation, make parent class
package-private
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6beafe0)
\
N -- N -- N refs/heads/BEAM-9646-cloud-vision (86175c8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 2 +-
.test-infra/dataproc/flink_cluster.sh | 1 -
.test-infra/dataproc/init-actions/flink.sh | 6 +-
.test-infra/jenkins/CommonTestProperties.groovy | 2 +-
.test-infra/jenkins/LoadTestConfig.groovy | 5 +-
.../jenkins/NexmarkBigqueryProperties.groovy | 1 -
.../jenkins/job_LoadTests_CoGBK_Java.groovy | 4 +
...s_CoGBK_Java_spark_structured_streaming.groovy} | 64 +-
.../job_LoadTests_Combine_Flink_Python.groovy | 6 +-
.../jenkins/job_LoadTests_Combine_Java.groovy | 3 +
...Combine_Java_spark_structured_streaming.groovy} | 60 +-
.../jenkins/job_LoadTests_Combine_Python.groovy | 5 +-
.../jenkins/job_LoadTests_GBK_Flink_Python.groovy | 10 +-
.test-infra/jenkins/job_LoadTests_GBK_Java.groovy | 7 +
.../jenkins/job_LoadTests_GBK_Java_Smoke.groovy | 1 +
...sts_GBK_Java_spark_structured_streaming.groovy} | 93 +-
.../jenkins/job_LoadTests_GBK_Python.groovy | 9 +-
.../job_LoadTests_GBK_Python_reiterate.groovy | 4 +-
.../job_LoadTests_ParDo_Flink_Python.groovy | 6 +-
.../jenkins/job_LoadTests_ParDo_Java.groovy | 4 +
...s_ParDo_Java_spark_structured_streaming.groovy} | 64 +-
.../jenkins/job_LoadTests_ParDo_Python.groovy | 6 +-
.../jenkins/job_LoadTests_ParDo_Python_37.groovy | 71 --
.../jenkins/job_LoadTests_Python_Smoke.groovy | 3 +-
.../job_LoadTests_coGBK_Flink_Python.groovy | 10 +-
.../jenkins/job_LoadTests_coGBK_Python.groovy | 6 +-
.../job_PerformanceTests_BigQueryIO_Python.groovy | 4 +-
.../jenkins/job_PerformanceTests_Python.groovy | 1 +
.../job_PostCommit_Java_Nexmark_Dataflow.groovy | 5 +
.../job_PostCommit_Java_Nexmark_Flink.groovy | 12 +-
...ob_PostCommit_Java_ValidatesRunner_Flink.groovy | 2 +-
...ommit_Java_ValidatesRunner_Flink_Java11.groovy} | 38 +-
.test-infra/metrics/README.md | 31 +-
.test-infra/metrics/apply_configmaps.sh | 26 -
.test-infra/metrics/beamprometheus-deploy.yaml | 125 --
.test-infra/metrics/build.gradle | 13 +
.test-infra/metrics/docker-compose.yml | 28 +-
.../alertmanager/config/alertmanager.yml | 41 -
.../prometheus/prometheus/config/prometheus.yml | 40 -
.../metrics/prometheus/prometheus/config/rules.yml | 35 -
CHANGES.md | 57 +-
build.gradle | 6 +
buildSrc/build.gradle | 1 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 75 +-
.../org/apache/beam/gradle/Repositories.groovy | 4 +
.../org/apache/beam/gradle/VendorJavaPlugin.groovy | 17 +-
gradle.properties | 4 +-
learning/katas/README.md | 2 +-
.../Introduction/Hello Beam/Hello Beam Test/go.mod | 47 +-
.../Introduction/Hello Beam/Hello Beam Test/go.sum | 58 +
.../Hello Beam/Hello Beam Test/pkg/task/task.go | 44 +-
.../Hello Beam/Hello Beam Test/task-info.yaml | 36 +
.../Hello Beam Test/task-remote-info.yaml | 2 +
.../Hello Beam/Hello Beam Test/task.md | 41 +
.../Hello Beam/Hello Beam Test/test/task_test.go | 49 +-
.../Introduction/Hello Beam/Hello Beam/cmd/main.go | 48 +-
.../go/Introduction/Hello Beam/Hello Beam/go.mod | 49 +-
.../go/Introduction/Hello Beam/Hello Beam/go.sum | 119 ++
.../Hello Beam/Hello Beam/pkg/task/task.go | 44 +-
.../Hello Beam/Hello Beam/task-info.yaml | 35 +
.../Hello Beam/Hello Beam/task-remote-info.yaml | 2 +
.../go/Introduction/Hello Beam/Hello Beam/task.md | 51 +
.../Hello Beam/Hello Beam/test/task_test.go | 49 +-
.../go/Introduction/Hello Beam/lesson-info.yaml | 22 +
.../Hello Beam/lesson-remote-info.yaml | 3 +
learning/katas/go/Introduction/section-info.yaml | 21 +
.../katas/go/Introduction/section-remote-info.yaml | 2 +
learning/katas/{ => go}/README.md | 17 +-
learning/katas/go/course-info.yaml | 26 +
learning/katas/go/course-remote-info.yaml | 2 +
.../fn-execution/src/main/proto/beam_fn_api.proto | 14 +-
.../pipeline/src/main/proto/beam_runner_api.proto | 76 +-
ownership/PYTHON_DEPENDENCY_OWNERS.yaml | 6 -
.../src/main/scripts/build_release_candidate.sh | 2 +-
.../core/construction/ArtifactResolver.java | 51 +
.../core/construction/DefaultArtifactResolver.java | 93 ++
.../runners/core/construction/Environments.java | 127 ++-
.../core/construction/ParDoTranslation.java | 90 +-
.../runners/core/construction/SplittableParDo.java | 6 -
.../construction/SplittableParDoNaiveBounded.java | 7 +-
.../beam/runners/core/construction/Timer.java | 33 +-
.../construction/DefaultArtifactResolverTest.java | 131 +++
.../core/construction/EnvironmentsTest.java | 7 +
.../core/construction/ParDoTranslationTest.java | 58 +-
...TimeBoundedSplittableProcessElementInvoker.java | 31 +-
.../apache/beam/runners/core/SimpleDoFnRunner.java | 27 +-
.../core/SplittableProcessElementInvoker.java | 9 +-
.../core/metrics/MonitoringInfoConstants.java | 2 +
.../runners/core/SplittableParDoProcessFnTest.java | 3 +-
.../direct/ExecutorServiceParallelExecutor.java | 36 +-
.../beam/runners/direct/DirectRunnerTest.java | 2 +-
runners/flink/1.9/build.gradle | 2 +-
runners/flink/flink_runner.gradle | 5 +-
.../runners/flink/FlinkExecutionEnvironments.java | 4 +
.../beam/runners/flink/FlinkPipelineOptions.java | 7 +
.../functions/FlinkExecutableStageFunction.java | 57 +-
.../functions/ImpulseSourceFunction.java | 12 +
.../flink/translation/utils/Workarounds.java | 25 +
.../wrappers/streaming/DoFnOperator.java | 271 +++--
.../streaming/ExecutableStageDoFnOperator.java | 324 ++++--
.../wrappers/streaming/SplittableDoFnOperator.java | 15 +-
.../wrappers/streaming/WindowDoFnOperator.java | 9 +-
.../streaming/stableinput/BufferingDoFnRunner.java | 97 +-
.../runners/flink/FlinkPipelineOptionsTest.java | 1 +
.../beam/runners/flink/FlinkSavepointTest.java | 34 +-
.../runners/flink/PortableTimersExecutionTest.java | 35 +-
.../FlinkExecutableStageFunctionTest.java | 14 +
.../functions/ImpulseSourceFunctionTest.java | 17 +-
.../wrappers/streaming/DoFnOperatorTest.java | 10 +-
.../streaming/ExecutableStageDoFnOperatorTest.java | 311 ++++-
.../wrappers/streaming/WindowDoFnOperatorTest.java | 4 +-
.../stableinput/BufferingDoFnRunnerTest.java | 171 +++
.../beam/runners/dataflow/DataflowRunner.java | 18 +-
.../dataflow/PrimitiveParDoSingleFactory.java | 38 +-
.../dataflow/DataflowPipelineTranslatorTest.java | 18 -
.../beam/runners/dataflow/DataflowRunnerTest.java | 25 +
.../google-cloud-dataflow-java/worker/build.gradle | 2 +-
.../worker/ApplianceShuffleEntryReader.java | 6 +-
.../dataflow/worker/BatchDataflowWorker.java | 9 +-
.../worker/BeamFnMapTaskExecutorFactory.java | 11 -
.../worker/ChunkingShuffleBatchReader.java | 8 +-
.../dataflow/worker/FnApiWindowMappingFn.java | 4 +-
.../dataflow/worker/StreamingDataflowWorker.java | 9 +-
.../fn/control/ProcessRemoteBundleOperation.java | 22 +-
.../dataflow/worker/fn/control/TimerReceiver.java | 268 -----
.../worker/fn/data/BeamFnDataGrpcService.java | 2 +-
.../fn/data/RemoteGrpcPortReadOperation.java | 2 +-
.../fn/data/RemoteGrpcPortWriteOperation.java | 2 +-
.../worker/graph/RegisterNodeFunction.java | 16 +-
.../JulHandlerPrintStreamAdapterFactory.java | 44 +-
.../dataflow/worker/status/DebugCapture.java | 9 +-
.../common/worker/CachingShuffleBatchReader.java | 61 +-
.../util/common/worker/ShuffleBatchReader.java | 5 +-
.../dataflow/worker/FnApiWindowMappingFnTest.java | 4 +
.../fn/control/BeamFnMapTaskExecutorTest.java | 4 +
.../RegisterAndProcessBundleOperationTest.java | 44 +-
.../worker/fn/control/TimerReceiverTest.java | 543 ---------
.../worker/fn/data/BeamFnDataGrpcServiceTest.java | 6 +-
.../fn/data/RemoteGrpcPortReadOperationTest.java | 8 +-
.../fn/data/RemoteGrpcPortWriteOperationTest.java | 6 +-
.../graph/LengthPrefixUnknownCodersTest.java | 2 +-
.../JulHandlerPrintStreamAdapterFactoryTest.java | 21 +
.../dataflow/worker/status/DebugCaptureTest.java | 2 +-
.../worker/BatchingShuffleEntryReaderTest.java | 18 +-
.../worker/CachingShuffleBatchReaderTest.java | 4 +-
.../artifact/ArtifactRetrievalService.java | 114 ++
.../artifact/ArtifactStagingService.java | 616 ++++++++++
.../control/DefaultJobBundleFactory.java | 44 +-
.../fnexecution/control/FnApiControlClient.java | 13 +-
.../control/InstructionRequestHandler.java | 2 +
.../fnexecution/control/OutputReceiverFactory.java | 2 +-
.../control/ProcessBundleDescriptors.java | 195 ++--
.../runners/fnexecution/control/RemoteBundle.java | 16 +
.../fnexecution/control/SdkHarnessClient.java | 271 +++--
.../SingleEnvironmentInstanceJobBundleFactory.java | 21 +-
.../fnexecution/control/StageBundleFactory.java | 10 +
.../fnexecution/control/TimerReceiverFactory.java | 91 +-
.../runners/fnexecution/data/GrpcDataService.java | 2 +-
.../translation/PipelineTranslatorUtils.java | 27 +-
.../artifact/ArtifactRetrievalServiceTest.java | 132 +++
.../artifact/ArtifactStagingServiceTest.java | 173 +++
.../control/FnApiControlClientTest.java | 19 -
.../fnexecution/control/RemoteExecutionTest.java | 710 ++++++++----
.../fnexecution/control/SdkHarnessClientTest.java | 99 +-
.../control/TimerReceiverFactoryTest.java} | 24 +-
.../fnexecution/data/GrpcDataServiceTest.java | 4 +-
.../beam/runners/portability/PortableRunner.java | 2 +
.../translation/SparkExecutableStageFunction.java | 72 +-
.../SparkExecutableStageFunctionTest.java | 20 +-
sdks/go/README.md | 2 +-
sdks/go/cmd/beamctl/cmd/artifact.go | 8 +-
sdks/go/cmd/beamctl/cmd/provision.go | 6 +-
.../stringsplit/offsetrange/offsetrange.go | 119 ++
sdks/go/examples/stringsplit/stringsplit.go | 246 ++++
sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go | 20 +-
sdks/go/pkg/beam/artifact/gcsproxy/staging.go | 20 +-
sdks/go/pkg/beam/artifact/materialize.go | 64 +-
sdks/go/pkg/beam/artifact/materialize_test.go | 81 +-
sdks/go/pkg/beam/artifact/server_test.go | 26 +-
sdks/go/pkg/beam/artifact/stage.go | 32 +-
sdks/go/pkg/beam/artifact/stage_test.go | 12 +-
sdks/go/pkg/beam/core/funcx/fn.go | 7 +-
sdks/go/pkg/beam/core/graph/fn.go | 21 +-
sdks/go/pkg/beam/core/graph/fn_test.go | 4 +-
sdks/go/pkg/beam/core/graph/mtime/time.go | 6 +-
sdks/go/pkg/beam/core/runtime/exec/coder.go | 4 +-
sdks/go/pkg/beam/core/runtime/exec/fn.go | 12 +-
sdks/go/pkg/beam/core/runtime/exec/pardo.go | 17 +-
sdks/go/pkg/beam/core/runtime/exec/reshuffle.go | 1 -
sdks/go/pkg/beam/core/runtime/exec/sdf.go | 315 +++++
.../beam/core/runtime/exec/sdf_invokers_test.go | 36 +-
sdks/go/pkg/beam/core/runtime/exec/sdf_test.go | 309 +++++
sdks/go/pkg/beam/core/runtime/exec/translate.go | 96 +-
sdks/go/pkg/beam/core/runtime/graphx/translate.go | 243 ++--
.../pkg/beam/core/runtime/graphx/translate_test.go | 54 +-
sdks/go/pkg/beam/core/runtime/harness/datamgr.go | 29 +-
.../pkg/beam/core/runtime/harness/datamgr_test.go | 14 +-
sdks/go/pkg/beam/core/runtime/harness/harness.go | 89 +-
.../pkg/beam/core/runtime/harness/harness_test.go | 165 +++
sdks/go/pkg/beam/core/runtime/harness/logging.go | 30 +-
.../pkg/beam/core/runtime/harness/logging_test.go | 6 +-
.../go/pkg/beam/core/runtime/harness/monitoring.go | 26 +-
sdks/go/pkg/beam/core/runtime/harness/session.go | 12 +-
sdks/go/pkg/beam/core/runtime/harness/statemgr.go | 58 +-
.../pkg/beam/core/runtime/harness/statemgr_test.go | 210 +++-
sdks/go/pkg/beam/core/runtime/options.go | 9 +-
sdks/go/pkg/beam/core/runtime/pipelinex/clone.go | 3 +-
sdks/go/pkg/beam/core/runtime/pipelinex/replace.go | 30 +-
.../beam/core/runtime/pipelinex/replace_test.go | 22 +-
sdks/go/pkg/beam/core/sdf/sdf.go | 7 +-
.../beam/model/fnexecution_v1/beam_fn_api.pb.go | 688 ++++++-----
.../beam/model/pipeline_v1/beam_runner_api.pb.go | 903 ++++++++-------
sdks/go/pkg/beam/options/jobopts/options.go | 4 +
sdks/go/pkg/beam/provision/provision.go | 20 +-
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 7 +-
.../pkg/beam/runners/dataflow/dataflowlib/job.go | 4 +-
.../beam/runners/dataflow/dataflowlib/translate.go | 26 +-
sdks/go/pkg/beam/runners/session/session.go | 30 +-
.../beam/runners/universal/runnerlib/execute.go | 4 +-
.../go/pkg/beam/runners/universal/runnerlib/job.go | 14 +-
.../pkg/beam/runners/universal/runnerlib/stage.go | 102 +-
sdks/go/pkg/beam/runners/universal/universal.go | 7 +-
sdks/go/pkg/beam/testing/passert/equals.go | 100 ++
sdks/go/pkg/beam/testing/passert/equals_test.go | 99 ++
sdks/go/pkg/beam/testing/passert/passert.go | 25 +-
sdks/go/pkg/beam/testing/passert/passert.shims.go | 28 +
sdks/go/test/build.gradle | 24 +
sdks/go/test/run_integration_tests.sh | 93 +-
sdks/java/bom/build.gradle | 4 +-
sdks/java/container/Dockerfile | 2 +
sdks/java/container/build.gradle | 36 +-
.../container/license_scripts/dep_urls_java.yaml | 203 ++++
.../container/license_scripts/license_script.sh | 53 +
.../license_scripts/pull_licenses_java.py | 323 ++++++
.../src/main/java/org/apache/beam/sdk/io/Read.java | 200 ++--
.../java/org/apache/beam/sdk/io/TFRecordIO.java | 55 +-
.../java/org/apache/beam/sdk/io/WriteFiles.java | 43 +-
.../org/apache/beam/sdk/io/range/ByteKeyRange.java | 2 +-
.../main/java/org/apache/beam/sdk/state/Timer.java | 22 +
.../java/org/apache/beam/sdk/transforms/DoFn.java | 27 +-
.../beam/sdk/transforms/FlatMapElements.java | 2 +-
.../reflect/ByteBuddyDoFnInvokerFactory.java | 13 +-
.../beam/sdk/transforms/reflect/DoFnInvoker.java | 12 +-
.../splittabledofn/ByteKeyRangeTracker.java | 98 +-
.../splittabledofn/OffsetRangeTracker.java | 35 +-
.../splittabledofn/RestrictionTracker.java | 76 +-
.../beam/sdk/transforms/splittabledofn/Sizes.java | 54 -
.../org/apache/beam/sdk/io/TFRecordIOTest.java | 130 +++
.../apache/beam/sdk/io/range/ByteKeyRangeTest.java | 7 +-
.../org/apache/beam/sdk/transforms/ParDoTest.java | 98 +-
.../splittabledofn/ByteKeyRangeTrackerTest.java | 132 ++-
.../splittabledofn/OffsetRangeTrackerTest.java | 80 +-
.../sdk/extensions/gcp/options/GcpOptions.java | 8 +-
.../beam/sdk/extensions/gcp/util/GcsUtil.java | 18 +-
sdks/java/extensions/ml/build.gradle | 2 -
.../beam/sdk/extensions/ml/AnnotateImages.java | 4 +-
.../beam/sdk/extensions/ml/AnnotateVideo.java | 105 ++
.../apache/beam/sdk/extensions/ml/CloudVision.java | 2 +
.../beam/sdk/extensions/ml/VideoIntelligence.java | 190 +++
.../beam/sdk/extensions/ml/package-info.java | 3 +
.../beam/sdk/extensions/ml/AnnotateVideoTest.java | 73 ++
.../sdk/extensions/ml/VideoIntelligenceIT.java | 83 ++
sdks/java/extensions/sql/build.gradle | 2 -
.../extensions/sql/expansion-service/build.gradle | 40 +
.../beam/sdk/extensions/sql/SqlTransform.java | 39 +-
.../extensions/sql/example/BeamSqlPojoExample.java | 8 +-
.../sdk/extensions/sql/example/model/Customer.java | 34 +-
.../sdk/extensions/sql/example/model/Order.java | 5 +
sdks/java/extensions/sql/zetasql/build.gradle | 2 +-
.../translation/AggregateScanConverter.java | 28 +-
.../LimitOffsetScanToLimitConverter.java | 6 +
.../LimitOffsetScanToOrderByLimitConverter.java | 5 +
.../sql/zetasql/ZetaSQLDialectSpecTest.java | 41 +
.../sdk/fn/data/BeamFnDataGrpcMultiplexer.java | 66 +-
.../sdk/fn/data/BeamFnDataInboundObserver.java | 35 +-
...amFnDataSizeBasedBufferingOutboundObserver.java | 38 +-
.../apache/beam/sdk/fn/data/LogicalEndpoint.java | 20 +-
.../sdk/fn/splittabledofn/RestrictionTrackers.java | 18 +-
.../sdk/fn/data/BeamFnDataGrpcMultiplexerTest.java | 49 +-
...DataSizeBasedBufferingOutboundObserverTest.java | 148 +--
...DataTimeBasedBufferingOutboundObserverTest.java | 84 +-
.../fn/splittabledofn/RestrictionTrackersTest.java | 14 +-
.../beam/fn/harness/BeamFnDataReadRunner.java | 7 +-
.../beam/fn/harness/BeamFnDataWriteRunner.java | 5 +-
.../beam/fn/harness/BoundedSourceRunner.java | 3 +
.../org/apache/beam/fn/harness/CombineRunners.java | 3 +
.../org/apache/beam/fn/harness/FlattenRunner.java | 3 +
.../apache/beam/fn/harness/FnApiDoFnRunner.java | 564 ++++++---
.../java/org/apache/beam/fn/harness/FnHarness.java | 38 +-
.../org/apache/beam/fn/harness/HandlesSplits.java | 8 +
.../org/apache/beam/fn/harness/MapFnRunners.java | 3 +
.../beam/fn/harness/PTransformRunnerFactory.java | 17 +
.../fn/harness/control/BeamFnControlClient.java | 19 +-
.../fn/harness/control/BundleSplitListener.java | 35 +-
.../fn/harness/control/ProcessBundleHandler.java | 231 ++--
.../beam/fn/harness/data/BeamFnDataGrpcClient.java | 12 +-
.../beam/fn/harness/data/BeamFnTimerClient.java | 52 +
.../fn/harness/data/BeamFnTimerGrpcClient.java | 99 ++
.../beam/fn/harness/state/FnApiStateAccessor.java | 22 +-
.../beam/fn/harness/AssignWindowsRunnerTest.java | 2 +
.../beam/fn/harness/BeamFnDataReadRunnerTest.java | 16 +-
.../beam/fn/harness/BeamFnDataWriteRunnerTest.java | 8 +-
.../beam/fn/harness/BoundedSourceRunnerTest.java | 2 +
.../apache/beam/fn/harness/CombineRunnersTest.java | 60 +-
.../apache/beam/fn/harness/FlattenRunnerTest.java | 4 +
.../beam/fn/harness/FnApiDoFnRunnerTest.java | 493 +++++---
.../apache/beam/fn/harness/MapFnRunnersTest.java | 6 +
.../harness/control/BundleSplitListenerTest.java | 57 +
.../harness/control/ProcessBundleHandlerTest.java | 110 +-
.../fn/harness/data/BeamFnDataGrpcClientTest.java | 4 +-
.../data/BeamFnDataInboundObserverTest.java | 22 +-
.../fn/harness/data/FakeBeamFnTimerClient.java | 115 ++
.../harness/data/QueueingBeamFnDataClientTest.java | 4 +-
sdks/java/io/google-cloud-platform/build.gradle | 1 +
.../sdk/io/gcp/bigquery/BigQueryAvroUtils.java | 37 +
.../io/gcp/bigquery/BigQueryStorageSourceBase.java | 7 +-
.../gcp/bigquery/BigQueryStorageTableSource.java | 12 +-
.../sdk/io/gcp/bigquery/DynamicDestinations.java | 6 +-
.../apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java | 685 +++++++++++
.../beam/sdk/io/gcp/healthcare/HL7v2Message.java | 213 ++++
.../sdk/io/gcp/healthcare/HL7v2MessageCoder.java | 65 ++
.../sdk/io/gcp/healthcare/HealthcareApiClient.java | 152 +++
.../sdk/io/gcp/healthcare/HealthcareIOError.java | 64 ++
.../io/gcp/healthcare/HealthcareIOErrorCoder.java | 59 +
.../healthcare/HealthcareIOErrorToTableRow.java | 91 ++
.../io/gcp/healthcare/HttpHealthcareApiClient.java | 460 ++++++++
.../beam/sdk/io/gcp/healthcare}/package-info.java | 13 +-
.../apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java | 1 +
.../gcp/bigquery/BigQueryIOStorageQueryTest.java | 67 +-
.../io/gcp/bigquery/BigQueryIOStorageReadTest.java | 183 ++-
.../io/gcp/healthcare/HL7V2MessagePagesTest.java | 95 ++
.../beam/sdk/io/gcp/healthcare/HL7v2IOReadIT.java | 196 ++++
.../beam/sdk/io/gcp/healthcare/HL7v2IOTest.java | 81 ++
.../sdk/io/gcp/healthcare/HL7v2IOTestUtil.java | 185 +++
.../beam/sdk/io/gcp/healthcare/HL7v2IOWriteIT.java | 93 ++
.../healthcare/HttpHealthcareApiClientTest.java | 56 +
.../java/org/apache/beam/sdk/io/mqtt/MqttIO.java | 5 +
sdks/python/apache_beam/coders/coders.py | 8 +-
sdks/python/apache_beam/dataframe/convert.py | 98 ++
sdks/python/apache_beam/dataframe/convert_test.py | 64 ++
sdks/python/apache_beam/dataframe/expressions.py | 23 +-
sdks/python/apache_beam/dataframe/frames.py | 6 +-
sdks/python/apache_beam/dataframe/transforms.py | 291 +++++
.../apache_beam/dataframe/transforms_test.py | 134 +++
.../examples/cookbook/datastore_wordcount.py | 131 +--
.../cookbook/datastore_wordcount_it_test.py | 19 +-
.../apache_beam/examples/snippets/snippets.py | 71 +-
.../apache_beam/examples/snippets/snippets_test.py | 85 +-
.../examples/snippets/snippets_test_py3.py | 21 +-
.../apache_beam/examples/wordcount_xlang_sql.py | 118 ++
sdks/python/apache_beam/io/external/kafka.py | 2 +-
.../io/gcp/big_query_query_to_table_pipeline.py | 2 +-
sdks/python/apache_beam/io/gcp/bigquery.py | 41 +-
.../apache_beam/io/gcp/bigquery_avro_tools.py | 1 +
.../apache_beam/io/gcp/bigquery_avro_tools_test.py | 7 +-
.../apache_beam/io/gcp/bigquery_file_loads.py | 6 +-
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 4 +-
.../io/gcp/bigquery_io_read_pipeline.py | 6 +-
.../apache_beam/io/gcp/bigquery_read_it_test.py | 4 +-
sdks/python/apache_beam/io/gcp/bigquery_test.py | 10 +-
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 6 +-
.../apache_beam/io/gcp/datastore/v1/datastoreio.py | 516 ---------
.../io/gcp/datastore/v1/datastoreio_test.py | 313 -----
.../io/gcp/datastore/v1/fake_datastore.py | 111 --
.../apache_beam/io/gcp/datastore/v1/helper.py | 335 ------
.../apache_beam/io/gcp/datastore/v1/helper_test.py | 296 -----
.../io/gcp/datastore/v1/query_splitter.py | 283 -----
.../io/gcp/datastore/v1/query_splitter_test.py | 244 ----
.../datastore/{v1 => v1new}/adaptive_throttler.py | 2 +-
.../{v1 => v1new}/adaptive_throttler_test.py | 2 +-
.../gcp/datastore/v1new/datastore_write_it_test.py | 3 +-
.../io/gcp/datastore/v1new/datastoreio.py | 16 +-
.../io/gcp/datastore/v1new/datastoreio_test.py | 60 +-
.../io/gcp/datastore/v1new/query_splitter_test.py | 102 +-
.../apache_beam/io/gcp/datastore/v1new/types.py | 20 +-
.../io/gcp/datastore/v1new/types_test.py | 3 +-
.../io/gcp/datastore/{v1 => v1new}/util.py | 0
.../io/gcp/datastore/{v1 => v1new}/util_test.py | 2 +-
.../io/gcp/datastore_write_it_pipeline.py | 213 ----
.../apache_beam/io/gcp/datastore_write_it_test.py | 86 --
.../hdfs_integration_test/hdfs_integration_test.sh | 5 +
sdks/python/apache_beam/io/iobase.py | 2 +-
sdks/python/apache_beam/io/localfilesystem.py | 3 +
sdks/python/apache_beam/io/textio_test.py | 23 +-
sdks/python/apache_beam/metrics/metric_test.py | 21 +
.../python/apache_beam/metrics/monitoring_infos.py | 60 +-
.../python/apache_beam/options/pipeline_options.py | 3 +-
.../options/pipeline_options_validator.py | 7 +-
.../options/pipeline_options_validator_test.py | 27 +
sdks/python/apache_beam/pipeline.py | 57 +-
sdks/python/apache_beam/runners/common.py | 10 +-
.../runners/dataflow/dataflow_runner.py | 65 +-
.../runners/dataflow/dataflow_runner_test.py | 30 +-
.../runners/dataflow/internal/apiclient.py | 21 +-
.../runners/dataflow/internal/apiclient_test.py | 2 +-
.../runners/dataflow/ptransform_overrides.py | 35 +
.../apache_beam/runners/direct/direct_userstate.py | 2 +-
.../apache_beam/runners/direct/test_stream_impl.py | 20 +-
.../runners/interactive/background_caching_job.py | 58 +-
.../interactive/background_caching_job_test.py | 4 +-
.../runners/interactive/caching/streaming_cache.py | 11 +-
.../interactive/display/pcoll_visualization.py | 2 +-
.../interactive/examples/UsCovidDataExample.ipynb | 511 +++++++++
.../runners/interactive/interactive_runner_test.py | 25 +-
.../runners/interactive/options/capture_control.py | 20 +-
.../interactive/options/capture_control_test.py | 68 +-
.../interactive/options/capture_limiters.py | 70 ++
.../interactive/testing/integration}/__init__.py | 0
.../goldens/7a35f487b2a5f3a9b9852a8659eeb4bd.png | Bin 0 -> 754885 bytes
.../testing/integration/notebook_executor.py | 141 +++
.../interactive/testing/integration/screen_diff.py | 227 ++++
.../test_notebooks/init_square_cube.ipynb | 178 +++
.../integration/tests/init_square_cube_test.py} | 17 +
.../runners/portability/artifact_service.py | 36 +-
.../runners/portability/artifact_service_test.py | 7 +-
.../runners/portability/fn_api_runner/execution.py | 219 +++-
.../runners/portability/fn_api_runner/fn_runner.py | 512 ++++-----
.../portability/fn_api_runner/fn_runner_test.py | 25 +
.../portability/fn_api_runner/translations.py | 85 +-
.../portability/fn_api_runner/worker_handlers.py | 63 +-
.../runners/portability/local_job_service.py | 39 +-
.../runners/portability/local_job_service_main.py | 113 +-
.../runners/portability/portable_runner.py | 65 +-
.../apache_beam/runners/worker/bundle_processor.py | 272 ++---
.../apache_beam/runners/worker/data_plane.py | 201 +++-
.../apache_beam/runners/worker/operations.pxd | 4 +-
.../apache_beam/runners/worker/operations.py | 126 +-
.../apache_beam/runners/worker/sdk_worker.py | 1 -
.../apache_beam/runners/worker/worker_pool_main.py | 6 +-
.../testing/benchmarks/chicago_taxi/preprocess.py | 2 +-
.../benchmarks/chicago_taxi/process_tfma.py | 2 +-
.../chicago_taxi/tfdv_analyze_and_validate.py | 2 +-
.../testing/load_tests/co_group_by_key_test.py | 2 +
.../apache_beam/testing/load_tests/combine_test.py | 2 +
.../testing/load_tests/group_by_key_test.py | 2 +
.../apache_beam/testing/load_tests/pardo_test.py | 2 +
.../apache_beam/testing/synthetic_pipeline.py | 48 +-
sdks/python/apache_beam/transforms/core.py | 47 +-
sdks/python/apache_beam/transforms/environments.py | 11 +-
sdks/python/apache_beam/transforms/external.py | 2 +-
.../apache_beam/transforms/periodicsequence.py | 192 ++++
.../transforms/periodicsequence_test.py | 102 ++
sdks/python/apache_beam/transforms/ptransform.py | 24 +-
sdks/python/apache_beam/transforms/sql.py | 74 ++
sdks/python/apache_beam/transforms/sql_test.py | 131 +++
sdks/python/apache_beam/transforms/trigger.py | 45 +-
sdks/python/apache_beam/transforms/trigger_test.py | 34 +
sdks/python/apache_beam/transforms/userstate.py | 12 +-
sdks/python/apache_beam/transforms/util.py | 5 +-
sdks/python/apache_beam/typehints/opcodes.py | 10 +-
.../apache_beam/typehints/typed_pipeline_test.py | 5 +-
.../typehints/typed_pipeline_test_py3.py | 17 +-
sdks/python/apache_beam/utils/annotations_test.py | 6 +
sdks/python/apache_beam/utils/subprocess_server.py | 2 +-
sdks/python/apache_beam/version.py | 2 +-
sdks/python/build-requirements.txt | 2 +-
sdks/python/build.gradle | 12 +
sdks/python/container/base_image_requirements.txt | 3 +-
.../container/license_scripts/dep_urls_py.yaml | 2 +
sdks/python/scripts/generate_pydoc.sh | 5 +-
sdks/python/scripts/run_integration_test.sh | 2 -
sdks/python/setup.py | 24 +-
sdks/python/test-suites/dataflow/py2/build.gradle | 87 +-
sdks/python/test-suites/dataflow/py37/build.gradle | 87 +-
sdks/python/tox.ini | 6 -
settings.gradle | 2 +
vendor/calcite-1_20_0/build.gradle | 3 +-
website/_config.yml | 2 +-
website/build.gradle | 10 +-
website/src/.htaccess | 2 +-
website/src/_data/authors.yml | 4 +
.../src/_includes/flink_java_pipeline_options.html | 5 +
.../_includes/flink_python_pipeline_options.html | 5 +
.../src/_includes/section-menu/documentation.html | 12 +
website/src/_posts/2020-04-15-beam-2.20.0.md | 77 ++
website/src/contribute/release-guide.md | 2 +-
website/src/documentation/io/built-in-hcatalog.md | 2 +-
website/src/documentation/patterns/side-inputs.md | 45 +-
.../pipelines/create-your-pipeline.md | 2 +-
website/src/documentation/programming-guide.md | 1205 +++++++++++++++++++-
website/src/documentation/runtime/environments.md | 57 +-
.../src/documentation/sdks/python-type-safety.md | 122 +-
website/src/get-started/downloads.md | 13 +-
website/src/get-started/quickstart-go.md | 1 +
website/src/get-started/wordcount-example.md | 2 +
485 files changed, 20820 insertions(+), 9350 deletions(-)
copy .test-infra/jenkins/{job_LoadTests_CoGBK_Java.groovy =>
job_LoadTests_CoGBK_Java_spark_structured_streaming.groovy} (78%)
copy .test-infra/jenkins/{job_LoadTests_Combine_Java.groovy =>
job_LoadTests_Combine_Java_spark_structured_streaming.groovy} (71%)
copy .test-infra/jenkins/{job_LoadTests_GBK_Java.groovy =>
job_LoadTests_GBK_Java_spark_structured_streaming.groovy} (77%)
copy .test-infra/jenkins/{job_LoadTests_ParDo_Java.groovy =>
job_LoadTests_ParDo_Java_spark_structured_streaming.groovy} (72%)
delete mode 100644 .test-infra/jenkins/job_LoadTests_ParDo_Python_37.groovy
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_Flink.groovy =>
job_PostCommit_Java_ValidatesRunner_Flink_Java11.groovy} (53%)
delete mode 100755 .test-infra/metrics/apply_configmaps.sh
delete mode 100644 .test-infra/metrics/beamprometheus-deploy.yaml
delete mode 100644
.test-infra/metrics/prometheus/alertmanager/config/alertmanager.yml
delete mode 100644
.test-infra/metrics/prometheus/prometheus/config/prometheus.yml
delete mode 100644 .test-infra/metrics/prometheus/prometheus/config/rules.yml
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam Test/go.mod (50%)
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam
Test/go.sum
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam Test/pkg/task/task.go (51%)
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam
Test/task-info.yaml
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam
Test/task-remote-info.yaml
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam
Test/task.md
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam Test/test/task_test.go
(52%)
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam/cmd/main.go (52%)
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam/go.mod (50%)
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam/go.sum
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go (51%)
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello
Beam/task-info.yaml
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello
Beam/task-remote-info.yaml
create mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam/task.md
copy sdks/go/cmd/beamctl/cmd/provision.go =>
learning/katas/go/Introduction/Hello Beam/Hello Beam/test/task_test.go (52%)
create mode 100644 learning/katas/go/Introduction/Hello Beam/lesson-info.yaml
create mode 100644 learning/katas/go/Introduction/Hello
Beam/lesson-remote-info.yaml
create mode 100644 learning/katas/go/Introduction/section-info.yaml
create mode 100644 learning/katas/go/Introduction/section-remote-info.yaml
copy learning/katas/{ => go}/README.md (60%)
create mode 100644 learning/katas/go/course-info.yaml
create mode 100644 learning/katas/go/course-remote-info.yaml
create mode 100644
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ArtifactResolver.java
create mode 100644
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/DefaultArtifactResolver.java
create mode 100644
runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/DefaultArtifactResolverTest.java
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunnerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/TimerReceiver.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/TimerReceiverTest.java
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/ArtifactRetrievalService.java
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/ArtifactStagingService.java
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/artifact/ArtifactRetrievalServiceTest.java
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/artifact/ArtifactStagingServiceTest.java
copy
runners/java-fn-execution/src/{main/java/org/apache/beam/runners/fnexecution/control/InstructionRequestHandler.java
=>
test/java/org/apache/beam/runners/fnexecution/control/TimerReceiverFactoryTest.java}
(61%)
create mode 100644 sdks/go/examples/stringsplit/offsetrange/offsetrange.go
create mode 100644 sdks/go/examples/stringsplit/stringsplit.go
create mode 100644 sdks/go/pkg/beam/core/runtime/exec/sdf.go
create mode 100644 sdks/go/pkg/beam/core/runtime/exec/sdf_test.go
create mode 100644 sdks/go/pkg/beam/core/runtime/harness/harness_test.go
create mode 100644 sdks/go/pkg/beam/testing/passert/equals.go
create mode 100644 sdks/go/pkg/beam/testing/passert/equals_test.go
create mode 100644 sdks/java/container/license_scripts/dep_urls_java.yaml
create mode 100755 sdks/java/container/license_scripts/license_script.sh
create mode 100644 sdks/java/container/license_scripts/pull_licenses_java.py
delete mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/Sizes.java
create mode 100644
sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/AnnotateVideo.java
create mode 100644
sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/VideoIntelligence.java
create mode 100644
sdks/java/extensions/ml/src/test/java/org/apache/beam/sdk/extensions/ml/AnnotateVideoTest.java
create mode 100644
sdks/java/extensions/ml/src/test/java/org/apache/beam/sdk/extensions/ml/VideoIntelligenceIT.java
create mode 100644 sdks/java/extensions/sql/expansion-service/build.gradle
create mode 100644
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/data/BeamFnTimerClient.java
create mode 100644
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/data/BeamFnTimerGrpcClient.java
create mode 100644
sdks/java/harness/src/test/java/org/apache/beam/fn/harness/control/BundleSplitListenerTest.java
create mode 100644
sdks/java/harness/src/test/java/org/apache/beam/fn/harness/data/FakeBeamFnTimerClient.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2Message.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2MessageCoder.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareApiClient.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareIOError.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareIOErrorCoder.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareIOErrorToTableRow.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HttpHealthcareApiClient.java
copy sdks/java/{extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml
=>
io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare}/package-info.java
(66%)
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7V2MessagePagesTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IOReadIT.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IOTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IOTestUtil.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IOWriteIT.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HttpHealthcareApiClientTest.java
create mode 100644 sdks/python/apache_beam/dataframe/convert.py
create mode 100644 sdks/python/apache_beam/dataframe/convert_test.py
create mode 100644 sdks/python/apache_beam/dataframe/transforms.py
create mode 100644 sdks/python/apache_beam/dataframe/transforms_test.py
create mode 100644 sdks/python/apache_beam/examples/wordcount_xlang_sql.py
delete mode 100644 sdks/python/apache_beam/io/gcp/datastore/v1/datastoreio.py
delete mode 100644
sdks/python/apache_beam/io/gcp/datastore/v1/datastoreio_test.py
delete mode 100644
sdks/python/apache_beam/io/gcp/datastore/v1/fake_datastore.py
delete mode 100644 sdks/python/apache_beam/io/gcp/datastore/v1/helper.py
delete mode 100644 sdks/python/apache_beam/io/gcp/datastore/v1/helper_test.py
delete mode 100644
sdks/python/apache_beam/io/gcp/datastore/v1/query_splitter.py
delete mode 100644
sdks/python/apache_beam/io/gcp/datastore/v1/query_splitter_test.py
rename sdks/python/apache_beam/io/gcp/datastore/{v1 =>
v1new}/adaptive_throttler.py (98%)
rename sdks/python/apache_beam/io/gcp/datastore/{v1 =>
v1new}/adaptive_throttler_test.py (97%)
rename sdks/python/apache_beam/io/gcp/datastore/{v1 => v1new}/util.py (100%)
rename sdks/python/apache_beam/io/gcp/datastore/{v1 => v1new}/util_test.py
(98%)
delete mode 100644
sdks/python/apache_beam/io/gcp/datastore_write_it_pipeline.py
delete mode 100644 sdks/python/apache_beam/io/gcp/datastore_write_it_test.py
create mode 100644
sdks/python/apache_beam/runners/interactive/examples/UsCovidDataExample.ipynb
create mode 100644
sdks/python/apache_beam/runners/interactive/options/capture_limiters.py
copy sdks/python/apache_beam/{testing/load_tests/streaming =>
runners/interactive/testing/integration}/__init__.py (100%)
create mode 100644
sdks/python/apache_beam/runners/interactive/testing/integration/goldens/7a35f487b2a5f3a9b9852a8659eeb4bd.png
create mode 100644
sdks/python/apache_beam/runners/interactive/testing/integration/notebook_executor.py
create mode 100644
sdks/python/apache_beam/runners/interactive/testing/integration/screen_diff.py
create mode 100644
sdks/python/apache_beam/runners/interactive/testing/integration/test_notebooks/init_square_cube.ipynb
rename sdks/python/apache_beam/{io/gcp/datastore/v1/__init__.py =>
runners/interactive/testing/integration/tests/init_square_cube_test.py} (69%)
create mode 100644 sdks/python/apache_beam/transforms/periodicsequence.py
create mode 100644 sdks/python/apache_beam/transforms/periodicsequence_test.py
create mode 100644 sdks/python/apache_beam/transforms/sql.py
create mode 100644 sdks/python/apache_beam/transforms/sql_test.py
create mode 100644 website/src/_posts/2020-04-15-beam-2.20.0.md
