This is an automated email from the ASF dual-hosted git repository. mwalenia pushed a change to branch BEAM-9646-cloud-vision in repository https://gitbox.apache.org/repos/asf/beam.git.
discard aa91ce7 Added changes to CHANGES.md discard 6f62ebd Add customizable number of keys to the transform discard 7b9a777 Set the same key for each element so that grouping is done correctly by existing transforms discard 20d82dd Fixes after code review discard bb70720 Add dependency to build.gradle discard edad9c3 Add @Experimental annotation, make parent class package-private discard f829148 Fix spotbugs errors discard 3911bba [BEAM-9646] Add Google Cloud vision integration transform add f387df7 add io-matrix add 8fe35ad add descriptions, links to guides add ff1f9f1 fix typo add fa805b6 format add 37ac9ee update CHANGES.md add 6766d0c [BEAM-10058] Provide less strict assertion to make the test more resistant against future changes in a model add 57dc7f4 Merge pull request #11795: [BEAM-10058] Provide less strict assertion to make the test more resistant against future changes in a model add 0f97499 [BEAM-9930] Update blog post for new Beam Summit Digital dates (#11772) add 79dcab9 [BEAM-10073] Add pubsub performance tests results to the grafana dashboards (#11809) add 9f8783d add def args add 18de412 Merge pull request #11786 from TheNeuralBit/mobile-gaming-fix add f83bc75 [BEAM-10065] Fix beam release guide template (#11797) add f452e09 Fix typo in doc "and and" -> "and" (#11810) add d86219f [BEAM-2939, BEAM-10057] Ensure that we can process an EmptyUnboundedSource and also prevent splitting on it. (#11781) add 8f53b99 Fix typo Portabillity -> Portability add 17ce646 Merge pull request #11789 from epicfaace/patch-2 add c2d6c36 [BEAM-9603] Add timer family support to FnApiDoFnRunner add 5839915 Address comments and add tests add d3dfa30 comments add b1791a2 Merge pull request #11756 from y1chi/BEAM-9603 add 90a79d6 [BEAM-7370] Upgrade sphinx to 3.0.3 (#11798) add 790ed44 Fix typo copyLicenseScrips -> copyLicenseScripts add 6e42344 Merge pull request #11800: Fix typo copyLicenseScrips -> copyLicenseScripts add fdd9409 Merge branch 'master' into bi add e1531b7 Merge pull request #11802 from [BEAM-9916] Update I/O documentation links and create more complete I/O matrix add c77bee6 Clear redundant debug statements in pr-11756 add 32c6579 Merge pull request #11822 from y1chi/clean-up add 6022079 [BEAM-10076] Fix dataflow worker status page rendering (#11812) add 27f0dba [BEAM-9770] BigQueryIO Patterns. Add Document update to snippit update. add 1a5dd77 Merge pull request #11815 from rezarokni/BEAM-9770 add 630a488 [BEAM-10016, BEAM-10094] Disable single test to get portable validates runner signal back to green add e6bbb40 Merge pull request #11825: [BEAM-10016, BEAM-10094] Disable single test to get portable validates runner signal back to green add 63b4d3a [BEAM-7770] Make ReplicaInfo part of Read on SolrIO add 61eebf0 [BEAM-7770] Add ReadAll transform for SolrIO add 5d88f77 Merge pull request #11357: [BEAM-7770] Add ReadAll transform for SolrIO add 9199598 [BEAM-10102] Fix query in pubsub read grafana dashoard add d0d813b Merge pull request #11827: [BEAM-10102] Fix query in pubsub read grafana dashoard add 1d84d00 [BEAM-10077] using filename + hash instead of UUID for staging name add 8aedab2 Merge pull request #11813: [BEAM-10077] using filename + hash instead of UUID for staging name add 5b6e84f Add a flag to tune the size of the grouping tables used by the dataflow workers (#11811) add 292dea3 [BEAM-9936] Reduce the code duplication across python/container/py* add 09a1677 [BEAM-9936] Create SDK harness containers with Python3.8 add a23b0e6 [BEAM-9936] Create SDK harness containers with Python 3.8 - PR #11738 add 1ec86e7 Add Pub/Sub Lite to in progress IOs add 6906192 Merge pull request #11833 from apache/aaltay-patch-1 add ec7278b [BEAM-10108] Update Flink versions in publish_docker_images.sh. add c27e033 Merge pull request #11829 from ibzib/BEAM-10108 add fa028e6 [BEAM-10074] | implement hashing functions add 706a06c Merge pull request #11817 from darshanj/BEAM-10074 add 64f8a40 [BEAM-10064] Fix google3 import error for BEAM-9383 add 1671c1f Merge pull request #11793 [BEAM-10064] Fix google3 import error for BEAM-9383 add eb59a84 Merge pull request #11796 from [BEAM-10003] Use local code for building code samples on website add 6cf105a Add to/from_runner_api_parameters to WriteToBigQuery add 6658d62 Merge pull request #11745 from [BEAM-9692] Add to/from_runner_api_parameters to WriteToBigQuery add 4817e84 [BEAM-9971] Do not use context classloader. (#11784) add ad15754 Add blog post announcing the Beam 2.21.0 release. add ff8430e Add release date. add c5b1183 Add release date. add e0f858f Remove HTML comment (causes formatting issues) add a0b5fb1 Merge pull request #11729 from ibzib/2-21-blog add 301c32b Update Beam website to release 2.21.0. add 9939a8f Move 2.20 download addresses to archive. add 5129587 Add release date. add e52f58d Merge branch 'website-2-21' of https://github.com/ibzib/beam into website-2-21 add 3850133 Merge pull request #11727 from ibzib/website-2-21 add 6654a3d [BEAM-9810] Tox suites for Python 3.8 add c0c786d [BEAM-9810] Fix 'dictionary keys changed during iteration' error add c2b9867 [BEAM-9810] Disable all failing tests until Dataflow runner supports Python 3.8 add c4f03a8 [BEAM-9810] Add a Tox (precommit) suite for Python 3.8 #11707 add 0588260 [BEAM-9363] Support TUMBLE aggregation (#11807) add 00736f3 [BEAM-10098] Enabling javadoc export for RabbitMqIO and KuduIO add a13ef24 Merge pull request #11837: [BEAM-10098] Enabling javadoc export for RabbitMqIO and KuduIO add 6ca6372 [BEAM-9646] Add Google Cloud vision integration transform add 6f67f71 Fix spotbugs errors add 16c9bdd Add @Experimental annotation, make parent class package-private add 1b1694b Add dependency to build.gradle add 23cf44d Fixes after code review add 372e385 Set the same key for each element so that grouping is done correctly by existing transforms add ef78421 Add customizable number of keys to the transform add 747ed7c Added changes to CHANGES.md add cd7d3de Update README.md This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (aa91ce7) \ N -- N -- N refs/heads/BEAM-9646-cloud-vision (cd7d3de) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../job_PerformanceTests_PubsubIO_Python.groovy | 5 +- ...ommit_Python_ValidatesContainer_Dataflow.groovy | 2 + .test-infra/metrics/beamgrafana-deploy.yaml | 2 +- .../Python_IO_IT_Tests_Dataflow.json | 256 +++++++++++++- CHANGES.md | 9 +- build.gradle | 4 +- .../org/apache/beam/gradle/BeamModulePlugin.groovy | 7 + .../src/main/groovy/MobileGamingCommands.groovy | 4 +- release/src/main/scripts/publish_docker_images.sh | 2 +- .../runners/core/construction/Environments.java | 23 +- .../core/construction/NativeTransforms.java | 5 +- runners/flink/job-server/flink_job_server.gradle | 4 + .../beam/runners/dataflow/DataflowRunner.java | 12 +- .../beam/runners/dataflow/util/PackageUtil.java | 25 +- .../beam/runners/dataflow/util/GCSUploadMain.java | 7 +- .../runners/dataflow/util/PackageUtilTest.java | 24 +- .../dataflow/worker/PartialGroupByKeyParDoFns.java | 10 +- .../dataflow/worker/StreamingDataflowWorker.java | 4 +- .../worker/util/common/worker/GroupingTables.java | 10 +- runners/spark/job-server/build.gradle | 4 + .../src/main/java/org/apache/beam/sdk/io/Read.java | 12 +- .../apache/beam/sdk/options/SdkHarnessOptions.java | 14 + .../beam/sdk/transforms/reflect/DoFnInvoker.java | 8 +- .../sql/impl/rel/BeamTableFunctionScanRel.java | 37 +- .../sql/impl/udf/BuiltinHashFunctions.java | 139 ++++++++ .../beam/sdk/extensions/sql/BeamSqlDslBase.java | 6 + .../udf/BeamSalUhfSpecialTypeAndValueTest.java | 69 ++++ .../sql/impl/udf/BeamSqlUdfExpressionTest.java | 41 +++ .../sql/zetasql/ZetaSQLDialectSpecTest.java | 25 ++ .../apache/beam/fn/harness/FnApiDoFnRunner.java | 131 +++++-- .../beam/fn/harness/PrecombineGroupingTable.java | 13 +- .../beam/fn/harness/FnApiDoFnRunnerTest.java | 264 +++++++++++---- sdks/java/io/kudu/build.gradle | 2 +- sdks/java/io/rabbitmq/build.gradle | 2 +- .../java/org/apache/beam/sdk/io/solr/SolrIO.java | 68 ++-- .../org/apache/beam/sdk/io/solr/SolrIOTest.java | 17 + sdks/python/apache_beam/io/gcp/bigquery.py | 72 ++++ sdks/python/apache_beam/io/gcp/bigquery_test.py | 67 ++++ .../python/apache_beam/io/hadoopfilesystem_test.py | 15 +- sdks/python/apache_beam/io/textio.py | 2 +- .../ml/gcp/videointelligenceml_test_it.py | 6 +- .../runners/dataflow/dataflow_runner_test.py | 57 ++++ .../runners/dataflow/internal/apiclient_test.py | 69 +++- .../runners/dataflow/template_runner_test.py | 3 + sdks/python/container/Dockerfile | 2 +- sdks/python/container/base_image_requirements.txt | 12 +- sdks/python/container/build.gradle | 1 + .../container/{py37/build.gradle => common.gradle} | 16 +- .../container/license_scripts/dep_urls_py.yaml | 4 +- sdks/python/container/py2/build.gradle | 53 +-- sdks/python/container/py35/build.gradle | 53 +-- sdks/python/container/py36/build.gradle | 53 +-- sdks/python/container/py37/build.gradle | 53 +-- .../python/container/py38}/build.gradle | 11 +- sdks/python/container/run_validatescontainer.sh | 20 +- sdks/python/test-suites/tox/py37/build.gradle | 3 - .../test-suites/tox/{py35 => py38}/build.gradle | 9 +- sdks/python/test-suites/tox/pycommon/build.gradle | 2 +- sdks/python/tox.ini | 29 +- settings.gradle | 2 + website/.gitignore | 2 +- website/CONTRIBUTE.md | 10 +- website/build.gradle | 10 +- ...ild_github_samples.sh => build_code_samples.sh} | 12 +- website/www/package.json | 2 +- website/www/site/config.toml | 2 +- website/www/site/content/en/blog/beam-2.21.0.md | 97 ++++++ .../content/en/blog/beam-summit-digital-2020.md | 10 +- .../site/content/en/contribute/release-guide.md | 60 ++-- .../site/content/en/documentation/io/built-in.md | 86 +---- .../documentation/io/built-in/google-bigquery.md | 50 +-- .../en/documentation/io/developing-io-python.md | 12 +- .../en/documentation/patterns/bigqueryio.md | 46 +++ .../en/documentation/patterns/custom-windows.md | 12 +- .../en/documentation/patterns/file-processing.md | 10 +- .../content/en/documentation/patterns/overview.md | 3 + .../en/documentation/patterns/pipeline-options.md | 6 +- .../en/documentation/patterns/side-inputs.md | 6 +- .../content/en/documentation/programming-guide.md | 94 ++--- .../en/documentation/sdks/python-type-safety.md | 22 +- .../transforms/python/elementwise/filter.md | 24 +- .../transforms/python/elementwise/flatmap.md | 36 +- .../transforms/python/elementwise/keys.md | 4 +- .../transforms/python/elementwise/kvswap.md | 4 +- .../transforms/python/elementwise/map.md | 32 +- .../transforms/python/elementwise/pardo.md | 12 +- .../transforms/python/elementwise/partition.md | 12 +- .../transforms/python/elementwise/regex.md | 36 +- .../transforms/python/elementwise/tostring.md | 12 +- .../transforms/python/elementwise/values.md | 4 +- .../python/elementwise/withtimestamps.md | 16 +- .../www/site/content/en/get-started/downloads.md | 14 +- .../en/get-started/mobile-gaming-example.md | 44 +-- .../content/en/get-started/wordcount-example.md | 24 +- website/www/site/content/en/roadmap/go-sdk.md | 2 +- website/www/site/data/io_matrix.yaml | 377 +++++++++++++++++++++ .../partials/section-menu/en/documentation.html | 13 + .../www/site/layouts/shortcodes/code_sample.html | 23 ++ .../www/site/layouts/shortcodes/github_sample.html | 15 - website/www/site/layouts/shortcodes/io-matrix.html | 55 +++ website/www/site/static/.htaccess | 2 +- .../blog/beamsummit/beamsummit-digital-2020.png | Bin 0 -> 72388 bytes .../images/blog/beamsummit/beamsummit-digital.png | Bin 63238 -> 0 bytes 103 files changed, 2352 insertions(+), 842 deletions(-) create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/udf/BuiltinHashFunctions.java copy sdks/python/container/{py37/build.gradle => common.gradle} (89%) copy {.test-infra/tools => sdks/python/container/py38}/build.gradle (84%) copy sdks/python/test-suites/tox/{py35 => py38}/build.gradle (85%) rename website/www/{build_github_samples.sh => build_code_samples.sh} (75%) create mode 100644 website/www/site/content/en/blog/beam-2.21.0.md create mode 100644 website/www/site/content/en/documentation/patterns/bigqueryio.md create mode 100644 website/www/site/data/io_matrix.yaml create mode 100644 website/www/site/layouts/shortcodes/code_sample.html delete mode 100644 website/www/site/layouts/shortcodes/github_sample.html create mode 100644 website/www/site/layouts/shortcodes/io-matrix.html create mode 100644 website/www/site/static/images/blog/beamsummit/beamsummit-digital-2020.png delete mode 100644 website/www/site/static/images/blog/beamsummit/beamsummit-digital.png