See
<https://builds.apache.org/job/beam_PreCommit_Python_pytest_Cron/4/display/redirect?page=changes>
Changes:
[nicolas.delsaux] Seems like i changed that line by error. Rolling that back.
[nicolas.delsaux] That code was useless Indeed, if there is an IOException when
trying to
[nicolas.delsaux] According to jbonofre, the transformation of deliverable into
[nicolas.delsaux] fixed one last formatting point
[iemejia] [BEAM-7640] Create amazon-web-services2 module and AwsOptions
[alireza4263] [BEAM-7545] Adding RowCount to TextTable.
[kcweaver] [BEAM-6692] portable Spark: reshuffle translation
[chamikara] [BEAM-7548] Fix flaky tests for ApproximateUnique (#8960)
[iemejia] [BEAM-7589] Use only one KinesisProducer instance per JVM
[iemejia] [BEAM-7589] Make KinesisIOIT compatible with all other ITs
[kcweaver] [BEAM-7656] Add sdk-worker-parallelism arg to flink job server shadow
[daniel.o.programmer] Update python containers to beam-master-20190605
[dcrhodes] [BEAM-7666] Memory monitor change
[dcrhodes] [BEAM-7666] Adds the counter
[lcwik] [BEAM-5605] Update Beam Java SDK backlog to track latest changes in Beam
[hannahjiang] BEAM-3645 add ParallelBundleProcessor
[hannahjiang] BEAM-3645 reflect comments
[hannahjiang] BEAM-3645 add changes from review comments
[hannahjiang] BEAM-3645 add thread lock when generating process_bundle_id
[valentyn] Use Beam's abstraction of pickler instead of dill in coder tests.
[github] Tiny typo fix
[htyleo] [BEAM-7665] Support TypeDefinition options in beam.Combine()
[boyuanz] Add estimate_size() to source_test.LineSource
[kamil.wasilewski] [BEAM-7536] Fixed BQ dataset name in collecting Load Tests
metrics
[kamil.wasilewski] [BEAM-7504] Added top_count parameter
[lukasz.gajowy] [BEAM-4420] Allow connecting to zookeeper using external ip
[kamil.wasilewski] [BEAM-7504] Create Combine Python Load Test Jenkins job
[iemejia] [BEAM-7640] Rename the package name for amazon-web-services2 from aws
to
[github] Remove unused NEWLINE in TextSink.TextWriter
[ehudm] [BEAM-7679] Add randomness to ITs' BQ dataset name
[kcweaver] [BEAM-7670] portable py: prevent race opening worker subprocess
[kamil.wasilewski] [BEAM-7535] Created Jenkins job for BQ performance tests
[kamil.wasilewski] [BEAM-7535] Delete existing data if the table already exists
[iemejia] [BEAM-7682] Fix Combine.GroupedValues javadoc code snippet
[lukasz.gajowy] [BEAM-4420] Add KafkaIO integration test pipeline
[iemejia] [BEAM-6740] Add PTransformTranslator for Combine.Globally
[iemejia] [BEAM-6740] Add extractAcummulatorCoder for Combine.Globally and fix
[iemejia] [BEAM-7640] Change tests to use PayloadTranslator instead of unused
[iemejia] [BEAM-6740] Refactor to remove duplicated code in CombineTranslation
[boyuanz] Fix RestrictionTracker docstring
[kmj] Use consistent names for auto-generated GCP client artifacts.
[htyleo] Revise the comments to only mention the TypeDefinition options.
[kmj] Alphabetize imports of GCP library entries
[kmj] Depend on proto- instead of grpc- package for google_cloud_bigtable
[boyuanz] fix lint
[hannahjiang] BEAM-3645 add thread lock
[cademarkegard] [BEAM-7690] Port WordCountTest off DoFnTester
[cyturel] [BEAM-7683] - fix withQueryFn when split is more than 0
[kamil.wasilewski] [BEAM-7550] Reimplement Python ParDo load test according to
the proposal
[kamil.wasilewski] [BEAM-7503] Added iteration parameter to CoGBK test in Python
[kamil.wasilewski] [BEAM-7503] Created CoGBK Python Load Test Jenkins job
[kamil.wasilewski] [BEAM-3959] Added a flake F821 test
[yanzhi.wyl] [BEAM-7694]Fix error spelling in annotation of
SparkTransformOverrides.
[ryan] Consider Elasticsearch as one word in camelCase.
[juta.staes] [BEAM-7630] add ITs for writing and reading bytes from pubsub
[rezarokni] Edit of Looping Timer Blog to fix issue with Timer State
[kcweaver] [BEAM-7708] don't expect SQL shell bundled dependencies to be
shadowed
[kcweaver] [BEAM-7676] uniquely identify SDK workers by the factory that created
[github] [BEAM-7709] Re-use node for explicit flattens
[boyuanz] Reformat CamelCase function naming style to underscore style for
[kcweaver] [BEAM-7710] [website] remove outdated reference to KeyedCombineFn
[heejong] [BEAM-7689] make a temporary directory unique for FileBaseSink
[boyuanz] fix lint
[yanzhi.wyl] update annotation for PTransformMatcher.
[33895511+aromanenko-dev] [BEAM-6480] Adds AvroIO sink for generic records.
(#9005)
[github] [SQL][Doc] fix broken gradle command.
[lcwik] Added new example on how to create a custom unbounded streaming source
[chambers] Update Python Dataflow runner to patch side input coders on the
unified
[iemejia] [BEAM-7653] Add PTransformTranslator for Combine.GroupedValues
[pachristopher] Update pyarrow version requirement in setup.py
[github] Update code comments to improve readability in docs (#9024)
[katarzyna.kucharczyk] [BEAM-5994] Fix condition which allows to publish
metrics.
[lcwik] [BEAM-7668] Add ability to query a pipeline definition from a gRPC
[chamikara] [BEAM-7389] Add Python snippet for Partition transform (#8904)
[lcwik] Ensure that publishing vendored artifacts checks the contents of the jar
[lcwik] [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Fix shading of vendored
[bmvishwas] [BEAM-7621] Null pointer exception when accessing null row fields in
[alireza4263] [BEAM-7729] Fixes the bug by checking the value first before
parsing it.
[boyuanz] Add _get_file_size and UT
[b_m.vishwas] [BEAM-7731] Adding helper function to handle if null else pattern
for
[sniemitz] Done notifications for BigtableIO.Write
[github] Label the ParDos with their name
[udim] [BEAM-7437] Raise RuntimeError for PY2 in
[github] Update Python dependencies page for 2.14.0
[ankurgoenka] [BEAM-7546] Increasing environment cache to avoid chances of
recreating
[ankurgoenka] [BEAM-7736] Free worker when work is skipped
[yoshiki.obata] [BEAM-7284] enabled to pickle MappingProxyType in order to
pickle
[github] Some trivial typos
[robertwb] [BEAM-7737] Fix microbenchmark scripts compiled check (#9066)
[ehudm] [BEAM-2264] Reuse GCP credentials in GCS calls.
[kmj] Fix stream position bug in BigQuery Storage stream source.
[lgajowy] [BEAM-6675] Generate JDBC statement and preparedStatementSetter
[lukasz.gajowy] [BEAM-4420] Extract method for publishing already collected
metrics
[iemejia] [BEAM-7715] Mark user facing APIs related to External transforms as
[ttanay100] [BEAM-7674] Combine batch and streaming BQ Streaming Insert ITs
[udim] [BEAM-7578] add py37 hdfs integration test (#8970)
[pabloem] Adding logging and adding one more retry
[ryan] [BEAM-4181] Add readFiles transform to TfRecordIO.
[michal.walenia] [BEAM-7751] Add a CoGroupByKey load testing Jenkins job
[lukasz.gajowy] [BEAM-4420] Collect & write metrics to BigQuery and console
[iemejia] [BEAM-7557] - Migrate DynamoDBIO to AWS SDK for Java 2
[hannahjiang] BEAM-3645 improve test cases
[pabloem] [BEAM-7530] Add it test to read None values from BigQuery (#8875)
[github] [BEAM-7499] Fixup for tricky Reify testing issue (#9077)
[alireza4263] [BEAM-7545] Reordering Beam Joins and check if the produced join
is
[github] [BEAM-7641] Collect xunit statistics for Py ITs (#8952)
[github] [BEAM-4948, BEAM-6267, BEAM-5559, BEAM-7289] Update the version of
guava
[udim] [BEAM-7484] Metrics collection in BigQuery perf tests (#8766)
[zyichi] Add transform_name_mapping pipeline option for python sdk
[valentyn] Split Python 3 Postcommits into several jobs
[pabloem] Matching on filename, not directory for fileio
[pabloem] Lint fixup
[pabloem] Lint fixup
[lukasz.gajowy] [BEAM-7775] Remove job_PerformanceTests_Spark
[lukasz.gajowy] [BEAM-7775] Remove job_PerformanceTests_Dataflow
[robertwb] [BEAM-7060] Support translation of native type variables.
[lostluck] Fix documentation on iterable coder spec.
[github] Add retractions doc to design doc page.
[lostluck] Add the ability to set the service account email for dataflow jobs
[dcavazos] Skip DoFn params test in Python 2 on Windows
[github] Add last_attempted_record_start to python OffsetRangeTracker (#9058)
[ehudm] [BEAM-7784] Fixup for Guava upgrade
[robertwb] Refactor portable JobService to allow better sharing of code.
[robertwb] Simplify known runner parsing code.
[robertwb] [BEAM-7722] Add a Python FlinkRunner that fetches and uses released
[echauchot] Add Nexmark talk (ApacheCon 2017) and Universal metrics talk
(ApacheCon
[je.ik] [BEAM-7785] synchronize watermark update with bundle processing
[echauchot] Make it explicit that the link starts to play the audio podcast
[robertwb] Pull out generic java job server helpers from flink.
[robertwb] [BEAM-7060] Translate the typing.Iterable type hint.
[robertwb] Minor fixes discovered in migration.
[robertwb] [BEAM-7060] Automated replace of typehints with typing.
[robertwb] Revert incorrect doc on flink runner version.
[jeff] [BEAM-5191] Support for BigQuery clustering
[zyichi] Check streaming option when validate transform_name_mapping
[github] [BEAM-7747] Close the file handle owned by fastavro.write.Writer in
[github] Remove 4 empty spaces from PR template that mess up Python postcommit
[github] Relax pydot requirements.
[valentyn] Default to PiplelineState.UNKNOWN when job state returned from
v1beta3
[valentyn] fixup: Address review feedback.
[ttanay100] Replace old badges for Python PostCommit test with split ones
[yanzhi.wyl] [BEAM-7696]prepare files to stage also in local master of spark
runner.
[robertwb] Post-rewrite lint fixes.
[robertwb] Manual fixes for over-agressive replace.
[robertwb] More conservative typing module translation.
[kamil.wasilewski] [BEAM-7502] Create ParDo Python Load Test Jenkins job
[kamil.wasilewski] [BEAM-7502] Renamed file with Python GBK Load Test job
definition
[kamil.wasilewski] [BEAM-7502] Reduced number of iterations to 1 in Java ParDo
job
[alireza4263] [BEAM-7783] Adding BeamTableStatistics.
[gleb] Add setFailOnNoMatchingTests(true)
[cmach] [BEAM-7794] DynamoDBIOTest is blocking forever
[github] Revert "[BEAM-7060] Migrate to native typing types where possible."
[github] Update Python 3 entry in Python SDK roadmap.
[robertwb] Revert "Revert "[BEAM-7060] Migrate to native typing types where
[robertwb] [BEAM-7798] Unit test for BundleBasedRunner type hints.
[robertwb] Temporary workaround for [BEAM-7473] (#9023)
[robertwb] [BEAM-7798] Typehints fix for bundle based runner.
[robertwb] Make SDFBoundedSource wrapper work with dynamic splitting (#8944)
[robertwb] [BEAM-7801] Fix wordcount_xlang postcommit.
[kamil.wasilewski] [BEAM-7662] Create Python Combine load test job on Flink
[kamil.wasilewski] [BEAM-7661] Create Python coGBK load test job on Flink
[kamil.wasilewski] [BEAM-7661] Generate the same number of hot keys from both
sources
[gleb] [BEAM-6783] byte[] breaks in BeamSQL codegen
[gleb] [SQL] Annotate BuiltinStringFunctions with Strict
[relax] Merge pull request #8943: Schema conversion cleanup
[lostluck] [BEAM-7781] Update from Go Version 1.10 to 1.12
[lostluck] [BEAM-7781] Use correct Go gradle tasks
[lostluck] [BEAM-7781] Update to latest GoGradle
[aryan.naraghi] Add progress reporting to the BigQuery source
[github] Update Python 2.7.x restriction
[lostluck] [BEAM-7726] Add tests for DataSource protocol
[lostluck] [BEAM-7726] Refactor to prepare for state backing.
[lostluck] [BEAM-7726] Rename some SideInput code to State
[lostluck] [BEAM-7726] Handle State Backed iterables
[pabloem] [BEAM-7680] Removed flaky assertion and separated unit tests in
[ehudm] [BEAM-7650] Add docker system prune to inventory job
[iemejia] [BEAM-7794] Ignore blocking DynamoDBIOtest class
[valentyn] Configure console output collection for Python 3.6, 3.7 postcommit
[aaltay] [BEAM-7797] Move upload_graph handling logic before template
[strokach] Don't map transforms to pubsub subscriptions unless neccessary
[rohde.samuel] Update the Dataflow API version to 0607
[aaltay] Weaken overly-strict microbenchmark check_complied test. (#9100)
[strokach] Use dateutil for more flexible parsing of RFC-3339 / ISO-8601
formatted
[ehudm] [BEAM-7814] Wait for BQ query results in tests
[b_m.vishwas] [BEAM-7808] Add method to convert avro field to beam field
[36090911+boyuanzz] Revert "[BEAM-7785] synchronize watermark update with
bundle processing"
[iemejia] [BEAM-7807] Forbid TestNG imports on Beam classes
[iemejia] [BEAM-7807] Update Beam tests to not import TestNG related classes
[rohde.samuel] Add hot key detection protos to Windmill
[kedin] Update website for v2.14.0
[alireza4263] [BEAM-7823] Implementing a Custom NonCumulativeCost Handler
[altay] Upgrade google-cloud-core and google-cloud-bigquery dependencies
[github] Update nexmark.md
[filiperegadas] [BEAM-7830]: make spark default duration consistent with
PipelineResult
[kenn] [Java] remove unused variable in DirectMetricsTest.java
[alireza4263] [BEAM-7783] BeamIOSourceRel using custom BeamStatistics
[ehudm] [BEAM-7641] Add xunit results to run_validatescontainer.sh
[kenn] [BEAM-7755] adding support for nested rows and arrays in BigQuery to
[filiperegadas] fixup! [BEAM-7830]: make spark default duration consistent with
[pabloem] [BEAM-6611] BigQuery file loads in Streaming for Python SDK (#8871)
[mxm] [BEAM-7366] Restore Spotless check for Flink Runner
[github] Nitpicky grammar fix in programming guide
[jozsi] Set job name in Jet Runner
[robertwb] [BEAM-7060] Use typing in type decorators of core.py
[b_m.vishwas] [BEAM-7623] Add support to select MAP with Row as values in Beam
SQL
[je.ik] [BEAM-7574] Use Map-based window combining
[je.ik] [BEAM-7574] use Merging and NonMerging accumulating strategies
[je.ik] [BEAM-7574] unify Combine.perKey and Combine.globally
[je.ik] [BEAM-7574] Explode windows to key when needed
[je.ik] [BEAM-7574] Optimize GC performance, finalize implementation
[pabloem] [BEAM-7814] Make BigqueryMatcher wait for results. (#9154)
[valentyn] Introduce state when job state reported by a runner cannot be
[valentyn] Introduce state when job state reported by a runner cannot be
[12602502+Ardagan] Provision datasources and dashboards from config.
[pabloem] [BEAM-7700] Java transform catalog (#9129)
[ryan] Add arbitrary value matchers to MetricsResultMatchers.
[ryan] [BEAM-6883] Improve long-running spark streaming test.
[github] Merge pull request #9183: [BEAM-7675] Unify Gradle test configuration
[iemejia] [BEAM-7837] Add adapter class for API incompatible changes in Spark 3
[chamikara] Sets a default environment for Dataflow runner
[github] [BEAM-6202] Change retry policy for dataflow's get_job (#9182)
[bhulette] Fix some copy paste errors
[dcavazos] Fixed type on singleton side inputs sample
[altay] Change retry policy for dataflow's list_messages
[github] [BEAM-6877] trivial_inference: make remaining tests pass (#8893)
[pabloem] [BEAM-7820] Add hot key detection to Batch Dataflow Worker (#9186)
[amaliujia] [BEAM-7728] [SQL] Support ParquetTable (#9054)
[zach.badgett] Add git scm extension to wipe workspace before cloning project
[altay] Added 2.14.0 blog post draft
[heejong] [BEAM-7813] FileIO.write() doesn't fail when writing to invalid bucket
[boyuanz] Add "--workerHarnessContainerImage=" to dataflow java11 legacy worker
[github] [BEAM-4948] Switches dataflow runner to use a container image with
[robertwb] [BEAM-3645] Add multiplexing for python FnApiRunner (#8979)
[github] [BEAM-7840] Provide MapTuple and FlatMapTuple for Python 3 users.
[gleb] [BEAM-6783] byte[] breaks in BeamSQL codegen
[michal.walenia] [BEAM-7079] Add Chicago Taxi Example runner script with Python
scripts
[altay] fixup
[github] Update design-documents.md
[altay] fixup
[github] Pin python-dateutil to current major version
[yifanzou] Moving to 2.16.0-SNAPSHOT on master branch.
[aaltay] [BEAM-7389] Add code examples for Filter page (#9184)
[ankurgoenka] [BEAM-7856] Suppress error on table bigquery table already exists
[amaliujia] [Java] remove unneeded junit dependency.
[alireza4263] [BEAM-7844] Custom MetadataHandler for NodeStats (RowRateWindow)
[ankurgoenka] [BEAM-7868] optionally skip hidden pipeline options
[michal.walenia] [BEAM-7079] Add Chicago Taxi Example gradle task and Jenkins
job
[katarzyna.kucharczyk] [BEAM-7664] Add all GBK Flink Python test cases to job.
[katarzyna.kucharczyk] [BEAM-7664] Changed worker amount, added scaling cluster
and renamed
[katarzyna.kucharczyk] [BEAM-7664] Fix typo in Combine Flink test
[36090911+boyuanzz] Make SDK worker resilient to bad logging services. (#9214)
[altay] Add known issues
[altay] fixup
[ttanay100] [BEAM-6611] Move limitations comment to TriggerCopyJobs
[davidyan] [BEAM-7876] create_svg from graphviz returns bytes and need to be
[apilloud] [SQL] Use reflection to instantiate planner.
[github] Updated the download page anchor
[ehudm] [BEAM-7814] Fix BigqueryMatcher._matches when called more than once.
[github] Add ApproximateUnique link
[kedin] Fix date on 2.14.0 download links
[github] Merge pull request #8929: [BEAM-7620] Make run_rc_validation.sh
[udim] [BEAM-7828] Fixes Key type conversion from/to client entity in Python
[iemejia] [BEAM-7878] Refine Spark runner dependencies
[michal.walenia] [BEAM-7869] Add load testing jobs to README.md
[alireza4263] [BEAM-7844] Implementing NodeStat Estimations for all the nodes
[pabloem] Fix minor typos (#9192)
[heejong] [BEAM-7846] add test for BEAM-7689
[udim] [BEAM-7577] Allow ValueProviders in Datastore Query filters (#8950)
[heejong] [BEAM-7877] Change the log level when deleting unknown temoprary files
[github] [BEAM-7060] Introduce Python3-only test modules (#9223)
[ehudm] [BEAM-7060] Fix :sdks:python:apache_beam:testing:load_tests:run breakage
[markliu] [BEAM-7889] Update RC validation guide due to run_rc_validation.sh
[github] Update design-documents.md
[github] Update Dataflow container images used by unreleased (dev) SDK.
[jozsi] Adapt Jet Runner page to runner being released now
[lukasz.gajowy] [BEAM-7776] Create kubernetes.sh script to use kubectl with
desired
[lukasz.gajowy] [BEAM-7776] Stop using PerfkitBenchmarker in MongoDBIOIT job
[lukasz.gajowy] [BEAM-7776] Move repeatable kubernetes jenkins steps to
[shoaib.zafaf] BEAM-7018: Added regex transform on Python SDK.
[aromanenko.dev] [BEAM-7894] Upgrade AWS KPL to version 0.13.1
[dcavazos] [BEAM-7389] Add code examples for Values page
[chernyak] Increase default chunk size for gRPC commit and get data streams. The
[alireza4263] [BEAM-7898] Remove default implementation of getRowCount and
change the
[dcavazos] [BEAM-7389] Add helper conversion samples and simplified tests
[chernyak] [BEAM-7901] Increase gRPC stream chunk sizes - fix test
[heejong] [BEAM-7667] report GCS throttling time to Dataflow autoscaler
[asamadian] [BEAM-7777] Wiring up BeamCostModel
[asamadian] [BEAM-7777] Implementing beamComputeSelfCost for all the rel nodes
[markliu] [BEAM-7899] Fix Python Dataflow VR tests by specify sdk_location
[github] [BEAM-7607] Per user request, making maxFilesPerBundle public (#9160)
[heejong] [BEAM-6683] add createCrossLanguageValidatesRunner task
[jiangkai] use vendor-bytebuddy in sdks-java-core
[michal.walenia] [BEAM-7721] Add a new module with BigQuery IO Read performance
test.
[michal.walenia] [BEAM-7721] Add cleanup to test and change the way of
reporting metrics
[michal.walenia] [BEAM-7721] Refactor metric gathering
[iemejia] [BEAM-7880] Upgrade Jackson databind to version 2.9.9.3
[github] [BEAM-7845] Install Python SDK using pip instead of setuptools.
[aaltay] [BEAM-7833] warn user when --region flag is not explicitly set (#9173)
[12602502+Ardagan] Update env variable name in sync script
[dcavazos] [BEAM-7389] Add code examples for MapTuple and FlatMapTuple
[github] [BEAM-7862] Moving FakeBigQueryServices to published artifacts (#9206)
[heejong] [BEAM-7915] show cross-language validate runner Flink badge on github
PR
[robertwb] Fully qualify use of internal typehints.
[lcwik] [BEAM-7912] Optimize GroupIntoBatches for batch Dataflow pipelines.
[aaltay] [BEAM-7678] Fixes bug in output element_type generation in Kv
[ehudm] [BEAM-7860] Python Datastore: fix key sort order
[github] Update python-sdk.md
[github] Update python-pipeline-dependencies.md
[markliu] [BEAM-6907] Reuse Python tarball in integration tests
[markliu] Add missing dependency and source copy in tox test
[markliu] Build sdk tarball before running installGcpTest task
[amaliujia] [BEAM-7918] adding nested row implementation for unnest and
uncollect
[heejong] [BEAM-7924] Failure in Python 2 postcommit:
crossLanguagePythonJavaFlink
[aaltay] Add helper functions for reading and writing to PubSub directly from
[chadrik] Add JobService.GetJobs to the job API
[lukasz.gajowy] [BEAM-4775] Ensure each JobInvocation test is run in isolation
[lukasz.gajowy] [BEAM-4775] Refactor SamzaRunner - separate pipeline result
creation
[lukasz.gajowy] [BEAM-4775] Introduce PortablePipelineResult interface
[lukasz.gajowy] [BEAM-4775] Use PortablePipelineResult in all portable runners
[robertwb] [BEAM -7741] Implement SetState for Python SDK (#9090)
[david.moravek] [BEAM-7613] HadoopFileSystem can work with more than one
cluster.
[pabloem] [BEAM-7820] Add basic hot key detection logging in Worker. (#9270)
[github] Fix: Make sure experiments list if modifiable
[alireza4263] [BEAM-7896] Implementing RateEstimation for KafkaTable with Unit
and
[github] Update beam_fn_api.proto
[udim] Add assertArrayCountEqual, which checks if two containers have the same
[robertwb] Move DirectRunner specific classes into direct runner package.
[katarzyna.kucharczyk] [BEAM-7528] Save metrics according to distribution name.
[markliu] [BEAM-7940] Fix beam_Release_Python_NightlySnapshot
[36090911+boyuanzz] [BEAM-7874], [BEAM-7873] Distributed FnApiRunner bugfixs
(#9218)
[chadrik] Fix python code style
[thinh.t.ha] [BEAM-3489] add PubSub messageId to PubsubMessage
[thinh.t.ha] [BEAM-3489] add Coder test
[jiangkai] upgrade calcite to 1.20.0
[jiangkai] api changes in calcite 1.20.0
[lukasz.gajowy] [BEAM-4775] Allow reading portableMetrics from JobInvocation
using grpc
[lukasz.gajowy] [BEAM-4775] Stash handle to pipeline result to query metrics
later
[lukasz.gajowy] [BEAM-4775] Read portable metrics from grpc response in Python
SDK
[lukasz.gajowy] [BEAM-4775] Return portable metrics from Flink runner
[lukasz.gajowy] [BEAM-7772] Remove Pkb from file-based IOIT jobs
[lukasz.gajowy] [BEAM-7772] Refactor: improve naming in the file
[lukasz.gajowy] [BEAM-7772] Refactor: autoformat for consistency (intelij rules
applied)
[echauchot] Address review comments: nit
[katarzyna.kucharczyk] [BEAM-7528] Refactor of load test utils and added
documentation
[iemejia] [BEAM-5822] Remove non vendored byte-buddy dependency from
[aromanenko.dev] [BEAM-7916] Add ElasticsearchIO query parameter to take a
ValueProvider
[github] [BEAM-5148] Optionally load bson (#9308)
[iemejia] [BEAM-3608] Remove remaining unused non-vendored guava dependencies
[markliu] [BEAM-7940] Quick fix sdks:python:depSnapshot
[dcavazos] [BEAM-7389] Add code examples for KvSwap page
[dcavazos] [BEAM-7389] Add code examples for Map page
[dcavazos] [BEAM-7389] Add outputs for Filter page examples
[dcavazos] [BEAM-7389] Add code examples for Keys page
[dcavazos] [BEAM-7389] Add code examples for WithTimestamps page
[dcavazos] [BEAM-7389] Add code examples for FlatMap page
[kedin] Refactor release guide.
[kedin] [SQL] Disable KafkaCSVTableIT
[markliu] [BEAM-7959] Simplify use of Gradle property and task in Python
[tomer] [BEAM-7854] Resolve parent folder recursively in LocalFileSystem match +
[tomer] Use regex to detect non glob prefix of path
[tomer] Change tests
[tomer] Refactor + put test back
[katarzyna.kucharczyk] [BEAM-7946] changed name of sdist task in LoadTests
gradle.
[iemejia] Update build plugins
[github] Added Paris meetup into the list of meetups
[michal.walenia] [BEAM-7721]Add write test to BigQueryIOIT
[michal.walenia] [BEAM-7721]Code reformat
[tomer] Change logic + fix tests
[tomer] Spotless
[tomer] Spotless
[iemejia] [BEAM-7954] Validate beam modules to disable guava imports from
vendored
[amaliujia] [BEAM-7965] add retracting mode to model proto.
[mxm] [BEAM-7420]: Allow including the flink runner without flink on the
[gleb] [BEAM-7816] Support Avro dates in Schemas
[gleb] [BEAM-7817] Support Avro enums in Schemas
[aryan.naraghi] Add fine-grained progress reporting
[jkai] release vendor calcite
[chamikara] [BEAM-7866] Fix python ReadFromMongoDB potential data loss issue
[seb.jambor] use LOG.info instead of System.out.println
[kcweaver] [BEAM-7973] py: shut down Flink job server automatically
[yifanzou] add blog post for 2.15.0 release
[chadrik] Fix the exception type of InMemoryJobService when job id not found
[mxm] [BEAM-7936] Update portable WordCount Gradle task on portability page
[markliu] Fix command format in Release Guide
[github] Update stager.py
[ostrokach] Rename assertArrayCountEqual to assertUnhashableCountEqual and fix
typo
[iemejia] [BEAM-7802] Make SQL example slightly simpler
[iemejia] [BEAM-7802] Inline AvroUtils methods to have only one public AvroUtils
[kedin] [SQL] Add custom table name resolution
[iemejia] [BEAM-7802] Fix minor issues (access modifiers + static) in
[iemejia] [BEAM-7802] Make Schema.toString method multi OS friendly
[iemejia] [BEAM-7802] Add AvroUtils.schemaCoder method to infer a Beam schema
from
[kedin] [SQL] Support complex identifiers in DataCatalog
[dcavazos] [BEAM-7389] Add code examples for Partition page
[github] Downgrade log message level
[kcweaver] [BEAM-7988] py: include runner name when runner is invalid
[bhulette] Add ReadFromParquetBatched and ReadAllFromParquetBatched
[ehudm] [BEAM-7986] Upgrade grpcio
[kyle.winkelman] [BEAM-7989] Remove side inputs from CacheVisitor calculation.
[lukecwik] [BEAM-7987] Drop empty Windmill workitem in WindowingWindmillReader
[chadrik] Update to also catch StatusRuntimeException
[thw] Move Design Documents index to cwiki
[aaltay] [BEAM-6694] Added Approximate Quantile Transfrom on Python SDK (#9153)
[kcweaver] [BEAM-7924] fix options attribute error
[wcn] [BEAM-7994] Fixing unsafe pointer usage for Go 1.13
[iemejia] [BEAM-7882] Invoke Spark API incompatible methods by reflection
[thinh.t.ha] [BEAM-3489] add messageId coders to
[thinh.t.ha] [BEAM-3489] update docs
[alex] [BEAM-7999] Fix BigQuery timestamp handling for Schema Aware PCollection
[ryan] [BEAM-5164]: Add documentation for ParquetIO.
[ryan] fixup! [BEAM-5164]: Add documentation for ParquetIO.
[ryan] fixup! [BEAM-5164]: Add documentation for ParquetIO.
[ryan] fixup! [BEAM-5164]: Add documentation for ParquetIO.
[lukasz.gajowy] [BEAM-7772] Remove PKB from JdbcIOIT Jenkins job
[lukasz.gajowy] [BEAM-7772] Remove PKB from HadoopFormatIOIT Jenkins job
[lukasz.gajowy] [BEAM-7772] Remove PKB in hdfs file based IOITs
[lukasz.gajowy] [BEAM-7772] Refactor: rename consistently with
[lukasz.gajowy] [BEAM-7772] Remove obsolete pkb config files
[lukasz.gajowy] [BEAM-7772] Refactor: reformat code, fix variable naming
[lukasz.gajowy] [BEAM-7772] Move all file based ioits to one file
[tvanderlippe] [BEAM-1009] Upgrade to Mockito 3.0.0 and Powermock 2.0.2
[kyle.winkelman] [BEAM-5519] Remove redundant groupByKey in Spark Streaming.
[lukasz.gajowy] [BEAM-5980] Refactor: keep all Flink Python Combine load test
scenarios
[lukasz.gajowy] [BEAM-5980] Remove redundant Combine Performance tests
[iemejia] [BEAM-7264] Remove set/getClientConfiguration in JdbcIO
[github] Update beam_fn_api.proto
[zyichi] Add mongodbio to beam website io links
[github] [BEAM-1974] Add Metrics user-oriented documentation (#9328)
[kamil.wasilewski] [BEAM-8004] Fully restart flink cluster to prevent
UnknownHostException
[mxm] [BEAM-8010] Remove translation of bounded reads from portable Flink
[udim] Add a datatype_inference module for inferring coders and schema from
[kenn] Remove BEAM-6675 from 2.14.0 release notes (#9385)
[lukecwik] [BEAM-8006] Add retracting to windowing strategy translation.
[michael.wolanski] Max publish batch byte size set to a value which takes
base64 encoding
[iemejia] [BEAM-8020] Remove translation of bounded reads from portable Spark
[kamil.wasilewski] [BEAM-8022] Add new section in Dockerized Jenkins README
[kamil.wasilewski] [BEAM-7872] Implement Flink class which takes responsibility
of creating
[kamil.wasilewski] [BEAM-7872] Rewrite create_flink_cluster.sh so that it
supports other
[mxm] [BEAM-8011] Remove workaround to run unbounded sources in portable
[zyichi] [BEAM-7804] Update python sdk transform programming guide.
[cmaden] SDK 2.15.0 Python dependencies
[github] [BEAM-7832] Add ZetaSQL as a dialect in BeamSQL (#9210)
[thw] [BEAM-7980] Containerized Python worker pool for external environment
[ankurgoenka] [BEAM-7856] Re Raise exception for code other than 409
[michael.wolanski] PR Comments
[zyichi] [BEAM-7642] Fix python sdk AfterProcessingTime unit discrepancy
[patwari.rahul] [BEAM-6114] Calcite Rules to Select Type of Join in BeamSQL
[lostluck] [BEAM-8018] Detect unexported fields earlier
[lostluck] !fixup typos
[kcweaver] [BEAM-8037] add beam_fn_api experiment when using FlinkRunner
[github] [BEAM-7969] Report FnAPI counters as deltas in streaming jobs. (#9330)
[pabloem] [BEAM-7742] Partition files in BQFL to cater to quotas & limits
(#9242)
[heejong] [BEAM-8008] show error message from expansion service in Java External
[tweise] [BEAM-8038] Fix worker pool exit hook
[yifanzou] update beam website for 2.15.0
[12602502+Ardagan] [BEAM-8036] fix failed postcommit (#9409)
[yifanzou] Add highlights and known issues.
[yifanzou] fix 2.15.0 release blog
[yifanzou] fix in the beam website download file
[gleb] [BEAM-6783] Unignore ZetaSQLDialectSpecTestZetaSQL#testSelectBytes
[kamil.wasilewski] [BEAM-8082] Do not publish any metrics in Load tests on Flink
[pabloem] Add PubSub BigQuery IT in Python for Batch Loads in Streaming (#9309)
[github] Release upper bound for dill
[github] Updated 2.15.0 blog post
[aaltay] [BEAM-7909] upgrade python lib versions to match to dataflow worker
[kcweaver] [BEAM-7859] set SDK worker parallelism to 1 in word count test
[amaliujia] [Beam ZetaSQL] remove @Ignore from fixed tests.
[charith.ellawala] [BEAM-8032] Fix exception in JDBC readRows when statement
preparator is
[relax] Merge pull request #9275: [BEAM-6858] Support side inputs injected into
[ttanay100] Skip PubSubBigQueryIT.test_file_loads on Dataflow
[gleb] [BEAM-8080] [SQL] Fix relocation of com.google.types
[gleb] [BEAM-8080] [SQL] Do not relocate com.google.type.*
[je.ik] [BEAM-6127] upgrade junit dependency
[je.ik] [BEAM-8092] changed guava's Optional to java.util
[aromanenko.dev] [BEAM-7978] Return BACKLOG_UNKNOWN in case of unknown watermark
[markliu] [BEAM-8079] Move release build verification to Jenkins
[kcweaver] [BEAM-8015] Get logs from Docker containers
[dcavazos] [BEAM-7389] Update to use util.Regex transform
[markliu] [BEAM-8079] Move release Gradle build to a Jenkins job (Part - 2)
[altay] Removed unused arg from ToString.Element
[kamil.wasilewski] [BEAM-7872] Create a class for publishing docker images
[36090911+boyuanzz] Integrating ZetaSketch's HLL++ algorithm with Beam (#9144)
[kcweaver] [BEAM-7864] Simplify/generalize Spark reshuffle translation
[github] Revert #9319 - Changes to release guide (#9436)
[chamikara] [BEAM-8023] Add value provider interfaces for BigQueryIO.Read using
[github] Remove the dot to make job logs link clickable.
[ryan] Remove unwanted byte-buddy forced version.
[ryan] [BEAM-7829] Add schema names when converting with AvroUtils.toAvroSchema
[ryan] [BEAM-7829] Avoid name collisions in generated avro
[iemejia] [BEAM-8107] Update commons-compress to version 1.19
[bhulette] Revert "Merge pull request #8943: Schema conversion cleanup"
[bhulette] Revert "Update portable schema representation and java
SchemaTranslation
[github] Add SchemaCoder known issue to 2.15.0 release blog
[chadrik] The coder returned for typehints.List should be IterableCoder
[chadrik] Clarify the story surrounding the list return value guarantee
[rahulpatwari8383] [BEAM-6114] Handle Unsupported Lookup Joins
[bhulette] Bump container version
[mxm] [BEAM-5428] Modify cache token Proto design to only include tokens in
[charith.ellawala] Set default statement preparator for readRows
[kcweaver] [BEAM-7966] Write portable Flink application jar
[lcwik] [BEAM-5428] Update generated proto bindings for Go.
[markliu] [BEAM-7993] Run Portable PreCommit tests sequentially
[ankurgoenka] [BEAM-7972] Always use Global window in reshuffle and then apply
window
[dcavazos] [BEAM-7389] Update to use util.ToString transform
[relax] Support side-input injection in FnApiDoFnRunner.
[yifanzou] [BEAM-8116] Update website instructions in the build_rc script.
[yifanzou] [BEAM-8117] add notes when generating the gpg key.
[iemejia] [BEAM-8129] Update Spark runner to use spark version 2.4.4
[pabloem] [BEAM-8114] Fix failing Chicago Taxi Example job on Jenkins (#9456)
[thw] [BEAM-7980] Exactly once artifact retrieval for Python SDK worker pool
[yifanzou] [BEAM-8097] update the release doc
[tvalentyn] [BEAM-5878] support DoFns with Keyword-only arguments (#9237)
[lostluck] Makes subnetwork configurable
[chadrik] Allow arguments to be passed to python tests from tox cli.
[github] [BEAM-8014] Using OffsetRange as restriction for
[pabloem] [BEAM-7721] Add Jenkins job configuration for BigQueryIO performance
[github] Update fixed and sliding window examples to be consistent with the
[david.moravek] [BEAM-8025] Temporarily disable CassandraIOTest
[jozsi] Make Jet Runner related documentation improvements
[ttanay100] Add blog for GSoC '19
[clairem] Add static Pubsub.readMessagesWithCoderAndParseFn method
[aaltay] [BEAM-7819] Python - parse PubSub message_id into attributes property
[chadrik] Track PCollection boundedness in python sdk
[chadrik] Add tests
[aaltay] [BEAM-7909] Python3 docker containers #9351
[36090911+boyuanzz] [BEAM-7909] add Portable Precommit test for py36 and py37
(#9475)
[github] Revert "[BEAM-7819] Python - parse PubSub message_id into attributes
[sunjincheng121] [hotfix] Encode keys as NESTED for flink keyselector
[aaltay] [BEAM-7909] fix python container image name (#9478)
[sunjincheng121] [hotfix] Remove duplicate words for the doc of flink runner
[github] Revert "[hotfix] Encode keys as NESTED for flink keyselector"
[robertwb] Fix slice inference to not always assume List.
[github] [BEAM-7060] Type hints from Python 3 annotations (#9283)
[aaltay] [BEAM-7389] Add code examples for Regex page (#9262)
[robertwb] Remove obsolete field from proto.
[github] lint
[aaltay] [BEAM-7389] Add DoFn methods sample (#9257)
[lcwik] [BEAM-8161] Update joda time to 2.10.3 to get TZDB 2019b
[sunjincheng121] [BEAM-7947] Improves the interfaces of classes such as
FnDataService,
[sunjincheng121] fix code style
[github] Add Prerequisites(Requirements) to run gradlew
[github] Update portability.md
[jozsi] Accommodate changes proposed during review
[pabloem] Fix minor typos (#9498)
[github] [BEAM-7616] apitools use urllib with the global timeout. Set it to 60
[lostluck] [GoSDK] Improve singleton side input runtime error
[github] Fix typo in Cloud Pub/Sub topic path
[hannahjiang] [BEAM-8105] change default docker image name
[lostluck] Improve go worker harness error messages.
[lostluck] Add non-terminal option for go harness worker hook
[kcweaver] [BEAM-6865] make executable stage context management code reuseable
[kcweaver] [BEAM-7600] borrow SDK harness management code into Spark runner
[github] [BEAM-7969] Fix doublecount on Cross SDK boundary GRPC PCollections in
[relax] Merge pull request #9454: [BEAM-8111] Add ValidatesRunner test to
[robertwb] Fix docker workers on os-x.
[robertwb] Allow cross-language for Direct runner.
[ehudm] [BEAM-8156] Add convert_to_typing_type
[12602502+Ardagan] [BEAM-8167] Replace Postcommit_Python_Verify in Grafana
dashboards.
[kcweaver] refactor Flink and Spark contexts into separate classes
[hannahjiang] [BEAM-8105] update release guide with docker images
[kenn] [BEAM-8169] DataCatalogTableProvider to use GCP credentials from
[ehudm] [BEAM-3713] Convert ITs to not use save_main_session
[github] Merge pull request #9372: [Beam-6858] Validate that side-input
[github] Revert "[BEAM-5878] support DoFns with Keyword-only arguments (#9237)"
[kenn] fixup! [BEAM-8169] DataCatalogTableProvider to use GCP credentials from
[kenn] fixup! fixup! [BEAM-8169] DataCatalogTableProvider to use GCP
[lostluck] [BEAM-7970] Adding correct version of Go protobuf compiler.
[markliu] [BEAM-8144] Update hub setup in validates_rc_candidate.sh
[ostrokach] [BEAM-8170] Don't import PyArrow on Python 2 + Windows
[lostluck] Replace tabs with spaces
[boyuanz] Downgrade warning log to debug log.
[valentyn] Don't require pyarrow on Win32 installations.
[12602502+Ardagan] Update beamgrafana deploy to match latest deployment
[lostluck] [Go SDK] Off By One error on splitting
[kcweaver] s/state/jobFactory
[github] Set diskType on disks (#9515)
[lostluck] Revert "[Go SDK] Off By One error on splitting"
[markliu] [BEAM-8185] Properly backup sys config files in release script
[lostluck] [go-sdk] Never split on the 0th element.
[mxm] [BEAM-8187] Do not publish gradlew_orig in release
[markliu] [BEAM-8119] Properly cleanup Pubsub in release script
[valentyn] Temporarily replace super() calls with calls to __init__ of parent
class
[sunjincheng121] [BEAM-8034] Upgrade Flink Runner to 1.8.2
[valentyn] Warn users about Beam becoming Python3-only in the future to
encourage
[aaltay] BEAM-7995: python PGBKCVOperation using wrong timestamp (#9364)
[github] Update verify release branch in release-guide.md
[hannahjiang] [BEAM-8105] fixup: Mark's comment
[lostluck] Enables asynchronous execution
[lostluck] Renames flag to reduce ambiguity, as per review
[ehudm] [BEAM-5539] Upgrade google-cloud-pubsub and bigtable packages.
[robinyqiu] Exclude AvroSchemaTest for Apex and Flink runner for now because
they
[robinyqiu] Exclude ParDoTest.testSideInputAnnotationWithMultipleSideInputs
[ehudm] Add typing.Generator -> typehints.Iterator conversion
[ehudm] Fix bug in run_pylint.sh
[hannahjiang] [BEAM-8165] read version to set tag for default docker image
[hannahjiang] [BEAM-6185] add image artifacts to release process
[kcweaver] [BEAM-8208] [website] fix Python container build invocation
[ostrokach] [BEAM-8188] Allow PyArrow imports to fail on systems other than Win
+
[aaltay] [BEAM-7389] Add code examples for ParDo page (#9503)
[aaltay] [BEAM-7389] Add code examples for ToString page (#9289)
[robinyqiu] Use a 0-length byte array to represent empty sketch in HllCount
[ehudm] [BEAM-8196] Add wait_until_finish_duration
[hannahjiang] [BEAM-8165] add _sdk to container name
[kcweaver] [BEAM-7967] Execute portable Flink application jar
[markliu] Moving to 2.17.0-SNAPSHOT on master branch.
[12602502+Ardagan] Use common metric matchers for fnapi and legacy python
dataflow metric
[hannahjiang] [BEAM-8105] fix
[ankurgoenka] [BEAM-8217] Encode str to bytes py3 in sdk harness status server
[robinyqiu] Improve documentation on BigQueryIO
[chadrik] Create a more user friendly external transform API
[chadrik] KafkaIO: use StringUtf8Coder in Java to match python
[chadrik] Encode and decode using nested context
[chadrik] Add better error message for configuration type mismatch
[chadrik] Allow tests to specify a python 3 minor version, to isolate syntax
[chadrik] Split dataclass-based external transform tests to run in python 3.7
only
[chadrik] Add dataclasses as a native library for isort
[kamil.wasilewski] [BEAM-8178] Switch to a new gradle task that builds python
docker image
[kamil.wasilewski] [BEAM-7660] Create Python ParDo load test job on Flink
[ehudm] [BEAM-7911] Fix flakiness in avroio_test.py
[aaltay] [Beam 8031] Add snippets for pattern on custom windows (#9538)
[ehudm] [BEAM-7060] Prefer inspect module over funcsigs
[hannahjiang] [BEAM-8105] add a command to update version at gradle.properties
to
[tvalentyn] [BEAM-8225] Fix portable runner test Bugfix (#9557)
[katarzyna.kucharczyk] [BEAM-8228] Changed suffix of Dataflow counter metrics
names filtered
[lostluck] Direct - Fix ParDo with side input after CoGroupByKey
[tudor.marian] Fix state sampler dropping the stage and step name from reported
current
[chadrik] Exclude python 3-specific test files from running in post-commit
[12602502+Ardagan] [BEAM-8226] Alter getInputElementsConsumed computation for
[aaltay] [BEAM-7389] Generate notebook for filter (#9561)
[chadrik] gen_protos: don't run futurize twice when grpcio-tools is not
installed
[chadrik] gen_protos: use disutils log so that output is printed during normal
use
[chadrik] gen_protos: log the reason that protos are being regenerated
[chadrik] gen_protos: cleanup pb2 files when there are more than expected
[chadrik] gen_protos: stop checking protos twice for sdist
[pabloem] Add test for versioned encoding
[github] Update index.md
[luisortramirez2211] BEAM-8138: Code snippet in FileIO Java docs fixed
[github] Update execution-model.md
[romsc93] BEAM-7596: Fixed 'concepts' section to make it parallel
[kirillkozlov] [BEAM-8241] SQL code gen is more restrictive than Calcite
[hannahjiang] BEAM-8105 fixup
[hannahjiang] BEAM-8165 pass root as a param and keep default root
[lcwik] [BEAM-8240] Ensure that the pipeline proto contains the
[lcwik] [BEAM-8093] Commit change to ensure py36-gcp runs tests
[ehudm] [BEAM-8093] Fail tox environments the don't set command
[kcweaver] [BEAM-8252] sync python SDK GCP client API with the current Google
[github] Added comments
[github] [BEAM-8224] Fix bug in _fn_takes_side_inputs (#9563)
[lukasz.gajowy] [BEAM-8021] Stop publishing unneeded artifacts
[github] Fix typo for procesing --> processing in programming-guide.md
[github] Fix traceback.format_exc(e) calls
[kcweaver] [BEAM-8267] remove spurious error on Py worker shutdown
[zyichi] Minor fix for python mongodbio display data
[mxm] Compatibility matrix: Remove unnecessary dependency on docker task of
[robertwb] Refactor job server subprocess logic to be used elsewhere.
[kcweaver] [BEAM-8252] Python: add worker_region and worker_zone options
[b_m.vishwas] [BEAM-8203] Add support for AvroTable in SQL
[yifanzou] Add explanation of triaging release blockers
[ningk] [BEAM-7760] Added Interactive Beam module
[lostluck] [BEAM-8242] Support -buildmode=pie with unregistered Go functions
[hannahjiang] [BEAM-8165] use default tag
[robertwb] Add java jar based expansion service.
[kcweaver] [BEAM-8233] [BEAM-8214] [BEAM-8232] Document environment_type flag
[chamikara] Several changes to support cross-language transforms on
DataflowRunner
[ehudm] [BEAM-8279] Disable IOTypeHints.from_callable
[lukasz.gajowy] [BEAM-3845] Remove deprecated Class.newInstance() method usage
[lukasz.gajowy] [BEAM-8003] Update IOIT documentation. Remove all mentions of
Perfkit.
[lukasz.gajowy] [BEAM-8003] Update descriptions of file-based ioits
[lukasz.gajowy] [BEAM-8003] Remove "performanceTest" gradle task (obsolete)
[mxm] [BEAM-7945] Allow runner to configure semi_persist_dir which is used in
[pabloem] Update capability-matrix.yml (#9234)
[pabloem] Update index.md (#9576)
[lostluck] [BEAM-7629] Go SDK additional Validation for DoFns (1st impl)
[lostluck] [BEAM-7629] Additional checks for context param and error return.
[aaltay] [BEAM-7389] Generate notebook for FlatMap (#9584)
[kcweaver] [BEAM-8286] replace dead intersphinx link for google-cloud-python
[apilloud] [BEAM-8285] Upgrade to ZetaSQL 2019.09.1
[github] fix typo
[mxm] [BEAM-8157] Introduce encode_nested method on Python SDK Coder
[mxm] [BEAM-8157] Ensure key encoding for state requests is consistent across
[ihr] Add clarification about authorized views
[kamil.wasilewski] [BEAM-7660] Parameter names changes
[kirillkozlov] [BEAM-8275] Beam SQL should support BigQuery in DIRECT_READ mode
[cmaden] Document creating and implementing custom windows
[robertwb] Avoid comparison with param.default in type signature analysis.
[robertwb] More documentation.
[bhulette] replace unnecessary identity calls in sql tests
[relax] Merge pull request #9604: Restore portable schema changes, with simple
[mxm] [BEAM-7962] Drop support for Flink 1.5 and 1.6
[mxm] [BEAM-7962] Update version compatibility section on Flink Runner page
[thw] Update Flink runner page to refer to classic instead of legacy runner
[iemejia] [BEAM-8299] Upgrade Jackson to version 2.9.10
[robertwb] [BEAM-8301] Fix incomparable defaults.
[zyichi] Remove several supported dataflow python streaming items from
[chamikara] Sets workerHarnessContaienrImage as the containerImage of the
[robertwb] [BEAM-8302] Fix PostCommit_XVR_Flink
[robertwb] More guarding against missing grpc.
[aaltay] [BEAM-8160] Add FnApi execution mode instruction (#9628)
[kcweaver] [BEAM-8293] prescriptive log message for artifact retrieval failure
[jiangkai] [BEAM-5820] Switch Beam SQL to vendored Calcite
[michal.walenia] [BEAM-8256] Set fixed number of workers for file-based IOITs
[jiangkai] not include shadow direct runner dependency in nexmark
[jozsi] Remove the 'experimental' word from Jet Runner modules/directories
[mxm] [BEAM-8157] Increase parallelism for Python PVR tests
[lukasz.gajowy] [BEAM-8021] Add Automatic-Module-Name headers to beam's
artifacts.
[pabloem] [BEAM-8131] Provide Kubernetes setup for Prometheus (#9482)
[aromanenko.dev] [BEAM-8100] Exception handling for AsJsons and ParseJsons
transforms
[zyichi] [BEAM-8311] Fix py mongodbio display data
[github] Fixing issue in AvroCoder documentation.
[kenn] [BEAM-3372] Remove duplicated zone in Dataflow worker options (#9578)
[mxm] [BEAM-5428] Implement Runner support for cache tokens
[lostluck] [Go SDK] Fix Off by One error splitting DataSource
[lostluck] Update sdks/go/pkg/beam/core/runtime/exec/datasource.go
[dcavazos] [BEAM-7389] Use includes for buttons
[ankurgoenka] Add artifacts-dir param to flink runner
[github] [BEAM-8313] Rename certain proto fields to be consistent across
[lcwik] [BEAM-8315] Clean-up unused fields for splitting SDFs
[dcavazos] [BEAM-7389] Update include buttons to support multiple languages
[github] Update capability-matrix.yml
[github] [BEAM-8313] Follow up on PR comments for #9663 (#9668)
[pabloem] [BEAM-1296] Providing a small dataset for "Apache Beam Mobile Gaming …
[bhulette] Add test demonstrating BEAM-8317
[valentyn] Clean up a few places where we pass arbitrary kwargs to PTransform
[bhulette] Always use UTC when creating instances of baselocal
[github] Addressed review comments
[github] Added a test for BigQuery SQL read in EXPORT mode
[tvalentyn] [BEAM-7919] Add MongoDB IO integration test for py3.5 (#9639)
[robertwb] Pull out reusable parts of BeamFileSystemArtifact*.java into abstract
[kcweaver] [BEAM-8321] fix Flink portable jar test
[chamikara] Adds a pipeline option to Python SDK for controlling the number of
[angoenka] [BEAM-8314] Add aggregation logic to beam_fn_api metric counter
updat…
[rakeshkumar] BEAM-7739: Rename ValueStateSpec to ReadModifyWrite
[github] Fix traceback.format_exc(e) call
[mxm] [BEAM-5428] Implement cross-bundle user state caching in the Python SDK
[mxm] [BEAM-5428] Add pipeline option to enable caching / Disable caching by
[ajhalaria] [BEAM-8300] - Add readObject to initialize producer if it is null
[lukecwik] [BEAM-6923] limit gcs buffer size to 1MB for artifact upload (#9647)
[valentyn] Restrict dill's upper bound.
[lostluck] Helper to get the value of a KV type
[robertwb] [BEAM-6896] Loosen PyYAML dependency.
[lcwik] [BEAM-8021] Swap build-tools to be compile only so it isn't a "required"
[valentyn] Move Flink VR test code into a common file to be shared across
multiple
[valentyn] Include common portable runner tasks in python 3.x suites.
[valentyn] Evaluate the name of Python container task based on interpreter
version.
[valentyn] Add Python 3.5 ValidatesRunner Flink Jenkins suite.
[valentyn] Match test suite name in the UI with the trigger command.
[valentyn] Update postcommit status links in README.md.
[valentyn] Rename existing Python_PVR_Flink precommit job to Python2_PVR_Flink.
[b_m.vishwas] UT for BeamSql DDL with field having map with row as value
[bhulette] Add TimestampConversion to AvroCoder ReflectData, and accompanying
test
[bhulette] Add similar test to PubsubIO
[robertwb] [BEAM-8312] ArtifactRetrievalService serving artifacts from jar
[chadrik] Split out lint job from monolithic python preCommit tests on jenkins
[github] Fix Python RC staging URL in release script
[mxm] [BEAM-8303] Ensure FileSystems registration code runs in non UDF Flink
[mxm] Fix broken :runners:java-fn-execution:spotbugsMain task
[kcweaver] [BEAM-8286] move datastore intersphinx link again
[yifanzou] Fix download_files in the python RC validation script
[kcweaver] [BEAM-8312] Flink jars retrieve artifacts from classloader
[github] Fix wrong temp_location in run_rc_validation.sh
[github] Quietly skip cleanup_pubsub in automation script
[apilloud] [BEAM-8334] Expose Language Options for testing
[lostluck] [Go-SDK] Check correct cache for wrapped structs.
[enrico] Add job server request timeout (default to 60 seconds)
[lukecwik] Handle newlines in dataflow side input labels. (#9712)
[dcavazos] [BEAM-7389] Created elementwise for consistency with docs
[github] [BEAM-8346] Increase timeout of beam_PostRelease_Python_Candidate to
[kirillkozlov] [BEAM-8343] Moved BeamTable classes to a more appropriate
location
[klk] [SQL] Add support for GCS entries in DataCatalog
[klk] Port GCS SQL table to latest Data Catalog client
[kenn] [BEAM-8111] Enable CloudObjectsTest$DefaultCoders (#9446)
[relax] Merge pull request #9612: [BEAM-6855] Side inputs are not supported when
[mxm] [BEAM-5707] Modify Flink streaming impulse function to include a counter
[bhulette] Add RowJsonSerializer and tests
[aaltay] [BEAM-7760] Added pipeline_instrument module (#9619)
[amyrvold] [BEAM-6228]: Website build from source release fails due to git
[relax] Merge pull request #9424: Make RowCoder a sub-class of SchemaCoder<Row>
[lvwanqi11] Add support to pass params along to external worker pool.
[lvwanqi11] Add unittest for external environment.
[chadrik] Add a standard boolean coder
[kirillkozlov] [BEAM-6995] Modify BeamBasicAggregationRule to only be applied
when
[kenn] Move ZetaSQL to Calcite translator to separate module
[dcavazos] Fix closing region tags
[angoenka] [BEAM-7657] fix misleading sdk_worker_parallelism description (#9709)
[cmaden] Custom container docs
[markliu] Update beam website for 2.16.0
[github] Update downloads.md to correct release date
[aaltay] [BEAM-7389] Update docs with matching code files (#9692)
[github] Merge pull request #9671: Beam 2.16.0 release Blog post
[chadrik] Reject values outside of 0, 1
[kenn] Add ToJson transform (#9744)
[sunjincheng121] [BEAM-8164][website] Correct document for building the python
SDK
[iemejia] [BEAM-7635] Migrate SnsIO to AWS SDK for Java 2
[chadrik] Add BooleanCoder to standard coders in python registry
[chadrik] [BEAM-7738] Add external transform support to PubsubIO
[chadrik] Adapt io.external.pubsub reader and writer to new API
[chadrik] Adjust the way that the new API is used to handle the encoded data
[markliu] Merge pull request: [BEAM-8309] Document SDK 2.16.0 Python
dependencies
[github] Use assertIn for inclusion check.
[tweise] [BEAM-6829] Use transform/pcollection name for metric namespace if none
[chiarato] Changing visibility of PubsubTestClient
[github] [BEAM-8333] Adding lull logging for SDK harness (#9746)
[kcweaver] [BEAM-8370] update outdated Flink gradle command
[kcweaver] [BEAM-8369] update link to new container docs
[kenn] Remove DoubleCoder from dataflow known coders (#9755)
[kirillkozlov] [BEAM-8343] Added nessesary methods to BeamSqlTable to enable
support
[sunjincheng121] [BEAM-8375]Correct the command for Start a Flink job server in
[david.moravek] [BEAM-7730] Introduce Flink 1.9 Runner.
[david.moravek] [BEAM-7730] Reuse sources overrides from previous flink
versions.
[david.moravek] [BEAM-7730] Fix spotless plugin for flink runner.
[github] Updating the flink runner docs
[cmaden] Document ZetaSQL dialect and add generated ZetaSQL dialect reference
[kirillkozlov] Add project push-down capability to TestTableProvider
[github] [BEAM-8378] Downgrade build-scan plugin to 2.3 (#9762)
[kcweaver] [BEAM-8183] restructure Flink portable jars to support multiple
[tvalentyn] [BEAM-8287] Python 3 docs updates for 2.16.0 (#9700)
[lukecwik] Change the default status port of BatchDataflowWorker to 8081 (#9763)
[github] Updates Dataflow container images used by unreleased SDKs.
[klk] Tweak ZetaSQL docs wording
[leonardo.miguel] [BEAM-7926] Update dependencies in Java Katas
[xiliu] [BEAM-8340]: Support stable id in stateful transforms (#9733)
[ryan] [BEAM-7073]: Add unit test for Avro logical type datum.
[iemejia] [BEAM-8391] Update type id write methods in `AwsModule` from
deprecated
[kamil.wasilewski] [BEAM-8368] Add a <0.14.0 bound for pyarrow
[jeff] [BEAM-8393] Fix Java BigQueryIO clustering support for multiple
[thw] [BEAM-8387] Remove sdk-worker-parallelism option from JobServerDriver
[dcavazos] [BEAM-7389] Remove old element-wise snippets directory
[robertwb] Implement ZipFile backed artifact service.
[robertwb] Make compatible with Java artifact services.
[robertwb] [BEAM-8372] Job server submitting UberJars directly to Flink Runner.
[robertwb] Account for post BEAM-8183 file structure.
[github] [BEAM-4046] Remove old project name mappings. (#8915)
[drobert] [BEAM-8390] usable useCorrelationId with RabbitMq
[kcweaver] [BEAM-8398] Upgrade Google Cloud Java Client APIs
[rahulpatwari8383] [BEAM-8394]: Add withDataSourceConfiguration() method in
JdbcIO.ReadRows
[mxm] [BEAM-8403] Guard request id generation to prevent concurrent worker
[michal.walenia] [BEAM-8176] Fix BigQuery Java test jobs
[apilloud] [BEAM-8401] Upgrade to ZetaSQL 2019.10.1
[kirillkozlov] Replaced depricated calls
[lukecwik] Fixing a logging bug (#9795)
[lukecwik] Fixing a logging bug in auth (#9796)
[github] Showing the Java environment used by Dataflow in the UI (#8680)
[robertwb] [BEAM-8372] Follow-up on Flink UberJar submission.
[echauchot] Fix javadoc of Pardo
[github] [Go SDK] Fix subtle split error. Add more tests. (#9750)
[lostluck] [BEAM-8355] Add standard bool coder to Go SDK
[pabloem] [BEAM-8385] Add a KafkaIOIT hash for 100M records (#9770)
[lostluck] add missing license to exec/coder_test.go
[ehudm] [BEAM-4132] Support multi-output type inference
[tvalentyn] [BEAM-8350] Upgrade to Pylint 2.4 (#9725)
[github] [BEAM-8367] Using insertId for BQ streaming inserts (#9797)
[kirillkozlov] Updated price model and added a rule to perform project
push-down for
[ehudm] Reduce the verbosity of local_job_service_main
[nlofeudo] Avoid polluting Stackdriver logs with noise during autoscaling
events.
[echauchot] Add DoFn lifecycle sequence diagram to website
[echauchot] Add Source lifecycle sequence diagram to website
[echauchot] Fix menus and width for lifecycle images
[lukasz.gajowy] [BEAM-8422] Notify builds@ on ABORTED job final state
[michal.walenia] [BEAM-8281] Refactor the IOIT settings
[lukasz.gajowy] [BEAM-8424] Lower the timeout value for Dataflow
ValidatesRunner tests
[lukasz.gajowy] [BEAM-8424] Set timeout value of java 11 dataflow validates
runner tests
[kcweaver] fix PubsubJsonClientTest NPE
[nlofeudo] Spotless Java applied
[lvwanqi11] [BEAM-8417] Expose ExternalWorkerHandler hostname
[valentyn] Add ValidatesRunner test for Impulse transform
[valentyn] Use base64 string for representing impulse payload in legacy DF
runner
[lukecwik] Small improvement. (#9818)
[kirillkozlov] Add project push-down for BigQuery
[robertwb] [BEAM-8416] Add locking to zipfile reading as well as zipfile
writing.
[amaliujia] Test DataCatalogBigQueryIT on ZetaSQL Planner
[chadrik] Must use Boolean arg to make use of the BooleanCoder in Java
[kcweaver] [BEAM-8412] xlang test: set sdk worker parallelism to 1
[amaliujia] [BEAM-8433] DataCatalogBigQueryIT runs for both Calcite and ZetaSQL
[github] Adds a section on how to make reviewer's job easier in Contributor
Guide
[davidyan] [BEAM-8415] Improving error message when applying PTransform with a
name
[robertwb] [BEAM-8396] Default to LOOPBACK mode for local flink runner.
[github] Fixing BQ streaming inserts test (#9824)
[cmach] [BEAM-8410] JdbcIO should support setConnectionInitSqls in its
[klk] Move test dependency to testCompile scope
[klk] Make CalciteQueryPlanner public, since users can choose query planner
[klk] Inline DataCatalogBigQueryIT to its only subclass
[chadrik] Fix docs
[b_m.vishwas] [BEAM-5690] Fix Zero value issue with countByKey transform in
Spark
[sunjincheng121] [BEAM-8442] Remove duplicate code for bundle register in
Python SDK
[kcweaver] [BEAM-8430] Change py default sdk_worker_parallelism to 1
[b_m.vishwas] [BEAM-5690] Addressing review comments
[robertwb] [BEAM-8416] Hold the lock for the duration, not just the creation, of
[mxm] [BEAM-8439] Delay bundle start until pushed back data is emitted
[klk] Fix some not allowed nulls in ZetaSQL translator
[klk] Use UnsupportedOperationException for not implemented methods in ZetaSQL
[kenn] Use JUnit Parameterized runner for exercising different SQL planners
[robertwb] Make beam_fn_api opt-out rather than opt-in for runners.
[robertwb] Workaround for BEAM-8436, BEAM-8444, and BEAM-8448.
[github] [BEAM-8405] Support embedded Datastore entities (#9805)
[kenn] [BEAM-8456] Add pipeline option to have Data Catalog truncate
[kenn] Inline Data Catalog TableUtils to its only use
[alex] [BEAM-5967] Add handling of DynamicMessage in ProtoCoder
[alex] [BEAM-5967] Add Nullable annotations and corrected JavaDoc after review
[alex] Backported ProtoDomainTest for schema aware PR
[alex] Split the ProtoCoder into ProtoCoder and DynamicProtoCoder.
[mxm] [BEAM-7730] Migrate docs and integration tests to Flink 1.9
[david.moravek] [BEAM-7730] Upgrade Flink 1.9.x to 1.9.1.
[aaltay] [BEAM-8449] Document known issue with macOS installation (#9847)
[aaltay] [BEAM-7389] Show code snippet outputs as stdout (#9790)
[kirillkozlov] Updated PushDown rule and TestTableProvider to support predicate
[pabloem] Fixing BQ write transform
[kirillkozlov] Updated cost model to favor IOs with filters pushed-down, added
more
[echauchot] Add introductory content to the lifecycle sequence diagrams
[je.ik] [BEAM-7520] Add ValidatesRunner test for timer ordering
[je.ik] [BEAM-7520] Synchronize watermarks with bundle processing and add timer
[je.ik] [BEAM-7520] add UsesStrictTimerOrdering category, ignore test for samza
[david.moravek] [BEAM-7730] Add ability to override resources for a specific
flink
[pabloem] Merge pull request #9854 from [BEAM-8457] Label Dataflow jobs from
[lostluck] [Go SDK] Fix Logging call depths.
[lostluck] [Go SDK] Track failed instructions in harness.
[lostluck] [Go SDK] Fix log on failed data channel read.
[github] [BEAM-7981] Fix double iterable stripping (#9708)
[je.ik] [BEAM-7520] ignore UsesStrictTimerOrdering for both batch and streaming
[chadrik] Mark kafka and pubsub external transforms as experimental
[migryz] Moving to 2.18.0-SNAPSHOT on master branch.
[migryz] Revert "Moving to 2.18.0-SNAPSHOT on master branch."
[migryz] Moving to 2.18.0-SNAPSHOT on master branch.
[ehudm] Add assertions on pcoll types
[xinyuliu.us] [BEAM-8341]: basic bundling support for portable runner (#9777)
[robertwb] [BEAM-8480] Explicitly set restriction coder for bounded reader
wrapper
[github] Revert "Merge pull request #9854 from [BEAM-8457] Label Dataflow jobs
[aaltay] [BEAM-7389] Add code snippet for CoGroupByKey (#9791)
[valentyn] Explicitly disable runner_api path in
test_remote_runners_display_data.
[valentyn] Extract helper classes to module level.
[valentyn] Simplify test_remote_runners_display_data assertion logic.
[ehudm] Don't try to infer recursively
[ehudm] [BEAM-8269] Prefer class decorators over from_callable
[lostluck] [Go SDK] Fix post commits: runtime error in logger
[kamil.wasilewski] [BEAM-8462] Bump up Flink version inside gradle task name
[kamil.wasilewski] [BEAM-8462] Put Hadoop jar into the FLINK_INSTALL_DIR/lib
directory
[kamil.wasilewski] [BEAM-8462] Add --region=global parameter to gcloud dataproc
calls
[bhulette] Combine serializer and deserializer
[bhulette] Pass along exception
[bhulette] expose of method
[bhulette] fixup!
[bhulette] Merge incorrectly split strings
[lostluck] [Go SDK] Propagate data channel failures.
[lostluck] [Go SDK] Simplify datamgr tests.
[kenn] Make BigQueryUtils, PubsubUtils, GcsUtils implementations of
[github] [Go SDK] Delete spammy log in datamgr.go
[lostluck] In Go, base counter names on PTransform unique_name
[mxm] [BEAM-8372] Support both flink_master and flink_master_url parameter in
[mxm] [BEAM-8372] Match the Java SDK's default of 'flink_master'
[robertwb] Move use_sdf_bounded_source tests to iobase.
[github] Merge #9858 for [BEAM-8446] Adding a test checking the wait for BQ jobs
[github] Improve user support story - and add community events link (#9875)
[github] Job timeout after 120 minutes instead of 100
[valentyn] fixup: remove sorting of expected_data
[lostluck] [BEAM-8493] Add standard double coder to Go SDK.
[hsuryawirawan] Change from study_project.xml to yaml files for Beam Katas
Python
[hsuryawirawan] Exclude Katas remote-info.yaml files from RAT check
[ryan] Move sequence diagrams to PlantUML SVG.
[hsuryawirawan] Add note for the "Built-in IOs" tasks to inform that there is
no kata to
[mxm] [BEAM-8383] Fix gauge metrics in Python SDK and Flink runner
[lukecwik] [BEAM-8432] Move javaVersion to gradle.properties (#9830)
[lostluck] Enables Dataflow workers to use private IPs
[pabloem] Update the Beam Katas Java to use the new course metadata yaml format
[lukecwik] BEAM-3658 Port SpannerIOReadTest off DoFnTester (#9898)
[mxm] [BEAM-8383] Add metrics for the Python state cache
[github] Increasing timeout due to still constant timeouts.
[thw] [BEAM-8471] Flink native job submission for portable pipelines
[kirillkozlov] Added more tests, bug fixing
[tweise] [BEAM-8471] Put back FlinkJobServerDriver.fromParams to fix post commit
[kirillkozlov] Addressed review comments
[kcweaver] [BEAM-8509] TestPortableRunner uses JobServerDriver interface
[kamil.wasilewski] [BEAM-4775] Extract code responsible for parsing
MonitoringInfo
[kamil.wasilewski] [BEAM-4775] Expose metrics gathered by FnApiRunner in
LocalJobServicer
[kamil.wasilewski] [BEAM-4775] Publish only user metrics in load tests
[kamil.wasilewski] [BEAM-4775] Use result property of MetricResult in load tests
[lukasz.gajowy] [BEAM-6303] Add withSuffix(".parquet") to Parquet write javadoc.
[mxm] [BEAM-7730] Add Flink 1.9 to the list of supported Flink versions
[dcavazos] [BEAM-7389] Add code snippets for Distinct
[kirillkozlov] Fixed style and readability issues
[kirillkozlov] BigQuery should return row fields in the selected order
[bhulette] Bump Python container versions
[robertwb] Cleanup metrics tests.
[lcwik] [BEAM-8515, BEAM-2699] Add equals/hashCode to ValueProvider classes.
[lcwik] [BEAM-7230] Cache DataSource instances based upon equivalent
[kcweaver] [BEAM-8516] only consider current version in sdist build
[kcweaver] [BEAM-8472] get default GCP region option (Python)
[thw] Fail when pipeline options conversion fails due to incompatible
[github] [BEAM-8517] Fix timezone conversion issue due to usage of different
[boyuanz] Not inject pubsub into Impulse when in fnapi streaming mode.
[tweise] [BEAM-8507] Augment 'flink_master' pipeline option for Python
[github] Remove --flink-master-url alias.
[derek.he] [BEAM-8306] estimate byte size by product count
[github] [BEAM-8474] A microbenchmark for Python FnApiRunner: (#9891)
[robertwb] [BEAM-8396] Restore Flink runner LOOPBACK default.
[robertwb] Remove reference to --flink-master-url.
[robertwb] Revert "Remove --flink-master-url alias."
[kamil.wasilewski] [BEAM-8526] Turn on metrics publishing in load tests on Flink
[mxm] Remove occurrences of flink-master-url and add test for
[je.ik] [BEAM-8530] disable UsesStrictEventTimeOrdering for portable dataflow
[github] [BEAM-8434] Translate trigger transcripts into validates runner tests.
[kcweaver] [BEAM-8533] Revert "[BEAM-8442] Remove duplicate code for bundle
[bhulette] Add separate testSQLRead and testSQLWrite test in BigQueryIT
[bhulette] Add PubsubJsonIT tests that pull and verify pubsub messages
[bhulette] Add TestPubsub#{waitForNMessages, assertThatTopicEventuallyReceives}
[pabloem] [BEAM-8451] Fix interactive beam max recursion err (#9865)
[kcweaver] [BEAM-8521] only build Python 2.7 for xlang test
[kenn] [BEAM-8514] ZetaSql should use cost based optimization (#9874)
[kenn] [BEAM-7434] [BEAM-5895] and [BEAM-5894] Upgrade to rabbit amqp-client
[jeff] [BEAM-8540] Fix CSVSink example in FileIO docs
[jbonofre] [BEAM-8513] Allow reads from exchange-bound queue without declaring
the
[robertwb] [BEAM-8544] Use ccache for compiling the Beam Python SDK.
[robertwb] Align base image requirements with Beam's setup.py.
[udim] [BEAM-7917] Fix datastore writes failing on retry (#9294)
[robertwb] Additional logging.
[pabloem] Merge pull request #9741 from [BEAM-7926] Visualize PCollection
[robertwb] [BEAM-8532] Output timestamp should be the inclusive, not exclusive,
end
[aaltay] [BEAM-7389] Add code snippets for CombinePerKey (#9921)
[kenn] [BEAM-8146,BEAM-8204,BEAM-8205] Add equals and hashCode to SchemaCoder
[klk] Add Jira priority explanations to Beam site
[klk] Add release blocking section to Beam site
[pabloem] [BEAM-8491] Add ability for replacing transforms with multiple outputs
[github] [BEAM-8467] Enabling reading compressed files (#9861)
[yoshiki.obata] update trigger phrase for beam_PreCommit_Website_Stage_GCS in
README
[github] [BEAM-3713] Add pytest for unit tests (#9756)
[github] [BEAM-8368] [BEAM-8392] Update pyarrow to the latest version 0.15.1
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-10 (beam) in workspace
<https://builds.apache.org/job/beam_PreCommit_Python_pytest_Cron/ws/>
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init
> <https://builds.apache.org/job/beam_PreCommit_Python_pytest_Cron/ws/src> #
> timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git
> +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* #
> timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git
> +refs/heads/*:refs/remotes/origin/*
> +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 9ed9dc793162965e9b1d960ba276b52b675350c1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9ed9dc793162965e9b1d960ba276b52b675350c1
Commit message: "[BEAM-8368] [BEAM-8392] Update pyarrow to the latest version
0.15.1 (#9970)"
> git rev-list --no-walk 2581565fe96f2a1b4be01993831898afefb3e4c7 # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the
SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $
<https://builds.apache.org/job/beam_PreCommit_Python_pytest_Cron/ws/src/gradlew>
--continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g
-Dorg.gradle.jvmargs=-Xmx4g :pythonPreCommitPytest
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for
details
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
> Task :buildSrc:assemble
> Task :buildSrc:spotlessGroovy
> Task :buildSrc:spotlessGroovyCheck
> Task :buildSrc:spotlessGroovyGradle
> Task :buildSrc:spotlessGroovyGradleCheck
> Task :buildSrc:spotlessCheck
> Task :buildSrc:pluginUnderTestMetadata
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validateTaskProperties FROM-CACHE
> Task :buildSrc:check
> Task :buildSrc:build
Configuration on demand is an incubating feature.
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine the dependencies of task
':sdks:python:test-suites:tox:py35:preCommitPy35Pytest'.
> Task with path 'lint' not found in project
> ':sdks:python:test-suites:tox:py35'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10s
Publishing build scan...
https://gradle.com/s/ghibe5x2ixy7o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org
For additional commands, e-mail: builds-h...@beam.apache.org
