This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/checkout-4
in repository https://gitbox.apache.org/repos/asf/beam.git
discard e941089a715 Bump actions/checkout from 3 to 4
add d26a78297f2 [YAML] Add JSON parsing to PubSub IO. (#28754)
add 6722ff197e8 Schema-aware Avro reading and writing.
add 714d3a9aa04 Add yaml providers for avro.
add 07e26fd014d test,lint
add ce217d36b13 Merge pull request #28855 [YAML] Schemify avroio and add a
yaml provider.
add 57821c191d3 Add python wheel staging to GitHub Actions
build_release_candidate
add 795d366c395 Merge pull request #28822: Add python wheel staging to
GitHub Actions build_release_candidate
add e665c1c73bd [Fix] Broken Healthcare Notebook Link (#28876)
add ee7a2aea975 Fix perf alert tool call (#28872)
add fba098b69db Bump postcss (#28866)
add cb802661f2a [Python]Update state cache size to 100 MB (#28877)
add fc93a36d869 Make parquetio schema aware.
add e94aeb71fff Add parquet yaml providers.
add 9602af91dcd Update error message to account for more detail.
add d7db27efccc Merge branch 'master' into yaml-parquet
add a6947eb3b79 Merge pull request #28850 [BEAM-28684] [YAML] Add Parquet
reading and writing.
add 7531501ff27 Fix interactive tests for Beam Notebooks release (#28880)
add b117ac8b083 Update executionResult (#28884)
add d5b8fb82958 Add edge segment size to filter out change points that are
observed on the data edge (#28780)
add a3bf007cef1 Append UUID when test_name is None for perf tool (#28882)
add 92d5bf87796 Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 in
/sdks (#28891)
add 76fbb8e9686 Bump org.nosphere.apache.rat from 0.8.0 to 0.8.1 (#28675)
add 604629798a3 Add dataclasses to perf alert tool and refactor code.
(#28889)
add d17063412ea Update BigQueryIO Documentation (#28591)
add 66bae3fa5ba Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#28890)
add ad5c2f55bfe Bump golang.org/x/sync from 0.3.0 to 0.4.0 in /sdks
(#28856)
add 5491b5cdabc Update container version in BeamModulePlugin.groovy
(#28879)
add be3909a62d7 Bump cloud.google.com/go/bigquery from 1.55.0 to 1.56.0 in
/sdks (#28892)
add 9eac03027bc Bump github.com/testcontainers/testcontainers-go in /sdks
(#28761)
add 4ead788840c Reenable TestFhirIO.* on GHA (#28435)
add 21f822f6580 Catch EOFError along with StopIteration (#28896)
add f63e9ed28e4 Additional context for decoding errors. (#27024)
add dafe928a15b Add Java variants of WriteTo{Csv,Json}. (#28380)
add 2bbb3485c78 Populate top-level display data in yaml main. (#28512)
add 3844972d681 Revert "Populate top-level display data in yaml main.
(#28512)"
add ab5992940c1 Merge pull request #28906 from
apache/revert-28512-display-data
add 3d574b485ab Fix sourceJar task dependencies (#28900)
add 152f9c52b57 Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 in /sdks
(#28912)
add e01de664594 remove unused os import (#28914)
add e130352d7c7 Remove issue_comment trigger for load test, performance
test and other jobs (#28915)
add a7abe61288c Remove issue_comment trigger in most postcommit (#28916)
add 2bccee14cf9 Bump google.golang.org/api from 0.144.0 to 0.146.0 in
/sdks (#28911)
add a859863d392 update dataflow containers (#28904)
add d2420ad6ee5 Fix buildDirectory (#28925)
add 65eaf45026e fix lint (#28917)
add 99c87a2c68b Updates the Transform Service to accept Python extra
packages through the Java API (#28783)
add 88139bdc9a0 Retrieve workflow runs for GA Post-Commits Status
dashboard (#28844)
add e3c1591bbb8 Bump google.golang.org/grpc from 1.58.2 to 1.58.3 in /sdks
(#28938)
add 62efb967884 Bump golang.org/x/net from 0.16.0 to 0.17.0 in /sdks
(#28939)
add d437afab961 Run self-assign and pr-updates only in create event
(#28928)
add a54afde4a05 Fix missing backslash in arguments of job (#28942)
add e56d1a1157b Enable missing-timeout pylint check (#28920)
add 9cdcb036cc9 move nested windmill state classes to top level files and
organize di… (#28506)
add 6a456f04b68 Add user counters and msec metrics to prism UI. (#28929)
add 2bd23b16666 Add counter for MLTransform and data processing transforms
(#28927)
add c62a81b1c57 Reapply "Populate top-level display data in yaml main.
(#28512)"
add de48383bdca Fix typo in docstring.
add b82dd4849e4 Merge pull request #28931 from robertwb/display-data
add 7a1245d4011 Bump github.com/google/go-cmp from 0.5.9 to 0.6.0 in /sdks
(#28940)
add 2bfcb9fda37 Remove unnecessary a space character that got through
Javadoc CI ;) (#28863)
add aedfa461fb3 [#20970] Fix gRPC leak by closing ResidualSource at
BoundedToUnboundedSourceAdapter.Reader#init() in Dataflow worker (#28548)
add bd46c76dd08 Bump snappy-java version to 1.1.10.4 (#28655)
add ebad3deb5a0 Add GitHub Workflow Replacements for Jenkins
job_LoadTests_Python_CoGBK subgroup (#28764)
add dfdbcb9006b Add GitHub Workflow Replacements for Jenkins
job_LoadTests_Java_CoGBK subgroup (#28742)
add 1e1f4f0849d Bump actions/checkout from 3 to 4
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e941089a715)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/checkout-4 (1e1f4f0849d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../beam_LoadTests_Go_CoGBK_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Go_CoGBK_Flink_batch.yml | 2 -
.../beam_LoadTests_Go_Combine_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Go_Combine_Flink_Batch.yml | 2 -
.../beam_LoadTests_Go_GBK_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Go_GBK_Flink_Batch.yml | 2 -
.../beam_LoadTests_Go_ParDo_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Go_ParDo_Flink_Batch.yml | 2 -
.../beam_LoadTests_Go_SideInput_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Go_SideInput_Flink_Batch.yml | 2 -
...> beam_LoadTests_Java_CoGBK_Dataflow_Batch.yml} | 51 +-
...eam_LoadTests_Java_CoGBK_Dataflow_Streaming.yml | 2 -
...s_Java_CoGBK_Dataflow_V2_Batch_JavaVersions.yml | 132 +
...a_CoGBK_Dataflow_V2_Streaming_JavaVersions.yml} | 101 +-
..._Java_CoGBK_SparkStructuredStreaming_Batch.yml} | 51 +-
.../beam_LoadTests_Java_Combine_Dataflow_Batch.yml | 2 -
...m_LoadTests_Java_Combine_Dataflow_Streaming.yml | 2 -
...Java_Combine_SparkStructuredStreaming_Batch.yml | 2 -
.../beam_LoadTests_Java_GBK_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Java_GBK_Dataflow_Streaming.yml | 2 -
...LoadTests_Java_GBK_Dataflow_V2_Batch_Java11.yml | 2 -
...LoadTests_Java_GBK_Dataflow_V2_Batch_Java17.yml | 2 -
...Tests_Java_GBK_Dataflow_V2_Streaming_Java11.yml | 2 -
...Tests_Java_GBK_Dataflow_V2_Streaming_Java17.yml | 2 -
...sts_Java_GBK_SparkStructuredStreaming_Batch.yml | 2 -
.../beam_LoadTests_Java_ParDo_Dataflow_Batch.yml | 2 -
...eam_LoadTests_Java_ParDo_Dataflow_Streaming.yml | 2 -
...s_Java_ParDo_SparkStructuredStreaming_Batch.yml | 2 -
...beam_LoadTests_Python_CoGBK_Dataflow_Batch.yml} | 65 +-
..._LoadTests_Python_CoGBK_Dataflow_Streaming.yml} | 64 +-
...=> beam_LoadTests_Python_CoGBK_Flink_Batch.yml} | 73 +-
...eam_LoadTests_Python_Combine_Dataflow_Batch.yml | 2 -
.../beam_LoadTests_Python_Combine_Flink_Batch.yml | 2 -
...am_LoadTests_Python_Combine_Flink_Streaming.yml | 2 -
...LoadTests_Python_FnApiRunner_Microbenchmark.yml | 2 -
.../beam_LoadTests_Python_GBK_Dataflow_Batch.yml | 2 -
...eam_LoadTests_Python_GBK_Dataflow_Streaming.yml | 2 -
.../workflows/beam_PerformanceTests_AvroIOIT.yml | 2 -
.../beam_PerformanceTests_AvroIOIT_HDFS.yml | 2 -
...PerformanceTests_BigQueryIO_Batch_Java_Avro.yml | 2 -
...PerformanceTests_BigQueryIO_Batch_Java_Json.yml | 2 -
..._PerformanceTests_BigQueryIO_Streaming_Java.yml | 2 -
...eam_PerformanceTests_BiqQueryIO_Read_Python.yml | 2 -
...formanceTests_BiqQueryIO_Write_Python_Batch.yml | 2 -
.github/workflows/beam_PerformanceTests_Cdap.yml | 2 -
.../beam_PerformanceTests_Compressed_TextIOIT.yml | 2 -
...m_PerformanceTests_Compressed_TextIOIT_HDFS.yml | 2 -
.../beam_PerformanceTests_HadoopFormat.yml | 2 -
.github/workflows/beam_PerformanceTests_JDBC.yml | 2 -
.../workflows/beam_PerformanceTests_Kafka_IO.yml | 2 -
.../beam_PerformanceTests_ManyFiles_TextIOIT.yml | 2 -
...am_PerformanceTests_ManyFiles_TextIOIT_HDFS.yml | 2 -
.../beam_PerformanceTests_MongoDBIO_IT.yml | 2 -
.../beam_PerformanceTests_ParquetIOIT.yml | 2 -
.../beam_PerformanceTests_ParquetIOIT_HDFS.yml | 2 -
...erformanceTests_PubsubIOIT_Python_Streaming.yml | 2 -
...m_PerformanceTests_SQLBigQueryIO_Batch_Java.yml | 2 -
..._PerformanceTests_SpannerIO_Read_2GB_Python.yml | 2 -
...manceTests_SpannerIO_Write_2GB_Python_Batch.yml | 2 -
.../beam_PerformanceTests_SparkReceiver_IO.yml | 2 -
.../beam_PerformanceTests_TFRecordIOIT.yml | 2 -
.../beam_PerformanceTests_TFRecordIOIT_HDFS.yml | 2 -
.../workflows/beam_PerformanceTests_TextIOIT.yml | 2 -
.../beam_PerformanceTests_TextIOIT_HDFS.yml | 2 -
.../beam_PerformanceTests_TextIOIT_Python.yml | 2 -
...PerformanceTests_WordCountIT_PythonVersions.yml | 2 -
.../workflows/beam_PerformanceTests_XmlIOIT.yml | 2 -
.../beam_PerformanceTests_XmlIOIT_HDFS.yml | 2 -
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 2 -
.../beam_PostCommit_BeamMetrics_Publish.yml | 2 -
.github/workflows/beam_PostCommit_Go.yml | 2 -
.github/workflows/beam_PostCommit_Go_VR_Flink.yml | 2 -
.github/workflows/beam_PostCommit_Go_VR_Samza.yml | 2 -
.github/workflows/beam_PostCommit_Go_VR_Spark.yml | 2 -
.github/workflows/beam_PostCommit_Java.yml | 2 -
.../beam_PostCommit_Java_Avro_Versions.yml | 2 -
.../beam_PostCommit_Java_BigQueryEarlyRollout.yml | 2 -
.../workflows/beam_PostCommit_Java_DataflowV1.yml | 2 -
.../workflows/beam_PostCommit_Java_DataflowV2.yml | 2 -
...beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 -
.../beam_PostCommit_Java_Examples_Dataflow_V2.yml | 2 -
...m_PostCommit_Java_Examples_Dataflow_V2_Java.yml | 2 -
.../beam_PostCommit_Java_Examples_Direct.yml | 2 -
.../beam_PostCommit_Java_Examples_Flink.yml | 2 -
.../beam_PostCommit_Java_Examples_Spark.yml | 2 -
.../beam_PostCommit_Java_Hadoop_Versions.yml | 2 -
.../beam_PostCommit_Java_InfluxDbIO_IT.yml | 2 -
.../beam_PostCommit_Java_Jpms_Dataflow_Java11.yml | 2 -
.../beam_PostCommit_Java_Jpms_Dataflow_Java17.yml | 2 -
.../beam_PostCommit_Java_Jpms_Direct_Java11.yml | 2 -
.../beam_PostCommit_Java_Jpms_Direct_Java17.yml | 2 -
.../beam_PostCommit_Java_Jpms_Flink_Java11.yml | 2 -
.../beam_PostCommit_Java_Jpms_Spark_Java11.yml | 2 -
.../beam_PostCommit_Java_Nexmark_Dataflow.yml | 2 -
.../beam_PostCommit_Java_Nexmark_Dataflow_V2.yml | 2 -
...am_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml | 2 -
.../beam_PostCommit_Java_Nexmark_Direct.yml | 2 -
.../beam_PostCommit_Java_Nexmark_Flink.yml | 2 -
.../beam_PostCommit_Java_Nexmark_Spark.yml | 2 -
.../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 -
.../workflows/beam_PostCommit_Java_PVR_Samza.yml | 2 -
.../beam_PostCommit_Java_PVR_Spark3_Streaming.yml | 2 -
.../beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 -
.github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 -
.../beam_PostCommit_Java_SingleStoreIO_IT.yml | 2 -
.../beam_PostCommit_Java_Tpcds_Dataflow.yml | 2 -
.../workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 -
.../workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 -
...am_PostCommit_Java_ValidatesRunner_Dataflow.yml | 2 -
..._Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 -
...mit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 -
...PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 -
..._Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 -
...beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 -
...it_Java_ValidatesRunner_Direct_JavaVersions.yml | 2 -
.../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 2 -
...ostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 -
.../beam_PostCommit_Java_ValidatesRunner_Samza.yml | 2 -
.../beam_PostCommit_Java_ValidatesRunner_Spark.yml | 2 -
...va_ValidatesRunner_SparkStructuredStreaming.yml | 2 -
...ostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 -
...am_PostCommit_Java_ValidatesRunner_Twister2.yml | 2 -
.../beam_PostCommit_Java_ValidatesRunner_ULR.yml | 2 -
.github/workflows/beam_PostCommit_Javadoc.yml | 2 -
.../beam_PostCommit_PortableJar_Flink.yml | 2 -
.../beam_PostCommit_PortableJar_Spark.yml | 2 -
.github/workflows/beam_PostCommit_Python.yml | 2 -
.../beam_PostCommit_Python_Examples_Dataflow.yml | 2 -
.../beam_PostCommit_Python_Examples_Direct.yml | 2 -
.../beam_PostCommit_Python_Examples_Flink.yml | 2 -
.../beam_PostCommit_Python_Examples_Spark.yml | 2 -
.../beam_PostCommit_Python_MongoDBIO_IT.yml | 2 -
.../beam_PostCommit_Python_Nexmark_Direct.yml | 2 -
...stCommit_Python_ValidatesContainer_Dataflow.yml | 2 -
..._Python_ValidatesContainer_Dataflow_With_RC.yml | 6 +-
..._PostCommit_Python_ValidatesRunner_Dataflow.yml | 2 -
...eam_PostCommit_Python_ValidatesRunner_Flink.yml | 2 -
...eam_PostCommit_Python_ValidatesRunner_Samza.yml | 2 -
...eam_PostCommit_Python_ValidatesRunner_Spark.yml | 2 -
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml | 2 -
.../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 2 -
.../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 2 -
.github/workflows/beam_PostCommit_SQL.yml | 2 -
.../workflows/beam_PostCommit_Sickbay_Python.yml | 2 -
.../beam_PostCommit_TransformService_Direct.yml | 2 -
.github/workflows/beam_PostCommit_Website_Test.yml | 2 -
.github/workflows/beam_PostCommit_XVR_Direct.yml | 2 -
.github/workflows/beam_PostCommit_XVR_Flink.yml | 2 -
.../beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 2 -
...eam_PostCommit_XVR_JavaUsingPython_Dataflow.yml | 2 -
..._PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml | 2 -
...eam_PostCommit_XVR_PythonUsingJava_Dataflow.yml | 2 -
.github/workflows/beam_PostCommit_XVR_Samza.yml | 2 -
.github/workflows/beam_PostCommit_XVR_Spark3.yml | 2 -
.../workflows/beam_Publish_Docker_Snapshots.yml | 2 -
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 2 -
.github/workflows/build_release_candidate.yml | 76 +-
...ava_CoGBK_Dataflow_Batch_100b_Multiple_Keys.txt | 27 +
.../java_CoGBK_Dataflow_Batch_100b_Single_Key.txt | 27 +
.../java_CoGBK_Dataflow_Batch_10kB.txt | 27 +
.../java_CoGBK_Dataflow_Batch_2MB.txt | 27 +
...taflow_V2_Streaming_Java_100b_Multiple_Keys.txt | 29 +
..._Dataflow_V2_Streaming_Java_100b_Single_Key.txt | 29 +
.../java_CoGBK_Dataflow_V2_Streaming_Java_10kB.txt | 29 +
.../java_CoGBK_Dataflow_V2_Streaming_Java_2MB.txt | 29 +
...tructuredStreaming_Batch_100b_Multiple_Keys.txt | 25 +
...rkStructuredStreaming_Batch_100b_Single_Key.txt | 25 +
...a_CoGBK_SparkStructuredStreaming_Batch_10kB.txt | 25 +
...va_CoGBK_SparkStructuredStreaming_Batch_2MB.txt | 25 +
...hon_CoGBK_Dataflow_Batch_100b_Multiple_Keys.txt | 28 +
...python_CoGBK_Dataflow_Batch_100b_Single_Key.txt | 28 +
.../python_CoGBK_Dataflow_Batch_10kB.txt | 28 +
.../python_CoGBK_Dataflow_Batch_2MB.txt | 28 +
...GBK_Dataflow_Flink_Batch_100b_Multiple_Keys.txt | 28 +
..._CoGBK_Dataflow_Flink_Batch_100b_Single_Key.txt | 28 +
.../python_CoGBK_Dataflow_Flink_Batch_10kB.txt | 28 +
...CoGBK_Dataflow_Streaming_100b_Multiple_Keys.txt | 30 +
...on_CoGBK_Dataflow_Streaming_100b_Single_Key.txt | 30 +
.../python_CoGBK_Dataflow_Streaming_10kB.txt | 30 +
.../python_CoGBK_Dataflow_Streaming_2MB.txt | 30 +
.github/workflows/pr-bot-pr-updates.yml | 1 +
.github/workflows/self-assign.yml | 2 +-
.test-infra/metrics/docker-compose.yml | 5 +-
.test-infra/metrics/sync/github/requirements.txt | 6 +-
.test-infra/metrics/sync/github/sync_workflows.py | 400 +--
build.gradle.kts | 4 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 30 +-
examples/notebooks/healthcare/beam_nlp.ipynb | 2 +-
.../scripts/download_github_actions_artifacts.py | 15 +-
.../core/construction/TransformUpgrader.java | 2 +-
.../UnboundedReadFromBoundedSource.java | 14 +-
.../UnboundedReadFromBoundedSourceTest.java | 180 ++
runners/flink/flink_runner.gradle | 23 +-
.../dataflow/worker/StreamingDataflowWorker.java | 2 +
.../worker/StreamingModeExecutionContext.java | 483 ++--
.../beam/runners/dataflow/worker/Weighers.java | 2 +-
.../dataflow/worker/WindmillStateInternals.java | 2830 --------------------
.../dataflow/worker/WorkerCustomSources.java | 7 +-
.../dataflow/worker/streaming/ActiveWorkState.java | 2 +-
.../worker/streaming/ComputationState.java | 2 +-
.../worker/windmill/state/CachingStateTable.java | 275 ++
.../worker/windmill/state/ConcatIterables.java | 55 +
.../dataflow/worker/windmill/state/IdTracker.java | 253 ++
.../worker/windmill/state/PagingIterable.java | 129 +
.../dataflow/worker/windmill/state/RangeCoder.java | 76 +
.../worker/windmill/state/RangeSetCoder.java | 46 +
.../state/SimpleWindmillState.java} | 33 +-
.../dataflow/worker/windmill/state/StateTag.java | 109 +
.../windmill/state/TimestampedValueWithId.java | 42 +
.../worker/windmill/state/ToIterableFunction.java | 79 +
.../windmill/state/ValuesAndContPosition.java | 46 +
.../state/WeightedList.java} | 45 +-
.../worker/windmill/state/WindmillBag.java | 209 ++
.../windmill/state/WindmillCombiningState.java | 170 ++
.../worker/windmill/state/WindmillMap.java | 449 ++++
.../worker/windmill/state/WindmillMultimap.java | 732 +++++
.../worker/windmill/state/WindmillOrderedList.java | 305 +++
.../worker/windmill/state/WindmillSet.java | 146 +
.../worker/windmill/state/WindmillState.java | 68 +
.../{ => windmill/state}/WindmillStateCache.java | 338 +--
.../windmill/state/WindmillStateInternals.java | 171 ++
.../{ => windmill/state}/WindmillStateReader.java | 530 +---
.../worker/windmill/state/WindmillStateUtil.java | 51 +
.../worker/windmill/state/WindmillValue.java | 160 ++
.../windmill/state/WindmillWatermarkHold.java | 259 ++
.../worker/windmill/state/WrappedFuture.java | 64 +
.../worker/StreamingModeExecutionContextTest.java | 2 +
.../worker/streaming/ActiveWorkStateTest.java | 2 +-
.../state}/WindmillStateCacheTest.java | 135 +-
.../state}/WindmillStateInternalsTest.java | 282 +-
.../state}/WindmillStateReaderTest.java | 6 +-
sdks/go.mod | 60 +-
sdks/go.sum | 127 +-
sdks/go/pkg/beam/runners/prism/internal/web/web.go | 40 +-
sdks/go/test/integration/integration.go | 7 -
sdks/go/test/integration/io/fhirio/fhirio_test.go | 19 +-
.../extensions/python/PythonExternalTransform.java | 39 +-
.../schemaio-expansion-service/build.gradle | 6 +
.../csv/providers/CsvWriteTransformProvider.java | 145 +
.../beam/sdk/io/csv/providers/package-info.java | 23 +-
.../beam/sdk/io/gcp/bigtable/BigtableIOTest.java | 2 +-
sdks/java/io/json/build.gradle | 35 +
.../java/org/apache/beam/sdk/io/json/JsonIO.java | 283 ++
.../org/apache/beam/sdk/io/json/package-info.java | 23 +-
.../json/providers/JsonWriteTransformProvider.java | 142 +
.../beam/sdk/io/json/providers/package-info.java | 23 +-
.../apache/beam/sdk/io/json/JsonIOWriteTest.java | 145 +
sdks/java/maven-archetypes/examples/build.gradle | 7 +
.../maven-archetypes/gcp-bom-examples/build.gradle | 6 +
sdks/java/transform-service/docker-compose/.env | 8 +
.../docker-compose/docker-compose.yml | 3 +-
sdks/java/transform-service/launcher/build.gradle | 3 +
.../launcher/TransformServiceLauncher.java | 135 +-
.../launcher/TransformServiceLauncherTest.java | 185 ++
.../sdk/transformservice/ExpansionService.java | 114 +-
.../sdk/transformservice/ExpansionServiceTest.java | 7 +-
sdks/python/.pylintrc | 1 -
sdks/python/apache_beam/io/avroio.py | 271 +-
sdks/python/apache_beam/io/avroio_test.py | 31 +-
sdks/python/apache_beam/io/gcp/bigquery.py | 2 +-
sdks/python/apache_beam/io/parquetio.py | 99 +-
sdks/python/apache_beam/io/parquetio_test.py | 16 +
sdks/python/apache_beam/ml/transforms/base.py | 39 +-
sdks/python/apache_beam/ml/transforms/base_test.py | 25 +
sdks/python/apache_beam/pipeline.py | 28 +-
sdks/python/apache_beam/pvalue.py | 3 +
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../apache-beam-jupyterlab-sidepanel/yarn.lock | 33 +-
.../Linux/29c9237ddf4f3d5988a503069b4d3c47.png | Bin 65527 -> 64875 bytes
.../Linux/7a35f487b2a5f3a9b9852a8659eeb4bd.png | Bin 728485 -> 687768
bytes
.../portability/flink_uber_jar_job_server.py | 4 +-
.../apache_beam/runners/worker/bundle_processor.py | 9 +-
.../apache_beam/runners/worker/sdk_worker_main.py | 4 +-
.../apache_beam/testing/analyzers/__init__.py | 5 +
.../apache_beam/testing/analyzers/constants.py | 1 +
.../testing/analyzers/github_issues_utils.py | 63 +-
.../apache_beam/testing/analyzers/perf_analysis.py | 150 +-
.../testing/analyzers/perf_analysis_test.py | 79 +-
.../testing/analyzers/perf_analysis_utils.py | 172 +-
.../testing/analyzers/tests_config.yaml | 49 +-
.../testing/benchmarks/nexmark/nexmark_launcher.py | 3 +-
.../apache_beam/testing/load_tests/build.gradle | 2 +-
.../testing/load_tests/load_test_metrics_utils.py | 3 +-
sdks/python/apache_beam/transforms/display.py | 23 +-
sdks/python/apache_beam/transforms/ptransform.py | 16 +
.../typehints/arrow_type_compatibility.py | 6 +-
.../typehints/arrow_type_compatibility_test.py | 2 +-
sdks/python/apache_beam/typehints/schemas.py | 4 +
.../utils/transform_service_launcher.py | 14 +
sdks/python/apache_beam/yaml/json_utils.py | 183 ++
sdks/python/apache_beam/yaml/main.py | 17 +-
sdks/python/apache_beam/yaml/standard_io.yaml | 28 +
sdks/python/apache_beam/yaml/yaml_io.py | 16 +-
sdks/python/apache_beam/yaml/yaml_io_test.py | 123 +
sdks/python/apache_beam/yaml/yaml_provider.py | 6 +-
sdks/python/expansion-service-container/boot.go | 102 +-
sdks/python/setup.py | 4 +-
settings.gradle.kts | 1 +
.../documentation/io/built-in/google-bigquery.md | 14 +-
.../en/documentation/runtime/environments.md | 3 +-
300 files changed, 8951 insertions(+), 5126 deletions(-)
copy .github/workflows/{beam_LoadTests_Java_ParDo_Dataflow_Batch.yml =>
beam_LoadTests_Java_CoGBK_Dataflow_Batch.yml} (64%)
create mode 100644
.github/workflows/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_JavaVersions.yml
copy .github/workflows/{beam_LoadTests_Java_GBK_Dataflow_Batch.yml =>
beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_JavaVersions.yml} (51%)
copy
.github/workflows/{beam_LoadTests_Java_ParDo_SparkStructuredStreaming_Batch.yml
=> beam_LoadTests_Java_CoGBK_SparkStructuredStreaming_Batch.yml} (64%)
copy .github/workflows/{beam_LoadTests_Python_GBK_Dataflow_Batch.yml =>
beam_LoadTests_Python_CoGBK_Dataflow_Batch.yml} (60%)
copy .github/workflows/{beam_LoadTests_Python_GBK_Dataflow_Batch.yml =>
beam_LoadTests_Python_CoGBK_Dataflow_Streaming.yml} (60%)
copy .github/workflows/{beam_LoadTests_Python_Combine_Flink_Batch.yml =>
beam_LoadTests_Python_CoGBK_Flink_Batch.yml} (68%)
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_Batch_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_Batch_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_Batch_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_Batch_2MB.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_V2_Streaming_Java_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_V2_Streaming_Java_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_V2_Streaming_Java_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_Dataflow_V2_Streaming_Java_2MB.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_SparkStructuredStreaming_Batch_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_SparkStructuredStreaming_Batch_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_SparkStructuredStreaming_Batch_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_CoGBK_SparkStructuredStreaming_Batch_2MB.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Batch_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Batch_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Batch_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Batch_2MB.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Flink_Batch_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Flink_Batch_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Flink_Batch_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Streaming_100b_Multiple_Keys.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Streaming_100b_Single_Key.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Streaming_10kB.txt
create mode 100644
.github/workflows/load-tests-job-configs/python-cogbk/python_CoGBK_Dataflow_Streaming_2MB.txt
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillStateInternals.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/CachingStateTable.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/ConcatIterables.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/IdTracker.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/PagingIterable.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/RangeCoder.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/RangeSetCoder.java
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/{Weighers.java
=> windmill/state/SimpleWindmillState.java} (50%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/StateTag.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/TimestampedValueWithId.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/ToIterableFunction.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/ValuesAndContPosition.java
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/{Weighers.java
=> windmill/state/WeightedList.java} (53%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillBag.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillCombiningState.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMap.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMultimap.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillOrderedList.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillSet.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillState.java
rename
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/{
=> windmill/state}/WindmillStateCache.java (92%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillStateInternals.java
rename
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/{
=> windmill/state}/WindmillStateReader.java (70%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillStateUtil.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillValue.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillWatermarkHold.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WrappedFuture.java
rename
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/{
=> windmill/state}/WindmillStateCacheTest.java (75%)
rename
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/{
=> windmill/state}/WindmillStateInternalsTest.java (97%)
rename
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/{
=> windmill/state}/WindmillStateReaderTest.java (99%)
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/providers/CsvWriteTransformProvider.java
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/Weighers.java
=>
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/providers/package-info.java
(53%)
create mode 100644 sdks/java/io/json/build.gradle
create mode 100644
sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/JsonIO.java
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/Weighers.java
=>
sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/package-info.java
(53%)
create mode 100644
sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/JsonWriteTransformProvider.java
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/Weighers.java
=>
sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/package-info.java
(53%)
create mode 100644
sdks/java/io/json/src/test/java/org/apache/beam/sdk/io/json/JsonIOWriteTest.java
create mode 100644
sdks/java/transform-service/launcher/src/test/java/org/apache/beam/sdk/transformservice/launcher/TransformServiceLauncherTest.java
create mode 100644 sdks/python/apache_beam/yaml/json_utils.py
