This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.17
in repository https://gitbox.apache.org/repos/asf/beam.git
discard cb14c3c75b6 Bump github.com/nats-io/nats-server/v2 from 2.10.12 to
2.10.17 in /sdks
add 6ec1fb23ece move heartbeat processor to where it is being used (#31298)
add 3588d195335 Add spark mapstate (#31669)
add fb1e9cf72d4 Allow override beam version for PythonExternalTransform
via pipeline option (#31691)
add d80a58c09f3 Eliminate the use of testRuntimeMigration for
sdks:java:io:common (#31693)
add 20aa916931f Properly close Storage API batch connections (#31710)
add f8b63ff3d55 Create CsvIOParseResult (#31706)
add 2f81e16f3ea [#31403] Python wrapper to download, use, or build and run
prism. (#31583)
add 9bdcb672d08 Create CsvIOParseConfiguration class (#31714)
add c3756c04e39 Remove excessive logging in test. (#31715)
add 5a090958992 Move CsvIOParseResult (#31722)
add ef5060416d9 [Dataflow Streaming] Enabled Heartbeat by Default (#31689)
add bae6fcfcb49 Add a unit test for MakePipelineOptionsFileAndEnvVar
(#31732)
add 47384772cac Dont read from cache in sensitive workflows (#31734)
add a3a62b866e0 Add scaffold CsvIOParseHelpers (#31720)
add 2c7211351f5 Remove testRuntimeMigration configuration for test-utils
dependencies (#31713)
add 957a7cd22b6 created CsvIOReadFiles class (#31738)
add 3b8ddda10a0 Solace Read connector: adding Basic Authentication support
(#31541)
add bacb9ec48b5 Set SchemaCoder for key in WithKeys transform
add bf8afb8d580 Merge pull request #31711: Set SchemaCoder for key in
WithKeys transform
add 5805734b3fc Disable caching for one more workflow (#31744)
add be900f940d7 Update joda-time to 2.10.14 (#31743)
add 45733be4fe4 Do not build shadowjar and disable javadoc for it modules
(#31741)
add 8e11d0a4815 Update Go Version to 1.22.4 (#31747)
add 209d50e6765 Support custom JdbcReadWithPartitionsHelper (#31733)
add dc63b0ffc99 Update Staticcheck to 2023.1.7 (#31750)
add b5035a08836 Update Playground Python Dockerfile (#31745)
add 366db73bf19 Disable SoftDeletePolicy when creating a default bucket
(#31748)
add e28e00ac608 Fix installation invocation for OpenJDK (#31752)
add 8a4a250076b Updates Dataflow service API version. (#31751)
add 1e873f42e14 Use highmem runner for
beam_PostRelease_NightlySnapshot.yml (#31749)
add b68b29a3ff6 Basic yaml-defined provider.
add d2df083a029 Refactor jinja templatiziation to common location.
add 3212688e2e6 Merge pull request #31684 Basic yaml-defined provider.
add d6139904db8 Use GCP libraries-bom version for all grpc (#31760)
add f15ca986811 Use ByteBuffer instead of BytesString which is unsupported
in Schema Coders (#31746)
add dbe72830b11 Add a test for getting state with MultimapSideInput
StateKey (#31757)
add f45b5d88e1d Add SerializableSupplier to the core beam.sdk.util package
(#31766)
add 54b882d285e Replace LGPL dep in Go SDK with an MIT alternative (#31769)
add c1ca5156e64 Polish DoFn.Setup
add dea440f46d5 Merge pull request #31764 from liferoad/polish-dofn
add a1a22835710 Fix CHANGES.md from #31769 which incorrectly added to a
released version (#31770)
add 0c89a0edb9c Fix playground snippets (#31778)
add a5eee589697 Fix flaky StreamingDataflowWorkerTest which wasn't waiting
for enough commits. (#31781)
add c08afeae60d Enable MapState and SetState for dataflow streaming engine
pipelines with legacy runner by building on top of MultimapState. (#31453)
add d1df1d7ecc9 Bump cloud.google.com/go/storage from 1.41.0 to 1.43.0 in
/sdks (#31772)
add 02600f55d21 Set Snowflake escape char to backslash since it is the
default used by CSVParser (fixes #24467) (#31779)
add ac423af5699 Pass-through IcebergIO catalog properties (#31726)
add 631d40d0e79 Stage PrismRunner implementation and dependencies (#31794)
add 8a88f1583f0 Solace Read connector: adding implementations of
SempClient and SempClientFactory (#31542)
add 746f3c5557e Use go 1.22 for self-hosted GHAs (#31767)
add de4645d4507 Add support for StringSet metric in Java SDK. (#31789)
add 516bbc77ef3 Add support for WindowStrategy Pane and AllowedLatness
features (#31806)
add dc10b77ce00 Update Go Version to 1.22.5 (#31812)
add 9721aca8f50 Fix PostCommit Java ValidatesRunner Samza job (#31773)
add 1db2373debc correctly close the javadoc tag in JmsIO.Write (#31801)
add 88aa25391ec Solace Read connector: integration tests with
testcontainers (#31543)
add 9cbdda1b4e5 add in redistribute option for Kafka Read
add cf37997d1dd Merge pull request #31347: Add in redistribute option for
Kafka Read
add e15cd9e040e Don't cache when building release candidates (#31810)
add ef143aed418 Add link to security model (#31811)
add 78bab0dd15e Avoid length-prefix-bytes substitutions for Flink
boundaries.
add dda0fbf57be Merge pull request #31579 Avoid length-prefix-bytes
substitutions for Flink boundaries.
add fa9c618cdbd Allow pr-bot to re-assign reviewers when stopped (#31436)
add a4558dfd8c8 Bump certifi from 2024.2.2 to 2024.7.4 in
/sdks/python/container/py38 (#31790)
add 566a3ca96c4 Publish and export Javadoc for Solace (#31809)
add 81538672cfe Support class executes the Prism binary (#31795)
add b12943380b5 Exclude StringSet tests from portable runners and Dataflow
LegacyRunner (#31818)
add f72f6ce0e81 Remove CsvIOParseResult (#31819)
add 5579a16de7d Introduce support for emitting lineage in BiqQueryIOs
add dded4f06d82 Be spotless
add 024692647b4 A couple improvements to BQ source lineage.
add c827bbac387 Update contains test.
add c9adc8ee6c6 Merge pull request #31805 Introduce support for emitting
lineage in BQ Source.
add 018bcdf592c Add missing params to Python Bigtable MutationsBatcher
(#31791)
add a2260949431 Avoid publishing string set metrics on the Dataflow legacy
runner. (#31825)
add b9a44126622 Add changelog notes regarding Solace read connector
(#31826)
add 6c829db657b Bump google.golang.org/grpc from 1.64.0 to 1.64.1 in /sdks
(#31817)
add 4df89c704b0 Allow Firestore project to be configurable (#31808)
add 080c80a9573 Moving to 2.59.0-SNAPSHOT on master branch.
add 43b237e5040 Modernize python type hints for apache_beam
add cd495e9cf8a Modernize python type hints for apache_beam/coders
add 6143cd0e0a8 Modernize python type hints for apache_beam/dataframe
add d75916b6342 Modernize python type hints for
apache_beam/examples/cookbook
add c842252b4b5 Modernize python type hints for apache_beam/internal
add 33bde4de9c5 Modernize python type hints for
apache_beam/internal/metrics
add d73982af054 Modernize python type hints for apache_beam/io
add 8f6f24dc1ff Modernize python type hints for apache_beam/io/azure
add b8029e9aa1e Modernize python type hints for apache_beam/io/flink
add f49a29a4685 Modernize python type hints for apache_beam/io/gcp
add 5d5a09b6b3e Modernize python type hints for apache_beam/metrics
add fbafe8d780b Modernize python type hints for apache_beam/ml/gcp
add 0eab29802a3 Modernize python type hints for apache_beam/options
add 842b8ecab01 Modernize python type hints for apache_beam/runners
add 0763d7e50f3 Modernize python type hints for apache_beam/runners/direct
add 3cf0c5512a3 Modernize python type hints for
apache_beam/runners/interactive
add b41698210df Modernize python type hints for apache_beam/runners/job
add 8b540eb5075 Modernize python type hints for
apache_beam/runners/portability
add 8fdbe88dc6e Modernize python type hints for apache_beam/runners/worker
add 77d81895109 Modernize python type hints for
apache_beam/testing/benchmarks
add acfd72c7c06 Modernize python type hints for
apache_beam/testing/load_tests
add 4402f2d44e9 Modernize python type hints for apache_beam/transforms
add 79d4ffd20ba Modernize python type hints for apache_beam/typehints
add f2ffa5ec49e Modernize python type hints for apache_beam/utils
add abdb1b742a9 Fix circular references, mypy complaints.
add d4de077a2a9 Fix bad type declarations.
add 14c52d66ec6 Fix bad typing in PubSub tests.
add 64e6194b948 Preserve existing linter comments.
add a0ba8dea7d8 isort
add 36961405769 Merge pull request #31755 Modernize type hints.
add dd0912460c4 add doc warning against using icebergio directly (#31833)
add 7c0cf39001a Merge pull request #31823 Add lineage information for
BigQuery sinks.
add 8d5c3b5ee2c Locate and download Prism binary (#31796)
add 00bf1c6d036 Change orphan file log to warning (#31835)
add 041ccdbe5d0 playground python image update openjdk to 17 (#31843)
add 9ee961fc0c2 Use fileNameTemplate attribute for file prefix (#31844)
add b34c014888b Dedup SerializableSupplier (#31829)
add e646c28d2ac [CsvIO] Implemented CsvIOParseHelpers:parseCell (#31802)
add 50a3403a474 Export string sets in monitoring infos. (#31838)
add 441840afb5b Fix Python test auth (#31850)
add eb59788cf4b [CsvIO] Changed CsvIOParseConfiguration to include DLQ
(#31852)
add b573c8ffc8b Add a little more info on snapshot containers (#31861)
add 4561fd10634 [CsvIO] Implement CsvIOParseHelpers::validate(CSVFormat)
(#31853)
add 4d429dde47b Add options to control number of Storage API connections
when using multiplexing (#31721)
add ff2731bc7e9 Switch to use self-hosted runner for build_wheels action
(#31866)
add ba27c36c073 [CsvIO] implement CsvIOParseHelpers::validate(CSVFormat,
Schema) (#31869)
add 7c07f7ff453 Create CsvIOParse scaffold (#31878)
add 86ea7806eca Remove auth@1 in tests running on self hosted runner
(#31881)
add 36f16314052 [CsvIO] Create scaffold CsvIOParseKV class (#31880)
add faff55c2882 Fix auth for clean up actions (#31888)
add 0b61035f36f Increase retry backoff for Storage API batch to survive
AppendRows quota refill (#31837)
add 408f67cfc83 Create CsvIOParseHelpers::mapFieldPositions method (#31889)
add 3e7614a50e5 Bump braces from 3.0.2 to 3.0.3 in /scripts/ci/pr-bot
(#31886)
add ea761115998 [CsvIO] Create CsvIOStringToCsvRecord Class (#31857)
add d74a2f53e47 Fix ReadAllFromBigQuery leak temp dataset (#31895)
add 5ea57c3f8bd Update CHANGES.md with 2.59.0 Section (#31831)
add 2547e0b482e Bump github.com/nats-io/nats-server/v2 from 2.10.12 to
2.10.16 in /sdks (#31611)
add 7658009b996 Bump github.com/nats-io/nats-server/v2 from 2.10.12 to
2.10.17 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (cb14c3c75b6)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.17
(7658009b996)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.github/ACTIONS.md | 4 +
.../test-properties.json | 2 +-
.../actions/setup-environment-action/action.yml | 7 +-
.../arc/images/Dockerfile | 2 +-
.../IO_Iceberg_Integration_Tests.json | 3 +-
.../beam_PostCommit_Java_PVR_Flink_Streaming.json | 2 +-
.../beam_PostCommit_Java_PVR_Samza.json | 3 +-
.../beam_PostCommit_Java_PVR_Spark3_Streaming.json | 2 +-
.../beam_PostCommit_Java_PVR_Spark_Batch.json | 2 +-
.github/workflows/IO_Iceberg_Integration_Tests.yml | 6 -
.github/workflows/IO_Iceberg_Performance_Tests.yml | 6 -
.github/workflows/IO_Iceberg_Unit_Tests.yml | 6 -
.github/workflows/beam_CancelStaleDataflowJobs.yml | 6 -
.github/workflows/beam_CleanUpGCPResources.yml | 9 +-
.../workflows/beam_CleanUpPrebuiltSDKImages.yml | 6 -
.../beam_PerformanceTests_AvroIOIT_HDFS.yml | 6 -
.github/workflows/beam_PerformanceTests_Cdap.yml | 6 -
...m_PerformanceTests_Compressed_TextIOIT_HDFS.yml | 6 -
.../beam_PerformanceTests_HadoopFormat.yml | 6 -
.github/workflows/beam_PerformanceTests_JDBC.yml | 6 -
.../workflows/beam_PerformanceTests_Kafka_IO.yml | 6 -
...am_PerformanceTests_ManyFiles_TextIOIT_HDFS.yml | 6 -
.../beam_PerformanceTests_MongoDBIO_IT.yml | 6 -
.../beam_PerformanceTests_ParquetIOIT_HDFS.yml | 6 -
.../beam_PerformanceTests_SingleStoreIO.yml | 6 -
.../beam_PerformanceTests_SparkReceiver_IO.yml | 6 -
.../beam_PerformanceTests_TFRecordIOIT_HDFS.yml | 6 -
.../beam_PerformanceTests_XmlIOIT_HDFS.yml | 6 -
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 6 -
.../beam_PostCommit_Java_InfluxDbIO_IT.yml | 6 -
.../beam_PostCommit_Java_SingleStoreIO_IT.yml | 6 -
.../workflows/beam_PostRelease_NightlySnapshot.yml | 2 +-
.../beam_PreCommit_Java_Solace_IO_Direct.yml | 2 +-
.github/workflows/beam_Publish_Website.yml | 2 -
.github/workflows/beam_Release_NightlySnapshot.yml | 3 +-
.../workflows/beam_StressTests_Java_KafkaIO.yml | 6 -
.github/workflows/build_release_candidate.yml | 5 +-
.github/workflows/build_runner_image.yml | 8 -
.github/workflows/build_wheels.yml | 52 +-
.../workflows/deploy_release_candidate_pypi.yaml | 3 +-
.github/workflows/finalize_release.yml | 3 +-
.github/workflows/go_tests.yml | 9 +-
.github/workflows/java_tests.yml | 6 +-
.github/workflows/local_env_tests.yml | 18 +-
.github/workflows/python_tests.yml | 7 +-
.github/workflows/typescript_tests.yml | 6 +-
.github/workflows/update_python_dependencies.yml | 4 +-
.test-infra/tools/stale_bq_datasets_cleaner.sh | 4 +-
CHANGES.md | 43 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 18 +-
contributor-docs/code-change-guide.md | 11 +
dev-support/docker/Dockerfile | 2 +-
gradle.properties | 4 +-
it/cassandra/build.gradle | 14 +-
.../it/cassandra/CassandraResourceManager.java | 4 +-
it/common/build.gradle | 2 -
it/conditions/build.gradle | 3 +-
it/elasticsearch/build.gradle | 12 +-
it/google-cloud-platform/build.gradle | 23 +-
it/jdbc/build.gradle | 14 +-
it/kafka/build.gradle | 13 +-
it/mongodb/build.gradle | 16 +-
it/neo4j/build.gradle | 11 +-
it/splunk/build.gradle | 17 +-
it/testcontainers/build.gradle | 8 +-
it/truthmatchers/build.gradle | 9 +-
.../apache/beam/model/pipeline/v1/metrics.proto | 19 +
playground/backend/containers/python/Dockerfile | 3 +-
.../org/apache/beam/runners/core/StateTags.java | 8 +
.../runners/core/metrics/DefaultMetricResults.java | 14 +-
.../beam/runners/core/metrics/MetricUpdates.java | 34 +-
.../runners/core/metrics/MetricsContainerImpl.java | 106 +++-
.../core/metrics/MetricsContainerStepMap.java | 6 +
.../core/metrics/MonitoringInfoConstants.java | 4 +
.../core/metrics/MonitoringInfoEncodings.java | 26 +
.../core/metrics/SimpleMonitoringInfoBuilder.java | 11 +
.../beam/runners/core/metrics/StringSetCell.java | 111 ++++
.../beam/runners/core/metrics/StringSetData.java | 97 ++++
.../beam/runners/core/StateInternalsTest.java | 6 +
.../core/metrics/MetricsContainerImplTest.java | 46 ++
.../core/metrics/MetricsContainerStepMapTest.java | 102 ++++
.../core/metrics/MonitoringInfoEncodingsTest.java | 28 +
.../runners/core/metrics/StringSetCellTest.java | 97 ++++
.../runners/core/metrics/StringSetDataTest.java | 102 ++++
.../apache/beam/runners/direct/DirectMetrics.java | 45 +-
.../beam/runners/direct/DirectMetricsTest.java | 26 +-
.../metrics/CustomMetricQueryResults.java | 11 +
.../extensions/metrics/MetricsHttpSinkTest.java | 7 +-
runners/flink/job-server/flink_job_server.gradle | 1 +
.../flink/adapter/BeamAdapterCoderUtils.java | 16 +
.../runners/flink/adapter/BeamAdapterUtils.java | 22 +
.../flink/adapter/BeamFlinkDataSetAdapter.java | 1 -
.../flink/adapter/BeamFlinkDataStreamAdapter.java | 1 -
.../flink/adapter/BeamFlinkDataSetAdapterTest.java | 50 ++
runners/google-cloud-dataflow-java/build.gradle | 2 +
.../beam/runners/dataflow/DataflowMetrics.java | 41 +-
.../beam/runners/dataflow/DataflowRunner.java | 35 +-
.../beam/runners/dataflow/DataflowMetricsTest.java | 59 ++
.../beam/runners/dataflow/DataflowRunnerTest.java | 59 --
.../dataflow/worker/BatchModeExecutionContext.java | 24 +-
.../dataflow/worker/DataflowMetricsContainer.java | 6 +
.../worker/MetricsToCounterUpdateConverter.java | 18 +-
.../dataflow/worker/StreamingDataflowWorker.java | 183 ++++---
.../worker/StreamingStepMetricsContainer.java | 32 +-
.../windmill/client/grpc/GrpcWindmillServer.java | 53 +-
.../client/grpc/GrpcWindmillStreamFactory.java | 101 ++--
.../client/grpc/StreamingEngineClient.java | 22 +-
.../windmill/client/grpc/WindmillStreamSender.java | 19 +-
.../worker/windmill/state/AbstractWindmillMap.java | 10 +-
.../worker/windmill/state/CachingStateTable.java | 53 +-
.../worker/windmill/state/WindmillMap.java | 24 +-
.../windmill/state/WindmillMapViaMultimap.java | 164 ++++++
.../worker/windmill/state/WindmillMultimap.java | 4 +-
.../worker/windmill/state/WindmillSet.java | 36 +-
.../worker/windmill/state/WindmillStateCache.java | 46 +-
.../windmill/state/WindmillStateInternals.java | 14 +-
.../worker/BatchModeExecutionContextTest.java | 34 ++
.../worker/StreamingDataflowWorkerTest.java | 2 +-
.../worker/StreamingModeExecutionContextTest.java | 5 +-
.../worker/StreamingStepMetricsContainerTest.java | 60 +++
.../dataflow/worker/WindmillStateTestUtils.java | 2 +-
.../dataflow/worker/WorkerCustomSourcesTest.java | 5 +-
.../client/grpc/GrpcWindmillServerTest.java | 37 +-
.../client/grpc/StreamingEngineClientTest.java | 5 +-
.../client/grpc/WindmillStreamSenderTest.java | 14 +-
.../windmill/state/WindmillStateCacheTest.java | 2 +-
.../windmill/state/WindmillStateInternalsTest.java | 225 +++++++-
.../budget/EvenGetWorkBudgetDistributorTest.java | 3 +-
.../refresh/DispatchedActiveWorkRefresherTest.java | 2 +-
.../wire/LengthPrefixUnknownCoders.java | 18 +-
.../runners/jet/FailedRunningPipelineResults.java | 6 +
.../beam/runners/jet/metrics/JetMetricResults.java | 54 +-
.../runners/jet/metrics/JetMetricsContainer.java | 24 +-
.../beam/runners/jet/metrics/StringSetImpl.java | 51 ++
runners/portability/java/build.gradle | 1 +
.../beam/runners/portability/PortableMetrics.java | 40 +-
.../runners/portability/PortableRunnerTest.java | 17 +
runners/prism/build.gradle | 1 +
.../io/json => runners/prism/java}/build.gradle | 26 +-
.../apache/beam/runners/prism/PrismExecutor.java | 160 ++++++
.../apache/beam/runners/prism/PrismLocator.java | 221 ++++++++
.../beam/runners/prism/PrismPipelineOptions.java | 44 ++
.../org/apache/beam/runners/prism/PrismRunner.java | 86 +++
.../apache/beam/runners/prism/package-info.java | 9 +-
.../beam/runners/prism/PrismExecutorTest.java | 99 ++++
.../beam/runners/prism/PrismLocatorTest.java | 125 +++++
.../apache/beam/runners/prism/PrismRunnerTest.java | 86 +++
runners/samza/build.gradle | 2 +
runners/samza/job-server/build.gradle | 1 +
runners/spark/job-server/spark_job_server.gradle | 2 +
.../spark/stateful/SparkStateInternals.java | 148 ++++-
.../spark/stateful/SparkStateInternalsTest.java | 10 +-
scripts/ci/pr-bot/package-lock.json | 28 +-
scripts/ci/pr-bot/processPrUpdate.ts | 23 +-
scripts/ci/pr-bot/shared/commentStrings.ts | 2 +-
scripts/ci/pr-bot/shared/userCommand.ts | 64 ++-
sdks/go.mod | 21 +-
sdks/go.sum | 48 +-
sdks/go/container/tools/pipeline_options.go | 5 +
...ipeline_options.go => pipeline_options_test.go} | 49 +-
sdks/go/pkg/beam/core/core.go | 2 +-
sdks/go/pkg/beam/core/runtime/xlangx/expand.go | 33 +-
.../beam/runners/prism/internal/handlerunner.go | 4 +
.../prism/internal/jobservices/management.go | 4 +-
.../runners/prism/internal/worker/worker_test.go | 81 ++-
sdks/go/pkg/beam/util/gcsx/gcs.go | 15 +-
sdks/go/pkg/beam/util/gcsx/gcs_test.go | 10 +
sdks/go/run_with_go_version.sh | 2 +-
.../integration/internal/containers/containers.go | 32 +-
.../java/org/apache/beam/sdk/metrics/Lineage.java} | 30 +-
.../beam/sdk/metrics/MetricQueryResults.java | 13 +-
.../org/apache/beam/sdk/metrics/MetricResult.java | 2 +-
.../java/org/apache/beam/sdk/metrics/Metrics.java | 46 ++
.../apache/beam/sdk/metrics/MetricsContainer.java | 6 +
.../org/apache/beam/sdk/metrics/StringSet.java} | 23 +-
.../apache/beam/sdk/metrics/StringSetResult.java | 61 +++
.../java/org/apache/beam/sdk/state/StateSpecs.java | 23 +
.../beam/sdk/testing/SerializableMatchers.java | 14 +-
.../beam/sdk/testing/UsesStringSetMetrics.java} | 14 +-
.../org/apache/beam/sdk/transforms/WithKeys.java | 17 +
.../beam/sdk/util}/SerializableSupplier.java | 2 +-
.../org/apache/beam/sdk/metrics/MetricsTest.java | 65 ++-
.../beam/sdk/metrics/StringSetResultTest.java | 64 +++
.../org/apache/beam/sdk/transforms/ParDoTest.java | 28 +-
.../apache/beam/sdk/transforms/WithKeysTest.java | 52 ++
.../extensions/python/PythonExternalTransform.java | 13 +-
.../python/PythonExternalTransformOptions.java | 10 +
.../beam/sdk/extensions/python/PythonService.java | 19 +-
.../fn/harness/control/ExecutionStateSampler.java | 9 +
.../harness/control/ExecutionStateSamplerTest.java | 22 +
sdks/java/io/amazon-web-services/build.gradle | 2 +-
sdks/java/io/amazon-web-services2/build.gradle | 2 +-
sdks/java/io/bigquery-io-perf-tests/build.gradle | 6 +-
sdks/java/io/cassandra/build.gradle | 2 +-
sdks/java/io/cdap/build.gradle | 5 +-
.../java/org/apache/beam/sdk/io/cdap/CdapIOIT.java | 4 +-
sdks/java/io/common/build.gradle | 17 +-
.../beam/sdk/io/common/DatabaseTestHelper.java | 70 ++-
.../org/apache/beam/sdk/io/common/HashingFn.java | 4 +-
.../org/apache/beam/sdk/io/common/IOITHelper.java | 0
.../beam/sdk/io/common/IOTestPipelineOptions.java | 0
.../beam/sdk/io/common/NetworkTestHelper.java | 0
.../io/common/PostgresIOTestPipelineOptions.java | 0
.../beam/sdk/io/common/SchemaAwareJavaBeans.java | 12 +-
.../org/apache/beam/sdk/io/common/TestRow.java | 17 +-
.../apache/beam/sdk/io/common/package-info.java} | 10 +-
sdks/java/io/csv/build.gradle | 2 +-
.../java/org/apache/beam/sdk/io/csv/CsvIO.java | 63 +++
.../org/apache/beam/sdk/io/csv/CsvIOParse.java | 57 ++
.../beam/sdk/io/csv/CsvIOParseConfiguration.java | 90 ++++
.../apache/beam/sdk/io/csv/CsvIOParseHelpers.java | 158 ++++++
.../org/apache/beam/sdk/io/csv/CsvIOParseKV.java | 40 ++
.../org/apache/beam/sdk/io/csv/CsvIOReadFiles.java | 48 ++
.../beam/sdk/io/csv/CsvIOStringToCsvRecord.java | 61 +++
.../sdk/io/csv/CsvIOParseConfigurationTest.java} | 16 +-
.../beam/sdk/io/csv/CsvIOParseHelpersTest.java | 597 +++++++++++++++++++++
.../apache/beam/sdk/io/csv/CsvIOParseKVTest.java} | 16 +-
.../apache/beam/sdk/io/csv/CsvIOParseTest.java} | 19 +-
.../beam/sdk/io/csv/CsvIOReadFilesTest.java} | 19 +-
.../sdk/io/csv/CsvIOStringToCsvRecordTest.java | 143 +++++
sdks/java/io/debezium/build.gradle | 4 +-
sdks/java/io/elasticsearch/build.gradle | 2 +-
sdks/java/io/file-based-io-tests/build.gradle | 4 +-
.../java/org/apache/beam/sdk/io/avro/AvroIOIT.java | 4 +-
.../apache/beam/sdk/io/parquet/ParquetIOIT.java | 4 +-
.../java/org/apache/beam/sdk/io/text/TextIOIT.java | 8 +-
.../apache/beam/sdk/io/tfrecord/TFRecordIOIT.java | 4 +-
.../java/org/apache/beam/sdk/io/xml/XmlIOIT.java | 4 +-
sdks/java/io/file-schema-transform/build.gradle | 2 +-
sdks/java/io/google-ads/build.gradle | 2 +-
sdks/java/io/google-cloud-platform/build.gradle | 11 +-
.../beam/sdk/io/gcp/bigquery/AppendClientInfo.java | 19 +-
.../beam/sdk/io/gcp/bigquery/BigQueryHelpers.java | 20 +
.../beam/sdk/io/gcp/bigquery/BigQueryOptions.java | 27 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 9 +
.../sdk/io/gcp/bigquery/BigQuerySourceBase.java | 7 +-
.../io/gcp/bigquery/BigQueryStorageSourceBase.java | 13 +-
.../beam/sdk/io/gcp/bigquery/CreateTables.java | 8 +-
.../bigquery/StorageApiWriteUnshardedRecords.java | 21 +-
.../bigquery/StorageApiWritesShardedRecords.java | 6 +
.../beam/sdk/io/gcp/bigquery/WriteRename.java | 6 +
.../beam/sdk/io/gcp/bigquery/WriteTables.java | 6 +
.../dofn/ReadChangeStreamPartitionDoFn.java | 1 +
.../sdk/io/gcp/firestore/FirestoreOptions.java | 11 +
.../sdk/io/gcp/firestore/FirestoreV1ReadFn.java | 9 +-
.../sdk/io/gcp/firestore/FirestoreV1WriteFn.java | 9 +-
.../internal/SubscriptionPartitionLoader.java | 2 +-
.../sdk/io/gcp/bigquery/BigQueryIOReadTest.java | 27 +-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 34 +-
.../gcp/bigquery/StorageApiSinkSchemaUpdateIT.java | 2 +-
.../dofn/ReadChangeStreamPartitionDoFnTest.java | 1 +
.../gcp/firestore/it/FirestoreTestingHelper.java | 5 +-
.../gcp/pubsublite/internal/FakeSerializable.java | 2 +-
.../internal/SubscriptionPartitionLoaderTest.java | 2 +-
sdks/java/io/hadoop-format/build.gradle | 4 +-
.../sdk/io/hadoop/format/HadoopFormatIOIT.java | 4 +-
sdks/java/io/hbase/build.gradle | 2 +-
sdks/java/io/hcatalog/build.gradle | 11 +-
.../beam/sdk/io/iceberg/IcebergCatalogConfig.java | 197 +------
.../org/apache/beam/sdk/io/iceberg/IcebergIO.java | 8 +
.../IcebergReadSchemaTransformProvider.java | 36 +-
.../IcebergSchemaTransformCatalogConfig.java | 107 ----
.../IcebergWriteSchemaTransformProvider.java | 42 +-
.../apache/beam/sdk/io/iceberg/IcebergIOIT.java | 16 +-
.../beam/sdk/io/iceberg/IcebergIOReadTest.java | 11 +-
.../beam/sdk/io/iceberg/IcebergIOWriteTest.java | 31 +-
.../IcebergReadSchemaTransformProviderTest.java | 34 +-
.../IcebergSchemaTransformTranslationTest.java | 49 +-
.../IcebergWriteSchemaTransformProviderTest.java | 34 +-
.../apache/beam/sdk/io/iceberg/ScanSourceTest.java | 28 +-
sdks/java/io/influxdb/build.gradle | 2 +-
sdks/java/io/jdbc/build.gradle | 4 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 64 ++-
.../sdk/io/jdbc/JdbcReadWithPartitionsHelper.java | 48 ++
.../java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java | 46 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java | 3 +-
.../org/apache/beam/sdk/io/jdbc/JdbcIOTest.java | 81 ++-
.../org/apache/beam/sdk/io/jdbc/JdbcUtilTest.java | 11 +-
sdks/java/io/jms/build.gradle | 4 +-
.../java/org/apache/beam/sdk/io/jms/JmsIO.java | 9 +-
.../java/org/apache/beam/sdk/io/jms/CommonJms.java | 5 +-
sdks/java/io/json/build.gradle | 2 +-
sdks/java/io/kafka/build.gradle | 4 +-
.../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 160 +++++-
.../KafkaIOReadImplementationCompatibility.java | 18 +
.../beam/sdk/io/kafka/KafkaIOExternalTest.java | 17 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 4 +-
...KafkaIOReadImplementationCompatibilityTest.java | 4 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOTest.java | 58 +-
.../sdk/io/kafka/upgrade/KafkaIOTranslation.java | 22 +
.../io/kafka/upgrade/KafkaIOTranslationTest.java | 1 +
sdks/java/io/kinesis/build.gradle | 2 +-
sdks/java/io/kudu/build.gradle | 2 +-
sdks/java/io/mongodb/build.gradle | 4 +-
.../apache/beam/sdk/io/mongodb/MongoDBIOIT.java | 4 +-
sdks/java/io/mqtt/build.gradle | 2 +-
sdks/java/io/neo4j/build.gradle | 4 +-
sdks/java/io/rabbitmq/build.gradle | 2 +-
sdks/java/io/redis/build.gradle | 2 +-
sdks/java/io/rrio/build.gradle | 2 +-
.../org/apache/beam/io/requestresponse/Call.java | 1 +
.../DefaultSerializableBackoffSupplier.java | 1 +
.../beam/io/requestresponse/RequestResponseIO.java | 1 +
.../requestresponse/WindowedCallShouldBackoff.java | 1 +
.../io/requestresponse/RequestResponseIOTest.java | 1 +
.../WindowedCallShouldBackoffTest.java | 1 +
sdks/java/io/singlestore/build.gradle | 5 +-
.../io/singlestore/SingleStoreIOPerformanceIT.java | 4 +-
sdks/java/io/snowflake/build.gradle | 4 +-
.../apache/beam/sdk/io/snowflake/SnowflakeIO.java | 6 +-
.../services/SnowflakeBatchServiceImpl.java | 6 +-
sdks/java/io/solace/build.gradle | 15 +-
.../broker/BasicAuthJcsmpSessionService.java | 148 +++++
.../BasicAuthJcsmpSessionServiceFactory.java | 74 +++
.../sdk/io/solace/broker/BasicAuthSempClient.java | 102 ++++
.../solace/broker/BasicAuthSempClientFactory.java | 82 +++
.../beam/sdk/io/solace/broker/BrokerResponse.java | 62 +++
.../solace/broker/SempBasicAuthClientExecutor.java | 217 ++++++++
.../io/solace/broker/SolaceMessageReceiver.java | 72 +++
.../org/apache/beam/sdk/io/solace/data/Semp.java | 74 +++
.../org/apache/beam/sdk/io/solace/data/Solace.java | 15 +-
.../broker/SempBasicAuthClientExecutorTest.java | 202 +++++++
.../beam/sdk/io/solace/data/SolaceDataUtils.java | 5 +-
.../sdk/io/solace/it/SolaceContainerManager.java | 188 +++++++
.../apache/beam/sdk/io/solace/it/SolaceIOIT.java | 129 +++++
sdks/java/io/sparkreceiver/2/build.gradle | 7 +-
.../sdk/io/sparkreceiver/SparkReceiverIOIT.java | 4 +-
sdks/java/testing/nexmark/build.gradle | 2 +-
sdks/java/testing/tpcds/build.gradle | 2 +-
sdks/python/apache_beam/coders/observable_test.py | 2 +-
sdks/python/apache_beam/coders/row_coder.py | 3 +-
sdks/python/apache_beam/coders/slow_stream.py | 32 +-
.../apache_beam/coders/standard_coders_test.py | 2 +-
sdks/python/apache_beam/coders/typecoders.py | 20 +-
sdks/python/apache_beam/dataframe/convert.py | 38 +-
sdks/python/apache_beam/dataframe/frame_base.py | 20 +-
sdks/python/apache_beam/dataframe/partitionings.py | 9 +-
sdks/python/apache_beam/dataframe/schemas.py | 9 +-
sdks/python/apache_beam/dataframe/schemas_test.py | 85 +--
.../examples/cookbook/bigtableio_it_test.py | 2 +-
.../examples/cookbook/datastore_wordcount.py | 4 +-
.../apache_beam/internal/cloudpickle_pickler.py | 4 +-
sdks/python/apache_beam/internal/dill_pickler.py | 7 +-
sdks/python/apache_beam/internal/metrics/cells.py | 27 +-
sdks/python/apache_beam/internal/metrics/metric.py | 66 +--
sdks/python/apache_beam/internal/module_test.py | 4 +-
sdks/python/apache_beam/internal/pickler.py | 3 +-
sdks/python/apache_beam/internal/util.py | 11 +-
.../apache_beam/io/azure/blobstoragefilesystem.py | 4 -
sdks/python/apache_beam/io/filebasedsource.py | 24 +-
sdks/python/apache_beam/io/fileio.py | 14 +-
sdks/python/apache_beam/io/filesystem.py | 77 +--
.../io/flink/flink_streaming_impulse_source.py | 2 +-
.../apache_beam/io/gcp/bigquery_avro_tools.py | 26 +-
.../apache_beam/io/gcp/bigquery_read_internal.py | 49 +-
.../apache_beam/io/gcp/bigquery_read_it_test.py | 4 +-
.../apache_beam/io/gcp/bigquery_schema_tools.py | 5 +-
sdks/python/apache_beam/io/gcp/bigtableio.py | 5 +-
.../apache_beam/io/gcp/datastore/v1new/helper.py | 4 +-
.../apache_beam/io/gcp/datastore/v1new/types.py | 19 +-
sdks/python/apache_beam/io/gcp/gcsfilesystem.py | 8 +-
sdks/python/apache_beam/io/gcp/pubsub.py | 58 +-
sdks/python/apache_beam/io/gcp/pubsub_test.py | 10 +-
sdks/python/apache_beam/io/hadoopfilesystem.py | 12 +-
sdks/python/apache_beam/io/iobase.py | 79 +--
sdks/python/apache_beam/io/jdbc.py | 18 +-
sdks/python/apache_beam/io/kafka.py | 38 +-
sdks/python/apache_beam/io/localfilesystem.py | 8 +-
sdks/python/apache_beam/io/parquetio_test.py | 3 +-
sdks/python/apache_beam/io/restriction_trackers.py | 9 +-
sdks/python/apache_beam/io/textio.py | 66 +--
sdks/python/apache_beam/metrics/metric.py | 94 ++--
sdks/python/apache_beam/metrics/metricbase.py | 12 +-
.../python/apache_beam/ml/gcp/naturallanguageml.py | 36 +-
.../python/apache_beam/options/pipeline_options.py | 19 +
sdks/python/apache_beam/options/value_provider.py | 2 +-
sdks/python/apache_beam/pvalue.py | 125 ++---
.../apache_beam/runners/direct/bundle_factory.py | 34 +-
.../direct/consumer_tracking_pipeline_visitor.py | 15 +-
.../apache_beam/runners/direct/direct_runner.py | 5 +-
.../runners/direct/evaluation_context.py | 123 ++---
sdks/python/apache_beam/runners/direct/executor.py | 101 ++--
.../runners/direct/sdf_direct_runner.py | 17 +-
.../apache_beam/runners/direct/test_stream_impl.py | 5 +-
.../runners/direct/transform_evaluator.py | 62 +--
.../runners/direct/watermark_manager.py | 84 ++-
.../runners/interactive/background_caching_job.py | 4 +-
.../runners/interactive/cache_manager.py | 4 +-
.../runners/interactive/cache_manager_test.py | 2 +-
.../runners/interactive/display/pipeline_graph.py | 17 +-
.../interactive/display/pipeline_graph_renderer.py | 30 +-
.../runners/interactive/options/capture_control.py | 10 +-
.../interactive/options/capture_limiters.py | 17 +-
.../runners/interactive/recording_manager.py | 143 +++--
.../testing/integration/notebook_executor.py | 3 +-
.../interactive/testing/integration/screen_diff.py | 7 +-
sdks/python/apache_beam/runners/job/utils.py | 6 +-
.../python/apache_beam/runners/pipeline_context.py | 140 +++--
.../runners/portability/abstract_job_service.py | 149 +++--
.../runners/portability/artifact_service.py | 18 +-
.../runners/portability/flink_runner_test.py | 3 +
.../runners/portability/fn_api_runner/execution.py | 307 +++++------
.../runners/portability/fn_api_runner/fn_runner.py | 325 +++++------
.../portability/fn_api_runner/fn_runner_test.py | 2 +-
.../portability/fn_api_runner/watermark_manager.py | 11 +-
.../apache_beam/runners/portability/job_server.py | 6 +-
.../runners/portability/local_job_service.py | 41 +-
.../runners/portability/portable_runner.py | 76 ++-
.../runners/portability/prism_runner.py | 216 ++++++++
.../{flink_runner_test.py => prism_runner_test.py} | 136 +----
.../apache_beam/runners/portability/stager_test.py | 2 +-
sdks/python/apache_beam/runners/runner.py | 42 +-
sdks/python/apache_beam/runners/sdf_utils.py | 26 +-
.../apache_beam/runners/worker/log_handler.py | 35 +-
sdks/python/apache_beam/runners/worker/logger.py | 29 +-
.../apache_beam/runners/worker/statecache.py | 76 +--
.../apache_beam/runners/worker/statesampler.py | 50 +-
.../runners/worker/statesampler_slow.py | 51 +-
.../runners/worker/worker_id_interceptor.py | 3 +-
.../apache_beam/runners/worker/worker_pool_main.py | 37 +-
.../apache_beam/runners/worker/worker_status.py | 4 +-
.../testing/benchmarks/nexmark/monitor.py | 3 +-
.../testing/benchmarks/nexmark/nexmark_launcher.py | 3 +-
.../testing/benchmarks/nexmark/nexmark_perf.py | 4 +-
.../testing/benchmarks/nexmark/nexmark_util.py | 18 +-
.../testing/load_tests/load_test_metrics_utils.py | 32 +-
.../transforms/combinefn_lifecycle_pipeline.py | 2 +-
sdks/python/apache_beam/transforms/combiners.py | 2 +-
.../python/apache_beam/transforms/external_java.py | 5 +-
sdks/python/apache_beam/transforms/resources.py | 39 +-
sdks/python/apache_beam/transforms/sideinputs.py | 28 +-
sdks/python/apache_beam/transforms/trigger.py | 7 +-
sdks/python/apache_beam/transforms/userstate.py | 149 +++--
.../apache_beam/transforms/userstate_test.py | 2 +-
sdks/python/apache_beam/transforms/window.py | 164 +++---
sdks/python/apache_beam/typehints/decorators.py | 80 ++-
.../typehints/native_type_compatibility.py | 2 +-
sdks/python/apache_beam/typehints/schemas.py | 8 +-
sdks/python/apache_beam/typehints/typehints.py | 2 +-
sdks/python/apache_beam/utils/profiler.py | 24 +-
sdks/python/apache_beam/utils/proto_utils.py | 34 +-
sdks/python/apache_beam/utils/python_callable.py | 6 +-
sdks/python/apache_beam/utils/sharded_key.py | 5 +-
sdks/python/apache_beam/utils/shared.py | 28 +-
sdks/python/apache_beam/utils/timestamp.py | 160 ++----
sdks/python/apache_beam/utils/urns.py | 60 +--
sdks/python/apache_beam/version.py | 2 +-
sdks/python/apache_beam/yaml/json_utils.py | 3 +-
sdks/python/apache_beam/yaml/main.py | 15 +-
sdks/python/apache_beam/yaml/yaml_provider.py | 57 ++
.../apache_beam/yaml/yaml_provider_unit_test.py | 58 ++
sdks/python/apache_beam/yaml/yaml_transform.py | 25 +-
.../container/py38/base_image_requirements.txt | 2 +-
sdks/typescript/package.json | 2 +-
settings.gradle.kts | 1 +
.../content/en/documentation/programming-guide.md | 1 +
.../transforms/python/elementwise/flatmap.md | 2 +-
.../transforms/python/elementwise/map.md | 4 +-
459 files changed, 9979 insertions(+), 4239 deletions(-)
create mode 100644
runners/core-java/src/main/java/org/apache/beam/runners/core/metrics/StringSetCell.java
create mode 100644
runners/core-java/src/main/java/org/apache/beam/runners/core/metrics/StringSetData.java
create mode 100644
runners/core-java/src/test/java/org/apache/beam/runners/core/metrics/StringSetCellTest.java
create mode 100644
runners/core-java/src/test/java/org/apache/beam/runners/core/metrics/StringSetDataTest.java
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=>
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/AbstractWindmillMap.java
(71%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMapViaMultimap.java
create mode 100644
runners/jet/src/main/java/org/apache/beam/runners/jet/metrics/StringSetImpl.java
copy {sdks/java/io/json => runners/prism/java}/build.gradle (59%)
create mode 100644
runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismExecutor.java
create mode 100644
runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismLocator.java
create mode 100644
runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismPipelineOptions.java
create mode 100644
runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismRunner.java
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=>
runners/prism/java/src/main/java/org/apache/beam/runners/prism/package-info.java
(71%)
create mode 100644
runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismExecutorTest.java
create mode 100644
runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismLocatorTest.java
create mode 100644
runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismRunnerTest.java
copy sdks/go/container/tools/{pipeline_options.go => pipeline_options_test.go}
(54%)
copy
sdks/java/{io/rrio/src/main/java/org/apache/beam/io/requestresponse/DefaultSerializableBackoffSupplier.java
=> core/src/main/java/org/apache/beam/sdk/metrics/Lineage.java} (51%)
copy
sdks/java/{io/rrio/src/main/java/org/apache/beam/io/requestresponse/SerializableSupplier.java
=> core/src/main/java/org/apache/beam/sdk/metrics/StringSet.java} (65%)
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/metrics/StringSetResult.java
copy
sdks/java/{io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=> core/src/main/java/org/apache/beam/sdk/testing/UsesStringSetMetrics.java}
(70%)
rename sdks/java/{io/rrio/src/main/java/org/apache/beam/io/requestresponse =>
core/src/main/java/org/apache/beam/sdk/util}/SerializableSupplier.java (96%)
create mode 100644
sdks/java/core/src/test/java/org/apache/beam/sdk/metrics/StringSetResultTest.java
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/DatabaseTestHelper.java (79%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/HashingFn.java (98%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/IOITHelper.java (100%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/IOTestPipelineOptions.java (100%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/NetworkTestHelper.java (100%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/PostgresIOTestPipelineOptions.java
(100%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java (98%)
rename sdks/java/io/common/src/{test =>
main}/java/org/apache/beam/sdk/io/common/TestRow.java (83%)
copy
sdks/java/io/{google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=> common/src/main/java/org/apache/beam/sdk/io/common/package-info.java} (71%)
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParse.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseConfiguration.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpers.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseKV.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOReadFiles.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOStringToCsvRecord.java
copy
sdks/java/io/{google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=>
csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOParseConfigurationTest.java}
(69%)
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpersTest.java
copy
sdks/java/io/{google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=> csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOParseKVTest.java} (71%)
copy
sdks/java/io/{google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=> csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOParseTest.java} (68%)
rename
sdks/java/io/{google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/SerializableSupplier.java
=> csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOReadFilesTest.java} (67%)
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOStringToCsvRecordTest.java
delete mode 100644
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergSchemaTransformCatalogConfig.java
create mode 100644
sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcReadWithPartitionsHelper.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionService.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionServiceFactory.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthSempClient.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthSempClientFactory.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BrokerResponse.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SempBasicAuthClientExecutor.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SolaceMessageReceiver.java
create mode 100644
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/data/Semp.java
create mode 100644
sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/broker/SempBasicAuthClientExecutorTest.java
create mode 100644
sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/it/SolaceContainerManager.java
create mode 100644
sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/it/SolaceIOIT.java
create mode 100644 sdks/python/apache_beam/runners/portability/prism_runner.py
copy sdks/python/apache_beam/runners/portability/{flink_runner_test.py =>
prism_runner_test.py} (67%)
