This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/beam.git.
*** WARNING: tag nightly-master was modified! ***
from 71c7760 (commit)
to a1e644a (commit)
from 71c7760 Moving to 2.25.0-SNAPSHOT on master branch.
add 2def4cc [BEAM-10694] Work around serialization issue with
ReaderContext by memoizing the serialized form and propagating it forward.
(#12556)
add 5f9396b [BEAM-9680] Add Filter with ParDo lesson to Go SDK Katas
(#12506)
add 198a583 [BEAM-9547] Lift associative aggregations. (#12469)
add a098130 Merge pull request #12427 from [BEAM-2855] nexmark python
suite implement queries 0, 1, 2 and 9
add de4464b Update nexmark dashboard links.
add e028256 Merge pull request #12475 from tysonjh/www-nexmark
add 1b985bb [BEAM-10500] Make KeyedTimerDataCoder encode output timestamp
(#12535)
add 72a9093 [BEAM-10648] Remove unused BigQuery queryTempDataset value
add a6907c0 Merge pull request #12478 from regadas/bigquery_remove_unused
add 23f3b49 Import WordExtractingDoFn from wordcount_with_metrics
add 70fcd57 Merge pull request #12569 from apache/aaltay-patch-1
add f1e780d [BEAM-10646] Remove SparkPortableExecutionTest.testExecution.
add 1709588 [BEAM-10646] Don't wait for test to time out if pipeline
fails.
add 228c621 Merge pull request #12477 from ibzib/BEAM-10646
add 699f872 Extending ApproximateQuantiles functionality to deal with
non-uniform weights. (#12420)
add b423fe3 fix logic issue in metric name namespace filtering (#12570)
add 0f04de7 Better error on BQ schema parse (#12549)
add 5ae71f9 [BEAM-10545] HtmlView module
add 213c2c7 Use primitive string[] to replace Array<string> type
add bf85a7a Merge pull request #12460 from [BEAM-10545] HtmlView module
add b37f1f6 [BEAM-10702] Do not implicitly decompress artifacts
add b214d1b Adds a Julia Set test on portable local runner
add c5da208 Address review comments
add caabb8b Merge pull request #12571 from apache/beam-10702:
[BEAM-10702] Do not implicitly decompress artifacts
add 29cd482 Fix bug that evicting computed PCollections was changing list
while iterating.
add 9ab86d6 Merge pull request #12558 from [BEAM-10695] Fix
InteractiveEnvironment cannot evict PCollection
add 547f724 Fix Py3 incompatibility in stager.py.
add 02bf081 [BEAM-10704] Fix Py3 incompatibility in stager.py. #12579
add 55874cc [BEAM-10691] Use FlinkStateInternals#addWatermarkHoldUsage
for timer output timestamp
add b5846c6 Merge pull request #12551: [BEAM-10691] Use
FlinkStateInternals#addWatermarkHoldUsage for timer output timestamp
add bc0ba7e [BEAM-9615] Map user types to Schema reps. (#12554)
add 6001642 fixed a typo in S3TestUtils (#12582)
add d2b2d9d Use unbounded wrapper for Kafka Read.
add c22417b Merge pull request #12577 from boyuanzz/use_wrapper
add 5b0bacd Enable dataflow streaming engine when running runner_v2 and
streaming.
add e1ec7f6 Fix formatter.
add 8ffce6b Merge pull request #12561 from boyuanzz/py
add 20b5112 [BEAM-9547] Implement some methods for deferred Series.
(#12534)
add e593261 [BEAM-10670] Use fraction of remainder if consumed fraction
is unknown
add 32c2227 [BEAM-10670] Improve splitting logic to prefer splits upto
the the desired number of splits and also remove check that isn't possible
since the restriction could be done unknowingly with the last tryClaim call.
add 874b9b7 [BEAM-10670] Fix passing forward the self-checkpoint from the
UnboundedSource to respect the contract of tryClaim
add 64449e8 Merge pull request #12548 from lukecwik/beam10670.3
add 7730b45 [BEAM-10688] Euphoria assumes that all type descriptors are
resolvable to coders (and serializable) which isn't true.
add dce1eb8 Merge pull request #12547 from lukecwik/beam10688
add 7a2411c Fix broken build.
add a1e644a Merge pull request #12590 from lostluck/fixvendored
No new revisions were added by this update.
Summary of changes:
build.gradle | 1 +
.../common_transforms/filter}/lesson-info.yaml | 2 +-
.../filter/lesson-remote-info.yaml | 3 +
.../filter/pardo}/cmd/main.go | 4 +-
.../filter}/pardo/pkg/task/task.go | 10 +-
.../filter/pardo}/task-info.yaml | 4 +-
.../filter/pardo/task-remote-info.yaml | 2 +
.../filter/pardo}/task.md | 9 +-
.../filter/pardo}/test/task_test.go | 6 +-
.../common_transforms/section-info.yaml} | 2 +-
.../go/common_transforms/section-remote-info.yaml | 2 +
learning/katas/go/course-info.yaml | 1 +
learning/katas/go/course-remote-info.yaml | 2 +-
.../wrappers/streaming/DoFnOperator.java | 33 +--
.../streaming/state/FlinkStateInternals.java | 21 +-
.../wrappers/streaming/DoFnOperatorTest.java | 2 +-
.../wrappers/streaming/WindowDoFnOperatorTest.java | 2 -
.../beam/runners/jobsubmission/JobInvocation.java | 2 +-
.../beam/runners/samza/runtime/KeyedTimerData.java | 6 +-
.../runners/samza/runtime/KeyedTimerDataTest.java | 8 +-
.../runners/spark/SparkPortableExecutionTest.java | 96 +-------
sdks/go/gogradle.lock | 30 ++-
.../pkg/beam/core/runtime/graphx/schema/schema.go | 250 ++++++++++++++++---
.../beam/core/runtime/graphx/schema/schema_test.go | 207 +++++++++++++++-
.../src/main/java/org/apache/beam/sdk/io/Read.java | 32 ++-
.../core/client/operator/base/Operator.java | 2 +-
.../euphoria/core/client/type/TypeAwareness.java | 5 +-
.../euphoria/core/client/lib/SplitTest.java | 3 +
.../core/client/operator/CountByKeyTest.java | 3 +
.../core/client/operator/DistinctTest.java | 3 +
.../euphoria/core/client/operator/FilterTest.java | 3 +
.../euphoria/core/client/operator/FlatMapTest.java | 3 +
.../euphoria/core/client/operator/JoinTest.java | 3 +
.../core/client/operator/MapElementsTest.java | 3 +
.../core/client/operator/ReduceByKeyTest.java | 3 +
.../core/client/operator/ReduceWindowTest.java | 3 +
.../core/client/operator/SumByKeyTest.java | 3 +
.../core/client/operator/TopPerKeyTest.java | 3 +
.../euphoria/core/client/operator/UnionTest.java | 3 +
.../euphoria/core/client/util/IOUtilsTest.java | 3 +
.../euphoria/core/client/util/SumsTest.java | 3 +
.../core/docs/DocumentationExamplesTest.java | 3 +
.../core/testkit/BroadcastHashJoinTest.java | 3 +
.../euphoria/core/testkit/CountByKeyTest.java | 3 +
.../euphoria/core/testkit/DistinctTest.java | 3 +
.../euphoria/core/testkit/FilterTest.java | 3 +
.../euphoria/core/testkit/FlatMapTest.java | 3 +
.../extensions/euphoria/core/testkit/JoinTest.java | 3 +
.../euphoria/core/testkit/MapElementsTest.java | 3 +
.../euphoria/core/testkit/ReduceByKeyTest.java | 3 +
.../euphoria/core/testkit/ReduceWindowTest.java | 3 +
.../euphoria/core/testkit/SumByKeyTest.java | 3 +
.../euphoria/core/testkit/TopPerKeyTest.java | 3 +
.../euphoria/core/testkit/UnionTest.java | 3 +
.../core/translate/BeamMetricsTranslationTest.java | 3 +
.../translate/BroadcastHashJoinTranslatorTest.java | 3 +
.../SingleJvmAccumulatorProviderTest.java | 3 +
.../translate/TimestampExtractTransformTest.java | 3 +
.../collector/SingleValueCollectorTest.java | 3 +
.../org/apache/beam/sdk/io/aws/s3/S3TestUtils.java | 5 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 2 -
.../apache/beam/sdk/io/hcatalog/HCatalogIO.java | 49 +++-
.../beam/sdk/io/hcatalog/HCatalogIOTest.java | 26 ++
.../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 4 +-
.../beam/sdk/io/kafka/KafkaIOExternalTest.java | 66 ++---
sdks/python/apache_beam/dataframe/doctests.py | 3 +-
sdks/python/apache_beam/dataframe/expressions.py | 18 +-
sdks/python/apache_beam/dataframe/frame_base.py | 5 -
sdks/python/apache_beam/dataframe/frames.py | 232 +++++++++++++++---
sdks/python/apache_beam/dataframe/frames_test.py | 23 +-
.../apache_beam/dataframe/pandas_doctests_test.py | 34 +--
sdks/python/apache_beam/dataframe/transforms.py | 22 ++
.../apache_beam/examples/streaming_wordcount.py | 2 +-
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 12 +-
sdks/python/apache_beam/metrics/metric.py | 11 +-
.../runners/dataflow/dataflow_runner.py | 10 +-
.../apache-beam-jupyterlab-sidepanel/.eslintrc.js | 12 +-
.../.prettierrc.json | 3 +-
.../apache-beam-jupyterlab-sidepanel/package.json | 8 +-
.../src/__tests__/common/HtmlView.test.tsx | 111 +++++++++
.../src/__tests__/kernel/KernelModel.test.ts | 21 +-
.../src/common/HtmlView.tsx | 119 +++++++++
.../apache-beam-jupyterlab-sidepanel/tsconfig.json | 1 +
.../apache-beam-jupyterlab-sidepanel/yarn.lock | 146 ++++++++++-
.../runners/interactive/interactive_environment.py | 4 +-
.../interactive/interactive_environment_test.py | 20 ++
.../runners/portability/artifact_service.py | 4 +-
.../portability/fn_api_runner/worker_handlers.py | 7 +-
.../apache_beam/runners/portability/stager.py | 2 +-
.../benchmarks/nexmark/models/auction_bid.py | 57 +++++
.../benchmarks/nexmark/models/auction_price.py | 56 +++++
.../benchmarks/nexmark/models/field_name.py} | 27 ++-
.../benchmarks/nexmark/models/nexmark_model.py | 153 ++++++++++--
.../testing/benchmarks/nexmark/nexmark_launcher.py | 24 +-
.../testing/benchmarks/nexmark/nexmark_util.py | 130 +++++++++-
.../queries/{query2.py => nexmark_query_util.py} | 59 +++--
.../testing/benchmarks/nexmark/queries/query0.py | 15 +-
.../testing/benchmarks/nexmark/queries/query1.py | 21 +-
.../testing/benchmarks/nexmark/queries/query2.py | 29 +--
.../benchmarks/nexmark/queries/query9.py} | 19 +-
.../benchmarks/nexmark/queries/winning_bids.py | 198 +++++++++++++++
sdks/python/apache_beam/transforms/stats.py | 266 +++++++++++++++------
sdks/python/apache_beam/transforms/stats_test.py | 62 ++++-
sdks/python/scripts/generate_pydoc.sh | 1 +
sdks/python/test-suites/portable/common.gradle | 25 ++
.../en/documentation/sdks/java/testing/nexmark.md | 24 +-
106 files changed, 2426 insertions(+), 562 deletions(-)
copy learning/katas/{python/Windowing/Adding Timestamp =>
go/common_transforms/filter}/lesson-info.yaml (99%)
create mode 100644
learning/katas/go/common_transforms/filter/lesson-remote-info.yaml
copy learning/katas/go/{core_transforms/combine/simple_function =>
common_transforms/filter/pardo}/cmd/main.go (90%)
copy learning/katas/go/{core_transforms/map =>
common_transforms/filter}/pardo/pkg/task/task.go (89%)
copy learning/katas/go/{core_transforms/additional_outputs/additional_outputs
=> common_transforms/filter/pardo}/task-info.yaml (96%)
create mode 100644
learning/katas/go/common_transforms/filter/pardo/task-remote-info.yaml
copy learning/katas/go/{core_transforms/map/pardo_onetomany =>
common_transforms/filter/pardo}/task.md (75%)
copy learning/katas/go/{core_transforms/combine/combinefn =>
common_transforms/filter/pardo}/test/task_test.go (88%)
copy learning/katas/{python/Windowing/Adding Timestamp/lesson-info.yaml =>
go/common_transforms/section-info.yaml} (98%)
create mode 100644 learning/katas/go/common_transforms/section-remote-info.yaml
create mode 100644
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/__tests__/common/HtmlView.test.tsx
create mode 100644
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/common/HtmlView.tsx
create mode 100644
sdks/python/apache_beam/testing/benchmarks/nexmark/models/auction_bid.py
create mode 100644
sdks/python/apache_beam/testing/benchmarks/nexmark/models/auction_price.py
copy sdks/python/apache_beam/{runners/dataflow/__init__.py =>
testing/benchmarks/nexmark/models/field_name.py} (62%)
copy sdks/python/apache_beam/testing/benchmarks/nexmark/queries/{query2.py =>
nexmark_query_util.py} (52%)
copy sdks/python/apache_beam/{runners/dataflow/__init__.py =>
testing/benchmarks/nexmark/queries/query9.py} (65%)
create mode 100644
sdks/python/apache_beam/testing/benchmarks/nexmark/queries/winning_bids.py
