George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463411782
##
File path: sdks/python/setup.py
##
@@ -128,6 +128,7 @@ def get_version():
cythonize = lambda *args, **kwargs: []
REQUIRED_PACKAGES = [
+
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463411782
##
File path: sdks/python/setup.py
##
@@ -128,6 +128,7 @@ def get_version():
cythonize = lambda *args, **kwargs: []
REQUIRED_PACKAGES = [
+
lostluck commented on pull request #12429:
URL: https://github.com/apache/beam/pull/12429#issuecomment-666901638
Going to close it, largely because while in terms of Beam Semantics it's
unlikely to trigger, not guarding a divide by zero seems fundamentally wrong.
Something could change
lostluck closed pull request #12429:
URL: https://github.com/apache/beam/pull/12429
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
kennknowles merged pull request #12387:
URL: https://github.com/apache/beam/pull/12387
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
lostluck merged pull request #12425:
URL: https://github.com/apache/beam/pull/12425
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463383155
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -164,70 +167,109 @@ class DicomSearch(PTransform):
}
"""
- def __init__(self,
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463382756
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -372,52 +420,63 @@ def __init__(self, destination_dict, input_type,
credential=None):
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463382633
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -372,52 +420,63 @@ def __init__(self, destination_dict, input_type,
credential=None):
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463382681
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -426,6 +485,44 @@ def process(self, element):
out = {}
out['status'] =
George-Wu commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463382501
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -114,55 +115,60 @@
class DicomSearch(PTransform):
"""A PTransform used for
ZijieSong946 commented on a change in pull request #12348:
URL: https://github.com/apache/beam/pull/12348#discussion_r463375064
##
File path:
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlBeamTranslationUtils.java
##
@@
lostluck commented on a change in pull request #12350:
URL: https://github.com/apache/beam/pull/12350#discussion_r462445798
##
File path: sdks/go/pkg/beam/core/runtime/exec/datasource.go
##
@@ -302,31 +308,76 @@ func (n *DataSource) Split(splits []int64, frac float64,
bufSize
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666858821
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666851408
only PullLicenses is failing @chamikaramj PTAL
This is an automated message from the Apache Git Service.
To
ibzib commented on pull request #12385:
URL: https://github.com/apache/beam/pull/12385#issuecomment-666850930
> I have already spent a long time trying to fix quotes, so I can't help but
wondering: why do we need flinkCompatibilityMatrixPROCESS in the first place,
when it is not being run
pabloem merged pull request #12082:
URL: https://github.com/apache/beam/pull/12082
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
ibzib commented on pull request #12385:
URL: https://github.com/apache/beam/pull/12385#issuecomment-666849394
I got `flinkCompatibilityMatrixPROCESS` to pass on my machine by escaping
the arguments via `${1@Q}`. Apparently whatever shell Jenkins is using does not
support this. I will have
ibzib commented on a change in pull request #12385:
URL: https://github.com/apache/beam/pull/12385#discussion_r463348975
##
File path: sdks/python/apache_beam/runners/portability/flink_runner_test.py
##
@@ -53,361 +53,380 @@
from apache_beam.transforms import userstate
from
ibzib commented on a change in pull request #12385:
URL: https://github.com/apache/beam/pull/12385#discussion_r463348975
##
File path: sdks/python/apache_beam/runners/portability/flink_runner_test.py
##
@@ -53,361 +53,380 @@
from apache_beam.transforms import userstate
from
ibzib commented on pull request #12385:
URL: https://github.com/apache/beam/pull/12385#issuecomment-666846231
> I couldn't find the published test results in Jenkins. Do we have to add
this separately?
https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Commit/6126/
ibzib commented on a change in pull request #12385:
URL: https://github.com/apache/beam/pull/12385#discussion_r463346402
##
File path: sdks/python/apache_beam/runners/portability/flink_runner_test.py
##
@@ -53,361 +53,380 @@
from apache_beam.transforms import userstate
from
ibzib commented on a change in pull request #12385:
URL: https://github.com/apache/beam/pull/12385#discussion_r463345888
##
File path: sdks/python/apache_beam/runners/portability/flink_runner_test.py
##
@@ -53,361 +53,380 @@
from apache_beam.transforms import userstate
from
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666830620
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
chamikaramj commented on pull request #12198:
URL: https://github.com/apache/beam/pull/12198#issuecomment-666825962
Thanks for clarifying. LGTM.
This is an automated message from the Apache Git Service.
To respond to the
chamikaramj commented on a change in pull request #12149:
URL: https://github.com/apache/beam/pull/12149#discussion_r463341503
##
File path: sdks/python/apache_beam/io/external/snowflake.py
##
@@ -0,0 +1,185 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666812322
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666812699
No, my mistake. It fails locally too, I was running it incorrectly.
This is an automated message from the Apache
tysonjh commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666808017
More test failures. NPEs in what seems like unrelated code, but notably the
pubsub one again which passes locally when I run it for some reason.
TheNeuralBit merged pull request #12399:
URL: https://github.com/apache/beam/pull/12399
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
pabloem commented on pull request #12082:
URL: https://github.com/apache/beam/pull/12082#issuecomment-666774552
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please
TheNeuralBit commented on a change in pull request #12422:
URL: https://github.com/apache/beam/pull/12422#discussion_r463325571
##
File path:
sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java
##
@@ -34,34 +35,65 @@
import
boyuanzz opened a new pull request #12430:
URL: https://github.com/apache/beam/pull/12430
**Please** add a meaningful description for your change here
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution
jiyongjung0 commented on pull request #12404:
URL: https://github.com/apache/beam/pull/12404#issuecomment-666770943
Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on
simonepri commented on pull request #12429:
URL: https://github.com/apache/beam/pull/12429#issuecomment-666770785
R: @youngoli, @lostluck
Feel free to close it without merging if you don't feel it is needed.
This is
amaliujia merged pull request #12417:
URL: https://github.com/apache/beam/pull/12417
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
youngoli commented on pull request #12350:
URL: https://github.com/apache/beam/pull/12350#issuecomment-666769902
Good idea.
This is an automated message from the Apache Git Service.
To respond to the message, please log on
aaltay commented on pull request #12304:
URL: https://github.com/apache/beam/pull/12304#issuecomment-666768573
Change LGTM. Could you rebase?
This is an automated message from the Apache Git Service.
To respond to the
aaltay commented on pull request #11967:
URL: https://github.com/apache/beam/pull/11967#issuecomment-666767908
What is the next step for this PR?
This is an automated message from the Apache Git Service.
To respond to the
aaltay commented on pull request #12042:
URL: https://github.com/apache/beam/pull/12042#issuecomment-666767694
What are the next steps for this PR?
This is an automated message from the Apache Git Service.
To respond to the
pabloem commented on a change in pull request #12331:
URL: https://github.com/apache/beam/pull/12331#discussion_r463282351
##
File path: sdks/python/apache_beam/io/gcp/dicomio.py
##
@@ -372,52 +420,63 @@ def __init__(self, destination_dict, input_type,
credential=None):
aaltay commented on pull request #12320:
URL: https://github.com/apache/beam/pull/12320#issuecomment-666767097
@dranderson1117 - Could someone from healthcare team do a first pass review?
This is an automated message from
simonepri opened a new pull request #12429:
URL: https://github.com/apache/beam/pull/12429
ExtractOutput it is not called when the input PCollection is empty so the if
statement that this PR removes is reduntant.
Post-Commit Tests Status (on master
aaltay commented on pull request #12404:
URL: https://github.com/apache/beam/pull/12404#issuecomment-666766550
I think we can merge this as is, and let #12198 extend the range a bit more.
Based on PR description this sounds like a blocker for using
google-cloud-storage.
TheNeuralBit closed pull request #12298:
URL: https://github.com/apache/beam/pull/12298
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
TheNeuralBit commented on pull request #12298:
URL: https://github.com/apache/beam/pull/12298#issuecomment-666766488
Closing in favor of https://github.com/apache/beam/pull/12297
This is an automated message from the Apache
TheNeuralBit commented on pull request #12234:
URL: https://github.com/apache/beam/pull/12234#issuecomment-666766596
Closing in favor of #12297
This is an automated message from the Apache Git Service.
To respond to the
aaltay merged pull request #12404:
URL: https://github.com/apache/beam/pull/12404
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
TheNeuralBit commented on pull request #12235:
URL: https://github.com/apache/beam/pull/12235#issuecomment-666766546
Closing in favor of #12297
This is an automated message from the Apache Git Service.
To respond to the
TheNeuralBit closed pull request #12235:
URL: https://github.com/apache/beam/pull/12235
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
TheNeuralBit closed pull request #12234:
URL: https://github.com/apache/beam/pull/12234
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
jiyongjung0 commented on pull request #12404:
URL: https://github.com/apache/beam/pull/12404#issuecomment-666765978
Thank you for the reviews. I'll wait for #12198 which has wider effect.
This is an automated message from
aaltay merged pull request #12372:
URL: https://github.com/apache/beam/pull/12372
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
simonepri commented on pull request #12421:
URL: https://github.com/apache/beam/pull/12421#issuecomment-666763106
> In practice, having no elements at all is so uncommon, it's not reasonable
to include that in basic utility transforms, since they never apply, and no
data = no compute is a
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463311197
##
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN:
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463311197
##
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN:
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463308917
##
File path: sdks/python/apache_beam/coders/coder_impl.py
##
@@ -530,6 +530,88 @@ def estimate_size(self, unused_value, nested=False):
return 1
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463308693
##
File path: sdks/python/apache_beam/coders/coder_impl.py
##
@@ -530,6 +530,88 @@ def estimate_size(self, unused_value, nested=False):
return 1
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666747008
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
ibzib opened a new pull request #12428:
URL: https://github.com/apache/beam/pull/12428
R: @mxm
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
lostluck commented on pull request #12350:
URL: https://github.com/apache/beam/pull/12350#issuecomment-666735372
Consider updating the PR description to match the current state :)
This is an automated message from the Apache
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666735046
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
lostluck merged pull request #12402:
URL: https://github.com/apache/beam/pull/12402
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
pabloem commented on a change in pull request #12082:
URL: https://github.com/apache/beam/pull/12082#discussion_r463295587
##
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
##
@@ -457,6 +456,22 @@
* for
pabloem commented on pull request #12082:
URL: https://github.com/apache/beam/pull/12082#issuecomment-666732227
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please
robertwb commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463291968
##
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN:
mf2199 commented on a change in pull request #11295:
URL: https://github.com/apache/beam/pull/11295#discussion_r463294587
##
File path: sdks/python/apache_beam/io/gcp/bigtableio.py
##
@@ -1,151 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-#
pabloem commented on pull request #12203:
URL: https://github.com/apache/beam/pull/12203#issuecomment-666729271
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
robertwb merged pull request #12362:
URL: https://github.com/apache/beam/pull/12362
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
robinyqiu commented on a change in pull request #12348:
URL: https://github.com/apache/beam/pull/12348#discussion_r463286427
##
File path:
sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlBeamTranslationUtils.java
##
@@ -184,6
y1chi commented on pull request #12427:
URL: https://github.com/apache/beam/pull/12427#issuecomment-666716230
Could you rename the title to be more meaningful and add `[BEAM-2855]` in
the front so that it will be hooked with the right JIRA ticket
TheNeuralBit commented on a change in pull request #12399:
URL: https://github.com/apache/beam/pull/12399#discussion_r463279232
##
File path: sdks/python/apache_beam/examples/sql_taxi.py
##
@@ -0,0 +1,97 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
pabloem commented on pull request #12151:
URL: https://github.com/apache/beam/pull/12151#issuecomment-666703403
One more thing: Please add a description of the change to `CHANGES.md`, so
we can have this on the release notes. Then I can merge.
pabloem commented on a change in pull request #12151:
URL: https://github.com/apache/beam/pull/12151#discussion_r463273363
##
File path:
sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeStreamingServiceConfig.java
##
@@ -20,26 +20,51 @@
pabloem commented on a change in pull request #12151:
URL: https://github.com/apache/beam/pull/12151#discussion_r463219932
##
File path:
sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeIO.java
##
@@ -719,7 +906,9 @@ private void checkArguments()
robinyqiu commented on pull request #12403:
URL: https://github.com/apache/beam/pull/12403#issuecomment-96191
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message,
lostluck merged pull request #12421:
URL: https://github.com/apache/beam/pull/12421
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
danielxjd commented on a change in pull request #12223:
URL: https://github.com/apache/beam/pull/12223#discussion_r463259859
##
File path:
sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java
##
@@ -235,12 +277,195 @@ public ReadFiles
leiyiz commented on pull request #12427:
URL: https://github.com/apache/beam/pull/12427#issuecomment-40310
R: @pabloem
R: @y1chi
This is an automated message from the Apache Git Service.
To respond to the message,
leiyiz opened a new pull request #12427:
URL: https://github.com/apache/beam/pull/12427
changed queries 0-2, implemented query9, coders for all models, new models
for query results, changed launcher to count and print results to file
Thank you for
udim commented on pull request #12009:
URL: https://github.com/apache/beam/pull/12009#issuecomment-31741
Run Python2_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the
leiyiz closed pull request #12365:
URL: https://github.com/apache/beam/pull/12365
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
lostluck commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463208404
##
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN:
TheNeuralBit commented on pull request #12426:
URL: https://github.com/apache/beam/pull/12426#issuecomment-666590444
This is relevant for BEAM-10571. Some existing payloads use `List>`
fields, which we could replace with Map when using schemas to encode the
payload.
TheNeuralBit edited a comment on pull request #12426:
URL: https://github.com/apache/beam/pull/12426#issuecomment-666590444
This is relevant for BEAM-10571. Some existing payloads use `List>`
fields, which we could replace with `Map` when using schemas to encode the
payload.
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463194957
##
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##
@@ -855,10 +855,21 @@ message StandardCoders {
// BOOLEAN:
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463182344
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/__tests__/kernel/KernelModel.test.ts
##
@@ -0,0
TheNeuralBit commented on a change in pull request #12426:
URL: https://github.com/apache/beam/pull/12426#discussion_r463194527
##
File path:
runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/CommonCoderTest.java
##
@@ -366,12 +368,15 @@
TheNeuralBit opened a new pull request #12426:
URL: https://github.com/apache/beam/pull/12426
This PR adds support for MapType in the Python RowCoder, as well as support
for encoding nulls inside of container types (NullableCoder).
Post-Commit Tests Status (on master branch)
KevinGG commented on pull request #12372:
URL: https://github.com/apache/beam/pull/12372#issuecomment-666586747
> LGTM. I do not believe I am a good typescript reviewer. It would be good
if you can ask on dev@ and see if there could be better reviewers for future
PRs.
Thanks! For
KevinGG commented on pull request #12372:
URL: https://github.com/apache/beam/pull/12372#issuecomment-666584845
> Retracting my LGTM, I have some questions.
>
> And a high level question, what is the purpose of this module? (Could you
add more details to what you wrote as part of
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463182344
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/__tests__/kernel/KernelModel.test.ts
##
@@ -0,0
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463188165
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/__tests__/kernel/KernelModel.test.ts
##
@@ -0,0
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463185961
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/kernel/KernelModel.ts
##
@@ -0,0 +1,161 @@
+//
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463183798
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/kernel/KernelModel.ts
##
@@ -0,0 +1,161 @@
+//
davidcavazos commented on pull request #12118:
URL: https://github.com/apache/beam/pull/12118#issuecomment-666576290
All tests are passing, @pabloem can you take a look whenever you have a
chance?
This is an automated
lostluck commented on pull request #12425:
URL: https://github.com/apache/beam/pull/12425#issuecomment-666575819
R: @youngoli
cc: @robertwb @damondouglas @codeBehindMe
The others are cc'd as this has value for setting up Python Portable Runner
usage, for local testing of
lostluck edited a comment on pull request #12425:
URL: https://github.com/apache/beam/pull/12425#issuecomment-666575819
R: @youngoli
cc: @robertwb @damondouglas @codeBehindMe
The others are cc'd as this has value for setting up Python Portable Runner
usage, for local testing
KevinGG commented on a change in pull request #12372:
URL: https://github.com/apache/beam/pull/12372#discussion_r463182344
##
File path:
sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/src/__tests__/kernel/KernelModel.test.ts
##
@@ -0,0
kennknowles commented on pull request #12416:
URL: https://github.com/apache/beam/pull/12416#issuecomment-666574292
But it also probably isn't a flake, since it is a pretty simple unit test.
Most likely the assumptions of the mock were overconstrained so changes in the
Pubsub client
1 - 100 of 213 matches
Mail list logo
