[beam] branch master updated (2adf96b -> a76a62d)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2adf96b [BEAM-1833] Fixes BEAM-1833 add 0f9acce [BEAM-1080] Skip tests that required GCP credentials add a76a62d Merge pull request #10890 from apache/aaltay-patch-1 No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch master updated (aad3ec5 -> 2adf96b)
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from aad3ec5 Merge pull request #10873 Add capabilities and requirements to beam protos. add 2adf96b [BEAM-1833] Fixes BEAM-1833 No new revisions were added by this update. Summary of changes: CHANGES.md | 3 ++ sdks/python/apache_beam/pipeline.py| 52 +- .../runners/dataflow/dataflow_runner.py| 2 +- 3 files changed, 44 insertions(+), 13 deletions(-)
[beam] branch master updated (e73e1d1 -> aad3ec5)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from e73e1d1 [BEAM-7246] Added Google Spanner Write Transform (#10712) add 4e9ed97 Add capabilities and requirements to beam protos. add d4e75d2 Apply suggestions from code review add aad3ec5 Merge pull request #10873 Add capabilities and requirements to beam protos. No new revisions were added by this update. Summary of changes: model/pipeline/src/main/proto/beam_runner_api.proto | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-)
[beam] branch master updated (745b425 -> e73e1d1)
This is an automated email from the ASF dual-hosted git repository. chamikara pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 745b425 Exclude tests that are not passing under currect Avro IO requirements. (#10876) add e73e1d1 [BEAM-7246] Added Google Spanner Write Transform (#10712) No new revisions were added by this update. Summary of changes: CHANGES.md | 2 + .../apache_beam/io/gcp/experimental/spannerio.py | 528 - .../io/gcp/experimental/spannerio_test.py | 245 +- 3 files changed, 771 insertions(+), 4 deletions(-)
[beam] branch master updated (54b75fa -> 745b425)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 54b75fa Merge pull request #10891: Removed compile time generation of test Thrift class. add 745b425 Exclude tests that are not passing under currect Avro IO requirements. (#10876) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/io/avroio_test.py | 4 sdks/python/setup.py | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
[beam] branch master updated (b483ddb -> 54b75fa)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from b483ddb [BEAM-9291] Upload graph option in dataflow's python sdk (#10829) add bb2fc76 Removed compile time generation of test Thrift class. add 54b75fa Merge pull request #10891: Removed compile time generation of test Thrift class. No new revisions were added by this update. Summary of changes: sdks/java/io/thrift/build.gradle |6 +- .../beam/sdk/io/thrift/TestThriftStruct.java | 1232 .../apache/beam/sdk/io/thrift/ThriftIOTest.java|1 - 3 files changed, 1236 insertions(+), 3 deletions(-) create mode 100644 sdks/java/io/thrift/src/test/java/org/apache/beam/sdk/io/thrift/TestThriftStruct.java
[beam] 01/01: [BEAM-1080] Skip tests that required GCP credentials
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch aaltay-patch-1
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 0f9accec37ec51177760802d7951ce51dc7f8695
Author: Ahmet Altay
AuthorDate: Tue Feb 18 14:00:08 2020 -0800
[BEAM-1080] Skip tests that required GCP credentials
---
sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py
b/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py
index dffef3a..bec5a52 100644
--- a/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py
+++ b/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py
@@ -787,7 +787,7 @@ class UtilTest(unittest.TestCase):
assert encoding == version_to_encoding[sys.version_info[0]]
- @unittest.skipIf(apiclient is None, 'GCP dependencies are not installed')
+ @unittest.skip("Enable once BEAM-1080 is fixed.")
def test_graph_is_uploaded(self):
pipeline_options = PipelineOptions([
'--project',
[beam] branch aaltay-patch-1 created (now 0f9acce)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch aaltay-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git. at 0f9acce [BEAM-1080] Skip tests that required GCP credentials This branch includes the following new commits: new 0f9acce [BEAM-1080] Skip tests that required GCP credentials The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Revert "[BEAM-6522] [BEAM-7455] Unskip Avro IO tests that are now passing. (#10838)"
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch revert-10838-avro
in repository https://gitbox.apache.org/repos/asf/beam.git
commit dd82f9d9bc1b0b0d03fe1c910a44b52962fb1964
Author: tvalentyn
AuthorDate: Tue Feb 18 11:51:18 2020 -0800
Revert "[BEAM-6522] [BEAM-7455] Unskip Avro IO tests that are now passing.
(#10838)"
This reverts commit 49863a465b368d3a860b1d7c9ee46faa8e6b55e7.
---
sdks/python/apache_beam/examples/fastavro_it_test.py | 8 +++-
sdks/python/apache_beam/io/avroio_test.py| 4
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/sdks/python/apache_beam/examples/fastavro_it_test.py
b/sdks/python/apache_beam/examples/fastavro_it_test.py
index d038d3e..2279ec3 100644
--- a/sdks/python/apache_beam/examples/fastavro_it_test.py
+++ b/sdks/python/apache_beam/examples/fastavro_it_test.py
@@ -49,6 +49,8 @@ from __future__ import division
import json
import logging
+import os
+import sys
import unittest
import uuid
@@ -86,6 +88,10 @@ def record(i):
}
+@unittest.skipIf(
+sys.version_info[0] >= 3 and os.environ.get('RUN_SKIPPED_PY3_TESTS') !=
'1',
+'Due to a known issue in avro-python3 package, this'
+'test is skipped until BEAM-6522 is addressed. ')
class FastavroIT(unittest.TestCase):
SCHEMA_STRING = '''
@@ -175,7 +181,7 @@ class FastavroIT(unittest.TestCase):
if l != r:
raise BeamAssertException('Assertion failed: %s == %s' % (l, r))
-assertEqual(sorted(v.keys()), ['avro', 'fastavro'])
+assertEqual(v.keys(), ['avro', 'fastavro'])
avro_values = v['avro']
fastavro_values = v['fastavro']
assertEqual(avro_values, fastavro_values)
diff --git a/sdks/python/apache_beam/io/avroio_test.py
b/sdks/python/apache_beam/io/avroio_test.py
index bfa213c..79036ca 100644
--- a/sdks/python/apache_beam/io/avroio_test.py
+++ b/sdks/python/apache_beam/io/avroio_test.py
@@ -445,6 +445,10 @@ class AvroBase(object):
assert_that(readback, equal_to([json.dumps(r) for r in self.RECORDS]))
+@unittest.skipIf(
+sys.version_info[0] == 3 and os.environ.get('RUN_SKIPPED_PY3_TESTS') !=
'1',
+'This test still needs to be fixed on Python 3. '
+'TODO: BEAM-6522.')
class TestAvro(AvroBase, unittest.TestCase):
def __init__(self, methodName='runTest'):
super(TestAvro, self).__init__(methodName)
[beam] branch revert-10838-avro created (now dd82f9d)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch revert-10838-avro in repository https://gitbox.apache.org/repos/asf/beam.git. at dd82f9d Revert "[BEAM-6522] [BEAM-7455] Unskip Avro IO tests that are now passing. (#10838)" This branch includes the following new commits: new dd82f9d Revert "[BEAM-6522] [BEAM-7455] Unskip Avro IO tests that are now passing. (#10838)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] branch master updated (945b0bc -> b483ddb)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 945b0bc [BEAM-9258] Integrate Google Cloud Data loss prevention functionality for Python SDK (#10849) add b483ddb [BEAM-9291] Upload graph option in dataflow's python sdk (#10829) No new revisions were added by this update. Summary of changes: .../runners/dataflow/dataflow_runner_test.py | 11 + .../runners/dataflow/internal/apiclient.py | 10 .../runners/dataflow/internal/apiclient_test.py| 27 ++ 3 files changed, 48 insertions(+)
[beam] branch master updated (2223871 -> 945b0bc)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2223871 Merge pull request #10880: Change static Map fields in ReflectUtils to be concurrent add 945b0bc [BEAM-9258] Integrate Google Cloud Data loss prevention functionality for Python SDK (#10849) No new revisions were added by this update. Summary of changes: CHANGES.md | 1 + sdks/python/apache_beam/ml/__init__.py | 17 ++ sdks/python/apache_beam/ml/gcp/__init__.py | 17 ++ sdks/python/apache_beam/ml/gcp/cloud_dlp.py | 226 +++ sdks/python/apache_beam/ml/gcp/cloud_dlp_test.py | 136 ++ sdks/python/setup.py | 1 + 6 files changed, 398 insertions(+) create mode 100644 sdks/python/apache_beam/ml/__init__.py create mode 100644 sdks/python/apache_beam/ml/gcp/__init__.py create mode 100644 sdks/python/apache_beam/ml/gcp/cloud_dlp.py create mode 100644 sdks/python/apache_beam/ml/gcp/cloud_dlp_test.py
[beam] branch master updated (2fceda3 -> 2223871)
This is an automated email from the ASF dual-hosted git repository. gleb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2fceda3 Merge pull request #10290: [BEAM-8561] Add ThriftIO to support IO for Thrift files add d50e12f Change static Map fields in ReflectUtils to be concurrent add 2223871 Merge pull request #10880: Change static Map fields in ReflectUtils to be concurrent No new revisions were added by this update. Summary of changes: .../main/java/org/apache/beam/sdk/schemas/utils/ReflectUtils.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[beam] branch master updated (52fea0e -> 2fceda3)
This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 52fea0e Merge pull request #10862: [BEAM-9320] Add AlwaysFetched
annotation
add 4cdbb1e [BEAM-8561] Add ThriftIO to support IO for Thrift files
add 2fceda3 Merge pull request #10290: [BEAM-8561] Add ThriftIO to
support IO for Thrift files
No new revisions were added by this update.
Summary of changes:
.github/autolabeler.yml| 1 +
.../src/main/resources/beam/suppressions.xml | 4 +
sdks/java/io/thrift/build.gradle | 57 +
.../org/apache/beam/sdk/io/thrift/ThriftCoder.java | 103
.../org/apache/beam/sdk/io/thrift/ThriftIO.java| 283 +
.../apache/beam/sdk/io/thrift}/package-info.java | 8 +-
.../apache/beam/sdk/io/thrift/ThriftIOTest.java| 256 +++
.../io/thrift/src/test/resources/ThriftIOTest/data | Bin 0 -> 92 bytes
.../src/test/resources/thrift/thrift_test.thrift} | 19 +-
settings.gradle| 1 +
10 files changed, 722 insertions(+), 10 deletions(-)
create mode 100644 sdks/java/io/thrift/build.gradle
create mode 100644
sdks/java/io/thrift/src/main/java/org/apache/beam/sdk/io/thrift/ThriftCoder.java
create mode 100644
sdks/java/io/thrift/src/main/java/org/apache/beam/sdk/io/thrift/ThriftIO.java
copy sdks/java/io/{jdbc/src/main/java/org/apache/beam/sdk/io/jdbc =>
thrift/src/main/java/org/apache/beam/sdk/io/thrift}/package-info.java (78%)
create mode 100644
sdks/java/io/thrift/src/test/java/org/apache/beam/sdk/io/thrift/ThriftIOTest.java
create mode 100644 sdks/java/io/thrift/src/test/resources/ThriftIOTest/data
copy
sdks/java/{core/src/main/java/org/apache/beam/sdk/testing/UsesCounterMetrics.java
=> io/thrift/src/test/resources/thrift/thrift_test.thrift} (73%)
