[beam] branch aaltay-patch-2 created (now a5f9d85)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch aaltay-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git. at a5f9d85 Fixing a logging bug in auth This branch includes the following new commits: new a5f9d85 Fixing a logging bug in auth The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Fixing a logging bug in auth
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch aaltay-patch-2
in repository https://gitbox.apache.org/repos/asf/beam.git
commit a5f9d850dd054fc43e3b8468f0e9f03951d1e8e1
Author: Ahmet Altay
AuthorDate: Mon Oct 14 17:34:54 2019 -0700
Fixing a logging bug in auth
---
sdks/python/apache_beam/internal/gcp/auth.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sdks/python/apache_beam/internal/gcp/auth.py
b/sdks/python/apache_beam/internal/gcp/auth.py
index f8c1254..5142a34 100644
--- a/sdks/python/apache_beam/internal/gcp/auth.py
+++ b/sdks/python/apache_beam/internal/gcp/auth.py
@@ -104,7 +104,7 @@ class _Credentials(object):
logging.info("Setting socket default timeout to 60 seconds.")
socket.setdefaulttimeout(60)
logging.info(
- "socket default timeout is % seconds.", socket.getdefaulttimeout())
+ "socket default timeout is %s seconds.", socket.getdefaulttimeout())
cls._credentials = cls._get_service_credentials()
cls._credentials_init = True
[beam] branch aaltay-patch-1 created (now 971da36)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch aaltay-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git. at 971da36 Fixing a logging bug No new revisions were added by this update.
[beam] branch master updated (b8bef24 -> 2a367c3)
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from b8bef24 Merge pull request #9773 from RyanSkraba/BEAM-7073-avro-sql-unit-test add 2a367c3 [BEAM-4046] Remove old project name mappings. (#8915) No new revisions were added by this update. Summary of changes: build.gradle| 4 - deprecation-warning.txt | 18 - gradlew | 184 -- gradlew.bat | 93 +++- gradlew_orig| 190 gradlew_orig.bat| 102 -- project-mappings| 115 - 7 files changed, 236 insertions(+), 470 deletions(-) delete mode 100644 deprecation-warning.txt delete mode 100755 gradlew_orig delete mode 100644 gradlew_orig.bat delete mode 100644 project-mappings
[beam] branch master updated: [BEAM-7073]: Add unit test for Avro logical type datum.
This is an automated email from the ASF dual-hosted git repository.
amaliujia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 8a661a4 [BEAM-7073]: Add unit test for Avro logical type datum.
new b8bef24 Merge pull request #9773 from
RyanSkraba/BEAM-7073-avro-sql-unit-test
8a661a4 is described below
commit 8a661a45947af9f86d9e365097ee84767934fb5d
Author: Ryan Skraba
AuthorDate: Fri Oct 11 16:22:55 2019 +0200
[BEAM-7073]: Add unit test for Avro logical type datum.
---
.../test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java | 6 ++
1 file changed, 6 insertions(+)
diff --git
a/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java
b/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java
index cedeb77..3679e21 100644
---
a/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java
+++
b/sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java
@@ -502,8 +502,14 @@ public class AvroUtilsTest {
@Test
public void testGenericRecordToBeamRow() {
+GenericRecord genericRecord = getGenericRecord();
Row row = AvroUtils.toBeamRowStrict(getGenericRecord(), null);
assertEquals(getBeamRow(), row);
+
+// Alternatively, a timestamp-millis logical type can have a joda datum.
+genericRecord.put("timestampMillis", new
DateTime(genericRecord.get("timestampMillis")));
+row = AvroUtils.toBeamRowStrict(getGenericRecord(), null);
+assertEquals(getBeamRow(), row);
}
@Test
[beam] branch master updated (0a00a8f -> 6ab60dc)
This is an automated email from the ASF dual-hosted git repository. thw pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0a00a8f Merge pull request #9786 from davidcavazos/remove-old-element-wise-snippets add 71b9292 [BEAM-8387] Remove sdk-worker-parallelism option from JobServerDriver add 6ab60dc Merge pull request #9785: [BEAM-8387] Remove sdk-worker-parallelism option from JobServerDriver No new revisions were added by this update. Summary of changes: .../java/org/apache/beam/runners/flink/FlinkJobInvoker.java | 3 --- .../apache/beam/runners/flink/FlinkJobServerDriverTest.java | 3 --- .../runners/fnexecution/jobsubmission/JobServerDriver.java | 13 - 3 files changed, 19 deletions(-)
[beam] branch master updated: [BEAM-7389] Remove old element-wise snippets directory
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 3ed2684 [BEAM-7389] Remove old element-wise snippets directory
new 0a00a8f Merge pull request #9786 from
davidcavazos/remove-old-element-wise-snippets
3ed2684 is described below
commit 3ed2684a74c50e9e081bc24a937c7da07df8a821
Author: David Cavazos
AuthorDate: Mon Oct 14 10:07:35 2019 -0700
[BEAM-7389] Remove old element-wise snippets directory
---
.../snippets/transforms/element_wise/__init__.py | 18 --
.../snippets/transforms/element_wise/filter.py | 182
.../transforms/element_wise/filter_test.py | 80 ---
.../snippets/transforms/element_wise/keys.py | 42
.../snippets/transforms/element_wise/keys_test.py | 55 -
.../snippets/transforms/element_wise/kvswap.py | 42
.../transforms/element_wise/kvswap_test.py | 55 -
.../snippets/transforms/element_wise/map.py| 226
.../snippets/transforms/element_wise/map_test.py | 90
.../snippets/transforms/element_wise/pardo.py | 126 ---
.../snippets/transforms/element_wise/pardo_test.py | 120 ---
.../snippets/transforms/element_wise/partition.py | 136
.../transforms/element_wise/partition_test.py | 84
.../snippets/transforms/element_wise/regex.py | 236 -
.../snippets/transforms/element_wise/regex_test.py | 173 ---
.../snippets/transforms/element_wise/values.py | 42
.../transforms/element_wise/values_test.py | 55 -
17 files changed, 1762 deletions(-)
diff --git
a/sdks/python/apache_beam/examples/snippets/transforms/element_wise/__init__.py
b/sdks/python/apache_beam/examples/snippets/transforms/element_wise/__init__.py
deleted file mode 100644
index 6569e3f..000
---
a/sdks/python/apache_beam/examples/snippets/transforms/element_wise/__init__.py
+++ /dev/null
@@ -1,18 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-#http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-from __future__ import absolute_import
diff --git
a/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py
b/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py
deleted file mode 100644
index 44b11b8..000
---
a/sdks/python/apache_beam/examples/snippets/transforms/element_wise/filter.py
+++ /dev/null
@@ -1,182 +0,0 @@
-# coding=utf-8
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-#http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-from __future__ import absolute_import
-from __future__ import print_function
-
-
-def filter_function(test=None):
- # [START filter_function]
- import apache_beam as beam
-
- def is_perennial(plant):
-return plant['duration'] == 'perennial'
-
- with beam.Pipeline() as pipeline:
-perennials = (
-pipeline
-| 'Gardening plants' >> beam.Create([
-{'icon': '', 'name': 'Strawberry', 'duration': 'perennial'},
-{'icon': '凌', 'name': 'Carrot', 'duration': 'biennial'},
-{'icon': '', 'name': 'Eggplant', 'duration': 'perennial'},
-{'icon': '', 'name': 'Tomato', 'duration': 'annual'},
-{'icon': '凜', 'name': 'Potato', 'duration': 'perennial'},
-])
-| 'Filter perennials' >> beam.Filter(is_perennial)
-| beam.Map(print)
-)
-# [END
[beam] branch master updated: [BEAM-8368] Add a <0.14.0 bound for pyarrow
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 383f7cd [BEAM-8368] Add a <0.14.0 bound for pyarrow
new 246cc8d Merge pull request #9768 from kamilwu/pyarrow-downgrade
383f7cd is described below
commit 383f7cda632f80b0dd9fe3336af858b71a607b64
Author: Kamil Wasilewski
AuthorDate: Fri Oct 11 10:14:00 2019 +0200
[BEAM-8368] Add a <0.14.0 bound for pyarrow
---
sdks/python/setup.py | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/sdks/python/setup.py b/sdks/python/setup.py
index 3e42485..4548488 100644
--- a/sdks/python/setup.py
+++ b/sdks/python/setup.py
@@ -119,7 +119,10 @@ REQUIRED_PACKAGES = [
'oauth2client>=2.0.1,<4',
'protobuf>=3.5.0.post1,<4',
# [BEAM-6287] pyarrow is not supported on Windows for Python 2
-('pyarrow>=0.11.1,<0.15.0; python_version >= "3.0" or '
+# [BEAM-8392] pyarrow 0.14.0 and 0.15.0 triggers an exception when
importing
+# apache_beam on macOS 10.15. Update version bounds as soon as the fix is
+# ready
+('pyarrow>=0.11.1,<0.14.0; python_version >= "3.0" or '
'platform_system != "Windows"'),
'pydot>=1.2.0,<2',
'python-dateutil>=2.8.0,<3',
[beam] branch master updated (8543801 -> 1c0ab42)
This is an automated email from the ASF dual-hosted git repository. apilloud pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 8543801 Merge pull request #9783: [BEAM-8391] Update type id write methods in add 453db82 Add project push-down capability to TestTableProvider add 1c0ab42 Merge pull request #9743: [BEAM-8365] Project push-down for test table provider No new revisions were added by this update. Summary of changes: .../sql/meta/provider/test/TestTableProvider.java | 28 + .../meta/provider/test/TestTableProviderTest.java | 121 + 2 files changed, 149 insertions(+) create mode 100644 sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/test/TestTableProviderTest.java
[beam] branch master updated (5a851b7 -> 8543801)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5a851b7 Merge pull request #8677 from Riduidel/fix/rabbitmq-message-not-serializable add b68d70a [BEAM-8391] Update type id write methods in `AwsModule` from deprecated (in jackson 2.9) to supported ones. add 8543801 Merge pull request #9783: [BEAM-8391] Update type id write methods in No new revisions were added by this update. Summary of changes: .../main/java/org/apache/beam/sdk/io/aws/options/AwsModule.java | 9 ++--- .../main/java/org/apache/beam/sdk/io/aws2/options/AwsModule.java | 8 ++-- 2 files changed, 12 insertions(+), 5 deletions(-)
[beam] branch master updated (5a851b7 -> 8543801)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5a851b7 Merge pull request #8677 from Riduidel/fix/rabbitmq-message-not-serializable add b68d70a [BEAM-8391] Update type id write methods in `AwsModule` from deprecated (in jackson 2.9) to supported ones. add 8543801 Merge pull request #9783: [BEAM-8391] Update type id write methods in No new revisions were added by this update. Summary of changes: .../main/java/org/apache/beam/sdk/io/aws/options/AwsModule.java | 9 ++--- .../main/java/org/apache/beam/sdk/io/aws2/options/AwsModule.java | 8 ++-- 2 files changed, 12 insertions(+), 5 deletions(-)
