[beam] branch master updated (3fafec1 -> 1518361)
This is an automated email from the ASF dual-hosted git repository. thw pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 3fafec1 [BEAM-5320] [BEAM-6106] Finish Python 3 porting for testing module (#7262) add bd0103c [BEAM-5978] Changing parallalim for wordcount to 1 new 1518361 Merge pull request #7174: [BEAM-5978] Changing parallelism for wordcount to 1 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/python/build.gradle | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-)
[beam] 01/01: Merge pull request #7174: [BEAM-5978] Changing parallelism for wordcount to 1
This is an automated email from the ASF dual-hosted git repository. thw pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 15183612e6a3f6b0eb4dca88a6bf90a92a456805 Merge: 3fafec1 bd0103c Author: Thomas Weise AuthorDate: Wed Dec 12 17:49:28 2018 -0800 Merge pull request #7174: [BEAM-5978] Changing parallelism for wordcount to 1 sdks/python/build.gradle | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-)
[beam] branch master updated: [BEAM-5320] [BEAM-6106] Finish Python 3 porting for testing module (#7262)
This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 3fafec1 [BEAM-5320] [BEAM-6106] Finish Python 3 porting for testing
module (#7262)
3fafec1 is described below
commit 3fafec15d88cd774e581243fa52f90370586e633
Author: Robbe Sneyders
AuthorDate: Thu Dec 13 01:42:49 2018 +0100
[BEAM-5320] [BEAM-6106] Finish Python 3 porting for testing module (#7262)
* Port testing module to Python 3
---
.../apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py| 2 +-
sdks/python/apache_beam/testing/synthetic_pipeline.py | 2 +-
sdks/python/apache_beam/testing/synthetic_pipeline_test.py | 2 +-
sdks/python/apache_beam/testing/test_utils.py | 7 +--
sdks/python/tox.ini| 2 +-
5 files changed, 9 insertions(+), 6 deletions(-)
diff --git
a/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py
b/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py
index 21e9c48..f7eb93b 100644
--- a/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py
+++ b/sdks/python/apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py
@@ -41,7 +41,7 @@ class BigqueryTornadoesIT(unittest.TestCase):
# The default checksum is a SHA-1 hash generated from sorted rows reading
# from expected Bigquery table.
- DEFAULT_CHECKSUM = '83789a7c1bca7959dcf23d3bc37e9204e594330f'
+ DEFAULT_CHECKSUM = 'd860e636050c559a16a791aff40d6ad809d4daf0'
@attr('IT')
def test_bigquery_tornadoes_it(self):
diff --git a/sdks/python/apache_beam/testing/synthetic_pipeline.py
b/sdks/python/apache_beam/testing/synthetic_pipeline.py
index 2f8980b..b560f35 100644
--- a/sdks/python/apache_beam/testing/synthetic_pipeline.py
+++ b/sdks/python/apache_beam/testing/synthetic_pipeline.py
@@ -71,7 +71,7 @@ def div_round_up(a, b):
def rotate_key(element):
"""Returns a new key-value pair of the same size but with a different key."""
(key, value) = element
- return key[-1] + key[:-1], value
+ return key[-1:] + key[:-1], value
class SyntheticStep(beam.DoFn):
diff --git a/sdks/python/apache_beam/testing/synthetic_pipeline_test.py
b/sdks/python/apache_beam/testing/synthetic_pipeline_test.py
index fe5e94a..e786553 100644
--- a/sdks/python/apache_beam/testing/synthetic_pipeline_test.py
+++ b/sdks/python/apache_beam/testing/synthetic_pipeline_test.py
@@ -125,7 +125,7 @@ class SyntheticPipelineTest(unittest.TestCase):
if writes_output:
read_output = []
for file_name in glob.glob(output_location + '*'):
-with open(file_name, 'r') as f:
+with open(file_name, 'rb') as f:
read_output.extend(f.read().splitlines())
self.assertEqual(10, len(read_output))
diff --git a/sdks/python/apache_beam/testing/test_utils.py
b/sdks/python/apache_beam/testing/test_utils.py
index 1f0e99e..f9aa128 100644
--- a/sdks/python/apache_beam/testing/test_utils.py
+++ b/sdks/python/apache_beam/testing/test_utils.py
@@ -75,11 +75,14 @@ class TempDir(object):
def compute_hash(content, hashing_alg=DEFAULT_HASHING_ALG):
- """Compute a hash value from a list of string."""
+ """Compute a hash value of a list of objects by hashing their string
+ representations."""
+ content = [str(x).encode('utf-8') if not isinstance(x, bytes) else x
+ for x in content]
content.sort()
m = hashlib.new(hashing_alg)
for elem in content:
-m.update(str(elem).encode('utf-8'))
+m.update(elem)
return m.hexdigest()
diff --git a/sdks/python/tox.ini b/sdks/python/tox.ini
index a357a2f..2818870 100644
--- a/sdks/python/tox.ini
+++ b/sdks/python/tox.ini
@@ -58,7 +58,7 @@ setenv =
BEAM_EXPERIMENTAL_PY3=1
RUN_SKIPPED_PY3_TESTS=0
modules =
-
apache_beam.typehints,apache_beam.coders,apache_beam.options,apache_beam.tools,apache_beam.utils,apache_beam.internal,apache_beam.metrics,apache_beam.portability,apache_beam.pipeline_test,apache_beam.pvalue_test,apache_beam.runners,apache_beam.io.hadoopfilesystem_test,apache_beam.io.hdfs_integration_test,apache_beam.io.gcp.tests.utils_test,apache_beam.io.gcp.big_query_query_to_table_it_test,apache_beam.io.gcp.bigquery_io_read_it_test,apache_beam.io.gcp.bigquery_test,apache_beam.io.gcp.
[...]
+
apache_beam.typehints,apache_beam.coders,apache_beam.options,apache_beam.tools,apache_beam.utils,apache_beam.internal,apache_beam.metrics,apache_beam.portability,apache_beam.pipeline_test,apache_beam.pvalue_test,apache_beam.runners,apache_beam.io.hadoopfilesystem_test,apache_beam.io.hdfs_integration_test,apache_beam.io.gcp.tests.utils_test,apache_beam.io.gcp.big_query_query_to_table_it_test,apache_beam.io.gcp.bigquery_io_read_it_test,apache_beam.io.gcp.bigquery_test,apache_beam.io.gcp.
[...]
commands =
python --version
pip
[beam] branch master updated (3400ba6 -> 8b20602)
This is an automated email from the ASF dual-hosted git repository.
reuvenlax pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 3400ba6 Merge pull request #7229: [BEAM-5419] Add Flink multi-version
builds
add 2a40c57 Add remaining Schema support for AVRO records: * Add
support for SpecificRecord using ByteBuddy codegen. * Add helper methods for
GenericRecord. * Fix uncovered bugs involving nullable support.
add 0a74b17 Add period to sentence.
new 8b20602 Merge pull request #7233: [BEAM-4454] Add remaining
functionality for AVRO schemas
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build.gradle | 1 +
...y.java => AvroSpecificRecordGetterFactory.java} | 13 +-
.../beam/sdk/schemas/AvroSpecificRecordSchema.java | 51 +
... AvroSpecificRecordUserTypeCreatorFactory.java} | 11 +-
.../sdk/schemas/GetterBasedSchemaProvider.java | 41 +---
.../apache/beam/sdk/schemas/JavaBeanSchema.java| 8 +-
.../apache/beam/sdk/schemas/JavaFieldSchema.java | 6 -
.../java/org/apache/beam/sdk/schemas/Schema.java | 8 +
.../sdk/schemas/SetterBasedCreatorFactory.java | 57 +
.../beam/sdk/schemas/utils/AvroByteBuddyUtils.java | 123 +++
... AvroSpecificRecordTypeInformationFactory.java} | 7 +-
.../apache/beam/sdk/schemas/utils/AvroUtils.java | 243 -
.../beam/sdk/schemas/utils/ByteBuddyUtils.java | 50 +
.../sdk/schemas/utils/JavaBeanGetterFactory.java | 3 +-
.../utils/JavaBeanTypeInformationFactory.java | 3 +-
.../beam/sdk/schemas/utils/JavaBeanUtils.java | 43 ++--
.../sdk/schemas/utils/StaticSchemaInference.java | 6 +-
.../avro/org/apache/beam/sdk/schemas/test.avsc | 29 +++
.../apache/beam/sdk/schemas/AvroSchemaTest.java| 162 ++
.../beam/sdk/schemas/utils/AvroUtilsTest.java | 41 +++-
.../beam/sdk/schemas/utils/JavaBeanUtilsTest.java | 43 +++-
21 files changed, 804 insertions(+), 145 deletions(-)
copy
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/{utils/PojoValueGetterFactory.java
=> AvroSpecificRecordGetterFactory.java} (69%)
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/AvroSpecificRecordSchema.java
copy
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/{PojoTypeUserTypeCreatorFactory.java
=> AvroSpecificRecordUserTypeCreatorFactory.java} (72%)
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SetterBasedCreatorFactory.java
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/AvroByteBuddyUtils.java
copy
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/{PojoValueTypeInformationFactory.java
=> AvroSpecificRecordTypeInformationFactory.java} (78%)
create mode 100644
sdks/java/core/src/test/avro/org/apache/beam/sdk/schemas/test.avsc
create mode 100644
sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/AvroSchemaTest.java
[beam] 01/01: Merge pull request #7233: [BEAM-4454] Add remaining functionality for AVRO schemas
This is an automated email from the ASF dual-hosted git repository. reuvenlax pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 8b20602f88959becbdd1c366d1e63a7fbbc8f62d Merge: 3400ba6 0a74b17 Author: reuvenlax AuthorDate: Wed Dec 12 13:16:03 2018 -0800 Merge pull request #7233: [BEAM-4454] Add remaining functionality for AVRO schemas build.gradle | 1 + ...y.java => AvroSpecificRecordGetterFactory.java} | 13 +- .../beam/sdk/schemas/AvroSpecificRecordSchema.java | 51 + ... AvroSpecificRecordUserTypeCreatorFactory.java} | 16 +- .../sdk/schemas/GetterBasedSchemaProvider.java | 41 +--- .../apache/beam/sdk/schemas/JavaBeanSchema.java| 8 +- .../apache/beam/sdk/schemas/JavaFieldSchema.java | 6 - .../java/org/apache/beam/sdk/schemas/Schema.java | 8 + .../sdk/schemas/SetterBasedCreatorFactory.java | 57 + .../beam/sdk/schemas/utils/AvroByteBuddyUtils.java | 123 +++ ... AvroSpecificRecordTypeInformationFactory.java} | 7 +- .../apache/beam/sdk/schemas/utils/AvroUtils.java | 243 - .../beam/sdk/schemas/utils/ByteBuddyUtils.java | 50 + .../sdk/schemas/utils/JavaBeanGetterFactory.java | 3 +- .../utils/JavaBeanTypeInformationFactory.java | 3 +- .../beam/sdk/schemas/utils/JavaBeanUtils.java | 43 ++-- .../sdk/schemas/utils/StaticSchemaInference.java | 6 +- .../avro/org/apache/beam/sdk/schemas/test.avsc | 29 +++ .../apache/beam/sdk/schemas/AvroSchemaTest.java| 162 ++ .../beam/sdk/schemas/utils/AvroUtilsTest.java | 41 +++- .../beam/sdk/schemas/utils/JavaBeanUtilsTest.java | 43 +++- 21 files changed, 806 insertions(+), 148 deletions(-)
[beam] branch master updated (20df151 -> 3400ba6)
This is an automated email from the ASF dual-hosted git repository.
mxm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 20df151 Merge pull request #7265: Fixing publishing problem
introduced in #7197
add 07bae67 [BEAM-5419] Add Flink multi-version build layout
add 6308a6d [BEAM-5267] Make source Flink v1.6 compatible
add d0254f0 [BEAM-5267] Add Flink v1.6 target for Flink Runner
new 3400ba6 Merge pull request #7229: [BEAM-5419] Add Flink multi-version
builds
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
{model/pipeline => runners/flink/1.6}/build.gradle | 22 ++--
.../flink/1.6/job-server-container}/build.gradle | 9 +-
.../flink/1.6/job-server}/build.gradle | 20 ++--
runners/flink/build.gradle | 129 ++---
.../flink/{build.gradle => flink_runner.gradle}| 34 +-
runners/flink/job-server-container/build.gradle| 39 +--
...ld.gradle => flink_job_server_container.gradle} | 18 ++-
runners/flink/job-server/build.gradle | 103 ++--
.../{build.gradle => flink_job_server.gradle} | 98 ++--
.../state/FlinkKeyGroupStateInternals.java | 3 +-
settings.gradle| 10 ++
11 files changed, 170 insertions(+), 315 deletions(-)
copy {model/pipeline => runners/flink/1.6}/build.gradle (59%)
copy {sdks/java/build-tools =>
runners/flink/1.6/job-server-container}/build.gradle (73%)
copy {sdks/java/io/common => runners/flink/1.6/job-server}/build.gradle (65%)
copy runners/flink/{build.gradle => flink_runner.gradle} (89%)
copy runners/flink/job-server-container/{build.gradle =>
flink_job_server_container.gradle} (71%)
copy runners/flink/job-server/{build.gradle => flink_job_server.gradle} (52%)
[beam] 01/01: Merge pull request #7229: [BEAM-5419] Add Flink multi-version builds
This is an automated email from the ASF dual-hosted git repository.
mxm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 3400ba6e703b02c093721a0232a536c7c5ea9061
Merge: 20df151 d0254f0
Author: Maximilian Michels
AuthorDate: Wed Dec 12 21:52:20 2018 +0100
Merge pull request #7229: [BEAM-5419] Add Flink multi-version builds
runners/flink/1.6/build.gradle | 33 ++
.../flink/1.6/job-server-container/build.gradle| 25
runners/flink/1.6/job-server/build.gradle | 30 +
runners/flink/build.gradle | 129 ++---
.../flink/{build.gradle => flink_runner.gradle}| 34 +-
runners/flink/job-server-container/build.gradle| 39 +--
...ld.gradle => flink_job_server_container.gradle} | 18 ++-
runners/flink/job-server/build.gradle | 103 ++--
.../{build.gradle => flink_job_server.gradle} | 98 ++--
.../state/FlinkKeyGroupStateInternals.java | 3 +-
settings.gradle| 10 ++
11 files changed, 230 insertions(+), 292 deletions(-)
[beam] 01/01: Merge pull request #7265: Fixing publishing problem introduced in #7197
This is an automated email from the ASF dual-hosted git repository. scott pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 20df151f8128110689df6248291805890139794c Merge: dc10f75 41719ac Author: Scott Wegner AuthorDate: Wed Dec 12 12:50:19 2018 -0800 Merge pull request #7265: Fixing publishing problem introduced in #7197 buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch master updated (dc10f75 -> 20df151)
This is an automated email from the ASF dual-hosted git repository. scott pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from dc10f75 Merge pull request #7244: [BEAM-6138] Add a MonitoringInfoSpec proto and SimpleMonitoringInfoBuilder to pro… add 41719ac Fixing publishing problem introduced in #7197 new 20df151 Merge pull request #7265: Fixing publishing problem introduced in #7197 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] 01/01: Merge pull request #7244: [BEAM-6138] Add a MonitoringInfoSpec proto and SimpleMonitoringInfoBuilder to pro…
This is an automated email from the ASF dual-hosted git repository. scott pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit dc10f75a7bda91f251d43309a21327fc2ab26154 Merge: a61f2c5 11513c3 Author: Scott Wegner AuthorDate: Wed Dec 12 10:44:28 2018 -0800 Merge pull request #7244: [BEAM-6138] Add a MonitoringInfoSpec proto and SimpleMonitoringInfoBuilder to pro… .../fn-execution/src/main/proto/beam_fn_api.proto | 269 ++--- .../core/metrics/SimpleMonitoringInfoBuilder.java | 219 + .../metrics/SimpleMonitoringInfoBuilderTest.java | 87 +++ 3 files changed, 482 insertions(+), 93 deletions(-)
[beam] branch master updated (a61f2c5 -> dc10f75)
This is an automated email from the ASF dual-hosted git repository. scott pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a61f2c5 Merge pull request #7242: Add a MonitoringInfoLabelProps proto to attach the proper key string name to MonitoringInfo label Enums add c314dfe Add a MonitoringInfoSpec proto and SimpleMonitoringInfoBuilder to provide specs and validate MonitoringInfos are properly populated. add 11513c3 Merge remote-tracking branch 'upstream/master' into pr7244 new dc10f75 Merge pull request #7244: [BEAM-6138] Add a MonitoringInfoSpec proto and SimpleMonitoringInfoBuilder to pro… The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fn-execution/src/main/proto/beam_fn_api.proto | 269 ++--- .../core/metrics/SimpleMonitoringInfoBuilder.java | 219 + .../metrics/SimpleMonitoringInfoBuilderTest.java | 87 +++ 3 files changed, 482 insertions(+), 93 deletions(-) create mode 100644 runners/core-java/src/main/java/org/apache/beam/runners/core/metrics/SimpleMonitoringInfoBuilder.java create mode 100644 runners/core-java/src/test/java/org/apache/beam/runners/core/metrics/SimpleMonitoringInfoBuilderTest.java
[beam] 01/01: Merge pull request #7242: Add a MonitoringInfoLabelProps proto to attach the proper key string name to MonitoringInfo label Enums
This is an automated email from the ASF dual-hosted git repository. scott pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit a61f2c59076eca6777c995798de9bbc207c780f1 Merge: 5574f47 8828b16 Author: Scott Wegner AuthorDate: Wed Dec 12 08:45:07 2018 -0800 Merge pull request #7242: Add a MonitoringInfoLabelProps proto to attach the proper key string name to MonitoringInfo label Enums .../fn-execution/src/main/proto/beam_fn_api.proto | 34 ++ 1 file changed, 29 insertions(+), 5 deletions(-)
[beam] branch master updated (5574f47 -> a61f2c5)
This is an automated email from the ASF dual-hosted git repository. scott pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5574f47 Merge pull request #7243: [BEAM-6205] Setup gradle task ro run fnapi worker test with use_execu… add 8828b16 Add a MonitoringInfoLabelProps proto to attach the proper key string name to MonitoringInfo label Enums new a61f2c5 Merge pull request #7242: Add a MonitoringInfoLabelProps proto to attach the proper key string name to MonitoringInfo label Enums The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fn-execution/src/main/proto/beam_fn_api.proto | 34 ++ 1 file changed, 29 insertions(+), 5 deletions(-)
[beam] branch master updated (798b3b3 -> 5574f47)
This is an automated email from the ASF dual-hosted git repository. scott pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 798b3b3 Merge pull request #7241: [BEAM-6204] Clean up duplicated SocketAddressFactory class. add 5e506bf [BEAM-6205] Setup gradle task ro run fnapi worker test with use_executable_stage_bundle_execution new 5574f47 Merge pull request #7243: [BEAM-6205] Setup gradle task ro run fnapi worker test with use_execu… The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: runners/google-cloud-dataflow-java/build.gradle | 4 +++- runners/google-cloud-dataflow-java/examples/build.gradle | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
[beam] 01/01: Merge pull request #7243: [BEAM-6205] Setup gradle task ro run fnapi worker test with use_execu…
This is an automated email from the ASF dual-hosted git repository. scott pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 5574f4739d2d205e909f8f6ad4bb6fdf8fa37857 Merge: 798b3b3 5e506bf Author: Scott Wegner AuthorDate: Wed Dec 12 08:35:47 2018 -0800 Merge pull request #7243: [BEAM-6205] Setup gradle task ro run fnapi worker test with use_execu… runners/google-cloud-dataflow-java/build.gradle | 4 +++- runners/google-cloud-dataflow-java/examples/build.gradle | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
[beam] branch master updated (a53f56a -> 798b3b3)
This is an automated email from the ASF dual-hosted git repository. scott pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a53f56a Merge pull request #7261 from echauchot/BEAM-6216-flink-local add 9e8ac83 [BEAM-6240] Clean up duplicated SocketAddressFactory class. new 798b3b3 Merge pull request #7241: [BEAM-6204] Clean up duplicated SocketAddressFactory class. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../dataflow/worker/fn/SocketAddressFactory.java | 68 -- .../worker/fn/SocketAddressFactoryTest.java| 55 - 2 files changed, 123 deletions(-) delete mode 100644 runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/SocketAddressFactory.java delete mode 100644 runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/SocketAddressFactoryTest.java
[beam] 01/01: Merge pull request #7241: [BEAM-6204] Clean up duplicated SocketAddressFactory class.
This is an automated email from the ASF dual-hosted git repository. scott pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 798b3b3edb5549602d979f28afffea86649f0ed1 Merge: a53f56a 9e8ac83 Author: Scott Wegner AuthorDate: Wed Dec 12 08:34:20 2018 -0800 Merge pull request #7241: [BEAM-6204] Clean up duplicated SocketAddressFactory class. .../dataflow/worker/fn/SocketAddressFactory.java | 68 -- .../worker/fn/SocketAddressFactoryTest.java| 55 - 2 files changed, 123 deletions(-)
[beam] branch asf-site updated: Publishing website 2018/12/12 16:11:34 at commit a53f56a
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new b0b5d8c Publishing website 2018/12/12 16:11:34 at commit a53f56a b0b5d8c is described below commit b0b5d8c4d0ff8c4e3f5a98dfe890bc1cea7ee450 Author: jenkins AuthorDate: Wed Dec 12 16:11:35 2018 + Publishing website 2018/12/12 16:11:34 at commit a53f56a --- .../documentation/sdks/java/testing/nexmark/index.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/generated-content/documentation/sdks/java/testing/nexmark/index.html b/website/generated-content/documentation/sdks/java/testing/nexmark/index.html index a552cca..8e7a411 100644 --- a/website/generated-content/documentation/sdks/java/testing/nexmark/index.html +++ b/website/generated-content/documentation/sdks/java/testing/nexmark/index.html @@ -545,7 +545,7 @@ or may be published to Pub/Sub or Kafka. Flink runner specific configuration --manageResources=false --monitorJobs=true \ ---flinkMaster=local --parallelism=#numcores +--flinkMaster=[local] --parallelism=#numcores @@ -911,7 +911,7 @@ configure logging. --streaming=false --manageResources=false --monitorJobs=true ---flinkMaster=local" +--flinkMaster=[local]" @@ -926,7 +926,7 @@ configure logging. --streaming=true --manageResources=false --monitorJobs=true ---flinkMaster=local" +--flinkMaster=[local]"
[beam] branch master updated (16b34cf -> a53f56a)
This is an automated email from the ASF dual-hosted git repository. echauchot pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 16b34cf Merge pull request #7260 Fix combiner lifting for external job service. add bc11c06 [BEAM-6216] Update flinkMaster URL in the nexmark wep-page to reflect change in FlinkExecutionEnvironments. add ffcd9f4 [BEAM-6216] Update flinkMaster URL in nexmark postCommit script to reflect change in FlinkExecutionEnvironments. new a53f56a Merge pull request #7261 from echauchot/BEAM-6216-flink-local The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Flink.groovy | 10 +- website/src/documentation/sdks/nexmark.md| 6 +++--- 2 files changed, 8 insertions(+), 8 deletions(-)
[beam] 01/01: Merge pull request #7261 from echauchot/BEAM-6216-flink-local
This is an automated email from the ASF dual-hosted git repository. echauchot pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit a53f56abc32456cd9d5d2bc2d0dbc7894a353fa0 Merge: 16b34cf ffcd9f4 Author: Etienne Chauchot AuthorDate: Wed Dec 12 17:10:06 2018 +0100 Merge pull request #7261 from echauchot/BEAM-6216-flink-local Merge pull request #7261: [BEAM-6216] Update flinkMaster URL to reflect change in FlinkExecutionEnvironments .test-infra/jenkins/job_PostCommit_Java_Nexmark_Flink.groovy | 10 +- website/src/documentation/sdks/nexmark.md| 6 +++--- 2 files changed, 8 insertions(+), 8 deletions(-)
[beam] 01/01: Merge pull request #7260 Fix combiner lifting for external job service.
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 16b34cfc55b37012cd810843a61cc58b05c0c662 Merge: 289d2b2 c00dfa5 Author: Robert Bradshaw AuthorDate: Wed Dec 12 16:47:20 2018 +0100 Merge pull request #7260 Fix combiner lifting for external job service. Fix combiner lifting for external job service. .../apache_beam/runners/portability/fn_api_runner_transforms.py | 5 + 1 file changed, 5 insertions(+)
[beam] branch master updated (289d2b2 -> 16b34cf)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 289d2b2 Merge pull request #7258: [BEAM-6213] Fix matching of glob patterns in windows local filesystem add c00dfa5 Fix combiner lifting for external job service. new 16b34cf Merge pull request #7260 Fix combiner lifting for external job service. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache_beam/runners/portability/fn_api_runner_transforms.py | 5 + 1 file changed, 5 insertions(+)
[beam] branch master updated (ec55000 -> 289d2b2)
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from ec55000 Merge pull request #7078 from [BEAM-6094] External portable BeamPython. add 9853bd7 [BEAM-6213] Fix matching of glob patterns in windows local filesystem new 289d2b2 Merge pull request #7258: [BEAM-6213] Fix matching of glob patterns in windows local filesystem The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../core/src/main/java/org/apache/beam/sdk/io/LocalFileSystem.java | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)
[beam] 01/01: Merge pull request #7258: [BEAM-6213] Fix matching of glob patterns in windows local filesystem
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 289d2b2d95748140479dfd8552fcfd10d740af9f Merge: ec55000 9853bd7 Author: Alexey Romanenko <33895511+aromanenko-...@users.noreply.github.com> AuthorDate: Wed Dec 12 15:30:32 2018 +0100 Merge pull request #7258: [BEAM-6213] Fix matching of glob patterns in windows local filesystem .../core/src/main/java/org/apache/beam/sdk/io/LocalFileSystem.java | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)
[beam] 01/01: Merge pull request #7078 from [BEAM-6094] External portable BeamPython.
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit ec55000f2686aae9856c53863cf70b85bc1ed38d Merge: 8c8b7c3 efd261a Author: Robert Bradshaw AuthorDate: Wed Dec 12 13:02:10 2018 +0100 Merge pull request #7078 from [BEAM-6094] External portable BeamPython. [BEAM-6094] Implement external environment for portable BeamPython. .../fn-execution/src/main/proto/beam_fn_api.proto | 20 ++ .../pipeline/src/main/proto/beam_runner_api.proto | 6 + .../FlinkDefaultExecutableStageContext.java| 3 + ...actory.java => ExternalEnvironmentFactory.java} | 116 +++ .../environment/ProcessEnvironmentFactory.java | 2 +- sdks/python/apache_beam/pipeline.py| 9 +- sdks/python/apache_beam/portability/python_urns.py | 16 + .../python/apache_beam/runners/pipeline_context.py | 2 +- .../runners/portability/flink_runner_test.py | 3 +- .../runners/portability/fn_api_runner.py | 376 ++--- .../runners/portability/fn_api_runner_test.py | 19 +- .../portability/fn_api_runner_transforms.py| 48 ++- .../runners/portability/local_job_service.py | 25 +- .../runners/portability/local_job_service_main.py | 4 +- .../runners/portability/portable_runner.py | 120 +-- .../runners/portability/portable_runner_test.py| 43 ++- .../apache_beam/runners/worker/sdk_worker.py | 8 +- .../runners/worker/worker_id_interceptor.py| 5 +- 18 files changed, 560 insertions(+), 265 deletions(-)
[beam] branch master updated (8c8b7c3 -> ec55000)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 8c8b7c3 Merge pull request #7254: [BEAM-2943] Fix typo in filename on
FlinkRunner page
add 73a4325 Use environment to control worker startup in FnApiRunner.
add efd261a [BEAM-6094] Implement external environment for Portable Beam.
new ec55000 Merge pull request #7078 from [BEAM-6094] External portable
BeamPython.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../fn-execution/src/main/proto/beam_fn_api.proto | 20 ++
.../pipeline/src/main/proto/beam_runner_api.proto | 6 +
.../FlinkDefaultExecutableStageContext.java| 3 +
...actory.java => ExternalEnvironmentFactory.java} | 116 +++
.../environment/ProcessEnvironmentFactory.java | 2 +-
sdks/python/apache_beam/pipeline.py| 9 +-
sdks/python/apache_beam/portability/python_urns.py | 16 +
.../python/apache_beam/runners/pipeline_context.py | 2 +-
.../runners/portability/flink_runner_test.py | 3 +-
.../runners/portability/fn_api_runner.py | 376 ++---
.../runners/portability/fn_api_runner_test.py | 19 +-
.../portability/fn_api_runner_transforms.py| 48 ++-
.../runners/portability/local_job_service.py | 25 +-
.../runners/portability/local_job_service_main.py | 4 +-
.../runners/portability/portable_runner.py | 120 +--
.../runners/portability/portable_runner_test.py| 43 ++-
.../apache_beam/runners/worker/sdk_worker.py | 8 +-
.../runners/worker/worker_id_interceptor.py| 5 +-
18 files changed, 560 insertions(+), 265 deletions(-)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/{ProcessEnvironmentFactory.java
=> ExternalEnvironmentFactory.java} (59%)
