[GitHub] beam pull request #1799: [BEAM-1285] Apache rat plugin will print out files ...

2017-01-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/1799 [BEAM-1285] Apache rat plugin will print out files with unapproved licenses to console Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #1801: Beam Fn API Prototype Implementation

2017-01-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/1801 Beam Fn API Prototype Implementation An overview is available at https://s.apache.org/beam-fn-api Code package descriptions: org.apache.beam.fn.harness.FnHarness: main entry point

[GitHub] beam pull request #1766: [BEAM-1266] Remove value only outputs in Dataflow j...

2017-01-11 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/1766 [BEAM-1266] Remove value only outputs in Dataflow job representation Always define outputs in terms of full windowed value representations Be sure to do all of the following to help us

[GitHub] beam pull request #1766: [BEAM-1266] Remove value only outputs in Dataflow j...

2017-01-12 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/1766 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #1772: Use varargs for PipelineOptionsFactory.fromArgs

2017-01-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/1772 Use varargs for PipelineOptionsFactory.fromArgs This simplifies usage of PipelineOptionsFactory within tests. Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #1709: [BEAM-1226] Add support for well known coder types ...

2016-12-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/1709 [BEAM-1226] Add support for well known coder types to Apache Beam Python SDK This uses specific cloud object representations for the following types: kind:pair (TupleCoder with two components

[GitHub] beam pull request #2511: [BEAM-115] Update timer/state fields on ParDoPayloa...

2017-04-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2511 [BEAM-115] Update timer/state fields on ParDoPayload to use a map field for consistent name usage Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2518: [BEAM-1347] Update protos related to State API for ...

2017-04-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2518 [BEAM-1347] Update protos related to State API for prototyping purposes Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #2616: [BEAM-1871] Move over GcsPath to gcp-core

2017-04-20 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2616 [BEAM-1871] Move over GcsPath to gcp-core Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like

[GitHub] beam pull request #2592: [BEAM-2015] Remove shared profile in runners/pom.xm...

2017-04-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2592 [BEAM-2015] Remove shared profile in runners/pom.xml and fix Dataflow ValidatesRunner PostCommit Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2598: [BEAM-1871] Remove another depedendency by moving T...

2017-04-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2598 [BEAM-1871] Remove another depedendency by moving TestCredential Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR

[GitHub] beam pull request #2605: [BEAM-1786, BEAM-1871] Add the ability to register ...

2017-04-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2605 [BEAM-1786, BEAM-1871] Add the ability to register coder factories Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #2637: [BEAM-1786] Post Dataflow worker CoderRegistry clea...

2017-04-21 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2637 [BEAM-1786] Post Dataflow worker CoderRegistry clean-up Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2638: [BEAM-1871] Remove unnecessary runtime dependencies...

2017-04-21 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2638 [BEAM-1871] Remove unnecessary runtime dependencies for Google Cloud Storage Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make

[GitHub] beam pull request #2670: [BEAM-1871] Move Bigquery/Pubsub options to sdks/ja...

2017-04-24 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2670 [BEAM-1871] Move Bigquery/Pubsub options to sdks/java/io/google-cloud-platform Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3638: [BEAM-1347] Add utility to be able to model inbound...

2017-07-25 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3638 [BEAM-1347] Add utility to be able to model inbound reading as a single input stream Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] beam pull request #3621: [BEAM-2662] Fix Java quickstart for Spark to includ...

2017-07-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3621 [BEAM-2662] Fix Java quickstart for Spark to include Spark version Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA

[GitHub] beam pull request #3622: [BEAM-2642] Update Google Auth to 0.7.1

2017-07-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3622 [BEAM-2642] Update Google Auth to 0.7.1 Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies. Follow this checklist to help us incorporate your contribution quickly

[GitHub] beam pull request #3620: [BEAM-2662] Fix Java quickstart for Spark to includ...

2017-07-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3620 [BEAM-2662] Fix Java quickstart for Spark to include Spark version Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make sure there is a [JIRA

[GitHub] beam pull request #3596: [BEAM-2642] Update Google Auth to 0.7.1

2017-07-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3596 [BEAM-2642] Update Google Auth to 0.7.1 Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies. Follow this checklist to help us incorporate your contribution quickly

[GitHub] beam pull request #3482: [BEAM-1348] Remove deprecated concepts in Fn API (n...

2017-06-30 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3482 [BEAM-1348] Remove deprecated concepts in Fn API (now replaced with Runner API concepts). Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3494: [BEAM-2553] Update Maven exec plugin to 1.6.0 to in...

2017-07-05 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3494 [BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging improvements Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3495: [BEAM-2552] Update Dataflow Java container version ...

2017-07-05 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3495 [BEAM-2552] Update Dataflow Java container version for 2.1.0 release Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #3495: [BEAM-2552] Update Dataflow Java container version ...

2017-07-05 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3495 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #2708: [BEAM-1871] Move RetryHttpRequestInitializer and re...

2017-04-26 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2708 [BEAM-1871] Move RetryHttpRequestInitializer and remove deprecated methods Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make

[GitHub] beam pull request #2706: [BEAM-1871] Move functionality from AvroUtils to Av...

2017-04-26 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2706 [BEAM-1871] Move functionality from AvroUtils to AvroSource hiding visibility of internal method, also remove deprecated method. Be sure to do all of the following to help us incorporate your

[GitHub] beam pull request #2710: [BEAM-1871] Remove deprecated methods from AvroCode...

2017-04-26 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2710 [BEAM-1871] Remove deprecated methods from AvroCoder Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2713: [BEAM-1871] Clean-up org.apache.beam.sdk.util, move...

2017-04-26 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2713 [BEAM-1871] Clean-up org.apache.beam.sdk.util, move BitSetCoder from util to coder Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x

[GitHub] beam pull request #2722: [BEAM-1871] Move several options/auth classes aroun...

2017-04-26 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2722 [BEAM-1871] Move several options/auth classes around in gcp-core Note that I kept a duplicate of GcsOptions because of its direct usage within the Dataflow Worker Be sure to do all

[GitHub] beam pull request #2640: [BEAM-1871] Move Xml IO and related classes to new ...

2017-04-21 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2640 [BEAM-1871] Move Xml IO and related classes to new sdks/java/io/xml package. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2678: [BEAM-1871] Move Bigquery/Pubsub options to sdks/ja...

2017-04-25 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2678 [BEAM-1871] Move Bigquery/Pubsub options to sdks/java/io/google-cloud-platform Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2682: [BEAM-1871] Move ByteStringCoder to sdks/java/exten...

2017-04-25 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2682 [BEAM-1871] Move ByteStringCoder to sdks/java/extensions/protobuf Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] beam pull request #2762: [BEAM-2031] Add support to pass through zero or mor...

2017-04-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2762 [BEAM-2031] Add support to pass through zero or more Hadoop configurations through as PipelineOptions Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] beam pull request #2765: [BEAM-2119] Fix FileSystems to have the local files...

2017-04-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2765 [BEAM-2119] Fix FileSystems to have the local filesystem installed by default. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x

[GitHub] beam pull request #2776: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...

2017-04-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2776 [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HadoopFileSystem. TODO: * Add multiplexing FileSystem that is able to route requests based upon

[GitHub] beam pull request #2776: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...

2017-04-28 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2776 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #2777: [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2...

2017-04-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2777 [BEAM-2005, BEAM-2030, BEAM-2031, BEAM-2032, BEAM-2033, BEAM-2070] Base implementation of HadoopFileSystem. TODO: * Add multiplexing FileSystem that is able to route requests based upon

[GitHub] beam pull request #2772: [BEAM-2005] Swap to use Lists within MatchResult in...

2017-04-28 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2772 [BEAM-2005] Swap to use Lists within MatchResult instead of arrays. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] beam pull request #2664: [BEAM-1871] Move ProtoCoder to new sdks/java/extens...

2017-04-24 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2664 [BEAM-1871] Move ProtoCoder to new sdks/java/extensions/protobuf package. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make

[GitHub] beam pull request #2670: [BEAM-1871] Move Bigquery/Pubsub options to sdks/ja...

2017-04-24 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2670 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #2747: [BEAM-1871] Hide internal implementation details of...

2017-04-27 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2747 [BEAM-1871] Hide internal implementation details of how we create a DefaultBucket for GCP Temp Location Moved relevant contents of GcpProjectUtil and DefaultProject into GcpOptions.GcpTempLocation

[GitHub] beam pull request #2748: [BEAM-1871] Remove irrelevant runtime dependencies

2017-04-27 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2748 [BEAM-1871] Remove irrelevant runtime dependencies Confirmed that sdks/java/io/gcp uses 1.22 for the removed libraries. Be sure to do all of the following to help us incorporate your

[GitHub] beam pull request #2752: [BEAM-2031] Add support to PipelineOptionsFactory/P...

2017-04-27 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2752 [BEAM-2031] Add support to PipelineOptionsFactory/ProxyInvocationHandler for auto registration of Jackson modules This is towards adding a mixin module for Hadoop Configuration so that it can live

[GitHub] beam pull request #2757: [BEAM-2005] Move getScheme from FileSystemRegistrar...

2017-04-27 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2757 [BEAM-2005] Move getScheme from FileSystemRegistrar to FileSystem Note that I needed to update FileSystems to instantiate the FileSystem(s) upfront instead of remembering the mapping from scheme

[GitHub] beam pull request #2722: [BEAM-1871] Move several options/auth classes aroun...

2017-04-27 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2722 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3723: [BEAM-1347] Plumb through a yet to be created state...

2017-08-16 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3723 [BEAM-1347] Plumb through a yet to be created state client through PTransformRunnerFactory Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make

[GitHub] beam pull request #3724: [BEAM-1347] Convert an InputStream into an Iterable...

2017-08-16 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3724 [BEAM-1347] Convert an InputStream into an Iterable using the Beam Fn data specification Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make

[GitHub] beam pull request #3736: [BEAM-1347] Provide an abstraction which creates an...

2017-08-18 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3736 [BEAM-1347] Provide an abstraction which creates an Iterator view over the Beam Fn State API Combining this with the DataStreams.DataStreamDecoder converts the Beam Fn State API

[GitHub] beam pull request #2862: [BEAM-1871] Move GCP specific serialization CloudOb...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2862 [BEAM-1871] Move GCP specific serialization CloudObject and supporting translation code to Dataflow runner module Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #2877: [BEAM-2135] Update Spark to support serializing/des...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2877 [BEAM-2135] Update Spark to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #2881: [BEAM-2135] Update Dataflow to support serializing/...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2881 [BEAM-2135] Update Dataflow to support serializing/deserializing custom user types configured via Jackson modules This also updates the runner harness and existing tests to use a properly

[GitHub] beam pull request #2880: [BEAM-2165] Update Apex to support serializing/dese...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2880 [BEAM-2165] Update Apex to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] beam pull request #2879: [BEAM-2165] Update Flink to support serializing/des...

2017-05-03 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2879 [BEAM-2165] Update Flink to support serializing/deserializing custom user types configured via Jackson modules Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #3100: Cherry pick #3093: [BEAM-2256] Add the last previou...

2017-05-11 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3134: [BEAM-1345] Remove FileSystems.setDefaultConfigInWo...

2017-05-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3134 [BEAM-1345] Remove FileSystems.setDefaultConfigInWorkers since Dataflow no longer depends on this Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3066: Cherry-pick #3062: Skip generating empty jars for p...

2017-05-10 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3066 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3066: Cherry-pick #3063: Skip generating empty jars for p...

2017-05-10 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3066 Cherry-pick #3063: Skip generating empty jars for parent poms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] beam pull request #3061: Cherry-pick #3034: [BEAM-2242] Ensure that jars are...

2017-05-10 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3061 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3034: [BEAM-2242] Ensure that the test jar is shaded corr...

2017-05-09 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3034 [BEAM-2242] Ensure that the test jar is shaded correctly This required listing the maven jar plugin earlier within the file so that the jar/test jar ran before shade within the package phase

[GitHub] beam pull request #3143: [BEAM-2299] Run maven install on Windows machine fo...

2017-05-15 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3143 [BEAM-2299] Run maven install on Windows machine for build/test coverage on Windows. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3152: [BEAM-2252] Use pluginManagement instead of redefin...

2017-05-15 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3152 [BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin to inherit build order from root pom.xml Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #3120: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...

2017-05-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3120 Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bounded iterator's hasNext idempotent Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3119: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...

2017-05-12 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3119 Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bounded iterator's hasNext idempotent Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3119: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...

2017-05-12 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3119 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3159: [BEAM-2252] Use resource transformer plugin to use ...

2017-05-15 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3159 [BEAM-2252] Use resource transformer plugin to use versions defined in root pom.xml within maven archetype poms Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #3100: Cherry pick #3093: [BEAM-2256] Add the last previou...

2017-05-11 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3100 Cherry pick #3093: [BEAM-2256] Add the last previous range filter Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR

[GitHub] beam pull request #3120: Cherry pick #2854: [BEAM-2095] Made SourceRDD.Bound...

2017-05-12 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3061: Cherry-pick #3034: [BEAM-2242] Ensure that jars are...

2017-05-10 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3061 Cherry-pick #3034: [BEAM-2242] Ensure that jars are shaded correctly by running the jar plugin before the shade plugin Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam-site pull request #257: [BEAM-2500] List Amazon S3 File System as a pla...

2017-06-22 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam-site/pull/257 [BEAM-2500] List Amazon S3 File System as a planned I/O. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lukecwik/incubator-beam-site asf

[GitHub] beam pull request #3430: [BEAM-1187] Improve logging to contain the number o...

2017-06-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3430 [BEAM-1187] Improve logging to contain the number of retries done due to IOException and unsuccessful response codes. Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #3432: [BEAM-1347] Create a DoFnRunner specific for the Fn...

2017-06-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3432 [BEAM-1347] Create a DoFnRunner specific for the Fn API Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title

[GitHub] beam pull request #3397: [BEAM-1348] Fix type error introduced into Python S...

2017-06-19 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3397 [BEAM-1348] Fix type error introduced into Python SDK because of PR/3268 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #3212: [BEAM-2354] Enable a native pubsub source within th...

2017-05-23 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3212 [BEAM-2354] Enable a native pubsub source within the Dataflow Runner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] beam pull request #3212: [BEAM-2354] Enable a native pubsub source within th...

2017-05-23 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/3212 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #3223: [BEAM-2354] Add a ReadStringsFromPubSub/WriteString...

2017-05-24 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3223 [BEAM-2354] Add a ReadStringsFromPubSub/WriteStringsToPubSub PTransform Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #3260: [BEAM-1347] Remove the usage of a thread local on a...

2017-05-30 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3260 [BEAM-1347] Remove the usage of a thread local on a potentially hot path Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #3268: [BEAM-1348] Model the Fn State Api as per https://s...

2017-05-31 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3268 [BEAM-1348] Model the Fn State Api as per https://s.apache.org/beam-fn-state-api-and-bundle-processing Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] beam pull request #3296: [BEAM-1347] Add additional logging

2017-06-05 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3296 [BEAM-1347] Add additional logging Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like

[GitHub] beam pull request #3297: [BEAM-1347] Migrate to Runner API constructs within...

2017-06-05 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3297 [BEAM-1347] Migrate to Runner API constructs within the Java SDK harness Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure

[GitHub] beam pull request #3312: Swap to use an Impulse primitive + DoFn for Create ...

2017-06-07 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3312 Swap to use an Impulse primitive + DoFn for Create when executing with the Fn API. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x

[GitHub] beam pull request #3366: [BEAM-1348] Mark Runner API like types declared wit...

2017-06-14 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3366 [BEAM-1348] Mark Runner API like types declared within Fn API as deprecated Add additional documentation. Be sure to do all of the following to help us incorporate your contribution

[GitHub] beam pull request #3375: [BEAM-1347] Break apart ProcessBundleHandler to use...

2017-06-15 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3375 [BEAM-1347] Break apart ProcessBundleHandler to use service locator pattern based upon URNs. This cleans up ProcessBundleHandler and allows for separate improvements of the various PTransform

[GitHub] beam pull request #2794: [BEAM-2130] Ensure the options id is never null.

2017-04-30 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2794 [BEAM-2130] Ensure the options id is never null. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2803: [BEAM-1871] Hide CloudResourceManager client in sdk...

2017-05-01 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2803 [BEAM-1871] Hide CloudResourceManager client in sdks/java/extensions/gcp-core Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2844: [BEAM-2135] Fix pointers to sdks/java/io/hadoop-fil...

2017-05-02 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2844 [BEAM-2135] Fix pointers to sdks/java/io/hadoop-file-system Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2929: [BEAM-2187] Fix build break in SparkRuntimeContextT...

2017-05-05 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2929 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #2929: [BEAM-2187] Fix build break in SparkRuntimeContextT...

2017-05-05 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2929 [BEAM-2187] Fix build break in SparkRuntimeContextTest Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2941: PR to run tests for #2930 on Jenkins

2017-05-06 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2941 PR to run tests for #2930 on Jenkins Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] beam pull request #2941: PR to run tests for #2930 on Jenkins

2017-05-06 Thread lukecwik
Github user lukecwik closed the pull request at: https://github.com/apache/beam/pull/2941 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #2905: [BEAM-1871] Move out test only utilities used only ...

2017-05-04 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2905 [BEAM-1871] Move out test only utilities used only in examples/java out of sdks/java/core. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2910: [BEAM-2174] Update CoderRegistry to allow creating ...

2017-05-04 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2910 [BEAM-2174] Update CoderRegistry to allow creating coders through CoderFactory for a wider range of types Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] beam pull request #2827: [BEAM-2135] Move gcp-core to google-cloud-platform-...

2017-05-02 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2827 [BEAM-2135] Move gcp-core to google-cloud-platform-core Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2825: Remove Coder.asCloudObject and related methods

2017-05-02 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2825 Remove Coder.asCloudObject and related methods Rebase #2806 on top of apache/master If this passes, will merge in #2806 You can merge this pull request into a Git repository by running

[GitHub] beam pull request #2804: [BEAM-1871] Remove deprecated org.apache.beam.sdk.o...

2017-05-01 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2804 [BEAM-1871] Remove deprecated org.apache.beam.sdk.options.GcsOptions after updating Dataflow worker Worker has been migrated to org.apache.beam.sdk.extensions.gcp.options.GcsOptions

[GitHub] beam pull request #2980: PR to run tests for #2933 using updated Dataflow wo...

2017-05-08 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/2980 PR to run tests for #2933 using updated Dataflow worker image Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] beam pull request #3271: [BEAM-1348] Add Runner API constructs to ProcessBun...

2017-05-31 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3271 [BEAM-1348] Add Runner API constructs to ProcessBundleDescriptor. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR

[GitHub] beam pull request #3856: [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED...

2017-09-15 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3856 [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first declared field. Follow this checklist to help us incorporate your contribution quickly and easily: - [x]

[GitHub] beam pull request #3849: [BEAM-2956] Attempt to correctly report the Dataflo...

2017-09-13 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3849 [BEAM-2956] Attempt to correctly report the Dataflow distribution in GCP related modules. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Make

[GitHub] beam pull request #3944: [BEAM-3016] Fix blocking issue within run() when ch...

2017-10-04 Thread lukecwik
GitHub user lukecwik opened a pull request: https://github.com/apache/beam/pull/3944 [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. Follow this checklist to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #3944: [BEAM-3016] Fix blocking issue within run() when ch...

2017-10-06 Thread lukecwik
GitHub user lukecwik reopened a pull request: https://github.com/apache/beam/pull/3944 [BEAM-3016] Fix blocking issue within run() when channel terminates while blocking within DirectStreamObserver. Follow this checklist to help us incorporate your contribution quickly and easily

  1   2   >