[jira] [Commented] (BEAM-469) NullableCoder optimized encoding via passthrough context

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768525#comment-15768525 ] ASF GitHub Bot commented on BEAM-469: - GitHub user dhalperi opened a pull request:

[GitHub] incubator-beam pull request #1680: [BEAM-XXX] Make KVCoder more efficient by...

2016-12-21 Thread dhalperi
GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/1680 [BEAM-XXX] Make KVCoder more efficient by removing unnecessary nesting See [BEAM-469] for more information about why this is correct. You can merge this pull request into a Git

[jira] [Commented] (BEAM-1201) Remove producesSortedKeys from Source

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768483#comment-15768483 ] ASF GitHub Bot commented on BEAM-1201: -- GitHub user dhalperi opened a pull request:

[jira] [Updated] (BEAM-1201) Remove producesSortedKeys from Source

2016-12-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-1201: -- Labels: backward-incompatible (was: ) > Remove producesSortedKeys from Source >

[jira] [Updated] (BEAM-1201) Remove producesSortedKeys from BoundedSource

2016-12-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-1201: -- Summary: Remove producesSortedKeys from BoundedSource (was: Remove producesSortedKeys from

[GitHub] incubator-beam pull request #1679: [BEAM-1201] Remove BoundedSource.produces...

2016-12-21 Thread dhalperi
GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/1679 [BEAM-1201] Remove BoundedSource.producesSortedKeys R: @jkff You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/incubator-beam

[jira] [Commented] (BEAM-469) NullableCoder optimized encoding via passthrough context

2016-12-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768476#comment-15768476 ] Daniel Halperin commented on BEAM-469: -- Sorry I missed this JIRA comment, [~mariusz89016]! A bit late,

[jira] [Commented] (BEAM-646) Get runners out of the apply()

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768470#comment-15768470 ] ASF GitHub Bot commented on BEAM-646: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request #1582: [BEAM-646] Add Parameters to finishSpecif...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1582 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/3] incubator-beam git commit: Add Parameters to finishSpecifying

2016-12-21 Thread tgroh
Repository: incubator-beam Updated Branches: refs/heads/master 57d9bbd79 -> 7984fe3fc Add Parameters to finishSpecifying Remove the need to use getProducingTransformInternal in TypedPValue. Ensure that all nodes are finished specifying before a call to TransformHierarchy#visit. This ensures

[2/3] incubator-beam git commit: Use CountingSource in ForceStreamingTest

2016-12-21 Thread tgroh
Use CountingSource in ForceStreamingTest Removes the requirement to have a FakeUnboundedSource, plus the read is fully specified. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b053be46 Tree:

[3/3] incubator-beam git commit: This closes #1582

2016-12-21 Thread tgroh
This closes #1582 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7984fe3f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7984fe3f Diff:

[jira] [Commented] (BEAM-1112) Python E2E Integration Test Framework - Batch Only

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768461#comment-15768461 ] ASF GitHub Bot commented on BEAM-1112: -- Github user markflyhigh closed the pull request at:

[jira] [Commented] (BEAM-1112) Python E2E Integration Test Framework - Batch Only

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768462#comment-15768462 ] ASF GitHub Bot commented on BEAM-1112: -- GitHub user markflyhigh reopened a pull request:

[GitHub] incubator-beam pull request #1639: [BEAM-1112] Python E2E Test Framework And...

2016-12-21 Thread markflyhigh
GitHub user markflyhigh reopened a pull request: https://github.com/apache/incubator-beam/pull/1639 [BEAM-1112] Python E2E Test Framework And Wordcount E2E Test Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make

[GitHub] incubator-beam pull request #1639: [BEAM-1112] Python E2E Test Framework And...

2016-12-21 Thread markflyhigh
Github user markflyhigh closed the pull request at: https://github.com/apache/incubator-beam/pull/1639 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/2] incubator-beam git commit: Add support for date partitioned table names

2016-12-21 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk bb09c07b6 -> 409d067b3 Add support for date partitioned table names These names have the format "tablename$mmdd". Previously the dollar sign caused this to be deemed invalid. Project:

[2/2] incubator-beam git commit: Closes #1534

2016-12-21 Thread robertwb
Closes #1534 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/409d067b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/409d067b Diff:

[1/2] incubator-beam git commit: Fixing inconsistencies in PipelineOptions

2016-12-21 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 3454d691f -> bb09c07b6 Fixing inconsistencies in PipelineOptions The following options have changed: * job_name - Default is 'beamapp-username-date-microseconds'. Test was added. * staging_location and temp_location -

[2/2] incubator-beam git commit: Closes #1526

2016-12-21 Thread robertwb
Closes #1526 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bb09c07b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bb09c07b Diff:

[jira] [Created] (BEAM-1202) Coders should have meaningful equals methods

2016-12-21 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1202: - Summary: Coders should have meaningful equals methods Key: BEAM-1202 URL: https://issues.apache.org/jira/browse/BEAM-1202 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-1202) Coders should have meaningful equals methods

2016-12-21 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-1202: -- Assignee: (was: Davor Bonaci) > Coders should have meaningful equals methods >

[jira] [Created] (BEAM-1201) Remove producesSortedKeys from Source

2016-12-21 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-1201: - Summary: Remove producesSortedKeys from Source Key: BEAM-1201 URL: https://issues.apache.org/jira/browse/BEAM-1201 Project: Beam Issue Type: Improvement

[46/51] [abbrv] incubator-beam git commit: Allow setting timer by ID in DirectTimerInternals

2016-12-21 Thread kenn
Allow setting timer by ID in DirectTimerInternals Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7f14c463 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7f14c463 Diff:

[49/51] [abbrv] incubator-beam git commit: This closes #1669: Preliminaries for timers in the direct runner

2016-12-21 Thread kenn
This closes #1669: Preliminaries for timers in the direct runner Hold output watermark according to pending timers Allow setting timer by ID in DirectTimerInternals Add UsesTestStream for use with JUnit @Category Add static Window.withOutputTimeFn to match build method Use informative

[50/51] [abbrv] incubator-beam git commit: Merge remote-tracking branch 'origin/master' into python-sdk

2016-12-21 Thread kenn
Merge remote-tracking branch 'origin/master' into python-sdk Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/77742979 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/77742979 Diff:

[41/51] [abbrv] incubator-beam git commit: Fix windowing in direct runner Stateful ParDo

2016-12-21 Thread kenn
Fix windowing in direct runner Stateful ParDo Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4fb16e8f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4fb16e8f Diff:

[44/51] [abbrv] incubator-beam git commit: Add static Window.withOutputTimeFn to match build method

2016-12-21 Thread kenn
Add static Window.withOutputTimeFn to match build method Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8188040d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8188040d Diff:

[45/51] [abbrv] incubator-beam git commit: Add UsesTestStream for use with JUnit @Category

2016-12-21 Thread kenn
Add UsesTestStream for use with JUnit @Category Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4d71924c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4d71924c Diff:

[36/51] [abbrv] incubator-beam git commit: This closes #1673: Require TimeDomain to delete a timer

2016-12-21 Thread kenn
This closes #1673: Require TimeDomain to delete a timer Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4843dc59 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4843dc59 Diff:

[51/51] [abbrv] incubator-beam git commit: This closes #1676: Merge master (57d9bbd) into python-sdk

2016-12-21 Thread kenn
ExecutionContext and related classes to runners-core Update Dataflow worker to beam-master-20161221 Require TimeDomain to delete a timer Provide local tags in PInput, POutput expansions Remove deprecated InMemoryTimerInternals from SDK Remove deprecated AggregatorFactory from SDK Update Dataflow

[38/51] [abbrv] incubator-beam git commit: Support set and delete of timer by ID in InMemoryTimerInternals

2016-12-21 Thread kenn
Support set and delete of timer by ID in InMemoryTimerInternals Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/df2e540d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/df2e540d Diff:

[35/51] [abbrv] incubator-beam git commit: Update Dataflow worker to beam-master-20161221

2016-12-21 Thread kenn
Update Dataflow worker to beam-master-20161221 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/64336e40 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/64336e40 Diff: http://git-wip

[47/51] [abbrv] incubator-beam git commit: Hold output watermark according to pending timers

2016-12-21 Thread kenn
Hold output watermark according to pending timers Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dfe2e62d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dfe2e62d Diff:

[43/51] [abbrv] incubator-beam git commit: Add informative Instant formatter to BoundedWindow

2016-12-21 Thread kenn
Add informative Instant formatter to BoundedWindow Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/280a6a8f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/280a6a8f Diff:

[28/51] [abbrv] incubator-beam git commit: This closes #1665: Remove deprecated AggregatorFactory from SDK

2016-12-21 Thread kenn
This closes #1665: Remove deprecated AggregatorFactory from SDK Update Dataflow worker to beam-master-20161220 Remove deprecated AggregatorFactory from SDK Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #1676: Merge remote-tracking branch 'origin/mast...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1676 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[37/51] [abbrv] incubator-beam git commit: This closes #1666: Move ExecutionContext and related classes to runners-core

2016-12-21 Thread kenn
This closes #1666: Move ExecutionContext and related classes to runners-core Move ExecutionContext and related classes to runners-core Update Dataflow worker to beam-master-20161221 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org

[48/51] [abbrv] incubator-beam git commit: Use informative Instant formatter in WatermarkHold

2016-12-21 Thread kenn
Use informative Instant formatter in WatermarkHold Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fa4958a6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/fa4958a6 Diff:

[42/51] [abbrv] incubator-beam git commit: This closes #1670: Fixes for direct runner expansion and evaluation of stateful ParDo

2016-12-21 Thread kenn
This closes #1670: Fixes for direct runner expansion and evaluation of stateful ParDo Actually propagate and commit state in direct runner Fix windowing in direct runner Stateful ParDo Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[40/51] [abbrv] incubator-beam git commit: Actually propagate and commit state in direct runner

2016-12-21 Thread kenn
Actually propagate and commit state in direct runner Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/55176c38 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/55176c38 Diff:

[10/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-extensions-join-library module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-extensions-join-library module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/24ad1831 Tree:

[09/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-extensions-sorter module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-extensions-sorter module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/63331aa8 Tree:

[02/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-core module to TestPipeline as a JUnit rule. Plus, fixed some checkstyle errors from previous modules' migration.

2016-12-21 Thread kenn
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/75a4c918/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/CreateTest.java -- diff --git

[08/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-google-cloud-platform module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-google-cloud-platform module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6dea0992 Tree:

[34/51] [abbrv] incubator-beam git commit: Move ExecutionContext and related classes to runners-core

2016-12-21 Thread kenn
Move ExecutionContext and related classes to runners-core Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9d2b8e09 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9d2b8e09 Diff:

[27/51] [abbrv] incubator-beam git commit: Update Dataflow worker to beam-master-20161220

2016-12-21 Thread kenn
Update Dataflow worker to beam-master-20161220 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a3f68d34 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a3f68d34 Diff:

[14/51] [abbrv] incubator-beam git commit: This closes #1664: [BEAM-1176] Migrating tests to use TestPipeline as a JUnit rule

2016-12-21 Thread kenn
This closes #1664: [BEAM-1176] Migrating tests to use TestPipeline as a JUnit rule Migrated the beam-sdks-java-io-java8tests module to TestPipeline as a JUnit rule + fixed WithTimestampsJava8Test.withTimestampsLambdaShouldApplyTimestamps. Migrated the beam-sdks-java-io-mongodb module to

[39/51] [abbrv] incubator-beam git commit: This closes #1160: Support set and delete of timer by ID in InMemoryTimerInternals

2016-12-21 Thread kenn
This closes #1160: Support set and delete of timer by ID in InMemoryTimerInternals Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7ee8c86d Tree:

[23/51] [abbrv] incubator-beam git commit: Fixup usage of canonical name with name since canonical name != name for inner classes.

2016-12-21 Thread kenn
Fixup usage of canonical name with name since canonical name != name for inner classes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/96d39314 Tree:

[05/51] [abbrv] incubator-beam git commit: Migrated the beam-runners-core module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-runners-core module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b6710251 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b6710251 Diff:

[01/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-core module to TestPipeline as a JUnit rule. Plus, fixed some checkstyle errors from previous modules' migration.

2016-12-21 Thread kenn
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 3b4fd5c7d -> 3454d691f http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/75a4c918/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ViewTest.java

[30/51] [abbrv] incubator-beam git commit: Provide local tags in PInput, POutput expansions

2016-12-21 Thread kenn
Provide local tags in PInput, POutput expansions Output an ordered colleciton in PInput and POutput expansions. This provides information that is necessary to reconstruct a PInput or POutput from its expansion. Implement PCollectionList.equals, PCollectionTuple.equals Project:

[31/51] [abbrv] incubator-beam git commit: This closes #1569

2016-12-21 Thread kenn
This closes #1569 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/aadcf3a1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/aadcf3a1 Diff:

[33/51] [abbrv] incubator-beam git commit: Require TimeDomain to delete a timer

2016-12-21 Thread kenn
Require TimeDomain to delete a timer Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/35a02740 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/35a02740 Diff:

[13/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-kinesis module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-kinesis module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/950aa7e1 Tree:

[24/51] [abbrv] incubator-beam git commit: Provide a better error message for non-existing gcpTempLocation

2016-12-21 Thread kenn
Provide a better error message for non-existing gcpTempLocation gcpTempLocation will default to using the value for tmpLocation, as long as the value is a valid GCP path. Non-valid GCP paths are silently discarded. This change removes existence validation from the default value logic such that

[07/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-jdbc module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-jdbc module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ccbe679 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ccbe679

[03/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-core module to TestPipeline as a JUnit rule. Plus, fixed some checkstyle errors from previous modules' migration.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-core module to TestPipeline as a JUnit rule. Plus, fixed some checkstyle errors from previous modules' migration. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/75a4c918 Tree:

[22/51] [abbrv] incubator-beam git commit: This closes #1581: [BEAM-1117] Port direct runner StatefulParDo to KeyedWorkItem

2016-12-21 Thread kenn
This closes #1581: [BEAM-1117] Port direct runner StatefulParDo to KeyedWorkItem Port direct runner StatefulParDo to KeyedWorkItem Propagate key through ParDo if DoFn is key-preserving Move responsibility for knowing about keyedness into EvaluationContext Add some key-preserving to

[06/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-jms module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-jms module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8d478c0f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8d478c0f

[11/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-java8tests module to TestPipeline as a JUnit rule + fixed WithTimestampsJava8Test.withTimestampsLambdaShouldApplyTimestamps.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-java8tests module to TestPipeline as a JUnit rule + fixed WithTimestampsJava8Test.withTimestampsLambdaShouldApplyTimestamps. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[18/51] [abbrv] incubator-beam git commit: Port direct runner StatefulParDo to KeyedWorkItem

2016-12-21 Thread kenn
Port direct runner StatefulParDo to KeyedWorkItem Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1f018ab6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1f018ab6 Diff:

[17/51] [abbrv] incubator-beam git commit: Move responsibility for knowing about keyedness into EvaluationContext

2016-12-21 Thread kenn
Move responsibility for knowing about keyedness into EvaluationContext This will allow transform evaluators to inquire about whether various collections are keyed. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[26/51] [abbrv] incubator-beam git commit: Remove deprecated AggregatorFactory from SDK

2016-12-21 Thread kenn
Remove deprecated AggregatorFactory from SDK Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/aab46a0e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/aab46a0e Diff:

[32/51] [abbrv] incubator-beam git commit: This closes #1668: Remove deprecated InMemoryTimerInternals from SDK

2016-12-21 Thread kenn
This closes #1668: Remove deprecated InMemoryTimerInternals from SDK Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0d0a5e28 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0d0a5e28 Diff:

[15/51] [abbrv] incubator-beam git commit: Add some key-preserving to KeyedPValueTrackingVisitor

2016-12-21 Thread kenn
Add some key-preserving to KeyedPValueTrackingVisitor Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/81702e67 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/81702e67 Diff:

[20/51] [abbrv] incubator-beam git commit: Move InMemoryTimerInternals to runners-core

2016-12-21 Thread kenn
Move InMemoryTimerInternals to runners-core Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/445c1205 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/445c1205 Diff:

[25/51] [abbrv] incubator-beam git commit: [BEAM-1097] Provide a better error message for non-existing gcpTempLocation

2016-12-21 Thread kenn
[BEAM-1097] Provide a better error message for non-existing gcpTempLocation This closes #1522 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/acd2196c Tree:

[19/51] [abbrv] incubator-beam git commit: Restore SDK's InMemoryTimerInternals, deprecated

2016-12-21 Thread kenn
Restore SDK's InMemoryTimerInternals, deprecated Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/69d2c47b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/69d2c47b Diff:

[21/51] [abbrv] incubator-beam git commit: This closes #1652: Move InMemoryTimerInternals to runners-core

2016-12-21 Thread kenn
This closes #1652: Move InMemoryTimerInternals to runners-core * github/pr/1652: Restore SDK's InMemoryTimerInternals, deprecated Move InMemoryTimerInternals to runners-core Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[12/51] [abbrv] incubator-beam git commit: Migrated the beam-sdks-java-io-mongodb module to TestPipeline as a JUnit rule.

2016-12-21 Thread kenn
Migrated the beam-sdks-java-io-mongodb module to TestPipeline as a JUnit rule. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fce4f658 Tree:

[jira] [Commented] (BEAM-1198) ViewFn: explicitly decouple runner materialization of side inputs from SDK-specific mapping

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768387#comment-15768387 ] ASF GitHub Bot commented on BEAM-1198: -- GitHub user kennknowles opened a pull request:

[GitHub] incubator-beam pull request #1678: [BEAM-1198, BEAM-846, BEAM-260] Refactor ...

2016-12-21 Thread kennknowles
GitHub user kennknowles opened a pull request: https://github.com/apache/incubator-beam/pull/1678 [BEAM-1198, BEAM-846, BEAM-260] Refactor Dataflow translator to decouple input and output graphs more easily Be sure to do all of the following to help us incorporate your contribution

[jira] [Assigned] (BEAM-1198) ViewFn: explicitly decouple runner materialization of side inputs from SDK-specific mapping

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-1198: - Assignee: Kenneth Knowles > ViewFn: explicitly decouple runner materialization of side

[GitHub] incubator-beam pull request #1677: Add a Test for Flatten with Heterogeneous...

2016-12-21 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/1677 Add a Test for Flatten with Heterogeneous Coders Add a category, and suppress in the Flink and Apex runners Be sure to do all of the following to help us incorporate your

[GitHub] incubator-beam pull request #1676: Merge remote-tracking branch 'origin/mast...

2016-12-21 Thread aaltay
GitHub user aaltay opened a pull request: https://github.com/apache/incubator-beam/pull/1676 Merge remote-tracking branch 'origin/master' into python-sdk There are frequent Apex runner failures the python-sdk branch, bringing up pom.xml changes to update those. You can merge this

[jira] [Created] (BEAM-1200) PubsubIO should allow for a user to supply the function which computes the watermark that is reported

2016-12-21 Thread Luke Cwik (JIRA)
Luke Cwik created BEAM-1200: --- Summary: PubsubIO should allow for a user to supply the function which computes the watermark that is reported Key: BEAM-1200 URL: https://issues.apache.org/jira/browse/BEAM-1200

[jira] [Commented] (BEAM-1117) Support for new Timer API in Direct runner

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768294#comment-15768294 ] ASF GitHub Bot commented on BEAM-1117: -- Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request #1669: [BEAM-1117] Preliminaries for timers in t...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1669 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[5/7] incubator-beam git commit: Add static Window.withOutputTimeFn to match build method

2016-12-21 Thread kenn
Add static Window.withOutputTimeFn to match build method Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8188040d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8188040d Diff:

[6/7] incubator-beam git commit: Add UsesTestStream for use with JUnit @Category

2016-12-21 Thread kenn
Add UsesTestStream for use with JUnit @Category Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4d71924c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4d71924c Diff:

[2/7] incubator-beam git commit: Hold output watermark according to pending timers

2016-12-21 Thread kenn
Hold output watermark according to pending timers Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dfe2e62d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dfe2e62d Diff:

[3/7] incubator-beam git commit: Use informative Instant formatter in WatermarkHold

2016-12-21 Thread kenn
Use informative Instant formatter in WatermarkHold Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fa4958a6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/fa4958a6 Diff:

[7/7] incubator-beam git commit: This closes #1669: Preliminaries for timers in the direct runner

2016-12-21 Thread kenn
This closes #1669: Preliminaries for timers in the direct runner Hold output watermark according to pending timers Allow setting timer by ID in DirectTimerInternals Add UsesTestStream for use with JUnit @Category Add static Window.withOutputTimeFn to match build method Use informative

[4/7] incubator-beam git commit: Allow setting timer by ID in DirectTimerInternals

2016-12-21 Thread kenn
Allow setting timer by ID in DirectTimerInternals Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7f14c463 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7f14c463 Diff:

[1/7] incubator-beam git commit: Add informative Instant formatter to BoundedWindow

2016-12-21 Thread kenn
Repository: incubator-beam Updated Branches: refs/heads/master ff3951699 -> 57d9bbd79 Add informative Instant formatter to BoundedWindow Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/280a6a8f Tree:

[jira] [Commented] (BEAM-1194) DataflowRunner should test a variety of valid tempLocation/stagingLocation/etc formats.

2016-12-21 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768284#comment-15768284 ] ASF GitHub Bot commented on BEAM-1194: -- Github user dhalperi closed the pull request at:

[GitHub] incubator-beam pull request #1671: [BEAM-1194] Add DataflowLocationIT

2016-12-21 Thread dhalperi
Github user dhalperi closed the pull request at: https://github.com/apache/incubator-beam/pull/1671 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1671: [BEAM-1194] Add DataflowLocationIT

2016-12-21 Thread dhalperi
GitHub user dhalperi reopened a pull request: https://github.com/apache/incubator-beam/pull/1671 [BEAM-1194] Add DataflowLocationIT Opening PR to test. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/incubator-beam

[GitHub] incubator-beam pull request #1675: [Beam-1186] Migrating the remaining tests...

2016-12-21 Thread staslev
GitHub user staslev opened a pull request: https://github.com/apache/incubator-beam/pull/1675 [Beam-1186] Migrating the remaining tests to use TestPipeline as a JUnit rule. Be sure to do all of the following to help us incorporate your contribution quickly and easily:

[GitHub] incubator-beam pull request #1639: [BEAM-1112] Python E2E Test Framework And...

2016-12-21 Thread markflyhigh
Github user markflyhigh closed the pull request at: https://github.com/apache/incubator-beam/pull/1639 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Created] (BEAM-1199) Condense recordAsOutput, finishSpecifyingOutput from POutput

2016-12-21 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1199: - Summary: Condense recordAsOutput, finishSpecifyingOutput from POutput Key: BEAM-1199 URL: https://issues.apache.org/jira/browse/BEAM-1199 Project: Beam Issue

[GitHub] incubator-beam pull request #1639: [BEAM-1112] Python E2E Test Framework And...

2016-12-21 Thread markflyhigh
GitHub user markflyhigh reopened a pull request: https://github.com/apache/incubator-beam/pull/1639 [BEAM-1112] Python E2E Test Framework And Wordcount E2E Test Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make

[jira] [Comment Edited] (BEAM-846) Decouple side input window mapping from WindowFn

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15613384#comment-15613384 ] Kenneth Knowles edited comment on BEAM-846 at 12/21/16 9:19 PM: Design

[jira] [Updated] (BEAM-210) Allow control of empty ON_TIME panes analogous to final panes

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-210: - Component/s: (was: beam-model) > Allow control of empty ON_TIME panes analogous to final panes

[jira] [Updated] (BEAM-260) WindowMappingFn: Know the getSideInputWindow upper bound to release side input resources

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-260: - Component/s: (was: beam-model) beam-model-fn-api > WindowMappingFn: Know the

[jira] [Updated] (BEAM-846) Decouple side input window mapping from WindowFn

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-846: - Component/s: (was: beam-model) > Decouple side input window mapping from WindowFn >

[jira] [Updated] (BEAM-1193) Give Coders URNs and document their binary formats outside the Java code base

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-1193: -- Component/s: (was: beam-model) > Give Coders URNs and document their binary formats outside

[jira] [Updated] (BEAM-653) Refine specification for WindowFn.isCompatible()

2016-12-21 Thread Kenneth Knowles (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-653: - Component/s: beam-model-runner-api > Refine specification for WindowFn.isCompatible() >

  1   2   3   >