Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Jean-Baptiste Onofré 

Got your message, so it was a mistake and already fixed.

No worries. Thanks,
Regards
JB

On 10/14/2016 06:35 AM, Jean-Baptiste Onofré wrote:

I saw a push of a eclipse branch on the Beam git repo.

Maybe I missed it, but I didn't see discussion about such branch on the
dev mailing list.

Regards
JB

On 10/13/2016 06:52 PM, Daniel Kulp wrote:


I just submitted a pull request that fixes the code as well as
cherry-picks the yaml change from the last branch.

Dan




On Oct 13, 2016, at 10:48 AM, Jean-Baptiste Onofré 
wrote:

Indeed the .travis.yml has not been merged. I gonna fix that.

Sorry about that.

Regards
JB

On 10/13/2016 04:37 PM, Daniel Kulp wrote:


This is in m2e.That said, it looks like the travis.yml file
wasn’t merged from my “eclipse” branch so Travis wasn’t actually
running agains the eclipse compiler.   That would have caught
this.   JB and I will investigate how that got lost in the merge to
master.

A "mvn -Peclipse-jdt clean install” in direct-java would show the
same error.


Dan




On Oct 13, 2016, at 10:05 AM, Jean-Baptiste Onofré
 wrote:

Hi Dan,

You mean directly building in Eclipse I guess using m2e ?

Regards
JB

On 10/13/2016 03:59 PM, Daniel Kulp wrote:


Just an FYI:   this commit has caused things to not build in
Eclipse, but I’m not exactly sure why.   The errors are in place
where methods of the exact signature just moved into an internal
class so I’m not yet sure why it’s causing an issue.

DescriptionResourcePathLocationType
Bound mismatch: The type Read.Bounded is not a valid
substitute for the bounded parameter > of the type
AppliedPTransform
BoundedReadEvaluatorFactory.java
/beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct
line 134Java Problem


Dan




On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:

Remove KeyedResourcePool

This interface is no longer used. Instead, the runner ensures that
bundles will be provided containing the appropriate input to the
TestStreamEvaluatorFactory.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0

Tree:
http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
Diff:
http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0

Branch: refs/heads/master
Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
Parents: 7306e16
Author: Thomas Groh 
Authored: Wed Oct 5 13:12:48 2016 -0700
Committer: Luke Cwik 
Committed: Thu Oct 6 15:14:38 2016 -0700

--

.../direct/BoundedReadEvaluatorFactory.java |  40 +++--
.../beam/runners/direct/DirectRunner.java   |   2 +
.../beam/runners/direct/EmptyInputProvider.java |  49 ++
.../direct/ExecutorServiceParallelExecutor.java |  27 ++-
.../runners/direct/FlattenEvaluatorFactory.java |  18 +-
.../beam/runners/direct/KeyedResourcePool.java  |  47 --
.../runners/direct/LockedKeyedResourcePool.java |  95 ---
.../beam/runners/direct/RootInputProvider.java  |  41 +
.../runners/direct/RootProviderRegistry.java|  65 
.../direct/RootTransformEvaluatorFactory.java   |  42 -
.../direct/TestStreamEvaluatorFactory.java  |  39 +++--
.../direct/TransformEvaluatorRegistry.java  |  17 +-
.../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
.../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
.../direct/FlattenEvaluatorFactoryTest.java |   3 +-
.../direct/LockedKeyedResourcePoolTest.java | 163
---
.../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
.../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
18 files changed, 269 insertions(+), 449 deletions(-)
--



http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java

--

diff --git
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java

index 4936ad9..326a535 100644
---
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java

+++
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java

@@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
* A {@link TransformEvaluatorFactory} that produces {@link
TransformEvaluator TransformEvaluators}
* for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
*/
-final class BoundedReadEvaluatorFactory implements
RootTransformEvaluatorFactory {
+final class BoundedReadEvaluatorFactory implements
TransformEvaluatorFactory {
 private final EvaluationContext evaluationContext;

 BoundedReadEvaluatorFactory(EvaluationContext

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Jean-Baptiste Onofré 

I saw a push of a eclipse branch on the Beam git repo.

Maybe I missed it, but I didn't see discussion about such branch on the 
dev mailing list.


Regards
JB

On 10/13/2016 06:52 PM, Daniel Kulp wrote:


I just submitted a pull request that fixes the code as well as cherry-picks the 
yaml change from the last branch.

Dan




On Oct 13, 2016, at 10:48 AM, Jean-Baptiste Onofré  wrote:

Indeed the .travis.yml has not been merged. I gonna fix that.

Sorry about that.

Regards
JB

On 10/13/2016 04:37 PM, Daniel Kulp wrote:


This is in m2e.That said, it looks like the travis.yml file wasn’t merged 
from my “eclipse” branch so Travis wasn’t actually running agains the eclipse 
compiler.   That would have caught this.   JB and I will investigate how that 
got lost in the merge to master.

A "mvn -Peclipse-jdt clean install” in direct-java would show the same error.


Dan




On Oct 13, 2016, at 10:05 AM, Jean-Baptiste Onofré  wrote:

Hi Dan,

You mean directly building in Eclipse I guess using m2e ?

Regards
JB

On 10/13/2016 03:59 PM, Daniel Kulp wrote:


Just an FYI:   this commit has caused things to not build in Eclipse, but I’m 
not exactly sure why.   The errors are in place where methods of the exact 
signature just moved into an internal class so I’m not yet sure why it’s 
causing an issue.

Description ResourcePathLocationType
Bound mismatch: The type Read.Bounded is not a valid substitute for the bounded 
parameter > of the type 
AppliedPTransform BoundedReadEvaluatorFactory.java
/beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  line 134Java Problem


Dan




On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:

Remove KeyedResourcePool

This interface is no longer used. Instead, the runner ensures that
bundles will be provided containing the appropriate input to the
TestStreamEvaluatorFactory.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0

Branch: refs/heads/master
Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
Parents: 7306e16
Author: Thomas Groh 
Authored: Wed Oct 5 13:12:48 2016 -0700
Committer: Luke Cwik 
Committed: Thu Oct 6 15:14:38 2016 -0700

--
.../direct/BoundedReadEvaluatorFactory.java |  40 +++--
.../beam/runners/direct/DirectRunner.java   |   2 +
.../beam/runners/direct/EmptyInputProvider.java |  49 ++
.../direct/ExecutorServiceParallelExecutor.java |  27 ++-
.../runners/direct/FlattenEvaluatorFactory.java |  18 +-
.../beam/runners/direct/KeyedResourcePool.java  |  47 --
.../runners/direct/LockedKeyedResourcePool.java |  95 ---
.../beam/runners/direct/RootInputProvider.java  |  41 +
.../runners/direct/RootProviderRegistry.java|  65 
.../direct/RootTransformEvaluatorFactory.java   |  42 -
.../direct/TestStreamEvaluatorFactory.java  |  39 +++--
.../direct/TransformEvaluatorRegistry.java  |  17 +-
.../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
.../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
.../direct/FlattenEvaluatorFactoryTest.java |   3 +-
.../direct/LockedKeyedResourcePoolTest.java | 163 ---
.../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
.../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
18 files changed, 269 insertions(+), 449 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
--
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
index 4936ad9..326a535 100644
--- 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
+++ 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
@@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
* A {@link TransformEvaluatorFactory} that produces {@link TransformEvaluator 
TransformEvaluators}
* for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
*/
-final class BoundedReadEvaluatorFactory implements 
RootTransformEvaluatorFactory {
+final class BoundedReadEvaluatorFactory implements TransformEvaluatorFactory {
 private final EvaluationContext evaluationContext;

 BoundedReadEvaluatorFactory(EvaluationContext evaluationContext) {
   this.evaluationContext = evaluationContext;
 }

-  @Override
-  public Collection>

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Daniel Kulp 

I just submitted a pull request that fixes the code as well as cherry-picks the 
yaml change from the last branch.  

Dan



> On Oct 13, 2016, at 10:48 AM, Jean-Baptiste Onofré  wrote:
> 
> Indeed the .travis.yml has not been merged. I gonna fix that.
> 
> Sorry about that.
> 
> Regards
> JB
> 
> On 10/13/2016 04:37 PM, Daniel Kulp wrote:
>> 
>> This is in m2e.That said, it looks like the travis.yml file wasn’t 
>> merged from my “eclipse” branch so Travis wasn’t actually running agains the 
>> eclipse compiler.   That would have caught this.   JB and I will investigate 
>> how that got lost in the merge to master.
>> 
>> A "mvn -Peclipse-jdt clean install” in direct-java would show the same error.
>> 
>> 
>> Dan
>> 
>> 
>> 
>>> On Oct 13, 2016, at 10:05 AM, Jean-Baptiste Onofré  
>>> wrote:
>>> 
>>> Hi Dan,
>>> 
>>> You mean directly building in Eclipse I guess using m2e ?
>>> 
>>> Regards
>>> JB
>>> 
>>> On 10/13/2016 03:59 PM, Daniel Kulp wrote:
 
 Just an FYI:   this commit has caused things to not build in Eclipse, but 
 I’m not exactly sure why.   The errors are in place where methods of the 
 exact signature just moved into an internal class so I’m not yet sure why 
 it’s causing an issue.
 
 DescriptionResourcePathLocationType
 Bound mismatch: The type Read.Bounded is not a valid substitute 
 for the bounded parameter >>> InputT,OutputT>> of the type AppliedPTransform  
   BoundedReadEvaluatorFactory.java
 /beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  
 line 134Java Problem
 
 
 Dan
 
 
 
 
 On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:
> Remove KeyedResourcePool
> 
> This interface is no longer used. Instead, the runner ensures that
> bundles will be provided containing the appropriate input to the
> TestStreamEvaluatorFactory.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
> Commit: 
> http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
> Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
> Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0
> 
> Branch: refs/heads/master
> Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
> Parents: 7306e16
> Author: Thomas Groh 
> Authored: Wed Oct 5 13:12:48 2016 -0700
> Committer: Luke Cwik 
> Committed: Thu Oct 6 15:14:38 2016 -0700
> 
> --
> .../direct/BoundedReadEvaluatorFactory.java |  40 +++--
> .../beam/runners/direct/DirectRunner.java   |   2 +
> .../beam/runners/direct/EmptyInputProvider.java |  49 ++
> .../direct/ExecutorServiceParallelExecutor.java |  27 ++-
> .../runners/direct/FlattenEvaluatorFactory.java |  18 +-
> .../beam/runners/direct/KeyedResourcePool.java  |  47 --
> .../runners/direct/LockedKeyedResourcePool.java |  95 ---
> .../beam/runners/direct/RootInputProvider.java  |  41 +
> .../runners/direct/RootProviderRegistry.java|  65 
> .../direct/RootTransformEvaluatorFactory.java   |  42 -
> .../direct/TestStreamEvaluatorFactory.java  |  39 +++--
> .../direct/TransformEvaluatorRegistry.java  |  17 +-
> .../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
> .../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
> .../direct/FlattenEvaluatorFactoryTest.java |   3 +-
> .../direct/LockedKeyedResourcePoolTest.java | 163 ---
> .../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
> .../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
> 18 files changed, 269 insertions(+), 449 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> --
> diff --git 
> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>  
> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> index 4936ad9..326a535 100644
> --- 
> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> +++ 
> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> @@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
> * A {@link TransformEvaluatorFactory} that produces {@link 
> TransformEvaluator TransformEvaluators}
> * for the {@link Bounded Read.Bounded} primitive {@link PTransform}

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Jean-Baptiste Onofré 

Indeed the .travis.yml has not been merged. I gonna fix that.

Sorry about that.

Regards
JB

On 10/13/2016 04:37 PM, Daniel Kulp wrote:


This is in m2e.That said, it looks like the travis.yml file wasn’t merged 
from my “eclipse” branch so Travis wasn’t actually running agains the eclipse 
compiler.   That would have caught this.   JB and I will investigate how that 
got lost in the merge to master.

A "mvn -Peclipse-jdt clean install” in direct-java would show the same error.


Dan




On Oct 13, 2016, at 10:05 AM, Jean-Baptiste Onofré  wrote:

Hi Dan,

You mean directly building in Eclipse I guess using m2e ?

Regards
JB

On 10/13/2016 03:59 PM, Daniel Kulp wrote:


Just an FYI:   this commit has caused things to not build in Eclipse, but I’m 
not exactly sure why.   The errors are in place where methods of the exact 
signature just moved into an internal class so I’m not yet sure why it’s 
causing an issue.

Description ResourcePathLocationType
Bound mismatch: The type Read.Bounded is not a valid substitute for the bounded 
parameter > of the type 
AppliedPTransform BoundedReadEvaluatorFactory.java
/beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  line 134Java Problem


Dan




On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:

Remove KeyedResourcePool

This interface is no longer used. Instead, the runner ensures that
bundles will be provided containing the appropriate input to the
TestStreamEvaluatorFactory.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0

Branch: refs/heads/master
Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
Parents: 7306e16
Author: Thomas Groh 
Authored: Wed Oct 5 13:12:48 2016 -0700
Committer: Luke Cwik 
Committed: Thu Oct 6 15:14:38 2016 -0700

--
.../direct/BoundedReadEvaluatorFactory.java |  40 +++--
.../beam/runners/direct/DirectRunner.java   |   2 +
.../beam/runners/direct/EmptyInputProvider.java |  49 ++
.../direct/ExecutorServiceParallelExecutor.java |  27 ++-
.../runners/direct/FlattenEvaluatorFactory.java |  18 +-
.../beam/runners/direct/KeyedResourcePool.java  |  47 --
.../runners/direct/LockedKeyedResourcePool.java |  95 ---
.../beam/runners/direct/RootInputProvider.java  |  41 +
.../runners/direct/RootProviderRegistry.java|  65 
.../direct/RootTransformEvaluatorFactory.java   |  42 -
.../direct/TestStreamEvaluatorFactory.java  |  39 +++--
.../direct/TransformEvaluatorRegistry.java  |  17 +-
.../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
.../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
.../direct/FlattenEvaluatorFactoryTest.java |   3 +-
.../direct/LockedKeyedResourcePoolTest.java | 163 ---
.../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
.../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
18 files changed, 269 insertions(+), 449 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
--
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
index 4936ad9..326a535 100644
--- 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
+++ 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
@@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
 * A {@link TransformEvaluatorFactory} that produces {@link TransformEvaluator 
TransformEvaluators}
 * for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
 */
-final class BoundedReadEvaluatorFactory implements 
RootTransformEvaluatorFactory {
+final class BoundedReadEvaluatorFactory implements TransformEvaluatorFactory {
  private final EvaluationContext evaluationContext;

  BoundedReadEvaluatorFactory(EvaluationContext evaluationContext) {
this.evaluationContext = evaluationContext;
  }

-  @Override
-  public Collection> getInitialInputs(AppliedPTransform transform) {
-return createInitialSplits((AppliedPTransform) transform);
-  }
-
-  private  Collection> createInitialSplits(
-  AppliedPTransform> transform) {
-BoundedSource source = transform.getTransform().getSource();
-return Collections.>singleton(
-evaluationContext
-.>createRootBundle()
-
.a

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Daniel Kulp 

This is in m2e.That said, it looks like the travis.yml file wasn’t merged 
from my “eclipse” branch so Travis wasn’t actually running agains the eclipse 
compiler.   That would have caught this.   JB and I will investigate how that 
got lost in the merge to master.

A "mvn -Peclipse-jdt clean install” in direct-java would show the same error.


Dan



> On Oct 13, 2016, at 10:05 AM, Jean-Baptiste Onofré  wrote:
> 
> Hi Dan,
> 
> You mean directly building in Eclipse I guess using m2e ?
> 
> Regards
> JB
> 
> On 10/13/2016 03:59 PM, Daniel Kulp wrote:
>> 
>> Just an FYI:   this commit has caused things to not build in Eclipse, but 
>> I’m not exactly sure why.   The errors are in place where methods of the 
>> exact signature just moved into an internal class so I’m not yet sure why 
>> it’s causing an issue.
>> 
>> Description  ResourcePathLocationType
>> Bound mismatch: The type Read.Bounded is not a valid substitute for 
>> the bounded parameter > InputT,OutputT>> of the type AppliedPTransform  
>> BoundedReadEvaluatorFactory.java
>> /beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  line 
>> 134Java Problem
>> 
>> 
>> Dan
>> 
>> 
>> 
>> 
>> On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:
>>> Remove KeyedResourcePool
>>> 
>>> This interface is no longer used. Instead, the runner ensures that
>>> bundles will be provided containing the appropriate input to the
>>> TestStreamEvaluatorFactory.
>>> 
>>> 
>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
>>> Commit: 
>>> http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
>>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
>>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0
>>> 
>>> Branch: refs/heads/master
>>> Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
>>> Parents: 7306e16
>>> Author: Thomas Groh 
>>> Authored: Wed Oct 5 13:12:48 2016 -0700
>>> Committer: Luke Cwik 
>>> Committed: Thu Oct 6 15:14:38 2016 -0700
>>> 
>>> --
>>> .../direct/BoundedReadEvaluatorFactory.java |  40 +++--
>>> .../beam/runners/direct/DirectRunner.java   |   2 +
>>> .../beam/runners/direct/EmptyInputProvider.java |  49 ++
>>> .../direct/ExecutorServiceParallelExecutor.java |  27 ++-
>>> .../runners/direct/FlattenEvaluatorFactory.java |  18 +-
>>> .../beam/runners/direct/KeyedResourcePool.java  |  47 --
>>> .../runners/direct/LockedKeyedResourcePool.java |  95 ---
>>> .../beam/runners/direct/RootInputProvider.java  |  41 +
>>> .../runners/direct/RootProviderRegistry.java|  65 
>>> .../direct/RootTransformEvaluatorFactory.java   |  42 -
>>> .../direct/TestStreamEvaluatorFactory.java  |  39 +++--
>>> .../direct/TransformEvaluatorRegistry.java  |  17 +-
>>> .../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
>>> .../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
>>> .../direct/FlattenEvaluatorFactoryTest.java |   3 +-
>>> .../direct/LockedKeyedResourcePoolTest.java | 163 ---
>>> .../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
>>> .../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
>>> 18 files changed, 269 insertions(+), 449 deletions(-)
>>> --
>>> 
>>> 
>>> http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>>> --
>>> diff --git 
>>> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>>>  
>>> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>>> index 4936ad9..326a535 100644
>>> --- 
>>> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>>> +++ 
>>> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>>> @@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
>>>  * A {@link TransformEvaluatorFactory} that produces {@link 
>>> TransformEvaluator TransformEvaluators}
>>>  * for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
>>>  */
>>> -final class BoundedReadEvaluatorFactory implements 
>>> RootTransformEvaluatorFactory {
>>> +final class BoundedReadEvaluatorFactory implements 
>>> TransformEvaluatorFactory {
>>>   private final EvaluationContext evaluationContext;
>>> 
>>>   BoundedReadEvaluatorFactory(EvaluationContext evaluationContext) {
>>> this.evaluationContext = evaluationContext;
>>>   }
>>> 
>>> -  @Override
>>> -  public Collection> 
>>> getInitialInputs(AppliedPTransform transform) {
>>> -return createInitialSplits((AppliedPTransform) transform);
>>> -  }
>>> -
>>> -

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Jean-Baptiste Onofré 

Hi Dan,

You mean directly building in Eclipse I guess using m2e ?

Regards
JB

On 10/13/2016 03:59 PM, Daniel Kulp wrote:


Just an FYI:   this commit has caused things to not build in Eclipse, but I’m 
not exactly sure why.   The errors are in place where methods of the exact 
signature just moved into an internal class so I’m not yet sure why it’s 
causing an issue.

Description ResourcePathLocationType
Bound mismatch: The type Read.Bounded is not a valid substitute for the bounded 
parameter > of the type 
AppliedPTransform BoundedReadEvaluatorFactory.java
/beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  line 134Java Problem


Dan




On 2016-10-06 18:31 (-0400), lc...@apache.org wrote:

Remove KeyedResourcePool

This interface is no longer used. Instead, the runner ensures that
bundles will be provided containing the appropriate input to the
TestStreamEvaluatorFactory.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0

Branch: refs/heads/master
Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
Parents: 7306e16
Author: Thomas Groh 
Authored: Wed Oct 5 13:12:48 2016 -0700
Committer: Luke Cwik 
Committed: Thu Oct 6 15:14:38 2016 -0700

--
 .../direct/BoundedReadEvaluatorFactory.java |  40 +++--
 .../beam/runners/direct/DirectRunner.java   |   2 +
 .../beam/runners/direct/EmptyInputProvider.java |  49 ++
 .../direct/ExecutorServiceParallelExecutor.java |  27 ++-
 .../runners/direct/FlattenEvaluatorFactory.java |  18 +-
 .../beam/runners/direct/KeyedResourcePool.java  |  47 --
 .../runners/direct/LockedKeyedResourcePool.java |  95 ---
 .../beam/runners/direct/RootInputProvider.java  |  41 +
 .../runners/direct/RootProviderRegistry.java|  65 
 .../direct/RootTransformEvaluatorFactory.java   |  42 -
 .../direct/TestStreamEvaluatorFactory.java  |  39 +++--
 .../direct/TransformEvaluatorRegistry.java  |  17 +-
 .../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
 .../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
 .../direct/FlattenEvaluatorFactoryTest.java |   3 +-
 .../direct/LockedKeyedResourcePoolTest.java | 163 ---
 .../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
 .../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
 18 files changed, 269 insertions(+), 449 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
--
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
index 4936ad9..326a535 100644
--- 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
+++ 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
@@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
  * A {@link TransformEvaluatorFactory} that produces {@link TransformEvaluator 
TransformEvaluators}
  * for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
  */
-final class BoundedReadEvaluatorFactory implements 
RootTransformEvaluatorFactory {
+final class BoundedReadEvaluatorFactory implements TransformEvaluatorFactory {
   private final EvaluationContext evaluationContext;

   BoundedReadEvaluatorFactory(EvaluationContext evaluationContext) {
 this.evaluationContext = evaluationContext;
   }

-  @Override
-  public Collection> getInitialInputs(AppliedPTransform transform) {
-return createInitialSplits((AppliedPTransform) transform);
-  }
-
-  private  Collection> createInitialSplits(
-  AppliedPTransform> transform) {
-BoundedSource source = transform.getTransform().getSource();
-return Collections.>singleton(
-evaluationContext
-.>createRootBundle()
-
.add(WindowedValue.valueInGlobalWindow(BoundedSourceShard.of(source)))
-.commit(BoundedWindow.TIMESTAMP_MAX_VALUE));
-  }
-
   @SuppressWarnings({"unchecked", "rawtypes"})
   @Override
   @Nullable
@@ -132,4 +117,27 @@ final class BoundedReadEvaluatorFactory implements 
RootTransformEvaluatorFactory

 abstract BoundedSource getSource();
   }
+
+  static class InputProvider implements RootInputProvider {
+private final EvaluationContext evaluationContext;
+
+InputProvider(EvaluationContext evaluationContex

Re: [3/4] incubator-beam git commit: Remove KeyedResourcePool

2016-10-13 Thread Daniel Kulp 

Just an FYI:   this commit has caused things to not build in Eclipse, but I’m 
not exactly sure why.   The errors are in place where methods of the exact 
signature just moved into an internal class so I’m not yet sure why it’s 
causing an issue.

Description ResourcePathLocationType
Bound mismatch: The type Read.Bounded is not a valid substitute for 
the bounded parameter > 
of the type AppliedPTransform 
BoundedReadEvaluatorFactory.java
/beam-runners-direct-java/src/main/java/org/apache/beam/runners/direct  line 
134Java Problem


Dan




On 2016-10-06 18:31 (-0400), lc...@apache.org wrote: 
> Remove KeyedResourcePool
> 
> This interface is no longer used. Instead, the runner ensures that
> bundles will be provided containing the appropriate input to the
> TestStreamEvaluatorFactory.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
> Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/41fb16f0
> Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/41fb16f0
> Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/41fb16f0
> 
> Branch: refs/heads/master
> Commit: 41fb16f014a79d2b9c149c5b369db12b61c4c774
> Parents: 7306e16
> Author: Thomas Groh 
> Authored: Wed Oct 5 13:12:48 2016 -0700
> Committer: Luke Cwik 
> Committed: Thu Oct 6 15:14:38 2016 -0700
> 
> --
>  .../direct/BoundedReadEvaluatorFactory.java |  40 +++--
>  .../beam/runners/direct/DirectRunner.java   |   2 +
>  .../beam/runners/direct/EmptyInputProvider.java |  49 ++
>  .../direct/ExecutorServiceParallelExecutor.java |  27 ++-
>  .../runners/direct/FlattenEvaluatorFactory.java |  18 +-
>  .../beam/runners/direct/KeyedResourcePool.java  |  47 --
>  .../runners/direct/LockedKeyedResourcePool.java |  95 ---
>  .../beam/runners/direct/RootInputProvider.java  |  41 +
>  .../runners/direct/RootProviderRegistry.java|  65 
>  .../direct/RootTransformEvaluatorFactory.java   |  42 -
>  .../direct/TestStreamEvaluatorFactory.java  |  39 +++--
>  .../direct/TransformEvaluatorRegistry.java  |  17 +-
>  .../direct/UnboundedReadEvaluatorFactory.java   |  56 ---
>  .../direct/BoundedReadEvaluatorFactoryTest.java |   3 +-
>  .../direct/FlattenEvaluatorFactoryTest.java |   3 +-
>  .../direct/LockedKeyedResourcePoolTest.java | 163 ---
>  .../direct/TestStreamEvaluatorFactoryTest.java  |   3 +-
>  .../UnboundedReadEvaluatorFactoryTest.java  |   8 +-
>  18 files changed, 269 insertions(+), 449 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/41fb16f0/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> --
> diff --git 
> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
>  
> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> index 4936ad9..326a535 100644
> --- 
> a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> +++ 
> b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/BoundedReadEvaluatorFactory.java
> @@ -39,28 +39,13 @@ import org.apache.beam.sdk.values.PCollection;
>   * A {@link TransformEvaluatorFactory} that produces {@link 
> TransformEvaluator TransformEvaluators}
>   * for the {@link Bounded Read.Bounded} primitive {@link PTransform}.
>   */
> -final class BoundedReadEvaluatorFactory implements 
> RootTransformEvaluatorFactory {
> +final class BoundedReadEvaluatorFactory implements TransformEvaluatorFactory 
> {
>private final EvaluationContext evaluationContext;
>  
>BoundedReadEvaluatorFactory(EvaluationContext evaluationContext) {
>  this.evaluationContext = evaluationContext;
>}
>  
> -  @Override
> -  public Collection> 
> getInitialInputs(AppliedPTransform transform) {
> -return createInitialSplits((AppliedPTransform) transform);
> -  }
> -
> -  private  Collection> createInitialSplits(
> -  AppliedPTransform> transform) {
> -BoundedSource source = transform.getTransform().getSource();
> -return Collections.>singleton(
> -evaluationContext
> -.>createRootBundle()
> -
> .add(WindowedValue.valueInGlobalWindow(BoundedSourceShard.of(source)))
> -.commit(BoundedWindow.TIMESTAMP_MAX_VALUE));
> -  }
> -
>@SuppressWarnings({"unchecked", "rawtypes"})
>@Override
>@Nullable
> @@ -132,4 +117,27 @@ final class BoundedReadEvaluatorFactory implements 
> RootTransformEvaluatorFactory
>  
>  abstract BoundedSource getSource();
>}
> +
> +  static class InputProvider implements RootInputProvider {
> +private final EvaluationContex