[jira] [Commented] (BEAM-3714) JdbcIO.read() should create a forward-only, read-only result set

2018-02-28 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381404#comment-16381404
 ] 

Innocent commented on BEAM-3714:


Hi Eugene, Thanks for your support, I am not sure about the Fetch Size part. 
according to this documentation from Oracle 
[https://docs.oracle.com/cd/A81042_01/DOC/java.816/a81354/resltse5.htm] Fetch 
size is set to 10 by default. I do not have much experience with this did you 
had a specific value/ range of values in mind when suggesting that it should be 
set to a big value ?

> JdbcIO.read() should create a forward-only, read-only result set
> 
>
> Key: BEAM-3714
> URL: https://issues.apache.org/jira/browse/BEAM-3714
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-jdbc
>Reporter: Eugene Kirpichov
>Assignee: Innocent
>Priority: Major
>
> [https://stackoverflow.com/questions/48784889/streaming-data-from-cloudsql-into-dataflow/48819934#48819934]
>  - a user is trying to load a large table from MySQL, and the MySQL JDBC 
> driver requires special measures when loading large result sets.
> JdbcIO currently calls simply "connection.prepareStatement(query)" 
> https://github.com/apache/beam/blob/bb8c12c4956cbe3c6f2e57113e7c0ce2a5c05009/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java#L508
>  - it should specify type TYPE_FORWARD_ONLY and concurrency CONCUR_READ_ONLY 
> - these values should always be used.
> Seems that different databases have different requirements for streaming 
> result sets.
> E.g. MySQL requires setting fetch size; PostgreSQL says "The Connection must 
> not be in autocommit mode." 
> https://jdbc.postgresql.org/documentation/head/query.html#query-with-cursor . 
> Oracle, I think, doesn't have any special requirements but I don't know. 
> Fetch size should probably still be set to a reasonably large value.
> Seems that the common denominator of these requirements is: set fetch size to 
> a reasonably large but not maximum value; disable autocommit (there's nothing 
> to commit in read() anyway).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-3714) JdbcIO.read() should create a forward-only, read-only result set

2018-02-28 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-3714:
--

Assignee: Innocent  (was: Jean-Baptiste Onofré)

> JdbcIO.read() should create a forward-only, read-only result set
> 
>
> Key: BEAM-3714
> URL: https://issues.apache.org/jira/browse/BEAM-3714
> Project: Beam
>  Issue Type: Bug
>  Components: io-java-jdbc
>Reporter: Eugene Kirpichov
>Assignee: Innocent
>Priority: Major
>
> [https://stackoverflow.com/questions/48784889/streaming-data-from-cloudsql-into-dataflow/48819934#48819934]
>  - a user is trying to load a large table from MySQL, and the MySQL JDBC 
> driver requires special measures when loading large result sets.
> JdbcIO currently calls simply "connection.prepareStatement(query)" 
> https://github.com/apache/beam/blob/bb8c12c4956cbe3c6f2e57113e7c0ce2a5c05009/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java#L508
>  - it should specify type TYPE_FORWARD_ONLY and concurrency CONCUR_READ_ONLY 
> - these values should always be used.
> Seems that different databases have different requirements for streaming 
> result sets.
> E.g. MySQL requires setting fetch size; PostgreSQL says "The Connection must 
> not be in autocommit mode." 
> https://jdbc.postgresql.org/documentation/head/query.html#query-with-cursor . 
> Oracle, I think, doesn't have any special requirements but I don't know. 
> Fetch size should probably still be set to a reasonably large value.
> Seems that the common denominator of these requirements is: set fetch size to 
> a reasonably large but not maximum value; disable autocommit (there's nothing 
> to commit in read() anyway).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-2232) ApiSurface tests should run on the jar, not the pre-shaded code.

2017-12-30 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-2232:
--

Assignee: Kenneth Knowles  (was: Innocent)

> ApiSurface tests should run on the jar, not the pre-shaded code.
> 
>
> Key: BEAM-2232
> URL: https://issues.apache.org/jira/browse/BEAM-2232
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>
> Currently, errors in the core SDK ApiSurface definition and loading are 
> caught only by tests of the ApiSurface of a module that depends on it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-1939) Serialize more coders via URN + Class name

2017-12-30 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-1939:
--

Assignee: Kenneth Knowles  (was: Innocent)

> Serialize more coders via URN + Class name
> --
>
> Key: BEAM-1939
> URL: https://issues.apache.org/jira/browse/BEAM-1939
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Kenneth Knowles
>Priority: Trivial
>
> If the size of serializing Standard Coders becomes too large, an arbitrary 
> Standard Coder can be encoded, alongside its components, via an URN and 
> looking up the class when it is to be deserialized.
> See 
> https://github.com/tgroh/beam/commit/070854845346d8e4df824e4aa374688bd095c2c6 
> as an example



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-2232) ApiSurface tests should run on the jar, not the pre-shaded code.

2017-07-13 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-2232:
--

Assignee: Innocent

> ApiSurface tests should run on the jar, not the pre-shaded code.
> 
>
> Key: BEAM-2232
> URL: https://issues.apache.org/jira/browse/BEAM-2232
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, errors in the core SDK ApiSurface definition and loading are 
> caught only by tests of the ApiSurface of a module that depends on it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-1939) Serialize more coders via URN + Class name

2017-07-11 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081751#comment-16081751
 ] 

Innocent commented on BEAM-1939:


Hi Thomas,  I think this ticket needs to be update to reflect the current state 
of the project. It looks like the notion of StandardCoders were replaced. Also 
the current code is way different from what the ticket has in its description. 
These make it very difficult grab what the issue is about since can get enough 
context from a code that do no longer exist.

> Serialize more coders via URN + Class name
> --
>
> Key: BEAM-1939
> URL: https://issues.apache.org/jira/browse/BEAM-1939
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Innocent
>Priority: Trivial
>
> If the size of serializing Standard Coders becomes too large, an arbitrary 
> Standard Coder can be encoded, alongside its components, via an URN and 
> looking up the class when it is to be deserialized.
> See 
> https://github.com/tgroh/beam/commit/070854845346d8e4df824e4aa374688bd095c2c6 
> as an example



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2231) ApiSurface should be lazy

2017-07-10 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080713#comment-16080713
 ] 

Innocent commented on BEAM-2231:


Thank you for your reply, will further check tonight.

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2230) Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined outside of the general ApiSurface class

2017-07-10 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080710#comment-16080710
 ] 

Innocent commented on BEAM-2230:


Create a v0 pull request for this https://github.com/apache/beam/pull/3533

> Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined 
> outside of the general ApiSurface class
> 
>
> Key: BEAM-2230
> URL: https://issues.apache.org/jira/browse/BEAM-2230
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currenlty, ApiSurface.getSdkApiSurface() is highly specialized and also not 
> correct.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-1939) Serialize more coders via URN + Class name

2017-07-10 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080705#comment-16080705
 ] 

Innocent commented on BEAM-1939:


Thank you for your comment will look at it later today.

> Serialize more coders via URN + Class name
> --
>
> Key: BEAM-1939
> URL: https://issues.apache.org/jira/browse/BEAM-1939
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Innocent
>Priority: Trivial
>
> If the size of serializing Standard Coders becomes too large, an arbitrary 
> Standard Coder can be encoded, alongside its components, via an URN and 
> looking up the class when it is to be deserialized.
> See 
> https://github.com/tgroh/beam/commit/070854845346d8e4df824e4aa374688bd095c2c6 
> as an example



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (BEAM-2231) ApiSurface should be lazy

2017-07-08 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079226#comment-16079226
 ] 

Innocent edited comment on BEAM-2231 at 7/8/17 4:56 PM:


Please Kenn provide more info on this especially because I think 

{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code} that will ultimately perform tuning via  {code:java}
addExposedTypes() {code} is this ticket only about performing tuning right from 
the recursive method 
{code:java} getAnyExposurePath(){code} the reason I am asking is because it is 
tagged as a bug and I might be missing the most important part.


was (Author: evindj):
Please Kenn provide more info on this especially because I think 
{noformat}
*no* further _formatting_ is done here
{noformat}

{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code}
that will ultimately perform tuning via  {code:java}
addExposedTypes()
{code} is this ticket only about performing tuning right from the recursive 
method 
{code:java}
getAnyExposurePath()
{code} the reason I am asking is because it is tagged as a bug and I might be 
missing the most important part.

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (BEAM-2231) ApiSurface should be lazy

2017-07-08 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079226#comment-16079226
 ] 

Innocent edited comment on BEAM-2231 at 7/8/17 4:56 PM:


Please Kenn provide more info on this especially because I think 

{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code} that will ultimately perform tuning via  {code:java}
addExposedTypes() {code} is this ticket only about performing tuning right from 
the recursive method ?
{code:java} getAnyExposurePath(){code} The reason I am asking is because it is 
tagged as a bug and I might be missing the most important part.


was (Author: evindj):
Please Kenn provide more info on this especially because I think 

{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code} that will ultimately perform tuning via  {code:java}
addExposedTypes() {code} is this ticket only about performing tuning right from 
the recursive method 
{code:java} getAnyExposurePath(){code} the reason I am asking is because it is 
tagged as a bug and I might be missing the most important part.

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (BEAM-2231) ApiSurface should be lazy

2017-07-08 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079226#comment-16079226
 ] 

Innocent edited comment on BEAM-2231 at 7/8/17 4:55 PM:


Please Kenn provide more info on this especially because I think 
{noformat}
*no* further _formatting_ is done here
{noformat}

{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code}
that will ultimately perform tuning via  {code:java}
addExposedTypes()
{code} is this ticket only about performing tuning right from the recursive 
method 
{code:java}
getAnyExposurePath()
{code} the reason I am asking is because it is tagged as a bug and I might be 
missing the most important part.


was (Author: evindj):
Please Kenn provide more info on this especially because I think 
{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code}
that will ultimately perform tuning via  {code:java}
addExposedTypes()
{code} is this ticket only about performing tuning right from the recursive 
method 
{code:java}
getAnyExposurePath()
{code} the reason I am asking is because it is tagged as a bug and I might be 
missing the most important part.

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2231) ApiSurface should be lazy

2017-07-08 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079226#comment-16079226
 ] 

Innocent commented on BEAM-2231:


Please Kenn provide more info on this especially because I think 
{code:java}
getAnyExposurePath(Class exposedClass, Set excluded)
{code}
that will ultimately perform tuning via  {code:java}
addExposedTypes()
{code} is this ticket only about performing tuning right from the recursive 
method 
{code:java}
getAnyExposurePath()
{code} the reason I am asking is because it is tagged as a bug and I might be 
missing the most important part.

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-940) ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-30 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070987#comment-16070987
 ] 

Innocent commented on BEAM-940:
---

A pull request is opened for this [https://github.com/apache/beam/pull/3470]


> ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or 
> OnTimerMethod
> ---
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2230) Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined outside of the general ApiSurface class

2017-06-30 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070955#comment-16070955
 ] 

Innocent commented on BEAM-2230:


Please Kenn, provide more info here, is it specialized because the way pruning 
patterns, classname or prefix are setup or it is because of a different reason? 
Also can you elaborate on how incorrect it is ?

> Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined 
> outside of the general ApiSurface class
> 
>
> Key: BEAM-2230
> URL: https://issues.apache.org/jira/browse/BEAM-2230
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currenlty, ApiSurface.getSdkApiSurface() is highly specialized and also not 
> correct.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2231) ApiSurface should be lazy

2017-06-30 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070948#comment-16070948
 ] 

Innocent commented on BEAM-2231:


Hi Kenn, I looked a bit at the implementation of this class and found that 
```includingPackage(String packageName, ClassLoader)```  is the only method 
that recursively load classes without pruning. Also just to make sure that we 
are on the same page regarding pruning,if I am trying to load a package like 
```com.apache``` does pruning in this case means I am going to ignore every 
package that does not start with ```com.apache``` or it is the opposite?

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-2230) Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined outside of the general ApiSurface class

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-2230:
--

Assignee: Innocent

> Core SDK ApiSurface should be only org.apache.beam.sdk and should be defined 
> outside of the general ApiSurface class
> 
>
> Key: BEAM-2230
> URL: https://issues.apache.org/jira/browse/BEAM-2230
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currenlty, ApiSurface.getSdkApiSurface() is highly specialized and also not 
> correct.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-2231) ApiSurface should be lazy

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-2231:
--

Assignee: Innocent

> ApiSurface should be lazy
> -
>
> Key: BEAM-2231
> URL: https://issues.apache.org/jira/browse/BEAM-2231
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>
> Currently, the ApiSurface loads classes recursively, when they should be 
> pruned before loading by the pruning pattern. This has caused crashes because 
> some classes that are never referenced in our code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-1785) Port game example in Beam repo to use InjectorUnboundedSource

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-1785:
--

Assignee: Innocent

> Port game example in Beam repo to use InjectorUnboundedSource
> -
>
> Key: BEAM-1785
> URL: https://issues.apache.org/jira/browse/BEAM-1785
> Project: Beam
>  Issue Type: Improvement
>  Components: examples-java
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Minor
>
> We developed an UnboundedSource so users didn't have to run a separate 
> injector, but it isn't in the Beam example. We should put it there, which 
> eliminates a big chunk of complexity.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-1939) Serialize more coders via URN + Class name

2017-06-29 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069461#comment-16069461
 ] 

Innocent commented on BEAM-1939:


Hi Thomas, can you please provide more information on this ticket? I did not 
found any Coders.java class in runner-core/constructor may be this have changed 
since the ticket was created. Did you have any particular class in mind when 
you created the ticket?
Thanks.

> Serialize more coders via URN + Class name
> --
>
> Key: BEAM-1939
> URL: https://issues.apache.org/jira/browse/BEAM-1939
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Innocent
>Priority: Trivial
>
> If the size of serializing Standard Coders becomes too large, an arbitrary 
> Standard Coder can be encoded, alongside its components, via an URN and 
> looking up the class when it is to be deserialized.
> See 
> https://github.com/tgroh/beam/commit/070854845346d8e4df824e4aa374688bd095c2c6 
> as an example



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-1187) GCP Transport not performing timed backoff after connection failure

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-1187:
--

Assignee: Luke Cwik  (was: Innocent)

> GCP Transport not performing timed backoff after connection failure
> ---
>
> Key: BEAM-1187
> URL: https://issues.apache.org/jira/browse/BEAM-1187
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core, sdk-java-gcp
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Minor
>
> The http request retries are failing and seemingly being immediately retried 
> if there is a connection exception. Note that below all the times are the 
> same, and also that we are logging too much. This seems to be related to the 
> interaction by the chaining http request initializer combining the Credential 
> initializer followed by the RetryHttpRequestInitializer. Also, note that we 
> never log "Request failed with IOException, will NOT retry" which implies 
> that the retry logic never made it to the RetryHttpRequestInitializer.
> Action items are:
> 1) Ensure that the RetryHttpRequestInitializer is used
> 2) Ensure that calls do backoff
> 3) PR/3430 -Reduce the logging to one terminal statement saying that we 
> retried X times and final failure was YYY-
> Dump of console output:
> Dec 20, 2016 9:12:20 AM 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner fromOptions
> INFO: PipelineOptions.filesToStage was not specified. Defaulting to files 
> from the classpath: will stage 1 files. Enable logging at DEBUG level to see 
> which files will be staged.
> Dec 20, 2016 9:12:21 AM 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner run
> INFO: Executing pipeline on the Dataflow Service, which will have billing 
> implications related to Google Compute Engine usage and other Google Cloud 
> Services.
> Dec 20, 2016 9:12:21 AM com.google.cloud.dataflow.sdk.util.PackageUtil 
> stageClasspathElements
> INFO: Uploading 1 files from PipelineOptions.filesToStage to staging location 
> to prepare for execution.
> Dec 20, 2016 9:12:21 AM com.google.cloud.dataflow.sdk.util.PackageUtil 
> stageClasspathElements
> INFO: Uploading PipelineOptions.filesToStage complete: 1 files newly 
> uploaded, 0 files cached
> Dec 20, 2016 9:12:22 AM com.google.api.client.http.HttpRequest execute
> WARNING: exception thrown while executing request
> java.net.ConnectException: Connection refused
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>   at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>   at 
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>   at java.net.Socket.connect(Socket.java:589)
>   at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
>   at sun.net.www.http.HttpClient.(HttpClient.java:211)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:308)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:326)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
>   at 
> com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
>   at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
>   at 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:632)
>   at 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:201)
>   at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:181)
>   at 
>

[jira] [Assigned] (BEAM-1187) GCP Transport not performing timed backoff after connection failure

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-1187:
--

Assignee: Innocent

> GCP Transport not performing timed backoff after connection failure
> ---
>
> Key: BEAM-1187
> URL: https://issues.apache.org/jira/browse/BEAM-1187
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-java-core, sdk-java-gcp
>Reporter: Luke Cwik
>Assignee: Innocent
>Priority: Minor
>
> The http request retries are failing and seemingly being immediately retried 
> if there is a connection exception. Note that below all the times are the 
> same, and also that we are logging too much. This seems to be related to the 
> interaction by the chaining http request initializer combining the Credential 
> initializer followed by the RetryHttpRequestInitializer. Also, note that we 
> never log "Request failed with IOException, will NOT retry" which implies 
> that the retry logic never made it to the RetryHttpRequestInitializer.
> Action items are:
> 1) Ensure that the RetryHttpRequestInitializer is used
> 2) Ensure that calls do backoff
> 3) PR/3430 -Reduce the logging to one terminal statement saying that we 
> retried X times and final failure was YYY-
> Dump of console output:
> Dec 20, 2016 9:12:20 AM 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner fromOptions
> INFO: PipelineOptions.filesToStage was not specified. Defaulting to files 
> from the classpath: will stage 1 files. Enable logging at DEBUG level to see 
> which files will be staged.
> Dec 20, 2016 9:12:21 AM 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner run
> INFO: Executing pipeline on the Dataflow Service, which will have billing 
> implications related to Google Compute Engine usage and other Google Cloud 
> Services.
> Dec 20, 2016 9:12:21 AM com.google.cloud.dataflow.sdk.util.PackageUtil 
> stageClasspathElements
> INFO: Uploading 1 files from PipelineOptions.filesToStage to staging location 
> to prepare for execution.
> Dec 20, 2016 9:12:21 AM com.google.cloud.dataflow.sdk.util.PackageUtil 
> stageClasspathElements
> INFO: Uploading PipelineOptions.filesToStage complete: 1 files newly 
> uploaded, 0 files cached
> Dec 20, 2016 9:12:22 AM com.google.api.client.http.HttpRequest execute
> WARNING: exception thrown while executing request
> java.net.ConnectException: Connection refused
>   at java.net.PlainSocketImpl.socketConnect(Native Method)
>   at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>   at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>   at 
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>   at java.net.Socket.connect(Socket.java:589)
>   at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
>   at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
>   at sun.net.www.http.HttpClient.(HttpClient.java:211)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:308)
>   at sun.net.www.http.HttpClient.New(HttpClient.java:326)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
>   at 
> sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
>   at 
> com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
>   at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
>   at 
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
>   at 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:632)
>   at 
> com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:201)
>   at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:181)
>   at 
>

[jira] [Assigned] (BEAM-1939) Serialize more coders via URN + Class name

2017-06-29 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-1939:
--

Assignee: Innocent

> Serialize more coders via URN + Class name
> --
>
> Key: BEAM-1939
> URL: https://issues.apache.org/jira/browse/BEAM-1939
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Reporter: Thomas Groh
>Assignee: Innocent
>Priority: Trivial
>
> If the size of serializing Standard Coders becomes too large, an arbitrary 
> Standard Coder can be encoded, alongside its components, via an URN and 
> looking up the class when it is to be deserialized.
> See 
> https://github.com/tgroh/beam/commit/070854845346d8e4df824e4aa374688bd095c2c6 
> as an example



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-940) ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-28 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066409#comment-16066409
 ] 

Innocent commented on BEAM-940:
---

Is it there a way to create an OnTimerMethod using DoFn and timerId? I am 
looking for a way to key the cache with OnTimerMethod. Can you please elaborate 
 a bit on the idea you had? I think doing this with OnTimerMethod is the best 
way to make this very straightforward. I have the impression that using a tuple 
might improve this part of the code but might require an additional Tuple class 
 I think we should go that route  only if this is not achievable with the 
OnTimerMethod.

> ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or 
> OnTimerMethod
> ---
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (BEAM-940) ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-26 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent updated BEAM-940:
--
Comment: was deleted

(was: Do you mean LoadingCache> was 
renamed to ByteBuddyOnTimerInvokerFactory? or you are refering to 
ByteBuddyDoFnOnTimerInvokerFactory.)

> ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or 
> OnTimerMethod
> ---
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-2482) CodedValueMutationDetector should use the coders structural value

2017-06-20 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-2482:
--

Assignee: Innocent

> CodedValueMutationDetector should use the coders structural value
> -
>
> Key: BEAM-2482
> URL: https://issues.apache.org/jira/browse/BEAM-2482
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Innocent
>Priority: Minor
>  Labels: newbie, starter
>
> Currently the CodedValueMutationDetector[1] checks to see if the objects are 
> Java equals and then compares whether the bytes are equal. Instead of relying 
> on the bytes, we should rely on the coders structural value explicitly[2]. 
> This allows for a coder to provide a fast path equality check with the 
> default still comparing byte array representations.
> 1: 
> https://github.com/apache/beam/blob/eae0d05bd7c088accd927dcfe3e511efbb11c9fd/sdks/java/core/src/main/java/org/apache/beam/sdk/util/MutationDetectors.java
> 2: 
> https://github.com/apache/beam/blob/01b3f87f977d44eac23eb5488074bbc638858a9d/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L252



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-766) Trigger AST should use @AutoValue

2017-06-19 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054958#comment-16054958
 ] 

Innocent commented on BEAM-766:
---

I thought about applying the @AutoValue annotation on the Trigger class in 
Windowing Trigger since every other Trigger derives directly from it. looking 
at the implementation of equals in the class, I have doubt that the 
implementation generated by @AutoValue will be equivalent to that one. This 
might actually be wrong any guidance?

> Trigger AST should use @AutoValue
> -
>
> Key: BEAM-766
> URL: https://issues.apache.org/jira/browse/BEAM-766
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>  Labels: easy, starter
>
> Triggers are already enriched POJOs that could use {{@AutoValue}}. Certainly 
> once they are converted to a simple AST this is the way to go.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-940) ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-19 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054908#comment-16054908
 ] 

Innocent commented on BEAM-940:
---

Do you mean LoadingCache> was renamed 
to ByteBuddyOnTimerInvokerFactory? or you are refering to 
ByteBuddyDoFnOnTimerInvokerFactory.

> ByteBuddyOnTimerInvokerFactory: key the cache with a (Class, id) tuple or 
> OnTimerMethod
> ---
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (BEAM-940) OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-15 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent updated BEAM-940:
--
Comment: was deleted

(was: I could not find any map (reference to the key) or anything matching 
LoadingCache>. I checked 
OnTimerInvokers (I think the class should be called OnTimerInvokerGenerator) 
and ByteBuddyDoFnOnTimerInvokerFactory there is definitely something that I am 
missing.)

> OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod
> 
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-940) OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-15 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051270#comment-16051270
 ] 

Innocent commented on BEAM-940:
---

I could not find any map (reference to the key) or anything matching 
LoadingCache>. I checked 
OnTimerInvokers (I think the class should be called OnTimerInvokerGenerator) 
and ByteBuddyDoFnOnTimerInvokerFactory there is definitely something that I am 
missing.

> OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod
> 
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-940) OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-15 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051271#comment-16051271
 ] 

Innocent commented on BEAM-940:
---

I could not find any map (reference to the key) or anything matching 
LoadingCache>. I checked 
OnTimerInvokers (I think the class should be called OnTimerInvokerGenerator) 
and ByteBuddyDoFnOnTimerInvokerFactory there is definitely something that I am 
missing.

> OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod
> 
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-766) Trigger AST should use @AutoValue

2017-06-15 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-766:
-

Assignee: Innocent

> Trigger AST should use @AutoValue
> -
>
> Key: BEAM-766
> URL: https://issues.apache.org/jira/browse/BEAM-766
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>  Labels: easy, starter
>
> Triggers are already enriched POJOs that could use {{@AutoValue}}. Certainly 
> once they are converted to a simple AST this is the way to go.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (BEAM-940) OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod

2017-06-15 Thread Innocent (JIRA) 

 [ 
https://issues.apache.org/jira/browse/BEAM-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Innocent reassigned BEAM-940:
-

Assignee: Innocent

> OnTimerInvokers: key the cache with a (Class, id) tuple or OnTimerMethod
> 
>
> Key: BEAM-940
> URL: https://issues.apache.org/jira/browse/BEAM-940
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> Right now it is a {{LoadingCache>}}. 
> It is correct but just a bit less straightforward then we might like.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-765) TriggerStateMachine does not need to be separate from OnceTriggerStateMachine

2017-06-15 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050941#comment-16050941
 ] 

Innocent commented on BEAM-765:
---

Created a pull request for this.

> TriggerStateMachine does not need to be separate from OnceTriggerStateMachine
> -
>
> Key: BEAM-765
> URL: https://issues.apache.org/jira/browse/BEAM-765
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> In runners-core, the state machine implementation of triggers does not need 
> to have the fine-grained type-like enforcement of whether a trigger is a 
> {{OnceTrigger}} or not. It may simplify the code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-765) TriggerStateMachine does not need to be separate from OnceTriggerStateMachine

2017-06-13 Thread Innocent (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048632#comment-16048632
 ] 

Innocent commented on BEAM-765:
---

Hi Kenneth, would you please add more details? I noticed that some classes 
extends TriggerStateMachine and other extends OnceTriggerStateMachine. In both 
cases everything is consistent with the associate semantic.(OnceTrigger for 
Triggers which fire at most once). Is the intention to make all implementation 
classes derive from TriggerStateMachine and use another mechanism to enforce 
the OnceTrigger Semantic?

> TriggerStateMachine does not need to be separate from OnceTriggerStateMachine
> -
>
> Key: BEAM-765
> URL: https://issues.apache.org/jira/browse/BEAM-765
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-core
>Reporter: Kenneth Knowles
>Assignee: Innocent
>Priority: Trivial
>
> In runners-core, the state machine implementation of triggers does not need 
> to have the fine-grained type-like enforcement of whether a trigger is a 
> {{OnceTrigger}} or not. It may simplify the code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)