[jira] [Closed] (BEAM-1097) Dataflow error message for non-existing gcpTempLocation is misleading
[
https://issues.apache.org/jira/browse/BEAM-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Wegner closed BEAM-1097.
--
Resolution: Fixed
Fix Version/s: 0.5.0-incubating
> Dataflow error message for non-existing gcpTempLocation is misleading
> -
>
> Key: BEAM-1097
> URL: https://issues.apache.org/jira/browse/BEAM-1097
> Project: Beam
> Issue Type: Bug
> Components: examples-java, runner-dataflow
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
> Fix For: 0.5.0-incubating
>
>
> The error message for specifying a GCP tempLocation which doesn't exist is
> misleading. Rather than mentioning the given path doesn't exist, it says none
> ways specified.
> This is particularly frustrating because it's one of the few configuration
> values necessary to get started with an example or starter archetype, and
> it's easy to introduce a typo as it's specified on the commandline. In my
> case, I was specifying "gs://swegner-tmp" instead of "gs://swegner-test".
> Repro:
> 1. Clone the starter archetype: {noformat}mvn archetype:generate
> -DarchetypeGroupId=org.apache.beam
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-starter{noformat}
> 2. Add beam-runners-google-cloud-dataflow-java as a dependency in the
> generated pom.xml
> 3. Build: {noformat}mvn install{noformat}
> 4. Run: {noformat}mvn exec:java -DmainClass=swegner.StarterPipeline
> -Dexec.args="--runner=DataflowRunner
> --tempLocation=gs://swegner-tmp"{noformat}
> Expected: An error message along the lines of: "The specified GCP temp
> location 'gs://swegner-tmp' does not exist under project 'myGcpProject'"
> bq. [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
> counter-names-test: An exception occured while executing the Java class.
> null: InvocationTargetException: Failed to construct instance from factory
> method DataflowRunner#fromOptions(interface
> org.apache.beam.sdk.options.PipelineOptions): DataflowRunner requires
> gcpTempLocation, and it is missing in PipelineOptions. -> [Help 1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1160) Disabling Read transform validation cause empty file patterns to unexpected succeed
Scott Wegner created BEAM-1160: -- Summary: Disabling Read transform validation cause empty file patterns to unexpected succeed Key: BEAM-1160 URL: https://issues.apache.org/jira/browse/BEAM-1160 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Assignee: Scott Wegner Typically, input file patterns are validated during Pipeline construction, but standard Read transforms include an option to disable validation. This is generally useful but can lead to cases where a Pipeline executes successfully with empty inputs. We should fail execution on empty file-based inputs even when validation is disabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-1097) Dataflow error message for non-existing gcpTempLocation is misleading
Scott Wegner created BEAM-1097:
--
Summary: Dataflow error message for non-existing gcpTempLocation
is misleading
Key: BEAM-1097
URL: https://issues.apache.org/jira/browse/BEAM-1097
Project: Beam
Issue Type: Bug
Components: examples-java, runner-dataflow
Reporter: Scott Wegner
Assignee: Scott Wegner
Priority: Minor
The error message for specifying a GCP tempLocation which doesn't exist is
misleading. Rather than mentioning the given path doesn't exist, it says none
ways specified.
This is particularly frustrating because it's one of the few configuration
values necessary to get started with an example or starter archetype, and it's
easy to introduce a typo as it's specified on the commandline. In my case, I
was specifying "gs://swegner-tmp" instead of "gs://swegner-test".
Repro:
1. Clone the starter archetype: {noformat}mvn archetype:generate
-DarchetypeGroupId=org.apache.beam
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-starter{noformat}
2. Add beam-runners-google-cloud-dataflow-java as a dependency in the generated
pom.xml
3. Build: {noformat}mvn install{noformat}
4. Run: {noformat}mvn exec:java -DmainClass=swegner.StarterPipeline
-Dexec.args="--runner=DataflowRunner --tempLocation=gs://swegner-tmp"{noformat}
Expected: An error message along the lines of: "The specified GCP temp location
'gs://swegner-tmp' does not exist under project 'myGcpProject'"
bq. [ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
counter-names-test: An exception occured while executing the Java class. null:
InvocationTargetException: Failed to construct instance from factory method
DataflowRunner#fromOptions(interface
org.apache.beam.sdk.options.PipelineOptions): DataflowRunner requires
gcpTempLocation, and it is missing in PipelineOptions. -> [Help 1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-1032) Starter archetype pipeline has no runner on its classpath
Scott Wegner created BEAM-1032: -- Summary: Starter archetype pipeline has no runner on its classpath Key: BEAM-1032 URL: https://issues.apache.org/jira/browse/BEAM-1032 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Assignee: Scott Wegner Priority: Minor When generating a new maven project using the starter archetype, the pipeline does not execute because it doesn't have a runner jar on it's classpath: $ mvn exec:java -Dexec.mainClass=swegner.StarterPipeline ... [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project yo: An exception occured while executing the Java class. null: InvocationTargetException: No Runner was specified and the DirectRunner was not found on the classpath. In the examples archetype, we have the direct runner as a dependency. We should do the same for the starter archetype. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (BEAM-436) DirectRunner RunnableOnService tempLocation configuration insufficient
[ https://issues.apache.org/jira/browse/BEAM-436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684080#comment-15684080 ] Scott Wegner edited comment on BEAM-436 at 11/21/16 4:56 PM: - I spoke w/ [~kenn] on Friday, and he pointed out that once we have a runner-independent graph surgery API, these tests can transition from RunnableOnService to NeedsRunner, and we can initialize using a temp directory similar to other tests (TextIO NeedsRunner tests, for example). Since the graph surgery work is underway, I propose to defer this work and later convert these tests to NeedsRunner. ([~tgroh] is there a JIRA I can link?) was (Author: swegner): I spoke w/ [~kenn] on Friday, and he pointed out that once we have a runner-independent graph surgery API, these tests can transition from RunnableOnService to NeedsRunner, and we can initialize using a temp directory similar to other tests (TextIO NeedsRunner tests, for example). Since the graph surgery work is underway, I propose to defer this work and later convert these tests to NeedsRunner. > DirectRunner RunnableOnService tempLocation configuration insufficient > -- > > Key: BEAM-436 > URL: https://issues.apache.org/jira/browse/BEAM-436 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Scott Wegner >Assignee: Scott Wegner > > The way we specify temp storage locations for RunnableOnService tests is not > sufficient, specifically for DirectRunner execution. Right now, > RunnableOnService tests are run for DirectRunner and DataflowRunner, who set > their temp locations differently: > * DirectRunner doesn't specify a temp location directly, but test classes > will use a JUnit @Rule TemporaryDirectory. Individual tests set it as > necessary for tempLocation, and set a fake gs:// path for individual GCP IO > tests. > * DataflowRunner tests pass an actual GCS path as tempRoot, and > TestDataflowRunner will initialize stagingLocation to this path. > This setup makes it difficult to write RunnableOnService tests which pass for > both runners. We should separate temp location setup out of individual test > classes so that RunnableOnService tests "just work" on any runner. > One solution would be to add logic inside TestPipeline#testingPipelineOptions: > # If --tempRoot is specified, use it to set tempLocation and stagingLocation. > Otherwise, use a JUnit TemporaryDirectory to set it > # If tempLocation is a GCS path, use it to set stagingLocation. Otherwise, > use a fake gcs path (i.e. gs://foo) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-436) DirectRunner RunnableOnService tempLocation configuration insufficient
[ https://issues.apache.org/jira/browse/BEAM-436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684080#comment-15684080 ] Scott Wegner commented on BEAM-436: --- I spoke w/ [~kenn] on Friday, and he pointed out that once we have a runner-independent graph surgery API, these tests can transition from RunnableOnService to NeedsRunner, and we can initialize using a temp directory similar to other tests (TextIO NeedsRunner tests, for example). Since the graph surgery work is underway, I propose to defer this work and later convert these tests to NeedsRunner. > DirectRunner RunnableOnService tempLocation configuration insufficient > -- > > Key: BEAM-436 > URL: https://issues.apache.org/jira/browse/BEAM-436 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Scott Wegner >Assignee: Scott Wegner > > The way we specify temp storage locations for RunnableOnService tests is not > sufficient, specifically for DirectRunner execution. Right now, > RunnableOnService tests are run for DirectRunner and DataflowRunner, who set > their temp locations differently: > * DirectRunner doesn't specify a temp location directly, but test classes > will use a JUnit @Rule TemporaryDirectory. Individual tests set it as > necessary for tempLocation, and set a fake gs:// path for individual GCP IO > tests. > * DataflowRunner tests pass an actual GCS path as tempRoot, and > TestDataflowRunner will initialize stagingLocation to this path. > This setup makes it difficult to write RunnableOnService tests which pass for > both runners. We should separate temp location setup out of individual test > classes so that RunnableOnService tests "just work" on any runner. > One solution would be to add logic inside TestPipeline#testingPipelineOptions: > # If --tempRoot is specified, use it to set tempLocation and stagingLocation. > Otherwise, use a JUnit TemporaryDirectory to set it > # If tempLocation is a GCS path, use it to set stagingLocation. Otherwise, > use a fake gcs path (i.e. gs://foo) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-795) org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in Dataflow RunnableOnService
[ https://issues.apache.org/jira/browse/BEAM-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677994#comment-15677994 ] Scott Wegner commented on BEAM-795: --- This is done. [~jasonkuster] can you close? > org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in > Dataflow RunnableOnService > --- > > Key: BEAM-795 > URL: https://issues.apache.org/jira/browse/BEAM-795 > Project: Beam > Issue Type: Bug >Reporter: Jason Kuster >Assignee: Scott Wegner >Priority: Critical > > Test is failing in beam_PostCommit_RunnableOnService_GoogleCloudDataflow > since yesterday. Culprit change seems to be > https://github.com/apache/incubator-beam/commit/ad03d07ae783f054a31e8b2e14100afff8cdf747 > Affected Jenkins runs: > https://builds.apache.org/job/beam_PostCommit_RunnableOnService_GoogleCloudDataflow/ > 1381 - 1386, 1388 as of bug creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-767) Add DisplayData builder '.nested(..)' operation
[ https://issues.apache.org/jira/browse/BEAM-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-767. - > Add DisplayData builder '.nested(..)' operation > --- > > Key: BEAM-767 > URL: https://issues.apache.org/jira/browse/BEAM-767 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: Not applicable > > > See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 > In the Java SDK, components can register display data directly, as well as > include display data from sub-components via the 'include(..)' operation. In > [PR #1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' > to display data metadata and also include a 'delegate(..)' action. > During the change for paths, we noticed an emerging pattern where components > will typically add the .getClass() of a subcomponent before including it. We > can support this convention better by embedding it in a single configurable > operation, '.nested(..)' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-767) Add DisplayData builder '.nested(..)' operation
[ https://issues.apache.org/jira/browse/BEAM-767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677988#comment-15677988 ] Scott Wegner commented on BEAM-767: --- After chatting with @lukecwik, we decided that it was confusing to have a .nested(..) operation which does two things (effectively .add(key, comp.getClass()).include(key, comp)). We thought it would be surprising to have an automatically-added class item. Furthermore, the display data API today is purely additive, and it was confusing having configuration that could configure and subtract in the same fluent API (.nested(..).withLabel(..).withoutClassItem()). We concluded that the current API is preferable because it is easier to understand and use, albeit being more syntax. > Add DisplayData builder '.nested(..)' operation > --- > > Key: BEAM-767 > URL: https://issues.apache.org/jira/browse/BEAM-767 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: Not applicable > > > See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 > In the Java SDK, components can register display data directly, as well as > include display data from sub-components via the 'include(..)' operation. In > [PR #1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' > to display data metadata and also include a 'delegate(..)' action. > During the change for paths, we noticed an emerging pattern where components > will typically add the .getClass() of a subcomponent before including it. We > can support this convention better by embedding it in a single configurable > operation, '.nested(..)' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-767) Add DisplayData builder '.nested(..)' operation
[ https://issues.apache.org/jira/browse/BEAM-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-767. --- Resolution: Won't Fix Fix Version/s: Not applicable > Add DisplayData builder '.nested(..)' operation > --- > > Key: BEAM-767 > URL: https://issues.apache.org/jira/browse/BEAM-767 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: Not applicable > > > See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 > In the Java SDK, components can register display data directly, as well as > include display data from sub-components via the 'include(..)' operation. In > [PR #1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' > to display data metadata and also include a 'delegate(..)' action. > During the change for paths, we noticed an emerging pattern where components > will typically add the .getClass() of a subcomponent before including it. We > can support this convention better by embedding it in a single configurable > operation, '.nested(..)' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-410) ApproximateQuantiles$QuantileBuffer defines compareTo but not equals
[ https://issues.apache.org/jira/browse/BEAM-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605727#comment-15605727 ] Scott Wegner commented on BEAM-410: --- I believe these are separate issues. From the FindBugs description of this issue: bq. Class defines compareTo(...) and uses Object.equals() To fix this issue, we could add .equals() and .hashCode() implementation to ApproximateQuanties.QuantileBuffer. The other issue has to do with using an Integer in Math.ceil(..) > ApproximateQuantiles$QuantileBuffer defines compareTo but not equals > > > Key: BEAM-410 > URL: https://issues.apache.org/jira/browse/BEAM-410 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Neelesh Srinivas Salian >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]: > Class defines compareTo(...) and uses Object.equals() > Applies to: > [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-745) Display Data fails on transforms with nested components of the same type
[ https://issues.apache.org/jira/browse/BEAM-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-745: -- Labels: backward-incompatible (was: ) > Display Data fails on transforms with nested components of the same type > > > Key: BEAM-745 > URL: https://issues.apache.org/jira/browse/BEAM-745 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner > Labels: backward-incompatible > Fix For: 0.3.0-incubating > > > Display Data supports the notion of "sub components", components within a > transform class which can contribute their own display data. We add a > namespace to display data items based on the originating component, which > keeps the display data items unique within the step. > There are instances where a component is included multiple times within a > step. We handle the case of the same instance being shared by simply ignoring > it the second time. However, we don't handle the case of a separate instance > being added of the same class. Currently the separate instances will add > display data with the same namespace and key, causing a failure. > This can come up for example when infrastructure at different levels wrap and > re-wrap a component. We saw this with a bounded source being adapted multiple > times, Bounded -> Unbounded -> Bounded -> Unbounded. The > BoundedToUnboundedSourceAdapter was included multiple times with separate > instances and caused a failure while populating display data. > See: https://github.com/apache/incubator-beam/pull/1088 for a failing test > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-745) Display Data fails on transforms with nested components of the same type
[ https://issues.apache.org/jira/browse/BEAM-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-745: -- Fix Version/s: 0.3.0-incubating > Display Data fails on transforms with nested components of the same type > > > Key: BEAM-745 > URL: https://issues.apache.org/jira/browse/BEAM-745 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.3.0-incubating > > > Display Data supports the notion of "sub components", components within a > transform class which can contribute their own display data. We add a > namespace to display data items based on the originating component, which > keeps the display data items unique within the step. > There are instances where a component is included multiple times within a > step. We handle the case of the same instance being shared by simply ignoring > it the second time. However, we don't handle the case of a separate instance > being added of the same class. Currently the separate instances will add > display data with the same namespace and key, causing a failure. > This can come up for example when infrastructure at different levels wrap and > re-wrap a component. We saw this with a bounded source being adapted multiple > times, Bounded -> Unbounded -> Bounded -> Unbounded. The > BoundedToUnboundedSourceAdapter was included multiple times with separate > instances and caused a failure while populating display data. > See: https://github.com/apache/incubator-beam/pull/1088 for a failing test > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-745) Display Data fails on transforms with nested components of the same type
[ https://issues.apache.org/jira/browse/BEAM-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-745. --- Resolution: Fixed > Display Data fails on transforms with nested components of the same type > > > Key: BEAM-745 > URL: https://issues.apache.org/jira/browse/BEAM-745 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.3.0-incubating > > > Display Data supports the notion of "sub components", components within a > transform class which can contribute their own display data. We add a > namespace to display data items based on the originating component, which > keeps the display data items unique within the step. > There are instances where a component is included multiple times within a > step. We handle the case of the same instance being shared by simply ignoring > it the second time. However, we don't handle the case of a separate instance > being added of the same class. Currently the separate instances will add > display data with the same namespace and key, causing a failure. > This can come up for example when infrastructure at different levels wrap and > re-wrap a component. We saw this with a bounded source being adapted multiple > times, Bounded -> Unbounded -> Bounded -> Unbounded. The > BoundedToUnboundedSourceAdapter was included multiple times with separate > instances and caused a failure while populating display data. > See: https://github.com/apache/incubator-beam/pull/1088 for a failing test > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-795) org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in Dataflow RunnableOnService
[ https://issues.apache.org/jira/browse/BEAM-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595912#comment-15595912 ] Scott Wegner commented on BEAM-795: --- Thanks for opening this. Looking into it now. > org.apache.beam.sdk.io.ReadTest.testStreamingPrimitiveDisplayData failing in > Dataflow RunnableOnService > --- > > Key: BEAM-795 > URL: https://issues.apache.org/jira/browse/BEAM-795 > Project: Beam > Issue Type: Bug >Reporter: Jason Kuster >Assignee: Scott Wegner >Priority: Critical > > Test is failing in beam_PostCommit_RunnableOnService_GoogleCloudDataflow > since yesterday. Culprit change seems to be > https://github.com/apache/incubator-beam/commit/ad03d07ae783f054a31e8b2e14100afff8cdf747 > Affected Jenkins runs: > https://builds.apache.org/job/beam_PostCommit_RunnableOnService_GoogleCloudDataflow/ > 1381 - 1386, 1388 as of bug creation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-767) Add DisplayData builder '.nested(..)' operation
[ https://issues.apache.org/jira/browse/BEAM-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-767: -- Description: See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 In the Java SDK, components can register display data directly, as well as include display data from sub-components via the 'include(..)' operation. In [PR #1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' to display data metadata and also include a 'delegate(..)' action. During the change for paths, we noticed an emerging pattern where components will typically add the .getClass() of a subcomponent before including it. We can support this convention better by embedding it in a single configurable operation, '.nested(..)' was: See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 In the Java SDK, components can register display data directly, as well as include display data from sub-components via the 'include(..)' operation. In [BEAM-1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' to display data metadata and also include a 'delegate(..)' action. During the change for paths, we noticed an emerging pattern where components will typically add the .getClass() of a subcomponent before including it. We can support this convention better by embedding it in a single configurable operation, '.nested(..)' > Add DisplayData builder '.nested(..)' operation > --- > > Key: BEAM-767 > URL: https://issues.apache.org/jira/browse/BEAM-767 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > > See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 > In the Java SDK, components can register display data directly, as well as > include display data from sub-components via the 'include(..)' operation. In > [PR #1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' > to display data metadata and also include a 'delegate(..)' action. > During the change for paths, we noticed an emerging pattern where components > will typically add the .getClass() of a subcomponent before including it. We > can support this convention better by embedding it in a single configurable > operation, '.nested(..)' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-767) Add DisplayData builder '.nested(..)' operation
Scott Wegner created BEAM-767: - Summary: Add DisplayData builder '.nested(..)' operation Key: BEAM-767 URL: https://issues.apache.org/jira/browse/BEAM-767 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Assignee: Scott Wegner See: https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515 In the Java SDK, components can register display data directly, as well as include display data from sub-components via the 'include(..)' operation. In [BEAM-1008|https://github.com/apache/incubator-beam/pull/1088], we add 'paths' to display data metadata and also include a 'delegate(..)' action. During the change for paths, we noticed an emerging pattern where components will typically add the .getClass() of a subcomponent before including it. We can support this convention better by embedding it in a single configurable operation, '.nested(..)' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-745) Display Data fails on transforms with nested components of the same type
Scott Wegner created BEAM-745: - Summary: Display Data fails on transforms with nested components of the same type Key: BEAM-745 URL: https://issues.apache.org/jira/browse/BEAM-745 Project: Beam Issue Type: Bug Reporter: Scott Wegner Assignee: Scott Wegner Display Data supports the notion of "sub components", components within a transform class which can contribute their own display data. We add a namespace to display data items based on the originating component, which keeps the display data items unique within the step. There are instances where a component is included multiple times within a step. We handle the case of the same instance being shared by simply ignoring it the second time. However, we don't handle the case of a separate instance being added of the same class. Currently the separate instances will add display data with the same namespace and key, causing a failure. This can come up for example when infrastructure at different levels wrap and re-wrap a component. We saw this with a bounded source being adapted multiple times, Bounded -> Unbounded -> Bounded -> Unbounded. The BoundedToUnboundedSourceAdapter was included multiple times with separate instances and caused a failure while populating display data. See: https://github.com/apache/incubator-beam/pull/1088 for a failing test case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-691) AvroIO missing display data for Dataflow runner in streaming mode
[ https://issues.apache.org/jira/browse/BEAM-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-691. - Resolution: Fixed Fix Version/s: 0.3.0-incubating > AvroIO missing display data for Dataflow runner in streaming mode > - > > Key: BEAM-691 > URL: https://issues.apache.org/jira/browse/BEAM-691 > Project: Beam > Issue Type: Bug > Components: runner-dataflow >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.3.0-incubating > > > AvroIO's Read display data does not show up in Dataflow runner for streaming > pipelines. This is because the runner has a different graph translation for > when StreamingOptions.isStreaming() is true, and the display data for this > case is not implemented. > We should add display data to the primitive transforms for this translation, > and also explore ways to better test display data integration for different > possible graph translations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (BEAM-688) Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with undeclared dependency error
[
https://issues.apache.org/jira/browse/BEAM-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Wegner reassigned BEAM-688:
-
Assignee: Scott Wegner (was: James Malone)
> Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with
> undeclared dependency error
>
>
> Key: BEAM-688
> URL: https://issues.apache.org/jira/browse/BEAM-688
> Project: Beam
> Issue Type: Bug
> Components: build-system, sdk-java-extensions
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
>
> When building repeatedly using {{mvn install}}, the second build will fail on
> target beam-sdks-java-maven-archetypes-starter with error:
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
> beam-sdks-java-maven-archetypes-starter ---
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
> {noformat}
> For some reason this only happens on the second build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (BEAM-396) Coder.NonDeterministicException doesn't inherit from Exception
[ https://issues.apache.org/jira/browse/BEAM-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-396. - > Coder.NonDeterministicException doesn't inherit from Exception > -- > > Key: BEAM-396 > URL: https://issues.apache.org/jira/browse/BEAM-396 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Chandni Singh >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: 0.3.0-incubating > > > [FindBugs > NM_CLASS_NOT_EXCEPTION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L67]: > Class is not derived from an Exception, even though it is named as such. > Applies to > [Coder.NonDeterministicException|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L263] > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-405) BigQueryIO$StreamingWriteFn uses inefficient keySet iterator
[ https://issues.apache.org/jira/browse/BEAM-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-405. - > BigQueryIO$StreamingWriteFn uses inefficient keySet iterator > > > Key: BEAM-405 > URL: https://issues.apache.org/jira/browse/BEAM-405 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Lucas Amorim >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: 0.2.0-incubating > > > [FindBugs > WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: > Inefficient use of keySet iterator instead of entrySet iterator > Applies to: > [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. > This can be made more efficient by using the entrySet iterator. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-397) Format string uses \n rather than %n
[ https://issues.apache.org/jira/browse/BEAM-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-397. - > Format string uses \n rather than %n > > > Key: BEAM-397 > URL: https://issues.apache.org/jira/browse/BEAM-397 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Lucas Amorim >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: 0.2.0-incubating > > > [FindBugs > VA_FORMAT_STRING_USES_NEWLINE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: > Format string should use %n rather than \n. > Applies to: > [Coder.NonDeterministicExeption.getMessage|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L296] > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-575) "Latest String" Aggregator
[ https://issues.apache.org/jira/browse/BEAM-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-575. - Resolution: Fixed Fix Version/s: 0.3.0-incubating > "Latest String" Aggregator > -- > > Key: BEAM-575 > URL: https://issues.apache.org/jira/browse/BEAM-575 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.3.0-incubating > > > Beam SDK recently gained support for "display data", which is > construction-time metadata about a pipeline. Display data is limited in that > it doesn't provide a way to publish runtime metadata. > Aggregators are the current method for pipelines to publish runtime metadata, > but currently focuses on counter-type metadata. It would be valuable to > expand the support for dynamically-computed properties like "pub-sub > subscription id" or "sink export job status". This would fit nicely into the > aggregators API by providing a "Latest" combiner with String-typed data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-557) Test-scoped dependencies should be excluded from shading package relocation
[ https://issues.apache.org/jira/browse/BEAM-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-557. - > Test-scoped dependencies should be excluded from shading package relocation > --- > > Key: BEAM-557 > URL: https://issues.apache.org/jira/browse/BEAM-557 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > Fix For: 0.3.0-incubating > > > Currently, guava-testlib is being relocated as part of the shading process, > but test-scope dependencies aren't bundled in the uber-jar. As a result, the > output JAR is unusable without recreating the same shading rules in a > consuming project. > Note that this does not effect our maven test process because tests are run > on the unshaded JAR. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-687) Repeated 'mvn install' fails with "java.lang.IllegalStateException: endPosTable already set"
[ https://issues.apache.org/jira/browse/BEAM-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-687. - Resolution: Fixed Fix Version/s: 0.3.0-incubating > Repeated 'mvn install' fails with "java.lang.IllegalStateException: > endPosTable already set" > > > Key: BEAM-687 > URL: https://issues.apache.org/jira/browse/BEAM-687 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.3.0-incubating > > > See [mail > thread|http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201609.mbox/%3CCAOdmQRgb8_aDUu%3DKt7LgJfLHJwffmZoBOWZ%3D69AcrTXgZXncmA%40mail.gmail.com%3E]. > It appears that we're hitting [this > issue|https://issues.apache.org/jira/browse/MCOMPILER-236] with > maven-compiler-plugin, AutoValue, and incremental builds. According to the > bug thread, the issue is fixed with maven-compiler-plugin version 3.5.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-692) DisplayData integration tests don't exercise branches in runner pipeline translation
[ https://issues.apache.org/jira/browse/BEAM-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-692. - Resolution: Won't Fix Fix Version/s: Not applicable > DisplayData integration tests don't exercise branches in runner pipeline > translation > > > Key: BEAM-692 > URL: https://issues.apache.org/jira/browse/BEAM-692 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: Not applicable > > > Display Data has various integration tests (for example > [AvroIOTest.testPrimitiveReadDisplayData()|https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/io/AvroIOTest.java#L287]) > which test display data under a runner as follows: > # Build a PTransform instance to test > # Attach the transform to a pipeline and translate the pipeline with a runner > and a default set of PipelineOptions > # Aggregate display data from the translated pipeline primitive transforms > and verify it includes expected items. > This validates that runners retain a core set of display data through any > PTransform replacements. However, this falls short because pipeline > translation can vary per-runner based on conditions set in PipelineOptions > (for example, > [StreamingOptions.isStreaming()|https://github.com/apache/incubator-beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L325]). > > The tests are runner-agnostic and thus unaware of runner-specific translation > branching. As a result, it's easy to implement pipeline translation logic > with untested display data. > [BEAM-691|https://issues.apache.org/jira/browse/BEAM-691] is one such > instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-678) Python executable should be overridable
[
https://issues.apache.org/jira/browse/BEAM-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Wegner closed BEAM-678.
-
Resolution: Fixed
Fix Version/s: Not applicable
> Python executable should be overridable
> ---
>
> Key: BEAM-678
> URL: https://issues.apache.org/jira/browse/BEAM-678
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
> Fix For: Not applicable
>
>
> Since [PR 962|https://github.com/apache/incubator-beam/pull/962],
> dependency.py will use {{sys.executable}} to source the python executable to
> use for managing dependencies. However, as stated in the [python
> docs|https://docs.python.org/2/library/sys.html#sys.executable]:
> bq. If Python is unable to retrieve the real path to its executable,
> sys.executable will be an empty string or None.
> To mitigate this, we should allow setting the python executable to use via an
> environment variable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-692) DisplayData integration tests don't exercise branches in runner pipeline translation
[ https://issues.apache.org/jira/browse/BEAM-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534167#comment-15534167 ] Scott Wegner commented on BEAM-692: --- Closing this for now. The real solution is to retain display data for composite transforms. This will be available once we have a new runners API. /cc [~kenn] > DisplayData integration tests don't exercise branches in runner pipeline > translation > > > Key: BEAM-692 > URL: https://issues.apache.org/jira/browse/BEAM-692 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner > > Display Data has various integration tests (for example > [AvroIOTest.testPrimitiveReadDisplayData()|https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/io/AvroIOTest.java#L287]) > which test display data under a runner as follows: > # Build a PTransform instance to test > # Attach the transform to a pipeline and translate the pipeline with a runner > and a default set of PipelineOptions > # Aggregate display data from the translated pipeline primitive transforms > and verify it includes expected items. > This validates that runners retain a core set of display data through any > PTransform replacements. However, this falls short because pipeline > translation can vary per-runner based on conditions set in PipelineOptions > (for example, > [StreamingOptions.isStreaming()|https://github.com/apache/incubator-beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L325]). > > The tests are runner-agnostic and thus unaware of runner-specific translation > branching. As a result, it's easy to implement pipeline translation logic > with untested display data. > [BEAM-691|https://issues.apache.org/jira/browse/BEAM-691] is one such > instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-692) DisplayData integration tests don't exercise branches in runner pipeline translation
Scott Wegner created BEAM-692: - Summary: DisplayData integration tests don't exercise branches in runner pipeline translation Key: BEAM-692 URL: https://issues.apache.org/jira/browse/BEAM-692 Project: Beam Issue Type: Bug Reporter: Scott Wegner Assignee: Scott Wegner Display Data has various integration tests (for example [AvroIOTest.testPrimitiveReadDisplayData()|https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/io/AvroIOTest.java#L287]) which test display data under a runner as follows: # Build a PTransform instance to test # Attach the transform to a pipeline and translate the pipeline with a runner and a default set of PipelineOptions # Aggregate display data from the translated pipeline primitive transforms and verify it includes expected items. This validates that runners retain a core set of display data through any PTransform replacements. However, this falls short because pipeline translation can vary per-runner based on conditions set in PipelineOptions (for example, [StreamingOptions.isStreaming()|https://github.com/apache/incubator-beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L325]). The tests are runner-agnostic and thus unaware of runner-specific translation branching. As a result, it's easy to implement pipeline translation logic with untested display data. [BEAM-691|https://issues.apache.org/jira/browse/BEAM-691] is one such instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-691) AvroIO missing display data for Dataflow runner in streaming mode
Scott Wegner created BEAM-691: - Summary: AvroIO missing display data for Dataflow runner in streaming mode Key: BEAM-691 URL: https://issues.apache.org/jira/browse/BEAM-691 Project: Beam Issue Type: Bug Components: runner-dataflow Reporter: Scott Wegner Assignee: Scott Wegner AvroIO's Read display data does not show up in Dataflow runner for streaming pipelines. This is because the runner has a different graph translation for when StreamingOptions.isStreaming() is true, and the display data for this case is not implemented. We should add display data to the primitive transforms for this translation, and also explore ways to better test display data integration for different possible graph translations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-688) Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with undeclared dependency error
[
https://issues.apache.org/jira/browse/BEAM-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530210#comment-15530210
]
Scott Wegner commented on BEAM-688:
---
Yes, [~dhalp...@google.com] and [~jbonofre] you both seem to be correct. Here's
what I see from playing around a bit:
{noformat}
mvn clean install -DskipTests # install dependencies
cd sdks/java/maven-archetypes/starter
mvn clean # start with clean project
mvn dependency:analyze -DignoreNonCompile=false # 1. clean => warning
mvn dependency:analyze -DignoreNonCompile=true # 2. clean => no warning
mvn install # build project
mvn dependency:analyze -DignoreNonCompile=false # 3. prebuilt => warning
mvn dependency:analyze -DignoreNonCompile=true # 4. prebuilt => warning
{noformat}
The difference between ignoreNonCompile true/false is expected, but the
difference between clean/prebuilt (#2 vs #4) seems like a bug.
> Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with
> undeclared dependency error
>
>
> Key: BEAM-688
> URL: https://issues.apache.org/jira/browse/BEAM-688
> Project: Beam
> Issue Type: Bug
> Components: build-system, sdk-java-extensions
>Reporter: Scott Wegner
>Assignee: James Malone
>Priority: Minor
>
> When building repeatedly using {{mvn install}}, the second build will fail on
> target beam-sdks-java-maven-archetypes-starter with error:
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
> beam-sdks-java-maven-archetypes-starter ---
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
> {noformat}
> For some reason this only happens on the second build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-688) Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with undeclared dependency error
[
https://issues.apache.org/jira/browse/BEAM-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530082#comment-15530082
]
Scott Wegner commented on BEAM-688:
---
I began seeing symptoms of this when upgrading the maven-compiler-plugin
version in [PR-1020|https://github.com/apache/incubator-beam/pull/1020] for
[BEAM-687|https://issues.apache.org/jira/browse/BEAM-687].
> Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with
> undeclared dependency error
>
>
> Key: BEAM-688
> URL: https://issues.apache.org/jira/browse/BEAM-688
> Project: Beam
> Issue Type: Bug
> Components: build-system, sdk-java-extensions
>Reporter: Scott Wegner
>Assignee: James Malone
>Priority: Minor
>
> When building repeatedly using {{mvn install}}, the second build will fail on
> target beam-sdks-java-maven-archetypes-starter with error:
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
> beam-sdks-java-maven-archetypes-starter ---
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
> {noformat}
> For some reason this only happens on the second build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-688) Repeated 'mvn install' fails on beam-sdks-java-maven-archetypes-starter with undeclared dependency error
Scott Wegner created BEAM-688:
-
Summary: Repeated 'mvn install' fails on
beam-sdks-java-maven-archetypes-starter with undeclared dependency error
Key: BEAM-688
URL: https://issues.apache.org/jira/browse/BEAM-688
Project: Beam
Issue Type: Bug
Components: build-system, sdk-java-extensions
Reporter: Scott Wegner
Assignee: James Malone
Priority: Minor
When building repeatedly using {{mvn install}}, the second build will fail on
target beam-sdks-java-maven-archetypes-starter with error:
{noformat}
[INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
beam-sdks-java-maven-archetypes-starter ---
[WARNING] Used undeclared dependencies found:
[WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
{noformat}
For some reason this only happens on the second build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-687) Repeated 'mvn install' fails with "java.lang.IllegalStateException: endPosTable already set"
Scott Wegner created BEAM-687: - Summary: Repeated 'mvn install' fails with "java.lang.IllegalStateException: endPosTable already set" Key: BEAM-687 URL: https://issues.apache.org/jira/browse/BEAM-687 Project: Beam Issue Type: Bug Components: build-system Reporter: Scott Wegner Assignee: Scott Wegner See [mail thread|http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201609.mbox/%3CCAOdmQRgb8_aDUu%3DKt7LgJfLHJwffmZoBOWZ%3D69AcrTXgZXncmA%40mail.gmail.com%3E]. It appears that we're hitting [this issue|https://issues.apache.org/jira/browse/MCOMPILER-236] with maven-compiler-plugin, AutoValue, and incremental builds. According to the bug thread, the issue is fixed with maven-compiler-plugin version 3.5.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-680) Python Dataflow stages stale requirements-cache dependencies
[
https://issues.apache.org/jira/browse/BEAM-680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524090#comment-15524090
]
Scott Wegner commented on BEAM-680:
---
/cc [~robertwb]
This came up as an issue with
[dependency_test.test_with_requirements_file()|https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/utils/dependency_test.py#L108]
in [PR 1005|https://github.com/apache/incubator-beam/pull/1005].
We use pip to download all required dependencies, but generate the full listing
by scanning the cache directory. Perhaps there is a way to ask pip for the
transitive dependency list as well.
> Python Dataflow stages stale requirements-cache dependencies
>
>
> Key: BEAM-680
> URL: https://issues.apache.org/jira/browse/BEAM-680
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
>Reporter: Scott Wegner
>Priority: Minor
>
> When executing a python pipeline using a requirements.txt file, the Dataflow
> runner will stage all dependencies downloaded to its requirements cache
> directory, including those specified in the requirements.txt, and any
> previously cached dependencies. This results in bloated staging directory if
> previous pipeline runs from the same machine included different dependencies.
> Repro:
> # Initialize a virtualenv and pip install apache_beam
> # Create an empty requirements.txt file
> # Create a simple pipeline using DataflowPipelineRunner and a
> requirements.txt file, for example:
> [my_pipeline.py|https://gist.github.com/swegner/6df00df1423b48206c4ab5a7e917218a]
> # {{touch /tmp/dataflow-requirements-cache/extra-file.txt}}
> # Run the pipeline with a specified staging directory
> # Check the staged files for the job
> 'extra-file.txt' will be uploaded with the job, along with any other cached
> dependencies under /tmp/dataflow-requirements-cache.
> We should only be staging the dependencies necessary for a pipeline, not all
> previously-cached dependencies found on the machine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-680) Python Dataflow stages stale requirements-cache dependencies
Scott Wegner created BEAM-680:
-
Summary: Python Dataflow stages stale requirements-cache
dependencies
Key: BEAM-680
URL: https://issues.apache.org/jira/browse/BEAM-680
Project: Beam
Issue Type: Bug
Components: sdk-py
Reporter: Scott Wegner
Priority: Minor
When executing a python pipeline using a requirements.txt file, the Dataflow
runner will stage all dependencies downloaded to its requirements cache
directory, including those specified in the requirements.txt, and any
previously cached dependencies. This results in bloated staging directory if
previous pipeline runs from the same machine included different dependencies.
Repro:
# Initialize a virtualenv and pip install apache_beam
# Create an empty requirements.txt file
# Create a simple pipeline using DataflowPipelineRunner and a requirements.txt
file, for example:
[my_pipeline.py|https://gist.github.com/swegner/6df00df1423b48206c4ab5a7e917218a]
# {{touch /tmp/dataflow-requirements-cache/extra-file.txt}}
# Run the pipeline with a specified staging directory
# Check the staged files for the job
'extra-file.txt' will be uploaded with the job, along with any other cached
dependencies under /tmp/dataflow-requirements-cache.
We should only be staging the dependencies necessary for a pipeline, not all
previously-cached dependencies found on the machine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-678) Python executable should be overridable
Scott Wegner created BEAM-678:
-
Summary: Python executable should be overridable
Key: BEAM-678
URL: https://issues.apache.org/jira/browse/BEAM-678
Project: Beam
Issue Type: Bug
Components: sdk-py
Reporter: Scott Wegner
Assignee: Scott Wegner
Priority: Minor
Since [PR 962|https://github.com/apache/incubator-beam/pull/962], dependency.py
will use {{sys.executable}} to source the python executable to use for managing
dependencies. However, as stated in the [python
docs|https://docs.python.org/2/library/sys.html#sys.executable]:
bq. If Python is unable to retrieve the real path to its executable,
sys.executable will be an empty string or None.
To mitigate this, we should allow setting the python executable to use via an
environment variable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-414) IntraBundleParallelization needs to be removed
[ https://issues.apache.org/jira/browse/BEAM-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425499#comment-15425499 ] Scott Wegner commented on BEAM-414: --- Is the plan still to remove IntraBundleParallelization? I also noticed it does not support the new DoFn (only supports OldDoFn). If we don't plan on deprecating, we should update it to support the new DoFn. /cc [~kenn] > IntraBundleParallelization needs to be removed > -- > > Key: BEAM-414 > URL: https://issues.apache.org/jira/browse/BEAM-414 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Minor > Labels: newbie, starter > > IntraBundleParallelization needs to be removed because it does not work since > it breaks bundle processing semantics by expecting that context information > is not mutated by the runner between element processing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-474) Tests which evaluate DisplayData but don't run a pipeline should be marked @NeedsRunner not @RunnableOnService
[ https://issues.apache.org/jira/browse/BEAM-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399726#comment-15399726 ] Scott Wegner commented on BEAM-474: --- These tests verify that the pipeline translation in each runner preserves display data in translated graph. We don't actually run the pipeline, but the tests currently use @RunnableOnService and TestPipeline in order to validate the implementation of each runner. If these shouldn't be @RunnableOnService or @NeedsRunner-- what are they? Is there any reason that @RunnableOnService / @NeedsRunner require that the tests run a pipeline, or can we update the definition? > Tests which evaluate DisplayData but don't run a pipeline should be marked > @NeedsRunner not @RunnableOnService > -- > > Key: BEAM-474 > URL: https://issues.apache.org/jira/browse/BEAM-474 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Luke Cwik >Priority: Minor > > Tests which evaluate DisplayData but don't run a pipeline should be marked > @NeedsRunner not @RunnableOnService > @RunnableOnService represents tests which actually execute a Pipeline while > @NeedsRunner represents tests that need a runner to construct the pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-474) Tests which evaluate DisplayData but don't run a pipeline should be marked @NeedsRunner not @RunnableOnService
[ https://issues.apache.org/jira/browse/BEAM-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394011#comment-15394011 ] Scott Wegner commented on BEAM-474: --- My understanding when we wrote the tests was different. I thought the distinction was: * @NeedsRunner tests require some runner to execute, and will default to DirectRunner * @RunnableOnService tests form a validation suite that runners are implemented correctly, and should be run on all runners If my interpretation was incorrect (and it sounds like it was), then we should fix the tests. I just want to make sure that they will still be executed against all runners. > Tests which evaluate DisplayData but don't run a pipeline should be marked > @NeedsRunner not @RunnableOnService > -- > > Key: BEAM-474 > URL: https://issues.apache.org/jira/browse/BEAM-474 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Luke Cwik >Priority: Minor > > Tests which evaluate DisplayData but don't run a pipeline should be marked > @NeedsRunner not @RunnableOnService > @RunnableOnService represents tests which actually execute a Pipeline while > @NeedsRunner represents tests that need a runner to construct the pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-445) Beam-examples-java build failed through local "mvn install"
[
https://issues.apache.org/jira/browse/BEAM-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15379622#comment-15379622
]
Scott Wegner commented on BEAM-445:
---
I've opened an issue with Bigtable to see if they can upgrade their dependency:
https://github.com/GoogleCloudPlatform/cloud-bigtable-client/issues/917
> Beam-examples-java build failed through local "mvn install"
> ---
>
> Key: BEAM-445
> URL: https://issues.apache.org/jira/browse/BEAM-445
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-gcp
> Environment: linux
>Reporter: Mark Liu
>Assignee: Daniel Halperin
>Priority: Critical
>
> Build project under beam/examples/java with command "mvn clean install
> -DskipTests" failed with following error:
> [ERROR] Failed to execute goal on project beam-examples-java: Could not
> resolve dependencies for project
> org.apache.beam:beam-examples-java:jar:0.2.0-incubating-SNAPSHOT: Could not
> transfer artifact
> io.netty:netty-tcnative-boringssl-static:jar:${os.detected.classifier}:1.1.33.Fork13
> from/to central (http://repo.maven.apache.org/maven2): Illegal character in
> path at index 138:
> http://repo.maven.apache.org/maven2/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork13/netty-tcnative-boringssl-static-1.1.33.Fork13-${os.detected.classifier}.jar
> Reason: can't resolve ${os.detected.classifier} in
> beam/sdks/java/io/google-cloud-platform/pom file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-445) Beam-examples-java build failed through local "mvn install"
[
https://issues.apache.org/jira/browse/BEAM-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373951#comment-15373951
]
Scott Wegner commented on BEAM-445:
---
Looking into this..
The issue is that netty-tcnative is a transitive dependency of
io/google-cloud-platform, and requires the os-maven-plugin extension present to
resolve the artifact classifier. This works for maven builds which only use
gcp-io through its own pom, but fails on those that bring it in as a dependency
artifact, i.e. examples/java. This will also imposes a dependency on customer
projects, confirmed by adding gcp-io as a dependency on the
maven-archetype-starter project.
>From http://netty.io/wiki/forked-tomcat-native.html, there is also an "uber
>jar": "While this tends to be a fairly large jar, it greatly simplifies the
>getting started experience since an application need not be concerned with
>obtaining the correct jar for the platform."
I recommend we target this jar instead in order to simplify user consumption.
> Beam-examples-java build failed through local "mvn install"
> ---
>
> Key: BEAM-445
> URL: https://issues.apache.org/jira/browse/BEAM-445
> Project: Beam
> Issue Type: Bug
> Environment: linux
>Reporter: Mark Liu
>Priority: Critical
>
> Build project under beam/examples/java with command "mvn clean install
> -DskipTests" failed with following error:
> [ERROR] Failed to execute goal on project beam-examples-java: Could not
> resolve dependencies for project
> org.apache.beam:beam-examples-java:jar:0.2.0-incubating-SNAPSHOT: Could not
> transfer artifact
> io.netty:netty-tcnative-boringssl-static:jar:${os.detected.classifier}:1.1.33.Fork13
> from/to central (http://repo.maven.apache.org/maven2): Illegal character in
> path at index 138:
> http://repo.maven.apache.org/maven2/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork13/netty-tcnative-boringssl-static-1.1.33.Fork13-${os.detected.classifier}.jar
> Reason: can't resolve ${os.detected.classifier} in
> beam/sdks/java/io/google-cloud-platform/pom file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BEAM-445) Beam-examples-java build failed through local "mvn install"
[
https://issues.apache.org/jira/browse/BEAM-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373833#comment-15373833
]
Scott Wegner commented on BEAM-445:
---
I also started seeing this today, not sure why this just started failing. I can
see that os-maven-plugin was added to sdks/java/io/google-cloud-platform.
Building this module works, but other modules which depend on it fail. One fix
is to add os-maven-plugin to the root module, but I wonder if there's a better
way to encapsulate the dependency. /cc [~ianzhou]
> Beam-examples-java build failed through local "mvn install"
> ---
>
> Key: BEAM-445
> URL: https://issues.apache.org/jira/browse/BEAM-445
> Project: Beam
> Issue Type: Bug
> Environment: linux
>Reporter: Mark Liu
>Priority: Critical
>
> Build project under beam/examples/java with command "mvn clean install
> -DskipTests" failed with following error:
> [ERROR] Failed to execute goal on project beam-examples-java: Could not
> resolve dependencies for project
> org.apache.beam:beam-examples-java:jar:0.2.0-incubating-SNAPSHOT: Could not
> transfer artifact
> io.netty:netty-tcnative-boringssl-static:jar:${os.detected.classifier}:1.1.33.Fork13
> from/to central (http://repo.maven.apache.org/maven2): Illegal character in
> path at index 138:
> http://repo.maven.apache.org/maven2/io/netty/netty-tcnative-boringssl-static/1.1.33.Fork13/netty-tcnative-boringssl-static-1.1.33.Fork13-${os.detected.classifier}.jar
> Reason: can't resolve ${os.detected.classifier} in
> beam/sdks/java/io/google-cloud-platform/pom file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-436) DirectRunner RunnableOnService tempLocation configuration insufficient
Scott Wegner created BEAM-436: - Summary: DirectRunner RunnableOnService tempLocation configuration insufficient Key: BEAM-436 URL: https://issues.apache.org/jira/browse/BEAM-436 Project: Beam Issue Type: Bug Components: runner-direct Reporter: Scott Wegner Assignee: Davor Bonaci The way we specify temp storage locations for RunnableOnService tests is not sufficient, specifically for DirectRunner execution. Right now, RunnableOnService tests are run for DirectRunner and DataflowRunner, who set their temp locations differently: * DirectRunner doesn't specify a temp location directly, but test classes will use a JUnit @Rule TemporaryDirectory. Individual tests set it as necessary for tempLocation, and set a fake gs:// path for individual GCP IO tests. * DataflowRunner tests pass an actual GCS path as tempRoot, and TestDataflowRunner will initialize stagingLocation to this path. This setup makes it difficult to write RunnableOnService tests which pass for both runners. We should separate temp location setup out of individual test classes so that RunnableOnService tests "just work" on any runner. One solution would be to add logic inside TestPipeline#testingPipelineOptions: # If --tempRoot is specified, use it to set tempLocation and stagingLocation. Otherwise, use a JUnit TemporaryDirectory to set it # If tempLocation is a GCS path, use it to set stagingLocation. Otherwise, use a fake gcs path (i.e. gs://foo) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-421) StateTags$CombiningValueStateTag inherits equals from super class but without incorporating its state
[ https://issues.apache.org/jira/browse/BEAM-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-421: -- Description: [FindBugs EQ_DOESNT_OVERRIDE_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L555]: Class doesn't override equals in superclass Applies to: [StateTags$CombiningValueStateTag|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/state/StateTags.java#L344]. CombiningValueStateTag inherits its equals implementation from its parent class, KeyedCombiningValueStateTag. CombiningValueStateTag has additional fields which are not included in the equality implementation, which may be incorrect. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L477]: Non-transient non-serializable instance field in serializable class Applies to: [WatermarkHold.timerInternals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/runners/core-java/src/main/java/org/apache/beam/sdk/util/WatermarkHold.java#L81]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > StateTags$CombiningValueStateTag inherits equals from super class but without > incorporating its state > - > > Key: BEAM-421 > URL: https://issues.apache.org/jira/browse/BEAM-421 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > EQ_DOESNT_OVERRIDE_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L555]: > Class doesn't override equals in superclass > Applies to: > [StateTags$CombiningValueStateTag|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/state/StateTags.java#L344]. > CombiningValueStateTag inherits its equals implementation from its parent > class, KeyedCombiningValueStateTag. CombiningValueStateTag has additional > fields which are not included in the equality implementation, which may be > incorrect. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-421) StateTags$CombiningValueStateTag inherits equals from super class but without incorporating its state
Scott Wegner created BEAM-421: - Summary: StateTags$CombiningValueStateTag inherits equals from super class but without incorporating its state Key: BEAM-421 URL: https://issues.apache.org/jira/browse/BEAM-421 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L477]: Non-transient non-serializable instance field in serializable class Applies to: [WatermarkHold.timerInternals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/runners/core-java/src/main/java/org/apache/beam/sdk/util/WatermarkHold.java#L81]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-420) Non-transient non-serializable instance field in WatermarkHold
[ https://issues.apache.org/jira/browse/BEAM-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-420: -- Description: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L477]: Non-transient non-serializable instance field in serializable class Applies to: [WatermarkHold.timerInternals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/runners/core-java/src/main/java/org/apache/beam/sdk/util/WatermarkHold.java#L81]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L363]: Non-transient non-serializable instance field in serializable class Applies to: [CombineFnUtil$NonSerializableBoundedKeyedCombineFn.context|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java#L170]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Non-transient non-serializable instance field in WatermarkHold > -- > > Key: BEAM-420 > URL: https://issues.apache.org/jira/browse/BEAM-420 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L477]: > Non-transient non-serializable instance field in serializable class > Applies to: > [WatermarkHold.timerInternals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/runners/core-java/src/main/java/org/apache/beam/sdk/util/WatermarkHold.java#L81]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-420) Non-transient non-serializable instance field in WatermarkHold
Scott Wegner created BEAM-420: - Summary: Non-transient non-serializable instance field in WatermarkHold Key: BEAM-420 URL: https://issues.apache.org/jira/browse/BEAM-420 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L363]: Non-transient non-serializable instance field in serializable class Applies to: [CombineFnUtil$NonSerializableBoundedKeyedCombineFn.context|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java#L170]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-417) AvroUtils$AvroMetadata exposes mutable syncMarker
[ https://issues.apache.org/jira/browse/BEAM-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-417. - Resolution: Invalid Assignee: Scott Wegner Fix Version/s: Not applicable > AvroUtils$AvroMetadata exposes mutable syncMarker > - > > Key: BEAM-417 > URL: https://issues.apache.org/jira/browse/BEAM-417 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: Not applicable > > > [FindBugs > EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L351]: > May expose internal representation by returning reference to mutable object > Applies to: > [AvroUtils$AvroMetadata.getSyncMarker|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/AvroUtils.java#L92]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-402) StructuralByteArray exposes mutable internal byte array
[ https://issues.apache.org/jira/browse/BEAM-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-402. - Resolution: Invalid Assignee: Scott Wegner Fix Version/s: Not applicable > StructuralByteArray exposes mutable internal byte array > --- > > Key: BEAM-402 > URL: https://issues.apache.org/jira/browse/BEAM-402 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: Not applicable > > > [FindBugs > EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112]: > May expose internal representation by returning reference to mutable object > Applies to: > [StructuralByteArray|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StructuralByteArray.java]. > Based on usage, it's not entirely clear whether this needs to be fixed. If > not, the FindBugs suppression should be moved out of > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112] > and inline as a @SuppressFBWarnings annotation. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-419) Non-transient non-serializable instance field in CombineFnUtil$NonSerializableBoundedKeyedCombineFn
[ https://issues.apache.org/jira/browse/BEAM-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-419: -- Description: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L363]: Non-transient non-serializable instance field in serializable class Applies to: [CombineFnUtil$NonSerializableBoundedKeyedCombineFn.context|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java#L170]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SE_TRANSIENT_FIELD_NOT_RESTORED|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L357]: Transient field that isn't set by deserialization. Applies to: [BitSetCoder.byteArrayCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/BitSetCoder.java#L35]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Non-transient non-serializable instance field in > CombineFnUtil$NonSerializableBoundedKeyedCombineFn > --- > > Key: BEAM-419 > URL: https://issues.apache.org/jira/browse/BEAM-419 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L363]: > Non-transient non-serializable instance field in serializable class > Applies to: > [CombineFnUtil$NonSerializableBoundedKeyedCombineFn.context|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java#L170]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-419) Non-transient non-serializable instance field in CombineFnUtil$NonSerializableBoundedKeyedCombineFn
Scott Wegner created BEAM-419: - Summary: Non-transient non-serializable instance field in CombineFnUtil$NonSerializableBoundedKeyedCombineFn Key: BEAM-419 URL: https://issues.apache.org/jira/browse/BEAM-419 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_TRANSIENT_FIELD_NOT_RESTORED|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L357]: Transient field that isn't set by deserialization. Applies to: [BitSetCoder.byteArrayCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/BitSetCoder.java#L35]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-417) AvroUtils$AvroMetadata exposes mutable syncMarker
Scott Wegner created BEAM-417: - Summary: AvroUtils$AvroMetadata exposes mutable syncMarker Key: BEAM-417 URL: https://issues.apache.org/jira/browse/BEAM-417 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs FE_FLOATING_POINT_EQUALITY|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L298]: Test for floating point equality Applies to: [Mean$CountSum.equals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Mean.java#L165]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-418) BitSetCoder transient field byteArrayCoder not set after deserialization
Scott Wegner created BEAM-418: - Summary: BitSetCoder transient field byteArrayCoder not set after deserialization Key: BEAM-418 URL: https://issues.apache.org/jira/browse/BEAM-418 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L351]: May expose internal representation by returning reference to mutable object Applies to: [AvroUtils$AvroMetadata.getSyncMarker|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/AvroUtils.java#L92]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-418) BitSetCoder transient field byteArrayCoder not set after deserialization
[ https://issues.apache.org/jira/browse/BEAM-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-418: -- Description: [FindBugs SE_TRANSIENT_FIELD_NOT_RESTORED|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L357]: Transient field that isn't set by deserialization. Applies to: [BitSetCoder.byteArrayCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/BitSetCoder.java#L35]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L351]: May expose internal representation by returning reference to mutable object Applies to: [AvroUtils$AvroMetadata.getSyncMarker|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/AvroUtils.java#L92]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > BitSetCoder transient field byteArrayCoder not set after deserialization > > > Key: BEAM-418 > URL: https://issues.apache.org/jira/browse/BEAM-418 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > SE_TRANSIENT_FIELD_NOT_RESTORED|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L357]: > Transient field that isn't set by deserialization. > Applies to: > [BitSetCoder.byteArrayCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/BitSetCoder.java#L35]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-417) AvroUtils$AvroMetadata exposes mutable syncMarker
[ https://issues.apache.org/jira/browse/BEAM-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-417: -- Description: [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L351]: May expose internal representation by returning reference to mutable object Applies to: [AvroUtils$AvroMetadata.getSyncMarker|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/AvroUtils.java#L92]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs FE_FLOATING_POINT_EQUALITY|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L298]: Test for floating point equality Applies to: [Mean$CountSum.equals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Mean.java#L165]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > AvroUtils$AvroMetadata exposes mutable syncMarker > - > > Key: BEAM-417 > URL: https://issues.apache.org/jira/browse/BEAM-417 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L351]: > May expose internal representation by returning reference to mutable object > Applies to: > [AvroUtils$AvroMetadata.getSyncMarker|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/AvroUtils.java#L92]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-413) Mean$CountSum tests for floating point equality
[ https://issues.apache.org/jira/browse/BEAM-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-413: -- Description: [FindBugs FE_FLOATING_POINT_EQUALITY|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L298]: Test for floating point equality Applies to: [Mean$CountSum.equals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Mean.java#L165]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L304]: Non-transient non-serializable instance field in serializable class Applies to: [DisplayData.entries|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/display/DisplayData.java#L58]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. Summary: Mean$CountSum tests for floating point equality (was: Mean.CountSum tests for floating point inequality) > Mean$CountSum tests for floating point equality > --- > > Key: BEAM-413 > URL: https://issues.apache.org/jira/browse/BEAM-413 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > FE_FLOATING_POINT_EQUALITY|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L298]: > Test for floating point equality > Applies to: > [Mean$CountSum.equals|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Mean.java#L165]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-413) Mean.CountSum tests for floating point inequality
Scott Wegner created BEAM-413: - Summary: Mean.CountSum tests for floating point inequality Key: BEAM-413 URL: https://issues.apache.org/jira/browse/BEAM-413 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L304]: Non-transient non-serializable instance field in serializable class Applies to: [DisplayData.entries|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/display/DisplayData.java#L58]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-412) DisplayData is marked Serializable but contains non-serializable field
[ https://issues.apache.org/jira/browse/BEAM-412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-412: -- Description: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L304]: Non-transient non-serializable instance field in serializable class Applies to: [DisplayData.entries|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/display/DisplayData.java#L58]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]: Class defines compareTo(...) and uses Object.equals() Applies to: [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > DisplayData is marked Serializable but contains non-serializable field > -- > > Key: BEAM-412 > URL: https://issues.apache.org/jira/browse/BEAM-412 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L304]: > Non-transient non-serializable instance field in serializable class > Applies to: > [DisplayData.entries|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/display/DisplayData.java#L58]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-412) DisplayData is marked Serializable but contains non-serializable field
Scott Wegner created BEAM-412: - Summary: DisplayData is marked Serializable but contains non-serializable field Key: BEAM-412 URL: https://issues.apache.org/jira/browse/BEAM-412 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]: Class defines compareTo(...) and uses Object.equals() Applies to: [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-410) ApproximateQuantiles$QuantileBuffer defines compareTo but not equals
[ https://issues.apache.org/jira/browse/BEAM-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-410: -- Description: [FindBugs EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]: Class defines compareTo(...) and uses Object.equals() Applies to: [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs ICAST_INT_CAST_TO_DOUBLE_PASSED_TO_CEIL|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L274]: Integral value cast to double and then passed to Math.ceil Applies to: [ApproximateQuanitles|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L349]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > ApproximateQuantiles$QuantileBuffer defines compareTo but not equals > > > Key: BEAM-410 > URL: https://issues.apache.org/jira/browse/BEAM-410 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]: > Class defines compareTo(...) and uses Object.equals() > Applies to: > [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-410) ApproximateQuantiles$QuantileBuffer defines compareTo but not equals
Scott Wegner created BEAM-410: - Summary: ApproximateQuantiles$QuantileBuffer defines compareTo but not equals Key: BEAM-410 URL: https://issues.apache.org/jira/browse/BEAM-410 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs ICAST_INT_CAST_TO_DOUBLE_PASSED_TO_CEIL|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L274]: Integral value cast to double and then passed to Math.ceil Applies to: [ApproximateQuanitles|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L349]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-409) Incorrect use of Math.ceil in ApproximateQuantiles
[ https://issues.apache.org/jira/browse/BEAM-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-409: -- Description: [FindBugs ICAST_INT_CAST_TO_DOUBLE_PASSED_TO_CEIL|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L274]: Integral value cast to double and then passed to Math.ceil Applies to: [ApproximateQuanitles|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L349]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L269]: Inconsistent synchronization Applies to: [WindowSupplier.windows|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/WindowSupplier.java#L63]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. Summary: Incorrect use of Math.ceil in ApproximateQuantiles (was: Inconsistent synchronization in WindowSupplier.windows) > Incorrect use of Math.ceil in ApproximateQuantiles > -- > > Key: BEAM-409 > URL: https://issues.apache.org/jira/browse/BEAM-409 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > ICAST_INT_CAST_TO_DOUBLE_PASSED_TO_CEIL|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L274]: > Integral value cast to double and then passed to Math.ceil > Applies to: > [ApproximateQuanitles|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L349]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-409) Inconsistent synchronization in WindowSupplier.windows
[ https://issues.apache.org/jira/browse/BEAM-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-409: -- Description: [FindBugs IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L269]: Inconsistent synchronization Applies to: [WindowSupplier.windows|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/WindowSupplier.java#L63]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs DM_NEXTINT_VIA_NEXTDOUBLE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L209]: Use the nextInt method of Random rather than nextDouble to generate a random integer Applies to: [ProxyInvocationHandler.hashCode|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L96]. This random value is used as a pseudo-unique value for #hashCode(), although the default Object.hasCode() implementation does the same thing, so this should be unnecessary. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Inconsistent synchronization in WindowSupplier.windows > -- > > Key: BEAM-409 > URL: https://issues.apache.org/jira/browse/BEAM-409 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L269]: > Inconsistent synchronization > Applies to: > [WindowSupplier.windows|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/WindowSupplier.java#L63]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-409) Inconsistent synchronization in WindowSupplier.windows
Scott Wegner created BEAM-409: - Summary: Inconsistent synchronization in WindowSupplier.windows Key: BEAM-409 URL: https://issues.apache.org/jira/browse/BEAM-409 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs DM_NEXTINT_VIA_NEXTDOUBLE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L209]: Use the nextInt method of Random rather than nextDouble to generate a random integer Applies to: [ProxyInvocationHandler.hashCode|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L96]. This random value is used as a pseudo-unique value for #hashCode(), although the default Object.hasCode() implementation does the same thing, so this should be unnecessary. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-408) ProxyInvocationHandler uses inefficient Math.random() for random int
[ https://issues.apache.org/jira/browse/BEAM-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-408: -- Description: [FindBugs DM_NEXTINT_VIA_NEXTDOUBLE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L209]: Use the nextInt method of Random rather than nextDouble to generate a random integer Applies to: [ProxyInvocationHandler.hashCode|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L96]. This random value is used as a pseudo-unique value for #hashCode(), although the default Object.hasCode() implementation does the same thing, so this should be unnecessary. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L148]: Inconsistent synchronization Applies to: [OffsetRangeTracker.copy|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java#L263]. Its mutating methods are all marked synchronized otherwise. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > ProxyInvocationHandler uses inefficient Math.random() for random int > > > Key: BEAM-408 > URL: https://issues.apache.org/jira/browse/BEAM-408 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > DM_NEXTINT_VIA_NEXTDOUBLE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L209]: > Use the nextInt method of Random rather than nextDouble to generate a random > integer > Applies to: > [ProxyInvocationHandler.hashCode|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L96]. > This random value is used as a pseudo-unique value for #hashCode(), although > the default Object.hasCode() implementation does the same thing, so this > should be unnecessary. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-408) ProxyInvocationHandler uses inefficient Math.random() for random int
Scott Wegner created BEAM-408: - Summary: ProxyInvocationHandler uses inefficient Math.random() for random int Key: BEAM-408 URL: https://issues.apache.org/jira/browse/BEAM-408 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L148]: Inconsistent synchronization Applies to: [OffsetRangeTracker.copy|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java#L263]. Its mutating methods are all marked synchronized otherwise. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-407) Inconsistent synchronization in OffsetRangeTracker.copy
[ https://issues.apache.org/jira/browse/BEAM-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-407: -- Description: [FindBugs IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L148]: Inconsistent synchronization Applies to: [OffsetRangeTracker.copy|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java#L263]. Its mutating methods are all marked synchronized otherwise. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: Inefficient use of keySet iterator instead of entrySet iterator Applies to: [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. This can be made more efficient by using the entrySet iterator. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Inconsistent synchronization in OffsetRangeTracker.copy > --- > > Key: BEAM-407 > URL: https://issues.apache.org/jira/browse/BEAM-407 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > IS2_INCONSISTENT_SYNC|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L148]: > Inconsistent synchronization > Applies to: > [OffsetRangeTracker.copy|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java#L263]. > Its mutating methods are all marked synchronized otherwise. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-407) Inconsistent synchronization in OffsetRangeTracker.copy
Scott Wegner created BEAM-407: - Summary: Inconsistent synchronization in OffsetRangeTracker.copy Key: BEAM-407 URL: https://issues.apache.org/jira/browse/BEAM-407 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: Inefficient use of keySet iterator instead of entrySet iterator Applies to: [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. This can be made more efficient by using the entrySet iterator. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-406) Unused local variable in Write$Bound
[ https://issues.apache.org/jira/browse/BEAM-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-406. - Resolution: Incomplete Fix Version/s: Not applicable > Unused local variable in Write$Bound > > > Key: BEAM-406 > URL: https://issues.apache.org/jira/browse/BEAM-406 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > Fix For: Not applicable > > > [FindBugs > WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: > Inefficient use of keySet iterator instead of entrySet iterator > Applies to: > [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. > This can be made more efficient by using the entrySet iterator. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-406) Unused local variable in Write$Bound
Scott Wegner created BEAM-406: - Summary: Unused local variable in Write$Bound Key: BEAM-406 URL: https://issues.apache.org/jira/browse/BEAM-406 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: Inefficient use of keySet iterator instead of entrySet iterator Applies to: [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. This can be made more efficient by using the entrySet iterator. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-405) BigQueryIO$StreamingWriteFn uses inefficient keySet iterator
[ https://issues.apache.org/jira/browse/BEAM-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-405: -- Description: [FindBugs WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: Inefficient use of keySet iterator instead of entrySet iterator Applies to: [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. This can be made more efficient by using the entrySet iterator. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112]: May expose internal representation by returning reference to mutable object Applies to: [StructuralByteArray|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StructuralByteArray.java]. Based on usage, it's not entirely clear whether this needs to be fixed. If not, the FindBugs suppression should be moved out of [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112] and inline as a @SuppressFBWarnings annotation. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > BigQueryIO$StreamingWriteFn uses inefficient keySet iterator > > > Key: BEAM-405 > URL: https://issues.apache.org/jira/browse/BEAM-405 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > WMI_WRONG_MAP_ITERATOR|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L124]: > Inefficient use of keySet iterator instead of entrySet iterator > Applies to: > [BigQueryIO$StreamingWriteFn.finishBundle|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BigQueryIO.java#L2080]. > This can be made more efficient by using the entrySet iterator. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-405) BigQueryIO$StreamingWriteFn uses inefficient keySet iterator
Scott Wegner created BEAM-405: - Summary: BigQueryIO$StreamingWriteFn uses inefficient keySet iterator Key: BEAM-405 URL: https://issues.apache.org/jira/browse/BEAM-405 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112]: May expose internal representation by returning reference to mutable object Applies to: [StructuralByteArray|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StructuralByteArray.java]. Based on usage, it's not entirely clear whether this needs to be fixed. If not, the FindBugs suppression should be moved out of [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112] and inline as a @SuppressFBWarnings annotation. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-402) StructuralByteArray exposes mutable internal byte array
[ https://issues.apache.org/jira/browse/BEAM-402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-402: -- Description: [FindBugs EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112]: May expose internal representation by returning reference to mutable object Applies to: [StructuralByteArray|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StructuralByteArray.java]. Based on usage, it's not entirely clear whether this needs to be fixed. If not, the FindBugs suppression should be moved out of [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112] and inline as a @SuppressFBWarnings annotation. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SBSC_USE_STRINGBUFFER_CONCATENATION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L107]: Method invokes inefficient new String(String) constructor Applies to: [StandardCoder.toString|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StandardCoder.java#L104]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > StructuralByteArray exposes mutable internal byte array > --- > > Key: BEAM-402 > URL: https://issues.apache.org/jira/browse/BEAM-402 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > EI_EXPOSE_REP|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112]: > May expose internal representation by returning reference to mutable object > Applies to: > [StructuralByteArray|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StructuralByteArray.java]. > Based on usage, it's not entirely clear whether this needs to be fixed. If > not, the FindBugs suppression should be moved out of > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L112] > and inline as a @SuppressFBWarnings annotation. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-401) StandardCoder uses inefficient string concatenation
[ https://issues.apache.org/jira/browse/BEAM-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-401: -- Description: [FindBugs SBSC_USE_STRINGBUFFER_CONCATENATION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L107]: Method invokes inefficient new String(String) constructor Applies to: [StandardCoder.toString|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StandardCoder.java#L104]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L100]: equals() method does not check for null argument Applies to: [SerializableCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/SerializableCoder.java]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > StandardCoder uses inefficient string concatenation > --- > > Key: BEAM-401 > URL: https://issues.apache.org/jira/browse/BEAM-401 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > SBSC_USE_STRINGBUFFER_CONCATENATION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L107]: > Method invokes inefficient new String(String) constructor > Applies to: > [StandardCoder.toString|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StandardCoder.java#L104]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-401) StandardCoder uses inefficient string concatenation
Scott Wegner created BEAM-401: - Summary: StandardCoder uses inefficient string concatenation Key: BEAM-401 URL: https://issues.apache.org/jira/browse/BEAM-401 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L100]: equals() method does not check for null argument Applies to: [SerializableCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/SerializableCoder.java]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-399) SerializableCoder.equals does not handle null
[ https://issues.apache.org/jira/browse/BEAM-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-399: -- Description: [FindBugs NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L100]: equals() method does not check for null argument Applies to: [SerializableCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/SerializableCoder.java]. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs DC_DOUBLECHECK|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Possible double check of field Applies to: [JAXBCoder.getContext|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/JAXBCoder.java#L113]. For details on why this is incorrect, see: http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > SerializableCoder.equals does not handle null > - > > Key: BEAM-399 > URL: https://issues.apache.org/jira/browse/BEAM-399 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L100]: > equals() method does not check for null argument > Applies to: > [SerializableCoder|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/SerializableCoder.java]. > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-399) SerializableCoder.equals does not handle null
Scott Wegner created BEAM-399: - Summary: SerializableCoder.equals does not handle null Key: BEAM-399 URL: https://issues.apache.org/jira/browse/BEAM-399 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs DC_DOUBLECHECK|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Possible double check of field Applies to: [JAXBCoder.getContext|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/JAXBCoder.java#L113]. For details on why this is incorrect, see: http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-398) JAXBCoder uses incorrect Double-Checked Locking
[ https://issues.apache.org/jira/browse/BEAM-398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357621#comment-15357621 ] Scott Wegner commented on BEAM-398: --- When this is fixed, it should also fix the [IS2_INCONSISTENT_SYNC issue|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L89]. > JAXBCoder uses incorrect Double-Checked Locking > --- > > Key: BEAM-398 > URL: https://issues.apache.org/jira/browse/BEAM-398 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > DC_DOUBLECHECK|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: > Possible double check of field > Applies to: > [JAXBCoder.getContext|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/JAXBCoder.java#L113]. > For details on why this is incorrect, see: > http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-398) JAXBCoder uses incorrect Double-Checked Locking
[ https://issues.apache.org/jira/browse/BEAM-398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-398: -- Description: [FindBugs DC_DOUBLECHECK|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Possible double check of field Applies to: [JAXBCoder.getContext|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/JAXBCoder.java#L113]. For details on why this is incorrect, see: http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs VA_FORMAT_STRING_USES_NEWLINE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Format string should use %n rather than \n. Applies to: [Coder.NonDeterministicExeption.getMessage|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L296] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. Summary: JAXBCoder uses incorrect Double-Checked Locking (was: JAXBCoder uses Double-Checked Locking) > JAXBCoder uses incorrect Double-Checked Locking > --- > > Key: BEAM-398 > URL: https://issues.apache.org/jira/browse/BEAM-398 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > DC_DOUBLECHECK|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: > Possible double check of field > Applies to: > [JAXBCoder.getContext|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/JAXBCoder.java#L113]. > For details on why this is incorrect, see: > http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-398) JAXBCoder uses Double-Checked Locking
Scott Wegner created BEAM-398: - Summary: JAXBCoder uses Double-Checked Locking Key: BEAM-398 URL: https://issues.apache.org/jira/browse/BEAM-398 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs VA_FORMAT_STRING_USES_NEWLINE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Format string should use %n rather than \n. Applies to: [Coder.NonDeterministicExeption.getMessage|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L296] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-397) Format string uses \n rather than %n
[ https://issues.apache.org/jira/browse/BEAM-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-397: -- Description: [FindBugs VA_FORMAT_STRING_USES_NEWLINE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: Format string should use %n rather than \n. Applies to: [Coder.NonDeterministicExeption.getMessage|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L296] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs NM_CLASS_NOT_EXCEPTION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L67]: Class is not derived from an Exception, even though it is named as such. Applies to [Coder.NonDeterministicException|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L263] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Format string uses \n rather than %n > > > Key: BEAM-397 > URL: https://issues.apache.org/jira/browse/BEAM-397 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > VA_FORMAT_STRING_USES_NEWLINE|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L72]: > Format string should use %n rather than \n. > Applies to: > [Coder.NonDeterministicExeption.getMessage|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L296] > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-397) Format string uses \n rather than %n
Scott Wegner created BEAM-397: - Summary: Format string uses \n rather than %n Key: BEAM-397 URL: https://issues.apache.org/jira/browse/BEAM-397 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs NM_CLASS_NOT_EXCEPTION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L67]: Class is not derived from an Exception, even though it is named as such. Applies to [Coder.NonDeterministicException|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L263] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-396) Coder.NonDeterministicException doesn't inherit from Exception
[ https://issues.apache.org/jira/browse/BEAM-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-396: -- Description: [FindBugs NM_CLASS_NOT_EXCEPTION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L67]: Class is not derived from an Exception, even though it is named as such. Applies to [Coder.NonDeterministicException|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L263] This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. was: [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L30]: Non-transient non-serializable instance field in serializable class. Applies to AvroCoder fields: * decoder * encoder * reader These should likely be marked transient and ensure they are properly initialized after deserialization. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. > Coder.NonDeterministicException doesn't inherit from Exception > -- > > Key: BEAM-396 > URL: https://issues.apache.org/jira/browse/BEAM-396 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Scott Wegner >Priority: Minor > Labels: findbugs, newbie, starter > > [FindBugs > NM_CLASS_NOT_EXCEPTION|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L67]: > Class is not derived from an Exception, even though it is named as such. > Applies to > [Coder.NonDeterministicException|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java#L263] > This is a good starter bug. When fixing, please remove the corresponding > entries from > [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] > and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-396) Coder.NonDeterministicException doesn't inherit from Exception
Scott Wegner created BEAM-396: - Summary: Coder.NonDeterministicException doesn't inherit from Exception Key: BEAM-396 URL: https://issues.apache.org/jira/browse/BEAM-396 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L30]: Non-transient non-serializable instance field in serializable class. Applies to AvroCoder fields: * decoder * encoder * reader These should likely be marked transient and ensure they are properly initialized after deserialization. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-394) AvroCoder is Serializable with non-serializable fields
Scott Wegner created BEAM-394: - Summary: AvroCoder is Serializable with non-serializable fields Key: BEAM-394 URL: https://issues.apache.org/jira/browse/BEAM-394 Project: Beam Issue Type: Bug Components: sdk-java-core Reporter: Scott Wegner Priority: Minor [FindBugs SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L30]: Non-transient non-serializable instance field in serializable class. Applies to AvroCoder fields: * decoder * encoder * reader These should likely be marked transient and ensure they are properly initialized after deserialization. This is a good starter bug. When fixing, please remove the corresponding entries from [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml] and verify the build passes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-176) Enable FindBugs static analysis during maven build
[ https://issues.apache.org/jira/browse/BEAM-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-176. - Resolution: Fixed Assignee: Scott Wegner Fix Version/s: 0.2.0-incubating This is now enabled! > Enable FindBugs static analysis during maven build > -- > > Key: BEAM-176 > URL: https://issues.apache.org/jira/browse/BEAM-176 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > Fix For: 0.2.0-incubating > > Attachments: FindBugsResult_4_6_16_9_42_AM.html, > FindBugsResult_4_6_16_9_42_AM.xml > > > [FindBugs|http://findbugs.sourceforge.net/] is a java static analysis tools > to find common coding issues that may result in bugs. > Running it over Beam repository with default configuration surfaces 195 > "bugs", 20 categorized as "high priority". Browsing through, some are obvious > issues that should be fixed. For example: > bq. > [FileBasedSink.java:635|https://github.com/apache/incubator-beam/blob/9793fa2c860279b8c90254260c87e5665b000254/sdks/java/core/src/main/java/com/google/cloud/dataflow/sdk/io/FileBasedSink.java#L635]: > More arguments are passed than are actually used in the format string > bq. > [ApproximateQuantiles.java:346|https://github.com/apache/incubator-beam/blob/9793fa2c860279b8c90254260c87e5665b000254/sdks/java/core/src/main/java/com/google/cloud/dataflow/sdk/transforms/ApproximateQuantiles.java#L346]: > Integral value cast to double and then passed to Math.ceil > FindBugs also has a [maven > plugin|http://gleclaire.github.io/findbugs-maven-plugin/index.html], which > can be used to either generate a report of found-issues, or fail the build. > I recommend we enable FindBugs as a quality gate during our build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-330) Maven exec warning message on Dataflow WordCount Example
[
https://issues.apache.org/jira/browse/BEAM-330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Wegner closed BEAM-330.
-
Resolution: Fixed
This is now fixed.
> Maven exec warning message on Dataflow WordCount Example
>
>
> Key: BEAM-330
> URL: https://issues.apache.org/jira/browse/BEAM-330
> Project: Beam
> Issue Type: Bug
> Components: examples-java
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
>
> When running the WordCount example with DataflowPipelineRunner, sometimes the
> Maven process will emit a warning when trying to tear down the process:
> {quote}{noformat}
> [DEBUG] interrupting thread
> Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
> [DEBUG] joining on thread
> Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
> [WARNING] thread Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
> was interrupted but is still alive after waiting at least 14999msecs
> [WARNING] thread Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
> will linger despite being asked to die via interruption
> [WARNING] NOTE: 1 thread(s) did not finish despite being asked to via
> interruption. This is not a problem with exec:java, it is a problem with the
> running code. Although not serious, it should be remedied.
> [WARNING] Couldn't destroy threadgroup
> org.codehaus.mojo.exec.ExecJavaMojo$IsolatedThreadGroup[name=org.apache.beam.examples.WordCount,maxpri=10]
> {noformat}{quote}
> This appears to be some bad interaction between exec-maven-plugin and
> DataflowPipelineRunner, possibly due to exec-maven-plugin's use of Guice and
> our shading of it.
> The problem doesn't always reproduce, except when executing multiple Maven
> targets, such as "mvn install exec:java \[...\]"
> Disabling exec:java's cleanupDaemonThreads indeed suppresses the issue. I
> recommend we add this configuration to the root pom.xml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (BEAM-331) WindowedWordCount.AddTimestampFn has nondeterministic timestamp bounds
[ https://issues.apache.org/jira/browse/BEAM-331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-331. - Resolution: Fixed This is now fixed. > WindowedWordCount.AddTimestampFn has nondeterministic timestamp bounds > -- > > Key: BEAM-331 > URL: https://issues.apache.org/jira/browse/BEAM-331 > Project: Beam > Issue Type: Bug >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > > The timestamps added in the WindowedWordCount example are based on when the > bundles are executed, which makes the min/max bounds non-deterministic. > It would be more desirable to capture the min/max at construction time. We > would like this to be able to use this as an example of adding display data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-339) Archetype project version shouldn't be coupled to Beam version
Scott Wegner created BEAM-339: - Summary: Archetype project version shouldn't be coupled to Beam version Key: BEAM-339 URL: https://issues.apache.org/jira/browse/BEAM-339 Project: Beam Issue Type: Bug Components: sdk-java-extensions Reporter: Scott Wegner Assignee: Scott Wegner Priority: Minor tl;dr: The maven-archetype project has a version reference of 0.1-SNAPSHOT. This is for the user project and shouldn't be tied to Beam versions. In the maven-archetype projects, we have a test which injects property values and verify that the generated project matches the expected. One of the injected properties is "version", which is currently set to "0.1-snapshot" to match the Beam project versions. The version property represents the version of the user project being created and thus shouldn't be tied to the Beam versioning. We should change it such that the intended usage is more clear and to test that the version isn't being set from the Beam version. See: https://github.com/apache/incubator-beam/pull/444/files/ba55042275bd9b525ee8716e4e1007b7924a647f#r66819150 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-331) WindowedWordCount.AddTimestampFn has nondeterministic timestamp bounds
Scott Wegner created BEAM-331: - Summary: WindowedWordCount.AddTimestampFn has nondeterministic timestamp bounds Key: BEAM-331 URL: https://issues.apache.org/jira/browse/BEAM-331 Project: Beam Issue Type: Bug Reporter: Scott Wegner Assignee: Scott Wegner Priority: Minor The timestamps added in the WindowedWordCount example are based on when the bundles are executed, which makes the min/max bounds non-deterministic. It would be more desirable to capture the min/max at construction time. We would like this to be able to use this as an example of adding display data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BEAM-330) Maven exec warning message on Dataflow WordCount Example
Scott Wegner created BEAM-330:
-
Summary: Maven exec warning message on Dataflow WordCount Example
Key: BEAM-330
URL: https://issues.apache.org/jira/browse/BEAM-330
Project: Beam
Issue Type: Bug
Components: examples-java
Reporter: Scott Wegner
Assignee: Scott Wegner
Priority: Minor
When running the WordCount example with DataflowPipelineRunner, sometimes the
Maven process will emit a warning when trying to tear down the process:
{quote}{noformat}
[DEBUG] interrupting thread
Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
[DEBUG] joining on thread
Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
[WARNING] thread Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
was interrupted but is still alive after waiting at least 14999msecs
[WARNING] thread Thread[pool-1-thread-1,5,org.apache.beam.examples.WordCount]
will linger despite being asked to die via interruption
[WARNING] NOTE: 1 thread(s) did not finish despite being asked to via
interruption. This is not a problem with exec:java, it is a problem with the
running code. Although not serious, it should be remedied.
[WARNING] Couldn't destroy threadgroup
org.codehaus.mojo.exec.ExecJavaMojo$IsolatedThreadGroup[name=org.apache.beam.examples.WordCount,maxpri=10]
{noformat}{quote}
This appears to be some bad interaction between exec-maven-plugin and
DataflowPipelineRunner, possibly due to exec-maven-plugin's use of Guice and
our shading of it.
The problem doesn't always reproduce, except when executing multiple Maven
targets, such as "mvn install exec:java \[...\]"
Disabling exec:java's cleanupDaemonThreads indeed suppresses the issue. I
recommend we add this configuration to the root pom.xml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BEAM-327) Dataflow runner should have configuration for System.out/err handling
Scott Wegner created BEAM-327: - Summary: Dataflow runner should have configuration for System.out/err handling Key: BEAM-327 URL: https://issues.apache.org/jira/browse/BEAM-327 Project: Beam Issue Type: Bug Components: runner-dataflow Reporter: Scott Wegner Assignee: Scott Wegner We would like to support the following scenarios: # Respect global logging filter configuration for System.out/System.err log messages. # Suppress all log message for a given source, including System.out/err (Level.OFF) # Set the log level for messages emitted from System.out/err. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-248) Register DisplayData from anonymous implementation PTransforms
[ https://issues.apache.org/jira/browse/BEAM-248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-248. - Resolution: Fixed Done > Register DisplayData from anonymous implementation PTransforms > -- > > Key: BEAM-248 > URL: https://issues.apache.org/jira/browse/BEAM-248 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner > > Most SDK PTransforms are implemented in terms of lower-level PTransforms, > often with anonymous user-fn implementations at the leaf-level. Currently > display data is only being registered on the composite node and not within > the anonymous implementation. As a result, the details are lost. > We should register display data both in the composite and internal leaf > nodes, particularly when the implementation is anonymous. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-117) Implement the API for Static Display Metadata
[ https://issues.apache.org/jira/browse/BEAM-117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-117: -- Assignee: Ben Chambers (was: Scott Wegner) > Implement the API for Static Display Metadata > - > > Key: BEAM-117 > URL: https://issues.apache.org/jira/browse/BEAM-117 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Ben Chambers >Assignee: Ben Chambers > > As described in the following doc, we would like the SDK to allow associating > display metadata with PTransforms. > https://docs.google.com/document/d/11enEB9JwVp6vO0uOYYTMYTGkr3TdNfELwWqoiUg5ZxM/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-238) Add link URLs for sources / sinks
[ https://issues.apache.org/jira/browse/BEAM-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-238. - Resolution: Fixed Closing this. > Add link URLs for sources / sinks > - > > Key: BEAM-238 > URL: https://issues.apache.org/jira/browse/BEAM-238 > Project: Beam > Issue Type: Bug > Components: website >Reporter: Scott Wegner >Assignee: Jean-Baptiste Onofré > > Where applicable, annotate sources/sink display data with link urls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-250) Exclude TextSource.minBundleSize default value from display data.
[ https://issues.apache.org/jira/browse/BEAM-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-250. - Resolution: Fixed Done > Exclude TextSource.minBundleSize default value from display data. > - > > Key: BEAM-250 > URL: https://issues.apache.org/jira/browse/BEAM-250 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > > TextSource currently registers minBundleSize as display data. We should > exclude it when the value is default of 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-251) Exclude PipelineOptions with @JsonIgnore from display data
[ https://issues.apache.org/jira/browse/BEAM-251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-251. - Resolution: Fixed This is done. > Exclude PipelineOptions with @JsonIgnore from display data > -- > > Key: BEAM-251 > URL: https://issues.apache.org/jira/browse/BEAM-251 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-core >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > > JsonIgnore properties are generally not useful and often very long. We should > exclude them from display data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-308) PipelineOptions throws when using package-private PipelineOptions interfaces from multiple packages
[
https://issues.apache.org/jira/browse/BEAM-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300563#comment-15300563
]
Scott Wegner commented on BEAM-308:
---
PR with failing test cases:
https://github.com/apache/incubator-beam/pull/389/files
/cc [~bchambers] [~lcwik]
> PipelineOptions throws when using package-private PipelineOptions interfaces
> from multiple packages
> ---
>
> Key: BEAM-308
> URL: https://issues.apache.org/jira/browse/BEAM-308
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
>
> If a PipelineOptions instance is used as multiple PipelineOptions
> package-private interfaces from different packages, {{PipelineOptions.as}}
> will throw an exception:
> {quote}
> java.lang.IllegalArgumentException: non-public interfaces from different
> packages
> at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:652)
> at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:592)
> at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:244)
> at java.lang.reflect.WeakCache.get(WeakCache.java:141)
> at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:455)
> at java.lang.reflect.Proxy.getProxyClass(Proxy.java:405)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.validateWellFormed(PipelineOptionsFactory.java:620)
> at
> org.apache.beam.sdk.options.ProxyInvocationHandler.as(ProxyInvocationHandler.java:209)
> at
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:135)
> at com.sun.proxy.$Proxy6.as(Unknown Source)
> {quote}
> This fails because ProxyInvocationHandler attempts to create a Java Proxy
> object implementing the full set of interfaces, which has the
> [restriction|https://docs.oracle.com/javase/7/docs/api/java/lang/reflect/Proxy.html#getProxyClass(java.lang.ClassLoader,%20java.lang.Class...)]:
>
> bq. All non-public interfaces must be in the same package; otherwise, it
> would not be possible for the proxy class to implement all of the interfaces,
> regardless of what package it is defined in.
> This can be triggered in a couple edge-case scenarios:
> # {{PipelineOptions.as}} is called on the same PipelineOptions instance with
> multiple package-private interfaces.
> # {{PipelineOptionsFactory.register}} is called with a package-private
> interface, and then {{PipelineOptions.as}} is called with a different
> package-private instance.
> We hit the second scenario in [DataflowSDK unit
> tests|https://github.com/GoogleCloudPlatform/DataflowJavaSDK/pull/286#issuecomment-221438063].
> It's hard to trigger, but possible.
>
> I propose we make the behavior for package-private options explicit:
> # Give a better exception message if we hit this issue in
> {{PipelineOptions.as}} listing the non-public interfaces and what packages
> they're in.
> # Explicitly reject non-public interfaces from
> {{PipelineOptionsFactory.register}}, since this state is global and is easier
> to cause issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
