[
https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927915#comment-15927915
]
Tobias Feldhaus commented on BEAM-1718:
---------------------------------------
This is the triggering method in a class extending DoFn:
{code}
@Override
public Duration getAllowedTimestampSkew() {
return Duration.millis(Long.MAX_VALUE);
}
{code}
In the
[Javadoc|https://github.com/apache/beam/blob/f360f47f9ca4f4054e9fb583c2a0f5dda9ee19ea/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java#L326]
it states:
"For infinite skew, return Duration.millis(Long.MAX_VALUE)"
This is the stacktrace:
{code}
java.lang.ArithmeticException: The calculation caused an overflow:
-9223372036854775 + -9223372036854775807
at org.joda.time.field.FieldUtils.safeAdd(FieldUtils.java:88)
at org.joda.time.chrono.BaseChronology.add(BaseChronology.java:328)
at org.joda.time.Instant.withDurationAdded(Instant.java:174)
at org.joda.time.Instant.withDurationAdded(Instant.java:192)
at org.joda.time.Instant.minus(Instant.java:246)
at
org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.checkTimestamp(SimpleDoFnRunner.java:587)
at
org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.outputWithTimestamp(SimpleDoFnRunner.java:558)
at
ch.localsearch.dataintel.logfiles.FrontendPipeline$ReadObjects.processElement(FrontendPipeline.java:138)
at
ch.localsearch.dataintel.logfiles.FrontendPipeline$ReadObjects$auxiliary$pWS7oKLF.invokeProcessElement(Unknown
Source)
at
org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:199)
at
org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:161)
at
org.apache.beam.runners.core.PushbackSideInputDoFnRunner.processElement(PushbackSideInputDoFnRunner.java:111)
at
org.apache.beam.runners.core.PushbackSideInputDoFnRunner.processElementInReadyWindows(PushbackSideInputDoFnRunner.java:77)
at
org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:134)
at
org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:51)
at
org.apache.beam.runners.direct.TransformExecutor.processElements(TransformExecutor.java:139)
at
org.apache.beam.runners.direct.TransformExecutor.run(TransformExecutor.java:107)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
I think this is a basic error and/or the Javadoc is misleading.
> Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew()
> causes Overflow/Underflow
> -----------------------------------------------------------------------------------------------------
>
> Key: BEAM-1718
> URL: https://issues.apache.org/jira/browse/BEAM-1718
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 0.5.0
> Reporter: Tobias Feldhaus
> Assignee: Thomas Groh
>
> Overriding getAllowedTimestampSkew() in DoFn and returning
> Duration.millis(Long.MAX_VALUE) (as suggested in the JavaDoc for allowing
> infinite skew) causes an Overflow/Underflow
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
