[
https://issues.apache.org/jira/browse/FLINK-16227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053166#comment-17053166
]
Robert Metzger commented on FLINK-16227:
----------------------------------------
Another issue with this test surfaced here:
https://dev.azure.com/rmetzger/Flink/_build/results?buildId=5990&view=logs&j=68a897ab-3047-5660-245a-cce8f83859f6&t=d47e27f5-9721-5d5f-1cf3-62adbf3d115d
{code}
2020-03-06T01:53:40.5992521Z
==============================================================================
2020-03-06T01:53:40.5993540Z Running 'Streaming bucketing end-to-end test'
2020-03-06T01:53:40.5994090Z
==============================================================================
2020-03-06T01:53:40.6007142Z TEST_DATA_DIR:
/home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-40600099298
2020-03-06T01:53:40.7839292Z Flink dist directory:
/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT
2020-03-06T01:53:40.8004840Z Setting up SSL with: internal JDK dynamic
2020-03-06T01:53:40.8044255Z Using SAN
dns:fv-az678.ut43s2juxjyexmvox34o2ig20f.fx.internal.cloudapp.net,ip:10.1.0.4,ip:172.17.0.1,ip:172.18.0.1
2020-03-06T01:53:42.1002112Z Certificate was added to keystore
2020-03-06T01:53:44.4867298Z Certificate was added to keystore
2020-03-06T01:53:45.1790583Z Certificate reply was installed in keystore
2020-03-06T01:53:45.4049658Z Setting up SSL with: rest JDK dynamic
2020-03-06T01:53:45.4087013Z Using SAN
dns:fv-az678.ut43s2juxjyexmvox34o2ig20f.fx.internal.cloudapp.net,ip:10.1.0.4,ip:172.17.0.1,ip:172.18.0.1
2020-03-06T01:53:46.7157558Z Certificate was added to keystore
2020-03-06T01:53:49.0606058Z Certificate was added to keystore
2020-03-06T01:53:49.7343063Z Certificate reply was installed in keystore
2020-03-06T01:53:49.9341650Z Mutual ssl auth: false
2020-03-06T01:53:50.0191284Z Starting cluster.
2020-03-06T01:53:50.2687106Z Starting standalonesession daemon on host fv-az678.
2020-03-06T01:53:53.4847093Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:53:53.5306709Z Waiting for Dispatcher REST endpoint to come up...
2020-03-06T01:53:54.5796584Z Waiting for Dispatcher REST endpoint to come up...
2020-03-06T01:53:56.3721187Z Waiting for Dispatcher REST endpoint to come up...
2020-03-06T01:53:57.5797893Z Waiting for Dispatcher REST endpoint to come up...
2020-03-06T01:53:58.8026513Z Waiting for Dispatcher REST endpoint to come up...
2020-03-06T01:53:59.9472093Z Dispatcher REST endpoint is up.
2020-03-06T01:54:01.5553181Z [INFO] 1 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:54:01.5558710Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:54:04.7618054Z [INFO] 2 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:54:04.7632985Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:54:10.0221144Z [INFO] 3 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:54:10.0236528Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:54:10.2740697Z Number of running task managers 2 is not yet 4.
2020-03-06T01:54:14.4714275Z Number of running task managers 3 is not yet 4.
2020-03-06T01:54:18.6094979Z Number of running task managers has reached 4.
2020-03-06T01:54:29.8066570Z Job (d6cc5747b81d9cff05abdcad35645039) is running.
2020-03-06T01:54:29.8067497Z Waiting for job (d6cc5747b81d9cff05abdcad35645039)
to have at least 5 completed checkpoints ...
2020-03-06T01:54:40.8744845Z Killing 1 TM
2020-03-06T01:54:41.0843979Z TaskManager 46047 killed.
2020-03-06T01:54:41.2330609Z Number of running task managers 4 is not yet 3.
2020-03-06T01:54:45.3792262Z Number of running task managers 4 is not yet 3.
2020-03-06T01:54:49.5089221Z Number of running task managers 4 is not yet 3.
2020-03-06T01:54:53.6601323Z Number of running task managers 4 is not yet 3.
2020-03-06T01:54:57.7821195Z Number of running task managers has reached 3.
2020-03-06T01:54:57.7822107Z Restarting 1 TM
2020-03-06T01:54:59.4070399Z [INFO] 3 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:54:59.4081118Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:54:59.5576004Z Number of running task managers 3 is not yet 4.
2020-03-06T01:55:04.0385251Z Number of running task managers has reached 4.
2020-03-06T01:55:04.0386044Z Killing 2 TMs
2020-03-06T01:55:04.6170140Z TaskManager 46743 killed.
2020-03-06T01:55:05.2196770Z TaskManager 47637 killed.
2020-03-06T01:55:05.3721773Z Number of running task managers 4 is not yet 2.
2020-03-06T01:55:09.4897558Z Number of running task managers 4 is not yet 2.
2020-03-06T01:55:13.6145632Z Number of running task managers 4 is not yet 2.
2020-03-06T01:55:17.7360623Z Number of running task managers 3 is not yet 2.
2020-03-06T01:55:21.8593811Z Number of running task managers 3 is not yet 2.
2020-03-06T01:55:25.9804403Z Number of running task managers has reached 2.
2020-03-06T01:55:25.9805734Z Starting 2 TMs
2020-03-06T01:55:27.5597523Z [INFO] 2 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:55:27.5602890Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:55:30.7546069Z [INFO] 3 instance(s) of taskexecutor are already
running on fv-az678.
2020-03-06T01:55:30.7554895Z Starting taskexecutor daemon on host fv-az678.
2020-03-06T01:55:30.9662610Z Number of running task managers 2 is not yet 4.
2020-03-06T01:55:35.1279213Z Number of running task managers 3 is not yet 4.
2020-03-06T01:55:39.3370727Z Number of running task managers has reached 4.
2020-03-06T01:55:41.9612458Z Job (d6cc5747b81d9cff05abdcad35645039) is running.
2020-03-06T01:55:41.9613955Z Waiting until all values have been produced
2020-03-06T01:55:41.9633776Z Truncating buckets
2020-03-06T01:55:47.0661363Z Number of produced values 24685/60000
2020-03-06T01:55:47.0688098Z Truncating buckets
2020-03-06T01:55:52.1749576Z Truncating buckets
2020-03-06T01:55:57.2776012Z Truncating buckets
2020-03-06T01:56:02.3867590Z Truncating buckets
2020-03-06T01:56:07.4942731Z Truncating buckets
2020-03-06T01:56:12.6044103Z Truncating buckets
2020-03-06T01:56:17.7062657Z Truncating buckets
2020-03-06T01:56:22.8158118Z Truncating buckets
2020-03-06T01:56:28.0017052Z Number of produced values 69265/60000
2020-03-06T01:56:29.2206317Z Cancelling job d6cc5747b81d9cff05abdcad35645039.
2020-03-06T01:56:30.2041445Z Cancelled job d6cc5747b81d9cff05abdcad35645039.
2020-03-06T01:56:30.2188289Z Waiting for job (d6cc5747b81d9cff05abdcad35645039)
to reach terminal state CANCELED ...
2020-03-06T01:56:30.3042160Z Job (d6cc5747b81d9cff05abdcad35645039) reached
terminal state CANCELED
2020-03-06T01:56:30.3043340Z Job d6cc5747b81d9cff05abdcad35645039 was
cancelled, time to verify
2020-03-06T01:56:30.4041460Z FAIL Bucketing Sink: Output hash mismatch. Got
aa12afbdb9732094890a1c4429223ed4, expected 01aba5ff77a0ef5e5cf6a727c248bdc3.
2020-03-06T01:56:30.4043117Z head hexdump of actual:
2020-03-06T01:56:30.4450251Z 0000000 ( 7 , 1 0 , 0 , S o
m e p a y
2020-03-06T01:56:30.4451524Z 0000010 l o a d . . . ) \n (
7 , 1 0 , 1
2020-03-06T01:56:30.4452512Z 0000020 , S o m e p a y l
o a d . . .
2020-03-06T01:56:30.4453117Z 0000030 ) \n ( 7 , 1 0 , 2 ,
S o m e p
2020-03-06T01:56:30.4453853Z 0000040 a y l o a d . . . )
\n ( 7 , 1 0
2020-03-06T01:56:30.4454578Z 0000050 , 3 , S o m e p a
y l o a d .
2020-03-06T01:56:30.4455398Z 0000060 . . ) \n ( 7 , 1 0 ,
4 , S o m e
2020-03-06T01:56:30.4455950Z 0000070 p a y l o a d . .
. ) \n ( 7 ,
2020-03-06T01:56:30.4456527Z 0000080 1 0 , 5 , S o m e
p a y l o a
2020-03-06T01:56:30.4457465Z 0000090 d . . . ) \n ( 7 , 1
0 , 6 , S o
2020-03-06T01:56:30.4458039Z 00000a0 m e p a y l o a d
. . . ) \n (
2020-03-06T01:56:30.4458682Z 00000b0 7 , 1 0 , 7 , S o m
e p a y l
2020-03-06T01:56:30.4459268Z 00000c0 o a d . . . ) \n ( 7
, 1 0 , 8 ,
2020-03-06T01:56:30.4459860Z 00000d0 S o m e p a y l o
a d . . . )
2020-03-06T01:56:30.4460453Z 00000e0 \n ( 7 , 1 0 , 9 , S
o m e p a
2020-03-06T01:56:30.4461576Z 00000f0 y l o a d . . . ) \n
2020-03-06T01:56:30.4462056Z 00000fa
2020-03-06T01:56:30.8279606Z Stopping taskexecutor daemon (pid: 48485) on host
fv-az678.
2020-03-06T01:56:31.0797555Z Stopping standalonesession daemon (pid: 45322) on
host fv-az678.
2020-03-06T01:56:31.6251323Z Stopping taskexecutor daemon (pid: 45652) on host
fv-az678.
2020-03-06T01:56:31.6269512Z Skipping taskexecutor daemon (pid: 46047), because
it is not running anymore on fv-az678.
2020-03-06T01:56:31.6270530Z Stopping taskexecutor daemon (pid: 46391) on host
fv-az678.
2020-03-06T01:56:31.6275038Z Skipping taskexecutor daemon (pid: 46743), because
it is not running anymore on fv-az678.
2020-03-06T01:56:31.6276149Z Skipping taskexecutor daemon (pid: 47637), because
it is not running anymore on fv-az678.
2020-03-06T01:56:31.6277074Z Stopping taskexecutor daemon (pid: 48161) on host
fv-az678.
2020-03-06T01:56:31.6308900Z [FAIL] Test script contains errors.
2020-03-06T01:56:31.6342805Z Checking of logs skipped.
2020-03-06T01:56:31.6343803Z
2020-03-06T01:56:31.6345373Z [FAIL] 'Streaming bucketing end-to-end test'
failed after 2 minutes and 51 seconds! Test exited with exit code 1
2020-03-06T01:56:31.6345950Z
2020-03-06T01:56:32.0156922Z No taskexecutor daemon to stop on host fv-az678.
2020-03-06T01:56:32.2517024Z No standalonesession daemon to stop on host
fv-az678.
2020-03-06T01:56:32.6989012Z
2020-03-06T01:56:32.7107754Z ##[error]Bash exited with code '1'.
2020-03-06T01:56:32.7150868Z ##[section]Finishing: Run e2e tests
{code}
> Streaming bucketing end-to-end test / test_streaming_bucketing.sh unstable
> --------------------------------------------------------------------------
>
> Key: FLINK-16227
> URL: https://issues.apache.org/jira/browse/FLINK-16227
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, Tests
> Affects Versions: 1.11.0
> Reporter: Robert Metzger
> Priority: Major
> Labels: test-stability
>
> This nightly cron job has failed:
> https://travis-ci.org/apache/flink/jobs/653454540
> {code}
> ==============================================================================
> Running 'Streaming bucketing end-to-end test'
> ==============================================================================
> TEST_DATA_DIR:
> /home/travis/build/apache/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-05739414867
> Flink dist directory:
> /home/travis/build/apache/flink/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT
> Setting up SSL with: internal JDK dynamic
> Using SAN
> dns:travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7,ip:10.20.0.145,ip:172.17.0.1
> Certificate was added to keystore
> Certificate was added to keystore
> Certificate reply was installed in keystore
> MAC verified OK
> Setting up SSL with: rest JDK dynamic
> Using SAN
> dns:travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7,ip:10.20.0.145,ip:172.17.0.1
> Certificate was added to keystore
> Certificate was added to keystore
> Certificate reply was installed in keystore
> MAC verified OK
> Mutual ssl auth: false
> Starting cluster.
> Starting standalonesession daemon on host
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Starting taskexecutor daemon on host
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Waiting for Dispatcher REST endpoint to come up...
> Waiting for Dispatcher REST endpoint to come up...
> Waiting for Dispatcher REST endpoint to come up...
> Waiting for Dispatcher REST endpoint to come up...
> Waiting for Dispatcher REST endpoint to come up...
> Dispatcher REST endpoint is up.
> [INFO] 1 instance(s) of taskexecutor are already running on
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Starting taskexecutor daemon on host
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> [INFO] 2 instance(s) of taskexecutor are already running on
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Starting taskexecutor daemon on host
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> [INFO] 3 instance(s) of taskexecutor are already running on
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Starting taskexecutor daemon on host
> travis-job-b9e26d64-0a62-42c7-9802-6c49defb4ad7.
> Number of running task managers 1 is not yet 4.
> Number of running task managers 2 is not yet 4.
> Number of running task managers has reached 4.
> java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
> at java.lang.Class.getDeclaredMethod(Class.java:2128)
> at
> org.apache.flink.api.java.ClosureCleaner.usesCustomSerialization(ClosureCleaner.java:164)
> at
> org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:89)
> at
> org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:71)
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1820)
> at
> org.apache.flink.streaming.api.datastream.DataStream.clean(DataStream.java:188)
> at
> org.apache.flink.streaming.api.datastream.DataStream.addSink(DataStream.java:1328)
> at
> org.apache.flink.streaming.tests.BucketingSinkTestProgram.main(BucketingSinkTestProgram.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
> at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
> at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.conf.Configuration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> at
> org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> ... 23 more
> Job () is running.
> Waiting for job () to have at least 5 completed checkpoints ...
> No output has been received in the last 10m0s, this potentially indicates a
> stalled build or something wrong with the build itself.
> Check the details on how to adjust your build configuration on:
> https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
> The build has been terminated
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
