[
https://issues.apache.org/jira/browse/BEAM-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432144#comment-16432144
]
Kamil Szewczyk commented on BEAM-4041:
--------------------------------------
[~alanmyrvold] [~chamikara] [~jasonkuster] [~ŁukaszG]
I have also took a look at that issue and noticed that we spin our
prerformance tests each six hours, so that many jobs try to run at the same
time. It's possible to easily run them each six hours, but with little time
shift ( like +15 min), by modifying cron job settings. This could probably
help, but I couldn't believe that providing just a few external IPs more at
single point of time is an issue for GCE. It's just my guess that maybe it
could help.
> Performance tests fail due to kubernetes load balancer problems
> ---------------------------------------------------------------
>
> Key: BEAM-4041
> URL: https://issues.apache.org/jira/browse/BEAM-4041
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Łukasz Gajowy
> Assignee: Jason Kuster
> Priority: Major
>
> Recently, as we added more IOITs to be run on jenkins using kubernetes, some
> of them started to fail randomly, because they couldn't retrieve LoadBalancer
> address. Normally obtaining the address took about one minute. Perfkit waits
> for the address (actively checking for it) for 3 minutes. This should be
> enough for getting the address, yet it recently started to exceed the 3
> minutes limit. I also noticed that this error didn't happen when there were
> fewer tests.
> Example logs:
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/31/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
