Quick update on this effort: Since yesterday, I'm experimenting with
running the end to end tests with each pull request and "master" push.
I hope that this helps to uncover issues earlier (without waiting for the
nightly test execution)
The tests run for almost 3 hours, so the overall build
@Leonard: On Azure, I'm not splitting the execution of the end to end tests
anymore. We won't have the overhead of compiling the same profile multiple
times anymore.
@all: We have recently merged a first version of the Azure configuration
files to Flink [1]. This will allow us to build pull
+1 for the migration.
*10 parallel builds with 300 minute timeouts * is very useful for tasks that
takes long time like e2e tests.
And in Travis, looks like we compile entire project for every cron task even if
they use same profile, eg:
`name: e2e - misc - hadoop 2.8
name: e2e
+1 for migrating to Azure pipelines as this can have shorter build time,
and faster response.
Best,
Congxian
Xiyuan Wang 于2019年12月9日周一 上午10:13写道：
> Hi Robert,
> Thanks for bring up this topic. The 2 ARM machines(16cores) which I
> donated is just for POC test. We(Huawei) can donate more
Hi Robert,
Thanks for bring up this topic. The 2 ARM machines(16cores) which I
donated is just for POC test. We(Huawei) can donate more once moving to
official Azure pipeline. :)
Robert Metzger 于2019年12月6日周五 上午3:25写道：
> Thanks for your comments Yun.
> If there's strong support for idea 2, it
Thanks for your comments Yun.
If there's strong support for idea 2, it would actually make my
life easier: the migration would be easier to do.
I also noticed that the uploads to transfer.sh were broken, but this should
be fixed in the "rmetzger.flink" builds (coming from rmetzger/flink). The
Hi Robert
Really exciting to see this new more powerful CI tool to get rid of the 50
minutes limit of traivs-CI free account.
After reading the wiki, I support idea 2 of AZP-setup version-2.
However, after I dig into some failing builds at
https://dev.azure.com/rmetzger/Flink/_build , I
I've created a first draft of my plans in the wiki:
https://cwiki.apache.org/confluence/display/FLINK/%5Bpreview%5D+Azure+Pipelines.
I'm looking forward to your comments.
On Thu, Dec 5, 2019 at 12:37 PM Robert Metzger wrote:
> Thank you all for the positive feedback. I will start putting
Thank you all for the positive feedback. I will start putting together a
page in the wiki.
@Jark: Azure Pipelines provides a free services, that is even better than
what Travis provides for free: 10 parallel builds with 6 hours timeouts.
@Chesnay: I will answer your questions in the
+1 I had good experiences with Azure pipelines in the past.
On Thu, Dec 5, 2019 at 11:35 AM Aljoscha Krettek
wrote:
> +1
>
> Thanks for the effort! The tooling seems to be quite a bit nicer and I
> like that we can grow by adding more machines.
>
> Best,
> Aljoscha
>
> > On 5. Dec 2019, at
+1
Thanks for the effort! The tooling seems to be quite a bit nicer and I like
that we can grow by adding more machines.
Best,
Aljoscha
> On 5. Dec 2019, at 03:18, Jark Wu wrote:
>
> +1 for Azure pipeline because it promises better performance.
>
> However, I have 2 concerns:
>
> 1) Travis
+1 for Azure pipeline because it promises better performance.
However, I have 2 concerns:
1) Travis provides personal free service for testing personal branches.
Usually, contributors use this feature to test PoC or run CRON jobs for
pull requests.
Using local machine will cost a lot of
+1
Till Rohrmann 于2019年12月4日周三 下午10:43写道：
> +1 for moving to Azure pipelines as it promises better scalability and
> tooling. Looking forward to having faster builds and hence shorter feedback
> cycles :-)
>
> Cheers,
> Till
>
> On Wed, Dec 4, 2019 at 1:24 PM Chesnay Schepler
> wrote:
>
> >
+1 for moving to Azure pipelines as it promises better scalability and
tooling. Looking forward to having faster builds and hence shorter feedback
cycles :-)
Cheers,
Till
On Wed, Dec 4, 2019 at 1:24 PM Chesnay Schepler wrote:
> @robert Can you expand how the azure setup interacts with CiBot?
@robert Can you expand how the azure setup interacts with CiBot? Do we
have to continue mirroring builds into flink-ci? How will the cronjob
configuration work? We should have a general idea on how to implement
this before proceeding.
Additionally, moving /all /jobs into flink-ci requires
From what I've seen so far Azure will provide us a better experience,
so I'd say +1 for the transition as a whole.
I'd delay merge at least until the feature branch is cut.
Given the parental leave it may even make sense to only start merging in
January afterwards, to reduce the total time
Thanks Robert for driving this. There is another big pain point of current
travis,
which is its cache mechanism will fail from time to time. Almost around 50%
of
the build fails are caused by cache problem. I opened this issue to travis
but
got no response yet. So big +1 from my side.
Just one
Thanks Robert for the updates! And thanks a lot for all the efforts to
investigate, experiment and tune Azure Pipelines for Flink building.
Big +1 for it.
It would be great that the community building can be extended with custom
machines so that the tests would not be queued for long with daily
Hi all,
as a follow up from our discussion on reducing the build time [1], I would
like to propose migrating our build infrastructure to Azure Pipelines (away
from Travis).
I believe that we have reached the limits of what Travis can provide the
Flink community, and I don't want the build system
19 matches
Mail list logo
