Re: [sympy] Re: Azure pipelines

[Nathan Goldbaum]

Yes. For example, here’s how to do it with codecov:

https://docs.codecov.io/docs/merging-reports


On Thu, Jan 31, 2019 at 12:07 PM Aaron Meurer  wrote:

> Code coverage requires the tests to run. So if we are going to do it, we
> have to do it where the tests are run anyway (meaning it might as well be
> on Travis). Can code coverage easily be aggregated across multiple builds,
> since we have the tests split up?
>
> Aaron Meurer
>
> On Thu, Jan 31, 2019 at 9:47 AM Jason Moore  wrote:
>
>> I think that the history on code coverage is that people have attempted
>> to get it fully working but we've never completed a PR on it. I'm not sure
>> the increased time it might take was a show stopper.
>>
>> Jason
>> moorepants.info
>> +01 530-601-9791
>>
>>
>> On Thu, Jan 31, 2019 at 8:09 AM Oscar Gustafsson <
>> oscar.gustafs...@gmail.com> wrote:
>>
>>> What about code coverage? If I understand it correctly the tests take to
>>> long on Travis to include code coverage (just an impression I got while
>>> browsing Issues, no idea how it works in practice). Maybe there is
>>> something nifty built in on Azure Pipelines that can be used for that? Or
>>> if it just a matter of CPU time, maybe one can run the code coverage tests
>>> on Azure and the other tests on Travis? I really would like if it was
>>> possible to get a codecov.io or something up and running.
>>>
>>> BR Oscar
>>>
>>> Den onsdag 30 januari 2019 kl. 19:35:49 UTC+1 skrev Aaron Meurer:



 On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  wrote:

> Thanks Isuru. If you see any issues again, I'd like to dig into it.
> Could have been transient, or could be a browser-specific problem we need
> to address.
>
> Also, I'm interested to know more about the complexity of our YAML
> syntax. (I don't disagree, I'm mainly curious what specifically you 
> tripped
> over.)
>

 You can see from https://github.com/sympy/sympy/pull/15297/commits how
 I struggled with the syntax, for instance, trying to figure out how to make
 the conditions work.

 My biggest gripe is that the documentation doesn't have a real search
 feature. There are two search boxes on
 https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
 One searches only the titles, and the other searches all of Azure DevOps. I
 ended up downloading the PDF documentation and using that because it was
 the only way I could get a reasonable search. Even then, I had a hard time
 with the documentation because of lack of concrete examples. I remember
 searching for a long time until I figured out how to make a build an
 allowed failure.

 My biggest hurdle was working with environment variables, and the fact
 that they are not persisted across scripts. This means that things like
 source activate for conda/virtualenv do not work. A lot of time was wasted
 from me not knowing this fact.

 Aaron Meurer


> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando
> wrote:
>>
>> The azure build log link didn't work for me when Aaron sent it 2 days
>> ago and now it opens at the correct line just fine.
>>
>> Isuru
>>
>> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper 
>> wrote:
>>
>>> (Azure Pipelines PM chiming in.)
>>>
>>> Aaron, can you send me a screenshot what you see when you click that
>>> log link? For me it opens directly to the line, as it's intended to do. 
>>> I'm
>>> using Firefox, and most of the team uses Chrome - it's possible we have 
>>> a
>>> bug though.
>>>
>>> With no additional clicks, here's what I see:
>>>
>>> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:

 The question is if its worth it, given the other issues Azure has.
 Keep in mind that we have to maintain whatever we set up.

 For benchmarks, I don't know if any public CI is appropriate. You
 really need a dedicated machine so you can get consistent runtimes.
 Code quality tests are done as part of the test suite already
 (test_code_quality.py).

 I'm not opposed to the idea of using some other CI for other tests.
 I
 think it should be driven by a real need, though, not just saying
 we
 should use the resource because it is there. There are other
 options
 too, such as CircleCI and AppVeyor.

 Aaron Meurer

 On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee 
 wrote:
 >
 > There had been some issues with azure pipelines with conda
 environment setups, and I would also agree that some features are not
 better than travis.
 >
 > However, I have a question that, if they are offering free
 resources for open source projects,
 > 

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

If someone wants to add codecov or some other coverage reporting I can do
whatever steps are needed that require admin access (I don't have time to
do the full setup).

Aaron Meurer

On Thu, Jan 31, 2019 at 11:11 AM Nathan Goldbaum 
wrote:

> Yes. For example, here’s how to do it with codecov:
>
> https://docs.codecov.io/docs/merging-reports
>
>
> On Thu, Jan 31, 2019 at 12:07 PM Aaron Meurer  wrote:
>
>> Code coverage requires the tests to run. So if we are going to do it, we
>> have to do it where the tests are run anyway (meaning it might as well be
>> on Travis). Can code coverage easily be aggregated across multiple builds,
>> since we have the tests split up?
>>
>> Aaron Meurer
>>
>> On Thu, Jan 31, 2019 at 9:47 AM Jason Moore  wrote:
>>
>>> I think that the history on code coverage is that people have attempted
>>> to get it fully working but we've never completed a PR on it. I'm not sure
>>> the increased time it might take was a show stopper.
>>>
>>> Jason
>>> moorepants.info
>>> +01 530-601-9791
>>>
>>>
>>> On Thu, Jan 31, 2019 at 8:09 AM Oscar Gustafsson <
>>> oscar.gustafs...@gmail.com> wrote:
>>>
 What about code coverage? If I understand it correctly the tests take
 to long on Travis to include code coverage (just an impression I got while
 browsing Issues, no idea how it works in practice). Maybe there is
 something nifty built in on Azure Pipelines that can be used for that? Or
 if it just a matter of CPU time, maybe one can run the code coverage tests
 on Azure and the other tests on Travis? I really would like if it was
 possible to get a codecov.io or something up and running.

 BR Oscar

 Den onsdag 30 januari 2019 kl. 19:35:49 UTC+1 skrev Aaron Meurer:
>
>
>
> On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper 
> wrote:
>
>> Thanks Isuru. If you see any issues again, I'd like to dig into it.
>> Could have been transient, or could be a browser-specific problem we need
>> to address.
>>
>> Also, I'm interested to know more about the complexity of our YAML
>> syntax. (I don't disagree, I'm mainly curious what specifically you 
>> tripped
>> over.)
>>
>
> You can see from https://github.com/sympy/sympy/pull/15297/commits
> how I struggled with the syntax, for instance, trying to figure out how to
> make the conditions work.
>
> My biggest gripe is that the documentation doesn't have a real search
> feature. There are two search boxes on
> https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
> One searches only the titles, and the other searches all of Azure DevOps. 
> I
> ended up downloading the PDF documentation and using that because it was
> the only way I could get a reasonable search. Even then, I had a hard time
> with the documentation because of lack of concrete examples. I remember
> searching for a long time until I figured out how to make a build an
> allowed failure.
>
> My biggest hurdle was working with environment variables, and the fact
> that they are not persisted across scripts. This means that things like
> source activate for conda/virtualenv do not work. A lot of time was wasted
> from me not knowing this fact.
>
> Aaron Meurer
>
>
>> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando
>> wrote:
>>>
>>> The azure build log link didn't work for me when Aaron sent it 2
>>> days ago and now it opens at the correct line just fine.
>>>
>>> Isuru
>>>
>>> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper 
>>> wrote:
>>>
 (Azure Pipelines PM chiming in.)

 Aaron, can you send me a screenshot what you see when you click
 that log link? For me it opens directly to the line, as it's intended 
 to
 do. I'm using Firefox, and most of the team uses Chrome - it's 
 possible we
 have a bug though.

 With no additional clicks, here's what I see:

 On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer
 wrote:
>
> The question is if its worth it, given the other issues Azure has.
> Keep in mind that we have to maintain whatever we set up.
>
> For benchmarks, I don't know if any public CI is appropriate. You
> really need a dedicated machine so you can get consistent
> runtimes.
> Code quality tests are done as part of the test suite already
> (test_code_quality.py).
>
> I'm not opposed to the idea of using some other CI for other
> tests. I
> think it should be driven by a real need, though, not just saying
> we
> should use the resource because it is there. There are other
> options
> too, such as CircleCI and AppVeyor.
>
> Aaron Meurer
>

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

Code coverage requires the tests to run. So if we are going to do it, we
have to do it where the tests are run anyway (meaning it might as well be
on Travis). Can code coverage easily be aggregated across multiple builds,
since we have the tests split up?

Aaron Meurer

On Thu, Jan 31, 2019 at 9:47 AM Jason Moore  wrote:

> I think that the history on code coverage is that people have attempted to
> get it fully working but we've never completed a PR on it. I'm not sure the
> increased time it might take was a show stopper.
>
> Jason
> moorepants.info
> +01 530-601-9791
>
>
> On Thu, Jan 31, 2019 at 8:09 AM Oscar Gustafsson <
> oscar.gustafs...@gmail.com> wrote:
>
>> What about code coverage? If I understand it correctly the tests take to
>> long on Travis to include code coverage (just an impression I got while
>> browsing Issues, no idea how it works in practice). Maybe there is
>> something nifty built in on Azure Pipelines that can be used for that? Or
>> if it just a matter of CPU time, maybe one can run the code coverage tests
>> on Azure and the other tests on Travis? I really would like if it was
>> possible to get a codecov.io or something up and running.
>>
>> BR Oscar
>>
>> Den onsdag 30 januari 2019 kl. 19:35:49 UTC+1 skrev Aaron Meurer:
>>>
>>>
>>>
>>> On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  wrote:
>>>
 Thanks Isuru. If you see any issues again, I'd like to dig into it.
 Could have been transient, or could be a browser-specific problem we need
 to address.

 Also, I'm interested to know more about the complexity of our YAML
 syntax. (I don't disagree, I'm mainly curious what specifically you tripped
 over.)

>>>
>>> You can see from https://github.com/sympy/sympy/pull/15297/commits how
>>> I struggled with the syntax, for instance, trying to figure out how to make
>>> the conditions work.
>>>
>>> My biggest gripe is that the documentation doesn't have a real search
>>> feature. There are two search boxes on
>>> https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
>>> One searches only the titles, and the other searches all of Azure DevOps. I
>>> ended up downloading the PDF documentation and using that because it was
>>> the only way I could get a reasonable search. Even then, I had a hard time
>>> with the documentation because of lack of concrete examples. I remember
>>> searching for a long time until I figured out how to make a build an
>>> allowed failure.
>>>
>>> My biggest hurdle was working with environment variables, and the fact
>>> that they are not persisted across scripts. This means that things like
>>> source activate for conda/virtualenv do not work. A lot of time was wasted
>>> from me not knowing this fact.
>>>
>>> Aaron Meurer
>>>
>>>
 On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando
 wrote:
>
> The azure build log link didn't work for me when Aaron sent it 2 days
> ago and now it opens at the correct line just fine.
>
> Isuru
>
> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper 
> wrote:
>
>> (Azure Pipelines PM chiming in.)
>>
>> Aaron, can you send me a screenshot what you see when you click that
>> log link? For me it opens directly to the line, as it's intended to do. 
>> I'm
>> using Firefox, and most of the team uses Chrome - it's possible we have a
>> bug though.
>>
>> With no additional clicks, here's what I see:
>>
>> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:
>>>
>>> The question is if its worth it, given the other issues Azure has.
>>> Keep in mind that we have to maintain whatever we set up.
>>>
>>> For benchmarks, I don't know if any public CI is appropriate. You
>>> really need a dedicated machine so you can get consistent runtimes.
>>> Code quality tests are done as part of the test suite already
>>> (test_code_quality.py).
>>>
>>> I'm not opposed to the idea of using some other CI for other tests.
>>> I
>>> think it should be driven by a real need, though, not just saying we
>>> should use the resource because it is there. There are other options
>>> too, such as CircleCI and AppVeyor.
>>>
>>> Aaron Meurer
>>>
>>> On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote:
>>> >
>>> > There had been some issues with azure pipelines with conda
>>> environment setups, and I would also agree that some features are not
>>> better than travis.
>>> >
>>> > However, I have a question that, if they are offering free
>>> resources for open source projects,
>>> > we may allocate some of the tests to azure, which can either
>>> reduce overall build time,
>>> > or making some space for running benchmarks or code quality tests.
>>> >
>>> > Would there be any reason making this idea naive?
>>> >
>>> > --
>>> > You received this message because you are 

Re: [sympy] Re: Azure pipelines

[Jason Moore]

I think that the history on code coverage is that people have attempted to
get it fully working but we've never completed a PR on it. I'm not sure the
increased time it might take was a show stopper.

Jason
moorepants.info
+01 530-601-9791


On Thu, Jan 31, 2019 at 8:09 AM Oscar Gustafsson 
wrote:

> What about code coverage? If I understand it correctly the tests take to
> long on Travis to include code coverage (just an impression I got while
> browsing Issues, no idea how it works in practice). Maybe there is
> something nifty built in on Azure Pipelines that can be used for that? Or
> if it just a matter of CPU time, maybe one can run the code coverage tests
> on Azure and the other tests on Travis? I really would like if it was
> possible to get a codecov.io or something up and running.
>
> BR Oscar
>
> Den onsdag 30 januari 2019 kl. 19:35:49 UTC+1 skrev Aaron Meurer:
>>
>>
>>
>> On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  wrote:
>>
>>> Thanks Isuru. If you see any issues again, I'd like to dig into it.
>>> Could have been transient, or could be a browser-specific problem we need
>>> to address.
>>>
>>> Also, I'm interested to know more about the complexity of our YAML
>>> syntax. (I don't disagree, I'm mainly curious what specifically you tripped
>>> over.)
>>>
>>
>> You can see from https://github.com/sympy/sympy/pull/15297/commits how I
>> struggled with the syntax, for instance, trying to figure out how to make
>> the conditions work.
>>
>> My biggest gripe is that the documentation doesn't have a real search
>> feature. There are two search boxes on
>> https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
>> One searches only the titles, and the other searches all of Azure DevOps. I
>> ended up downloading the PDF documentation and using that because it was
>> the only way I could get a reasonable search. Even then, I had a hard time
>> with the documentation because of lack of concrete examples. I remember
>> searching for a long time until I figured out how to make a build an
>> allowed failure.
>>
>> My biggest hurdle was working with environment variables, and the fact
>> that they are not persisted across scripts. This means that things like
>> source activate for conda/virtualenv do not work. A lot of time was wasted
>> from me not knowing this fact.
>>
>> Aaron Meurer
>>
>>
>>> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando
>>> wrote:

 The azure build log link didn't work for me when Aaron sent it 2 days
 ago and now it opens at the correct line just fine.

 Isuru

 On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper  wrote:

> (Azure Pipelines PM chiming in.)
>
> Aaron, can you send me a screenshot what you see when you click that
> log link? For me it opens directly to the line, as it's intended to do. 
> I'm
> using Firefox, and most of the team uses Chrome - it's possible we have a
> bug though.
>
> With no additional clicks, here's what I see:
>
> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:
>>
>> The question is if its worth it, given the other issues Azure has.
>> Keep in mind that we have to maintain whatever we set up.
>>
>> For benchmarks, I don't know if any public CI is appropriate. You
>> really need a dedicated machine so you can get consistent runtimes.
>> Code quality tests are done as part of the test suite already
>> (test_code_quality.py).
>>
>> I'm not opposed to the idea of using some other CI for other tests. I
>> think it should be driven by a real need, though, not just saying we
>> should use the resource because it is there. There are other options
>> too, such as CircleCI and AppVeyor.
>>
>> Aaron Meurer
>>
>> On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote:
>> >
>> > There had been some issues with azure pipelines with conda
>> environment setups, and I would also agree that some features are not
>> better than travis.
>> >
>> > However, I have a question that, if they are offering free
>> resources for open source projects,
>> > we may allocate some of the tests to azure, which can either reduce
>> overall build time,
>> > or making some space for running benchmarks or code quality tests.
>> >
>> > Would there be any reason making this idea naive?
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "sympy" group.
>> > To unsubscribe from this group and stop receiving emails from it,
>> send an email to sympy+un...@googlegroups.com.
>> > To post to this group, send email to sy...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sympy.
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.
>>
>> 

Re: [sympy] Re: Azure pipelines

[Oscar Gustafsson]

What about code coverage? If I understand it correctly the tests take to 
long on Travis to include code coverage (just an impression I got while 
browsing Issues, no idea how it works in practice). Maybe there is 
something nifty built in on Azure Pipelines that can be used for that? Or 
if it just a matter of CPU time, maybe one can run the code coverage tests 
on Azure and the other tests on Travis? I really would like if it was 
possible to get a codecov.io or something up and running.

BR Oscar

Den onsdag 30 januari 2019 kl. 19:35:49 UTC+1 skrev Aaron Meurer:
>
>
>
> On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  > wrote:
>
>> Thanks Isuru. If you see any issues again, I'd like to dig into it. Could 
>> have been transient, or could be a browser-specific problem we need to 
>> address.
>>
>> Also, I'm interested to know more about the complexity of our YAML 
>> syntax. (I don't disagree, I'm mainly curious what specifically you tripped 
>> over.)
>>
>
> You can see from https://github.com/sympy/sympy/pull/15297/commits how I 
> struggled with the syntax, for instance, trying to figure out how to make 
> the conditions work. 
>
> My biggest gripe is that the documentation doesn't have a real search 
> feature. There are two search boxes on 
> https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
>  
> One searches only the titles, and the other searches all of Azure DevOps. I 
> ended up downloading the PDF documentation and using that because it was 
> the only way I could get a reasonable search. Even then, I had a hard time 
> with the documentation because of lack of concrete examples. I remember 
> searching for a long time until I figured out how to make a build an 
> allowed failure. 
>
> My biggest hurdle was working with environment variables, and the fact 
> that they are not persisted across scripts. This means that things like 
> source activate for conda/virtualenv do not work. A lot of time was wasted 
> from me not knowing this fact. 
>
> Aaron Meurer
>
>
>> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando wrote:
>>>
>>> The azure build log link didn't work for me when Aaron sent it 2 days 
>>> ago and now it opens at the correct line just fine.
>>>
>>> Isuru
>>>
>>> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper  wrote:
>>>
 (Azure Pipelines PM chiming in.)

 Aaron, can you send me a screenshot what you see when you click that 
 log link? For me it opens directly to the line, as it's intended to do. 
 I'm 
 using Firefox, and most of the team uses Chrome - it's possible we have a 
 bug though.

 With no additional clicks, here's what I see:

 On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:
>
> The question is if its worth it, given the other issues Azure has. 
> Keep in mind that we have to maintain whatever we set up. 
>
> For benchmarks, I don't know if any public CI is appropriate. You 
> really need a dedicated machine so you can get consistent runtimes. 
> Code quality tests are done as part of the test suite already 
> (test_code_quality.py). 
>
> I'm not opposed to the idea of using some other CI for other tests. I 
> think it should be driven by a real need, though, not just saying we 
> should use the resource because it is there. There are other options 
> too, such as CircleCI and AppVeyor. 
>
> Aaron Meurer 
>
> On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote: 
> > 
> > There had been some issues with azure pipelines with conda 
> environment setups, and I would also agree that some features are not 
> better than travis. 
> > 
> > However, I have a question that, if they are offering free resources 
> for open source projects, 
> > we may allocate some of the tests to azure, which can either reduce 
> overall build time, 
> > or making some space for running benchmarks or code quality tests. 
> > 
> > Would there be any reason making this idea naive? 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "sympy" group. 
> > To unsubscribe from this group and stop receiving emails from it, 
> send an email to sympy+un...@googlegroups.com. 
> > To post to this group, send email to sy...@googlegroups.com. 
> > Visit this group at https://groups.google.com/group/sympy. 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>
 -- 
 You received this message because you are subscribed to the Google 
 Groups "sympy" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to sympy+un...@googlegroups.com.
 To post to this group, send email to 

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  wrote:

> Thanks Isuru. If you see any issues again, I'd like to dig into it. Could
> have been transient, or could be a browser-specific problem we need to
> address.
>
> Also, I'm interested to know more about the complexity of our YAML syntax.
> (I don't disagree, I'm mainly curious what specifically you tripped over.)
>

You can see from https://github.com/sympy/sympy/pull/15297/commits how I
struggled with the syntax, for instance, trying to figure out how to make
the conditions work.

My biggest gripe is that the documentation doesn't have a real search
feature. There are two search boxes on
https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/index?view=azdevops.
One searches only the titles, and the other searches all of Azure DevOps. I
ended up downloading the PDF documentation and using that because it was
the only way I could get a reasonable search. Even then, I had a hard time
with the documentation because of lack of concrete examples. I remember
searching for a long time until I figured out how to make a build an
allowed failure.

My biggest hurdle was working with environment variables, and the fact that
they are not persisted across scripts. This means that things like source
activate for conda/virtualenv do not work. A lot of time was wasted from me
not knowing this fact.

Aaron Meurer


> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando wrote:
>>
>> The azure build log link didn't work for me when Aaron sent it 2 days ago
>> and now it opens at the correct line just fine.
>>
>> Isuru
>>
>> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper  wrote:
>>
>>> (Azure Pipelines PM chiming in.)
>>>
>>> Aaron, can you send me a screenshot what you see when you click that log
>>> link? For me it opens directly to the line, as it's intended to do. I'm
>>> using Firefox, and most of the team uses Chrome - it's possible we have a
>>> bug though.
>>>
>>> With no additional clicks, here's what I see:
>>>
>>> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:

 The question is if its worth it, given the other issues Azure has.
 Keep in mind that we have to maintain whatever we set up.

 For benchmarks, I don't know if any public CI is appropriate. You
 really need a dedicated machine so you can get consistent runtimes.
 Code quality tests are done as part of the test suite already
 (test_code_quality.py).

 I'm not opposed to the idea of using some other CI for other tests. I
 think it should be driven by a real need, though, not just saying we
 should use the resource because it is there. There are other options
 too, such as CircleCI and AppVeyor.

 Aaron Meurer

 On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote:
 >
 > There had been some issues with azure pipelines with conda
 environment setups, and I would also agree that some features are not
 better than travis.
 >
 > However, I have a question that, if they are offering free resources
 for open source projects,
 > we may allocate some of the tests to azure, which can either reduce
 overall build time,
 > or making some space for running benchmarks or code quality tests.
 >
 > Would there be any reason making this idea naive?
 >
 > --
 > You received this message because you are subscribed to the Google
 Groups "sympy" group.
 > To unsubscribe from this group and stop receiving emails from it,
 send an email to sympy+un...@googlegroups.com.
 > To post to this group, send email to sy...@googlegroups.com.
 > Visit this group at https://groups.google.com/group/sympy.
 > To view this discussion on the web visit
 https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.

 > For more options, visit https://groups.google.com/d/optout.

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "sympy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to sympy+un...@googlegroups.com.
>>> To post to this group, send email to sy...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/sympy.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/sympy/702492e8-8233-417a-aeba-a9cde5aeb668%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at 

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

Perhaps the link failed because the tests were still running when I tested
it.

Aaron Meurer

On Wed, Jan 30, 2019 at 8:32 AM Matt Cooper  wrote:

> Thanks Isuru. If you see any issues again, I'd like to dig into it. Could
> have been transient, or could be a browser-specific problem we need to
> address.
>
> Also, I'm interested to know more about the complexity of our YAML syntax.
> (I don't disagree, I'm mainly curious what specifically you tripped over.)
>
> On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando wrote:
>>
>> The azure build log link didn't work for me when Aaron sent it 2 days ago
>> and now it opens at the correct line just fine.
>>
>> Isuru
>>
>> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper  wrote:
>>
> (Azure Pipelines PM chiming in.)
>>>
>>> Aaron, can you send me a screenshot what you see when you click that log
>>> link? For me it opens directly to the line, as it's intended to do. I'm
>>> using Firefox, and most of the team uses Chrome - it's possible we have a
>>> bug though.
>>>
>>> With no additional clicks, here's what I see:
>>>
>>> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:

 The question is if its worth it, given the other issues Azure has.
 Keep in mind that we have to maintain whatever we set up.

 For benchmarks, I don't know if any public CI is appropriate. You
 really need a dedicated machine so you can get consistent runtimes.
 Code quality tests are done as part of the test suite already
 (test_code_quality.py).

 I'm not opposed to the idea of using some other CI for other tests. I
 think it should be driven by a real need, though, not just saying we
 should use the resource because it is there. There are other options
 too, such as CircleCI and AppVeyor.

 Aaron Meurer

 On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote:
 >
 > There had been some issues with azure pipelines with conda
 environment setups, and I would also agree that some features are not
 better than travis.
 >
 > However, I have a question that, if they are offering free resources
 for open source projects,
 > we may allocate some of the tests to azure, which can either reduce
 overall build time,
 > or making some space for running benchmarks or code quality tests.
 >
 > Would there be any reason making this idea naive?
 >
 > --
 > You received this message because you are subscribed to the Google
 Groups "sympy" group.
 > To unsubscribe from this group and stop receiving emails from it,
 send an email to sympy+un...@googlegroups.com.
 > To post to this group, send email to sy...@googlegroups.com.
 > Visit this group at https://groups.google.com/group/sympy.
 > To view this discussion on the web visit
 https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.

 > For more options, visit https://groups.google.com/d/optout.

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "sympy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to sympy+un...@googlegroups.com.
>>> To post to this group, send email to sy...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/sympy.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/sympy/702492e8-8233-417a-aeba-a9cde5aeb668%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/7b98fac0-f9c8-413d-9885-c3eb33aafc25%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2BqKs4yZWM0yT-v_Mn23OGwYa%2BjkjyDxUTbRO2W9aXmyw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Matt Cooper]

Thanks Isuru. If you see any issues again, I'd like to dig into it. Could 
have been transient, or could be a browser-specific problem we need to 
address.

Also, I'm interested to know more about the complexity of our YAML syntax. 
(I don't disagree, I'm mainly curious what specifically you tripped over.)

On Wednesday, January 30, 2019 at 10:17:15 AM UTC-5, Isuru Fernando wrote:
>
> The azure build log link didn't work for me when Aaron sent it 2 days ago 
> and now it opens at the correct line just fine.
>
> Isuru
>
> On Wed, Jan 30, 2019 at 7:34 AM Matt Cooper  > wrote:
>
>> (Azure Pipelines PM chiming in.)
>>
>> Aaron, can you send me a screenshot what you see when you click that log 
>> link? For me it opens directly to the line, as it's intended to do. I'm 
>> using Firefox, and most of the team uses Chrome - it's possible we have a 
>> bug though.
>>
>> With no additional clicks, here's what I see:
>>
>> On Tuesday, January 29, 2019 at 1:59:25 PM UTC-5, Aaron Meurer wrote:
>>>
>>> The question is if its worth it, given the other issues Azure has. 
>>> Keep in mind that we have to maintain whatever we set up. 
>>>
>>> For benchmarks, I don't know if any public CI is appropriate. You 
>>> really need a dedicated machine so you can get consistent runtimes. 
>>> Code quality tests are done as part of the test suite already 
>>> (test_code_quality.py). 
>>>
>>> I'm not opposed to the idea of using some other CI for other tests. I 
>>> think it should be driven by a real need, though, not just saying we 
>>> should use the resource because it is there. There are other options 
>>> too, such as CircleCI and AppVeyor. 
>>>
>>> Aaron Meurer 
>>>
>>> On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote: 
>>> > 
>>> > There had been some issues with azure pipelines with conda environment 
>>> setups, and I would also agree that some features are not better than 
>>> travis. 
>>> > 
>>> > However, I have a question that, if they are offering free resources 
>>> for open source projects, 
>>> > we may allocate some of the tests to azure, which can either reduce 
>>> overall build time, 
>>> > or making some space for running benchmarks or code quality tests. 
>>> > 
>>> > Would there be any reason making this idea naive? 
>>> > 
>>> > -- 
>>> > You received this message because you are subscribed to the Google 
>>> Groups "sympy" group. 
>>> > To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to sympy+un...@googlegroups.com. 
>>> > To post to this group, send email to sy...@googlegroups.com. 
>>> > Visit this group at https://groups.google.com/group/sympy. 
>>> > To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.
>>>  
>>>
>>> > For more options, visit https://groups.google.com/d/optout. 
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "sympy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sympy+un...@googlegroups.com .
>> To post to this group, send email to sy...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/sympy.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/sympy/702492e8-8233-417a-aeba-a9cde5aeb668%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/7b98fac0-f9c8-413d-9885-c3eb33aafc25%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sympy] Re: Azure pipelines

[Sidhant Nagpal]

I found Azure to be more resilient to timeout errors, but I also
haven't seen such errors in Travis lately. At the same time, there
have been some instances where the Azure tests have failed to
start upon opening of Pull Requests.

I also prefer to view logs on Travis whenever a PR fails a CI, as
their UI is simpler and easier to browse. I agree that Azure feels 
more convoluted and rightly difficult to maintain, reading your account.

One feature on Azure, which could be useful, is that it allows
publishing test analytics. I remember seeing an issue for this:
https://github.com/sympy/sympy/issues/15526

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/27acfec0-f1a0-4bf5-9957-3f16f642e244%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

The question is if its worth it, given the other issues Azure has.
Keep in mind that we have to maintain whatever we set up.

For benchmarks, I don't know if any public CI is appropriate. You
really need a dedicated machine so you can get consistent runtimes.
Code quality tests are done as part of the test suite already
(test_code_quality.py).

I'm not opposed to the idea of using some other CI for other tests. I
think it should be driven by a real need, though, not just saying we
should use the resource because it is there. There are other options
too, such as CircleCI and AppVeyor.

Aaron Meurer

On Tue, Jan 29, 2019 at 3:09 AM S.Y. Lee  wrote:
>
> There had been some issues with azure pipelines with conda environment 
> setups, and I would also agree that some features are not better than travis.
>
> However, I have a question that, if they are offering free resources for open 
> source projects,
> we may allocate some of the tests to azure, which can either reduce overall 
> build time,
> or making some space for running benchmarks or code quality tests.
>
> Would there be any reason making this idea naive?
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6LLL%2BG%2BCq%3D1Fr7YidbfQsA-wRwPOPeAA%3DAu4tdOZCTxAg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[sympy] Re: Azure pipelines

[S.Y. Lee]

There had been some issues with azure pipelines with conda environment 
setups, and I would also agree that some features are not better than 
travis.

However, I have a question that, if they are offering free resources for 
open source projects,
we may allocate some of the tests to azure, which can either reduce overall 
build time,
or making some space for running benchmarks or code quality tests.

Would there be any reason making this idea naive?

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/3bd7d189-b67e-457f-ac3b-994d3c5deda0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

You can view Azure Pipelines logs in the browser if you click the name
of the job (like here if you click "doctests"
https://dev.azure.com/SymPy/SymPy/_build/results?buildId=1396).

However, that reminds me about another annoyance of Azure, which is
that you can't link to a specific script or job log. You can only link
to the log for the whole build. Actually I discovered just now that if
you click a line number on a log it offers to create a link to that
line, but I tested it and it doesn't seem to actually work (it creates
a huge url like
https://dev.azure.com/SymPy/SymPy/_build/results?buildId=1396=logs=60d3e802-b22a-5dd9-4fed-c37ef5e6d2bc=f72b1324-e456-5562-0912-687e803699f2=33=34=1=1
that doesn't actually open the log at that point).

On Travis you just click a line number and it anchors to that line,
making a simple link like
https://travis-ci.org/sympy/sympy/jobs/485626173#L1696. And when you
download the raw log you get a plain text file
https://api.travis-ci.org/v3/job/485626173/log.txt.

Aaron Meurer

On Mon, Jan 28, 2019 at 1:27 PM Kalevi Suominen  wrote:
>
> I have found Travis logs quite useful as the browser can show them. Azure logs
> come in zip files which are rather inconvenient to deal with. I don't think 
> that
> dropping Azure would be a loss for me.
>
> Kalevi Suominen
>
> On Monday, January 28, 2019 at 10:01:51 PM UTC+2, Aaron Meurer wrote:
>>
>> So Azure Pipelines has been running now for about 3 1/2 months. I
>> would like to get feedback on how people like it.
>>
>> For myself, it seems that Travis has fixed most of the failing builds
>> issues that it was having. I've also noticed that Travis actually
>> finishes faster than Azure, possibly because of the higher concurrent
>> build limit.
>>
>> Frankly, I'm not too happy with Azure. The web interface is about as
>> complicated as they could possibly make it. It took me half an hour
>> just to find something as simple as the setting to disable email
>> notifications. The YAML spec is also much more complicated, and
>> despite being so, has some serious limitations, such as environment
>> variables not being shared across "scripts". The most annoying thing
>> by far is that people with push access don't have the ability to
>> restart failed builds. I think I as an admin can give people access,
>> but 1) this is annoying to do, and 2) I really can't even figure out
>> where I should do that (I can't stress just how complicated and
>> unintuitive their web interface is).
>>
>> Travis for its part is extremely simple, both in its web interface and
>> YAML spec. It's also very widely used, meaning most issues you would
>> encounter with it you can quickly find a workaround by Googling. For
>> instance, there is a workaround required to get Python 3.7 working,
>> but a Google search for "Python 3.7 Travis" turns up
>> https://github.com/travis-ci/travis-ci/issues/9815, which has the
>> workaround in the first comment. Contrast that with "Azure pipelines
>> pypy", which only turns up my issue requesting official support, with
>> no workarounds 
>> (https://github.com/Microsoft/azure-pipelines-tasks/issues/8514).
>>
>> So for my part, I would like to stop using it. I'm adding Python 3.7
>> support to Travis here, which was the only thing that was only on
>> Azure https://github.com/sympy/sympy/pull/15867 (yes, this will add
>> more Travis builds, but we should be able to drop 3.4 support after
>> this next release).
>>
>> But I would also like to hear others' opinions on it. Have you noticed
>> that Azure is any better than Travis in some way? I don't follow every
>> PR, so I don't notice every issue that comes up with Travis or Azure.
>>
>> Aaron Meurer
>>
>> On Fri, Nov 9, 2018 at 12:43 PM Aaron Meurer  wrote:
>> >
>> > Azure has fewer concurrent builds than Travis (10 instead of 15), but
>> > it also has longer builds 60 instead of 50, meaning we can split the
>> > tests 2 ways instead of 4. I think Azure also boots up faster and
>> > possibly has faster machines, though I haven't tested it.
>> >
>> > Aaron Meurer
>> > On Thu, Nov 8, 2018 at 9:06 PM Sidhant Nagpal  
>> > wrote:
>> > >
>> > > Right, the test time is likely to be affected further on the account of 
>> > > lesser concurrent builds on Azure.
>> > >
>> > > --
>> > > You received this message because you are subscribed to the Google 
>> > > Groups "sympy" group.
>> > > To unsubscribe from this group and stop receiving emails from it, send 
>> > > an email to sympy+unsubscr...@googlegroups.com.
>> > > To post to this group, send email to sympy@googlegroups.com.
>> > > Visit this group at https://groups.google.com/group/sympy.
>> > > To view this discussion on the web visit 
>> > > https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com.
>> > > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sympy" group.
> To unsubscribe from this group 

Re: [sympy] Re: Azure pipelines

[Kalevi Suominen]

I have found Travis logs quite useful as the browser can show them. Azure 
logs 
come in zip files which are rather inconvenient to deal with. I don't think 
that
dropping Azure would be a loss for me.

Kalevi Suominen

On Monday, January 28, 2019 at 10:01:51 PM UTC+2, Aaron Meurer wrote:
>
> So Azure Pipelines has been running now for about 3 1/2 months. I 
> would like to get feedback on how people like it. 
>
> For myself, it seems that Travis has fixed most of the failing builds 
> issues that it was having. I've also noticed that Travis actually 
> finishes faster than Azure, possibly because of the higher concurrent 
> build limit. 
>
> Frankly, I'm not too happy with Azure. The web interface is about as 
> complicated as they could possibly make it. It took me half an hour 
> just to find something as simple as the setting to disable email 
> notifications. The YAML spec is also much more complicated, and 
> despite being so, has some serious limitations, such as environment 
> variables not being shared across "scripts". The most annoying thing 
> by far is that people with push access don't have the ability to 
> restart failed builds. I think I as an admin can give people access, 
> but 1) this is annoying to do, and 2) I really can't even figure out 
> where I should do that (I can't stress just how complicated and 
> unintuitive their web interface is). 
>
> Travis for its part is extremely simple, both in its web interface and 
> YAML spec. It's also very widely used, meaning most issues you would 
> encounter with it you can quickly find a workaround by Googling. For 
> instance, there is a workaround required to get Python 3.7 working, 
> but a Google search for "Python 3.7 Travis" turns up 
> https://github.com/travis-ci/travis-ci/issues/9815, which has the 
> workaround in the first comment. Contrast that with "Azure pipelines 
> pypy", which only turns up my issue requesting official support, with 
> no workarounds (
> https://github.com/Microsoft/azure-pipelines-tasks/issues/8514). 
>
> So for my part, I would like to stop using it. I'm adding Python 3.7 
> support to Travis here, which was the only thing that was only on 
> Azure https://github.com/sympy/sympy/pull/15867 (yes, this will add 
> more Travis builds, but we should be able to drop 3.4 support after 
> this next release). 
>
> But I would also like to hear others' opinions on it. Have you noticed 
> that Azure is any better than Travis in some way? I don't follow every 
> PR, so I don't notice every issue that comes up with Travis or Azure. 
>
> Aaron Meurer 
>
> On Fri, Nov 9, 2018 at 12:43 PM Aaron Meurer  wrote: 
> > 
> > Azure has fewer concurrent builds than Travis (10 instead of 15), but 
> > it also has longer builds 60 instead of 50, meaning we can split the 
> > tests 2 ways instead of 4. I think Azure also boots up faster and 
> > possibly has faster machines, though I haven't tested it. 
> > 
> > Aaron Meurer 
> > On Thu, Nov 8, 2018 at 9:06 PM Sidhant Nagpal  
> wrote: 
> > > 
> > > Right, the test time is likely to be affected further on the account 
> of lesser concurrent builds on Azure. 
> > > 
> > > -- 
> > > You received this message because you are subscribed to the Google 
> Groups "sympy" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> an email to sympy+unsubscr...@googlegroups.com. 
> > > To post to this group, send email to sympy@googlegroups.com. 
> > > Visit this group at https://groups.google.com/group/sympy. 
> > > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com.
>  
>
> > > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/bcf320e4-3dae-4165-83db-c61ee1f91a4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Jason Moore]

Note that I haven't paid attention to the Azure system at all, but it
sounds like it offers nothing (in fact maybe less). Everyone is already
familiar with Travis, so I'd say drop Azure.

Thanks for the time and effort on trying it out.

Jason
moorepants.info
+01 530-601-9791


On Mon, Jan 28, 2019 at 12:01 PM Aaron Meurer  wrote:

> So Azure Pipelines has been running now for about 3 1/2 months. I
> would like to get feedback on how people like it.
>
> For myself, it seems that Travis has fixed most of the failing builds
> issues that it was having. I've also noticed that Travis actually
> finishes faster than Azure, possibly because of the higher concurrent
> build limit.
>
> Frankly, I'm not too happy with Azure. The web interface is about as
> complicated as they could possibly make it. It took me half an hour
> just to find something as simple as the setting to disable email
> notifications. The YAML spec is also much more complicated, and
> despite being so, has some serious limitations, such as environment
> variables not being shared across "scripts". The most annoying thing
> by far is that people with push access don't have the ability to
> restart failed builds. I think I as an admin can give people access,
> but 1) this is annoying to do, and 2) I really can't even figure out
> where I should do that (I can't stress just how complicated and
> unintuitive their web interface is).
>
> Travis for its part is extremely simple, both in its web interface and
> YAML spec. It's also very widely used, meaning most issues you would
> encounter with it you can quickly find a workaround by Googling. For
> instance, there is a workaround required to get Python 3.7 working,
> but a Google search for "Python 3.7 Travis" turns up
> https://github.com/travis-ci/travis-ci/issues/9815, which has the
> workaround in the first comment. Contrast that with "Azure pipelines
> pypy", which only turns up my issue requesting official support, with
> no workarounds (
> https://github.com/Microsoft/azure-pipelines-tasks/issues/8514).
>
> So for my part, I would like to stop using it. I'm adding Python 3.7
> support to Travis here, which was the only thing that was only on
> Azure https://github.com/sympy/sympy/pull/15867 (yes, this will add
> more Travis builds, but we should be able to drop 3.4 support after
> this next release).
>
> But I would also like to hear others' opinions on it. Have you noticed
> that Azure is any better than Travis in some way? I don't follow every
> PR, so I don't notice every issue that comes up with Travis or Azure.
>
> Aaron Meurer
>
> On Fri, Nov 9, 2018 at 12:43 PM Aaron Meurer  wrote:
> >
> > Azure has fewer concurrent builds than Travis (10 instead of 15), but
> > it also has longer builds 60 instead of 50, meaning we can split the
> > tests 2 ways instead of 4. I think Azure also boots up faster and
> > possibly has faster machines, though I haven't tested it.
> >
> > Aaron Meurer
> > On Thu, Nov 8, 2018 at 9:06 PM Sidhant Nagpal 
> wrote:
> > >
> > > Right, the test time is likely to be affected further on the account
> of lesser concurrent builds on Azure.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups "sympy" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an email to sympy+unsubscr...@googlegroups.com.
> > > To post to this group, send email to sympy@googlegroups.com.
> > > Visit this group at https://groups.google.com/group/sympy.
> > > To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com
> .
> > > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2Bum-5ySbE2XrETKTuphFTjTo-C6RW3-TLt-kOHO5QvGg%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAP7f1AjVmksVzxKVjbGDcX0Q9%3DnzFEgABNtL4i2DD7X9L3JWiA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

So Azure Pipelines has been running now for about 3 1/2 months. I
would like to get feedback on how people like it.

For myself, it seems that Travis has fixed most of the failing builds
issues that it was having. I've also noticed that Travis actually
finishes faster than Azure, possibly because of the higher concurrent
build limit.

Frankly, I'm not too happy with Azure. The web interface is about as
complicated as they could possibly make it. It took me half an hour
just to find something as simple as the setting to disable email
notifications. The YAML spec is also much more complicated, and
despite being so, has some serious limitations, such as environment
variables not being shared across "scripts". The most annoying thing
by far is that people with push access don't have the ability to
restart failed builds. I think I as an admin can give people access,
but 1) this is annoying to do, and 2) I really can't even figure out
where I should do that (I can't stress just how complicated and
unintuitive their web interface is).

Travis for its part is extremely simple, both in its web interface and
YAML spec. It's also very widely used, meaning most issues you would
encounter with it you can quickly find a workaround by Googling. For
instance, there is a workaround required to get Python 3.7 working,
but a Google search for "Python 3.7 Travis" turns up
https://github.com/travis-ci/travis-ci/issues/9815, which has the
workaround in the first comment. Contrast that with "Azure pipelines
pypy", which only turns up my issue requesting official support, with
no workarounds (https://github.com/Microsoft/azure-pipelines-tasks/issues/8514).

So for my part, I would like to stop using it. I'm adding Python 3.7
support to Travis here, which was the only thing that was only on
Azure https://github.com/sympy/sympy/pull/15867 (yes, this will add
more Travis builds, but we should be able to drop 3.4 support after
this next release).

But I would also like to hear others' opinions on it. Have you noticed
that Azure is any better than Travis in some way? I don't follow every
PR, so I don't notice every issue that comes up with Travis or Azure.

Aaron Meurer

On Fri, Nov 9, 2018 at 12:43 PM Aaron Meurer  wrote:
>
> Azure has fewer concurrent builds than Travis (10 instead of 15), but
> it also has longer builds 60 instead of 50, meaning we can split the
> tests 2 ways instead of 4. I think Azure also boots up faster and
> possibly has faster machines, though I haven't tested it.
>
> Aaron Meurer
> On Thu, Nov 8, 2018 at 9:06 PM Sidhant Nagpal  
> wrote:
> >
> > Right, the test time is likely to be affected further on the account of 
> > lesser concurrent builds on Azure.
> >
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "sympy" group.
> > To unsubscribe from this group and stop receiving emails from it, send an 
> > email to sympy+unsubscr...@googlegroups.com.
> > To post to this group, send email to sympy@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sympy.
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2Bum-5ySbE2XrETKTuphFTjTo-C6RW3-TLt-kOHO5QvGg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

Azure has fewer concurrent builds than Travis (10 instead of 15), but
it also has longer builds 60 instead of 50, meaning we can split the
tests 2 ways instead of 4. I think Azure also boots up faster and
possibly has faster machines, though I haven't tested it.

Aaron Meurer
On Thu, Nov 8, 2018 at 9:06 PM Sidhant Nagpal  wrote:
>
> Right, the test time is likely to be affected further on the account of 
> lesser concurrent builds on Azure.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6K68ZW3tg-uYtdyPP23T%2BfJcS9V_zwX00_t-5D%2BHh2eJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Sidhant Nagpal]

Right, the test time is likely to be affected further on the account of lesser 
concurrent builds on Azure.

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/08646799-88cc-4375-af17-58fd6682fa9d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sympy] Re: Azure pipelines

[Aaron Meurer]

To be fair, the runtimes aren't exactly comparable because they
currently run different things. Azure runs 3.7 whereas Travis doesn't,
and Azure still needs to run the slow tests, docs build, and PyPy
tests.

Regarding the optional dependency build, it is currently failing
because of some broken conda packages. I'm hoping that once
conda-forge finishes its compiler migration it will work.

See https://github.com/sympy/sympy/issues/15385 for more information.

Aaron Meurer

On Thu, Nov 8, 2018 at 7:49 AM Sidhant Nagpal  wrote:
>
> Nice, the tests seem to run faster on Azure on individual jobs compared to 
> Travis.
> Although, the overall testing time seems to be comparable for both.
>
> https://dev.azure.com/SymPy/SymPy/_build/results?buildId=266=logs
> https://travis-ci.org/sympy/sympy/builds/444273129?utm_source=github_status_medium=notification
>
> By the way, is there a blocker for making optional dependency tests to work 
> on Azure?
> (The status badge for Azure builds reads "partially succeeded")
>
> On Saturday, October 13, 2018 at 12:46:54 AM UTC+5:30, Aaron Meurer wrote:
>>
>> You may have noticed some Azure pipelines builds in the pull requests.
>> Till now, they have only run the doctests, but I have just merged a PR
>> which adds the rest of the test suite (there are still a few things
>> missing, see https://github.com/sympy/sympy/issues/15385).
>>
>> For now, the Azure builds are not required to merge pull requests. The
>> idea here is to experiment, to see if it is worth using Azure over
>> Travis. So even though they are not required, take a look if the tests
>> fail there, and also make a note if Azure or Travis is more reliable.
>>
>> My plan is
>>
>> 1. Run both Travis and Azure in parallel.
>>
>> 2. Once it is clear Azure tests are passing make them required to
>> merge, just like Travis.
>>
>> 3. Add the missing stuff to Azure 
>> (https://github.com/sympy/sympy/issues/15385).
>>
>> 4. If it becomes clear that Azure is more reliable than Travis
>> (finishes faster, no build restarts), we can remove the Travis builds.
>> But we should let both run for at least a month before doing this.
>>
>> 5. If we remove Travis, we will need to keep at least the Travis docs
>> build, because doctr doesn't support Azure yet (and I don't plan to
>> add the docs build to Azure because of this).
>>
>> Or if it turns out that Azure really is worse than Travis, we can
>> scrap it, or maybe use it only for some specific builds. I already
>> have some gripes with it, and the UI is not as clean as on Travis. But
>> I'd like to hear what other think.
>>
>> Aaron Meurer
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sympy/fbbe2fff-af5f-4a1e-bee7-cc7aa577fc99%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6L73w94qCSgZNtodvBnNQmB3DFiHsDN5%2BmP%3DA8-2Rwcfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[sympy] Re: Azure pipelines

[Sidhant Nagpal]

Nice, the tests seem to run faster on Azure on individual jobs compared to 
Travis.
Although, the overall testing time seems to be comparable for both.

https://dev.azure.com/SymPy/SymPy/_build/results?buildId=266=logs
https://travis-ci.org/sympy/sympy/builds/444273129?utm_source=github_status_medium=notification

By the way, is there a blocker for making optional dependency tests to work 
on Azure?
(The status badge for Azure builds reads "partially succeeded")

On Saturday, October 13, 2018 at 12:46:54 AM UTC+5:30, Aaron Meurer wrote:
>
> You may have noticed some Azure pipelines builds in the pull requests. 
> Till now, they have only run the doctests, but I have just merged a PR 
> which adds the rest of the test suite (there are still a few things 
> missing, see https://github.com/sympy/sympy/issues/15385). 
>
> For now, the Azure builds are not required to merge pull requests. The 
> idea here is to experiment, to see if it is worth using Azure over 
> Travis. So even though they are not required, take a look if the tests 
> fail there, and also make a note if Azure or Travis is more reliable. 
>
> My plan is 
>
> 1. Run both Travis and Azure in parallel. 
>
> 2. Once it is clear Azure tests are passing make them required to 
> merge, just like Travis. 
>
> 3. Add the missing stuff to Azure (
> https://github.com/sympy/sympy/issues/15385). 
>
> 4. If it becomes clear that Azure is more reliable than Travis 
> (finishes faster, no build restarts), we can remove the Travis builds. 
> But we should let both run for at least a month before doing this. 
>
> 5. If we remove Travis, we will need to keep at least the Travis docs 
> build, because doctr doesn't support Azure yet (and I don't plan to 
> add the docs build to Azure because of this). 
>
> Or if it turns out that Azure really is worse than Travis, we can 
> scrap it, or maybe use it only for some specific builds. I already 
> have some gripes with it, and the UI is not as clean as on Travis. But 
> I'd like to hear what other think. 
>
> Aaron Meurer 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/fbbe2fff-af5f-4a1e-bee7-cc7aa577fc99%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sympy] Re: Azure pipelines

[Aaron Meurer]

I have made Azure pipelines required to merge pull requests. Let me
know if there are any issues.

Aaron Meurer
On Fri, Oct 12, 2018 at 1:16 PM Aaron Meurer  wrote:
>
> You may have noticed some Azure pipelines builds in the pull requests.
> Till now, they have only run the doctests, but I have just merged a PR
> which adds the rest of the test suite (there are still a few things
> missing, see https://github.com/sympy/sympy/issues/15385).
>
> For now, the Azure builds are not required to merge pull requests. The
> idea here is to experiment, to see if it is worth using Azure over
> Travis. So even though they are not required, take a look if the tests
> fail there, and also make a note if Azure or Travis is more reliable.
>
> My plan is
>
> 1. Run both Travis and Azure in parallel.
>
> 2. Once it is clear Azure tests are passing make them required to
> merge, just like Travis.
>
> 3. Add the missing stuff to Azure 
> (https://github.com/sympy/sympy/issues/15385).
>
> 4. If it becomes clear that Azure is more reliable than Travis
> (finishes faster, no build restarts), we can remove the Travis builds.
> But we should let both run for at least a month before doing this.
>
> 5. If we remove Travis, we will need to keep at least the Travis docs
> build, because doctr doesn't support Azure yet (and I don't plan to
> add the docs build to Azure because of this).
>
> Or if it turns out that Azure really is worse than Travis, we can
> scrap it, or maybe use it only for some specific builds. I already
> have some gripes with it, and the UI is not as clean as on Travis. But
> I'd like to hear what other think.
>
> Aaron Meurer

-- 
You received this message because you are subscribed to the Google Groups 
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at https://groups.google.com/group/sympy.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2B1a5oa_Y%3Dtcxd2PaROPts_h_ZiyjUAD7%2B2SB3O00p_kA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.