Re: [Bioc-devel] Check time on macOS (MIRit)

[Martin Grigorov]

Hi Kasper,

According to
https://github.com/Bioconductor/BBS/blob/9ebb2cf20e70e4bebf24a2a824a67193de821b08/3.19/bioc-mac-arm64/kjohnson3/config.sh#L17-L20
the machine has 24 vCPUs.
The automatic/weekly build utilizes 12-16 out of those at any given time!
With the respective RAM and disk usage.

By "by hand" do you mean to run the build/check of a given package on a
"calm" machine ?
It would be expected that it will be faster if it has all/most resources
just for it.

Martin

On Wed, Feb 7, 2024 at 5:12 PM Kasper Daniel Hansen <
kasperdanielhan...@gmail.com> wrote:

> To be a bit more clear and vague at the same time: currently, some of our
> macOS builders exhibit weird slowdowns when the build system is run
> automatically compared to running builds "by hand". This is pretty weird,
> and hopefully we can figure out the cause and fix it. Specifically, builds
> on kjohnson3 is (at the moment) weirdly slow.
>
> Best,
> Kasper
>
> On Wed, Feb 7, 2024 at 7:24 AM Jacopo Ronchi 
> wrote:
>
> > Ok, understood. Thanks
> >
> > Il mer 7 feb 2024, 12:46 Vincent Carey  ha
> > scritto:
> >
> > > We need a bit of internal discussion before committing to this
> notion.  I
> > > will get back to the list on this.
> > >
> > > On Wed, Feb 7, 2024 at 6:34 AM Jacopo Ronchi 
> > > wrote:
> > >
> > >> Dear Vincent, thanks for your answer. So, at least for now, can I
> ignore
> > >> the warning on macOS for the submission of MIRit?
> > >>
> > >> Il mer 7 feb 2024, 12:26 Vincent Carey 
> ha
> > >> scritto:
> > >>
> > >>> Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
> > >>> disparities of platform
> > >>> performance and are working to improve the build process.  It will
> take
> > >>> time.  I believe that
> > >>> achievement of acceptable timings on the linux platform should be the
> > >>> key goal at
> > >>> this time.
> > >>>
> > >>> On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi <
> jacopo.ron...@unimib.it>
> > >>> wrote:
> > >>>
> >  Dear developers,
> > 
> >  I am struggling to reduce R CMD check times for my package, MIRit,
> on
> >  macOS. The check on linux takes 4 minutes, while on macOS it needs
> > 9/10
> >  minutes. I have severely reduced the size of examples and tests,
> which
> >  now
> >  use smaller datasets and cached resources.
> > 
> >  On linux, unit tests require 30 seconds more or less, and examples
> 40
> >  seconds. On the other hand, macOS tests take 100 seconds, and 75 for
> > the
> >  examples. The issue is that sometimes the check runs in less than 10
> >  minutes, sometimes it needs a little bit more.
> > 
> >  However I am out of ideas on how to further reduce examples and
> > tests...
> >  The vignette is really extensive, but since SPB uses the
> > "--no-vignette"
> >  option this should not influence heavily on this behavior, at least
> to
> >  my
> >  understanding.
> > 
> >  Sorry for bothering you,
> >  Best regards,
> >  Jacopo
> > 
> >  [[alternative HTML version deleted]]
> > 
> >  ___
> >  Bioc-devel@r-project.org mailing list
> >  https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > 
> > >>>
> > >>> The information in this e-mail is intended only for the person to
> whom
> > >>> it is
> > >>> addressed. If you believe this e-mail was sent to you in error and
> the
> > >>> e-mail
> > >>> contains patient information, please contact the Partners Compliance
> > HelpLine
> > >>> at
> > >>> http://www.partners.org/complianceline . If the e-mail was sent to
> you
> > >>> in error
> > >>> but does not contain patient information, please contact the sender
> and
> > >>> properly
> > >>> dispose of the e-mail.
> > >>
> > >>
> > > The information in this e-mail is intended only for th...{{dropped:16}}
> >
> > ___
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
>
>
> --
> Best,
> Kasper
>
> [[alternative HTML version deleted]]
>
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Check time on macOS (MIRit)

[Kasper Daniel Hansen]

To be a bit more clear and vague at the same time: currently, some of our
macOS builders exhibit weird slowdowns when the build system is run
automatically compared to running builds "by hand". This is pretty weird,
and hopefully we can figure out the cause and fix it. Specifically, builds
on kjohnson3 is (at the moment) weirdly slow.

Best,
Kasper

On Wed, Feb 7, 2024 at 7:24 AM Jacopo Ronchi 
wrote:

> Ok, understood. Thanks
>
> Il mer 7 feb 2024, 12:46 Vincent Carey  ha
> scritto:
>
> > We need a bit of internal discussion before committing to this notion.  I
> > will get back to the list on this.
> >
> > On Wed, Feb 7, 2024 at 6:34 AM Jacopo Ronchi 
> > wrote:
> >
> >> Dear Vincent, thanks for your answer. So, at least for now, can I ignore
> >> the warning on macOS for the submission of MIRit?
> >>
> >> Il mer 7 feb 2024, 12:26 Vincent Carey  ha
> >> scritto:
> >>
> >>> Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
> >>> disparities of platform
> >>> performance and are working to improve the build process.  It will take
> >>> time.  I believe that
> >>> achievement of acceptable timings on the linux platform should be the
> >>> key goal at
> >>> this time.
> >>>
> >>> On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi 
> >>> wrote:
> >>>
>  Dear developers,
> 
>  I am struggling to reduce R CMD check times for my package, MIRit, on
>  macOS. The check on linux takes 4 minutes, while on macOS it needs
> 9/10
>  minutes. I have severely reduced the size of examples and tests, which
>  now
>  use smaller datasets and cached resources.
> 
>  On linux, unit tests require 30 seconds more or less, and examples 40
>  seconds. On the other hand, macOS tests take 100 seconds, and 75 for
> the
>  examples. The issue is that sometimes the check runs in less than 10
>  minutes, sometimes it needs a little bit more.
> 
>  However I am out of ideas on how to further reduce examples and
> tests...
>  The vignette is really extensive, but since SPB uses the
> "--no-vignette"
>  option this should not influence heavily on this behavior, at least to
>  my
>  understanding.
> 
>  Sorry for bothering you,
>  Best regards,
>  Jacopo
> 
>  [[alternative HTML version deleted]]
> 
>  ___
>  Bioc-devel@r-project.org mailing list
>  https://stat.ethz.ch/mailman/listinfo/bioc-devel
> 
> >>>
> >>> The information in this e-mail is intended only for the person to whom
> >>> it is
> >>> addressed. If you believe this e-mail was sent to you in error and the
> >>> e-mail
> >>> contains patient information, please contact the Partners Compliance
> HelpLine
> >>> at
> >>> http://www.partners.org/complianceline . If the e-mail was sent to you
> >>> in error
> >>> but does not contain patient information, please contact the sender and
> >>> properly
> >>> dispose of the e-mail.
> >>
> >>
> > The information in this e-mail is intended only for th...{{dropped:16}}
>
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>


-- 
Best,
Kasper

[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Check time on macOS (MIRit)

[Jacopo Ronchi]

Ok, understood. Thanks

Il mer 7 feb 2024, 12:46 Vincent Carey  ha
scritto:

> We need a bit of internal discussion before committing to this notion.  I
> will get back to the list on this.
>
> On Wed, Feb 7, 2024 at 6:34 AM Jacopo Ronchi 
> wrote:
>
>> Dear Vincent, thanks for your answer. So, at least for now, can I ignore
>> the warning on macOS for the submission of MIRit?
>>
>> Il mer 7 feb 2024, 12:26 Vincent Carey  ha
>> scritto:
>>
>>> Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
>>> disparities of platform
>>> performance and are working to improve the build process.  It will take
>>> time.  I believe that
>>> achievement of acceptable timings on the linux platform should be the
>>> key goal at
>>> this time.
>>>
>>> On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi 
>>> wrote:
>>>
 Dear developers,

 I am struggling to reduce R CMD check times for my package, MIRit, on
 macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
 minutes. I have severely reduced the size of examples and tests, which
 now
 use smaller datasets and cached resources.

 On linux, unit tests require 30 seconds more or less, and examples 40
 seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
 examples. The issue is that sometimes the check runs in less than 10
 minutes, sometimes it needs a little bit more.

 However I am out of ideas on how to further reduce examples and tests...
 The vignette is really extensive, but since SPB uses the "--no-vignette"
 option this should not influence heavily on this behavior, at least to
 my
 understanding.

 Sorry for bothering you,
 Best regards,
 Jacopo

 [[alternative HTML version deleted]]

 ___
 Bioc-devel@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/bioc-devel

>>>
>>> The information in this e-mail is intended only for the person to whom
>>> it is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance 
>>> HelpLine
>>> at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you
>>> in error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>
>>
> The information in this e-mail is intended only for th...{{dropped:16}}

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Check time on macOS (MIRit)

[Vincent Carey]

We need a bit of internal discussion before committing to this notion.  I
will get back to the list on this.

On Wed, Feb 7, 2024 at 6:34 AM Jacopo Ronchi 
wrote:

> Dear Vincent, thanks for your answer. So, at least for now, can I ignore
> the warning on macOS for the submission of MIRit?
>
> Il mer 7 feb 2024, 12:26 Vincent Carey  ha
> scritto:
>
>> Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
>> disparities of platform
>> performance and are working to improve the build process.  It will take
>> time.  I believe that
>> achievement of acceptable timings on the linux platform should be the key
>> goal at
>> this time.
>>
>> On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi 
>> wrote:
>>
>>> Dear developers,
>>>
>>> I am struggling to reduce R CMD check times for my package, MIRit, on
>>> macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
>>> minutes. I have severely reduced the size of examples and tests, which
>>> now
>>> use smaller datasets and cached resources.
>>>
>>> On linux, unit tests require 30 seconds more or less, and examples 40
>>> seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
>>> examples. The issue is that sometimes the check runs in less than 10
>>> minutes, sometimes it needs a little bit more.
>>>
>>> However I am out of ideas on how to further reduce examples and tests...
>>> The vignette is really extensive, but since SPB uses the "--no-vignette"
>>> option this should not influence heavily on this behavior, at least to my
>>> understanding.
>>>
>>> Sorry for bothering you,
>>> Best regards,
>>> Jacopo
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> ___
>>> Bioc-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance HelpLine
>> at
>> http://www.partners.org/complianceline . If the e-mail was sent to you
>> in error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>
>

-- 
The information in this e-mail is intended only for the person to whom it 
is
addressed. If you believe this e-mail was sent to you in error and the 
e-mail
contains patient information, please contact the Partners Compliance 
HelpLine at
http://www.partners.org/complianceline 
 . If the e-mail was sent to you in 
error
but does not contain patient information, please contact the sender 
and properly
dispose of the e-mail.

[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Check time on macOS (MIRit)

[Jacopo Ronchi]

Dear Vincent, thanks for your answer. So, at least for now, can I ignore
the warning on macOS for the submission of MIRit?

Il mer 7 feb 2024, 12:26 Vincent Carey  ha
scritto:

> Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
> disparities of platform
> performance and are working to improve the build process.  It will take
> time.  I believe that
> achievement of acceptable timings on the linux platform should be the key
> goal at
> this time.
>
> On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi 
> wrote:
>
>> Dear developers,
>>
>> I am struggling to reduce R CMD check times for my package, MIRit, on
>> macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
>> minutes. I have severely reduced the size of examples and tests, which now
>> use smaller datasets and cached resources.
>>
>> On linux, unit tests require 30 seconds more or less, and examples 40
>> seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
>> examples. The issue is that sometimes the check runs in less than 10
>> minutes, sometimes it needs a little bit more.
>>
>> However I am out of ideas on how to further reduce examples and tests...
>> The vignette is really extensive, but since SPB uses the "--no-vignette"
>> option this should not influence heavily on this behavior, at least to my
>> understanding.
>>
>> Sorry for bothering you,
>> Best regards,
>> Jacopo
>>
>> [[alternative HTML version deleted]]
>>
>> ___
>> Bioc-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>
> The information in this e-mail is intended only for th...{{dropped:16}}

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

Re: [Bioc-devel] Check time on macOS (MIRit)

[Vincent Carey]

Thanks Jacopo.  You shouldn't have to struggle.  We are aware of
disparities of platform
performance and are working to improve the build process.  It will take
time.  I believe that
achievement of acceptable timings on the linux platform should be the key
goal at
this time.

On Wed, Feb 7, 2024 at 5:18 AM Jacopo Ronchi 
wrote:

> Dear developers,
>
> I am struggling to reduce R CMD check times for my package, MIRit, on
> macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
> minutes. I have severely reduced the size of examples and tests, which now
> use smaller datasets and cached resources.
>
> On linux, unit tests require 30 seconds more or less, and examples 40
> seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
> examples. The issue is that sometimes the check runs in less than 10
> minutes, sometimes it needs a little bit more.
>
> However I am out of ideas on how to further reduce examples and tests...
> The vignette is really extensive, but since SPB uses the "--no-vignette"
> option this should not influence heavily on this behavior, at least to my
> understanding.
>
> Sorry for bothering you,
> Best regards,
> Jacopo
>
> [[alternative HTML version deleted]]
>
> ___
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

-- 
The information in this e-mail is intended only for the ...{{dropped:18}}

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

[Bioc-devel] Check time on macOS (MIRit)

[Jacopo Ronchi]

Dear developers,

I am struggling to reduce R CMD check times for my package, MIRit, on
macOS. The check on linux takes 4 minutes, while on macOS it needs 9/10
minutes. I have severely reduced the size of examples and tests, which now
use smaller datasets and cached resources.

On linux, unit tests require 30 seconds more or less, and examples 40
seconds. On the other hand, macOS tests take 100 seconds, and 75 for the
examples. The issue is that sometimes the check runs in less than 10
minutes, sometimes it needs a little bit more.

However I am out of ideas on how to further reduce examples and tests...
The vignette is really extensive, but since SPB uses the "--no-vignette"
option this should not influence heavily on this behavior, at least to my
understanding.

Sorry for bothering you,
Best regards,
Jacopo

[[alternative HTML version deleted]]

___
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel