Re: perf.haskell.org functional again

[Sebastian Graf]

Hi,

just came here to resurrect the thread and point out that nofib currently
isn't really run in parallel:
https://ghc.haskell.org/trac/ghc/ticket/15976#ticket
Also it's unclear to me how that would be possible without rewriting the
whole benchmark harness in Python or Haskell, because make parallelism
would probably interleave outputs of different rules, giving nofib-analyse
a hard time.

Greetings
Sebastian

Am Mo., 30. Okt. 2017 um 00:14 Uhr schrieb Joachim Breitner <
m...@joachim-breitner.de>:

> Hi,
>
> Am Sonntag, den 29.10.2017, 23:58 +0100 schrieb Sebastian Graf:
> > Hi,
> >
> > just wanted to throw in the idea of parallelising the benchmark suite
> > (hurts to even write that, but cachegrind) to speed up the build, if
> > ever need be.
>
> https://github.com/nomeata/gipeda/blob/master/ghc/run-speed.sh#L143
>
> good idea indeed – I’ve been doing that since I started running
> cachegrind ;-)
>
>
> Joachim
>
> --
> Joachim Breitner
>   m...@joachim-breitner.de
>   http://www.joachim-breitner.de/
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Joachim Breitner]

Hi,

Am Sonntag, den 29.10.2017, 23:58 +0100 schrieb Sebastian Graf:
> Hi,
> 
> just wanted to throw in the idea of parallelising the benchmark suite
> (hurts to even write that, but cachegrind) to speed up the build, if
> ever need be.

https://github.com/nomeata/gipeda/blob/master/ghc/run-speed.sh#L143

good idea indeed – I’ve been doing that since I started running
cachegrind ;-)


Joachim

-- 
Joachim Breitner
  m...@joachim-breitner.de
  http://www.joachim-breitner.de/


signature.asc
Description: This is a digitally signed message part
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Sebastian Graf]

Hi,

just wanted to throw in the idea of parallelising the benchmark suite
(hurts to even write that, but cachegrind) to speed up the build, if ever
need be.

Cheers,
Sebastian

On Wed, Oct 25, 2017 at 2:13 AM, Manuel M T Chakravarty <
c...@justtesting.org> wrote:

> Am 24.10.2017 um 14:46 schrieb Joachim Breitner  >:
>
> Is CircleCI the future now?
>
>
> Yes, as per
>
>   https://ghc.haskell.org/trac/ghc/wiki/ContinuousIntegration
>
> In general, yes. But it’s running fine for now, so I would not
> prematurely throw it over.
>
> My requirements are:
>
> * needs to run on every commit (not just every push), including
>   branches.
> * needs to be able to push to a repository¹, so it needs access to
>   some secret token for depolyment.
>   Alternatively, the CI could simply keep track of the build log
>   and the perf.haskell.org dashboard scrapes it from there.
> * Occasionally, I find that I want to rebuild a fair number of commits
>   that are already in the repository. So a good way to trigger
>   rebuilds would be nice.
>
>
> No need to mess with a working system, but as long as performance counters
> are the basis, I think, these constraints can all be met. In particular,
> CircleCI has facilities to allow deployments including managing of secrets.
> (That is a pretty common requirements for CIs.)
>
> Cheers,
> Manuel
>
>
> Greetings,
> Joachim
>
> ¹ https://github.com/nomeata/ghc-speed-logs/commits/master
>
> Am Dienstag, den 24.10.2017, 12:18 +1100 schrieb Manuel M T
> Chakravarty:
>
> Hi Joachim,
>
> Great! Just because you mention CI infrastructure and our effort around
> GHC CI at the moment, do you think, it would make sense to move this to
> CircleCI eventually?
>
> Cheers,
> Manuel
>
> Am 24.10.2017 um 11:02 schrieb Joachim Breitner  >:
>
> Hi,
>
> after a system upgrade to avoid weird linker errors, and after some fixes
> in the nofib submodule, http://perf.haskell.org/ghc is running again.
>
> I am collecting instruction counts instead of runtime, because the latter
> was just too often varying too wildly. I hope this will yield less false
> alarms.
>
> I am also running nofib with mode=fast. This way, building GHC, running
> the testsuite and nofib takes a bit over one hour. I hope this can keep up
> with y'all's commits (when it took 2h it couldn't).
>
> Now nothing of the setup requires a quiet dedicated machine, so if there
> is a need, we could move these builds into the cloud or into some CI
> infrastructure - but no changes are immediately planned.
>
> Enjoy, Joachim
>
> --
> Joachim Breitner
>  m...@joachim-breitner.de
>  http://www.joachim-breitner.de/
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Manuel M T Chakravarty]

> Am 24.10.2017 um 14:46 schrieb Joachim Breitner  >:
> 
> Is CircleCI the future now?

Yes, as per

  https://ghc.haskell.org/trac/ghc/wiki/ContinuousIntegration 


> In general, yes. But it’s running fine for now, so I would not
> prematurely throw it over.
> 
> My requirements are: 
> 
> * needs to run on every commit (not just every push), including 
>   branches.
> * needs to be able to push to a repository¹, so it needs access to 
>   some secret token for depolyment.
>   Alternatively, the CI could simply keep track of the build log
>   and the perf.haskell.org  dashboard scrapes it 
> from there.
> * Occasionally, I find that I want to rebuild a fair number of commits
>   that are already in the repository. So a good way to trigger
>   rebuilds would be nice.

No need to mess with a working system, but as long as performance counters are 
the basis, I think, these constraints can all be met. In particular, CircleCI 
has facilities to allow deployments including managing of secrets. (That is a 
pretty common requirements for CIs.)

Cheers,
Manuel

> 
> Greetings,
> Joachim
> 
> ¹ https://github.com/nomeata/ghc-speed-logs/commits/master 
> 
> 
> Am Dienstag, den 24.10.2017, 12:18 +1100 schrieb Manuel M T
> Chakravarty:
>> Hi Joachim,
>> 
>> Great! Just because you mention CI infrastructure and our effort around GHC 
>> CI at the moment, do you think, it would make sense to move this to CircleCI 
>> eventually?
>> 
>> Cheers,
>> Manuel
>> 
>>> Am 24.10.2017 um 11:02 schrieb Joachim Breitner >> >:
>>> 
>>> Hi,
>>> 
>>> after a system upgrade to avoid weird linker errors, and after some fixes 
>>> in the nofib submodule, http://perf.haskell.org/ghc 
>>>  is running again.
>>> 
>>> I am collecting instruction counts instead of runtime, because the latter 
>>> was just too often varying too wildly. I hope this will yield less false 
>>> alarms.
>>> 
>>> I am also running nofib with mode=fast. This way, building GHC, running the 
>>> testsuite and nofib takes a bit over one hour. I hope this can keep up with 
>>> y'all's commits (when it took 2h it couldn't).
>>> 
>>> Now nothing of the setup requires a quiet dedicated machine, so if there is 
>>> a need, we could move these builds into the cloud or into some CI 
>>> infrastructure - but no changes are immediately planned.
>>> 
>>> Enjoy, Joachim 
> -- 
> Joachim Breitner
>  m...@joachim-breitner.de 
>  http://www.joachim-breitner.de/ 
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org 
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs 
> 
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Boespflug, Mathieu]

Hi Joachim,

> But that’s what I am saying: By measuring dynamic instructions (using
> cachegrind), the execution time is no longer relevant.

I'm skeptical that this metric is a reliable enough proxy for actual
runtime of user programs to make tracking it and only it sufficient.
For the reasons Sven Panne gave in a previous email. But I'll be very
interested to hear your experience report over time.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Joachim Breitner]

Hi,

Is CircleCI the future now?

In general, yes. But it’s running fine for now, so I would not
prematurely throw it over.

My requirements are: 

 * needs to run on every commit (not just every push), including 
   branches.
 * needs to be able to push to a repository¹, so it needs access to 
   some secret token for depolyment.
   Alternatively, the CI could simply keep track of the build log
   and the perf.haskell.org dashboard scrapes it from there.
 * Occasionally, I find that I want to rebuild a fair number of commits
   that are already in the repository. So a good way to trigger
   rebuilds would be nice.

Greetings,
Joachim

¹ https://github.com/nomeata/ghc-speed-logs/commits/master

Am Dienstag, den 24.10.2017, 12:18 +1100 schrieb Manuel M T
Chakravarty:
> Hi Joachim,
> 
> Great! Just because you mention CI infrastructure and our effort around GHC 
> CI at the moment, do you think, it would make sense to move this to CircleCI 
> eventually?
> 
> Cheers,
> Manuel
> 
> > Am 24.10.2017 um 11:02 schrieb Joachim Breitner :
> > 
> > Hi,
> > 
> > after a system upgrade to avoid weird linker errors, and after some fixes 
> > in the nofib submodule, http://perf.haskell.org/ghc is running again.
> > 
> > I am collecting instruction counts instead of runtime, because the latter 
> > was just too often varying too wildly. I hope this will yield less false 
> > alarms.
> > 
> > I am also running nofib with mode=fast. This way, building GHC, running the 
> > testsuite and nofib takes a bit over one hour. I hope this can keep up with 
> > y'all's commits (when it took 2h it couldn't).
> > 
> > Now nothing of the setup requires a quiet dedicated machine, so if there is 
> > a need, we could move these builds into the cloud or into some CI 
> > infrastructure - but no changes are immediately planned.
> > 
> > Enjoy, Joachim 
-- 
Joachim Breitner
  m...@joachim-breitner.de
  http://www.joachim-breitner.de/


signature.asc
Description: This is a digitally signed message part
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Manuel M T Chakravarty]

Hi Joachim,

Great! Just because you mention CI infrastructure and our effort around GHC CI 
at the moment, do you think, it would make sense to move this to CircleCI 
eventually?

Cheers,
Manuel

> Am 24.10.2017 um 11:02 schrieb Joachim Breitner :
> 
> Hi,
> 
> after a system upgrade to avoid weird linker errors, and after some fixes in 
> the nofib submodule, http://perf.haskell.org/ghc is running again.
> 
> I am collecting instruction counts instead of runtime, because the latter was 
> just too often varying too wildly. I hope this will yield less false alarms.
> 
> I am also running nofib with mode=fast. This way, building GHC, running the 
> testsuite and nofib takes a bit over one hour. I hope this can keep up with 
> y'all's commits (when it took 2h it couldn't).
> 
> Now nothing of the setup requires a quiet dedicated machine, so if there is a 
> need, we could move these builds into the cloud or into some CI 
> infrastructure - but no changes are immediately planned.
> 
> Enjoy, Joachim 
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf.haskell.org functional again

[Moritz Angermann]

Hooray! Thanks for doing this!

Sent from my iPhone

> On 24 Oct 2017, at 8:02 AM, Joachim Breitner  wrote:
> 
> Hi,
> 
> after a system upgrade to avoid weird linker errors, and after some fixes in 
> the nofib submodule, http://perf.haskell.org/ghc is running again.
> 
> I am collecting instruction counts instead of runtime, because the latter was 
> just too often varying too wildly. I hope this will yield less false alarms.
> 
> I am also running nofib with mode=fast. This way, building GHC, running the 
> testsuite and nofib takes a bit over one hour. I hope this can keep up with 
> y'all's commits (when it took 2h it couldn't).
> 
> Now nothing of the setup requires a quiet dedicated machine, so if there is a 
> need, we could move these builds into the cloud or into some CI 
> infrastructure - but no changes are immediately planned.
> 
> Enjoy, Joachim 
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs