[ACTIVITY] Monday 14th - Friday 18th March

[Bernie Ogden]

Port to microinstance - TCWG-432 [5/10]
* Non-lab side of minimal trust benchmarking
* More investigation of runtime anomalies
* Reordered builder phases to do useful work while waiting for targets
* Updated everything to work with benchmarking LAVA user (rather than
running as me)

Automated backport benchmarking - TCWG-352 [2/10]
* Cycles of review/development/testing

Controlled image builds - TCWG-360 [1/10]
* More failures to get image to boot on Juno

Log critical data - TCWG-349 [1/10]
* Everything now logged, except where it depends on TCWG-360

Misc - [1/10]


=Plan=

Finish non-lab side of minimal trust benchmarking
Commit backport benchmarking, review permitting
Tweak microinstance in reaction to lab work
More Juno image work
More runtime anomaly work


=Availability=

Off from this Friday, back for three days from Monday 4th April
Return to ARM on Thursday 7th April
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 29th Februrary - 11th March

[Bernie Ogden]

Port to microinstance - TCWG-432 [17/10]
* Investigating difference between LAVA and 'desktop Juno' runtimes
** Some of this was down to piles of /dev/console output - redirecting
to file improved SPEC build time by 75%!
** Some cases make sense, others remain unexplained
** Might just go away if we update the Juno image
* Wrote up how to do benchmarking for minimal-trust cases
** Needs both lab and development work
* Merged another large tranche of changes back to benchmarking branch
** Microinstance more or less functional, main instance benchmarking
seems unbroken
** But some more tweaks to make as Lab work happens
* Prepared backport benchmarking for merge

Misc [3/10]


=Plan=

* Submit backport benchmarking for review
* Tweak uinstance in reaction to lab work
* Implement the non-lab side of minimal-trust benchmarking
* Return to looking at Juno image generation
* Look some more at LAVA/desktop runtime differences
* Implement small improvements, if time
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 22nd - 26th February

[Bernie Ogden]

Port to microinstance - TCWG-432 [7/10]
* Merged last few months of development back to benchmarking branch
* Restored support for multiple targets per builder
* Updated builder landed, altered jobs to work with it
** Removed assumption that host filesystem is non-persistent
** Stacked up test runs for the weekend

Transfer secret management to LAVA [1/10]
* LAVA jobs now use a within-LAVA key to access sources

Misc [2/10]
* Unsuccessful fiddling with heat-monitoring tools on Juno
* Usual background of mail and meetings


=Plan=

* Fallout from weekend test runs
** Some failure is going on, need to investigate
* Update docs and Jenkins configs w.r.t. last week's activity
* Further investigation on a couple of LAVA issues that are causing me pain
** Un-deserializable bundles
** Inaccessible image reports
* Continue assessing target stability/looking at inconsistent results
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 15th - 19th Februrary

[Bernie Ogden]

Bug with compiler flag handling - (no ticket) [2/10]
* Coremark-Pro was ignoring compiler flags
* Fixed that, made flag handling consistent across all benchmarks

Release benchmarking via Jenkins - TCWG-348 [1/10]
* Seems to work with test workload

Port to microinstance - TCG-432 [3/10]
* Looked at some inconsistent results
* Worried that one of the Junos may be sick, but unproven for now

Backport benchmarking via Jenkins - TCWG-352 [1/10]
* Finished 'general benchmarking' job
* Switched backport job to build/test cross-compilers
* Recent backport results bundles are corrupted, unable to work out why

Document benchmarking infrastructure - TCWG-496 [1/10]
* Documented Jenkins interface

Misc [2/10]


=Plan=

If updated builder becomes available, convert uinstance job to use it
Continue assessing target stability/looking at inconsistent results
Rework LAVA scripts to permit multiple targets per builder
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 8th - 12th February

[Bernie Ogden]

Port to microinstance - TCWG-432 [2/10]
* Better reporting/error trapping in dispatch scripts
* Some back and forth with Lab about builder implementation
* Started assessing target stability
* Fixed a few bugs in reporting/bundle generation

Document benchmarking infrastructure - TCWG-496 [2/10]
* All done apart from Jenkins interface
* Need to finish Jenkins interface first...

Generate graphs for benchmarking - TCWG-268 [1/10]
* Generate a bunch of filters and charts to track Coremark-Pro
* Wrote up what I learned about LAVA's reporting interface

EEMBC benchmarks - (no ticket) [2/10]
* Checked over what we had, decided it was probably complete
* Imported to our repos
* Wrote up what I know about this
* Moved Coremark-Pro into its own repo
** Broke some code that had worked by coincidence
** Fixed it

Misc
* Usual meetings/mail/etc
* Some fun with what turned out to be a known bug in the
linaro-media-create now used in main LAVA instance
* Educated guess as to why Jenkins-triggered jobs produced no bundle
** Raised a ticket
** Think I know how to fix on my side


=Plan=

If updated builder becomes available, convert uinstance jobs to use it
Test release benchmark job
Finish backport benchmarking
Document Jenkins interface
Assess target stability
More image reports in microinstance
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 1st - 5th February

[Bernie Ogden]

Automated release benchmark Jenkins job - TCWG-348 [2/10]
* Drafted a job, it succeeds in dispatching to the uinstance
* Needs testing

Port to microinstance - TCWG-432 [5/10]
* Almost entirely fixing up the CPU2006 -> LAVA reporting
* Should now be able to report CPU2000 results as well

Misc [3/10]
* Mail/meetings/etc
* Connect slides


=Plan=

If builder is available, convert uinstance jobs to use it
Test release benchmark job
Produce image reports in microinstance
Attempt to finish backport benchmarking
Actually finish documentation
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 18th - 22nd January

[Bernie Ogden]

Port to microinstance - TCWG-432 [2/10]
* More fallout from attempts to fix race condition
* Began cleaning job submission YAML

Controlled image builds - TCWG-360 [1/10]
* (Re)learned how to generate images from hwpack + filesystem

Struck down by flu [7/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 25th - 29th January

[Bernie Ogden]

Port to microinstance - TCWG-432 [4/10]
* Went a few more rounds with the race condition, finally nailed it
* Tidied up transfer of Coremark results to LAVA
* Added metadata to LAVA results
* Sketched some charts using LAVA reporting
* Improved runtime benchmark logging

Benchmarking infrastructure documentation - TCWG-496 [2/10]
* Most everything drafted, some docs finished

Controlled image builds - TCWG-360 [2/10]
* Reached a better understanding of what a hwpack is
* Generated a few images, none of them booted

Misc [2/10]


=Plan=

Benchmarking-102 slides for Connect
Finish documentation
Backport benchmarking
If builder shows up in microinstance, evaluate stability of its Junos
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 11th-15th January

[Bernie Ogden]

Port to microinstance - TCWG-432 [3/10]
* Fallout from attempts to fix race condition
* Various minor fixes - simplifications, better reporting

Backport benchmarking - TCWG-352 [1/10]
* Decoupled 'target triple' from 'toolchain' name
** Immediately, to let me benchmark with Juno-built native gcc
** But this has been causing minor pain for a while

Benchmarking infrastructure documentation - TCWG-496 [1/10]
* Finished drafts of LAVA wrapper and microinstance access docs

Unexpected 1/2 day off [1/10]

Misc [4/10]
* Unusual number of meetings
* Fiddling with my git workflow
* Some thinking about Benchmark-102 for Connect
* Figured out why our SPEC objects were disappearing
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 4th-8th January

[Bernie Ogden]

Centralized benchmark source - TCWG-354 [1/10]
* Fixed some bugs in Coremark reporting
* Experimented some more with clang builds, set this aside for now

Port to microinstance - TCWG-432 [3/10]
* Fixed some false assumptions exposed by change to uinstance
* Worked on LAVA side of coremark reporting

Backport benchmarking - TCWG-352 [4/10]
* Fixed some bugs that appeared over the holiday
* Got job dispatching benchmark run to microinstance, with pointer to
backport-containing toolchain
* But resulting job fails

Benchmarking infra documentation - TCWG-496 [1/10]
* Started documenting LAVA wrapper

Misc [1/10]


=Plan=

LAVA side of coremark reporting
Fix backport-dispatched benchmark job
Much more documentation
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: gcc-linaro-5.1 vs gcc-linaro-4.8

[Bernie Ogden]

Hello,

I'm not sure from the information below whether you have observed a
performance gap, or are expecting to observe one. Have you seen a
performance gap?

Regards,

Bernie

On 5 January 2016 at 10:29, Xiaofeng Ren  wrote:
> Hello All,
>
> I found one difference between gcc-linaro-5.1 vs gcc-linaro-4.8 while I’m
> doing lmbench benchmark test for our LS1043 (cortex-A53).
>
> While using gcc-linaro-4.8, gcc will generate advanced SIMD instructions
> (like as ld1, etc), however, gcc-linaro-5.1 will not generate advance SIMD
> instructions. This will cause big performance gap between gcc-4.8 and
> gcc-5.1 for lmbench memory bandwidth “fcp” test (bw_mem program).
>
>
>
> My compiler flags is “-O3 -mcpu=cortex-a53”. I also tried several different
> compiler flags (“-O3 -mcpu=cortex-a53+fp+simd”, “-O2 -ftree-vectorize
> -mcpu=cortex-a53”,  “-O3 -ftree-vectorize -mcpu=cortex-a53”), all of them
> doesn’t work.
>
>
>
> Gcc-5.1 toolchain was downloaded from following link:
>
>
>
> https://snapshots.linaro.org/openembedded/sources/gcc-linaro-5.1-snapshot-2015.06-1-x86_64_aarch64-linux-gnu.tar.xz
>
>
>
> Can I have your comments on this?
>
>
>
>
>
> Thanks
>
> Ron
>
>
> ___
> linaro-toolchain mailing list
> linaro-toolchain@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/linaro-toolchain
>
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 7th-11th December

[Bernie Ogden]

Centralised benchmark source - TCWG-354 [6/10]
* Understood CoremarkPro run rules/behaviour better
* Experimented with shortening runs while still giving meaningful results
* Cleaned up build/run scaffolding

Port to microinstance - TCWG-432 [1/10]
* Got access to microinstance, learned a bit about SSL in the process

Backport benchmarking - TCWG-352 [2/10]
* Fixed 'build with sysroot' code for current deliverable shape
* Fixed a problem with multiple manifests


=Plan=

Review, test, debug build-triggers-benchmark job
Finish off Coremark Pro integration
Review security with shared uinstance/main instance code
Expose more data, benchmarks to bundles
Debug/test Jenkins job in microinstance
Create bootable image for at least 1 target, or know what the problems are
Write up noise control report (if time)
More support for SPEC-on-Android?


=Absence=

Holiday 22nd December - 1st January
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 30th November - 4th December

[Bernie Ogden]

Holiday [2/10]

Port to microinstance - TCWG-432 [1/10]
* Set up reporting for CPU2006
* Learned how to generate metadata, but not how to use it

Trigger benchmarks on backports - TCWG-352 [2/10]
* Figured out the rough shape
* Created, didn't test, rough implementation

TCWG-354 [3/10]
* Build/run scaffolding for CoreMark Pro
* Working for manual runs

Misc [2/10]
* Input validation for dispatcher script
* Meeting with Ade on LAVA benchmarking
* Meetings/mail/etc


=Plan=

Review, test, debug build-triggers-benchmark job
Check CoreMark Pro run configuration, enable for automatic runs
Review security with shared uinstance/main instance code
Expose more data, benchmarks to bundles
Debug/test Jenkins job in microinstance
Create bootable image for at least 1 target, or know what the problems are
Write up noise control report (if time)
More support for SPEC-on-Android?
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 23rd - 27th November

[Bernie Ogden]

Controlled image builds - TCWG-360 [1/10]
* Tried, failed to generate bootable images

Jenkins benchmarking job - TCWG-348 [5/10]
* Jenkins job functional on kvms in main instance
* Wrote job-dispatch script for non-Jenkins use cases

Juno crashdump [1/10]
* Struggled with, worked around network problems
* Juno running, waiting for it to crash

SPEC-on-Android [1/10]
* Helped Qian to find root cause of a problem

Misc [2/10]


=Plan=

Review security with shared uinstance/main instance code
Expose more data, benchmarks to bundles
Debug/test Jenkins job in microinstance
Create bootable image for at least 1 target, or know what the problems are
Write up noise control report (if time)
Probably more support for SPEC-on-Android
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 16th - 20th November

[Bernie Ogden]

Controlled image builds - TCWG-360 [2/10]
* A few more test/debug cycles with ci-loop-built image

Jenkins benchmarking job - TCWG-348 [3/10]
* YAML-ised Jenkins job, more test/debug cycles

Juno crashdump [1/10]
* Got a usable dump (via alt-sysrq-c) with latest patches plus some fiddling

SPEC-on-Android [1/10]
* Looked at Qian's work to date, didn't come up with any bright ideas

Misc [3/10]


=Plan=

Review security with shared uinstance/main instance code
Expose more data, benchmarks to bundles
Continue debug/test of Jenkins job
Create bootable image for at least 1 target, or know what the problems are
Write up noise control report (if time)
Set Juno off, try to get a dump of my crash
Probably more support for SPEC-on-Android


=Absences=

'ARM Day' next Monday (30th)
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 9th-13th November

[Bernie Ogden]

Implement LAVA jobs for microinstance - TCWG-432 [6/10]
* Refactoring to permit sharing of code between uinstance & main
instance, as far as possible
* Further refactoring for sane submission of bundles without inserting
LAVA assumptions in the wrong places
* Tested as far as possible in main instance, using light hacks and fakebench

Jenkins benchmarking job - TCWG-348 [1/10]
* Converted pbl hacks into a sane patch for yaml-to-json.py

Controlled image builds - TCWG-360 [1/10]
* Submitted aarch64 filesystem build for review
* Generated armhf and amd64 filesystems
* Started learning how to generate hwpack

Misc [2/10]


=Plan=

Review security with shared uinstance/main instance code
Expose more data, benchmarks to bundles
Create YAML definition for Jenkins benchmarking job
Generate (controlled) hwpack for at least one target, or know what the
problems are
Write up noise control report (if time)
Have another at crashdump (if time, if new kexec patches)
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 2nd - 6th November

[Bernie Ogden]

Holiday [2/10]

Juno crash analysis [2/10]
* Spent some time fiddling with kexec on AArch64
* Worked in one very specific case
* Another patch series is (apparently) coming, will look out for it
and try again

SPEC-on-Android [2/10]
* Supporting Qian on getting this working
* Wrote a readme for the repository, fixed a Makefile bug that Qian's
cross-compiler happened to tickle

Jenkins benchmarking job - TCWG-348 [1/10]
* Tested, tidied up pbl hacks to generate JSON
* Tested my pbl with Jenkins prototype jobs
* A few minor bug fixes/enhancements for pbl

LAVA jobs for uinstance - TCWG-432 [1/10]
* Reworked jobs to support uinstance, maintaining backward
compatibility as far as possible
* Started adding support to submit results to bundle stream

Misc [2/10]
* Debian FS ready to submit
* Usual meetings/mail/etc background


=Plan=

Look at doing pbl hacks properly in Fathi's in-development refactored p-b-l
Pull together Jenkins/LAVA/pbl, ready to test when uinstance is available
Write up noise control report
(If time, if patches land) have another go at crashdump
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 26th - 30th October

[Bernie Ogden]

Noise control experiments - TCWG-358 [3/10]
* Some analysis of data to date

Debian filesystem - TCWG-360 [3/10]
* Got stuck on LAVA interactions
* Now booting-to-LAVA-usability, needs some cleanup and testing with
real benchmark runs

Benchmarking-via-Jenkins - TCWG-348 [1/10]
* Picked back up on understanding that LAVA uinstance is a-coming
* Hacked pbl.py (post-build-lava) to generate suitable JSON
** As a bonus, this can work as a CLI job-submission tool


=Plan=

Holiday Friday (pending approval)
Set up crashdumping on my Juno, try to learn why it crashes
Finish Debian filesystem
Get Jenkins generating and submitting jobs suitable for uinstance
Write up noise control report (probably will get bumped to next week)
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 19th - 23rd October

[Bernie Ogden]

Mustang benchmarking bringup - (no ticket) [2/10]

Controlled Debian build - TCWG-360 [1/10]
* Constructed filesystem, works OK as chroot
* Next step is to boot it

Investigate Workload Automation framework - TCWG-361 [1/10]
* Seems to basically work
* SPEC support dubious

Misc
* LAVA uinstance background (TCWG-396, < 1/10)
* Noise control experiments (TCWG-358, < 1/10)
* ARM management duties [2/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 12th - 16th October

[Bernie Ogden]

Investigate effectiveness of noise-control measures - TCWG-358 [4/10]
* Recovered from a couple more crashes, existing logging sheds no light
* Explored the existing data a bit, need more to draw any conclusions

Controlled image builds - TCWG-360 [2/10]
* Looked into controlled, repeatable debian builds, B
* Got the general shape of it, checking my understanding with Fathi

LAVA microinstance - TCWG-396 [1/10]
* Sent doc for review, revised it a bit

Misc [3/10]
* Exported benchmark/results handling rules to a place other
Linaro-ans can read them
* Updated example LAVA jobs to use tcwg tag (for load balancing purposes)
* Meetings, mail, etc


=Plan=

Generate more noise data, watching out for crashes
Produce a candidate Debian image, document how I did it
If time, learn about Workload Automation
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 4th - 9th October

[Bernie Ogden]

LAVA uinstance for benchmarking - TCWG-396 [3/10]
* Long meeting with Renato + LAVA/lab people
* Much thinking about the security side
* Wrote up rough draft of what I understand the design to be

Investigate effectiveness of noise control measures - TCWG-358 [3/10]
* Set up new host node
* Dealt with one host node crash, one target crash
** Noticed that target was writing logs to tmpfs, changed that so I
can investigate next crash
* Both sides stayed up through the weekend, hurrah
* Some work on data-massaging scripts

Misc [4/10]

Heavier than usual week mail-wise, plus performance review prep


=Plan=

Clean up uinstance design, circulate to make sure we're all on same page
Look at data from experiments so far
Investigate B's Debian-building tools
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 28th August - 2nd October

[Bernie Ogden]

Connect recovery - [2/10]

Investigate effectiveness of noise-control measures - TCWG-358 [3/10]
* Host node crashed, tried to recover it, failed, started building a new one

Jenkins automation - TCWG-348 [3/10]
* Everything in place up to first interaction with LAVA kvm
* Considered LAVA team's micro-instance proposal

Misc [2/10]


=Plan=

Discuss micro-instance proposal
Work on whatever comes out of discussion
Finish building new host node, get experiments running again
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 14th - 18th September

[Bernie Ogden]

Investigate effectiveness of noise-control measures - TCWG-358 [6/10]
* Finished setup, started running experiments
* Initial data pretty noisy, more runs needed
* Kicked off more runs for week of Connect

Ensure that all critical data is logged - TCWG-349 [1/10]
* Logged a few more variables, tested, merged
* Put down until we have a controlled image build

Misc - [3/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 7th-11th September

[Bernie Ogden]

Ensure all critical (benchmarking) data is logged - TCWG-349 [1/10]
* Added logging of several factors
* Documented what we do and don't log

Noise control experiments on Juno - TCWG-349 [4/10]
* Rediscovered that my Juno is an r0, not an r1
* Rebuilt target image in a carefully scripted way
* Fiddled about with differences between local and LAVA targets
* Ran a couple of trials to test infrastructure

Connect preparation - [2/10]
* General sorting out of tickets et al
* Final pass through benchmarking presentation

Misc - [3/10]


=Plan=

Merge 'logging' branch
Final pass through logging documentation
Run some noise experiments on Juno
If time, work on Jenkins benchmarking jobs
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 31st August - 4th September

[Bernie Ogden]

Holiday [4/10]

Multinode wrapper - TCWG-350 [2/10]
* Merged to benchmarking branch, roughly documented
* Tested & added job definition templates

Setting up VPN [2/10]
* Much struggling with mutt, pgp, VM vs real system, Mac vs Linux
* Still not working, but wrote up what I've learned on the Collaborate page

Misc [2/10]
* ARM admin, meetings, mail, etc etc


=Plan=

Last pass through benchmarking presentation
Finish multinode/updating benchmarking docs
Back to Juno noise control experiments
Back to Jenkins - get it to drive multinode
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 17th - 21st August

[Bernie Ogden]

Upcoming Absences
* Away from this Wednesday until next Tuesday, inclusive

Benchmark infrastructure - TCWG-360 [1/10]
* Finished filling out cards/bugs for benchmarking work
* Which makes the rest of this report more fine-grained

Multinode wrapper - TCWG-888 [2/10]
* Completed oversubscription workaround
* Herded most tests through
* Uncovered one significant bug, seems fixed

Centralized source/results storage - TCWG-722 [1/10]
* Wrote draft of source/results handling rules
* Confirmed that old repos are subsets of new repos

Noise control experiments - TCWG-897 [2/10]
* Learned more about firmware, openembedded
* Juno running a minimal filesystem, needs some tidying up

Misc [4/10]
* Including a further 1/10 of ARM management


=Plan=

* Herd remaining multinode tests through
** Depends on LAVA lab coming back from power cut
** And queues being short enough
* Progress (finish?) Juno bring-up
* Share source/results rules draft
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 10th - 14th August

[Bernie Ogden]

Benchmark infrastructure - TCWG-360 [6/10]
* Fixed remaining known critical issues in multinode
** Testing stymied by LAVA oversubscription
** Attempted to implement some workarounds for oversubscription
* Some discussion/investigation on generating filesystems for benchmarking
* Started brain-dumping state of infrastructure into cards.linaro.org

Bringing up Juno for noise control experiments [1/10]
* Some struggles with firmware/bootloader configuration

Misc [3/10]

Featuring some ARM management duties


= Plan =

* Get benchmarking tests through, if LAVA permits
* Fix any critical issues discovered
* Finish brain-dump into cards.l.o
* Bring up Juno for noise control experiments
* Some more ARM management
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 27th - 31st July

[Bernie Ogden]

Benchmark infrastructure - TCWG-360 [8/10]
* Testing found many problems in multinode
* Iterating to solutions

Misc [2/10]


=Plan=

Holiday next week.

Then back to fixing multinode, incorporating into jenkins, noise
control experiments
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 13th - 17th June

[Bernie Ogden]

Benchmark infrastructure - TCWG-360 [5/10]
* Worked through my Jenkins issues with Fathi, raised some tickets at him
* Converting LAVA end into multinode job
** Having some trouble with multinode API

Benchmarking 101 presentation [3/10]
* 1/2 day of discussions/reading, full day of redrafting
* Looked for Michael Hope's similar 2012 presentation
** Found slides, not video


=Plan=

* Complete multinode job
* Integrate into Jenkins to the extent that I'm comfortable
* Complete 'shareable' draft of benchmarking-101
** And see if I have enough left over for -102, maybe -103
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 6th - 10th June

[Bernie Ogden]

Benchmark infrastructure - TCWG-360 [5/10]
* More thinking/prototyping sufficiently-secure Jenkins benchmarking
* Converting LAVA end into multinode job

Benchmarking presentation [2/10]
* A couple of helpful discussions
* Read a couple of helpful docs

Misc [3/10]


=Plan=

* Complete multinode job
* Settle on a plan for Jenkins
* Redraft presentation
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 22nd - 26th June

[Bernie Ogden]

Benchmark automation - TCWG-360 [3/10]
* Created a partial Jenkins prototype
* Considered some security issues

Benchmarking presentation [5/10]
* Drafted some slides, did some reading

Misc [2/10]


=Plan=

More of the above
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 15th-19th June

[Bernie Ogden]

Benchmark automation - TCWG-360 [7/10]
* Arndales stopped booting
** Package servers for elderly filesystem had gone
** Investigated some approaches to creating more stable filesystems
** Realized I could just updated image to point at old-releases, so
did that for now
* _More_ time thinking about interactions with Jenkins  LAVA. Fathi
gave me some Jenkins jobs to prototype in.
* Brain-dumped some of the present state of things into Collaborate

Misc - [3/10]


=Plan=

Jenkins prototyping
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 1st-5th June

[Bernie Ogden]

Holiday [2/10]

Benchmark Automation - TCWG-360 [4/10]
* Learned about central Jenkins instance
* Considered ways of integrating benchmark scripts with said instance
* Considered alternative way for benchmark scripts to interoperate with LAVA

Misc [4/10]
* Large mail backlog
* Travel prep

=Plan=

Team sprint
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 11th - 15th May

[Bernie Ogden]

Benchmark automation - TCWG-360 [7/10]
* Convinced benchmark scripts to run entirely within LAVA
* Did a couple of benchmark runs for Kugan
* Cleaned up my CPU2000 patches and committed to central repository

Misc [3/10]


=Plan=

Clean up, commit CPU2006 patches
Pick a direction on streaming vs listeners, write down the reasoning
Follow up (close out?) results storage and restricted targets
Write more tests, refactoring where appropriate
If time, and it looks sane, wrap a LAVA job around benchmark scripts
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 4th - 8th May

[Bernie Ogden]

(Public) Holiday [2/10]

Benchmark automation - TCWG-360 [6/10]
* More unit tests
* Refactoring - test writing is an effective spotlight
* Experimented with LAVA log streaming as a 'listener' replacement
** Reduces complexity, for (possibly insignificant) costs in
generality and scalability
* Gave some thought to results storage, benchmarking of restricted targets

Misc [2/10]


=Plan=

Pick a direction on streaming vs listeners, write down the reasoning
Follow up (close out?) results storages and restricted targets
Write more tests, refactoring where appropriate
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 27th April - 1st May

[Bernie Ogden]

Holiday [2/10]

Benchmark automation - TCWG-360 [5/10]
* Finished rolling backbench patches into review
* Submitted some test suite patches to reduce the number of untested cases
* Began writing tests for benchmarking in general

Misc [3/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 20th - 24th April

[Bernie Ogden]

benchmark automation - TCWG-360 [3/10]
* Shifted more generic patches from backport branch to review
* One complete run of backport benchmarking before lab went down

Misc [7/10]
* Most of this (4 or 5 tenths) is reaction to lab downtime


=Plan=

Benchmarking test suite
Fish out, finish off thoughts about benchmark access control


NB Next Monday (4th May) is a public holiday in UK
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: Linaro Toolchain Product Release Process Changes (Effective 2015.04)

[Bernie Ogden]

 TCWG will introduce the concept of a 'release candidate source archive' and
 'release candidate binary toolchain archive' six weeks before the projected
 release date of the Quarterly Binary Toolchain Release.  For example, the
 2015.05 Quarterly Binary Toolchain and Quarterly Source Archive will be
 based upon the 2015.04 source archive snapshot.

 Release candidate source and binary archives will be posted to
 snapshots.linaro.org.

So the 'release candidate binary toolchain archive' will be in the
form of source, or of binary tarball? My guess would be that, if
binary toolchain users are going to look at a prerelease at all,
they'll only do so if it's already built for them.
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 13th - 17th April

[Bernie Ogden]

Benchmark automation - TCWG-360 [5/10]
* Flushed the rest of the easy stuff from my 'small fixups' to my
'staging' branch
* Started rolling generic patches from my backport-benchmark branch into gerrit
* Prodded at the prototype backport-benchmark job until the pieces worked
** Ran out of time to try an end-to-end run
* Thought a little about permissions for access to source/results
* Use of nc.traditional appears to have introduced a race condition

catomics - TCWG-436 [2/10]
* Tried a run on A15 - same story as A57, good improvement on g8 libc
ubenchmark that doesn't translate into SPEC subset
* Performance counting the ubenchmark suggests variation around cache
accesses on the catomic vs not-catomic code, didn't have any time to
think about what, if anything, this means

Misc [3/10]
* Featuring an unusual level of ARM interruption
* Especially concerns about change from multiarch to non-multiarch
sysroot in binary releases


=Plan=

Confirm backport benchmarking working, roll remaining generic patches
into gerrit
Investigate the nc.traditional race
Add a retry loop around LAVA boot (occasionally see LAVA-fail here)
Create unit tests
Follow up on (non-)multiarch sysroot issue
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 30th March - 3rd April

[Bernie Ogden]

catomics - TCWG-436 [4/10]
* Fiddled catomics to not use atomics at all, set off a new SPEC run
* Not using atomics seems wrong in general, but OK for malloc/free

Benchmark automation - TCWG-360 [1/10]
* Quick look at Tyler's LAVA console streamer
* Ported benchmark scripts to work on Debian
** Turns out there are two versions of netcat and Debian and Ubuntu
default to different ones
* Checked lab transition plan for benchmark-affecting points

Misc [1/10]

Holiday [4/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 5th - 10th April

[Bernie Ogden]

catomics - TCWG-436 [4/10]
* Fiddled with SPEC runs with unstripped binaries on tiny Juno discs
* Got a run for the interesting subset of SPEC, non-atomic catomics
still show no effect
* Tried a few catomics variations with the glibc malloc
microbenchmark, here non-atomic catomics show a significant effect
* Tried firing a perf shotgun at the malloc microbenchmark to see if
it suggests anything - results pending

Benchmark automation - TCWG-360 [3/10]
* Fixed Jira benchmarking
* Flushed some of the backlog out of my 'small fixups' branch

Misc [1/10]

Holiday [2/10]
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 23rd - 27th March

[Bernie Ogden]

catomics - TCWG-436 [5/10]
* Got pointed at a suitable set of benchmarks, results still underwhelming
* However, patches were using relaxed atomics rather than no atomics at all
* Fiddled abe into building sysroots for me (I get libstdc++ that way)

Misc - [5/10]
* Tidied up some 'perf shotgun' scripting from the juno cache
investigation, so I've got the tools for next time
* Started sorting out my backups - but didn't finish before build-01's
death destroyed a bunch of work
* Raised priority of sorting out my backups, now just a matter of
waiting on some large rysncs
* Pieced my world back together on dev-01

=Plan=

Holiday Wednesday, public holidays next Friday and Monday
See how catomics do when we're conditionally-not-atomic-at-all
Investigate a bit to see if I can see if there's a reason we were
using relaxed atomics
Resurrect Jira benchmarking on dev-01
 * Will include some porting, scripts don't work out of box on dev-01
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: [ACTIVITY] 9th - 13th March

[Bernie Ogden]

Urgh - I filled in the tenths from the wrong week. For project
management, the correct numbers are:
LDTS-1238 [4/10]
TCWG-436 [1/10]
TCWG-360 [1/10]
Misc [4/10] (about 1/2 of this was juno-01 fixing)

On 23 March 2015 at 08:57, Bernie Ogden bernie.og...@linaro.org wrote:
 Juno cache effects - LDTS-1238 [6/10]
 * Seems to be mainly due to (expected) instruction scheduling
 limitations, and prefetcher effects
 * Reported back, hopefully this will wrap up now

 catomics - TCWG-436 [1/10]
 * Shepherding benchmark runs in LAVA, usual problems with ssh-agent,
 juno contention and random target failure
 * Almost no actual data produced

 benchmark automation - TCWG-360 [1/10]
 * User support, some discussion about extent of our juno usage
 * Something weird happened in Jenkins, _might_ have been a one-off due
 to slaves moving around

 Misc - [2/10]
 * Featuring juno-01 fixing

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 16th - 20th March

[Bernie Ogden]

catomics - TCWG-436 [6/10]
* Started a series of runs on a local board I'd borrowed
** Then had to give it back before they'd really got anywhere
* Got some, possibly dubious, results back from A15 from previous week
** If the results are worth anything, they suggest that catomics don't
achieve anything
* Started again with a subset of SPEC on juno-01, as it was on my desk
for the weekend anyway
** Results again underwhelming
** Maybe I picked the wrong subset, maybe A57 is too smart

Misc [4/10]
* Including a little 'juno cache effects' followup, a little juno-01
work, and a lot of mail catchup


=Plan=

* Get back to benchmark automation
** Apply a bunch of small improvements I've got on a branch
** Get a working Jenkins backport benchmarking prototype
** Sort out sources/results storage

* Think about why catomics may not be showing any effect
** Starting to believe that this is a red herring
** But might be interesting to try 'little' class cores
** But that does involve finding a reliable target I can hold for a long time

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 9th - 13th March

[Bernie Ogden]

Juno cache effects - LDTS-1238 [6/10]
* Seems to be mainly due to (expected) instruction scheduling
limitations, and prefetcher effects
* Reported back, hopefully this will wrap up now

catomics - TCWG-436 [1/10]
* Shepherding benchmark runs in LAVA, usual problems with ssh-agent,
juno contention and random target failure
* Almost no actual data produced

benchmark automation - TCWG-360 [1/10]
* User support, some discussion about extent of our juno usage
* Something weird happened in Jenkins, _might_ have been a one-off due
to slaves moving around

Misc - [2/10]
* Featuring juno-01 fixing

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 2nd - 6th March

[Bernie Ogden]

Juno cache effects - LDTS-1238 [6/10]
* Ran more experiments
* Cobbled together some gdb/python script to run perf stat within an
address range
* Becoming more confident in my hypotheses

catomics - TCWG-436 [1/10]
* Found and fixed some more sysroot benchmark bugs
* Kicked off a bunch of spec runs, waiting for the results to come back

Benchmark automation - TCWG-360 [1/10]
* A couple of post-lab-maintenance fixups
* Took a first look at result consistency with a broken run intended
for catomics

Misc - [2/10]
* Featuring the start of fixing juno-01


=Plan=

Finish fixing juno-01
Carry on looking at LDTS-1238, hopeful of confirming hypotheses
Analyse catomics results (assuming that they land)
Carry on with Jenkins, if time

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 23th - 27th Februrary

[Bernie Ogden]

Holiday Friday [2/10]

catomics - TCWG-436 [2/10]
* Fixed broken benchmark-against-sysroot code in scripts
* Built a lot of sysroots

Juno cache effects - LDTS-1238 [4/10]
* Quite a bit of reading and learning some more perf and other tools
* Formed a few hypothesis, started testing
* Looking promising, but I find these things often go wrong, so we'll see

Misc [2/10]
* Featuring first successful Jenkins run of all of SPEC2006


=Plan=

Make sure I've really fixed the last sysroot bug, actually run
benchmarks on catomics
Attempt to close out the cache effects issue
Carry on with Jenkins

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: [ACTIVITY] 16th - 20th February

[Bernie Ogden]

Oh -- and I'm on holiday this Friday (27th).

On 23 February 2015 at 10:12, Bernie Ogden bernie.og...@linaro.org wrote:
 Connect detox [2/10]

 ABE benchmarking automation - TCWG-360 [5/10]
 * Backport benchmarking
 ** Harder than expected, have to build the backport into a binary,
 then benchmark that binary, passing information between jobs.
 ** AFAIK I'm blazing the Jenkins-chainging trail for us.
 * Release benchmarking
 ** New LAVA release broke my image (but quickly fixed by the LAVA folks)
 ** Now seems to be running into issues with Jenkins/builders/an
 anonymous job-slayer
 * Misc fixes and improvements
 ** Main highlight being support for building with separated-out sysroot

 Misc - [3/10]


 =Plan=

 Kick off catomics benchmark runs
 (Attempt to) close out cache effects on Juno issue
 Carry on with Jenkins


 =Issues=

 * Some of the Jenkins builders seem unreliable
 * My lightly hacked version of the BinaryRelease job is unreliable
 ** Is the BinaryRelease job itself is reliable

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 16th - 20th February

[Bernie Ogden]

Connect detox [2/10]

ABE benchmarking automation - TCWG-360 [5/10]
* Backport benchmarking
** Harder than expected, have to build the backport into a binary,
then benchmark that binary, passing information between jobs.
** AFAIK I'm blazing the Jenkins-chainging trail for us.
* Release benchmarking
** New LAVA release broke my image (but quickly fixed by the LAVA folks)
** Now seems to be running into issues with Jenkins/builders/an
anonymous job-slayer
* Misc fixes and improvements
** Main highlight being support for building with separated-out sysroot

Misc - [3/10]


=Plan=

Kick off catomics benchmark runs
(Attempt to) close out cache effects on Juno issue
Carry on with Jenkins


=Issues=

* Some of the Jenkins builders seem unreliable
* My lightly hacked version of the BinaryRelease job is unreliable
** Is the BinaryRelease job itself is reliable

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: [ACTIVITY] 19th - 23rd January

[Bernie Ogden]

Correction - that was 26th - 30th January.

On 2 February 2015 at 14:34, Bernie Ogden bernie.og...@linaro.org wrote:
 ABE benchmarking automation - TCWG-360 [8/10]
 * Pursued sneaky bugs through twisty mazes of bash
 * Learned that ControlMaster does not play nice with my scripts
 * Learned more about IPC pitfalls
 * Cleaned up build behaviour of scripts somewhat
 * Implemented support for benchmarking against sysroot (needed by Will
 and by me)
 * Generally tried to help Charles
 * A bit of Jenkins testing

 Misc [2/10]

 * Featuring part 1 of an office move


 =Plan=

 Holiday Wednesday, 1/2 of Thursday
 Part 2 of office move
 Support benchmark users
 Look more at cache effects on Juno
 Get Jenkins working
 Benchmark catomic patches (as much as I can without putting even more
 pressure on the Junos)
 Write up/think about where to go with libm exercising
 Final preparation for Connect

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 2nd - 13th February

[Bernie Ogden]

Holiday [3/10]
ABE benchmarking automation - TCWG-360 [3/10]
* Initial Jenkins implementation
Investigating cache effects on Juno - LDTS-1238 [2/10]
* One effect due to write streaming, others still to look at
Misc [2/10]
* Unpacking, some background catomics

Connect [10/10]

=Plan=

Carry on with Jenkins implementation
Run some benchmarks on catomics
Juno cache effects
Write up/think about where to go with libm exercising

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 19th - 23rd January

[Bernie Ogden]

ABE benchmarking automation - TCWG-360 [8/10]
* Pursued sneaky bugs through twisty mazes of bash
* Learned that ControlMaster does not play nice with my scripts
* Learned more about IPC pitfalls
* Cleaned up build behaviour of scripts somewhat
* Implemented support for benchmarking against sysroot (needed by Will
and by me)
* Generally tried to help Charles
* A bit of Jenkins testing

Misc [2/10]

* Featuring part 1 of an office move


=Plan=

Holiday Wednesday, 1/2 of Thursday
Part 2 of office move
Support benchmark users
Look more at cache effects on Juno
Get Jenkins working
Benchmark catomic patches (as much as I can without putting even more
pressure on the Junos)
Write up/think about where to go with libm exercising
Final preparation for Connect

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 12th - 16th January

[Bernie Ogden]

ABE benchmarking automation - TCWG-360 [5/10]
* Converted Juno runs from OE to Ubuntu
* Fixed cross-compilation in SPEC2000
* Picked up a couple of users, spent some time supporting them
* Discussion about storage

catomics - TCWG-436 [3/10]
* Took a remarkably long time to make the last few patches build
* Looked through the generated code
* Learned some of the ways of test-schroot

Misc - [2/10]


=Plan=

Support Charles' release benchmarking
Try Jenkins again, now that release has gone
Carry on, hopefully close, storage discussion
Benchmark catomic patches
Write up/think about where to go with libm exercising

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 4th - 9th January

[Bernie Ogden]

ABE benchmarking automation - TCWG-360 [8/10]
* Added some sanity checks, fixed some bugs in LAVA URL handling
* Some struggling with Jenkins, with support from Rob
* Compress builds before scp-ing
* Resurrected SPEC, fixed some problems I'd forgotten about

Misc [2/10]

* Email catchup, start-of-year meetings
* Brief look at single-thread optimization in glibc (TCWG-436)
* Connect prep (including immunization boosters, I'm still a little woozy)


=Plan=

Test SPEC with present state of automation
Keep struggling with Jenkins
Address benchmark storage
Actually do something with TCWG-436
Write up/think about where to go next with libm exercising

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 15th - 19th December

[Bernie Ogden]

ABE benchmarking automation - TCWG-360 [6/10]
* TCWG/LAVA lab interactions working with just a couple hacks
* Jenkins job created (thanks Rob) and beginning to work

Misc - [4/10]
* Included a quick  look at Will's single-thread performance patches -
unfortunately, no time to actually do anything with them


=Plan=

Benchmarking storage/automation
Look though Will's single-thread performance patches
Switch to TCWG Junos
Write up/think about where to go next with libm exercising
Test benchmark repeatability


=Absences=

On holiday Monday 22nd Dec to Friday 2nd Jan

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 8th - 12th December

[Bernie Ogden]

ABE benchmarking - TCWG-360 [7/10]
* Finished post-move LAVA lab workarounds
* Should be flexible enough for TCWG lab too, but we'll see
* Refactored sources - now more robust and maintainable

lowlevellock.h comments - CARD-341 [1/10]
* A further round of changes, got an OK

Raised a bug on GCC re questionable generation of pow calls - TCWG-558 [0/10]
* Upstream fixed it really fast

Misc - [2/10]


=Plan=

Switch to TCWG Junos, perhaps start using gprof
Think about where to go next with libm exercising
Look though Will's single-thread performance patches
Test benchmark repeatability
Port benchmarking scripts to ABE repo
Start on storage/automation, if Rob has time


=Absences=

On holiday Monday 22nd Dec to Friday 2nd Jan

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: renaming completed

[Bernie Ogden]

Oh, it's as easy as:

git push /home/bernie.ogden/src/abe bernie_benchmarking:bernie/benchmarking

Benchmarking branch is now in the abe repository at bernie/benchmarking.

I'll pester Rob about automation tomorrow ;)

On 20 November 2014 at 10:44, Bernie Ogden bernie.og...@linaro.org wrote:
 I'll need to move my benchmarking branch. How long is 'a while'?

 On 20 November 2014 07:40, Maxim Kuvyrkov maxim.kuvyr...@linaro.org wrote:
 On Nov 19, 2014, at 2:01 PM, Rob Savoye rob.sav...@linaro.org wrote:

  I've completed renaming Cbuildv2 to Abe, as well as modifying all the
 board files for remote testing to match what the new DNS server is
 using. Please stop using the existing cbuild2 repository, as all changes
 are now done in the new repository. I'll leave the old repository for a
 while, I know there are branches there people depend on. For those, I'm
 sorry for the pain (having just gone through that myself), but please
 migrate any important branches to the new repository. URLs for that are
 here:

https://git.linaro.org/toolchain/abe.git

 I will be merging schroot-test work into abe.git/master in the coming weeks. 
  I will not be adding a schroot-test branch to abe.git repo.

 Thanks,

 --
 Maxim Kuvyrkov
 www.linaro.org



 ___
 linaro-toolchain mailing list
 linaro-toolchain@lists.linaro.org
 http://lists.linaro.org/mailman/listinfo/linaro-toolchain

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 1st - 5th December

[Bernie Ogden]

Holiday [6/10]

Misc [3/10]
* Mail backlog
* Moved all current AArch64 work off 'my' Juno, as ARM needed it back
* A little bit of a look at another possible memcpy performance issue

libm exercising - TCWG-558 [1/10]
* Reduced 'needless calls to pow' to a simple test case
** Found that this is actually an all-targets thing (at least AArch32,
AArch64, x86)
* Looked through benchfft output
** Looks like only one implementation calls libm much
** This is probably just bad code, but could do with a comparison run
on non-AArch64 to be sure

=Plan=

Switch to TCWG Junos
Think about where to go next with libm exercising
Complete 'same network' workaround, test benchmark repeatability
Port benchmarking scripts to ABE repo
Get storage/automation started, if Rob has time

=Absences=

On holiday Monday 22nd Dec to Friday 2nd Jan

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 24th - 28th November

[Bernie Ogden]

ABE benchmarking - TCWG-360 [4/10]
* Implemented most of a solution to the 'must be in same network' restriction

libm exercising - TCWG-558 [4/10]
* lulesh generates needless calls to pow on AArch64 (as opposed to
'pow is slow')
** Working on a reduced test case
* Ran a chunk of benchfft, left a process searching the perf reports
for libm calls
* More chroot/glibc fiddling
* Decided Graph500 was unlikely to be interesting

Misc - [2/10]

=Plan=

On holiday Monday - Wednesday
More lulesh, benchfft results
Think about where to go next with libm exercising
Complete 'same network' workaround, test benchmark repeatability
Port benchmarking scripts to ABE repo
Get storage/automation started, if Rob has time

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

Re: renaming completed

[Bernie Ogden]

I'll need to move my benchmarking branch. How long is 'a while'?

On 20 November 2014 07:40, Maxim Kuvyrkov maxim.kuvyr...@linaro.org wrote:
 On Nov 19, 2014, at 2:01 PM, Rob Savoye rob.sav...@linaro.org wrote:

  I've completed renaming Cbuildv2 to Abe, as well as modifying all the
 board files for remote testing to match what the new DNS server is
 using. Please stop using the existing cbuild2 repository, as all changes
 are now done in the new repository. I'll leave the old repository for a
 while, I know there are branches there people depend on. For those, I'm
 sorry for the pain (having just gone through that myself), but please
 migrate any important branches to the new repository. URLs for that are
 here:

https://git.linaro.org/toolchain/abe.git

 I will be merging schroot-test work into abe.git/master in the coming weeks.  
 I will not be adding a schroot-test branch to abe.git repo.

 Thanks,

 --
 Maxim Kuvyrkov
 www.linaro.org



 ___
 linaro-toolchain mailing list
 linaro-toolchain@lists.linaro.org
 http://lists.linaro.org/mailman/listinfo/linaro-toolchain

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 10th - 14th November

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [5/10]
* Fixed some bugs and did some general tidying up
* Pulled SPEC2000 into my framework
* Did some test runs on local machines, looks promising

libm exercising - TCWG-558 [3/10]
* Much fiddling with one benchmark (MCB)
* Experimented, thought about methodology

Meetings/mail/etc - [2/10]


=Plan=

cbuild2 benchmarking
* Tweak eembc - it worked before, but spec forced me to change the
scripts a little
* Test in LAVA/new TCWG infrastructure when available
* Test repeatability (depends on above)
* Possibly have another go at building tools on AArch64

libm exerising
* Work through the most interesting benchmarks with a fixed method
* Hopefully reach the end and do some analysis

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 27th - 31st October

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [8/10]
* Scripts build SPEC 2006 tools on x86_64 and arm, not yet on AArch64
* Scripts cross-build for x86 to arm and aarch64
* Cross-built binaries refuse to run

Meetings/mail/etc - [2/10]


=Plan=

cbuild2 benchmarking
* Make cross-built binaries run, collect reports
* Build tools for AArch64 (lower priority)

libm exerising
* Work through list of benchmarks, see what I find

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 20th - 24th October

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [2/10]
* Figured out how to cook my own OE images
* Started remembering how to build spec

libm exercising - CARD-1693 [2/10]
* Borrowed a usable Juno
* Found that lapack tests segfault on AArch64
* Ran linpack hpl, didn't observe it exercising libm much
** Haven't ruled out Bernie-error yet, though

Upstream - CARD-341 [1/10]
* Respun lowlevellock.h comments


Meetings/mail/etc [5/10]
* A lot of time (3/10) in performance review and other annual ARM admin
* Some back and forth about difficulties of userspace access to ID registers


=Plan=

Get spec2006 working via my scripts
Find a good HPL libm exerciser
Spend a lot less time on ARM admin

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 13th - 17th October

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [3/10]
* A few small enhancements and bug-fixes
* Tried to run spec2006 on Juno, looks tricky

libm profiling - CARD-1693 [3/10]
* Pulled together lapack + blas
* Looked a bit at how it exercises libm on x86
* Tried to look at how it exercises libm on Juno, looks tricky

Meetings/mail/etc [4/10]
* Featuring some fun with backups and ARM performance review season


=Plan=

* cbuild2 benchmarking - keep chipping away at spec/juno
* libm profiling - attempt to assemble an AArch64 setup that works for me
* further performance review/corporate admin will take up extra time
this week, but should then fall back to normal levels


=Issues=

State of the setup on our Junos

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 6th - 10th October

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [7/10]
* Fixed/worked around some especially resilient bugs
** Mostly relate to running benchmarks through LAVA, which may be less
important in the near future
* Wrote a doc
* Upstreamed some code that works about as I want it to

Meetings/mail/etc [3/10]


=Plan=

cbuild2 benchmarking
* Work through TODO list from connect
* Tidy up behaviour on shutdown
* Storage issues
* (Maybe) persuade a LAVA Juno to work for benchmarking or configure spec

Hopefully pick up something else

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 29th September - 3rd October

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [7/10]
* Cleaned up and upstreamed some code that works
* Fixed a couple of issues raised by LAVA people
** No longer scraping the logs
** No longer assuming stable IP addresses

lowlevellock.h comments - CARD-341 [1/10]
* Respun based on Carlos' comments

Meetings/mail/etc - [2/10]


=Plan=

cbuild2 benchmarking
* Upstream code-with-fixes
* Write a doc
* Invite people to start using this thing
** Assuming that the storage questions get resolved
* Work through list of tweaks

Get up to speed on glibc vector math thread

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 8th - 26th September

[Bernie Ogden]

cbuild2 benchmarking - TCWG-360 [9/10]
* Knocked off a lot of rough edges
* Now working fairly robustly

lowlevellock.h comments - CARD-341 [4/10]
* Got a bit stuck trying to follow condvar locking
* But wasn't really needed to describe the code in question

LCA [10/10]

LCA recovery day [2/10]

Meetings/mail/etc [5/10]


=Plan=

cbuild2 benchmarking:
* Push upstream for others to look at
* Sort out storage story for benchmark sources and results
* Try to get working on a Juno (in LAVA, therefore on OE)
* Look at our Jenkins scripts with a view to figuring out how to hook in
* Work through list of bits 'n' pieces to fix -
  + Don't scrape logs (bad for server)
  + Don't fail on network timeouts
  + Extra target control knobs - disable ASLR, set a nasty nice value,
maybe hugepages
  + Set appropriate flags automatically for cross-builds

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 1st - 4th September

[Bernie Ogden]

=Progress=

cbuild2 benchmarking - TCWG-360 [8/10]
* Got an end-to-end run in my test environment
* Patched tests to run more selectively (in review)

Meetings/mail/etc - [2/10]


=Plan=

cbuild2 benchmarking:
* Clean up some lose ends
* Sort out source/results storage
glibc:
* Finish comments in lowlevellock.h (if time)

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 18th - 22nd August 2014

[Bernie Ogden]

=Progress=

cbuild2 benchmarking - TCWG-360 [5/10]
* Proposed a plan for benchmark storage
* A bit more thought on cross-running
* Prototyped script to launch/run/release on LAVA targets (e.g. wg)
** The 'run' part is pretty much independent of the 'LAVA' part

lowlevellock.h commentary -  [2/10]
* In response to a patch review, added some comments to this fairly obscure file
* Took longer than expected, but the end's in sight so I guess I'll finish
* I know a bit more about futexes and threads now

Meetings/mail/etc [3/10]


=Plan=

Holiday. Back on 1st September.

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 4th - 8th August 2014

[Bernie Ogden]

=Progress=

cbuild2 benchmarking- TCWG-360 [6/10]
* Existed patches tidied up and gerritified
* A few drive-by bugfixes
* Cross-building support added and also gerritified

lowlevellock patch [1/10]
* Resubmitted bugfix, reacted to review
* Some wrestling with space-mangling evil in mail clients

Meetings/mail/etc [3/10]


=Plan=

Holiday Tuesday afternoon

cbuild2 benchmarking
* Get remote targets working
* Maybe look at results gathering

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 28th July - 1st August 2014

[Bernie Ogden]

=Progress=

cbuild2 benchmarking - TCWG-360 [7/10]
* Integrated benchmarking more into the core of cbuild2
* 'Audited' the benchmarks lying around on toolchain64
* Persuaded cbuild2 to use git-over-ssh

Meetings/mail/etc [3/10]

=Plan=

cbuild2 benchmarking
* Convert eembc integration into reviewable patches
* Get cross-compilation working
* Get remote targets working

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] July 7th - 11th

[Bernie Ogden]

=Progress=

memcpy regression on A9 - TCWG-390 [6/10]
* Not down to the branch predictor after all
* Can be fixed for A9 with explicit pld, but this isn't free

cbuild2 benchmarking - TCWG-360 [1/10]
* Dug out my branch, merged in trunk, refreshed memory

Meetings/mail/etc [3/10]
* Including a little patch review for Roland's lowlevellock.h (TCWG-435)


=Plan=

Close TCWG-390 as wontfix
Lots more cbuild2 benchmark automation
Keep up with lowlevellock.h

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] June 30th - July 4th

[Bernie Ogden]

=Progress=

memcpy regression on A9 - TCWG-390 [7/10]
* Tweaked bench.py some more
* Cobbled together a bare metal version of cortex-strings benchmark
* Was surprised to find that the problem does not relate to Linux
* Have some evidence that points at the branch predictor

Meetings/mail/etc [3/10]


=Plan=

See if A9 performance is recoverable without sacrificing other targets
Push some of my improvements into the cortex-strings repo
Possibly write up my cortex-strings-benchmarking-in-lava method
Back to cbuild benchmark automation

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] June 23 - 27

[Bernie Ogden]

=Progress=

memset performance improvement - TCWG-156 [2/10]
* Bugfixed/improved cortex-strings-bench-on-lava

memcpy regression on A9 - TCWG-390 [6/10]
* Scanned through lots of data
* Learned some perf and some streamline
* Still don't know what's going on here

lowlevellock performance bugs - TCWG-435 [0/10]
* Stalled until someone reacts to my pings

Meetings/mail/etc [2/10]
* Including some questions about whether we need softfloat support in
binary releases


=Plan=

Keep prodding at memcpy with perf/streamline
Explore repeatability of cortex-strings benchmark
See how much quicker I can make cortex-strings benchmark without
compromising repeatability
Possibly try out a bare metal version of (a subset of) cortex-strings benchmark

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 16 - 20 June

[Bernie Ogden]

=Progress=

memcpy regression on A9 - TCWG-390 [6/10]
* cortex-strings benchmarks suggest this isn't just an outlier, nor
entirely about preload
* Quick test on A15 suggests a similar (less severe) issue
* Got cortex-strings benchmarks more-or-less running via lava

memset performance improvement - TCWG-156 [1/10]
* GNU benchtests not as repeatable as hoped

lowlevellock performance bugs - TCWG-435 [1/10]
* Converted a bunch of inline functions into macros for coldfire

Meetings/mail/etc [2/10]


=Plan=

Carry on generating memcpy data, develop some hypotheses
Bugfix lava-ization of cortex-strings benchmarks as  when
Explore repeatability of cortex-strings benchmark
See how much quicker I can make cortex-strings benchmark without
compromising repeatability
Try to get lowlevellock.h committed for at least arm/aarch64

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] 09 - 13 June

[Bernie Ogden]

=Progress=

memcpy regression on A9 - REQ-363 [3/10]
* Verified regression on a couple of targets, some investigation around cause

lowlevellock performance bugs - TCWG-435 [3/10]
* Upstreamed, some reworking based on review, obsessed about odd code
change  intermittent test failure

memset improvement - TCWG-156 [1/10]
* Started looking at glibc benchtests as a quicker way of doing
1st-round benchmarking

Meetings/mail/etc [3/10]


=Plan=

More of the above -
* Get some memset benchmarks running
* Carry on with lowlevellock patches (breaks m68k build, apparently)
* More memcpy investigation

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 23

[Bernie Ogden]

=Progress=

lowlevellock performance bugs - TCWG-435 [1/10]
* All ready to go

cbuild benchmarking - TCWG-360 [4/10]
* Completed a draft implementation for spec2k
* Parked pending review

Meetings/mail/etc [5/10]


=Plan=

Send lowlevellock patches to list
Get back to benchmarking/improving cortex-strings memset

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 21

[Bernie Ogden]

== Progress==

lowlevellock performance bugs - TCWG-435 [5/10]
* Tried various methods to build/test glibc for aarch64
* Eventually succeeded (tests passed)

cbuild benchmarking - TCWG-360 [3/10]
* cbuildized spec2xxx scripts working as far as 'run'

Meetings/mail/etc [2/10]


== Plan ==

Holiday for one week

After that:
* Clean up cbuildized spec2xxx scripts, cbuildize them some more 
discuss with Rob
* Send lowlevellock patch upstream
* If time, put together some more experimental memset implementations

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 19

[Bernie Ogden]

== Progress==

Holiday [2/10]

Fixing performance bugs in lowlevellock - TCWG-435 [4/10]
* Initial patch for lowlevellock.h largely done, awaiting an aarch64 test run
* About 1/2 of the lowlevellock.h's remain post-patch, need to ask
some questions about them
cortex-strings memset - TCWG-156 [1/10]
* Fixed an alternative implementation, a little more benchmarking investigation
cbuild benchmarking branch [1/10]
* Hunting for correct benchmark source, added a little more error checking

Meetings/mail/etc [2/10]


== Plan ==

More of the same -
* Post lowlevellock.h patch and ask some questions
* Finish exploring memset alternatives and work out how to benchmark them
* Carry on with cbuild benchmarking branch


Holiday 26th - 30th May

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 18

[Bernie Ogden]

(Resending to correct address)

== Progress==

TCWG-435 needless busy-wait in lowlevellock.c (0/10)
* Patches for lowlevellock.c sent to list
* Patch for lowlevellock.h still to do

TCWG-156 cortex-strings memset (5/10)
* Dug through a bunch of docs and fiddled with the source

Looking at the cbuild benchmarking branch (2/10)
* Some back and forth around getting access to lava lab
* Poked around the code, made some small improvements

== Misc ==

Meetings/mail/etc 3/10


== Plan ==

(Public) holiday on Monday
Put together lowlevellock.h patch
Do something more substantial with cbuild
Finish fiddling with the memset source

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 17

[Bernie Ogden]

== Progress==

TCWG-435 needless busy-wait in lowlevellock.c (3/10)
* Patches ready, testing a bit slow  fiddly

TCWG-156 cortex-strings memset (3/10)
* Worried about noisy benchmarks
* Learned to use Lava
* Found some helpful internal docs
* Managed to tweak the code slightly


== Misc ==

Meetings/mail/etc 2/10
(Public) holiday  2/10


== Plan ==

Stop worrying and follow the general curve of the noisy benchmarks
(But also try to convince Maxim's spec scripts + lava to generate some
less noisy results)
Another public holiday next Monday

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 16

[Bernie Ogden]

== Progress ==

TCWG-156 cortex-strings memset (3/10)

* Got a full set of benchmarks (for my 2 targets)
* Cleaned up code
* Sped up small memsets (for A9, A15 results pending)

glibc performance bug in lowlevellock.c (1/10 - I'll make a card next week)

* Learned to build and test glibc
* Understood the bug and worked out how to fix

Misc

* Meetings/mail (1/10)
* Infrastructure/workflow fiddling (1/10)
* 2 days holiday


== Plan ==

More TCWG-156
Finish off glibc performance bug
Public holidays Friday  Monday

___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain

[ACTIVITY] Week 14

[Bernie Ogden]

TCWG-156 (5/10)
* Hacked v7 memcpy into a memset
* Much fiddling with builds, targets
* Kicked off a benchmark run

Misc
* Meetings (1/10)
* Finding hardware/setting up working environments/figuring out workflows
(4/10)
___
linaro-toolchain mailing list
linaro-toolchain@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-toolchain