Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-22 Thread Josh Hursey 
Our runs last night were successful with that MTT update. I'll keep an eye
on it.

On Wed, Mar 21, 2018 at 11:37 AM, Barrett, Brian via devel <
devel@lists.open-mpi.org> wrote:

> Yeah, it’s failing in exactly the reason I thought it would (and why we
> didn’t support md5sum files in the first place).
>
> Josh and I updated the MTT client yesterday and I pushed the change
> today.  Please update your client and it should work.
>
> Brian
>
> On Mar 20, 2018, at 10:35 PM, Boris Karasev  wrote:
>
> This issue occurs again:
>
> Running command: /bin/md5sum /scratch/20180321_050534_
> 23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/
> openmpi-master-201803200242-9eb426e.tar.gz
> e418ee1ea52345ba7ba72c4d4b7cd1ac /scratch/20180321_050534_
> 23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/
> openmpi-master-201803200242-9eb426e.tar.gz
> *** Child process stdout closed
> *** Child process now dead
> Command complete, exit status: 0
> *** ERROR: md5sum from checksum file does not match actual ( !=
> e418ee1ea52345ba7ba72c4d4b7cd1ac)*** ERROR: Module aborted:
> MTT::MPI::Get::OMPI_Snapshot:Get: *** ERROR: md5sum from checksum file
> does not match actual ( != e418ee1ea52345ba7ba72c4d4b7cd1ac) at
> /hpc/newhome/mtt/svn/mt
>
> Currently, the files contain those commits:
> https://download.open-mpi.org/nightly/open-mpi/master/latest_snapshot.txt
> :
> master-201803200242-9eb426e
>
> https://download.open-mpi.org/nightly/open-mpi/master/[
> md5sums.txt|sha1sums.txt]:
> master-201803081852-70c59f7
> master-201803140349-5f58e7b
> master-201803180329-a2d1419
> master-201803110243-50d07e9
> master-201803170305-bf3dd8a
> master-201803082122-0f345c0
> master-201803160306-e08e580
>
>
> --
>
> Best regards, Boris Karasev.
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/devel
>
>
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/devel
>



-- 
Josh Hursey
IBM Spectrum MPI Developer
___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-21 Thread Barrett, Brian via devel 
Yeah, it’s failing in exactly the reason I thought it would (and why we didn’t 
support md5sum files in the first place).

Josh and I updated the MTT client yesterday and I pushed the change today.  
Please update your client and it should work.

Brian

On Mar 20, 2018, at 10:35 PM, Boris Karasev 
mailto:karase...@gmail.com>> wrote:

This issue occurs again:

Running command: /bin/md5sum 
/scratch/20180321_050534_23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803200242-9eb426e.tar.gz
e418ee1ea52345ba7ba72c4d4b7cd1ac 
/scratch/20180321_050534_23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803200242-9eb426e.tar.gz
*** Child process stdout closed
*** Child process now dead
Command complete, exit status: 0
*** ERROR: md5sum from checksum file does not match actual ( != 
e418ee1ea52345ba7ba72c4d4b7cd1ac)*** ERROR: Module aborted: 
MTT::MPI::Get::OMPI_Snapshot:Get: *** ERROR: md5sum from checksum file does not 
match actual ( != e418ee1ea52345ba7ba72c4d4b7cd1ac) at /hpc/newhome/mtt/svn/mt

Currently, the files contain those commits:
https://download.open-mpi.org/nightly/open-mpi/master/latest_snapshot.txt:
master-201803200242-9eb426e

https://download.open-mpi.org/nightly/open-mpi/master/[md5sums.txt|sha1sums.txt]:
master-201803081852-70c59f7
master-201803140349-5f58e7b
master-201803180329-a2d1419
master-201803110243-50d07e9
master-201803170305-bf3dd8a
master-201803082122-0f345c0
master-201803160306-e08e580


--

Best regards, Boris Karasev.

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-20 Thread Boris Karasev 
This issue occurs again:

Running command: /bin/md5sum
/scratch/20180321_050534_23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803200242-9eb426e.tar.gz
e418ee1ea52345ba7ba72c4d4b7cd1ac
/scratch/20180321_050534_23183_145104/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803200242-9eb426e.tar.gz
*** Child process stdout closed
*** Child process now dead
Command complete, exit status: 0
*** ERROR: md5sum from checksum file does not match actual ( !=
e418ee1ea52345ba7ba72c4d4b7cd1ac)*** ERROR: Module aborted:
MTT::MPI::Get::OMPI_Snapshot:Get: *** ERROR: md5sum from checksum file does
not match actual ( != e418ee1ea52345ba7ba72c4d4b7cd1ac) at
/hpc/newhome/mtt/svn/mt

Currently, the files contain those commits:
https://download.open-mpi.org/nightly/open-mpi/master/latest_snapshot.txt:
master-201803200242-9eb426e

https://download.open-mpi.org/nightly/open-mpi/master/[md5sums.txt|sha1sums.txt]:

master-201803081852-70c59f7
master-201803140349-5f58e7b
master-201803180329-a2d1419
master-201803110243-50d07e9
master-201803170305-bf3dd8a
master-201803082122-0f345c0
master-201803160306-e08e580


-- 

Best regards, Boris Karasev.
___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-16 Thread Barrett, Brian via devel 
Eventual consistency for the win.  It looks like I forgot to set a short cache 
time for the CDN that fronts the artifact repository.  So the previous day’s 
file was returned.  I fixed that and flushed the cache on the CDN, so it should 
work now.

Brian

On Mar 15, 2018, at 10:41 PM, Boris Karasev 
mailto:karase...@gmail.com>> wrote:

Hello,
I have an issue with md5sums.txt/sha1sums.txt files for master. They are 
received but they looks outdated:
…
Value returning:
Running command: /bin/md5sum 
/scratch/20180316_063515_26360_144526/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803160306-e08e580.tar.gz
3b05d9b122995c63c74dcc290e654624  
/scratch/20180316_063515_26360_144526/sources/mpi_get__ompi-nightly-master/tarballs/openmpi-master-201803160306-e08e580.tar.gz
*** Child process stdout closed
*** Child process now dead
Command complete, exit status: 0
*** ERROR: md5sum from checksum file does not match actual ( != 
3b05d9b122995c63c74dcc290e654624)*** ERROR: Module aborted: 
MTT::MPI::Get::OMPI_Snapshot:Get: *** ERROR: md5sum from checksum file does not 
match actual ( != 3b05d9b122995c63c74dcc290e654624) at 
/hpc/newhome/mtt/svn/mtt.git/trunk/lib/MTT/Messages.pm li
ne 133.

I checked the file: 
https://download.open-mpi.org/nightly/open-mpi/master/latest_snapshot.txt
On that moment it contains this: "master-201803160306-e08e580"
But the files md5sums.txt/sha1sums.txt did not contained the checksum for 
"master-201803160306-e08e580".
The last checksum contained in the files is intended for 
"openmpi-master-201803081852-70c59f7".

Thanks.


--

Best regards, Boris Karasev.

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-13 Thread Josh Hursey 
Reminder that this switch happens on Sunday, March 18th. Please update your
MTT INI files.

On Mon, Feb 26, 2018 at 4:09 PM, Barrett, Brian via devel <
devel@lists.open-mpi.org> wrote:

> On Sunday, March 18th, the Open MPI team is going to make a change in
> where nightly tarballs are stored that will likely impact MTT test
> configuration.  If you have an automated system (including MTT) that
> fetches nightly tarballs, you will likely need to make a change in the next
> two weeks to avoid breakage.
>
> Over the last year, we’ve been working behind the scenes to improve some
> of the workflows around building and releasing tarballs (both nightly and
> release), including moving them out of our web tree and into Amazon S3 (and
> the Amazon CloudFront CDN for downloads).  It’s time to make the next step,
> moving the nightly tarballs out of the web tree.
>
> As of December, the nightly tarball builder uploads the build results to
> S3, which can be accessed from:
>
>https://download.open-mpi.org/nightly/open-mpi/ branch>/
>
> So to get the latest 3.0 nightly tarball version, you’d download
> https://download.open-mpi.org/nightly/open-mpi/v3.0.x/latest_snapshot.txt.
> The build artifact tree under https://download.open-mpi.org/
> nightly/open-mpi/ matches the tree under https://www.open-mpi.org/nightly/,
> so scripts should work with only the change in root of the tree.
>
> On Sunday, March 18th, we’ll stop mirroring tarballs to www.open-mpi.org
> and rewrite the web pages to direct users to download.open-mpi.org/ for
> downloading nightly tarballs of Open MPI.  We will add redirects from the
> old tarballs and latest_snapshot.txt files to the new location, but not all
> clients follow redirects by default.  So we’re asking everyone to
> proactively update their MTT scripts.  It should just be updating lines
> like:
>
>   ompi_snapshot_url = https://www.open-mpi.org/nightly/master
>
> to read:
>
>   ompi_snapshot_url = https://download.open-mpi.org/
> nightly/open-mpi/master
>
>
> Thanks,
>
> Brian
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/devel




-- 
Josh Hursey
IBM Spectrum MPI Developer
___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-08 Thread Barrett, Brian via devel 
Sorry about that; we screwed up understanding the “API” between MTT and the 
Open MPI download path.  We made a change in the nightly tarball builder and 
things should be working now.

Brian

> On Mar 8, 2018, at 2:31 AM, Christoph Niethammer  wrote:
> 
> Hello,
> 
> After the change of the nighly tarball URL I have issues with mtt getting the 
> checksum files:
> 
> ...
> Running command: wget --no-check-certificate -nv
>   https://download.open-mpi.org/nightly/open-mpi/v2.x/md5sums.txt
> md5sums.txt
> https://download.open-mpi.org/nightly/open-mpi/v2.x/md5sums.txt:
>   2018-03-07 23:16:19 ERROR 403: Forbidden.
> ...
> 
> Fetching the tarballs itself works fine.
> Anything else I have to change in the setup?
> 
> 
> Best
> 
> Christoph Niethammer
> 
> 
> - Original Message -
> From: "Open MPI Developers" 
> To: "Open MPI Developers" 
> Cc: "Barrett, Brian" 
> Sent: Monday, February 26, 2018 11:09:42 PM
> Subject: [OMPI devel] Upcoming nightly tarball URL changes
> 
> On Sunday, March 18th, the Open MPI team is going to make a change in where 
> nightly tarballs are stored that will likely impact MTT test configuration.  
> If you have an automated system (including MTT) that fetches nightly 
> tarballs, you will likely need to make a change in the next two weeks to 
> avoid breakage.
> 
> Over the last year, we’ve been working behind the scenes to improve some of 
> the workflows around building and releasing tarballs (both nightly and 
> release), including moving them out of our web tree and into Amazon S3 (and 
> the Amazon CloudFront CDN for downloads).  It’s time to make the next step, 
> moving the nightly tarballs out of the web tree.
> 
> As of December, the nightly tarball builder uploads the build results to S3, 
> which can be accessed from:
> 
>   https://download.open-mpi.org/nightly/open-mpi// name>
> 
> So to get the latest 3.0 nightly tarball version, you’d download 
> https://download.open-mpi.org/nightly/open-mpi/v3.0.x/latest_snapshot.txt.  
> The build artifact tree under https://download.open-mpi.org/nightly/open-mpi/ 
> matches the tree under https://www.open-mpi.org/nightly/, so scripts should 
> work with only the change in root of the tree.
> 
> On Sunday, March 18th, we’ll stop mirroring tarballs to www.open-mpi.org and 
> rewrite the web pages to direct users to download.open-mpi.org/ for 
> downloading nightly tarballs of Open MPI.  We will add redirects from the old 
> tarballs and latest_snapshot.txt files to the new location, but not all 
> clients follow redirects by default.  So we’re asking everyone to proactively 
> update their MTT scripts.  It should just be updating lines like:
> 
>  ompi_snapshot_url = https://www.open-mpi.org/nightly/master
> 
> to read:
> 
>  ompi_snapshot_url = https://download.open-mpi.org/nightly/open-mpi/master
> 
> 
> Thanks,
> 
> Brian
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/devel

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-03-08 Thread Christoph Niethammer 
Hello,

After the change of the nighly tarball URL I have issues with mtt getting the 
checksum files:

...
Running command: wget --no-check-certificate -nv
   https://download.open-mpi.org/nightly/open-mpi/v2.x/md5sums.txt
md5sums.txt
https://download.open-mpi.org/nightly/open-mpi/v2.x/md5sums.txt:
   2018-03-07 23:16:19 ERROR 403: Forbidden.
...

Fetching the tarballs itself works fine.
Anything else I have to change in the setup?


Best

Christoph Niethammer


- Original Message -
From: "Open MPI Developers" 
To: "Open MPI Developers" 
Cc: "Barrett, Brian" 
Sent: Monday, February 26, 2018 11:09:42 PM
Subject: [OMPI devel] Upcoming nightly tarball URL changes

On Sunday, March 18th, the Open MPI team is going to make a change in where 
nightly tarballs are stored that will likely impact MTT test configuration.  If 
you have an automated system (including MTT) that fetches nightly tarballs, you 
will likely need to make a change in the next two weeks to avoid breakage.

Over the last year, we’ve been working behind the scenes to improve some of the 
workflows around building and releasing tarballs (both nightly and release), 
including moving them out of our web tree and into Amazon S3 (and the Amazon 
CloudFront CDN for downloads).  It’s time to make the next step, moving the 
nightly tarballs out of the web tree.

As of December, the nightly tarball builder uploads the build results to S3, 
which can be accessed from:

   https://download.open-mpi.org/nightly/open-mpi//

So to get the latest 3.0 nightly tarball version, you’d download 
https://download.open-mpi.org/nightly/open-mpi/v3.0.x/latest_snapshot.txt.  The 
build artifact tree under https://download.open-mpi.org/nightly/open-mpi/ 
matches the tree under https://www.open-mpi.org/nightly/, so scripts should 
work with only the change in root of the tree.

On Sunday, March 18th, we’ll stop mirroring tarballs to www.open-mpi.org and 
rewrite the web pages to direct users to download.open-mpi.org/ for downloading 
nightly tarballs of Open MPI.  We will add redirects from the old tarballs and 
latest_snapshot.txt files to the new location, but not all clients follow 
redirects by default.  So we’re asking everyone to proactively update their MTT 
scripts.  It should just be updating lines like:

  ompi_snapshot_url = https://www.open-mpi.org/nightly/master

to read:

  ompi_snapshot_url = https://download.open-mpi.org/nightly/open-mpi/master


Thanks,

Brian
___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel
___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel

Re: [OMPI devel] Upcoming nightly tarball URL changes

2018-02-27 Thread Jeff Squyres (jsquyres) 
Thanks Brian -- I have updated Cisco's MTT URLs.

PSA: Everyone go update your MTT URLs today!



> On Feb 26, 2018, at 5:09 PM, Barrett, Brian via devel 
>  wrote:
> 
> On Sunday, March 18th, the Open MPI team is going to make a change in where 
> nightly tarballs are stored that will likely impact MTT test configuration.  
> If you have an automated system (including MTT) that fetches nightly 
> tarballs, you will likely need to make a change in the next two weeks to 
> avoid breakage.
> 
> Over the last year, we’ve been working behind the scenes to improve some of 
> the workflows around building and releasing tarballs (both nightly and 
> release), including moving them out of our web tree and into Amazon S3 (and 
> the Amazon CloudFront CDN for downloads).  It’s time to make the next step, 
> moving the nightly tarballs out of the web tree.
> 
> As of December, the nightly tarball builder uploads the build results to S3, 
> which can be accessed from:
> 
>   https://download.open-mpi.org/nightly/open-mpi// name>
> 
> So to get the latest 3.0 nightly tarball version, you’d download 
> https://download.open-mpi.org/nightly/open-mpi/v3.0.x/latest_snapshot.txt.  
> The build artifact tree under https://download.open-mpi.org/nightly/open-mpi/ 
> matches the tree under https://www.open-mpi.org/nightly/, so scripts should 
> work with only the change in root of the tree.
> 
> On Sunday, March 18th, we’ll stop mirroring tarballs to www.open-mpi.org and 
> rewrite the web pages to direct users to download.open-mpi.org/ for 
> downloading nightly tarballs of Open MPI.  We will add redirects from the old 
> tarballs and latest_snapshot.txt files to the new location, but not all 
> clients follow redirects by default.  So we’re asking everyone to proactively 
> update their MTT scripts.  It should just be updating lines like:
> 
>  ompi_snapshot_url = https://www.open-mpi.org/nightly/master
> 
> to read:
> 
>  ompi_snapshot_url = https://download.open-mpi.org/nightly/open-mpi/master
> 
> 
> Thanks,
> 
> Brian
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/devel


-- 
Jeff Squyres
jsquy...@cisco.com

___
devel mailing list
devel@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/devel