The published hash is a SHA512. You can verify the integrity of the packages by running `sha512sum` on the archive and comparing the computed hash with the published one. Unfortunately however, I don't know what tool is used to generate the hash and I can't reproduce the format, so I ended up manually comparing the hashes.
On Mon, Apr 4, 2016 at 2:39 PM, Nicholas Chammas <nicholas.cham...@gmail.com> wrote: > An additional note: The Spark packages being served off of CloudFront (i.e. > the “direct download” option on spark.apache.org) are also corrupt. > > Btw what’s the correct way to verify the SHA of a Spark package? I’ve tried > a few commands on working packages downloaded from Apache mirrors, but I > can’t seem to reproduce the published SHA for spark-1.6.1-bin-hadoop2.6.tgz. > > > On Mon, Apr 4, 2016 at 11:45 AM Ted Yu <yuzhih...@gmail.com> wrote: >> >> Maybe temporarily take out the artifacts on S3 before the root cause is >> found. >> >> On Thu, Mar 24, 2016 at 7:25 AM, Nicholas Chammas >> <nicholas.cham...@gmail.com> wrote: >>> >>> Just checking in on this again as the builds on S3 are still broken. :/ >>> >>> Could it have something to do with us moving release-build.sh? >>> >>> >>> On Mon, Mar 21, 2016 at 1:43 PM Nicholas Chammas >>> <nicholas.cham...@gmail.com> wrote: >>>> >>>> Is someone going to retry fixing these packages? It's still a problem. >>>> >>>> Also, it would be good to understand why this is happening. >>>> >>>> On Fri, Mar 18, 2016 at 6:49 PM Jakob Odersky <ja...@odersky.com> wrote: >>>>> >>>>> I just realized you're using a different download site. Sorry for the >>>>> confusion, the link I get for a direct download of Spark 1.6.1 / >>>>> Hadoop 2.6 is >>>>> http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz >>>>> >>>>> On Fri, Mar 18, 2016 at 3:20 PM, Nicholas Chammas >>>>> <nicholas.cham...@gmail.com> wrote: >>>>> > I just retried the Spark 1.6.1 / Hadoop 2.6 download and got a >>>>> > corrupt ZIP >>>>> > file. >>>>> > >>>>> > Jakob, are you sure the ZIP unpacks correctly for you? Is it the same >>>>> > Spark >>>>> > 1.6.1/Hadoop 2.6 package you had a success with? >>>>> > >>>>> > On Fri, Mar 18, 2016 at 6:11 PM Jakob Odersky <ja...@odersky.com> >>>>> > wrote: >>>>> >> >>>>> >> I just experienced the issue, however retrying the download a second >>>>> >> time worked. Could it be that there is some load balancer/cache in >>>>> >> front of the archive and some nodes still serve the corrupt >>>>> >> packages? >>>>> >> >>>>> >> On Fri, Mar 18, 2016 at 8:00 AM, Nicholas Chammas >>>>> >> <nicholas.cham...@gmail.com> wrote: >>>>> >> > I'm seeing the same. :( >>>>> >> > >>>>> >> > On Fri, Mar 18, 2016 at 10:57 AM Ted Yu <yuzhih...@gmail.com> >>>>> >> > wrote: >>>>> >> >> >>>>> >> >> I tried again this morning : >>>>> >> >> >>>>> >> >> $ wget >>>>> >> >> >>>>> >> >> >>>>> >> >> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >> --2016-03-18 07:55:30-- >>>>> >> >> >>>>> >> >> >>>>> >> >> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >> Resolving s3.amazonaws.com... 54.231.19.163 >>>>> >> >> ... >>>>> >> >> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >> >>>>> >> >> gzip: stdin: unexpected end of file >>>>> >> >> tar: Unexpected EOF in archive >>>>> >> >> tar: Unexpected EOF in archive >>>>> >> >> tar: Error is not recoverable: exiting now >>>>> >> >> >>>>> >> >> On Thu, Mar 17, 2016 at 8:57 AM, Michael Armbrust >>>>> >> >> <mich...@databricks.com> >>>>> >> >> wrote: >>>>> >> >>> >>>>> >> >>> Patrick reuploaded the artifacts, so it should be fixed now. >>>>> >> >>> >>>>> >> >>> On Mar 16, 2016 5:48 PM, "Nicholas Chammas" >>>>> >> >>> <nicholas.cham...@gmail.com> >>>>> >> >>> wrote: >>>>> >> >>>> >>>>> >> >>>> Looks like the other packages may also be corrupt. I’m getting >>>>> >> >>>> the >>>>> >> >>>> same >>>>> >> >>>> error for the Spark 1.6.1 / Hadoop 2.4 package. >>>>> >> >>>> >>>>> >> >>>> >>>>> >> >>>> >>>>> >> >>>> >>>>> >> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.4.tgz >>>>> >> >>>> >>>>> >> >>>> Nick >>>>> >> >>>> >>>>> >> >>>> >>>>> >> >>>> On Wed, Mar 16, 2016 at 8:28 PM Ted Yu <yuzhih...@gmail.com> >>>>> >> >>>> wrote: >>>>> >> >>>>> >>>>> >> >>>>> On Linux, I got: >>>>> >> >>>>> >>>>> >> >>>>> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >>>>> >>>>> >> >>>>> gzip: stdin: unexpected end of file >>>>> >> >>>>> tar: Unexpected EOF in archive >>>>> >> >>>>> tar: Unexpected EOF in archive >>>>> >> >>>>> tar: Error is not recoverable: exiting now >>>>> >> >>>>> >>>>> >> >>>>> On Wed, Mar 16, 2016 at 5:15 PM, Nicholas Chammas >>>>> >> >>>>> <nicholas.cham...@gmail.com> wrote: >>>>> >> >>>>>> >>>>> >> >>>>>> >>>>> >> >>>>>> >>>>> >> >>>>>> >>>>> >> >>>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >>>>>> >>>>> >> >>>>>> Does anyone else have trouble unzipping this? How did this >>>>> >> >>>>>> happen? >>>>> >> >>>>>> >>>>> >> >>>>>> What I get is: >>>>> >> >>>>>> >>>>> >> >>>>>> $ gzip -t spark-1.6.1-bin-hadoop2.6.tgz >>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: unexpected end of file >>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: uncompress failed >>>>> >> >>>>>> >>>>> >> >>>>>> Seems like a strange type of problem to come across. >>>>> >> >>>>>> >>>>> >> >>>>>> Nick >>>>> >> >>>>> >>>>> >> >>>>> >>>>> >> >> >>>>> >> > >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
