An additional note: The Spark packages being served off of CloudFront (i.e. the “direct download” option on spark.apache.org) are also corrupt.
Btw what’s the correct way to verify the SHA of a Spark package? I’ve tried a few commands on working packages downloaded from Apache mirrors, but I can’t seem to reproduce the published SHA for spark-1.6.1-bin-hadoop2.6.tgz <http://www.apache.org/dist/spark/spark-1.6.1/spark-1.6.1-bin-hadoop2.6.tgz.sha> . On Mon, Apr 4, 2016 at 11:45 AM Ted Yu <yuzhih...@gmail.com> wrote: > Maybe temporarily take out the artifacts on S3 before the root cause is > found. > > On Thu, Mar 24, 2016 at 7:25 AM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> Just checking in on this again as the builds on S3 are still broken. :/ >> >> Could it have something to do with us moving release-build.sh >> <https://github.com/apache/spark/commits/master/dev/create-release/release-build.sh> >> ? >> >> >> On Mon, Mar 21, 2016 at 1:43 PM Nicholas Chammas < >> nicholas.cham...@gmail.com> wrote: >> >>> Is someone going to retry fixing these packages? It's still a problem. >>> >>> Also, it would be good to understand why this is happening. >>> >>> On Fri, Mar 18, 2016 at 6:49 PM Jakob Odersky <ja...@odersky.com> wrote: >>> >>>> I just realized you're using a different download site. Sorry for the >>>> confusion, the link I get for a direct download of Spark 1.6.1 / >>>> Hadoop 2.6 is >>>> http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz >>>> >>>> On Fri, Mar 18, 2016 at 3:20 PM, Nicholas Chammas >>>> <nicholas.cham...@gmail.com> wrote: >>>> > I just retried the Spark 1.6.1 / Hadoop 2.6 download and got a >>>> corrupt ZIP >>>> > file. >>>> > >>>> > Jakob, are you sure the ZIP unpacks correctly for you? Is it the same >>>> Spark >>>> > 1.6.1/Hadoop 2.6 package you had a success with? >>>> > >>>> > On Fri, Mar 18, 2016 at 6:11 PM Jakob Odersky <ja...@odersky.com> >>>> wrote: >>>> >> >>>> >> I just experienced the issue, however retrying the download a second >>>> >> time worked. Could it be that there is some load balancer/cache in >>>> >> front of the archive and some nodes still serve the corrupt packages? >>>> >> >>>> >> On Fri, Mar 18, 2016 at 8:00 AM, Nicholas Chammas >>>> >> <nicholas.cham...@gmail.com> wrote: >>>> >> > I'm seeing the same. :( >>>> >> > >>>> >> > On Fri, Mar 18, 2016 at 10:57 AM Ted Yu <yuzhih...@gmail.com> >>>> wrote: >>>> >> >> >>>> >> >> I tried again this morning : >>>> >> >> >>>> >> >> $ wget >>>> >> >> >>>> >> >> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >> --2016-03-18 07:55:30-- >>>> >> >> >>>> >> >> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >> Resolving s3.amazonaws.com... 54.231.19.163 >>>> >> >> ... >>>> >> >> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >> >>>> >> >> gzip: stdin: unexpected end of file >>>> >> >> tar: Unexpected EOF in archive >>>> >> >> tar: Unexpected EOF in archive >>>> >> >> tar: Error is not recoverable: exiting now >>>> >> >> >>>> >> >> On Thu, Mar 17, 2016 at 8:57 AM, Michael Armbrust >>>> >> >> <mich...@databricks.com> >>>> >> >> wrote: >>>> >> >>> >>>> >> >>> Patrick reuploaded the artifacts, so it should be fixed now. >>>> >> >>> >>>> >> >>> On Mar 16, 2016 5:48 PM, "Nicholas Chammas" >>>> >> >>> <nicholas.cham...@gmail.com> >>>> >> >>> wrote: >>>> >> >>>> >>>> >> >>>> Looks like the other packages may also be corrupt. I’m getting >>>> the >>>> >> >>>> same >>>> >> >>>> error for the Spark 1.6.1 / Hadoop 2.4 package. >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.4.tgz >>>> >> >>>> >>>> >> >>>> Nick >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> On Wed, Mar 16, 2016 at 8:28 PM Ted Yu <yuzhih...@gmail.com> >>>> wrote: >>>> >> >>>>> >>>> >> >>>>> On Linux, I got: >>>> >> >>>>> >>>> >> >>>>> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >>>>> >>>> >> >>>>> gzip: stdin: unexpected end of file >>>> >> >>>>> tar: Unexpected EOF in archive >>>> >> >>>>> tar: Unexpected EOF in archive >>>> >> >>>>> tar: Error is not recoverable: exiting now >>>> >> >>>>> >>>> >> >>>>> On Wed, Mar 16, 2016 at 5:15 PM, Nicholas Chammas >>>> >> >>>>> <nicholas.cham...@gmail.com> wrote: >>>> >> >>>>>> >>>> >> >>>>>> >>>> >> >>>>>> >>>> >> >>>>>> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >>>>>> >>>> >> >>>>>> Does anyone else have trouble unzipping this? How did this >>>> happen? >>>> >> >>>>>> >>>> >> >>>>>> What I get is: >>>> >> >>>>>> >>>> >> >>>>>> $ gzip -t spark-1.6.1-bin-hadoop2.6.tgz >>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: unexpected end of file >>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: uncompress failed >>>> >> >>>>>> >>>> >> >>>>>> Seems like a strange type of problem to come across. >>>> >> >>>>>> >>>> >> >>>>>> Nick >>>> >> >>>>> >>>> >> >>>>> >>>> >> >> >>>> >> > >>>> >>> >