I think it is a race condition caused by netty deactivating a channel while it is active. Switched to nio and it works fine --conf spark.shuffle.blockTransferService=nio
On Thu, Nov 20, 2014 at 10:44 AM, Hector Yee <hector....@gmail.com> wrote: > I'm still seeing the fetch failed error and updated > https://issues.apache.org/jira/browse/SPARK-3633 > > On Thu, Nov 20, 2014 at 10:21 AM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> +1 (non-binding) >> >> . ran simple things on spark-shell >> . ran jobs in yarn client & cluster modes, and standalone cluster mode >> >> On Wed, Nov 19, 2014 at 2:51 PM, Andrew Or <and...@databricks.com> wrote: >> > Please vote on releasing the following candidate as Apache Spark version >> > 1.1.1. >> > >> > This release fixes a number of bugs in Spark 1.1.0. Some of the notable >> ones >> > are >> > - [SPARK-3426] Sort-based shuffle compression settings are incompatible >> > - [SPARK-3948] Stream corruption issues in sort-based shuffle >> > - [SPARK-4107] Incorrect handling of Channel.read() led to data >> truncation >> > The full list is at http://s.apache.org/z9h and in the CHANGES.txt >> attached. >> > >> > Additionally, this candidate fixes two blockers from the previous RC: >> > - [SPARK-4434] Cluster mode jar URLs are broken >> > - [SPARK-4480][SPARK-4467] Too many open files exception from shuffle >> spills >> > >> > The tag to be voted on is v1.1.1-rc2 (commit 3693ae5d): >> > http://s.apache.org/p8 >> > >> > The release files, including signatures, digests, etc can be found at: >> > http://people.apache.org/~andrewor14/spark-1.1.1-rc2/ >> > >> > Release artifacts are signed with the following key: >> > https://people.apache.org/keys/committer/andrewor14.asc >> > >> > The staging repository for this release can be found at: >> > https://repository.apache.org/content/repositories/orgapachespark-1043/ >> > >> > The documentation corresponding to this release can be found at: >> > http://people.apache.org/~andrewor14/spark-1.1.1-rc2-docs/ >> > >> > Please vote on releasing this package as Apache Spark 1.1.1! >> > >> > The vote is open until Saturday, November 22, at 23:00 UTC and passes if >> > a majority of at least 3 +1 PMC votes are cast. >> > [ ] +1 Release this package as Apache Spark 1.1.1 >> > [ ] -1 Do not release this package because ... >> > >> > To learn more about Apache Spark, please see >> > http://spark.apache.org/ >> > >> > Cheers, >> > Andrew >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> > For additional commands, e-mail: dev-h...@spark.apache.org >> >> >> >> -- >> Marcelo >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> > > > -- > Yee Yang Li Hector <http://google.com/+HectorYee> > *google.com/+HectorYee <http://google.com/+HectorYee>* > -- Yee Yang Li Hector <http://google.com/+HectorYee> *google.com/+HectorYee <http://google.com/+HectorYee>*
