[
https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010339#comment-14010339
]
Patrick Wendell commented on SPARK-1518:
----------------------------------------
Re: versioning - I was just asking whether this changes the oldest version we
are compatible with. That's a question we should ask of all Hadoop patches. I
just tested Spark and it doesn't compile against 0.20.X, so this is a no-op in
terms of compatibility anyways.
It would be good to list the oldest upstream Hadoop version we support. Many
people still run Spark against Hadoop 1.X/CDH3 variants. We get high download
rates for those pre-built packages. I think this is a little different than
e.g. CDH where people upgrade all components at the same time... people
download newer versions of Spark and run it with old filesystems very often.
> Spark master doesn't compile against hadoop-common trunk
> --------------------------------------------------------
>
> Key: SPARK-1518
> URL: https://issues.apache.org/jira/browse/SPARK-1518
> Project: Spark
> Issue Type: Bug
> Reporter: Marcelo Vanzin
> Assignee: Colin Patrick McCabe
> Priority: Critical
>
> FSDataOutputStream::sync() has disappeared from trunk in Hadoop;
> FileLogger.scala is calling it.
> I've changed it locally to hsync() so I can compile the code, but haven't
> checked yet whether those are equivalent. hsync() seems to have been there
> forever, so it hopefully works with all versions Spark cares about.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
