[
https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013591#comment-14013591
]
Sean Owen commented on SPARK-1518:
----------------------------------
Sorry for one more message here to reply to Matei -- yes it's the "laptop" use
case except I'd describe that as a not-uncommon production deployment! it's the
embedded-client scenario. It is more than adding one hadoop-client dependency,
because you need to emulate the excludes, etc that Spark has to. (But yeah then
it works.) I agree supporting a bunch of Hadoop versions gets painful, as a
result. This was why I was suggesting way up top that supporting old versions
may become more trouble than its worht.
> Spark master doesn't compile against hadoop-common trunk
> --------------------------------------------------------
>
> Key: SPARK-1518
> URL: https://issues.apache.org/jira/browse/SPARK-1518
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Marcelo Vanzin
> Assignee: Colin Patrick McCabe
> Priority: Critical
>
> FSDataOutputStream::sync() has disappeared from trunk in Hadoop;
> FileLogger.scala is calling it.
> I've changed it locally to hsync() so I can compile the code, but haven't
> checked yet whether those are equivalent. hsync() seems to have been there
> forever, so it hopefully works with all versions Spark cares about.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
