[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

2015-03-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365087#comment-14365087 ] Steve Loughran commented on SPARK-3685: --- YARN-1197 covers supporting resizing

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

2015-03-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371299#comment-14371299 ] Steve Loughran commented on SPARK-6433: --- Similarly, the original sql package's

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

2015-03-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371283#comment-14371283 ] Steve Loughran commented on SPARK-6433: --- They have diverged, the original sql one

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

2015-03-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371350#comment-14371350 ] Steve Loughran commented on SPARK-6433: --- There's one interesting question: whether

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

2015-03-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371280#comment-14371280 ] Steve Loughran commented on SPARK-6433: --- ..sorry, I'd missed that previous report.

[jira] [Commented] (SPARK-1200) Make it possible to use unmanaged AM in yarn-client mode

2015-03-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369339#comment-14369339 ] Steve Loughran commented on SPARK-1200: --- You know, we could benefit all YARN apps if

[jira] [Created] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

2015-03-20 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-6433: - Summary: hive tests to import spark-sql test JAR for QueryTest access Key: SPARK-6433 URL: https://issues.apache.org/jira/browse/SPARK-6433 Project: Spark

[jira] [Created] (SPARK-6389) YARN app diagnostics report doesn't report NPEs

2015-03-17 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-6389: - Summary: YARN app diagnostics report doesn't report NPEs Key: SPARK-6389 URL: https://issues.apache.org/jira/browse/SPARK-6389 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

2015-03-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365419#comment-14365419 ] Steve Loughran commented on SPARK-3351: --- Thomas: what Yarn version was this against?

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

2015-03-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365547#comment-14365547 ] Steve Loughran commented on SPARK-3351: --- The NPE is certainly gone. I don't know

[jira] [Commented] (SPARK-6568) spark-shell.cmd --jars option does not accept the jar that has space in its path

2015-03-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386552#comment-14386552 ] Steve Loughran commented on SPARK-6568: --- Can you show the full stack trace?

[jira] [Commented] (SPARK-799) Windows versions of the deploy scripts

2015-03-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386571#comment-14386571 ] Steve Loughran commented on SPARK-799: -- Proving python versions of the launcher

[jira] [Commented] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

2015-03-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386585#comment-14386585 ] Steve Loughran commented on SPARK-2356: --- It's coming from {{

[jira] [Commented] (SPARK-6646) Spark 2.0: Rearchitecting Spark for Mobile Platforms

2015-04-01 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390373#comment-14390373 ] Steve Loughran commented on SPARK-6646: --- Obviously the barrier will be data source

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-03-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14385890#comment-14385890 ] Steve Loughran commented on SPARK-1537: --- # I've just tried to see where YARN-2444

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384520#comment-14384520 ] Steve Loughran commented on SPARK-6479: --- Henry: utterly unrelated. I was merely

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503752#comment-14503752 ] Steve Loughran commented on SPARK-7009: --- most of the others seemed fix by

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502657#comment-14502657 ] Steve Loughran commented on SPARK-7009: --- It's only 30 lines of diff including the

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502675#comment-14502675 ] Steve Loughran commented on SPARK-7009: --- Looking at the [openJDK

[jira] [Commented] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

2015-04-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502589#comment-14502589 ] Steve Loughran commented on SPARK-1911: --- This doesn't fix the problem, merely

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504639#comment-14504639 ] Steve Loughran commented on SPARK-7009: --- yes, what we are trying to do is come up

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503510#comment-14503510 ] Steve Loughran commented on SPARK-7009: --- ...so, open a JIRA for that and close this

[jira] [Commented] (SPARK-1437) Jenkins should build with Java 6

2015-05-03 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525761#comment-14525761 ] Steve Loughran commented on SPARK-1437: --- ..be good for the pull request test runs to

[jira] [Commented] (SPARK-5754) Spark AM not launching on Windows

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533315#comment-14533315 ] Steve Loughran commented on SPARK-5754: --- I had a look at what we did for slider via

[jira] [Commented] (SPARK-5034) Spark on Yarn launch failure on HDInsight on Windows

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533349#comment-14533349 ] Steve Loughran commented on SPARK-5034: --- looks related to SPARK-5754 ; linking

[jira] [Commented] (SPARK-5754) Spark AM not launching on Windows

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533360#comment-14533360 ] Steve Loughran commented on SPARK-5754: --- I think windows will need its own escaping

[jira] [Created] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

2015-05-08 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-7481: - Summary: Add Hadoop 2.6+ profile to pull in object store FS accessors Key: SPARK-7481 URL: https://issues.apache.org/jira/browse/SPARK-7481 Project: Spark

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

2015-05-08 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534174#comment-14534174 ] Steve Loughran commented on SPARK-7481: --- This doesn't contain any endorsement of the

[jira] [Commented] (SPARK-1423) Add scripts for launching Spark on Windows Azure

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533351#comment-14533351 ] Steve Loughran commented on SPARK-1423: --- should this be resolved as WONTFIX? Add

[jira] [Updated] (SPARK-5034) Spark on Yarn launch failure on HDInsight on Windows

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-5034: -- Summary: Spark on Yarn launch failure on HDInsight on Windows (was: Spark on Yarn within

[jira] [Commented] (SPARK-6961) Cannot save data to parquet files when executing from Windows from a Maven Project

2015-05-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533340#comment-14533340 ] Steve Loughran commented on SPARK-6961: --- I know what the cause is, it's something

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

2015-05-08 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534622#comment-14534622 ] Steve Loughran commented on SPARK-7481: --- hadoop openstack 100K +httpclient (400K)

[jira] [Created] (SPARK-7508) JettyUtils-generated servlets to log report all errors

2015-05-09 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-7508: - Summary: JettyUtils-generated servlets to log report all errors Key: SPARK-7508 URL: https://issues.apache.org/jira/browse/SPARK-7508 Project: Spark

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-05-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541589#comment-14541589 ] Steve Loughran commented on SPARK-1537: --- + YARN-3539 is resolved; the [v1 timeline

[jira] [Commented] (SPARK-7669) Builds against Hadoop 2.6+ get inconsistent curator dependencies

2015-05-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545636#comment-14545636 ] Steve Loughran commented on SPARK-7669: --- snippet of the maven dependencies with

[jira] [Created] (SPARK-7669) Builds against Hadoop 2.6+ get inconsistent curator dependencies

2015-05-15 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-7669: - Summary: Builds against Hadoop 2.6+ get inconsistent curator dependencies Key: SPARK-7669 URL: https://issues.apache.org/jira/browse/SPARK-7669 Project: Spark

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-04-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492303#comment-14492303 ] Steve Loughran commented on SPARK-1537: --- HADOOP-11826 patches the hadoop

[jira] [Commented] (SPARK-6907) Create an isolated classloader for the Hive Client.

2015-05-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550808#comment-14550808 ] Steve Loughran commented on SPARK-6907: --- Having just looked at this code, I'm a bit

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

2015-06-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575070#comment-14575070 ] Steve Loughran commented on SPARK-8064: --- I'm working on this Upgrade Hive to 1.2

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-06-09 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579152#comment-14579152 ] Steve Loughran commented on SPARK-1537: --- Full application log. Application hasn't

[jira] [Commented] (SPARK-8275) HistoryServer caches incomplete App UIs

2015-06-09 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579134#comment-14579134 ] Steve Loughran commented on SPARK-8275: --- I have a test for this as part of the

[jira] [Created] (SPARK-8275) HistoryServer caches incomplete App UIs

2015-06-09 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-8275: - Summary: HistoryServer caches incomplete App UIs Key: SPARK-8275 URL: https://issues.apache.org/jira/browse/SPARK-8275 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-8276) NPE in YarnClientSchedulerBackend.stop

2015-06-09 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-8276: - Summary: NPE in YarnClientSchedulerBackend.stop Key: SPARK-8276 URL: https://issues.apache.org/jira/browse/SPARK-8276 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-8275) HistoryServer caches incomplete App UIs

2015-06-09 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579273#comment-14579273 ] Steve Loughran commented on SPARK-8275: --- The cache code is from google; history

[jira] [Comment Edited] (SPARK-8275) HistoryServer caches incomplete App UIs

2015-06-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579273#comment-14579273 ] Steve Loughran edited comment on SPARK-8275 at 6/10/15 11:43 AM:

[jira] [Issue Comment Deleted] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2015-06-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-1537: -- Comment: was deleted (was: Full application log. Application hasn't actually stopped, which is

[jira] [Commented] (SPARK-7889) Jobs progress of apps on complete page of HistoryServer shows uncompleted

2015-06-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595998#comment-14595998 ] Steve Loughran commented on SPARK-7889: --- Added a [new pull

[jira] [Resolved] (SPARK-3284) saveAsParquetFile not working on windows

2015-06-11 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-3284. --- Resolution: Duplicate closing as duplicate of SPARK-6961 saveAsParquetFile not working on

[jira] [Commented] (SPARK-6961) Cannot save data to parquet files when executing from Windows from a Maven Project

2015-06-11 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14581886#comment-14581886 ] Steve Loughran commented on SPARK-6961: --- issue here is WINUTILS.EXE isn't on the

[jira] [Commented] (SPARK-7889) Jobs progress of apps on complete page of HistoryServer shows uncompleted

2015-06-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590215#comment-14590215 ] Steve Loughran commented on SPARK-7889: --- Is JIRA about (a) the status on the

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-06-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589659#comment-14589659 ] Steve Loughran commented on SPARK-7009: --- The issue isn't that python can't read

[jira] [Commented] (SPARK-8275) HistoryServer caches incomplete App UIs

2015-06-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587753#comment-14587753 ] Steve Loughran commented on SPARK-8275: --- your right, tagging as duplicate.

[jira] [Created] (SPARK-8394) HistoryServer doesn't read kerberos opts from config

2015-06-16 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-8394: - Summary: HistoryServer doesn't read kerberos opts from config Key: SPARK-8394 URL: https://issues.apache.org/jira/browse/SPARK-8394 Project: Spark Issue

[jira] [Commented] (SPARK-8394) HistoryServer doesn't read kerberos opts from config

2015-06-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587882#comment-14587882 ] Steve Loughran commented on SPARK-8394: --- Fixed in commit

[jira] [Resolved] (SPARK-8394) HistoryServer doesn't read kerberos opts from config

2015-06-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-8394. --- Resolution: Fixed fixed by Marcelo HistoryServer doesn't read kerberos opts from config

[jira] [Commented] (SPARK-4352) Incorporate locality preferences in dynamic allocation requests

2015-06-01 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567345#comment-14567345 ] Steve Loughran commented on SPARK-4352: --- As usual, when YARN-1042 is done, life gets

[jira] [Created] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2015-07-02 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-8789: - Summary: improve SQLQuerySuite resilience by dropping tables in setup Key: SPARK-8789 URL: https://issues.apache.org/jira/browse/SPARK-8789 Project: Spark

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

2015-07-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646699#comment-14646699 ] Steve Loughran commented on SPARK-6152: --- Chill and Kryo need to be in sync; there's

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

2015-08-03 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652730#comment-14652730 ] Steve Loughran commented on SPARK-8064: --- Also: we had to produce a custom release of

[jira] [Commented] (SPARK-8064) Upgrade Hive to 1.2

2015-08-03 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652699#comment-14652699 ] Steve Loughran commented on SPARK-8064: --- While assigned to me, lots of people helped

[jira] [Created] (SPARK-9417) sbt-launch to fetch sbt binaries over https not http

2015-07-28 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-9417: - Summary: sbt-launch to fetch sbt binaries over https not http Key: SPARK-9417 URL: https://issues.apache.org/jira/browse/SPARK-9417 Project: Spark Issue

[jira] [Commented] (SPARK-9417) sbt-launch to fetch sbt binaries over https not http

2015-07-28 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644959#comment-14644959 ] Steve Loughran commented on SPARK-9417: --- Marking as related to SPARK-9254, which

[jira] [Commented] (SPARK-9019) spark-submit fails on yarn with kerberos enabled

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662211#comment-14662211 ] Steve Loughran commented on SPARK-9019: --- # If this problem exists (I don't have test

[jira] [Commented] (SPARK-7789) sql on security hbase:Token generation only allowed for Kerberos authenticated clients

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662221#comment-14662221 ] Steve Loughran commented on SPARK-7789: --- With SPARK-8064, Hive 1.2.1 is the hive lib

[jira] [Updated] (SPARK-7252) Add support for creating new Hive and HBase delegation tokens

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-7252: -- Affects Version/s: (was: 1.3.1) 1.5.0 Add support for creating new

[jira] [Commented] (SPARK-6882) Spark ThriftServer2 Kerberos failed encountering java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662117#comment-14662117 ] Steve Loughran commented on SPARK-6882: --- Spark 1.5 will use Hive 1.2.1; this problem

[jira] [Resolved] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-5111. --- Resolution: Done Fix Version/s: 1.5.0 HiveContext and Thriftserver cannot work in

[jira] [Reopened] (SPARK-8385) java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation

2015-07-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened SPARK-8385: --- I can reproduce this too, and have actually managed to do it in the spark tests, somehow {code}

[jira] [Commented] (SPARK-8385) java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation

2015-07-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643652#comment-14643652 ] Steve Loughran commented on SPARK-8385: --- What's happening? Hadoop is trying to enum

[jira] [Resolved] (SPARK-8385) java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation

2015-07-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-8385. --- Resolution: Won't Fix java.lang.UnsupportedOperationException: Not implemented by the TFS

[jira] [Comment Edited] (SPARK-8385) java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation

2015-07-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643652#comment-14643652 ] Steve Loughran edited comment on SPARK-8385 at 7/28/15 1:01 AM:

[jira] [Created] (SPARK-9070) JavaDataFrameSuite teardown NPEs if setup failed

2015-07-15 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-9070: - Summary: JavaDataFrameSuite teardown NPEs if setup failed Key: SPARK-9070 URL: https://issues.apache.org/jira/browse/SPARK-9070 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8898) Jets3t hangs with more than 1 core

2015-07-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631133#comment-14631133 ] Steve Loughran commented on SPARK-8898: --- which Hadoop version are you using? Hadoop

[jira] [Commented] (SPARK-8898) Jets3t hangs with more than 1 core

2015-07-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633404#comment-14633404 ] Steve Loughran commented on SPARK-8898: --- OK. The last jets3t update (0.90) in Hadoop

[jira] [Commented] (SPARK-8276) NPE in YarnClientSchedulerBackend.stop

2015-07-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633479#comment-14633479 ] Steve Loughran commented on SPARK-8276: --- Afraid I can't, sorry. I'm assuming its

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971188#comment-14971188 ] Steve Loughran commented on SPARK-11265: Pull request is :

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970930#comment-14970930 ] Steve Loughran commented on SPARK-11265: I can trigger a failure in a unit test now, once you get

[jira] [Updated] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

2015-10-23 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-6270: -- Affects Version/s: 1.5.1 > Standalone Master hangs when streaming job completes and event

[jira] [Created] (SPARK-11265) YarnClient cant get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11265: -- Summary: YarnClient cant get tokens to talk to Hive in a secure cluster Key: SPARK-11265 URL: https://issues.apache.org/jira/browse/SPARK-11265 Project: Spark

[jira] [Updated] (SPARK-11265) YarnClient can't get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11265: --- Summary: YarnClient can't get tokens to talk to Hive in a secure cluster (was: YarnClient

[jira] [Commented] (SPARK-11265) YarnClient cant get tokens to talk to Hive in a secure cluster

2015-10-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969818#comment-14969818 ] Steve Loughran commented on SPARK-11265: Initial report from Chester Chen {noformat} This is

[jira] [Commented] (SPARK-11317) YARN HBase token code shouldn't swallow invocation target exceptions

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974905#comment-14974905 ] Steve Loughran commented on SPARK-11317: I'll do this as soon as SPARK-11265 is in; I've factored

[jira] [Created] (SPARK-11373) Add metrics to the History Server and providers

2015-10-28 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11373: -- Summary: Add metrics to the History Server and providers Key: SPARK-11373 URL: https://issues.apache.org/jira/browse/SPARK-11373 Project: Spark Issue

[jira] [Commented] (SPARK-11373) Add metrics to the History Server and providers

2015-10-28 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978322#comment-14978322 ] Steve Loughran commented on SPARK-11373: # This has tangible benefit for the SPARK-1537 YARN ATS

[jira] [Created] (SPARK-11375) History Server "no histories" message to be dynamically generated by ApplicationHistoryProviders

2015-10-28 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11375: -- Summary: History Server "no histories" message to be dynamically generated by ApplicationHistoryProviders Key: SPARK-11375 URL:

[jira] [Commented] (SPARK-11375) History Server "no histories" message to be dynamically generated by ApplicationHistoryProviders

2015-10-28 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978329#comment-14978329 ] Steve Loughran commented on SPARK-11375: This could be implemented with a new method on

[jira] [Updated] (SPARK-11317) YARN HBase token code shouldn't swallow invocation target exceptions

2015-10-28 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11317: --- Affects Version/s: 1.5.1 > YARN HBase token code shouldn't swallow invocation target

[jira] [Updated] (SPARK-11317) YARN HBase token code shouldn't swallow invocation target exceptions

2015-10-28 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11317: --- Description: As with SPARK-11265; the HBase token retrieval code of SPARK-6918 1. swallows

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2015-10-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980278#comment-14980278 ] Steve Loughran commented on SPARK-2089: --- Bear in mind that if you start specifying locations, it

[jira] [Created] (SPARK-11314) Add service API and test service for Yarn Cluster schedulers

2015-10-26 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11314: -- Summary: Add service API and test service for Yarn Cluster schedulers Key: SPARK-11314 URL: https://issues.apache.org/jira/browse/SPARK-11314 Project: Spark

[jira] [Updated] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11265: --- Summary: YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster (was:

[jira] [Commented] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974239#comment-14974239 ] Steve Loughran commented on SPARK-11265: What's changed? The spark code uses reflection to get

[jira] [Comment Edited] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974239#comment-14974239 ] Steve Loughran edited comment on SPARK-11265 at 10/26/15 2:08 PM: --

[jira] [Created] (SPARK-11315) Add YARN extension service to publish Spark events to YARN timeline service

2015-10-26 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11315: -- Summary: Add YARN extension service to publish Spark events to YARN timeline service Key: SPARK-11315 URL: https://issues.apache.org/jira/browse/SPARK-11315

[jira] [Updated] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11265: --- Description: As reported on the dev list, trying to run a YARN client which wants to talk to

[jira] [Comment Edited] (SPARK-11265) YarnClient can't get tokens to talk to Hive 1.2.1 in a secure cluster

2015-10-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974239#comment-14974239 ] Steve Loughran edited comment on SPARK-11265 at 10/26/15 2:08 PM: --

[jira] [Created] (SPARK-11317) YARN HBase token code shouldn't swallow invocation target exceptions

2015-10-26 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11317: -- Summary: YARN HBase token code shouldn't swallow invocation target exceptions Key: SPARK-11317 URL: https://issues.apache.org/jira/browse/SPARK-11317 Project:

[jira] [Created] (SPARK-11323) Add History Service Provider to service application histories from YARN timeline server

2015-10-26 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-11323: -- Summary: Add History Service Provider to service application histories from YARN timeline server Key: SPARK-11323 URL: https://issues.apache.org/jira/browse/SPARK-11323

[jira] [Commented] (SPARK-11373) Add metrics to the History Server and providers

2015-11-09 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997215#comment-14997215 ] Steve Loughran commented on SPARK-11373: [~charlesyeh] I've just put up a pull request of what I

[jira] [Updated] (SPARK-10793) Make spark's use/subclassing of hive more maintainable

2015-11-11 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-10793: --- Summary: Make spark's use/subclassing of hive more maintainable (was: Make sparks

  1   2   3   4   5   6   7   8   9   >