Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-161062442
Updated patch. This does add a web UI filter, one which is being attached
to incomplete apps, and triggering checks of the underlying history, the goal
being when
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-160677674
This patch moves counters of actions into a metrics source, including
timing of operations. This is designed to be ready to integrate with
SPARK-11373: metrics
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-160670545
(catching up: looks like the git tool I'm using decided to make up a new
remote branch when I renamed the local. Fixing to track the old one and pushing
up again
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-160725719
Test run had 0 failing tests and a lot of stack related to connection
refused in jenkins ... possibly an"environmental" failure
---
If your project
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-160595631
Thinking about this some more, how about instead of trying to do something
clever with the REST UIs, web filters and stuff, we just have a background
thread
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46008320
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9571#issuecomment-159996648
jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9571#issuecomment-159996634
Added reworked design
* all metrics go through the `MetricsSystem`; the providers return an
optional `Source` from the `start()` call
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9571#issuecomment-16264
(Note that the POMs changed to pull in some more of the codahale servlets,
though only the health checks & thread dump are being registered. Hooking up
t
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-159970929
The test failing is the web UI one; after hitting the web UI there's a
check on the cache metrics. The fact that `lookup.count == 1` means the 2nd GET
didn't
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-159722837
OK, here's the latest revision
# lots of tests on the cache itself -all working. This includes cnew hecks
on eviction.
# merged in the @squito test
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-159691027
I'm having a go, though I'm feeling somewhat defeated
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9571#discussion_r45874777
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala ---
@@ -121,20 +132,36 @@ class HistoryServer
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9571#discussion_r45874984
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +709,69 @@ private[history] class FsHistoryProvider
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-159637773
@squito âI'm adding your test case to this patch. It's never worked, has
it?
---
If your project is set up for it, you can reply to this email and have your
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9297#issuecomment-159617284
While I support this feature, people already see problems with
scanning/loading app histories off the filesystem. This has the potential to
increase
Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/9913
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-159434408
This is a converged patch which
1. Splits out all caching into `ApplicationCache`, for isolation and
testing.
1. Now asks the history server (which
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9860#issuecomment-159237981
One thing about auto-refresh is it generates load on the back end on every
refresh, even if its a tab in the browser that isn't even visible. With a
manual reload
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9875#issuecomment-159241477
Is the situation where the client can only have tokens need to be covered?
As users may not have keytabs, only kinit-granted tickets, and they still have
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6545#issuecomment-158969374
...I'm looking at this and my PR; I think they are differently
interdependent.
1. This patch adds a new method to `ApplicationHistoryProvider
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-158971646
As discussed on #6545, I think we could actually switch to live UI
playback, provided the number of cached UIs was kept down.
What must be done here
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9913
[SPARK-7889] [Core] WiP History Caching: xu tingjun's patch against master
This is @XuTingjun's patch (#6545) reapplied to trunk with a bit of cleanup
(the probe in the history provider
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9860#issuecomment-158425481
Wont be of much merit on the completed history server UIs, so ought to be
something which can be completely omitted off the page
---
If your project is set up
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r45225663
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r45226057
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r45226337
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r45226729
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r45226180
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-157813169
OK, this is what I've done
1. AM doesn't propagate app/attempt IDs via system properties. It still
sets them for other things to use, including
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-157814298
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-157698220
will do...ignore the latest push as its a just a rebase of the existing
work prior to any other edits
---
If your project is set up for it, you can reply
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9608#discussion_r44526124
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -810,7 +810,7 @@ private[spark] object Utils extends Logging {
* Get
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9297#discussion_r44556620
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -96,6 +114,7 @@ private[spark] object JsonProtocol
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9608#discussion_r44526017
--- Diff: core/src/main/scala/org/apache/spark/HttpFileServer.scala ---
@@ -42,10 +42,11 @@ private[spark] class HttpFileServer(
fileDir.mkdir
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9608#discussion_r44539552
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -810,7 +810,7 @@ private[spark] object Utils extends Logging {
* Get
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9571#issuecomment-155900033
...I thought it did. If I'm wrong, then yes, all you need is a context.
Once that's in there, the HistorySource should be what's needed
---
If your project
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9608#discussion_r44597535
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -810,7 +810,7 @@ private[spark] object Utils extends Logging {
* Get
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-155937044
This just rolls back to the explicit app/attempt ID. it's got better
guarantees of what's coming down
---
If your project is set up for it, you can reply
Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/8766
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/8748
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9571
[SPARK-11373] [CORE] WiP Add metrics to the History Server and providers
This adds metrics to the history server, with the `FsHistoryProvider`
metering its load, performance
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9272#discussion_r44013419
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala ---
@@ -150,6 +152,14 @@ private[hive] class ClientWrapper
Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/9466
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9272#discussion_r43867512
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala ---
@@ -150,6 +152,14 @@ private[hive] class ClientWrapper
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9272#issuecomment-153688412
This is unrelated to the SPARK-11265 patch; that's all about getting
reflection to find the right methods. This is about UGI setup.
---
If your project is set up
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9272#discussion_r43867986
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/ClientWrapper.scala ---
@@ -150,6 +152,14 @@ private[hive] class ClientWrapper
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9438#issuecomment-153688927
that's what comes of trying to code at a conference; will cancel and
resubmit
---
If your project is set up for it, you can reply to this email and have your
Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/9438
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9466
[SPARK-11265] [YARN] YarnClient can't get tokens to talk to Hive in a
secureâ¦
Backport to branch-1.5 of SPARK-11265 patch.
The sole change is in `Client`, where there's no longer
GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/9438
[SPARK-11265] [YARN] YarnClient can't get tokens to talk to Hive in a
secure cluster - backport to branch-1.5
This is a backport of the [SPARK-11265] patch to Branch-1.5; won't compile
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9232#issuecomment-152815974
Will do. Same JIRA or a new backport one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-152816395
and I've already gone and moved to strings. never mind.
the existing attempt IDs are nice for humans in the web UI, and potentially
in the rest, but don't
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-152727218
Test failure is
{{org.apache.spark.sql.execution.datasources.parquet.ParquetFilterSuite}}; very
unlikely to be related
---
If your project is set up for it, you
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43488794
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,81 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43488706
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -1337,55 +1337,9 @@ object Client extends Logging {
conf
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43488997
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,31 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43488942
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,81 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43498840
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,41 @@ private[spark] abstract class
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-152536617
OK, new patch with uses String values for appId and app attempt ID in the
service extension binding âbut explicitly derives these from the YARN app &
att
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43498798
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,41 @@ private[spark] abstract class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43504882
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -17,17 +17,17 @@
package
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43488737
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,81 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43377401
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43374435
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -17,17 +17,17 @@
package
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43373740
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43376459
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,41 @@ private[spark] abstract class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43439082
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,41 @@ private[spark] abstract class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43439849
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,55 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43368611
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,55 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43369014
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -1337,55 +1337,9 @@ object Client extends Logging {
conf
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43246322
--- Diff:
yarn/src/test/scala/org/apache/spark/scheduler/cluster/SimpleExtensionService.scala
---
@@ -0,0 +1,39 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43246273
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,38 @@ private[spark] abstract class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43246543
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,137 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43246410
--- Diff:
yarn/src/test/scala/org/apache/spark/scheduler/cluster/StubApplicationAttemptId.scala
---
@@ -0,0 +1,50 @@
+/*
--- End diff
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43244486
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,137 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43242989
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtilSuite.scala
---
@@ -245,4 +247,55 @@ class YarnSparkHadoopUtilSuite
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43244882
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,137 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43244977
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,137 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43281460
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -17,17 +17,17 @@
package
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43280181
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,136 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43280831
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,136 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43281778
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,38 @@ private[spark] abstract class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43279624
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,136 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43281918
--- Diff:
yarn/src/test/scala/org/apache/spark/scheduler/cluster/ExtensionServiceIntegrationSuite.scala
---
@@ -0,0 +1,88 @@
+/*
+ * Licensed
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43279925
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala
---
@@ -0,0 +1,136 @@
+/*
+ * Licensed
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/8744#issuecomment-151935694
> BTW, do we need to take special care with #9297 when integrated with ATS?
1. an older history server will fail when it gets a newer event. The
curr
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9182#issuecomment-151931420
rebase to follow
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/5423#issuecomment-151933545
.. rebase in sync with (and containing) publisher and service extension
patches ...
---
If your project is set up for it, you can reply to this email and have
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43240828
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,99 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9238#discussion_r43100632
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -73,4 +73,11 @@ private[history] abstract class
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9221#issuecomment-151425243
you are right. I'd probably have to do it by releasing an upstream artifact
which will wait until run on your jenkins instance before posting your
credentials
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9297#discussion_r43098850
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -504,9 +542,28 @@ private[spark] object JsonProtocol {
case
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9238#discussion_r43109175
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -299,6 +300,54 @@ private[history] class FsHistoryProvider
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/9238#issuecomment-151462264
1. Adding new abstract methods is going to break providers which use the
current plugin point. Adding base implementations which throw an "unsupported"
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43189656
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,117 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43189502
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,117 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43194609
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,99 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43201346
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,99 @@ class YarnSparkHadoopUtil extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r43128655
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,117 @@ class YarnSparkHadoopUtil extends
801 - 900 of 1115 matches
Mail list logo
