Github user steveloughran closed the pull request at:
https://github.com/apache/spark/pull/6935
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-183169031
ps can you close this one now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r52184854
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,668 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-181426395
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-181440993
**[Test build #2522 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2522/consoleFull)**
for PR 6935 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-181440518
**[Test build #2522 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2522/consoleFull)**
for PR 6935 at commit
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r52184490
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,668 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r52202867
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -415,8 +488,59 @@ private[history] class FsHistoryProvider(conf:
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-180374115
rebased against master; switch from scan of HTML view to REST API to
enumerate listings of complete/incomplete apps, add @squito's ? arg redirection
and test
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-179849703
...thx for the feedback. I'm fixing the merge which is now triggering a
regression âmaybe a race condition in test startupâ
apps should go from
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-179850363
..I should add that it depends on the head attempt on the list being
complete; the filter in HistoryServer is very sensitive to ordering. If there's
an incomplete
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-178139026
jenkins, add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-178142661
hi @steveloughran sorry for another really long delay on my end. mostly
this looks fine, there are some style nits and a couple of comments that need
updating. I also
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51446687
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -42,6 +42,35 @@ import org.apache.spark.util.{Clock, SystemClock,
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51449903
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,484 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51451010
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -282,6 +297,197 @@ class HistoryServerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51445356
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51444707
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +517,55 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51447527
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -666,6 +804,40 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51447480
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -666,6 +804,40 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51447272
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -613,13 +739,23 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51450618
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -44,7 +57,7 @@ import
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51450584
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -21,16 +21,29 @@ import java.net.{HttpURLConnection, URL}
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51444027
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,667 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51446839
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -386,24 +437,49 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51450652
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -21,16 +21,29 @@ import java.net.{HttpURLConnection, URL}
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r51451496
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,667 @@
+/*
+ * Licensed to the Apache Software
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-176209826
@squito âhave you had a chance to look at the latest version? I think
I've addressed all your issues, and it is building and testing against the
current master
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-175048241
I'm going to add a warning note here pointing to
[HDFS-5478](https://issues.apache.org/jira/browse/HDFS-5478), some versions of
HDFS aren't picking up changes
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48548650
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-167676684
hi @steveloughran , I think maybe some commits didn't get pushed? I
noticed a few places where things didn't seem to quite line up to your
comments. (also there is a
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48363721
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364543
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48367142
--- Diff: docs/monitoring.md ---
@@ -69,36 +83,53 @@ follows:
+### Spark configuration options
+
Property
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48368082
--- Diff: docs/monitoring.md ---
@@ -69,36 +83,53 @@ follows:
+### Spark configuration options
+
Property
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364727
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364720
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364439
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +517,55 @@ private[history] class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364502
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48366527
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -21,15 +21,28 @@ import java.net.{HttpURLConnection, URL}
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48366891
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,191 @@ class HistoryServerSuite extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48363399
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364066
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +517,55 @@ private[history] class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48366085
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48366097
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r48364810
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-166966798
I've updated the patch with the comments, and reworked how the updated
probe works, removing the need to have provider-specific state cached, returned
in the
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-164740423
yeah, I found it surprisingly complex too ... if I known at the start I'd
have steered clear. And don't worry about any delay; there are some other spark
PRs of
Github user 3ourroom commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-164741165
NAVER - http://www.naver.com/
3ourr...@naver.com ëê» ë³´ë´ì ë©ì¼ ì´
ë¤ìê³¼ ê°ì ì´ì ë¡
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-164540400
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-164540371
Jenkins, add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47552578
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47553423
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47549529
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47550432
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47551365
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -33,7 +33,35 @@ private[spark] case class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47551552
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -73,4 +102,18 @@ private[history] abstract class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47551814
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -385,24 +448,48 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47551842
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -385,24 +448,48 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47552043
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +517,55 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47553168
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47553490
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47553757
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47554225
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -21,15 +21,28 @@ import java.net.{HttpURLConnection, URL}
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47555787
--- Diff: docs/monitoring.md ---
@@ -69,36 +83,53 @@ follows:
+### Spark configuration options
+
Property
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47556139
--- Diff: docs/monitoring.md ---
@@ -69,36 +83,53 @@ follows:
+### Spark configuration options
+
Property
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47554002
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/ApplicationCacheSuite.scala
---
@@ -0,0 +1,476 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47554403
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -55,7 +68,7 @@ class HistoryServerSuite extends SparkFunSuite
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47554883
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,191 @@ class HistoryServerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47554923
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,191 @@ class HistoryServerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47555335
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,191 @@ class HistoryServerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47572920
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47573757
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,658 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47573952
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -371,9 +434,9 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47574113
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +517,55 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47574767
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala ---
@@ -103,7 +93,7 @@ class HistoryServer(
// Note we don't use
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47574667
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +826,52 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-164591472
thanks for updating again @steveloughran . Sorry for the long break in my
reviews. Looks really close, I think I just found one bug with the redirects.
Other than
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163168780
**[Test build #47424 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47424/consoleFull)**
for PR 6935 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163196873
**[Test build #47424 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47424/consoleFull)**
for PR 6935 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163196933
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163196935
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163203560
Failing test is pyspark âpretty unlikely to be related
```
==
FAIL:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46989032
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,648 @@
+/*
+ * Licensed to the Apache Software
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46993661
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,648 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46995257
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,648 @@
+/*
+ * Licensed to the Apache Software
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46995458
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -73,4 +101,17 @@ private[history] abstract class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46995405
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -33,7 +33,35 @@ private[spark] case class
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46995031
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,648 @@
+/*
+ * Licensed to the Apache
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46995495
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationCache.scala ---
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46997833
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +827,54 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46998222
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala ---
@@ -146,22 +136,51 @@ class HistoryServer(
override def stop() {
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46999417
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,202 @@ class HistoryServerSuite extends
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r46997864
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +827,54 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47004101
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +827,54 @@ private[history] class FsHistoryProvider(conf:
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47006645
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,202 @@ class HistoryServerSuite extends
Github user squito commented on the pull request:
https://github.com/apache/spark/pull/6935#issuecomment-163002749
hi @steveloughran thanks for the updates. The only real issue I see now is
some inconsistency between whether the checks are time stamp or filesize based
(or both).
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47005658
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -430,8 +519,54 @@ private[history] class FsHistoryProvider(conf:
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47010882
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala ---
@@ -281,6 +296,202 @@ class HistoryServerSuite extends
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/6935#discussion_r47012771
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -73,4 +101,17 @@ private[history] abstract class
1 - 100 of 110 matches
Mail list logo
