[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20138 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-17 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r162160545 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -663,6 +665,95 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-16 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161882635 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -663,6 +665,95 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-16 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161880321 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -405,49 +404,70 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161660926 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -663,6 +665,95 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-14 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161443974 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -405,49 +404,70 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161099778 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -663,6 +665,95 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161099310 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -834,6 +906,9 @@ private[history] case class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161098356 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -544,73 +621,75 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161095561 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -405,49 +404,70 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-11 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r161082423 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -405,49 +404,70 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-04 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r159691083 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -806,6 +867,17 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-04 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r159690722 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -544,73 +621,75 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-04 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20138#discussion_r159705360 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -806,6 +867,17 @@ private[history] class

[GitHub] spark pull request #20138: [SPARK-20664][core] Delete stale application data...

2018-01-02 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20138 [SPARK-20664][core] Delete stale application data from SHS. Detect the deletion of event log files from storage, and remove data about the related application attempt in the SHS. Also