[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12247 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-12 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-209024772 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59419309 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -129,8 +129,17 @@ trait SchemaRelationProvider { * Implemented

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208651263 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208651259 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208651028 **[Test build #55548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55548/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59303566 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -341,6 +347,33 @@ class FileStreamSourceSuite extends

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208616091 **[Test build #55548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55548/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208607923 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208607918 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208607307 **[Test build #55539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55539/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59297483 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -129,8 +129,17 @@ trait SchemaRelationProvider { * Implemented

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59297156 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -129,8 +129,17 @@ trait SchemaRelationProvider { * Implemented

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59296806 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -129,8 +129,17 @@ trait SchemaRelationProvider { * Implemented

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59295994 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -129,8 +129,17 @@ trait SchemaRelationProvider { * Implemented

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208593200 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208593199 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208592951 **[Test build #55537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55537/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208591595 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208591596 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208591339 **[Test build #55536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55536/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208576061 **[Test build #55539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55539/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59286840 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -341,6 +347,33 @@ class FileStreamSourceSuite extends

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59282240 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -341,6 +347,33 @@ class FileStreamSourceSuite extends

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59280899 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -341,6 +347,33 @@ class FileStreamSourceSuite extends

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208559400 **[Test build #55537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55537/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208559378 Updated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-208558013 **[Test build #55536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55536/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59256046 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala --- @@ -67,12 +62,33 @@ class FileStreamSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59255827 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -123,8 +123,16 @@ case class DataSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59255253 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala --- @@ -67,12 +62,33 @@ class FileStreamSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59254961 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/MemorySinkSuite.scala --- @@ -59,7 +59,7 @@ class MemorySinkSuite extends StreamTest with

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59254687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -123,8 +123,16 @@ case class DataSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59254285 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala --- @@ -341,6 +347,33 @@ class FileStreamSourceSuite extends

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59253946 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala --- @@ -67,12 +62,33 @@ class FileStreamSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59253976 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/MemorySinkSuite.scala --- @@ -59,7 +59,7 @@ class MemorySinkSuite extends StreamTest with

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12247#discussion_r59253837 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -123,8 +123,16 @@ case class DataSource( }

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-08 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207516450 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207216362 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207216360 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207216132 **[Test build #55308 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55308/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207196573 **[Test build #55308 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55308/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207195658 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207152941 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207152943 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207152790 **[Test build #55270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55270/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207136914 **[Test build #55270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55270/consoleFull)** for PR 12247 at commit

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/12247 [SPARK-14474][SQL]Move FileSource offset log into checkpointLocation ## What changes were proposed in this pull request? Now that we have a single location for storing checkpointed state.