Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15188
/cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
GitHub user marmbrus opened a pull request:
https://github.com/apache/spark/pull/15188
[SPARK-17627] Mark Streaming Providers Experimental
All of structured streaming is experimental in its first release. We
missed the annotation on two of the APIs.
You can merge this pull
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15102
> Either way, who are you to presume that a user doesn't know what she is
doing when she configured a consumer to start at a particular position for
an added partition?
I f
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79690215
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15102
> My bigger concern is that it looks like you guys are continuing to hack
in a particular direction, without addressing my points or answering whether
you're willing to let me help work
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79676581
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14553#discussion_r79493142
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Source.scala
---
@@ -30,16 +30,37 @@ trait Source {
/** Returns the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14553#discussion_r79498410
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Source.scala
---
@@ -30,16 +30,37 @@ trait Source {
/** Returns the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14553#discussion_r79492259
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -72,13 +74,17 @@ class StreamExecution
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14803#discussion_r79488089
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -197,10 +197,13 @@ case class DataSource
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14803#discussion_r79488934
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala
---
@@ -608,6 +608,34 @@ class FileStreamSourceSuite extends
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79088749
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79089396
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79088295
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79088253
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala
---
@@ -0,0 +1,186 @@
+/*
+ * Licensed to
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79089541
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79089110
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79089641
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79088325
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15102#discussion_r79088914
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
---
@@ -0,0 +1,446 @@
+/*
--- End diff
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15102
> This already does depend on most of the existing Kafka DStream
implementation
I pushed for this code to be copied rather than refactored because I think
this is the right direct
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/15054#discussion_r79056087
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -360,6 +360,7 @@ trait CheckAnalysis extends
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15023
Thanks for understanding! I do hope you guys upgrade eventually, there's a
lot of good stuff and 2.0.1 should be out in the near future. Please do report
any issues you see :)
---
If
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15023
Thanks for spending the time to backport this, but it does seem a little
risky to include changes to the configuration system in a maintenance release.
As such, I'd probably error on the si
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14728#discussion_r75786696
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -17,21 +17,18 @@
package
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14124
@cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14356
/cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
GitHub user marmbrus opened a pull request:
https://github.com/apache/spark/pull/14356
[SPARK-16724] Expose DefinedByConstructorParams
We don't generally make things in catalyst/execution private. Instead they
are just undocumented due to their lack of stability guarantees.
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14252
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14087
/cc @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14087#discussion_r71025786
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala
---
@@ -331,6 +331,24 @@ class FileStreamSourceSuite extends
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14214
Thanks for working on this, but I'm tempted to close this as "won't fix".
Its likely we are going to have to rewrite the incremental planner completely
for 2.1 and
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
Oh, I see what you are saying, although I'm not sure I agree with the
conclusion. Given that tests can run in parallel I don't think you actually
want to toggle back and forth between
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
I think thats where we are today. All query tests use LA and the harness
configures that. The problem before this PR was this one suite was setting LA
(due to its base class), and then UTC (due
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
All the tests in SQL are written to assume `Los_Angeles`, so I think this
is actually desired. Otherwise people have to configure their machine
specially to run spark tests.
---
If your project
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2e97f3a08 -> 7de183d97
[SPARK-16531][SQL][TEST] Remove timezone setting from
DataFrameTimeWindowingSuite
## What changes were proposed in this pull request?
It's unnecessary. `QueryTest` already sets it.
Author: Burak Yavuz
Closes
Repository: spark
Updated Branches:
refs/heads/master 01f09b161 -> 0744d84c9
[SPARK-16531][SQL][TEST] Remove timezone setting from
DataFrameTimeWindowingSuite
## What changes were proposed in this pull request?
It's unnecessary. `QueryTest` already sets it.
Author: Burak Yavuz
Closes #141
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
Thanks, merging to master and 2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
We should put this in 2.0 for whoever merges.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14170
LGTM, can you make a JIRA? Its a little scary to change tests w/o one in
case there is flakiness.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14139
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13890#discussion_r70139445
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -74,13 +74,71 @@ object RDDConversions
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14094
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14094#discussion_r69986165
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -45,6 +47,7 @@ class FileStreamSource
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14094#discussion_r69985831
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -26,6 +27,7 @@ import
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14030#discussion_r69819131
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ForeachSink.scala
---
@@ -30,7 +32,42 @@ import org.apache.spark.sql.{DataFrame
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/14030#discussion_r69819064
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -155,7 +155,7 @@ private[sql] object Dataset {
class Dataset[T] private[sql
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13873
/cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13890
/cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14002
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/14000
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13901
No tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13939
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13939#discussion_r68841421
--- Diff:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveWindowFunctionQuerySuite.scala
---
@@ -569,6 +572,7 @@ class
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13939#discussion_r68840940
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ---
@@ -196,6 +185,10 @@ private[sql] class HiveSessionCatalog
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13862
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13862#discussion_r68148358
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
---
@@ -43,13 +48,16 @@ case class PartitionedFile
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13718#discussion_r67740050
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -211,6 +217,7 @@ class StreamSuite extends StreamTest
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13718#discussion_r67736904
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -211,6 +217,7 @@ class StreamSuite extends StreamTest
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13718#discussion_r67734059
--- Diff: core/src/main/scala/org/apache/spark/util/ManualClock.scala ---
@@ -57,9 +59,19 @@ private[spark] class ManualClock(private var time: Long
Repository: spark
Updated Branches:
refs/heads/master 905f774b7 -> 5cfabec87
[SPARK-16050][TESTS] Remove the flaky test: ConsoleSinkSuite
## What changes were proposed in this pull request?
ConsoleSinkSuite just collects content from stdout and compare them with the
expected string. However,
Repository: spark
Updated Branches:
refs/heads/branch-2.0 0b0b5fe54 -> 363db9f8b
[SPARK-16050][TESTS] Remove the flaky test: ConsoleSinkSuite
## What changes were proposed in this pull request?
ConsoleSinkSuite just collects content from stdout and compare them with the
expected string. Howe
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13776
LGTM, merging to master and 2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67578285
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13727
A few comments. Overall LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67575918
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67575723
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -276,7 +267,45 @@ class DataFrameReader private[sql](sparkSession
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67575684
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -276,7 +267,45 @@ class DataFrameReader private[sql](sparkSession
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13740
There are examples in `quietly`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13740
LGTM and we should merge before the RC.
How hard to add a test? You could redirect stdout temporarily?
---
If your project is set up for it, you can reply to this email and have your
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67573034
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67572740
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67572585
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67572462
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -228,4 +220,101 @@ class DataFrameReaderWriterSuite
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67572121
--- Diff:
sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameReaderWriterSuite.java
---
@@ -0,0 +1,158 @@
+/*
+* Licensed to the Apache
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67572068
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -368,6 +397,63 @@ class DataFrameReader private[sql](sparkSession
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13718
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13718#discussion_r67435503
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -545,6 +545,13 @@ object SQLConf {
.booleanConf
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13673
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13653#discussion_r67029686
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13653
Overall looks pretty good. Feel free to merge after addressing comments /
passing tests to avoid more conflicts.
---
If your project is set up for it, you can reply to this email and have your
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13653#discussion_r67023372
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13653#discussion_r67023216
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13653#discussion_r67023075
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -0,0 +1,401 @@
+/*
+ * Licensed to the Apache
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13653#discussion_r67022595
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -0,0 +1,288 @@
+/*
+ * Licensed to the Apache
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13638
Hmmm, does not apply cleanly to 1.6. @ueshin if you have time it might be
nice to backport.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Repository: spark
Updated Branches:
refs/heads/branch-2.0 d5e60748b -> 83aa17d44
[SPARK-15915][SQL] Logical plans should use canonicalized plan when override
sameResult.
## What changes were proposed in this pull request?
`DataFrame` with plan overriding `sameResult` but not using canonicali
Repository: spark
Updated Branches:
refs/heads/master bc02d0112 -> c5b735581
[SPARK-15915][SQL] Logical plans should use canonicalized plan when override
sameResult.
## What changes were proposed in this pull request?
`DataFrame` with plan overriding `sameResult` but not using canonicalized
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13638
Yeah, sounds reasonable. Merging to master, 2.0 and 1.6.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13638#discussion_r66884087
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13638#discussion_r66882083
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13638
Seems reasonable. Is this a regression from 1.6?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13638#discussion_r66876008
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/8416
@rxin I believe I fixed that limitation in my recent refactoring.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13424
Thanks! Merged into master and 2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Repository: spark
Updated Branches:
refs/heads/master aec502d91 -> 127a6678d
[SPARK-15489][SQL] Dataset kryo encoder won't load custom user settings
## What changes were proposed in this pull request?
Serializer instantiation will consider existing SparkConf
## How was this patch tested?
man
Repository: spark
Updated Branches:
refs/heads/branch-2.0 bc53422ad -> e6ebb547b
[SPARK-15489][SQL] Dataset kryo encoder won't load custom user settings
## What changes were proposed in this pull request?
Serializer instantiation will consider existing SparkConf
## How was this patch tested?
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13147
Thanks, merged to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Repository: spark
Updated Branches:
refs/heads/master fb219029d -> 667d4ea7b
[SPARK-6320][SQL] Move planLater method into GenericStrategy.
## What changes were proposed in this pull request?
This PR moves `QueryPlanner.planLater()` method into `GenericStrategy` for
extra strategies to be abl
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/13424
LGTM, can you update the description (it still says WIP).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
601 - 700 of 8851 matches
Mail list logo
