Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22895#discussion_r229563086
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -100,9 +100,14 @@ case class AvroDataToCatalyst
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22883
Seems reasonable. Ping @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22895#discussion_r229553580
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -100,9 +100,14 @@ case class AvroDataToCatalyst
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22895#discussion_r229553381
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -100,9 +100,14 @@ case class AvroDataToCatalyst
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22864
Looks like the parameter is not used from the first day?
https://github.com/apache/spark/pull/290/files#diff-b8adb646ef90f616c34eb5c98d1ebd16R140
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22883#discussion_r229535706
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -1105,6 +1095,15 @@ private[spark] class AppStatusListener
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22887
Hi @gjhkael ,
can you explain more about why you make the change?
Did you try `spark.SessionState.newHadoopConf
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22895
[SPARK-25886][SQL] Improve error message of `FailureSafeParser` and
`from_avro` in FAILFAST mode
## What changes were proposed in this pull request?
Currently in `FailureSafeParser
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22890
[SPARK-25883][SQL][Minor] Override method `prettyName` in
`from_avro`/`to_avro`
## What changes were proposed in this pull request?
Previously in from_avro/to_avro, we override
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22878
@xuanyuanking , thanks for the work!
You can try editing the previous commit message
https://help.github.com/articles/creating-a-commit-with-multiple-authors/ , and
then `push -f
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22861
Personally I am against accessing the main args in such way. It looks a bit
ugly.
But if we have to move everything to `BenchmarkBase`, then maybe this is
the way
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22847#discussion_r228600757
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -812,6 +812,17 @@ object SQLConf {
.intConf
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22848#discussion_r228559067
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala ---
@@ -89,12 +89,12 @@ private class LongLongTupleConverter
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22848
[SPARK-25851][SQL] Fix deprecated API warning in SQLListener
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/21596, Jackson is upgraded to
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22823#discussion_r228407600
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -910,12 +910,14 @@ class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r228392145
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/package.scala ---
@@ -31,10 +32,32 @@ package object avro {
* @since 2.4.0
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r228388968
--- Diff: docs/sql-data-sources-avro.md ---
@@ -177,6 +180,19 @@ Data source options of Avro can be set using the
`.option` method on `DataFrameR
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22814
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22814
@cloud-fan @HyukjinKwon I have changed the default value to "FAILFAST" and
remove the migration guide.
See the PR description f
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r228148950
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/avro/AvroFunctionsSuite.scala
---
@@ -61,6 +59,24 @@ class AvroFunctionsSuite extends
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r228086740
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -21,16 +21,31 @@ import org.apache.avro.Schema
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r228064742
--- Diff: docs/sql-data-sources-avro.md ---
@@ -177,6 +180,18 @@ Data source options of Avro can be set using the
`.option` method on `DataFrameR
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22814
The major code is ready.
I will add a new section in doc soon.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22814#discussion_r227755767
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala
---
@@ -60,9 +60,11 @@ trait
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22814
[WIP][SPARK-25819][SQL] Support parse mode option for the function
`from_avro`
## What changes were proposed in this pull request?
Current the function `from_avro` throws exception
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22663
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22808
Hi @dongjoon-hyun , I see. I have updated the title.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22808
@srowen @felixcheung @shivusondur
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22808
[SPARK-25675][UI][FOLLOWUP][TEST] Fix test failure in PagedTableSuite
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/22668, the PR was
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22668
@srowen I think it is easy fix, let me fix it now.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22237
@HyukjinKwon Got it. I won't take it since you will do it.
---
-
To unsubscribe, e-mail: reviews-uns
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22663
@dongjoon-hyun Sorry to reply late. I have updated the title and
description.
---
-
To unsubscribe, e-mail: reviews
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22663
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22746
This is cool +1 ð
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22746#discussion_r225921716
--- Diff: docs/sql-reference.md ---
@@ -0,0 +1,641 @@
+---
+layout: global
+title: Reference
+displayTitle: Reference
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22752#discussion_r225846631
--- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala
---
@@ -64,4 +64,11 @@ private[spark] object config
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22752#discussion_r225845782
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -449,7 +450,7 @@ private[history] class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22753#discussion_r225810815
--- Diff: docs/_layouts/global.html ---
@@ -184,7 +184,8 @@ {{ page.title
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22753#discussion_r225809921
--- Diff: docs/_layouts/global.html ---
@@ -184,7 +184,8 @@ {{ page.title
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22742
Hi @heuermh ,
I left some comments in JIRA yesterday. I tried the test case in
branch-2.3(with tag v2.3.1 and v2.3.0), the case is still reproduced by running:
```
./build/sbt
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22753
@srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22753
[SPARK-25754][DOC] Change CDN for MathJax
## What changes were proposed in this pull request?
Currently when we open our doc site:
https://spark.apache.org/docs/latest/index.html
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22746#discussion_r225783658
--- Diff: docs/sql-reference.md ---
@@ -0,0 +1,641 @@
+---
+layout: global
+title: Reference
+displayTitle: Reference
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22744
@srowen @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22744
[SPARK-25741][WebUI] Long URLs are not rendered properly in web UI
## What changes were proposed in this pull request?
When the URL for description column in the table of job/stage
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22727#discussion_r225418151
--- Diff: sbin/start-thriftserver.sh ---
@@ -39,6 +39,10 @@ function usage {
pattern+="\|Spark Command: "
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22727#discussion_r225392908
--- Diff:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
---
@@ -71,6 +71,12 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22727#discussion_r225175351
--- Diff:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
---
@@ -71,6 +71,12 @@ object
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22727
@srowen @dongjoon-hyun @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22727
[SPARK-25735][CORE][MINOR]Improve start-thriftserver.sh: print clean usage
and exit with code 1
## What changes were proposed in this pull request?
Currently if we run
```
sh
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22714
BTW there is another approach: https://github.com/apache/spark/pull/21512
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22699
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22709#discussion_r224831076
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala
---
@@ -67,21 +71,28 @@ object SchemaConverters
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22709
[SPARK-25718][SQL]Detect recursive reference in Avro schema and throw
exception
## What changes were proposed in this pull request?
Avro schema allows recursive reference, e.g. the
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22699
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22699
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22700#discussion_r224549311
--- Diff: sbin/start-master.sh ---
@@ -31,7 +31,8 @@ if [[ "$@" = *--help ]] || [[ "$@" = *-h ]]; then
echo "Usag
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22699
Agree. Let me remove them in this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22700#discussion_r224537568
--- Diff: sbin/start-master.sh ---
@@ -31,7 +31,8 @@ if [[ "$@" = *--help ]] || [[ "$@" = *-h ]]; then
echo "Usag
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22699#discussion_r224508786
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryServerArguments.scala
---
@@ -34,26 +34,25 @@ private[history] class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22699#discussion_r224506748
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryServerArguments.scala
---
@@ -34,26 +34,25 @@ private[history] class
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22699#discussion_r224506025
--- Diff: sbin/start-history-server.sh ---
@@ -28,7 +28,22 @@ if [ -z "${SPARK_HOME}" ]; then
export SPARK_HOME="$(cd "`di
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22700#discussion_r224502646
--- Diff: sbin/start-master.sh ---
@@ -31,7 +31,8 @@ if [[ "$@" = *--help ]] || [[ "$@" = *-h ]]; then
echo "Usag
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22700
[SPARK-25712][Core][Minor] Improve usage message
## What changes were proposed in this pull request?
Currently if we run
```
./sbin/start-master.sh -h
```
We get
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22699
[SPARK-25711][Core] Allow history server to show usage
## What changes were proposed in this pull request?
Currently, if we try run
```
./start-history-server.sh -h
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22668
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22668#discussion_r224323509
--- Diff: core/src/main/scala/org/apache/spark/ui/PagedTable.scala ---
@@ -154,9 +150,6 @@ private[ui] trait PagedTable[T
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22684#discussion_r224186347
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala
---
@@ -138,39 +138,75 @@ private[sql] object
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22595
I read the original PR https://github.com/apache/spark/pull/14617, and it
is wired that the two columns are hidden in the very beginning.
Also the screenshot in PR description doesn&#
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22668#discussion_r224101829
--- Diff: core/src/main/scala/org/apache/spark/ui/PagedTable.scala ---
@@ -123,10 +123,9 @@ private[ui] trait PagedTable[T
Github user gengliangwang closed the pull request at:
https://github.com/apache/spark/pull/22687
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22687
It's OK. Close this one.
Thanks for reviewing @viirya @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-uns
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22687
@viirya @HyukjinKwon I did the code changes and then I found the condition
is not reachable, as I have stated in PR description.
Just feel that it won't hurt to have such handli
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22687
[SPARK-25702][SQL] Push down filters with `Not` operator in Parquet
## What changes were proposed in this pull request?
Currently, in ParquetFilters, predicates inside `Not` operator
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22684
@dbtsai @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22684
[SPARK-25699][SQL] Partially push down conjunctive predicated in Orc
## What changes were proposed in this pull request?
Inspired by https://github.com/apache/spark/pull/22574
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22663
@dongjoon-hyun Is the changes OK to you?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22643
@dongjoon-hyun please take another look, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22663#discussion_r223221336
--- Diff: core/benchmarks/KryoBenchmark-results.txt ---
@@ -0,0 +1,29
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22663
[SPARK-25490][SQL][TEST] Refactor KryoBenchmark to use main method
## What changes were proposed in this pull request?
Also make the scale smaller(reduce to 20%) to avoid OOM
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22643
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22642
@dongjoon-hyun thanks, I have created new Jira ticket:
https://issues.apache.org/jira/browse/SPARK-25653
---
-
To
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22643
[SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest
## What changes were proposed in this pull request?
There was 5 suites extends `HadoopFsRelationTest`, for testing
&quo
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22642
[SPARK-25613][SPARK-25614][TEST] Add tag ExtendedHiveTest for
HiveSparkSubmitSuite
## What changes were proposed in this pull request?
The total run time of `HiveSparkSubmitSuite
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22640
[SPARK-25621][SPARK-25622][TEST] Reduce test time of
BucketedReadWithHiveSupportSuite
## What changes were proposed in this pull request?
By reduce the number of loops.
- `read
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22611
@HyukjinKwon Thanks for the review :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22611#discussion_r16887
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala ---
@@ -100,6 +77,50 @@ private[avro] class AvroFileFormat
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22609
Hi @mridulm ,
it seems that the changes will affect the metrics in `SparkStatusTracker`,
e.g. number of active tasks in `getExecutorInfos` or `getStageInfo` .
Maybe we
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22599
Discuss with @cloud-fan offline. Rename method `benchmark` as
`runBenchmarkSuite`. Also add comment to guide developers to use
`runBenchmark` for each scenario in implementations
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22611
[SPARK-25595] Ignore corrupt Avro files if flag IGNORE_CORRUPT_FILES enabled
## What changes were proposed in this pull request?
With flag IGNORE_CORRUPT_FILES enabled, schema
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22599#discussion_r221637241
--- Diff:
core/src/test/scala/org/apache/spark/benchmark/BenchmarkBase.scala ---
@@ -25,7 +25,10 @@ import java.io.{File, FileOutputStream
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22599#discussion_r221636295
--- Diff:
core/src/test/scala/org/apache/spark/benchmark/BenchmarkBase.scala ---
@@ -25,7 +25,10 @@ import java.io.{File, FileOutputStream
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22599
@wangyum @dongjoon-hyun @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22599
[SPARK-25581][SQL] Rename method `benchmark` as `benchmarkSuite` in
`BenchmarkBase`
## What changes were proposed in this pull request?
Rename method `benchmark` in `BenchmarkBase
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22484#discussion_r221417293
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SqlBasedBenchmark.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22484#discussion_r221416202
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SqlBasedBenchmark.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22484#discussion_r221415701
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SqlBasedBenchmark.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22484#discussion_r221415642
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SqlBasedBenchmark.scala
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22237
I manually ran some queries.
My point was about the value of the column `from_json`, `null` vs
`Row(null, null, ...)`.
Now I am +1 with the proposal of this PR, making the column
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22237
Hi @MaxGekk ,
I just reviewed this PR. I noticed that there is one behavior change. The
column value of `from_json(corrupt_record...)` become `Row(null, nulll, ...)`,
instead of `null
101 - 200 of 801 matches
Mail list logo
