spark git commit: [SPARK-21786][SQL][FOLLOWUP] Add compressionCodec test for CTAS

2018-09-07 Thread dongjoon
TAS not supported. Now it has been fixed in #20522 , the test case should be enabled too. ## How was this patch tested? This only re-enables the test cases of previous PR. Closes #22302 from fjh100456/compressionCodec. Authored-by: fjh100456 Signed-off-by: Dongjoon Hyun (cherry picked from com

spark git commit: [SPARK-21786][SQL][FOLLOWUP] Add compressionCodec test for CTAS

2018-09-07 Thread dongjoon
TAS not supported. Now it has been fixed in #20522 , the test case should be enabled too. ## How was this patch tested? This only re-enables the test cases of previous PR. Closes #22302 from fjh100456/compressionCodec. Authored-by: fjh100456 Signed-off-by: Dongjoon Hyun Project: http://git-

spark git commit: [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore

2018-09-09 Thread dongjoon
ce").show +---+ | A| +---+ | 3| | 2| | 4| | 1| | 0| +---+ ``` See #22148 for more details about parquet data source reader. ## How was this patch tested? Unit tests added. Closes #22262 from seancxmao/SPARK-25175. Authored-by: seancxmao Signed-off-by: Dongjoon Hyun Project: http://gi

spark git commit: [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore

2018-09-09 Thread dongjoon
ce").show +---+ | A| +---+ | 3| | 2| | 4| | 1| | 0| +---+ ``` See #22148 for more details about parquet data source reader. ## How was this patch tested? Unit tests added. Closes #22262 from seancxmao/SPARK-25175. Authored-by: seancxmao Signed-off-by: Dongjoon Hyun (cherry picked

spark-website git commit: Add Dongjoon Hyun to the list of committers

2018-08-30 Thread dongjoon
Repository: spark-website Updated Branches: refs/heads/asf-site 7c294769a -> 92a85c6c2 Add Dongjoon Hyun to the list of committers Author: Dongjoon Hyun Closes #140 from dongjoon-hyun/add_dongjoon. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-

spark git commit: [DOC] Update some outdated links

2018-09-04 Thread dongjoon
ttp://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn Fix files which use these links. Closes #22321 from kisimple/docfix. Authored-by: blueszheng Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/re

spark git commit: [SPARK-25389][SQL] INSERT OVERWRITE DIRECTORY STORED AS should prevent duplicate fields

2018-09-11 Thread dongjoon
ewly added test cases. Closes #22378 from dongjoon-hyun/SPARK-25389. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit 77579aa8c35b0d98bbeac3c828bf68a1d190d13e) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: ht

spark git commit: [SPARK-25543][K8S] Print debug message iff execIdsRemovedInThisRound is not empty.

2018-09-30 Thread dongjoon
red-by: Prashant Sharma Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4da541a5 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4da541a5 Diff: http://git-wip-us.apache.org/repos/asf/spark/d

spark git commit: [SPARK-25543][K8S] Print debug message iff execIdsRemovedInThisRound is not empty.

2018-09-30 Thread dongjoon
red-by: Prashant Sharma Signed-off-by: Dongjoon Hyun (cherry picked from commit 4da541a5d23b039eb549dd849cf121bdc8676e59) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c886f050 Tree: http://git-

spark git commit: [SPARK-25700][SQL] Creates ReadSupport in only Append Mode in Data Source V2 write path

2018-10-11 Thread dongjoon
.com/Possible-bug-in-DatasourceV2-td25343.html ## How was this patch tested? Unit test and manual tests. Closes #22688 from HyukjinKwon/append-revert-2. Authored-by: hyukjinkwon Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.

spark git commit: [SPARK-25664][SQL][TEST] Refactor JoinBenchmark to use main method

2018-10-12 Thread dongjoon
:runMain org.apache.spark.sql.execution.benchmark.JoinBenchmark" ``` ## How was this patch tested? manual tests Closes #22661 from wangyum/SPARK-25664. Lead-authored-by: Yuming Wang Co-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: h

spark git commit: [SPARK-25760][SQL] Set AddJarCommand return empty

2018-10-18 Thread dongjoon
st/resources/TestUDTF.jar; ADD JAR /Users/yumwang/spark/sql/hive/src/test/resources/TestUDTF.jar spark-sql> ``` Closes #22747 from wangyum/AddJarCommand. Authored-by: Yuming Wang Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apach

spark git commit: [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans

2018-10-18 Thread dongjoon
ion of `ClusteringEvaluator` in order to evaluate the clustering. ## How was this patch tested? NA Closes #22756 from mgaido91/SPARK-25758. Authored-by: Marco Gaido Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/re

spark git commit: [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans

2018-10-18 Thread dongjoon
ion of `ClusteringEvaluator` in order to evaluate the clustering. ## How was this patch tested? NA Closes #22756 from mgaido91/SPARK-25758. Authored-by: Marco Gaido Signed-off-by: Dongjoon Hyun (cherry picked from commit c2962546d9a5900a5628a31b83d2c4b22c3a7936) Signed-off-by: Dongjoon H

spark git commit: [SPARK-24499][DOC][FOLLOW-UP] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread dongjoon
ark SQL 2.4 to 3.0` when merging to 2.4 ## How was this patch tested? N/A Closes #22769 from gatorsmile/test2.4. Authored-by: gatorsmile Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fd5b2

spark git commit: [SPARK-21402][SQL][BACKPORT-2.2] Fix java array of structs deserialization

2018-10-18 Thread dongjoon
ata, not on the java bean. ## How was this patch tested? Added a test case. Built complete project on travis. dongjoon-hyun cloud-fan Closes #22768 from vofque/SPARK-21402-2.2. Lead-authored-by: Vladimir Kuriatkov Co-authored-by: Vladimir Kuriatkov Signed-off-by: Dongjoon Hyun Project: http://git-

spark git commit: [SPARK-25795][R][EXAMPLE] Fix CSV SparkR SQL Example

2018-10-22 Thread dongjoon
in/resources/people.csv", "csv", sep=";", > inferSchema=T, header=T) > namesAndAges <- select(df, "name", "age") ``` Closes #22791 from dongjoon-hyun/SPARK-25795. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://gi

spark git commit: [SPARK-25795][R][EXAMPLE] Fix CSV SparkR SQL Example

2018-10-22 Thread dongjoon
s/src/main/resources/people.csv", "csv", sep=";", > inferSchema=T, header=T) > namesAndAges <- select(df, "name", "age") ``` Closes #22791 from dongjoon-hyun/SPARK-25795. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked fro

spark git commit: [SPARK-25795][R][EXAMPLE] Fix CSV SparkR SQL Example

2018-10-22 Thread dongjoon
s/src/main/resources/people.csv", "csv", sep=";", > inferSchema=T, header=T) > namesAndAges <- select(df, "name", "age") ``` Closes #22791 from dongjoon-hyun/SPARK-25795. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked fro

spark git commit: [SPARK-25269][SQL] SQL interface support specify StorageLevel when cache table

2018-10-19 Thread dongjoon
rom wangyum/SPARK-25269. Authored-by: Yuming Wang Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9ad0f6ea Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9ad0f6ea Diff: http://git-

spark git commit: [SPARK-25757][BUILD] Upgrade netty-all from 4.1.17.Final to 4.1.30.Final

2018-10-20 Thread dongjoon
2GB to merge everything, which will create OOM errors. And we can avoid this issue by upgrade the netty. https://github.com/netty/netty/pull/8038 ## How was this patch tested? Manual tests in some spark jobs. Closes #22765 from lipzhu/SPARK-25757. Authored-by: Zhu, Lipeng Signed-off-by:

spark git commit: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeRowConversion

2018-10-20 Thread dongjoon
xec` ## How was this patch tested? existing tests Closes #22750 from cloud-fan/minor. Authored-by: Wenchen Fan Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ab5752cb Tree: http://git-

spark git commit: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use main method

2018-10-20 Thread dongjoon
quot;sql/test:runMain org.apache.spark.sql.execution.benchmark.WideSchemaBenchmark" ``` ## How was this patch tested? manual tests Closes #22501 from wangyum/SPARK-25492. Lead-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apa

spark git commit: [SPARK-25779][SQL][TESTS] Remove SQL query tests for function documentation by DESCRIBE FUNCTION at SQLQueryTestSuite

2018-10-20 Thread dongjoon
nual. Closes #22776 from HyukjinKwon/SPARK-25779. Authored-by: hyukjinkwon Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b8c6ba9e Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b8c6ba

spark git commit: [SPARK-25711][CORE] Improve start-history-server.sh: show usage User-Friendly and remove deprecated options

2018-10-13 Thread dongjoon
nts) spark.history.fs.updateIntervalHow often to reload log data from storage (in seconds, default: 10) ``` ## How was this patch tested? Manual test Closes #22699 from gengliangwang/refactorSHSUsage. Authored-by: Gengliang Wang Signed-off-by: Dongj

spark git commit: [SPARK-25726][SQL][TEST] Fix flaky test in SaveIntoDataSourceCommandSuite

2018-10-13 Thread dongjoon
" contained "123" (SaveIntoDataSourceCommandSuite.scala:42) ``` ## How was this patch tested? Pass the Jenkins with the updated test case Closes #22716 from dongjoon-hyun/SPARK-25726. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit 6bbceb9fefe815d1

spark git commit: [SPARK-25726][SQL][TEST] Fix flaky test in SaveIntoDataSourceCommandSuite

2018-10-13 Thread dongjoon
" contained "123" (SaveIntoDataSourceCommandSuite.scala:42) ``` ## How was this patch tested? Pass the Jenkins with the updated test case Closes #22716 from dongjoon-hyun/SPARK-25726. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit 6bbceb9fefe815d1

spark git commit: [SPARK-25726][SQL][TEST] Fix flaky test in SaveIntoDataSourceCommandSuite

2018-10-13 Thread dongjoon
" contained "123" (SaveIntoDataSourceCommandSuite.scala:42) ``` ## How was this patch tested? Pass the Jenkins with the updated test case Closes #22716 from dongjoon-hyun/SPARK-25726. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/a

spark git commit: [SPARK-25726][SQL][TEST] Fix flaky test in SaveIntoDataSourceCommandSuite

2018-10-13 Thread dongjoon
" contained "123" (SaveIntoDataSourceCommandSuite.scala:42) ``` ## How was this patch tested? Pass the Jenkins with the updated test case Closes #22716 from dongjoon-hyun/SPARK-25726. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit 6bbceb9fefe815d1

spark git commit: [SPARK-25727][SQL] Add outputOrdering to otherCopyArgs in InMemoryRelation

2018-10-13 Thread dongjoon
ode.scala:599) at org.apache.spark.sql.catalyst.trees.TreeNode.jsonValue(TreeNode.scala:604) at org.apache.spark.sql.catalyst.trees.TreeNode.toJSON(TreeNode.scala:590) ``` ## How was this patch tested? Added a test Closes #22715 from gatorsmile/copyArgs1. Authored-by: gatorsmile Signed-off-by: Dongjoon

spark git commit: [SPARK-25727][SQL] Add outputOrdering to otherCopyArgs in InMemoryRelation

2018-10-13 Thread dongjoon
ode.scala:599) at org.apache.spark.sql.catalyst.trees.TreeNode.jsonValue(TreeNode.scala:604) at org.apache.spark.sql.catalyst.trees.TreeNode.toJSON(TreeNode.scala:590) ``` ## How was this patch tested? Added a test Closes #22715 from gatorsmile/copyArgs1. Authored-by: gatorsmile Signed-off-by: Dongjoon Hyun

spark git commit: [SPARK-25812][UI][TEST] Fix test failure in PagedTableSuite

2018-10-23 Thread dongjoon
rom gengliangwang/fixPagedTableSuite. Authored-by: Gengliang Wang Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/65a8d1b8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/65a8d1b8 Diff: http://

spark git commit: [SPARK-25656][SQL][DOC][EXAMPLE] Add a doc and examples about extra data source options

2018-10-23 Thread dongjoon
rce specific options to the underlying data source. According to [the review comment](https://github.com/apache/spark/pull/22622#discussion_r222911529), this PR aims to add more detailed information and examples ## How was this patch tested? Manual. Closes #22801 from dongjoon-hyun/SPARK-25

spark git commit: [SPARK-25797][SQL][DOCS][BACKPORT-2.3] Add migration doc for solving issues caused by view canonicalization approach change

2018-10-28 Thread dongjoon
yll serve --watch ``` Closes #22851 from seancxmao/SPARK-25797-2.3. Authored-by: seancxmao Signed-off-by: Dongjoon Hyun (cherry picked from commit 3e0160bacfbe4597f15ca410ca832617cdeeddca) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://

spark git commit: [SPARK-25797][SQL][DOCS][BACKPORT-2.3] Add migration doc for solving issues caused by view canonicalization approach change

2018-10-28 Thread dongjoon
yll serve --watch ``` Closes #22851 from seancxmao/SPARK-25797-2.3. Authored-by: seancxmao Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3e0160ba Tree: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-25893][SQL] Show a directional error message for unsupported Hive Metastore versions

2018-10-31 Thread dongjoon
re version (2.4). Please set spark.sql.hive.metastore.version with a valid version. ``` ## How was this patch tested? Manual. Closes #22902 from dongjoon-hyun/SPARK-25893. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-25891][PYTHON] Upgrade to Py4J 0.10.8.1

2018-10-31 Thread dongjoon
had better have this to get the official support. Also, there are some patches related to garbage collections. https://www.py4j.org/changelog.html#py4j-0-10-8-and-py4j-0-10-8-1 ## How was this patch tested? Pass the Jenkins. Closes #22901 from dongjoon-hyun/SPARK-25891. Authored-by: Dongjoon H

spark git commit: [SPARK-25848][SQL][TEST] Refactor CSVBenchmarks to use main method

2018-10-30 Thread dongjoon
:runMain org.apache.spark.sql.execution.datasources.csv.CSVBenchmark"` ## How was this patch tested? manual tests Closes #22845 from heary-cao/CSVBenchmarks. Authored-by: caoxuewen Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apa

spark git commit: [SPARK-25833][SQL][DOCS] Update migration guide for Hive view compatibility

2018-10-31 Thread dongjoon
lly. ``` SKIP_API=1 jekyll serve ``` Closes #22868 from seancxmao/SPARK-25833. Authored-by: seancxmao Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/49bea5a7 Tree: http://git-wip-us.apache.org/repos/

spark git commit: [SPARK-25618][SQL][TEST] Reduce time taken to execute KafkaContinuousSourceStressForDontFailOnDataLossSuite

2018-10-31 Thread dongjoon
wal/SPARK-25618. Authored-by: Dilip Biswal Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/34c3bc9f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/34c3bc9f Diff: http://git-wip-us.apache.

spark git commit: [SPARK-25663][SPARK-25661][SQL][TEST] Refactor BuiltInDataSourceWriteBenchmark, DataSourceWriteBenchmark and AvroWriteBenchmark to use main method

2018-10-31 Thread dongjoon
;avro/test:runMain org.apache.spark.sql.execution.benchmark.AvroWriteBenchmark" ``` ## How was this patch tested? manual tests Closes #22861 from yucai/BuiltInDataSourceWriteBenchmark. Lead-authored-by: yucai Co-authored-by: Yucai Yu Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoo

spark git commit: [SPARK-25656][SQL][DOC][EXAMPLE][BRANCH-2.4] Add a doc and examples about extra data source options

2018-10-25 Thread dongjoon
nce it's not supported in ORC 1.5.2. ## How was this patch tested? Manual. Closes #22839 from dongjoon-hyun/SPARK-25656-2.4. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-25665][SQL][TEST] Refactor ObjectHashAggregateExecBenchmark to…

2018-10-25 Thread dongjoon
; ``` Closes #22804 from peter-toth/SPARK-25665. Lead-authored-by: Peter Toth Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ccd07b73 Tree: http://git-wip-us.apa

spark git commit: [SPARK-25931][SQL] Benchmarking creation of Jackson parser

2018-11-03 Thread dongjoon
gt; `Select a subset of 10 columns`. Closes #22920 from MaxGekk/json-benchmark-follow-up. Lead-authored-by: Maxim Gekk Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/com

spark git commit: [SPARK-25918][SQL] LOAD DATA LOCAL INPATH should handle a relative path

2018-11-02 Thread dongjoon
NTO TABLE t") org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: kv1.txt; ``` ## How was this patch tested? Pass the Jenkins Closes #22927 from dongjoon-hyun/SPARK-LOAD. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit e91b607719886b57d1550a70

spark git commit: [SPARK-25918][SQL] LOAD DATA LOCAL INPATH should handle a relative path

2018-11-02 Thread dongjoon
NTO TABLE t") org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: kv1.txt; ``` ## How was this patch tested? Pass the Jenkins Closes #22927 from dongjoon-hyun/SPARK-LOAD. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-25840][BUILD] `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread dongjoon
ally, I did before.) This PR aims to recover that script to work in any way. This doesn't aim for source artifacts to reproduce the compiled artifacts. ## How was this patch tested? Manual. ``` $ rm LICENSE-binary $ dev/make-distribution.sh ``` Closes #22840 from dongjoon-hyun/SPARK-25840. Aut

spark git commit: [SPARK-25840][BUILD] `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread dongjoon
ally, I did before.) This PR aims to recover that script to work in any way. This doesn't aim for source artifacts to reproduce the compiled artifacts. ## How was this patch tested? Manual. ``` $ rm LICENSE-binary $ dev/make-distribution.sh ``` Closes #22840 from dongjoon-hyun/SPARK-25840. Authored

spark git commit: [MINOR][TEST][BRANCH-2.4] Regenerate golden file `datetime.sql.out`

2018-10-25 Thread dongjoon
ual [reverting](https://github.com/dongjoon-hyun/spark/commit/5d744499667fcd08825bca0ac6d5d90d6e110ebc#diff-79dd276be45ede6f34e24ad7005b0a7cR87). This doens't cause test failure because the difference is inside `comments` and blank lines. We had better fix this minor issue before RC5. ##

spark git commit: [MINOR][DOC] Update the building doc to use Maven 3.5.4 and Java 8 only

2018-10-20 Thread dongjoon
ent to describe Java 8 only. ## How was this patch tested? N/A (This is a document only change.) Closes #22781 from dongjoon-hyun/SPARK-JDK-DOC. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit fc9ba9dcc6ad47fbd05f093b94e7e1358d5f) Signed-off-by: Dongjoon H

spark git commit: [MINOR][DOC] Update the building doc to use Maven 3.5.4 and Java 8 only

2018-10-20 Thread dongjoon
ibe Java 8 only. ## How was this patch tested? N/A (This is a document only change.) Closes #22781 from dongjoon-hyun/SPARK-JDK-DOC. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/

spark git commit: [DOC][MINOR] Fix minor error in the code of graphx guide

2018-10-20 Thread dongjoon
s fixed error relates to `[SPARK-12995][GraphX] Remove deprecate APIs from Pregel` ## How was this patch tested? N/A Closes #22780 from WeichenXu123/minor_doc_update1. Authored-by: WeichenXu Signed-off-by: Dongjoon Hyun (cherry picked from commit 3b4f35f568eb3844d2a789c8a409bc705477df6b) Signed-off

spark git commit: [DOC][MINOR] Fix minor error in the code of graphx guide

2018-10-20 Thread dongjoon
s fixed error relates to `[SPARK-12995][GraphX] Remove deprecate APIs from Pregel` ## How was this patch tested? N/A Closes #22780 from WeichenXu123/minor_doc_update1. Authored-by: WeichenXu Signed-off-by: Dongjoon Hyun (cherry picked from commit 3b4f35f568eb3844d2a789c8a409bc705477df6b) Signed-off

spark git commit: [DOC][MINOR] Fix minor error in the code of graphx guide

2018-10-20 Thread dongjoon
s fixed error relates to `[SPARK-12995][GraphX] Remove deprecate APIs from Pregel` ## How was this patch tested? N/A Closes #22780 from WeichenXu123/minor_doc_update1. Authored-by: WeichenXu Signed-off-by: Dongjoon Hyun (cherry picked from commit 3b4f35f568eb3844d2a789c8a409bc705477df6b) Signed-off

spark git commit: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.5.1)

2018-11-05 Thread dongjoon
5.1. ## How was this patch tested? AppVeyor Closes #22948 from HyukjinKwon/SPARK-25944. Authored-by: hyukjinkwon Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/486acda8 Tree: http://git-

spark git commit: [SPARK-22827][SQL][FOLLOW-UP] Throw `SparkOutOfMemoryError` in `HashAggregateExec`, too.

2018-11-08 Thread dongjoon
ror` to avoid killing the entire executor when an `OutOfMemoryError` is thrown. We should throw `SparkOutOfMemoryError` in `HashAggregateExec`, too. ## How was this patch tested? Existing tests. Closes #22969 from ueshin/issues/SPARK-22827/oome. Authored-by: Takuya UESHIN Signed-off-by: Dongj

spark git commit: [SPARK-25971][SQL] Ignore partition byte-size statistics in SQLQueryTestSuite

2018-11-08 Thread dongjoon
, 2 rows +Partition Statistics [not included in comparison] bytes, 2 rows ``` ## How was this patch tested? Pass the Jenkins with the newly updated test cases. Closes #22972 from dongjoon-hyun/SPARK-25971. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-

spark git commit: [SPARK-25964][SQL][MINOR] Revise OrcReadBenchmark/DataSourceReadBenchmark case names and execution instructions

2018-11-08 Thread dongjoon
DataSourceReadBenchmark, for the benchmark "String with Nulls Scan" . ## How was this patch tested? Re-run benchmarks Closes #22965 from gengliangwang/fixHiveOrcReadBenchmark. Lead-authored-by: Gengliang Wang Co-authored-by: Gengliang Wang Co-authored-by: Dongjoon Hyun Signed-off-by: D

spark git commit: [MINOR] update HiveExternalCatalogVersionsSuite to test 2.4.0

2018-11-08 Thread dongjoon
How was this patch tested? N/A Closes #22984 from cloud-fan/minor. Authored-by: Wenchen Fan Signed-off-by: Dongjoon Hyun (cherry picked from commit 973f7c01df0788b6f5d21224d96c33f14c5b8c64) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://

spark git commit: [MINOR] update HiveExternalCatalogVersionsSuite to test 2.4.0

2018-11-08 Thread dongjoon
How was this patch tested? N/A Closes #22984 from cloud-fan/minor. Authored-by: Wenchen Fan Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/973f7c01 Tree: http://git-wip-us.apache.org/repos/asf/spark/t

spark git commit: [SPARK-25098][SQL] Trim the string when cast stringToTimestamp and stringToDate

2018-11-06 Thread dongjoon
sts Closes https://github.com/apache/spark/pull/22089 Closes #22943 from wangyum/SPARK-25098. Authored-by: Yuming Wang Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9e9fa2f6 Tree: http://git-

spark git commit: [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWideTable to use main method

2018-11-06 Thread dongjoon
ult: ``` SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.WideTableBenchmark" ``` ## How was this patch tested? manual tests Closes #22823 from yucai/BenchmarkWideTable. Lead-authored-by: yucai Co-authored-by: Yucai Yu Co-authored-by:

spark git commit: [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0

2018-11-14 Thread dongjoon
rom dbtsai/scala2.12. Authored-by: DB Tsai Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ad853c56 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ad853c56 Diff: http://git-wip-us.apache.

spark git commit: [SPARK-25965][SQL][TEST] Add avro read benchmark

2018-11-14 Thread dongjoon
ark Closes #22966 from gengliangwang/avroReadBenchmark. Lead-authored-by: Gengliang Wang Co-authored-by: Gengliang Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/com

spark git commit: [SPARK-25479][TEST] Refactor DatasetBenchmark to use main method

2018-10-04 Thread dongjoon
`sh SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.DatasetBenchmark" ``` ## How was this patch tested? manual tests Closes #22488 from wangyum/SPARK-25479. Lead-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: h

spark git commit: [SPARK-25538][SQL] Zero-out all bytes when writing decimal

2018-10-03 Thread dongjoon
ink is more efficient later. ## How was this patch tested? Running the test attached in the JIRA + added UT Closes #22602 from mgaido91/SPARK-25582. Authored-by: Marco Gaido Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.

spark git commit: [SPARK-25538][SQL] Zero-out all bytes when writing decimal

2018-10-03 Thread dongjoon
ink is more efficient later. ## How was this patch tested? Running the test attached in the JIRA + added UT Closes #22602 from mgaido91/SPARK-25582. Authored-by: Marco Gaido Signed-off-by: Dongjoon Hyun (cherry picked from commit d7ae36a810bfcbedfe7360eb2cdbbc3ca970e4d0) Signed-off-by: Dongj

spark git commit: [SPARK-25583][DOC][BRANCH-2.3] Add history-server related configuration in the documentation.

2018-10-03 Thread dongjoon
i31/SPARK-25583. Authored-by: Shahid Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7102aeeb Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7102aeeb Diff: http://git-wip-us.apache.org/re

spark git commit: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-03 Thread dongjoon
ong time. For Parquet data source, it's expected to be added with next Parquet release update. ## How was this patch tested? Manual. ```scala SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.BloomFilterBenchmark" ``` Closes #22605

spark git commit: [SPARK-25583][DOC] Add history-server related configuration in the documentation.

2018-10-02 Thread dongjoon
7b5.png) Closes #22601 from shahidki31/historyConf. Authored-by: Shahid Signed-off-by: Dongjoon Hyun (cherry picked from commit 71876633f3af706408355b5fb561b58dbc593360) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.

spark git commit: [SPARK-25583][DOC] Add history-server related configuration in the documentation.

2018-10-02 Thread dongjoon
png) Closes #22601 from shahidki31/historyConf. Authored-by: Shahid Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/71876633 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/71876633 D

spark git commit: [SPARK-25581][SQL] Rename method `benchmark` as `runBenchmarkSuite` in `BenchmarkBase`

2018-10-02 Thread dongjoon
339 ## How was this patch tested? Unit test. Closes #22599 from gengliangwang/renameBenchmarkSuite. Authored-by: Gengliang Wang Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7b4e94f1 Tree: http://

spark git commit: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

2018-10-02 Thread dongjoon
ses before merging Related Job: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-branch-2.2-lint/913/console Closes #22596 from samdvr/SPARK-25576. Lead-authored-by: Sam Davarnia Co-authored-by: Sam Davarnia <> Co-authored-by: Dongjoon Hyun Co-authored-by: Sam

spark git commit: [SPARK-23285][DOC][FOLLOWUP] Fix missing markup tag

2018-09-28 Thread dongjoon
ual via `SKIP_API=1 jekyll build`. Closes #22585 from dongjoon-hyun/SPARK-23285. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0b33f086 Tree: http://git-wip-us.apache.

spark git commit: [SPARK-25542][CORE][TEST] Move flaky test in OpenHashMapSuite to OpenHashSetSuite and make it against OpenHashSet

2018-09-28 Thread dongjoon
ted? Existing tests. Closes #22569 from viirya/SPARK-25542. Authored-by: Liang-Chi Hsieh Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b7d80349 Tree: http://git-wip-us.apache.org/repos/asf/spark/t

spark git commit: [SPARK-25542][CORE][TEST] Move flaky test in OpenHashMapSuite to OpenHashSetSuite and make it against OpenHashSet

2018-09-28 Thread dongjoon
ted? Existing tests. Closes #22569 from viirya/SPARK-25542. Authored-by: Liang-Chi Hsieh Signed-off-by: Dongjoon Hyun (cherry picked from commit b7d80349b0e367d78cab238e62c2ec353f0f12b3) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-

spark git commit: [SPARK-23285][DOC][FOLLOWUP] Fix missing markup tag

2018-09-28 Thread dongjoon
ted? Manual via `SKIP_API=1 jekyll build`. Closes #22585 from dongjoon-hyun/SPARK-23285. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun (cherry picked from commit 0b33f08683a41f6f3a6ec02c327010c0722cc1d1) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-25559][SQL] Remove the unsupported predicates in Parquet when possible

2018-09-28 Thread dongjoon
sai/removeUnsupportedPredicatesInParquet. Lead-authored-by: DB Tsai Co-authored-by: Dongjoon Hyun Co-authored-by: DB Tsai Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5d726b86 Tree: http://git-

spark git commit: [SPARK-25508][SQL][TEST] Refactor OrcReadBenchmark to use main method

2018-09-29 Thread dongjoon
ult: ``` SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "hive/test:runMain org.apache.spark.sql.hive.orc.OrcReadBenchmark" ``` ## How was this patch tested? manual tests Closes #22580 from yucai/SPARK-25508. Lead-authored-by: yucai Co-authored-by: Yucai Yu Co-authored-by: Dongjoon Hyun Signed-off-by:

spark-website git commit: Add a section to update test suites

2018-10-01 Thread dongjoon
Repository: spark-website Updated Branches: refs/heads/asf-site 8b7444182 -> 4ea5a5d8f Add a section to update test suites Author: Dongjoon Hyun Closes #151 from dongjoon-hyun/test_suite. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-

spark git commit: [SPARK-25476][SPARK-25510][TEST] Refactor AggregateBenchmark and add a new trait to better support Dataset and DataFrame API

2018-10-01 Thread dongjoon
; ``` ## How was this patch tested? manual tests Closes #22484 from wangyum/SPARK-25476. Lead-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/

spark git commit: [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to use main method

2018-10-03 Thread dongjoon
`sh SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.UnsafeArrayDataBenchmark" ``` ## How was this patch tested? manual tests Closes #22491 from wangyum/SPARK-25483. Lead-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyu

spark git commit: [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build.

2018-10-05 Thread dongjoon
dev build. Tested on local dev build. Closes #22634 from vanzin/SPARK-25646. Authored-by: Marcelo Vanzin Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/58287a39 Tree: http://git-wip-us.apache.

spark git commit: [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build.

2018-10-05 Thread dongjoon
sed in a dev build. Tested on local dev build. Closes #22634 from vanzin/SPARK-25646. Authored-by: Marcelo Vanzin Signed-off-by: Dongjoon Hyun (cherry picked from commit 58287a39864db463eeef17d1152d664be021d9ef) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method

2018-10-06 Thread dongjoon
`sh SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.MiscBenchmark" ``` ## How was this patch tested? manual tests Closes #22500 from wangyum/SPARK-25488. Lead-authored-by: Yuming Wang Co-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyu

spark git commit: [SPARK-25644][SS][FOLLOWUP][BUILD] Fix Scala 2.12 build error due to foreachBatch

2018-10-06 Thread dongjoon
ds.writeStream.foreachBatch((_, _) => {}).partitionBy("value").start() [error] ^ ``` ## How was this patch tested? Manual. Since this failure occurs in Scala-2.12 profile and test cases, Jenkins will not test this. We need to build with Scala-2.12 and run the tests

spark git commit: [SPARK-25644][SS][FOLLOWUP][BUILD] Fix Scala 2.12 build error due to foreachBatch

2018-10-06 Thread dongjoon
ds.writeStream.foreachBatch((_, _) => {}).partitionBy("value").start() [error] ^ ``` ## How was this patch tested? Manual. Since this failure occurs in Scala-2.12 profile and test cases, Jenkins will not test this. We need to build with Scala-2.12 and run the tests

spark git commit: [SPARK-25062][SQL] Clean up BlockLocations in InMemoryFileIndex

2018-10-06 Thread dongjoon
ume more memory. This PR adds the conversion to the latter case and decreases memory consumption. ## How was this patch tested? Added unit test. Closes #22603 from peter-toth/SPARK-25062. Authored-by: Peter Toth Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-25658][SQL][TEST] Refactor HashByteArrayBenchmark to use main method

2018-10-07 Thread dongjoon
catalyst/test:runMain org.apache.spark.sql.HashByteArrayBenchmark" ``` ## How was this patch tested? manual tests Closes #22652 from wangyum/SPARK-25658. Lead-authored-by: Yuming Wang Co-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: h

spark git commit: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaking change

2018-10-07 Thread dongjoon
ing changes. cc: WeichenXu123 ## How was this patch tested? Existing unit tests. Closes #22618 from mengxr/SPARK-25321.master. Authored-by: WeichenXu Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/com

spark git commit: [SPARK-25657][SQL][TEST] Refactor HashBenchmark to use main method

2018-10-07 Thread dongjoon
:runMain org.apache.spark.sql.HashBenchmark" ``` ## How was this patch tested? manual tests Closes #22651 from wangyum/SPARK-25657. Lead-authored-by: Yuming Wang Co-authored-by: Yuming Wang Co-authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/a

spark git commit: [SPARK-25425][SQL][BACKPORT-2.4] Extra options should override session options in DataSource V2

2018-09-19 Thread dongjoon
tra options in DataSource V2. Extra options are more specific and set via `.option()`, and should overwrite more generic session options. ## How was this patch tested? Added tests for read and write paths. Closes #22474 from MaxGekk/session-options-2.4. Authored-by: Maxim Gekk Signed-off-by: Dongj

spark git commit: Revert "[SPARK-23173][SQL] rename spark.sql.fromJsonForceNullableSchema"

2018-09-19 Thread dongjoon
61bc7) Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9fefb47f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9fefb47f Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9fefb47f

spark git commit: Revert "[SPARK-23173][SQL] rename spark.sql.fromJsonForceNullableSchema"

2018-09-19 Thread dongjoon
t: http://git-wip-us.apache.org/repos/asf/spark/commit/cb1b55cf Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/cb1b55cf Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/cb1b55cf Branch: refs/heads/master Commit: cb1b55cf771018f1560f6b173cdd7c6ca8061bc7 Parents: a71f6a1 Author:

spark git commit: [SPARK-25460][BRANCH-2.4][SS] DataSourceV2: SS sources do not respect SessionConfigSupport

2018-09-24 Thread dongjoon
red-by: hyukjinkwon Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ec384284 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ec384284 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ec384284 Bra

spark git commit: [SPARK-24519][CORE] Compute SHUFFLE_MIN_NUM_PARTS_TO_HIGHLY_COMPRESS only once

2018-09-26 Thread dongjoon
fig is computed only once. ## How was this patch tested? Removed a test case that's no longer valid. Closes #22521 from rxin/SPARK-24519. Authored-by: Reynold Xin Signed-off-by: Dongjoon Hyun Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-24519][CORE] Compute SHUFFLE_MIN_NUM_PARTS_TO_HIGHLY_COMPRESS only once

2018-09-26 Thread dongjoon
fig is computed only once. ## How was this patch tested? Removed a test case that's no longer valid. Closes #22521 from rxin/SPARK-24519. Authored-by: Reynold Xin Signed-off-by: Dongjoon Hyun (cherry picked from commit e702fb1d5218d062fcb8e618b92dad7958eb4062) Signed-off-by: Dongjoon Hyun Proj

spark git commit: [SPARK-25536][CORE] metric value for METRIC_OUTPUT_RECORDS_WRITTEN is incorrect

2018-09-26 Thread dongjoon
rom 'task.metrics.inputMetrics.recordsRead' to 'task.metrics.outputMetrics.recordsWritten'. This bug was introduced in SPARK-22190. https://github.com/apache/spark/pull/19426 ## How was this patch tested? Existing tests Closes #22555 from shahidki31/SPARK-25536. Authored-by: Shahid Signed-off-by: Dongjoon Hyun Proj

spark git commit: [SPARK-25536][CORE] metric value for METRIC_OUTPUT_RECORDS_WRITTEN is incorrect

2018-09-26 Thread dongjoon
rom 'task.metrics.inputMetrics.recordsRead' to 'task.metrics.outputMetrics.recordsWritten'. This bug was introduced in SPARK-22190. https://github.com/apache/spark/pull/19426 ## How was this patch tested? Existing tests Closes #22555 from shahidki31/SPARK-25536. Authored-by: Shahid Signed-off-by: Dongjoon H

spark git commit: [SPARK-25536][CORE] metric value for METRIC_OUTPUT_RECORDS_WRITTEN is incorrect

2018-09-26 Thread dongjoon
rom 'task.metrics.inputMetrics.recordsRead' to 'task.metrics.outputMetrics.recordsWritten'. This bug was introduced in SPARK-22190. https://github.com/apache/spark/pull/19426 ## How was this patch tested? Existing tests Closes #22555 from shahidki31/SPARK-25536. Authored-by: Shahid Signed-off-by: Dongjoon H

  1   2   3   4   5   6   7   8   9   >