[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20894 LGTM Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91399/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91399/testReport)** for PR 20894 at commit [`3b37712`](https://github.com/apache/spark/commit/3b37712ded664aaf716306574f50072e58b9bbd1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91399/testReport)** for PR 20894 at commit [`3b37712`](https://github.com/apache/spark/commit/3b37712ded664aaf716306574f50072e58b9bbd1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @gatorsmile @cloud-fan @HyukjinKwon @gengliangwang Could you look at the PR one more time. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91155/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)** for PR 20894 at commit [`26ae4f9`](https://github.com/apache/spark/commit/26ae4f9b624a92414f089743a4a747230b654738). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #91155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)** for PR 20894 at commit [`26ae4f9`](https://github.com/apache/spark/commit/26ae4f9b624a92414f089743a4a747230b654738). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90993/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90993/testReport)** for PR 20894 at commit [`7dce1e7`](https://github.com/apache/spark/commit/7dce1e72f080044ba471fa08bb572452d4b0c907). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class TaskKilled(` * `case class KafkaContinuousInputPartition(` * `trait HasValidationIndicatorCol extends Params ` * `trait DivModLike extends BinaryArithmetic ` * `case class Divide(left: Expression, right: Expression) extends DivModLike ` * `case class Remainder(left: Expression, right: Expression) extends DivModLike ` * `case class ExprCode(var code: Block, var isNull: ExprValue, var value: ExprValue)` * `trait Block extends JavaCode ` * ` implicit class BlockHelper(val sc: StringContext) extends AnyVal ` * `case class CodeBlock(codeParts: Seq[String], blockInputs: Seq[JavaCode]) extends Block ` * `case class Blocks(blocks: Seq[Block]) extends Block ` * `case class MapEntries(child: Expression) extends UnaryExpression with ExpectsInputTypes ` * `class ReadOnlySQLConf(context: TaskContext) extends SQLConf ` * `class TaskContextConfigProvider(context: TaskContext) extends ConfigProvider ` * `case class RateStreamContinuousInputPartition(` * `case class ContinuousShuffleReadPartition(index: Int, queueSize: Int) extends Partition ` * `class ContinuousShuffleReadRDD(` * `trait ContinuousShuffleReader ` * `class MemoryStreamInputPartition(records: Array[UnsafeRow])` * `class ContinuousMemoryStreamInputPartition(` * `class RateStreamMicroBatchInputPartition(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90993/testReport)** for PR 20894 at commit [`7dce1e7`](https://github.com/apache/spark/commit/7dce1e72f080044ba471fa08bb572452d4b0c907). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @gatorsmile @cloud-fan @HyukjinKwon @gengliangwang May I ask you to look at the PR again. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90809/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90809/testReport)** for PR 20894 at commit [`11c7591`](https://github.com/apache/spark/commit/11c75913f4afadd0437945e0a41870a141326375). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90808/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90808 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90808/testReport)** for PR 20894 at commit [`9606711`](https://github.com/apache/spark/commit/9606711c55333c2d299f694ddf168304a22cbd48). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90809/testReport)** for PR 20894 at commit [`11c7591`](https://github.com/apache/spark/commit/11c75913f4afadd0437945e0a41870a141326375). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90808 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90808/testReport)** for PR 20894 at commit [`9606711`](https://github.com/apache/spark/commit/9606711c55333c2d299f694ddf168304a22cbd48). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90756/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90756/testReport)** for PR 20894 at commit [`795a878`](https://github.com/apache/spark/commit/795a8781129d5265906d537ad87a3e9b0abc890f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90756/testReport)** for PR 20894 at commit [`795a878`](https://github.com/apache/spark/commit/795a8781129d5265906d537ad87a3e9b0abc890f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90738/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90738/testReport)** for PR 20894 at commit [`04199e0`](https://github.com/apache/spark/commit/04199e0fd28ae0f5e21050eb4d08ec60d238cab5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90738/testReport)** for PR 20894 at commit [`04199e0`](https://github.com/apache/spark/commit/04199e0fd28ae0f5e21050eb4d08ec60d238cab5). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20894 I understand the default value of `enforceSchema` must be true to keep backward compatibility, but shall we suggest users to set it to false to avoid incorrect result in the document? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 Right, will proceed reviewing this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90682/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90682/testReport)** for PR 20894 at commit [`e3b4275`](https://github.com/apache/spark/commit/e3b4275d71d2230b9833d92435157a54cdc0b7e0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20894 In this PR, when documenting the option `enforceSchema`, we need to emphasize the schemas are checked by positions instead of by names. @ssimeonov `UNION` is a good example. Many Spark users feel confused about it. In the last release (SPARK 2.3), we added a new Dataset API `unionByName`. I think that is why we should also consider this in the follow-up PR. When we fetch the data from the external data sources, we respect the embedded/built-in schema and do the column reordering when building the plan. In addition, this sounds a pretty general issue for reading/writing from/to the external data sources. We can consider this option in the other built-in data sources and also in the design of data source API v2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90682/testReport)** for PR 20894 at commit [`e3b4275`](https://github.com/apache/spark/commit/e3b4275d71d2230b9833d92435157a54cdc0b7e0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 cc @cloud-fan, I would like to listen what you think too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 Can we check if there's any committer positive on this change before going ahead? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20894 ping @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @gengliangwang @gatorsmile May I ask you to look at this PR one more time. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90598/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90598/testReport)** for PR 20894 at commit [`21f8b10`](https://github.com/apache/spark/commit/21f8b10dda4b0ef71ba69cc6147d1cf8614812f1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90598/testReport)** for PR 20894 at commit [`21f8b10`](https://github.com/apache/spark/commit/21f8b10dda4b0ef71ba69cc6147d1cf8614812f1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90589/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90589/testReport)** for PR 20894 at commit [`21f8b10`](https://github.com/apache/spark/commit/21f8b10dda4b0ef71ba69cc6147d1cf8614812f1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 Documentation does solve the problem in a way because we are going to claim what Spark supports and does. How about adding a script or something to find out by reading ahead or another way? I don't see too much value on it. If you guys think it's worth, please go ahead. I think I said don't block on me. I am okay. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user ssimeonov commented on the issue: https://github.com/apache/spark/pull/20894 @HyukjinKwon we are one of the Spark users experiencing this problem in the real world: dealing with dirty data produced by a variety of third party systems. Documentation doesn't solve anything here: it simply punts a complicated series of traversals and manual header checks. We've had to implement them to address the issue in the absence of this capability. I agree that there are (too) many reader (and writer) options in Spark. That's fundamentally a side effect of architecture choices: Spark offers no user-accessible lifecycle hooks for custom loading/validation (or saving, for that matter). This is a fundamental limitation, something I've discussed with @rxin and @marmbrus in the past. (For example, at Swoop, we've had to completely re-implement `DataFrameWriter` as `OpenDataFrameWriter` to expose options & the data being written so that the API can be extended via implicits.) Let's not blame this PR based a Spark architecture choice. In the absence of the abovementioned hooks, the only way to meaningfully extend Spark's loading behavior is by jamming more code into data sources and adding more options to control it. Given that (1) this is the Spark 2.x approach to data sources, (2) the issue this PR addresses can lead to silent correctness problems and (3) users have no easy way, within Spark, to identify and correct the issue, I believe this PR serves a very useful purpose in the Spark ecosystem. By analogy, consider SQL's UNION, which can cause problems because columns are combined not by name but by position, hence, `(x: Int, y: Int)` can be happily UNIONed with `(y: Int, x: Int)` to create, very likely, gobbledygook. The difference in this case is that (a) the SQL UNION behavior is standardized, (b) it has been well-documented for decades and (c) the operands' schema can be easily inspected via Spark. This is a great situation where users who end up in trouble can be told to RTFM. By contrast, the Spark CSV data source behavior is (a) arbitrary, (b) undocumented and (c) Spark provides no useful tools for users to check whether they'll get in trouble. Great job, @MaxGekk! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90589/testReport)** for PR 20894 at commit [`21f8b10`](https://github.com/apache/spark/commit/21f8b10dda4b0ef71ba69cc6147d1cf8614812f1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90581/testReport)** for PR 20894 at commit [`2bd2713`](https://github.com/apache/spark/commit/2bd27136ae9095beec429ff15a6a5f1be0464419). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90581/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90581/testReport)** for PR 20894 at commit [`2bd2713`](https://github.com/apache/spark/commit/2bd27136ae9095beec429ff15a6a5f1be0464419). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 I don't have a particularly better solution for now. Documentation _might be_ one option in such case given past discussions and suggestions from other committers and reviewers. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 > For example, there are so many options that can be potentially added (other univocity parser options). You are right, so many things can be added but in this particular case we are trying to solve a problem related to correctness. This is the issue to which our customers faced in real life, and lost a few days to restore their data only because Spark produces wrong result **silently**. Any solution which checks CSV header against schema will require an option or global parameter to disable it for keeping backward compatibility. @HyukjinKwon What kind of solution would be appropriate for this problem from your point of view? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20894 I understood the rationale, problem and the approach and I don't feel strongly - I am less sure since 1. this option is specific to CSV about columns. It sounds adding complexity. 2. however, I doubt if the severity of the issue is worth enough adding an option and that amount of change. I believe I usually have been staying against in most of such cases in particular in JIRA level (probably worth enough checking Won't fix JIRAs). For clarification, please proceed if you guys feel it's needed and don't block on me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @HyukjinKwon @gengliangwang @gatorsmile Please, have a look at it again. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90215/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90215/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90215/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90049/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90049/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90049/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
> The case exists in all data source format, right?
Not in all, for example, JSON datasource is more tolerant to field order in
json records. Let's say if you have the schema:
```
val schema = new StructType().add("f1", IntegerType).add("f2", IntegerType)
```
you can read files from the same folder with different order of fields:
*1.json*
```
{"f1":1, "f2":2}
```
*2.json*
```
{"f2":22, "f1":11}
```
```
spark.read.schema(schema).json("json-dir")
res0.show
+---+---+
| f1| f2|
+---+---+
| 11| 22|
| 1| 2|
+---+---+
```
> If user didn't provide schema, should we check the header among CSV files?
If the user didn't provide the schema, it will be inferred (if
`inferSchema` is set, proper types will be inferred otherwise string types).
So, the inferred schema will be verified against actual CSV headers with this
changes.
> Users should be responsible for the specifying data schema.
Yes but it can be inferred too and checked during parsing.
> The proposed behavior can only help users to avoid manually checking the
CSV headers.
Yes, this is the problem reported by our customers. They have multiple CSV
files received from different sources. So, some files have different order of
columns. And Spark returns wrong result silently. The expected behavior must be
an error (with file name) or right result (data in columns must belong to right
columns in loaded dataframe like in JSON datasource).
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20894 Some questions here: 1. The case exists in all data source format, right? 2. If user didn't provide schema, should we check the header among CSV files? 3. Users should be responsible for the specifying data schema. The proposed behavior can only help users to avoid manually checking the CSV headers. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90011/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90011/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #90011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90011/testReport)** for PR 20894 at commit [`ad6cda4`](https://github.com/apache/spark/commit/ad6cda4c9ffe46831d956c5fc92a272d98a4e731). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20894 cc @gengliangwang Could you review this CSV PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89940/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89940/testReport)** for PR 20894 at commit [`1fffc16`](https://github.com/apache/spark/commit/1fffc1614c5028fcbaf88bb07b9e75d56646aec1). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes ` * `case class ArrayJoin(` * `case class ArrayPosition(left: Expression, right: Expression)` * `case class ElementAt(left: Expression, right: Expression) extends GetMapValueUtil ` * `case class Concat(children: Seq[Expression]) extends Expression ` * `case class Flatten(child: Expression) extends UnaryExpression ` * `abstract class GetMapValueUtil extends BinaryExpression with ImplicitCastInputTypes ` * `case class GetMapValue(child: Expression, key: Expression)` * `case class MonthsBetween(` * `trait QueryPlanConstraints extends ConstraintHelper ` * `trait ConstraintHelper ` * `case class CachedRDDBuilder(` * `case class InMemoryRelation(` * `case class WriteToContinuousDataSource(` * `case class WriteToContinuousDataSourceExec(writer: StreamWriter, query: SparkPlan)` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89940/testReport)** for PR 20894 at commit [`1fffc16`](https://github.com/apache/spark/commit/1fffc1614c5028fcbaf88bb07b9e75d56646aec1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 @gatorsmile @HyukjinKwon Could you look at the PR again, please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89501/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89501/testReport)** for PR 20894 at commit [`9b2d403`](https://github.com/apache/spark/commit/9b2d403085f45b0d975cc3e7d5ac559aa81e0c64). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89501/testReport)** for PR 20894 at commit [`9b2d403`](https://github.com/apache/spark/commit/9b2d403085f45b0d975cc3e7d5ac559aa81e0c64). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89497/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89497/testReport)** for PR 20894 at commit [`9b2d403`](https://github.com/apache/spark/commit/9b2d403085f45b0d975cc3e7d5ac559aa81e0c64). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89497/testReport)** for PR 20894 at commit [`9b2d403`](https://github.com/apache/spark/commit/9b2d403085f45b0d975cc3e7d5ac559aa81e0c64). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89496/testReport)** for PR 20894 at commit [`a5f2916`](https://github.com/apache/spark/commit/a5f2916ef1f3516eed07e2ccea37688aef322689). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89496/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20894 **[Test build #89496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89496/testReport)** for PR 20894 at commit [`a5f2916`](https://github.com/apache/spark/commit/a5f2916ef1f3516eed07e2ccea37688aef322689). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20894 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89365/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20894 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
