[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/21410
  
Thanks! Merged to master.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91151/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91151 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91151/testReport)**
 for PR 21410 at commit 
[`ac76544`](https://github.com/apache/spark/commit/ac7654415a3ec82c6cf3306e664cf09018c66db6).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91151 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91151/testReport)**
 for PR 21410 at commit 
[`ac76544`](https://github.com/apache/spark/commit/ac7654415a3ec82c6cf3306e664cf09018c66db6).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[gatorsmile]

Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/21410
  
LGTM except one minor comment.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[MaxGekk]

Github user MaxGekk commented on the issue:

https://github.com/apache/spark/pull/21410
  
@gatorsmile Could you look at the PR, please. The changes should help us in 
trouble shooting of customer's issues.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[MaxGekk]

Github user MaxGekk commented on the issue:

https://github.com/apache/spark/pull/21410
  
> Is there a way to identify where in the schema the issue is occurring?

We can catch the exceptions on each level of schema tree traversal, and 
show sub-trees in each catch. For example: `array>>` , the first exception will point out `struct`, 
the second one `array` and up to the "root" schema. 

> e.g., a.b.c where this is happening, is required to easily isolate the 
issue in the input data and resolve it.

I guess in the case of arrays and maps, you want to see indexes and keys. 
Could you provide concrete example with values and a schema (array, struct, 
map), and what kind of info the error should contain.

Just in case, I would propose to make such improvements in a separate PR.   


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91062/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91062 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91062/testReport)**
 for PR 21410 at commit 
[`26cc2f8`](https://github.com/apache/spark/commit/26cc2f84ee6324db23936e20816d240031211311).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91062 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91062/testReport)**
 for PR 21410 at commit 
[`26cc2f8`](https://github.com/apache/spark/commit/26cc2f84ee6324db23936e20816d240031211311).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[MaxGekk]

Github user MaxGekk commented on the issue:

https://github.com/apache/spark/pull/21410
  
jenkins, retest this, please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[ssimeonov]

Github user ssimeonov commented on the issue:

https://github.com/apache/spark/pull/21410
  
This is an excellent start and a worthy improvement. 

Is there a way to identify where in the schema the issue is occurring? For 
example, when you have a schema with many nested fields, the failing value is 
helpful but the breadcrumb trail, e.g., `a.b.c` where this is happening, is 
required to easily isolate the issue in the input data and resolve it. 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91034/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91034 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91034/testReport)**
 for PR 21410 at commit 
[`26cc2f8`](https://github.com/apache/spark/commit/26cc2f84ee6324db23936e20816d240031211311).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[AmplabJenkins]

Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/21410
  
Can one of the admins verify this patch?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21410: [SPARK-24366][SQL] Improving of error messages for type ...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/21410
  
**[Test build #91034 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91034/testReport)**
 for PR 21410 at commit 
[`26cc2f8`](https://github.com/apache/spark/commit/26cc2f84ee6324db23936e20816d240031211311).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org