[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Naresh Kumar (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362088#comment-16362088
 ] 

Naresh Kumar commented on SPARK-23403:
--

 I am sure that exception is occurred because of bad creation of RDD[Row], can 
you please tell me how to handle missing fields while creating explicit schema 
using StructType 

> java.lang.ArrayIndexOutOfBoundsException: 10
> 
>
> Key: SPARK-23403
> URL: https://issues.apache.org/jira/browse/SPARK-23403
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Shell
>Affects Versions: 2.2.0
>Reporter: Naresh Kumar
>Priority: Major
>
> java.lang.ArrayIndexOutOfBoundsException: 10, while retriving records from 
> Dataframe in spark-shell



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Naresh Kumar (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362076#comment-16362076
 ] 

Naresh Kumar commented on SPARK-23403:
--

yes , csv has multiple missing fields , you can see the file Content below.

00433543cd1b9c6f45f38b63aa013399,1-56a9192009049c73a2fe0a129fd807d2,2493,1259,,1480435851831,
00433543cd1b9c6f45f38b63aa023131,1-d335fa3bddcf4aa82d7571ac9d637e26,4155,,,721480435854844,237
00433543cd1b9c6f45f38b63aa06c44d,1-348922ba236fa0272e66ef7dcac2ab0a,12464,11,acceptable,,105,,99,1480435871046,
00433543cd1b9c6f45f38b63aa07cb70,1-52ecbdba397f70b37b7e8792302dbd56,12468,11,acceptable,,125,,95,1480435875052,
00433543cd1b9c6f45f38b63aa0a2053,1-90a7e486051f07c3e0703c2ae775f5b2,2499,1298,,1480435881841,
00433543cd1b9c6f45f38b63aa0b0568,1-000fd8f5457cea84b999668a50165d29,12478,11,acceptable,,175,,100,1480435885069,
00433543cd1b9c6f45f38b63aa0e1347,1-9ded0b0acd5371681690759e39ffecbc,2502,1084,,1480435896845,
00433543cd1b9c6f45f38b63aa107551,1-379bb80aac977a7db6e830379eef0f05,12500,11,acceptable,,77,,87,1480435907109,
00433543cd1b9c6f45f38b63aa111c74,1-91984d5ddab7c1ea03e3a4fabfab4eff,12502,11,acceptable,,71,,86,1480435909113,
00433543cd1b9c6f45f38b63aa155690,1-33f4e9733e9c34535113c557d25ae08c,12519,11,acceptable,,76,,85,1480435926143,
00433543cd1b9c6f45f38b63aa159058,1-4c6ba82ea579a0676c15cc9635dd,12520,11,acceptable,,76,,97,1480435927144,
00433543cd1b9c6f45f38b63aa1ae3db,1-3a0ec91bff1d8045243d4ffa52c91837,2512,1023,,1480435946862,
00433543cd1b9c6f45f38b63aa1eb150,1-8d93424b0b9957b87a27cd077d7802fe,4190,,,601480435959910,230
00433543cd1b9c6f45f38b63aa1f8c43,1-c15f12d63c9e36bd481a71e773bdc4ec,12556,11,acceptable,,71,,88,1480435963217,
00433543cd1b9c6f45f38b63aa20c9f8,1-8d197b3cf6238352ece8e600118353f7,12561,11,acceptable,,74,,95,1480435968226,
00433543cd1b9c6f45f38b63aa229da6,1-8aba52859f5b3383ea0cb6712bcc94c1,4195,,,661480435974920,244
00433543cd1b9c6f45f38b63aa236280,1-791b4f657f8e66f4321c2a405f9bd081,4196,,,711480435977922,253
00433543cd1b9c6f45f38b63aa29cca1,1-74a604a89eab53e86737b4a18d87715d,12598,11,acceptable,,170,,80,1480436005298,
00433543cd1b9c6f45f38b63aa2c725a,1-dab7cc1ae0b2fcccda51ddd908688303,12606,11,acceptable,,186,,87,1480436013321,
00433543cd1b9c6f45f38b63aa2ea119,1-00e8ab9617cc3f22649f1a13351eb26b,2527,1097,,1480436021887,
00433543cd1b9c6f45f38b63aa31aabe,1-baef457b493f89d36651506bc1177b22,12624,11,acceptable,,75,,96,1480436031373,
00433543cd1b9c6f45f38b63aa334299,1-9035a6c7ef75d01212145c182f16,2530,1003,,1480436036894,
00433543cd1b9c6f45f38b63aa3399a0,1-723f439a098dc51cb16f3872c5ab9c39,12631,11,acceptable,,75,,85,1480436038391,
00433543cd1b9c6f45f38b63aa342c75,1-cf1867ac82259f787f7ca229af4135c3,12635,11,acceptable,,80,,93,1480436042404,

> java.lang.ArrayIndexOutOfBoundsException: 10
> 
>
> Key: SPARK-23403
> URL: https://issues.apache.org/jira/browse/SPARK-23403
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Shell
>Affects Versions: 2.2.0
>Reporter: Naresh Kumar
>Priority: Major
>
> java.lang.ArrayIndexOutOfBoundsException: 10, while retriving records from 
> Dataframe in spark-shell



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Liang-Chi Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362030#comment-16362030
 ] 

Liang-Chi Hsieh commented on SPARK-23403:
-

Have you checked the content of the csv file? Is there any line with missing 
field?

> java.lang.ArrayIndexOutOfBoundsException: 10
> 
>
> Key: SPARK-23403
> URL: https://issues.apache.org/jira/browse/SPARK-23403
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Shell
>Affects Versions: 2.2.0
>Reporter: Naresh Kumar
>Priority: Major
>
> java.lang.ArrayIndexOutOfBoundsException: 10, while retriving records from 
> Dataframe in spark-shell



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org