[
https://issues.apache.org/jira/browse/SPARK-21868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rahul sharma updated SPARK-21868:
----------------------------------
Description:
I have a sample spark job which I am successfully able to run on java 8 but
when I run same program on java 9 early access, it fails with
NumberFormatException.
SparkConf conf = new SparkConf();
conf.setMaster("local[*]").setAppName("dataframe join example");
SparkSession session =
SparkSession.builder().config(conf).getOrCreate();
Dataset<Row> ds =
session.read().text(Thread.currentThread().getContextClassLoader().
getResource("employee").getPath());
System.out.println(ds.count());
Maven spark dependencies:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>2.1.0</version>
</dependency>
Error:
Exception in thread "main" java.lang.NumberFormatException: For input string:
"ea" at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Integer.parseInt(Integer.java:695) at
java.base/java.lang.Integer.parseInt(Integer.java:813) at
scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229) at
scala.collection.immutable.StringOps.toInt(StringOps.scala:31) at
org.apache.spark.SparkContext.warnDeprecatedVersions(SparkContext.scala:353) at
org.apache.spark.SparkContext.(SparkContext.scala:186) at
org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:120) at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
at
Java Details:
java -version
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+156)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+156, mixed mode)
are there different set of steps to run spark job on java 9?
https://stackoverflow.com/questions/45945128/spark-job-fails-on-java-9-numberformatexception-for-input-string-ea/45948077#45948077
was:
I have a sample spark job which I am successfully able to run on java 8 but
when I run same program on java 9 early access, it fails with
NumberFormatException.
SparkConf conf = new SparkConf();
conf.setMaster("local[*]").setAppName("dataframe join example");
SparkSession session =
SparkSession.builder().config(conf).getOrCreate();
Dataset<Row> ds =
session.read().text(Thread.currentThread().getContextClassLoader().
getResource("employee").getPath());
System.out.println(ds.count());
Maven spark dependencies:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>2.1.0</version>
</dependency>
Error:
Exception in thread "main" java.lang.NumberFormatException: For input string:
"ea" at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.base/java.lang.Integer.parseInt(Integer.java:695) at
java.base/java.lang.Integer.parseInt(Integer.java:813) at
scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229) at
scala.collection.immutable.StringOps.toInt(StringOps.scala:31) at
org.apache.spark.SparkContext.warnDeprecatedVersions(SparkContext.scala:353) at
org.apache.spark.SparkContext.(SparkContext.scala:186) at
org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:120) at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
at
Java Details:
java -version
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+156)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+156, mixed mode)
> Spark job fails on java 9 NumberFormatException for input string ea
> -------------------------------------------------------------------
>
> Key: SPARK-21868
> URL: https://issues.apache.org/jira/browse/SPARK-21868
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: rahul sharma
> Priority: Minor
>
> I have a sample spark job which I am successfully able to run on java 8 but
> when I run same program on java 9 early access, it fails with
> NumberFormatException.
> SparkConf conf = new SparkConf();
> conf.setMaster("local[*]").setAppName("dataframe join example");
> SparkSession session =
> SparkSession.builder().config(conf).getOrCreate();
> Dataset<Row> ds =
> session.read().text(Thread.currentThread().getContextClassLoader().
> getResource("employee").getPath());
> System.out.println(ds.count());
> Maven spark dependencies:
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.10</artifactId>
> <version>2.1.0</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-sql_2.10</artifactId>
> <version>2.1.0</version>
> </dependency>
> Error:
> Exception in thread "main" java.lang.NumberFormatException: For input string:
> "ea" at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Integer.parseInt(Integer.java:695) at
> java.base/java.lang.Integer.parseInt(Integer.java:813) at
> scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229) at
> scala.collection.immutable.StringOps.toInt(StringOps.scala:31) at
> org.apache.spark.SparkContext.warnDeprecatedVersions(SparkContext.scala:353)
> at org.apache.spark.SparkContext.(SparkContext.scala:186) at
> org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
> at scala.Option.getOrElse(Option.scala:120) at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
> at
> Java Details:
> java -version
> java version "9-ea"
> Java(TM) SE Runtime Environment (build 9-ea+156)
> Java HotSpot(TM) 64-Bit Server VM (build 9-ea+156, mixed mode)
> are there different set of steps to run spark job on java 9?
> https://stackoverflow.com/questions/45945128/spark-job-fails-on-java-9-numberformatexception-for-input-string-ea/45948077#45948077
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
