Re: Re: how to construct a ClassTag object as a method parameter in Java

[balvisio]

Hi, 

I think same issue is happening with the constructor of the
PartitionPruningRDD class. It hasn't been fixed in version 1.0.1 Should this
be reported to JIRA?



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/how-to-construct-a-ClassTag-object-as-a-method-parameter-in-Java-tp6768p9920.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Re: how to construct a ClassTag object as a method parameter in Java

[Michael Armbrust]

Ah, this is a bug that was fixed in 1.0.

I think you should be able to workaround it by using a fake class tag:
scala.reflect.ClassTag$.MODULE$.AnyRef()


On Mon, Jun 2, 2014 at 8:22 PM, bluejoe2008 bluejoe2...@gmail.com wrote:

  spark 0.9.1
 textInput is a JavaRDD object
 i am programming in Java

 2014-06-03
 --
 bluejoe2008

  *From:* Michael Armbrust mich...@databricks.com
 *Date:* 2014-06-03 10:09
 *To:* user user@spark.apache.org
 *Subject:* Re: how to construct a ClassTag object as a method parameter
 in Java
  What version of Spark are you using?  Also are you sure the type of
 textInput is a JavaRDD and not an RDD?

 It looks like the 1.0 Java API
 http://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaRDDLike.html#mapPartitionsWithIndex(org.apache.spark.api.java.function.Function2,%20boolean)does
 not require a class tag.


 On Mon, Jun 2, 2014 at 5:59 PM, bluejoe2008 bluejoe2...@gmail.com wrote:

  hi,all
i am programming with Spark in Java, and now i have a question:
 when i made a method call on a JavaRDD such as:

 textInput.mapPartitionsWithIndex(
 new Function2Object, IteratorString, IteratorInteger()
 {...},
 false,
 PARAM3
 );

 what value should i pass as the PARAM3 parameter?
 it is required as a ClassTag value, then how can i define such a value in
 Java? i really have no idea...

 best regards,
 bluejoe2008

Re: how to construct a ClassTag object as a method parameter in Java

[Gino Bustelo]

A better way seems to be to use ClassTag$.apply(Class). 

I'm going by memory since I'm on my phone, but I just did that today. 

Gino B.

 On Jun 3, 2014, at 11:04 AM, Michael Armbrust mich...@databricks.com wrote:
 
 Ah, this is a bug that was fixed in 1.0.
 
 I think you should be able to workaround it by using a fake class tag: 
 scala.reflect.ClassTag$.MODULE$.AnyRef()
 
 
 On Mon, Jun 2, 2014 at 8:22 PM, bluejoe2008 bluejoe2...@gmail.com wrote:
 spark 0.9.1
 textInput is a JavaRDD object
 i am programming in Java
  
 2014-06-03
 bluejoe2008
  
 From: Michael Armbrust
 Date: 2014-06-03 10:09
 To: user
 Subject: Re: how to construct a ClassTag object as a method parameter in Java
 What version of Spark are you using?  Also are you sure the type of 
 textInput is a JavaRDD and not an RDD?
 
 It looks like the 1.0 Java API does not require a class tag.
 
 
 On Mon, Jun 2, 2014 at 5:59 PM, bluejoe2008 bluejoe2...@gmail.com wrote:
 hi,all
 i am programming with Spark in Java, and now i have a question:
 when i made a method call on a JavaRDD such as:
  
 textInput.mapPartitionsWithIndex(
 new Function2Object, IteratorString, IteratorInteger()
 {...},
 false,
 PARAM3
 );
  
 what value should i pass as the PARAM3 parameter?
 it is required as a ClassTag value, then how can i define such a value in 
 Java? i really have no idea...
  
 best regards,
 bluejoe2008
 

Re: Re: how to construct a ClassTag object as a method parameter in Java

[bluejoe2008]

spark 0.9.1
textInput is a JavaRDD object
i am programming in Java

2014-06-03 


bluejoe2008

From: Michael Armbrust
Date: 2014-06-03 10:09
To: user
Subject: Re: how to construct a ClassTag object as a method parameter in Java
What version of Spark are you using?  Also are you sure the type of textInput 
is a JavaRDD and not an RDD?


It looks like the 1.0 Java API does not require a class tag.



On Mon, Jun 2, 2014 at 5:59 PM, bluejoe2008 bluejoe2...@gmail.com wrote:

hi,all
i am programming with Spark in Java, and now i have a question:
when i made a method call on a JavaRDD such as:

textInput.mapPartitionsWithIndex(
new Function2Object, IteratorString, IteratorInteger()
{...},
false,
PARAM3
);

what value should i pass as the PARAM3 parameter?
it is required as a ClassTag value, then how can i define such a value in Java? 
i really have no idea...

best regards,
bluejoe2008