Yep, that's what I'm doing. (def OBJECT-CLASS-TAG (.apply ClassTag$/MODULE$ java.lang.Object))
ps - I'm planning to open source this Clojure DSL soon as well On Sun, Jun 1, 2014 at 5:10 PM, Matei Zaharia <matei.zaha...@gmail.com> wrote: > Ah, got it. In general it will always be safe to pass the ClassTag for > java.lang.Object here — this is what our Java API does to say that type > info is not known. So you can always pass that. Look at the Java code for > how to get this ClassTag. > > Matei > > On Jun 1, 2014, at 4:33 PM, Soren Macbeth <so...@yieldbot.com> wrote: > > > I'm writing a Clojure DSL for Spark. I use kryo to serialize my clojure > > functions and for efficiency I hook into Spark's kryo serializer. In > order > > to do that I get a SerializerInstance from SparkEnv and call the > serialize > > and deserialize methods. I was able to workaround it by making ClassTag > > object in clojure, but it's less than ideal. > > > > > > On Sun, Jun 1, 2014 at 4:25 PM, Matei Zaharia <matei.zaha...@gmail.com> > > wrote: > > > >> BTW passing a ClassTag tells the Serializer what the type of object > being > >> serialized is when you compile your program, which will allow for more > >> efficient serializers (especially on streams). > >> > >> Matei > >> > >> On Jun 1, 2014, at 4:24 PM, Matei Zaharia <matei.zaha...@gmail.com> > wrote: > >> > >>> Why do you need to call Serializer from your own program? It’s an > >> internal developer API so ideally it would only be called to extend > Spark. > >> Are you looking to implement a custom Serializer? > >>> > >>> Matei > >>> > >>> On Jun 1, 2014, at 3:40 PM, Soren Macbeth <so...@yieldbot.com> wrote: > >>> > >>>> > >> > https://github.com/apache/spark/blob/v1.0.0/core/src/main/scala/org/apache/spark/serializer/Serializer.scala#L64-L66 > >>>> > >>>> These changes to the SerializerInstance make it really gross to call > >>>> serialize and deserialize from non-scala languages. I'm not sure what > >> the > >>>> purpose of a ClassTag is, but if we could get some other arities that > >> don't > >>>> require classtags that would help a ton. > >>> > >> > >> > >
