Re: Java Interop on steroids?

[Chris Nuernberger]

OK that makes more sense, BeamFn is not an interface. I imagine they argued
about that decision at some point.

In your example, MyFn looks to be a general class as it can work with any
clojure var. Do you have a package with many of these type of stub classes
defined? I mean, you could name it BeamFnIntegerString thus encoding the
specializations of the baseclass into the type. One of my questions was
about the range of types that end up specializing the BeamFn<>.

Great link about the proxy attack, that is very interesting.  I want to
dive into that one a bit more.

On Tue, Jul 2, 2019 at 8:14 PM atdixon  wrote:

> I'm glad someone else is thinking on this too!
>
> #2 - For my case at the moment (Apache Beam), I believe we will always
> know the types in advance so using a Java class is workable but of course a
> (proxy++) would be ideal. Beam asks for us to extend abstract generic class
> so we must use (proxy). It also asks for our instances to be Serializable
> (again, proxy explicitly refuses to help here
>  but I believe this, too,
> should be surmountable, without the security implications of that link). In
> any case, what we do looks like this:
>
> // pseudo-ish code
> public class MyFn extends BeamFn implements Serializable
> {
>public MyFn(Var fn) { this.fn = fn; }
>@Override public String invoke(Integer input) { return (String) fn.
> invoke(input); }
> }
>
> On the Clojure side:
>
> ;; pseudo-ish code
>
> (defn my-fn [^Integer val] ...return a string...)
>
> ...
>(register-beam-step (MyFn. #'my-fn))
> ...
>
> Note how we pass the Clojure function as a Var; this is b/c Beam wants to
> send the function over the wire. IFn is not serializable; Var is however
> and can be resolved back to Clojure function on the other end during
> deserialization.
>
> Now of course this whole dance could be eliminated with a specialized
> library that included a proxy++ function that included the ability for
> clients to specify generic type parameter values and serialization support
> (the mechanics of which would need ironing out but I think should be
> possible.)
>
> And to your point about types not known at runtime... this proxy would
> support that use case on-the-fly, which opens a bunch of possible
> interesting options, as well.
>
>
> On Tuesday, July 2, 2019 at 8:24:41 PM UTC-5, Chris Nuernberger wrote:
>>
>> eglue,
>>
>> 1.  I think this is a great idea if it is really necessary.  I would be
>> in favor of a reify++ alone to simplify things.  I find reify amazing at
>> code compression and heavily use it via type specific macros to implement
>> interfaces that for instance support a particular primitive type.
>> 2.  Is a possible workaround to define java interfaces that implement the
>> type specific generic interfaces and then reify those explicitly or is the
>> set of possible interface specialization types unknown a-priori?
>> 3.  The case where something is unbounded or unknown a-priori I would
>> think would often end up with a java class as on of the specializations.
>> In this case, regardless of the cause, one answer might be an upgraded
>> reify pathway.
>> 4.  Are these perhaps cases where you can create just a little bit of
>> java as a generator somehow to generate the interface you need to reify?
>>
>>
>> I would personally find reify a much nicer pathway than calling clojure
>> vars from java.
>>
>> I also think there could be low hanging fruit (or just good unknown
>> libraries) in the pathway for calling clojure vars from java.
>>
>>
>> Interesting problem (at least to me), thanks!
>>
>> On Thu, Jun 20, 2019 at 10:03 PM eglue  wrote:
>>
>>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
>>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>>
>>> However -- there are many popular Java frameworks that love to reflect
>>> on their annotations and their generic type signatures.
>>>
>>> To name a heavyweight: Spring. But also, of late: big data frameworks,
>>> many written in Java, love reflecting on generic type signatures. My org is
>>> looking at Beam and Flink, for example.
>>>
>>> These frameworks use types not for the static checking really but as
>>> parameters governing their own dynamic behavior. For example, Spring will
>>> use types at runtime to simply match objects to where they should be
>>> dynamically injected. Beam will look at your type signatures and do runtime
>>> validations to ensure it can process things appropriately. Of course this
>>> is unfortunate, using types this way, when it is all really just data.
>>> Clojure does -- or would do -- it better, simpler, directer, and all of
>>> that.
>>>
>>> Yet we would like to leverage these frameworks. Or rather, we must for
>>> various pragmatic and business reasons.
>>>
>>> And any time we need to "communicate" to these frameworks "through"
>>> their desired fashion of generic types and annotations, we can, of course,
>>> create the appropriate .java files to represent 

Re: Java Interop on steroids?

[atdixon]

I'm glad someone else is thinking on this too!

#2 - For my case at the moment (Apache Beam), I believe we will always know 
the types in advance so using a Java class is workable but of course a 
(proxy++) would be ideal. Beam asks for us to extend abstract generic class 
so we must use (proxy). It also asks for our instances to be Serializable 
(again, proxy explicitly refuses to help here 
 but I believe this, too, 
should be surmountable, without the security implications of that link). In 
any case, what we do looks like this:

// pseudo-ish code
public class MyFn extends BeamFn implements Serializable {
   public MyFn(Var fn) { this.fn = fn; }
   @Override public String invoke(Integer input) { return (String) fn.invoke
(input); }
}

On the Clojure side:

;; pseudo-ish code

(defn my-fn [^Integer val] ...return a string...)

...
   (register-beam-step (MyFn. #'my-fn))
...

Note how we pass the Clojure function as a Var; this is b/c Beam wants to 
send the function over the wire. IFn is not serializable; Var is however 
and can be resolved back to Clojure function on the other end during 
deserialization.

Now of course this whole dance could be eliminated with a specialized 
library that included a proxy++ function that included the ability for 
clients to specify generic type parameter values and serialization support 
(the mechanics of which would need ironing out but I think should be 
possible.)

And to your point about types not known at runtime... this proxy would 
support that use case on-the-fly, which opens a bunch of possible 
interesting options, as well.


On Tuesday, July 2, 2019 at 8:24:41 PM UTC-5, Chris Nuernberger wrote:
>
> eglue,
>
> 1.  I think this is a great idea if it is really necessary.  I would be in 
> favor of a reify++ alone to simplify things.  I find reify amazing at code 
> compression and heavily use it via type specific macros to implement 
> interfaces that for instance support a particular primitive type.
> 2.  Is a possible workaround to define java interfaces that implement the 
> type specific generic interfaces and then reify those explicitly or is the 
> set of possible interface specialization types unknown a-priori? 
> 3.  The case where something is unbounded or unknown a-priori I would 
> think would often end up with a java class as on of the specializations.  
> In this case, regardless of the cause, one answer might be an upgraded 
> reify pathway.  
> 4.  Are these perhaps cases where you can create just a little bit of java 
> as a generator somehow to generate the interface you need to reify?
>
>
> I would personally find reify a much nicer pathway than calling clojure 
> vars from java.  
>
> I also think there could be low hanging fruit (or just good unknown 
> libraries) in the pathway for calling clojure vars from java.
>
>
> Interesting problem (at least to me), thanks!
>
> On Thu, Jun 20, 2019 at 10:03 PM eglue > 
> wrote:
>
>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>
>> However -- there are many popular Java frameworks that love to reflect on 
>> their annotations and their generic type signatures.
>>
>> To name a heavyweight: Spring. But also, of late: big data frameworks, 
>> many written in Java, love reflecting on generic type signatures. My org is 
>> looking at Beam and Flink, for example.
>>
>> These frameworks use types not for the static checking really but as 
>> parameters governing their own dynamic behavior. For example, Spring will 
>> use types at runtime to simply match objects to where they should be 
>> dynamically injected. Beam will look at your type signatures and do runtime 
>> validations to ensure it can process things appropriately. Of course this 
>> is unfortunate, using types this way, when it is all really just data. 
>> Clojure does -- or would do -- it better, simpler, directer, and all of 
>> that.
>>
>> Yet we would like to leverage these frameworks. Or rather, we must for 
>> various pragmatic and business reasons.
>>
>> And any time we need to "communicate" to these frameworks "through" their 
>> desired fashion of generic types and annotations, we can, of course, create 
>> the appropriate .java files to represent what is needed (and then do the 
>> invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this 
>> works.
>>
>> However this is quite tedious because in these frameworks I mentioned you 
>> end up having to create these Java files quite a bit. For example, when 
>> expressing a streaming data pipeline to Beam, you may specify multiple 
>> transforms, each a function with its own type signature.
>>
>> A little searching and it seems Clojure has shied away from generating 
>> generic type information in places where it could offer this capability. 
>>
>> For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, 
>> and other dynamic bytecode generation features of Clojure.

Re: Java Interop on steroids?

[Chris Nuernberger]

5.  If you need a concrete class definition that then implements a set of
type specific interfaces this would seem to fall into a category of
gen-class assuming you could specify the interfaces with type
specifications.  I can't immediately place a way to do this with anything
mentioned above.  It may be then important to be able to override Object
methods but maybe you already can with gen-class.

On Tue, Jul 2, 2019 at 7:24 PM Chris Nuernberger 
wrote:

> eglue,
>
> 1.  I think this is a great idea if it is really necessary.  I would be in
> favor of a reify++ alone to simplify things.  I find reify amazing at code
> compression and heavily use it via type specific macros to implement
> interfaces that for instance support a particular primitive type.
> 2.  Is a possible workaround to define java interfaces that implement the
> type specific generic interfaces and then reify those explicitly or is the
> set of possible interface specialization types unknown a-priori?
> 3.  The case where something is unbounded or unknown a-priori I would
> think would often end up with a java class as on of the specializations.
> In this case, regardless of the cause, one answer might be an upgraded
> reify pathway.
> 4.  Are these perhaps cases where you can create just a little bit of java
> as a generator somehow to generate the interface you need to reify?
>
>
> I would personally find reify a much nicer pathway than calling clojure
> vars from java.
>
> I also think there could be low hanging fruit (or just good unknown
> libraries) in the pathway for calling clojure vars from java.
>
>
> Interesting problem (at least to me), thanks!
>
> On Thu, Jun 20, 2019 at 10:03 PM eglue  wrote:
>
>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>
>> However -- there are many popular Java frameworks that love to reflect on
>> their annotations and their generic type signatures.
>>
>> To name a heavyweight: Spring. But also, of late: big data frameworks,
>> many written in Java, love reflecting on generic type signatures. My org is
>> looking at Beam and Flink, for example.
>>
>> These frameworks use types not for the static checking really but as
>> parameters governing their own dynamic behavior. For example, Spring will
>> use types at runtime to simply match objects to where they should be
>> dynamically injected. Beam will look at your type signatures and do runtime
>> validations to ensure it can process things appropriately. Of course this
>> is unfortunate, using types this way, when it is all really just data.
>> Clojure does -- or would do -- it better, simpler, directer, and all of
>> that.
>>
>> Yet we would like to leverage these frameworks. Or rather, we must for
>> various pragmatic and business reasons.
>>
>> And any time we need to "communicate" to these frameworks "through" their
>> desired fashion of generic types and annotations, we can, of course, create
>> the appropriate .java files to represent what is needed (and then do the
>> invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this
>> works.
>>
>> However this is quite tedious because in these frameworks I mentioned you
>> end up having to create these Java files quite a bit. For example, when
>> expressing a streaming data pipeline to Beam, you may specify multiple
>> transforms, each a function with its own type signature.
>>
>> A little searching and it seems Clojure has shied away from generating
>> generic type information in places where it could offer this capability.
>>
>> For example, in `proxy` ... or I suppose also in `gen-class`, `reify`,
>> and other dynamic bytecode generation features of Clojure.
>>
>> However it seems to me that `proxy` (and these others) could allow one to
>> pass in a representation of desired type arguments, annotations, etc. and
>> then we could remain in Clojure to interop with these popular frameworks.
>>
>> I respect Clojure's efforts to keep its core small and wait for worthy
>> features to prove themselves.
>>
>> So my question is not when is Clojure going to do this, but rather:
>>
>> Are there any precedents in the community for someone building out the
>> kind of richer Java interop that I'm nodding toward here?
>>
>> For example, does anyone know of an attempt out there to build a `proxy`
>> plus-plus, that would allow one to extend a generic class with provided
>> type parameters and have this metadata properly rendered in the bytecode
>> that proxy produces?
>>
>> If not, as a practical and hopefully quick and workable solution, I was
>> thinking it'd be possible to take the bytecode emitted by proxy and re-run
>> it through ASM to create a *new* class with simply the proxy-produced class
>> bytes filled-in with the desired, provided type parameters. I bet this
>> could be sufficient and fast, with the slight overhead of the extra class.
>>
>> To do this, I think I'd need access to these proxy-made bytes... either
>> by having proxy answer 

Re: Java Interop on steroids?

[Chris Nuernberger]

eglue,

1.  I think this is a great idea if it is really necessary.  I would be in
favor of a reify++ alone to simplify things.  I find reify amazing at code
compression and heavily use it via type specific macros to implement
interfaces that for instance support a particular primitive type.
2.  Is a possible workaround to define java interfaces that implement the
type specific generic interfaces and then reify those explicitly or is the
set of possible interface specialization types unknown a-priori?
3.  The case where something is unbounded or unknown a-priori I would think
would often end up with a java class as on of the specializations.  In this
case, regardless of the cause, one answer might be an upgraded reify
pathway.
4.  Are these perhaps cases where you can create just a little bit of java
as a generator somehow to generate the interface you need to reify?


I would personally find reify a much nicer pathway than calling clojure
vars from java.

I also think there could be low hanging fruit (or just good unknown
libraries) in the pathway for calling clojure vars from java.


Interesting problem (at least to me), thanks!

On Thu, Jun 20, 2019 at 10:03 PM eglue  wrote:

> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
> ̶H̶i̶c̶k̶e̶y̶  guy.
>
> However -- there are many popular Java frameworks that love to reflect on
> their annotations and their generic type signatures.
>
> To name a heavyweight: Spring. But also, of late: big data frameworks,
> many written in Java, love reflecting on generic type signatures. My org is
> looking at Beam and Flink, for example.
>
> These frameworks use types not for the static checking really but as
> parameters governing their own dynamic behavior. For example, Spring will
> use types at runtime to simply match objects to where they should be
> dynamically injected. Beam will look at your type signatures and do runtime
> validations to ensure it can process things appropriately. Of course this
> is unfortunate, using types this way, when it is all really just data.
> Clojure does -- or would do -- it better, simpler, directer, and all of
> that.
>
> Yet we would like to leverage these frameworks. Or rather, we must for
> various pragmatic and business reasons.
>
> And any time we need to "communicate" to these frameworks "through" their
> desired fashion of generic types and annotations, we can, of course, create
> the appropriate .java files to represent what is needed (and then do the
> invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this
> works.
>
> However this is quite tedious because in these frameworks I mentioned you
> end up having to create these Java files quite a bit. For example, when
> expressing a streaming data pipeline to Beam, you may specify multiple
> transforms, each a function with its own type signature.
>
> A little searching and it seems Clojure has shied away from generating
> generic type information in places where it could offer this capability.
>
> For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and
> other dynamic bytecode generation features of Clojure.
>
> However it seems to me that `proxy` (and these others) could allow one to
> pass in a representation of desired type arguments, annotations, etc. and
> then we could remain in Clojure to interop with these popular frameworks.
>
> I respect Clojure's efforts to keep its core small and wait for worthy
> features to prove themselves.
>
> So my question is not when is Clojure going to do this, but rather:
>
> Are there any precedents in the community for someone building out the
> kind of richer Java interop that I'm nodding toward here?
>
> For example, does anyone know of an attempt out there to build a `proxy`
> plus-plus, that would allow one to extend a generic class with provided
> type parameters and have this metadata properly rendered in the bytecode
> that proxy produces?
>
> If not, as a practical and hopefully quick and workable solution, I was
> thinking it'd be possible to take the bytecode emitted by proxy and re-run
> it through ASM to create a *new* class with simply the proxy-produced class
> bytes filled-in with the desired, provided type parameters. I bet this
> could be sufficient and fast, with the slight overhead of the extra class.
>
> To do this, I think I'd need access to these proxy-made bytes... either by
> having proxy answer them somehow, or offering a hook to contribute to the
> defined bytecode before it is committed to the classloader, or by having
> DynamicClassLoader have these bytes on hand for inquiring parties, or
> something else along these lines. This would likely be something that
> Clojure core would have to expose .. correct me if I'm wrong.
>
> Would love to hear any other immediate thoughts on this.
>
> I think once you realize that this generic type information is not even
> being used for "static typing" by these frameworks but rather as an (albeit
> poor) means to 

Re: Java Interop on steroids?

[Didier]

Oh, I didn't know about that, pretty funky stuff.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/37f95c64-257b-43c3-9c7c-0bf0f8087e54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java Interop on steroids?

[atdixon]

> Do the framework you're talking about do static analysis of the types? 
Because generic types are erased at runtime, so there wouldn't ever be a 
way for proxy to set them in.

They aren't entirely erased. They're erased from the code, but Java 
compilers are obligated to emit generic type signatures (as metadata) in 
the compiled class file.

The frameworks I mentioned reflect on these generic type parameters *at 
runtime* to drive their dynamic behavior. 

More information here:

https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getGenericSuperclass--
https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.9.1

Searching back through this group, I found that in 2012 someone proposed a 
patch to Clojure supporting this in `proxy` (link: 
https://groups.google.com/d/msg/clojure/Xv1pKATfP0c/od_uwpHlNhMJ) but it 
looks like it never made it in.


On Saturday, June 22, 2019 at 11:32:36 PM UTC-5, Didier wrote:
>
> > They did cite a significant performance boost as a side effect.
>
> I think it isn't very clear from the wording. They didn't just rewrite it 
> in Java, they also changed the architecture:
>
> > Storm 2.0.0 introduces a new core featuring a leaner threading model, a 
> blazing fast messaging subsystem and a lightweight back pressure model
>
> So it is hard I think to do an apples to apples comparison. In my opinion, 
> a system like Storm will fundamentally be more limited by its architecture 
> in performance than the language it uses.
>
> That said, even if I think the performance improvements are probably 
> mostly due to architectural changes they also made. It shouldn't come as a 
> surprise that Java would be faster than Clojure in most cases. I don't want 
> to make false pretenses. Clojure and Java are not equal in semantics, and 
> Java's mutable, eager and object grouped methods semantics are almost 
> always going to be more performant. Clojure makes a trade offs of 
> performance and memory for simplicity.
>
> When people say Clojure can match Java in performance, it almost always 
> implies using escape hatches and changing the semantics back to imperative 
> code. The good news though, when you use Clojure's semantics, you benefit 
> in simplicity and the performance impact is marginal, so it is still fast 
> enough for almost all use cases.
>
> Now, back to type annotations. I really don't think Storm 2.0 performance 
> improvements were due to usage of reflection that hadn't been addressed 
> from Clojure. But who knows.
>
> Do the framework you're talking about do static analysis of the types? 
> Because generic types are erased at runtime, so there wouldn't ever be a 
> way for proxy to set them in.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/1cf5ad80-0504-4634-b186-b624206c0145%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java Interop on steroids?

[Didier]

> They did cite a significant performance boost as a side effect.

I think it isn't very clear from the wording. They didn't just rewrite it in 
Java, they also changed the architecture:

> Storm 2.0.0 introduces a new core featuring a leaner threading model, a 
> blazing fast messaging subsystem and a lightweight back pressure model

So it is hard I think to do an apples to apples comparison. In my opinion, a 
system like Storm will fundamentally be more limited by its architecture in 
performance than the language it uses.

That said, even if I think the performance improvements are probably mostly due 
to architectural changes they also made. It shouldn't come as a surprise that 
Java would be faster than Clojure in most cases. I don't want to make false 
pretenses. Clojure and Java are not equal in semantics, and Java's mutable, 
eager and object grouped methods semantics are almost always going to be more 
performant. Clojure makes a trade offs of performance and memory for simplicity.

When people say Clojure can match Java in performance, it almost always implies 
using escape hatches and changing the semantics back to imperative code. The 
good news though, when you use Clojure's semantics, you benefit in simplicity 
and the performance impact is marginal, so it is still fast enough for almost 
all use cases.

Now, back to type annotations. I really don't think Storm 2.0 performance 
improvements were due to usage of reflection that hadn't been addressed from 
Clojure. But who knows.

Do the framework you're talking about do static analysis of the types? Because 
generic types are erased at runtime, so there wouldn't ever be a way for proxy 
to set them in.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/5a4ad6dc-922a-42ff-b058-faaa94219553%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java Interop on steroids?

[Chris Nuernberger]

Nathan,

For the subset of problems for which clojure has insufficient power to 
optimize well, I think a good percentage of those would be best off solved 
by numeric languages like numpy <https://github.com/cnuernber/libpython-clj> 
or a numeric compiler like TVM <https://github.com/techascent/tvm-clj/> as 
this gives you a pathway to much more scalability and performance in the 
end.  In short, they aren't best solved by Java either but best solved via 
translation to, execution in, and then translation back from a numeric 
form.  Aided by 4 years of understanding the problem this sort of 
transformation should definitely be possible.
​
As for being able to contribute without knowing clojure I think this will 
probably overall lead to an ossification of design over time but the larger 
set of involved people is definitely a great thing in and of itself.  
Potentially the audience for Storm is mainly other jvm languages.

The best person I found who learned clojure fast was already very good at 
python and used Koans <http://clojurekoans.com/>. I think it should be 
reasonable to stay within some of the extent of the language described in 
things like the Koans for the majority of the codebase and still provide a 
pathway for new people.

That's a lot of thinking for a weekend :-).  Thanks for the post it was 
interesting!

Chris
​

On Friday, June 21, 2019 at 5:18:06 PM UTC-6, Nathan Fisher wrote:
>
> Storm recently moved away from Clojure in its core.
>
> https://storm.apache.org/2019/05/30/storm200-released.html
>
> I wonder how much of the legacy Clojure core could be optimised or if they 
> reached an upper limit imposed by the runtime/architecture. That being said 
> I suspect for 90% of orgs they'll never hit that boundary.
>
> On Fri, Jun 21, 2019 at 16:40, Chris Nuernberger  > wrote:
>
>> Sean, 
>>
>> That is an interesting blog post.  Sorry if I am not following everything 
>> but why not use the annotation support in gen-class for those types of 
>> things?
>>
>>
>> https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34
>>
>> On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
>>>
>>> You might be interested in how we provide type-based annotations on 
>>> Clojure functions so that tooling (in our case New Relic) sees those 
>>> annotations:
>>>
>>>  
>>>
>>>
>>> https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/
>>>
>>>  
>>>
>>> I agree that this could be a lot easier.
>>>
>>>  
>>>
>>> Sean Corfield -- (904) 302-SEAN
>>> An Architect's View -- http://corfield.org/
>>>
>>> "If you're not annoying somebody, you're not really alive."
>>> -- Margaret Atwood
>>>
>>>  
>>> --
>>> *From:* clo...@googlegroups.com  on behalf of 
>>> eglue 
>>> *Sent:* Thursday, June 20, 2019 9:03:45 PM
>>> *To:* Clojure
>>> *Subject:* Java Interop on steroids? 
>>>  
>>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
>>> ̶H̶i̶c̶k̶e̶y̶  guy. 
>>>
>>> However -- there are many popular Java frameworks that love to reflect 
>>> on their annotations and their generic type signatures.
>>>
>>> To name a heavyweight: Spring. But also, of late: big data frameworks, 
>>> many written in Java, love reflecting on generic type signatures. My org is 
>>> looking at Beam and Flink, for example.
>>>
>>> These frameworks use types not for the static checking really but as 
>>> parameters governing their own dynamic behavior. For example, Spring will 
>>> use types at runtime to simply match objects to where they should be 
>>> dynamically injected. Beam will look at your type signatures and do runtime 
>>> validations to ensure it can process things appropriately. Of course this 
>>> is unfortunate, using types this way, when it is all really just data. 
>>> Clojure does -- or would do -- it better, simpler, directer, and all of 
>>> that.
>>>
>>> Yet we would like to leverage these frameworks. Or rather, we must for 
>>> various pragmatic and business reasons.
>>>
>>> And any time we need to "communicate" to these frameworks "through" 
>>> their desired fashion of generic types and annotations, we can, of course, 
>>> create the appropriate .java files to represent what is needed (and then do 
>>> the invocation back 

Re: Java Interop on steroids?

[Aaron Dixon]

That is likely more to do with doing a full rewrite with 5+ years of
learned experience under their belt than any insurmountable performance
issue by Clojure.

(Storm was mostly Clojure from day one but always had Java code in the mix,
as well; in other words, they've always had Java if they had needed it...
so crediting their perf gains to the PL switch is suspect imo.)

On Sat, Jun 22, 2019 at 2:48 PM Nathan Fisher 
wrote:

> They did cite a significant performance boost as a side effect.
>
> On Fri, Jun 21, 2019 at 21:04, Sean Corfield  wrote:
>
>> I got the impression this was the primary reason for Storm’s rewrite:
>>
>>
>>
>> While Storm's Clojure implementation served it well for many years, it
>> was often cited as a barrier for entry to new contributors. Storm's
>> codebase is now more accessible to developers who don't want to learn
>> Clojure in order to contribute.
>>
>>
>>
>> Sean Corfield -- (904) 302-SEAN
>> An Architect's View -- http://corfield.org/
>>
>> "If you're not annoying somebody, you're not really alive."
>> -- Margaret Atwood
>>
>>
>> --
>> *From:* clojure@googlegroups.com  on behalf of
>> Nathan Fisher 
>> *Sent:* Friday, June 21, 2019 4:17:43 PM
>> *To:* clojure@googlegroups.com
>> *Subject:* Re: Java Interop on steroids?
>>
>> Storm recently moved away from Clojure in its core.
>>
>> https://storm.apache.org/2019/05/30/storm200-released.html
>>
>> I wonder how much of the legacy Clojure core could be optimised or if
>> they reached an upper limit imposed by the runtime/architecture. That being
>> said I suspect for 90% of orgs they'll never hit that boundary.
>>
>> On Fri, Jun 21, 2019 at 16:40, Chris Nuernberger 
>> wrote:
>>
>>> Sean,
>>>
>>> That is an interesting blog post.  Sorry if I am not following
>>> everything but why not use the annotation support in gen-class for those
>>> types of things?
>>>
>>>
>>> https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34
>>>
>>> On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
>>>>
>>>> You might be interested in how we provide type-based annotations on
>>>> Clojure functions so that tooling (in our case New Relic) sees those
>>>> annotations:
>>>>
>>>>
>>>>
>>>>
>>>> https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/
>>>>
>>>>
>>>>
>>>> I agree that this could be a lot easier.
>>>>
>>>>
>>>>
>>>> Sean Corfield -- (904) 302-SEAN
>>>> An Architect's View -- http://corfield.org/
>>>>
>>>> "If you're not annoying somebody, you're not really alive."
>>>> -- Margaret Atwood
>>>>
>>>>
>>>> --
>>>> *From:* clo...@googlegroups.com  on behalf of
>>>> eglue 
>>>> *Sent:* Thursday, June 20, 2019 9:03:45 PM
>>>> *To:* Clojure
>>>> *Subject:* Java Interop on steroids?
>>>>
>>>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
>>>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>>>
>>>> However -- there are many popular Java frameworks that love to reflect
>>>> on their annotations and their generic type signatures.
>>>>
>>>> To name a heavyweight: Spring. But also, of late: big data frameworks,
>>>> many written in Java, love reflecting on generic type signatures. My org is
>>>> looking at Beam and Flink, for example.
>>>>
>>>> These frameworks use types not for the static checking really but as
>>>> parameters governing their own dynamic behavior. For example, Spring will
>>>> use types at runtime to simply match objects to where they should be
>>>> dynamically injected. Beam will look at your type signatures and do runtime
>>>> validations to ensure it can process things appropriately. Of course this
>>>> is unfortunate, using types this way, when it is all really just data.
>>>> Clojure does -- or would do -- it better, simpler, directer, and all of
>>>> that.
>>>>
>>>> Yet we would like to leverage these frameworks. Or rather, we must for
>>>> various pragmatic and business reasons.
>>>>
>>>>

Re: Java Interop on steroids?

[Nathan Fisher]

They did cite a significant performance boost as a side effect.

On Fri, Jun 21, 2019 at 21:04, Sean Corfield  wrote:

> I got the impression this was the primary reason for Storm’s rewrite:
>
>
>
> While Storm's Clojure implementation served it well for many years, it was
> often cited as a barrier for entry to new contributors. Storm's codebase is
> now more accessible to developers who don't want to learn Clojure in order
> to contribute.
>
>
>
> Sean Corfield -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>
> --
> *From:* clojure@googlegroups.com  on behalf of
> Nathan Fisher 
> *Sent:* Friday, June 21, 2019 4:17:43 PM
> *To:* clojure@googlegroups.com
> *Subject:* Re: Java Interop on steroids?
>
> Storm recently moved away from Clojure in its core.
>
> https://storm.apache.org/2019/05/30/storm200-released.html
>
> I wonder how much of the legacy Clojure core could be optimised or if they
> reached an upper limit imposed by the runtime/architecture. That being said
> I suspect for 90% of orgs they'll never hit that boundary.
>
> On Fri, Jun 21, 2019 at 16:40, Chris Nuernberger 
> wrote:
>
>> Sean,
>>
>> That is an interesting blog post.  Sorry if I am not following everything
>> but why not use the annotation support in gen-class for those types of
>> things?
>>
>>
>> https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34
>>
>> On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
>>>
>>> You might be interested in how we provide type-based annotations on
>>> Clojure functions so that tooling (in our case New Relic) sees those
>>> annotations:
>>>
>>>
>>>
>>>
>>> https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/
>>>
>>>
>>>
>>> I agree that this could be a lot easier.
>>>
>>>
>>>
>>> Sean Corfield -- (904) 302-SEAN
>>> An Architect's View -- http://corfield.org/
>>>
>>> "If you're not annoying somebody, you're not really alive."
>>> -- Margaret Atwood
>>>
>>>
>>> --
>>> *From:* clo...@googlegroups.com  on behalf of
>>> eglue 
>>> *Sent:* Thursday, June 20, 2019 9:03:45 PM
>>> *To:* Clojure
>>> *Subject:* Java Interop on steroids?
>>>
>>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
>>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>>
>>> However -- there are many popular Java frameworks that love to reflect
>>> on their annotations and their generic type signatures.
>>>
>>> To name a heavyweight: Spring. But also, of late: big data frameworks,
>>> many written in Java, love reflecting on generic type signatures. My org is
>>> looking at Beam and Flink, for example.
>>>
>>> These frameworks use types not for the static checking really but as
>>> parameters governing their own dynamic behavior. For example, Spring will
>>> use types at runtime to simply match objects to where they should be
>>> dynamically injected. Beam will look at your type signatures and do runtime
>>> validations to ensure it can process things appropriately. Of course this
>>> is unfortunate, using types this way, when it is all really just data.
>>> Clojure does -- or would do -- it better, simpler, directer, and all of
>>> that.
>>>
>>> Yet we would like to leverage these frameworks. Or rather, we must for
>>> various pragmatic and business reasons.
>>>
>>> And any time we need to "communicate" to these frameworks "through"
>>> their desired fashion of generic types and annotations, we can, of course,
>>> create the appropriate .java files to represent what is needed (and then do
>>> the invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes,
>>> this works.
>>>
>>> However this is quite tedious because in these frameworks I mentioned
>>> you end up having to create these Java files quite a bit. For example, when
>>> expressing a streaming data pipeline to Beam, you may specify multiple
>>> transforms, each a function with its own type signature.
>>>
>>> A little searching and it seems Clojure has shied away from generating
>>> generic type information in pla

Re: Java Interop on steroids?

[atdixon]

Here is my problem, distilled. This code should tell the full story:

static class Apple {}

Apple a = new Apple() {};

Type[] x = ((ParameterizedType)a.getClass().getGenericSuperclass())
.getActualTypeArguments();

// x is a Type array containing String, Integer

HOWEVER, via Clojure `proxy`, I don't have a way to tell it which type 
arguments to use--

(proxy [Apple [String Integer]] []) ;; something like this is not supported

Certain frameworks, however, consult these type arguments to govern their 
own dynamic behavior.

Having Clojure's proxy support specifying type arguments would allow for 
this not uncommon Java interop need.

On Saturday, June 22, 2019 at 5:38:36 AM UTC-5, Matching Socks wrote:
>
> By "generic type information", you mean the X in List ?
>
>
> On Friday, June 21, 2019 at 12:03:46 AM UTC-4, atdixon wrote:
>>
>> However -- there are many popular Java frameworks that love to reflect on 
>> their annotations and their generic type signatures.
>>
>> To name a heavyweight: Spring. But also, of late: big data frameworks, 
>> many written in Java, love reflecting on generic type signatures. My org is 
>> looking at Beam and Flink, for example.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/8c5ae6a3-2626-47ed-9439-adbab9dec469%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java Interop on steroids?

[atdixon]

Here is my problem, distilled. This code should tell the full story:

static class Apple {}

Apple a = new Apple() {};

Type[] x = ((ParameterizedType)a.getClass().getGenericSuperclass())
.getActualTypeArguments();

// x is a Type array containing String, String

HOWEVER, via Clojure `proxy`, I don't have a way to tell it which type 
arguments to use--

(proxy [Apple [String Integer]] []) ;; something like this is not supported

Certain frameworks, however, consult these type arguments to govern their 
own dynamic behavior.

Having Clojure's proxy support specifying type arguments would allow for 
this not uncommon Java interop need.



On Saturday, June 22, 2019 at 5:38:36 AM UTC-5, Matching Socks wrote:
>
> By "generic type information", you mean the X in List ?
>
>
> On Friday, June 21, 2019 at 12:03:46 AM UTC-4, atdixon wrote:
>>
>> However -- there are many popular Java frameworks that love to reflect on 
>> their annotations and their generic type signatures.
>>
>> To name a heavyweight: Spring. But also, of late: big data frameworks, 
>> many written in Java, love reflecting on generic type signatures. My org is 
>> looking at Beam and Flink, for example.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/b4fb12fe-6dc6-47a5-a271-f453941fcb2d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java Interop on steroids?

[Matching Socks]

By "generic type information", you mean the X in List ?


On Friday, June 21, 2019 at 12:03:46 AM UTC-4, atdixon wrote:
>
> However -- there are many popular Java frameworks that love to reflect on 
> their annotations and their generic type signatures.
>
> To name a heavyweight: Spring. But also, of late: big data frameworks, 
> many written in Java, love reflecting on generic type signatures. My org is 
> looking at Beam and Flink, for example.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/8fd67aff-8229-4588-9b2b-c07df8f5f055%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: Java Interop on steroids?

[Sean Corfield]

I got the impression this was the primary reason for Storm’s rewrite:

While Storm's Clojure implementation served it well for many years, it was 
often cited as a barrier for entry to new contributors. Storm's codebase is now 
more accessible to developers who don't want to learn Clojure in order to 
contribute.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of Nathan 
Fisher 
Sent: Friday, June 21, 2019 4:17:43 PM
To: clojure@googlegroups.com
Subject: Re: Java Interop on steroids?

Storm recently moved away from Clojure in its core.

https://storm.apache.org/2019/05/30/storm200-released.html

I wonder how much of the legacy Clojure core could be optimised or if they 
reached an upper limit imposed by the runtime/architecture. That being said I 
suspect for 90% of orgs they'll never hit that boundary.

On Fri, Jun 21, 2019 at 16:40, Chris Nuernberger 
mailto:ch...@techascent.com>> wrote:
Sean,

That is an interesting blog post.  Sorry if I am not following everything but 
why not use the annotation support in gen-class for those types of things?

https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34

On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
You might be interested in how we provide type-based annotations on Clojure 
functions so that tooling (in our case New Relic) sees those annotations:

https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/

I agree that this could be a lot easier.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clo...@googlegroups.com  on behalf of eglue 

Sent: Thursday, June 20, 2019 9:03:45 PM
To: Clojure
Subject: Java Interop on steroids?

Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
̶H̶i̶c̶k̶e̶y̶  guy.

However -- there are many popular Java frameworks that love to reflect on their 
annotations and their generic type signatures.

To name a heavyweight: Spring. But also, of late: big data frameworks, many 
written in Java, love reflecting on generic type signatures. My org is looking 
at Beam and Flink, for example.

These frameworks use types not for the static checking really but as parameters 
governing their own dynamic behavior. For example, Spring will use types at 
runtime to simply match objects to where they should be dynamically injected. 
Beam will look at your type signatures and do runtime validations to ensure it 
can process things appropriately. Of course this is unfortunate, using types 
this way, when it is all really just data. Clojure does -- or would do -- it 
better, simpler, directer, and all of that.

Yet we would like to leverage these frameworks. Or rather, we must for various 
pragmatic and business reasons.

And any time we need to "communicate" to these frameworks "through" their 
desired fashion of generic types and annotations, we can, of course, create the 
appropriate .java files to represent what is needed (and then do the invocation 
back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this works.

However this is quite tedious because in these frameworks I mentioned you end 
up having to create these Java files quite a bit. For example, when expressing 
a streaming data pipeline to Beam, you may specify multiple transforms, each a 
function with its own type signature.

A little searching and it seems Clojure has shied away from generating generic 
type information in places where it could offer this capability.

For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and 
other dynamic bytecode generation features of Clojure.

However it seems to me that `proxy` (and these others) could allow one to pass 
in a representation of desired type arguments, annotations, etc. and then we 
could remain in Clojure to interop with these popular frameworks.

I respect Clojure's efforts to keep its core small and wait for worthy features 
to prove themselves.

So my question is not when is Clojure going to do this, but rather:

Are there any precedents in the community for someone building out the kind of 
richer Java interop that I'm nodding toward here?

For example, does anyone know of an attempt out there to build a `proxy` 
plus-plus, that would allow one to extend a generic class with provided type 
parameters and have this metadata properly rendered in the bytecode that proxy 
produces?

If not, as a practical and hopefully quick and workable solution, I was 
thinking it'd be possible to take the bytecode emitted by proxy and re-run it 
through ASM to create a *new* class with simply the proxy-pr

RE: Java Interop on steroids?

[Sean Corfield]

Two reasons:

  1.  When I searched for ‘annotations in Clojure’ I pretty much came up empty 
– so I didn’t know you could do that sort of stuff with gen-class 
  2.  I was very focused on adding annotations to specific functions that I 
wanted New Relic to trace so I didn’t think much about class level stuff
  3.  We’ve since dropped the New Relic tracing stuff from our code (because we 
got the data we needed and moved on)

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of Chris 
Nuernberger 
Sent: Friday, June 21, 2019 12:40:28 PM
To: Clojure
Subject: Re: Java Interop on steroids?

Sean,

That is an interesting blog post.  Sorry if I am not following everything but 
why not use the annotation support in gen-class for those types of things?

https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34

On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
You might be interested in how we provide type-based annotations on Clojure 
functions so that tooling (in our case New Relic) sees those annotations:

https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/

I agree that this could be a lot easier.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clo...@googlegroups.com 
> on behalf of eglue 
>
Sent: Thursday, June 20, 2019 9:03:45 PM
To: Clojure
Subject: Java Interop on steroids?

Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
̶H̶i̶c̶k̶e̶y̶  guy.

However -- there are many popular Java frameworks that love to reflect on their 
annotations and their generic type signatures.

To name a heavyweight: Spring. But also, of late: big data frameworks, many 
written in Java, love reflecting on generic type signatures. My org is looking 
at Beam and Flink, for example.

These frameworks use types not for the static checking really but as parameters 
governing their own dynamic behavior. For example, Spring will use types at 
runtime to simply match objects to where they should be dynamically injected. 
Beam will look at your type signatures and do runtime validations to ensure it 
can process things appropriately. Of course this is unfortunate, using types 
this way, when it is all really just data. Clojure does -- or would do -- it 
better, simpler, directer, and all of that.

Yet we would like to leverage these frameworks. Or rather, we must for various 
pragmatic and business reasons.

And any time we need to "communicate" to these frameworks "through" their 
desired fashion of generic types and annotations, we can, of course, create the 
appropriate .java files to represent what is needed (and then do the invocation 
back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this works.

However this is quite tedious because in these frameworks I mentioned you end 
up having to create these Java files quite a bit. For example, when expressing 
a streaming data pipeline to Beam, you may specify multiple transforms, each a 
function with its own type signature.

A little searching and it seems Clojure has shied away from generating generic 
type information in places where it could offer this capability.

For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and 
other dynamic bytecode generation features of Clojure.

However it seems to me that `proxy` (and these others) could allow one to pass 
in a representation of desired type arguments, annotations, etc. and then we 
could remain in Clojure to interop with these popular frameworks.

I respect Clojure's efforts to keep its core small and wait for worthy features 
to prove themselves.

So my question is not when is Clojure going to do this, but rather:

Are there any precedents in the community for someone building out the kind of 
richer Java interop that I'm nodding toward here?

For example, does anyone know of an attempt out there to build a `proxy` 
plus-plus, that would allow one to extend a generic class with provided type 
parameters and have this metadata properly rendered in the bytecode that proxy 
produces?

If not, as a practical and hopefully quick and workable solution, I was 
thinking it'd be possible to take the bytecode emitted by proxy and re-run it 
through ASM to create a *new* class with simply the proxy-produced class bytes 
filled-in with the desired, provided type parameters. I bet this could be 
sufficient and fast, with the slight overhead of the extra class.

To do this, I think I'd need access to these proxy-made bytes... either by 
having proxy answer them somehow, or offering a hook to contribute t

Re: Java Interop on steroids?

[Nathan Fisher]

Storm recently moved away from Clojure in its core.

https://storm.apache.org/2019/05/30/storm200-released.html

I wonder how much of the legacy Clojure core could be optimised or if they
reached an upper limit imposed by the runtime/architecture. That being said
I suspect for 90% of orgs they'll never hit that boundary.

On Fri, Jun 21, 2019 at 16:40, Chris Nuernberger 
wrote:

> Sean,
>
> That is an interesting blog post.  Sorry if I am not following everything
> but why not use the annotation support in gen-class for those types of
> things?
>
>
> https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34
>
> On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
>>
>> You might be interested in how we provide type-based annotations on
>> Clojure functions so that tooling (in our case New Relic) sees those
>> annotations:
>>
>>
>>
>>
>> https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/
>>
>>
>>
>> I agree that this could be a lot easier.
>>
>>
>>
>> Sean Corfield -- (904) 302-SEAN
>> An Architect's View -- http://corfield.org/
>>
>> "If you're not annoying somebody, you're not really alive."
>> -- Margaret Atwood
>>
>>
>> ------
>> *From:* clo...@googlegroups.com  on behalf of
>> eglue 
>> *Sent:* Thursday, June 20, 2019 9:03:45 PM
>> *To:* Clojure
>> *Subject:* Java Interop on steroids?
>>
>> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶
>> ̶H̶i̶c̶k̶e̶y̶  guy.
>>
>> However -- there are many popular Java frameworks that love to reflect on
>> their annotations and their generic type signatures.
>>
>> To name a heavyweight: Spring. But also, of late: big data frameworks,
>> many written in Java, love reflecting on generic type signatures. My org is
>> looking at Beam and Flink, for example.
>>
>> These frameworks use types not for the static checking really but as
>> parameters governing their own dynamic behavior. For example, Spring will
>> use types at runtime to simply match objects to where they should be
>> dynamically injected. Beam will look at your type signatures and do runtime
>> validations to ensure it can process things appropriately. Of course this
>> is unfortunate, using types this way, when it is all really just data.
>> Clojure does -- or would do -- it better, simpler, directer, and all of
>> that.
>>
>> Yet we would like to leverage these frameworks. Or rather, we must for
>> various pragmatic and business reasons.
>>
>> And any time we need to "communicate" to these frameworks "through" their
>> desired fashion of generic types and annotations, we can, of course, create
>> the appropriate .java files to represent what is needed (and then do the
>> invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this
>> works.
>>
>> However this is quite tedious because in these frameworks I mentioned you
>> end up having to create these Java files quite a bit. For example, when
>> expressing a streaming data pipeline to Beam, you may specify multiple
>> transforms, each a function with its own type signature.
>>
>> A little searching and it seems Clojure has shied away from generating
>> generic type information in places where it could offer this capability.
>>
>> For example, in `proxy` ... or I suppose also in `gen-class`, `reify`,
>> and other dynamic bytecode generation features of Clojure.
>>
>> However it seems to me that `proxy` (and these others) could allow one to
>> pass in a representation of desired type arguments, annotations, etc. and
>> then we could remain in Clojure to interop with these popular frameworks.
>>
>> I respect Clojure's efforts to keep its core small and wait for worthy
>> features to prove themselves.
>>
>> So my question is not when is Clojure going to do this, but rather:
>>
>> Are there any precedents in the community for someone building out the
>> kind of richer Java interop that I'm nodding toward here?
>>
>> For example, does anyone know of an attempt out there to build a `proxy`
>> plus-plus, that would allow one to extend a generic class with provided
>> type parameters and have this metadata properly rendered in the bytecode
>> that proxy produces?
>>
>> If not, as a practical and hopefully quick and workable solution, I was
>> thinking it'd be possible to take the bytecode emitted by pr

Re: Java Interop on steroids?

[Chris Nuernberger]

Sean, 

That is an interesting blog post.  Sorry if I am not following everything 
but why not use the annotation support in gen-class for those types of 
things?

https://github.com/clojure/clojure/blob/8af7e9a92570eb28c58b15481ae9c271d891c028/test/clojure/test_clojure/genclass/examples.clj#L34

On Friday, June 21, 2019 at 1:29:56 PM UTC-6, Sean Corfield wrote:
>
> You might be interested in how we provide type-based annotations on 
> Clojure functions so that tooling (in our case New Relic) sees those 
> annotations:
>
>  
>
>
> https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/
>
>  
>
> I agree that this could be a lot easier.
>
>  
>
> Sean Corfield -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
> --
> *From:* clo...@googlegroups.com   > on behalf of eglue >
> *Sent:* Thursday, June 20, 2019 9:03:45 PM
> *To:* Clojure
> *Subject:* Java Interop on steroids? 
>  
> Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
> ̶H̶i̶c̶k̶e̶y̶  guy. 
>
> However -- there are many popular Java frameworks that love to reflect on 
> their annotations and their generic type signatures.
>
> To name a heavyweight: Spring. But also, of late: big data frameworks, 
> many written in Java, love reflecting on generic type signatures. My org is 
> looking at Beam and Flink, for example.
>
> These frameworks use types not for the static checking really but as 
> parameters governing their own dynamic behavior. For example, Spring will 
> use types at runtime to simply match objects to where they should be 
> dynamically injected. Beam will look at your type signatures and do runtime 
> validations to ensure it can process things appropriately. Of course this 
> is unfortunate, using types this way, when it is all really just data. 
> Clojure does -- or would do -- it better, simpler, directer, and all of 
> that.
>
> Yet we would like to leverage these frameworks. Or rather, we must for 
> various pragmatic and business reasons.
>
> And any time we need to "communicate" to these frameworks "through" their 
> desired fashion of generic types and annotations, we can, of course, create 
> the appropriate .java files to represent what is needed (and then do the 
> invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this 
> works.
>
> However this is quite tedious because in these frameworks I mentioned you 
> end up having to create these Java files quite a bit. For example, when 
> expressing a streaming data pipeline to Beam, you may specify multiple 
> transforms, each a function with its own type signature.
>
> A little searching and it seems Clojure has shied away from generating 
> generic type information in places where it could offer this capability. 
>
> For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and 
> other dynamic bytecode generation features of Clojure.
>
> However it seems to me that `proxy` (and these others) could allow one to 
> pass in a representation of desired type arguments, annotations, etc. and 
> then we could remain in Clojure to interop with these popular frameworks.
>
> I respect Clojure's efforts to keep its core small and wait for worthy 
> features to prove themselves.
>
> So my question is not when is Clojure going to do this, but rather:
>
> Are there any precedents in the community for someone building out the 
> kind of richer Java interop that I'm nodding toward here?
>
> For example, does anyone know of an attempt out there to build a `proxy` 
> plus-plus, that would allow one to extend a generic class with provided 
> type parameters and have this metadata properly rendered in the bytecode 
> that proxy produces?
>
> If not, as a practical and hopefully quick and workable solution, I was 
> thinking it'd be possible to take the bytecode emitted by proxy and re-run 
> it through ASM to create a *new* class with simply the proxy-produced class 
> bytes filled-in with the desired, provided type parameters. I bet this 
> could be sufficient and fast, with the slight overhead of the extra class.
>
> To do this, I think I'd need access to these proxy-made bytes... either by 
> having proxy answer them somehow, or offering a hook to contribute to the 
> defined bytecode before it is committed to the classloader, or by having 
> DynamicClassLoader have these bytes on hand for inquiring parties, or 
> something else along these lines. This would likely be something that 
> Clojure core would have to expose .. correct me if I'm wrong.
>
>

RE: Java Interop on steroids?

[Sean Corfield]

You might be interested in how we provide type-based annotations on Clojure 
functions so that tooling (in our case New Relic) sees those annotations:

https://corfield.org/blog/2013/05/01/instrumenting-clojure-for-new-relic-monitoring/

I agree that this could be a lot easier.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of eglue 

Sent: Thursday, June 20, 2019 9:03:45 PM
To: Clojure
Subject: Java Interop on steroids?

Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
̶H̶i̶c̶k̶e̶y̶  guy.

However -- there are many popular Java frameworks that love to reflect on their 
annotations and their generic type signatures.

To name a heavyweight: Spring. But also, of late: big data frameworks, many 
written in Java, love reflecting on generic type signatures. My org is looking 
at Beam and Flink, for example.

These frameworks use types not for the static checking really but as parameters 
governing their own dynamic behavior. For example, Spring will use types at 
runtime to simply match objects to where they should be dynamically injected. 
Beam will look at your type signatures and do runtime validations to ensure it 
can process things appropriately. Of course this is unfortunate, using types 
this way, when it is all really just data. Clojure does -- or would do -- it 
better, simpler, directer, and all of that.

Yet we would like to leverage these frameworks. Or rather, we must for various 
pragmatic and business reasons.

And any time we need to "communicate" to these frameworks "through" their 
desired fashion of generic types and annotations, we can, of course, create the 
appropriate .java files to represent what is needed (and then do the invocation 
back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this works.

However this is quite tedious because in these frameworks I mentioned you end 
up having to create these Java files quite a bit. For example, when expressing 
a streaming data pipeline to Beam, you may specify multiple transforms, each a 
function with its own type signature.

A little searching and it seems Clojure has shied away from generating generic 
type information in places where it could offer this capability.

For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and 
other dynamic bytecode generation features of Clojure.

However it seems to me that `proxy` (and these others) could allow one to pass 
in a representation of desired type arguments, annotations, etc. and then we 
could remain in Clojure to interop with these popular frameworks.

I respect Clojure's efforts to keep its core small and wait for worthy features 
to prove themselves.

So my question is not when is Clojure going to do this, but rather:

Are there any precedents in the community for someone building out the kind of 
richer Java interop that I'm nodding toward here?

For example, does anyone know of an attempt out there to build a `proxy` 
plus-plus, that would allow one to extend a generic class with provided type 
parameters and have this metadata properly rendered in the bytecode that proxy 
produces?

If not, as a practical and hopefully quick and workable solution, I was 
thinking it'd be possible to take the bytecode emitted by proxy and re-run it 
through ASM to create a *new* class with simply the proxy-produced class bytes 
filled-in with the desired, provided type parameters. I bet this could be 
sufficient and fast, with the slight overhead of the extra class.

To do this, I think I'd need access to these proxy-made bytes... either by 
having proxy answer them somehow, or offering a hook to contribute to the 
defined bytecode before it is committed to the classloader, or by having 
DynamicClassLoader have these bytes on hand for inquiring parties, or something 
else along these lines. This would likely be something that Clojure core would 
have to expose .. correct me if I'm wrong.

Would love to hear any other immediate thoughts on this.

I think once you realize that this generic type information is not even being 
used for "static typing" by these frameworks but rather as an (albeit poor) 
means to receive semantic information from their clients (as parameters to 
govern their own dynamic behavior), then the need/value of being able to remain 
in Clojure and communicate to these libraries through generic params and 
annotations perhaps becomes more understandable..










--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, 

Java Interop on steroids?

[eglue]

Don't get me wrong, I'm as much against types as the next R̶i̶c̶h̶ 
̶H̶i̶c̶k̶e̶y̶  guy.

However -- there are many popular Java frameworks that love to reflect on 
their annotations and their generic type signatures.

To name a heavyweight: Spring. But also, of late: big data frameworks, many 
written in Java, love reflecting on generic type signatures. My org is 
looking at Beam and Flink, for example.

These frameworks use types not for the static checking really but as 
parameters governing their own dynamic behavior. For example, Spring will 
use types at runtime to simply match objects to where they should be 
dynamically injected. Beam will look at your type signatures and do runtime 
validations to ensure it can process things appropriately. Of course this 
is unfortunate, using types this way, when it is all really just data. 
Clojure does -- or would do -- it better, simpler, directer, and all of 
that.

Yet we would like to leverage these frameworks. Or rather, we must for 
various pragmatic and business reasons.

And any time we need to "communicate" to these frameworks "through" their 
desired fashion of generic types and annotations, we can, of course, create 
the appropriate .java files to represent what is needed (and then do the 
invocation back to Clojure via IFn.invoke or Compiler.eval, etc). Yes, this 
works.

However this is quite tedious because in these frameworks I mentioned you 
end up having to create these Java files quite a bit. For example, when 
expressing a streaming data pipeline to Beam, you may specify multiple 
transforms, each a function with its own type signature.

A little searching and it seems Clojure has shied away from generating 
generic type information in places where it could offer this capability. 

For example, in `proxy` ... or I suppose also in `gen-class`, `reify`, and 
other dynamic bytecode generation features of Clojure.

However it seems to me that `proxy` (and these others) could allow one to 
pass in a representation of desired type arguments, annotations, etc. and 
then we could remain in Clojure to interop with these popular frameworks.

I respect Clojure's efforts to keep its core small and wait for worthy 
features to prove themselves.

So my question is not when is Clojure going to do this, but rather:

Are there any precedents in the community for someone building out the kind 
of richer Java interop that I'm nodding toward here?

For example, does anyone know of an attempt out there to build a `proxy` 
plus-plus, that would allow one to extend a generic class with provided 
type parameters and have this metadata properly rendered in the bytecode 
that proxy produces?

If not, as a practical and hopefully quick and workable solution, I was 
thinking it'd be possible to take the bytecode emitted by proxy and re-run 
it through ASM to create a *new* class with simply the proxy-produced class 
bytes filled-in with the desired, provided type parameters. I bet this 
could be sufficient and fast, with the slight overhead of the extra class.

To do this, I think I'd need access to these proxy-made bytes... either by 
having proxy answer them somehow, or offering a hook to contribute to the 
defined bytecode before it is committed to the classloader, or by having 
DynamicClassLoader have these bytes on hand for inquiring parties, or 
something else along these lines. This would likely be something that 
Clojure core would have to expose .. correct me if I'm wrong.

Would love to hear any other immediate thoughts on this.

I think once you realize that this generic type information is not even 
being used for "static typing" by these frameworks but rather as an (albeit 
poor) means to receive semantic information from their clients (as 
parameters to govern their own dynamic behavior), then the need/value of 
being able to remain in Clojure and communicate to these libraries through 
generic params and annotations perhaps becomes more understandable..









-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/128dd732-b79e-4c35-999f-691cdc42512b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.