Hi, Did you try doing this across JVMs? Its possible to serialize a lambda within a JVM (easily), but not across JVMs.
Marko. http://markorodriguez.com On May 11, 2015, at 9:47 AM, Bryn Cooke <[email protected]> wrote: > Hmm, I just gave it a go and didn't have to do anything to successfully > serialize a Lambda. It just worked. > > As long as you don't reference anything outside the lambda it's fine. If you > do then you get a java.io.NotSerializableException. > > Bryn > > > On 11/05/15 16:22, Marko Rodriguez wrote: >> Hi Bryn, >> >> Huh. That is how you "dehydrate()" a closure in Groovy. What is the field >> name of the "it" in a Java8 lambda? If we could figure that out, then we >> could have nice "static dehydrate()" and "static hydrate()" helpers in >> gremlin-core/function. >> >> Good stuff, >> Marko. >> >> http://markorodriguez.com >> >> On May 11, 2015, at 9:19 AM, Bryn Cooke <[email protected]> wrote: >> >>> Hi Marko, >>> >>> Just in case Kryo 3 doesn't serialialize lambdas in the way you expect it >>> would be worth trying to null out the reference to the outer object in the >>> lambda object using reflection. I *believe* that this is the only thing >>> stopping serialization using the standard Java Seriazable mechanism. >>> >>> Bryn >>> >>> On 11/05/15 16:06, Marko Rodriguez wrote: >>>> Hi, >>>> >>>>> SubgraphStrategyTest contains test cases that access both vertices >>>>> incident >>>>> on edges which isn't allowed under GraphComputer semantics. It silently >>>>> passes in GraphComputerTest because the TP3 implementations ignore those >>>>> cases. I believe implementations should throw an exception on such access. >>>>> This example demonstrates how easy it is to end up with difficult to find >>>>> bugs otherwise. We would put a high burden on the user to write unit tests >>>>> for their strategies to make sure they detect those cases where it makes a >>>>> difference. >>>> Yes. I agree that StarGraph should throw and exception when, right now, it >>>> returns Collections.emptyIterator(). >>>> >>>> I will add to the GraphComputer TestSuite verification of the topology of >>>> the "star vertex" so that all vendors have the same accessible data and >>>> the same Exceptions thrown. >>>> >>>>> Moreover, we should think about such FilterStrategies in general. Should >>>>> there be a way to restrict them to just stargraph access to ensure that >>>>> they always work in GraphComputer? Basically, I think it would be great to >>>>> avoid a case where a strategy works in OLTP but doesn't in OLAP because >>>>> most people would probably implement it for OLTP first. This is >>>>> particularly true for SubgraphStrategies where you try to restrict access. >>>> There are various situations in which OLAP and OLTP do not line up. >>>> However, I don't think we should restrict OLTP to be that which OLAP can >>>> handle. Primarily because many of the limitations of OLAP will be solved >>>> in the future. Here is the list as I know them: >>>> >>>> 1. OLAP does not support mutating steps (addV(), addE(), etc.). This is >>>> because we haven't come up with a good model for mutating in OLAP. >>>> - The latest idea on this is the "GraphMutator" interface that >>>> all OLAP vendors should supply, blah blah. >>>> - In essence, it will happens, just not right now (3.1+). >>>> 2. OLAP does not support lambdas. This is because Java8 lambdas are not >>>> serializable. >>>> - We can fake this with passing Groovy strings, but its not >>>> "true" Gremlin-Java8. >>>> - Kryo3 supposedly figured out a way to serialize Java8 lambdas >>>> -- perhaps our savior? >>>> - In essence, it will happen, just not right now. >>>> 3. OLAP does not support mid-traversal barriers (i.e. >>>> g.V.count().is(gt(100))). This is because a barrier step requires a >>>> MapReduce and we can't go MapReduce -> VertexProgram. >>>> - There is a ticket to support OLAP->OLTP->OLAP which will all >>>> this. >>>> - In essence, it will happen, just not right now. >>>> 4. OLAP does not support non-graph object processing (i.e. >>>> __.(1,2,3,4).order(local).by(incr).sum() in OLAP (i.e. no graph objects)). >>>> - This is possible (and actually not that hard) and there is a >>>> ticket for it. >>>> - In essence, it will happen, just not right now. >>>> 5. OLAP localTraversals can not leave "the star graph." >>>> - This is because we don't have that data available locally, >>>> though, I don't see why we can't solve this with "TraverserSet -- scoping" >>>> in OLAP. >>>> - The solution would be complex to write, but doable. The way >>>> TraveralMatrix works in OLAP, the code is staged for this. >>>> - In essence, it will happen, just not right now. >>>> 6. OLAP does not support MatchStep. >>>> - There is currently no thoughts on solving this, but I suspect >>>> its a natural fall out once 4 is complete. >>>> - In essence, it will happen, just not right now. >>>> >>>> So, while OLAP and OLTP differ, I would not restrict OLTP to be what OLAP >>>> can do. What we could do is make the ComputerVerificationStrategy's >>>> analysis easily accessible to users. For instance: >>>> >>>> ComputerVerificationStrategy.check(myTraversal) // which is simply a >>>> call to ComputerVerificationStrategy.instance().apply(myTraversal) :) >>>> >>>> This way, a user can easily determine if their traversal will work on OLAP >>>> without having to actually try and execute the job. One of the things that >>>> Daniel Kuppitz is working on is making sure that >>>> ComputerVerificationStrategy is able to identify the 5 areas above and >>>> throw the appropriate exception explaining "why" to the user. As we move >>>> forward with 3.1, 3.2, 3.3, etc. hopefully the list becomes smaller and >>>> smaller. >>>> >>>> Thanks, >>>> Marko. >>>> >>>> http://markorodriguez.com >>>> >>>> >>>> >> >
