I don't know what the problem could be here. I'm not aware of others having problems building master atm.
On Mon, Jan 4, 2016 at 11:41 PM, Jonathan Ellithorpe <j...@cs.stanford.edu> wrote: > Just tried that, but I'm still getting this "Output directory > target/test-output/~traversers already exists" error above. > > I noticed the following in the logs: > > [INFO] Slf4jLogger$anonfun$receive$1$anonfun$applyOrElse$3 - Shutting down > remote daemon. > [INFO] Slf4jLogger$anonfun$receive$1$anonfun$applyOrElse$3 - Remote daemon > shut down; proceeding with flushing remote transports. > [INFO] Slf4jLogger$anonfun$receive$1$anonfun$applyOrElse$3 - Remoting shut > down. > [WARN] HadoopGremlinPlugin - Be sure to set the environmental variable: > HADOOP_GREMLIN_LIBS > [WARN] FileUtil - Failed to delete file or dir > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/m]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b4000000c2]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b2000000c6]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b0000000bf]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b5000000c5]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b3000000c4]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b6000000c3]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612b1000000c1]: > it still exists. > [WARN] FileUtil - Failed to delete file or dir > > [/home/jdellit/tmp/incubator-tinkerpop/spark-gremlin/target/test-output/~traversers/.nfs00000000066612af000000c0]: > it still exists. > [INFO] Logging$class - Running Spark version 1.5.1 > [INFO] Logging$class - Changing view acls to: jdellit > [INFO] Logging$class - Changing modify acls to: jdellit > > The appearance of the .nfsXXX files leads me to suspect that files are > being deleted by one thread that are still open on another thread. If that > other thread attempts a write then an .nfsXXX file is generated. > > Are other folks running these tests on NFS? > > Jonathan > > > On Mon, Jan 4, 2016 at 4:00 PM Stephen Mallette <spmalle...@gmail.com> > wrote: > > > how about good ol' mvn clean install? that works for me (and others) and > > travis on master. > > > > i guess "mvn clean compile" and "mvn test" fails because gremlin-shaded > > doesn't get built as the core of its work is bound to the "package" > phase, > > so you get all those errors. it seems that you have to minimally execute > > the "package" phase of the maven life cycle > > > > On Mon, Jan 4, 2016 at 4:47 PM, Jonathan Ellithorpe <j...@cs.stanford.edu > > > > wrote: > > > > > Hi Stephen, thanks for the heads up. I remember getting stuck trying to > > > find a point in the tree from which to base my changes. Either there > was > > an > > > error compiling tinkerpop, or the unit tests did not pass. For example > on > > > tag 3.1.0-incubating I get the following error when executing mvn clean > > > compile: > > > > > > [ERROR] COMPILATION ERROR : > > > [INFO] ------------------------------------------------------------- > > > [ERROR] > > > > > > > > > /home/jdellit/tmp/incubator-tinkerpop/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONReader.java:[41,53] > > > package org.apache.tinkerpop.shaded.jackson.core.type does not exist > > > . > > > . > > > . > > > > > > "mvn clean package -DskipTests" does work. > > > > > > Then when I run "mvn test" I get: > > > > > > shouldGetVersion(org.apache.tinkerpop.gremlin.util.GremlinTest) Time > > > elapsed: 0.016 sec <<< ERROR! > > > java.lang.ExceptionInInitializerError: null > > > at com.jcabi.manifests.Manifests.read(Manifests.java:274) > > > at org.apache.tinkerpop.gremlin.util.Gremlin.<clinit>(Gremlin.java:32) > > > at > > > > > > > > > org.apache.tinkerpop.gremlin.util.GremlinTest.shouldGetVersion(GremlinTest.java:39) > > > > > > And several other errors while running tests in Gremlin Core. > Strangely, > > > however, running "mvn package" at this point does not produce those > > errors, > > > even though its running the same tests. It encounters a different error > > for > > > Spark Gremlin: > > > > > > > > > > > > shouldGracefullyHandleBadGremlinHadoopLibs(org.apache.tinkerpop.gremlin.hadoop.groovy.plugin.HadoopGremlinPluginTest) > > > Time elapsed: 3.318 sec <<< ERROR! > > > > > > org.apache.tinkerpop.gremlin.groovy.plugin.RemoteException: > > > java.util.concurrent.ExecutionException: > > > org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory > > > target/test-output/~traversers already exists > > > at > > > > > > > > > java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) > > > at > > java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) > > > at > > > > > > > > > org.apache.tinkerpop.gremlin.hadoop.groovy.plugin.HadoopRemoteAcceptor.submit(HadoopRemoteAcceptor.java:99) > > > at > > > > > > > > > org.apache.tinkerpop.gremlin.hadoop.groovy.plugin.HadoopGremlinPluginTest.shouldGracefullyHandleBadGremlinHadoopLibs(HadoopGremlinPluginTest.java:169) > > > Caused by: org.apache.hadoop.mapred.FileAlreadyExistsException: Output > > > directory target/test-output/~traversers already exists > > > at > > > > > > > > > org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1011) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:998) > > > at > > > > > > > > > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) > > > at > > > > > > > > > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) > > > at org.apache.spark.rdd.RDD.withScope(RDD.scala:306) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:998) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:938) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:930) > > > at > > > > > > > > > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) > > > at > > > > > > > > > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) > > > at org.apache.spark.rdd.RDD.withScope(RDD.scala:306) > > > at > > > > > > > > > org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:930) > > > at > > > > > > > > > org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:809) > > > at > > > > > > > > > org.apache.tinkerpop.gremlin.spark.process.computer.SparkExecutor.saveMapReduceRDD(SparkExecutor.java:208) > > > at > > > > > > > > > org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submit$21(SparkGraphComputer.java:211) > > > > > > I am confused as to why mvn package results in the success of certain > > tests > > > that fail for mvn package. > > > > > > Jonathan > > > > > > On Mon, Jan 4, 2016 at 7:08 AM Stephen Mallette <spmalle...@gmail.com> > > > wrote: > > > > > > > Jonathan, just wanted to throw a heads up your way so that you're > aware > > > of > > > > our expecting timing. If all goes as planned, we will head into code > > > > freeze for 3.1.1-incubating in about three weeks. If you are still > > > > planning on submitting PRs for: > > > > > > > > https://issues.apache.org/jira/browse/TINKERPOP3-997 > > > > https://issues.apache.org/jira/browse/TINKERPOP3-998 > > > > > > > > we'd need to see it in that time frame. I don't mean to apply > > pressure, > > > I > > > > just don't want to miss the chance to get these fixes in for > > > > 3.1.1-incubating. > > > > > > > > > > > > > > > > On Wed, Dec 9, 2015 at 1:24 AM, Jonathan Ellithorpe < > > j...@cs.stanford.edu > > > > > > > > wrote: > > > > > > > > > Hi Stephen, working on that now, thanks for pinging me on this. > > > > > > > > > > On Tue, Dec 8, 2015 at 4:48 PM Stephen Mallette < > > spmalle...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi Jonathan, just wondering if you still plan to look at offering > > PRs > > > > > for: > > > > > > > > > > > > https://issues.apache.org/jira/browse/TINKERPOP-998 > > > > > > https://issues.apache.org/jira/browse/TINKERPOP-997 > > > > > > > > > > > > I'll stay away from those, if you think you will be working on > > them. > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 30, 2015 at 1:39 PM, Stephen Mallette < > > > > spmalle...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Using VertexProperyFeatures.FEATURE_{ADD, REMOVE}_PROPERTY > > > perhaps > > > > > > > would be more consistent with the logic used everywhere else... > > > > > > > > > > > > > > yeah - i'm +1 for this approach. it makes more sense given > > > ADD/REMOVE > > > > > > > already being the pattern for graph Element instances. > > > > > > > > > > > > > > On Mon, Nov 30, 2015 at 1:31 PM, Jonathan Ellithorpe < > > > > > > j...@cs.stanford.edu> > > > > > > > wrote: > > > > > > > > > > > > > >> I think it's either that or change FEATURE_META_PROPERTY to a > > > > > symmetric > > > > > > >> VertexFeatures.FEATURE_{ADD, REMOVE}_METAPROPERTY to pair with > > > > > > >> VertexFeatures.FEATURE_{ADD, REMOVE}_PROPERTY. > > > > > > >> > > > > > > >> Using VertexProperyFeatures.FEATURE_{ADD, REMOVE}_PROPERTY > > perhaps > > > > > would > > > > > > >> be > > > > > > >> more consistent with the logic used everywhere else... > > > > > > >> > > > > > > >> > > > > > > >> On Mon, Nov 30, 2015 at 6:30 AM Stephen Mallette < > > > > > spmalle...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >> > ugh - mess. maybe we should just keep the add/remove > symmetry > > > and > > > > > > >> > deprecate FEATURE_META_PROPERTY then. > > > > > > >> > > > > > > > >> > > > > > > > >> > On Sat, Nov 28, 2015 at 4:11 PM, Jonathan Ellithorpe < > > > > > > >> j...@cs.stanford.edu> > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> > > 1) Yes, I can submit a PR for fixing the SIMPLE feature > > > > > requirement > > > > > > >> set. > > > > > > >> > > 2) I also agree with deprecating > > > > > > >> > > VertexPropertyFeatures.FEATURE_ADD_PROPERTY, but looking > at > > > the > > > > > > code I > > > > > > >> > > think I realized there is a slight complication here. That > > is, > > > > > what > > > > > > >> to do > > > > > > >> > > with VertexPropertyFeatures.FEATURE_REMOVE_PROPERTY. Does > > > > > > >> > > VertexFeatures.FEATURE_META_PROPERTIES imply both ADD and > > > > REMOVE, > > > > > or > > > > > > >> only > > > > > > >> > > ADD? In the later case, we would need to leave > > > > > > >> > > VertexPropertyFeatures.FEATURE_REMOVE_PROPERTIES. > > Personally, > > > > > seeing > > > > > > >> as > > > > > > >> > how > > > > > > >> > > VertexFeatures, extending ElementFeatures, has a > > > > > > FEATURE_ADD_PROPERTY > > > > > > >> and > > > > > > >> > > FEATURE_REMOVE_PROPERTY, that the FEATURE_META_PROPERTIES > be > > > > > changed > > > > > > >> to > > > > > > >> > > FEATURE_ADD_METAPROPERTY and FEATURE_REMOVE_METAPROPERTY. > > > > > > >> > > > > > > > > >> > > Jonathan > > > > > > >> > > > > > > > > >> > > On Fri, Nov 27, 2015 at 4:55 AM Stephen Mallette < > > > > > > >> spmalle...@gmail.com> > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > >> > > > ...damn - hot key sent my post too soon - trying again: > > > > > > >> > > > > > > > > > >> > > > Hi Jonathan, thanks for bringing this up. It would be > > nice > > > if > > > > > we > > > > > > >> could > > > > > > >> > > > expand coverage of our test suite by simply improving > the > > > way > > > > in > > > > > > >> which > > > > > > >> > > > features are applied. I was about to suggest a big set > of > > > > > changes > > > > > > >> > when I > > > > > > >> > > > realized that FeatureRequirementSet.SIMPLE is just > defined > > > > > wrong. > > > > > > >> It > > > > > > >> > > > shouldn't have this entry: > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > addFeatureRequirement.Factory.create(Graph.Features.VertexPropertyFeatures.FEATURE_ADD_PROPERTY, > > > > > > >> > > > Graph.Features.VertexPropertyFeatures.class)); > > > > > > >> > > > > > > > > > >> > > > it should just be: > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > add(FeatureRequirement.Factory.create(Graph.Features.VertexFeatures.FEATURE_ADD_PROPERTY, > > > > > > >> > > > Graph.Features.VertexFeatures.class)); > > > > > > >> > > > > > > > > > >> > > > I've created an issue for that to track things: > > > > > > >> > > > > > > > > > >> > > > https://issues.apache.org/jira/browse/TINKERPOP3-997 > > > > > > >> > > > > > > > > > >> > > > because it is a "breaking" change as it will open up > tests > > > and > > > > > > >> possibly > > > > > > >> > > > cause existing implementations to fail. If you'd like > to > > > > > submit a > > > > > > >> PR > > > > > > >> > for > > > > > > >> > > > this little fix, as you were the reporter for it and as > > > > someone > > > > > > who > > > > > > >> can > > > > > > >> > > > test it in a way that is currently failing for them, > just > > > let > > > > me > > > > > > >> know. > > > > > > >> > > > > > > > > > >> > > > As for the this issue: > > > > > > >> > > > > Graph.Features.VertexPropertyFeatures.FEATURE_ADD_PROPERTY > > > > > > >> > > > <==> > > Graph.Features.VertexFeatures.FEATURE_META_PROPERTIES - > > > > > yeah > > > > > > - > > > > > > >> we > > > > > > >> > > need > > > > > > >> > > > to deprecate one of those as they are the same thing. > Not > > > > sure > > > > > if > > > > > > >> > anyone > > > > > > >> > > > has any preferences on that. in one sense, > > > > FEATURE_ADD_PROPERTY > > > > > > is > > > > > > >> > > better > > > > > > >> > > > because it matches the approach for Vertex/Edge. > > > > > > >> > > > > > > > > >> > > On the other hand, the > > > > > > >> > > > documentation refers to this feature as > "meta-properties". > > > I > > > > > > guess > > > > > > >> i > > > > > > >> > > would > > > > > > >> > > > go with keeping FEATURE_META_PROPERTIES and deprecating > > > > > > >> > > > FEATURE_ADD_PROPERTY. I've created an issue as such: > > > > > > >> > > > > > > > > > >> > > > https://issues.apache.org/jira/browse/TINKERPOP3-998 > > > > > > >> > > > > > > > > >> > > If no one has any objections in the next 72 hours (Monday, > > > > > November > > > > > > >> 30, > > > > > > >> > > > 2015 at 7:45am) I'll assume lazy consensus and we can > move > > > > > forward > > > > > > >> with > > > > > > >> > > > this one. > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > On Fri, Nov 27, 2015 at 7:35 AM, Stephen Mallette < > > > > > > >> > spmalle...@gmail.com> > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > >> > > > > Hi Jonathan, thanks for bringing this up. It would be > > > nice > > > > if > > > > > > we > > > > > > >> > could > > > > > > >> > > > > expand coverage of our test suite by simply improving > > the > > > > way > > > > > in > > > > > > >> > which > > > > > > >> > > > > features are applied. I was about to suggest a big > set > > of > > > > > > changes > > > > > > >> > > when I > > > > > > >> > > > > realized that FeatureRequirementSet.SIMPLE is just > > defined > > > > > > >> wrong. It > > > > > > >> > > > > shouldn't have > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > addFeatureRequirement.Factory.create(Graph.Features.VertexPropertyFeatures.FEATURE_ADD_PROPERTY, > > > > > > >> > > > > Graph.Features.VertexPropertyFeatures.class)); > > > > > > >> > > > > > > > > > > >> > > > > it should just be: > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Mon, Nov 23, 2015 at 7:39 PM, Jonathan Ellithorpe < > > > > > > >> > > > j...@cs.stanford.edu> > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > > > > >> > > > >> Hello all, > > > > > > >> > > > >> > > > > > > >> > > > >> I am currently working on an experimental > > implementation > > > of > > > > > > >> > TinkerPop3 > > > > > > >> > > > on > > > > > > >> > > > >> an in-memory key-value store called RAMCloud. In the > > > > process > > > > > of > > > > > > >> > > running > > > > > > >> > > > >> the > > > > > > >> > > > >> unit tests I noticed that turning on support for > > > > persistence > > > > > > did > > > > > > >> not > > > > > > >> > > > >> trigger any new unit tests in GraphTests. Looking > into > > > the > > > > > > >> matter, I > > > > > > >> > > > found > > > > > > >> > > > >> that the unit test that tests this, > > shouldPersistOnClose, > > > > was > > > > > > not > > > > > > >> > > > >> executing > > > > > > >> > > > >> because meta properties support is included in its > > > feature > > > > > > >> > > requirements, > > > > > > >> > > > >> but I do not have support for meta properties. Oddly, > > > > though, > > > > > > >> this > > > > > > >> > > > >> features > > > > > > >> > > > >> requirement seems to be superfluous, since the test > > does > > > > not > > > > > > >> utilize > > > > > > >> > > > meta > > > > > > >> > > > >> properties. > > > > > > >> > > > >> > > > > > > >> > > > >> An orthogonal issue seems to be that > > > > > > >> > > > >> > > > Graph.Features.VertexPropertyFeatures.FEATURE_ADD_PROPERTY > > > > > <==> > > > > > > >> > > > >> Graph.Features.VertexFeatures.FEATURE_META_PROPERTIES > > > > > > >> > > > >> > > > > > > >> > > > >> Best, > > > > > > >> > > > >> Jonathan > > > > > > >> > > > >> > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
