[
https://issues.apache.org/jira/browse/GIRAPH-528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595591#comment-13595591
]
Hudson commented on GIRAPH-528:
-------------------------------
Integrated in Giraph-trunk-Commit #784 (See
[https://builds.apache.org/job/Giraph-trunk-Commit/784/])
GIRAPH-528: Decouple vertex implementation from edge storage (apresta)
(Revision 3f5009aea0797563304dd8ba1aa6fb06ebaab83b)
Result = FAILURE
alessandro :
http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=3f5009aea0797563304dd8ba1aa6fb06ebaab83b
Files :
*
giraph-core/src/test/java/org/apache/giraph/vertex/TestIntIntNullIntVertex.java
* giraph-core/src/test/java/org/apache/giraph/graph/TestVertexAndEdges.java
*
giraph-examples/src/test/java/org/apache/giraph/examples/SimpleTriangleClosingVertexTest.java
* giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsVertex.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableIntArrayIterator.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomVertexInputFormat.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTriangleClosingVertex.java
* giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
* giraph-core/src/test/java/org/apache/giraph/utils/ComparisonUtilsTest.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/ConnectedComponentsVertex.java
* giraph-core/src/test/java/org/apache/giraph/vertex/TestMultiGraphVertex.java
* giraph-core/src/test/java/org/apache/giraph/edge/TestMultiGraphEdges.java
*
giraph-core/src/test/java/org/apache/giraph/io/TestLongDoubleDoubleAdjacencyListVertexInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/TextDoubleDoubleAdjacencyListVertexInputFormat.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleOutDegreeCountVertex.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/HashMultimapEdges.java
* giraph-hive/src/main/java/org/apache/giraph/hive/output/HiveVertexWriter.java
* giraph-core/src/main/java/org/apache/giraph/io/VertexWriter.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleDoubleDoubleTextInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/graph/GiraphTransferRegulator.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleShortestPathsVertex.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/RepresentativeVertexPageRankBenchmark.java
*
giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/TestAccumuloVertexFormat.java
* giraph-core/src/main/java/org/apache/giraph/worker/DefaultWorkerContext.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java
* giraph-examples/src/test/java/org/apache/giraph/vertex/TestVertexTypes.java
* giraph-core/src/main/java/org/apache/giraph/partition/SimplePartition.java
* giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java
* giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankComputation.java
*
giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayEdges.java
* giraph-core/src/main/java/org/apache/giraph/edge/EdgeNoValue.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphEdgeListVertexPageRankBenchmark.java
* giraph-examples/pom.xml
*
giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/ByteArrayEdges.java
* giraph-core/src/test/java/org/apache/giraph/comm/ConnectionTest.java
* giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
*
giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/vertex/EdgeListVertexBase.java
*
giraph-core/src/main/java/org/apache/giraph/partition/DiskBackedPartitionStore.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueDoubleEdgeTextOutputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/vertex/package-info.java
* giraph-core/src/main/java/org/apache/giraph/edge/LongNullArrayEdges.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleFloatDoubleVertex.java
* giraph-core/src/main/java/org/apache/giraph/vertex/SimpleMutableVertex.java
* giraph-core/src/main/java/org/apache/giraph/vertex/HashMapVertex.java
*
giraph-core/src/test/java/org/apache/giraph/edge/TestStrictRandomAccessEdges.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java
*
giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java
* giraph-core/src/main/java/org/apache/giraph/graph/DefaultVertexResolver.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleDoubleDoubleTextInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongNullEdgeArrayIterable.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/ByteArrayVertexPageRankBenchmark.java
* giraph-core/src/main/java/org/apache/giraph/edge/HashMapEdges.java
*
giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphEdgeListVertex.java
*
giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexOutputFormat.java
*
giraph-examples/src/test/java/org/apache/giraph/examples/SimpleShortestPathsVertexTest.java
* giraph-core/src/main/java/org/apache/giraph/edge/Edge.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/AdjacencyListTextVertexInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/IdWithValueTextOutputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/IntNullNullNullTextInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
*
giraph-core/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStoreByPartition.java
* giraph-core/src/main/java/org/apache/giraph/utils/EdgeComparator.java
* giraph-core/src/test/java/org/apache/giraph/vertex/TestMutableVertex.java
* giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java
* giraph-core/src/main/java/org/apache/giraph/edge/EdgeFactory.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/AdjacencyListTextVertexOutputFormat.java
*
giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleInDegreeCountVertex.java
* giraph-core/src/test/java/org/apache/giraph/comm/TestMessageStores.java
* giraph-core/src/main/java/org/apache/giraph/comm/SendMutationsCache.java
*
giraph-examples/src/test/java/org/apache/giraph/examples/ConnectedComponentsVertexTest.java
* giraph-core/src/main/java/org/apache/giraph/edge/ArrayListEdges.java
* giraph-core/src/main/java/org/apache/giraph/vertex/MutableVertex.java
* giraph-core/src/test/java/org/apache/giraph/io/TestEdgeInput.java
* giraph-core/src/main/java/org/apache/giraph/graph/VertexResolver.java
* giraph-core/src/main/java/org/apache/giraph/comm/MsgList.java
* giraph-examples/src/test/java/org/apache/giraph/TestBspBasic.java
*
giraph-core/src/main/java/org/apache/giraph/edge/MultiRandomAccessVertexEdges.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongFloatEdgeArrayIterable.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/JsonBase64VertexOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/edge/LongDoubleHashMapEdges.java
*
giraph-core/src/main/java/org/apache/giraph/comm/WorkerClientRequestProcessor.java
*
giraph-hbase/src/test/java/org/apache/giraph/io/hbase/TestHBaseRootMarkerVertextFormat.java
* giraph-core/src/main/java/org/apache/giraph/graph/Edge.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java
*
giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/io/VertexValueReader.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleFloatDoubleTextInputFormat.java
* giraph-core/src/test/java/org/apache/giraph/BspCase.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMasterComputeVertex.java
* giraph-hive/src/main/java/org/apache/giraph/hive/output/VertexToHive.java
* giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java
* giraph-core/src/main/java/org/apache/giraph/bsp/BspOutputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/AggregatorsBenchmark.java
* giraph-core/src/main/java/org/apache/giraph/utils/ComparisonUtils.java
* giraph-core/src/test/java/org/apache/giraph/utils/MockUtils.java
* giraph-core/src/main/java/org/apache/giraph/edge/MutableEdge.java
* giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/DefaultEdge.java
*
giraph-core/src/main/java/org/apache/giraph/edge/StrictRandomAccessVertexEdges.java
* giraph-core/src/main/java/org/apache/giraph/utils/EmptyIterable.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdEdges.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertex.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueFloatEdgeTextOutputFormat.java
*
giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/partition/Partition.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java
* giraph-core/src/main/java/org/apache/giraph/graph/VertexChanges.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableDoubleArrayIterator.java
* giraph-core/src/test/java/org/apache/giraph/bsp/BspUtilsTest.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java
* giraph-core/src/main/java/org/apache/giraph/vertex/RepresentativeVertex.java
* giraph-core/src/main/java/org/apache/giraph/vertex/SimpleVertex.java
* pom.xml
*
giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatGiraphRunner.java
* giraph-core/src/main/java/org/apache/giraph/edge/package-info.java
* giraph-core/src/main/java/org/apache/giraph/vertex/EdgeListVertex.java
*
giraph-examples/src/test/java/org/apache/giraph/examples/TryMultiIpcBindingPortsTest.java
*
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java
*
giraph-core/src/test/java/org/apache/giraph/partition/TestGiraphTransferRegulator.java
* giraph-core/src/main/java/org/apache/giraph/partition/ByteArrayPartition.java
* giraph-core/src/main/java/org/apache/giraph/utils/ReflectionUtils.java
* giraph-core/src/test/java/org/apache/giraph/conf/TestObjectCreation.java
*
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdMessages.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleVertexWithWorkerContext.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/IntIntNullIntTextInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
* giraph-core/src/main/java/org/apache/giraph/graph/EdgeStore.java
*
giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeOutputFormat.java
* giraph-examples/src/main/java/org/apache/giraph/examples/SimpleFailVertex.java
* giraph-core/src/test/java/org/apache/giraph/master/TestMasterObserver.java
* giraph-core/src/main/java/org/apache/giraph/comm/SendPartitionCache.java
*
giraph-core/src/test/java/org/apache/giraph/edge/TestMultiRandomAccessEdges.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphRepresentativeVertexPageRankBenchmark.java
* giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleFloatDoubleEdgeListVertex.java
* giraph-core/src/main/java/org/apache/giraph/comm/SendEdgeCache.java
* giraph-core/src/test/java/org/apache/giraph/partition/TestPartitionStores.java
* giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/PartitionContextTestVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java
* giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java
* giraph-core/src/test/java/org/apache/giraph/edge/TestStrictGraphEdges.java
* giraph-core/src/test/java/org/apache/giraph/edge/TestNullValueEdges.java
* giraph-core/src/main/java/org/apache/giraph/graph/VertexMutations.java
* giraph-core/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java
* giraph-core/src/main/java/org/apache/giraph/io/ReverseEdgeDuplicator.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCombinerVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/ReuseObjectsVertexEdges.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/vertex/ByteArrayVertex.java
* giraph-core/src/main/java/org/apache/giraph/graph/MutableEdge.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomEdgeInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/edge/VertexEdges.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleFloatDoubleTextInputFormat.java
*
giraph-core/src/test/java/org/apache/giraph/io/TestTextDoubleDoubleAdjacencyListVertexInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataOutput.java
* giraph-examples/src/main/java/org/apache/giraph/examples/IdentityVertex.java
* giraph-core/src/main/java/org/apache/giraph/edge/LongNullHashSetEdges.java
* giraph-core/src/main/java/org/apache/giraph/graph/DefaultEdge.java
*
giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/JsonBase64VertexInputFormat.java
* giraph-core/pom.xml
* giraph-core/src/main/java/org/apache/giraph/utils/EdgeIterables.java
* giraph-core/src/test/java/org/apache/giraph/io/TestJsonBase64Format.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/graph/ReverseEdgeDuplicator.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/utils/WritableUtils.java
*
giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
* giraph-core/src/main/java/org/apache/giraph/vertex/IntNullNullNullVertex.java
* giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java
* giraph-core/src/main/java/org/apache/giraph/graph/EdgeNoValue.java
* giraph-core/src/main/java/org/apache/giraph/vertex/Vertex.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphByteArrayVertexPageRankBenchmark.java
* giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMsgVertex.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/LongDoubleDoubleAdjacencyListVertexInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/graph/Vertex.java
* CHANGELOG
*
giraph-core/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/HashMapVertexShortestPathsBenchmark.java
* giraph-examples/src/main/java/org/apache/giraph/examples/VerifyMessage.java
*
giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongArrayIterator.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdData.java
* giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkVertex.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java
* giraph-core/src/main/java/org/apache/giraph/vertex/IntIntNullIntVertex.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleNullDoubleVertex.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsComputation.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphRepresentativeVertex.java
*
giraph-examples/src/test/java/org/apache/giraph/examples/RandomWalkWithRestartVertexTest.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
*
giraph-core/src/test/java/org/apache/giraph/io/TestAdjacencyListTextVertexOutputFormat.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexReader.java
*
giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogEdgeInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/edge/LongDoubleArrayEdges.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphByteArrayVertex.java
*
giraph-core/src/main/java/org/apache/giraph/io/formats/IntNullReverseTextEdgeInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/edge/ConfigurableVertexEdges.java
*
giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTextVertexOutputFormat.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeReader.java
* giraph-core/src/main/java/org/apache/giraph/vertex/ByteArrayVertexBase.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
*
giraph-core/src/test/java/org/apache/giraph/io/TestIdWithValueTextOutputFormat.java
* giraph-core/src/main/java/org/apache/giraph/bsp/BspUtils.java
*
giraph-core/src/main/java/org/apache/giraph/vertex/RepresentativeVertexBase.java
*
giraph-core/src/main/java/org/apache/giraph/benchmark/HashMapVertexPageRankBenchmark.java
* giraph-core/src/main/java/org/apache/giraph/graph/EdgeFactory.java
> Decouple vertex implementation from edge storage
> ------------------------------------------------
>
> Key: GIRAPH-528
> URL: https://issues.apache.org/jira/browse/GIRAPH-528
> Project: Giraph
> Issue Type: Improvement
> Reporter: Alessandro Presta
> Assignee: Alessandro Presta
> Attachments: GIRAPH-528.patch, GIRAPH-528.patch, GIRAPH-528.patch,
> GIRAPH-528.patch, GIRAPH-528.patch, GIRAPH-528.patch
>
>
> This is meant to address the following issues:
> 1) The Vertex hierarchy is too complex and sometimes hard to work with
> (Vertex, SimpleVertex, MutableVertex, SimpleMutableVertex...).
> 2) Changing the underlying edge storage implementation for an existing
> algorithm requires editing your vertex to extend a different one.
> 3) In the general case (e.g. when not using ByteArrayVertex with the current
> EdgeStore), moving edges from the EdgeStore to the vertices is an additional
> step that can be avoided.
> My proposal is the following:
> - Make EdgeStore an interface. An implementation should deal with
> (concurrent) insertion of edges during input superstep; iteration over a
> vertex's edges during computation; insertion/deletion of edges during
> mutations (optional?); checkpointing.
> - The default EdgeStore will be the current byte-array implementation, which
> is generic (works with any choice of <I, V, E, M>) and reasonably optimized.
> - Only one Vertex class, which the user extends for the sole purpose of
> defining compute(). I don't necessarily agree that it should be an interface,
> because we still want to provide methods like getEdges(), sendMessage(), and
> those should delegate to the EdgeStore/MessageStore of choice.
> - Switching edge storage implementation is done by passing the EdgeStore
> class as an option. One can also define his own ad-hoc EdgeStore (e.g.,
> backed by primitive arrays).
> I think we should also extend this idea to MessageStore, making it possible
> to override that functionality too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
