I'm not sure I understand the reason of two different data types for incoming and outgoing messages. can you relate on that?
On Wed, May 15, 2013 at 8:40 PM, Maja Kabiljo <[email protected]> wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11182/ > ----------------------------------------------------------- > > Review request for giraph. > > > Description > ------- > > Currently, our Vertex class holds a lot of stuff which shouldn't be there, > related to global graph and worker state. We should decouple vertex and > computation, vertex should be provided by the infrastructure and user > should implement the computation. > > In many real-world applications there are several different stages, where > different kind of computation is done and different type of messages are > sent. This can be done currently by having complicated compute() and > encoding the message type inside of the message. Much better would be to > provide a way to change which Computation is used by Giraph. Applications > can then be considered as pieces which can be put together in a pipeline > coordinated by master. > > Main changes: > - Added Computation<I, V, E, M1, M2> class, where M1 is incoming and M2 > outgoing message type. Computation has compute(vertex, messages) method > which should do what Vertex.compute(messages) was doing. Methods left in > Vertex are only the ones which handle the Vertex data, all other stuff are > moved to Computation. Also Vertex doesn't need GraphState anymore. > - MasterCompute has methods setComputation and setCombiner - if these are > not called everything works as before - Computation and Combiner from > previous superstep are used. > - BasicComputation<I, V, E, M> class is what current application should > extend - here incoming and outgoing message type is the same. Switching > existing application to these stuff is pretty straightforward: > * Instead of extending Vertex, one should extend BasicComputation > * In compute method, wherever some vertex data methods are called (i.e. > get/setValue, edges methods) we prefix that with vertex.* > - Cleaned up GraphState - that one was used all over the place and had > many fields which were used only by ComputeCallable > - No need for PartitionContext anymore - Computation can hold whichever > reusable objects we need and they can be accessed directly from compute() > - This touches a lot of classes because I removed M parameter from almost > everywhere, since that one is not constant anymore; had to rename all > *Vertex* classes to *Computation*. Classes which hold the main change are > BspService/Master/Worker, Vertex, Computation, BasicComputation, > MasterCompute, SuperstepClasses, GraphState, ComputeCallable, > ImmutableClassesGiraphConfiguration. To see how new stuff are used, take a > look at TestSwitchClasses. > > > This addresses bug GIRAPH-667. > https://issues.apache.org/jira/browse/GIRAPH-667 > > > Diffs > ----- > > > giraph-accumulo/src/main/java/org/apache/giraph/io/accumulo/AccumuloVertexInputFormat.java > 221fabd3016782db70c15bc6c56d41913164d664 > > giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/TestAccumuloVertexFormat.java > 6698c9a514a191b2aa900b4f1a833685af02391d > > giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeInputFormat.java > 5855cfcd4225db9289ebdfe3e85e52b627bd429b > > giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeOutputFormat.java > f999497a8782143afc0c42ac693470d197528075 > > giraph-core/src/main/java/org/apache/giraph/benchmark/AggregatorsBenchmark.java > 12304bb426dc8c11d2a3bda161b969c64d7e3f15 > > giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java > 0f8d284998974beba3884240a8ede71bb81c1098 > > giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankVertex.java > 9900a44bbe01eb9d5b1838fe4cd6c84db848e8b5 > > giraph-core/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java > 5c7e019dbed30a6e66734a301b3ab9017a2b7c65 > > giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java > 8e6c877c84d0a43a935ae6259fd3663d02ee8c56 > > giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsVertex.java > c1b77d1ff0a4803c3b8c93732f1395e30fe30fac > > giraph-core/src/main/java/org/apache/giraph/benchmark/WeightedPageRankBenchmark.java > 3fc514a2fb90048d1aca57198efed3c89d283bec > > giraph-core/src/main/java/org/apache/giraph/benchmark/WeightedPageRankVertex.java > 70f0f619ea1f3283cac6ce96d94d3bd508171179 > giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java > 178c96fd8106017751df1d3aa0295b238611ae54 > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedService.java > 2281903e943370c6b66508ca0cf8584b8bbc1aec > > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java > 5f84ece17082b1d0d5ede7aeabfe557945d4e8d0 > > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java > 1c7bde4c83cd7cb9507d426b5ee2ade34202088d > giraph-core/src/main/java/org/apache/giraph/comm/SendCache.java > 1e8bdf99200a5b48efa1d7b2b5e5ea5ca0526f83 > giraph-core/src/main/java/org/apache/giraph/comm/SendEdgeCache.java > fbc911f5665a7a6f304166a3f1acde00a7d71296 > giraph-core/src/main/java/org/apache/giraph/comm/SendMessageCache.java > 7d2a888254d9c52ca8b445923b7a1ff619c83f77 > giraph-core/src/main/java/org/apache/giraph/comm/SendMutationsCache.java > 67f74f1f29ef73e9c86298d645b82e8e9bcd45af > giraph-core/src/main/java/org/apache/giraph/comm/SendPartitionCache.java > 31cf05271dc6aa899bc8fff956555797d200e2ac > giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java > 743a6f83d6727c7365eeadb1b30c5e0fc41274c7 > giraph-core/src/main/java/org/apache/giraph/comm/WorkerClient.java > ebabf45482bd7c4f2fc47b2f2958a6245d886a82 > > giraph-core/src/main/java/org/apache/giraph/comm/WorkerClientRequestProcessor.java > bc0637faa52d6f79d6616da4a17a86fdd2166494 > giraph-core/src/main/java/org/apache/giraph/comm/WorkerServer.java > e373b2cb525046ddc61c8e74f21d7fc150134191 > > giraph-core/src/main/java/org/apache/giraph/comm/messages/ByteArrayMessagesPerVertexStore.java > 3cd1175054aa2401becc42f93f54b6d7e83ee372 > > giraph-core/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStore.java > 26abb94ebd62609fb98144f5b8453f023f34b58a > > giraph-core/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStoreByPartition.java > 02bfb1fbaa0448541683b6ca194cea6167f33301 > > giraph-core/src/main/java/org/apache/giraph/comm/messages/InMemoryMessageStoreFactory.java > PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/comm/messages/MessageStoreFactory.java > 3c13c4795fe45e354d49dbab620ed9e7a0f03920 > > giraph-core/src/main/java/org/apache/giraph/comm/messages/OneMessagePerVertexStore.java > 7db02667febf5db3080782f9e816cdff28e7382f > > giraph-core/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java > 3fe44308a0b0c1989bfeb1b8601a5f75e5336cb3 > > giraph-core/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java > fc6560dd454c5f8db5f32203edbfd723012750e3 > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java > 319f41a99b3dfd94be0f412e599e0c5cc0208d16 > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java > 1f04bcff4ccbce7cacb453689bc5361753f1d935 > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerAggregatorRequestProcessor.java > d1cce64c8dba8e17f50bc03dfde671e0e2ec41ac > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java > 9c09524577840f2080234664db909ca9f3ea516e > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java > db4ff5d0abbb15353107636d706127c561d8ef39 > > giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java > ed0861ef8eab0a70f04a561add12e10eea2fb34a > > giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/WorkerRequestServerHandler.java > b4e7ddae59d1c9d76462eca34205300486f43f67 > > giraph-core/src/main/java/org/apache/giraph/comm/requests/SendPartitionCurrentMessagesRequest.java > 037f4a055c8978b9708024beb24c1a09f91e07d5 > > giraph-core/src/main/java/org/apache/giraph/comm/requests/SendPartitionMutationsRequest.java > a96842d37093bf9a6d5a7474d158f3093b95c23e > > giraph-core/src/main/java/org/apache/giraph/comm/requests/SendVertexRequest.java > 1de3cbb372a659c64826b4d948f06ec90d2ccf82 > > giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerMessagesRequest.java > 04b633b47b63e0d4b62891aa8b5ebedec67ea892 > > giraph-core/src/main/java/org/apache/giraph/comm/requests/WorkerRequest.java > 4d9382f0b6b7279e31ba84dfc4c24f4a0e144d24 > > giraph-core/src/main/java/org/apache/giraph/comm/requests/WritableRequest.java > fad20b0714b55bf5b0a00903fa2a0686eb673c3b > giraph-core/src/main/java/org/apache/giraph/conf/AllOptions.java > cceaaef8a141e4867abd9c69ab6fb74959abedc2 > > giraph-core/src/main/java/org/apache/giraph/conf/DefaultImmutableClassesGiraphConfigurable.java > 77564ee8d954266f4bf5f5f798ad4f712b43c725 > giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java > 10e49756718c217fdedf223ae853876c39c64f4d > > giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java > 754fad9207263d62f1dc378e9b9de48d7aa0c0ad > giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java > bbf50e57ec76eba0a8553f1a7890d161701d358a > > giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfigurable.java > e4351a2c555e11793b0eea4ded6a0e8b82237cb7 > > giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java > a9add4f44a79e8bbba764be748d26d9311e1dd00 > > giraph-core/src/main/java/org/apache/giraph/edge/ConfigurableOutEdges.java > 631b209d2d02f5eb54be2a1fe74ab26dff7715f4 > giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java > 9e2d246fe5afa0010c94e9f79bf3515fc4bd2d63 > > giraph-core/src/main/java/org/apache/giraph/edge/MutableEdgesIterable.java > 82486f49166dc568e1dd7a665d48310326b49951 > > giraph-core/src/main/java/org/apache/giraph/edge/MutableEdgesWrapper.java > cd845d0cbd5f2296a12fa4a25969469c0759b702 > giraph-core/src/main/java/org/apache/giraph/graph/Computation.java > PRE-CREATION > giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java > 0fc5fdfda88b56e617c4eb8317bb5f76dc0ec41c > > giraph-core/src/main/java/org/apache/giraph/graph/DefaultVertexResolver.java > 52df38dd3fca3deb0b21175d056cd677d5eb78e8 > > giraph-core/src/main/java/org/apache/giraph/graph/DefaultVertexValueFactory.java > 47902d13a4bcfd26603fa27e1fb41d2ef9abf800 > > giraph-core/src/main/java/org/apache/giraph/graph/GiraphTransferRegulator.java > 3c2286d958b307456f31d86ec86a52b8a434f44d > giraph-core/src/main/java/org/apache/giraph/graph/GraphMapper.java > 4181d24547f0fd54dc001b8368b223746c54a383 > giraph-core/src/main/java/org/apache/giraph/graph/GraphState.java > 93ad5dfc435f561dc2f18d4305d28244a4021f45 > giraph-core/src/main/java/org/apache/giraph/graph/GraphStateAware.java > 76cef433283f543f30f2cb9a8036d32e4b45862a > giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java > 82e1b1e745491f1bc6b6fa8ea2509a0de794ac28 > giraph-core/src/main/java/org/apache/giraph/graph/Vertex.java > 61624e5846f382dc046ae7acad00637e4ef47278 > giraph-core/src/main/java/org/apache/giraph/graph/VertexChanges.java > 947463699a6b355a20dd278ac7ff2019fb159337 > giraph-core/src/main/java/org/apache/giraph/graph/VertexMutations.java > 75c0aef06212303a1669bd9c30e8e14e54da8870 > giraph-core/src/main/java/org/apache/giraph/graph/VertexResolver.java > 1fc0ddcc23b6487d892110622d5272ccebdbb099 > > giraph-core/src/main/java/org/apache/giraph/graph/VertexValueFactory.java > e62bb01139dc3f3da46282f8881227bd05541e68 > > giraph-core/src/main/java/org/apache/giraph/integration/SuperstepHashPartitionerFactory.java > c03d718312bf2d0b61e01bca94449adbcd53640e > giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java > 6d87bf2f2748b37717878d8d85e68bd30e33d650 > giraph-core/src/main/java/org/apache/giraph/io/GiraphInputFormat.java > 58d79a63d403a2cc7c274d42abf5814d46f7f3c3 > giraph-core/src/main/java/org/apache/giraph/io/SimpleVertexWriter.java > e4c34962dceb996e59580b62b7cc26f67a2b1282 > giraph-core/src/main/java/org/apache/giraph/io/VertexOutputFormat.java > 154f7e41de45a1f63d3934e79e714e82b2b78e20 > giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java > 3f6bb3fdc866e8ef9a181ff2f30fee54272459d5 > giraph-core/src/main/java/org/apache/giraph/io/VertexValueReader.java > 0b06a4a05de5658be3b14c924c0cb4f9665f34b0 > giraph-core/src/main/java/org/apache/giraph/io/VertexWriter.java > a4285c1176dd50e993afa23256da963c7c546340 > > giraph-core/src/main/java/org/apache/giraph/io/filters/DefaultEdgeInputFilter.java > ad52496f9b8f2a919a6f165eb48b325b95515bf5 > > giraph-core/src/main/java/org/apache/giraph/io/filters/DefaultVertexInputFilter.java > 2976cbc79510b8ac663dab33ca2398dbf6cee152 > > giraph-core/src/main/java/org/apache/giraph/io/filters/VertexInputFilter.java > d9af103aa9dded4ec80d58652a9b4882cdbf9d98 > > giraph-core/src/main/java/org/apache/giraph/io/formats/AdjacencyListTextVertexOutputFormat.java > f71ef25acbae07a43653a68473ff0f54072ea82e > > giraph-core/src/main/java/org/apache/giraph/io/formats/IdWithValueTextOutputFormat.java > 6dd74688d317edeedc91fd3c150996e308b1f785 > > giraph-core/src/main/java/org/apache/giraph/io/formats/IntIntTextVertexValueInputFormat.java > a7dbef8fb0fca599c04e344a144f429bac18145b > > giraph-core/src/main/java/org/apache/giraph/io/formats/JsonBase64VertexOutputFormat.java > 7d8fcf65267bc126ccfcf223a26f517c94c2a85c > > giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexInputFormat.java > 7dfd6078f8b9408674ba03690f4597a3f975ed79 > > giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexOutputFormat.java > d0a330538807227a95da2ff853f07b36cae86a04 > > giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomIntNullVertexInputFormat.java > c9390bacd2c40c2945d157f42dae2e032d4a305f > > giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomVertexInputFormat.java > 5d293ebacb58b0bbfd7856a40e3ae66d0151b9b1 > > giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexInputFormat.java > 0f2d929a6d5fa31156ead71b5b09a806ec73c043 > > giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexOutputFormat.java > c4ed65c2eb866cf2ad49881773fcdf022ac00bde > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextDoubleDoubleAdjacencyListVertexInputFormat.java > 6e62b714d9f3f61fa67b91539c75222e8bfa2e03 > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexInputFormat.java > b08e6f767a2f342b8a3b22d9d687de513acc8509 > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java > a3073f9bd62baad934b64d9dfe189f5b32e7cb95 > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexValueInputFormat.java > 6d133ae5c10b0e0dbfe26e4f126749cd35c1fdd8 > > giraph-core/src/main/java/org/apache/giraph/io/formats/multi/EdgeInputFormatDescription.java > 569cee93d9819bbc08975a2c87f095b47d85004e > > giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiEdgeInputFormat.java > c377fbcd59bf076f68074e2fa2849d01f9f8a463 > > giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java > 72929d961252d53e979df1fe3f8449f9ca0fde55 > > giraph-core/src/main/java/org/apache/giraph/io/formats/multi/VertexInputFormatDescription.java > bdd5a74001b04a92de0da68e163d0e1799958400 > > giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java > c0a2cd1dbbc8127d226ba40322a3246f9623c0cc > > giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexOutputFormat.java > 8110209eea16a5b0fc4fe45b82c2737849c367a9 > > giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java > 3a8ac506ea4a8ef290528cad3db4af0b462a583e > > giraph-core/src/main/java/org/apache/giraph/io/iterables/EdgeReaderWrapper.java > efd8fe742318fdad86d0b358c7b465316d30e8cb > > giraph-core/src/main/java/org/apache/giraph/io/iterables/VertexReaderWrapper.java > 614f9453ab434fefef3ab4d5a8f1daee4e8edcd5 > > giraph-core/src/main/java/org/apache/giraph/io/superstep_output/MultiThreadedSuperstepOutput.java > a8dff876ba45689705cacea2ebd47a9b07eb1aaf > > giraph-core/src/main/java/org/apache/giraph/io/superstep_output/NoOpSuperstepOutput.java > 82684b2265857da1b68296d56a6d447cb201a990 > > giraph-core/src/main/java/org/apache/giraph/io/superstep_output/SynchronizedSuperstepOutput.java > f94bd56d0b877fa778732b502431e753c09b8733 > > giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java > cc6b126ca3e2fa30f8e66fff4648da30d5c465a8 > giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java > f00116a7cea96be8c18c327eda767a2a2698d9dc > > giraph-core/src/main/java/org/apache/giraph/master/MasterAggregatorHandler.java > f769c3a24962e0ac9e275db838d4ed9cad2e9007 > giraph-core/src/main/java/org/apache/giraph/master/MasterCompute.java > 1c36e03def9262d712ea04a4609801d74f4b6c27 > giraph-core/src/main/java/org/apache/giraph/master/MasterThread.java > ba2f8eb7a093e616f15fc0875fa0b03999f1861b > giraph-core/src/main/java/org/apache/giraph/master/SuperstepClasses.java > PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/partition/BasicPartition.java > dc9192ed85c10c083153f0ecf0ae9bd005e1b84b > > giraph-core/src/main/java/org/apache/giraph/partition/ByteArrayPartition.java > c4669d38868af4c244ee750536e2080748357eba > > giraph-core/src/main/java/org/apache/giraph/partition/DefaultPartitionContext.java > c22c802e26807fa27425a93e6033e92a02d23e59 > > giraph-core/src/main/java/org/apache/giraph/partition/DiskBackedPartitionStore.java > a4739f1d147ff689afbe5a1e8d7b94897c9e3d88 > > giraph-core/src/main/java/org/apache/giraph/partition/GraphPartitionerFactory.java > e2e04ddf44145009717e843a29f6d4a12eed01db > > giraph-core/src/main/java/org/apache/giraph/partition/HashMasterPartitioner.java > 5faf3675c17d40b1e015110a886ef6a37cba0031 > > giraph-core/src/main/java/org/apache/giraph/partition/HashPartitionerFactory.java > f7343a11f5632fcbd742c43e70d2dde29b2363f3 > > giraph-core/src/main/java/org/apache/giraph/partition/HashRangePartitionerFactory.java > 227e23483920a62df8592f384301c8ac7ae9aadc > > giraph-core/src/main/java/org/apache/giraph/partition/HashRangeWorkerPartitioner.java > a6e764d86c5a9f4a96e1ec09f867b94e84544b62 > > giraph-core/src/main/java/org/apache/giraph/partition/HashWorkerPartitioner.java > 599ea0c88ca1a3eb5d2ef099d58f62ad0b0fd9fb > > giraph-core/src/main/java/org/apache/giraph/partition/MasterGraphPartitioner.java > 130ee07b2ffcbe6181528447ee4fba8283ac33ec > giraph-core/src/main/java/org/apache/giraph/partition/Partition.java > 1ca0b61cec3f6938211864da8c0503cc184b86e3 > > giraph-core/src/main/java/org/apache/giraph/partition/PartitionContext.java > 412f6e3aa1a3a533425313470e95a008f1178a95 > > giraph-core/src/main/java/org/apache/giraph/partition/PartitionStore.java > 4206ce343ec36dce39fa9a558f392d063b0c8e5f > > giraph-core/src/main/java/org/apache/giraph/partition/RangeMasterPartitioner.java > f9b0329983abe715fd7d1fd33c7be8f54cbc2bc9 > > giraph-core/src/main/java/org/apache/giraph/partition/RangePartitionerFactory.java > 29f7898120eafac08ee17327da2072e3ad5a5670 > > giraph-core/src/main/java/org/apache/giraph/partition/RangeSplitHint.java > 9634c3386669441d43121e1872c5a90bf47ffcb2 > > giraph-core/src/main/java/org/apache/giraph/partition/RangeWorkerPartitioner.java > 5a494a53f0549277a5ed86d033435d0d8cf3d094 > > giraph-core/src/main/java/org/apache/giraph/partition/ReusesObjectsPartition.java > 4153d554881f1586545242ba7c00f951630077ca > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleIntRangePartitionerFactory.java > 9ac2e11e780499dac6ea9235ca73cacaaf7fa4fd > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleLongRangePartitionerFactory.java > 5772a7b5cd3cf6fa49c09934900630fab3a8094c > > giraph-core/src/main/java/org/apache/giraph/partition/SimplePartition.java > d6a46bd3c178506865b85b8b596d95dde59064b2 > > giraph-core/src/main/java/org/apache/giraph/partition/SimplePartitionStore.java > 74cc3a7d8e719fb1c728e77eaf345e75ca973508 > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleRangeMasterPartitioner.java > bf34ecde3dde7fd4d868d396f97e473cbcda76ba > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleRangeWorkerPartitioner.java > f94c14bedc20605ca2bef29d01440423d8ba2a02 > > giraph-core/src/main/java/org/apache/giraph/partition/WorkerGraphPartitioner.java > 5a78b1d54109d80d94911971c522c543cecc8e33 > > giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdData.java > 2cfa661ee7204e73e6aa3705f742e9f58acfc4e9 > > giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdEdges.java > 2c5f2f7f69b06001f004992c396bf49e6fd6785b > > giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdMessages.java > 0280c58b4e11ed356e77761771db930303f53a80 > > giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java > 6016ba40241e5b339b44a594e51d6278eaee1967 > > giraph-core/src/main/java/org/apache/giraph/utils/InMemoryVertexInputFormat.java > 65d99db8edea57c5c2d75c8c346cead581d5d783 > > giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java > be2d2a9c127250fb302da9eedc9af82613d049e2 > giraph-core/src/main/java/org/apache/giraph/utils/ReflectionUtils.java > d70eecbd2429d5824b57e057a390a575e8ea83f2 > giraph-core/src/main/java/org/apache/giraph/utils/TestGraph.java > 3577a9e4482054e91fb5094ac93207ff7fc44ea5 > giraph-core/src/main/java/org/apache/giraph/utils/VertexIdIterator.java > 0c9ee076aba989e32288bc9f9f6d182625a3a5cf > giraph-core/src/main/java/org/apache/giraph/utils/WritableUtils.java > c607ca326046e693b8a21b50a968ef6404b56a6d > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java > 03a4876c1cec2d8598a3bb086ca92a91050bb159 > > giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java > 351a114390a90528a6f4c1b46dcdb77c50e845aa > > giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallableFactory.java > 33fb5159cc87e0653c0e8933d706159aa97954a4 > > giraph-core/src/main/java/org/apache/giraph/worker/InputSplitsCallable.java > a8298c5d9e2c0631ef08dd044eaf0a69ae089e0a > > giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java > 1c292ad1db3d2b0149c3f568c12a7279d831b1ca > > giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallableFactory.java > cf5e8ad85a6663baec90e0d81c6fcffbcdf9e4ef > > giraph-core/src/main/java/org/apache/giraph/worker/WorkerAggregatorHandler.java > 9a8a8b8bc4de65708a6416e7643ef4b22897fad3 > giraph-core/src/main/java/org/apache/giraph/worker/WorkerContext.java > d3ffaeadf5b32f039dc8abe620b1fcf4fcac08e0 > giraph-core/src/test/java/org/apache/giraph/comm/ConnectionTest.java > 91b842afbaac2962bbf8f46ec1cabbcfc5b94146 > giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java > 58aa7d1c63523522f27e375a7e65baf770e5fe05 > giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java > f1f8e26e2fe99edea1de6a173bf9aab9ad4d47ca > giraph-core/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java > c27156fb98f86afec352ed1cf70b95e307805d34 > giraph-core/src/test/java/org/apache/giraph/comm/TestMessageStores.java > 53b8a2488171bb3c116b9296d5b7467bf890dc8e > giraph-core/src/test/java/org/apache/giraph/conf/TestObjectCreation.java > 9075145eaa7e6744ec656890c8c95ca387ab22e0 > > giraph-core/src/test/java/org/apache/giraph/graph/TestVertexAndEdges.java > 15d2bb0aad6f17bca90631dc7b95ea5535dc95f0 > > giraph-core/src/test/java/org/apache/giraph/io/TestAdjacencyListTextVertexOutputFormat.java > 6849e3a938bbe20d7e29ad432baf55504d4cf80a > giraph-core/src/test/java/org/apache/giraph/io/TestEdgeInput.java > cb1a8daa94ef235cb22d4d2d2ddaed7d74e79fa2 > giraph-core/src/test/java/org/apache/giraph/io/TestFilters.java > 83a366da7f3f7b3c80f055d077394f444bff218d > > giraph-core/src/test/java/org/apache/giraph/io/TestIdWithValueTextOutputFormat.java > ff4d2b1f572049c747becea19fc61ca0e2d4e493 > giraph-core/src/test/java/org/apache/giraph/io/TestJsonBase64Format.java > a6aa78a0399dc6c839b5f1334e8e812b73808d7a > > giraph-core/src/test/java/org/apache/giraph/io/TestLongDoubleDoubleAdjacencyListVertexInputFormat.java > 601c824684d7298eedad9093dc83762362482e6c > > giraph-core/src/test/java/org/apache/giraph/io/TestTextDoubleDoubleAdjacencyListVertexInputFormat.java > 4ee8ea19b30d55cd3c81bb5281f27a15d45a6412 > > giraph-core/src/test/java/org/apache/giraph/master/TestComputationCombinerTypes.java > PRE-CREATION > > giraph-core/src/test/java/org/apache/giraph/master/TestMasterObserver.java > 9d6b21536ee85ea6835e55e1e5bb477219105e65 > > giraph-core/src/test/java/org/apache/giraph/master/TestSwitchClasses.java > PRE-CREATION > > giraph-core/src/test/java/org/apache/giraph/partition/TestGiraphTransferRegulator.java > 9b655af1f5c252d285862c75cf44be4ae3c29d05 > > giraph-core/src/test/java/org/apache/giraph/partition/TestPartitionStores.java > 5a93d41e30bb90e3a7f2f2dbce7c44f9d5c8d88c > > giraph-core/src/test/java/org/apache/giraph/utils/ComputationCountEdges.java > PRE-CREATION > giraph-core/src/test/java/org/apache/giraph/utils/MockUtils.java > 0039ad62502a740db156c37bcc4445aa5801b6b2 > > giraph-core/src/test/java/org/apache/giraph/vertices/IntIntNullVertexDoNothing.java > c98d5804fa3921b23f4fc54a6554b076c8a1a44c > > giraph-core/src/test/java/org/apache/giraph/vertices/VertexCountEdges.java > 9060bc7077ac23a9b362d10698c5facae5dcf55a > > giraph-core/src/test/java/org/apache/giraph/vertices/VertexDoNothing.java > fac3fce3eec0280ac10f65b3f4ebfb166fa84d1a > giraph-core/src/test/java/org/apache/giraph/yarn/TestYarnJob.java > f9d5544edaee5471a4087c9ec81b4a08ddea2a20 > > giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestVertex.java > d08519ba3dc6e5517e99dea640f0f9d16c6594f4 > > giraph-examples/src/main/java/org/apache/giraph/examples/ConnectedComponentsVertex.java > dbeb6bf98721cf11744f45682681599f18fe0487 > > giraph-examples/src/main/java/org/apache/giraph/examples/IdentityVertex.java > 30cca86547de3806c271099eea87e95429b8f548 > > giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleDoubleTextInputFormat.java > 62bea5a7f186d9e731d4fa25f11f238d47d33407 > > giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleNullTextInputFormat.java > fdc90505b52a43e70ebc0802ca6af46d6c2c128d > > giraph-examples/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleDoubleTextInputFormat.java > 7dc8475c095207cc7fed199cc05ed80205085549 > > giraph-examples/src/main/java/org/apache/giraph/examples/PageRankVertex.java > 9678b31a2747d899888d99933716aa5fe3d40757 > > giraph-examples/src/main/java/org/apache/giraph/examples/PartitionContextTestVertex.java > f617d8ea577c1083bf6db72c047dec336f801667 > > giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkVertex.java > 2d2c988b7163b7d3a4584df91e211c1d3f73b873 > > giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkVertexMasterCompute.java > 9e5dbbf9416f771ee957b0d3765c002dced0b2a9 > > giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkWithRestartVertex.java > 6f3eb6cdc43b14c2d3776704c9a84178ba12c450 > > giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkWorkerContext.java > 2566f43b7fbb91933034df7b3645435807c20fcd > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java > 03d977b1bed88a8dc61498ad44ea7fb895eefe1b > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCombinerVertex.java > f4475aeef0784bad456cb91fef22d651d7bc4d21 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleFailVertex.java > c0e206c304e0df2608893beee70fb3d52980f024 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleInDegreeCountVertex.java > 3bcf3f5324aec428c00454e4d578333e05e6ced8 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleLongDoubleDoubleDoubleIdentityVertex.java > c7349d15c847b4eedca5a14d073487b97d2cf6eb > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMasterComputeVertex.java > 8a21ad788ad5b3bf4c24ee63a7d99d9d4448fe30 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMsgVertex.java > 024fe9d0841a1b6c99ded1d2d2a18e27c6d77858 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java > a468491c6a8acd3256fb9bfe14edc559fe743f4b > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleOutDegreeCountVertex.java > c830fa2a6060995776e0227a53bbb969779b9b0f > > giraph-examples/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java > 7a63e8d7052437a9e1d0c859805d5c58b60f289d > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleShortestPathsVertex.java > 13d1d7c1e37571fe0207cc93659e26bb4ddad580 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java > 6f8b352d2e2500390c21d775b662610ce669badb > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTextVertexOutputFormat.java > 157e6ef6c4e005b20d06b510190ea3070c6d4335 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTriangleClosingVertex.java > f44cb188117b21e068aedcc6d03b1ef1210b8900 > > giraph-examples/src/main/java/org/apache/giraph/examples/SimpleVertexWithWorkerContext.java > 8a6f77562b9ffa3c7fa5ae87a9afbf000ef80305 > > giraph-examples/src/main/java/org/apache/giraph/examples/VerifyMessage.java > 6e3c589be8d55dbc6dbe4a51c92a77ad282fe09f > > giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueDoubleEdgeTextOutputFormat.java > d3281535b2107ac2eeacd20318459544974b69cd > > giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueNullEdgeTextOutputFormat.java > 85f3556b247a6aa51a5a392e70986377b1fc6e44 > giraph-examples/src/test/java/org/apache/giraph/TestAutoCheckpoint.java > 652913b4965a3ccc8f5294bc4976d8f369489807 > giraph-examples/src/test/java/org/apache/giraph/TestBspBasic.java > 38ba27fae8c9725545c63ea97b8b7e080cfdb0d3 > > giraph-examples/src/test/java/org/apache/giraph/TestGraphPartitioner.java > 12f0d8d65a18b7cf167b59635561f11fcf397a7c > > giraph-examples/src/test/java/org/apache/giraph/TestManualCheckpoint.java > 766e1af3fb838991f757a6d5cc7cf797407d4d64 > giraph-examples/src/test/java/org/apache/giraph/TestMaxSuperstep.java > 0989ac5e9fbfe5c1e23484015cbac29478d86bb9 > giraph-examples/src/test/java/org/apache/giraph/TestMutateGraph.java > da85fc4fa8e8d309cf5e332975d0a696916ea43e > > giraph-examples/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java > 6da9c500717b00ed58da21e06a96e06eb873422d > > giraph-examples/src/test/java/org/apache/giraph/TestPartitionContext.java > 4b042dfda701e61df0e79cb117475a090f876ab8 > > giraph-examples/src/test/java/org/apache/giraph/aggregators/TestAggregatorsHandling.java > 53cdeab311b1a5ec2b5ab92fa55c1dca790d8dd5 > > giraph-examples/src/test/java/org/apache/giraph/examples/ConnectedComponentsVertexTest.java > 5d712594ef83d7e0a7501f96b0008cf8a44bae50 > > giraph-examples/src/test/java/org/apache/giraph/examples/ConnectedComponentsVertexTestInMemory.java > e4c029c4ac5bb1fc3b59f5a2af98c353ce330d32 > > giraph-examples/src/test/java/org/apache/giraph/examples/PageRankVertexTest.java > 2a392915093cc64ccb5f9d4c037a509bbfe9c0f1 > > giraph-examples/src/test/java/org/apache/giraph/examples/RandomWalkWithRestartVertexTest.java > a2a891e28575c858c3208187eef7e4f9c8bf3431 > > giraph-examples/src/test/java/org/apache/giraph/examples/SimpleShortestPathsVertexTest.java > 10065777cf51ac4b02f9ebbbe1f3a77b31ae9eb2 > > giraph-examples/src/test/java/org/apache/giraph/examples/SimpleTriangleClosingVertexTest.java > bfe8002cde95c823133936438442db218754648c > > giraph-examples/src/test/java/org/apache/giraph/examples/TestPageRank.java > 922e7363caa8b8a90033c9f300a20edc9f093283 > > giraph-examples/src/test/java/org/apache/giraph/examples/TryMultiIpcBindingPortsTest.java > 23d055e2e9f8e4b9af26354e3f060b97748112a0 > > giraph-examples/src/test/java/org/apache/giraph/vertex/TestVertexTypes.java > 0898d635e5a98f7e5ee871e41df2b751c907e1d4 > > giraph-hbase/src/test/java/org/apache/giraph/io/hbase/TestHBaseRootMarkerVertextFormat.java > 5091b48ae4f6c01396747ccc67a6c03a0127dce1 > > giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeInputFormat.java > 5ecb7dde8e8a3a02b3abacf4a002564b7e30e87e > > giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeOutputFormat.java > 859f519f9f4dfdf7d0e590a77e018392ae21d673 > > giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatGiraphRunner.java > 2182a7133a7e011453f2f9a38d0e342a42870ad9 > > giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java > 08b263fddc2ad16246bd51ef684481fd3567d393 > > giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexOutputFormat.java > 37b3b74aecd6b5f7d71e1c5afb8219fe36141261 > giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java > da9ee2fc048c791388e3f5681f757d50f52674f7 > > giraph-hive/src/main/java/org/apache/giraph/hive/common/DefaultConfigurableAndTableSchemaAware.java > 4ba69c9ed474243bedf9cf9fc751d866e3e8ae55 > > giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/AbstractHiveToEdge.java > 73fc553f1c92ed11eede41d5dc1af636202f5f0b > > giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeInputFormat.java > f35b6ea021e9547319561b23fd1bc5fa8dcc734d > > giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeReader.java > ba992672f20007e4abc949dcc43deb2ce0e219e7 > > giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/AbstractHiveToVertex.java > b08a09fdcfba9bd6a648f4cd4df7b95093c1615a > > giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertex.java > c670876980396093bd46fb20c5c6c94b42c2253b > > giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexInputFormat.java > 824d8a6a1ac52915cf44fc6c6ab9e81de793ef39 > > giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexReader.java > 719d8ad1c200cf74d8bfc8921f1549157c26dbc5 > > giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/SimpleHiveToVertex.java > 6df269d451f3470dc6d46d29a2e60dac452a214f > > giraph-hive/src/main/java/org/apache/giraph/hive/output/AbstractVertexToHive.java > 52fe93f1375dd7c7278e7f7c93ccf9c950a7ff9d > > giraph-hive/src/main/java/org/apache/giraph/hive/output/HiveVertexOutputFormat.java > 05441f4c0aec209b60cd970e94ce5f6116feff0b > > giraph-hive/src/main/java/org/apache/giraph/hive/output/HiveVertexWriter.java > 357cf89c95096537f2ee7a99fdce389dbeafb79a > > giraph-hive/src/main/java/org/apache/giraph/hive/output/SimpleVertexToHive.java > fdfb3630a293a73036da396c1769e53f11601fe8 > > giraph-hive/src/main/java/org/apache/giraph/hive/output/VertexToHive.java > 85e438c8d111953c9960f39d7e16c1e92259d7ab > > giraph-hive/src/main/java/org/apache/giraph/hive/output/examples/HiveOutputIntIntVertex.java > 7955915c6b560fbb1c9bdca7e91bc3d8ccba8ab0 > > giraph-hive/src/test/java/org/apache/giraph/hive/input/HiveEdgeInputTest.java > 2ad87fa469933337399a2d858ff659822c754f62 > > giraph-hive/src/test/java/org/apache/giraph/hive/input/HiveVertexInputTest.java > b6ab139054d5f597809f374454f7443b42f1e8dd > > giraph-hive/src/test/java/org/apache/giraph/hive/output/HiveOutputTest.java > 673c9d503075e99956d8db5696bcd245bbb22ded > > giraph-hive/src/test/java/org/apache/giraph/hive/vertexes/VertexCountEdges.java > a6db2afad47262216758894cd1f1da1b538f4376 > > giraph-hive/src/test/java/org/apache/giraph/hive/vertexes/VertexSumEdges.java > d8527b4f693c05fc2629fd4f58354bec50c71bc0 > > giraph-hive/src/test/java/org/apache/giraph/hive/vertexes/package-info.java > 552bb91ac4ec2d0f5719970023d2b83ba3be7987 > > Diff: https://reviews.apache.org/r/11182/diff/ > > > Testing > ------- > > mvn clean install > Added some tests for new features > Real application run on a cluster > > > Thanks, > > Maja Kabiljo > > -- Claudio Martella [email protected]
