-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13756/
-----------------------------------------------------------
Review request for giraph, Armando, Claudio Martella, and Nitay Joffe.
Bugs: GIRAPH-732
https://issues.apache.org/jira/browse/GIRAPH-732
Repository: giraph-git
Description
-------
This patch is a possible solution to provide an EdgeOutputFormat API in Giraph.
The idea is to add the same logic as the one for the vertices for edges. For
this reason I add saveEdges after saveVertices.
This requires the user to have to different paths for the resulting files. For
this reason I have implemented TextEdgeOutputFormat
and modified TextVertexOutputFormat so that a subdirectory can be specified.
The code I provide should be compliant with previous versions of hadoop as well
as YARN (I was yet not able to test it on YARN).
It is also retro-compatible since when not specified, TextVertexOutputFormat
behaves as before.
I am also providing an actual usable implementation with the associated tests
(SrcIdDstIdEdgeValueOutputFormat).
Also very interesting is the fact that this implementation is totally
transparent to the classes implementing TextVertexOutputFormat.
passed "maven verify"
Diffs
-----
giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1bd79b5
giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 71fe885
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
23bcd32
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java c276c2a
giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
49a2ebc
giraph-core/src/main/java/org/apache/giraph/io/EdgeOutputFormat.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/EdgeWriter.java PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphTextOutputFormat.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeOutputFormat.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java
c91d543
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeOutputFormat.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java
745764b
giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
da1e7fb
giraph-core/src/test/java/org/apache/giraph/io/TestSrcIdDstIdEdgeValueTextOutputFormat.java
PRE-CREATION
Diff: https://reviews.apache.org/r/13756/diff/
Testing
-------
Thanks,
Armando
