-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10564/
-----------------------------------------------------------
(Updated April 25, 2013, 12:27 a.m.)
Review request for giraph.
Changes
-------
I revisited my approach - instead of each input format having to know its index
and having configuration parameters set based on that index, I am putting all
input format specific settings inside of one Configuration parameter, in JSON
format, like
giraph.edgeInputFormatDescriptions=[["EIF1",{"p":"v1"}],["EIF2",{"p":"v2","q":"v"}]]
This would mean that we are using two edge input formats - EIF1 and EIF2. What
happens in the infrastructure is that when EIF1 is created we also create a
copy of Configuration in which we set p=v1,
and for EIF2 p=v2 and q=v. This means that input formats can just call
conf.get("p") and they will be getting the parameters they need - they don't
need to know anything about existence of other input formats or their index.
Again I did only edge input part, and added support only in HiveGiraphRunner,
hoping for constructive feedback before proceeding :-)
Description
-------
For now, I did this only for Edge input, once I get some feedback I'll do the
exactly same thing for vertex input.
Also, I added direct support only to HiveGiraphRunner, we can extend it later
to others as well.
This addresses bug GIRAPH-639.
https://issues.apache.org/jira/browse/GIRAPH-639
Diffs (updated)
-----
giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java 178c96f
giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 4a0e8f7
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
795cd0c
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 6a5949e
giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
f5a926f
giraph-core/src/main/java/org/apache/giraph/io/internal/EdgeInputFormatDescription.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/internal/InputFormatDescription.java
PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java
cc6b126
giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
2c1a679
giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java
029cb5d
giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
01937ab
giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java
de1e774
giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallableFactory.java
4a1705b
giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java
8d67c1d
Diff: https://reviews.apache.org/r/10564/diff/
Testing
-------
mvn clean verify
Run application with two edge input tables - verified results.
Thanks,
Maja Kabiljo
