[
https://issues.apache.org/jira/browse/GIRAPH-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Presta updated GIRAPH-547:
-------------------------------------
Description:
This is a somewhat long term item.
Because of some optimized edge storage implementations (byte array, primitive
array), we have a contract with the user that Edge objects returned by
getEdges() are read-only.
One concrete example where in-place modification would be useful: in the
weighted version of PageRank, you can store the weight sum and normalize each
message sent, or you could more efficiently normalize the out-edges once in
superstep 0.
The Pregel paper describes an OutEdgeIterator that allows for in-place
modification of edges. I can see how that would be easy to implement in C++,
where there is no need to reuse objects.
Giraph "unofficially" supports this if one is using generic collections to
represent edges (e.g. ArrayList or HashMap).
It may be trickier in some optimized implementations, but in principle it
should be doable.
One way would be to have some special MutableEdge implementation which calls
back to the edge data structure in order to save modifications:
{code}
for (Edge<I, E> edge : getEdges()) {
edge.setValue(newValue);
}
{code}
Another option would be to add a special set() method to our edge iterator,
where one can replace the current edge:
{code}
for (EdgeIterator<I, E> it = getEdges().iterator(); it.hasNext();) {
Edge<I, E> edge = it.next();
edge.setValue(newValue);
it.set(edge);
}
{code}
We could actually implement the first version as syntactic sugar on top of the
second version (the special MutableEdge would need a reference to the iterator
in order to call set(this)).
was:
This is a somewhat long term item.
Because of some optimized edge storage implementations (byte array, primitive
array), we have a contract with the user that Edge objects returned by
getEdges() are read-only.
One concrete example where this is needed: in the weighted version of PageRank,
you can store the weight sum and normalize each message sent, or you could more
efficiently normalize the out-edges once in superstep 0.
The Pregel paper describes an OutEdgeIterator that allows for in-place
modification of edges. I can see how that would be easy to implement in C++,
where there is no need to reuse objects.
Giraph "unofficially" supports this if one is using generic collections to
represent edges (e.g. ArrayList or HashMap).
It may be trickier in some optimized implementations, but in principle it
should be doable.
One way would be to have some special MutableEdge implementation which calls
back to the edge data structure in order to save modifications:
{code}
for (Edge<I, E> edge : getEdges()) {
edge.setValue(newValue);
}
{code}
Another option would be to add a special set() method to our edge iterator,
where one can replace the current edge:
{code}
for (EdgeIterator<I, E> it = getEdges().iterator(); it.hasNext();) {
Edge<I, E> edge = it.next();
edge.setValue(newValue);
it.set(edge);
}
{code}
We could actually implement the first version as syntactic sugar on top of the
second version (the special MutableEdge would need a reference to the iterator
in order to call set(this)).
> Allow in-place modification of edges
> ------------------------------------
>
> Key: GIRAPH-547
> URL: https://issues.apache.org/jira/browse/GIRAPH-547
> Project: Giraph
> Issue Type: New Feature
> Reporter: Alessandro Presta
> Assignee: Alessandro Presta
>
> This is a somewhat long term item.
> Because of some optimized edge storage implementations (byte array, primitive
> array), we have a contract with the user that Edge objects returned by
> getEdges() are read-only.
> One concrete example where in-place modification would be useful: in the
> weighted version of PageRank, you can store the weight sum and normalize each
> message sent, or you could more efficiently normalize the out-edges once in
> superstep 0.
> The Pregel paper describes an OutEdgeIterator that allows for in-place
> modification of edges. I can see how that would be easy to implement in C++,
> where there is no need to reuse objects.
> Giraph "unofficially" supports this if one is using generic collections to
> represent edges (e.g. ArrayList or HashMap).
> It may be trickier in some optimized implementations, but in principle it
> should be doable.
> One way would be to have some special MutableEdge implementation which calls
> back to the edge data structure in order to save modifications:
> {code}
> for (Edge<I, E> edge : getEdges()) {
> edge.setValue(newValue);
> }
> {code}
> Another option would be to add a special set() method to our edge iterator,
> where one can replace the current edge:
> {code}
> for (EdgeIterator<I, E> it = getEdges().iterator(); it.hasNext();) {
> Edge<I, E> edge = it.next();
> edge.setValue(newValue);
> it.set(edge);
> }
> {code}
> We could actually implement the first version as syntactic sugar on top of
> the second version (the special MutableEdge would need a reference to the
> iterator in order to call set(this)).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
