Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Avery Ching

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7756
---


Overall, looks great.  Can you address the questions/comments and then I'll 
re-review?


trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java
https://reviews.apache.org/r/5077/#comment17058

Just out of curiosity, why this change?



trunk/src/main/java/org/apache/giraph/utils/FileUtils.java
https://reviews.apache.org/r/5077/#comment17059

Why delete it?



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17060

Empty params and return.



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17061

Empty params and return.



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17062

Empty params and return.



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17063

Empty params and return.



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17064

@return



trunk/src/test/java/org/apache/giraph/BspCase.java
https://reviews.apache.org/r/5077/#comment17065

@return



trunk/src/test/java/org/apache/giraph/TestBspBasic.java
https://reviews.apache.org/r/5077/#comment17066

shouldn't it be 49 not 491?


- Avery


On 2012-05-09 11:37:47, Sebastian Schelter wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/5077/
 ---
 
 (Updated 2012-05-09 11:37:47)
 
 
 Review request for giraph.
 
 
 Summary
 ---
 
 All temporary files that the tests generate are now written to 
 /tmp/_giraphTests including zooKeeper files, checkpoints etc. 
 
 This behavior will be automatically configured whenever 
 InternalVertexRunner.run() or BspCase.prepareJob() is used.
 
 Usually I can't stop myself once I have my refactoring hat on, therefore I 
 also tidied up a lot of minor stuff, removed code duplications etc.
 
 
 This addresses bug GIRAPH-20.
 https://issues.apache.org/jira/browse/GIRAPH-20
 
 
 Diffs
 -
 
   trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1332106 
   trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1332106 
   trunk/src/test/java/org/apache/giraph/BspCase.java 1332106 
   trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1332106 
   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/5077/diff
 
 
 Testing
 ---
 
 successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
 
 
 Thanks,
 
 Sebastian
 




Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Sebastian Schelter


 On 2012-05-10 06:57:01, Avery Ching wrote:
  Overall, looks great.  Can you address the questions/comments and then I'll 
  re-review?

Thanks for the quick review!

I'll address your comments, merge this with the current trunk and posted a new 
patch.


 On 2012-05-10 06:57:01, Avery Ching wrote:
  trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java, 
  lines 85-87
  https://reviews.apache.org/r/5077/diff/1/?file=108155#file108155line85
 
  Just out of curiosity, why this change?

If one uses writeUTF() and then reads the resulting file with a buffered 
reader, each line starts with a broken char. Directly writing the bytes out 
solved this.


 On 2012-05-10 06:57:01, Avery Ching wrote:
  trunk/src/main/java/org/apache/giraph/utils/FileUtils.java, line 87
  https://reviews.apache.org/r/5077/diff/1/?file=108156#file108156line87
 
  Why delete it?

I'll remove this.


 On 2012-05-10 06:57:01, Avery Ching wrote:
  trunk/src/test/java/org/apache/giraph/TestBspBasic.java, line 242
  https://reviews.apache.org/r/5077/diff/1/?file=108160#file108160line242
 
  shouldn't it be 49 not 491?

its a small L not a 1 :)


- Sebastian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7756
---


On 2012-05-09 11:37:47, Sebastian Schelter wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/5077/
 ---
 
 (Updated 2012-05-09 11:37:47)
 
 
 Review request for giraph.
 
 
 Summary
 ---
 
 All temporary files that the tests generate are now written to 
 /tmp/_giraphTests including zooKeeper files, checkpoints etc. 
 
 This behavior will be automatically configured whenever 
 InternalVertexRunner.run() or BspCase.prepareJob() is used.
 
 Usually I can't stop myself once I have my refactoring hat on, therefore I 
 also tidied up a lot of minor stuff, removed code duplications etc.
 
 
 This addresses bug GIRAPH-20.
 https://issues.apache.org/jira/browse/GIRAPH-20
 
 
 Diffs
 -
 
   trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1332106 
   trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1332106 
   trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1332106 
   trunk/src/test/java/org/apache/giraph/BspCase.java 1332106 
   trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1332106 
   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
 1332106 
   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/5077/diff
 
 
 Testing
 ---
 
 successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
 
 
 Thanks,
 
 Sebastian
 




[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272152#comment-13272152
 ] 

jirapos...@reviews.apache.org commented on GIRAPH-20:
-



bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.   Overall, looks great.  Can you address the questions/comments and then 
I'll re-review?

Thanks for the quick review!

I'll address your comments, merge this with the current trunk and posted a new 
patch.


bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java, 
lines 85-87
bq.   https://reviews.apache.org/r/5077/diff/1/?file=108155#file108155line85
bq.  
bq.   Just out of curiosity, why this change?

If one uses writeUTF() and then reads the resulting file with a buffered 
reader, each line starts with a broken char. Directly writing the bytes out 
solved this.


bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java, line 87
bq.   https://reviews.apache.org/r/5077/diff/1/?file=108156#file108156line87
bq.  
bq.   Why delete it?

I'll remove this.


bq.  On 2012-05-10 06:57:01, Avery Ching wrote:
bq.   trunk/src/test/java/org/apache/giraph/TestBspBasic.java, line 242
bq.   https://reviews.apache.org/r/5077/diff/1/?file=108160#file108160line242
bq.  
bq.   shouldn't it be 49 not 491?

its a small L not a 1 :)


- Sebastian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7756
---


On 2012-05-09 11:37:47, Sebastian Schelter wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/5077/
bq.  ---
bq.  
bq.  (Updated 2012-05-09 11:37:47)
bq.  
bq.  
bq.  Review request for giraph.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 
bq.  
bq.  This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.
bq.  
bq.  Usually I can't stop myself once I have my refactoring hat on, therefore I 
also tidied up a lot of minor stuff, removed code duplications etc.
bq.  
bq.  
bq.  This addresses bug GIRAPH-20.
bq.  https://issues.apache.org/jira/browse/GIRAPH-20
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 
1332106 
bq.trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1332106 
bq.trunk/src/test/java/org/apache/giraph/BspCase.java 1332106 
bq.trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1332106 
bq.trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
1332106 
bq.trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/5077/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Sebastian
bq.  
bq.



 Move temporary test files from the project directory
 

 Key: GIRAPH-20
 URL: https://issues.apache.org/jira/browse/GIRAPH-20
 Project: Giraph
  Issue Type: Improvement
  Components: test
Affects Versions: 0.2.0
Reporter: Owen O'Malley
Assignee: Sebastian Schelter
 Attachments: GIRAPH-20.patch


 We shouldn't use the project directory as the location for temporary files 
 generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: 

[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Roman K (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272201#comment-13272201
 ] 

Roman K commented on GIRAPH-141:


Sorry for taking your time, but I am not sure I understand you correctly.
Are you talking about Avery's idea, to sub-class the BasicVertex class with 
some BasicMultiVertexClass and etc. or about introducing some MultiValueEdge 
class which implements Writable and could be hidden inside the existing 
simple graph structure?

 mulitgraph support in giraph
 

 Key: GIRAPH-141
 URL: https://issues.apache.org/jira/browse/GIRAPH-141
 Project: Giraph
  Issue Type: Improvement
  Components: graph
Reporter: André Kelpe

 The current vertex API only supports simple graphs, meaning that there can 
 only ever be one edge between two vertices. Many graphs like the road network 
 are in fact multigraphs, where many edges can connect two vertices at the 
 same time.
 Support for this could be added by introducing an IteratorEdgeWritable 
 getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
 Connector between the edge and the vertex is also a good idea, so that you 
 could do something like:
 {code} 
 for (final ConnectorEdgeWritable, VertexWritable conn: getEdgeValues(){
  final EdgeWritable edge = conn.getEdge();
  final VertexWritable otherVertex = conn.getOther();
  doInterestingStuff(otherVertex);
  doMoreInterestingStuff(edge);
 }
 {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (GIRAPH-141) mulitgraph support in giraph

2012-05-10 Thread Sebastian Schelter (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272209#comment-13272209
 ] 

Sebastian Schelter commented on GIRAPH-141:
---

No need to excuse.

Maybe we simply misunderstand each other. I just wanted to say that it might be 
a lot of effort to implement real multigraph support deep into the system as a 
lot of special cases might have to be kept in mind when doing this, e.g. when 
mutating or partitioning the graph. I just wanted to say that we should first 
search for simple ways to add multigraph support transparently on top of what 
we already have.

 mulitgraph support in giraph
 

 Key: GIRAPH-141
 URL: https://issues.apache.org/jira/browse/GIRAPH-141
 Project: Giraph
  Issue Type: Improvement
  Components: graph
Reporter: André Kelpe

 The current vertex API only supports simple graphs, meaning that there can 
 only ever be one edge between two vertices. Many graphs like the road network 
 are in fact multigraphs, where many edges can connect two vertices at the 
 same time.
 Support for this could be added by introducing an IteratorEdgeWritable 
 getEdgeValue() or a similar construct. Maybe introducing a slim object like a 
 Connector between the edge and the vertex is also a good idea, so that you 
 could do something like:
 {code} 
 for (final ConnectorEdgeWritable, VertexWritable conn: getEdgeValues(){
  final EdgeWritable edge = conn.getEdge();
  final VertexWritable otherVertex = conn.getOther();
  doInterestingStuff(otherVertex);
  doMoreInterestingStuff(edge);
 }
 {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Sebastian Schelter

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/
---

(Updated 2012-05-10 09:32:10.140325)


Review request for giraph.


Changes
---

Updated the patch to reflect Avery's comments.

The line FileUtils:87 has to be kept, we initially delete the new file (if it 
existed) as it will be recreated later.


Summary
---

All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 

This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.

Usually I can't stop myself once I have my refactoring hat on, therefore I also 
tidied up a lot of minor stuff, removed code duplications etc.


This addresses bug GIRAPH-20.
https://issues.apache.org/jira/browse/GIRAPH-20


Diffs (updated)
-

  trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1336504 
  trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
  trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 1336504 
  trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
  trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 1336504 
  trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
  trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 

Diff: https://reviews.apache.org/r/5077/diff


Testing
---

successfully passed local and pseudo-distributed tests with Hadoop 0.20.203


Thanks,

Sebastian



[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272217#comment-13272217
 ] 

jirapos...@reviews.apache.org commented on GIRAPH-20:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/
---

(Updated 2012-05-10 09:32:10.140325)


Review request for giraph.


Changes
---

Updated the patch to reflect Avery's comments.

The line FileUtils:87 has to be kept, we initially delete the new file (if it 
existed) as it will be recreated later.


Summary
---

All temporary files that the tests generate are now written to 
/tmp/_giraphTests including zooKeeper files, checkpoints etc. 

This behavior will be automatically configured whenever 
InternalVertexRunner.run() or BspCase.prepareJob() is used.

Usually I can't stop myself once I have my refactoring hat on, therefore I also 
tidied up a lot of minor stuff, removed code duplications etc.


This addresses bug GIRAPH-20.
https://issues.apache.org/jira/browse/GIRAPH-20


Diffs (updated)
-

  trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
1336504 
  trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
  trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 1336504 
  trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
  trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 1336504 
  trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
  trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
  trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
  trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 

Diff: https://reviews.apache.org/r/5077/diff


Testing
---

successfully passed local and pseudo-distributed tests with Hadoop 0.20.203


Thanks,

Sebastian



 Move temporary test files from the project directory
 

 Key: GIRAPH-20
 URL: https://issues.apache.org/jira/browse/GIRAPH-20
 Project: Giraph
  Issue Type: Improvement
  Components: test
Affects Versions: 0.2.0
Reporter: Owen O'Malley
Assignee: Sebastian Schelter
 Attachments: GIRAPH-20.patch


 We shouldn't use the project directory as the location for temporary files 
 generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (GIRAPH-153) HBase/Accumulo Input and Output formats

2012-05-10 Thread Brian Femiano (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272309#comment-13272309
 ] 

Brian Femiano commented on GIRAPH-153:
--

I'll take down the confluence entry until this is approved. We don't want 
anyone getting the wrong idea. 

 HBase/Accumulo Input and Output formats
 ---

 Key: GIRAPH-153
 URL: https://issues.apache.org/jira/browse/GIRAPH-153
 Project: Giraph
  Issue Type: New Feature
  Components: bsp
Affects Versions: 0.1.0
 Environment: Single host OSX 10.6.8 2.2Ghz Intel i7, 8GB
Reporter: Brian Femiano
 Attachments: GIRAPH-153.1.patch, GIRAPH-153.2.patch, 
 GIRAPH-153.3.patch, GIRAPH-153.patch


 Four abstract classes that wrap their respective delegate input/output 
 formats for
 easy hooks into vertex input format subclasses. I've included some sample 
 programs that show two very simple graph
 algorithms. I have a graph generator that builds out a very simple directed 
 structure, starting with a few 'root' nodes.
 Root nodes are defined as nodes which are not listed as a child anywhere in 
 the graph. 
 Algorithm 1) AccumuloRootMarker.java  -- Accumulo as read/write source. 
 Every vertex starts thinking it's a root. At superstep 0, send a message down 
 to each
 child as a non-root notification. After superstep 1, only root nodes will 
 have never been messaged. 
 Algorithm 2) TableRootMarker -- HBase as read/write source. Expands on A1 by 
 bundling the notification logic followed by root node propagation. Once we've 
 marked the appropriate nodes as roots, tell every child which roots it can be 
 traced back to via one or more spanning trees. This will take N + 2 
 supersteps where N is the maximum number of hops from any root to any leaf, 
 plus 2 supersteps for the initial root flagging. 
 I've included all relevant code plus DistributedCacheHelper.java for 
 recursive cache file and archive searches. It is more hadoop centric than 
 giraph, but these jobs use it so I figured why not commit here. 
 These have been tested through local JobRunner, pseudo-distributed on the 
 aforementioned hardware, and full distributed on EC2. More details in the 
 comments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Review Request: GIRAPH-20 Move temporary test files from the project directory

2012-05-10 Thread Avery Ching

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5077/#review7772
---

Ship it!


+1, thanks Sebastian!  I'm assuming it also passes 'mvn verify', but please 
double check before you commit.

- Avery


On 2012-05-10 09:32:10, Sebastian Schelter wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/5077/
 ---
 
 (Updated 2012-05-10 09:32:10)
 
 
 Review request for giraph.
 
 
 Summary
 ---
 
 All temporary files that the tests generate are now written to 
 /tmp/_giraphTests including zooKeeper files, checkpoints etc. 
 
 This behavior will be automatically configured whenever 
 InternalVertexRunner.run() or BspCase.prepareJob() is used.
 
 Usually I can't stop myself once I have my refactoring hat on, therefore I 
 also tidied up a lot of minor stuff, removed code duplications etc.
 
 
 This addresses bug GIRAPH-20.
 https://issues.apache.org/jira/browse/GIRAPH-20
 
 
 Diffs
 -
 
   trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 
 1336504 
   trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1336504 
   trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java 
 1336504 
   trunk/src/main/java/org/apache/giraph/utils/FileUtils.java PRE-CREATION 
   trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 
 1336504 
   trunk/src/test/java/org/apache/giraph/BspCase.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java 1336506 
   trunk/src/test/java/org/apache/giraph/TestBspBasic.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java 1336506 
   trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java 1336504 
   trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java 1336504 
   trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1336504 
 
 Diff: https://reviews.apache.org/r/5077/diff
 
 
 Testing
 ---
 
 successfully passed local and pseudo-distributed tests with Hadoop 0.20.203
 
 
 Thanks,
 
 Sebastian
 




[jira] [Commented] (GIRAPH-20) Move temporary test files from the project directory

2012-05-10 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/GIRAPH-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13272418#comment-13272418
 ] 

Hudson commented on GIRAPH-20:
--

Integrated in Giraph-trunk-Commit #109 (See 
[https://builds.apache.org/job/Giraph-trunk-Commit/109/])
GIRAPH-20 Move temporary test files from the project directory (Revision 
1336743)

 Result = SUCCESS
ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1336743
Files : 
* /incubator/giraph/trunk/CHANGELOG
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/TextAggregatorWriter.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/utils/FileUtils.java
* 
/incubator/giraph/trunk/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/BspCase.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestAutoCheckpoint.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/TestBspBasic.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestGraphPartitioner.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestJsonBase64Format.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestManualCheckpoint.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestMutateGraphVertex.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestNotEnoughMapTasks.java
* /incubator/giraph/trunk/src/test/java/org/apache/giraph/TestZooKeeperExt.java
* 
/incubator/giraph/trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java


 Move temporary test files from the project directory
 

 Key: GIRAPH-20
 URL: https://issues.apache.org/jira/browse/GIRAPH-20
 Project: Giraph
  Issue Type: Improvement
  Components: test
Affects Versions: 0.2.0
Reporter: Owen O'Malley
Assignee: Sebastian Schelter
 Fix For: 0.2.0

 Attachments: GIRAPH-20.patch


 We shouldn't use the project directory as the location for temporary files 
 generated by the tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira