-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6609/
-----------------------------------------------------------
(Updated Oct. 10, 2012, 5:49 p.m.)
Review request for giraph.
Changes
-------
updated description
Description (updated)
-------
Passes
mvn -Phadoop_non_secure clean verify && mvn -Phadoop_0.20.203 clean verify &&
mvn -Phadoop_1.0 clean verify && mvn -Phadoop0.23 clean verify && mvn
-Phadoop_2.0.0 clean verify && mvn -Phadoop_2.0.1 clean verify && mvn
-Phadoop_2.0.2 clean verify
How to use:
use
-Dgiraph.useNetty=true -Dgiraph.authenticate=true
on your job's command line to use this feature.
How to compile and test (using Hadoop 2.0.0 for example):
mvn -Phadoop_2.0.0 clean test
$HADOOP_RUNTIME/bin/hadoop jar
$GIRAPH_DIR/target/giraph-0.2-SNAPSHOT-for-hadoop-2.0.0-SNAPSHOT-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useNetty=true
-Dgiraph.authenticate=true -e 100 -s 100 -v -V 100 -w 6
Look for something similar to the following in the workers' syslogs:
2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.handler.SaslClientHandler: decode: Got a response
of type SASL_TOKEN_MESSAGE_REQUEST from
server:Eugenes-MacBook-Pro.local/172.16.175.1:30010
2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.handler.SaslClientHandler: handleUpstream:
Responding to server's token of length: 40
2012-10-10 09:24:49,924 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.handler.SaslClientHandler: handleUpstream:
Response to server is null: authentication should now be complete.
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.ByteCounter: handleUpstream: 0 buffer size = 17,
total bytes = 207
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.handler.SaslClientHandler: decode: Got a response
of type SASL_COMPLETE_REQUEST from
server:Eugenes-MacBook-Pro.local/172.16.175.1:30010
2012-10-10 09:24:49,925 DEBUG [New I/O client worker #1-2]
org.apache.giraph.comm.netty.handler.SaslClientHandler: handleUpstream: Server
has sent us the SaslComplete message. Allowing normal work to proceed.
2012-10-10 09:24:49,926 DEBUG [main] org.apache.giraph.comm.netty.NettyClient:
authenticateOnChannel: Authentication on channel: [id: 0x32486cdd,
/172.16.175.1:54768 => Eugenes-MacBook-Pro.local/172.16.175.1:30010] has
completed successfully.
..
..
2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6]
org.apache.giraph.comm.netty.handler.AuthorizeServerHandler: messageReceived:
Got class org.apache.giraph.comm.requests.SendWorkerMessagesRequest
2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6]
org.apache.giraph.comm.netty.handler.AuthorizeServerHandler: messageReceived:
authenticated client: FmpvYl8xMzQ5NzE1MTIxNDc1XzAyMDU= is authorized to do
request on server.
2012-10-10 09:24:56,844 INFO [main] org.apache.giraph.comm.netty.NettyClient:
waitAllRequests: Finished all requests. MBytes/sec sent = 0.3824, MBytes/sec
received = 0.0124, MBytesSent = 0.0019, MBytesReceived = 0.0001, ave sent req
MBytes = 0.0004, ave received req MBytes = 0, secs waited = 0.0040
2012-10-10 09:24:56,844 DEBUG [New I/O server worker #1-6]
org.apache.giraph.comm.netty.handler.RequestServerHandler: messageReceived:
Processing client 3, requestId 101, SEND_WORKER_MESSAGES_REQUEST took 30000 ns
This addresses bug GIRAPH-211.
https://issues.apache.org/jira/browse/GIRAPH-211
Diffs
-----
pom.xml 51944fa
src/main/java/org/apache/giraph/GiraphConfiguration.java 797ba6e
src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1849504
src/main/java/org/apache/giraph/comm/CommunicationsInterface.java d60f512
src/main/java/org/apache/giraph/comm/SecureRPCCommunications.java af9720e
src/main/java/org/apache/giraph/comm/WorkerClient.java c3ec4fe
src/main/java/org/apache/giraph/comm/netty/NettyClient.java 93f5671
src/main/java/org/apache/giraph/comm/netty/NettyServer.java 886d68b
src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java 5b71401
src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientServer.java
4710f3a
src/main/java/org/apache/giraph/comm/netty/SaslNettyClient.java PRE-CREATION
src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java PRE-CREATION
src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java
3174447
src/main/java/org/apache/giraph/comm/netty/handler/SaslClientHandler.java
PRE-CREATION
src/main/java/org/apache/giraph/comm/netty/handler/SaslServerHandler.java
PRE-CREATION
src/main/java/org/apache/giraph/comm/requests/RequestType.java b412aa2
src/main/java/org/apache/giraph/comm/requests/SaslCompleteRequest.java
PRE-CREATION
src/main/java/org/apache/giraph/comm/requests/SaslTokenMessageRequest.java
PRE-CREATION
src/main/java/org/apache/giraph/graph/BspServiceWorker.java bb498ce
src/test/java/org/apache/giraph/comm/SaslConnectionTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/6609/diff/
Testing (updated)
-------
Tested as above.
Thanks,
Eugene Koontz
