Hi
I am working on porting spark onto Hadoop 2.2.0, With some renaming and
call into new YARN API works done. I can run up the spark master. While I
encounter the issue that Executor Actor could not connecting to Driver actor.
After some investigation, I found the root cause is that the
akka-remote do not support protobuf 2.5.0 before 2.3. And hadoop move to
protobuf 2.5.0 from 2.1-beta.
The issue is that if I exclude the akka dependency from hadoop and
force protobuf dependency to 2.4.1, the compile/packing will fail since hadoop
common jar require a new interface from protobuf 2.5.0.
So any suggestion on this?
Best Regards,
Raymond Liu
