Hi hadoop users, I just want to verify that there is no way to put a binary on HDFS and execute it using the hadoop java api. If not, I would appreciate advice in getting in creating an implementation that uses native libraries.
"In contrast to the POSIX model, there are no *sticky*, *setuid* or *setgid* bits for files as there is no notion of executable files." Is there no workaround? A little bit more about what I'm trying to do. I have a binary that converts my image to another image format. I currently want to put it in the distributed cache and tell the reducer to execute the binary on the data on hdfs. However, since I can't set the execute permission bit on that file, it seems that I cannot do that. Since I cannot use the binary, it seems like I have to use my own implementation to do this. The challenge is that these libraries that I can use to do this are .a and .so files. Would I have to use JNI and package the libraries in the distributed cache and then have the reducer find and use those libraries on the task nodes? Actually, I wouldn't want to use JNI, I'd probably want to use java native access (JNA) to do this. Has anyone used JNA with hadoop and been successful? Are there problems I'll encounter? Please let me know. Thanks, -Julian
