Hi I was able to carbonize the hadoop HDFS datanode and started and joined a carbon datanode in the WSO2 SS pack successfull Now i am into with testing of cluster created by file uploads, views and make dirs with the hadoop CLI. I used hadoop version 2.4.1.
Thanks On Wed, Aug 13, 2014 at 2:18 PM, Poshitha Dabare <[email protected]> wrote: > Hi, > > I am trying to spin up a carbonised datanode with authentication enabled > in 'core-site.xml' file. When trying to spin the datanode by calling > DataNode class in package org.apache.hadoop.hdfs.server.datanode it is > unable. It we inspect the ''hadoop' CLI sh file it trys to spin up datanode > using Class SecureDataNodeStarter which takes arguments PID files and also > creates secured resourced and pass them to DataNode class. Without these > arguments and resources ( whith them null) DataNode can not start with > secure resources and authentication. I am trying to automate the process > that takes place in the 'hadoop' CLI .sh file in the SS 'bin' folder , in > java code level by specifying the arguments in the HDFS datanode wrapper > class in HDFS component > > any suggestions ? > > thanks, > > > > > > > > > On Wed, Aug 6, 2014 at 10:32 AM, Poshitha Dabare <[email protected]> > wrote: > >> *Description* >> WSO2 Storage Server currently support only spinning up HDFS Name nodes >> at startup. After starting the WSO2 SS name node , the datanode should be >> spinnned off with using the 'hadoop' CLI tool provided in the SS >> installations 'bin' folder (ref: >> https://docs.wso2.com/display/SS110/Setting+up+Storage+Server+with+HDFS >> ). This is felt as a certain shortcoming in the SS HDFS provisioning >> process and should be addressed in SS future version 1.5.0. The solution is >> to implement the datanode as Carbon Component which loads at startup and >> spin of the itself based on a command line parameter given. >> >> Benefits of this also include, >> >> Not necessary to install KDC for the datanode to operate on the OS. >> Easy to use and configure HDFS datanode >> The datanode can be started as a single SS pack >> >> >> >> >> On Wed, Aug 6, 2014 at 9:03 AM, Prabath Abeysekera <[email protected]> >> wrote: >> >>> Hi Poshitha, >>> >>> Would you be able to add a small description on why this is being >>> implemented and the design considerations, etc you might have had to come >>> up with while you;re into it? >>> >>> Cheers, >>> Prabath >>> >>> >>> On Tue, Aug 5, 2014 at 3:44 PM, Poshitha Dabare <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I was able to start the datanode comepenent is SS as OSGI bundle. Also >>>> i added the switch to wso2server file as "Denable.hdfs.datanode" that >>>> enables the datanode at startup. I tried the system in the same machine by >>>> disabling authentication. Tested the file system by creating a folder and >>>> uploading file , i was able to download the same file in name node and data >>>> node. >>>> >>>> -- >>>> Poshitha Dabare >>>> Software Engineer >>>> >>>> Mob: 075 536 8005 >>>> >>> >>> >>> >>> -- >>> Prabath Abeysekara >>> Associate Technical Lead, Data TG. >>> WSO2 Inc. >>> Email: [email protected] >>> Mobile: +94774171471 >>> >> >> >> >> -- >> Poshitha Dabare >> Software Engineer >> >> Mob: 075 536 8005 >> > > > > -- > Poshitha Dabare > Software Engineer > > Mob: 075 536 8005 > -- Poshitha Dabare Software Engineer Mob: 075 536 8005
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
