Re: Bundles for deployment of hadoop service in osgi container
Hello Guillaume Yes i tried your link.I downloaded the fabric-hadoop folder.I deployed it by giving the command mvn clean deploy.And after that i started the karaf root shell by starting karaf.bat file inside ..\fusesource\bin\karaf.bat.Thats it.Now i have wordcount example with me.I have exported it to jar file.But i am unable to find what to do afterwards.How to take output of wordcount example to help my osgi service? On Wed, Jun 27, 2012 at 11:12 PM, ramesh chandra tablet...@gmail.comwrote: On 06/27/2012 12:22 PM, Jean-Baptiste Onofré wrote: Hi Somya, I'm still working on the patches. However, if you want, I can deploy the bundles on my repo (http://maven.nanthrax.net) if you want to test. Regards JB On 06/27/2012 10:58 AM, somya singhal wrote: Hello I have recently coma across a link -: http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features I am trying to install hadoop bundle in osgi for quite a few days.But i am unable to do so.Anyone can please suggest me as it is written in the link,from where can i get Hadoop modules (common, annotations, hdfs, mapreduce, etc)??? Somya Singhal Btech(4th year,csi) IIT ROORKEE Hello Somya, You can follow the *headers* as it is the best way to find missing dependencies. And you can also wrap all the dependencies , common, annotations, hdfs etc and any other external dependency that is missing and install as osgi. osgi:install wrap:mvn:Groupid/Artifactid/versoion Hopefully this approach might help regards. Ramesh
Re: Bundles for deployment of hadoop service in osgi container
The fabric-hadoop module uses OSGi managed services to create the
hadoop services so you need to add a file
etc/org.fusesource.fabric.hadoop.cfg with the following parameters for
example:
fs.default.name=hdfs\://localhost\:9000
dfs.replication=1
mapred.job.tracker=localhost\:9001
dfs.name.dir=${karaf.data}/hadoop/dfs/name
dfs.http.address=localhost\:9002
dfs.data.dir=${karaf.data}/hadoop/dfs/data
dfs.name.edits.dir=${karaf.data}/hadoop/dfs/name
nameNode=true
dataNode=true
You should then have an hdfs file system ready to use.
If you want map reduce, you can add
jobTracker=true
taskTracker=true
Note that hadoop is designed to have nameNode / dataNode / jobTracker
/ taskTracker on different servers ...
On Thu, Jun 28, 2012 at 11:50 AM, somya singhal
28somyasing...@gmail.com wrote:
Hello Guillaume
Yes i tried your link.I downloaded the fabric-hadoop folder.I deployed it by
giving the command mvn clean deploy.And after that i started the karaf root
shell by starting karaf.bat file inside ..\fusesource\bin\karaf.bat.Thats
it.Now i have wordcount example with me.I have exported it to jar file.But i
am unable to find what to do afterwards.How to take output of wordcount
example to help my osgi service?
On Wed, Jun 27, 2012 at 11:12 PM, ramesh chandra tablet...@gmail.com
wrote:
On 06/27/2012 12:22 PM, Jean-Baptiste Onofré wrote:
Hi Somya,
I'm still working on the patches. However, if you want, I can deploy the
bundles on my repo (http://maven.nanthrax.net) if you want to test.
Regards
JB
On 06/27/2012 10:58 AM, somya singhal wrote:
Hello
I have recently coma across a link -:
http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features
I am trying to install hadoop bundle in osgi for quite a few days.But i
am unable to do so.Anyone can please suggest me as it is written in the
link,from where can i get Hadoop modules (common, annotations, hdfs,
mapreduce, etc)???
Somya Singhal
Btech(4th year,csi)
IIT ROORKEE
Hello Somya,
You can follow the headers as it is the best way to find missing
dependencies. And you can also
wrap all the dependencies , common, annotations, hdfs etc and any other
external dependency that is missing and install as osgi.
osgi:install wrap:mvn:Groupid/Artifactid/versoion
Hopefully this approach might help
regards.
Ramesh
--
Guillaume Nodet
Blog: http://gnodet.blogspot.com/
FuseSource, Integration everywhere
http://fusesource.com
Re: Bundles for deployment of hadoop service in osgi container
Also, note that the fabric-hadoop bundle will register a few karaf
commands for hadoop and also a url handler so you can access hdfs
directly from osgi using hdfs://[path].
On Thu, Jun 28, 2012 at 1:05 PM, Guillaume Nodet gno...@gmail.com wrote:
The fabric-hadoop module uses OSGi managed services to create the
hadoop services so you need to add a file
etc/org.fusesource.fabric.hadoop.cfg with the following parameters for
example:
fs.default.name=hdfs\://localhost\:9000
dfs.replication=1
mapred.job.tracker=localhost\:9001
dfs.name.dir=${karaf.data}/hadoop/dfs/name
dfs.http.address=localhost\:9002
dfs.data.dir=${karaf.data}/hadoop/dfs/data
dfs.name.edits.dir=${karaf.data}/hadoop/dfs/name
nameNode=true
dataNode=true
You should then have an hdfs file system ready to use.
If you want map reduce, you can add
jobTracker=true
taskTracker=true
Note that hadoop is designed to have nameNode / dataNode / jobTracker
/ taskTracker on different servers ...
On Thu, Jun 28, 2012 at 11:50 AM, somya singhal
28somyasing...@gmail.com wrote:
Hello Guillaume
Yes i tried your link.I downloaded the fabric-hadoop folder.I deployed it by
giving the command mvn clean deploy.And after that i started the karaf root
shell by starting karaf.bat file inside ..\fusesource\bin\karaf.bat.Thats
it.Now i have wordcount example with me.I have exported it to jar file.But i
am unable to find what to do afterwards.How to take output of wordcount
example to help my osgi service?
On Wed, Jun 27, 2012 at 11:12 PM, ramesh chandra tablet...@gmail.com
wrote:
On 06/27/2012 12:22 PM, Jean-Baptiste Onofré wrote:
Hi Somya,
I'm still working on the patches. However, if you want, I can deploy the
bundles on my repo (http://maven.nanthrax.net) if you want to test.
Regards
JB
On 06/27/2012 10:58 AM, somya singhal wrote:
Hello
I have recently coma across a link -:
http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features
I am trying to install hadoop bundle in osgi for quite a few days.But i
am unable to do so.Anyone can please suggest me as it is written in the
link,from where can i get Hadoop modules (common, annotations, hdfs,
mapreduce, etc)???
Somya Singhal
Btech(4th year,csi)
IIT ROORKEE
Hello Somya,
You can follow the headers as it is the best way to find missing
dependencies. And you can also
wrap all the dependencies , common, annotations, hdfs etc and any other
external dependency that is missing and install as osgi.
osgi:install wrap:mvn:Groupid/Artifactid/versoion
Hopefully this approach might help
regards.
Ramesh
--
Guillaume Nodet
Blog: http://gnodet.blogspot.com/
FuseSource, Integration everywhere
http://fusesource.com
--
Guillaume Nodet
Blog: http://gnodet.blogspot.com/
FuseSource, Integration everywhere
http://fusesource.com
Re: Bundles for deployment of hadoop service in osgi container
Have you tried the links I gave you ? On Wed, Jun 27, 2012 at 10:58 AM, somya singhal 28somyasing...@gmail.com wrote: Hello I have recently coma across a link -: http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features I am trying to install hadoop bundle in osgi for quite a few days.But i am unable to do so.Anyone can please suggest me as it is written in the link,from where can i get Hadoop modules (common, annotations, hdfs, mapreduce, etc)??? Somya Singhal Btech(4th year,csi) IIT ROORKEE -- Guillaume Nodet Blog: http://gnodet.blogspot.com/ FuseSource, Integration everywhere http://fusesource.com
Re: Bundles for deployment of hadoop service in osgi container
Hi Somya, I'm still working on the patches. However, if you want, I can deploy the bundles on my repo (http://maven.nanthrax.net) if you want to test. Regards JB On 06/27/2012 10:58 AM, somya singhal wrote: Hello I have recently coma across a link -: http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features I am trying to install hadoop bundle in osgi for quite a few days.But i am unable to do so.Anyone can please suggest me as it is written in the link,from where can i get Hadoop modules (common, annotations, hdfs, mapreduce, etc)??? Somya Singhal Btech(4th year,csi) IIT ROORKEE -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
Re: Bundles for deployment of hadoop service in osgi container
On 06/27/2012 12:22 PM, Jean-Baptiste Onofré wrote: Hi Somya, I'm still working on the patches. However, if you want, I can deploy the bundles on my repo (http://maven.nanthrax.net) if you want to test. Regards JB On 06/27/2012 10:58 AM, somya singhal wrote: Hello I have recently coma across a link -: http://search-hadoop.com/m/7TZE59pm6vsubj=Re+PROPOSAL+Hadoop+OSGi+compliant+and+Apache+Karaf+features I am trying to install hadoop bundle in osgi for quite a few days.But i am unable to do so.Anyone can please suggest me as it is written in the link,from where can i get Hadoop modules (common, annotations, hdfs, mapreduce, etc)??? Somya Singhal Btech(4th year,csi) IIT ROORKEE Hello Somya, You can follow the /headers/ as it is the best way to find missing dependencies. And you can also wrap all the dependencies , common, annotations, hdfs etc and any other external dependency that is missing and install as osgi. osgi:install wrap:mvn:Groupid/Artifactid/versoion Hopefully this approach might help regards. Ramesh
