you need to assemble the code to get spark working (unless you are using hadoop 1.0.4).
to run the code you can follow any of the standalone guides here: https://spark.apache.org/docs/0.9.0/quick-start.html#a-standalone-app-in-scalayou would still need sbt though. Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Tue, Mar 18, 2014 at 2:32 AM, Chengi Liu <chengi.liu...@gmail.com> wrote: > Yeah.. The http_proxy is set up.. and so is https_proxy.. > Basically, my maven projects, git pulls etc everything is working fine.. > except this. > > Here is another question which might help me to bypass this issue.... > If I create a jar using eclipse... how do i run that jar in code. Like in > hadoop, I create a jar and then that jar like hadoop jar jar_name > Earlier what i was trying to do was basically, write a code inside > spark_examples directory.. do a new sbt build and create new jars.. and use > run_examples script to run my code.. But since sbt assembly is having > connection issue maybe someone can help me with how to build jars and > deploy code on spark (not using spark shell) > Thanks > > > On Mon, Mar 17, 2014 at 11:27 PM, Mayur Rustagi > <mayur.rust...@gmail.com>wrote: > >> is it translating to sbt? >> are you also setting command line proxy HTTP_PROXY >> easiest is to build a small code & just test it out by building in >> command line.. >> >> Mayur Rustagi >> Ph: +1 (760) 203 3257 >> http://www.sigmoidanalytics.com >> @mayur_rustagi <https://twitter.com/mayur_rustagi> >> >> >> >> On Tue, Mar 18, 2014 at 2:15 AM, Chengi Liu <chengi.liu...@gmail.com>wrote: >> >>> Hi Sean, >>> Yeah.. I am seeing erros across all repos and yepp.. this error is >>> mainly because of connectivity issue... >>> How do I set up proxy.. I did set up proxy as suggested by Mayur: >>> >>> export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=yourserver >>> -Dhttp.proxyPort=8080 -Dhttp.proxyUser=username >>> -Dhttp.proxyPassword=password" >>> >>> >>> How do I rectify this error. :( >>> >>> >>> >>> On Mon, Mar 17, 2014 at 6:07 PM, Sean Owen <so...@cloudera.com> wrote: >>> >>>> It's in the main Maven repo: >>>> http://central.maven.org/maven2/io/netty/netty-all/ >>>> >>>> I assume you're seeing errors accessing all repos? the last few you >>>> quote are not where they are intended to be, you're just seeing it >>>> fail through all of them. I think it remains a connectivity problem >>>> from your env to the repos, possibly because of a proxy? >>>> -- >>>> Sean Owen | Director, Data Science | London >>>> >>>> >>>> On Mon, Mar 17, 2014 at 8:39 PM, Chengi Liu <chengi.liu...@gmail.com> >>>> wrote: >>>> > I have set it up.. still it fails.. Question: >>>> > >>>> https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/ >>>> > >>>> > 4.0.13 is not there? Instead 4.0.18 is there?? Is this a bug? >>>> > >>>> > >>>> > On Mon, Mar 17, 2014 at 11:01 AM, Mayur Rustagi < >>>> mayur.rust...@gmail.com> >>>> > wrote: >>>> >> >>>> >> >>>> >> >>>> http://mail-archives.apache.org/mod_mbox/spark-user/201403.mbox/%3ccaaqhkj48japuzqc476es67c+rrfime87uprambdoofhcl0k...@mail.gmail.com%3E >>>> >> >>>> >> You also have to specify git proxy as code may be copied off git >>>> also. >>>> >> >>>> >> Mayur Rustagi >>>> >> Ph: +1 (760) 203 3257 >>>> >> http://www.sigmoidanalytics.com >>>> >> @mayur_rustagi >>>> >> >>>> >> >>>> >> >>>> >> On Mon, Mar 17, 2014 at 1:25 PM, Chengi Liu <chengi.liu...@gmail.com >>>> > >>>> >> wrote: >>>> >>> >>>> >>> Hi, >>>> >>> I am trying to compile the spark project using sbt/sbt assembly.. >>>> >>> And i see this error: >>>> >>> [info] Resolving io.netty#netty-all;4.0.13.Final ... >>>> >>> [error] Server access Error: Connection timed out >>>> >>> url= >>>> https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom >>>> >>> [error] Server access Error: Connection timed out >>>> >>> url= >>>> https://oss.sonatype.org/service/local/staging/deploy/maven2/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom >>>> >>> [error] Server access Error: Connection timed out >>>> >>> url= >>>> https://repository.cloudera.com/artifactory/cloudera-repos/io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.pom >>>> >>> >>>> >>> >>>> >>> >>>> >>> I followed the url and seemslike 4.0.13 is not present..Am i missing >>>> >>> something.. Also, I am behind proxy.. can that be an issue? >>>> >>> How do i resolve this. >>>> >>> Thanks >>>> >>> >>>> >> >>>> > >>>> >>> >>> >> >
