Hi Prateek,
Were you able to figure why this is happening? I'm seeing the same error on
my spark standalone cluster.
Any pointers anyone?
On Fri, Dec 11, 2015 at 2:05 PM, prateek arora
wrote:
>
>
> Hi
>
> I am trying to access Spark Using REST API but got below error :
>
> Command :
>
> curl ht
No we are using standard spark w/ datastax cassandra. I'm able to see some
json when I do http://10.1.40.16:7080/json/v1/applications
but getting the following errors when I do
http://10.1.40.16:7080/api/v1/applications
HTTP ERROR 503
Problem accessing /api/v1/applications. Reason:
Service
We are using Monit to kick off spark streaming jobs n seems to work fine.
On Monday, September 28, 2015, Chen Song wrote:
> I am also interested specifically in monitoring and alerting on Spark
> streaming jobs. It will be helpful to get some general guidelines or advice
> on this, from people w
Does Azkaban support scheduling long running jobs like spark steaming jobs?
Will Azkaban kill a job if it's running for a long time.
On Friday, August 7, 2015, Vikram Kone wrote:
> Hien,
> Is Azkaban being phased out at linkedin as rumored? If so, what's linkedin
> going
95. You can leverage
> the SLA feature to kill a job if it ran longer than expected.
>
> BTW, we just solved the scalability issue by supporting multiple
> executors. Within a week or two, the code for that should be merged in the
> main trunk.
>
> Hien
>
> On Tue, Oct 6, 2015
t 2015 à 17:51, Hien Luu a
>> écrit :
>>
>>> Looks like Oozie can satisfy most of your requirements.
>>>
>>>
>>>
>>> On Fri, Aug 7, 2015 at 8:43 AM, Vikram Kone
>>> wrote:
>>>
>>>> Hi,
>>>> I&#x
Hi Feng,
Does airflow allow remote submissions of spark jobs via spark-submit?
On Wed, Nov 18, 2015 at 6:01 PM, Fengdong Yu
wrote:
> Hi,
>
> we use ‘Airflow' as our job workflow scheduler.
>
>
>
>
> On Nov 19, 2015, at 9:47 AM, Vikram Kone wrote:
>
> Hi
Hi,
I'm seeing a strange problem. I have a spark cluster in standalone mode. I
submit spark jobs from a remote node as follows from the terminal
spark-submit --master spark://10.1.40.18:7077 --class com.test.Ping
spark-jobs.jar
when the app is running , when I press ctrl-C on the console termina
Spark 1.4.1
On Friday, November 20, 2015, Ted Yu wrote:
> Which Spark release are you using ?
>
> Can you pastebin the stack trace of the process running on your machine ?
>
> Thanks
>
> On Nov 20, 2015, at 6:46 PM, Vikram Kone > wrote:
>
> Hi,
> I'm se
l was shutdown when spark stopped
>
> I hopes this help
>
> Stephane
>
>
> On Fri, Nov 20, 2015 at 7:46 PM, Vikram Kone > wrote:
>
>> Hi,
>> I'm seeing a strange problem. I have a spark cluster in standalone mode.
>> I submit spark jobs fr
I tried adding shutdown hook to my code but it didn't help. Still same issue
On Fri, Nov 20, 2015 at 7:08 PM, Ted Yu wrote:
> Which Spark release are you using ?
>
> Can you pastebin the stack trace of the process running on your machine ?
>
> Thanks
>
> On Nov 20
Hi,
I'm looking for open source workflow tools/engines that allow us to
schedule spark jobs on a datastax cassandra cluster. Since there are tonnes
of alternatives out there like Ozzie, Azkaban, Luigi , Chronos etc, I
wanted to check with people here to see what they are using today.
Some of the r
ore than
full feature set
On Friday, August 7, 2015, Hien Luu wrote:
> Looks like Oozie can satisfy most of your requirements.
>
>
>
> On Fri, Aug 7, 2015 at 8:43 AM, Vikram Kone > wrote:
>
>> Hi,
>> I'm looking for open source workflow tools/engines that al
workflows (independent of spark-defined job flows).
>>
>>
>> On Fri, Aug 7, 2015 at 7:00 PM, Jörn Franke > > wrote:
>>
>>> Check also falcon in combination with oozie
>>>
>>> Le ven. 7 août 2015 à 17:51, Hien Luu a
>>> écrit :
>&g
nd it way better than Oozie. I haven't tried Chronos, and it seemed
>>>> quite involved to set up. Haven't tried Luigi either.
>>>>
>>>> Spark job server is good but as you say lacks some stuff like
>>>> scheduling and DAG type workflows (independent of
nts.
This became a bit of a brain dump on the topic. I hope that it is
useful. Don't hesitate to get back if I can help.
Regards,
Lars Albertsson
On Fri, Aug 7, 2015 at 5:43 PM, Vikram Kone wrote:
> Hi,
> I'm looking for open source workflow tools/engines that allow us to sche
Hi,
We are planning to install Spark in stand alone mode on cassandra cluster.
The problem, is since Cassandra has a no-SPOF architecture ie any node can
become the master for the cluster, it creates the problem for Spark master
since it's not a peer-peer architecture where any node can become the
I have a python script that is used to submit spark jobs using the
spark-submit tool. I want to execute the command and write the output both
to STDOUT and a logfile in real time. i'm using python 2.7 on a ubuntu
server.
This is what I have so far in my SubmitJob.py script
#!/usr/bin/python
# Sub
18 matches
Mail list logo
