Hi, I am trying to make an initial run on Amaterasu with https://github.com/arunma/amaterasu-v2-demo (just an unmodified fork of https://github.com/shintoio/amaterasu-v2-demo). Seems like the spark job fails with error (as I see from the logs). Not surprisingly, I am unable so see a json on the /tmp/test1.
I am not familiar with Mesos. Tried to check for clues on /var/log/mesos on the vagrant box with no luck. I am just running a single node mesos on vagrant ( https://github.com/shintoio/amaterasu-vagrant). Greatly appreciate if you could help me with some hints. Earlier I ran a `./gradlew buildHomeDir` and modified the Vagrantfile to point to my local build directory of Amaterasu. Cheers, Arun [vagrant@node1 ama]$ ./ama-start.sh --repo=" https://github.com/arunma/amaterasu-v2-demo.git"; --branch="master" --env="test" --report="code" serving amaterasu from /ama/lib on user supplied port ./ama-start.sh: line 29: popd: directory stack empty /\ / \ /\ / /\ / \ _ _ / / / /\ \ /_\ _ __ __ _ | |_ ___ _ _ __(_( _(_(_ )_) / _ \ | ' \ / _` || _|/ -_)| '_|/ _` |(_-<| || | /_/ \_\|_|_|_|\__,_| \__|\___||_| \__,_|/__/ \_,_| Continuously deployed data pipelines Version 0.2.0-incubating repo: https://github.com/arunma/amaterasu-v2-demo.git java -cp ./bin/leader-0.2.0-incubating-all.jar -Djava.library.path=/usr/lib org.apache.amaterasu.leader.mesos.JobLauncher --home . --repo https://github.com/arunma/amaterasu-v2-demo.git --branch master --env test --report code 2017-09-30 09:22:46.184:INFO::main: Logging initialized @688ms 2017-09-30 09:22:46.262:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT 2017-09-30 09:22:46.300:INFO:oejsh.ContextHandler:main: Started o.e.j.s.ServletContextHandler@385e9564{/,file:/ama/dist/,AVAILABLE} 2017-09-30 09:22:46.317:INFO:oejs.ServerConnector:main: Started ServerConnector@1dac5ef{HTTP/1.1}{0.0.0.0:8000} 2017-09-30 09:22:46.317:INFO:oejs.Server:main: Started @822ms SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. I0930 09:22:46.591053 5481 sched.cpp:232] Version: 1.4.0 I0930 09:22:46.592723 5514 sched.cpp:336] New master detected at master@192.168.33.11:5050 I0930 09:22:46.593138 5514 sched.cpp:352] No credentials provided. Attempting to register without authentication I0930 09:22:46.596042 5511 sched.cpp:759] Framework registered with a82be91b-d3fb-4ccd-9dc3-145f60ac8316-0000 ===> Executor 0000000002-6bfdbb03-88b7-44af-9cd0-2a12ddda7b62 registered ===> a provider for group spark was created ===> launching task: 0000000002 ===> ================= started action start ================= ===> val data = 1 to 1000 ===> val rdd = sc.parallelize(data) ===> val odd = rdd.filter(n => n%2 != 0).toDF("number") ===> ================= finished action start ================= ===> complete task: 0000000002 ===> launching task: 0000000003 ===> Executor 0000000003-6f7eb9e9-8977-4d47-906e-e0802bb7ffe8 registered ===> a provider for group spark was created ===> launching task: 0000000003 ===> Executor 0000000003-605f3b31-e15e-482d-8c93-737f1605e9f6 registered ===> a provider for group spark was created ===> launching task: 0000000003 ===> moving to err action null 2017-09-30 09:24:07.355:INFO:oejs.ServerConnector:Thread-59: Stopped ServerConnector@1dac5ef{HTTP/1.1}{0.0.0.0:8000} 2017-09-30 09:24:07.359:INFO:oejsh.ContextHandler:Thread-59: Stopped o.e.j.s.ServletContextHandler@385e9564{/,file:/ama/dist/,UNAVAILABLE} I0930 09:24:07.361192 5512 sched.cpp:2021] Asked to stop the driver I0930 09:24:07.361409 5512 sched.cpp:1203] Stopping framework a82be91b-d3fb-4ccd-9dc3-145f60ac8316-0000 kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec] W00t amaterasu job is finished!!!
