Actually you are launching app in local mode.
Instead of
launch -local /home/cloudera/myapexapp-1.0-SNAPSHOT.apa
launch as:
launch /home/cloudera/myapexapp-1.0-SNAPSHOT.apa
Then it should launch an application using Yarn.
Thanks & Regards,
Vikram
On Fri, May 5, 2017 at 5:04 PM, Sushil Apex <sushil.aa...@gmail.com> wrote:
> @tushar
>
> I have used apex-cli in one terminal and launched my application with
> below command, in another terminal, i checked for application id but it is
> coming zero active applications, I must be missing some thing here.
>
>
>
> apex> launch -local /home/cloudera/myapexapp-1.0-SNAPSHOT.apa
> 1. Kafka2HDFS2
> 2. MyFirstApplication
> Choose application: 1
>
>
>
> Apex CLI 3.5.0 06.12.2016 @ 22:11:51 PST rev: 6de8828 branch:
> 6de8828e4f3d5734d0a6f9c1be0aa7057cb60ac8
> apex> list-apps *
> {"apps": []}
> 0 active, total 0 applications.
>
>
> @chaitanya
> hdfs and yarn are running, but the container folder is blank
>
> [cloudera@quickstart bin]$ cd /var/log/hadoop-yarn/container
> [cloudera@quickstart container]$ ls
> [cloudera@quickstart container]$
>
>
>
> On Fri, May 5, 2017 at 4:41 PM, Tushar Gosavi <tus...@datatorrent.com>
> wrote:
>
>> If log aggregations is enabled you can get the logs using yarn logs
>> command. you can get the application id using apex-cli.
>>
>> - Tushar.
>>
>>
>> On Fri, May 5, 2017 at 4:33 PM, Sushil Apex <sushil.aa...@gmail.com>
>> wrote:
>>
>>> To add here
>>> I am using Cloudera VM for hdfs and running Kafka from apache with
>>> single machine zookeeper
>>>
>>> On Fri, May 5, 2017 at 4:32 PM, Sushil Apex <sushil.aa...@gmail.com>
>>> wrote:
>>>
>>>> @vikram, I am using apex cli
>>>> not sure where ts logs will be generated, checked /var/log/hadoop-yarn
>>>> but did found any updated logs for apex
>>>>
>>>> I have changed the application name but that didn't helped
>>>>
>>>> On Fri, May 5, 2017 at 4:03 PM, vikram patil <patilvik...@gmail.com>
>>>> wrote:
>>>>
>>>>> Can you share your Apex logs? Have you launched an application using
>>>>> apex cli?
>>>>>
>>>>> Also, can you try once by changing name of your application and
>>>>> relaunch again. Please kill or shut down your existing running
>>>>> application before this.
>>>>>
>>>>> -Vikram
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, May 5, 2017 at 3:56 PM, Sushil Apex <sushil.aa...@gmail.com>
>>>>> wrote:
>>>>> > Tried with EARLIEST option also, no luck :(
>>>>> >
>>>>> > On Fri, May 5, 2017 at 3:49 PM, Sushil Apex <sushil.aa...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> @Vikram the folder /tmp/FromKafka is already created.
>>>>> >> @Chaitanya yes I am pusing messages in kafka topic using
>>>>> consoleProducer
>>>>> >> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic
>>>>> >> kafka2hdfs
>>>>> >>
>>>>> >> @Ajay
>>>>> >> will try with the initial offset to EARLIEST.
>>>>> >>
>>>>> >> Thank you
>>>>> >> Sushil
>>>>> >>
>>>>> >> On Fri, May 5, 2017 at 2:54 PM, AJAY GUPTA <ajaygit...@gmail.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> Also, you may be required to specify the initial offset to
>>>>> EARLIEST.
>>>>> >>>
>>>>> >>> Ajay
>>>>> >>>
>>>>> >>> On Fri, May 5, 2017 at 2:35 PM, vikram patil <
>>>>> patilvik...@gmail.com>
>>>>> >>> wrote:
>>>>> >>>>
>>>>> >>>> Hi Sushil,
>>>>> >>>>
>>>>> >>>> Have you provided configuration specifying hdfs directory and
>>>>> file for
>>>>> >>>> an application?
>>>>> >>>> You may have to create /tmp/fromKafka directory in hdfs.
>>>>> >>>> Thanks & Regards,
>>>>> >>>> Vikram
>>>>> >>>>
>>>>> >>>> On Fri, May 5, 2017 at 2:30 PM, Sushil Apex <
>>>>> sushil.aa...@gmail.com>
>>>>> >>>> wrote:
>>>>> >>>> >
>>>>> >>>> > I am using Apex 3.5.0 and kafka 0.9
>>>>> >>>> > malhar library used is 3.6.0
>>>>> >>>> >
>>>>> >>>> > I am following the example from
>>>>> >>>> > https://github.com/DataTorrent/examples/blob/master/tutorial
>>>>> s/kafka/src/main/java/com/example/myapexapp/KafkaApp.java
>>>>> >>>> >
>>>>> >>>> > I am putting messages in Kafka topic using consolekafkaProducer
>>>>> >>>> > provided by Kafka, but I am not able to read these messages in
>>>>> Apex
>>>>> >>>> > DAG(created based on above link).
>>>>> >>>> >
>>>>> >>>> > I am running the apex apa file through apex-cli
>>>>> >>>> >
>>>>> >>>> > Apex CLI 3.5.0 06.12.2016 @ 22:11:51 PST rev: 6de8828 branch:
>>>>> >>>> > 6de8828e4f3d5734d0a6f9c1be0aa7057cb60ac8
>>>>> >>>> > apex> launch --local /home/cloudera/myapexapp-1.0-SNAPSHOT.apa
>>>>> >>>> > 1. Kafka2HDFS
>>>>> >>>> > 2. MyFirstApplication
>>>>> >>>> > Choose application: 1
>>>>> >>>> >
>>>>> >>>> > and nothing happens after this, I can see the messages put in
>>>>> >>>> > consoleConsumer in kafka logs
>>>>> >>>> >
>>>>> >>>> > Properties used are
>>>>> >>>> >
>>>>> >>>> > <property>
>>>>> >>>> > <name>dt.operator.kafkaIn.prop.topics</name>
>>>>> >>>> > <value>kafka2hdfs</value>
>>>>> >>>> > </property>
>>>>> >>>> >
>>>>> >>>> > <property>
>>>>> >>>> > <name>dt.operator.kafkaIn.prop.consumer.zookeeper</name>
>>>>> >>>> > <value>localhost:2181</value>
>>>>> >>>> > </property>
>>>>> >>>> > <property>
>>>>> >>>> > <name>dt.operator.kafkaIn.prop.clusters</name>
>>>>> >>>> > <value>localhost:9092</value>
>>>>> >>>> > </property>
>>>>> >>>> > <property>
>>>>> >>>> > <name>dt.operator.kafkaIn.prop.initialPartitionCount</name>
>>>>> >>>> > <value>1</value>
>>>>> >>>> > </property>
>>>>> >>>> >
>>>>> >>>> >
>>>>> >>>> > Application Code
>>>>> >>>> >
>>>>> >>>> > KafkaSinglePortByteArrayInputOperator in
>>>>> >>>> > = dag.addOperator("kafkaIn", new
>>>>> >>>> > KafkaSinglePortByteArrayInputOperator());
>>>>> >>>> >
>>>>> >>>> > LineOutputOperator out = dag.addOperator("fileOut", new
>>>>> >>>> > LineOutputOperator());
>>>>> >>>> >
>>>>> >>>> > dag.addStream("dataf", in.outputPort, out.input);
>>>>> >>>> >
>>>>> >>>> >
>>>>> >>>> > I am not able to understand what config I am missing here?
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>
