Re: Getting partition null or empty error @Apache Apex-Kafka Integration

[Tushar Gosavi]

Hi Anuj,

I have observed similar problem in the past. most probably the
application master is unable to connect to kafka cluster.
can you check if you are using correct version of kafka client, And
your kafka cluster is accessible with configuration
you have provided.

Regards,
-Tushar.


On Tue, Jul 12, 2016 at 11:00 AM, ANUJ THAKWANI  wrote:
> Hi Ram,
>
> Attaching Application.java for your reference.
>
> Application fails after 7-8 seconds.
>
>
>
> On Tue, Jul 12, 2016 at 10:55 AM, Munagala Ramanath 
> wrote:
>>
>> Anuj,
>>
>> Could you provide additional details such as: What does your DAG look like
>> ? What operators (custom as well as from Malhar)
>> are you using ? Does this exception happen immediately upon launch or
>> after
>> some time ?
>>
>> Ram
>>
>> On Mon, Jul 11, 2016 at 10:12 PM, ANUJ THAKWANI 
>> wrote:
>>
>> > Hi,
>> >
>> > I have subscribed to the dev list. Let me know the way forward.
>> >
>> > I am a newbie in this project .
>> >
>> > On Tue, Jul 12, 2016 at 10:31 AM, Amol Kekre 
>> > wrote:
>> >
>> > >
>> > > Anuj,
>> > > You are not yet subscribed to dev list, and hence will have trouble
>> > > posting to this forum. Please subscribe by clicking on "subscribe" on
>> > > the
>> > > dev list as listed on http://apex.apache.org/community.html.
>> > >
>> > > OR you can send an email to dev-subscr...@apex.apache.org with a
>> > > subject
>> > > text.
>> > >
>> > > Thks
>> > > Amol
>> > >
>> > >
>> > > On Mon, Jul 11, 2016 at 9:39 PM, ANUJ THAKWANI
>> > > 
>> > > wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> I am getting below given exception while launching kafka Application.
>> > >> Please let me know how to fix this.
>> > >>
>> > >> java.lang.IllegalStateException: Partitioner returns null or empty.
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>> > >> at
>> > >>
>> > >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
>> > >> 2016-07-11 22:24:48,250 ERROR
>> > >> com.datatorrent.stram.StreamingAppMaster: Exiting Application Master
>> > >> java.lang.IllegalStateException: Partitioner returns null or empty.
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>> > >> at
>> > >>
>> > >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> > >> at
>> > >>
>> >
>> > com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
>> > >>
>> > >
>> > >
>> >
>
>

[jira] [Commented] (APEXCORE-489) Getting exception while launching kafka application

[Anuj (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372212#comment-15372212
 ] 

Anuj commented on APEXCORE-489:
---

Attaching Application.java for your reference.

Application fails after 7-8 seconds



 @Override
  public void populateDAG(DAG dag, Configuration conf)
  {
// Sample DAG with 2 operators
// Replace this code with the DAG you want to build

  KafkaSinglePortStringInputOperator input =  
dag.addOperator("MessageReader", new KafkaSinglePortStringInputOperator());
ConsoleOutputOperator cons = dag.addOperator("console", new 
ConsoleOutputOperator());

dag.addStream("randomData", input.outputPort, 
cons.input).setLocality(Locality.CONTAINER_LOCAL);
  }

> Getting exception while launching kafka application
> ---
>
> Key: APEXCORE-489
> URL: https://issues.apache.org/jira/browse/APEXCORE-489
> Project: Apache Apex Core
>  Issue Type: Question
>Reporter: Anuj
>
> Getting below given exception while launching kafka application.
> 2016-07-12 10:33:02,243 INFO com.datatorrent.stram.StreamingAppMaster: 
> Initializing Application Master.
> 2016-07-12 10:33:02,263 INFO com.datatorrent.stram.StreamingAppMasterService: 
> Application master, appId=158, clustertimestamp=1465548058955, attemptId=1
> 2016-07-12 10:33:03,779 INFO 
> com.datatorrent.contrib.kafka.AbstractKafkaInputOperator: [ONE_TO_ONE]: 
> Initializing partition(s)
> 2016-07-12 10:33:03,781 INFO org.apache.hadoop.service.AbstractService: 
> Service com.datatorrent.stram.StreamingAppMasterService failed in state 
> INITED; cause: java.lang.IllegalStateException: Partitioner returns null or 
> empty.
> java.lang.IllegalStateException: Partitioner returns null or empty.
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
> 2016-07-12 10:33:03,784 ERROR com.datatorrent.stram.StreamingAppMaster: 
> Exiting Application Master
> java.lang.IllegalStateException: Partitioner returns null or empty.
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: How to specify affinity rules from application properties file

[Akshay Gore]

There was a typo in the application name because of which the property was
not applying properly.

On Tue, Jul 12, 2016 at 10:33 AM, Munagala Ramanath 
wrote:

> Could you share how you resolved the issue ?
>
> Ram
>
> On Mon, Jul 11, 2016 at 9:59 PM, Akshay Gore 
> wrote:
>
> > Thank you Pradeep for the response. I have already resolved the issue.
> >
> > Regards,
> > Akshay
> >
> > On Tue, Jul 12, 2016 at 10:10 AM, Pradeep Kumbhar <
> prad...@datatorrent.com
> > >
> > wrote:
> >
> > > Should
> > > "dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET"
> > > be
> > > "dt.application.PiDemo.attr.AFFINITY_RULES_SET" ?
> > > or it's just an example.
> > >
> > >
> > > On Mon, Jul 11, 2016 at 4:02 PM, Akshay Gore 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am trying to explore affinity feature in Apex. For this, I am using
> > > > pi-demo application from apex-malhar. As per the affinity document
> > > > <
> > > >
> > >
> >
> https://github.com/apache/apex-core/blob/master/docs/application_development.md
> > > > >,
> > > > I have updated the demo application's properties file as follows:
> > > >
> > > > 
> > > >
> > > >
> > >
> >
> dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET
> > > >   
> > > > {
> > > >   "affinityRules": [
> > > >  {
> > > >   "operatorsList": [
> > > >  "rand",
> > > >  "picalc"
> > > >],
> > > >"locality":
> > > > "NODE_LOCAL",
> > > >"type":
> "AFFINITY",
> > > >"relaxLocality":
> > false
> > > >  }
> > > >]
> > > >}
> > > > 
> > > >
> > > >
> > > > After launching the demo application, I don't see containers for
> > operator
> > > > "rand" and "picalc" on the same node. Although it's working fine
> when I
> > > set
> > > > the rule from application code. Can anyone please help me here?
> > > >
> > > > Thanks,
> > > > Akshay
> > > >
> > >
> > >
> > >
> > > --
> > > *regards,*
> > > *~pradeep*
> > >
> >
>

[jira] [Commented] (APEXCORE-489) Getting exception while launching kafka application

[Munagala V. Ramanath (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372195#comment-15372195
 ] 

Munagala V. Ramanath commented on APEXCORE-489:
---

Could you provide additional details such as: What does your DAG look like ? 
What operators (custom as well as from Malhar)
are you using ? Does this exception happen immediately upon launch or after 
some time ?

> Getting exception while launching kafka application
> ---
>
> Key: APEXCORE-489
> URL: https://issues.apache.org/jira/browse/APEXCORE-489
> Project: Apache Apex Core
>  Issue Type: Question
>Reporter: Anuj
>
> Getting below given exception while launching kafka application.
> 2016-07-12 10:33:02,243 INFO com.datatorrent.stram.StreamingAppMaster: 
> Initializing Application Master.
> 2016-07-12 10:33:02,263 INFO com.datatorrent.stram.StreamingAppMasterService: 
> Application master, appId=158, clustertimestamp=1465548058955, attemptId=1
> 2016-07-12 10:33:03,779 INFO 
> com.datatorrent.contrib.kafka.AbstractKafkaInputOperator: [ONE_TO_ONE]: 
> Initializing partition(s)
> 2016-07-12 10:33:03,781 INFO org.apache.hadoop.service.AbstractService: 
> Service com.datatorrent.stram.StreamingAppMasterService failed in state 
> INITED; cause: java.lang.IllegalStateException: Partitioner returns null or 
> empty.
> java.lang.IllegalStateException: Partitioner returns null or empty.
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
> 2016-07-12 10:33:03,784 ERROR com.datatorrent.stram.StreamingAppMaster: 
> Exiting Application Master
> java.lang.IllegalStateException: Partitioner returns null or empty.
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>   at 
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Getting partition null or empty error @Apache Apex-Kafka Integration

[Munagala Ramanath]

Anuj,

Could you provide additional details such as: What does your DAG look like
? What operators (custom as well as from Malhar)
are you using ? Does this exception happen immediately upon launch or after
some time ?

Ram

On Mon, Jul 11, 2016 at 10:12 PM, ANUJ THAKWANI 
wrote:

> Hi,
>
> I have subscribed to the dev list. Let me know the way forward.
>
> I am a newbie in this project .
>
> On Tue, Jul 12, 2016 at 10:31 AM, Amol Kekre  wrote:
>
> >
> > Anuj,
> > You are not yet subscribed to dev list, and hence will have trouble
> > posting to this forum. Please subscribe by clicking on "subscribe" on the
> > dev list as listed on http://apex.apache.org/community.html.
> >
> > OR you can send an email to dev-subscr...@apex.apache.org with a subject
> > text.
> >
> > Thks
> > Amol
> >
> >
> > On Mon, Jul 11, 2016 at 9:39 PM, ANUJ THAKWANI 
> > wrote:
> >
> >> Hi,
> >>
> >> I am getting below given exception while launching kafka Application.
> >> Please let me know how to fix this.
> >>
> >> java.lang.IllegalStateException: Partitioner returns null or empty.
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
> >> at
> >>
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
> >> at
> >>
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
> >> at
> >>
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
> >> at
> >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> >> at
> >>
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
> >> 2016-07-11 22:24:48,250 ERROR
> >> com.datatorrent.stram.StreamingAppMaster: Exiting Application Master
> >> java.lang.IllegalStateException: Partitioner returns null or empty.
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
> >> at
> >>
> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
> >> at
> >>
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
> >> at
> >>
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
> >> at
> >>
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
> >> at
> >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> >> at
> >>
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
> >>
> >
> >
>

[jira] [Created] (APEXCORE-489) Getting exception while launching kafka application

[Anuj (JIRA)]

Anuj created APEXCORE-489:
-

 Summary: Getting exception while launching kafka application
 Key: APEXCORE-489
 URL: https://issues.apache.org/jira/browse/APEXCORE-489
 Project: Apache Apex Core
  Issue Type: Question
Reporter: Anuj


Getting below given exception while launching kafka application.

2016-07-12 10:33:02,243 INFO com.datatorrent.stram.StreamingAppMaster: 
Initializing Application Master.
2016-07-12 10:33:02,263 INFO com.datatorrent.stram.StreamingAppMasterService: 
Application master, appId=158, clustertimestamp=1465548058955, attemptId=1
2016-07-12 10:33:03,779 INFO 
com.datatorrent.contrib.kafka.AbstractKafkaInputOperator: [ONE_TO_ONE]: 
Initializing partition(s)
2016-07-12 10:33:03,781 INFO org.apache.hadoop.service.AbstractService: Service 
com.datatorrent.stram.StreamingAppMasterService failed in state INITED; cause: 
java.lang.IllegalStateException: Partitioner returns null or empty.
java.lang.IllegalStateException: Partitioner returns null or empty.
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
at 
com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
at 
com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
2016-07-12 10:33:03,784 ERROR com.datatorrent.stram.StreamingAppMaster: Exiting 
Application Master
java.lang.IllegalStateException: Partitioner returns null or empty.
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
at 
com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
at 
com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Getting partition null or empty error @Apache Apex-Kafka Integration

[ANUJ THAKWANI]

Hi,

I have subscribed to the dev list. Let me know the way forward.

I am a newbie in this project .

On Tue, Jul 12, 2016 at 10:31 AM, Amol Kekre  wrote:

>
> Anuj,
> You are not yet subscribed to dev list, and hence will have trouble
> posting to this forum. Please subscribe by clicking on "subscribe" on the
> dev list as listed on http://apex.apache.org/community.html.
>
> OR you can send an email to dev-subscr...@apex.apache.org with a subject
> text.
>
> Thks
> Amol
>
>
> On Mon, Jul 11, 2016 at 9:39 PM, ANUJ THAKWANI 
> wrote:
>
>> Hi,
>>
>> I am getting below given exception while launching kafka Application.
>> Please let me know how to fix this.
>>
>> java.lang.IllegalStateException: Partitioner returns null or empty.
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>> at
>> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>> at
>> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>> at
>> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
>> 2016-07-11 22:24:48,250 ERROR
>> com.datatorrent.stram.StreamingAppMaster: Exiting Application Master
>> java.lang.IllegalStateException: Partitioner returns null or empty.
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
>> at
>> com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
>> at
>> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
>> at
>> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>> at
>> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>> at
>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>> at
>> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
>>
>
>

Re: Running application built against custom Malhar version

[Priyanka Gugale]

As per maven documentation, the "system" scope is similar to "provided"
scope. That means jre or platform is suppose to provide the jar at runtime.
Normally, Malhar library jar is included in uber jar and is not at provided
scope when we build an apa. That's why your application can't locate the
classes at runtime.

Why you are using system scope? You can simply "install" the malhar-library
in your local repository and your application will pick it from there.

-Priyanka

On Mon, Jul 11, 2016 at 9:58 PM, Ganelin, Ilya 
wrote:

> I’m attempting to build a new application leveraging a custom Malhar
> library. I’ve compiled the updated Malhar library, confirmed that the
> classes I’ve added are present, and added this specific library as a system
> dependency in my pom of the newly created application:
>
> 
>   org.apache.apex
>   malhar-library
>   3.5.0-SNAPSHOT
>   system
>
> code/apex-malhar/library/target/malhar-library-3.5.0-SNAPSHOT.jar
> 
>
> However, when I then build this new application and upload the generated
> APA to the cluster and attempt to run it with the DataTorrent UI, I get a
> NoClassDefFoundError. I would appreciate some help understanding what I’m
> doing incorrectly – I believe the dependency on the newly added class
> should be provided since the APA is an UberJar. Am I missing something
> obvious here?
>
> An error occurred trying to launch the application. Server message:
> java.lang.NoClassDefFoundError:
> com/datatorrent/lib/stream/MultipleStreamMerger at
> com.example.merger.Application.populateDAG(Application.java:41) at
> com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(LogicalPlanConfiguration.java:1171)
> at
> com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLauncher.java:404)
> at
> com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:479)
> at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452) at
> com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104) at
> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893) at
> com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447) Caused by:
> java.lang.ClassNotFoundException:
> com.datatorrent.lib.stream.MultipleStreamMerger at
> java.net.URLClassLoader$1.run(URLClassLoader.java:366) at
> java.net.URLClassLoader$1.run(URLClassLoader.java:355) at
> java.security.AccessController.doPrivileged(Native Method) at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:425) at
> java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 9 more Fatal
> error encountered
>
>
>
> 
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Re: How to specify affinity rules from application properties file

[Munagala Ramanath]

Could you share how you resolved the issue ?

Ram

On Mon, Jul 11, 2016 at 9:59 PM, Akshay Gore  wrote:

> Thank you Pradeep for the response. I have already resolved the issue.
>
> Regards,
> Akshay
>
> On Tue, Jul 12, 2016 at 10:10 AM, Pradeep Kumbhar  >
> wrote:
>
> > Should
> > "dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET"
> > be
> > "dt.application.PiDemo.attr.AFFINITY_RULES_SET" ?
> > or it's just an example.
> >
> >
> > On Mon, Jul 11, 2016 at 4:02 PM, Akshay Gore 
> > wrote:
> >
> > > Hi,
> > >
> > > I am trying to explore affinity feature in Apex. For this, I am using
> > > pi-demo application from apex-malhar. As per the affinity document
> > > <
> > >
> >
> https://github.com/apache/apex-core/blob/master/docs/application_development.md
> > > >,
> > > I have updated the demo application's properties file as follows:
> > >
> > > 
> > >
> > >
> >
> dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET
> > >   
> > > {
> > >   "affinityRules": [
> > >  {
> > >   "operatorsList": [
> > >  "rand",
> > >  "picalc"
> > >],
> > >"locality":
> > > "NODE_LOCAL",
> > >"type": "AFFINITY",
> > >"relaxLocality":
> false
> > >  }
> > >]
> > >}
> > > 
> > >
> > >
> > > After launching the demo application, I don't see containers for
> operator
> > > "rand" and "picalc" on the same node. Although it's working fine when I
> > set
> > > the rule from application code. Can anyone please help me here?
> > >
> > > Thanks,
> > > Akshay
> > >
> >
> >
> >
> > --
> > *regards,*
> > *~pradeep*
> >
>

Re: How to specify affinity rules from application properties file

[Akshay Gore]

Thank you Pradeep for the response. I have already resolved the issue.

Regards,
Akshay

On Tue, Jul 12, 2016 at 10:10 AM, Pradeep Kumbhar 
wrote:

> Should
> "dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET"
> be
> "dt.application.PiDemo.attr.AFFINITY_RULES_SET" ?
> or it's just an example.
>
>
> On Mon, Jul 11, 2016 at 4:02 PM, Akshay Gore 
> wrote:
>
> > Hi,
> >
> > I am trying to explore affinity feature in Apex. For this, I am using
> > pi-demo application from apex-malhar. As per the affinity document
> > <
> >
> https://github.com/apache/apex-core/blob/master/docs/application_development.md
> > >,
> > I have updated the demo application's properties file as follows:
> >
> > 
> >
> >
> dt.application.AffinityRulesSampleApplication.attr.AFFINITY_RULES_SET
> >   
> > {
> >   "affinityRules": [
> >  {
> >   "operatorsList": [
> >  "rand",
> >  "picalc"
> >],
> >"locality":
> > "NODE_LOCAL",
> >"type": "AFFINITY",
> >"relaxLocality": false
> >  }
> >]
> >}
> > 
> >
> >
> > After launching the demo application, I don't see containers for operator
> > "rand" and "picalc" on the same node. Although it's working fine when I
> set
> > the rule from application code. Can anyone please help me here?
> >
> > Thanks,
> > Akshay
> >
>
>
>
> --
> *regards,*
> *~pradeep*
>

Getting partition null or empty error @Apache Apex-Kafka Integration

[ANUJ THAKWANI]

Hi,

I am getting below given exception while launching kafka Application.
Please let me know how to fix this.

java.lang.IllegalStateException: Partitioner returns null or empty.
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
at 
com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
at 
com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)
2016-07-11 22:24:48,250 ERROR
com.datatorrent.stram.StreamingAppMaster: Exiting Application Master
java.lang.IllegalStateException: Partitioner returns null or empty.
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.initPartitioning(PhysicalPlan.java:605)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.addLogicalOperator(PhysicalPlan.java:1497)
at 
com.datatorrent.stram.plan.physical.PhysicalPlan.(PhysicalPlan.java:344)
at 
com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:362)
at 
com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)

[jira] [Commented] (APEXMALHAR-2019) Creation of S3 Input Module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372154#comment-15372154
 ] 

ASF GitHub Bot commented on APEXMALHAR-2019:


Github user yogidevendra commented on a diff in the pull request:

https://github.com/apache/apex-malhar/pull/263#discussion_r70375602
  
--- Diff: 
library/src/main/java/com/datatorrent/lib/io/fs/S3BlockReader.java ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package com.datatorrent.lib.io.fs;
+
+import java.io.IOException;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.Path;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.datatorrent.api.Context;
+import com.datatorrent.lib.io.block.BlockMetadata;
+import com.datatorrent.lib.io.block.BlockReader;
+import com.datatorrent.lib.io.block.ReaderContext;
+
+/**
+ * S3BlockReader extends from BlockReader and serves the functionality of 
read objects and
+ * parse Block metadata
+ */
+public class S3BlockReader extends BlockReader
+{
+  protected transient String s3bucketUri;
+
+  public S3BlockReader()
+  {
+this.readerContext = new S3BlockReaderContext();
+  }
+
+  @Override
+  public void setup(Context.OperatorContext context)
+  {
+super.setup(context);
+s3bucketUri = fs.getScheme() + "://" + extractBucket(uri);
+  }
+
+  @VisibleForTesting
+  protected String extractBucket(String s3uri)
+  {
+return s3uri.substring(s3uri.indexOf('@') + 1, s3uri.indexOf("/", 
s3uri.indexOf('@')));
+  }
+
+  @Override
+  protected FSDataInputStream setupStream(BlockMetadata.FileBlockMetadata 
block) throws IOException
+  {
+FSDataInputStream ins = fs.open(new Path(s3bucketUri + 
block.getFilePath()));
+ins.seek(block.getOffset());
+return ins;
+  }
+
+  /**
+   * BlockReadeContext for reading S3 Blocks.
+   * This should use read API without offset.
+   */
+  private static class S3BlockReaderContext extends 
ReaderContext.FixedBytesReaderContext
+  {
+@Override
+protected Entity readEntity() throws IOException
--- End diff --

@chaithu14 Could you please add javadoc to readEntity() mentioning the 
difference between the super implementation?


> Creation of S3 Input Module
> ---
>
> Key: APEXMALHAR-2019
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2019
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: Chaitanya
>Assignee: Chaitanya
>Priority: Minor
>
> This module serves the functionality of parallel read files from S3 bucket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #263: APEXMALHAR-2019 S3-Input Implemented S3 Input...

[yogidevendra]

Github user yogidevendra commented on a diff in the pull request:

https://github.com/apache/apex-malhar/pull/263#discussion_r70375602
  
--- Diff: 
library/src/main/java/com/datatorrent/lib/io/fs/S3BlockReader.java ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package com.datatorrent.lib.io.fs;
+
+import java.io.IOException;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.Path;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.datatorrent.api.Context;
+import com.datatorrent.lib.io.block.BlockMetadata;
+import com.datatorrent.lib.io.block.BlockReader;
+import com.datatorrent.lib.io.block.ReaderContext;
+
+/**
+ * S3BlockReader extends from BlockReader and serves the functionality of 
read objects and
+ * parse Block metadata
+ */
+public class S3BlockReader extends BlockReader
+{
+  protected transient String s3bucketUri;
+
+  public S3BlockReader()
+  {
+this.readerContext = new S3BlockReaderContext();
+  }
+
+  @Override
+  public void setup(Context.OperatorContext context)
+  {
+super.setup(context);
+s3bucketUri = fs.getScheme() + "://" + extractBucket(uri);
+  }
+
+  @VisibleForTesting
+  protected String extractBucket(String s3uri)
+  {
+return s3uri.substring(s3uri.indexOf('@') + 1, s3uri.indexOf("/", 
s3uri.indexOf('@')));
+  }
+
+  @Override
+  protected FSDataInputStream setupStream(BlockMetadata.FileBlockMetadata 
block) throws IOException
+  {
+FSDataInputStream ins = fs.open(new Path(s3bucketUri + 
block.getFilePath()));
+ins.seek(block.getOffset());
+return ins;
+  }
+
+  /**
+   * BlockReadeContext for reading S3 Blocks.
+   * This should use read API without offset.
+   */
+  private static class S3BlockReaderContext extends 
ReaderContext.FixedBytesReaderContext
+  {
+@Override
+protected Entity readEntity() throws IOException
--- End diff --

@chaithu14 Could you please add javadoc to readEntity() mentioning the 
difference between the super implementation?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372065#comment-15372065
 ] 

ASF GitHub Bot commented on APEXCORE-488:
-

Github user vrozov commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70368327
  
--- Diff: 
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -608,13 +609,16 @@ protected void serviceStart() throws Exception
 
 try {
   Configuration config = getConfig();
+  YarnConfiguration conf = new YarnConfiguration();
--- End diff --

Why is it necessary to have Configuration and YarnConfiguration? Can this 
code be simplified to `YarnConfiguration = 
StramClientUtils.getYarnConfiguration(getConfig());`


> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #357: APEXCORE-488: Issues in SSL communication with ...

[vrozov]

Github user vrozov commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70368327
  
--- Diff: 
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -608,13 +609,16 @@ protected void serviceStart() throws Exception
 
 try {
   Configuration config = getConfig();
+  YarnConfiguration conf = new YarnConfiguration();
--- End diff --

Why is it necessary to have Configuration and YarnConfiguration? Can this 
code be simplified to `YarnConfiguration = 
StramClientUtils.getYarnConfiguration(getConfig());`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-core pull request #356: APEXCORE-487 - Upgrade apex and malhar dependen...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/356


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (APEXCORE-485) Upgrade maven surefire plugin to the latest version

[David Yan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Yan resolved APEXCORE-485.

   Resolution: Fixed
Fix Version/s: 3.5.0

> Upgrade maven surefire plugin to the latest version
> ---
>
> Key: APEXCORE-485
> URL: https://issues.apache.org/jira/browse/APEXCORE-485
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.5.0
>
>
> maven-surefire-plugin 2.14.1 is 3 years old and there is significant amount 
> of bug fixes and enhancements provided by maven-surefire-plugin and junit 4.7 
> provider.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-485) Upgrade maven surefire plugin to the latest version

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371963#comment-15371963
 ] 

ASF GitHub Bot commented on APEXCORE-485:
-

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/355


> Upgrade maven surefire plugin to the latest version
> ---
>
> Key: APEXCORE-485
> URL: https://issues.apache.org/jira/browse/APEXCORE-485
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.5.0
>
>
> maven-surefire-plugin 2.14.1 is 3 years old and there is significant amount 
> of bug fixes and enhancements provided by maven-surefire-plugin and junit 4.7 
> provider.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #355: APEXCORE-485 - Upgrade maven surefire plugin to...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/355


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

[sandeshh]

Github user sandeshh closed the pull request at:

https://github.com/apache/apex-core/pull/350


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371959#comment-15371959
 ] 

ASF GitHub Bot commented on APEXCORE-222:
-

Github user sandeshh closed the pull request at:

https://github.com/apache/apex-core/pull/350


> Delegate Buffer Server purge to StreamingContainer
> --
>
> Key: APEXCORE-222
> URL: https://issues.apache.org/jira/browse/APEXCORE-222
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Thomas Weise
>Assignee: Sandesh
>
> Currently the purge requests are sent to the buffer servers from the app 
> master. This interaction exists parallel to the heartbeat protocol. Instead, 
> the committed window ID that is propagated through the heartbeat response can 
> be used in StreamingContainer to initiate the purge with the local buffer 
> server, similar to how the committed callback on the operator occurs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371960#comment-15371960
 ] 

ASF GitHub Bot commented on APEXCORE-222:
-

GitHub user sandeshh reopened a pull request:

https://github.com/apache/apex-core/pull/350

APEXCORE-222 Purging the stale data present in the bufferserver, dire…

…ctly from the streaming container.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sandeshh/apex-core APEXCORE-222

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/350.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #350


commit 36e3d333d0c4f22d07701cefacaacae0ab81a0eb
Author: sandeshh 
Date:   2016-06-28T03:58:07Z

APEXCORE-222 purging of the buffer server is done from the streaming 
container, instead of StreamingContainerManager




> Delegate Buffer Server purge to StreamingContainer
> --
>
> Key: APEXCORE-222
> URL: https://issues.apache.org/jira/browse/APEXCORE-222
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Thomas Weise
>Assignee: Sandesh
>
> Currently the purge requests are sent to the buffer servers from the app 
> master. This interaction exists parallel to the heartbeat protocol. Instead, 
> the committed window ID that is propagated through the heartbeat response can 
> be used in StreamingContainer to initiate the purge with the local buffer 
> server, similar to how the committed callback on the operator occurs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core issue #355: APEXCORE-485 - Upgrade maven surefire plugin to the la...

[vrozov]

Github user vrozov commented on the issue:

https://github.com/apache/apex-core/pull/355
  
@davidyan74 David, can you merge this one as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371943#comment-15371943
 ] 

ASF GitHub Bot commented on APEXCORE-488:
-

Github user pradeepdalvi commented on the issue:

https://github.com/apache/apex-core/pull/357
  
@PramodSSImmaneni Could you please review this?


> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core issue #357: APEXCORE-488: Issues in SSL communication with StrAM

[pradeepdalvi]

Github user pradeepdalvi commented on the issue:

https://github.com/apache/apex-core/pull/357
  
@PramodSSImmaneni Could you please review this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371902#comment-15371902
 ] 

ASF GitHub Bot commented on APEXCORE-488:
-

Github user pradeepdalvi commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70356242
  
--- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAgent.java 
---
@@ -204,7 +204,14 @@ private UriBuilder 
getStramWebURIBuilder(WebServicesClient webServicesClient, St
 if (info != null) {
   //ws = wsClient.resource("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
   // the filter should convert to the right version
-  ub = UriBuilder.fromUri("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
+  String url;
+  if (!info.appMasterTrackingUrl.startsWith("http://;)
+  && !info.appMasterTrackingUrl.startsWith("https://;)) {
+url = "http://; + info.appMasterTrackingUrl;
--- End diff --

Its not falling back. When protocol scheme is not specified in Tracking 
URL, it shall use the default protocol scheme i.e. 'http://'


> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #357: APEXCORE-488: Issues in SSL communication with ...

[pradeepdalvi]

Github user pradeepdalvi commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70356242
  
--- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAgent.java 
---
@@ -204,7 +204,14 @@ private UriBuilder 
getStramWebURIBuilder(WebServicesClient webServicesClient, St
 if (info != null) {
   //ws = wsClient.resource("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
   // the filter should convert to the right version
-  ub = UriBuilder.fromUri("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
+  String url;
+  if (!info.appMasterTrackingUrl.startsWith("http://;)
+  && !info.appMasterTrackingUrl.startsWith("https://;)) {
+url = "http://; + info.appMasterTrackingUrl;
--- End diff --

Its not falling back. When protocol scheme is not specified in Tracking 
URL, it shall use the default protocol scheme i.e. 'http://'


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-487) Upgrade apex and malhar dependencies in the archtype

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371897#comment-15371897
 ] 

ASF GitHub Bot commented on APEXCORE-487:
-

Github user vrozov closed the pull request at:

https://github.com/apache/apex-core/pull/356


> Upgrade apex and malhar dependencies in the archtype
> 
>
> Key: APEXCORE-487
> URL: https://issues.apache.org/jira/browse/APEXCORE-487
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-487) Upgrade apex and malhar dependencies in the archtype

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371898#comment-15371898
 ] 

ASF GitHub Bot commented on APEXCORE-487:
-

GitHub user vrozov reopened a pull request:

https://github.com/apache/apex-core/pull/356

APEXCORE-487 - Upgrade apex and malhar dependencies in the archtype

@davidyan74 Please review

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vrozov/apex-core APEXCORE-487

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #356


commit 68be2f81ca41aea885620ebd6f3cd3c8eac22b07
Author: Vlad Rozov 
Date:   2016-07-11T21:06:27Z

APEXCORE-487 - Upgrade apex and malhar dependencies in the archtype




> Upgrade apex and malhar dependencies in the archtype
> 
>
> Key: APEXCORE-487
> URL: https://issues.apache.org/jira/browse/APEXCORE-487
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #356: APEXCORE-487 - Upgrade apex and malhar dependen...

[vrozov]

Github user vrozov closed the pull request at:

https://github.com/apache/apex-core/pull/356


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (APEXCORE-484) Increase JVM PermGen size while running engine Unit test

[David Yan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Yan resolved APEXCORE-484.

   Resolution: Fixed
Fix Version/s: 3.5.0

> Increase JVM PermGen size while running engine Unit test
> 
>
> Key: APEXCORE-484
> URL: https://issues.apache.org/jira/browse/APEXCORE-484
> Project: Apache Apex Core
>  Issue Type: Task
>Affects Versions: 3.5.0
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.5.0
>
>
> Build in travis-ci environment fails with OutOfMemoryError due to 
> insufficient PermGen:
> {noformat}
> 2016-07-09 18:29:22,507 [main] INFO  event.ExecutionEventLogger mojoStarted - 
> 2016-07-09 18:29:22,507 [main] INFO  event.ExecutionEventLogger mojoStarted - 
> --- maven-assembly-plugin:2.2-beta-5:single (app-package-assembly) @ mydtapp 
> ---
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logResult - 
> 
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logResult - 
> BUILD FAILURE
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> 
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> Total time: 15.632 s
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> Finished at: 2016-07-09T18:29:29+00:00
> 2016-07-09 18:29:32,083 [main] INFO  event.ExecutionEventLogger logStats - 
> Final Memory: 141M/1787M
> 2016-07-09 18:29:32,084 [main] INFO  event.ExecutionEventLogger sessionEnded 
> - 
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 logSummary - 
> PermGen space -> [Help 1]
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 execute - 
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 execute - To 
> see the full stack trace of the errors, re-run Maven with the -e switch.
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> Re-run Maven using the -X switch to enable full debug logging.
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - For 
> more information about the errors and possible solutions, please read the 
> following articles:
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #357: APEXCORE-488: Issues in SSL communication with ...

[sandeshh]

Github user sandeshh commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70355804
  
--- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAgent.java 
---
@@ -204,7 +204,14 @@ private UriBuilder 
getStramWebURIBuilder(WebServicesClient webServicesClient, St
 if (info != null) {
   //ws = wsClient.resource("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
   // the filter should convert to the right version
-  ub = UriBuilder.fromUri("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
+  String url;
+  if (!info.appMasterTrackingUrl.startsWith("http://;)
+  && !info.appMasterTrackingUrl.startsWith("https://;)) {
+url = "http://; + info.appMasterTrackingUrl;
--- End diff --

If it is a secure cluster, then fallback should be https?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-484) Increase JVM PermGen size while running engine Unit test

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371891#comment-15371891
 ] 

ASF GitHub Bot commented on APEXCORE-484:
-

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/354


> Increase JVM PermGen size while running engine Unit test
> 
>
> Key: APEXCORE-484
> URL: https://issues.apache.org/jira/browse/APEXCORE-484
> Project: Apache Apex Core
>  Issue Type: Task
>Affects Versions: 3.5.0
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
>
> Build in travis-ci environment fails with OutOfMemoryError due to 
> insufficient PermGen:
> {noformat}
> 2016-07-09 18:29:22,507 [main] INFO  event.ExecutionEventLogger mojoStarted - 
> 2016-07-09 18:29:22,507 [main] INFO  event.ExecutionEventLogger mojoStarted - 
> --- maven-assembly-plugin:2.2-beta-5:single (app-package-assembly) @ mydtapp 
> ---
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logResult - 
> 
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logResult - 
> BUILD FAILURE
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> 
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> Total time: 15.632 s
> 2016-07-09 18:29:29,911 [main] INFO  event.ExecutionEventLogger logStats - 
> Finished at: 2016-07-09T18:29:29+00:00
> 2016-07-09 18:29:32,083 [main] INFO  event.ExecutionEventLogger logStats - 
> Final Memory: 141M/1787M
> 2016-07-09 18:29:32,084 [main] INFO  event.ExecutionEventLogger sessionEnded 
> - 
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 logSummary - 
> PermGen space -> [Help 1]
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 execute - 
> 2016-07-09 18:29:32,084 [main] ERROR support.StramTestSupport$1 execute - To 
> see the full stack trace of the errors, re-run Maven with the -e switch.
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> Re-run Maven using the -X switch to enable full debug logging.
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - For 
> more information about the errors and possible solutions, please read the 
> following articles:
> 2016-07-09 18:29:32,086 [main] ERROR support.StramTestSupport$1 execute - 
> [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371892#comment-15371892
 ] 

ASF GitHub Bot commented on APEXCORE-488:
-

Github user sandeshh commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70355804
  
--- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAgent.java 
---
@@ -204,7 +204,14 @@ private UriBuilder 
getStramWebURIBuilder(WebServicesClient webServicesClient, St
 if (info != null) {
   //ws = wsClient.resource("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
   // the filter should convert to the right version
-  ub = UriBuilder.fromUri("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
+  String url;
+  if (!info.appMasterTrackingUrl.startsWith("http://;)
+  && !info.appMasterTrackingUrl.startsWith("https://;)) {
+url = "http://; + info.appMasterTrackingUrl;
--- End diff --

If it is a secure cluster, then fallback should be https?


> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #354: APEXCORE-484 - Increase JVM PermGen size while ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/354


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-core pull request #357: APEXCORE-488: Issues in SSL communication with ...

[sandeshh]

Github user sandeshh commented on a diff in the pull request:

https://github.com/apache/apex-core/pull/357#discussion_r70355340
  
--- Diff: engine/src/main/java/com/datatorrent/stram/client/StramAgent.java 
---
@@ -204,7 +204,14 @@ private UriBuilder 
getStramWebURIBuilder(WebServicesClient webServicesClient, St
 if (info != null) {
   //ws = wsClient.resource("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
   // the filter should convert to the right version
-  ub = UriBuilder.fromUri("http://; + 
info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
+  String url;
+  if (!info.appMasterTrackingUrl.startsWith("http://;)
+  && !info.appMasterTrackingUrl.startsWith("https://;)) {
+url = "http://; + info.appMasterTrackingUrl;
--- End diff --

It should be logged, saying falling back to http


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371883#comment-15371883
 ] 

ASF GitHub Bot commented on APEXCORE-488:
-

GitHub user pradeepdalvi opened a pull request:

https://github.com/apache/apex-core/pull/357

APEXCORE-488: Issues in SSL communication with StrAM

 - Fixed Application Master trackingURL
 - StramAgent shall not assume always HTTP

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pradeepdalvi/apex-core APEXCORE-488.ssl-issues

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/357.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #357


commit d8b64f323e2e9d59dda7935807893a95802f328a
Author: Pradeep A. Dalvi 
Date:   2016-07-11T22:59:32Z

APEXCORE-488: Issues in SSL communication with StrAM
 - Fixed Application Master trackingURL
 - StramAgent shall not assume always HTTP




> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #357: APEXCORE-488: Issues in SSL communication with ...

[pradeepdalvi]

GitHub user pradeepdalvi opened a pull request:

https://github.com/apache/apex-core/pull/357

APEXCORE-488: Issues in SSL communication with StrAM

 - Fixed Application Master trackingURL
 - StramAgent shall not assume always HTTP

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pradeepdalvi/apex-core APEXCORE-488.ssl-issues

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/357.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #357


commit d8b64f323e2e9d59dda7935807893a95802f328a
Author: Pradeep A. Dalvi 
Date:   2016-07-11T22:59:32Z

APEXCORE-488: Issues in SSL communication with StrAM
 - Fixed Application Master trackingURL
 - StramAgent shall not assume always HTTP




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-core issue #354: APEXCORE-484 - Increase JVM PermGen size while running...

[vrozov]

Github user vrozov commented on the issue:

https://github.com/apache/apex-core/pull/354
  
@davidyan74 This should help with other open PRs build failures. Can you 
review and merge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-malhar pull request #338: APEXMALHAR-2139

[aayushia]

GitHub user aayushia opened a pull request:

https://github.com/apache/apex-malhar/pull/338

APEXMALHAR-2139 

Emit only the keys for which values changed or were added in a given window.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aayushia/apex-malhar APEXMALHAR-2139

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/338.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #338


commit 7d36050d4602018b7f9b6cef563b01973b8ee5e6
Author: Aayushi Acharya 
Date:   2016-07-11T22:55:57Z

APEXMALHAR-2139 #resolve #comment Emit only the keys for which values 
changed or were added in a given window.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (APEXMALHAR-2139) UniqueCounter changes

[Aayushi Acharya (JIRA)]

Aayushi Acharya created APEXMALHAR-2139:
---

 Summary: UniqueCounter changes
 Key: APEXMALHAR-2139
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2139
 Project: Apache Apex Malhar
  Issue Type: Improvement
Reporter: Aayushi Acharya
 Fix For: 3.5.0


1.Update the unique counter operator from Malhar 
https://github.com/apache/apex-malhar/blob/master/library/src/main/java/com/datatorrent/lib/algo/UniqueCounter.java
 to emit one tuple at a time vs dumping a map.
2.Emit only the keys for which values changed or were added in a given window.
3.Add test cases for 1 and 2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core issue #348: [APEXMALHAR-1978] Replaced usages of {groupId} with {p...

[vrozov]

Github user vrozov commented on the issue:

https://github.com/apache/apex-core/pull/348
  
@ilganeli The issue was fixed in the Malhar project. Please see my comments 
in JIRA for details. Changes in this PR are not necessary as files modified are 
archetype prototype pom.xml files and `{groupId}` will be replaced with user 
specified groupId during archetype plugin execution.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (APEXMALHAR-1978) Replace ${groupId} with ${project.groupId} in modules and project pom

[Vlad Rozov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov resolved APEXMALHAR-1978.

   Resolution: Done
Fix Version/s: 3.4.0

Was done as part of e38e144f2b3d25289c1aef128eb0c24b63362647 commit

> Replace ${groupId} with ${project.groupId} in modules and project pom
> -
>
> Key: APEXMALHAR-1978
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-1978
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: Vlad Rozov
>Priority: Minor
> Fix For: 3.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (APEXMALHAR-2138) Multiple declaration of org.mockito.mockito-all-1.8.5 in Malhar library pom

[Vlad Rozov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov updated APEXMALHAR-2138:
---
Description: 
{noformat}
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
be unique: org.mockito:mockito-all:jar -> duplicate declaration of version 
1.8.5 @ org.apache.apex:malhar-library:[unknown-version], 
/Users/vrozov/Projects/DataTorrent/malhar/library/pom.xml, line 332, column 17
{noformat}

> Multiple declaration of org.mockito.mockito-all-1.8.5 in Malhar library pom 
> 
>
> Key: APEXMALHAR-2138
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2138
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Vlad Rozov
>Assignee: Priyanka Gugale
>Priority: Minor
>
> {noformat}
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.mockito:mockito-all:jar -> duplicate declaration of version 
> 1.8.5 @ org.apache.apex:malhar-library:[unknown-version], 
> /Users/vrozov/Projects/DataTorrent/malhar/library/pom.xml, line 332, column 17
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (APEXMALHAR-2138) Multiple declaration of org.mockito.mockito-all-1.8.5 in Malhar library pom

[Vlad Rozov (JIRA)]

Vlad Rozov created APEXMALHAR-2138:
--

 Summary: Multiple declaration of org.mockito.mockito-all-1.8.5 in 
Malhar library pom 
 Key: APEXMALHAR-2138
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2138
 Project: Apache Apex Malhar
  Issue Type: Bug
Affects Versions: 3.5.0
Reporter: Vlad Rozov
Assignee: Priyanka Gugale
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-487) Upgrade apex and malhar dependencies in the archtype

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371687#comment-15371687
 ] 

ASF GitHub Bot commented on APEXCORE-487:
-

GitHub user vrozov opened a pull request:

https://github.com/apache/apex-core/pull/356

APEXCORE-487 - Upgrade apex and malhar dependencies in the archtype

@davidyan74 Please review

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vrozov/apex-core APEXCORE-487

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #356






> Upgrade apex and malhar dependencies in the archtype
> 
>
> Key: APEXCORE-487
> URL: https://issues.apache.org/jira/browse/APEXCORE-487
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (APEXCORE-488) Issues in SSL communication with StrAM

[Pradeep A. Dalvi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep A. Dalvi updated APEXCORE-488:
--
Assignee: Pradeep A. Dalvi

> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-488) Issues in SSL communication with StrAM

[Pradeep A. Dalvi (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371644#comment-15371644
 ] 

Pradeep A. Dalvi commented on APEXCORE-488:
---

I would like to fix these issues.

> Issues in SSL communication with StrAM
> --
>
> Key: APEXCORE-488
> URL: https://issues.apache.org/jira/browse/APEXCORE-488
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>
> Couple of issues in SSL communication with StrAM to track application progress
>  - trackingURL without protocol scheme, makes YARN to pick up default HTTP. 
> This happens even if the yarn.http.policy is set to HTTPS_ONLY.
>  - StramAgent assumes always HTTP communication



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Bleeding edge branch ?

[Sanjay Pujare]

As the name suggests the "bleeding-edge" branch ideally should use bleeding
edge versions so I would like to see Java 8 used there (and Hadoop 3 when
it does eventually come out) to make the maintenance effort worthwhile...

On Mon, Jul 11, 2016 at 12:05 PM, David Yan  wrote:

> I'm -0 on Java 8, but I'm +1 on the rest, and I'm especially strong +1 for
> upgrading the Hadoop dependency version.
>
> Here are my reasons:
>
> - Hadoop 3 will require Java 8, but Hadoop 2.7.2 still supports Java 7 and
> there will probably be some time (I'm guessing more than one year) for
> Hadoop 3 to become GA and for major distros to support Hadoop 3. The
> maintenance effort for having two branches, one for Java 7 and one for Java
> 8 is not worth it at this time.
>
> - Apex currently uses Hadoop 2.2 dependencies, marked "provided". And
> Hadoop 2.4 has been released more than two years ago, and it added a lot of
> features in the API that Apex can make use of. Most distros already bundle
> Hadoop 2.6 or later. Although some old versions of Cloudera that include
> hadoop version earlier than 2.4 still have not reached end-of-life yet, the
> number of users using those old versions is probably very small.
>
> David
>
>
> On Mon, Jul 11, 2016 at 8:59 AM, Munagala Ramanath 
> wrote:
>
> > We've had a number of issues recently related to dependencies on old
> > versions
> > of various packages/libraries such as Hadoop itself, Google guava,
> > HTTPClient,
> > mbassador, etc.
> >
> > How about we create a "bleeding-edge" branch in both Core and Malhar
> which
> > will use the latest versions of these various dependencies, upgrade to
> Java
> > 8 so
> > we can use the new Java features, etc. ?
> >
> > This will give us an opportunity to discover these sorts of problems
> early
> > and,
> > when we are ready to pull the trigger for a major version, we have a
> branch
> > ready
> > for merge with, hopefully, minimal additional effort.
> >
> > There will be no guarantees w.r.t. this branch so people using it use it
> at
> > their own
> > risk.
> >
> > Ram
> >
>

Re: Bleeding edge branch ?

[Sandesh Hegde]

+1 with some variation

Support next version, compared to one supported by the Apex main, of the
Hadoop instead of the latest Hadoop. This makes moving the Apex main to
next version of the Hadoop easy.



On Mon, Jul 11, 2016 at 10:33 AM Sanjay Pujare 
wrote:

> strong +1 (will be nice to have some dedicated resource to maintain this
> branch)
>
> On Mon, Jul 11, 2016 at 8:59 AM, Munagala Ramanath 
> wrote:
>
> > We've had a number of issues recently related to dependencies on old
> > versions
> > of various packages/libraries such as Hadoop itself, Google guava,
> > HTTPClient,
> > mbassador, etc.
> >
> > How about we create a "bleeding-edge" branch in both Core and Malhar
> which
> > will use the latest versions of these various dependencies, upgrade to
> Java
> > 8 so
> > we can use the new Java features, etc. ?
> >
> > This will give us an opportunity to discover these sorts of problems
> early
> > and,
> > when we are ready to pull the trigger for a major version, we have a
> branch
> > ready
> > for merge with, hopefully, minimal additional effort.
> >
> > There will be no guarantees w.r.t. this branch so people using it use it
> at
> > their own
> > risk.
> >
> > Ram
> >
>

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371350#comment-15371350
 ] 

ASF GitHub Bot commented on APEXCORE-222:
-

Github user sandeshh closed the pull request at:

https://github.com/apache/apex-core/pull/350


> Delegate Buffer Server purge to StreamingContainer
> --
>
> Key: APEXCORE-222
> URL: https://issues.apache.org/jira/browse/APEXCORE-222
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Thomas Weise
>Assignee: Sandesh
>
> Currently the purge requests are sent to the buffer servers from the app 
> master. This interaction exists parallel to the heartbeat protocol. Instead, 
> the committed window ID that is propagated through the heartbeat response can 
> be used in StreamingContainer to initiate the purge with the local buffer 
> server, similar to how the committed callback on the operator occurs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (APEXMALHAR-2137) Create an Integrated With page

[Amol Kekre (JIRA)]

Amol Kekre created APEXMALHAR-2137:
--

 Summary: Create an Integrated With page
 Key: APEXMALHAR-2137
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2137
 Project: Apache Apex Malhar
  Issue Type: Task
Reporter: Amol Kekre



A page that lists the logogs of all the technologies we integrate with. This 
way users can visually see all the technologies in a single page



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Bleeding edge branch ?

[Siyuan Hua]

+1

On Mon, Jul 11, 2016 at 8:59 AM, Munagala Ramanath 
wrote:

> We've had a number of issues recently related to dependencies on old
> versions
> of various packages/libraries such as Hadoop itself, Google guava,
> HTTPClient,
> mbassador, etc.
>
> How about we create a "bleeding-edge" branch in both Core and Malhar which
> will use the latest versions of these various dependencies, upgrade to Java
> 8 so
> we can use the new Java features, etc. ?
>
> This will give us an opportunity to discover these sorts of problems early
> and,
> when we are ready to pull the trigger for a major version, we have a branch
> ready
> for merge with, hopefully, minimal additional effort.
>
> There will be no guarantees w.r.t. this branch so people using it use it at
> their own
> risk.
>
> Ram
>

[GitHub] apex-malhar pull request #309: [APEXMALHAR-2106] Support multiple streams in...

[ilganeli]

GitHub user ilganeli reopened a pull request:

https://github.com/apache/apex-malhar/pull/309

[APEXMALHAR-2106] Support multiple streams in StreamMerger

* Created a module which creates a binary tree of StreamMerger operators to 
support merging multiple streams
* StreamMerger operators are presently allocated as CONTAINER_LOCAL. I 
would like for them to be THREAD_LOCAL but I was getting the following error 
when attempting this:
```
Caused by: javax.validation.ValidationException: Locality THREAD_LOCAL 
invalid for operator OperatorMeta{name=Merger_Tier_0_#_0, 
operator=StreamMerger{name=null}, attributes={}} with multiple input streams as 
they origin from different owner OIO operators
```

* An example application is included 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilganeli/incubator-apex-malhar 
APEXMALHAR-2106F

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #309


commit e56d2b00f2114842877ec403b8a60df82b4014a9
Author: Ilya Ganelin 
Date:   2016-06-02T21:56:39Z

Created a module to merge multiple streams into a single stream. Created a 
test application demonstrating this in action. Still need to add more tests.

Fixed error with attempting to use odd number of input streams. Input ports 
for StreamMerger are now optional.

Dramatically simplified implementation by replacing manual tier creation 
with a queue of unconnected ports.

Updated locality settings.

Added test to count number of tuples received at output and test equality 
against all tuples sent across all merged streams.

Added explicit check for values.

Changes to pom.

Restored pom

commit d89df40ffd385c686b67ceef0aefb4f36f6fb5a0
Author: Ilya Ganelin 
Date:   2016-07-11T15:40:53Z

Merge remote-tracking branch 'upstream/master' into APEXMALHAR-2106F

commit f654ff6aeb3961cbdc566b27eadc4d347856
Author: Ilya Ganelin 
Date:   2016-07-11T15:49:01Z

Updated to use async execution and fixed comments from PR.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-malhar pull request #309: [APEXMALHAR-2106] Support multiple streams in...

[ilganeli]

Github user ilganeli closed the pull request at:

https://github.com/apache/apex-malhar/pull/309


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXMALHAR-2106) Support merging multiple streams with StreamMerger

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371060#comment-15371060
 ] 

ASF GitHub Bot commented on APEXMALHAR-2106:


GitHub user ilganeli reopened a pull request:

https://github.com/apache/apex-malhar/pull/309

[APEXMALHAR-2106] Support multiple streams in StreamMerger

* Created a module which creates a binary tree of StreamMerger operators to 
support merging multiple streams
* StreamMerger operators are presently allocated as CONTAINER_LOCAL. I 
would like for them to be THREAD_LOCAL but I was getting the following error 
when attempting this:
```
Caused by: javax.validation.ValidationException: Locality THREAD_LOCAL 
invalid for operator OperatorMeta{name=Merger_Tier_0_#_0, 
operator=StreamMerger{name=null}, attributes={}} with multiple input streams as 
they origin from different owner OIO operators
```

* An example application is included 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilganeli/incubator-apex-malhar 
APEXMALHAR-2106F

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #309


commit e56d2b00f2114842877ec403b8a60df82b4014a9
Author: Ilya Ganelin 
Date:   2016-06-02T21:56:39Z

Created a module to merge multiple streams into a single stream. Created a 
test application demonstrating this in action. Still need to add more tests.

Fixed error with attempting to use odd number of input streams. Input ports 
for StreamMerger are now optional.

Dramatically simplified implementation by replacing manual tier creation 
with a queue of unconnected ports.

Updated locality settings.

Added test to count number of tuples received at output and test equality 
against all tuples sent across all merged streams.

Added explicit check for values.

Changes to pom.

Restored pom

commit d89df40ffd385c686b67ceef0aefb4f36f6fb5a0
Author: Ilya Ganelin 
Date:   2016-07-11T15:40:53Z

Merge remote-tracking branch 'upstream/master' into APEXMALHAR-2106F

commit f654ff6aeb3961cbdc566b27eadc4d347856
Author: Ilya Ganelin 
Date:   2016-07-11T15:49:01Z

Updated to use async execution and fixed comments from PR.




> Support merging multiple streams with StreamMerger 
> ---
>
> Key: APEXMALHAR-2106
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2106
> Project: Apache Apex Malhar
>  Issue Type: New Feature
>Reporter: Ilya Ganelin
>Assignee: Ilya Ganelin
>
> To properly implement the Flatten transformation (and other Stream 
> combination operations), Apex must support merging data from multiple 
> sources. The StreamMerger operator can be improved to merge multiple streams, 
> rather than just the two streams it can handle in the present implementation. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Bleeding edge branch ?

[Munagala Ramanath]

We've had a number of issues recently related to dependencies on old
versions
of various packages/libraries such as Hadoop itself, Google guava,
HTTPClient,
mbassador, etc.

How about we create a "bleeding-edge" branch in both Core and Malhar which
will use the latest versions of these various dependencies, upgrade to Java
8 so
we can use the new Java features, etc. ?

This will give us an opportunity to discover these sorts of problems early
and,
when we are ready to pull the trigger for a major version, we have a branch
ready
for merge with, hopefully, minimal additional effort.

There will be no guarantees w.r.t. this branch so people using it use it at
their own
risk.

Ram

[jira] [Commented] (APEXMALHAR-2106) Support merging multiple streams with StreamMerger

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371022#comment-15371022
 ] 

ASF GitHub Bot commented on APEXMALHAR-2106:


Github user ilganeli closed the pull request at:

https://github.com/apache/apex-malhar/pull/309


> Support merging multiple streams with StreamMerger 
> ---
>
> Key: APEXMALHAR-2106
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2106
> Project: Apache Apex Malhar
>  Issue Type: New Feature
>Reporter: Ilya Ganelin
>Assignee: Ilya Ganelin
>
> To properly implement the Flatten transformation (and other Stream 
> combination operations), Apex must support merging data from multiple 
> sources. The StreamMerger operator can be improved to merge multiple streams, 
> rather than just the two streams it can handle in the present implementation. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #309: [APEXMALHAR-2106] Support multiple streams in...

[ilganeli]

Github user ilganeli closed the pull request at:

https://github.com/apache/apex-malhar/pull/309


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (APEXCORE-486) Facing Method not found issue while launching Kafka application

[Vlad Rozov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov resolved APEXCORE-486.
-
Resolution: Not A Bug

Apache Apex requires mbassador-1.1.9. Please check your application for 
dependency on the mbassador-1.2.x version that is not binary compatible with 
the prior versions.

> Facing Method not found issue while launching Kafka application
> ---
>
> Key: APEXCORE-486
> URL: https://issues.apache.org/jira/browse/APEXCORE-486
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Anuj
>
> Facing Method not found issue while launching Kafka application.
> java.lang.NoSuchMethodError: 
> net.engio.mbassy.bus.config.BusConfiguration.Default(III)Lnet/engio/mbassy/bus/config/BusConfiguration;
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:360)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (APEXCORE-486) Facing Method not found issue while launching Kafka application

[Vlad Rozov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov updated APEXCORE-486:

Labels:   (was: newbie)

> Facing Method not found issue while launching Kafka application
> ---
>
> Key: APEXCORE-486
> URL: https://issues.apache.org/jira/browse/APEXCORE-486
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Anuj
>
> Facing Method not found issue while launching Kafka application.
> java.lang.NoSuchMethodError: 
> net.engio.mbassy.bus.config.BusConfiguration.Default(III)Lnet/engio/mbassy/bus/config/BusConfiguration;
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:360)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:2979)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:550)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:101)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #355: APEXCORE-485 - Upgrade maven surefire plugin to...

[vrozov]

GitHub user vrozov opened a pull request:

https://github.com/apache/apex-core/pull/355

APEXCORE-485 - Upgrade maven surefire plugin to the latest version

@tweise Please review

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vrozov/apex-core APEXCORE-485

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/355.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #355






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Windowed Operator PR

[Bhupesh Chawda]

+1 for merging it.

~ Bhupesh

On Mon, Jul 11, 2016 at 11:10 AM, Yogi Devendra <
devendra.vyavah...@gmail.com> wrote:

> +1 for merging and incremental additions.
>
> ~ Yogi
>
> On 11 July 2016 at 11:04, Chinmay Kolhatkar 
> wrote:
>
> > +1 on merging.
> >
> > On Mon, Jul 11, 2016 at 11:00 AM, Amol Kekre 
> wrote:
> >
> > > +1 on merging it. This is a big feature addition.
> > >
> > > Thks
> > > Amol
> > >
> > >
> > > On Fri, Jul 8, 2016 at 4:17 PM, Siyuan Hua 
> > wrote:
> > >
> > > > +1
> > > >
> > > > We should merge this ASAP.
> > > > I don't think we could solve all the problems in one PR and I think
> > > David's
> > > > PR is good enough that we can keep working on this incrementally and
> in
> > > > parallel.
> > > >
> > > > Regards,
> > > > Siyuan
> > > >
> > > > On Fri, Jul 8, 2016 at 4:10 PM, David Yan 
> > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > The Windowed Operator PR is ready to be merged. Thank you very much
> > for
> > > > all
> > > > > your feedback so far.
> > > > >
> > > > > https://github.com/apache/apex-malhar/pull/319
> > > > >
> > > > > Merging this PR will make projects related to the WindowedOperator
> go
> > > on
> > > > > more easily, which includes High level API, Apache Calcite support,
> > > Apex
> > > > > runner in Beam and Dedup operator. Please speak up now If you think
> > > there
> > > > > are reasons for not merging it.
> > > > >
> > > > > Also please note that all the classes and interfaces are marked
> > > > "Evolving"
> > > > > so we can always change them later.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > David
> > > > >
> > > >
> > >
> >
>