Re: [Feature Proposal] Move Apache Apex website to Jekyll

[Tushar Gosavi]

Hi Ananth,

The new pages looks awsome. I would like to collbrate on improving the
website. Do let me know how can I help?

- Tushar.

On Thu, Jun 21, 2018 at 8:03 AM Ananth G  wrote:
>
> Hello All,
>
> Per some JIRA tickets and other PR review comments, having a single
> cohesive website along with blogging capabilities seems to be long due for
> Apache Apex.
>
> I would like to propose moving the Apache Apex website to Jekyll to address
> the following high level concerns:
>
> - Aim to provide a better build experience for the website
> - Make it easier to contribute blogs
> - Make the approval process easier by making it more markup reliant
> - Organise the content better in terms of content layout so that we can
> maintain it better.
>
>
> Here is a "skeleton" preview of the Jekyll based site and compatible with
> GitHub pages.
> - https://apacheapex.github.io/index.html
> - Each page can have a customised sidebar like
> https://apacheapex.github.io/architecture.html
> and https://apacheapex.github.io/writing_to_kudu_using_apex.html
> - There is a menu for blogs giving latest and archived views of blogs.
> https://apacheapex.github.io/latestblogentries.html
> - Mobile compatible as well.
>
>
> Here are some call outs regarding the implementation:
>
> - The above url is only a temporary host to showcase the skeleton structure
> and will be migrated to the apex-site url as a separate branch once I get
> some consensus.
> - There is a BSD styled licensed component  that I need guidance on
> https://github.com/apacheapex/apacheapex.github.io/blob/master/LICENSE-BSD-NAVGOCO.txt
> .
> - We will need more contribution from someone who has bootstrap experience
> to fix a few aspects on the home page (scrolling powered by section and the
> top level Apache Apex header sections)
> - The effort is going to span a few months as it is an entire website
> migration and there is a lot of content that needs to be collated and
> possibly updated as well as we progress
> - The site is based on the following project
> https://github.com/tomjoht/documentation-theme-jekyll
>
>
> My ask as part of the approval process:
>
> - A high level approval to the approach
> - Agreement on the high level menu and the drop downs at the top - Comments
> welcome
> - Usage of Jekyll as the main mechanism to generate content
> - Commenting is disabled consciously
> - BSD styled license inclusion for sidebar NAVGOCO component
> - Migration of the content from multiple locations (Malhar documentation ,
> Apex Documentation ) into a single cohesive set.
> - Altering of the build process for the website - I do not have an idea as
> to how compiled content in Jenkins is pushed to apache domain.
>
>
> May I request if anyone else is interested in contributing to this effort
> as well? I will wait for comments for the next 5 days and create JIRA
> tickets accordingly.
>
> Regards,
> Ananth



-- 
“I'd have blown my top, because I want to beat this damn thing,
 as long as I've gone this far. I can't just leave it after I've found
 out so much about it. I have to keep going to find out ultimately
what is the matter with it in the end."
Richard P. Feynman

Re: Core release 3.7.0

[Tushar Gosavi]

+1 for the release

Regards,
- Tushar.


On Thu, Jan 18, 2018 at 3:28 AM, Amol Kekre  wrote:

> +1
>
> Thks,
> Amol
>
>
> E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
>
> www.datatorrent.com
>
>
> On Wed, Jan 17, 2018 at 1:55 PM, Pramod Immaneni 
> wrote:
>
> > +1
> >
> > > On Jan 17, 2018, at 7:25 AM, Thomas Weise  wrote:
> > >
> > > Last release was 3.6.0 in May and following issues are ready for
> release:
> > >
> > > https://issues.apache.org/jira/issues/?jql=fixVersion%
> > 20%3D%203.7.0%20AND%20project%20%3D%20APEXCORE%20ORDER%20BY%
> 20status%20ASC
> > >
> > > Any opinions on cutting a release?
> > >
> > > Any committer interested running the release?
> > >
> > > Thanks,
> > > Thomas
> >
> >
>

Re: [VOTE] Apache Apex Malhar 3.8.0 release candidate RC1

[Tushar Gosavi]

+1 (binding)

- Verified signatures
- Built with -DskipTests
- Successfully ran pi example.
- Checked NOTICE, LICENSE and README.md files.

Regards,
-Tushar.


On Fri, Nov 10, 2017 at 11:48 AM, Bhupesh Chawda 
wrote:

> +1
>
> - Signatures verified
> - Build successful without tests
> - LICENSE, NOTICE and README.md exists
>
> ~ Bhupesh
>
>
>
> ___
>
> Bhupesh Chawda
>
> E: bhup...@datatorrent.com | Twitter: @bhupeshsc
>
> www.datatorrent.com  |  apex.apache.org
>
>
>
> On Thu, Nov 9, 2017 at 11:45 PM, Ananth G  wrote:
>
> > Hello All,
> >
> > A gentle reminder. Could you please vote for the release candidate so
> that
> > we can proceed with the release.
> >
> > Regards,
> > Ananth
> > > On 8 Nov 2017, at 3:39 am, Thomas Weise  wrote:
> > >
> > > +1 (binding)
> > >
> > > - verified signatures
> > > - run pi demo
> > >
> > > The release candidate should come with staged javadoc (it will be
> needed
> > to
> > > update the download page). Please see recent release threads, the vote
> > > example in the release instructions should probably be updated.
> > >
> > > Thanks,
> > > Thomas
> > >
> > >
> > >
> > > On Sun, Nov 5, 2017 at 10:23 AM, Ananth G 
> > wrote:
> > >
> > >> Dear Community,
> > >>
> > >> Please vote on the following Apache Apex Malhar 3.8.0 release
> candidate.
> > >>
> > >> This is a source release with binary artifacts published to Maven.
> > >>
> > >> This release is based on Apex Core 3.6 and resolves 63 issues.
> > >>
> > >> List of all issues fixed: https://issues.apache.org/
> > >> jira/secure/ReleaseNote.jspa?projectId=12318824=12340282 <
> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > >> projectId=12318824=12340282>
> > >> User documentation: http://apex.apache.org/docs/malhar-3.8/ <
> > >> http://apex.apache.org/docs/malhar-3.8/>
> > >>
> > >> Staging directory: https://dist.apache.org/repos/
> > >> dist/dev/apex/apache-apex-malhar-3.8.0-RC1/  > >> repos/dist/dev/apex/apache-apex-malhar-3.8.0-RC1/>
> > >> Source zip:
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> > >> malhar-3.8.0-RC1/apache-apex-malhar-3.8.0-source-release.zip <
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-
> > >> apex-malhar-3.8.0-RC1/apache-apex-malhar-3.8.0-source-
> > release.zip>Source
> > >> tar.gz:
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> > >> malhar-3.8.0-RC1/apache-apex-malhar-3.8.0-source-release.tar.gz <
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-
> > >> apex-malhar-3.8.0-RC1/apache-apex-malhar-3.8.0-source-release.tar.gz>
> > >> Maven staging repository:
> > >> https://repository.apache.org/content/repositories/
> orgapacheapex-1030/
> > <
> > >> https://repository.apache.org/content/repositories/
> orgapacheapex-1030/>
> > >>
> > >> Git source:
> > >> https://github.com/apache/apex-malhar/tree/v3.8.0-RC1 <
> > >> https://github.com/apache/apex-malhar/tree/v3.8.0-RC1> (commit:
> > >> 552a0b7e50d1441db3e98104491f4abce65d506e)
> > >>
> > >> PGP key:
> > >> http://pgp.mit.edu/pks/lookup?search=ananthg%40apache.org=index <
> > >> http://pgp.mit.edu/pks/lookup?search=anan...@apache.org=index>
> > >> KEYS file:
> > >> https://dist.apache.org/repos/dist/release/apex/KEYS <
> > >> https://dist.apache.org/repos/dist/release/apex/KEYS>
> > >>
> > >> More information at:
> > >> http://apex.apache.org 
> > >>
> > >> Please try the release and vote; vote will be open for at least 72
> > hours.
> > >>
> > >> [ ] +1 approve (and what verification was done)
> > >> [ ] -1 disapprove (and reason why)
> > >>
> > >> http://www.apache.org/foundation/voting.html  > >> foundation/voting.html>
> > >>
> > >> How to verify release candidate:
> > >>
> > >> http://apex.apache.org/verification.html  > >> verification.html>
> > >>
> > >> Regards,
> > >> Ananth
> >
> >
>

Re: [ANNOUNCE] New Apache Apex Committer: Ananth Gundabattula

[Tushar Gosavi]

Congratulations Ananth.

- Tushar.


On Fri, Nov 3, 2017 at 9:28 PM, Ashwin Chandra Putta <
ashwinchand...@gmail.com> wrote:

> Congratulations Ananth. Well deserved!
>
> On Fri, Nov 3, 2017 at 2:08 AM, Chinmay Kolhatkar  >
> wrote:
>
> > Congrats Ananth!!
> >
> > On Fri, Nov 3, 2017 at 2:37 PM, Priyanka Gugale 
> wrote:
> >
> > > Congrats Ananth!!
> > >
> > > -Priyanka
> > >
> > > On Fri, Nov 3, 2017 at 2:20 PM, Thomas Weise  wrote:
> > >
> > > > The Project Management Committee (PMC) for Apache Apex is pleased to
> > > > announce Ananth Gundabattula as new committer.
> > > >
> > > > Ananth has been contributing to the project for about a year.
> > Highlights:
> > > >
> > > > * Cassandra and Kudu operators with in-depth analysis/design work
> > > > * Good collaboration, adherence to contributor guidelines and
> ownership
> > > of
> > > > work
> > > > * Work beyond feature focus such as fixing pre-existing test issues
> > that
> > > > impact CI
> > > > * Presented at YOW Data and Dataworks Summit Australia
> > > > * Enthusiast, contributes on his own time
> > > >
> > > > Welcome, Ananth, and congratulations!
> > > > Thomas, for the Apache Apex PMC.
> > > >
> > >
> >
>
>
>
> --
>
> Regards,
> Ashwin.
>

Re: Malhar release 3.8.0

[Tushar Gosavi]

Hi Vlad,

As far as I remember, I had access to staging maven area while doing
previous apex release. You will need to update .m2/settings.xml with apache
credential to access the maven repository.

Regards,
-Tushar.


On Thu, Oct 26, 2017 at 11:02 PM, Vlad Rozov <vro...@apache.org> wrote:

> Please send your PGP public key to one of PMC members to be added to KEYS.
> I don't remember if only PMC have access to staging Apache maven, it may be
> the case. Tushar, did you have write access to the staging Apache maven
> when you did the release?
>
> What do we do with https://issues.apache.org/jira/browse/APEXMALHAR-2461?
>
> Thank you,
>
> Vlad
>
>
> On 10/25/17 15:28, Ananth G wrote:
>
>> I would like to volunteer to be the release manager for this. Given I
>> have not done this before I might have a few questions along the way in the
>> mailing list.
>>
>> A couple of questions regarding the release process:
>>
>> - In the link https://apex.apache.org/release.html , in the section
>> titled “Build and deploy release candidate” there is a mention of adding
>> GPG keys.
>>  - Is it mandatory for the release manager gpg public key to be
>> present in the list
>>  - If it is how do I get my key added to that list
>> - In the same section of the above link there is a mention of configuring
>> the server apache.staging.https in the maven settings file.
>>  - I am not able to reach this server ? Is this expected?
>>  - The userid and password to be configured are our committer ids
>> ?
>>
>> Regards
>> Ananth
>>
>> On 26 Oct 2017, at 4:04 am, Ananth G <ananthg.a...@gmail.com> wrote:
>>>
>>> +1 for malhar release.
>>>
>>>
>>> Regards,
>>> Ananth
>>>
>>> On 26 Oct 2017, at 3:20 am, Bhupesh Chawda <bhup...@datatorrent.com>
>>>> wrote:
>>>>
>>>> +1 for malhar release
>>>>
>>>> ~ Bhupesh
>>>>
>>>>
>>>> ___
>>>>
>>>> Bhupesh Chawda
>>>>
>>>> E: bhup...@datatorrent.com | Twitter: @bhupeshsc
>>>>
>>>> www.datatorrent.com  |  apex.apache.org
>>>>
>>>>
>>>>
>>>> On Wed, Oct 25, 2017 at 9:37 PM, Chinmay Kolhatkar <
>>>> chin...@datatorrent.com>
>>>> wrote:
>>>>
>>>> +1.
>>>>>
>>>>> - Chinmay.
>>>>>
>>>>> On 25 Oct 2017 9:20 pm, "Chaitanya Chebolu" <chaita...@datatorrent.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> +1 on new release.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> On Wed, Oct 25, 2017 at 9:09 PM, Vlad Rozov <vro...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> +1.
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Vlad
>>>>>>>
>>>>>>>
>>>>>>> On 10/25/17 08:21, Amol Kekre wrote:
>>>>>>>>
>>>>>>>> +1 on a new malhar release.
>>>>>>>>
>>>>>>>> Thks,
>>>>>>>> Amol
>>>>>>>>
>>>>>>>>
>>>>>>>> E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
>>>>>>>>
>>>>>>>> www.datatorrent.com
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 24, 2017 at 9:12 PM, Tushar Gosavi <
>>>>>>>>
>>>>>>> tus...@datatorrent.com>
>>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> +1 on creating a new malhar release.
>>>>>>>>
>>>>>>>>> - Tushar.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Oct 25, 2017 at 4:39 AM, Pramod Immaneni <
>>>>>>>>>
>>>>>>>> pra...@datatorrent.com
>>>>>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> +1 on creating a new release. I, unfortunately, do not have the
>>>>>>>>> time
>>>>>>>>>
>>>>>>>>>> currently to participate in the release activities.
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 23, 2017 at 7:15 PM, Thomas Weise <t...@apache.org>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>
>>>>>> The last release was back in March, there are quite a few JIRAs that
>>>>>>>>>> have
>>>>>>>>>> been completed since and should be released.
>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=fixVersion%
>>>>>>>>>>> 20%3D%203.8.0%20AND%20project%20%3D%20APEXMALHAR%20ORDER%
>>>>>>>>>>> 20BY%20status%20ASC
>>>>>>>>>>>
>>>>>>>>>>>  From looking at the list there is nothing that should stand in
>>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>> way
>>>>>>
>>>>>>> of a
>>>>>>>>>>
>>>>>>>>>> release?
>>>>>>>>>>>
>>>>>>>>>>> Also, once the release is out it would be a good opportunity to
>>>>>>>>>>>
>>>>>>>>>> effect
>>>>>>
>>>>>>> the
>>>>>>>>>>
>>>>>>>>>> major version change.
>>>>>>>>>>>
>>>>>>>>>>> Anyone interested to be the release manager?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Chaitanya*
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> E: chaita...@datatorrent.com | Twitter: @chaithu1403
>>>>>>
>>>>>> www.datatorrent.com  |  apex.apache.org
>>>>>>
>>>>>>
>

Re: Malhar release 3.8.0

[Tushar Gosavi]

+1 on creating a new malhar release.

- Tushar.


On Wed, Oct 25, 2017 at 4:39 AM, Pramod Immaneni 
wrote:

> +1 on creating a new release. I, unfortunately, do not have the time
> currently to participate in the release activities.
>
> On Mon, Oct 23, 2017 at 7:15 PM, Thomas Weise  wrote:
>
> > The last release was back in March, there are quite a few JIRAs that have
> > been completed since and should be released.
> >
> > https://issues.apache.org/jira/issues/?jql=fixVersion%
> > 20%3D%203.8.0%20AND%20project%20%3D%20APEXMALHAR%20ORDER%
> > 20BY%20status%20ASC
> >
> > From looking at the list there is nothing that should stand in the way
> of a
> > release?
> >
> > Also, once the release is out it would be a good opportunity to effect
> the
> > major version change.
> >
> > Anyone interested to be the release manager?
> >
> > Thanks,
> > Thomas
> >
>

[jira] [Resolved] (APEXMALHAR-2508) CacheManager throws exception when the operator receives a shutdown request

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi resolved APEXMALHAR-2508.
---
   Resolution: Fixed
Fix Version/s: 3.8.0

> CacheManager throws exception when the operator receives a shutdown request
> ---
>
> Key: APEXMALHAR-2508
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2508
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Reporter: Ajay Gupta
>Assignee: Ajay Gupta
>Priority: Minor
> Fix For: 3.8.0
>
>
> CacheManager initializes the Timer (refresher) only if refreshTime is not 
> null. However, in deactivate, before cancelling the timer, there is no null 
> check leading to NullPointerException.
> Solution : Add a null check before cancelling the timer.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (APEXMALHAR-2536) Support for setters on nested Pojo

[Tushar Gosavi (JIRA)]

Tushar Gosavi created APEXMALHAR-2536:
-

 Summary: Support for setters on nested Pojo
 Key: APEXMALHAR-2536
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2536
 Project: Apache Apex Malhar
  Issue Type: Improvement
Reporter: Tushar Gosavi
Priority: Minor


Getters support getting values from nested Pojo by adding suitable 
methods/fields in the chain

for example
{code}
class Inner {
  private int b;
  public int getB() { return b; }
  public void setB(int b) { this.b = b; }
}

class Nested {
  private in a;
  private Inner inner;
  public int getA() { return a; }
  public void getA(int a) { this.a = a; }
  public Inner getInner() { return inner; }
  public void setInner(Inner i) { this.inner = i; }
}
{code}

With above objects  If getter is specified as `inner.b` then the expression is 
created with
`object.getInner().getB()` which compiles and gives result correctly. 

If setter is specified with same expression it fails to compile as inner is a 
private object.
We plan to include similar functionality for setters by following chain of 
objects and constructing valid java expressions.






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (APEXMALHAR-2535) Timeouts in AbstractEnricher specified as int which limits duration of time which could be specified.

[Tushar Gosavi (JIRA)]

Tushar Gosavi created APEXMALHAR-2535:
-

 Summary: Timeouts in AbstractEnricher specified as int which 
limits duration of time which could be specified.
 Key: APEXMALHAR-2535
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2535
 Project: Apache Apex Malhar
  Issue Type: Bug
Reporter: Tushar Gosavi
Priority: Minor


cacheExpirationInterval and cacheCleanupInterval timeout properties are using 
int as datatype which limits the amount of time duration could be specified 
using them. I am
going to change they to long also provide a way for operator to specify 
expiryType on cacheStore





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (APEXCORE-720) In DAGExecution plugin, Context.getDAG returns null

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi resolved APEXCORE-720.

   Resolution: Fixed
Fix Version/s: 3.7.0

> In DAGExecution plugin, Context.getDAG returns null
> ---
>
> Key: APEXCORE-720
> URL: https://issues.apache.org/jira/browse/APEXCORE-720
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Chinmay Kolhatkar
>Assignee: Chinmay Kolhatkar
> Fix For: 3.7.0
>
>
> When plugin calls context.getDAG method, it returns null.
> The method should return a LogicalPlan object.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (APEXCORE-718) How to Use one Kafka input operator and Feed to multiple operator

[Tushar Gosavi (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006246#comment-16006246
 ] 

Tushar Gosavi commented on APEXCORE-718:


Please use us...@apex.apache.org mailing list for asking questions about apex 
before opening an JIRA.  Use JIRA for bug/improvement related to code and 
documentation.

The stream for which exception is thrown is MapData, and the stream is not 
present in the code snipped you have provided. In general you can't use the 
same port twice if using addStream api. For sending data to multiple operators 
you can connected same output port to multiple input ports.

dag.addStreams("streamname", outputPorts, in1, in2, in3 ...)

You can not connect single input port to multiple output ports.



> How to Use one Kafka input operator and Feed to multiple operator 
> --
>
> Key: APEXCORE-718
> URL: https://issues.apache.org/jira/browse/APEXCORE-718
> Project: Apache Apex Core
>  Issue Type: Question
>Reporter: ROHIT RAJKUMAR GARG
>
> I want to read from a topic using the kafka input operator and then feed it 
> to multiple operator . in short it will be a sort of branch 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[ANNOUNCE] Apache Apex Core 3.6.0 released

[Tushar Gosavi]

Dear Community,

The Apache Apex community is pleased to announce release 3.6.0 of Apex
Core (the engine).

Apache Apex is an enterprise grade big data-in-motion platform that unifies
stream and batch processing. Apex was built for scalability and low-latency
processing, high availability and operability.

This release adds the support for custom control tuples, experimental
support for plugins
and number of important bug fixes and improvements. See full release notes

for
all changes in this release.

Apex provides features that similar platforms currently don’t offer, such
as fine grained, incremental recovery to only reset the portion of a
topology that is affected by a failure, support for elastic scaling based
on the ability to acquire (and release) resources as needed as well as the
ability to alter topology and operator properties on running applications.

Apex early on brought the combination of high throughput, low latency and
fault tolerance with strong processing guarantees to the stream data
processing space and gained maturity through important production use cases
at several organizations. See the powered by page and resources on the
project web site for more information:

http://apex.apache.org/powered-by-apex.html
http://apex.apache.org/docs.html

The Apex engine is supplemented by Malhar, the library of pre-built
operators, including adapters that integrate with many existing
technologies as sources and destinations, like message buses, databases,
files or social media feeds.

An easy way to get started with Apex is to pick one of the examples as starting
point. They cover many common and recurring tasks, such as data consumption
from different sources, output to various sinks, partitioning and fault
tolerance:

https://github.com/apache/apex-malhar/tree/master/examples/


Apex Malhar and Core (the engine) are separate repositories and releases.
We expect more frequent releases of Malhar to roll out new connectors and
other operators based on a stable engine API. The latest Malhar release
3.7.0 works on existing Apex Core 3.4.0.

The source release can be found at:
http://apex.apache.org/downloads.html

We welcome your help and feedback. For more information on the project and
how to get involved, visit our website at:

http://apex.apache.org/

Regards,
The Apache Apex community

Re: release activties

[Tushar Gosavi]

The issue with java documentation is resolved after opening
https://issues.apache.org/jira/browse/INFRA-14117. Now java doc for 3.6.0
is available at
https://ci.apache.org/projects/apex-core/apex-core-javadoc-release-3.6/index.html

- Tushar.


On Mon, May 8, 2017 at 11:11 AM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> I have updated https://svn.apache.org/repos/infra/
> infrastructure/buildbot/aegis/buildmaster/master1/projects/apex.conf file
> by adding release-3.6 branch for Apex Core. For some reason builder is not
> registered for release-3.6 and force build through IRC is also not working
> as builder is not registered.
>
> I am communicating through IRC for the resolution of the issue. No
> response till now. I may open an INRA JIRA for it.
>
> Regards,
> -Tushar.
>
>
> On Sat, May 6, 2017 at 10:23 PM, Thomas Weise <t...@apache.org> wrote:
>
>> The javadocs are not pushed yet.
>>
>> https://ci.apache.org/projects/apex-core/apex-core-javadoc-
>> release-3.6/index.html
>>
>> Last time, Ram had to trigger buildbot manually to make it happen.
>>
>> Ram, what is the magic to do that? We should also document it in case the
>> trigger on commit or nightly build does not happen.
>>
>>
>>
>> On Fri, May 5, 2017 at 6:41 AM, Thomas Weise <t...@apache.org> wrote:
>>
>> > Move of RC to release is done and new JIRA version added.
>> >
>> >
>> > On Fri, May 5, 2017 at 6:34 AM, Vlad Rozov <v.ro...@datatorrent.com>
>> > wrote:
>> >
>> >> I'll help with moving RC candidate from dev to release. Do you have
>> >> permission to publish artifacts to maven?
>> >>
>> >> Thank you,
>> >>
>> >> Vlad
>> >>
>> >> On 5/5/17 06:21, Thomas Weise wrote:
>> >>
>> >>> You should be able to complete the release activities except the JIRA
>> >>> version. What do you mean by "release svn repository"?
>> >>>
>> >>> Also, release process states that you should turn off JIRA
>> notifications
>> >>> when closing issues (to not flood mailing list).
>> >>>
>> >>> Subject of closing vote should start with [RESULT]
>> >>>
>> >>> Yes, the semantic versioning check should be updated for both
>> branches.
>> >>>
>> >>> Thomas
>> >>>
>> >>>
>> >>> On Fri, May 5, 2017 at 2:34 AM, Tushar Gosavi <tus...@datatorrent.com
>> >
>> >>> wrote:
>> >>>
>> >>> As a committer, I do not have write permissions to release svn
>> >>>> repository.
>> >>>> Need help from PMC member to update the release repository. Any
>> >>>> volunteers??
>> >>>>
>> >>>> Also one more clarification needed with respect to updating semantic
>> >>>> versioning check. Does it needs to be done for both release-3.6.0 and
>> >>>> master branch?
>> >>>>
>> >>>> I will push the documentations after release repository is updated.
>> >>>>
>> >>>> Thanks,
>> >>>> -Tushar.
>> >>>>
>> >>>>
>> >>>> On Fri, May 5, 2017 at 11:35 AM, Tushar Gosavi <
>> tus...@datatorrent.com>
>> >>>> wrote:
>> >>>>
>> >>>> The vote is concluded and passes. Thanks everyone for voting and
>> >>>>>
>> >>>> verifying
>> >>>>
>> >>>>> the release.
>> >>>>>
>> >>>>> binding +1 (4)
>> >>>>>
>> >>>>> Vlad Rozov
>> >>>>> Thomas Weise
>> >>>>> Amol Kekre
>> >>>>> Pramod Immaneni
>> >>>>>
>> >>>>>
>> >>>>> non-binding +1 (2)
>> >>>>> Bhupesh Chawda
>> >>>>> Chaitanya Chebolu
>> >>>>>
>> >>>>> No other votes.
>> >>>>>
>> >>>>> Will complete the release activities.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Tushar.
>> >>>>>
>> >>>>>
>> >>>>> On Thu, May 4, 2017 at 12:05 PM, Chaitanya Chebolu <
>> >>>>> chaita...@datatorrent.com> wrote:
>&

Re: release activties

[Tushar Gosavi]

I have updated
https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/apex.conf
file by adding release-3.6 branch for Apex Core. For some reason builder is
not registered for release-3.6 and force build through IRC is also not
working as builder is not registered.

I am communicating through IRC for the resolution of the issue. No response
till now. I may open an INRA JIRA for it.

Regards,
-Tushar.


On Sat, May 6, 2017 at 10:23 PM, Thomas Weise <t...@apache.org> wrote:

> The javadocs are not pushed yet.
>
> https://ci.apache.org/projects/apex-core/apex-core-
> javadoc-release-3.6/index.html
>
> Last time, Ram had to trigger buildbot manually to make it happen.
>
> Ram, what is the magic to do that? We should also document it in case the
> trigger on commit or nightly build does not happen.
>
>
>
> On Fri, May 5, 2017 at 6:41 AM, Thomas Weise <t...@apache.org> wrote:
>
> > Move of RC to release is done and new JIRA version added.
> >
> >
> > On Fri, May 5, 2017 at 6:34 AM, Vlad Rozov <v.ro...@datatorrent.com>
> > wrote:
> >
> >> I'll help with moving RC candidate from dev to release. Do you have
> >> permission to publish artifacts to maven?
> >>
> >> Thank you,
> >>
> >> Vlad
> >>
> >> On 5/5/17 06:21, Thomas Weise wrote:
> >>
> >>> You should be able to complete the release activities except the JIRA
> >>> version. What do you mean by "release svn repository"?
> >>>
> >>> Also, release process states that you should turn off JIRA
> notifications
> >>> when closing issues (to not flood mailing list).
> >>>
> >>> Subject of closing vote should start with [RESULT]
> >>>
> >>> Yes, the semantic versioning check should be updated for both branches.
> >>>
> >>> Thomas
> >>>
> >>>
> >>> On Fri, May 5, 2017 at 2:34 AM, Tushar Gosavi <tus...@datatorrent.com>
> >>> wrote:
> >>>
> >>> As a committer, I do not have write permissions to release svn
> >>>> repository.
> >>>> Need help from PMC member to update the release repository. Any
> >>>> volunteers??
> >>>>
> >>>> Also one more clarification needed with respect to updating semantic
> >>>> versioning check. Does it needs to be done for both release-3.6.0 and
> >>>> master branch?
> >>>>
> >>>> I will push the documentations after release repository is updated.
> >>>>
> >>>> Thanks,
> >>>> -Tushar.
> >>>>
> >>>>
> >>>> On Fri, May 5, 2017 at 11:35 AM, Tushar Gosavi <
> tus...@datatorrent.com>
> >>>> wrote:
> >>>>
> >>>> The vote is concluded and passes. Thanks everyone for voting and
> >>>>>
> >>>> verifying
> >>>>
> >>>>> the release.
> >>>>>
> >>>>> binding +1 (4)
> >>>>>
> >>>>> Vlad Rozov
> >>>>> Thomas Weise
> >>>>> Amol Kekre
> >>>>> Pramod Immaneni
> >>>>>
> >>>>>
> >>>>> non-binding +1 (2)
> >>>>> Bhupesh Chawda
> >>>>> Chaitanya Chebolu
> >>>>>
> >>>>> No other votes.
> >>>>>
> >>>>> Will complete the release activities.
> >>>>>
> >>>>> Thanks,
> >>>>> Tushar.
> >>>>>
> >>>>>
> >>>>> On Thu, May 4, 2017 at 12:05 PM, Chaitanya Chebolu <
> >>>>> chaita...@datatorrent.com> wrote:
> >>>>>
> >>>>> +1
> >>>>>>
> >>>>>> 1) Verified the file Integrity check.
> >>>>>> 2) Verified the source code using tar file.
> >>>>>>  - Extracted tar file and compiled.
> >>>>>>  - No binary files
> >>>>>>  - LICENSE, NOTICE, README, CHANGELOG.md exist.
> >>>>>>  - rat check
> >>>>>> 3) Launched PI demo.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Chaitanya
> >>>>>>
> >>>>>> On Wed, May 3, 2017 at 7:07 AM, Pramod Immaneni <
> >>>>>> pra...@dat

Re: [VOTE] Apache Apex Core Release 3.6.0 (RC1)

[Tushar Gosavi]

I do not seems to have permissions to create versions in JIRA as specified
release steps (http://apex.apache.org/release.html). Can someone create new
release tags for 3.7.0 and 3.6.1.

Thank You,
-Tushar.


On Fri, May 5, 2017 at 3:04 PM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> As a committer, I do not have write permissions to release svn repository.
> Need help from PMC member to update the release repository. Any volunteers??
>
> Also one more clarification needed with respect to updating semantic
> versioning check. Does it needs to be done for both release-3.6.0 and
> master branch?
>
> I will push the documentations after release repository is updated.
>
> Thanks,
> -Tushar.
>
>
> On Fri, May 5, 2017 at 11:35 AM, Tushar Gosavi <tus...@datatorrent.com>
> wrote:
>
>> The vote is concluded and passes. Thanks everyone for voting and
>> verifying the release.
>>
>> binding +1 (4)
>>
>> Vlad Rozov
>> Thomas Weise
>> Amol Kekre
>> Pramod Immaneni
>>
>>
>> non-binding +1 (2)
>> Bhupesh Chawda
>> Chaitanya Chebolu
>>
>> No other votes.
>>
>> Will complete the release activities.
>>
>> Thanks,
>> Tushar.
>>
>>
>> On Thu, May 4, 2017 at 12:05 PM, Chaitanya Chebolu <
>> chaita...@datatorrent.com> wrote:
>>
>>> +1
>>>
>>> 1) Verified the file Integrity check.
>>> 2) Verified the source code using tar file.
>>> - Extracted tar file and compiled.
>>> - No binary files
>>> - LICENSE, NOTICE, README, CHANGELOG.md exist.
>>> - rat check
>>> 3) Launched PI demo.
>>>
>>> Regards,
>>> Chaitanya
>>>
>>> On Wed, May 3, 2017 at 7:07 AM, Pramod Immaneni <pra...@datatorrent.com>
>>> wrote:
>>>
>>> > +1 (binding)
>>> >
>>> > verified file integrity
>>> > no unexpected binary files
>>> > existence of README.md, NOTICE, LICENSE and CHANGELOG.md files
>>> > checked build and licenses
>>> > launched and ran pi demo
>>> >
>>> > Minor nitpicks
>>> >
>>> > CHANGELOG.md has a line in the beginning of the file that has a date in
>>> > future "Version 3.6.0 - 2017-05-04"
>>> > Please have your key signed by some of the others in the KEYS file,
>>> right
>>> > now it is self-signed
>>> >
>>> > Thanks
>>> >
>>> > On Mon, May 1, 2017 at 11:49 AM, Tushar Gosavi <tus...@datatorrent.com
>>> >
>>> > wrote:
>>> >
>>> > > Dear Community,
>>> > >
>>> > > Please vote on the following Apache Apex Core 3.6.0 release
>>> candidate 1.
>>> > >
>>> > > This release adds the support for custom control tuples, experimental
>>> > > support for plugins
>>> > > and other improvements and important bug fixes.
>>> > >
>>> > > This is a source release with binary artifacts published to Maven.
>>> > >
>>> > > List of all issues fixed: https://s.apache.org/HQ0r
>>> > >
>>> > > Staging directory
>>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-core
>>> -3.6.0-RC1/
>>> > > Source zip:
>>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
>>> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.zip
>>> > > Source tar.gz:
>>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
>>> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.tar.gz
>>> > > Maven staging repository:
>>> > > https://repository.apache.org/content/repositories/orgapache
>>> apex-1028
>>> > >
>>> > > Git source:
>>> > > https://git-wip-us.apache.org/repos/asf?p=apex-core.git;a=
>>> > > commit;h=refs/tags/v3.6.0-RC1
>>> > > (commit: 5a517348ae497c06150f32ce39b6915588e92510)
>>> > >
>>> > > PGP key:
>>> > > http://pgp.mit.edu:11371/pks/lookup?op=vindex=tushar@
>>> apache.org
>>> > > KEYS file:
>>> > > https://dist.apache.org/repos/dist/release/apex/KEYS
>>> > >
>>> > > More information at:
>>> > > http://apex.apache.org
>>> > >
>>> > > Please try the release and vote; vote will be open for at least 72
>>> hours.
>>> > >
>>> > > [ ] +1 approve (and what verification was done)
>>> > > [ ] -1 disapprove (and reason why)
>>> > >
>>> > > http://www.apache.org/foundation/voting.html
>>> > >
>>> > > How to verify release candidate:
>>> > >
>>> > > http://apex.apache.org/verification.html
>>> > >
>>> > > Thanks,
>>> > > Tushar.
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>>
>>> *Chaitanya*
>>>
>>> Software Engineer
>>>
>>> E: chaita...@datatorrent.com | Twitter: @chaithu1403
>>>
>>> www.datatorrent.com  |  apex.apache.org
>>>
>>
>>
>

[jira] [Closed] (APEXCORE-585) Latency should be calculated only after the first window has been complete

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-585.
--

Closing after 3.6.0 release

> Latency should be calculated only after the first window has been complete
> --
>
> Key: APEXCORE-585
> URL: https://issues.apache.org/jira/browse/APEXCORE-585
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: David Yan
>Assignee: David Yan
>Priority: Minor
> Fix For: 3.6.0
>
>
> From the latency unit test, it tries to calculate the latency by looking at 
> the difference between current window id and upstream's current window id 
> when the end window stats is not available. But when the current window id is 
> 0 (first window has not been complete), it would return some wacky latency 
> value because the current window id is 0.
> This would not be an issue if the window id was not a simple sequence.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-715) Remove unnecessary @Evolving annotation in engine

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-715.
--

Closing after 3.6.0 release

> Remove unnecessary @Evolving annotation in engine
> -
>
> Key: APEXCORE-715
> URL: https://issues.apache.org/jira/browse/APEXCORE-715
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> \@Evolving annotation in the engine are not necessary and introduce 
> unnecessarily practice and wrong impression that some interfaces and/or 
> classes are more stable compared to others. All interfaces and classes in the 
> engine are not subject to the semantic version checks and may change from a 
> release to a release. The same applies to the buffer server.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-617) InputNodeTest intermittently fails with ConcurrentModificationException

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-617.
--

Closing after 3.6.0 release

> InputNodeTest intermittently fails with ConcurrentModificationException
> ---
>
> Key: APEXCORE-617
> URL: https://issues.apache.org/jira/browse/APEXCORE-617
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> InputNodeTest.emitTestHelper uses deprecated inherently unsafe Thread.stop(). 
> Thread.stop() does not immediately stops the thread, the JVM waits for a 
> safepoint to raise ThreadDeath exception on the target thread. This leads to 
> a race condition and possible ConcurrentModificationException.   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-641) Subscribers/DataListeners may not be scheduled to execute even when they have data to process

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-641.
--

Closing after 3.6.0 release

> Subscribers/DataListeners may not be scheduled to execute even when they have 
> data to process
> -
>
> Key: APEXCORE-641
> URL: https://issues.apache.org/jira/browse/APEXCORE-641
> Project: Apache Apex Core
>  Issue Type: Bug
>  Components: Buffer Server
>Affects Versions: 3.2.0, 3.3.0, 3.2.1, 3.4.0, 3.5.0, 3.6.0
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> Buffer server iterates over DataListeners aka LogicalNodes and each 
> LogicalNode tries to send to it's downstream all data that Publisher added to 
> the DataList. When an output port is connected to multiple partitions or 
> downstream operators (2 or more DataListeners/LogicalNodes) there may be more 
> data published to the DataList after first few DataListeners in the listeners 
> set iterated over DataList and reached the last block published so far. The 
> data published while the last DataListeners sends data to it's downstream 
> will not be processed by other DataListeners until Publisher adds more data 
> to the DataList. This may lead to blocked operators, as Buffer server may 
> stop processing data completely in case Publisher fills more than one Data 
> block while a single DataListener sends data to it's downstream and there are 
> more Subscribers/DataListeners than number of in memory blocks allowed (8). 
> In such case, Publisher will be suspended, and there will be no task 
> scheduled to process data already published to the DataList.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-596) Committed method on operators not called when stream locality is THREAD_LOCAL

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-596.
--

Closing after 3.6.0 release

> Committed method on operators not called when stream locality is THREAD_LOCAL
> -
>
> Key: APEXCORE-596
> URL: https://issues.apache.org/jira/browse/APEXCORE-596
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Francis Fernandes
>Assignee: Francis Fernandes
>Priority: Minor
> Fix For: 3.6.0
>
>
> When the locality of the stream connecting the two operators is 
> Locality.THREAD_LOCAL, the committed method is not called for some operators. 
> These operators implement the Operator.CheckpointListener. e.g. 
> AbstractFileOutputOperator
> For thread local during activate  we do not set the thread in the node's 
> context
> Because the thread is not set, we skip this operator in the 
> processHeartBeatResponse and the committed is not called
> {code}
> if (thread == null || !thread.isAlive()) {
>   continue;
> }
> {code}
> We need this condition for invalid operators (operator failures) in case of 
> other localities. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-662) Raise StramEvent for heartbeat miss

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-662.
--

Closing after 3.6.0 release

> Raise StramEvent for heartbeat miss
> ---
>
> Key: APEXCORE-662
> URL: https://issues.apache.org/jira/browse/APEXCORE-662
> Project: Apache Apex Core
>  Issue Type: Task
>    Reporter: Tushar Gosavi
>Assignee: Hitesh Kapoor
>Priority: Minor
> Fix For: 3.6.0
>
>
> One of the cause of container kill is heartbeat miss. we can raise an 
> StramEvent in case of heartbeat miss to make debugging easier.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-683) Apex client should support application packages on HDFS

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-683.
--

Closing after 3.6.0 release

> Apex client should support application packages on HDFS
> ---
>
> Key: APEXCORE-683
> URL: https://issues.apache.org/jira/browse/APEXCORE-683
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> apex client supports only local file system, it is natural for native yarn 
> application to support hdfs and other dfs file systems:
> {noformat}
> hdfs dfs -ls hdfs:///user/vrozov
> Found 4 items
> drwx--   - vrozov supergroup  0 2016-12-24 16:00 
> hdfs:///user/vrozov/.Trash
> drwxrwxrwx   - vrozov supergroup  0 2017-02-28 19:20 
> hdfs:///user/vrozov/.flink
> -rw-r--r--   3 vrozov supergroup   18862977 2017-03-25 09:20 
> hdfs:///user/vrozov/appPackage.apa
> drwxrwxrwx   - vrozov supergroup  0 2016-10-25 21:49 
> hdfs:///user/vrozov/datatorrent
> [vrozov@node0 apex]$ apex
> Apex CLI 3.6.0-SNAPSHOT 25.03.2017 @ 09:17:45 PDT rev: b4a4e05 branch: master
> apex> get-app-package-info  hdfs:///user/vrozov/appPackage.apa
> java.io.FileNotFoundException: hdfs:/user/vrozov/appPackage.apa (No such file 
> or directory)
>   at java.util.zip.ZipFile.open(Native Method)
>   at java.util.zip.ZipFile.(ZipFile.java:215)
>   at java.util.zip.ZipFile.(ZipFile.java:145)
>   at java.util.jar.JarFile.(JarFile.java:154)
>   at java.util.jar.JarFile.(JarFile.java:118)
>   at com.datatorrent.stram.client.AppPackage.(AppPackage.java:185)
>   at com.datatorrent.stram.client.AppPackage.(AppPackage.java:258)
>   at 
> com.datatorrent.stram.cli.ApexCli.newAppPackageInstance(ApexCli.java:467)
>   at 
> com.datatorrent.stram.cli.ApexCli$GetAppPackageInfoCommand.execute(ApexCli.java:3502)
>   at com.datatorrent.stram.cli.ApexCli$3.run(ApexCli.java:1520)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-655) Support RELEASE as archetype version when creating a project

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-655.
--

Closing after 3.6.0 release

> Support RELEASE as archetype version when creating a project 
> -
>
> Key: APEXCORE-655
> URL: https://issues.apache.org/jira/browse/APEXCORE-655
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Thomas Weise
>Priority: Minor
> Fix For: 3.6.0
>
>
> Rather than keep on replacing the Apex archetype version number in various 
> examples, Maven supports LATEST to refer to the latest released archetype 
> version. 
> This requires a change to decouple resolution of Apex dependency version from 
> archetype version to replace the Apex version when the archetype is created 
> vs. when the project is created.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-511) DAG.addOperator, addStream and addModule should check for null and empty names

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-511.
--

Closing after 3.6.0 release

> DAG.addOperator, addStream and addModule should check for null and empty names
> --
>
> Key: APEXCORE-511
> URL: https://issues.apache.org/jira/browse/APEXCORE-511
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Oliver Winke
>Priority: Minor
>  Labels: newbie
> Fix For: 3.6.0
>
>
> Operators, Streams and Modules should have valid not empty names.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-701) Upgrade Malhar version in archetype to 3.7.0

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-701.
--

Closing after 3.6.0 release

> Upgrade Malhar version in archetype to 3.7.0 
> -
>
> Key: APEXCORE-701
> URL: https://issues.apache.org/jira/browse/APEXCORE-701
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Trivial
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-671) DTConfiguration utility class ValueEntry access level was changed

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-671.
--

Closing after 3.6.0 release

> DTConfiguration utility class ValueEntry access level was changed
> -
>
> Key: APEXCORE-671
> URL: https://issues.apache.org/jira/browse/APEXCORE-671
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
>Priority: Trivial
> Fix For: 3.6.0
>
>
> DTConfiguration.ValueEntry that was public was changed to private. This needs 
> to be reverted back and a better solution needs to be found in future for 
> items that would fall into stram/api.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-644) get-app-package-operators with parent option does not work

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-644.
--

Closing after 3.6.0 release

> get-app-package-operators with parent option does not work
> --
>
> Key: APEXCORE-644
> URL: https://issues.apache.org/jira/browse/APEXCORE-644
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Yatin Chaubal
>Assignee: Sergey Golovko
>Priority: Minor
> Fix For: 3.6.0
>
>
> Issue: get-app-package-operators with -parent option doesnot work
>  
> Steps:
> 1) Start dtcli/apex
> 2) Run get-app-package-operators -parent com.datatorrent.demos.pi 
> /home/hduser/tf2jan/apa/pi-demo-3.4.0.apa
> Expected out output: valid JSON 
> Actual output: 
> {noformat}
> com.datatorrent.stram.cli.ApexCli$CliException: 
> /home/hduser/tf2jan/com.datatorrent.demos.pi does not match any file
> at com.datatorrent.stram.cli.ApexCli.expandFileName(ApexCli.java:918)
> at com.datatorrent.stram.cli.ApexCli.access$000(ApexCli.java:152)
> at 
> com.datatorrent.stram.cli.ApexCli$GetAppPackageOperatorsCommand.execute(ApexCli.java:3827)
> at com.datatorrent.stram.cli.ApexCli$3.run(ApexCli.java:1492)
> {noformat}
> Reference:
> Without -parent option this work fine
> apex> get-app-package-operators  /home/hduser/tf2jan/apa/pi-demo-3.4.0.apa
> {
>   "operatorClasses": [
> {
>   "name": "com.datatorrent.common.util.DefaultDelayOperator",
>   "properties": [],
>   "portTypeInfo": [
> {



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-294) Graceful application shutdown

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-294.
--

Closing after 3.6.0 release

> Graceful application shutdown
> -
>
> Key: APEXCORE-294
> URL: https://issues.apache.org/jira/browse/APEXCORE-294
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Thomas Weise
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> By injecting the end stream tuple into input operators, to replace the 
> current mechanism of forced operator undeploy.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-575) Improve application relaunch time.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-575.
--

Closing after 3.6.0 release

> Improve application relaunch time.
> --
>
> Key: APEXCORE-575
> URL: https://issues.apache.org/jira/browse/APEXCORE-575
> Project: Apache Apex Core
>  Issue Type: Improvement
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> Improve application relaunch time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-594) Plugin support in Apex

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-594.
--

Closing after 3.6.0 release

> Plugin support in Apex
> --
>
> Key: APEXCORE-594
> URL: https://issues.apache.org/jira/browse/APEXCORE-594
> Project: Apache Apex Core
>  Issue Type: New Feature
>    Reporter: Tushar Gosavi
> Fix For: 3.6.0
>
>
> Investigate adding new functionality to apex through user plugins.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-627) Unit test AtMostOnceTest intermittently fails

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-627.
--

Closing after 3.6.0 release

> Unit test AtMostOnceTest intermittently fails
> -
>
> Key: APEXCORE-627
> URL: https://issues.apache.org/jira/browse/APEXCORE-627
> Project: Apache Apex Core
>  Issue Type: Bug
> Environment: The test is reproducible on macOS Sierra, Processor 2.2 
> GHz Intel Core i7, Memory 16GB 1600 MHz DDR3.
>Reporter: Sergey Golovko
>Assignee: Sergey Golovko
>Priority: Minor
> Fix For: 3.6.0
>
>
> The test AtMostOnceTest is not able to reach the criteria to stop the test. 
> And it continue to recover an input operator and rerun the test in a loop.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-616) Application fails to start Kerberised cluster

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-616.
--

Closing after 3.6.0 release

> Application fails to start Kerberised cluster
> -
>
> Key: APEXCORE-616
> URL: https://issues.apache.org/jira/browse/APEXCORE-616
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
> Environment: CDH - any 
>Reporter: Venkatesh Kottapalli
>Assignee: Pramod Immaneni
> Fix For: 3.6.0
>
>
> Application fails to start in a Kerberised cluster with the following 
> exception.
> 2017-01-20 19:28:07,408 [main] ERROR stram.StreamingAppMaster main - Exiting 
> Application Master
> java.lang.NoClassDefFoundError: 
> com/sun/jersey/client/apache4/ApacheHttpClient4Handler
>   at 
> com.datatorrent.stram.util.WebServicesClient.(WebServicesClient.java:153)
>   at 
> com.datatorrent.stram.util.WebServicesClient.(WebServicesClient.java:140)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getAppMasterContainerInfo(StreamingContainerManager.java:481)
>   at 
> com.datatorrent.stram.StreamingContainerManager.init(StreamingContainerManager.java:448)
>   at 
> com.datatorrent.stram.StreamingContainerManager.(StreamingContainerManager.java:420)
>   at 
> com.datatorrent.stram.StreamingContainerManager.getInstance(StreamingContainerManager.java:3065)
>   at 
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:552)
>   at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>   at 
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
> Caused by: java.lang.ClassNotFoundException: 
> com.sun.jersey.client.apache4.ApacheHttpClient4Handler
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>   at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>   ... 9 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-702) Mark ApexPlugin as Evolving

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-702.
--

Closing after 3.6.0 release

> Mark ApexPlugin as Evolving
> ---
>
> Key: APEXCORE-702
> URL: https://issues.apache.org/jira/browse/APEXCORE-702
> Project: Apache Apex Core
>  Issue Type: Sub-task
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
>Priority: Minor
> Fix For: 3.6.0
>
>
> Make ApexPlugin and related classes as Evolving and this part of code is new 
> and might change in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-590) Failed to restart application on MapR

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-590.
--

Closing after 3.6.0 release

> Failed to restart application on MapR
> -
>
> Key: APEXCORE-590
> URL: https://issues.apache.org/jira/browse/APEXCORE-590
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pradeep A. Dalvi
>Assignee: Pradeep A. Dalvi
> Fix For: 3.6.0
>
>
> For restarting application, we try to copy previous app state i.e. 
> checkpoints directory from original app. However checkpoints are not being 
> copied due to incorrect check of source and destination directory path.
> 16/12/16 13:28:32 ERROR fs.MapRFileSystem: Failed to delete path 
> maprfs:/user/dtadmin/datatorrent/apps/application_1481890072066_0006/checkpoints,
>  error: No such file or directory (2)
> 16/12/16 13:28:32 INFO stram.FSRecoveryHandler: Creating 
> maprfs:/user/dtadmin/datatorrent/apps/application_1481890072066_0006/recovery/log
> 16/12/16 13:28:32 INFO stram.StramClient: Ignoring 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/events 
> as it already exists under 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/events
> 16/12/16 13:28:32 INFO stram.StramClient: Ignoring 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/recovery
>  as it already exists under 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/recovery
> 16/12/16 13:28:32 INFO stram.StramClient: Ignoring 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/checkpoints
>  as it already exists under 
> maprfs:///user/dtadmin/datatorrent/apps/application_1481890072066_0004/checkpoints
> 16/12/16 13:28:32 INFO stram.StramClient: Set the environment for the 
> application master



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-608) Streaming Containers use stale RPC proxy after connection is closed

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-608.
--

Closing after 3.6.0 release

> Streaming Containers use stale RPC proxy after connection is closed
> ---
>
> Key: APEXCORE-608
> URL: https://issues.apache.org/jira/browse/APEXCORE-608
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> When Application is killed and Application Master is terminated, Streaming 
> Containers initiate container exit sequence and use disconnected RPC proxy to 
> report errors back to already terminated Application Master.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-678) Shutdown of application should start from input nodes

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-678.
--

Closing after 3.6.0 release

> Shutdown of application should start from input nodes
> -
>
> Key: APEXCORE-678
> URL: https://issues.apache.org/jira/browse/APEXCORE-678
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Bhupesh Chawda
>Assignee: Bhupesh Chawda
> Fix For: 3.6.0
>
>
> Streaming container calls shutdown() for all nodes instead of just input 
> nodes.
> {code}
>   private void stopInputNodes()
>   {
> for (Entry<Integer, Node> e : nodes.entrySet()) {
>   Node node = e.getValue();
>   if (node instanceof InputNode) {
> final Thread thread = e.getValue().context.getThread();
> if (thread == null || !thread.isAlive()) {
>   continue;
> }
>   }
>   node.shutdown(true);
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-674) Change access specifier of DTConfiguration.ValueEntry to private

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-674.
--

Closing after 3.6.0 release

> Change access specifier of DTConfiguration.ValueEntry to private
> 
>
> Key: APEXCORE-674
> URL: https://issues.apache.org/jira/browse/APEXCORE-674
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: Ajay Gupta
>Assignee: Ajay Gupta
>Priority: Trivial
> Fix For: 3.6.0
>
>
> There is no need for ValueEntry class to be public. DTConfiguration provides 
> methods to access the member variables of ValueEntry class. Also, the setter 
> functions set() and setInternal() will be changed to return void instead of 
> ValueEntry.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-570) Prevent upstream operators from getting too far ahead when downstream operators are slow

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-570.
--

Closing after 3.6.0 release

> Prevent upstream operators from getting too far ahead when downstream 
> operators are slow
> 
>
> Key: APEXCORE-570
> URL: https://issues.apache.org/jira/browse/APEXCORE-570
> Project: Apache Apex Core
>  Issue Type: Improvement
>  Components: Buffer Server
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
> Fix For: 3.6.0
>
>
> If the downstream operators are slower than upstream operators then the 
> upstream operators will get ahead and the gap can continue to increase. 
> Provide an option to slow down or temporarily pause the upstream operators 
> when they get too far ahead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-654) Recovery window is not updated when Delay Operator is used along with Partitioned Operators

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-654.
--

Closing after 3.6.0 release

> Recovery window is not updated when Delay Operator is used along with 
> Partitioned Operators
> ---
>
> Key: APEXCORE-654
> URL: https://issues.apache.org/jira/browse/APEXCORE-654
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
> Environment: Hadoop 2.7.2
> Apache Apex 3.5.0
> Apache Apex Malhar 3.6.0
>Reporter: Ambarish Pande
>Assignee: Bhupesh Chawda
>  Labels: DelayOperator
> Fix For: 3.6.0
>
> Attachments: ProblemDag.png
>
>
> Checkpointing is not happening when DefaultDelayOperator is used in a DAG in 
> which some upstream operators are Partitioned.
> When used without partitioning, I can see the operators being check-pointed 
> properly.
> Here is the link of the App source code and also the built apa file.
> https://github.com/ambarishpande/delay-operator-test



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-579) Custom control tuple support

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-579.
--

Closing after 3.6.0 release

> Custom control tuple support
> 
>
> Key: APEXCORE-579
> URL: https://issues.apache.org/jira/browse/APEXCORE-579
> Project: Apache Apex Core
>  Issue Type: New Feature
>Reporter: David Yan
>Assignee: Bhupesh Chawda
> Fix For: 3.6.0
>
>
> The discussion thread is here:
> https://lists.apache.org/thread.html/9accc404a3cb17af2de73b860ed7dff4523a837b4d3c1c73fe0858c5@%3Cdev.apex.apache.org%3E
> We will implement option 1 for now and park option 2 until we have a real use 
> case.
> We probably need a LinkedHashMap to store all the control tuples received in 
> memory (to preserve the order and uniqueness).
> The assumption is that we will not have a large number of control tuples 
> within a streaming window and the memory footprint of control tuples is  
> small.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-605) Suppress bootstrap compiler warning

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-605.
--

Closing after 3.6.0 release

> Suppress bootstrap compiler warning
> ---
>
> Key: APEXCORE-605
> URL: https://issues.apache.org/jira/browse/APEXCORE-605
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> Apex uses animal-sniffer-maven-plugin to enforce 1.7 API compatibility. The 
> following warning caused by {{1.7}} compiler plugin option 
> should be suppressed.
> {noformat}
> [WARNING] bootstrap class path not set in conjunction with -source 1.7
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-592) Returning description field in defaultProperties during apex cli call get-app-package-info

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-592.
--

Closing after 3.6.0 release

> Returning description field in defaultProperties during apex cli call 
> get-app-package-info
> --
>
> Key: APEXCORE-592
> URL: https://issues.apache.org/jira/browse/APEXCORE-592
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Ajay Gupta
>Assignee: Ajay Gupta
>Priority: Minor
> Fix For: 3.6.0
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently for an operator property, only the value field is returned. The 
> operator property can have a description describing the property. This task 
> is created to return this filed for that operator property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-624) Shutdown does not work because of incorrect logic in the AppMaster

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-624.
--

Closing after 3.6.0 release

> Shutdown does not work because of incorrect logic in the AppMaster
> --
>
> Key: APEXCORE-624
> URL: https://issues.apache.org/jira/browse/APEXCORE-624
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
>Priority: Minor
> Fix For: 3.3.1, 3.2.2, 3.5.1, 3.6.0, 3.4.1
>
>
> com.datatorrent.stram.StreamingAppMasterService.execute() calculates 
> numRequestedContainers incorrectly in some cases (e.g. RM container 
> allocation failure) which prevents an application from shutting down when it 
> is requested externally. An example is where we ask RM to remove previous 
> container allocation request (where the count should be decremented but is 
> NOT) and add a new one (where the count should be and IS incremented). 
> Another example is the "alreadyAllocated" case where we release the container 
> and still increment numRequestedContainers which seems wrong. 
> This bug is showing up in multiple Apex deployments.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-663) Application restart not working.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-663.
--

Closing after 3.6.0 release

> Application restart not working.
> 
>
> Key: APEXCORE-663
> URL: https://issues.apache.org/jira/browse/APEXCORE-663
> Project: Apache Apex Core
>  Issue Type: Bug
>    Reporter: Tushar Gosavi
>Assignee: Sandesh
> Fix For: 3.6.0
>
>
> Application restart is not working. Steps to reproduce
> - launch PiDemo
> - Wait for some time (till recovery windowId reaches 120 >)
> - kill the app
> - Try to relaunch using 'launch -originalAppId'



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-563) Have a pointer to log file name and offset in container/operator failure events

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-563.
--

Closing after 3.6.0 release

> Have a pointer to log file name and offset in container/operator failure 
> events
> ---
>
> Key: APEXCORE-563
> URL: https://issues.apache.org/jira/browse/APEXCORE-563
> Project: Apache Apex Core
>  Issue Type: New Feature
>Reporter: Sanjay M Pujare
>Assignee: Priyanka Gugale
> Fix For: 3.6.0
>
>
> The default DailyRollingFileAppender does not take into account of how many 
> backup files to keep and it will result in unbounded growth of log files, 
> especially for long running applications.
> The below is an interesting add-on to the default DailyRollingFileAppender 
> that supports maxBackupIndex.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-680) Review container heartbeat timeout log level messages

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-680.
--

Closing after 3.6.0 release

> Review container heartbeat timeout log level messages 
> --
>
> Key: APEXCORE-680
> URL: https://issues.apache.org/jira/browse/APEXCORE-680
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: Vlad Rozov
>Assignee: Hitesh Kapoor
>Priority: Trivial
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-577) Plugin support to inspect DAG before launch.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-577.
--

Closing after 3.6.0 release

> Plugin support to inspect DAG before launch.
> 
>
> Key: APEXCORE-577
> URL: https://issues.apache.org/jira/browse/APEXCORE-577
> Project: Apache Apex Core
>  Issue Type: Sub-task
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> Support an Visitor like api for DAG, which could be used to write plugins for 
> Apex.
> change DAG interface to add a visitor
> {code}
> registerVisitor(DagVisitor visitor)
> {code}
> The DagVisitor interface
> {code}
> interface DagVisitor {
>   boolean startDAG(DAG* dag);
>   boolean visitOperator(OperatorMeta *ometa)
>   boolean visitStream(StreamMeta *smeta);
>   boolean endDAG()
> }
> {code}
> dev thread discussions
> https://lists.apache.org/thread.html/1c37e0954cee029c9de537cc35ecb35beebdf49aba17de89bd2ce9ed@%3Cdev.apex.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-583) Buffer Server LogicalNode should not be reused by Subscribers

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-583.
--

Closing after 3.6.0 release

> Buffer Server LogicalNode should not be reused by Subscribers
> -
>
> Key: APEXCORE-583
> URL: https://issues.apache.org/jira/browse/APEXCORE-583
> Project: Apache Apex Core
>  Issue Type: Bug
>  Components: Buffer Server
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> Currently LogicalNode supports only one Subscriber and it can't be reused by 
> a new Subscriber.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-700) Make the plugin registration interface uniform

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-700.
--

Closing after 3.6.0 release

> Make the plugin registration interface uniform
> --
>
> Key: APEXCORE-700
> URL: https://issues.apache.org/jira/browse/APEXCORE-700
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
>Priority: Minor
> Fix For: 3.6.0
>
>
> The user-facing plugin registration for DAG setup plugins is slightly 
> different from runtime plugins. It would be better to have a uniform way to 
> do this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-597) BufferServer needs to shutdown all created execution services

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-597.
--

Closing after 3.6.0 release

> BufferServer needs to shutdown all created execution services
> -
>
> Key: APEXCORE-597
> URL: https://issues.apache.org/jira/browse/APEXCORE-597
> Project: Apache Apex Core
>  Issue Type: Bug
>  Components: Buffer Server
>    Reporter: Tushar Gosavi
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> JVM does not shutdown cleanly after receiving shutdown request from Stram. 
> The issue may be because BufferServer is not shutdown. The issue is seen 
> after commit d1646e42bdf5594ef34070594733a7ca10123a3f
> The demo application to recreate the issue is at
> https://github.com/tushargosavi/apex-malhar/blob/shutdownapp/demos/pi/src/main/java/com/datatorrent/demos/pi/Application.java



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-591) SubscribeRequestTuple has wrong buffer size when mask is zero

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-591.
--

Closing after 3.6.0 release

> SubscribeRequestTuple has wrong buffer size when mask is zero
> -
>
> Key: APEXCORE-591
> URL: https://issues.apache.org/jira/browse/APEXCORE-591
> Project: Apache Apex Core
>  Issue Type: Bug
>  Components: Buffer Server
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-645) StramLocalCluster does not wait for master thread termination

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-645.
--

Closing after 3.6.0 release

> StramLocalCluster does not wait for master thread termination
> -
>
> Key: APEXCORE-645
> URL: https://issues.apache.org/jira/browse/APEXCORE-645
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> When StramLocalCluster is started asynchroniously it does not wait for the 
> master thread to terminate



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-598) Embedded mode execution does not use APPLICATION_PATH for checkpointing

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-598.
--

Closing after 3.6.0 release

> Embedded mode execution does not use APPLICATION_PATH for checkpointing
> ---
>
> Key: APEXCORE-598
> URL: https://issues.apache.org/jira/browse/APEXCORE-598
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
> Fix For: 3.6.0
>
>
> Checkpoints are written to a default location, and when tests execute in 
> parallel they will interfere with each other.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-572) Remove dependency on hadoop-common test.jar

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-572.
--

Closing after 3.6.0 release

> Remove dependency on hadoop-common test.jar
> ---
>
> Key: APEXCORE-572
> URL: https://issues.apache.org/jira/browse/APEXCORE-572
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> The dependency on hadoop-common test-jar can be avoided by implementing a few 
> helper methods. It is not necessary to have such dependency.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-558) Do not use yellow color to display command strings in help output

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-558.
--

Closing after 3.6.0 release

> Do not use yellow color to display command strings in help output
> -
>
> Key: APEXCORE-558
> URL: https://issues.apache.org/jira/browse/APEXCORE-558
> Project: Apache Apex Core
>  Issue Type: Bug
> Environment: MacOS, Linux
>Reporter: Sanjay M Pujare
>Assignee: Sanjay M Pujare
>Priority: Minor
> Fix For: 3.6.0
>
>
> Apex CLI Help output (at least on MacOS and Linux terminals) shows command 
> strings in yellow which is extremely hard to read and reduces usability of 
> the CLI. Should be changed to red or purple or just be bolded in black.
> In the following output "alias" and "begin-macro" are in yellow and hard to 
> read.
> apex> help
> GLOBAL COMMANDS EXCEPT WHEN CHANGING LOGICAL PLAN:
> alias alias-name command
>   Create a command alias
> begin-macro name
>   



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-610) Avoid multiple getBytes() calls in Tuple.writeString

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-610.
--

Closing after 3.6.0 release

> Avoid multiple getBytes() calls in Tuple.writeString
> 
>
> Key: APEXCORE-610
> URL: https://issues.apache.org/jira/browse/APEXCORE-610
> Project: Apache Apex Core
>  Issue Type: Bug
>  Components: Buffer Server
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
>Priority: Trivial
> Fix For: 3.6.0
>
>
> Tuple.writeString calls getBytes() multiple times for identifier. each call 
> to getBytes returns new byte array hence the call is costly in terms of 
> memory usage.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-676) Show description for DefaultProperties only when user requests it

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-676.
--

Closing after 3.6.0 release

> Show description for DefaultProperties only when user requests it
> -
>
> Key: APEXCORE-676
> URL: https://issues.apache.org/jira/browse/APEXCORE-676
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sandesh
>Assignee: Ajay Gupta
>Priority: Minor
> Fix For: 3.6.0
>
>
> Apex CLI shows description for default properties, this feature was added 
> recently as a part of
> https://issues.apache.org/jira/browse/APEXCORE-592
> But description should be shown only when a user requests it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-456) Explicitly limit Server.Subscriber to one way communication

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-456.
--

Closing after 3.6.0 release

> Explicitly limit Server.Subscriber to one way communication
> ---
>
> Key: APEXCORE-456
> URL: https://issues.apache.org/jira/browse/APEXCORE-456
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> In the DAG data always flows from upstream to downstream operators and buffer 
> server Server.Subscriber class may explicitly shutdown input stream, does not 
> need to allocate read buffer and listen to OP_READ events. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-687) Update docs, change supported Hadoop version to 2.6

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-687.
--

Closing after 3.6.0 release

> Update docs, change supported Hadoop version to 2.6
> ---
>
> Key: APEXCORE-687
> URL: https://issues.apache.org/jira/browse/APEXCORE-687
> Project: Apache Apex Core
>  Issue Type: Documentation
>  Components: Documentation
>Reporter: Dean Lockgaard
>Assignee: Dean Lockgaard
>Priority: Minor
> Fix For: 3.6.0
>
>
> The doc at 
> https://github.com/apache/apex-core/blob/master/docs/compatibility.md refers 
> to Hadoop version 2.2, but it is now 2.6.  There may be other instances in 
> the docs where this version number would need to be changed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-695) Remove unnecessary interface modifiers

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-695.
--

Closing after 3.6.0 release

> Remove unnecessary interface modifiers
> --
>
> Key: APEXCORE-695
> URL: https://issues.apache.org/jira/browse/APEXCORE-695
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> public modifier is redundant in interfaces



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-690) Embedded app launcher does not apply HEARTBEAT_MONITORING setting

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-690.
--

Closing after 3.6.0 release

> Embedded app launcher does not apply HEARTBEAT_MONITORING setting
> -
>
> Key: APEXCORE-690
> URL: https://issues.apache.org/jira/browse/APEXCORE-690
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
> Fix For: 3.6.0
>
>
> The setting is only applied when true (which is the default). The purpose of 
> the setting is to be able to turn it off though, such as for debugging the 
> DAG in the IDE.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-604) Extend DAG API to allow accessing DAG objects.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-604.
--

Closing after 3.6.0 release

> Extend DAG API to allow accessing DAG objects.
> --
>
> Key: APEXCORE-604
> URL: https://issues.apache.org/jira/browse/APEXCORE-604
> Project: Apache Apex Core
>  Issue Type: Sub-task
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> I am planing extend the DAG api to export internals of the DAG.
> Current DAG does not provide a way to get the list of operators and
> streams with their attributes. Also streamMeta does not provide
> API to access end ports.
> This type of information is needed when external translator (like
> Samoa) are constructing the DAG,
> and we need to apply some transformation or configure the DAG before
> it is run. Planing to extend
> DAG API with following.
> InputPortMeta
> public Operator.InputPort getPortObject();
> OutputPortMeta
> public Operator.OutputPort getPortObject();
> streamMeta
> public  T getSource();
> public  Collection getSinks();
> DAG
> public abstract  Collection getOperators();
> public abstract  Collection getStreams();



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-593) apex cli get-app-package-info could not retrieve properties defined in properties.xml

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-593.
--

Closing after 3.6.0 release

> apex cli get-app-package-info could not retrieve properties defined in 
> properties.xml
> -
>
> Key: APEXCORE-593
> URL: https://issues.apache.org/jira/browse/APEXCORE-593
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Yogi Devendra
>Assignee: Vikram Patil
>Priority: Minor
> Fix For: 3.6.0
>
>
> If application defines properties in properties.xml; such properties should 
> be available to populateDAG() method when invoked from processAppDirectory() 
> in get-app-package-info.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-471) Requests for container allocation are not resubmitted

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-471.
--

Closing after 3.6.0 release

> Requests for container allocation are not resubmitted
> -
>
> Key: APEXCORE-471
> URL: https://issues.apache.org/jira/browse/APEXCORE-471
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Vlad Rozov
>Assignee: Sandesh
> Fix For: 3.6.0
>
>
> When Yarn cluster has a limited number of available resources, requests 
> should be resubmitted. BlacklistBasedResourceRequestHandler does not properly 
> handle case when resources are limited.
> Apex has 2 schedulers, BlacklistedResource* is a default option on CDH.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-691) Use type inference for generic instance creation

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-691.
--

Closing after 3.6.0 release

> Use type inference for generic instance creation
> 
>
> Key: APEXCORE-691
> URL: https://issues.apache.org/jira/browse/APEXCORE-691
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> The current Java language level allows using type inference for generic 
> instance creation making constructs like
> {noformat}
> Map <String, String> map = new HashMap<String, String>();
> {noformat}
> unnecessary.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-634) Unifier attributes are not set for modules in DAG

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-634.
--

Closing after 3.6.0 release

> Unifier attributes are not set for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>Priority: Minor
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-677) Avoid starting StramLocalCluster in StreamingContainerManagerTest.testAppDataSources

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-677.
--

Closing after 3.6.0 release

> Avoid starting StramLocalCluster in 
> StreamingContainerManagerTest.testAppDataSources
> 
>
> Key: APEXCORE-677
> URL: https://issues.apache.org/jira/browse/APEXCORE-677
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Trivial
> Fix For: 3.6.0
>
>
> testAppDataSources does not wait for the StramLocalCluster to be initialized, 
> so it does not need to start it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-611) Stram Event Log Levels

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-611.
--

Closing after 3.6.0 release

> Stram Event Log Levels
> --
>
> Key: APEXCORE-611
> URL: https://issues.apache.org/jira/browse/APEXCORE-611
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Ajay Gupta
>Assignee: Ajay Gupta
>Priority: Minor
> Fix For: 3.6.0
>
>
> Provide log levels for Stram events. Such as INFO, WARN, ERROR
> Eg: 
> 1. Start Container, Start Operator are INFO level events
> 2. OperatorError is ERROR level event
> 3. Stop Container, Stop Operator are WARN level events
> Log level for events can help in user experience when showing the event list 
> in a GUI. eg: In datatorrent gateway UI, we want to provide color-coded log 
> levels so that user can focus more on ERROR and WARN events.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-580) Interface for processing and emitting control tuples

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-580.
--

Closing after 3.6.0 release

> Interface for processing and emitting control tuples
> 
>
> Key: APEXCORE-580
> URL: https://issues.apache.org/jira/browse/APEXCORE-580
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: David Yan
>Assignee: Bhupesh Chawda
> Fix For: 3.6.0
>
>
> DefaultOutputPort needs to have a emitControl method so that operator code 
> can call to emit a control tuple.
> DefaultInputPort needs to have a processControl method so that operator would 
> be able to act on the arrival of a control tuple.
> Similar to a regular data tuple, we also need to provide a way for the user 
> to provide custom serialization for the control tuple.
> We need to design this so that the default behavior is to propagate control 
> tuples to all output ports, and it should allow the user to easily change 
> that behavior. The user can selectively propagate control tuples to certain 
> output ports, or block the propagation altogether.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-692) Apex Dev Setup doc should reference download page

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-692.
--

Closing after 3.6.0 release

> Apex Dev Setup doc should reference download page
> -
>
> Key: APEXCORE-692
> URL: https://issues.apache.org/jira/browse/APEXCORE-692
> Project: Apache Apex Core
>  Issue Type: Documentation
>  Components: Documentation
>Reporter: Dean Lockgaard
>Assignee: Dean Lockgaard
>Priority: Minor
> Fix For: 3.6.0
>
>
> In the Sandbox section of the Apache Apex Development Environment Setup 
> documentation 
> (https://apex.apache.org/docs/apex/apex_development_setup/#sandbox), 
> instructions are provided for a vendor-specific Sandbox.
> These instructions should be changed to reference the Apache Apex Sandbox 
> instead (https://hub.docker.com/r/apacheapex/sandbox).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-504) Possible race condition in StreamingContainerAgent.getStreamCodec()

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-504.
--

Closing after 3.6.0 release

> Possible race condition in StreamingContainerAgent.getStreamCodec()
> ---
>
> Key: APEXCORE-504
> URL: https://issues.apache.org/jira/browse/APEXCORE-504
> Project: Apache Apex Core
>  Issue Type: Bug
>Affects Versions: 3.3.0, 3.2.1, 3.4.0, 3.5.0
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.3.1, 3.2.2, 3.5.1, 3.6.0, 3.4.1
>
>
> StreamingContainerAgent.getStreamCodec() may be called from multiple RPC 
> heartbeat processing threads for the same inputPortMeta (for an input port of 
> a partitioned operator) that leads to a race condition in the following code:
> {noformat}
>   StreamCodec codec = inputPortMeta.getValue(PortContext.STREAM_CODEC);
>   if (codec == null) {
> // it cannot be this object that gets returned. Depending on this 
> value is dangerous 
> codec = inputPortMeta.getPortObject().getStreamCodec();
> if (codec != null) {
>   // don't create codec multiple times - it will assign a new 
> identifier
>   inputPortMeta.getAttributes().put(PortContext.STREAM_CODEC, codec);
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-686) AppPackage fails when .apa contains zero length stored entry

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-686.
--

Closing after 3.6.0 release

> AppPackage fails when .apa contains zero length stored entry
> 
>
> Key: APEXCORE-686
> URL: https://issues.apache.org/jira/browse/APEXCORE-686
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> JarInputStream/ZipInputStream does not support valid formats of .jar/.zip file
> {noformat}
> java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor
>   at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:309)
>   at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:121)
>   at java.util.jar.JarInputStream.(JarInputStream.java:83)
>   at java.util.jar.JarInputStream.(JarInputStream.java:62)
>   at com.datatorrent.stram.client.AppPackage.(AppPackage.java:215)
>   at com.datatorrent.stram.client.AppPackage.(AppPackage.java:196)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-595) Master incorrectly updates committedWindowId when all partitions are terminated.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-595.
--

Closing after 3.6.0 release

> Master incorrectly updates committedWindowId when all partitions are 
> terminated.
> 
>
> Key: APEXCORE-595
> URL: https://issues.apache.org/jira/browse/APEXCORE-595
> Project: Apache Apex Core
>  Issue Type: Bug
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> When all partitions are terminated application master incorrectly updates the 
> committedWindowId to Long.MAX_VALUE and send purge requet to StramChild 
> causing exception in StramChild.
> The problematic code is in StreamingContainerManager.updateCheckpoints.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-581) Delivery of Custom Control Tuples

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-581.
--

Closing after 3.6.0 release

> Delivery of Custom Control Tuples
> -
>
> Key: APEXCORE-581
> URL: https://issues.apache.org/jira/browse/APEXCORE-581
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: David Yan
>Assignee: Bhupesh Chawda
> Fix For: 3.6.0
>
>
> The behavior should be as follow:
> - The control tuples should only be sent to downstream at streaming window 
> boundaries
> - The control tuples should be sent to all partitions downstream
> - The control tuples should be sent in the same order of arrival.
> - Within a streaming window, do not send the same control tuple twice, even 
> if the same control tuple is received multiple times within that window. This 
> is possible if the operator has two input ports. (The LinkedHashMap should be 
> easily able to ensure both order and uniqueness.)
> - The delivery of control tuples needs to stop at DelayOperator. 
> - When a streaming window is committed, remove the associated LinkedHashMap 
> that belong to windows with IDs that are less than the committed window
> - It's safe to assume the control tuples are rare enough and can fit in memory
> This will involve an additional MessageType to represent a custom control 
> tuple. 
> We probably need to have a data structure (possibly a LinkedHashMap) per 
> streaming window that stores the control tuple in the buffer server.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-709) Refactor changes done through APEXCORE-575

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-709.
--

Closing after 3.6.0 release

> Refactor changes done through APEXCORE-575
> --
>
> Key: APEXCORE-709
> URL: https://issues.apache.org/jira/browse/APEXCORE-709
> Project: Apache Apex Core
>  Issue Type: Bug
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
>Priority: Minor
> Fix For: 3.6.0
>
>
> Handle additional review comments on pull request 422.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-649) Infrastructure for user define stram event listeners.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-649.
--

Closing after 3.6.0 release

> Infrastructure for user define stram event listeners.
> -
>
> Key: APEXCORE-649
> URL: https://issues.apache.org/jira/browse/APEXCORE-649
> Project: Apache Apex Core
>  Issue Type: Sub-task
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> As  suggested while working on Visitor API, I have came up with following 
> proposal. The idea is to support user defined DAG listeners.  The plan is to
> support limitated set of events for now and we could add more events
> in future.
> For the details functionality propvided check attached document.
> Please provide feedback on provided proposal.
> https://docs.google.com/document/d/1SAIE0EjnCumrB1jKJSnbGvcml47Po8ZHFthfcbNJQgU/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-522) Promote singleton usage pattern for String2String, Long2String and other StringCodecs

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-522.
--

Closing after 3.6.0 release

> Promote singleton usage pattern for String2String, Long2String and other 
> StringCodecs
> -
>
> Key: APEXCORE-522
> URL: https://issues.apache.org/jira/browse/APEXCORE-522
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> String2String codec does not hold state, a single instance of the same 
> String2String class may be reused.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-480) Change the container log name from dt.log to apex.log

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-480.
--

Closing after 3.6.0 release

> Change the container log name from dt.log to apex.log
> -
>
> Key: APEXCORE-480
> URL: https://issues.apache.org/jira/browse/APEXCORE-480
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Siyuan Hua
>Assignee: Thomas Weise
>Priority: Minor
> Fix For: 3.6.0
>
>
> The log file name should be changed to avoid growing user dependency on 
> legacy naming. New users should not be exposed to this. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-658) Replace dt. prefix with apex. for configuration keys

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-658.
--

Closing after 3.6.0 release

> Replace dt. prefix with apex. for configuration keys  
> --
>
> Key: APEXCORE-658
> URL: https://issues.apache.org/jira/browse/APEXCORE-658
> Project: Apache Apex Core
>  Issue Type: Task
>Reporter: Thomas Weise
>Assignee: Thomas Weise
> Fix For: 3.6.0
>
>
> Users should be able to configure their applications with keys that use apex. 
> as prefix and this is how it should be shown in examples also. It should be  
> possible to make this change w/o breaking backward compatibility.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (APEXCORE-648) Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair()

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi closed APEXCORE-648.
--

Closing after 3.6.0 release

> Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair()
> ---
>
> Key: APEXCORE-648
> URL: https://issues.apache.org/jira/browse/APEXCORE-648
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> DefaultStatefulStreamCodec.toDataStatePair() calls Kryo Output.toBytes() that 
>  creates new byte[] and copies serialized data to the newly allocated array. 
> It is not necessary as the output of toDataStatePair() will be copied again 
> in PayloadTuple.getSerializedTuple().



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE] Apache Apex Core Release 3.6.0 (RC1)

[Tushar Gosavi]

As a committer, I do not have write permissions to release svn repository.
Need help from PMC member to update the release repository. Any volunteers??

Also one more clarification needed with respect to updating semantic
versioning check. Does it needs to be done for both release-3.6.0 and
master branch?

I will push the documentations after release repository is updated.

Thanks,
-Tushar.


On Fri, May 5, 2017 at 11:35 AM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> The vote is concluded and passes. Thanks everyone for voting and verifying
> the release.
>
> binding +1 (4)
>
> Vlad Rozov
> Thomas Weise
> Amol Kekre
> Pramod Immaneni
>
>
> non-binding +1 (2)
> Bhupesh Chawda
> Chaitanya Chebolu
>
> No other votes.
>
> Will complete the release activities.
>
> Thanks,
> Tushar.
>
>
> On Thu, May 4, 2017 at 12:05 PM, Chaitanya Chebolu <
> chaita...@datatorrent.com> wrote:
>
>> +1
>>
>> 1) Verified the file Integrity check.
>> 2) Verified the source code using tar file.
>> - Extracted tar file and compiled.
>> - No binary files
>> - LICENSE, NOTICE, README, CHANGELOG.md exist.
>> - rat check
>> 3) Launched PI demo.
>>
>> Regards,
>> Chaitanya
>>
>> On Wed, May 3, 2017 at 7:07 AM, Pramod Immaneni <pra...@datatorrent.com>
>> wrote:
>>
>> > +1 (binding)
>> >
>> > verified file integrity
>> > no unexpected binary files
>> > existence of README.md, NOTICE, LICENSE and CHANGELOG.md files
>> > checked build and licenses
>> > launched and ran pi demo
>> >
>> > Minor nitpicks
>> >
>> > CHANGELOG.md has a line in the beginning of the file that has a date in
>> > future "Version 3.6.0 - 2017-05-04"
>> > Please have your key signed by some of the others in the KEYS file,
>> right
>> > now it is self-signed
>> >
>> > Thanks
>> >
>> > On Mon, May 1, 2017 at 11:49 AM, Tushar Gosavi <tus...@datatorrent.com>
>> > wrote:
>> >
>> > > Dear Community,
>> > >
>> > > Please vote on the following Apache Apex Core 3.6.0 release candidate
>> 1.
>> > >
>> > > This release adds the support for custom control tuples, experimental
>> > > support for plugins
>> > > and other improvements and important bug fixes.
>> > >
>> > > This is a source release with binary artifacts published to Maven.
>> > >
>> > > List of all issues fixed: https://s.apache.org/HQ0r
>> > >
>> > > Staging directory
>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-core
>> -3.6.0-RC1/
>> > > Source zip:
>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
>> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.zip
>> > > Source tar.gz:
>> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
>> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.tar.gz
>> > > Maven staging repository:
>> > > https://repository.apache.org/content/repositories/orgapacheapex-1028
>> > >
>> > > Git source:
>> > > https://git-wip-us.apache.org/repos/asf?p=apex-core.git;a=
>> > > commit;h=refs/tags/v3.6.0-RC1
>> > > (commit: 5a517348ae497c06150f32ce39b6915588e92510)
>> > >
>> > > PGP key:
>> > > http://pgp.mit.edu:11371/pks/lookup?op=vindex=tushar@
>> apache.org
>> > > KEYS file:
>> > > https://dist.apache.org/repos/dist/release/apex/KEYS
>> > >
>> > > More information at:
>> > > http://apex.apache.org
>> > >
>> > > Please try the release and vote; vote will be open for at least 72
>> hours.
>> > >
>> > > [ ] +1 approve (and what verification was done)
>> > > [ ] -1 disapprove (and reason why)
>> > >
>> > > http://www.apache.org/foundation/voting.html
>> > >
>> > > How to verify release candidate:
>> > >
>> > > http://apex.apache.org/verification.html
>> > >
>> > > Thanks,
>> > > Tushar.
>> > >
>> >
>>
>>
>>
>> --
>>
>> *Chaitanya*
>>
>> Software Engineer
>>
>> E: chaita...@datatorrent.com | Twitter: @chaithu1403
>>
>> www.datatorrent.com  |  apex.apache.org
>>
>
>

Re: [VOTE] Apache Apex Core Release 3.6.0 (RC1)

[Tushar Gosavi]

The vote is concluded and passes. Thanks everyone for voting and verifying
the release.

binding +1 (4)

Vlad Rozov
Thomas Weise
Amol Kekre
Pramod Immaneni


non-binding +1 (2)
Bhupesh Chawda
Chaitanya Chebolu

No other votes.

Will complete the release activities.

Thanks,
Tushar.


On Thu, May 4, 2017 at 12:05 PM, Chaitanya Chebolu <
chaita...@datatorrent.com> wrote:

> +1
>
> 1) Verified the file Integrity check.
> 2) Verified the source code using tar file.
> - Extracted tar file and compiled.
> - No binary files
> - LICENSE, NOTICE, README, CHANGELOG.md exist.
> - rat check
> 3) Launched PI demo.
>
> Regards,
> Chaitanya
>
> On Wed, May 3, 2017 at 7:07 AM, Pramod Immaneni <pra...@datatorrent.com>
> wrote:
>
> > +1 (binding)
> >
> > verified file integrity
> > no unexpected binary files
> > existence of README.md, NOTICE, LICENSE and CHANGELOG.md files
> > checked build and licenses
> > launched and ran pi demo
> >
> > Minor nitpicks
> >
> > CHANGELOG.md has a line in the beginning of the file that has a date in
> > future "Version 3.6.0 - 2017-05-04"
> > Please have your key signed by some of the others in the KEYS file, right
> > now it is self-signed
> >
> > Thanks
> >
> > On Mon, May 1, 2017 at 11:49 AM, Tushar Gosavi <tus...@datatorrent.com>
> > wrote:
> >
> > > Dear Community,
> > >
> > > Please vote on the following Apache Apex Core 3.6.0 release candidate
> 1.
> > >
> > > This release adds the support for custom control tuples, experimental
> > > support for plugins
> > > and other improvements and important bug fixes.
> > >
> > > This is a source release with binary artifacts published to Maven.
> > >
> > > List of all issues fixed: https://s.apache.org/HQ0r
> > >
> > > Staging directory
> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> core-3.6.0-RC1/
> > > Source zip:
> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.zip
> > > Source tar.gz:
> > > https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> > > core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.tar.gz
> > > Maven staging repository:
> > > https://repository.apache.org/content/repositories/orgapacheapex-1028
> > >
> > > Git source:
> > > https://git-wip-us.apache.org/repos/asf?p=apex-core.git;a=
> > > commit;h=refs/tags/v3.6.0-RC1
> > > (commit: 5a517348ae497c06150f32ce39b6915588e92510)
> > >
> > > PGP key:
> > > http://pgp.mit.edu:11371/pks/lookup?op=vindex=tus...@apache.org
> > > KEYS file:
> > > https://dist.apache.org/repos/dist/release/apex/KEYS
> > >
> > > More information at:
> > > http://apex.apache.org
> > >
> > > Please try the release and vote; vote will be open for at least 72
> hours.
> > >
> > > [ ] +1 approve (and what verification was done)
> > > [ ] -1 disapprove (and reason why)
> > >
> > > http://www.apache.org/foundation/voting.html
> > >
> > > How to verify release candidate:
> > >
> > > http://apex.apache.org/verification.html
> > >
> > > Thanks,
> > > Tushar.
> > >
> >
>
>
>
> --
>
> *Chaitanya*
>
> Software Engineer
>
> E: chaita...@datatorrent.com | Twitter: @chaithu1403
>
> www.datatorrent.com  |  apex.apache.org
>

Re: [VOTE] Apache Apex Core Release 3.6.0 (RC1)

[Tushar Gosavi]

Hi Thomas,

I had pushed documentation to the apex-site repository, published them now
under https://apex.apache.org/docs/apex-3.6/

For javadocs had updated the buildbot configuration, somehow the java
documentation is not prepared yet. looking into it.

Thanks,
- Tushar.


On Tue, May 2, 2017 at 8:50 PM, Thomas Weise <t...@apache.org> wrote:

> +1 (binding)
>
> - verified signatures and hashes
> - build passes: mvn clean apache-rat:check install -Dlicense.skip=false
> - successfully run the Apache Beam validate runner tests with this version
>
> Minor issue:
>
> The email should contain documentation links (javadoc and user
> documentation).
> Was the documentation published?
>
> Thanks,
> Thomas
>
>
> On Tue, May 2, 2017 at 6:31 AM, Bhupesh Chawda <bhup...@datatorrent.com>
> wrote:
>
> > +1
> >
> > Checked the following:
> > 1. Signatures and checksums okay
> > 2. Build successful with tests
> > 3. Presence of README.md, LICENSE, NOTICE and CHANGELOG.md files
> > 4. Could launch pi demo successfully
> >
> > ~ Bhupesh
> >
> >
> >
> >
> > ___
> >
> > Bhupesh Chawda
> >
> > E: bhup...@datatorrent.com | Twitter: @bhupeshsc
> >
> > www.datatorrent.com  |  apex.apache.org
> >
> >
> >
> > On Tue, May 2, 2017 at 2:01 AM, Vlad Rozov <v.ro...@datatorrent.com>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > - verified release signature and hashes
> > > - verified LICENSE, NOTICE, README.md and CHANGELOG.md are present
> > > - no unexpected binary files in the source distribution
> > > - verified build and tests run clean "mvn clean apache-rat:check verify
> > > -Dlicense.skip=false install"
> > >
> > > Thank you,
> > >
> > > Vlad
> > >
> > >
> > > On 5/1/17 11:49, Tushar Gosavi wrote:
> > >
> > >> Dear Community,
> > >>
> > >> Please vote on the following Apache Apex Core 3.6.0 release candidate
> 1.
> > >>
> > >> This release adds the support for custom control tuples, experimental
> > >> support for plugins
> > >> and other improvements and important bug fixes.
> > >>
> > >> This is a source release with binary artifacts published to Maven.
> > >>
> > >> List of all issues fixed: https://s.apache.org/HQ0r
> > >>
> > >> Staging directory
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-apex-
> core-3.6.0-RC1/
> > >> Source zip:
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-apex-core
> > >> -3.6.0-RC1/apache-apex-core-3.6.0-source-release.zip
> > >> Source tar.gz:
> > >> https://dist.apache.org/repos/dist/dev/apex/apache-apex-core
> > >> -3.6.0-RC1/apache-apex-core-3.6.0-source-release.tar.gz
> > >> Maven staging repository:
> > >> https://repository.apache.org/content/repositories/orgapacheapex-1028
> > >>
> > >> Git source:
> > >> https://git-wip-us.apache.org/repos/asf?p=apex-core.git;a=co
> > >> mmit;h=refs/tags/v3.6.0-RC1
> > >> (commit: 5a517348ae497c06150f32ce39b6915588e92510)
> > >>
> > >> PGP key:
> > >> http://pgp.mit.edu:11371/pks/lookup?op=vindex=
> tus...@apache.org
> > >> KEYS file:
> > >> https://dist.apache.org/repos/dist/release/apex/KEYS
> > >>
> > >> More information at:
> > >> http://apex.apache.org
> > >>
> > >> Please try the release and vote; vote will be open for at least 72
> > hours.
> > >>
> > >> [ ] +1 approve (and what verification was done)
> > >> [ ] -1 disapprove (and reason why)
> > >>
> > >> http://www.apache.org/foundation/voting.html
> > >>
> > >> How to verify release candidate:
> > >>
> > >> http://apex.apache.org/verification.html
> > >>
> > >> Thanks,
> > >> Tushar.
> > >>
> > >>
> > >
> >
>

[VOTE] Apache Apex Core Release 3.6.0 (RC1)

[Tushar Gosavi]

Dear Community,

Please vote on the following Apache Apex Core 3.6.0 release candidate 1.

This release adds the support for custom control tuples, experimental
support for plugins
and other improvements and important bug fixes.

This is a source release with binary artifacts published to Maven.

List of all issues fixed: https://s.apache.org/HQ0r

Staging directory
https://dist.apache.org/repos/dist/dev/apex/apache-apex-core-3.6.0-RC1/
Source zip:
https://dist.apache.org/repos/dist/dev/apex/apache-apex-core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.zip
Source tar.gz:
https://dist.apache.org/repos/dist/dev/apex/apache-apex-core-3.6.0-RC1/apache-apex-core-3.6.0-source-release.tar.gz
Maven staging repository:
https://repository.apache.org/content/repositories/orgapacheapex-1028

Git source:
https://git-wip-us.apache.org/repos/asf?p=apex-core.git;a=commit;h=refs/tags/v3.6.0-RC1
(commit: 5a517348ae497c06150f32ce39b6915588e92510)

PGP key:
http://pgp.mit.edu:11371/pks/lookup?op=vindex=tus...@apache.org
KEYS file:
https://dist.apache.org/repos/dist/release/apex/KEYS

More information at:
http://apex.apache.org

Please try the release and vote; vote will be open for at least 72 hours.

[ ] +1 approve (and what verification was done)
[ ] -1 disapprove (and reason why)

http://www.apache.org/foundation/voting.html

How to verify release candidate:

http://apex.apache.org/verification.html

Thanks,
Tushar.

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

On Sat, Apr 29, 2017 at 2:16 AM, Thomas Weise <t...@apache.org> wrote:

> The remote 3.6.0 branch stays at 3.6.0-SNAPSHOT until release is complete.
> The tag has the release version.
>
>
There is no step to push release branch to remote while being at
3.6.0-SNAPSHOT. Is this step missing from release document?



> Your local branch obviously needs to be same as you create the tag from it.
>
> Perhaps we need to word this differently in the instructions.
>
> Thomas
>
>
> On Fri, Apr 28, 2017 at 12:57 PM, Tushar Gosavi <tus...@datatorrent.com>
> wrote:
>
> > I am executing following command to prepare an RC for ApexCore 3.6.0.
> >
> > git checkout -b release-3.6 master
> >  git checkout master
> >  dv=3.6.0-SNAPSHOT
> >  rv=3.7.0-SNAPSHOT
> >  for a in `git grep -l "${dv}"`; do echo $a; sed -i
> > 's/'"${dv}"'/'"${rv}"'/g' $a; done
> >  git commit --author "Apex Dev <dev@apex.apache.org>" -am "Preparing for
> > 3.7.0 development"
> >  git push origin master
> >  git checkout release-3.6
> >  ruby ~/add-since.rb `pwd` -s 3.6.0
> >  # update changelog
> >  cp /tmp/CHANGELOG.md CHANGELOG.md
> >  git commit --author "Apex Dev <dev@apex.apache.org>" -am "Add @since
> tags
> > and update change log for release 3.6.0"
> >  dv=3.6.0-SNAPSHOT
> >  rv=3.6.0
> >  for a in `git grep -l "${dv}"`; do echo $a; sed -i
> > 's/'"${dv}"'/'"${rv}"'/g' $a; done
> >  git commit --author "Apex Dev <dev@apex.apache.org>" -am "Preparing to
> > release 3.6.0-RC1"
> >  git tag -a v3.6.0-RC1 -m "Release 3.6.0-RC1"
> >  git push origin v3.6.0-RC1
> >
> >
> > At this point git diff gives empty output between the tag and branch.
> > git diff v3.6.0-RC1 release-3.6
> >
> >
> > Also git log is same for tag and branch, but a comment on release page
> just
> > after  *git push origin v3.6.0-RC1* is
> >
> > The only difference between release branch and tag is this final version
> > number change. The branch stays at -SNAPSHOT version.
> >
> >
> > Is any step missing in above commands?
> >
> > Thanks & Regards,
> > -Tushar.
> >
> >
> >
> > On Thu, Apr 27, 2017 at 2:29 PM, Tushar Gosavi <tus...@datatorrent.com>
> > wrote:
> >
> > > All the issues marked as 3.6 have been resolved. I will go ahead and
> > > prepare RC1 candidate.
> > >
> > > Thanks,
> > > - Tushar.
> > >
> > > On Wed, Apr 26, 2017 at 2:30 PM, Tushar Gosavi <tus...@datatorrent.com
> >
> > > wrote:
> > >
> > >> These errors went away after switching to 1.7 version of java for
> build.
> > >>
> > >> Thanks,
> > >> -Tushar.
> > >>
> > >>
> > >> On Wed, Apr 26, 2017 at 12:50 PM, Tushar Gosavi <
> tus...@datatorrent.com
> > >
> > >> wrote:
> > >>
> > >>> I am following process specified at http://apex.apache.org/
> > release.html to
> > >>> prepare a release candidate for 3.6.
> > >>>
> > >>> I am getting error while executing command
> > >>> mvn clean apache-rat:check deploy -Papache-release -DskipTests
> > >>>
> > >>>
> > >>> 100 warnings
> > >>> [INFO] 
> > >>> 
> > >>> [INFO] Reactor Summary:
> > >>> [INFO]
> > >>> [INFO] Apache Apex  SUCCESS
> > >>> [06:51 min]
> > >>> [INFO] Apache Apex API  FAILURE
> [
> > >>>  5.255 s]
> > >>> [INFO] Apache Apex Common Library . SKIPPED
> > >>> [INFO] Apache Apex Buffer Server .. SKIPPED
> > >>> [INFO] Apache Apex Stream Processing Engine ... SKIPPED
> > >>> [INFO] Apache Apex Application Maven Archetype  SKIPPED
> > >>> [INFO] Apache Apex App Configuration Maven Archetype .. SKIPPED
> > >>> [INFO] 
> > >>> 
> > >>> [INFO] BUILD FAILURE
> > >>> [INFO] 
> > >>> 
> > >>

[jira] [Updated] (APEXCORE-715) Remove unnecessary @Evolving annotation in engine

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-715:
---
Fix Version/s: 3.6.0

> Remove unnecessary @Evolving annotation in engine
> -
>
> Key: APEXCORE-715
> URL: https://issues.apache.org/jira/browse/APEXCORE-715
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Fix For: 3.6.0
>
>
> \@Evolving annotation in the engine are not necessary and introduce 
> unnecessarily practice and wrong impression that some interfaces and/or 
> classes are more stable compared to others. All interfaces and classes in the 
> engine are not subject to the semantic version checks and may change from a 
> release to a release. The same applies to the buffer server.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

I am executing following command to prepare an RC for ApexCore 3.6.0.

git checkout -b release-3.6 master
 git checkout master
 dv=3.6.0-SNAPSHOT
 rv=3.7.0-SNAPSHOT
 for a in `git grep -l "${dv}"`; do echo $a; sed -i
's/'"${dv}"'/'"${rv}"'/g' $a; done
 git commit --author "Apex Dev <dev@apex.apache.org>" -am "Preparing for
3.7.0 development"
 git push origin master
 git checkout release-3.6
 ruby ~/add-since.rb `pwd` -s 3.6.0
 # update changelog
 cp /tmp/CHANGELOG.md CHANGELOG.md
 git commit --author "Apex Dev <dev@apex.apache.org>" -am "Add @since tags
and update change log for release 3.6.0"
 dv=3.6.0-SNAPSHOT
 rv=3.6.0
 for a in `git grep -l "${dv}"`; do echo $a; sed -i
's/'"${dv}"'/'"${rv}"'/g' $a; done
 git commit --author "Apex Dev <dev@apex.apache.org>" -am "Preparing to
release 3.6.0-RC1"
 git tag -a v3.6.0-RC1 -m "Release 3.6.0-RC1"
 git push origin v3.6.0-RC1


At this point git diff gives empty output between the tag and branch.
git diff v3.6.0-RC1 release-3.6


Also git log is same for tag and branch, but a comment on release page just
after  *git push origin v3.6.0-RC1* is

The only difference between release branch and tag is this final version
number change. The branch stays at -SNAPSHOT version.


Is any step missing in above commands?

Thanks & Regards,
-Tushar.



On Thu, Apr 27, 2017 at 2:29 PM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> All the issues marked as 3.6 have been resolved. I will go ahead and
> prepare RC1 candidate.
>
> Thanks,
> - Tushar.
>
> On Wed, Apr 26, 2017 at 2:30 PM, Tushar Gosavi <tus...@datatorrent.com>
> wrote:
>
>> These errors went away after switching to 1.7 version of java for build.
>>
>> Thanks,
>> -Tushar.
>>
>>
>> On Wed, Apr 26, 2017 at 12:50 PM, Tushar Gosavi <tus...@datatorrent.com>
>> wrote:
>>
>>> I am following process specified at http://apex.apache.org/release.html to
>>> prepare a release candidate for 3.6.
>>>
>>> I am getting error while executing command
>>> mvn clean apache-rat:check deploy -Papache-release -DskipTests
>>>
>>>
>>> 100 warnings
>>> [INFO] 
>>> 
>>> [INFO] Reactor Summary:
>>> [INFO]
>>> [INFO] Apache Apex  SUCCESS
>>> [06:51 min]
>>> [INFO] Apache Apex API  FAILURE [
>>>  5.255 s]
>>> [INFO] Apache Apex Common Library . SKIPPED
>>> [INFO] Apache Apex Buffer Server .. SKIPPED
>>> [INFO] Apache Apex Stream Processing Engine ... SKIPPED
>>> [INFO] Apache Apex Application Maven Archetype  SKIPPED
>>> [INFO] Apache Apex App Configuration Maven Archetype .. SKIPPED
>>> [INFO] 
>>> 
>>> [INFO] BUILD FAILURE
>>> [INFO] 
>>> 
>>> [INFO] Total time: 06:56 min
>>> [INFO] Finished at: 2017-04-26T12:41:35+05:30
>>> [INFO] Final Memory: 74M/903M
>>> [INFO] 
>>> 
>>> [ERROR] Failed to execute goal 
>>> org.apache.maven.plugins:maven-javadoc-plugin:2.9:jar
>>> (attach-javadocs) on project apex-api: MavenReportException: Error while
>>> creating archive:
>>> [ERROR] Exit code: 1 - /home/tushar/work/apex/core/ap
>>> i/src/main/java/com/datatorrent/api/Attribute.java:116: warning: no
>>> description for @param
>>> [ERROR] * @param 
>>> [ERROR] ^
>>> [ERROR] 
>>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:117:
>>> warning: no description for @param
>>> [ERROR] * @param key
>>> [ERROR] ^
>>> [ERROR] 
>>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:146:
>>> warning: no description for @throws
>>> [ERROR] * @throws java.lang.CloneNotSupportedException
>>> [ERROR] ^
>>> [ERROR] 
>>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/AutoMetric.java:59:
>>> error: self-closing element not allowed
>>> [ERROR] * It aggregates metrics from multiple physical partitions of an
>>> operator to a logical one.
>>> [ERROR] ^
>>>
>>> What is the correct command to build the release.
>>>
>>> Thanks,
>>> - Tushar.
>>>
>>>
>>
>

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

All the issues marked as 3.6 have been resolved. I will go ahead and
prepare RC1 candidate.

Thanks,
- Tushar.

On Wed, Apr 26, 2017 at 2:30 PM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> These errors went away after switching to 1.7 version of java for build.
>
> Thanks,
> -Tushar.
>
>
> On Wed, Apr 26, 2017 at 12:50 PM, Tushar Gosavi <tus...@datatorrent.com>
> wrote:
>
>> I am following process specified at http://apex.apache.org/release.html to
>> prepare a release candidate for 3.6.
>>
>> I am getting error while executing command
>> mvn clean apache-rat:check deploy -Papache-release -DskipTests
>>
>>
>> 100 warnings
>> [INFO] 
>> 
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Apex  SUCCESS
>> [06:51 min]
>> [INFO] Apache Apex API  FAILURE [
>>  5.255 s]
>> [INFO] Apache Apex Common Library . SKIPPED
>> [INFO] Apache Apex Buffer Server .. SKIPPED
>> [INFO] Apache Apex Stream Processing Engine ... SKIPPED
>> [INFO] Apache Apex Application Maven Archetype  SKIPPED
>> [INFO] Apache Apex App Configuration Maven Archetype .. SKIPPED
>> [INFO] 
>> 
>> [INFO] BUILD FAILURE
>> [INFO] 
>> 
>> [INFO] Total time: 06:56 min
>> [INFO] Finished at: 2017-04-26T12:41:35+05:30
>> [INFO] Final Memory: 74M/903M
>> [INFO] 
>> 
>> [ERROR] Failed to execute goal 
>> org.apache.maven.plugins:maven-javadoc-plugin:2.9:jar
>> (attach-javadocs) on project apex-api: MavenReportException: Error while
>> creating archive:
>> [ERROR] Exit code: 1 - /home/tushar/work/apex/core/ap
>> i/src/main/java/com/datatorrent/api/Attribute.java:116: warning: no
>> description for @param
>> [ERROR] * @param 
>> [ERROR] ^
>> [ERROR] 
>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:117:
>> warning: no description for @param
>> [ERROR] * @param key
>> [ERROR] ^
>> [ERROR] 
>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:146:
>> warning: no description for @throws
>> [ERROR] * @throws java.lang.CloneNotSupportedException
>> [ERROR] ^
>> [ERROR] 
>> /home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/AutoMetric.java:59:
>> error: self-closing element not allowed
>> [ERROR] * It aggregates metrics from multiple physical partitions of an
>> operator to a logical one.
>> [ERROR] ^
>>
>> What is the correct command to build the release.
>>
>> Thanks,
>> - Tushar.
>>
>>
>

[jira] [Resolved] (APEXCORE-594) Plugin support in Apex

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi resolved APEXCORE-594.

   Resolution: Fixed
Fix Version/s: 3.6.0

> Plugin support in Apex
> --
>
> Key: APEXCORE-594
> URL: https://issues.apache.org/jira/browse/APEXCORE-594
> Project: Apache Apex Core
>  Issue Type: New Feature
>    Reporter: Tushar Gosavi
> Fix For: 3.6.0
>
>
> Investigate adding new functionality to apex through user plugins.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (APEXCORE-700) Make the plugin registration interface uniform

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi resolved APEXCORE-700.

Resolution: Fixed

> Make the plugin registration interface uniform
> --
>
> Key: APEXCORE-700
> URL: https://issues.apache.org/jira/browse/APEXCORE-700
> Project: Apache Apex Core
>  Issue Type: Sub-task
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
>Priority: Minor
> Fix For: 3.6.0
>
>
> The user-facing plugin registration for DAG setup plugins is slightly 
> different from runtime plugins. It would be better to have a uniform way to 
> do this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

These errors went away after switching to 1.7 version of java for build.

Thanks,
-Tushar.


On Wed, Apr 26, 2017 at 12:50 PM, Tushar Gosavi <tus...@datatorrent.com>
wrote:

> I am following process specified at http://apex.apache.org/release.html to
> prepare a release candidate for 3.6.
>
> I am getting error while executing command
> mvn clean apache-rat:check deploy -Papache-release -DskipTests
>
>
> 100 warnings
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Apex  SUCCESS
> [06:51 min]
> [INFO] Apache Apex API  FAILURE [
>  5.255 s]
> [INFO] Apache Apex Common Library . SKIPPED
> [INFO] Apache Apex Buffer Server .. SKIPPED
> [INFO] Apache Apex Stream Processing Engine ... SKIPPED
> [INFO] Apache Apex Application Maven Archetype  SKIPPED
> [INFO] Apache Apex App Configuration Maven Archetype .. SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 06:56 min
> [INFO] Finished at: 2017-04-26T12:41:35+05:30
> [INFO] Final Memory: 74M/903M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-javadoc-plugin:2.9:jar
> (attach-javadocs) on project apex-api: MavenReportException: Error while
> creating archive:
> [ERROR] Exit code: 1 - /home/tushar/work/apex/core/api/src/main/java/com/
> datatorrent/api/Attribute.java:116: warning: no description for @param
> [ERROR] * @param 
> [ERROR] ^
> [ERROR] /home/tushar/work/apex/core/api/src/main/java/com/
> datatorrent/api/Attribute.java:117: warning: no description for @param
> [ERROR] * @param key
> [ERROR] ^
> [ERROR] /home/tushar/work/apex/core/api/src/main/java/com/
> datatorrent/api/Attribute.java:146: warning: no description for @throws
> [ERROR] * @throws java.lang.CloneNotSupportedException
> [ERROR] ^
> [ERROR] /home/tushar/work/apex/core/api/src/main/java/com/
> datatorrent/api/AutoMetric.java:59: error: self-closing element not
> allowed
> [ERROR] * It aggregates metrics from multiple physical partitions of an
> operator to a logical one.
> [ERROR] ^
>
> What is the correct command to build the release.
>
> Thanks,
> - Tushar.
>
>

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

I am following process specified at http://apex.apache.org/release.html to
prepare a release candidate for 3.6.

I am getting error while executing command
mvn clean apache-rat:check deploy -Papache-release -DskipTests


100 warnings
[INFO]

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Apex  SUCCESS [06:51
min]
[INFO] Apache Apex API  FAILURE [
 5.255 s]
[INFO] Apache Apex Common Library . SKIPPED
[INFO] Apache Apex Buffer Server .. SKIPPED
[INFO] Apache Apex Stream Processing Engine ... SKIPPED
[INFO] Apache Apex Application Maven Archetype  SKIPPED
[INFO] Apache Apex App Configuration Maven Archetype .. SKIPPED
[INFO]

[INFO] BUILD FAILURE
[INFO]

[INFO] Total time: 06:56 min
[INFO] Finished at: 2017-04-26T12:41:35+05:30
[INFO] Final Memory: 74M/903M
[INFO]

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-javadoc-plugin:2.9:jar (attach-javadocs) on
project apex-api: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 -
/home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:116:
warning: no description for @param
[ERROR] * @param 
[ERROR] ^
[ERROR]
/home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:117:
warning: no description for @param
[ERROR] * @param key
[ERROR] ^
[ERROR]
/home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/Attribute.java:146:
warning: no description for @throws
[ERROR] * @throws java.lang.CloneNotSupportedException
[ERROR] ^
[ERROR]
/home/tushar/work/apex/core/api/src/main/java/com/datatorrent/api/AutoMetric.java:59:
error: self-closing element not allowed
[ERROR] * It aggregates metrics from multiple physical partitions of an
operator to a logical one.
[ERROR] ^

What is the correct command to build the release.

Thanks,
- Tushar.

[jira] [Updated] (APEXCORE-563) Have a pointer to log file name and offset in container/operator failure events

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-563:
---
Summary: Have a pointer to log file name and offset in container/operator 
failure events  (was: Have a pointer to container log file name and offset in 
stram events that deliver a container or operator failure event.)

> Have a pointer to log file name and offset in container/operator failure 
> events
> ---
>
> Key: APEXCORE-563
> URL: https://issues.apache.org/jira/browse/APEXCORE-563
> Project: Apache Apex Core
>  Issue Type: New Feature
>Reporter: Sanjay M Pujare
>Assignee: Priyanka Gugale
> Fix For: 3.6.0
>
>
> The default DailyRollingFileAppender does not take into account of how many 
> backup files to keep and it will result in unbounded growth of log files, 
> especially for long running applications.
> The below is an interesting add-on to the default DailyRollingFileAppender 
> that supports maxBackupIndex.
> http://wiki.apache.org/logging-log4j/DailyRollingFileAppender



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (APEXCORE-636) Ability to refresh tokens using user's own kerberos credentials in a managed environment

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-636:
---
Summary: Ability to refresh tokens using user's own kerberos credentials in 
a managed environment  (was: Ability to refresh tokens using user's own 
kerberos credentials in a managed environment where the application is launched 
using an admin with impersonation)

> Ability to refresh tokens using user's own kerberos credentials in a managed 
> environment
> 
>
> Key: APEXCORE-636
> URL: https://issues.apache.org/jira/browse/APEXCORE-636
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>Assignee: devendra tagare
> Fix For: 3.6.0
>
>
> When applications run in secure mode, they use delegation tokens to access 
> Hadoop resources. These delegation tokens have a lifetime, typically 7 days, 
> after which they no longer work and the application will not be able to 
> communicate with Hadoop. Apex can automatically refresh these tokens before 
> they expire. To do this it requires Kerberos credentials which should be 
> supplied during launch time.
> In a managed environment the user launching the application may not be 
> intended runtime user for the application. Apex today supports impersonation 
> to achieve this. Typically, a management application uses its own 
> credentials, which typically have higher privilege, to launch the application 
> and impersonate as a regular user so that the application runs as the regular 
> user. However, the admin credentials are also packaged with the application 
> to for refreshing the tokens described above. This can cause a security 
> concern because a regular user has access to a higher privilege Kerberos 
> credentials.
> We need a way to specify alternate kerberos credentials to be used for token 
> refresh. Today there is a partially implemented feature for this which allows 
> specification of the refresh keytab using a property but not the principal. 
> We would need to add support for the principal as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Towards Apache Apex 3.6.0 release

[Tushar Gosavi]

Hi All,

I am going to cut first RC in few days. There are two JIRA pending
APEXCORE-700 and APEXCORE-641. APEXCORE-700 is close to merge. @Vlad, Can
we move APEXCORE-641 out of 3.6?

List of 3.6 issues.
https://issues.apache.org/jira/browse/APEXCORE/fixforversion
/12338818/?selectedTab=com.atlassian.jira.jira-projects-plug
in:version-issues-panel

Thanks,
- Tushar


On Fri, Apr 14, 2017 at 9:16 PM, Dean Lockgaard <dean.lockga...@gmail.com>
wrote:

> Vlad,
>
> Here is my thought process about these tickets.  Both 692 (Apex dev setup
> sandbox section to reference Apex website downloads page) and 687 (update
> supported Hadoop v2.6 in Apex docs) are Apex documentation issues, and so
> they are part of the Apex release process.  Furthermore, 692 directly
> references 693 (update Apex website downloads page with cleaned up and
> augmented list of 3rd party binaries), so it makes sense to have 693
> updated as well, though of course I agree that it is not a part of Apex
> core release nor a blocker for the release.
>
> Thanks,
> Dean
>
>
>
> On Fri, Apr 14, 2017 at 11:27 AM, Vlad Rozov <v.ro...@datatorrent.com>
> wrote:
>
> > Dean,
> >
> > 692 and 693 are web site documentation issues and are not part of the
> Apex
> > core 3.6.0 release. 687 can be covered in the release README (known
> issues).
> >
> > Thank you,
> >
> > Vlad
> >
> > On 4/13/17 14:11, Dean Lockgaard wrote:
> >
> >> I'd like to request that 687, 692 and 693 be included in the 3.6.0
> >> release.  I will send PRs for these shortly.
> >>
> >> Thanks,
> >> Dean
> >>
> >>
> >>
> >> On Fri, Apr 14, 2017 at 5:05 AM, Amol Kekre <a...@datatorrent.com>
> wrote:
> >>
> >> +1 to cut a release
> >>>
> >>> Thks
> >>> Amol
> >>>
> >>>
> >>> E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
> >>>
> >>> www.datatorrent.com
> >>>
> >>>
> >>> On Thu, Apr 13, 2017 at 9:22 AM, Pramod Immaneni <
> pra...@datatorrent.com
> >>> >
> >>> wrote:
> >>>
> >>> +1
> >>>>
> >>>> I would like to see 699 and 700 addressed as well.
> >>>>
> >>>> On Wed, Apr 12, 2017 at 10:16 PM, Tushar Gosavi <
> tus...@datatorrent.com
> >>>> >
> >>>> wrote:
> >>>>
> >>>> Hi,
> >>>>>
> >>>>> It has been four month since 3.5.0 Apex Core release. There are
> several
> >>>>>
> >>>> new
> >>>>
> >>>>> features added to core after 3.5.0. I would like to propose the 3.6.0
> >>>>> release of Apex Core, to make these features available to users.
> >>>>>
> >>>>> The list of issues fixed in 3.6.0 are:
> >>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>> 3D%20APEXCORE%20AND%20status%20in%20(Resolved%2C%20Closed)%
> >>>>> 20AND%20fixVersion%20%3D%203.6.0%20ORDER%20BY%20status%20ASC
> >>>>>
> >>>>> Apart from above JIRAs, which bug-fixes/features people will like to
> >>>>>
> >>>> see
> >>>
> >>>> in
> >>>>
> >>>>> this release. If you feel your JIRA should be included then please
> set
> >>>>>
> >>>> fix
> >>>>
> >>>>> version to 3.6.0 with estimated time for work completion, also
> discuss
> >>>>>
> >>>> it
> >>>
> >>>> here. Some of the pending pull requests could be incorporated in the
> >>>>> release. I feel following JIRA should be part of release
> APEXCORE-649,
> >>>>> APEXCORE-678.
> >>>>>
> >>>>> Let me know about your thoughts.
> >>>>>
> >>>>> Thanks,
> >>>>> Tushar.
> >>>>>
> >>>>>
> >
>

[jira] [Updated] (APEXCORE-656) Upgrade org.apache.httpcomponents.httpclient to 4.3.6 version

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-656:
---
Summary: Upgrade org.apache.httpcomponents.httpclient to  4.3.6 version  
(was: Upgrade org.apache.httpcomponents.httpclient)

> Upgrade org.apache.httpcomponents.httpclient to  4.3.6 version
> --
>
> Key: APEXCORE-656
> URL: https://issues.apache.org/jira/browse/APEXCORE-656
> Project: Apache Apex Core
>  Issue Type: Dependency upgrade
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Trivial
> Fix For: 3.6.0
>
>
> Upgrade org.apache.httpcomponents.httpclient dependency to the latest 4.3.x 
> version. Please see 
> [https://dist.apache.org/repos/dist/release/httpcomponents/httpclient/RELEASE_NOTES-4.3.x.txt].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (APEXCORE-634) Unifier attributes are not set for modules in DAG

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-634:
---
Summary: Unifier attributes are not set for modules in DAG  (was: Apex 
Platform unable to set unifier attributes for modules in DAG)

> Unifier attributes are not set for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>Priority: Minor
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (APEXCORE-594) Plugin support in Apex

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-594:
---
Summary: Plugin support in Apex  (was: Investigate adding new functionality 
to apex through user plugins.)

> Plugin support in Apex
> --
>
> Key: APEXCORE-594
> URL: https://issues.apache.org/jira/browse/APEXCORE-594
> Project: Apache Apex Core
>  Issue Type: New Feature
>    Reporter: Tushar Gosavi
>
> Investigate adding new functionality to apex through user plugins.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (APEXCORE-710) Plugin manager does not report exceptions from plugins

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-710:
---
Issue Type: Bug  (was: Sub-task)
Parent: (was: APEXCORE-594)

> Plugin manager does not report exceptions from plugins
> --
>
> Key: APEXCORE-710
> URL: https://issues.apache.org/jira/browse/APEXCORE-710
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>
> If there are exceptions during execution of a plugin they should be reported. 
> The plugin should possibly be taken down as it might be in a bad state. An 
> event may need to be raised in stram to notify users.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (APEXCORE-577) Plugin support to inspect DAG before launch.

[Tushar Gosavi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/APEXCORE-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tushar Gosavi updated APEXCORE-577:
---
Summary: Plugin support to inspect DAG before launch.  (was: Visitor API 
for DAG)

> Plugin support to inspect DAG before launch.
> 
>
> Key: APEXCORE-577
> URL: https://issues.apache.org/jira/browse/APEXCORE-577
> Project: Apache Apex Core
>  Issue Type: Sub-task
>    Reporter: Tushar Gosavi
>    Assignee: Tushar Gosavi
> Fix For: 3.6.0
>
>
> Support an Visitor like api for DAG, which could be used to write plugins for 
> Apex.
> change DAG interface to add a visitor
> {code}
> registerVisitor(DagVisitor visitor)
> {code}
> The DagVisitor interface
> {code}
> interface DagVisitor {
>   boolean startDAG(DAG* dag);
>   boolean visitOperator(OperatorMeta *ometa)
>   boolean visitStream(StreamMeta *smeta);
>   boolean endDAG()
> }
> {code}
> dev thread discussions
> https://lists.apache.org/thread.html/1c37e0954cee029c9de537cc35ecb35beebdf49aba17de89bd2ce9ed@%3Cdev.apex.apache.org%3E



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)