Re: A proposal for Malhar

2016-05-27 Thread Chinmay Kolhatkar
+1 for the idea of incubating space in malhar. This will not just ensure more contributions but a faster development of operators for putting into firsthand use. Though, I believe we need to build a process around that, right from the point where operator developer starts to develop till the

Re: APEXCORE-408 : Ability to schedule Sub-DAG from running application

2016-06-22 Thread Chinmay Kolhatkar
I think its a good idea to have a scheduling operator when you need to start a part of the DAG when some trigger happens (for eg. FileSplitter identifying new files in FS) and otherwise bring it down to save resources. On Wed, Jun 22, 2016 at 9:54 AM, Timothy Farkas <

Re: Update to apex.apache.org site?

2016-06-23 Thread Chinmay Kolhatkar
+1 for redesign. I think we can also put some though on redesigning of the docs page ( http://apex.apache.org/docs.html) And maybe also have link to latest downloadable on front page? On Thu, Jun 23, 2016 at 10:34 AM, York, Brennon wrote: > +1 to the redesign.

Re: [DISCUSSION] Custom Control Tuples

2016-06-27 Thread Chinmay Kolhatkar
I hope I'm not commenting too late on this thread. >From the above discussion, it looks like the requirement is to have a custom tuple which has following 2 capabilities to influence streaming engine on: 1. When to send it (between windows/within window) 2. Where to send it (all partitions, some

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

2016-05-23 Thread Chinmay Kolhatkar
+1. - File Integrity: PASS - No Binary file in the package: PASS - Compilation: PASS - Unit tests: PASS On Tue, May 24, 2016 at 1:20 AM, David Yan wrote: > +1 (binding) > > - Build passed > - All unit tests passed > - Successfully ran pi demo with Apex Core 3.4.0. > >

Re: Bleeding edge branch ?

2016-07-12 Thread Chinmay Kolhatkar
I'm -0 on this idea. Here is the reason: Unless we see a real case where users want to see everything on latest, this branch might quickly become low hanging fruit and eventually get obsolete because its anyway a "no gaurantee" branch. We have a bunch of dependencies which we'll have to take

Re: Using DSL api to construct sql queries

2016-07-22 Thread Chinmay Kolhatkar
Yes there is no clash with calcite integration. Calcite is a query planner converts SQL to a relational algebra... This is different. On Fri, Jul 22, 2016 at 11:03 AM, Priyanka Gugale wrote: > I don't know much about Calcite, but reading abstract, calcite seems to be > for

Re: Malhar 3.5.0 release

2016-08-01 Thread Chinmay Kolhatkar
APEXMALHAR-2128 is already done. Somehow the status did not go to resolved. Changed it now. On Fri, Jul 29, 2016 at 1:53 PM, Yogi Devendra wrote: > APEXMALHAR-2116 > https://github.com/apache/apex-malhar/pull/326 > > @Ram > Your review comments are incorporated.

Re: custom JAVA_HOME

2016-08-03 Thread Chinmay Kolhatkar
+1 for the idea. Are there any in the list that one can see as conflicting with our own environment variables? For e.g. LOGNAME? On Wed, Aug 3, 2016 at 4:49 AM, Pramod Immaneni wrote: > How about allowing specification of all environment variables supported by > YARN

Re: [Proposal] Named Checkpoints

2016-08-04 Thread Chinmay Kolhatkar
Nice feature. +1 for it. One question, instead of creating a new tuple type, can this be done by modifying current checkpoint tuple? On Thu, Aug 4, 2016 at 2:02 PM, Yogi Devendra wrote: > This will be awesome feature. I can see the usecases for production > scenario

APEXMALHAR-1818 : Apex-Calcite Integration

2016-08-12 Thread Chinmay Kolhatkar
Hi All, I wanted to give update on Apex-Calcite Integration work being done for visibility and feedback from the community. In the first phase, target is to use Calcite core library for SQL parsing and transformation of relation algebra to apex specific component (operators). Once this is

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

2016-08-11 Thread Chinmay Kolhatkar
Congrats Dev!! On Thu, Aug 11, 2016 at 10:50 AM, Shubham Pathak wrote: > Congratulations Dev ! > > Thanks, > Shubham > > On Thu, Aug 11, 2016 at 10:43 AM, Bhupesh Chawda > wrote: > > > Congrats Dev!! > > > > ~ Bhupesh > > > > On Thu, Aug 11,

Re: A proposal for Malhar

2016-07-12 Thread Chinmay Kolhatkar
+1. This is a really good starting point to cleanup malhar. On Wed, Jul 13, 2016 at 3:06 AM, David Yan wrote: > Hi Lakshmi, > > Thanks for volunteering. > > I think Pramod's suggestion of putting the operators into 3 buckets and > Siyuan's suggestion of starting a shared

Re: auto-generated emails

2016-07-12 Thread Chinmay Kolhatkar
Strongly +1 for this. For 1, can the mail be sent to someone who is mentioned in the PR? So for e.g., if I mention @PramodSSImmaneni , then Pramod will be part of PR email notification all further back and forth communication for that PR. @Pramod, just using your name as an example.. :) -

Re: Container & memory resource allocation

2016-07-19 Thread Chinmay Kolhatkar
Hi Pradeep, This is a great content to add to the documents. These are the common set of errors which might get googled and hence great to get indexed as well. You can take a look at: https://github.com/apache/apex-core/tree/master/docs The docs for apex reside there in markdown format.

Re: auto-generated emails

2016-07-18 Thread Chinmay Kolhatkar
mmits@apex which would > > have full archive of comments? > > > > New JIRA, PR can be sent to both dev, commits. > > > > ~ Yogi > > > > On 13 July 2016 at 10:22, Chinmay Kolhatkar <chin...@datatorrent.com> > > wrote: > > > > &

Re: Windowed Operator PR

2016-07-10 Thread Chinmay Kolhatkar
+1 on merging. On Mon, Jul 11, 2016 at 11:00 AM, Amol Kekre wrote: > +1 on merging it. This is a big feature addition. > > Thks > Amol > > > On Fri, Jul 8, 2016 at 4:17 PM, Siyuan Hua wrote: > > > +1 > > > > We should merge this ASAP. > > I don't

builds.apache.org

2016-08-06 Thread Chinmay Kolhatkar
Hi All, I just came across https://builds.apache.org/ and am curious to know what it is used for. There are quite some projects listed there. Anybody has any idea what is this used for? Can we leverage this service provided by ASF Build Services, if we see any use? Here is the wiki for the

Re: Upgrade Apache Bigtop to Apex Core 3.5.0

2017-01-24 Thread Chinmay Kolhatkar
; > > Thks > > Amol > > > > > > On Thu, Jan 19, 2017 at 8:55 PM, Priyanka Gugale < > priya...@datatorrent.com > > > > > wrote: > > > > > +1 > > > > > > On Fri, Jan 20, 2017 at 9:25 AM, Chinmay Kolhatkar < > >

Re: Schema Discovery Support in Apex Applications

2017-01-25 Thread Chinmay Kolhatkar
ache.org> wrote: > +1 to have this feature. > > -Priyanka > > On Tue, Jan 17, 2017 at 9:18 PM, Pramod Immaneni <pra...@datatorrent.com> > wrote: > > > +1 > > > > On Mon, Jan 16, 2017 at 1:23 AM, Chinmay Kolhatkar <chin...@apache.org> > > wro

Re: One Yarn with Multiple Apex Applications

2017-01-30 Thread Chinmay Kolhatkar
Hi Santhosh, It seems that your YARN does not have enough resources available for allocating memory for 2 application. When you kill the first application, the memory is regained by yarn and then allocated to the second application. You can try to give more memory to yarn if your system allows

Re: Schema Discovery Support in Apex Applications

2017-01-30 Thread Chinmay Kolhatkar
er properties of the > data that can be useful by the schema receiver to process the data (for > instance, a delimiter for CSV record stream). > > Thanks, > Sergey > > On 2017-01-25 01:47 (-0800), Chinmay Kolhatkar <chin...@datatorrent.com> > wrote: > > Thank you all f

Re: APEXMALHAR-2400 In PojoInnerJoin accumulation same field names are emitted as single field

2017-02-15 Thread Chinmay Kolhatkar
I agree with the requirement. But I think the requirement here is superceed by requirement at https://lists.apache.org/thread.html/d1e282047399558a4f04aab186c0d986114d353dcaf6d6c76a2a3a35@%3Cdev.apex.apache.org%3E Having PojoInnerJoin to emit a POJO instead of map overrides this. I suggest to do

Re: [Community Opinion] Use of Lombok Library

2017-02-22 Thread Chinmay Kolhatkar
Certainly good and convenient for application/operator developers to use. But could someone include this dependency in apex application project and use it? If yes, then I would prefer to keep this outside of Apex in general and let the user use it for convenience. -Chinmay. On Wed, Feb 22,

PojoInnerJoin Accumulation emitting Map

2017-02-15 Thread Chinmay Kolhatkar
Dear Community, Currently PojoInnerJoin accumulation is accepting 2 POJOs but emitting a Map. I think it should be emitting POJO instead of Map. Please share your thoughts about this. Thanks, Chinmay.

Re: PojoInnerJoin Accumulation emitting Map

2017-02-16 Thread Chinmay Kolhatkar
APEXMALHAR-2409 Created. I'll create a PR soon. On Wed, Feb 15, 2017 at 10:31 PM, Chinmay Kolhatkar <chin...@datatorrent.com > wrote: > Thanks guys. I'll create a jira and create a PR for it soon. > > Ajay, btw this is not much related to Schema discovery design... This is

Re: PojoInnerJoin Accumulation emitting Map

2017-02-15 Thread Chinmay Kolhatkar
t 7:34 AM, AJAY GUPTA <ajaygit...@gmail.com> wrote: > > > Yes, it should be emitting a POJO. This POJO can then be further used to > > join with a third POJO stream, thus behaving like a DB join. > > > > It would be best if we can incorporate this into Schema Disc

Re: Shutdown of an Apex app

2017-01-17 Thread Chinmay Kolhatkar
+1 for the feature in general. Though I have a question about it: Lets say 10th operator in the DAG initiates the shutdown.. And operator being in quite downstream as compared to Input Operator, it might be fairly behind in window it is processing. What happens to the data being already processed

Upgrade Apache Bigtop to Apex Core 3.5.0

2017-01-19 Thread Chinmay Kolhatkar
Dear Community, Now that Apex core 3.5.0 is released, is it good time to upgrade Apache Bigtop for Apex to 3.5.0? -Chinmay.

Re: Upgrade Apache Bigtop to Apex Core 3.5.0

2017-01-19 Thread Chinmay Kolhatkar
_ > Sent with difficulty, I mean handheld ;) > > On 19 Jan 2017 2:28 pm, "Tushar Gosavi" <tus...@datatorrent.com> wrote: > > > +1 for including Apex 3.5 in Bigtop. > > > > - Tushar. > > > > > > On Thu, Jan 19, 2017 at 2:23 PM, Chi

Schema Discovery Support in Apex Applications

2017-01-16 Thread Chinmay Kolhatkar
Hi All, Currently a DAG that is generated by user, if contains any POJOfied operators, TUPLE_CLASS attribute needs to be set on each and every port which receives or sends a POJO. For e.g., if a DAG is like File -> Parser -> Transform -> Dedup -> Formatter -> Kafka, then TUPLE_CLASS attribute

Re: Schema Discovery Support in Apex Applications

2017-01-16 Thread Chinmay Kolhatkar
er/operator developer would access the tuple object in case he wishes to > do something with it? > > > Ajay > > On Mon, Jan 16, 2017 at 2:53 PM, Chinmay Kolhatkar <chin...@apache.org> > wrote: > > > Hi All, > > > > Currently a DAG that is generated b

Re: Contribution Process before PR

2017-01-16 Thread Chinmay Kolhatkar
I agree that the discussions are important. There are currently discussions being happening over mailing list but some topics might be get missed and we should improve there. The tasks should be brought to community before the design starts and then mature the design through community discussion.

Re: Upgrade Apache Bigtop to Apex Core 3.5.0

2017-01-19 Thread Chinmay Kolhatkar
org/ ? > > On 1/19/17, 8:12 AM, "Pramod Immaneni" <pra...@datatorrent.com> wrote: > > +1 > > On Thu, Jan 19, 2017 at 12:53 AM, Chinmay Kolhatkar < > chin...@datatorrent.com > > wrote: > > > Dear Community, > > >

Re: Proposal: CompositeAccumulation for Windowed Operator

2017-02-27 Thread Chinmay Kolhatkar
t; > > > * accumulations.getSubOutput(maxTag, outputValues)* > > > > *accumulations.getSubOutput(minTag, outputValues)* > > > > > > Thanks > > > > Bright > > > > On Sun, Feb 26, 2017 at 10:33 PM, Chinmay Kolhatkar < >

Re: Java packages: legacy -> org.apache.apex

2017-02-27 Thread Chinmay Kolhatkar
Thomas, I agree with you that we need this migration to be done but I have a different opinion on how to execute this. I think if we do this in phases as described above, users might end up in more confusion. For doing this migration, I think it should follow these steps: 1. Whether for operator

Re: Fixed Width Record Parser

2016-09-08 Thread Chinmay Kolhatkar
Few points/questions: 1. Agree with Yogi. Approach 2 does not look clean. 2. Do we need "recordwidthlength"? 3. "recordseperator" should be "\n" and not "/n". 4. In general, providing schema as a JSON is tedious from user perspective. I suggest we find a simpler format for specifying schema. For

Re: [ANNOUNCE] New Apache Apex PMC Member: Chandni Singh

2016-09-12 Thread Chinmay Kolhatkar
Congrats Chandni!! On Tue, Sep 13, 2016 at 10:40 AM, Priyanka Gugale wrote: > Congrats Chandni. > > -Priyanka > > On Tue, Sep 13, 2016 at 10:39 AM, Yogi Devendra < > devendra.vyavah...@gmail.com> wrote: > > > Congratulations !!! > > > > ~ Yogi > > > > On 13 September 2016 at

Re: Jekyll for Apex website?

2016-09-15 Thread Chinmay Kolhatkar
If we are thinking abut revamping apex website, this seems to be pretty good option. The jelyllrb.com itself is pretty nice website and the code for that resides here: https://github.com/jekyll/jekyll/tree/master/site I personally support the idea of revamping the website with jekyll. Not just

Re: Python support

2016-09-16 Thread Chinmay Kolhatkar
Strongly +1 on this. One thing that proves this is useful for Apex is hadoop streaming where python is used write map-reduce jobs. This not only will increase the reach in development world but also would be appealing to administrators to write an app as they are usually aware of python. Few

Watermark generation in Windowed Operators

2016-09-16 Thread Chinmay Kolhatkar
Hi All, I was looking at Windowed Operator APIs and have to mention they're pretty nicely done. I have a question related to watermark generation. What I understood is that for completion of processing of an event window one has provision for sending of watermark tuple from some previous stage

Re: checkpoint statistics

2016-09-25 Thread Chinmay Kolhatkar
+1. very useful feature. We should also provide doc on how to use that information for tuning. On Sun, Sep 25, 2016 at 11:27 PM, Thomas Weise wrote: > +1 very useful during tuning and ongoing monitoring for cost of > checkpointing (both, serialization and io). Can also

Travis CI build failing

2016-09-28 Thread Chinmay Kolhatkar
Hi All, I'm getting a weird error in travis CI build related to connection to repository. https://s3.amazonaws.com/archive.travis-ci.org/jobs/163438909/log.txt Error is as following: Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project

Re: APEXMALHAR-1818 : Apex-Calcite Integration

2016-10-05 Thread Chinmay Kolhatkar
time to remove "Review Only" for this PR and propose this PR for including in apex malhar. Please share your opinion. Thanks, Chinmay. On Wed, Sep 28, 2016 at 9:03 PM, Chinmay Kolhatkar <chin...@datatorrent.com> wrote: > Dear Community, > > I've created a review only PR f

Re: APEXMALHAR-1818 : Apex-Calcite Integration

2016-09-19 Thread Chinmay Kolhatkar
ood to create a Review Only PR for first cut of calcite integration. Please share your opinion on above. Thanks, Chinmay. On Fri, Aug 12, 2016 at 9:55 PM, Chinmay Kolhatkar <chin...@datatorrent.com> wrote: > Hi All, > > I wanted to give update on Apex-Calcite Integration work be

Re: Python support

2016-09-22 Thread Chinmay Kolhatkar
I would like to help in contributing to this feature. On Wed, Sep 21, 2016 at 12:26 AM, Sasha Parfenov wrote: > +1 on both executing Python code in an operator and high level API for > constructing Pipelines in Python. > > There is a large user base of engineers and data

PrintWriter for LogicalPlan

2016-09-22 Thread Chinmay Kolhatkar
Hi All, For testing of calcite generated DAG, I want to verify whether the correct DAG is created with right operators and streams and whether properties are set properly. Is there any PrintWriter for the LogicalPlan object printing components of DAG? If not, I am planning to write one for

Re: PrintWriter for LogicalPlan

2016-09-22 Thread Chinmay Kolhatkar
On Thu, Sep 22, 2016 at 12:22 PM, Tushar Gosavi <tus...@datatorrent.com> wrote: > Hi Chinmay, > > take a look at following gist for example. > https://gist.github.com/anonymous/8df8e05ffc16e620bbc030e1034da3c4 > > - Tushar. > > > On Thu, Sep 22, 2016 at 12:05 PM, Chinm

Re: PrintWriter for LogicalPlan

2016-09-22 Thread Chinmay Kolhatkar
inted easily. > But for verification check if right components are added in DAG by > examining dag structure than printing it. > > - Tushar. > > > On Thu, Sep 22, 2016 at 11:39 AM, Chinmay Kolhatkar > <chin...@datatorrent.com> wrote: > > Hi All, > > > > Fo

Re: Watermark generation in Windowed Operators

2016-09-17 Thread Chinmay Kolhatkar
However, I think it's possible to implement some kind of watermark > > > > > generation in the WindowedOperator itself if that knowledge is not > > > > > available from the input. It's actually already doing that if you > > call > > > >

Re: Malhar 3.5.0 release

2016-08-27 Thread Chinmay Kolhatkar
>> > > > > 20%3D%203.5.0%20AND%20project%20%3D%20APEXMALHAR%20AND% > >> >> > > > 20resolution%20%3D% > >> >> > > > > 20Unresolved%20ORDER%20BY%20priority%20DESC > >> >> > > > > > >> >> > > > > Pl

Re: APEXMALHAR-1818 : Apex-Calcite Integration

2016-09-28 Thread Chinmay Kolhatkar
as well as cluster mode. ​I request everyone to please review the PR and provide feedback​. ​Thanks, Chinmay.​ On Tue, Sep 20, 2016 at 11:16 AM, Chinmay Kolhatkar <chin...@datatorrent.com > wrote: > Hi All, > > I wanted to give a quick update on Apex-Calcite integration work. > &g

Re: Kudu store operators

2016-10-02 Thread Chinmay Kolhatkar
+1. - Chinmay. On 3 Oct 2016 7:25 a.m., "Amol Kekre" wrote: > Ananth, > This would be great to have. +1 > > Thks > Amol > > On Sun, Oct 2, 2016 at 8:38 AM, Munagala Ramanath > wrote: > > > +1 > > > > Kudu looks impressive from the overview, though

Re: Fixed Width Record Parser

2016-10-03 Thread Chinmay Kolhatkar
ears more dirty to me. > > Regards, > Hitesh > > > > On Thu, Sep 8, 2016 at 1:37 PM, Yogi Devendra < > devendra.vyavah...@gmail.com> > wrote: > > > If we specify order of the fields and length for each field then start, > end > > can be computed. > >

Re: Apex Core PR

2016-10-01 Thread Chinmay Kolhatkar
Hi Vlad, In one of the PR the travis build is passing but Jenkins build is failing. https://github.com/apache/apex-malhar/pull/409 Is it observed somewhere else too? Any idea why that is the case? Thanks, Chinmay. On Sat, Oct 1, 2016 at 2:13 AM, Vlad Rozov wrote: >

Re: [VOTE] Hadoop upgrade

2016-10-04 Thread Chinmay Kolhatkar
+1 for 2.6 On Tue, Oct 4, 2016 at 6:19 AM, Bhupesh Chawda wrote: > +1 for 2.6 > > ~ Bhupesh > > On Oct 4, 2016 4:14 AM, "Siyuan Hua" wrote: > > > +1 for 2.6.x > > > > On Mon, Oct 3, 2016 at 3:41 PM, Pramod Immaneni > >

Re: Malhar release 3.6

2016-10-26 Thread Chinmay Kolhatkar
+1. On Thu, Oct 27, 2016 at 1:41 AM, Thomas Weise wrote: > Hi, > > I'm proposing another release of Malhar in November. There are 49 issues > marked for the release, including important bug fixes, new documentation, > SQL support and the work for windowed operator state

Suggestion to Compositional API of Apex

2016-10-26 Thread Chinmay Kolhatkar
Hi All, With a lot of Apache projects making use of inner class in Java 7 and Decarative programming feature of Java 8 to define stages in the pipeline, I would like to propose a small additional to our existing compositional APIs allowing it to define processing stages inline. I've created a

Re: [KUDOS] Contributed runner: Apache Apex!

2016-10-18 Thread Chinmay Kolhatkar
Very nice ..:) I would like to contribute as well. Thanks, Chinmay. On Tue, Oct 18, 2016 at 2:19 AM, David Yan wrote: > I would also like to contribute. Thanks! > > David > > On Mon, Oct 17, 2016 at 10:01 AM, Thomas Weise wrote: > > > FYI, the first cut

Re: Make cli support high-level API and SQL

2016-10-20 Thread Chinmay Kolhatkar
Are you talking about apex cli (the one present in apex-core) to support both? As both high-level API and SQL are in malhar, wouldn't there be circular dependency issue? Beside the possible issues, this is a great idea. Though, I'll suggest to have a seperate cli for high level and SQL in

[New Feature] Apex SQL Support - Calcite Integration

2016-10-23 Thread Chinmay Kolhatkar
Dear Community, Thank you for all the contribution in making the first PR to get merged for adding SQL Support in Apex. The support for SQL in Apex is present here: https://github.com/apache/apex-malhar/tree/master/sql As a part of PR review and other discussion a number of followup items that

Re: Proposal for apex/malhar extensions

2016-11-24 Thread Chinmay Kolhatkar
pra...@datatorrent.com> wrote: > Yes, I think it could be useful for core as well. > > On Wed, Nov 16, 2016 at 11:19 AM, Chinmay Kolhatkar <chin...@apache.org> > wrote: > > > @sanjay, yes we can define the process around this. > > > > @pramod, Well, I said apex-m

Re: JSON License and Apache Projects

2016-11-24 Thread Chinmay Kolhatkar
Yes... That's the mail.. There are couple if related conversations can be seen here too: https://lists.apache.org/list.html?legal-disc...@apache.org I suggest we take a look at it and do the needful from our end too. -Chinmay. On Fri, Nov 25, 2016 at 10:15 AM, Amol Kekre

APEXMALHAR-2354 Heuristic Watermark in windowed operator

2016-11-24 Thread Chinmay Kolhatkar
Dear Community, I'm working on adding support for heuristic watermark in Windowed Operator. Heuristic watermark give users of WindowedOperator a way to logically determine whether watermark condition is met or not by inspecting the tuples received. This can act as a replacement for or way to work

Re: APEXMALHAR-2354 Heuristic Watermark in windowed operator

2016-11-28 Thread Chinmay Kolhatkar
actually somewhat related to the separate debate on this list about > control tuples being delivered only at streaming window boundary. > > David > > On Thu, Nov 24, 2016 at 2:52 AM, Chinmay Kolhatkar <chin...@apache.org> > wrote: > > > Dear Community, >

Re: Apex internal documentation.

2016-11-28 Thread Chinmay Kolhatkar
+1... Good to have. On Tue, Nov 29, 2016 at 11:42 AM, Devendra Tagare wrote: > +1 > > ~Dev > > On Mon, Nov 28, 2016 at 10:10 PM, Priyanka Gugale > wrote: > > > +1 for documenting this. > > > > I had covered the basic cycle of launch in couple of

Re: [VOTE] Apache Apex Malhar Release 3.6.0 (RC1)

2016-11-28 Thread Chinmay Kolhatkar
+1. Verified md5, asc, sha. Malhar is building correctly. Successfully launched pi demo. On Sun, Nov 27, 2016 at 8:02 AM, Thomas Weise wrote: > Dear Community, > > Please vote on the following Apache Apex Malhar 3.6.0 release candidate. > > This is a source release with

Re: APEXMALHAR-2354 Heuristic Watermark in windowed operator

2016-11-28 Thread Chinmay Kolhatkar
t;tus...@datatorrent.com> > wrote: > > > +1, but can you change name of HeuristicWatermark to > > WatermarkGenerator as the purpose of > > this interface is to generate watermarks. > > > > - Tushar. > > > > > > On Thu, Nov 24, 2016 a

Re: Proposal for apex/malhar extensions

2016-11-16 Thread Chinmay Kolhatkar
pra...@datatorrent.com> wrote: > So it would be like a yellow pages for the apex ecosystem. Sounds like a > good idea. Why limit it to malhar? > > On Wed, Nov 16, 2016 at 3:17 AM, Chinmay Kolhatkar <chin...@apache.org> > wrote: > > > Dear Community, > > > > This is in

Proposal for apex/malhar extensions

2016-11-16 Thread Chinmay Kolhatkar
Dear Community, This is in relation to malhar cleanup work that is ongoing. In one of the talks during Apache BigData Europe, I got to know about Spark-Packages (https://spark-packages.org/) (I believe lot of you must be aware of it). Spark package is basically functionality over and above and

Re: ApacheCon Europe Call For Papers Open

2016-10-21 Thread Chinmay Kolhatkar
Hope I'm not responding to this very late. We have a PR open for initial cur of Apex-Calcite integration in Malhar: https://github.com/apache/apex-malhar/pull/432 Maybe that can give you some content to include in your topic. As for upcoming work, there are APEXMALHAR jiras created with "sql"

Re: ApacheCon Europe Call For Papers Open

2016-10-21 Thread Chinmay Kolhatkar
Any feedback is welcome.. :) On Fri, Oct 21, 2016 at 1:48 PM, Atri Sharma <atri.j...@gmail.com> wrote: > Aha,thanks. that makes sense. > > I saw Julian review the PR. Do you need me to still look at it? > > On Fri, Oct 21, 2016 at 1:47 PM, Chinmay Kolhatkar <chi

EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Chinmay Kolhatkar
Dear Community, While testing kafka related endpoint code for Apex-Calcite integration, I found an interesting embedded kafka implementation over the internet and I used it in my code. The implementation is based on ready to use EmbeddedZookeer and KafkaServer from mvn dependency

Re: (APEXMALHAR-2290) JDBC operator does not deploy after failure in certain cases

2016-10-13 Thread Chinmay Kolhatkar
sh > > On Thu, Oct 13, 2016 at 11:20 AM, Pradeep A. Dalvi <p...@apache.org> > wrote: > > > +1 for Option 1 to use conn.getMetaData() > > > > --prad > > > > On Wed, Oct 12, 2016 at 11:39 PM, Chinmay Kolhatkar <chin...@apache.org> > > w

Re: EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Chinmay Kolhatkar
> ActiveMQ and SQS (ElasticMQ) unit tests. My only question/concern is how > > well can you simulate different versions of Kafka. > > > > On 10/13/16, 7:25 AM, "Chinmay Kolhatkar" <chin...@apache.org> wrote: > > > > Dear Community, >

Re: (APEXMALHAR-2290) JDBC operator does not deploy after failure in certain cases

2016-10-12 Thread Chinmay Kolhatkar
Hi Hitesh, Instead of limiting the row count please use one of the following 2 approaches: 1. ResultSet rsColumns = null; DatabaseMetaData meta = conn.getMetaData(); rsColumns = meta.getColumns(null, null, "tablename", null); while (rsColumns.next()) {

Re: APEXMALHAR-2382 User needs to create dt_meta table while using JdbcPOJOInsertOutputOperator

2017-01-15 Thread Chinmay Kolhatkar
-1 for automatic schema creation... Moreover, I am wondering whether asking user to create a dt_meta table is right way. From an admins perspective, an ask for creation of meta table looks wrong to me. dt_meta table is created for the purpose of exactly once but it does not hold any user data..

Malhar build failing

2017-03-07 Thread Chinmay Kolhatkar
Hi, I'm finding malhar build to fail because of rat check failing in demos/ folder. Moreover, I cannot see demos/ folder in https://github.com/apache/apex-malhar I want to confirm whether its in apache repository itself OR its my environment issue. Please reply. Thanks, Chinmay.

Re: Package tarball as deliverable of apex-core

2017-08-07 Thread Chinmay Kolhatkar
; > -Priyanka > > On Tue, Aug 8, 2017 at 8:34 AM, Chinmay Kolhatkar <chin...@datatorrent.com > > > wrote: > > > Dear Community, > > > > Currently Apache Apex does not have a deliverable tarball which bundles > the > > dependent jars and apex cli which

Package tarball as deliverable of apex-core

2017-08-07 Thread Chinmay Kolhatkar
Dear Community, Currently Apache Apex does not have a deliverable tarball which bundles the dependent jars and apex cli which user can just download and get started with. Having this would be also be useful for current integration with Apache Bigtop as well. Currently in bigtop, the bigtop build

Re: Input needed for ApexCli.

2017-08-07 Thread Chinmay Kolhatkar
+1 for the best effort with warnings. On Sun, Aug 6, 2017 at 9:19 PM, Pradeep Kumbhar wrote: > +1 for the best effort with warnings as default behavior and an optional > flag for Validate & Fail (Ajay's suggestion) > > > On Sun, Aug 6, 2017 at 12:14 PM, Mohit Jotwani

Re: Proposal to upgrade Apex Core dependency in Malhar to 3.6

2017-05-12 Thread Chinmay Kolhatkar
+1 On Fri, May 12, 2017 at 1:41 PM, Mohit Jotwani wrote: > +1 > > Regards, > Mohit > > On Fri, May 12, 2017 at 10:49 AM, Bhupesh Chawda > wrote: > > > +1 > > > > ~ Bhupesh > > > > > > ___ > > >

Re: impersonation and application path

2017-05-18 Thread Chinmay Kolhatkar
+1 for pramod's proposal. On 19-May-2017 4:51 AM, "Sanjay Pujare" wrote: > +1 for Pramod's proposal for impersonation. > > I have an issue with Sandesh's suggestion about making the new behavior as > the default (or only) behavior. This will introduce incompatibility

Re: Maintenance of Bigtop Apex component

2017-06-04 Thread Chinmay Kolhatkar
Hi Thomas, Even I'm currently occupied and won't be able to work on this as of now. Also, you're already collaborator on docker-pool repository. -Chinmay. On Sat, Jun 3, 2017 at 3:08 PM, Aniruddha Thombare < anirud...@datatorrent.com> wrote: > Hi, > > I'm currently preoccupied, won't be able

Re: [ANNOUNCE] New Apache Apex Committer: Ananth Gundabattula

2017-11-03 Thread Chinmay Kolhatkar
Congrats Ananth!! On Fri, Nov 3, 2017 at 2:37 PM, Priyanka Gugale wrote: > Congrats Ananth!! > > -Priyanka > > On Fri, Nov 3, 2017 at 2:20 PM, Thomas Weise wrote: > > > The Project Management Committee (PMC) for Apache Apex is pleased to > > announce Ananth

Re: [ANNOUNCE] New Apache Apex PMC: Tushar Gosavi

2017-11-03 Thread Chinmay Kolhatkar
Congrats Tushar!! On Fri, Nov 3, 2017 at 10:21 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Congratulations Tushar!!! > > On Fri, Nov 3, 2017 at 10:06 PM, Venkatesh Kottapalli < > venkat...@datatorrent.com> wrote: > > > Congratulations Tushar! > > > > -Venky > > > > On Fri, Nov

Re: Malhar release 3.8.0

2017-10-25 Thread Chinmay Kolhatkar
+1. - Chinmay. On 25 Oct 2017 9:20 pm, "Chaitanya Chebolu" wrote: > +1 on new release. > > Thanks, > > On Wed, Oct 25, 2017 at 9:09 PM, Vlad Rozov wrote: > > > +1. > > > > Thank you, > > > > Vlad > > > > > > On 10/25/17 08:21, Amol Kekre wrote: >

Re: [RESULT] [VOTE] Apache Apex Core Release 3.7.0 (RC1)

2018-05-09 Thread Chinmay Kolhatkar
Yes Thomas. I'm going to create a docker image. I was hoping that I can create the docker for 3.7.0 from new changes in master which pulls dist atrato instead of creating from my repo. This will act as a pilot for building docker from apex-core repo as well. Let me know if this is fine. Regards,

Re: [Feature Proposal] Add metrics dropwizards like gauges, meters, histogram etc to Apex Platform

2018-05-09 Thread Chinmay Kolhatkar
+1 for approach 1. As Thomas mentioned, I think there metrics layer should be abstracted out from its implementation. This way one can plugin different metrics systems in apex. Also keep in mind that there is a lot of code which uses AutoMetrics annotations. We should help a smooth transition

Re: [RESULT] [VOTE] Apache Apex Core Release 3.7.0 (RC1)

2018-05-13 Thread Chinmay Kolhatkar
ince we don't have the docker files in apex-core/release-3.7, my > suggestion to create the branch in your repo, as done till now. > > You can still pilot the build for the latest image from apex-core/master > > Thanks > > > On Wed, May 9, 2018 at 7:38 AM, Chinmay Kolhatkar < >

Re: Proposal for UI component in stram

2018-06-16 Thread Chinmay Kolhatkar
branch shortly so that we can start working against it. -Chinmay. On Sat, Jun 16, 2018 at 7:07 PM Chinmay Kolhatkar wrote: > So here is how it'll be: > 1. ReactJS application can be compiled for production builds. The output > is set of html+css+compressed js files. It can also out

Re: Proposal for UI component in stram

2018-06-16 Thread Chinmay Kolhatkar
ay > > information in a browser, where they will be hosted is irrelevant. They > > can be hosted by stram, but it should be possible to host them > > externally without making modificaitons. If this is the case, I don't > > have any objections to the proposal. > > >

Re: Proposal for UI component in stram

2018-06-13 Thread Chinmay Kolhatkar
ozov > wrote: > > > > > > > >> Clarification: I am -0 on making UI part of the stram. I am +1 on > > > >> providing reference webUI application outside of the stram. > > > >> > > > >> Thank you, > > > >> > > >

Re: Proposal for UI component in stram

2018-06-16 Thread Chinmay Kolhatkar
Chinmay Kolhatkar wrote: > Hi All, > > I've created following jiras encompassing the requirements (To be picked > up in order): > https://issues.apache.org/jira/browse/APEXMALHAR-2558 > https://issues.apache.org/jira/browse/APEXMALHAR-2559 > > I'll start adding subtask

Re: [ANNOUNCE] New Apache Apex PMC Member: Chinmay Kolhatkar

2018-06-05 Thread Chinmay Kolhatkar
gt; >> > > > > > >> >> On Thu, May 24, 2018 at 10:34 AM, Hitesh Kapoor < > > > > > forhitesh...@gmail.com > > > > > >> > > > > > > >> >> wrote: > > > > > >> >> > > > > > >> >>> Congra

Proposal for UI component in stram

2018-06-06 Thread Chinmay Kolhatkar
Hi Community, I want to propose a feature in apex-core to have a UI component in stram. This is influenced by how basic runtime information is shown on UI by spark. This includes following features: 1. Webpage will be hosted by stram and will be available for user to view in one of the two ways

Apex-core build/release steps improvements proposal

2018-05-03 Thread Chinmay Kolhatkar
Hello Community, I want to propose following improvements for apex-core build and related steps: 1. Most (probably all of the open source project) has the a binary release package of the software and not just the source release package. Currently we have only source package. Luckily there are

Re: Core release 3.7.0

2018-01-18 Thread Chinmay Kolhatkar
+1 for the release. - Chinmay. On 18 Jan 2018 8:58 pm, "Tushar Gosavi" wrote: > +1 for the release > > Regards, > - Tushar. > > > On Thu, Jan 18, 2018 at 3:28 AM, Amol Kekre wrote: > > > +1 > > > > Thks, > > Amol > > > > > > E:a...@datatorrent.com

[jira] [Updated] (APEXMALHAR-2105) Enhance CSV Formatter to take in schema similar to Csv Parser

2016-06-08 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2105: -- Assignee: shubham pathak > Enhance CSV Formatter to take in schema simi

[jira] [Resolved] (APEXMALHAR-2105) Enhance CSV Formatter to take in schema similar to Csv Parser

2016-06-08 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar resolved APEXMALHAR-2105. --- Resolution: Done Fix Version/s: 3.5.0 The PR is merged

  1   2   3   >