Re: [DISCUSS] Maven version

lopment process needs it to be the case. >> Hence +1 for 3.5.0 >> >> Plus 3.5.x seems to provide additional goodies like colour outputs as well >> as a module progress indicators. >> >> Regards, >> Ananth >> >> >> On Thu, May 17, 2018 at 5:29 AM, Yo

Re: [DISCUSS] Maven version

1. +1 for 3.3.9. 2. Why do we need 3.5.x if 3.3.9 serves the purpose? ~ Yogi On 16 May 2018 at 20:20, Ananth G wrote: > +1 for maven version upgrade. > > +1 for version 3.5.x as there seems to be some fixes for issues related to > this property in version 3.3.9 >

Re: [VOTE] Major version change for Apex Library (Malhar)

One thing which is not clear to me is: if someone has any contributions planned for 3.x; will that contribution need 2 different PRs? If yes, then can we avoid this by giving sufficient time for the community to prepare for this change? -1 for immediate release. -1 for immediate separation for

Re: [RESULT] [VOTE] Major version change for Apex Library (Malhar)

I would suggest following: 1. Announce some end date for 3.x new features. (To give sufficient time for community members to plan their feature contributions.) 2. 3.x to 4.x migration should happen on this date. 3. Community members can submit 3.x PRs till this date. 4. New PRs after this date

[jira] [Updated] (APEXMALHAR-2494) Update demo apps with description

[ https://issues.apache.org/jira/browse/APEXMALHAR-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2494: -- Fix Version/s: 3.8.0 > Update demo apps with descript

[jira] [Resolved] (APEXMALHAR-2494) Update demo apps with description

[ https://issues.apache.org/jira/browse/APEXMALHAR-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2494. --- Resolution: Fixed > Update demo apps with descript

[jira] [Assigned] (APEXMALHAR-2476) Unable to override tupleSeparator property of GenericFileOutputOperator using configurations

[ https://issues.apache.org/jira/browse/APEXMALHAR-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra reassigned APEXMALHAR-2476: - Assignee: Yogi Devendra > Unable to override tupleSeparator prope

[jira] [Created] (APEXMALHAR-2476) Unable to override tupleSeparator property of GenericFileOutputOperator using configurations

Yogi Devendra created APEXMALHAR-2476: - Summary: Unable to override tupleSeparator property of GenericFileOutputOperator using configurations Key: APEXMALHAR-2476 URL: https://issues.apache.org/jira/browse

[jira] [Commented] (APEXMALHAR-2369) S3 output module for tuple based output

[ https://issues.apache.org/jira/browse/APEXMALHAR-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837265#comment-15837265 ] Yogi Devendra commented on APEXMALHAR-2369: --- [~chaithu] Could you please review this? >

Re: APEXMALHAR-2382 User needs to create dt_meta table while using JdbcPOJOInsertOutputOperator

uldn't worry about table creations as its one-time activity. > > > > > > > > Thanks, > > > > A > > > > > > _____ > > Sent with difficulty, I mean handheld ;) > > > > On 13 Jan 2017 6:37 pm, &q

Re: APEXMALHAR-2382 User needs to create dt_meta table while using JdbcPOJOInsertOutputOperator

operator guide as well as put a log message when we create table. > And in case you don't have privileges, the operator should throw meaningful > message. > > -Priyanka > > On Fri, Jan 13, 2017 at 4:07 PM, Yogi Devendra <yogideven...@apache.org> > wrote: > > > My

Re: APEXMALHAR-2382 User needs to create dt_meta table while using JdbcPOJOInsertOutputOperator

My suggestions: 1. Have a separate utility script for creating this table. 2. Have README for the utility script 3. Mention about the utility script in the operator javadocs. 4. Mention about the utility script in the application README. 5. If at all, you wish to ease out the

[jira] [Commented] (APEXCORE-593) apex cli get-app-package-info could not retrieve properties defined in properties.xml

[ https://issues.apache.org/jira/browse/APEXCORE-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766547#comment-15766547 ] Yogi Devendra commented on APEXCORE-593: I would take this up. [~tushargosavi] has agreed

[jira] [Created] (APEXCORE-593) apex cli get-app-package-info could not retrieve propeties defined in properties.xml

Yogi Devendra created APEXCORE-593: -- Summary: apex cli get-app-package-info could not retrieve propeties defined in properties.xml Key: APEXCORE-593 URL: https://issues.apache.org/jira/browse/APEXCORE-593

[jira] [Updated] (APEXCORE-593) apex cli get-app-package-info could not retrieve properties defined in properties.xml

[ https://issues.apache.org/jira/browse/APEXCORE-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXCORE-593: --- Summary: apex cli get-app-package-info could not retrieve properties defined

[jira] [Updated] (APEXMALHAR-2369) S3 output module for tuple based output

[ https://issues.apache.org/jira/browse/APEXMALHAR-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2369: -- Description: Currently, S3 output is available using S3OutputModule which

[jira] [Created] (APEXMALHAR-2369) S3 output module for tuple based output

Yogi Devendra created APEXMALHAR-2369: - Summary: S3 output module for tuple based output Key: APEXMALHAR-2369 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2369 Project: Apache Apex Malhar

[jira] [Resolved] (APEXMALHAR-2302) Exposing few properties of FSSplitter and BlockReader operators to FSRecordReaderModule to tune Application

[ https://issues.apache.org/jira/browse/APEXMALHAR-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2302. --- Resolution: Fixed Fix Version/s: 3.6.0 > Exposing few propert

[jira] [Resolved] (APEXMALHAR-2312) NullPointerException in FileSplitterInput only if the file path is specified for attribute instead of directory path

[ https://issues.apache.org/jira/browse/APEXMALHAR-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2312. --- Resolution: Fixed Fix Version/s: 3.6.0 > NullPointerExcept

[jira] [Commented] (APEXMALHAR-1818) Integrate Calcite to support SQL

[ https://issues.apache.org/jira/browse/APEXMALHAR-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548137#comment-15548137 ] Yogi Devendra commented on APEXMALHAR-1818: --- Reviewed the PR. Given some comments

[jira] [Created] (APEXMALHAR-2266) Documentation for quasi java expressions

Yogi Devendra created APEXMALHAR-2266: - Summary: Documentation for quasi java expressions Key: APEXMALHAR-2266 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2266 Project: Apache Apex

[jira] [Created] (APEXMALHAR-2237) Dynamic partitioning support for FSInputModule

Yogi Devendra created APEXMALHAR-2237: - Summary: Dynamic partitioning support for FSInputModule Key: APEXMALHAR-2237 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2237 Project: Apache Apex

[jira] [Created] (APEXMALHAR-2234) Add user documentation for FilterOperator

Yogi Devendra created APEXMALHAR-2234: - Summary: Add user documentation for FilterOperator Key: APEXMALHAR-2234 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2234 Project: Apache Apex

Re: [ANNOUNCE] New Apache Apex PMC Member: Chandni Singh

Congratulations !!! ~ Yogi On 13 September 2016 at 10:30, Tushar Gosavi wrote: > Congratulations Chandni > > - Tushar. > > > On Tue, Sep 13, 2016 at 9:59 AM, Bhupesh Chawda > wrote: > > Congrats Chandni! > > > > ~ Bhupesh > > > > On Tue, Sep

Re: Fixed Width Record Parser

me existing library for parsing. There is no > point in re-inventing the wheels and trying to make something robust can be > time consuming. > > -Chinmay. > > > On Wed, Sep 7, 2016 at 4:33 PM, Yogi Devendra < > devendra.vyavah...@gmail.com> > wrote: > > > Appro

[jira] [Comment Edited] (APEXMALHAR-2224) GenericFileOutputOperator rotateCall fails if no data is written to the file

[ https://issues.apache.org/jira/browse/APEXMALHAR-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470073#comment-15470073 ] Yogi Devendra edited comment on APEXMALHAR-2224 at 9/7/16 8:59 AM

[jira] [Commented] (APEXMALHAR-2224) GenericFileOutputOperator rotateCall fails if no data is written to the file

[ https://issues.apache.org/jira/browse/APEXMALHAR-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470073#comment-15470073 ] Yogi Devendra commented on APEXMALHAR-2224: --- One possible solution could be: check

[jira] [Created] (APEXMALHAR-2224) GenericFileOutputOperator rotateCall fails if no data is written to the file

Yogi Devendra created APEXMALHAR-2224: - Summary: GenericFileOutputOperator rotateCall fails if no data is written to the file Key: APEXMALHAR-2224 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2224

Re: [VOTE] Apache Apex Malhar Release 3.5.0 (RC1)

+1 Checked for - Signatures - No binary files - Rat check and build passed ~ Yogi On 29 August 2016 at 10:29, Thomas Weise wrote: > Dear Community, > > Please vote on the following Apache Apex Malhar 3.5.0 release candidate. > > This is a source release with binary

[jira] [Commented] (APEXMALHAR-2206) Some Application tests taking too long

[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445329#comment-15445329 ] Yogi Devendra commented on APEXMALHAR-2206: --- Should we mark fix version as 3.5.0

[jira] [Created] (APEXMALHAR-2207) JsonFormatterTest application test should check for presence of expected results

Yogi Devendra created APEXMALHAR-2207: - Summary: JsonFormatterTest application test should check for presence of expected results Key: APEXMALHAR-2207 URL: https://issues.apache.org/jira/browse/APEXMALHAR

[jira] [Commented] (APEXMALHAR-2206) Some Application tests taking too long

[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439309#comment-15439309 ] Yogi Devendra commented on APEXMALHAR-2206: --- Yes. It has ConsoleOutputOperator output

[jira] [Commented] (APEXMALHAR-2206) Some Application tests taking too long

[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439220#comment-15439220 ] Yogi Devendra commented on APEXMALHAR-2206: --- I propose to create separate newbie ticket

[jira] [Commented] (APEXMALHAR-2206) Some Application tests taking too long

[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439215#comment-15439215 ] Yogi Devendra commented on APEXMALHAR-2206: --- Problem regarding anti pattern in the unit

[jira] [Resolved] (APEXMALHAR-2206) Some Application tests taking too long

[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2206. --- Resolution: Fixed > Some Application tests taking too l

Re: Malhar 3.5.0 release

Fix for APEXMALHAR-2206 has been reviewed and merged into asf master. ~ Yogi On 26 August 2016 at 19:04, Yogi Devendra <devendra.vyavah...@gmail.com> wrote: > We noticed that lc.runAsync() is called without corresponding > lc.shutdown() at couple of places in unit tests. > >

Re: GenericFileOutputOpeator doesn't work for all hadoop file systems

I propose alternate approach to than the 3 options mentioned above: In AbstractFileOutputOperator we can introduce one flag saying isFileSystemAppendSupported. This flag should be set based on the filePath in setup or activate method. It can be done in 2 ways: 1. Adding if else rules based on

Re: Malhar 3.5.0 release

; > >> > > >> Regards, > > >> Chaitanya > > >> > > >> On Fri, Aug 5, 2016 at 1:10 PM, Priyanka Gugale < > > priya...@datatorrent.com > > >> > > > >> wrote: > > >> > > >> > APEXMA

[jira] [Updated] (APEXMALHAR-2195) LineReaderContext gives incorrect results for files not ending with the newline

[ https://issues.apache.org/jira/browse/APEXMALHAR-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2195: -- Fix Version/s: 3.5.0 > LineReaderContext gives incorrect results for fi

[jira] [Updated] (APEXMALHAR-2198) ReaderContext should handle different encodings for String

[ https://issues.apache.org/jira/browse/APEXMALHAR-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2198: -- Assignee: (was: Yogi Devendra) > ReaderContext should handle differ

Re: Default value for cooldownMillis in StatsAwareStatelessPartitioner

I tried 30 sec which worked well. ~ Yogi On 18 August 2016 at 22:28, Munagala Ramanath <r...@datatorrent.com> wrote: > Just out of curiosity what is the smallest value that fixes the issue ? > > Ram > > On Thu, Aug 18, 2016 at 3:55 AM, Yogi Devendra <yogideven...@apach

Default value for cooldownMillis in StatsAwareStatelessPartitioner

Hi, Current default value is 2000 ms. I tried an application which uses default value. It was noticed that it leads to continuous re-deployment of the operator. Reason could be because 2000 ms is not sufficient for re-deploy+reaching some steady state. How about changing default to 30 sec or 1

[jira] [Commented] (APEXMALHAR-2196) S3 Record reader module

[ https://issues.apache.org/jira/browse/APEXMALHAR-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422518#comment-15422518 ] Yogi Devendra commented on APEXMALHAR-2196: --- This module will be similar in functionality

[jira] [Created] (APEXMALHAR-2196) S3 Record reader module

Yogi Devendra created APEXMALHAR-2196: - Summary: S3 Record reader module Key: APEXMALHAR-2196 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2196 Project: Apache Apex Malhar Issue

[jira] [Commented] (APEXMALHAR-2195) LineReaderContext gives incorrect results for files not ending with the newline

[ https://issues.apache.org/jira/browse/APEXMALHAR-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422467#comment-15422467 ] Yogi Devendra commented on APEXMALHAR-2195: --- ReaderContext.LineReaderContext not able

[jira] [Created] (APEXMALHAR-2195) LineReaderContext gives incorrect results for files not ending with the newline

Yogi Devendra created APEXMALHAR-2195: - Summary: LineReaderContext gives incorrect results for files not ending with the newline Key: APEXMALHAR-2195 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2195

Re: Default value for tupleSeperator

d all the time. When such case arise, the > user can always configure empty string as delimiter in the operator he > would prefer to. > > -Venkatesh. > > > > On Aug 12, 2016, at 10:08 PM, Yogi Devendra < > devendra.vyavah...@gmail.com> wrote: > > > > @V

Re: Default value for tupleSeperator

19:09, Venkatesh Kottapalli < > venkat...@datatorrent.com> wrote: > > > > > > IMO, since we have an option to configure the line delimiter, leaving it > to the user is a good idea as there might be cases where either one of them > is used and not both. > > > >

Default value for tupleSeperator

Hi, Currently, for the usecases involving writing POJO to file output we use DAG simillar to following: POJOEmittingOperator->CSVFormatter->StringFileOutputOperator. Both CSVFormatter as well as StringFileOutputOperator have configurable tupleSeperator defaulting to newline. This introduces two

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Congratulations Dev !!! ~ Yogi On 11 August 2016 at 11:37, Chinmay Kolhatkar wrote: > Congrats Dev!! > > On Thu, Aug 11, 2016 at 10:50 AM, Shubham Pathak > wrote: > > > Congratulations Dev ! > > > > Thanks, > > Shubham > > > > On Thu, Aug 11,

[jira] [Updated] (APEXMALHAR-2161) Add tests for AbstractThroughputFileInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2161: -- Fix Version/s: 3.5.0 > Add tests for AbstractThroughputFileInputOpera

[jira] [Resolved] (APEXMALHAR-2161) Add tests for AbstractThroughputFileInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2161. --- Resolution: Fixed > Add tests for AbstractThroughputFileInputOpera

[jira] [Updated] (APEXMALHAR-2174) S3 File Reader reading more data than expected

[ https://issues.apache.org/jira/browse/APEXMALHAR-2174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2174: -- Fix Version/s: 3.5.0 > S3 File Reader reading more data than expec

Re: JDBC Poller implementation

+1 for Tushar's idea for common abstract class. Additionally, POJO based output should be considered for active development. CSV output can be deprecated. Since it can be achieved easily using POJO + CSV formatter. Operators are meant to be lego blocks for reusable functionality to achieve

Re: Malhar 3.5.0 release

l 29, 2016 at 1:53 PM, Yogi Devendra < > devendra.vyavah...@gmail.com > > wrote: > > > APEXMALHAR-2116 > > https://github.com/apache/apex-malhar/pull/326 > > > > @Ram > > Your review comments are incorporated. Could you please have a look at > > thi

[jira] [Commented] (APEXCORE-501) Sub Partitioning the parallel partitions

[ https://issues.apache.org/jira/browse/APEXCORE-501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408917#comment-15408917 ] Yogi Devendra commented on APEXCORE-501: Link for the discussion: https://lists.apache.org

[jira] [Created] (APEXCORE-501) Sub Partitioning the parallel partitions

Yogi Devendra created APEXCORE-501: -- Summary: Sub Partitioning the parallel partitions Key: APEXCORE-501 URL: https://issues.apache.org/jira/browse/APEXCORE-501 Project: Apache Apex Core

[jira] [Comment Edited] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

[ https://issues.apache.org/jira/browse/APEXMALHAR-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407815#comment-15407815 ] Yogi Devendra edited comment on APEXMALHAR-2176 at 8/4/16 2:10 PM

[jira] [Commented] (APEXMALHAR-2176) expressionFunctions for FilterOperator throws IndexOutOfBounds

[ https://issues.apache.org/jira/browse/APEXMALHAR-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407815#comment-15407815 ] Yogi Devendra commented on APEXMALHAR-2176: --- additional setter method for listItem

[jira] [Created] (APEXMALHAR-2175) POJOUtils is considering java reserved words as fields in the expression

Yogi Devendra created APEXMALHAR-2175: - Summary: POJOUtils is considering java reserved words as fields in the expression Key: APEXMALHAR-2175 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2175

Re: [Proposal] Named Checkpoints

This will be awesome feature. I can see the usecases for production scenario as well as developement/troubleshooting environments. Excellent value add. ~ Yogi On 4 August 2016 at 12:40, Sandesh Hegde wrote: > Hello Team, > > This thread is to discuss the Named

Re: Malhar 3.5.0 release

APEXMALHAR-2116 https://github.com/apache/apex-malhar/pull/326 @Ram Your review comments are incorporated. Could you please have a look at this? ~ Yogi On 29 July 2016 at 13:49, Thomas Weise wrote: > Hi, > > I would suggest to target the 3.5.0 release with an RC out

Re: APEXMALHAR-1701 Deduper in Malhar

+1 for Windowed Operator if we manage to take care of the problem you mentioned for this approach in the earlier email. IMO, "WindowedStorage to suit the dedup use case" would open up ways for any other similar use-cases for Windowed Operators which needs some custom handling. -1 for keeping

condition in FilterOperator

Hi, I am working on creating example app for com.datatorrent.lib.filter.FilterOperator For condition property, I gave following value: {$}.getAmount() >= 2 This gives following exception: java.lang.RuntimeException: org.codehaus.commons.compiler.CompileException: Line 1, Column 29: Cannot

[jira] [Resolved] (APEXMALHAR-2019) Creation of S3 Input Module

[ https://issues.apache.org/jira/browse/APEXMALHAR-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra resolved APEXMALHAR-2019. --- Resolution: Fixed > Creation of S3 Input Mod

[jira] [Updated] (APEXMALHAR-2019) Creation of S3 Input Module

[ https://issues.apache.org/jira/browse/APEXMALHAR-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2019: -- Fix Version/s: 3.5.0 > Creation of S3 Input Mod

Re: Windowed Operator PR

+1 for merging and incremental additions. ~ Yogi On 11 July 2016 at 11:04, Chinmay Kolhatkar wrote: > +1 on merging. > > On Mon, Jul 11, 2016 at 11:00 AM, Amol Kekre wrote: > > > +1 on merging it. This is a big feature addition. > > > > Thks > >

[jira] [Updated] (APEXMALHAR-2104) input ports in BytesFileOutputOperator should be optional

[ https://issues.apache.org/jira/browse/APEXMALHAR-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogi Devendra updated APEXMALHAR-2104: -- Priority: Minor (was: Major) > input ports in BytesFileOutputOperator sho

[jira] [Commented] (APEXMALHAR-2104) input ports in BytesFileOutputOperator should be optional

[ https://issues.apache.org/jira/browse/APEXMALHAR-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15309364#comment-15309364 ] Yogi Devendra commented on APEXMALHAR-2104: --- input port used for byte[] is defined

Overriding input port annotations from parent class

Hi, Here is my scenario: 1. Class A with DefaultInputPort input. 2. Class B extends A with DefaultInputPort stringInput. 3. Class A does NOT define optional=true for input. 4. In Class B, I want to specify that any one of input, stringInput should be connected. (OR making both of them as