Re: Redshift Output Operator

Created JIRA for this task: APEXMALHAR-2416 On Mon, Feb 13, 2017 at 4:14 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Hi All, > > I am proposing Amazon Redshift output module. > Please refer below link about the Redshift: https://aws.amazon.com/ > re

Re: Kafka Input Operator for 0.10.x

Created JIRA : https://issues.apache.org/jira/browse/APEXMALHAR-2459 On Wed, Mar 22, 2017 at 4:11 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Hi All, > >I would like to propose the Kafka Input Operator using 0.10.0.0 > consumer APIs. > > Changes in 0

Re: [VOTE] Apache Apex Core Release 3.6.0 (RC1)

+1 1) Verified the file Integrity check. 2) Verified the source code using tar file. - Extracted tar file and compiled. - No binary files - LICENSE, NOTICE, README, CHANGELOG.md exist. - rat check 3) Launched PI demo. Regards, Chaitanya On Wed, May 3, 2017 at 7:07 AM, Pramod Imma

Re: [jira] [Commented] (APEXMALHAR-2537) POM bundling issue in apex-malhar/kafka/kafka010

Hi Srikanth, Below dependency also doesn't work. Because the abstract operator is under kafka-common module. org.apache.apex malhar-kafka 3.8.0-SNAPSHOT * * You need to include the dependency as follows: org.apache.apex malhar-kafka 3.8.0-SNAPSHOT org.apache.apex malhar-library

Re: [VOTE] Major version change for Apex Library (Malhar)

I think the backward in-compatibility is there in both options. I think we should give sufficient time frame for community members/ users for major changes. So, I'm -1 on both options. On Fri, Sep 1, 2017 at 5:43 PM, Sandeep Deshmukh wrote: > -1 from my side for "1. Version 4.0 as major version

Re: Malhar release 3.8.0

+1 on new release. Thanks, On Wed, Oct 25, 2017 at 9:09 PM, Vlad Rozov wrote: > +1. > > Thank you, > > Vlad > > > On 10/25/17 08:21, Amol Kekre wrote: > >> +1 on a new malhar release. >> >> Thks, >> Amol >> >> >> E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre* >> >> www.datato

Re: [ANNOUNCE] New Apache Apex Committer: Ananth Gundabattula

Congrats Ananth!! On Fri, Nov 3, 2017 at 9:49 PM, Bhupesh Chawda wrote: > Congratulations Ananth! > > ~ Bhupesh > > > ___ > > Bhupesh Chawda > > E: bhup...@datatorrent.com | Twitter: @bhupeshsc > > www.datatorrent.com | apex.apache.org > > >

Re: [ANNOUNCE] New Apache Apex PMC: Tushar Gosavi

Congratulations Tushar!!! On Fri, Nov 3, 2017 at 10:06 PM, Venkatesh Kottapalli < venkat...@datatorrent.com> wrote: > Congratulations Tushar! > > -Venky > > On Fri, Nov 3, 2017 at 9:24 AM Hitesh Kapoor > wrote: > > > Congratulations Tushar. > > > > Regards, > > Hitesh Kapoor > > > > On 03-Nov-20

Dynamic partition is not working in Kafka Input Operator

Hi All, I am facing dynamic partition issues in 0.8 version of Kafka Input Operator. My application has the following DAG: KafkaSinglePortStringInputOperator(Input) -> ConsoleOutputOperator(Output) I launched the application with below configuration: Kafka topic created with single part

Re: Dynamic partition is not working in Kafka Input Operator

Hegde wrote: > Was this resolved? > > My understanding is that, Kafka Input operator doesn't support the changes > in Kafka partitions after the initial launch. > > On Mon, Jul 18, 2016 at 1:54 AM Chaitanya Chebolu < > chaita...@datatorrent.com> > wrote: > > >

Re: Malhar 3.5.0 release

APEXMALHAR-2100. PR is open: https://github.com/apache/apex-malhar/pull/330 Regards, Chaitanya On Fri, Aug 5, 2016 at 1:10 PM, Priyanka Gugale wrote: > APEXMALHAR-2171 > PR is open: https://github.com/apache/apex-malhar/pull/358 > > -Priyanka > > On Fri, Aug 5, 2016 at 12:32 PM, Yogi Devendra <

GenericFileOutputOpeator doesn't work for all hadoop file systems

Hi All, GenericFileOutputOpeator which is in Malhar repository works only for few file systems. GenericFileOutputOpeator is extended from AbstractFileOutputOperator. Reason: openStream() method which is in AbstractFileOutputOperator calls append operation. But, all the file systems doesn't su

Re: Malhar 3.5.0 release

y%20DESC > > > > Please remove the fix version unless you expect to complete the work in > > the next few days. > > > > And we need to complete the PR for high level API: > > > > https://issues.apache.org/jira/browse/APEXMALHAR-2142 > > > > Thanks,

Re: GenericFileOutputOpeator doesn't work for all hadoop file systems

> -1 for modifying Abstract class to take care of unsupported operations. > > > > -Priyanka > > > > On Wed, Aug 24, 2016 at 6:21 PM, Chaitanya Chebolu < > > chaita...@datatorrent.com> wrote: > > > > > Hi All, > > > > > &

Re: [ANNOUNCE] New Apache Apex PMC Member: Chandni Singh

Congrats Chandni! Regards, Chaitanya On Tue, Sep 13, 2016 at 7:08 AM, Dongming Liang wrote: > Congrats, Chandni! > > Thanks, > - Dongming > > Dongming LIANG > > dongming.li...@gmail.com > > On Mon, Sep 12, 2016 at 11:28 AM, Pradeep A. Dalvi > wrote: > > > Congratulations, Chandni!

Re: Kudu store operators

+1 Regards, Chaitanya On Mon, Oct 3, 2016 at 6:01 PM, Sanjay Pujare wrote: > +1 > > On Oct 3, 2016 5:33 PM, "Sandeep Deshmukh" > wrote: > > > +1 > > > > Regards, > > Sandeep > > > > On Mon, Oct 3, 2016 at 10:16 AM, Tushar Gosavi > > wrote: > > > > > +1, It will be great to have this operator.

Outer Join

Hi David, I am working on Outer Join Using Managed State. This will support event time as well as tuple process time. This covers Left Outer, Right Outer, Full Outer Join. I am planning to start design discussion on dev@apex. But, I have seen a PR(#414

Re: S3 Output Module

pproaches. Regards, Chaitanya On Tue, Mar 29, 2016 at 2:35 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > @ Tushar > > S3 Copy Output Module consists of following operators: > 1) BlockWriter : Writes the blocks into the HDFS. > 2) Synchronizer: Sends trigger to dow

Re: [jira] [Assigned] (APEXMALHAR-2303) S3 Line By Line Module

+1 for new approach i.e, adding the file length to FileBlockMetadata. On Thu, Oct 20, 2016 at 12:00 PM, Tushar Gosavi wrote: > I think this approach is clean compare to previous two approached you > have mentioned. Depending on exception/non standard error code to > determine eof is not > good a

Re: S3 Output Module

, 2016 at 8:11 PM, Chaitanya Chebolu < chaita...@datatorrent.com> wrote: > Hi All, > > I am proposing the below new design for S3 Output Module using multi part > upload feature: > > Input to this Module: FileMetadata, FileBlockMetadata, ReaderRecord > > Steps for uplo

Re: Malhar release 3.6

Hi Thomas, I am working on APEXMALHAR-2284 and will open a PR in couple of days. Regards, Chaitanya On Sun, Nov 6, 2016 at 10:51 PM, Thomas Weise wrote: > Is anyone working on APEXMALHAR-2284 ? > > On Fri, Oct 28, 2016 at 11:00 AM, Bhupesh Chawda > wrote: > > > +1 > > > > ~ Bhupesh > > > >

Re: [VOTE] Apache Apex Malhar Release 3.6.0 (RC1)

+1 Verified the following: 1) File Integrity Check. 2) Source Code Verification using tar.gz - Extracted tar file and compiled. - No binary files - LICENSE, NOTICE, README, CHANGELOG.md exist. - rat check 3) Launched PI demo. On Thu, Dec 1, 2016 at 1:19 PM, Vlad Rozov wrote: > +

Redshift Output Operator

Hi All, I am proposing Amazon Redshift output module. Please refer below link about the Redshift: https://aws.amazon.com/redshift/ Primary functionality of this module is load data into Redshift tables from data files using copy command. Refer the below link about the copy command: http://d

Re: [VOTE] Apache Apex Malhar Release 3.4.0 (RC1)

+1 Checked : 1) File Integrity Check. 2) Source Code Verification using tar.gz - Extracted tar file and compiled. - No binary files - Checked LICENSE, NOTICE, README, NOTICE. - rat check 3) Launched PI demo. Regards, Chaitanya On Mon, May 23, 2016 at 12:40 PM, Pradeep Kumbhar wr

Re: Inner Join Operator Using Managed State

t; > > > > class InnerJoinOperator > > > { > > >SpillableArrayListMultiMap stream1Data = new > > > SpillableArrayListMultiMap(ManagedTimeUnifiedStateImp); > > > > > >port1.process (tuple) { > > >stream1Data.put(tuple.g

new method emitMessage() in Kafka Operator

Hi All, I am proposing to add new API in 0.8 version of AbstractKafkaInputOperator: void emitMessage(KafkaMessage message). "message" has details like offset, kafkapartition, value of the message. By adding this, users have more control over the message. Callback of this method would be in