txxXogZA0/LJueYiC8wi1mulaGSA15EnW2/f2JYgu8JVEyP9KSJhPUdp1SBGmMjOU
> TNr
> ylmQ==
> X-Gm-Message-State: AE9vXwOBHlMhqJNYH93cPHVYROcXGNG74VY5L+y4iCsnUK5+
> oY0E4zPU24FICsPqh8bEifQAhqFs7vF+flvjqw==
> X-Received: by 10.36.53.214 with SMTP id k205mr2760283ita.85.
>
you trying to run, how many operators?
>
> ~ Bhupesh
>
> On Fri, Sep 16, 2016 at 6:54 AM, Jaspal Singh > wrote:
>
>> Hi Team,
>>
>> I am using the datatorrent sandbox and deploying a streaming application
>> integrating it with mapr streams. Even if I try t
inputs on this issue ?
Thanks
Jaspal
On Fri, Sep 16, 2016 at 8:35 PM, Jaspal Singh
wrote:
> Bhupesh,
>
> Actually we have the data torrent application setup in lab environment for
> POC and YARN RM is running on dbslt0076. We are trying to run a streaming
> application readin
> and verify that no hadoop jars have been bundled.
>
> ~ Bhupesh
>
>
> On Wed, Sep 21, 2016 at 1:45 AM, Jaspal Singh > wrote:
>
>> Hi Team,
>>
>> We are trying to build a DT application to read messages from mapr
>> streams and write them on to h
Hi Thomas,
In our case we are writing the results back to maprstreams topic based on
some validations.
Thanks
Jaspal
On Thursday, October 6, 2016, Thomas Weise wrote:
> Hi,
>
> which operators in your application are writing to external systems?
>
> When you look at the example from the blog
> stated assumptions.
>
>
> On Thu, Oct 6, 2016 at 3:37 PM, Jaspal Singh > wrote:
>
>> Hi Thomas,
>>
>> In our case we are writing the results back to maprstreams topic based
>> on some validations.
>>
>>
>> Thanks
>> Jaspal
offset ranges), the computation in the DAG is idempotent
> and the output operator can discard the results that were already published
> instead of producing duplicates.
>
> On Thu, Oct 6, 2016 at 3:57 PM, Jaspal Singh > wrote:
>
>> I think this is something called a customized
instance's checkpointed state).
>
> For cold restart, you would need to consider the property you mention and
> decide what is applicable to your use case.
>
> Thomas
>
>
> On Thu, Oct 6, 2016 at 4:16 PM, Jaspal Singh
> wrote:
>
>> Ok now I get it. Thanks fo
:
> Are you referring to the upstream operator in the DAG or the state of the
> previous application after relaunch? Since the data is stored in MapR
> streams, an operator that is a producer can also act as a consumer. Please
> clarify your question.
>
>
> On Fri, Oct 7, 2016 at
aapi/producer/Producer; at
java.lang.Class.getDeclaredFields0(Native Method) at
java.lang.Class.privateGetDeclaredFields(Class.java:2583) at
java.lang.Class.getDeclaredFields(Class.java:1916) at
Thanks
Jaspal
On Fri, Oct 7, 2016 at 10:42 AM, Jaspal Singh
wrote:
> Thomas,
>
> I was trying
ples/objects are moved by the
> Apex engine through the stream from operator to operator. There is nothing
> you need to do beyond connecting the operator ports with addStream when you
> specify the DAG.
>
>
> On Fri, Oct 7, 2016 at 9:01 AM, Jaspal Singh
> wrote:
>
&g
kaProducer instead of
>> kafka.javaapi.producer.Producer. That is old api and that is not
>> supported by MapR stream.
>>
>>
>> Regards,
>> Siyuan
>>
>> On Fri, Oct 7, 2016 at 9:01 AM, Jaspal Singh
>> wrote:
>>
>>> Thomas,
>>&
>> wrote:
>>>
>>>> Hey Jaspal,
>>>>
>>>> Did you add any code to existing KafkaSinglePortExactlyOnceOutputOperator
>>>> from malhar? If so please make sure the producer you use here is
>>>> org.apache.kafka.clients.producer.KafkaProducer ins
So I just changes the malhar-kafka version to 3.5.0, I was able to import
the AbstractOutputOperator. Let me try to launch it now.
Thanks for your inputs !!
On Fri, Oct 7, 2016 at 1:09 PM, Jaspal Singh
wrote:
> Should we use malhar-library version 3.5 then ?
>
>
> Thanks!!
>
=com.example.datatorrent.KafkaSinglePortExactlyOnceOutputOperator@4726f93f,
propertyPath='topic', message='may not be null', *
Thanks!!
On Fri, Oct 7, 2016 at 1:13 PM, Jaspal Singh
wrote:
> So I just changes the malhar-kafka version to 3.5.0, I was able to import
> the AbstractOutputOperator. L
le: /your/stream/path:streamname
>
> Regards,
> Siyuan
>
> On Fri, Oct 7, 2016 at 11:22 AM, Jaspal Singh
> wrote:
>
>> After making the change, we are getting the below error while application
>> launch:
>>
>> *An e
Name(), gson.toJson(tenant)));
}
producer.flush();
}
}
Thanks!!
On Fri, Oct 7, 2016 at 1:34 PM, hsy...@gmail.com wrote:
> Jaspal,
>
> I think you miss the kafkaOut :)
>
> Regards,
> Siyuan
>
> On Fri, Oct 7, 2016 at 11:32 AM, Jaspal Singh
> wrote:
>
&g
Hi Team,
We are pushing messages from Kafka to Datatorrent application using REST
API service as a producer. If say due to some issue with the service the
message couldn't be pushed/processed so we want to "retry it for n times"
before it is dropped.
Is there any retry functionality built within
; Apex Kafka Input Operator.
>
> If this is the flow, then the kafka input operator should be reading
> messages from Kafka without losing them. There is no retry attempts
> necessary.
>
> Let me know if the understanding of the flow is incorrect.
>
> Regards,
> Ashwin.
>
&g
:
>>
>> We have a Hbase client dependency and do not have any Hadoop dependencies
>> in project.
>>
>>
>>
>> *From:* Ashwin Chandra Putta [mailto:ashwinchand...@gmail.com]
>> *Sent:* Wednesday, October 19, 2016 2:40 PM
>> *To:* users@apex.apache.org
>> *Su
Where I need to set the table name. In property file or the application
code ?
Thanks!!
On Wednesday, October 19, 2016, Sanjay Pujare
wrote:
> Take a look at https://github.com/apache/apex-malhar/tree/master/
> contrib/src/main/java/com/datatorrent/contrib/hbase . There are multiple
> output o
example that uses HBase that may be helpful:
>
> https://github.com/apache/apex-malhar/blob/master/demos/
> twitter/src/main/java/com/datatorrent/demos/twitter/
> TwitterDumpHBaseApplication.java
>
> Thomas
>
> On Wed, Oct 19, 2016 at 6:36 PM, Jaspal Singh > wrote:
>
t.getText().getBytes());
> put.add("cf".getBytes(), "userid".getBytes(), t.getText().getBytes());
> return put;
> }
> }
> {code}
>
> - Tushar.
>
>
> On Thu, Oct 20, 2016 at 11:59 AM, Jaspal Singh
> wrote:
> > Hi Thomas, Thanks for sharing this exa
endencies-conflicts
>
>
> On Thu, Oct 20, 2016 at 11:39 AM, Thomas Weise wrote:
>
>> Please see the HBase dependency and its exclusions here:
>>
>> https://github.com/apache/apex-malhar/blob/master/contrib/pom.xml#L342
>>
>> Thanks,
>> Thomas
>>
Thomas,
Can you please highlight how the Gson has to be a transient member ? Below
is our operator code.
public class HbaseTableUpdate extends
AbstractHBasePutOutputOperator implements Serializable {
private static final transient Logger logger =
LoggerFactory.getLogger(HbaseTableUpdate.class);
eems to be serializable with kryo. Gson instance does not
> maintain any useful state, you can safely make it transient and
> initialise it in constructor of operator.
>
> - Tushar.
>
>
>
> On Fri, Oct 21, 2016 at 9:52 AM, Jaspal Singh
> wrote:
> > Thomas,
> >
There are couple of Kafka input operators like
KafkaSinglePortBytesInputOperator and KafkaSinglePortInputOperator that
extends AbstractKafkaInputOperator. They are part of malhar library.
They can right away consume messages from kafka topics.
Thanks!!
On Monday, October 24, 2016, Raja.Aravapall
Team,
Is there any exactly once Hbase put operator ??
Also can we define two tables using the set table property for Hbase store
??
Thanks!!
On Thursday, October 20, 2016, Jaspal Singh
wrote:
> Thomas,
>
> When we try excluding the transitive dependencies using 'exclusion *&#
com.example.datatorrent.HbaseTableUpdate
Thanks!!
On Thu, Oct 27, 2016 at 9:22 AM, Jaspal Singh
wrote:
> Team,
>
> Is there any exactly once Hbase put operator ??
>
> Also can we define two tables using the set table property for Hbase store
> ??
>
> Thanks!!
>
>
> On Thursday, Octobe
29 matches
Mail list logo
