Re: [EXTERNAL] Re: KafkaSinglePortInputOperator

2016-12-06 Thread Chaitanya Chebolu
Raja, Issue is the SSL properties(ssl.*.*) are not reflected to Kafka consumer. Could you please share the complete project ? Thanks, Chaitanya On Wed, Dec 7, 2016 at 7:39 AM, Raja.Aravapalli wrote: > Hi Chaitanya, > > > > Any other thoughts on how I can fix

Re: Query

2016-12-06 Thread Vlad Rozov
I'd recommend to use additional output port solution outlined by Bhupesh. There are few Apex applications on the field that leverage that solution. Thank you, Vlad On 12/4/16 11:45, Vishal Agrawal wrote: Thank you Bhupesh and Ram. Appreciate your quick response. I see

Re: [EXTERNAL] Re: KafkaSinglePortInputOperator

2016-12-06 Thread Raja . Aravapalli
I added the below line as said… I cannot see any exceptions also Still nothing is happening ☹ I am not sure, why these below as always showing as null… even though I set them in my Application.java class!! Any help on how to set these properties ??? ssl.keystore.location = null

Re: Cassandra Output Opretaor: Invalid null value for partition key part id

2016-12-06 Thread Priyanka Gugale
Ananth, Yes we can very well try executing each query in batch and illuminate only erroneous query/statement. That we can be detailed out. @Max, As operator doesn't handle this case, it's a good idea to use filter in upstream operator. We do have filter operator in Malhar. -Priyanka On Tue,

Re: [EXTERNAL] Re: KafkaSinglePortInputOperator

2016-12-06 Thread Chaitanya Chebolu
Raja, Please set the consumerProps to the KafkaSinglePortInputOperator. Add the below line in your application: KafkaSinglePortInputOperator in = dag.addOperator("in", new KafkaSinglePortInputOperator()); -- in.setConsumerProps(props); Please let me know, if you are still

Re: Cassandra Output Opretaor: Invalid null value for partition key part id

2016-12-06 Thread Max Bridgewater
I was wondering if there could be different strategies to choose from depending on the scenario. One could be to drop the events in that window. (Trying to avoid the word batch as I understand Apex doesn't do batch ;)) In my case, I would have preferred to update the upstream operator to filter

Re: [EXTERNAL] Re: KafkaSinglePortInputOperator

2016-12-06 Thread Raja . Aravapalli
Find below the log I am observing: 2016-12-06 05:17:37,264 INFO kafka.AbstractKafkaInputOperator (AbstractKafkaInputOperator.java:initPartitioner(311)) - Initialize Partitioner 2016-12-06 05:17:37,265 INFO kafka.AbstractKafkaInputOperator

Re: [EXTERNAL] Re: KafkaSinglePortInputOperator

2016-12-06 Thread Raja . Aravapalli
Thanks a log for the response Chaitanya!! Sharing more details for your reference and suggestions !! Appliation.java: import com.datatorrent.api.DAG; import com.datatorrent.api.StreamingApplication; import com.datatorrent.contrib.kafka.KafkaSinglePortByteArrayInputOperator; import

KafkaSinglePortInputOperator

2016-12-06 Thread Raja . Aravapalli
Hi Team, I am using “KafkaSinglePortInputOperator” to connect to a SSL Secured topic in Kafka 0.9!! Unfortunately… my apex application is not going to “RUNNING” state…!! Only staying in ACCEPTED State and then going into FAILED statie!! I don’t see much information in the logs…!! ☹ Can

Re: Cassandra Output Opretaor: Invalid null value for partition key part id

2016-12-06 Thread Ananth G
Dropping the entire batch even if one entry is invalid seems to be too stringent . Thoughts? Regards Ananth > On 6 Dec. 2016, at 5:08 pm, Priyanka Gugale wrote: > > Hi Max, > > Right now the operator doesn't provide a way to set retry limit and drop the > statement /