Thanks Joe. I updated my config and not seeing the issue anymore. On Wed, Mar 30, 2016 at 1:18 PM, Joe Witt <[email protected]> wrote:
> Hello > > From your screenshot it shows you have both success and failure > looping back to Kafka. Do not loop success and you should be fine. > > Thanks > Joe > > On Wed, Mar 30, 2016 at 11:16 AM, Madhukar Thota > <[email protected]> wrote: > > I was able to construct the Json with right data type output from > > ExecuteScript and sending to Kafka directly. The problem i am seeing is > if i > > send one record to kafka, kafka processor is writing the message again > and > > again and not ending the loop. How can i send exactly once message? Any > > help. > > > > Here is what i am doing in my script: > > > > import simplejson as json > > from org.apache.nifi.processor.io import StreamCallback > > from user_agents import parse > > > > > > def num(s): > > try: > > return int(s) > > except ValueError: > > try: > > return float(s) > > except ValueError: > > raise ValueError('argument is not a string of number') > > > > > > class PyStreamCallback(StreamCallback): > > def __init__(self): > > pass > > > > def process(self, inputStream, outputStream): > > obj = {'browser': > > str(parse(flowFile.getAttribute('useragent')).browser.family), > > 'browser_version': > > str(parse(flowFile.getAttribute('useragent')).browser.version_string), > > 'os': > > str(parse(flowFile.getAttribute('useragent')).os.family), > > 'os_version': > > str(parse(flowFile.getAttribute('useragent')).os.version_string), > > 'client_ip': flowFile.getAttribute('clientip')} > > if flowFile.getAttribute('http.param.t_resp') and > > flowFile.getAttribute('http.param.t_page') and > > flowFile.getAttribute('http.param.t_done'): > > obj['rt_firstbyte'] = > > num(flowFile.getAttribute('http.param.t_resp')) > > obj['rt_lastbyte'] = > > num(flowFile.getAttribute('http.param.t_page')) > > obj['rt_loadtime'] = > > num(flowFile.getAttribute('http.param.t_done')) > > outputStream.write(bytearray(json.dumps(obj, > > indent=4).encode('utf-8'))) > > > > > > flowFile = session.get() > > if (flowFile != None): > > flowFile = session.write(flowFile, PyStreamCallback()) > > session.transfer(flowFile, REL_SUCCESS) > > > > > > Thanks > > > > On Tue, Mar 29, 2016 at 2:30 AM, Conrad Crampton > > <[email protected]> wrote: > >> > >> Hi, > >> Depending on the final destination of the data (json) you could use the > >> JsonToAvro -> ConvertAvroSchema -> AvroToJson, with the > ConvertAvroSchema > >> doing the type conversion. I had to do this as I came across this > behaviour > >> previously. I use the Avro directly (after the conversion) as that was > my > >> final data format requirement, but I don’t see any reason if you want > Json > >> back that this wouldn’t work. I haven’t tried this by the way, but the > type > >> conversion certainly works for the final attributes in the Avro > documents. > >> Conrad > >> > >> From: Madhukar Thota <[email protected]> > >> Reply-To: "[email protected]" <[email protected]> > >> Date: Friday, 25 March 2016 at 14:01 > >> To: "[email protected]" <[email protected]> > >> Subject: Re: String conversion to Int, float double > >> > >> Any Other ways to achieve this? > >> > >> On Thu, Mar 24, 2016 at 4:48 PM, Bryan Bende <[email protected]> wrote: > >>> > >>> I think the problem is that all attributes are actually Strings > >>> internally, even after calling toNumber() that is only temporary while > the > >>> expression language is executing. > >>> > >>> So by the time it gets to AttributesToJson it doesn't have any > >>> information about the type of each attribute and they all end up as > Strings. > >>> I think we would have to come up with a way to pass some type > information > >>> along to AttributesToJson in order to get something other than Strings. > >>> > >>> -Bryan > >>> > >>> > >>> On Thu, Mar 24, 2016 at 3:30 PM, Madhukar Thota > >>> <[email protected]> wrote: > >>>> > >>>> Hi i am trying to convert string value to integer in UpdateAtrributes > >>>> using toNumber like this > >>>> > >>>> > >>>> ${http.param.t_resp:toNumber()} where http.param.t_resp = "132" > >>>> > >>>> but when the fileattribute pushed to Attributetojson processor , i am > >>>> stilling seeing it as string. Am i am doing something wrong? and also > how > >>>> can i convert string to float? > >>>> > >>>> > >>>> > >>>> > >>> > >> > >> > >> > >> ***This email originated outside SecureData*** > >> > >> Click here to report this email as spam. > >> > >> > >> > >> SecureData, combating cyber threats > >> > >> ________________________________ > >> > >> The information contained in this message or any of its attachments may > be > >> privileged and confidential and intended for the exclusive use of the > >> intended recipient. If you are not the intended recipient any > disclosure, > >> reproduction, distribution or other dissemination or use of this > >> communications is strictly prohibited. The views expressed in this > email are > >> those of the individual and not necessarily of SecureData Europe Ltd. > Any > >> prices quoted are only valid if followed up by a formal written quote. > >> > >> SecureData Europe Limited. Registered in England & Wales 04365896. > >> Registered Address: SecureData House, Hermitage Court, Hermitage Lane, > >> Maidstone, Kent, ME16 9NT > > > > >
