Hi, I have created a WebSocketReceiver, A Stream and a Kafka Publisher with the following configuration
*Receiver* <?xml version="1.0" encoding="UTF-8"?> <eventReceiver name="TweetReceiver" statistics="disable" trace="disable" xmlns="http://wso2.org/carbon/eventreceiver";> <from eventAdapterType="websocket"> <property name="websocket.server.url">ws://localhost:1880/node-red/twitterStream</property> <property name="events.duplicated.in.cluster">false</property> </from> <mapping customMapping="enable" type="json"> <property> <from jsonPath="$"/> <to name="twitterMessage" type="string"/> </property> </mapping> <to streamName="TweetStream" version="1.0.0"/> </eventReceiver> *Stream * { "streamId": "TweetStream:1.0.0", "name": "TweetStream", "version": "1.0.0", "nickName": "TweetStream", "description": "TweetStream", "metaData": [], "correlationData": [], "payloadData": [ { "name": "twitterMessage", "type": "STRING" } ] } <?xml version="1.0" encoding="UTF-8"?> <eventPublisher name="KafkaTweetPublisher" statistics="disable" trace="disable" xmlns="http://wso2.org/carbon/eventpublisher";> <from streamName="TweetStream" version="1.0.0"/> <mapping customMapping="disable" type="json"/> <to eventAdapterType="kafka"> <property name="topic">SmartCity</property> <property name="meta.broker.list">localhost:9092</property> </to> </eventPublisher> My issue is that there is not type for JSON object. I am forced to use string. And this causes the json object to be enclosed in quotes, as can be seen from the following json snippet. The field "twitterMessage" [marked in bold has its contents wrapped in quotes, which should not be the case. If there were a type json object, or object, these quotes would be avoided. I tried to parse it into an object, which obviously did not work, since it was a string. But for some reason, the jackson json library also cant parse the string, because it is not escaped properly The following post explains the issue. [https://developer.salesforce.com/forums/?id=906F0000000B3pnIAC] *There are two solutions* a. Escape the json string properly b. Add an object type to the list of types so that the JSON object is not enclosed in quotes and passed as is. The following is the json produced by the publisher. Please note the quotes immediately after *"twitterMessage":* This forces me to access each field from the tweet individually, leading to duplication of effort in generating the json. { "event":{ "payloadData":{ * "twitterMessage":"{"* payload":"RT @ipfconline1: The Internet Of Things Changing Our Life\n#IoT #IoE #M2M #SmartCity #Sensors #HealthCare\n#makeyourownlane #defstar5\u2026 ", "topic":"tweets\/techbhullar", "location":{ "place":"Amritsar, India" }, "tweet":{ "in_reply_to_status_id_str":null, "in_reply_to_status_id":null, "created_at":"Sat Jul 22 06:14:51 +0000 2017", "in_reply_to_user_id_str":null, "source":"<a href=\"http:\/\/twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>", "retweeted_status":{ "extended_tweet":{ "extended_entities":{ "media":[ { "display_url":"pic.twitter.com\/N1uVdV9zLv", "indices":[ 136, 159 ], "sizes":{ "small":{ "w":650, "h":242, "resize":"fit" }, "large":{ "w":650, "h":242, "resize":"fit" }, "thumb":{ "w":150, "h":150, "resize":"crop" }, "medium":{ "w":650, "h":242, "resize":"fit" } }, "id_str":"888383385558802436", "expanded_url":"https:\/\/twitter.com \/ipfconline1\/status\/888383389883150336\/photo\/1", "media_url_https":"https:\/\/pbs.twimg.com \/media\/DFQr-3iXYAQ_JM3.jpg", "id":888383385558802400, "type":"photo", "media_url":"http:\/\/pbs.twimg.com \/media\/DFQr-3iXYAQ_JM3.jpg", "url":"https:\/\/t.co\/N1uVdV9zLv" } ] }, "entities":{ "urls":[ ], "hashtags":[ { "indices":[ 41, 45 ], "text":"IoT" }, { "indices":[ 46, 50 ], "text":"IoE" }, { "indices":[ 51, 55 ], "text":"M2M" }, { "indices":[ 56, 66 ], "text":"SmartCity" }, { "indices":[ 67, 75 ], "text":"Sensors" }, { "indices":[ 76, 87 ], "text":"HealthCare" }, { "indices":[ 88, 104 ], "text":"makeyourownlane" }, { "indices":[ 105, 114 ], "text":"defstar5" }, { "indices":[ 115, 122 ], "text":"Mpgvip" } ], "media":[ { "display_url":"pic.twitter.com\/N1uVdV9zLv", "indices":[ 136, 159 ], "sizes":{ "small":{ "w":650, "h":242, "resize":"fit" }, "large":{ "w":650, "h":242, "resize":"fit" }, "thumb":{ "w":150, "h":150, "resize":"crop" }, "medium":{ "w":650, "h":242, "resize":"fit" } }, "id_str":"888383385558802436", "expanded_url":"https:\/\/twitter.com \/ipfconline1\/status\/888383389883150336\/photo\/1", "media_url_https":"https:\/\/pbs.twimg.com \/media\/DFQr-3iXYAQ_JM3.jpg", "id":888383385558802400, "type":"photo", "media_url":"http:\/\/pbs.twimg.com \/media\/DFQr-3iXYAQ_JM3.jpg", "url":"https:\/\/t.co\/N1uVdV9zLv" } ], "user_mentions":[ { "indices":[ 123, 135 ], "screen_name":"reach2ratan", "id_str":"262641807", "name":"Ratan Jyoti", "id":262641807 } ], "symbols":[ ] }, "full_text":"The Internet Of Things Changing Our Life\n#IoT #IoE #M2M #SmartCity #Sensors #HealthCare\n#makeyourownlane #defstar5 #Mpgvip @reach2ratan https:\/\/t.co\/N1uVdV9zLv", "display_text_range":[ 0, 135 ] }, "in_reply_to_status_id_str":null, "in_reply_to_status_id":null, "created_at":"Fri Jul 21 13:01:20 +0000 2017", "in_reply_to_user_id_str":null, "source":"<a href=\"http:\/\/www.hootsuite.com\" rel=\"nofollow\">Hootsuite<\/a>", "retweet_count":16, "retweeted":false, "geo":null, "filter_level":"low", "in_reply_to_screen_name":null, "is_quote_status":false, "id_str":"888383389883150336", "in_reply_to_user_id":null, "favorite_count":15, "id":888383389883150300, "text":"The Internet Of Things Changing Our Life\n#IoT #IoE #M2M #SmartCity #Sensors #HealthCare\n#makeyourownlane #defstar5\u2026 https:\/\/t.co\/qKfi2EgAZ5", "place":null, "lang":"en", "favorited":false, "possibly_sensitive":false, "coordinates":null, "truncated":true, "entities":{ "urls":[ { "display_url":"twitter.com \/i\/web\/status\/8\u2026", "indices":[ 116, 139 ], "expanded_url":"https:\/\/twitter.com \/i\/web\/status\/888383389883150336", "url":"https:\/\/t.co\/qKfi2EgAZ5" } ], "hashtags":[ { "indices":[ 41, 45 ], "text":"IoT" }, { "indices":[ 46, 50 ], "text":"IoE" }, { "indices":[ 51, 55 ], "text":"M2M" }, { "indices":[ 56, 66 ], "text":"SmartCity" }, { "indices":[ 67, 75 ], "text":"Sensors" }, { "indices":[ 76, 87 ], "text":"HealthCare" }, { "indices":[ 88, 104 ], "text":"makeyourownlane" }, { "indices":[ 105, 114 ], "text":"defstar5" } ], "user_mentions":[ ], "symbols":[ ] }, "display_text_range":[ 0, 140 ], "contributors":null, "user":{ "utc_offset":-25200, "friends_count":59115, "profile_image_url_https":"https:\/\/pbs.twimg.com \/profile_images\/729065804004769793\/St2_Pum9_normal.jpg", "listed_count":7863, "profile_background_image_url":"", "default_profile_image":false, "favourites_count":66194, "description":"Création de Sites Internet Formation et Conseil Informatique #ecommerce #webmaster #seo #DigitalMarketing #foad #seriousgame #AI #BigData", "created_at":"Thu Mar 03 23:46:02 +0000 2016", "is_translator":false, "profile_background_image_url_https":"", "protected":false, "screen_name":"ipfconline1", "id_str":"705539763349164032", "profile_link_color":"1DA1F2", "id":705539763349164000, "geo_enabled":false, "profile_background_color":"F5F8FA", "lang":"en", "profile_sidebar_border_color":"C0DEED", "profile_text_color":"333333", "verified":false, "profile_image_url":"http:\/\/pbs.twimg.com \/profile_images\/729065804004769793\/St2_Pum9_normal.jpg", "time_zone":"Pacific Time (US & Canada)", "url":"http:\/\/www.ipfconline.fr", "contributors_enabled":false, "profile_background_tile":false, "profile_banner_url":"https:\/\/pbs.twimg.com \/profile_banners\/705539763349164032\/1462657867", "statuses_count":75505, "follow_request_sent":null, "followers_count":60616, "profile_use_background_image":true, "default_profile":true, "following":null, "name":"ipfconline", "location":"Marseille, France", "profile_sidebar_fill_color":"DDEEF6", "notifications":null } }, "retweet_count":0, "retweeted":false, "geo":null, "filter_level":"low", "in_reply_to_screen_name":null, "is_quote_status":false, "id_str":"888643484852273153", "in_reply_to_user_id":null, "favorite_count":0, "id":888643484852273200, "text":"RT @ipfconline1: The Internet Of Things Changing Our Life\n#IoT #IoE #M2M #SmartCity #Sensors #HealthCare\n#makeyourownlane #defstar5\u2026 ", "place":null, "lang":"en", "favorited":false, "coordinates":null, "truncated":false, "timestamp_ms":"1500704091643", "entities":{ "urls":[ { "indices":[ 133, 133 ], "expanded_url":null, "url":"" } ], "hashtags":[ { "indices":[ 58, 62 ], "text":"IoT" }, { "indices":[ 63, 67 ], "text":"IoE" }, { "indices":[ 68, 72 ], "text":"M2M" }, { "indices":[ 73, 83 ], "text":"SmartCity" }, { "indices":[ 84, 92 ], "text":"Sensors" }, { "indices":[ 93, 104 ], "text":"HealthCare" }, { "indices":[ 105, 121 ], "text":"makeyourownlane" }, { "indices":[ 122, 131 ], "text":"defstar5" } ], "user_mentions":[ { "indices":[ 3, 15 ], "screen_name":"ipfconline1", "id_str":"705539763349164032", "name":"ipfconline", "id":705539763349164000 } ], "symbols":[ ] }, "contributors":null, "user":{ "utc_offset":-25200, "friends_count":103, "profile_image_url_https":"https:\/\/pbs.twimg.com \/profile_images\/870898833449865216\/kcVLpiK1_normal.jpg", "listed_count":5, "profile_background_image_url":"http:\/\/abs.twimg.com \/images\/themes\/theme1\/bg.png", "default_profile_image":false, "favourites_count":196, "description":"??Hello dear, my name is Prabh & about my hobbies i like learning of new web development languages. Moreover, I always like to gain & share new info.?", "created_at":"Sat Mar 21 14:30:57 +0000 2015", "is_translator":false, "profile_background_image_url_https":"https:\/\/abs.twimg.com \/images\/themes\/theme1\/bg.png", "protected":false, "screen_name":"techbhullar", "id_str":"3101659344", "profile_link_color":"7FDBB6", "id":3101659344, "geo_enabled":true, "profile_background_color":"000000", "lang":"en", "profile_sidebar_border_color":"000000", "profile_text_color":"000000", "verified":false, "profile_image_url":"http:\/\/pbs.twimg.com \/profile_images\/870898833449865216\/kcVLpiK1_normal.jpg", "time_zone":"Pacific Time (US & Canada)", "url":"https:\/\/www.peopleperhour.com \/freelancer\/prabhjot\/web-developer\/1685824", "contributors_enabled":false, "profile_background_tile":false, "profile_banner_url":"https:\/\/pbs.twimg.com \/profile_banners\/3101659344\/1498889350", "statuses_count":975, "follow_request_sent":null, "followers_count":141, "profile_use_background_image":false, "default_profile":false, "following":null, "name":"Tech Bhullar", "location":"Amritsar, India", "profile_sidebar_fill_color":"000000", "notifications":null } }, "lang":"en", "_msgid":"60cc03b2.19e0fc" } " } } }
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
