RE: Heatmap with Spark Streaming
Umesh, You can create a web-service in any of the languages supported by Spark and stream the result from this web-service to your D3-based client using Websocket or Server-Sent Events. For example, you can create a webservice using Play. This app will integrate with Spark streaming in the back-end. The front-end can be a D3-based or any Javascript app. Play makes it easy to stream data to a web client. So a client does not need to continuously poll data from the back-end server. Mohammed From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Thursday, July 30, 2015 2:07 AM To: UMESH CHAUDHARY Cc: user@spark.apache.org Subject: Re: Heatmap with Spark Streaming You can integrate it with any language (like php) and use ajax calls to update the charts. Thanks Best Regards On Thu, Jul 30, 2015 at 2:11 PM, UMESH CHAUDHARY umesh9...@gmail.commailto:umesh9...@gmail.com wrote: Thanks For the suggestion Akhil! I looked at https://github.com/mbostock/d3/wiki/Gallery to know more about d3, all examples described here are on static data, how we can update our heat map from updated data, if we store it in Hbase or Mysql. I mean, do we need to query back and fourth for it. Is there any pluggable and more quick component for heatmap with spark streaming? On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das ak...@sigmoidanalytics.commailto:ak...@sigmoidanalytics.com wrote: You can easily push data to an intermediate storage from spark streaming (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3 js. Thanks Best Regards On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY umesh9...@gmail.commailto:umesh9...@gmail.com wrote: I have just started using Spark Streaming and done few POCs. It is fairly easy to implement. I was thinking of presenting the data using some smart graphing dashboarding tools e.g. Graphite or Grafna, but they don't have heat-maps. I also looked at Zeppelinhttp://zeppelin-project.org/ , but unable to found any heat-map functionality. Could you please suggest any data visualization tools using Heat-map and Spark streaming.
Re: Heatmap with Spark Streaming
I do suggest that the non-spark related discussions be taken to a different this forum as it does not directly contribute to the contents of this user list. On Thu, Jul 30, 2015 at 8:52 PM, UMESH CHAUDHARY umesh9...@gmail.com wrote: Thanks for the valuable suggestion. I also started with JAX-RS restful service with Angular. Since play can help a lot in my scenario, I would prefer it along with Angular. Is this combination fine over d3?
Re: Heatmap with Spark Streaming
Thanks For the suggestion Akhil! I looked at https://github.com/mbostock/d3/wiki/Gallery to know more about d3, all examples described here are on static data, how we can update our heat map from updated data, if we store it in Hbase or Mysql. I mean, do we need to query back and fourth for it. Is there any pluggable and more quick component for heatmap with spark streaming? On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das ak...@sigmoidanalytics.com wrote: You can easily push data to an intermediate storage from spark streaming (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3 js. Thanks Best Regards On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY umesh9...@gmail.com wrote: I have just started using Spark Streaming and done few POCs. It is fairly easy to implement. I was thinking of presenting the data using some smart graphing dashboarding tools e.g. Graphite or Grafna, but they don't have heat-maps. I also looked at Zeppelin http://zeppelin-project.org/ , but unable to found any heat-map functionality. Could you please suggest any data visualization tools using Heat-map and Spark streaming.
Re: Heatmap with Spark Streaming
You can easily push data to an intermediate storage from spark streaming (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3 js. Thanks Best Regards On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY umesh9...@gmail.com wrote: I have just started using Spark Streaming and done few POCs. It is fairly easy to implement. I was thinking of presenting the data using some smart graphing dashboarding tools e.g. Graphite or Grafna, but they don't have heat-maps. I also looked at Zeppelin http://zeppelin-project.org/ , but unable to found any heat-map functionality. Could you please suggest any data visualization tools using Heat-map and Spark streaming.
Re: Heatmap with Spark Streaming
You can integrate it with any language (like php) and use ajax calls to update the charts. Thanks Best Regards On Thu, Jul 30, 2015 at 2:11 PM, UMESH CHAUDHARY umesh9...@gmail.com wrote: Thanks For the suggestion Akhil! I looked at https://github.com/mbostock/d3/wiki/Gallery to know more about d3, all examples described here are on static data, how we can update our heat map from updated data, if we store it in Hbase or Mysql. I mean, do we need to query back and fourth for it. Is there any pluggable and more quick component for heatmap with spark streaming? On Thu, Jul 30, 2015 at 1:23 PM, Akhil Das ak...@sigmoidanalytics.com wrote: You can easily push data to an intermediate storage from spark streaming (like HBase or a SQL/NoSQL DB etc) and then power your dashboards with d3 js. Thanks Best Regards On Tue, Jul 28, 2015 at 12:18 PM, UMESH CHAUDHARY umesh9...@gmail.com wrote: I have just started using Spark Streaming and done few POCs. It is fairly easy to implement. I was thinking of presenting the data using some smart graphing dashboarding tools e.g. Graphite or Grafna, but they don't have heat-maps. I also looked at Zeppelin http://zeppelin-project.org/ , but unable to found any heat-map functionality. Could you please suggest any data visualization tools using Heat-map and Spark streaming.
