I suppose we are all looking at some sort of use case for recommendation engine, whether that is commodities or some Financial Instrument.
They all depend on some form of criteria. The commodity or instrument does not matter. It could be a new "Super Mario Wii" release or some shares that are "performing well". So if you are a big retailer shop you would be going and ordering 100s of the new game or if you are a portfolio manager would be seeing a good "buy" there. So I guess the design would be pretty similar. Still there will be a batch layer and speed layer and some presentation layer hooked to a faster database like Druid or Hbase. Cheers Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 5 September 2016 at 16:28, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > no problem. Got it Alonso > > Thanks > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 5 September 2016 at 15:59, Alonso Isidoro Roman <alons...@gmail.com> > wrote: > >> i would like to work, not love :) sorry >> >> Alonso Isidoro Roman >> [image: https://]about.me/alonso.isidoro.roman >> >> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >> >> 2016-09-05 16:58 GMT+02:00 Alonso Isidoro Roman <alons...@gmail.com>: >> >>> By the way, i would love to work in your project, looks promising! >>> >>> >>> >>> Alonso Isidoro Roman >>> [image: https://]about.me/alonso.isidoro.roman >>> >>> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >>> >>> 2016-09-05 16:57 GMT+02:00 Alonso Isidoro Roman <alons...@gmail.com>: >>> >>>> Hi Mitch, >>>> >>>> >>>> 1. What do you mean "my own rating" here? You know the products. So >>>> what Amazon is going to provide by way of Kafka? >>>> >>>> The idea was to embed the functionality of a kafka producer within a >>>> rest service in order i can invoke this logic with my a rating. I did not >>>> create such functionality because i started to make another things, i get >>>> bored, basically. I created some unix commands with this code, using >>>> sbt-pack. >>>> >>>> >>>> >>>> 1. Assuming that you have created topic specifically for this >>>> purpose then that topic is streamed into Kafka, some algorithms is >>>> applied >>>> and results are saved in DB >>>> >>>> you got it! >>>> >>>> 1. You have some dashboard that will fetch data (via ???) from the >>>> DB and I guess Zeppelin can do it here? >>>> >>>> >>>> No, not any dashboard yet. Maybe it is not a good idea to connect >>>> mongodb with a dashboard through a web socket. It probably works, for a >>>> proof of concept, but, in a real project? i don't know yet... >>>> >>>> You can see what i did to push data within a kafka topic in this scala >>>> class >>>> <https://github.com/alonsoir/awesome-recommendation-engine/blob/master/src/main/scala/example/producer/AmazonProducerExample.scala>, >>>> you have to invoke pack within the scala shell to create this unix command. >>>> >>>> Regards! >>>> >>>> Alonso >>>> >>>> >>>> >>>> Alonso Isidoro Roman >>>> [image: https://]about.me/alonso.isidoro.roman >>>> >>>> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >>>> >>>> 2016-09-05 16:39 GMT+02:00 Mich Talebzadeh <mich.talebza...@gmail.com>: >>>> >>>>> Thank you Alonso, >>>>> >>>>> I looked at your project. Interesting >>>>> >>>>> As I see it this is what you are suggesting >>>>> >>>>> >>>>> 1. A kafka producer is going to ask periodically to Amazon in >>>>> order to know what products based on my own ratings and i am going to >>>>> introduced them into some kafka topic. >>>>> 2. A spark streaming process is going to read from that previous >>>>> topic. >>>>> 3. Apply some machine learning algorithms (ALS, content based >>>>> filtering colaborative filtering) on those datasets readed by the spark >>>>> streaming process. >>>>> 4. Save results in a mongo or cassandra instance. >>>>> 5. Use play framework to create an websocket interface between the >>>>> mongo instance and the visual interface. >>>>> >>>>> >>>>> As I understand >>>>> >>>>> Point 1: A kafka producer is going to ask periodically to Amazon in >>>>> order to know what products based on my own ratings ..... >>>>> >>>>> >>>>> 1. What do you mean "my own rating" here? You know the products. >>>>> So what Amazon is going to provide by way of Kafka? >>>>> 2. Assuming that you have created topic specifically for this >>>>> purpose then that topic is streamed into Kafka, some algorithms is >>>>> applied >>>>> and results are saved in DB >>>>> 3. You have some dashboard that will fetch data (via ???) from the >>>>> DB and I guess Zeppelin can do it here? >>>>> >>>>> >>>>> Do you Have a DFD diagram for your design in case. Something like >>>>> below (hope does not look pedantic, non intended). >>>>> >>>>> [image: Inline images 1] >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Dr Mich Talebzadeh >>>>> >>>>> >>>>> >>>>> LinkedIn * >>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>>> >>>>> >>>>> >>>>> http://talebzadehmich.wordpress.com >>>>> >>>>> >>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>>> any loss, damage or destruction of data or any other property which may >>>>> arise from relying on this email's technical content is explicitly >>>>> disclaimed. The author will in no case be liable for any monetary damages >>>>> arising from such loss, damage or destruction. >>>>> >>>>> >>>>> >>>>> On 5 September 2016 at 15:08, Alonso Isidoro Roman <alons...@gmail.com >>>>> > wrote: >>>>> >>>>>> Hi Mitch, i wrote few months ago a tiny project with this issue in >>>>>> mind. The idea is to apply ALS algorithm in order to get some valid >>>>>> recommendations from another users. >>>>>> >>>>>> >>>>>> The url of the project >>>>>> <https://github.com/alonsoir/awesome-recommendation-engine> >>>>>> >>>>>> >>>>>> >>>>>> Alonso Isidoro Roman >>>>>> [image: https://]about.me/alonso.isidoro.roman >>>>>> >>>>>> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >>>>>> >>>>>> 2016-09-05 15:41 GMT+02:00 Mich Talebzadeh <mich.talebza...@gmail.com >>>>>> >: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Has anyone done any work on Real time recommendation engines with >>>>>>> Spark and Scala. >>>>>>> >>>>>>> I have seen few PPTs with Python but wanted to see if these have >>>>>>> been done with Scala. >>>>>>> >>>>>>> I trust this question makes sense. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> p.s. My prime interest would be in Financial markets. >>>>>>> >>>>>>> >>>>>>> Dr Mich Talebzadeh >>>>>>> >>>>>>> >>>>>>> >>>>>>> LinkedIn * >>>>>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://talebzadehmich.wordpress.com >>>>>>> >>>>>>> >>>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility >>>>>>> for any loss, damage or destruction of data or any other property which >>>>>>> may >>>>>>> arise from relying on this email's technical content is explicitly >>>>>>> disclaimed. The author will in no case be liable for any monetary >>>>>>> damages >>>>>>> arising from such loss, damage or destruction. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
