Hi All, Following the announcement of the latest HDP Hadoop bundle by LazyPower and Amir, I built a demo to run a recommendation engine on top of it.
You can find the code here : https://github.com/SaMnCo/juju-hadoop-recommendation-engine Notes: * The script will install Mahout and Snappy compression lib. This will in the end be included in the charm. * It has to run once to learn the recommendation from the source file * The output is a big matrix and there is a small python script provided to extract meaningful results * Wait until the YARN resource manager is started before installing and running * Tested on AWS only * The initial workload uses 100k movie recommendations but there are 1M and 10M files available (see config in the demo script) * The inspiration for this was built for Hadoop Vanilla. Most things can be ported to that environment as well. If anyone is willing to help on a small web GUI, that could make a nice demo for our beloved SEs. Hope you like it! Best, Sam -- Samuel Cozannet Cloud, Big Data and IoT Strategy Team Strategic Program Manager Changing the Future of Cloud Ubuntu <http://ubuntu.com> / Canonical <http://canonical.com> UK LTD [email protected] +33 616 702 389
-- Juju mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
