PredictionIO is scalable BY SCALING ITS SUB-SERVICES. Running on a single machine sounds like no scaling has been executed or even planned.
How do you scale ANY system? 1) vertical scaling: make the instance larger with more cores, more disk, and most importantly more memory. Increase whatever resource you need most but all will be affected eventually. 2) move each service to its own instance. Move the DB, Spark, etc (depends on what you are using) Then you can scale the sub-service (the ones PIO uses) independently as needed. Without a scaling plan you must trim your data to fit the system you have. For instance save only a few months of data. Unfortunately PIO has no automatic way to do this, like a TTL. We created a template that you can run to trim your db by dropping old data. Unfortunately we have not kept up with PIO versions since we have moved to another ML server that DOES have TTLs. If anyone wants to upgrade the template it was last used with PIO 0.12.x and is here: https://github.com/actionml/db-cleaner If you continually add data to a bucket it will eventually overflow, how could it be any other way? From: Sami Serbey <sami.ser...@designer-24.com> <sami.ser...@designer-24.com> Reply: user@predictionio.apache.org <user@predictionio.apache.org> <user@predictionio.apache.org> Date: March 19, 2020 at 7:43:08 AM To: user@predictionio.apache.org <user@predictionio.apache.org> <user@predictionio.apache.org> Subject: Re: PredictionIO ASF Board Report for Mar 2020 Hello! My knowledge to predictionio is limited. I was able to set up a predictionIO server and run on it two templates, the recommendation and similar item template. The server is on production in my company and we were having good results. Suddenly, as we feed data to the server, our cloud machine memory got full and we can't have new data anymore nor we can process this data. An error message on ubuntu state: "No space left on device". I am deploying this server on a single machine without any cluster or the help of docker. Do you have any suggestion to solve this issue? Also, is there a way to clean the machine from old data it has? As a final note, my knowledge in the data engineer and machine learning field is limited. I understand scala and can work with spark. However, I am willing to dig deeper into predictionio. Do you think there is a way I can contribute to the community in one way or another? Or you're just looking for true experts in order to avoid moving the project to attic? Regards Sami Serbey ------------------------------ *From:* Donald Szeto <don...@apache.org> *Sent:* Tuesday, March 10, 2020 8:26 PM *To:* user@predictionio.apache.org <user@predictionio.apache.org>; d...@predictionio.apache.org <d...@predictionio.apache.org> *Subject:* PredictionIO ASF Board Report for Mar 2020 Hi all, Please take a look at the draft report below and make your comments or edits as you see fit. The draft will be submitted on Mar 11, 2020. Regards, Donald ## Description: The mission of Apache Predictionio is the creation and maintenance of software related to a machine learning server built on top of state-of-the-art open source stack, that enables developers to manage and deploy production-ready predictive services for various kinds of machine learning tasks ## Issues: Update: A community member, who's a committer and PMC of another Apache project, has expressed interest in helping. The member has been engaged and we are waiting for actions from that member. Last report: No PMC chair nominee was nominated a week after the PMC chair expressed intention to resign from the chair on the PMC mailing list. ## Membership Data: Apache PredictionIO was founded 2017-10-17 (2 years ago) There are currently 29 committers and 28 PMC members in this project. The Committer-to-PMC ratio is roughly 8:7. Community changes, past quarter: - No new PMC members. Last addition was Andrew Kyle Purtell on 2017-10-17. - No new committers were added. ## Project Activity: Sparse activities only on mailing list. Recent releases: 0.14.0 was released on 2019-03-11. 0.13.0 was released on 2018-09-20. 0.12.1 was released on 2018-03-11. ## Community Health: Update: A community member, who's a committer and PMC of another Apache project, has expressed interest in helping. The member has been engaged and we are waiting for actions from that member to see if a nomination to PMC and chair would be appropriate. Last report: We are seeking new leadership for the project at the moment to bring it out of maintenance mode. Moving to the attic would be the last option.
