There is no need to have Universal Recommender models put in S3, they are not used and only exist (in stub form) because PIO requires them. The actual model lives in Elasticsearch and uses special features of ES to perform the last phase of the algorithm and so cannot be replaced.
The stub PIO models have no data and will be tiny. putting them in HDFS or the local file system is recommended. From: Dave Novelli <[email protected]> <[email protected]> Reply: [email protected] <[email protected]> <[email protected]> Date: March 22, 2018 at 6:17:32 PM To: [email protected] <[email protected]> <[email protected]> Subject: Unclear problem with using S3 as a storage data source Hi all, I'm using the Universal Recommender template and I'm trying to switch storage data sources from local file to S3 for the model repository. I've read the page at https://predictionio.apache.org/system/anotherdatastore/ to try to understand the configuration requirements, but when I run pio train it's indicating an error and nothing shows up in the s3 bucket: [ERROR] [S3Models] Failed to insert a model to s3://pio-model/pio_modelAWJPjTYM0wNJe2iKBl0d I created a new bucket named "pio-model" and granted full public permissions. Seemingly relevant settings from pio-env.sh: PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=S3 ... PIO_STORAGE_SOURCES_S3_TYPE=s3 PIO_STORAGE_SOURCES_S3_REGION=us-west-2 PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio-model # I've tried with and without this #PIO_STORAGE_SOURCES_S3_ENDPOINT=http://s3.us-west-2.amazonaws.com # I've tried with and without this #PIO_STORAGE_SOURCES_S3_BASE_PATH=pio-model Any suggestions where I can start troubleshooting my configuration? Thanks, Dave
