Thank you for your guidance. We have to build and train on development machine which can be standalone or 3 node cluster and deploy on production environment which is completely different cluster. In this case does scratch-uri will work or we have to follow another process.
Please suggest me. Thank you Regards, Bansari On Wed, Oct 5, 2016 at 1:30 AM, Donald Szeto <[email protected]> wrote: > Hi Bansari, > > The --scratch-uri switch is only useful with "pio train/deploy" using YARN > cluster mode, which is your case. It tells PredictionIO where to copy > PredictionIO JARs and engine.json for YARN cluster mode to work properly. > > 1. Make sure HADOOP_CONF_DIR is set properly in conf/pio-env.sh. > 2. Provide an HDFS URL to --scratch-uri. You need to have write access to > this location. > > Regards, > Donald > > On Tue, Oct 4, 2016 at 11:21 AM, Pat Ferrel <[email protected]> wrote: > >> No idea about 'scratch-uri’ but once you build a model if you have >> specified (in pio-env.sh) that pio use hdfs for the model storage it will >> already be available to any machine that has access to hdfs. It somewhat >> depends on the template, the Universal Recommender uses Elasticsearch for >> model storage so any machine with access to ES will have the model. >> >> >> On Oct 4, 2016, at 10:09 AM, Bansari Shah <[email protected]> >> wrote: >> >> Hi Donald, >> >> I am running my spark cluster of 3 node with YARN and spark driver is >> within cluster. >> >> Thanks >> Regards, >> Bansari >> >> On Tue, Oct 4, 2016 at 9:59 PM, Donald Szeto <[email protected]> wrote: >> >>> Hi Bansari, >>> >>> How are you running your Spark cluster? Standalone, YARN, or Mesos? Are >>> you running the Spark driver on the client or within the cluster? >>> >>> Regards, >>> Donald >>> >>> On Tue, Oct 4, 2016 at 5:55 AM, Bansari Shah <[email protected]> >>> wrote: >>> >>>> Hi, >>>> Can you please guide me how to use 'scratch-uri' argument in case of >>>> transferring all necessary files to remote location. >>>> >>>> And can you please suggest me any way for deploying model on remote >>>> location which is trained and build on other machine. >>>> >>>> Please consider it. >>>> >>>> Thank you, >>>> >>>> Regards, >>>> Bansari >>>> >>> >>> >> >> >
