Thanks Mars and Donald. I think this gets me to next steps:
- Clone PredictionIO 0.12 and adjust the bin/compute-classpath.sh to
have aws-java-sdk-1.7.4 loaded first.
- Create custom binary distribution of PredicionIO 0.12.
- Add config var to point to custom binary distribution.
This is very helpful. Thank you!
*Shane Johnson | LIFT IQ*
*Founder | CEO*
*www.liftiq.com <http://www.liftiq.com/>* or *sh...@liftiq.com
mobile: (801) 360-3350
LinkedIn <https://www.linkedin.com/in/shanewjohnson/> | Twitter
<https://twitter.com/SWaldenJ> | Facebook
On Tue, Mar 6, 2018 at 4:52 PM, Mars Hall <mars.h...@salesforce.com> wrote:
> On Tue, Mar 6, 2018 at 11:39 PM, Shane Johnson <sh...@liftiq.com> wrote:
>> Do you know the version of hadoop-aws.jar and aws-java-sdk.jar that you
>> are using?
>> I do not know what version is being used. Is this something that I can
>> specify or control? I am using the PredictionIO buildpack
>> https://github.com/heroku/predictionio-buildpack. I am *not* specifying
>> these in my build.sbt currently.
> The versions are specified in the "bin/common/setup-runtime" script of the
> Currently they are:
> - hadoop-aws-2.7.3
> - aws-java-sdk-1.7.4
> If you fork the buildpack, those download URLs (currently hosted in my S3
> bucket) can be changed.
>> You are also right that you can modify the class path in
>> bin/compute-classpath.sh as a short term fix. The current order is
>> following the output of your target system's `ls`, so the order is not
>> guaranteed like you speak. Right before the last line (echo $CLASSPATH),
>> you can add a line to make sure that the JAR you want to be loaded first is
>> at the very beginning.
>> I believe this would need to be something that is edited on master (
>> https://github.com/apache/predictionio) as the buildpack leverages it
>> vs. a cloned version of the code that I can edit, am I thinking through
>> that correctly? I may need to circle back with Mars to see if there are any
>> other options to get this to work with Heroku. Is this something that can
>> be committed to master? The aws-java-sdk.jar is the one that we need to
>> load first.
> If you build your own custom binary distribution of PredictionIO 0.12 and
> upload it to a publicly accessible URL, then the buildpack can be
> configured to use it by setting config var `PREDICTIONIO_DIST_URL` which is
> used by the same setup script as above:
> This is how I tested my pre-release builds on Heroku.
> *Mars Hall
> 415-818-7039 <(415)%20818-7039>
> Customer Facing Architect
> Salesforce Platform / Heroku
> San Francisco, California