Re: PIO-20 problems

[Pat Ferrel]

The first set of requirement fail to install at python’s unittest. That sort-of 
stops me dead. Don’t have a lot of dedicated time right now to debug so I let 
Travis do the tests and they are passing now and all comments are resolved. I 
left the keystore in so we ca address in a separate Jira. This PR will be 
pushed this afternoon, at which time the fork will completely “healed”.


On Aug 13, 2016, at 1:41 PM, Donald Szeto  wrote:

Hey Pat, how are you running integration tests? All requirements are already 
inside the Docker image and should not need to be installed manually. Are you 
trying to rebuild the image from scratch? Is there somewhere in the integration 
test documentation that can be improved?

On Sat, Aug 13, 2016 at 12:31 PM, Alex Merritt > wrote:
Resolved, passing in Travis now.


On Fri, Aug 12, 2016 at 3:04 PM, Pat Ferrel > wrote:
Alex, can you look at these unit test failures on the PR, they seem to be in 
JDBCPEvents

https://travis-ci.org/apache/incubator-predictionio/builds/151905196 



On Aug 12, 2016, at 1:18 PM, Pat Ferrel > wrote:

Can't install unittest with pip or pip3 even though the rest of the 
prerequisites work. Simply refuses to find it. Pip3 search unittest give this:

WebTestRunner (0.2)- Web-based interface for selectively 
executing client-side Python UnitTests
unittest (0.0) -
nosetests-json-extended (0.1.0)- Create json logging output for 
pythonnosetests unittest framework

no description and 0.0 seems odd. Pip3 install unittest gives:

Collecting unittest
 Could not find a version that satisfies the requirement unittest (from 
versions: )
No matching distribution found for unittest

Tried forcing the version to 0.0 but again no luck

Ideas? In the meantime waiting for Travis to do it—sigh


On Aug 11, 2016, at 2:31 PM, Pat Ferrel > wrote:

With the keystore my template-based integration test passes and I’ve put the 
keystore back in. The diffs on PRs on Github seem completely wonky right now. 
Git diff is trustworthy at least.

Will try the python tests now too.

Thanks guys, working smoothly now.


On Aug 11, 2016, at 11:53 AM, Donald Szeto > wrote:

Hi all,

I went ahead and pulled Pat's branch, performed a clean build, and repeated
the quick start guide of the Scala parallel recommendation template. I
could produce the same problem, and root caused it to a missing
conf/keystore.jks file.

I think with SSL now optional, we should not be distributing a KeyStore
file. We can either quick fix it now by putting this file back, or modify
SSLConfiguration to not look for this file when SSL support is off.

Regards,
Donald

On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee > wrote:

> I want to clarify some points:
>
> 2) Deploying templates do not require SSL. If you execute
> ./make-distribution.sh using the current develop branch (provided you
> change the namespace in the template from io.prediction to
> org.apache.predictionio), you can deploy without SSL on localhost. The
> travis tests also do not use SSL and pass as can be seen in
> https://travis-ci.org/Ziemin/incubator-predictionio 
> .
>
> 3) You can run the tests locally the same way it is run on travis
> ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> README in the same directory, but maybe this was not clear enough. If you
> have any additional questions or run into issues executing the tests,
> please let me know.
>
> Thanks,
> Chan
>
> On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  > wrote:
>
>> I’m always disinclined to commit code that has not been tested. However
>> many things in the current state of the develop branch seem broken or at
>> least too big to address in a single commit.
>>
>> 1) Style test fail in Travis that are not runnable or at least have not
>> been documented for local build. I have fixed these.
>> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
>> have addressed this. Has anyone tried to build and test a template yet
>> without SSL? Unless someone can state otherwise I’m inclined to ignore this
>> failure for now since it is out of scope for this commit.
>> 3) the test framework has been integrated into Travis but again no
>> documented way to run it locally and some of the travis errors seem
>> spurious. I am also not inclined to wait for this to be fixed unless
>> someone can point out real problems it is discovering and send instructions
>> for how to run it locally.
>>
>> 

Re: PIO-20 problems

[Alex Merritt]

Resolved, passing in Travis now.

On Fri, Aug 12, 2016 at 3:04 PM, Pat Ferrel  wrote:

> Alex, can you look at these unit test failures on the PR, they seem to be
> in JDBCPEvents
>
> https://travis-ci.org/apache/incubator-predictionio/builds/151905196
>
>
> On Aug 12, 2016, at 1:18 PM, Pat Ferrel  wrote:
>
> Can't install unittest with pip or pip3 even though the rest of the
> prerequisites work. Simply refuses to find it. Pip3 search unittest give
> this:
>
> WebTestRunner (0.2)- Web-based interface for selectively
> executing client-side Python UnitTests
> unittest (0.0) -
> nosetests-json-extended (0.1.0)- Create json logging output for
> pythonnosetests unittest framework
>
> no description and 0.0 seems odd. Pip3 install unittest gives:
>
> Collecting unittest
>  Could not find a version that satisfies the requirement unittest (from
> versions: )
> No matching distribution found for unittest
>
> Tried forcing the version to 0.0 but again no luck
>
> Ideas? In the meantime waiting for Travis to do it—sigh
>
>
> On Aug 11, 2016, at 2:31 PM, Pat Ferrel  wrote:
>
> With the keystore my template-based integration test passes and I’ve put
> the keystore back in. The diffs on PRs on Github seem completely wonky
> right now. Git diff is trustworthy at least.
>
> Will try the python tests now too.
>
> Thanks guys, working smoothly now.
>
>
> On Aug 11, 2016, at 11:53 AM, Donald Szeto  wrote:
>
> Hi all,
>
> I went ahead and pulled Pat's branch, performed a clean build, and repeated
> the quick start guide of the Scala parallel recommendation template. I
> could produce the same problem, and root caused it to a missing
> conf/keystore.jks file.
>
> I think with SSL now optional, we should not be distributing a KeyStore
> file. We can either quick fix it now by putting this file back, or modify
> SSLConfiguration to not look for this file when SSL support is off.
>
> Regards,
> Donald
>
> On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee  wrote:
>
> > I want to clarify some points:
> >
> > 2) Deploying templates do not require SSL. If you execute
> > ./make-distribution.sh using the current develop branch (provided you
> > change the namespace in the template from io.prediction to
> > org.apache.predictionio), you can deploy without SSL on localhost. The
> > travis tests also do not use SSL and pass as can be seen in
> > https://travis-ci.org/Ziemin/incubator-predictionio.
> >
> > 3) You can run the tests locally the same way it is run on travis
> > ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> > README in the same directory, but maybe this was not clear enough. If you
> > have any additional questions or run into issues executing the tests,
> > please let me know.
> >
> > Thanks,
> > Chan
> >
> > On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel 
> wrote:
> >
> >> I’m always disinclined to commit code that has not been tested. However
> >> many things in the current state of the develop branch seem broken or at
> >> least too big to address in a single commit.
> >>
> >> 1) Style test fail in Travis that are not runnable or at least have not
> >> been documented for local build. I have fixed these.
> >> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
> >> have addressed this. Has anyone tried to build and test a template yet
> >> without SSL? Unless someone can state otherwise I’m inclined to ignore
> this
> >> failure for now since it is out of scope for this commit.
> >> 3) the test framework has been integrated into Travis but again no
> >> documented way to run it locally and some of the travis errors seem
> >> spurious. I am also not inclined to wait for this to be fixed unless
> >> someone can point out real problems it is discovering and send
> instructions
> >> for how to run it locally.
> >>
> >> Before the extra travis tests I was able to build and train a template,
> >> which fails on deploy with an SSL config error. Since the PR does not
> >> strictly touch any templates I am going to have to ignore this serious
> >> issue and address it in Jira.
> >>
> >> Unless someone vetos this I plan to reluctantly push the PR with these
> >> non-trivial issues remaining.
> >>
> >>
> >>
> >
>
>
>
>

Re: PIO-20 problems

[Pat Ferrel]

Alex, can you look at these unit test failures on the PR, they seem to be in 
JDBCPEvents 

https://travis-ci.org/apache/incubator-predictionio/builds/151905196


On Aug 12, 2016, at 1:18 PM, Pat Ferrel  wrote:

Can't install unittest with pip or pip3 even though the rest of the 
prerequisites work. Simply refuses to find it. Pip3 search unittest give this:

WebTestRunner (0.2)- Web-based interface for selectively 
executing client-side Python UnitTests
unittest (0.0) - 
nosetests-json-extended (0.1.0)- Create json logging output for 
pythonnosetests unittest framework

no description and 0.0 seems odd. Pip3 install unittest gives: 

Collecting unittest
 Could not find a version that satisfies the requirement unittest (from 
versions: )
No matching distribution found for unittest

Tried forcing the version to 0.0 but again no luck

Ideas? In the meantime waiting for Travis to do it—sigh


On Aug 11, 2016, at 2:31 PM, Pat Ferrel  wrote:

With the keystore my template-based integration test passes and I’ve put the 
keystore back in. The diffs on PRs on Github seem completely wonky right now. 
Git diff is trustworthy at least.

Will try the python tests now too.

Thanks guys, working smoothly now.


On Aug 11, 2016, at 11:53 AM, Donald Szeto  wrote:

Hi all,

I went ahead and pulled Pat's branch, performed a clean build, and repeated
the quick start guide of the Scala parallel recommendation template. I
could produce the same problem, and root caused it to a missing
conf/keystore.jks file.

I think with SSL now optional, we should not be distributing a KeyStore
file. We can either quick fix it now by putting this file back, or modify
SSLConfiguration to not look for this file when SSL support is off.

Regards,
Donald

On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee  wrote:

> I want to clarify some points:
> 
> 2) Deploying templates do not require SSL. If you execute
> ./make-distribution.sh using the current develop branch (provided you
> change the namespace in the template from io.prediction to
> org.apache.predictionio), you can deploy without SSL on localhost. The
> travis tests also do not use SSL and pass as can be seen in
> https://travis-ci.org/Ziemin/incubator-predictionio.
> 
> 3) You can run the tests locally the same way it is run on travis
> ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> README in the same directory, but maybe this was not clear enough. If you
> have any additional questions or run into issues executing the tests,
> please let me know.
> 
> Thanks,
> Chan
> 
> On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:
> 
>> I’m always disinclined to commit code that has not been tested. However
>> many things in the current state of the develop branch seem broken or at
>> least too big to address in a single commit.
>> 
>> 1) Style test fail in Travis that are not runnable or at least have not
>> been documented for local build. I have fixed these.
>> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
>> have addressed this. Has anyone tried to build and test a template yet
>> without SSL? Unless someone can state otherwise I’m inclined to ignore this
>> failure for now since it is out of scope for this commit.
>> 3) the test framework has been integrated into Travis but again no
>> documented way to run it locally and some of the travis errors seem
>> spurious. I am also not inclined to wait for this to be fixed unless
>> someone can point out real problems it is discovering and send instructions
>> for how to run it locally.
>> 
>> Before the extra travis tests I was able to build and train a template,
>> which fails on deploy with an SSL config error. Since the PR does not
>> strictly touch any templates I am going to have to ignore this serious
>> issue and address it in Jira.
>> 
>> Unless someone vetos this I plan to reluctantly push the PR with these
>> non-trivial issues remaining.
>> 
>> 
>> 
> 

Re: PIO-20 problems

[Pat Ferrel]

Can't install unittest with pip or pip3 even though the rest of the 
prerequisites work. Simply refuses to find it. Pip3 search unittest give this:

WebTestRunner (0.2)- Web-based interface for selectively 
executing client-side Python UnitTests
unittest (0.0) - 
nosetests-json-extended (0.1.0)- Create json logging output for 
pythonnosetests unittest framework

no description and 0.0 seems odd. Pip3 install unittest gives: 

Collecting unittest
  Could not find a version that satisfies the requirement unittest (from 
versions: )
No matching distribution found for unittest

Tried forcing the version to 0.0 but again no luck

Ideas? In the meantime waiting for Travis to do it—sigh


On Aug 11, 2016, at 2:31 PM, Pat Ferrel  wrote:

With the keystore my template-based integration test passes and I’ve put the 
keystore back in. The diffs on PRs on Github seem completely wonky right now. 
Git diff is trustworthy at least.

Will try the python tests now too.

Thanks guys, working smoothly now.


On Aug 11, 2016, at 11:53 AM, Donald Szeto  wrote:

Hi all,

I went ahead and pulled Pat's branch, performed a clean build, and repeated
the quick start guide of the Scala parallel recommendation template. I
could produce the same problem, and root caused it to a missing
conf/keystore.jks file.

I think with SSL now optional, we should not be distributing a KeyStore
file. We can either quick fix it now by putting this file back, or modify
SSLConfiguration to not look for this file when SSL support is off.

Regards,
Donald

On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee  wrote:

> I want to clarify some points:
> 
> 2) Deploying templates do not require SSL. If you execute
> ./make-distribution.sh using the current develop branch (provided you
> change the namespace in the template from io.prediction to
> org.apache.predictionio), you can deploy without SSL on localhost. The
> travis tests also do not use SSL and pass as can be seen in
> https://travis-ci.org/Ziemin/incubator-predictionio.
> 
> 3) You can run the tests locally the same way it is run on travis
> ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> README in the same directory, but maybe this was not clear enough. If you
> have any additional questions or run into issues executing the tests,
> please let me know.
> 
> Thanks,
> Chan
> 
> On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:
> 
>> I’m always disinclined to commit code that has not been tested. However
>> many things in the current state of the develop branch seem broken or at
>> least too big to address in a single commit.
>> 
>> 1) Style test fail in Travis that are not runnable or at least have not
>> been documented for local build. I have fixed these.
>> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
>> have addressed this. Has anyone tried to build and test a template yet
>> without SSL? Unless someone can state otherwise I’m inclined to ignore this
>> failure for now since it is out of scope for this commit.
>> 3) the test framework has been integrated into Travis but again no
>> documented way to run it locally and some of the travis errors seem
>> spurious. I am also not inclined to wait for this to be fixed unless
>> someone can point out real problems it is discovering and send instructions
>> for how to run it locally.
>> 
>> Before the extra travis tests I was able to build and train a template,
>> which fails on deploy with an SSL config error. Since the PR does not
>> strictly touch any templates I am going to have to ignore this serious
>> issue and address it in Jira.
>> 
>> Unless someone vetos this I plan to reluctantly push the PR with these
>> non-trivial issues remaining.
>> 
>> 
>> 
> 

Re: PIO-20 problems

[Pat Ferrel]

With the keystore my template-based integration test passes and I’ve put the 
keystore back in. The diffs on PRs on Github seem completely wonky right now. 
Git diff is trustworthy at least.

Will try the python tests now too.

Thanks guys, working smoothly now.


On Aug 11, 2016, at 11:53 AM, Donald Szeto  wrote:

Hi all,

I went ahead and pulled Pat's branch, performed a clean build, and repeated
the quick start guide of the Scala parallel recommendation template. I
could produce the same problem, and root caused it to a missing
conf/keystore.jks file.

I think with SSL now optional, we should not be distributing a KeyStore
file. We can either quick fix it now by putting this file back, or modify
SSLConfiguration to not look for this file when SSL support is off.

Regards,
Donald

On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee  wrote:

> I want to clarify some points:
> 
> 2) Deploying templates do not require SSL. If you execute
> ./make-distribution.sh using the current develop branch (provided you
> change the namespace in the template from io.prediction to
> org.apache.predictionio), you can deploy without SSL on localhost. The
> travis tests also do not use SSL and pass as can be seen in
> https://travis-ci.org/Ziemin/incubator-predictionio.
> 
> 3) You can run the tests locally the same way it is run on travis
> ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> README in the same directory, but maybe this was not clear enough. If you
> have any additional questions or run into issues executing the tests,
> please let me know.
> 
> Thanks,
> Chan
> 
> On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:
> 
>> I’m always disinclined to commit code that has not been tested. However
>> many things in the current state of the develop branch seem broken or at
>> least too big to address in a single commit.
>> 
>> 1) Style test fail in Travis that are not runnable or at least have not
>> been documented for local build. I have fixed these.
>> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
>> have addressed this. Has anyone tried to build and test a template yet
>> without SSL? Unless someone can state otherwise I’m inclined to ignore this
>> failure for now since it is out of scope for this commit.
>> 3) the test framework has been integrated into Travis but again no
>> documented way to run it locally and some of the travis errors seem
>> spurious. I am also not inclined to wait for this to be fixed unless
>> someone can point out real problems it is discovering and send instructions
>> for how to run it locally.
>> 
>> Before the extra travis tests I was able to build and train a template,
>> which fails on deploy with an SSL config error. Since the PR does not
>> strictly touch any templates I am going to have to ignore this serious
>> issue and address it in Jira.
>> 
>> Unless someone vetos this I plan to reluctantly push the PR with these
>> non-trivial issues remaining.
>> 
>> 
>> 
> 

Re: PIO-20 problems

[Donald Szeto]

Hi all,

I went ahead and pulled Pat's branch, performed a clean build, and repeated
the quick start guide of the Scala parallel recommendation template. I
could produce the same problem, and root caused it to a missing
conf/keystore.jks file.

I think with SSL now optional, we should not be distributing a KeyStore
file. We can either quick fix it now by putting this file back, or modify
SSLConfiguration to not look for this file when SSL support is off.

Regards,
Donald

On Thu, Aug 11, 2016 at 8:03 AM, Chan Lee  wrote:

> I want to clarify some points:
>
> 2) Deploying templates do not require SSL. If you execute
> ./make-distribution.sh using the current develop branch (provided you
> change the namespace in the template from io.prediction to
> org.apache.predictionio), you can deploy without SSL on localhost. The
> travis tests also do not use SSL and pass as can be seen in
> https://travis-ci.org/Ziemin/incubator-predictionio.
>
> 3) You can run the tests locally the same way it is run on travis
> ('python3 ${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the
> README in the same directory, but maybe this was not clear enough. If you
> have any additional questions or run into issues executing the tests,
> please let me know.
>
> Thanks,
> Chan
>
> On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:
>
>> I’m always disinclined to commit code that has not been tested. However
>> many things in the current state of the develop branch seem broken or at
>> least too big to address in a single commit.
>>
>> 1) Style test fail in Travis that are not runnable or at least have not
>> been documented for local build. I have fixed these.
>> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
>> have addressed this. Has anyone tried to build and test a template yet
>> without SSL? Unless someone can state otherwise I’m inclined to ignore this
>> failure for now since it is out of scope for this commit.
>> 3) the test framework has been integrated into Travis but again no
>> documented way to run it locally and some of the travis errors seem
>> spurious. I am also not inclined to wait for this to be fixed unless
>> someone can point out real problems it is discovering and send instructions
>> for how to run it locally.
>>
>> Before the extra travis tests I was able to build and train a template,
>> which fails on deploy with an SSL config error. Since the PR does not
>> strictly touch any templates I am going to have to ignore this serious
>> issue and address it in Jira.
>>
>> Unless someone vetos this I plan to reluctantly push the PR with these
>> non-trivial issues remaining.
>>
>>
>>
>

Re: PIO-20 problems

[Chan Lee]

I want to clarify some points:

2) Deploying templates do not require SSL. If you execute
./make-distribution.sh using the current develop branch (provided you
change the namespace in the template from io.prediction to
org.apache.predictionio), you can deploy without SSL on localhost. The
travis tests also do not use SSL and pass as can be seen in
https://travis-ci.org/Ziemin/incubator-predictionio.

3) You can run the tests locally the same way it is run on travis ('python3
${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the README in
the same directory, but maybe this was not clear enough. If you have any
additional questions or run into issues executing the tests, please let me
know.

Thanks,
Chan

On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:

> I’m always disinclined to commit code that has not been tested. However
> many things in the current state of the develop branch seem broken or at
> least too big to address in a single commit.
>
> 1) Style test fail in Travis that are not runnable or at least have not
> been documented for local build. I have fixed these.
> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
> have addressed this. Has anyone tried to build and test a template yet
> without SSL? Unless someone can state otherwise I’m inclined to ignore this
> failure for now since it is out of scope for this commit.
> 3) the test framework has been integrated into Travis but again no
> documented way to run it locally and some of the travis errors seem
> spurious. I am also not inclined to wait for this to be fixed unless
> someone can point out real problems it is discovering and send instructions
> for how to run it locally.
>
> Before the extra travis tests I was able to build and train a template,
> which fails on deploy with an SSL config error. Since the PR does not
> strictly touch any templates I am going to have to ignore this serious
> issue and address it in Jira.
>
> Unless someone vetos this I plan to reluctantly push the PR with these
> non-trivial issues remaining.
>
>
>