Re: Jenkins infra is not stable for MADlib PR

[Rahul Iyer]

I've fixed the POM file not found issue (build 99
)
- it was looking in wrong place.

On Tue, Sep 19, 2017 at 5:19 PM, Nandish Jayaram 
wrote:

> There is another Jenkins related failure:
> https://builds.apache.org/user/riyer/my-views/view/
> MADlib-Monitor/job/madlib-master-build/98/console
>
> [ERROR] The build could not read 1 project -> [Help 1][ERROR]
> [ERROR]   The project
> (/home/jenkins/jenkins-slave/workspace/madlib-master-build/
> incubator-madlib/pom.xml)
> has 1 error[ERROR] Non-readable POM
> /home/jenkins/jenkins-slave/workspace/madlib-master-build/
> incubator-madlib/pom.xml:
> /home/jenkins/jenkins-slave/workspace/madlib-master-build/
> incubator-madlib/pom.xml
> (No such file or directory)[ERROR] [ERROR] To see the full stack trace
> of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven
> using the -X switch to enable full debug logging.[ERROR] [ERROR] For
> more information about the errors and possible solutions, please read
> the following articles:[ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
>
>
>
> This seems to be different from what Jingyi pointed out. Any workaround for
> this,
> or should we raise a ticket instead?
>
> NJ
>
> On Tue, Sep 19, 2017 at 2:15 PM, Ed Espino  wrote:
>
> > Thanks for the info Roman.
> >
> > For the "madlib-pr-build" and "madlib-master-build" Apache Jenkins jobs
> and
> > to exclude the problematic agent "qnode3", I have changed the "Label
> > Expression" value for the "Restrict where this project can be run" option
> > from "ubuntu" to "ubuntu && ! qnode3".
> >
> > Cheers,
> > -=e
> >
> >
> > On Tue, Sep 19, 2017 at 1:58 PM, Roman Shaposhnik 
> > wrote:
> >
> > > A quick solution is to disable the offending host in your Jenkins
> > > configuration.
> > >
> > > Thanks,
> > > Roman.
> > >
> > > On Tue, Sep 19, 2017 at 12:15 PM, Jingyi Mei  wrote:
> > > > Hi Ed,
> > > >
> > > > Thanks for referring to the INFRA issue. Seems there is no quick
> > solution
> > > > for that yet. And since I kept getting the same error for my PR, I am
> > > going
> > > > to create another issue specifically for unblocking my build.
> > > >
> > > > Thank you very much!
> > > > Jingyi
> > > >
> > > > On Mon, Sep 18, 2017 at 8:23 PM, Ed Espino 
> wrote:
> > > >
> > > >> Hey Jingyi,
> > > >>
> > > >> Welcome to the project. These types of issues do happen from time to
> > > time.
> > > >> There is a recent Infrastructure issue which references the same
> > system
> > > >> that your MADlib PR build hit.
> > > >>
> > > >> Jira: INFRA-14979
> > > >> Summary: Jenkins jobs on qnode3 are failing with "No space left on
> > > device"
> > > >> Jira link: https://issues.apache.org/jira/browse/INFRA-14979
> > > >>
> > > >> If the problem persists, please file an Infrastructure (INFRA)
> ticket
> > > so it
> > > >> can be addressed. There is a good chance it impacts other projects
> as
> > > well.
> > > >>
> > > >> Thanks,
> > > >> -=e
> > > >>
> > > >>
> > > >> On Mon, Sep 18, 2017 at 2:25 PM, Jingyi Mei 
> wrote:
> > > >>
> > > >> > Hi developers,
> > > >> >
> > > >> > Recently, I kept getting build failure on Jenkins for infra
> reasons,
> > > >> error
> > > >> > message is as following:
> > > >> >
> > > >> > ---
> > > >> >
> > > >> > FATAL: Unable to produce a script file
> > > >> > java.io.IOException: No space left on device
> > > >> > at java.io.FileOutputStream.writeBytes(Native Method)
> > > >> > at java.io.FileOutputStream.write(FileOutputStream.java:
> > 326)
> > > >> > at sun.nio.cs.StreamEncoder.
> writeBytes(StreamEncoder.java:
> > > 221)
> > > >> > at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:
> > 316)
> > > >> > at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
> > > >> > at java.io.OutputStreamWriter.
> > close(OutputStreamWriter.java:
> > > 233)
> > > >> > at hudson.FilePath$17.invoke(FilePath.java:1380)
> > > >> > at hudson.FilePath$17.invoke(FilePath.java:1363)
> > > >> > at hudson.FilePath$FileCallableWrapper.call(
> > > FilePath.java:2739)
> > > >> > at hudson.remoting.UserRequest.
> > perform(UserRequest.java:153)
> > > >> > at hudson.remoting.UserRequest.
> perform(UserRequest.java:50)
> > > >> > at hudson.remoting.Request$2.run(Request.java:336)
> > > >> > at hudson.remoting.InterceptingExecutorService$1.call(
> > > >> > InterceptingExecutorService.java:68)
> > > >> > at java.util.concurrent.FutureTask.run(FutureTask.
> java:266)
> > > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > >> > ThreadPoolExecutor.java:1142)
> > > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(

Re: Jenkins infra is not stable for MADlib PR

[Nandish Jayaram]

There is another Jenkins related failure:
https://builds.apache.org/user/riyer/my-views/view/MADlib-Monitor/job/madlib-master-build/98/console

[ERROR] The build could not read 1 project -> [Help 1][ERROR]
[ERROR]   The project
(/home/jenkins/jenkins-slave/workspace/madlib-master-build/incubator-madlib/pom.xml)
has 1 error[ERROR] Non-readable POM
/home/jenkins/jenkins-slave/workspace/madlib-master-build/incubator-madlib/pom.xml:
/home/jenkins/jenkins-slave/workspace/madlib-master-build/incubator-madlib/pom.xml
(No such file or directory)[ERROR] [ERROR] To see the full stack trace
of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven
using the -X switch to enable full debug logging.[ERROR] [ERROR] For
more information about the errors and possible solutions, please read
the following articles:[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException



This seems to be different from what Jingyi pointed out. Any workaround for
this,
or should we raise a ticket instead?

NJ

On Tue, Sep 19, 2017 at 2:15 PM, Ed Espino  wrote:

> Thanks for the info Roman.
>
> For the "madlib-pr-build" and "madlib-master-build" Apache Jenkins jobs and
> to exclude the problematic agent "qnode3", I have changed the "Label
> Expression" value for the "Restrict where this project can be run" option
> from "ubuntu" to "ubuntu && ! qnode3".
>
> Cheers,
> -=e
>
>
> On Tue, Sep 19, 2017 at 1:58 PM, Roman Shaposhnik 
> wrote:
>
> > A quick solution is to disable the offending host in your Jenkins
> > configuration.
> >
> > Thanks,
> > Roman.
> >
> > On Tue, Sep 19, 2017 at 12:15 PM, Jingyi Mei  wrote:
> > > Hi Ed,
> > >
> > > Thanks for referring to the INFRA issue. Seems there is no quick
> solution
> > > for that yet. And since I kept getting the same error for my PR, I am
> > going
> > > to create another issue specifically for unblocking my build.
> > >
> > > Thank you very much!
> > > Jingyi
> > >
> > > On Mon, Sep 18, 2017 at 8:23 PM, Ed Espino  wrote:
> > >
> > >> Hey Jingyi,
> > >>
> > >> Welcome to the project. These types of issues do happen from time to
> > time.
> > >> There is a recent Infrastructure issue which references the same
> system
> > >> that your MADlib PR build hit.
> > >>
> > >> Jira: INFRA-14979
> > >> Summary: Jenkins jobs on qnode3 are failing with "No space left on
> > device"
> > >> Jira link: https://issues.apache.org/jira/browse/INFRA-14979
> > >>
> > >> If the problem persists, please file an Infrastructure (INFRA) ticket
> > so it
> > >> can be addressed. There is a good chance it impacts other projects as
> > well.
> > >>
> > >> Thanks,
> > >> -=e
> > >>
> > >>
> > >> On Mon, Sep 18, 2017 at 2:25 PM, Jingyi Mei  wrote:
> > >>
> > >> > Hi developers,
> > >> >
> > >> > Recently, I kept getting build failure on Jenkins for infra reasons,
> > >> error
> > >> > message is as following:
> > >> >
> > >> > ---
> > >> >
> > >> > FATAL: Unable to produce a script file
> > >> > java.io.IOException: No space left on device
> > >> > at java.io.FileOutputStream.writeBytes(Native Method)
> > >> > at java.io.FileOutputStream.write(FileOutputStream.java:
> 326)
> > >> > at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:
> > 221)
> > >> > at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:
> 316)
> > >> > at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
> > >> > at java.io.OutputStreamWriter.
> close(OutputStreamWriter.java:
> > 233)
> > >> > at hudson.FilePath$17.invoke(FilePath.java:1380)
> > >> > at hudson.FilePath$17.invoke(FilePath.java:1363)
> > >> > at hudson.FilePath$FileCallableWrapper.call(
> > FilePath.java:2739)
> > >> > at hudson.remoting.UserRequest.
> perform(UserRequest.java:153)
> > >> > at hudson.remoting.UserRequest.perform(UserRequest.java:50)
> > >> > at hudson.remoting.Request$2.run(Request.java:336)
> > >> > at hudson.remoting.InterceptingExecutorService$1.call(
> > >> > InterceptingExecutorService.java:68)
> > >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > >> > ThreadPoolExecutor.java:1142)
> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > >> > ThreadPoolExecutor.java:617)
> > >> > at java.lang.Thread.run(Thread.java:745)
> > >> > at ..remote call to qnode3(Native Method)
> > >> > at hudson.remoting.Channel.attachCallSiteStackTrace(
> > >> > Channel.java:1545)
> > >> > at hudson.remoting.UserResponse.
> > retrieve(UserRequest.java:253)
> > >> > at hudson.remoting.Channel.call(Channel.java:830)
> > >> > at hudson.FilePath.act(FilePath.java:986)
> > >> > Caused: java.io.IOException: remote file 

[GitHub] madlib issue #181: Add a script to start docker image

[asfgit]

Github user asfgit commented on the issue:

https://github.com/apache/madlib/pull/181
  

Refer to this link for build results (access rights to CI server needed): 
https://builds.apache.org/job/madlib-pr-build/249/



---

[GitHub] madlib pull request #182: MADLIB-1154. PostgreSQL 10 (beta 4) support.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/madlib/pull/182


---

[GitHub] madlib issue #187: Docs: MLP docs to be corrected post grouping support

[asfgit]

Github user asfgit commented on the issue:

https://github.com/apache/madlib/pull/187
  

Refer to this link for build results (access rights to CI server needed): 
https://builds.apache.org/job/madlib-pr-build/248/



---

[GitHub] madlib issue #182: MADLIB-1154. PostgreSQL 10 (beta 4) support.

[njayaram2]

Github user njayaram2 commented on the issue:

https://github.com/apache/madlib/pull/182
  
Will go ahead and merge this since @jingyimei has approved.


---

[GitHub] madlib pull request #187: Docs: MLP docs to be corrected post grouping suppo...

[njayaram2]

GitHub user njayaram2 opened a pull request:

https://github.com/apache/madlib/pull/187

Docs: MLP docs to be corrected post grouping support

The MLP docs created during grouping support had some inadvertant
changes. This commit fixes those issues.

Closes #187

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/njayaram2/madlib docs/mlp

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/madlib/pull/187.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #187






---

[GitHub] madlib issue #181: Add a script to start docker image

[edespino]

Github user edespino commented on the issue:

https://github.com/apache/madlib/pull/181
  
Jenkins ok to test


---

[GitHub] madlib issue #184: Knn dev 1129

[asfgit]

Github user asfgit commented on the issue:

https://github.com/apache/madlib/pull/184
  

Refer to this link for build results (access rights to CI server needed): 
https://builds.apache.org/job/madlib-pr-build/247/



---

[GitHub] madlib issue #184: Knn dev 1129

[edespino]

Github user edespino commented on the issue:

https://github.com/apache/madlib/pull/184
  
Jenkins ok to test


---

[GitHub] madlib pull request #186: Add error message for checking postgres install co...

[jingyimei]

Github user jingyimei commented on a diff in the pull request:

https://github.com/apache/madlib/pull/186#discussion_r139798284
  
--- Diff: src/madpack/madpack.py ---
@@ -512,7 +512,9 @@ def _plpy_check(py_min_ver):
 try:
 _internal_run_query("CREATE LANGUAGE plpythonu;", True)
 except:
-_error('Cannot create language plpythonu. Stopping 
installation...', False)
+_error("""Cannot create language plpythonu. Please check if you
+have configured and installed postgres with `--with-python`
--- End diff --

Sure, that should be more general.


---

[GitHub] madlib pull request #186: Add error message for checking postgres install co...

[edespino]

Github user edespino commented on a diff in the pull request:

https://github.com/apache/madlib/pull/186#discussion_r139794616
  
--- Diff: src/madpack/madpack.py ---
@@ -512,7 +512,9 @@ def _plpy_check(py_min_ver):
 try:
 _internal_run_query("CREATE LANGUAGE plpythonu;", True)
 except:
-_error('Cannot create language plpythonu. Stopping 
installation...', False)
+_error("""Cannot create language plpythonu. Please check if you
+have configured and installed postgres with `--with-python`
--- End diff --

Can you possibly use "portid" or "args.platform[0]" in place of "postgres"?


---

[GitHub] madlib issue #186: Add error message for checking postgres install configura...

[asfgit]

Github user asfgit commented on the issue:

https://github.com/apache/madlib/pull/186
  

Refer to this link for build results (access rights to CI server needed): 
https://builds.apache.org/job/madlib-pr-build/245/



---

Re: Jenkins infra is not stable for MADlib PR

[Jingyi Mei]

Hi Ed,

Thanks for referring to the INFRA issue. Seems there is no quick solution
for that yet. And since I kept getting the same error for my PR, I am going
to create another issue specifically for unblocking my build.

Thank you very much!
Jingyi

On Mon, Sep 18, 2017 at 8:23 PM, Ed Espino  wrote:

> Hey Jingyi,
>
> Welcome to the project. These types of issues do happen from time to time.
> There is a recent Infrastructure issue which references the same system
> that your MADlib PR build hit.
>
> Jira: INFRA-14979
> Summary: Jenkins jobs on qnode3 are failing with "No space left on device"
> Jira link: https://issues.apache.org/jira/browse/INFRA-14979
>
> If the problem persists, please file an Infrastructure (INFRA) ticket so it
> can be addressed. There is a good chance it impacts other projects as well.
>
> Thanks,
> -=e
>
>
> On Mon, Sep 18, 2017 at 2:25 PM, Jingyi Mei  wrote:
>
> > Hi developers,
> >
> > Recently, I kept getting build failure on Jenkins for infra reasons,
> error
> > message is as following:
> >
> > ---
> >
> > FATAL: Unable to produce a script file
> > java.io.IOException: No space left on device
> > at java.io.FileOutputStream.writeBytes(Native Method)
> > at java.io.FileOutputStream.write(FileOutputStream.java:326)
> > at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
> > at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:316)
> > at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
> > at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
> > at hudson.FilePath$17.invoke(FilePath.java:1380)
> > at hudson.FilePath$17.invoke(FilePath.java:1363)
> > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:153)
> > at hudson.remoting.UserRequest.perform(UserRequest.java:50)
> > at hudson.remoting.Request$2.run(Request.java:336)
> > at hudson.remoting.InterceptingExecutorService$1.call(
> > InterceptingExecutorService.java:68)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1142)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > at ..remote call to qnode3(Native Method)
> > at hudson.remoting.Channel.attachCallSiteStackTrace(
> > Channel.java:1545)
> > at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
> > at hudson.remoting.Channel.call(Channel.java:830)
> > at hudson.FilePath.act(FilePath.java:986)
> > Caused: java.io.IOException: remote file operation failed:
> > /home/jenkins/jenkins-slave/workspace/madlib-pr-build
> >
> > 
> >
> >
> > It seems that for a specific PR, it was always sent to a specific test
> > machine and that machine always didn't have enough space.
> >
> >
> > Does any one know how to fix this?
> >
> >
> > Thanks,
> >
> > Jingyi
> >
>
>
>
> --
> *Ed Espino*
>

[GitHub] madlib pull request #186: Add error message for checking postgres install co...

[jingyimei]

GitHub user jingyimei opened a pull request:

https://github.com/apache/madlib/pull/186

Add error message for checking postgres install configuration

MADlib needs to be installed on a postgres with python extension. If
postgres is not configured with the `--with-python` option, madpack
installation will failed for not finding $libdir/plpython2 directory.
This commit add some instructions when this failure happens.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jingyimei/madlib python_error_message

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/madlib/pull/186.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #186


commit f123105aa543953a12757eb228d33e10f6979b9f
Author: Jingyi Mei 
Date:   2017-09-19T19:01:02Z

Add error message for checking postgres install configuration

MADlib needs to be installed on a postgres with python configured. If
postgres is not configured with the `--with-python` option, madpack
installation will failed for not finding $libdir/plpython2 directory.
This commit add some instructions when this failure happens.




---