Re: Bringing up JDBC Tests to trunk

2015-12-06 Thread Josh Rosen
Can you write a script to download and install the JDBC driver to the local Maven repository if it's not already present? If we had that, we could just invoke it as part of dev/run-tests. On Thu, Dec 3, 2015 at 5:55 PM Luciano Resende wrote: > > > On Mon, Nov 30, 2015 at

Re: Bringing up JDBC Tests to trunk

2015-12-03 Thread Luciano Resende
On Mon, Nov 30, 2015 at 1:53 PM, Josh Rosen wrote: > The JDBC drivers are currently being pulled in as test-scope dependencies > of the `sql/core` module: > https://github.com/apache/spark/blob/f2fbfa444f6e8d27953ec2d1c0b3abd603c963f9/sql/core/pom.xml#L91 > > In SBT,

Re: Bringing up JDBC Tests to trunk

2015-12-01 Thread Jacek Laskowski
On Mon, Nov 30, 2015 at 10:53 PM, Josh Rosen wrote: > In SBT, these wind up on the Docker JDBC tests' classpath as a transitive > dependency of the `spark-sql` test JAR. However, what we should be doing is > adding them as explicit test dependencies of the

Re: Bringing up JDBC Tests to trunk

2015-11-30 Thread Josh Rosen
The JDBC drivers are currently being pulled in as test-scope dependencies of the `sql/core` module: https://github.com/apache/spark/blob/f2fbfa444f6e8d27953ec2d1c0b3abd603c963f9/sql/core/pom.xml#L91 In SBT, these wind up on the Docker JDBC tests' classpath as a transitive dependency of the

Re: Bringing up JDBC Tests to trunk

2015-11-22 Thread Luciano Resende
Hey Josh, Thanks for helping bringing this up, I have just pushed a WIP PR for bringing the DB2 tests to be running on Docker, and I have a question about how the jdbc drivers are actually being setup for the other datasources (MySQL and PostgreSQL), are these setup directly on the Jenkins slaves

Bringing up JDBC Tests to trunk

2015-10-21 Thread Luciano Resende
I have started looking into PR-8101 [1] and what is required to merge it into trunk which will also unblock me around SPARK-10521 [2]. So here is the minimal plan I was thinking about : - make the docker image version fixed so we make sure we are using the same image all the time - pull the

Re: Bringing up JDBC Tests to trunk

2015-10-21 Thread Josh Rosen
Hey Luciano, This sounds like a reasonable plan to me. One of my colleagues has written some Dockerized MySQL testing utilities, so I'll take a peek at those to see if there are any specifics of their solution that we should adapt for Spark. On Wed, Oct 21, 2015 at 1:16 PM, Luciano Resende