in the fat jar
On 5 Jun 2015 19:28, "Stephan Ewen" <se...@apache.org> wrote:
> In which way is the driver in the classpath?
>
> - fat jar?
> - in the nested /out folder in the slim jar?
>
>
>
> On Fri, Jun 5, 2015 at 7:23 PM, Flavio Pompermaier <pomperma...@okkam.it>
> wrote:
>
>> Actually I just need to load it in the main method (job manager) before
>> calling any flink operation, I retrieve the records in a mysql table
>> because they contain the path of files I'll need to read. Nothing more
>> nothing less
>> On 5 Jun 2015 19:06, "Robert Metzger" <rmetz...@apache.org> wrote:
>>
>>> Sure.
>>>
>>> So the DriverManager has a static variable called "registeredDrivers".
>>> When DriverManager.getConnection() is called, the method is looking up
>>> if an registered driver for that connection (in this case "mysql") is
>>> available.
>>>
>>> For drivers to be in that list, they have to register themselves using
>>> the DriverManager.registerDriver() method.
>>>
>>> Drivers can register themselves with a static constructor (which is
>>> executed when Java loads a class):
>>>
>>> public class PoolingDriver implements Driver {
>>> /** Register myself with the {@link DriverManager}. */
>>> static {
>>> try {
>>> DriverManager.registerDriver(new PoolingDriver());
>>> } catch(Exception e) {
>>> }
>>> }
>>>
>>> To execute that driver registration, you need to do:
>>> Class.forName("org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver
>>> ");
>>> because then Java is loading the class and executing the static
>>> constructor which is registering the driver at the connection manager.
>>>
>>> When executing Flink locally, you are using only one JVM. By calling the
>>> MySQL driver manually in the main() method of your flink job, you are
>>> registering the MySQL driver at the DriverManager of that JVM.
>>>
>>> However, when you run Flink in a distributed cluster, at the TaskManager
>>> JVMs, the MySQL driver is not loaded at the DriverManager there.
>>> Therefore, you have to make sure that Class.forName("
>>> org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver"); (this is
>>> not the correct class for the MySQL driver)
>>> has been called.
>>> One approach to do that is to call Class.forName() in the open() method
>>> of your function.
>>>
>>> Best,
>>> Robert
>>>
>>>
>>> On Fri, Jun 5, 2015 at 6:54 PM, Flavio Pompermaier <pomperma...@okkam.it
>>> > wrote:
>>>
>>>> HI Robert,
>>>> In the main method I connect to a mysql table that acts as a
>>>> data-source repository that I use to know which dataset I need to load. All
>>>> mysql classes are present in the shaded jar.
>>>> Could you explain a little bit more in detail the solution to fix this
>>>> problem please? Sorry but I didn't understand it :(
>>>>
>>>> Thanks,
>>>> Flavio
>>>> On 5 Jun 2015 18:33, "Robert Metzger" <rmetz...@apache.org> wrote:
>>>>
>>>>> Hi Stefano,
>>>>>
>>>>> I doubt that there are conflicting dependencies because Flink does not
>>>>> contain MySQL dependencies.
>>>>> Are you using Flink's JDBCInputFormat or custom code?
>>>>>
>>>>> For drivers to register at java.sql's DriverManager, their classes
>>>>> need to be loaded first. To load a class, you need to call
>>>>> Class.forName("<classname>");
>>>>>
>>>>> Maybe you are loading the class in the application's main() method
>>>>> (thats why it is working from eclipse) but not on the cluster instances
>>>>> which are supposed to read the data.
>>>>>
>>>>> On Fri, Jun 5, 2015 at 5:16 PM, Stefano Bortoli <s.bort...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> I answer on behalf of Flavio. He told me the driver jar was included.
>>>>>> Smells lik class-loading issue due to 'conflicting' dependencies. Is it
>>>>>> possible?
>>>>>>
>>>>>> Saluti,
>>>>>> Stefano
>>>>>>
>>>>>> 2015-06-05 16:24 GMT+02:00 Robert Metzger <rmetz...@apache.org>:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> is the MySQL driver part of the Jar file that you've build?
>>>>>>>
>>>>>>> On Fri, Jun 5, 2015 at 4:11 PM, Flavio Pompermaier <
>>>>>>> pomperma...@okkam.it> wrote:
>>>>>>>
>>>>>>>> Hi to all,
>>>>>>>>
>>>>>>>> I'm using a fresh build of flink-0.9-SNAPSHOT and in my flink job I
>>>>>>>> set up a mysql connection.
>>>>>>>> When I run the job from Eclipse everything is fine,
>>>>>>>> while when running the job from the Web UI I get the following
>>>>>>>> exception:
>>>>>>>>
>>>>>>>> java.sql.SQLException: No suitable driver found for
>>>>>>>> jdbc:mysql:/localhost:3306/mydb?autoReconnect=true
>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>>>>>>>
>>>>>>>> How can I fix that?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Flavio
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>
