Actually I just need to load it in the main method (job manager) before
calling any flink operation, I retrieve the records in a mysql table
because they contain the path of files I'll need to read. Nothing more
nothing less
On 5 Jun 2015 19:06, "Robert Metzger" <rmetz...@apache.org> wrote:
> Sure.
>
> So the DriverManager has a static variable called "registeredDrivers".
> When DriverManager.getConnection() is called, the method is looking up if
> an registered driver for that connection (in this case "mysql") is
> available.
>
> For drivers to be in that list, they have to register themselves using
> the DriverManager.registerDriver() method.
>
> Drivers can register themselves with a static constructor (which is
> executed when Java loads a class):
>
> public class PoolingDriver implements Driver {
> /** Register myself with the {@link DriverManager}. */
> static {
> try {
> DriverManager.registerDriver(new PoolingDriver());
> } catch(Exception e) {
> }
> }
>
> To execute that driver registration, you need to do:
> Class.forName("org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver
> ");
> because then Java is loading the class and executing the static
> constructor which is registering the driver at the connection manager.
>
> When executing Flink locally, you are using only one JVM. By calling the
> MySQL driver manually in the main() method of your flink job, you are
> registering the MySQL driver at the DriverManager of that JVM.
>
> However, when you run Flink in a distributed cluster, at the TaskManager
> JVMs, the MySQL driver is not loaded at the DriverManager there.
> Therefore, you have to make sure that Class.forName("
> org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver"); (this is not
> the correct class for the MySQL driver)
> has been called.
> One approach to do that is to call Class.forName() in the open() method of
> your function.
>
> Best,
> Robert
>
>
> On Fri, Jun 5, 2015 at 6:54 PM, Flavio Pompermaier <pomperma...@okkam.it>
> wrote:
>
>> HI Robert,
>> In the main method I connect to a mysql table that acts as a data-source
>> repository that I use to know which dataset I need to load. All mysql
>> classes are present in the shaded jar.
>> Could you explain a little bit more in detail the solution to fix this
>> problem please? Sorry but I didn't understand it :(
>>
>> Thanks,
>> Flavio
>> On 5 Jun 2015 18:33, "Robert Metzger" <rmetz...@apache.org> wrote:
>>
>>> Hi Stefano,
>>>
>>> I doubt that there are conflicting dependencies because Flink does not
>>> contain MySQL dependencies.
>>> Are you using Flink's JDBCInputFormat or custom code?
>>>
>>> For drivers to register at java.sql's DriverManager, their classes need
>>> to be loaded first. To load a class, you need to call
>>> Class.forName("<classname>");
>>>
>>> Maybe you are loading the class in the application's main() method
>>> (thats why it is working from eclipse) but not on the cluster instances
>>> which are supposed to read the data.
>>>
>>> On Fri, Jun 5, 2015 at 5:16 PM, Stefano Bortoli <s.bort...@gmail.com>
>>> wrote:
>>>
>>>> Hi Robert,
>>>>
>>>> I answer on behalf of Flavio. He told me the driver jar was included.
>>>> Smells lik class-loading issue due to 'conflicting' dependencies. Is it
>>>> possible?
>>>>
>>>> Saluti,
>>>> Stefano
>>>>
>>>> 2015-06-05 16:24 GMT+02:00 Robert Metzger <rmetz...@apache.org>:
>>>>
>>>>> Hi,
>>>>>
>>>>> is the MySQL driver part of the Jar file that you've build?
>>>>>
>>>>> On Fri, Jun 5, 2015 at 4:11 PM, Flavio Pompermaier <
>>>>> pomperma...@okkam.it> wrote:
>>>>>
>>>>>> Hi to all,
>>>>>>
>>>>>> I'm using a fresh build of flink-0.9-SNAPSHOT and in my flink job I
>>>>>> set up a mysql connection.
>>>>>> When I run the job from Eclipse everything is fine,
>>>>>> while when running the job from the Web UI I get the following
>>>>>> exception:
>>>>>>
>>>>>> java.sql.SQLException: No suitable driver found for
>>>>>> jdbc:mysql:/localhost:3306/mydb?autoReconnect=true
>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>>>>>
>>>>>> How can I fix that?
>>>>>>
>>>>>> Best,
>>>>>> Flavio
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
