Thanks so much for that excellent explanation. That all makes sense. I will open up that feature request. Definitely could see that being helpful.
On Fri, Aug 5, 2016 at 8:23 PM, Attila Szabo <[email protected]> wrote: > Hi Geren, > > To have all the details highlighted we need some Java "FUN" to understand: > So MySQLManager.java states perfectly that it's driver is > com.mysql.jdbc.Driver. (MySQL.java line 57 in the current trunk version) > And in it's super class SqlManager it also validates if the driver class > is there, by trying to load it. (SQLManager.java line 873 in the current > trunk version) > Although the static initializer of the MariaDB driver also loads the > org.maraidb.jdbc.Driver class (Driver.java line 67 in the latest mariadb > connector tgz file). It's done through the > DriverManager.registerDriver(java.sql.Driver) > method. > > com.mysql.jdbc.Driver states it's responsible for protocols starting with > "jdbc:mysql", meanwhile org.mariadb.jdbc.Driver states the very same (as > mariadb is a fork of MySQL). (this part can be found in both driver's > overridden acceptsUrl(java.lang.String) method. > > Thus from this POV we're in a bit of concurrent race between mariadb > driver, and mysql driver. And if by any reason the static intializer of > mariadb driver is executed earlier than MySQLManager (or anyone else) would > load the mysql jdbc driver, then mariadb would win, as it's states it can > handle properly MySQL database. > > On the top: > As I've read on the bug lists this bug is quite old in mariadb, so it is > absolutely possible that in any newer version it is fixed (or maybe not!). > > There is a possibility (the community has to investigate that scenario of > course), that Sqoop could attach a logWriter to DriverManager class, and > thus those information when drivers are loaded (and in which order) would > be visible on the std err/out or logfiles. > > If you think this feature request would help you in the future, I strongly > encourage you to open an FR JIRA ticket to the Sqoop development community > where we could discuss the possible solutions around this problem. > > Best regards, > Attila > > > > > On Fri, Aug 5, 2016 at 2:43 PM, Geren White <[email protected]> wrote: > >> Attila thank you so much for your responses. Ended up getting me in the >> right direction and I fixed the issue. It turns out that EMR installs >> mariadb connectors in the java installation of the server. After removing >> those the import started working. >> >> Any ideas why the mariadb connector would take precedence over the mysql >> one? It seems like this would be outside of the control of Sqoop. I was >> taking a look at the sqoop code and when specifying --driver and no manager >> it's going to use the GenericJdbcManager so would end up using whatever was >> handling the jdbc:mysql connection strings. Since mariadb connector was in >> the java installation I guess that takes precedence. If I specify both >> --driver and --connection-manager it blows up because the MysqlManager >> doesn't have the appropriate constructor. Specifying just the manager looks >> like it would force it to use the MysqlManager which would end up doing a >> Class.forName("com.mysql.jdbc.Driver"). Seems like this isn't enough to >> make it use the Mysql connector? >> >> Thanks, >> Geren >> > > > > -- > Best regards, > > Attila Szabo > Sotware Engineer > > <http://www.cloudera.com> > -- *Geren White | Senior Java Developer* *(e)* [email protected]
