vloup created ZEPPELIN-2891:
-------------------------------
Summary: Impossible to use jdbc interface with presto-jdbc >= 0.180
Key: ZEPPELIN-2891
URL: https://issues.apache.org/jira/browse/ZEPPELIN-2891
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.7.2
Environment: Official apache/zeppelin:0.7.2 docker image environment.
Deployment:
{{docker run \
--name zeppelin072 \
-v /etc/zeppelin/conf/:/zeppelin/conf \
-v /etc/zeppelin/notebook/:/notebook \
-v /etc/zeppelin/log/:/log \
--network=host \
-d \
apache/zeppelin:0.7.2}}
PrestoDB 0.183
Docker 17.03.1-ce running on Ubuntu 16.04.2 LTS (xenial) on amd64
Reporter: vloup
Priority: Blocker
Hello,
I have been trying since yesterday to integrate Presto (0.183) with Apache
Zeppelin (0.7.2) (using the presto-jdbc driver interface), and, so far, I am
hitting a roadblock.
Since presto version 0.180 and higher, with commit
defabcba4e1b89d5df536da51258cfdbfea906cb ("Improve properties/URL parameter
handling"), developers did tighten connection properties with
com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties.
This function rejects anything that is not specified into
com.facebook.presto.jdbc.ConnectionProperties.
Sadly, Zeppelin adds extra property keys such as "url" and "driver" which,
naturally, get rejected. (See org.apache.zeppelin.jdbc.JDBCInterpreter.java at
function "open")
A typical debug log would look like that:
{{ INFO [2017-08-31 07:14:21,156] ({pool-2-thread-2}
SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1504163661153
started by scheduler org.apache.zeppelin.jdbc.JDBCInterpreter43219549
DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: default.password
DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: default, value: password
DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.auth.type
DEBUG [2017-08-31 07:14:21,159] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.auth.type
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: common.max_count
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: common, value: max_count
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.principal
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.principal
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: default.user
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: default, value: user
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.localRepo
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.localRepo
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: default.url
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: default, value: url
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: default.driver
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: default, value: driver
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.keytab.location
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.keytab.location
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.concurrent.use
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.concurrent.use
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey:
zeppelin.jdbc.concurrent.max_connection
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value:
jdbc.concurrent.max_connection
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.output.limit
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.output.limit
ERROR [2017-08-31 07:14:21,162] ({pool-2-thread-2}
JDBCInterpreter.java[open]:177) - zeppelin will be ignored. driver.zeppelin and
zeppelin.url is mandatory.
DEBUG [2017-08-31 07:14:21,166] ({pool-2-thread-2}
JDBCInterpreter.java[open]:187) - JDBC PropretiesMap: {default={user=presto,
url=jdbc:presto://DOMAIN_NAME:4481/hive;LogLevel=6;LogPath=/log/, password=,
driver=com.facebook.presto.jdbc.PrestoDriver}, common={max_count=1000}}
DEBUG [2017-08-31 07:14:21,191] ({pool-1-thread-3}
Interpreter.java[getProperty]:165) - key: zeppelin.jdbc.concurrent.use, value:
true
DEBUG [2017-08-31 07:14:21,192] ({pool-1-thread-3}
Interpreter.java[getProperty]:165) - key:
zeppelin.jdbc.concurrent.max_connection, value: 10
DEBUG [2017-08-31 07:14:21,208] ({pool-2-thread-2}
JDBCInterpreter.java[interpret]:683) - Run SQL command 'use dataset_demo;
show tables;'
DEBUG [2017-08-31 07:14:21,208] ({pool-2-thread-2}
JDBCInterpreter.java[interpret]:691) - PropertyKey: default, SQL command: 'use
dataset_demo;
show tables;'
ERROR [2017-08-31 07:14:21,690] ({pool-2-thread-2}
JDBCInterpreter.java[executeSql]:631) - Cannot run use dataset_demo;
show tables;
java.sql.SQLException: Unrecognized connection property 'url'
at
com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292)
at
com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:89)
at
com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:79)
at com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at
org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:354)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:424)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:565)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)}}
As a workaround, it is possible to locally downgrade the presto-jdbc driver in
Zeppelin to version 0.179, but we do lack features included in later versions.
Comes my question: which one shall get patched?
I am doing a similar report on the presto google group.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
