[jira] [Updated] (DRILL-4281) Drill should support inbound impersonation
[ https://issues.apache.org/jira/browse/DRILL-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-4281: --- Fix Version/s: 1.6.0 > Drill should support inbound impersonation > -- > > Key: DRILL-4281 > URL: https://issues.apache.org/jira/browse/DRILL-4281 > Project: Apache Drill > Issue Type: Improvement >Reporter: Keys Botzum >Assignee: Sudheesh Katkam > Labels: doc-impacting, security > Fix For: 1.6.0 > > > Today Drill supports impersonation *to* external sources. For example I can > authenticate to Drill as myself and then Drill will access HDFS using > impersonation > In many scenarios we also need impersonation to Drill. For example I might > use some front end tool (such as Tableau) and authenticate to it as myself. > That tool (server version) then needs to access Drill to perform queries and > I want those queries to run as myself, not as the Tableau user. While in > theory the intermediate tool could store the userid & password for every user > to the Drill this isn't a scalable or very secure solution. > Note that HS2 today does support inbound impersonation as described here: > https://issues.apache.org/jira/browse/HIVE-5155 > The above is not the best approach as it is tied to the connection object > which is very coarse grained and potentially expensive. It would be better if > there was a call on the ODBC/JDBC driver to switch the identity on a existing > connection. Most modern SQL databases (Oracle, DB2) support such function. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4362) MapR profile - DRILL-3581 breaks build
[ https://issues.apache.org/jira/browse/DRILL-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-4362: --- Fix Version/s: 1.6.0 > MapR profile - DRILL-3581 breaks build > -- > > Key: DRILL-4362 > URL: https://issues.apache.org/jira/browse/DRILL-4362 > Project: Apache Drill > Issue Type: Bug >Reporter: Sudheesh Katkam >Assignee: Sudheesh Katkam > Fix For: 1.6.0 > > > The new rule in [this > commit|https://github.com/apache/drill/commit/422c5a83b8e69e4169d3ebc946401248073c8bf8#diff-8f76e2ed78ea4afabd0d911a33fec0fc] > that adds log4j to enforcer exclusions breaks the mapr build (*mvn clean > install -DskipTests -Pmapr*). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4588) Enable JMXReporter to Expose Metrics
[ https://issues.apache.org/jira/browse/DRILL-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-4588: --- Fix Version/s: 1.7.0 > Enable JMXReporter to Expose Metrics > > > Key: DRILL-4588 > URL: https://issues.apache.org/jira/browse/DRILL-4588 > Project: Apache Drill > Issue Type: Bug >Reporter: Sudheesh Katkam >Assignee: Sudheesh Katkam > Fix For: 1.7.0 > > > -There is a static initialization order issue that needs to be fixed.- > The code is commented out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3623) Limit 0 should avoid execution when querying a known schema
[ https://issues.apache.org/jira/browse/DRILL-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-3623: --- Fix Version/s: (was: Future) 1.7.0 > Limit 0 should avoid execution when querying a known schema > --- > > Key: DRILL-3623 > URL: https://issues.apache.org/jira/browse/DRILL-3623 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Hive >Affects Versions: 1.1.0 > Environment: MapR cluster >Reporter: Andries Engelbrecht >Assignee: Sudheesh Katkam > Labels: doc-impacting > Fix For: 1.7.0 > > > Running a select * from hive.table limit 0 does not return (hangs). > Select * from hive.table limit 1 works fine > Hive table is about 6GB with 330 files with parquet using snappy compression. > Data types are int, bigint, string and double. > Querying directory with parquet files through the DFS plugin works fine > select * from dfs.root.`/user/hive/warehouse/database/table` limit 0; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3562) Query fails when using flatten on JSON data where some documents have an empty array
[
https://issues.apache.org/jira/browse/DRILL-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15357603#comment-15357603
]
F Méthot commented on DRILL-3562:
-
Really would like to see this one fixed!
Here is a workaround we are doing to get to our data:
This will extract data without the null arrays:
select t.a.b.c as c from dfs.`flat.json` t where t.a.b.c[0]['d'] is not null
(d is an value name expected to be found within the array)
but flatten still won't work:
To get flatten working:
create table TEMP_JSON_DATA as (select t.a.b.c as c from dfs..`flat.json`
t where t.a.b.c[0]['d'] is not null);
then
select flatten(c) from TEMP_JSON_DATA;
(using parquet format for temp table)
For interactive analysis of data, is a pretty lame workaround, but for
scripting environment that worked out fine, if you automate dropping of the
temp table.
> Query fails when using flatten on JSON data where some documents have an
> empty array
>
>
> Key: DRILL-3562
> URL: https://issues.apache.org/jira/browse/DRILL-3562
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JSON
>Affects Versions: 1.1.0
>Reporter: Philip Deegan
> Fix For: Future
>
>
> Drill query fails when using flatten when some records contain an empty array
> {noformat}
> SELECT COUNT(*) FROM (SELECT FLATTEN(t.a.b.c) AS c FROM dfs.`flat.json` t)
> flat WHERE flat.c.d.e = 'f' limit 1;
> {noformat}
> Succeeds on
> { "a": { "b": { "c": [ { "d": { "e": "f" } } ] } } }
> Fails on
> { "a": { "b": { "c": [] } } }
> Error
> {noformat}
> Error: SYSTEM ERROR: ClassCastException: Cannot cast
> org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> {noformat}
> Is it possible to ignore the empty arrays, or do they need to be populated
> with dummy data?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-4171) When exceeded scripts "select * from mscIdentities limit 1;"throwException in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC overhead limit exceeded
[ https://issues.apache.org/jira/browse/DRILL-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4171. - Resolution: Workaround > When exceeded scripts "select * from mscIdentities limit 1;"throwException > in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC overhead > limit exceeded > --- > > Key: DRILL-4171 > URL: https://issues.apache.org/jira/browse/DRILL-4171 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.3.0 > Environment: drill 1.3 jdk7 x86_64 GNU/Linux >Reporter: david_hudavy > > 0: jdbc:drill:zk=local> select * from mscIdentities limit 1; > Exception in thread "WorkManager.StatusThread" java.lang.OutOfMemoryError: GC > overhead limit exceeded > at > java.util.concurrent.ConcurrentHashMap$Values.iterator(ConcurrentHashMap.java:1449) > at > org.apache.drill.exec.work.WorkManager$StatusThread.run(WorkManager.java:309) > Node ran out of Heap memory, exiting. > java.lang.OutOfMemoryError: GC overhead limit exceeded > at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2166) > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1956) > at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3308) > at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) > at > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3032) > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2280) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2546) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2504) > at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:101) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:128) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105) > at > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230) > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-4177) select * from table;Node ran out of Heap memory, exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded
[ https://issues.apache.org/jira/browse/DRILL-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva resolved DRILL-4177. - Resolution: Workaround > select * from table;Node ran out of Heap memory, > exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded > --- > > Key: DRILL-4177 > URL: https://issues.apache.org/jira/browse/DRILL-4177 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.3.0 > Environment: drill1.3 jdk7 >Reporter: david_hudavy > Labels: patch > Original Estimate: 12h > Remaining Estimate: 12h > > 0: jdbc:drill:zk=local> select * from table; > Node ran out of Heap memory, exiting. > java.lang.OutOfMemoryError: GC overhead limit exceeded > at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2149) > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1956) > at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3308) > at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) > at > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3032) > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2280) > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2546) > at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2504) > at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) > at > org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:101) > at > org.apache.drill.exec.physical.impl.ScanBatch.(ScanBatch.java:128) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) > at > org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131) > at > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174) > at > org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105) > at > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230) > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-4757) Column Name Clash experienced with JDBC
[
https://issues.apache.org/jira/browse/DRILL-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Haug updated DRILL-4757:
-
Priority: Critical (was: Major)
> Column Name Clash experienced with JDBC
> ---
>
> Key: DRILL-4757
> URL: https://issues.apache.org/jira/browse/DRILL-4757
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Affects Versions: 1.6.0, 1.7.0
>Reporter: Till Haug
>Priority: Critical
>
> If there are two columns with the same name in two different tables, there
> seems to be a conflict.
> Example 1:
> select t.emp_no as col_one, d.emp_no as col_two
> from mysqlaws.employees.titles as t, mysqlaws.employees.dept_manager as d
> where t.emp_no = d.emp_no
> Result 1:
> emp_no emp_no0
> 110022 null
> 110022 null
> 110039 null
> ...
> Expected Result 1:
> col_one col_two
> 110022 110022
> 110022 110022
> 110039 110039
> ...
> EXPLAIN PLAN FOR Example 1:
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(col_one=[$0], col_two=[$1])
> 00-02Project(col_one=[$2], col_two=[$6])
> 00-03 Jdbc(sql=[SELECT *
> FROM `employees`.`titles`
> INNER JOIN `employees`.`dept_manager` ON `titles`.`emp_no` =
> `dept_manager`.`emp_no`])
> | {
> "head" : {
>"version" : 1,
>"generator" : {
> "type" : "ExplainHandler",
> "info" : ""
>},
>"type" : "APACHE_DRILL_PHYSICAL",
>"options" : [ ],
>"queue" : 0,
>"resultMode" : "EXEC"
> },
> "graph" : [ {
>"pop" : "jdbc-scan",
>"@id" : 3,
>"sql" : "SELECT *\nFROM `employees`.`titles`\nINNER JOIN
> `employees`.`dept_manager` ON `titles`.`emp_no` = `dept_manager`.`emp_no`",
>"config" : {
> "type" : "jdbc",
> "driver" : "com.mysql.jdbc.Driver",
> "url" :
> "jdbc:mysql://vz-test.cbnbj0e1vrwg.eu-central-1.rds.amazonaws.com:8008",
> "username" : "vz_master",
> "password" : "vzpassword",
> "enabled" : true
>},
>"userName" : "",
>"cost" : 100.0
> }, {
>"pop" : "project",
>"@id" : 2,
>"exprs" : [ {
> "ref" : "`col_one`",
> "expr" : "`from_date`"
>}, {
> "ref" : "`col_two`",
> "expr" : "`from_date0`"
>} ],
>"child" : 3,
>"initialAllocation" : 100,
>"maxAllocation" : 100,
>"cost" : 100.0
> }, {
>"pop" : "project",
>"@id" : 1,
>"exprs" : [ {
> "ref" : "`col_one`",
> "expr" : "`col_one`"
>}, {
> "ref" : "`col_two`",
> "expr" : "`col_two`"
>} ],
>"child" : 2,
>"initialAllocation" : 100,
>"maxAllocation" : 100,
>"cost" : 100.0
> }, {
>"pop" : "screen",
>"@id" : 0,
>"child" : 1,
>"initialAllocation" : 100,
>"maxAllocation" : 100,
>"cost" : 100.0
> } ]
> }
> Example 2:
> select t.from_date as col_one, d.from_date as col_two
> from mysqlaws.employees.titles as t, mysqlaws.employees.dept_manager as d
> where t.emp_no = d.emp_no
> Result 2:
> col_one col_two
> 1985-01-01 null
> 1985-01-01 null
> 1991-10-01 null
> ...
> Expected Result 2:
> col_one col_two
> 1985-01-011985-01-01
> 1991-10-011985-01-01
> 1991-10-011991-10-01
> ...
> In Example 1 there is no rename happening and the col_two is all nulls.
> In Example 2 the rename is happening, but the col_two is still all nulls.
> When we run these queries directly against the databases (both mssql and
> mysql) they work as expected.
> If you'd like to directly reproduce it, feel free to use our server we set up
> with the following storage plugin
> {
> "type": "jdbc",
> "driver": "com.mysql.jdbc.Driver",
> "url":
> "jdbc:mysql://vz-test.cbnbj0e1vrwg.eu-central-1.rds.amazonaws.com:8008",
> "username": "vz_master",
> "password": "vzpassword",
> "enabled": false
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4757) Column Name Clash experienced with JDBC
Till Haug created DRILL-4757:
Summary: Column Name Clash experienced with JDBC
Key: DRILL-4757
URL: https://issues.apache.org/jira/browse/DRILL-4757
Project: Apache Drill
Issue Type: Bug
Components: Storage - JDBC
Affects Versions: 1.6.0, 1.7.0
Reporter: Till Haug
If there are two columns with the same name in two different tables, there
seems to be a conflict.
Example 1:
select t.emp_no as col_one, d.emp_no as col_two
from mysqlaws.employees.titles as t, mysqlaws.employees.dept_manager as d
where t.emp_no = d.emp_no
Result 1:
emp_no emp_no0
110022 null
110022 null
110039 null
...
Expected Result 1:
col_one col_two
110022 110022
110022 110022
110039 110039
...
EXPLAIN PLAN FOR Example 1:
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(col_one=[$0], col_two=[$1])
00-02Project(col_one=[$2], col_two=[$6])
00-03 Jdbc(sql=[SELECT *
FROM `employees`.`titles`
INNER JOIN `employees`.`dept_manager` ON `titles`.`emp_no` =
`dept_manager`.`emp_no`])
| {
"head" : {
"version" : 1,
"generator" : {
"type" : "ExplainHandler",
"info" : ""
},
"type" : "APACHE_DRILL_PHYSICAL",
"options" : [ ],
"queue" : 0,
"resultMode" : "EXEC"
},
"graph" : [ {
"pop" : "jdbc-scan",
"@id" : 3,
"sql" : "SELECT *\nFROM `employees`.`titles`\nINNER JOIN
`employees`.`dept_manager` ON `titles`.`emp_no` = `dept_manager`.`emp_no`",
"config" : {
"type" : "jdbc",
"driver" : "com.mysql.jdbc.Driver",
"url" :
"jdbc:mysql://vz-test.cbnbj0e1vrwg.eu-central-1.rds.amazonaws.com:8008",
"username" : "vz_master",
"password" : "vzpassword",
"enabled" : true
},
"userName" : "",
"cost" : 100.0
}, {
"pop" : "project",
"@id" : 2,
"exprs" : [ {
"ref" : "`col_one`",
"expr" : "`from_date`"
}, {
"ref" : "`col_two`",
"expr" : "`from_date0`"
} ],
"child" : 3,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 100.0
}, {
"pop" : "project",
"@id" : 1,
"exprs" : [ {
"ref" : "`col_one`",
"expr" : "`col_one`"
}, {
"ref" : "`col_two`",
"expr" : "`col_two`"
} ],
"child" : 2,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 100.0
}, {
"pop" : "screen",
"@id" : 0,
"child" : 1,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 100.0
} ]
}
Example 2:
select t.from_date as col_one, d.from_date as col_two
from mysqlaws.employees.titles as t, mysqlaws.employees.dept_manager as d
where t.emp_no = d.emp_no
Result 2:
col_one col_two
1985-01-01 null
1985-01-01 null
1991-10-01 null
...
Expected Result 2:
col_one col_two
1985-01-011985-01-01
1991-10-011985-01-01
1991-10-011991-10-01
...
In Example 1 there is no rename happening and the col_two is all nulls.
In Example 2 the rename is happening, but the col_two is still all nulls.
When we run these queries directly against the databases (both mssql and mysql)
they work as expected.
If you'd like to directly reproduce it, feel free to use our server we set up
with the following storage plugin
{
"type": "jdbc",
"driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://vz-test.cbnbj0e1vrwg.eu-central-1.rds.amazonaws.com:8008",
"username": "vz_master",
"password": "vzpassword",
"enabled": false
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4581) Various problems in the Drill startup scripts
[
https://issues.apache.org/jira/browse/DRILL-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15356590#comment-15356590
]
ASF GitHub Bot commented on DRILL-4581:
---
Github user shishkin commented on a diff in the pull request:
https://github.com/apache/drill/pull/478#discussion_r69077287
--- Diff: distribution/src/resources/runbit ---
@@ -15,11 +15,41 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-bin=`dirname "${BASH_SOURCE-$0}"`
-bin=`cd "$bin">/dev/null; pwd`
+# Invoke Drill using Java. Command line arguments are assumed to be
+# of the form
+# -Dname=value
+# and are passed to Java as JVM arguments.
+#
+# Environment Variables
+#
+# DRILLBIT_LOG_PATH Path to the Drillbit log file.
+# (Set in drillbit.sh)
+# DRILLBIT_QUERY_LOG_PATH Path the the query log file.
+# (Set in drillbit.sh)
+# DRILL_JAVA_OPTS Optional JVM arguments such as system
+# property overides used by both the
+# drillbit (server) and client,
+# set in drill-env.sh or the environment.
+# DRILLBIT_JAVA_OPTS Optional JVM arguments specifically for
+# the server (drillbit). Set in the
+# environment or in the user defined
+# drill-env.sh
+# SERVER_GC_OPTS Set in drill-env.sh, customized by
+# drillbit.sh.
+# CP Drillbit classpath set in drillbit.sh
+
+#bin=`dirname "${BASH_SOURCE-$0}"`
+#bin=`cd "$bin">/dev/null; pwd`
+
+LOG_OPTS="-Dlog.path=$DRILLBIT_LOG_PATH
-Dlog.query.path=$DRILLBIT_QUERY_LOG_PATH"
+DRILL_ALL_JAVA_OPTS="$DRILL_JAVA_OPTS $DRILLBIT_JAVA_OPTS $SERVER_GC_OPTS
$@ $LOG_OPTS"
--- End diff --
Glad to see `$@` in there. Finally there would be an option to override
memory settings from the command line rather than through files. This will
simplify Drill deployment on Mesos with Marathon.
> Various problems in the Drill startup scripts
> -
>
> Key: DRILL-4581
> URL: https://issues.apache.org/jira/browse/DRILL-4581
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Server
>Affects Versions: 1.6.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
>
> Noticed the following in drillbit.sh:
> 1) Comment: DRILL_LOG_DIRWhere log files are stored. PWD by default.
> Code: DRILL_LOG_DIR=/var/log/drill or, if it does not exist, $DRILL_HOME/log
> 2) Comment: DRILL_PID_DIRThe pid files are stored. /tmp by default.
> Code: DRILL_PID_DIR=$DRILL_HOME
> 3) Redundant checking of JAVA_HOME. drillbit.sh sources drill-config.sh which
> checks JAVA_HOME. Later, drillbit.sh checks it again. The second check is
> both unnecessary and prints a less informative message than the
> drill-config.sh check. Suggestion: Remove the JAVA_HOME check in drillbit.sh.
> 4) Though drill-config.sh carefully checks JAVA_HOME, it does not export the
> JAVA_HOME variable. Perhaps this is why drillbit.sh repeats the check?
> Recommended: export JAVA_HOME from drill-config.sh.
> 5) Both drillbit.sh and the sourced drill-config.sh check DRILL_LOG_DIR and
> set the default value. Drill-config.sh defaults to /var/log/drill, or if that
> fails, to $DRILL_HOME/log. Drillbit.sh just sets /var/log/drill and does not
> handle the case where that directory is not writable. Suggested: remove the
> check in drillbit.sh.
> 6) Drill-config.sh checks the writability of the DRILL_LOG_DIR by touching
> sqlline.log, but does not delete that file, leaving a bogus, empty client log
> file on the drillbit server. Recommendation: use bash commands instead.
> 7) The implementation of the above check is a bit awkward. It has a fallback
> case with somewhat awkward logic. Clean this up.
> 8) drillbit.sh, but not drill-config.sh, attempts to create /var/log/drill if
> it does not exist. Recommended: decide on a single choice, implement it in
> drill-config.sh.
> 9) drill-config.sh checks if $DRILL_CONF_DIR is a directory. If not, defaults
> it to $DRILL_HOME/conf. This can lead to subtle errors. If I use
> drillbit.sh --config /misspelled/path
> where I mistype the path, I won't get an error, I get the default config,
> which may not at all be what I want to run. Recommendation: if the value of
> DRILL_CONF_DRILL is passed into the script (as a variable or via --config),
> then that directory must exist. Else, use the default.
> 10) drill-config.sh exports, but may not set, HADOOP_HOME. This may be left
> over
