[jira] [Assigned] (DRILL-7244) Run-time rowgroup pruning match() fails on casting a Long to an Integer
[
https://issues.apache.org/jira/browse/DRILL-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati reassigned DRILL-7244:
Assignee: Venkata Jyothsna Donapati (was: Boaz Ben-Zvi)
> Run-time rowgroup pruning match() fails on casting a Long to an Integer
> ---
>
> Key: DRILL-7244
> URL: https://issues.apache.org/jira/browse/DRILL-7244
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Storage - Parquet
>Affects Versions: 1.17.0
>Reporter: Boaz Ben-Zvi
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.17.0
>
>
> See DRILL-7240 , where a temporary workaround was created, skipping pruning
> (and logging) instead of this failure:
> After a Parquet table is refreshed with selected "interesting" columns, a
> query whose WHERE clause contains a condition on a "non interesting" INT64
> column fails during run-time pruning (calling match()) with:
> {noformat}
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
> {noformat}
> A long term solution is to pass the whole (or the relevant part of the)
> schema to the runtime, instead of just passing the "interesting" columns.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7238) Drill does not use DirectScan for non-existent columns
[
https://issues.apache.org/jira/browse/DRILL-7238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-7238:
-
Labels: ready-to-commit (was: )
> Drill does not use DirectScan for non-existent columns
> --
>
> Key: DRILL-7238
> URL: https://issues.apache.org/jira/browse/DRILL-7238
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Venkata Jyothsna Donapati
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.17.0
>
>
> This query does not use the summary metadata cache file:
> select count(int_nulls_id), count(int_id), count(ss_ticket_number),
> count(extra) from store_sales_null_blocks_int;
> In this query, extra is a column that does not exist (non-existent column).
> Here is the explain plan:
> {noformat}
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
> 00-02StreamAgg(group=[{}], EXPR$0=[$SUM0($0)], EXPR$1=[$SUM0($1)],
> EXPR$2=[$SUM0($2)], EXPR$3=[$SUM0($3)])
> 00-03 UnionExchange
> 01-01StreamAgg(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[COUNT($1)], EXPR$2=[COUNT($2)], EXPR$3=[COUNT($3)])
> 01-02 Scan(table=[[dfs, parquet_metadata_cache,
> store_sales_null_blocks_int]], groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=/drill/testdata/metadata_cache/store_sales_null_blocks_int]],
> selectionRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> numFiles=1, numRowGroups=11, usedMetadataFile=true,
> cacheFileRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> columns=[`int_nulls_id`, `int_id`, `ss_ticket_number`, `extra`]]])
> {noformat}
> This is a regression from Drill 1.15.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7238) Drill does not use DirectScan for non-existent columns
[
https://issues.apache.org/jira/browse/DRILL-7238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-7238:
-
Fix Version/s: 1.17.0
> Drill does not use DirectScan for non-existent columns
> --
>
> Key: DRILL-7238
> URL: https://issues.apache.org/jira/browse/DRILL-7238
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Venkata Jyothsna Donapati
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.17.0
>
>
> This query does not use the summary metadata cache file:
> select count(int_nulls_id), count(int_id), count(ss_ticket_number),
> count(extra) from store_sales_null_blocks_int;
> In this query, extra is a column that does not exist (non-existent column).
> Here is the explain plan:
> {noformat}
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
> 00-02StreamAgg(group=[{}], EXPR$0=[$SUM0($0)], EXPR$1=[$SUM0($1)],
> EXPR$2=[$SUM0($2)], EXPR$3=[$SUM0($3)])
> 00-03 UnionExchange
> 01-01StreamAgg(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[COUNT($1)], EXPR$2=[COUNT($2)], EXPR$3=[COUNT($3)])
> 01-02 Scan(table=[[dfs, parquet_metadata_cache,
> store_sales_null_blocks_int]], groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=/drill/testdata/metadata_cache/store_sales_null_blocks_int]],
> selectionRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> numFiles=1, numRowGroups=11, usedMetadataFile=true,
> cacheFileRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> columns=[`int_nulls_id`, `int_id`, `ss_ticket_number`, `extra`]]])
> {noformat}
> This is a regression from Drill 1.15.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7238) Drill does not use DirectScan for non-existent columns
[
https://issues.apache.org/jira/browse/DRILL-7238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-7238:
-
Component/s: Metadata
> Drill does not use DirectScan for non-existent columns
> --
>
> Key: DRILL-7238
> URL: https://issues.apache.org/jira/browse/DRILL-7238
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Venkata Jyothsna Donapati
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
>
> This query does not use the summary metadata cache file:
> select count(int_nulls_id), count(int_id), count(ss_ticket_number),
> count(extra) from store_sales_null_blocks_int;
> In this query, extra is a column that does not exist (non-existent column).
> Here is the explain plan:
> {noformat}
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
> 00-02StreamAgg(group=[{}], EXPR$0=[$SUM0($0)], EXPR$1=[$SUM0($1)],
> EXPR$2=[$SUM0($2)], EXPR$3=[$SUM0($3)])
> 00-03 UnionExchange
> 01-01StreamAgg(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[COUNT($1)], EXPR$2=[COUNT($2)], EXPR$3=[COUNT($3)])
> 01-02 Scan(table=[[dfs, parquet_metadata_cache,
> store_sales_null_blocks_int]], groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=/drill/testdata/metadata_cache/store_sales_null_blocks_int]],
> selectionRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> numFiles=1, numRowGroups=11, usedMetadataFile=true,
> cacheFileRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
> columns=[`int_nulls_id`, `int_id`, `ss_ticket_number`, `extra`]]])
> {noformat}
> This is a regression from Drill 1.15.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7199) Optimize the time taken to populate column statistics for non-interesting columns
[ https://issues.apache.org/jira/browse/DRILL-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7199: - Labels: ready-to-commit (was: ) > Optimize the time taken to populate column statistics for non-interesting > columns > - > > Key: DRILL-7199 > URL: https://issues.apache.org/jira/browse/DRILL-7199 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Labels: ready-to-commit > Fix For: 1.17.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > Currently populating column statistics for non-interesting columns very long > since it is populated for every row group. Since non-interesting column > statistics are common for the table, it can be populated once and can be > reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7238) Drill does not use DirectScan for non-existent columns
Venkata Jyothsna Donapati created DRILL-7238:
Summary: Drill does not use DirectScan for non-existent columns
Key: DRILL-7238
URL: https://issues.apache.org/jira/browse/DRILL-7238
Project: Apache Drill
Issue Type: Bug
Reporter: Venkata Jyothsna Donapati
Assignee: Venkata Jyothsna Donapati
This query does not use the summary metadata cache file:
select count(int_nulls_id), count(int_id), count(ss_ticket_number),
count(extra) from store_sales_null_blocks_int;
In this query, extra is a column that does not exist (non-existent column).
Here is the explain plan:
{noformat}
| 00-00Screen
00-01 Project(EXPR$0=[$0], EXPR$1=[$1], EXPR$2=[$2], EXPR$3=[$3])
00-02StreamAgg(group=[{}], EXPR$0=[$SUM0($0)], EXPR$1=[$SUM0($1)],
EXPR$2=[$SUM0($2)], EXPR$3=[$SUM0($3)])
00-03 UnionExchange
01-01StreamAgg(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[COUNT($1)],
EXPR$2=[COUNT($2)], EXPR$3=[COUNT($3)])
01-02 Scan(table=[[dfs, parquet_metadata_cache,
store_sales_null_blocks_int]], groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=/drill/testdata/metadata_cache/store_sales_null_blocks_int]],
selectionRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
numFiles=1, numRowGroups=11, usedMetadataFile=true,
cacheFileRoot=/drill/testdata/metadata_cache/store_sales_null_blocks_int,
columns=[`int_nulls_id`, `int_id`, `ss_ticket_number`, `extra`]]])
{noformat}
This is a regression from Drill 1.15.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7225) Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata
[ https://issues.apache.org/jira/browse/DRILL-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829859#comment-16829859 ] Venkata Jyothsna Donapati commented on DRILL-7225: -- [~amansinha100] Case-2 results in NPE. Case-1 works fine which means there won't be any problem when two sub-directories under a directory have different schemas. > Merging of columnTypeInfo for file with different schema throws > NullPointerException during refresh metadata > > > Key: DRILL-7225 > URL: https://issues.apache.org/jira/browse/DRILL-7225 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > > Merging of columnTypeInfo from two files with different schemas throws > nullpointerexception. For example if a directory Orders has two files: > * orders.parquet (with columns order_id, order_name, order_date) > * orders_with_address.parquet (with columns order_id, order_name, address) > When refresh table metadata is triggered, metadata such as total_null_count > for columns in both the files is aggregated and updated in the > ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's > ColumnTypeInfo (i.e., order_id, order_name, order_date). While aggregating, > the existing ColumnTypeInfo is looked up for columns in the second file and > since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This > can be fixed by initializing ColumnTypeInfo for columns that are not yet > present. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7225) Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata
[ https://issues.apache.org/jira/browse/DRILL-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7225: - Description: Merging of columnTypeInfo from two files with different schemas throws nullpointerexception. For example if a directory Orders has two files: * orders.parquet (with columns order_id, order_name, order_date) * orders_with_address.parquet (with columns order_id, order_name, address) When refresh table metadata is triggered, metadata such as total_null_count for columns in both the files is aggregated and updated in the ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's ColumnTypeInfo (i.e., order_id, order_name, order_date). While aggregating, the existing ColumnTypeInfo is looked up for columns in the second file and since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This can be fixed by initializing ColumnTypeInfo for columns that are not yet present. was: Merging of columnTypeInfo from two files with different schemas throws nullpointerexception. For example if a directory Orders has two files: * orders.parquet (with columns order_id, order_name, order_date) * orders_with_address.parquet (with columns order_id, order_name, address) When refresh table metadata is triggered, metadata such as total_null_count for columns in both the files is aggregated and updated in the ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's ColumnTypeInfo. While aggregating, the existing ColumnTypeInfo is looked up for columns in the second file and since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This can be fixed by initializing ColumnTypeInfo for columns that are not yet present. > Merging of columnTypeInfo for file with different schema throws > NullPointerException during refresh metadata > > > Key: DRILL-7225 > URL: https://issues.apache.org/jira/browse/DRILL-7225 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > > Merging of columnTypeInfo from two files with different schemas throws > nullpointerexception. For example if a directory Orders has two files: > * orders.parquet (with columns order_id, order_name, order_date) > * orders_with_address.parquet (with columns order_id, order_name, address) > When refresh table metadata is triggered, metadata such as total_null_count > for columns in both the files is aggregated and updated in the > ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's > ColumnTypeInfo (i.e., order_id, order_name, order_date). While aggregating, > the existing ColumnTypeInfo is looked up for columns in the second file and > since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This > can be fixed by initializing ColumnTypeInfo for columns that are not yet > present. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7225) Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata
[ https://issues.apache.org/jira/browse/DRILL-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7225: - Description: Merging of columnTypeInfo from two files with different schemas throws nullpointerexception. For example if a directory Orders has two files: * orders.parquet (with columns order_id, order_name, order_date) * orders_with_address.parquet (with columns order_id, order_name, address) When refresh table metadata is triggered, metadata such as total_null_count for columns in both the files is aggregated and updated in the ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's ColumnTypeInfo. While aggregating, the existing ColumnTypeInfo is looked up for columns in the second file and since some of them don't exist in the ColumnTypeInfo, a npe is thrown. This can be fixed by initializing ColumnTypeInfo for columns that are not yet present. was:Merging of columnTypeInfo from two files with different schemas throws nullpointerexception > Merging of columnTypeInfo for file with different schema throws > NullPointerException during refresh metadata > > > Key: DRILL-7225 > URL: https://issues.apache.org/jira/browse/DRILL-7225 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > > Merging of columnTypeInfo from two files with different schemas throws > nullpointerexception. For example if a directory Orders has two files: > * orders.parquet (with columns order_id, order_name, order_date) > * orders_with_address.parquet (with columns order_id, order_name, address) > When refresh table metadata is triggered, metadata such as total_null_count > for columns in both the files is aggregated and updated in the > ColumnTypeInfo. Initially ColumnTypeInfo is initialized with the first file's > ColumnTypeInfo. While aggregating, the existing ColumnTypeInfo is looked up > for columns in the second file and since some of them don't exist in the > ColumnTypeInfo, a npe is thrown. This can be fixed by initializing > ColumnTypeInfo for columns that are not yet present. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7225) Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata
[ https://issues.apache.org/jira/browse/DRILL-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7225: - Description: Merging of columnTypeInfo from two files with different schemas throws nullpointerexception > Merging of columnTypeInfo for file with different schema throws > NullPointerException during refresh metadata > > > Key: DRILL-7225 > URL: https://issues.apache.org/jira/browse/DRILL-7225 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > > Merging of columnTypeInfo from two files with different schemas throws > nullpointerexception -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7225) Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata
Venkata Jyothsna Donapati created DRILL-7225: Summary: Merging of columnTypeInfo for file with different schema throws NullPointerException during refresh metadata Key: DRILL-7225 URL: https://issues.apache.org/jira/browse/DRILL-7225 Project: Apache Drill Issue Type: Bug Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7199) Optimize the time taken to populate column statistics for non-interesting columns
[ https://issues.apache.org/jira/browse/DRILL-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7199: - Description: Currently populating column statistics for non-interesting columns very long since it is populated for every row group. Since non-interesting column statistics are common for the table, it can be populated once and can be reused. (was: Currently populating column statistics for non-existent columns very long since it is populated for every row group. Since non-existent column statistics are common for the table, it can be populated once and can be reused.) > Optimize the time taken to populate column statistics for non-interesting > columns > - > > Key: DRILL-7199 > URL: https://issues.apache.org/jira/browse/DRILL-7199 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.17.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > Currently populating column statistics for non-interesting columns very long > since it is populated for every row group. Since non-interesting column > statistics are common for the table, it can be populated once and can be > reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7199) Optimize the time taken to populate column statistics for non-interesting columns
[ https://issues.apache.org/jira/browse/DRILL-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7199: - Fix Version/s: 1.17.0 > Optimize the time taken to populate column statistics for non-interesting > columns > - > > Key: DRILL-7199 > URL: https://issues.apache.org/jira/browse/DRILL-7199 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.17.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > Currently populating column statistics for non-existent columns very long > since it is populated for every row group. Since non-existent column > statistics are common for the table, it can be populated once and can be > reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7199) Optimize the time taken to populate column statistics for non-interesting columns
Venkata Jyothsna Donapati created DRILL-7199: Summary: Optimize the time taken to populate column statistics for non-interesting columns Key: DRILL-7199 URL: https://issues.apache.org/jira/browse/DRILL-7199 Project: Apache Drill Issue Type: Bug Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Currently populating column statistics for non-existent columns very long since it is populated for every row group. Since non-existent column statistics are common for the table, it can be populated once and can be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-7037) Apache Drill Crashes when a 50mb json string is queried via the REST API provided
[ https://issues.apache.org/jira/browse/DRILL-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816675#comment-16816675 ] Venkata Jyothsna Donapati edited comment on DRILL-7037 at 4/12/19 9:33 PM: --- [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [https://drill.apache.org/docs/create-temporary-table-as-cttas/] was (Author: vdonapati): [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/]] > Apache Drill Crashes when a 50mb json string is queried via the REST API > provided > - > > Key: DRILL-7037 > URL: https://issues.apache.org/jira/browse/DRILL-7037 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 1.14.0 > Environment: Windows 10 > 24GB RAM > 8 Cores > Used the REST API call to query drill >Reporter: Ayush Sharma >Priority: Blocker > > Apache Drill crashes with OutofMemoryException (24GB RAM) when a REST API > call is made by supplying a json of size 50MB in the query paramater of the > REST API. > The REST API even crashes for a 10MB query (16GB RAM) and works with a 5MB > query. > This is a blocker for us and will need immediate remediation. > We are also not aware of any sys.options which might bring the HEAP size down > drastically or currently making it go up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-7037) Apache Drill Crashes when a 50mb json string is queried via the REST API provided
[ https://issues.apache.org/jira/browse/DRILL-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816675#comment-16816675 ] Venkata Jyothsna Donapati edited comment on DRILL-7037 at 4/12/19 9:33 PM: --- [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/]] was (Author: vdonapati): [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/] []|http://example.com]/] > Apache Drill Crashes when a 50mb json string is queried via the REST API > provided > - > > Key: DRILL-7037 > URL: https://issues.apache.org/jira/browse/DRILL-7037 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 1.14.0 > Environment: Windows 10 > 24GB RAM > 8 Cores > Used the REST API call to query drill >Reporter: Ayush Sharma >Priority: Blocker > > Apache Drill crashes with OutofMemoryException (24GB RAM) when a REST API > call is made by supplying a json of size 50MB in the query paramater of the > REST API. > The REST API even crashes for a 10MB query (16GB RAM) and works with a 5MB > query. > This is a blocker for us and will need immediate remediation. > We are also not aware of any sys.options which might bring the HEAP size down > drastically or currently making it go up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-7037) Apache Drill Crashes when a 50mb json string is queried via the REST API provided
[ https://issues.apache.org/jira/browse/DRILL-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816675#comment-16816675 ] Venkata Jyothsna Donapati edited comment on DRILL-7037 at 4/12/19 9:33 PM: --- [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/] []|http://example.com]/] was (Author: vdonapati): [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/]] > Apache Drill Crashes when a 50mb json string is queried via the REST API > provided > - > > Key: DRILL-7037 > URL: https://issues.apache.org/jira/browse/DRILL-7037 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 1.14.0 > Environment: Windows 10 > 24GB RAM > 8 Cores > Used the REST API call to query drill >Reporter: Ayush Sharma >Priority: Blocker > > Apache Drill crashes with OutofMemoryException (24GB RAM) when a REST API > call is made by supplying a json of size 50MB in the query paramater of the > REST API. > The REST API even crashes for a 10MB query (16GB RAM) and works with a 5MB > query. > This is a blocker for us and will need immediate remediation. > We are also not aware of any sys.options which might bring the HEAP size down > drastically or currently making it go up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7037) Apache Drill Crashes when a 50mb json string is queried via the REST API provided
[ https://issues.apache.org/jira/browse/DRILL-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816675#comment-16816675 ] Venkata Jyothsna Donapati commented on DRILL-7037: -- [~er.ayushsha...@gmail.com] Drill provides a way to create temporary tables. Please take a look at [this|[https://drill.apache.org/docs/create-temporary-table-as-cttas/]] > Apache Drill Crashes when a 50mb json string is queried via the REST API > provided > - > > Key: DRILL-7037 > URL: https://issues.apache.org/jira/browse/DRILL-7037 > Project: Apache Drill > Issue Type: Bug > Components: Client - HTTP >Affects Versions: 1.14.0 > Environment: Windows 10 > 24GB RAM > 8 Cores > Used the REST API call to query drill >Reporter: Ayush Sharma >Priority: Blocker > > Apache Drill crashes with OutofMemoryException (24GB RAM) when a REST API > call is made by supplying a json of size 50MB in the query paramater of the > REST API. > The REST API even crashes for a 10MB query (16GB RAM) and works with a 5MB > query. > This is a blocker for us and will need immediate remediation. > We are also not aware of any sys.options which might bring the HEAP size down > drastically or currently making it go up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7063) Create separate summary file for schema, totalRowCount, totalNullCount (includes maintenance)
[ https://issues.apache.org/jira/browse/DRILL-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816546#comment-16816546 ] Venkata Jyothsna Donapati commented on DRILL-7063: -- [~vvysotskyi] I had a discussion with [~bbevens] and she is already working on adding the new commands and user behavior to the Drill documentation. > Create separate summary file for schema, totalRowCount, totalNullCount > (includes maintenance) > - > > Key: DRILL-7063 > URL: https://issues.apache.org/jira/browse/DRILL-7063 > Project: Apache Drill > Issue Type: Sub-task > Components: Metadata >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.16.0 > > Original Estimate: 252h > Remaining Estimate: 252h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7171) Count(*) query on leaf level directory is not reading summary cache file.
Venkata Jyothsna Donapati created DRILL-7171: Summary: Count(*) query on leaf level directory is not reading summary cache file. Key: DRILL-7171 URL: https://issues.apache.org/jira/browse/DRILL-7171 Project: Apache Drill Issue Type: Bug Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Since the leaf level directory doesn't store the metadata directories file, while reading summary if the directories cache file is not present, it is assumed that the cache is possibly corrupt and reading of the summary cache file is skipped. Metadata directories cache file should be created at the leaf level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-7166) Tests doing count(* ) with wildcards in table name are querying metadata cache and returning wrong results
[
https://issues.apache.org/jira/browse/DRILL-7166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati reassigned DRILL-7166:
Assignee: Venkata Jyothsna Donapati (was: Pritesh Maker)
> Tests doing count(* ) with wildcards in table name are querying metadata
> cache and returning wrong results
> --
>
> Key: DRILL-7166
> URL: https://issues.apache.org/jira/browse/DRILL-7166
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Affects Versions: 1.16.0
>Reporter: Abhishek Girish
>Assignee: Venkata Jyothsna Donapati
>Priority: Blocker
> Fix For: 1.16.0
>
>
> Tests:
> {code}
> Functional/metadata_caching/data/drill4376_1.q
> Functional/metadata_caching/data/drill4376_2.q
> Functional/metadata_caching/data/drill4376_3.q
> Functional/metadata_caching/data/drill4376_4.q
> Functional/metadata_caching/data/drill4376_5.q
> Functional/metadata_caching/data/drill4376_6.q
> Functional/metadata_caching/data/drill4376_8.q
> {code}
> Example pattern of queries:
> {code}
> select count(*) from `lineitem_hierarchical_intint/*8*/3*`;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7058) Refresh command to support subset of columns
[ https://issues.apache.org/jira/browse/DRILL-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7058: - Description: Modify the REFRESH TABLE METADATA command to specify selected columns which are deemed interesting in some form - either sorted/partitioned/clustered by and column metadata will be stored only for those columns. The proposed syntax is REFRESH TABLE METADATA *_[ COLUMNS (list of columns) / NONE ]_* For example, REFRESH TABLE METADATA COLUMNS (age, salary) `/tmp/employee` stores column metadata only for the age and salary columns. REFRESH TABLE METADATA COLUMNS NONE `/tmp/employee` will not store column metadata for any column. By default, if the optional 'COLUMNS' clause is omitted, column metadata is collected for all the columns. was: Modify the REFRESH TABLE METADATA command to specify selected columns which are deemed interesting in some form - either sorted/partitioned/clustered by and column metadata will be stored only for those columns. The proposed syntax is REFRESH TABLE METADATA *_[ COLUMNS (list of columns) / NONE ]_* For example, REFRESH TABLE METADATA COLUMNS (age, salary) `/tmp/employee` stores column metadata only for the age and salary columns. By default, if the optional 'COLUMNS' clause is omitted, column metadata is collected for all the columns. > Refresh command to support subset of columns > > > Key: DRILL-7058 > URL: https://issues.apache.org/jira/browse/DRILL-7058 > Project: Apache Drill > Issue Type: Sub-task > Components: Metadata >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Labels: doc-impacting > Fix For: 1.16.0 > > Original Estimate: 72h > Remaining Estimate: 72h > > Modify the REFRESH TABLE METADATA command to specify selected columns which > are deemed interesting in some form - either sorted/partitioned/clustered by > and column metadata will be stored only for those columns. The proposed > syntax is > REFRESH TABLE METADATA *_[ COLUMNS (list of columns) / NONE ]_* path> > For example, REFRESH TABLE METADATA COLUMNS (age, salary) `/tmp/employee` > stores column metadata only for the age and salary columns. REFRESH TABLE > METADATA COLUMNS NONE `/tmp/employee` will not store column metadata for any > column. > By default, if the optional 'COLUMNS' clause is omitted, column metadata is > collected for all the columns. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7058) Refresh command to support subset of columns
[ https://issues.apache.org/jira/browse/DRILL-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7058: - Description: Modify the REFRESH TABLE METADATA command to specify selected columns which are deemed interesting in some form - either sorted/partitioned/clustered by and column metadata will be stored only for those columns. The proposed syntax is REFRESH TABLE METADATA *_[ COLUMNS (list of columns) / NONE ]_* For example, REFRESH TABLE METADATA COLUMNS (age, salary) `/tmp/employee` stores column metadata only for the age and salary columns. By default, if the optional 'COLUMNS' clause is omitted, column metadata is collected for all the columns. > Refresh command to support subset of columns > > > Key: DRILL-7058 > URL: https://issues.apache.org/jira/browse/DRILL-7058 > Project: Apache Drill > Issue Type: Sub-task > Components: Metadata >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Labels: doc-impacting > Fix For: 1.16.0 > > Original Estimate: 72h > Remaining Estimate: 72h > > Modify the REFRESH TABLE METADATA command to specify selected columns which > are deemed interesting in some form - either sorted/partitioned/clustered by > and column metadata will be stored only for those columns. The proposed > syntax is > REFRESH TABLE METADATA *_[ COLUMNS (list of columns) / NONE ]_* path> > For example, REFRESH TABLE METADATA COLUMNS (age, salary) `/tmp/employee` > stores column metadata only for the age and salary columns. > By default, if the optional 'COLUMNS' clause is omitted, column metadata is > collected for all the columns. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7058) Refresh command to support subset of columns
[ https://issues.apache.org/jira/browse/DRILL-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7058: - Reviewer: Gautam Parai > Refresh command to support subset of columns > > > Key: DRILL-7058 > URL: https://issues.apache.org/jira/browse/DRILL-7058 > Project: Apache Drill > Issue Type: Sub-task > Components: Metadata >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.16.0 > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7063) Create separate summary file for schema, totalRowCount, totalNullCount (includes maintenance)
Venkata Jyothsna Donapati created DRILL-7063: Summary: Create separate summary file for schema, totalRowCount, totalNullCount (includes maintenance) Key: DRILL-7063 URL: https://issues.apache.org/jira/browse/DRILL-7063 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7066) Auto-refresh should pick up existing columns from metadata cache
Venkata Jyothsna Donapati created DRILL-7066: Summary: Auto-refresh should pick up existing columns from metadata cache Key: DRILL-7066 URL: https://issues.apache.org/jira/browse/DRILL-7066 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7062) Run-time row group pruning
Venkata Jyothsna Donapati created DRILL-7062: Summary: Run-time row group pruning Key: DRILL-7062 URL: https://issues.apache.org/jira/browse/DRILL-7062 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Boaz Ben-Zvi Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7058) Refresh command to support subset of columns
[ https://issues.apache.org/jira/browse/DRILL-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-7058: - Issue Type: Sub-task (was: New Feature) Parent: DRILL-7028 > Refresh command to support subset of columns > > > Key: DRILL-7058 > URL: https://issues.apache.org/jira/browse/DRILL-7058 > Project: Apache Drill > Issue Type: Sub-task > Components: Metadata >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.16.0 > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7065) Ensure backward compatibility is maintained
Venkata Jyothsna Donapati created DRILL-7065: Summary: Ensure backward compatibility is maintained Key: DRILL-7065 URL: https://issues.apache.org/jira/browse/DRILL-7065 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7064) Leverage the summary's totalRowCount and totalNullCount for COUNT() queries (also prevent eager expansion of files)
Venkata Jyothsna Donapati created DRILL-7064: Summary: Leverage the summary's totalRowCount and totalNullCount for COUNT() queries (also prevent eager expansion of files) Key: DRILL-7064 URL: https://issues.apache.org/jira/browse/DRILL-7064 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6695) Graceful shutdown removes spill directory before query finished
[ https://issues.apache.org/jira/browse/DRILL-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6695: - Fix Version/s: (was: 1.16.0) 1.17.0 > Graceful shutdown removes spill directory before query finished > > > Key: DRILL-6695 > URL: https://issues.apache.org/jira/browse/DRILL-6695 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.14.0 >Reporter: Krystal >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.17.0 > > Attachments: drillbit.log > > > Ran the following query from sqlline: > select a.columns[0], b.columns[1], a.columns[2], b.columns[3], a.columns[4] > from `test` a join `test` b on a.columns[0]=b.columns[0] and > a.columns[4]=b.columns[4] order by a.columns[0] limit 1000; > While the query was running, initiated a graceful shutdown from command line > on the foreman node. The query failed with the following error message: > Error: RESOURCE ERROR: Hash Join failed to open spill file: > /tmp/drill/spill/248a054a-ee63-e795-a44e-d9205df8e9b8_HashJoin_3-2-0/spill7_outer > Fragment 3:0 > Looks like somehow the spill directory gets deleted while query is still > running when graceful_shutdown is initiated. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7058) Refresh command to support subset of columns
Venkata Jyothsna Donapati created DRILL-7058: Summary: Refresh command to support subset of columns Key: DRILL-7058 URL: https://issues.apache.org/jira/browse/DRILL-7058 Project: Apache Drill Issue Type: New Feature Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7028) Reduce the planning time of queries on large Parquet tables with large metadata cache files
Venkata Jyothsna Donapati created DRILL-7028: Summary: Reduce the planning time of queries on large Parquet tables with large metadata cache files Key: DRILL-7028 URL: https://issues.apache.org/jira/browse/DRILL-7028 Project: Apache Drill Issue Type: Improvement Components: Metadata Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati Fix For: 1.16.0 If the Parquet table has a large number of small files, the metadata cache files grow larger and the planner tries to read the large metadata cache file which leads to the planning time overhead. Most of the time of execution is spent during the planning phase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6245: - Fix Version/s: (was: 1.15.0) 1.16.0 > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.16.0 > > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6822) Fix imports for DRILL-6381
[ https://issues.apache.org/jira/browse/DRILL-6822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6822: - Labels: ready-to-commit (was: ) > Fix imports for DRILL-6381 > -- > > Key: DRILL-6822 > URL: https://issues.apache.org/jira/browse/DRILL-6822 > Project: Apache Drill > Issue Type: Task >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Major > Labels: ready-to-commit > Fix For: 1.15.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6822) Fix imports for DRILL-6381
Venkata Jyothsna Donapati created DRILL-6822: Summary: Fix imports for DRILL-6381 Key: DRILL-6822 URL: https://issues.apache.org/jira/browse/DRILL-6822 Project: Apache Drill Issue Type: Task Reporter: Venkata Jyothsna Donapati Assignee: Venkata Jyothsna Donapati -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-5990) Apache Drill /status API returns OK ('Running') even with JRE while queries will not work - make status API reflect the fact that Drill is broken on JRE or stop Drill st
[
https://issues.apache.org/jira/browse/DRILL-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati reassigned DRILL-5990:
Assignee: Kunal Khatua (was: Venkata Jyothsna Donapati)
> Apache Drill /status API returns OK ('Running') even with JRE while queries
> will not work - make status API reflect the fact that Drill is broken on JRE
> or stop Drill starting up with JRE
> ---
>
> Key: DRILL-5990
> URL: https://issues.apache.org/jira/browse/DRILL-5990
> Project: Apache Drill
> Issue Type: Bug
> Components: Server, Client - HTTP, Execution - Monitoring, Query
> Planning Optimization
>Affects Versions: 1.10.0, 1.11.0
> Environment: Docker
>Reporter: Hari Sekhon
>Assignee: Kunal Khatua
>Priority: Major
>
> If running Apache Drill on versions 1.10 / 1.11 on JRE it appears that
> queries no longer run without JDK, but the /status monitoring API endpoint
> does not reflect the fact the Apache Drill will not work and still returns
> 'Running'.
> While 'Running' is technically true the process is up, it's effectively
> broken and Apache Drill should either reflect this in /status that is is
> broken or refuse to start up on JRE altogether.
> See this ticket for more information:
> https://github.com/HariSekhon/Dockerfiles/pull/15
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6415) Unit test TestGracefulShutdown.testRestApiShutdown times out
[
https://issues.apache.org/jira/browse/DRILL-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati reassigned DRILL-6415:
Assignee: Venkata Jyothsna Donapati
> Unit test TestGracefulShutdown.testRestApiShutdown times out
>
>
> Key: DRILL-6415
> URL: https://issues.apache.org/jira/browse/DRILL-6415
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build Test
>Reporter: Abhishek Girish
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
>
> {code}
> 16:03:40.415 [main] ERROR org.apache.drill.TestReporter - Test Failed (d:
> -18.3 KiB(72.9 KiB), h: -335.3 MiB(1.3 GiB), nh: 1.1 MiB(335.9 MiB)):
> testRestApiShutdown(org.apache.drill.test.TestGracefulShutdown)
> org.junit.runners.model.TestTimedOutException: test timed out after 18
> milliseconds
> at sun.misc.Unsafe.park(Native Method) ~[na:1.8.0_161]
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> ~[na:1.8.0_161]
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976)
> ~[na:1.8.0_161]
> at
> org.apache.drill.exec.work.WorkManager.waitToExit(WorkManager.java:203)
> ~[classes/:na]
> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:242)
> ~[classes/:na]
> at
> org.apache.drill.test.ClusterFixture.safeClose(ClusterFixture.java:454)
> ~[test-classes/:1.14.0-SNAPSHOT]
> at org.apache.drill.test.ClusterFixture.close(ClusterFixture.java:405)
> ~[test-classes/:1.14.0-SNAPSHOT]
> at
> org.apache.drill.test.TestGracefulShutdown.testRestApiShutdown(TestGracefulShutdown.java:294)
> ~[test-classes/:1.14.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_161]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_161]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_161]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_161]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> ~[junit-4.12.jar:4.12]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:154)
> ~[jmockit-1.39.jar:1.39]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:70)
> ~[jmockit-1.39.jar:1.39]
> at
> mockit.integration.junit4.internal.FakeFrameworkMethod.invokeExplosively(FakeFrameworkMethod.java:34)
> ~[jmockit-1.39.jar:1.39]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> ~[junit-4.12.jar:4.12]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> ~[junit-4.12.jar:4.12]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
> {code}
> {code}
> testRestApiShutdown(org.apache.drill.test.TestGracefulShutdown) Time
> elapsed: 180.028 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 18
> milliseconds
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976)
> at
> org.apache.drill.exec.work.WorkManager.waitToExit(WorkManager.java:203)
> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:242)
> at
> org.apache.drill.test.ClusterFixture.safeClose(ClusterFixture.java:454)
> at org.apache.drill.test.ClusterFixture.close(ClusterFixture.java:405)
> at
> org.apache.drill.test.TestGracefulShutdown.testRestApiShutdown(TestGracefulShutdown.java:294)
>
[jira] [Updated] (DRILL-6302) NPE in Drillbit.java in close method
[
https://issues.apache.org/jira/browse/DRILL-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6302:
-
Fix Version/s: 1.14.0
> NPE in Drillbit.java in close method
>
>
> Key: DRILL-6302
> URL: https://issues.apache.org/jira/browse/DRILL-6302
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.13.0
> Environment: git.commit.id=bb07ebbb9ba8742f44689f8bd8efb5853c5edea0
>Reporter: Khurram Faraaz
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> registrationHandle in the close method of Drillbit.java (line 228) is null,
> which causes an NPE and as a result the server does not close.
> [https://github.com/mapr/private-drill/blob/drill-1.12.0-mapr/exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java#L228]
> registrationHandle = coord.update(registrationHandle, State.QUIESCENT);
> Stack trace from drillbit.log
> {noformat}
> /opt/mapr/drill/drill-1.12.0/logs/drillbit.out
> {noformat}
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support
> was removed in 8.0
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.drill.exec.coord.zk.ZKClusterCoordinator.update(ZKClusterCoordinator.java:223)
> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:228)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:401)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:372)
> at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:368)
> {noformat}{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (DRILL-6009) No drillbits on index page
[ https://issues.apache.org/jira/browse/DRILL-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati closed DRILL-6009. Resolution: Cannot Reproduce > No drillbits on index page > -- > > Key: DRILL-6009 > URL: https://issues.apache.org/jira/browse/DRILL-6009 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > Attachments: empty_drillbits.JPG > > > After DRILL-4286 once I saw that index page showed no drillbits at all but it > was working, so at least one drillbit was online (empty_drillbits.JPG). After > refresh everything was fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6009) No drillbits on index page
[ https://issues.apache.org/jira/browse/DRILL-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420807#comment-16420807 ] Venkata Jyothsna Donapati commented on DRILL-6009: -- [~arina] I have tried to reproduce this issue and had no luck. Can you please provide some specific scenarios when you ran into this issue? > No drillbits on index page > -- > > Key: DRILL-6009 > URL: https://issues.apache.org/jira/browse/DRILL-6009 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > Attachments: empty_drillbits.JPG > > > After DRILL-4286 once I saw that index page showed no drillbits at all but it > was working, so at least one drillbit was online (empty_drillbits.JPG). After > refresh everything was fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6244) Ability to shutdown cross domain Drillbits
[ https://issues.apache.org/jira/browse/DRILL-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415951#comment-16415951 ] Venkata Jyothsna Donapati commented on DRILL-6244: -- [~kkhatua] > Ability to shutdown cross domain Drillbits > -- > > Key: DRILL-6244 > URL: https://issues.apache.org/jira/browse/DRILL-6244 > Project: Apache Drill > Issue Type: Improvement >Reporter: Venkata Jyothsna Donapati >Priority: Minor > > Currently Drill Web UI supports shutdown of current Drillbit and doesn't > provide an option to shutdown cross domain Drillbits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415907#comment-16415907 ] Venkata Jyothsna Donapati commented on DRILL-6245: -- [~harisekhon] No, This problem arises only when we try to establish a connection using https and then if we try to disable ssl and try to connect using http. Are you able to reproduce the issue otherwise too? > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414207#comment-16414207 ] Venkata Jyothsna Donapati commented on DRILL-6245: -- [~shamirwasia] , [~karthikm], [~parthc] > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414206#comment-16414206 ] Venkata Jyothsna Donapati commented on DRILL-6245: -- Yes, chrome prevents non secure hosts touching secure cookies. So when the https connection is established secure cookie is added. But when we try to establish a http connection after https connection, chrome prevents it from modifying the secure cookie and leads to this issue. Of course, manually deleting the secure cookie is resolving the issue. > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414180#comment-16414180
]
Venkata Jyothsna Donapati commented on DRILL-6286:
--
Please ignore the above pull request
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6286:
-
Comment: was deleted
(was: This is duplicate of DRILL-6261 . It caused by changes made in DRILL-5922.
)
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410493#comment-16410493
]
Venkata Jyothsna Donapati edited comment on DRILL-6286 at 3/22/18 10:24 PM:
This is duplicate of DRILL-6261 . It caused by changes made in DRILL-5922.
was (Author: vdonapati):
This is duplicate of
[DRILL-6261|https://issues.apache.org/jira/browse/DRILL-6261] . It caused by
changes made in DRILL-5292.
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410493#comment-16410493
]
Venkata Jyothsna Donapati edited comment on DRILL-6286 at 3/22/18 10:23 PM:
This is duplicate of
[DRILL-6261|https://issues.apache.org/jira/browse/DRILL-6261] . It caused by
changes made in DRILL-5292.
was (Author: vdonapati):
This is duplicate of
[[DRILL-6261|https://issues.apache.org/jira/browse/DRILL-6261]|https://issues.a/].
It caused by changes made in DRILL-5292.
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410493#comment-16410493
]
Venkata Jyothsna Donapati edited comment on DRILL-6286 at 3/22/18 10:22 PM:
This is duplicate of
[[DRILL-6261|https://issues.apache.org/jira/browse/DRILL-6261]|https://issues.a/].
It caused by changes made in DRILL-5292.
was (Author: vdonapati):
This is duplicate of https://issues.apache.org/jira/browse/DRILL-6261. It
caused by changes made in
[DRILL-5292|https://issues.apache.org/jira/browse/DRILL-5292].
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6286) Regression: incorrect reference to shutdown in drillbit.log
[
https://issues.apache.org/jira/browse/DRILL-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410493#comment-16410493
]
Venkata Jyothsna Donapati commented on DRILL-6286:
--
This is duplicate of https://issues.apache.org/jira/browse/DRILL-6261. It
caused by changes made in
[DRILL-5292|https://issues.apache.org/jira/browse/DRILL-5292].
> Regression: incorrect reference to shutdown in drillbit.log
> ---
>
> Key: DRILL-6286
> URL: https://issues.apache.org/jira/browse/DRILL-6286
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> drillbit.log refers to shutdown even in cases when no shutdown sequence was
> initiated:
> {noformat}
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete
> before shutting down
> 2018-03-16 11:55:52,693 [drill-executor-19] INFO
> o.apache.drill.exec.work.WorkManager - Waiting for 3 running fragments to
> complete before shutting down
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405472#comment-16405472
]
Venkata Jyothsna Donapati edited comment on DRILL-6252 at 3/19/18 9:21 PM:
---
[~vrozov] I have attached corresponding logs.
was (Author: vdonapati):
[~vrozov] Please look for attached logs.
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405472#comment-16405472
]
Venkata Jyothsna Donapati commented on DRILL-6252:
--
[~vrozov] Please look for attached logs.
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6008) Unable to shutdown Drillbit using short domain name
[ https://issues.apache.org/jira/browse/DRILL-6008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403110#comment-16403110 ] Venkata Jyothsna Donapati commented on DRILL-6008: -- [~arina] With DRILL-6044 changes the post request (shutdown request) is done on the host from the url. So this issue will not occur. Moreover, I have tried to shutdown using short host name from Postman(Rest API client) and it works fine (This works only in case of ssl, auth disabled). All I did was add XX.XX.XX.XXX cv1 cv1.lab in /etc/hosts in my local from where I'm using the Postman to shutdown (POST on http://cv1:8047/gracefulShutdown) > Unable to shutdown Drillbit using short domain name > --- > > Key: DRILL-6008 > URL: https://issues.apache.org/jira/browse/DRILL-6008 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Attachments: fqdn.JPG, method_is_not_allowed.JPG, > response_of_undefined.JPG > > > Could not shutdown drillbit on cluster where host name was used as drillbit's > address (fqdn.JPG). Pressing shutdown resulted in > (response_of_undefined.JPG). I have tried using ip address and also no luck > (method_is_not_allowed.JPG). > I could shutdown drillbit in embeddded mode but then I saw the following > errors (local_shutdown.JPG): looks like Web UI was trying to get drillbit > status though it was down. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6261) logging "Waiting for X queries to complete before shutting down" even before shutdown request is triggered
Venkata Jyothsna Donapati created DRILL-6261: Summary: logging "Waiting for X queries to complete before shutting down" even before shutdown request is triggered Key: DRILL-6261 URL: https://issues.apache.org/jira/browse/DRILL-6261 Project: Apache Drill Issue Type: Bug Reporter: Venkata Jyothsna Donapati After https://issues.apache.org/jira/browse/DRILL-5922 changes "Waiting for X queries to complete before shutting down" is logged every time a query runs instead of it being logged after a shutdown request is triggered. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6252:
-
Attachment: foreman_drillbit.log
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6252:
-
Attachment: nonforeman_drillbit.log
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6243) Alert box to confirm shutdown of drillbit after clicking shutdown button
[ https://issues.apache.org/jira/browse/DRILL-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati reassigned DRILL-6243: Assignee: Venkata Jyothsna Donapati > Alert box to confirm shutdown of drillbit after clicking shutdown button > - > > Key: DRILL-6243 > URL: https://issues.apache.org/jira/browse/DRILL-6243 > Project: Apache Drill > Issue Type: Improvement >Reporter: Venkata Jyothsna Donapati >Assignee: Venkata Jyothsna Donapati >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-4829) Configure the address to bind to
[ https://issues.apache.org/jira/browse/DRILL-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401063#comment-16401063 ] Venkata Jyothsna Donapati commented on DRILL-4829: -- This is incorporated in DRILL-6005. Configuring `drill.exec.allow_loopback_address_binding` option to true will allow binding to loopback address > Configure the address to bind to > > > Key: DRILL-4829 > URL: https://issues.apache.org/jira/browse/DRILL-4829 > Project: Apache Drill > Issue Type: Improvement >Reporter: Daniel Stockton >Assignee: Venkata Jyothsna Donapati >Priority: Minor > Fix For: 1.14.0 > > > 1.7 included the following patch to prevent Drillbits binding to the loopback > address: https://issues.apache.org/jira/browse/DRILL-4523 > "Drillbit is disallowed to bind to loopback address in distributed mode." > It would be better if this was configurable rather than rely on /etc/hosts, > since it's common for the hostname to resolve to loopback. > Would you accept a patch that adds this option to drill.override.conf? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[ https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401054#comment-16401054 ] Venkata Jyothsna Donapati commented on DRILL-6039: -- This happened only when we query on parquet files. And looks like this is the issue : https://issues.apache.org/jira/browse/DRILL-6252 and graceful shutdown has nothing to do with this > drillbit.sh graceful_stop does not wait for fragments to complete before > stopping the drillbit > -- > > Key: DRILL-6039 > URL: https://issues.apache.org/jira/browse/DRILL-6039 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.3.0 >Reporter: Krystal >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > > git.commit.id.abbrev=eb0c403 > I have 3-nodes cluster with drillbits running on each node. I kicked off a > long running query. In the middle of the query, I did a "./drillbit.sh > graceful_stop" on one of the non-foreman node. The node was stopped within a > few seconds and the query failed with error: > Error: SYSTEM ERROR: IOException: Filesystem closed > Fragment 4:15 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
Venkata Jyothsna Donapati created DRILL-6252: Summary: Foreman node is going down when the non foreman node is stopped Key: DRILL-6252 URL: https://issues.apache.org/jira/browse/DRILL-6252 Project: Apache Drill Issue Type: Bug Reporter: Venkata Jyothsna Donapati Two drillbits are running. I'm running a join query over parquet and tried to stop the non-foreman node using drillbit.sh stop. The query fails with *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The non-foreman node goes down. The foreman node also goes down. When I looked at the drillbit.log of both foreman and non-foreman I found that there is memory leak "Memory was leaked by query. Memory leaked: (2097152)\nAllocator(op:2:0:0:HashPartitionSender) 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are the stack traces for memory leaks [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (3145728) Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 (res/actual/peak/limit) Fragment 2:1 [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266) [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161] Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (3145728) Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 (res/actual/peak/limit) Ping me for the logs and more information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6244) Ability to shutdown cross domain Drillbits
[ https://issues.apache.org/jira/browse/DRILL-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398944#comment-16398944 ] Venkata Jyothsna Donapati commented on DRILL-6244: -- Following document explains the issues and some of the proposed solutions to solve this problem [https://docs.google.com/document/d/1LozFqXSuVHsOZ31gFNfI39h2W3IeJG84j9-eW6cq92Q/edit?usp=sharing] > Ability to shutdown cross domain Drillbits > -- > > Key: DRILL-6244 > URL: https://issues.apache.org/jira/browse/DRILL-6244 > Project: Apache Drill > Issue Type: Improvement >Reporter: Venkata Jyothsna Donapati >Priority: Minor > > Currently Drill Web UI supports shutdown of current Drillbit and doesn't > provide an option to shutdown cross domain Drillbits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6244) Ability to shutdown cross domain Drillbits
[ https://issues.apache.org/jira/browse/DRILL-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6244: - Priority: Minor (was: Major) > Ability to shutdown cross domain Drillbits > -- > > Key: DRILL-6244 > URL: https://issues.apache.org/jira/browse/DRILL-6244 > Project: Apache Drill > Issue Type: Improvement >Reporter: Venkata Jyothsna Donapati >Priority: Minor > > Currently Drill Web UI supports shutdown of current Drillbit and doesn't > provide an option to shutdown cross domain Drillbits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398936#comment-16398936 ] Venkata Jyothsna Donapati commented on DRILL-6245: -- This might the root cause: [https://stackoverflow.com/questions/2163828/reading-cookies-via-https-that-were-set-using-http] > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Priority: Minor > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6245) Clicking on anything redirects to main login page
Venkata Jyothsna Donapati created DRILL-6245: Summary: Clicking on anything redirects to main login page Key: DRILL-6245 URL: https://issues.apache.org/jira/browse/DRILL-6245 Project: Apache Drill Issue Type: Improvement Reporter: Venkata Jyothsna Donapati When the Drill Web UI is accessed using https and then by http protocol, the Web UI is always trying to redirect to main login page if anything is clicked on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6245) Clicking on anything redirects to main login page
[ https://issues.apache.org/jira/browse/DRILL-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6245: - Issue Type: Bug (was: Improvement) > Clicking on anything redirects to main login page > - > > Key: DRILL-6245 > URL: https://issues.apache.org/jira/browse/DRILL-6245 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Priority: Minor > > When the Drill Web UI is accessed using https and then by http protocol, the > Web UI is always trying to redirect to main login page if anything is clicked > on index page. However, this works fine if the cookies are cleared. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6244) Ability to shutdown cross domain Drillbits
Venkata Jyothsna Donapati created DRILL-6244: Summary: Ability to shutdown cross domain Drillbits Key: DRILL-6244 URL: https://issues.apache.org/jira/browse/DRILL-6244 Project: Apache Drill Issue Type: Improvement Reporter: Venkata Jyothsna Donapati Currently Drill Web UI supports shutdown of current Drillbit and doesn't provide an option to shutdown cross domain Drillbits. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6243) Alert box to confirm shutdown of drillbit after clicking shutdown button
[ https://issues.apache.org/jira/browse/DRILL-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6243: - Summary: Alert box to confirm shutdown of drillbit after clicking shutdown button (was: Alert box to confirm shutdown of drillbit) > Alert box to confirm shutdown of drillbit after clicking shutdown button > - > > Key: DRILL-6243 > URL: https://issues.apache.org/jira/browse/DRILL-6243 > Project: Apache Drill > Issue Type: Improvement >Reporter: Venkata Jyothsna Donapati >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6243) Alert box to confirm shutdown of drillbit
Venkata Jyothsna Donapati created DRILL-6243: Summary: Alert box to confirm shutdown of drillbit Key: DRILL-6243 URL: https://issues.apache.org/jira/browse/DRILL-6243 Project: Apache Drill Issue Type: Improvement Reporter: Venkata Jyothsna Donapati -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DRILL-6010) Working drillbit showing as in QUIESCENT state
[ https://issues.apache.org/jira/browse/DRILL-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati closed DRILL-6010. Resolution: Fixed > Working drillbit showing as in QUIESCENT state > -- > > Key: DRILL-6010 > URL: https://issues.apache.org/jira/browse/DRILL-6010 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > Attachments: online_vs_quiescent.JPG > > > After DRILL-4286 once I got a situation that after running all functional > tests three drillbits were in ONLINE state, another one in QUIESCENT. Though > from the one in quiescent state I could run queries and so it was online. > drillbit.sh stop could not shutdown it and had to do kill -9 of the process > (online_vs_quiescent.JPG). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6010) Working drillbit showing as in QUIESCENT state
[ https://issues.apache.org/jira/browse/DRILL-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397687#comment-16397687 ] Venkata Jyothsna Donapati commented on DRILL-6010: -- Yes, Looks like thats the issue. Closing the issue. > Working drillbit showing as in QUIESCENT state > -- > > Key: DRILL-6010 > URL: https://issues.apache.org/jira/browse/DRILL-6010 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > Attachments: online_vs_quiescent.JPG > > > After DRILL-4286 once I got a situation that after running all functional > tests three drillbits were in ONLINE state, another one in QUIESCENT. Though > from the one in quiescent state I could run queries and so it was online. > drillbit.sh stop could not shutdown it and had to do kill -9 of the process > (online_vs_quiescent.JPG). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6010) Working drillbit showing as in QUIESCENT state
[ https://issues.apache.org/jira/browse/DRILL-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397687#comment-16397687 ] Venkata Jyothsna Donapati edited comment on DRILL-6010 at 3/13/18 9:29 PM: --- Yes, Looks like thats the issue. Closing the issue. was (Author: vdonapati): Yes, Looks like thats the issue. Closing the issue. > Working drillbit showing as in QUIESCENT state > -- > > Key: DRILL-6010 > URL: https://issues.apache.org/jira/browse/DRILL-6010 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > Attachments: online_vs_quiescent.JPG > > > After DRILL-4286 once I got a situation that after running all functional > tests three drillbits were in ONLINE state, another one in QUIESCENT. Though > from the one in quiescent state I could run queries and so it was online. > drillbit.sh stop could not shutdown it and had to do kill -9 of the process > (online_vs_quiescent.JPG). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6149) Support Query Level Options
[ https://issues.apache.org/jira/browse/DRILL-6149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6149: - Issue Type: New Feature (was: Bug) > Support Query Level Options > --- > > Key: DRILL-6149 > URL: https://issues.apache.org/jira/browse/DRILL-6149 > Project: Apache Drill > Issue Type: New Feature >Reporter: Timothy Farkas >Assignee: Venkata Jyothsna Donapati >Priority: Major > > Currently the line between session options and query level options are > blurred. Ideally a session level option can be set and persisted for the > duration of a user's session. A query level option can be set for a query. > Right now there is no explicit query level option. A query level option is a > session option whose validator has a non zero ttl. We should change this to > have options that belong to the Query Scope as defined in the > OptionDefinition. The ttl should also be configurable by the user in their > **alter set** command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6022) Improve js part for graceful shutdown
[ https://issues.apache.org/jira/browse/DRILL-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-6022: - Issue Type: Improvement (was: Bug) > Improve js part for graceful shutdown > - > > Key: DRILL-6022 > URL: https://issues.apache.org/jira/browse/DRILL-6022 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > > DRILL-4286 introduces graceful shutdown but its js part needs improvement: > a. ajax call do not handle errors, so when error occurs it is just swallowed. > b. there are some unused and / or unnecessary variables usage > c. shutdown functionality is disabled when user is not an admin but some > other ajax calls are still being executed, for example, port number, number > of queries, grace period. All that can be also can be disabled when user is > not an admin. > d. there are many ajax calls which can be factored out in dedicated js file. > Other fixes: > a. all shutdown functionality reside in DrillRoot class, it can be factored > out in shutdown specific class where all shutdown functionality will be > allowed only for admin on class level, currently we marked in on the level > (see DRILL-6019). > b. issue described in DRILL-6021. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6005) Fix TestGracefulShutdown tests to skip check for loopback address usage in distributed mode
[
https://issues.apache.org/jira/browse/DRILL-6005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6005:
-
Issue Type: Improvement (was: Bug)
> Fix TestGracefulShutdown tests to skip check for loopback address usage in
> distributed mode
> ---
>
> Key: DRILL-6005
> URL: https://issues.apache.org/jira/browse/DRILL-6005
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Venkata Jyothsna Donapati
>Priority: Minor
> Fix For: 1.14.0
>
>
> After DRILL-4286 changes some of the newly added unit tests fail with
> {noformat}
> Drillbit is disallowed to bind to loopback address in distributed mode.
> {noformat}
> List of failed tests:
> {noformat}
> Tests in error:
> TestGracefulShutdown.testOnlineEndPoints:96 » IllegalState Cluster fixture
> set...
> TestGracefulShutdown.testStateChange:130 » IllegalState Cluster fixture
> setup ...
> TestGracefulShutdown.testRestApi:167 » IllegalState Cluster fixture setup
> fail...
> TestGracefulShutdown.testRestApiShutdown:207 » IllegalState Cluster fixture
> se...
> {noformat}
> This can be fixed if {{/etc/hosts}} file is edited.
> Source -
> https://stackoverflow.com/questions/40506221/how-to-start-drillbit-locally-in-distributed-mode
> Though these changes are required on production during running unit tests I
> don't think this check should be enforced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6227) Graceful shutdown should fail if unable to kill drillbit during some timeout rather then trying indefinitely
[
https://issues.apache.org/jira/browse/DRILL-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6227:
-
Issue Type: Improvement (was: Bug)
> Graceful shutdown should fail if unable to kill drillbit during some timeout
> rather then trying indefinitely
>
>
> Key: DRILL-6227
> URL: https://issues.apache.org/jira/browse/DRILL-6227
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.14.0
>
>
> In drillbit.sh graceful shutdown calls drillbit stop with {{kill_drillbit}}
> set to false.
> {code}
> kill_drillbit=false
> stop_bit $kill_drillbit
> {code}
> It means that {{waitForProcessEnd}} will be called with the same property.
> When {{waitForProcessEnd}} is called with {{kill_drillbit}} set to false,
> this method will try to kill drillbit using {{kill -0}} until succeeds. So if
> at some point it won't be able, it may run forever. Need to have some timeout
> when {{waitForProcessEnd}} will stop trying to kill drillbit and report an
> error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-4547) Javadoc fails with Java8
[
https://issues.apache.org/jira/browse/DRILL-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati reassigned DRILL-4547:
Assignee: Venkata Jyothsna Donapati
> Javadoc fails with Java8
>
>
> Key: DRILL-4547
> URL: https://issues.apache.org/jira/browse/DRILL-4547
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build Test
>Affects Versions: 1.6.0
>Reporter: Laurent Goujon
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
>
> Javadoc cannot be generated when using Java8 (likely because the parser is
> now more strict).
> Here's an example of issues when trying to generate javadocs in module
> {{drill-fmpp-maven-plugin}}
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:jar (attach-javadocs) on
> project drill-fmpp-maven-plugin: MavenReportException: Error while creating
> archive:
> [ERROR] Exit code: 1 -
> /Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:44:
> error: unknown tag: goal
> [ERROR] * @goal generate
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:45:
> error: unknown tag: phase
> [ERROR] * @phase generate-sources
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:25:
> error: unknown tag: goal
> [ERROR] * @goal help
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:26:
> error: unknown tag: requiresProject
> [ERROR] * @requiresProject false
> [ERROR] ^
> [ERROR]
> /Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:27:
> error: unknown tag: threadSafe
> [ERROR] * @threadSafe
> [ERROR] ^
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/bin/javadoc
> @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in
> '/Users/laurent/devel/drill/tools/fmpp/target/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :drill-fmpp-maven-plugin
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6023) Graceful shutdown improvements (umbrella jira)
[ https://issues.apache.org/jira/browse/DRILL-6023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379164#comment-16379164 ] Venkata Jyothsna Donapati commented on DRILL-6023: -- I'm planning to address the following four jiras for 1.13: DRILL-6044, DRILL-6040, DRILL-6039, DRILL-6021 > Graceful shutdown improvements (umbrella jira) > -- > > Key: DRILL-6023 > URL: https://issues.apache.org/jira/browse/DRILL-6023 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.14.0 > > > DRILL-4286 introduces graceful shutdown. > This is umbrella Jira for currently known graceful shutdown improvements. > All known Jiras are linked. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6010) Working drillbit showing as in QUIESCENT state
[ https://issues.apache.org/jira/browse/DRILL-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373474#comment-16373474 ] Venkata Jyothsna Donapati commented on DRILL-6010: -- I'm trying to understand the scenario but drillbit.sh stop basically does "kill -9" after waiting for 120 seconds. Is the page stale? > Working drillbit showing as in QUIESCENT state > -- > > Key: DRILL-6010 > URL: https://issues.apache.org/jira/browse/DRILL-6010 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva >Assignee: Venkata Jyothsna Donapati >Priority: Major > Fix For: 1.13.0 > > Attachments: online_vs_quiescent.JPG > > > After DRILL-4286 once I got a situation that after running all functional > tests three drillbits were in ONLINE state, another one in QUIESCENT. Though > from the one in quiescent state I could run queries and so it was online. > drillbit.sh stop could not shutdown it and had to do kill -9 of the process > (online_vs_quiescent.JPG). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6005) Fix TestGracefulShutdown tests to skip check for loopback address usage in distributed mode
[
https://issues.apache.org/jira/browse/DRILL-6005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370530#comment-16370530
]
Venkata Jyothsna Donapati commented on DRILL-6005:
--
[~arina] Do you mean we can allow loopback address in distributed mode?
> Fix TestGracefulShutdown tests to skip check for loopback address usage in
> distributed mode
> ---
>
> Key: DRILL-6005
> URL: https://issues.apache.org/jira/browse/DRILL-6005
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Venkata Jyothsna Donapati
>Priority: Major
> Fix For: 1.13.0
>
>
> After DRILL-4286 changes some of the newly added unit tests fail with
> {noformat}
> Drillbit is disallowed to bind to loopback address in distributed mode.
> {noformat}
> List of failed tests:
> {noformat}
> Tests in error:
> TestGracefulShutdown.testOnlineEndPoints:96 » IllegalState Cluster fixture
> set...
> TestGracefulShutdown.testStateChange:130 » IllegalState Cluster fixture
> setup ...
> TestGracefulShutdown.testRestApi:167 » IllegalState Cluster fixture setup
> fail...
> TestGracefulShutdown.testRestApiShutdown:207 » IllegalState Cluster fixture
> se...
> {noformat}
> This can be fixed if {{/etc/hosts}} file is edited.
> Source -
> https://stackoverflow.com/questions/40506221/how-to-start-drillbit-locally-in-distributed-mode
> Though these changes are required on production during running unit tests I
> don't think this check should be enforced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6010) Working drillbit showing as in QUIESCENT state
[ https://issues.apache.org/jira/browse/DRILL-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280400#comment-16280400 ] Venkata Jyothsna Donapati commented on DRILL-6010: -- [~arina] Can you please give more details like what caused the shutdown of that Drillbit? What is the graceperiod that is set( there is zookeeper delay in updating other drillbits about its state)? drillbit.sh stop typically waits for 120 sec before forcefully shutting down. Did you see it running even after 120 sec ? > Working drillbit showing as in QUIESCENT state > -- > > Key: DRILL-6010 > URL: https://issues.apache.org/jira/browse/DRILL-6010 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva > Attachments: online_vs_quiescent.JPG > > > After DRILL-4286 once I got a situation that after running all functional > tests three drillbits were in ONLINE state, another one in QUIESCENT. Though > from the one in quiescent state I could run queries and so it was online. > drillbit.sh stop could not shutdown it and had to do kill -9 of the process > (online_vs_quiescent.JPG). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6008) Unable to shutdown Drillbit
[ https://issues.apache.org/jira/browse/DRILL-6008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280351#comment-16280351 ] Venkata Jyothsna Donapati commented on DRILL-6008: -- [~arina] Is the hostname defined in /etc/hosts? For the method not allowed I'm assuming you are doing a GET instead of POST, this can be tested using restapi clients like postman. > Unable to shutdown Drillbit > --- > > Key: DRILL-6008 > URL: https://issues.apache.org/jira/browse/DRILL-6008 > Project: Apache Drill > Issue Type: Bug >Reporter: Arina Ielchiieva > Attachments: fqdn.JPG, method_is_not_allowed.JPG, > response_of_undefined.JPG > > > Could not shutdown drillbit on cluster where host name was used as drillbit's > address (fqdn.JPG). Pressing shutdown resulted in > (response_of_undefined.JPG). I have tried using ip address and also no luck > (method_is_not_allowed.JPG). > I could shutdown drillbit in embeddded mode but then I saw the following > errors (local_shutdown.JPG): looks like Web UI was trying to get drillbit > status though it was down. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-5547) Drill config options and session options do not work as intended
[ https://issues.apache.org/jira/browse/DRILL-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Jyothsna Donapati updated DRILL-5547: - Labels: ready-to-commit (was: ) > Drill config options and session options do not work as intended > > > Key: DRILL-5547 > URL: https://issues.apache.org/jira/browse/DRILL-5547 > Project: Apache Drill > Issue Type: Bug > Components: Server >Affects Versions: 1.10.0 >Reporter: Karthikeyan Manivannan >Assignee: Venkata Jyothsna Donapati > Labels: ready-to-commit > Fix For: Future > > > In Drill, session options should take precedence over config options. But > several of these session options are assigned hard-coded default values when > the option validators are initialized. Because of this config options will > never be read and honored even if the user did not specify the session > option. > ClassCompilerSelector.JAVA_COMPILER_VALIDATOR uses CompilerPolicy.DEFAULT as > the default value. This default value gets into the session options map via > the initialization of validators in SystemOptionManager. > Now any piece of code that tries to check if a session option is set will > never see a null, so it will always use that value and never try to look into > the config options. For example, in the following piece of code from > ClassCompilerSelector (), the policy will never be read from the config file. > policy = CompilerPolicy.valueOf((value != null) ? > value.string_val.toUpperCase() : > config.getString(JAVA_COMPILER_CONFIG).toUpperCase()); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
