[jira] [Commented] (DRILL-4761) Hit NoClassDefFoundError on class oadd/org/apache/log4j/Logger
[ https://issues.apache.org/jira/browse/DRILL-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380501#comment-16380501 ] John Humphreys commented on DRILL-4761: --- Seeing same with 1.12 JAR. :( > Hit NoClassDefFoundError on class oadd/org/apache/log4j/Logger > -- > > Key: DRILL-4761 > URL: https://issues.apache.org/jira/browse/DRILL-4761 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.6.0 >Reporter: Alfie >Priority: Minor > > I'm using drill-jdbc-all jar and see NoClassDefFoundError exception because > of cannot find class oadd/org/apache/log4j/Logger. > The class is not included in the jar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6194) Allow un-caching of parquet metadata or stop queries from failing when metadata is old.
John Humphreys created DRILL-6194: - Summary: Allow un-caching of parquet metadata or stop queries from failing when metadata is old. Key: DRILL-6194 URL: https://issues.apache.org/jira/browse/DRILL-6194 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Affects Versions: 1.10.0 Reporter: John Humphreys Let's say you have files stored in the standard hierarchical way and the data is held in parquet: * year/ ** month/ *** day/ filev2.parquet If you cache the metadata under year/ or one of the other levels, and then you replace filev2.parquet with filev3.parquet, you will get errors when running queries relating to file2.parquet not being present. I'm specifically seeing this when using maxdir(), and dir0/1/2 for year/month/day but I suspect its a general issue. Queries using cached metadata should not fail if the metadata is outdated; they should just choose not to use it. Otherwise there should be an uncache operator for the metadata so people can just decide to stop using it. It's not always efficient to run a metadata refresh before every single query you do, and its difficult to run one from every program that touches HDFS files immediately after it touches them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6116) MAXDIR - Unsupported operation exception // partition explorer interface?
[
https://issues.apache.org/jira/browse/DRILL-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379152#comment-16379152
]
John Humphreys commented on DRILL-6116:
---
[~kkhatua] - Running into this again. Did you ever manage to take a peek?
> MAXDIR - Unsupported operation exception // partition explorer interface?
> -
>
> Key: DRILL-6116
> URL: https://issues.apache.org/jira/browse/DRILL-6116
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.10.0
>Reporter: John Humphreys
>Priority: Major
> Attachments: maxdir_fail.json, maxdir_success.json
>
>
> Using MAXDIR in drill seems to cause errors when trying to apply other
> standard where clause filters. Am I doing something wrong here?
> **This Query Fails**
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> and entity = 1784928
> **With This Error**
> > SYSTEM ERROR: UnsupportedOperationException: The partition explorer
> > interface can only be used in functions that can be evaluated at
> > planning time. Make sure that the planner.enable_constant_folding
> > configuration option is set to true.
> >
> > Fragment 1:7
> >
> > [Error Id: d62c8227-db0e-4d31-9443-e570193d1010 on
> > psclxcpdevsys13.nomura.com:31010]
> **Wrapping it Makes it Work Without Error**
> select * from (
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> ) where entity = 1784928
> epoch_hour entity 1_average 1_95p
> 1517086800 1784928 5.3347222 8.82857142857143
> 1517014800 1784928 25.944984717977217 40.37453087409783
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6116) MAXDIR - Unsupported operation exception // partition explorer interface?
[
https://issues.apache.org/jira/browse/DRILL-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352917#comment-16352917
]
John Humphreys commented on DRILL-6116:
---
Sorry for the delay. I moved on to another way of doing things and had to
recreate the failure. It's actually pretty trivial to recreate though.
By the way, I use drill on top of MapR and they tested and confirmed there was
an issue too - so it has some form of validation already.
I attached the requested JSON profiles; one for the failure and one for the
success gained from wrapping it in a sub-query.
* https://issues.apache.org/jira/secure/attachment/12909275/maxdir_fail.json
*
^https://issues.apache.org/jira/secure/attachment/12909276/maxdir_success.json^
> MAXDIR - Unsupported operation exception // partition explorer interface?
> -
>
> Key: DRILL-6116
> URL: https://issues.apache.org/jira/browse/DRILL-6116
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.10.0
>Reporter: John Humphreys
>Priority: Major
> Attachments: maxdir_fail.json, maxdir_success.json
>
>
> Using MAXDIR in drill seems to cause errors when trying to apply other
> standard where clause filters. Am I doing something wrong here?
> **This Query Fails**
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> and entity = 1784928
> **With This Error**
> > SYSTEM ERROR: UnsupportedOperationException: The partition explorer
> > interface can only be used in functions that can be evaluated at
> > planning time. Make sure that the planner.enable_constant_folding
> > configuration option is set to true.
> >
> > Fragment 1:7
> >
> > [Error Id: d62c8227-db0e-4d31-9443-e570193d1010 on
> > psclxcpdevsys13.nomura.com:31010]
> **Wrapping it Makes it Work Without Error**
> select * from (
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> ) where entity = 1784928
> epoch_hour entity 1_average 1_95p
> 1517086800 1784928 5.3347222 8.82857142857143
> 1517014800 1784928 25.944984717977217 40.37453087409783
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6116) MAXDIR - Unsupported operation exception // partition explorer interface?
[
https://issues.apache.org/jira/browse/DRILL-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Humphreys updated DRILL-6116:
--
Attachment: maxdir_success.json
maxdir_fail.json
> MAXDIR - Unsupported operation exception // partition explorer interface?
> -
>
> Key: DRILL-6116
> URL: https://issues.apache.org/jira/browse/DRILL-6116
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC
>Affects Versions: 1.10.0
>Reporter: John Humphreys
>Priority: Major
> Attachments: maxdir_fail.json, maxdir_success.json
>
>
> Using MAXDIR in drill seems to cause errors when trying to apply other
> standard where clause filters. Am I doing something wrong here?
> **This Query Fails**
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> and entity = 1784928
> **With This Error**
> > SYSTEM ERROR: UnsupportedOperationException: The partition explorer
> > interface can only be used in functions that can be evaluated at
> > planning time. Make sure that the planner.enable_constant_folding
> > configuration option is set to true.
> >
> > Fragment 1:7
> >
> > [Error Id: d62c8227-db0e-4d31-9443-e570193d1010 on
> > psclxcpdevsys13.nomura.com:31010]
> **Wrapping it Makes it Work Without Error**
> select * from (
> SELECT epoch_hour, entity, `1_average`, `1_95p`
> FROM dfs.`/path/drill-hour-final/`
> where
> dir0 = '2018' and dir1 = '01' and dir2 = '27'
> and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
> dir0, '/', dir1, '/', dir2, '/'))
> ) where entity = 1784928
> epoch_hour entity 1_average 1_95p
> 1517086800 1784928 5.3347222 8.82857142857143
> 1517014800 1784928 25.944984717977217 40.37453087409783
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6116) MAXDIR - Unsupported operation exception // partition explorer interface?
John Humphreys created DRILL-6116:
-
Summary: MAXDIR - Unsupported operation exception // partition
explorer interface?
Key: DRILL-6116
URL: https://issues.apache.org/jira/browse/DRILL-6116
Project: Apache Drill
Issue Type: Bug
Components: Client - JDBC
Affects Versions: 1.10.0
Reporter: John Humphreys
Using MAXDIR in drill seems to cause errors when trying to apply other standard
where clause filters. Am I doing something wrong here?
**This Query Fails**
SELECT epoch_hour, entity, `1_average`, `1_95p`
FROM dfs.`/path/drill-hour-final/`
where
dir0 = '2018' and dir1 = '01' and dir2 = '27'
and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
dir0, '/', dir1, '/', dir2, '/'))
and entity = 1784928
**With This Error**
> SYSTEM ERROR: UnsupportedOperationException: The partition explorer
> interface can only be used in functions that can be evaluated at
> planning time. Make sure that the planner.enable_constant_folding
> configuration option is set to true.
>
> Fragment 1:7
>
> [Error Id: d62c8227-db0e-4d31-9443-e570193d1010 on
> psclxcpdevsys13.nomura.com:31010]
**Wrapping it Makes it Work Without Error**
select * from (
SELECT epoch_hour, entity, `1_average`, `1_95p`
FROM dfs.`/path/drill-hour-final/`
where
dir0 = '2018' and dir1 = '01' and dir2 = '27'
and dir3 = MAXDIR('dfs', concat('/path/drill-hour-final/',
dir0, '/', dir1, '/', dir2, '/'))
) where entity = 1784928
epoch_hour entity 1_average 1_95p
1517086800 1784928 5.3347222 8.82857142857143
1517014800 1784928 25.944984717977217 40.37453087409783
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
