[jira] [Updated] (DRILL-8373) Add log statements wherever a finite resource is freed
[ https://issues.apache.org/jira/browse/DRILL-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8373: Fix Version/s: Future (was: 1.21.0) > Add log statements wherever a finite resource is freed > -- > > Key: DRILL-8373 > URL: https://issues.apache.org/jira/browse/DRILL-8373 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.2 >Reporter: James Turton >Priority: Minor > Fix For: Future > > > When a system administrator is looking into a resource consumption question > it can be helpful if they can see the closing or freeing of resources such as > files and network handles in the Drill logs. This Jira proposes a change to > add such log statements at the debug level (or perhaps the info level?) > wherever possible. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8362: Fix Version/s: (was: 1.21.0) > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8368) Update Yauaa to 7.9.0
[ https://issues.apache.org/jira/browse/DRILL-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8368. --- Resolution: Fixed > Update Yauaa to 7.9.0 > - > > Key: DRILL-8368 > URL: https://issues.apache.org/jira/browse/DRILL-8368 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.3 >Reporter: Niels Basjes >Assignee: Niels Basjes >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8362: Fix Version/s: 1.21.0 > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.0 > > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8362. --- Resolution: Fixed > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.0 > > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8259) Support advanced HBase persistence storage options
[
https://issues.apache.org/jira/browse/DRILL-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8259:
Fix Version/s: (was: 1.21.0)
> Support advanced HBase persistence storage options
> --
>
> Key: DRILL-8259
> URL: https://issues.apache.org/jira/browse/DRILL-8259
> Project: Apache Drill
> Issue Type: New Feature
> Components: Storage - HBase
>Reporter: Cong Luo
>Assignee: Cong Luo
>Priority: Major
>
> Its contents are as follows
> {code:java}
> sys.store.provider: {
> class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
> hbase: {
> table : "drill_store",
> config: {
> "hbase.zookeeper.quorum": "zk_host3,zk_host2,zk_host1",
> "hbase.zookeeper.property.clientPort": "2181",
> "zookeeper.znode.parent": "/hbase-test"
> },
> table_config : {
> "durability": "ASYNC_WAL",
> "compaction_enabled": false,
> "split_enabled": false,
> "max_filesize": 10737418240,
> "memstore_flushsize": 536870912
> },
> column_config : {
> "versions": 1,
> "ttl": 2626560,
> "compression": "SNAPPY",
> "blockcache": true,
> "blocksize": 131072,
> "data_block_encoding": "FAST_DIFF",
> "in_memory": true,
> "dfs_replication": 3
> }
> }
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8241) Remove Deprecated JSON Reader
[ https://issues.apache.org/jira/browse/DRILL-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8241: Fix Version/s: (was: 1.21.0) > Remove Deprecated JSON Reader > - > > Key: DRILL-8241 > URL: https://issues.apache.org/jira/browse/DRILL-8241 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - JSON >Affects Versions: 1.20.1 >Reporter: Charles Givre >Priority: Major > > This is a master ticket to remove the deprecated v1 JSON reader from Drill. > This JSON reader is used in several places and removing it will ensure > consistent behavior across all data sources. > The V2, EVF based JSON reader has several advantages, including the > possibility of schema provisioning, limit pushdowns and others. > Here are the tasks which need to be completed to fully remove the v1 JSON > reader. > * Complete DRILL-5955 which adds support for the UNION vector to the EVF > Json reader. > * Convert the convert_fromJSON functions to V2 (DRILL-8239) > * Convert the Druid Storage Plugin to V2 (DRILL-8316) > * Convert MongoDB Storage Plugin to V2. (Note the MongoDB plugin uses an > EVF-based BSON reader as well as the V1 JSON reader) > * Remove all V1-based unit tests > * Migrate the JsonOptions from the HTTP Storage Plugin to global location to > allow other plugins and users of JSON to set JSON configuration at a more > granular level. (DRILL-8243) > * Remove extraneous configuration options. > * Bug fix HTTP UDFs (DRILL-8242) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8201) SchemaChange in HashAgg operator
[
https://issues.apache.org/jira/browse/DRILL-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8201:
Fix Version/s: (was: 1.21.0)
> SchemaChange in HashAgg operator
>
>
> Key: DRILL-8201
> URL: https://issues.apache.org/jira/browse/DRILL-8201
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Codegen, Storage - JSON
>Affects Versions: 1.20.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>
> Hash aggregate does not support schema change due to HashAggBatch
> implementation:
> {code:java}
> case UPDATE_AGGREGATOR:
> throw UserException.unsupportedError()
> .message(SchemaChangeException.schemaChanged(
> "Hash aggregate does not support schema change",
> incomingSchema,
> incoming.getSchema()).getMessage())
> .build(logger);
> default: {code}
> After JSON update to leverage EVF there is schema change for this oprator in
> the
> _TestE2EUnnestAndLateral#testMultipleBatchesLateral_WithGroupByInParent_ test
> case
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8239) Convert JSON UDF to EVF
[ https://issues.apache.org/jira/browse/DRILL-8239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8239: Fix Version/s: (was: 1.21.0) > Convert JSON UDF to EVF > --- > > Key: DRILL-8239 > URL: https://issues.apache.org/jira/browse/DRILL-8239 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.20.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > > In an effort to fully deprecate the old JsonReader, this PR converts the > convert_from JSON UDF to EVF. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8203) Enable Union Type for JSON2 reader
[ https://issues.apache.org/jira/browse/DRILL-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8203: Fix Version/s: (was: 1.21.0) > Enable Union Type for JSON2 reader > -- > > Key: DRILL-8203 > URL: https://issues.apache.org/jira/browse/DRILL-8203 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - JSON >Affects Versions: 1.20.0 >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Major > > Enable UNION TYPE Mode support for EVF JSON reader, which is controlled with > _exec.enable_union_type_ system/session option. > Need to leverage > _SingleMapWriter#unionEnabled_ functionality __ and bind to __ > _JsonLoaderOptions#unionEnabled_ config -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8196) JSON EVF2
[ https://issues.apache.org/jira/browse/DRILL-8196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8196: Fix Version/s: (was: 1.21.0) > JSON EVF2 > - > > Key: DRILL-8196 > URL: https://issues.apache.org/jira/browse/DRILL-8196 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - JSON >Affects Versions: 1.20.0 >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Major > > DRILL-8085 introduces EVF2 formt. Switch EVF1 JSON to EVF2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8197) Enable All Text Mode support for EVF JSON reader
[ https://issues.apache.org/jira/browse/DRILL-8197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8197: Fix Version/s: (was: 1.21.0) > Enable All Text Mode support for EVF JSON reader > > > Key: DRILL-8197 > URL: https://issues.apache.org/jira/browse/DRILL-8197 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - JSON >Affects Versions: 1.20.0 >Reporter: Vitalii Diravka >Assignee: Vitalii Diravka >Priority: Major > > Enable All Text Mode support for EVF JSON reader. > _JsonLoaderOptions#allTextMode_ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8180) Add Icons to Storage Plugin List
[ https://issues.apache.org/jira/browse/DRILL-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8180: Fix Version/s: (was: 1.21.0) > Add Icons to Storage Plugin List > > > Key: DRILL-8180 > URL: https://issues.apache.org/jira/browse/DRILL-8180 > Project: Apache Drill > Issue Type: Task > Components: Storage - Other, Web Server >Affects Versions: 1.20.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8189) Convert Image format to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8189: Fix Version/s: (was: 1.21.0) > Convert Image format to EVF2 > > > Key: DRILL-8189 > URL: https://issues.apache.org/jira/browse/DRILL-8189 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.0 >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > > Second refactor of the Image format. > In addition, continue to deal with minor bugs in the V2 framework. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8195) Add Timestamp Zone offset ISO-8601 format for JSON EVF
[
https://issues.apache.org/jira/browse/DRILL-8195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8195:
Fix Version/s: (was: 1.21.0)
> Add Timestamp Zone offset ISO-8601 format for JSON EVF
> ---
>
> Key: DRILL-8195
> URL: https://issues.apache.org/jira/browse/DRILL-8195
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - JSON
>Affects Versions: 1.20.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>
> Add new format ifor mongo-styled dates intorduced in DRILL-8143 to EVF based
> JSON reader
> The following test cases failed:
> _TestExtendedTypes, TestFrameworkTest, TestCsvWithoutHeadersWithSchema,
> TestNestedDateTimeTimestamp, TestJsonRecordReader_
> {code:java}
> java.lang.Exception: org.apache.drill.common.exceptions.UserRemoteException:
> DATA_READ ERROR: Type of JSON token is not compatible with its columnJSON
> token type: date
> JSON token: 2019-09-30T20:47:43.10+05
> Column: date
> Column type: TIMESTAMP
> Format plugin: json
> Format plugin: JSONFormatPlugin
> Plugin config name: json
> File name: classpath:/jsoninput/input2.json
> Line: 32
> Position: 42
> Near token: 2019-09-30T20:47:43.10+05
> Fragment: 0:0 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8188) Convert HDF5 format to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8188: Fix Version/s: (was: 1.21.0) > Convert HDF5 format to EVF2 > --- > > Key: DRILL-8188 > URL: https://issues.apache.org/jira/browse/DRILL-8188 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.0 >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > > Use EVF V2 instead of old V1. > Also, fixed a few bugs in V2 framework. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8101) Resolve the TIMESTAMP madness
[
https://issues.apache.org/jira/browse/DRILL-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8101:
Fix Version/s: (was: 1.21.0)
> Resolve the TIMESTAMP madness
> -
>
> Key: DRILL-8101
> URL: https://issues.apache.org/jira/browse/DRILL-8101
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Paul Rogers
>Priority: Major
>
> Drill's TIMESAMP type tries to be two different things at the same time,
> causing incorrect results when the two interpretations collide.
> Drill has the classic DATE and TIME data types. A DATE is just that: a day
> wherever you happen to be. Your birthday goes from midnight to midnight in
> the time zone where you find yourself. If you happen to travel around the
> world, you can make your birthday last almost 48 hours as midnight of your
> birthday starts at the international date line, circles the globe, followed
> by the midnight of the next day.
> Similarly, a time is a time where you are. 12:00PM is noon (more-or-less) as
> determined by the sun. 12:00PM occurs once in every time zone every day.
> Since there are many time zones, there are many noons each day.
> These are both examples of local time. Most databases combine these two ideas
> to get a DATETIME: a date and time wherever you are.
> In our modern world, knowing something occurred on 2022-01-02 12:00:00 is not
> good enough. Did it occur at that time in my time zone or yours? If the event
> is a user login, or a network breach, then it occurred once, at a specific
> time, it did not occur many times: once in each time zone. Hence, machines
> often use UTC time to coordinate.
> Unix-like systems also define the idea of a "timestamp", the number of
> seconds (or milliseconds or nanoseconds) since 1970-01-01 00:00:00. This is
> the time reported by Java in the {{System.currentTime()}} function. It is the
> time most often found in machine-generated logs. It may be as a number (ms
> since the epoch) or as an ISO-formatted string.
> Thus, users of Drill would expect to find a "timestamp" type that represents
> a UTC timestamp in Unix format. The will be disappointed, however.
> Drill's TIMESTAMP type is essentially a DATETIME type: it is a date/time in
> an unspecified timezone and that zone can be whatever you want it to be. UTC?
> Fine. Local? OK. Nairobi? Sure, why not.
> This works fine as long as _all_ your data is in the same time zone, and you
> don't need a concept of "now". As described in DRILL-8099 and DRILL-8100,
> this is how the authors of CTAS thought of it: read Parquet data straight
> into Drill with no conversion, then write it back out to JSON with no
> conversion. Both work with UTC, so the result is fine: who cares that the
> 32-bit number, when in Drill, had no implied time zone? It is just a number
> we read then write. All good.
> It is even possible to compute the difference of two DATETIMEs with
> unspecified time zone: that's what an INTERVAL does. As long as the times are
> actually in the same zone (UTC, say, or local, or Nairobi), then all is fine.
> Everything collapses, however, when someone wants to know, "but how long ago
> was that event"? "Long enough ago that I need to raise the escalation level?"
> Drill has the INTERVAL type to give us the difference, but how do I get
> "now"? Drill has {{CURRENT_TIMESTAMP}}. But, how we have a problem, what
> timezone is that time in? UTC? My local timezone? Nairobi? And, what if my
> data is UTC but {{CURRENT_TIMESTAMP}} is local? Or visa-versa? The whole
> house of cards comes crashing down.
> Over the years, this bug has appeared again and again. Sometimes people
> change the logic to assume TIMESTAMP is UTC. Sometimes things are changed to
> assume TIMESTAMP is local time (I've been guilty of this). Sometimes we just
> punt, and require that the machine (or test) run only in UTC, since that's
> the only place the two systems coincide.
> But, in fact, I believe that the original designers of Drill meant TIMESTAMP
> to have _no_ timezone: two TIMESTAMP values could be in entirely different
> (unknown) timezones! One can see vestiges of this in the value vector code.
> It seems the original engineers imagined a "TIMESTAMP_WITH_ZONE" type,
> similar to Java's (or Joda's) {{ZonedDateTime}} type. Other bits of code
> (Parquet) refers to a never-built "TIMESTAMPZ" type for a UTC timestamp. When
> faced with the {{CURRENT_TIMESTAMP}} issue, fixes started down the path of
> saying that TIMESTAMP is local time, but this is probably a misunderstanding
> of the original design, forced upon us by the gaps in that original design.
> Further, each time we make a change (such as DRILL-8099 and DRILL-8100), we
> change behavior, potentially breaking a kludge that someone found to
[jira] [Updated] (DRILL-8173) Replace httpcomponents by okhttp
[ https://issues.apache.org/jira/browse/DRILL-8173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8173: Fix Version/s: (was: 1.21.0) > Replace httpcomponents by okhttp > > > Key: DRILL-8173 > URL: https://issues.apache.org/jira/browse/DRILL-8173 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Priority: Minor > > The newest httpcomponent version is 5.5, however, the version used by drill > is 4.1.4, actually is a little bit out of date. Besides, the more popular > tool is okhttp, which is also adopted by drill. So I will replace all usages > of httpcomponent to okhttp, for dependency optimization purposes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-7978) Fixed Width Format Plugin
[ https://issues.apache.org/jira/browse/DRILL-7978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-7978: Fix Version/s: (was: 1.21.0) > Fixed Width Format Plugin > - > > Key: DRILL-7978 > URL: https://issues.apache.org/jira/browse/DRILL-7978 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Megan Foss >Priority: Major > > Developing format plugin to parse fixed width files. > Fixed Width Text File Definition: > https://www.oracle.com/webfolder/technetwork/data-quality/edqhelp/Content/introduction/getting_started/configuring_fixed_width_text_file_formats.htm -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8117. --- Resolution: Fixed > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-7718) Apache Issue with page nginx
[ https://issues.apache.org/jira/browse/DRILL-7718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-7718: Fix Version/s: (was: 1.21.0) > Apache Issue with page nginx > > > Key: DRILL-7718 > URL: https://issues.apache.org/jira/browse/DRILL-7718 > Project: Apache Drill > Issue Type: Test > Components: Server >Affects Versions: 1.17.0 > Environment: is there a solution to fix this https://kongashare.com >Reporter: Colbert >Priority: Major > Labels: newbie > > I have a problem if my Apache server is overloaded. even though the > specifications are very high. for example, my website is [sheet > music|https://kongashare.com] accessed by more than 1000 people, then not > responding. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8135) Option to prevent Impersonation for Metastore
[
https://issues.apache.org/jira/browse/DRILL-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8135:
Fix Version/s: (was: 1.21.0)
> Option to prevent Impersonation for Metastore
> -
>
> Key: DRILL-8135
> URL: https://issues.apache.org/jira/browse/DRILL-8135
> Project: Apache Drill
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: 1.20.0
> Environment: * drill-1.20.0-hadoop2
> * hadoop 2.9.2
> * ranger 1.2.0
>Reporter: Christian Pfarr
>Priority: Minor
> Labels: iceberg, metastore
> Attachments: analyze-table-refresh-metadata.json
>
>
> After enabling iceberg metastore within a kerberized hadoop cluster and
> enabled impersonation, i have to give all users full access to the metastore
> directory in hdfs.
> If not, i get an _Permission denied_ when a non-admistator tries to call
> {code:java}
> analyze table hdfs.`dataset` refresh metadata{code}
> I attached the profile with the permission denied error from hdfs/ranger.
> Im not sure if i should call this a bug, because drill should impersonate
> everything, and so its also impersonating the iceberg metastore. But as an
> admin i dont want to give all users full write access to the metastore. A
> unexperienced user could accidentally overwrite something and corrupt the
> iceberg table or delete metadata completly.
> Maybe we could add a option to drill-metastore-overwrite.conf
> {code:java}
> drill.metastore: {
> implementation.class: "org.apache.drill.metastore.iceberg.IcebergMetastore",
> iceberg: {
> location: {
> relative_path: "hdfs://nameservice/drill-metastore"
> },
> impersonation: {
> enabled: true,
> max_chained_user_hops: 3
> }
> }
> }{code}
> In my case, i would of course disable it, but to match the behaviour of the
> general impersonation it could look like this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-6800) Simplify packaging of Jdbc-all jar
[ https://issues.apache.org/jira/browse/DRILL-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-6800: Fix Version/s: (was: 1.21.0) > Simplify packaging of Jdbc-all jar > -- > > Key: DRILL-6800 > URL: https://issues.apache.org/jira/browse/DRILL-6800 > Project: Apache Drill > Issue Type: Improvement > Components: Client - JDBC >Reporter: Sorabh Hamirwasia >Priority: Major > Fix For: Future > > > Today Jdbc-all package is created using drill-java-exec as dependency and > then excluding unnecessary dependency. Also there is size check for jdbc-all > jar to avoid including any unwanted dependency. But configured size has > increased over time and doesn't really provide a good mechanism to enforce > small footprint of jdbc-all jar. Following are some recommendation to improve > it: > 1) Divide java-exec module into separate client/server and common module > 2) Have size check for client artifact only. > 3) Update jdbc-all pom to include newly created client artifact and jdbc > driver artifact. > * Have multiple profiles to include and exclude any profile specific > dependency. For > example MapR profile will exclude hadoop dependency whereas apache profile > will > include it. > * We can create 2 artifacts for jdbc-all: one with and other without (for > smaller jar size) Hadoop dependencies. > 4) Update client side protobuf to not have server side definitions like > QueryProfile / CoreOperatorType etc -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-5562) Vector types IntervalYear, IntervalDay and Interval are of the wrong width
[
https://issues.apache.org/jira/browse/DRILL-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-5562:
Fix Version/s: (was: 1.21.0)
> Vector types IntervalYear, IntervalDay and Interval are of the wrong width
> --
>
> Key: DRILL-5562
> URL: https://issues.apache.org/jira/browse/DRILL-5562
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Priority: Major
>
> Drill provides three interval types, described in `ValueVectorTypes.tdd`:
> * {{IntervalYear}}: a duration in months (sic)
> * {{IntervalDay}}: a duration in days and ms.
> * {{Interval}}: a duration in months, days and ms.
> The file defines the width of each "field" (ms, days, months) as an int: 4
> bytes. But, the total vector width is wrong:
> * {{IntervalYear}}: 8 bytes (should be 4: for months)
> * {{IntervalDay}}: 12 bytes (should be 8: for days and ms.)
> * {{Interval}}: 16 bytes (should be 12: for months, days and ms.)
> It could be that the extra 4 bytes is supposed to be for a time zone. But,
> time zones don't apply to intervals: an hour is the same duration everywhere
> on earth.
> Since an interval does not contain a point in time, a time-zone is not useful
> even for daylight savings time adjustments.
> The code for each type reflects the "missing" 4 bytes. For example, for the
> 12-byte {{IntervalDay}} vector:
> {code}
> public void set(int index, int days, int milliseconds) {
> final int offsetIndex = index * VALUE_WIDTH;
> data.setInt(offsetIndex, days);
> data.setInt((offsetIndex + 4), milliseconds);
> }
> {code}
> Note also that the Drill IntervalDay need not be two fields wide. Except on a
> leap second, a day has a fixed number of milliseconds. And, the only way to
> compensate for a leap second is to know a point in time, which the interval
> does not have. Even if measured across a leap second, an interval of a minute
> is always 60 seconds. It is only when doing:
> {code}
> end date/time = start date/time + interval
> {code}
> is the leap second even needed.
> Although the ISO format expresses intervals as a tuple of (year, month, day,
> hour, minute, second), the same value can be expressed as (months, ms) (with
> the proper conversions), so Drill's interval types need only be 4 and 8 bytes
> wide.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-6527) Update option name for Drill Parquet native reader
[ https://issues.apache.org/jira/browse/DRILL-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-6527: Fix Version/s: (was: 1.21.0) > Update option name for Drill Parquet native reader > -- > > Key: DRILL-6527 > URL: https://issues.apache.org/jira/browse/DRILL-6527 > Project: Apache Drill > Issue Type: Task > Components: Storage - Hive, Storage - Parquet >Affects Versions: 1.14.0 >Reporter: Vitalii Diravka >Priority: Minor > > The old option name to enable Drill parquet reader is > "store.hive.optimize_scan_with_native_readers". > Starting from DRILL-6454 one new native reader is introduced, therefore more > precise option name is added for parquet native reader too. > A new option name for parquet reader is > "store.hive.parquet.optimize_scan_with_native_reader". > The old one is deprecated and should be removed. > Since options are considered as API we can remove them only in major > releases, i.e. in Drill 2.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-7088) Remove deprecated decimal data types
[ https://issues.apache.org/jira/browse/DRILL-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-7088: Fix Version/s: (was: 1.21.0) > Remove deprecated decimal data types > > > Key: DRILL-7088 > URL: https://issues.apache.org/jira/browse/DRILL-7088 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.14.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Major > > In DRILL-6094 was introduced VarDecimal data type and was removed usage of > other decimal data types, but they were left to handle the case when user use > UDF which contains old decimal data types. > For Drill 2.0 these deprecated decimal data types should be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-5531) Nullable value vectors unnecessarily inherit from BaseDataValueVector
[
https://issues.apache.org/jira/browse/DRILL-5531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-5531:
Fix Version/s: (was: 1.21.0)
> Nullable value vectors unnecessarily inherit from BaseDataValueVector
> --
>
> Key: DRILL-5531
> URL: https://issues.apache.org/jira/browse/DRILL-5531
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Consider a typical nullable value vector: NullableVarCharVector. The members
> are:
> {code}
> private final UInt1Vector bits = new UInt1Vector(bitsField, allocator);
> private final VarCharVector values = new VarCharVector(field, allocator);
> {code}
> Inheritance is defined as:
> {code}
> class NullableVarCharVector extends BaseDataValueVector ...
> {code}
> But, notice the members of {{BaseDataValueVector}}:
> {code}
> protected DrillBuf data;
> {code}
> It is confusing that a nullable vector has a data buffer that is never used.
> Instead, the data is carried by the {{values}} member.
> Improvements:
> * Define a new {{BaseNullableVector}} class that does not extend
> {{BaseDataValueVector}}.
> * Move the {{bits}} member to the new base class rather than declaring it
> anew in each generated class.
> * Adjust member functions to match.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-5283) Support "is not present" as subtype of "is null" for JSON data
[ https://issues.apache.org/jira/browse/DRILL-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-5283: Fix Version/s: (was: 1.21.0) > Support "is not present" as subtype of "is null" for JSON data > -- > > Key: DRILL-5283 > URL: https://issues.apache.org/jira/browse/DRILL-5283 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.10.0 >Reporter: Paul Rogers >Priority: Major > > JSON files consist of a series of "objects", each of which has name/value > pairs. Values can be in one of three states: > * Not present (the value does not appear) > * Null (the name appears and the value is null) > * Non-null (the field is one of the JSON data types) > Drill, however, has only a single null state and so Drill collapses "not > present" and "null" into the same state. > The not-present and present-but-null states work identically for calculations > inside Drill. But, when doing a CTAS from JSON to JSON, the collapsed state > means that the user does not get out of Drill what was put in: all null > values either appear as null values, or do not appear at all (depending on > Drill version.) > This ticket asks to repurpose the "bit" fields in nullable vectors. Rename > the vector to "nullState". Then, use these values: > * 0: value is set > * 1: value is null > * 3: value is not present > The column is null if the null state is non-zero. The column is not null if > the null state is 0. > This change requires reversing the "polarity" of the bit field, and so is a > major change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-5360) Timestamp type documented as UTC, implemented as local time
[
https://issues.apache.org/jira/browse/DRILL-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-5360:
Fix Version/s: (was: 1.21.0)
> Timestamp type documented as UTC, implemented as local time
> ---
>
> Key: DRILL-5360
> URL: https://issues.apache.org/jira/browse/DRILL-5360
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Priority: Critical
>
> The Drill documentation implies that the {{Timestamp}} type is in UTC:
> bq. JDBC timestamp in year, month, date hour, minute, second, and optional
> milliseconds format: -MM-dd HH:mm:ss.SSS. ... TIMESTAMP literals: Drill
> stores values in Coordinated Universal Time (UTC). Drill supports time
> functions in the range 1971 to 2037. ... Drill does not support TIMESTAMP
> with time zone.
> The above is ambiguous. The first part talks about JDBC timestamps. From the
> JDK Javadoc:
> bq. Timestamp: A thin wrapper around java.util.Date. ... Date class is
> intended to reflect coordinated universal time (UTC)...
> So, a JDBC timestamp is intended to represent time in UTC. (The "indented to
> reflect" statement leaves open the possibility of misusing {{Date}} to
> represent times in other time zones. This was common practice in early Java
> development and was the reason for the eventual development of the Joda, then
> Java 8 date/time classes.)
> The Drill documentation implies that timestamp *literals* are in UTC, but a
> careful read of the documentation does allow an interpretation that the
> internal representation can be other than UTC. If this is true, then we would
> also rely on a liberal reading of the Java `Timestamp` class to also not be
> UTC. (Or, we rely on the Drill JDBC driver to convert from the (unknown)
> server time zone to a UTC value returned by the Drill JDBC client.)
> Still, a superficial reading (and common practice) would suggest that a Drill
> Timestamp should be in UTC.
> However, a test on a Mac, with an embedded Drillbit (run in the Pacific time
> zone, with Daylight Savings Time in effect) shows that the Timestamp binary
> value is actual local time:
> {code}
> long before = System.currentTimeMillis();
> long value = getDateValue(client, "SELECT NOW() FROM (VALUES(1))" );
> double hrsDiff = (value - before) / (1000.00 * 60 * 60);
> System.out.println("Hours: " + hrsDiff);
> {code}
> The above gets the actual UTC time from Java. Then, it runs a query that gets
> Drill's idea of the current time using the {{NOW()}} function. (The
> {{getDateValue}} function uses the new test framework to access the actual
> {{long}} value from the returned value vector.) Finally, we compute the
> difference between the two times, converted to hours. Output:
> {code}
> Hours: -6.975
> {code}
> As it turns out, this is the difference between UTC and PDT. So, the time is
> in local time, not UTC.
> Since the documentation and implementation are both ambiguous, it is hard to
> know the intent of the Drill Timestamp. Clearly, common practice is to use
> UTC. But, there is wiggle-room.
> If the Timestamp value is supposed to be local time, then Drill should
> provide a function to return the server's time zone offset (in ms) from UTC
> so that the client can to the needed local-to-UTC conversion to get a true
> timestamp.
> On the other hand, if the Timestamp is supposed to be UTC (per common
> practice), then {{NOW()}} should not report local time, it should return UTC.
> Further, if {{NOW()}} returns local time, but Timestamp literals are UTC,
> then it is hard to see how any query can be rationally written if one
> timestamp value is local, but a literal is UTC.
> So, job #1 is to define the Timestamp semantics. Then, use that to figure out
> where the bug lies to make implementation consistent with documentation (or
> visa-versa.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-4791) Provide a light-weight, versioned client API
[ https://issues.apache.org/jira/browse/DRILL-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4791: Fix Version/s: (was: 1.21.0) > Provide a light-weight, versioned client API > > > Key: DRILL-4791 > URL: https://issues.apache.org/jira/browse/DRILL-4791 > Project: Apache Drill > Issue Type: New Feature >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > > Drill's existing client APIs are "industrial strength" - they provide full > access to the sophisticated distributed, columnar RPCs which Drill uses > internall. However, they are too complex for most client needs. Provide a > simpler API optimized for clients: row-based result sets, synchronous, etc. > At the same time, Drill clients must currently link with the same version of > Drill code as is running on the Drill cluster. This forces clients to upgrade > in lock-step with the cluster. Allow Drill clients to be upgraded after (or > even before) the Drill cluster to simplify management of desktop apps that > use Drill. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4741) sqlline scripts should differentiate embedded vs remote config
[ https://issues.apache.org/jira/browse/DRILL-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4741: Fix Version/s: (was: 1.21.0) > sqlline scripts should differentiate embedded vs remote config > -- > > Key: DRILL-4741 > URL: https://issues.apache.org/jira/browse/DRILL-4741 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.8.0 >Reporter: Paul Rogers >Priority: Minor > > $DRILL_HOME/bin contains four sqlline-related scripts: > sqlline -- main script for running sqlline > drill-conf — Wrapper for sqlline, uses drill config to find Drill. Seems this > one needs fixing to use a config other than the hard-coded $DRILL_HOME/conf > location. > drill-embedded — Starts a drill “embedded” in SqlLine, using a local ZK. > drill-localhost — Wrapper for sqlline, uses a local ZK. > The last three turn around and call sqlline. > Behind the scenes, the script call drill-config.sh and drill-env.sh to do > setup. > Note, however that we run Sqlline and Drill in three distinct configurations: > sqlline as client: should run with light memory > drillbit as daemon: should run with full memory use > sqline with embedded drillbit: sqlline needs to run with Drillbit memory > options. > Today, sqlline always uses the Drillbit memory options (and VM options) which > results in too much memory and port conflicts when running client-only. > Provide sqlline specific VM and memory options. Then, the tricky bit, use > them only when Drill is not embedded. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4600) Document Public Compatibility Commitments
[ https://issues.apache.org/jira/browse/DRILL-4600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4600: Fix Version/s: (was: 1.21.0) > Document Public Compatibility Commitments > - > > Key: DRILL-4600 > URL: https://issues.apache.org/jira/browse/DRILL-4600 > Project: Apache Drill > Issue Type: New Feature >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4543) Advertise Drill-bit ports, status, capabilities in ZooKeeper
[ https://issues.apache.org/jira/browse/DRILL-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4543: Fix Version/s: (was: 1.21.0) > Advertise Drill-bit ports, status, capabilities in ZooKeeper > > > Key: DRILL-4543 > URL: https://issues.apache.org/jira/browse/DRILL-4543 > Project: Apache Drill > Issue Type: Sub-task > Components: Server >Reporter: Paul Rogers >Priority: Major > Labels: documentation > > Today Drill uses ZooKeeper (ZK) to advertise the existence of a Drill-bit, > providing the host name/IP Address of the Drill-bit and the ports used, > encoded in Protobuf format. All other information (status, CPUs, memory) are > assumed to be the same across all Drill-bits in the cluster as specified in > the Drill config file. (Amended to reflect 1.6 behavior.) > Moving forward, as Drill becomes more sophisticated, Drill should advertise > the specifics of each Drill-bit so that one Drill bit can differ from another. > For example, when running on YARN, we need a way for Drill to gracefully shut > down. Advertising a status of Ready or Unavailable will help. Ready is the > normal state. Unavailable means the Drill-bit will finish in-flight queries, > but won't accept new ones. (The actual status is a separate enhancement.) > In a YARN cluster, Drill should take advantage of machines with more memory, > but live with machines with less. (Perhaps some are newer, some are older or > more heavily loaded.) Drill should use ZK to identify its available memory > and CPUs so that the planner can use them. (Use of the info is a separate > enhancement.) > There may be times when two drill bits run on a single machine. If so, they > must use separate ports. So, each Drill-bit should advertise its ports in ZK. > For backward compatibility, the information is optional; if not present, the > receiver should assume the information defaults to that in the config file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8184) Unexpected exception during fragment initialization: ELASTICSEARCH != NONE
[ https://issues.apache.org/jira/browse/DRILL-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8184: Fix Version/s: (was: 1.21.0) > Unexpected exception during fragment initialization: ELASTICSEARCH != NONE > -- > > Key: DRILL-8184 > URL: https://issues.apache.org/jira/browse/DRILL-8184 > Project: Apache Drill > Issue Type: Bug > Components: Storage - ElasticSearch >Affects Versions: 1.20.0 > Environment: OS: centos 7 >Reporter: guihui >Priority: Major > > In the runtime environment of drill, there are two storage plugins, > Elasticsearch and hive. When querying a table in hive, the error message is > as follows: > Caused by: java.lang.AssertionError: ELASTICSEARCH != NONE > at > org.apache.calcite.adapter.elasticsearch.ElasticsearchAggregate.(ElasticsearchAggregate.java:66) > at > org.apache.calcite.adapter.elasticsearch.ElasticsearchAggregate.copy(ElasticsearchAggregate.java:112) > at org.apache.calcite.rel.core.Aggregate.copy(Aggregate.java:211) > at > org.apache.calcite.rel.rules.FilterAggregateTransposeRule.onMatch(FilterAggregateTransposeRule.java:128) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:633) > at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:327) > at > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:405) > at > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:351) > at > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:245) > at > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:308) > at > org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:173) > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:399) > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:181) > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:146) > at > org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:111) > at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593) > at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274) > Query sql as follow: > WITH res AS ( > SELECT sr_store_sk AS ctr_store_sk, SUM(SR_FEE) AS ctr_total_return > FROM `hive`.`tpcds`.`store_returns` > GROUP BY sr_store_sk > ) > SELECT t1.ctr_total_return > FROM res t1 > WHERE t1.ctr_total_return > ( > SELECT AVG(ctr_total_return) > FROM res t2 > WHERE t1.ctr_store_sk = t2.ctr_store_sk > ); > > The ddl of `hive`.`tpcds`.`store_returns` as follow: > create table store_returns > ( > sr_returned_date_sk int , > sr_return_time_sk int , > sr_item_sk int , > sr_customer_sk int , > sr_cdemo_sk int , > sr_hdemo_sk int , > sr_addr_sk int , > sr_store_sk int , > sr_reason_sk int , > sr_ticket_number int , > sr_return_quantity int , > sr_return_amt decimal(7,2) , > sr_return_tax decimal(7,2) , > sr_return_amt_inc_tax decimal(7,2) , > sr_fee decimal(7,2) , > sr_return_ship_cost decimal(7,2) , > sr_refunded_cash decimal(7,2) , > sr_reversed_charge decimal(7,2) , > sr_store_credit decimal(7,2) , > sr_net_loss decimal(7,2) > )row format delimited fields terminated by '|' > NULL DEFINED AS '' > STORED AS TEXTFILE; > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4538) Turn on Union type by default
[ https://issues.apache.org/jira/browse/DRILL-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4538: Fix Version/s: (was: 1.21.0) > Turn on Union type by default > - > > Key: DRILL-4538 > URL: https://issues.apache.org/jira/browse/DRILL-4538 > Project: Apache Drill > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8185) EVF 2 doen't handle map arrays or nested maps
[ https://issues.apache.org/jira/browse/DRILL-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8185: Fix Version/s: (was: 1.21.0) > EVF 2 doen't handle map arrays or nested maps > - > > Key: DRILL-8185 > URL: https://issues.apache.org/jira/browse/DRILL-8185 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.20.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > > When converting Avro, Luoc found two bugs in how EVF 2 (the projection > mechanism) handles map array and nested maps -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4536) Modify Project such that NULL_IF_NULL handling operates columnar
[ https://issues.apache.org/jira/browse/DRILL-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4536: Fix Version/s: (was: 1.21.0) > Modify Project such that NULL_IF_NULL handling operates columnar > > > Key: DRILL-4536 > URL: https://issues.apache.org/jira/browse/DRILL-4536 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4535) Remove declarative null type from execution
[ https://issues.apache.org/jira/browse/DRILL-4535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4535: Fix Version/s: (was: 1.21.0) > Remove declarative null type from execution > --- > > Key: DRILL-4535 > URL: https://issues.apache.org/jira/browse/DRILL-4535 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4534) Replace declarative null type with observed null type in execution layer
[ https://issues.apache.org/jira/browse/DRILL-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4534: Fix Version/s: (was: 1.21.0) > Replace declarative null type with observed null type in execution layer > > > Key: DRILL-4534 > URL: https://issues.apache.org/jira/browse/DRILL-4534 > Project: Apache Drill > Issue Type: Improvement >Reporter: Jacques Nadeau >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-4417) Umbrella: Faster Metadata Queries
[ https://issues.apache.org/jira/browse/DRILL-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-4417: Fix Version/s: (was: 1.21.0) > Umbrella: Faster Metadata Queries > - > > Key: DRILL-4417 > URL: https://issues.apache.org/jira/browse/DRILL-4417 > Project: Apache Drill > Issue Type: New Feature >Reporter: Jacques Nadeau >Priority: Major > > Drill's metadata return operations should be faster to provide a better BI > experience. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8374) Set the Drill development version to 1.21.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/DRILL-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8374. --- Resolution: Fixed > Set the Drill development version to 1.21.0-SNAPSHOT > > > Key: DRILL-8374 > URL: https://issues.apache.org/jira/browse/DRILL-8374 > Project: Apache Drill > Issue Type: Task >Reporter: James Turton >Assignee: James Turton >Priority: Trivial > > Changes the Maven version numbers in the Drill master branch from 2.0.0 to > 1.21.0. Discussion in the Drill mailing list established that the project > would prefer to do a release in the near future than to wait to build up a > changset for which a version jump to 2.0 would be appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8374) Set the Drill development version to 1.21.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/DRILL-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8374: Fix Version/s: (was: 1.21.0) > Set the Drill development version to 1.21.0-SNAPSHOT > > > Key: DRILL-8374 > URL: https://issues.apache.org/jira/browse/DRILL-8374 > Project: Apache Drill > Issue Type: Task >Reporter: James Turton >Assignee: James Turton >Priority: Trivial > > Changes the Maven version numbers in the Drill master branch from 2.0.0 to > 1.21.0. Discussion in the Drill mailing list established that the project > would prefer to do a release in the near future than to wait to build up a > changset for which a version jump to 2.0 would be appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8374) Set the Drill development version to 1.21.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/DRILL-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8374: - > Set the Drill development version to 1.21.0-SNAPSHOT > > > Key: DRILL-8374 > URL: https://issues.apache.org/jira/browse/DRILL-8374 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: James Turton >Assignee: James Turton >Priority: Trivial > Fix For: 1.21.0 > > > Changes the Maven version numbers in the Drill master branch from 2.0.0 to > 1.21.0. Discussion in the Drill mailing list established that the project > would prefer to do a release in the near future than to wait to build up a > changset for which a version jump to 2.0 would be appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8374) Set the Drill development version to 1.21.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/DRILL-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8374: Affects Version/s: (was: 1.21.0) > Set the Drill development version to 1.21.0-SNAPSHOT > > > Key: DRILL-8374 > URL: https://issues.apache.org/jira/browse/DRILL-8374 > Project: Apache Drill > Issue Type: Task >Reporter: James Turton >Assignee: James Turton >Priority: Trivial > Fix For: 1.21.0 > > > Changes the Maven version numbers in the Drill master branch from 2.0.0 to > 1.21.0. Discussion in the Drill mailing list established that the project > would prefer to do a release in the near future than to wait to build up a > changset for which a version jump to 2.0 would be appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8368) Update Yauaa to 7.9.0
[ https://issues.apache.org/jira/browse/DRILL-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8368: Fix Version/s: 1.21.0 > Update Yauaa to 7.9.0 > - > > Key: DRILL-8368 > URL: https://issues.apache.org/jira/browse/DRILL-8368 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.3 >Reporter: Niels Basjes >Assignee: Niels Basjes >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8368) Update Yauaa to 7.9.0
[ https://issues.apache.org/jira/browse/DRILL-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8368: Affects Version/s: 1.20.3 > Update Yauaa to 7.9.0 > - > > Key: DRILL-8368 > URL: https://issues.apache.org/jira/browse/DRILL-8368 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.3 >Reporter: Niels Basjes >Assignee: Niels Basjes >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8368) Update Yauaa to 7.9.0
[ https://issues.apache.org/jira/browse/DRILL-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8368: - > Update Yauaa to 7.9.0 > - > > Key: DRILL-8368 > URL: https://issues.apache.org/jira/browse/DRILL-8368 > Project: Apache Drill > Issue Type: Improvement >Reporter: Niels Basjes >Assignee: Niels Basjes >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8363) upgrade postgresql to 42.4.3 due to security issue
[ https://issues.apache.org/jira/browse/DRILL-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8363: Affects Version/s: 1.20.3 > upgrade postgresql to 42.4.3 due to security issue > -- > > Key: DRILL-8363 > URL: https://issues.apache.org/jira/browse/DRILL-8363 > Project: Apache Drill > Issue Type: Task > Components: Storage - JDBC >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > https://github.com/advisories/GHSA-562r-vg33-8x8h -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8363) upgrade postgresql to 42.4.3 due to security issue
[ https://issues.apache.org/jira/browse/DRILL-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8363. --- Resolution: Fixed > upgrade postgresql to 42.4.3 due to security issue > -- > > Key: DRILL-8363 > URL: https://issues.apache.org/jira/browse/DRILL-8363 > Project: Apache Drill > Issue Type: Task > Components: Storage - JDBC >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.0 > > > https://github.com/advisories/GHSA-562r-vg33-8x8h -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8363) upgrade postgresql to 42.4.3 due to security issue
[ https://issues.apache.org/jira/browse/DRILL-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8363: Fix Version/s: 1.21.0 > upgrade postgresql to 42.4.3 due to security issue > -- > > Key: DRILL-8363 > URL: https://issues.apache.org/jira/browse/DRILL-8363 > Project: Apache Drill > Issue Type: Task > Components: Storage - JDBC >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.0 > > > https://github.com/advisories/GHSA-562r-vg33-8x8h -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8363) upgrade postgresql to 42.4.3 due to security issue
[ https://issues.apache.org/jira/browse/DRILL-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8363: - > upgrade postgresql to 42.4.3 due to security issue > -- > > Key: DRILL-8363 > URL: https://issues.apache.org/jira/browse/DRILL-8363 > Project: Apache Drill > Issue Type: Task > Components: Storage - JDBC >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > https://github.com/advisories/GHSA-562r-vg33-8x8h -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8362: Fix Version/s: 1.21.0 > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.0 > > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8362: - > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8362) upgrade excel-streaming-reader v4.0.5
[ https://issues.apache.org/jira/browse/DRILL-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8362: Affects Version/s: 1.20.3 > upgrade excel-streaming-reader v4.0.5 > - > > Key: DRILL-8362 > URL: https://issues.apache.org/jira/browse/DRILL-8362 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.3 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > A few small issues have been fixed -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8379) Update Calcite to 1.33.0
[ https://issues.apache.org/jira/browse/DRILL-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8379. --- > Update Calcite to 1.33.0 > > > Key: DRILL-8379 > URL: https://issues.apache.org/jira/browse/DRILL-8379 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8379) Update Calcite to 1.33.0
[ https://issues.apache.org/jira/browse/DRILL-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8379: Affects Version/s: 1.21.0 > Update Calcite to 1.33.0 > > > Key: DRILL-8379 > URL: https://issues.apache.org/jira/browse/DRILL-8379 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8379) Update Calcite to 1.33.0
[ https://issues.apache.org/jira/browse/DRILL-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8379: Fix Version/s: 1.21.0 > Update Calcite to 1.33.0 > > > Key: DRILL-8379 > URL: https://issues.apache.org/jira/browse/DRILL-8379 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8396) Update checkstyle version
[ https://issues.apache.org/jira/browse/DRILL-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8396: Fix Version/s: 1.21.0 > Update checkstyle version > - > > Key: DRILL-8396 > URL: https://issues.apache.org/jira/browse/DRILL-8396 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Minor > Fix For: 1.21.0 > > > Update com.puppycrawl.tools:checkstyle version to the latest one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8397) Drill prints warnings to console when starting it
[
https://issues.apache.org/jira/browse/DRILL-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton reopened DRILL-8397:
-
> Drill prints warnings to console when starting it
> -
>
> Key: DRILL-8397
> URL: https://issues.apache.org/jira/browse/DRILL-8397
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When starting the drill in embedded mode, it prints the following warnings:
> {noformat}
> 11:19:55,482 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This
> is logback-classic version 1.4.5
> 11:19:55,499 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
> NOT find resource [logback-test.xml]
> 11:19:55,503 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
> resource [logback.xml] at
> [file:/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/conf/logback.xml]
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 73
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 78
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 91
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [STDOUT]
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
> 11:19:55,658 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [QUERY]
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,697 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@66ac5762 - No
> compression will be used
> 11:19:55,699 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [FILE]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@797cf65c - No
> compression will be used
> 11:19:55,701 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log]
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting
> additivity of logger [org.apache.drill] to false
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LevelModelHandler - org.apache.drill
> level set to INFO
> 11:19:55,702 |-INFO in
> ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching
> appender named [FILE] to Logger[org.apache.drill]
> 11:19:55,703 |-INFO in
>
[jira] [Updated] (DRILL-8397) Drill prints warnings to console when starting it
[
https://issues.apache.org/jira/browse/DRILL-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8397:
Affects Version/s: 1.21.0
> Drill prints warnings to console when starting it
> -
>
> Key: DRILL-8397
> URL: https://issues.apache.org/jira/browse/DRILL-8397
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When starting the drill in embedded mode, it prints the following warnings:
> {noformat}
> 11:19:55,482 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This
> is logback-classic version 1.4.5
> 11:19:55,499 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
> NOT find resource [logback-test.xml]
> 11:19:55,503 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
> resource [logback.xml] at
> [file:/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/conf/logback.xml]
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 73
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 78
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 91
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [STDOUT]
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
> 11:19:55,658 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [QUERY]
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,697 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@66ac5762 - No
> compression will be used
> 11:19:55,699 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [FILE]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@797cf65c - No
> compression will be used
> 11:19:55,701 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log]
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting
> additivity of logger [org.apache.drill] to false
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LevelModelHandler - org.apache.drill
> level set to INFO
> 11:19:55,702 |-INFO in
> ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching
> appender named [FILE] to
[jira] [Closed] (DRILL-8397) Drill prints warnings to console when starting it
[
https://issues.apache.org/jira/browse/DRILL-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8397.
---
Resolution: Fixed
> Drill prints warnings to console when starting it
> -
>
> Key: DRILL-8397
> URL: https://issues.apache.org/jira/browse/DRILL-8397
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When starting the drill in embedded mode, it prints the following warnings:
> {noformat}
> 11:19:55,482 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This
> is logback-classic version 1.4.5
> 11:19:55,499 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
> NOT find resource [logback-test.xml]
> 11:19:55,503 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
> resource [logback.xml] at
> [file:/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/conf/logback.xml]
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 73
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 78
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 91
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [STDOUT]
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
> 11:19:55,658 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [QUERY]
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,697 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@66ac5762 - No
> compression will be used
> 11:19:55,699 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [FILE]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@797cf65c - No
> compression will be used
> 11:19:55,701 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log]
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting
> additivity of logger [org.apache.drill] to false
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LevelModelHandler - org.apache.drill
> level set to INFO
> 11:19:55,702 |-INFO in
> ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching
> appender named [FILE] to Logger[org.apache.drill]
> 11:19:55,703 |-INFO in
>
[jira] [Updated] (DRILL-8396) Update checkstyle version
[ https://issues.apache.org/jira/browse/DRILL-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8396: Affects Version/s: 1.21.0 > Update checkstyle version > - > > Key: DRILL-8396 > URL: https://issues.apache.org/jira/browse/DRILL-8396 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Minor > > Update com.puppycrawl.tools:checkstyle version to the latest one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8397) Drill prints warnings to console when starting it
[
https://issues.apache.org/jira/browse/DRILL-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8397:
Affects Version/s: (was: 1.21.0)
> Drill prints warnings to console when starting it
> -
>
> Key: DRILL-8397
> URL: https://issues.apache.org/jira/browse/DRILL-8397
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When starting the drill in embedded mode, it prints the following warnings:
> {noformat}
> 11:19:55,482 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This
> is logback-classic version 1.4.5
> 11:19:55,499 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
> NOT find resource [logback-test.xml]
> 11:19:55,503 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
> resource [logback.xml] at
> [file:/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/conf/logback.xml]
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 73
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 78
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 91
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [STDOUT]
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
> 11:19:55,658 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [QUERY]
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,697 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@66ac5762 - No
> compression will be used
> 11:19:55,699 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [FILE]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@797cf65c - No
> compression will be used
> 11:19:55,701 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log]
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting
> additivity of logger [org.apache.drill] to false
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LevelModelHandler - org.apache.drill
> level set to INFO
> 11:19:55,702 |-INFO in
> ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching
> appender named [FILE] to Logger[org.apache.drill]
> 11:19:55,703
[jira] [Closed] (DRILL-8396) Update checkstyle version
[ https://issues.apache.org/jira/browse/DRILL-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8396. --- Resolution: Fixed > Update checkstyle version > - > > Key: DRILL-8396 > URL: https://issues.apache.org/jira/browse/DRILL-8396 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Minor > Fix For: 1.21.0 > > > Update com.puppycrawl.tools:checkstyle version to the latest one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8396) Update checkstyle version
[ https://issues.apache.org/jira/browse/DRILL-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8396: - > Update checkstyle version > - > > Key: DRILL-8396 > URL: https://issues.apache.org/jira/browse/DRILL-8396 > Project: Apache Drill > Issue Type: Task >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Minor > > Update com.puppycrawl.tools:checkstyle version to the latest one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8157) Replace buggy functions.cast_empty_string_to_null config option
[
https://issues.apache.org/jira/browse/DRILL-8157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8157:
Fix Version/s: (was: 1.21.0)
> Replace buggy functions.cast_empty_string_to_null config option
> ---
>
> Key: DRILL-8157
> URL: https://issues.apache.org/jira/browse/DRILL-8157
> Project: Apache Drill
> Issue Type: Bug
>Reporter: James Turton
>Assignee: James Turton
>Priority: Major
>
> Setting the config option {{drill.exec.functions.cast_empty_string_to_null}}
> causes errors when string literals containing dates and timestamps appear in
> a query because of its implementation as function over the entire query
> syntax tree. Reproduce as follows.
>
> {code:java}
> set `drill.exec.functions.cast_empty_string_to_null` = true;
> select * from cp.`tpch/lineitem.parquet` where l_shipdate = '1970-01-01';
> Error: SYSTEM ERROR: ClassCastException: class
> org.apache.drill.exec.expr.holders.NullableDateHolder cannot be cast to class
> org.apache.drill.exec.expr.holders.DateHolder
> (org.apache.drill.exec.expr.holders.NullableDateHolder and
> org.apache.drill.exec.expr.holders.DateHolder are in unnamed module of loader
> 'app'){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8288) Null Columns not being Written to GoogleSheets
[ https://issues.apache.org/jira/browse/DRILL-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8288: Fix Version/s: (was: 1.21.0) > Null Columns not being Written to GoogleSheets > -- > > Key: DRILL-8288 > URL: https://issues.apache.org/jira/browse/DRILL-8288 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 1.21.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > > When writing to GoogleSheets, null columns are not written which causes wrong > data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8286) GoogleSheets StoragePlugin displaying ClientID and ClientSecret in Config
[ https://issues.apache.org/jira/browse/DRILL-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8286: Fix Version/s: (was: 1.21.0) > GoogleSheets StoragePlugin displaying ClientID and ClientSecret in Config > - > > Key: DRILL-8286 > URL: https://issues.apache.org/jira/browse/DRILL-8286 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 1.21.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > > The GoogleSheets storage plugin is rendering the `clientID` and > `clientSecret` in the config body instead of in the credential provider. > This minor PR fixes that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8316) Convert Druid Storage Plugin to EVF & V2 JSON Reader
[ https://issues.apache.org/jira/browse/DRILL-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8316: Fix Version/s: (was: 1.21.0) > Convert Druid Storage Plugin to EVF & V2 JSON Reader > > > Key: DRILL-8316 > URL: https://issues.apache.org/jira/browse/DRILL-8316 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Druid >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8312) Convert Format Plugins to EVF V2
[ https://issues.apache.org/jira/browse/DRILL-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8312: Fix Version/s: (was: 1.21.0) > Convert Format Plugins to EVF V2 > > > Key: DRILL-8312 > URL: https://issues.apache.org/jira/browse/DRILL-8312 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.2 >Reporter: Charles Givre >Priority: Major > > This is a blanket ticket to convert all format plugins to EVF V2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8336) HTTP UDFs do not correctly handle results rooted in an array
[ https://issues.apache.org/jira/browse/DRILL-8336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8336: Fix Version/s: (was: 1.21.0) > HTTP UDFs do not correctly handle results rooted in an array > > > Key: DRILL-8336 > URL: https://issues.apache.org/jira/browse/DRILL-8336 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - HTTP >Affects Versions: 1.20.2 >Reporter: James Turton >Priority: Minor > > When the HTTP udfs http_get or http_request are used against an endpoint that > returns JSON data that is rooted with an array (or line-delimited object > sequence) then they return consecutive objects from that array with each > invocation until the end is reached, after which the next invocation fails > with an error. This can be reproduced by having a web server serve Drill's > donuts.json. > In addition, each invocation of these UDFs leaks a single file descriptor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8347) Complete the migration of the impersonation switch from SYSTEM to storage config
[ https://issues.apache.org/jira/browse/DRILL-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8347: Fix Version/s: (was: 1.21.0) > Complete the migration of the impersonation switch from SYSTEM to storage > config > > > Key: DRILL-8347 > URL: https://issues.apache.org/jira/browse/DRILL-8347 > Project: Apache Drill > Issue Type: Improvement > Components: Security >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Major > > DRILL-8155 introduced the configuration of the auth mode for a storage plugin > in the storage config. The legacy SYSTEM option drill.exec.impersonation was > not removed in that change and continues to control impersonation for > Hadoop-based plugins. All plugins should use the new authMode property of > their storage config to decided with mode of authentication, including > impersonation, at which point the drill.exec.impersonation SYSTEM option can > be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8351) UserRemoteException : SYSTEM ERROR: NullPointerException: Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a sc
[ https://issues.apache.org/jira/browse/DRILL-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8351. --- Resolution: Fixed > UserRemoteException :SYSTEM ERROR: NullPointerException: Schema is > currently null. You must call buildSchema(SelectionVectorMode) before this > container can return a schema. > > > Key: DRILL-8351 > URL: https://issues.apache.org/jira/browse/DRILL-8351 > Project: Apache Drill > Issue Type: Bug > Components: Client - Java > Environment: version 1.19 and 1.20.2 >Reporter: Ravi Kiran >Priority: Critical > Fix For: 1.21.0 > > Attachments: SUPPLIERS-20220720082618924.csv, > SUPPLIERS-20220720082618936-2.csv, SUPPLIERS-20220720082618946-1.csv, > SUPPLIERS-20220720082618958-2.csv, SUPPLIERS-2022072008261930.csv > > > Hi Team, > I have more than 100k records in my directory, but I am able to fetch only > 100k records, not able to fetch after 100k records by using limit and offset. > I am attaching files also. > > if i use 1.17 or 1.14 able to fetch all records. > > SELECT * FROM dfs. `C:\Users\ravik\Downloads\test_folder` LIMIT 1 OFFSET > 9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8351) UserRemoteException : SYSTEM ERROR: NullPointerException: Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a s
[ https://issues.apache.org/jira/browse/DRILL-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8351: Fix Version/s: 1.21.0 > UserRemoteException :SYSTEM ERROR: NullPointerException: Schema is > currently null. You must call buildSchema(SelectionVectorMode) before this > container can return a schema. > > > Key: DRILL-8351 > URL: https://issues.apache.org/jira/browse/DRILL-8351 > Project: Apache Drill > Issue Type: Bug > Components: Client - Java > Environment: version 1.19 and 1.20.2 >Reporter: Ravi Kiran >Priority: Critical > Fix For: 1.21.0 > > Attachments: SUPPLIERS-20220720082618924.csv, > SUPPLIERS-20220720082618936-2.csv, SUPPLIERS-20220720082618946-1.csv, > SUPPLIERS-20220720082618958-2.csv, SUPPLIERS-2022072008261930.csv > > > Hi Team, > I have more than 100k records in my directory, but I am able to fetch only > 100k records, not able to fetch after 100k records by using limit and offset. > I am attaching files also. > > if i use 1.17 or 1.14 able to fetch all records. > > SELECT * FROM dfs. `C:\Users\ravik\Downloads\test_folder` LIMIT 1 OFFSET > 9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8351) UserRemoteException : SYSTEM ERROR: NullPointerException: Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a
[ https://issues.apache.org/jira/browse/DRILL-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688847#comment-17688847 ] James Turton commented on DRILL-8351: - Hi, we're releasing Drill 1.21 instead of 2.0. The first RC should be uploaded today. > UserRemoteException :SYSTEM ERROR: NullPointerException: Schema is > currently null. You must call buildSchema(SelectionVectorMode) before this > container can return a schema. > > > Key: DRILL-8351 > URL: https://issues.apache.org/jira/browse/DRILL-8351 > Project: Apache Drill > Issue Type: Bug > Components: Client - Java > Environment: version 1.19 and 1.20.2 >Reporter: Ravi Kiran >Priority: Critical > Attachments: SUPPLIERS-20220720082618924.csv, > SUPPLIERS-20220720082618936-2.csv, SUPPLIERS-20220720082618946-1.csv, > SUPPLIERS-20220720082618958-2.csv, SUPPLIERS-2022072008261930.csv > > > Hi Team, > I have more than 100k records in my directory, but I am able to fetch only > 100k records, not able to fetch after 100k records by using limit and offset. > I am attaching files also. > > if i use 1.17 or 1.14 able to fetch all records. > > SELECT * FROM dfs. `C:\Users\ravik\Downloads\test_folder` LIMIT 1 OFFSET > 9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8355) Embedded drill unable to start with security configuration
[
https://issues.apache.org/jira/browse/DRILL-8355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8355.
---
Resolution: Information Provided
> Embedded drill unable to start with security configuration
> ---
>
> Key: DRILL-8355
> URL: https://issues.apache.org/jira/browse/DRILL-8355
> Project: Apache Drill
> Issue Type: Bug
> Components: Server, Security
>Affects Versions: 1.17.0
>Reporter: Nitul
>Priority: Major
>
> Security is enabled in drill in drill-override.conf:
> drill.exec: {
> sys.store.provider.local.path="/home/user/somedatafolder",
> security.user.auth: {
> enabled: true,
> packages += "com.app",
> impl: "myCustomAuthenticatorType"
> }
> },
> // below properties are used by the custom authenticator to fetch the
> password from external service
> drill.username: "username",
> drill.password.key: "passkey",
> drill.password.sentry.url: "https://somehost/api;,
> drill.exec.options: {
> security.admin.users: "username"
> }
> Error :
> Error: Failure in connecting to Drill:
> org.apache.drill.exec.rpc.NonTransientRpcException:
> javax.security.sasl.SaslException: Server requires authentication using
> [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled ,
> MaxWrappedSize: 65536 , WrapSizeLimit: 0]. (state=,code=0)
> java.sql.SQLNonTransientConnectionException: Failure in connecting to Drill:
> org.apache.drill.exec.rpc.NonTransientRpcException:
> javax.security.sasl.SaslException: Server requires authentication using
> [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled ,
> MaxWrappedSize: 65536 , WrapSizeLimit: 0].
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:178)
> at
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
> at
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
> at
> org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
> at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
> at
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
> at sqlline.Commands.connect(Commands.java:1364)
> at sqlline.Commands.connect(Commands.java:1244)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
> at sqlline.SqlLine.dispatch(SqlLine.java:730)
> at sqlline.SqlLine.initArgs(SqlLine.java:410)
> at sqlline.SqlLine.begin(SqlLine.java:515)
> at sqlline.SqlLine.start(SqlLine.java:267)
> at sqlline.SqlLine.main(SqlLine.java:206)
> Caused by: org.apache.drill.exec.rpc.NonTransientRpcException:
> javax.security.sasl.SaslException: Server requires authentication using
> [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled ,
> MaxWrappedSize: 65536 , WrapSizeLimit: 0].
> at
> org.apache.drill.exec.rpc.user.UserClient.connect(UserClient.java:207)
> at
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:458)
> at
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:402)
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:169)
> ... 18 more
> Caused by: javax.security.sasl.SaslException: Server requires authentication
> using [PLAIN]. Insufficient credentials?. [Details: Encryption: disabled ,
> MaxWrappedSize: 65536 , WrapSizeLimit: 0].
> at
> org.apache.drill.exec.rpc.user.UserClient.getAuthenticatorFactory(UserClient.java:366)
> at
> org.apache.drill.exec.rpc.user.UserClient.prepareSaslHandshake(UserClient.java:442)
> at
> org.apache.drill.exec.rpc.ConnectionMultiListener$HandshakeSendHandler.success(ConnectionMultiListener.java:160)
> at
> org.apache.drill.exec.rpc.ConnectionMultiListener$HandshakeSendHandler.success(ConnectionMultiListener.java:143)
> at
> org.apache.drill.exec.rpc.RequestIdMap$RpcListener.set(RequestIdMap.java:134)
> at
> org.apache.drill.exec.rpc.BasicClient$ClientHandshakeHandler.consumeHandshake(BasicClient.java:318)
> at
> org.apache.drill.exec.rpc.AbstractHandshakeHandler.decode(AbstractHandshakeHandler.java:57)
> at
>
[jira] [Updated] (DRILL-8361) Add opt-in support for nulling invalid data
[ https://issues.apache.org/jira/browse/DRILL-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8361: Fix Version/s: (was: 1.21.0) > Add opt-in support for nulling invalid data > --- > > Key: DRILL-8361 > URL: https://issues.apache.org/jira/browse/DRILL-8361 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > > Currently Drill fails a query if it encounters invalid data. This Jira > proposes opt-in support for nulling invalid data without failing the query. > The number of invalid data that were nulled should be logged at the warning > level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8393:
Fix Version/s: (was: 1.21.0)
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Priority: Major
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8383) Convert Image format plugin to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8383: Fix Version/s: (was: 1.21.0) > Convert Image format plugin to EVF2 > --- > > Key: DRILL-8383 > URL: https://issues.apache.org/jira/browse/DRILL-8383 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Other >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8401) Skip nested MAP column without children when creating parquet tables
[ https://issues.apache.org/jira/browse/DRILL-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8401. --- Resolution: Fixed > Skip nested MAP column without children when creating parquet tables > > > Key: DRILL-8401 > URL: https://issues.apache.org/jira/browse/DRILL-8401 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Parquet >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > This extends the work of DRILL-8272 in order to handle nested empty MAPs > which currently also break the Parquet writer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688812#comment-17688812 ] ASF GitHub Bot commented on DRILL-8117: --- cgivre merged PR #2756: URL: https://github.com/apache/drill/pull/2756 > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688803#comment-17688803 ] ASF GitHub Bot commented on DRILL-8117: --- jnturton commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430707391 Message to whoever squashes and merges here, in case it's not me: when cleaning up the squashed commit detail message please retain the co-author footer so that the repo will reflect @kingswanwho's contribution. ``` - Co-authored-by: kingswanwho ``` > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688802#comment-17688802 ] ASF GitHub Bot commented on DRILL-8117: --- jnturton commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430704625 Okay, I think the penny's finally dropped. I was also thinking about the markdown in /docs but couldn't fathom what we'd add. But the new UserExceptionMatcher usage can be described and also the try-with-resources style of creating single-use client fixtures. > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688779#comment-17688779 ] ASF GitHub Bot commented on DRILL-8117: --- kingswanwho commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430637603 > > > One other question. Should we document this in the developer documentation? > > > > > > I think we do have developer documentation describing cluster fixture tests, or do you mean something else? > > I was referring to the markdown files in the `/docs` folder. With this PR do those need to be updated? (It doesn't have to be a part of this PR.) Hi Charles, I have checked /docs developer information, this PR transfer test framework from BaseTestQuery to ClusterTest, and doesn't change the test logic of ClusterTest. James helps to find a clean way to create new ClientFixture, and handle UserException. I can help to update those information in /docs in a new PR. > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8401) Skip nested MAP column without children when creating parquet tables
[ https://issues.apache.org/jira/browse/DRILL-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688685#comment-17688685 ] ASF GitHub Bot commented on DRILL-8401: --- cgivre merged PR #2757: URL: https://github.com/apache/drill/pull/2757 > Skip nested MAP column without children when creating parquet tables > > > Key: DRILL-8401 > URL: https://issues.apache.org/jira/browse/DRILL-8401 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Parquet >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > This extends the work of DRILL-8272 in order to handle nested empty MAPs > which currently also break the Parquet writer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688630#comment-17688630 ] ASF GitHub Bot commented on DRILL-8117: --- cgivre commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430116439 > > One other question. Should we document this in the developer documentation? > > I think we do have developer documentation describing cluster fixture tests, or do you mean something else? I was referring to the markdown files in the `/docs` folder. With this PR do those need to be updated? (It doesn't have to be a part of this PR.) > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688628#comment-17688628 ] ASF GitHub Bot commented on DRILL-8117: --- jnturton commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430114758 > One other question. Should we document this in the developer documentation? I think we do have developer documentation describing cluster tests, or do you mean something else? > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688612#comment-17688612 ] ASF GitHub Bot commented on DRILL-8117: --- kingswanwho commented on PR #2756: URL: https://github.com/apache/drill/pull/2756#issuecomment-1430077393 > @jnturton @kingswanwho Should we close the other PR? Yes, I closed another PR > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Upgrade unit tests to the cluster fixture framework
[ https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688611#comment-17688611 ] ASF GitHub Bot commented on DRILL-8117: --- kingswanwho closed pull request #2499: DRILL-8117: Upgrade unit tests to the cluster fixture framework URL: https://github.com/apache/drill/pull/2499 > Upgrade unit tests to the cluster fixture framework > --- > > Key: DRILL-8117 > URL: https://issues.apache.org/jira/browse/DRILL-8117 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > Upgrade various unit tests to the cluster fixture framework and replace other > instances of deprecated code usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8395) Add Support for INSERT and Drop Table to GoogleSheets Plugin
[ https://issues.apache.org/jira/browse/DRILL-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8395. --- Resolution: Fixed > Add Support for INSERT and Drop Table to GoogleSheets Plugin > > > Key: DRILL-8395 > URL: https://issues.apache.org/jira/browse/DRILL-8395 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - GoogleSheets >Affects Versions: 1.20.3 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.21.0 > > > This PR adds support for INSERT queries which allow a user to append data to > an existing GoogleSheets tab. It also: > * Adds support for DROP TABLE queries which were not implemented > * Modifies CTAS queries so that if a user executes a CTAS query with a file > token, Drill will add a new tab to an existing document, but if the user > executes a CTAS with a file name, it will create an entirely new document. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8396) Update checkstyle version
[ https://issues.apache.org/jira/browse/DRILL-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8396. --- > Update checkstyle version > - > > Key: DRILL-8396 > URL: https://issues.apache.org/jira/browse/DRILL-8396 > Project: Apache Drill > Issue Type: Task >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Minor > > Update com.puppycrawl.tools:checkstyle version to the latest one. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8397) Drill prints warnings to console when starting it
[
https://issues.apache.org/jira/browse/DRILL-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8397.
---
Resolution: Fixed
> Drill prints warnings to console when starting it
> -
>
> Key: DRILL-8397
> URL: https://issues.apache.org/jira/browse/DRILL-8397
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Minor
>
> When starting the drill in embedded mode, it prints the following warnings:
> {noformat}
> 11:19:55,482 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This
> is logback-classic version 1.4.5
> 11:19:55,499 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could
> NOT find resource [logback-test.xml]
> 11:19:55,503 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found
> resource [logback.xml] at
> [file:/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/conf/logback.xml]
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 73
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 78
> 11:19:55,607 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> element is deprecated. Near [level] on line 91
> 11:19:55,608 |-WARN in ch.qos.logback.classic.joran.action.LevelAction -
> Please use "level" attribute within or elements instead.
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [STDOUT]
> 11:19:55,652 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
> 11:19:55,658 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [QUERY]
> 11:19:55,689 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,697 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@66ac5762 - No
> compression will be used
> 11:19:55,699 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json
> 11:19:55,699 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline_queries.json]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - Processing
> appender named [FILE]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.model.processor.AppenderModelHandler - About to
> instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
> 11:19:55,700 |-INFO in
> ch.qos.logback.core.rolling.FixedWindowRollingPolicy@797cf65c - No
> compression will be used
> 11:19:55,701 |-INFO in
> ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default
> type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
> property
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - Active log file name:
> /tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log
> 11:19:55,701 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE]
> - File property is set to
> [/tmp/drill/distribution/target/apache-drill-1.21.0-SNAPSHOT/apache-drill-1.21.0-SNAPSHOT/log/sqlline.log]
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting
> additivity of logger [org.apache.drill] to false
> 11:19:55,702 |-INFO in
> ch.qos.logback.classic.model.processor.LevelModelHandler - org.apache.drill
> level set to INFO
> 11:19:55,702 |-INFO in
> ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching
> appender named [FILE] to Logger[org.apache.drill]
> 11:19:55,703 |-INFO in
>
[jira] [Closed] (DRILL-8398) Fix GitHub Actions to use proper JDK version
[ https://issues.apache.org/jira/browse/DRILL-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8398. --- Resolution: Fixed > Fix GitHub Actions to use proper JDK version > > > Key: DRILL-8398 > URL: https://issues.apache.org/jira/browse/DRILL-8398 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.21.0 >Reporter: Vova Vysotskyi >Assignee: Vova Vysotskyi >Priority: Blocker > Fix For: 1.21.0 > > > In one of the changes GitHub Actions `actions/setup-java@v2` action was > removed, and after that provided java version (11) was used instead of the > specified one in the > matrix.java. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8399) MS Access Reader Misinterprets Data Types
[ https://issues.apache.org/jira/browse/DRILL-8399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8399. --- Resolution: Fixed > MS Access Reader Misinterprets Data Types > - > > Key: DRILL-8399 > URL: https://issues.apache.org/jira/browse/DRILL-8399 > Project: Apache Drill > Issue Type: Bug > Components: Format - MS Access >Affects Versions: 1.21.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 1.21.0 > > > The MS Access reader was assigning certain data types incorrectly, resulting > in various errors. This minor PR fixes that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8400) Fix pruning partitions with pushed transitive predicates
[
https://issues.apache.org/jira/browse/DRILL-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8400.
---
Resolution: Fixed
> Fix pruning partitions with pushed transitive predicates
>
>
> Key: DRILL-8400
> URL: https://issues.apache.org/jira/browse/DRILL-8400
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.3
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Major
> Fix For: 1.21.0
>
>
> See {{TestHivePartitionPruning.prunePartitionsBasedOnTransitivePredicates()}}
> test for details.
> The issue occurs for queries like these:
> {code:sql}
> SELECT * FROM hive.partition_pruning_test t1
> JOIN hive.partition_with_few_schemas t2 ON t1.`d` = t2.`d` AND t1.`e` =
> t2.`e`
> WHERE t2.`e` IS NOT NULL AND t1.`d` = 1
> {code}
> The expected behavior is to create additional filters based on the existing
> filters and join conditions. We have a {{TRANSITIVE_CLOSURE}} planning phase,
> which is responsible for such query transformations, but Drill pushes down
> filters from the WHERE condition before that phase, so the optimization is
> not performed.
> Ideally, we should move rules from the {{TRANSITIVE_CLOSURE}} phase to the
> {{LOGICAL}} phase so that the planner will choose the most optimal plan, but
> it wouldn't help until CALCITE-1048 is fixed (it is required to pull
> predicates when three has {{RelSubset}} nodes).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8400) Fix pruning partitions with pushed transitive predicates
[
https://issues.apache.org/jira/browse/DRILL-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8400:
Fix Version/s: 1.21.0
> Fix pruning partitions with pushed transitive predicates
>
>
> Key: DRILL-8400
> URL: https://issues.apache.org/jira/browse/DRILL-8400
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.3
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Major
> Fix For: 1.21.0
>
>
> See {{TestHivePartitionPruning.prunePartitionsBasedOnTransitivePredicates()}}
> test for details.
> The issue occurs for queries like these:
> {code:sql}
> SELECT * FROM hive.partition_pruning_test t1
> JOIN hive.partition_with_few_schemas t2 ON t1.`d` = t2.`d` AND t1.`e` =
> t2.`e`
> WHERE t2.`e` IS NOT NULL AND t1.`d` = 1
> {code}
> The expected behavior is to create additional filters based on the existing
> filters and join conditions. We have a {{TRANSITIVE_CLOSURE}} planning phase,
> which is responsible for such query transformations, but Drill pushes down
> filters from the WHERE condition before that phase, so the optimization is
> not performed.
> Ideally, we should move rules from the {{TRANSITIVE_CLOSURE}} phase to the
> {{LOGICAL}} phase so that the planner will choose the most optimal plan, but
> it wouldn't help until CALCITE-1048 is fixed (it is required to pull
> predicates when three has {{RelSubset}} nodes).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8400) Fix pruning partitions with pushed transitive predicates
[
https://issues.apache.org/jira/browse/DRILL-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8400:
Affects Version/s: 1.20.3
> Fix pruning partitions with pushed transitive predicates
>
>
> Key: DRILL-8400
> URL: https://issues.apache.org/jira/browse/DRILL-8400
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.3
>Reporter: Vova Vysotskyi
>Assignee: Vova Vysotskyi
>Priority: Major
>
> See {{TestHivePartitionPruning.prunePartitionsBasedOnTransitivePredicates()}}
> test for details.
> The issue occurs for queries like these:
> {code:sql}
> SELECT * FROM hive.partition_pruning_test t1
> JOIN hive.partition_with_few_schemas t2 ON t1.`d` = t2.`d` AND t1.`e` =
> t2.`e`
> WHERE t2.`e` IS NOT NULL AND t1.`d` = 1
> {code}
> The expected behavior is to create additional filters based on the existing
> filters and join conditions. We have a {{TRANSITIVE_CLOSURE}} planning phase,
> which is responsible for such query transformations, but Drill pushes down
> filters from the WHERE condition before that phase, so the optimization is
> not performed.
> Ideally, we should move rules from the {{TRANSITIVE_CLOSURE}} phase to the
> {{LOGICAL}} phase so that the planner will choose the most optimal plan, but
> it wouldn't help until CALCITE-1048 is fixed (it is required to pull
> predicates when three has {{RelSubset}} nodes).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8401) Skip nested MAP column without children when creating parquet tables
[ https://issues.apache.org/jira/browse/DRILL-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688526#comment-17688526 ] ASF GitHub Bot commented on DRILL-8401: --- jnturton opened a new pull request, #2757: URL: https://github.com/apache/drill/pull/2757 # [DRILL-8401](https://issues.apache.org/jira/browse/DRILL-8401): Skip nested MAP column without children when creating parquet tables ## Description This extends the work of [DRILL-8272](https://issues.apache.org/jira/browse/DRILL-8272) in order to handle nested empty MAPs which currently also break the Parquet writer. ## Documentation N/A ## Testing Extension of TestParquetWriter#testResultWithEmptyMap > Skip nested MAP column without children when creating parquet tables > > > Key: DRILL-8401 > URL: https://issues.apache.org/jira/browse/DRILL-8401 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Parquet >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.21.0 > > > This extends the work of DRILL-8272 in order to handle nested empty MAPs > which currently also break the Parquet writer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8401) Skip nested MAP column without children when creating parquet tables
James Turton created DRILL-8401: --- Summary: Skip nested MAP column without children when creating parquet tables Key: DRILL-8401 URL: https://issues.apache.org/jira/browse/DRILL-8401 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Affects Versions: 1.20.3 Reporter: James Turton Assignee: James Turton Fix For: 1.21.0 This extends the work of DRILL-8272 in order to handle nested empty MAPs which currently also break the Parquet writer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (DRILL-7980) Pushdowns subquery results as parameters
[
https://issues.apache.org/jira/browse/DRILL-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394315#comment-17394315
]
Vitalii Diravka edited comment on DRILL-7980 at 2/14/23 1:02 PM:
-
Looks like the query works fine (thanks [~volodymyr] for pointing this):
{code:java}
apache drill> SELECT first_name FROM cp.`employee.json` where first_name in
(SELECT first_name FROM cp.`employee.json` where first_name in ('Sheri',
'Derrick'));
++
| first_name |
++
| Sheri |
| Derrick|
++{code}
the query with star can't be submitted, because the number of columns are
unknown:
{code:java}
apache drill> SELECT * FROM cp.`employee.json` where first_name in (SELECT *
FROM cp.`employee.json` where first_name in ('Sheri', 'Derrick'));
Error: SYSTEM ERROR: NullPointerException: Cannot invoke
"org.apache.drill.exec.record.RecordBatchSizer$ColumnSize.getStdNetOrNetSizePerEntry()"
because "columnSize" is null{code}
For HTTP plugin the planner works in the same manner.
Mark the ticket as invalid.
was (Author: vitalii):
Looks like the query works fine (thanks [~volodymyr] for pointing this):
{code:java}
apache drill> SELECT first_name FROM cp.`employee.json` where first_name in
(SELECT first_name FROM cp.`employee.json` where first_name in ('Sheri',
'Derrick'));
++
| first_name |
++
| Sheri |
| Derrick|
++{code}
the query with start can't be submitted, because the number of columns are
unknown:
{code:java}
apache drill> SELECT * FROM cp.`employee.json` where first_name in (SELECT *
FROM cp.`employee.json` where first_name in ('Sheri', 'Derrick'));
Error: SYSTEM ERROR: NullPointerException: Cannot invoke
"org.apache.drill.exec.record.RecordBatchSizer$ColumnSize.getStdNetOrNetSizePerEntry()"
because "columnSize" is null{code}
For HTTP plugin the planner works in the same manner.
Mark the ticket as invalid.
> Pushdowns subquery results as parameters
>
>
> Key: DRILL-7980
> URL: https://issues.apache.org/jira/browse/DRILL-7980
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning Optimization
>Affects Versions: 1.19.0
>Reporter: Vitalii Diravka
>Assignee: Vitalii Diravka
>Priority: Major
>
> If a sub query is used to generate parameters for an API call you get 400
> errors and the paramters don't show up on URL in the error.
>
> For Example:
> If you have an API config that requires a "q=" paramter and want to populate
> that with a sub query like this...
>
> {code:java}
> SELECT *
> FROM api.example
> WHERE q IN (SELECT col FROM source){code}
> It doesn't work. The expected behavior would be multiple API calls (one for
> each value in "col") with the results aggregated back together for the
> projection.
> Query example:
> {code:java}
> vitalii@vitalii-UX331UN:~/IdeaProjects/drill$
> distribution/target/apache-drill-1.20.0-SNAPSHOT/apache-drill-1.20.0-SNAPSHOT/bin/drill-embedded
> Apache Drill 1.20.0-SNAPSHOT
> "A Drill is a terrible thing to waste."
> apache drill> SELECT * FROM cp.`employee.json` where first_name in ('Sheri',
> 'Derrick');
> +-+-++---+-++--+---++---+-+---+-+++---+
> | employee_id | full_name | first_name | last_name | position_id |
> position_title | store_id | department_id | birth_date | hire_date | salary |
> supervisor_id | education_level | marital_status | gender | management_role |
> +-+-++---+-++--+---++---+-+---+-+++---+
> | 1 | Sheri Nowmer | Sheri | Nowmer | 1 | President | 0 | 1 | 1961-08-26 |
> 1994-12-01 00:00:00.0 | 8.0 | 0 | Graduate Degree | S | F | Senior
> Management |
> | 2 | Derrick Whelply | Derrick | Whelply | 2 | VP Country Manager | 0 | 1 |
> 1915-07-03 | 1994-12-01 00:00:00.0 | 4.0 | 1 | Graduate Degree | M | M |
> Senior Management |
> +-+-++---+-++--+---++---+-+---+-+++---+
> 2 rows selected (0.156 seconds)
> apache drill> select * from (VALUES('Sheri', 'Derrick'));
> ++-+
> | EXPR$0 | EXPR$1 |
> ++-+
> | Sheri | Derrick |
> ++-+
> 1 row selected (0.143 seconds)
> apache drill> SELECT * FROM cp.`employee.json` where first_name in
