[jira] [Reopened] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread PJ Fanning (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

PJ Fanning reopened DRILL-8460:
---
  Assignee: (was: PJ Fanning)

This is not fixed. The CI build had some test failures that indicate that we 
may nor be able to upgrade.

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8457) Allow configuring csv parser in http storage plugin configuration

2023-10-31 Thread Zbigniew Tomanek (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zbigniew Tomanek closed DRILL-8457.
---
Fix Version/s: 1.22.0
   (was: Future)
 Reviewer: Charles Givre
   Resolution: Implemented

> Allow configuring csv parser in http storage plugin configuration
> -
>
> Key: DRILL-8457
> URL: https://issues.apache.org/jira/browse/DRILL-8457
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - HTTP
>Affects Versions: Future
>Reporter: Zbigniew Tomanek
>Priority: Minor
> Fix For: 1.22.0
>
>
> Currently there is no way to configure csv parser when http plugin is used. 
> Because of that some kind of files cannot be parsed (e.g. when any column has 
> more than 4096 chars or file has a delimiter different from `,`).
> Since in DataWalk we utilize http plugin quite often we've changed our 
> internal fork of Drill so following parser/format properties can be 
> configured using additional `csvOptions` field:
>  
> {code:json}
> {
>   "csvOptions": {
>     "delimiter": "\t",
>     "quote": "\"",
>     "quote_escape": "\"",
>     "line_separator": "\n",
>     "header_extraction_enabled": null,
>     "number_of_rows_to_skip": 0,
>     "number_of_records_to_read": -1,
>     "line_separator_detection_enabled": true,
>     "max_columns": 512,
>     "max_chars_per_column": 4096,
>     "skip_empty_lines": true,
>     "ignore_leading_whitespaces": true,
>     "ignore_trailing_whitespaces": true,
>     "null_value": null
>   }
> }{code}
> I'd be glad to get feedback whether creating PR with these changes would 
> bring any value to the Drill



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (DRILL-8457) Allow configuring csv parser in http storage plugin configuration

2023-10-31 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/DRILL-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781429#comment-17781429
 ] 

ASF GitHub Bot commented on DRILL-8457:
---

cgivre merged PR #2840:
URL: https://github.com/apache/drill/pull/2840




> Allow configuring csv parser in http storage plugin configuration
> -
>
> Key: DRILL-8457
> URL: https://issues.apache.org/jira/browse/DRILL-8457
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - HTTP
>Affects Versions: Future
>Reporter: Zbigniew Tomanek
>Priority: Minor
> Fix For: Future
>
>
> Currently there is no way to configure csv parser when http plugin is used. 
> Because of that some kind of files cannot be parsed (e.g. when any column has 
> more than 4096 chars or file has a delimiter different from `,`).
> Since in DataWalk we utilize http plugin quite often we've changed our 
> internal fork of Drill so following parser/format properties can be 
> configured using additional `csvOptions` field:
>  
> {code:json}
> {
>   "csvOptions": {
>     "delimiter": "\t",
>     "quote": "\"",
>     "quote_escape": "\"",
>     "line_separator": "\n",
>     "header_extraction_enabled": null,
>     "number_of_rows_to_skip": 0,
>     "number_of_records_to_read": -1,
>     "line_separator_detection_enabled": true,
>     "max_columns": 512,
>     "max_chars_per_column": 4096,
>     "skip_empty_lines": true,
>     "ignore_leading_whitespaces": true,
>     "ignore_trailing_whitespaces": true,
>     "null_value": null
>   }
> }{code}
> I'd be glad to get feedback whether creating PR with these changes would 
> bring any value to the Drill



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (DRILL-8446) Incorrect use of OperatingSystemMXBean

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reassigned DRILL-8446:
---

Assignee: James Turton

> Incorrect use of OperatingSystemMXBean
> --
>
> Key: DRILL-8446
> URL: https://issues.apache.org/jira/browse/DRILL-8446
> Project: Apache Drill
>  Issue Type: Bug
>Affects Versions: 1.21.1
>Reporter: Mahmoud Ouali Alami
>Assignee: James Turton
>Priority: Major
> Attachments: image-2023-07-04-15-36-42-905.png, 
> image-2023-07-04-16-24-59-662.png
>
>
> *Context :* 
> In Drill "CpuGaugeSet" class, we use an internal class instead of a public 
> class : com.sun.management.OperatingSystemMXBean;
> !image-2023-07-04-15-36-42-905.png|width=387,height=257!
> This can result to a NoClassDefFoundError:
> !image-2023-07-04-16-24-59-662.png|width=845,height=108!  
> *To do :* 
> Replace the private class "com.sun.managemenet.OperatingSystemMXBean" with 
> "java.lang.management.OperatingSystemMXBean",
>  
> Kind regards,
> Mahmoud
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (DRILL-8447) Jetty TLS support broken for server certificate with multiple SANs

2023-10-31 Thread James Turton (Jira)


[ 
https://issues.apache.org/jira/browse/DRILL-8447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781421#comment-17781421
 ] 

James Turton commented on DRILL-8447:
-

Hi, in DRILL-8452 we did a lot of library uprgades including, by chance, the 
changes you mention here. Can you run a test using a snapshot of the Drill 
master branch? I can also upload a snapshot build for you if that would be 
helpful.

> Jetty TLS support broken for server certificate with multiple SANs
> --
>
> Key: DRILL-8447
> URL: https://issues.apache.org/jira/browse/DRILL-8447
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Web Server
>Affects Versions: 1.21.1
>Reporter: Lev C
>Assignee: James Turton
>Priority: Minor
>
> The update of Jetty dependency to version 9.4 (DRILL-7135) has led to Drill 
> no longer supporting a TLS server certificate with multiple Subject Alternate 
> Name (SAN) values.
> If you try to use a keystore containing a single cert with multiple SANs, you 
> get the following error on startup of the Drillbit:
>  
> {code:java}
> [main] ERROR o.apache.drill.exec.server.Drillbit - Failure during initial 
> startup of Drillbit.
> java.lang.illegalStateException: KeyStores with multiple certificates are not 
> supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. 
> (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or 
> org.eclipse.jetty.util.SslContextFactory$Client instead){code}
> It appears that Jetty version 9.4 has deprecated the 
> org.eclipse.jetty.util.ssl.SslContextFactory class, and Drill should now use 
> the org.eclipse.jetty.util.ssl.SslContextFactory.Server class instead.
>  
> I was unable to find a Drill configuration file to change which class is used 
> (eg an instance of jetty-ssl.xml), and it looks like the specific 
> SslContextFactory class is hardcoded in 
> org.apache.drill.exec.server.rest.ssl.SslContextFactoryConfigurator:
> {code:java}
> public SslContextFactory configureNewSslContextFactory() throws Exception {   
>  
>   SSLConfig sslConf = new SSLConfigBuilder()
> .config(config)
> .mode(SSLConfig.Mode.SERVER)
> .initializeSSLContext(false)
> .validateKeyStore(true)
> .build();
>   final SslContextFactory sslContextFactory = new SslContextFactory();
>   if (sslConf.isSslValid()) {
> useOptionsConfiguredByUser(sslContextFactory, sslConf);
>   } else {
> useAutoGeneratedSelfSignedCertificate(sslContextFactory);
>   }
>   return sslContextFactory;  
> } {code}
> Relevant links for other products affected by the same issue:
> https://issues.apache.org/jira/browse/NIFI-7730 
> [https://opennms.discourse.group/t/jetty-with-ssl-throws-error-with-keystores-with-multiple-certificates-are-not-supported/1489]
> [https://community.microfocus.com/adtd/sws-qc/f/itrc-895/513861/java-lang-illegalstateexception-keystores-with-multiple-certificates-are-not-supported-on-the-base-class-org-eclipse-jetty-util-ssl-sslcontextfactory]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (DRILL-8447) Jetty TLS support broken for server certificate with multiple SANs

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reassigned DRILL-8447:
---

Assignee: James Turton

> Jetty TLS support broken for server certificate with multiple SANs
> --
>
> Key: DRILL-8447
> URL: https://issues.apache.org/jira/browse/DRILL-8447
> Project: Apache Drill
>  Issue Type: Bug
>  Components:  Server, Web Server
>Affects Versions: 1.21.1
>Reporter: Lev C
>Assignee: James Turton
>Priority: Minor
>
> The update of Jetty dependency to version 9.4 (DRILL-7135) has led to Drill 
> no longer supporting a TLS server certificate with multiple Subject Alternate 
> Name (SAN) values.
> If you try to use a keystore containing a single cert with multiple SANs, you 
> get the following error on startup of the Drillbit:
>  
> {code:java}
> [main] ERROR o.apache.drill.exec.server.Drillbit - Failure during initial 
> startup of Drillbit.
> java.lang.illegalStateException: KeyStores with multiple certificates are not 
> supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. 
> (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or 
> org.eclipse.jetty.util.SslContextFactory$Client instead){code}
> It appears that Jetty version 9.4 has deprecated the 
> org.eclipse.jetty.util.ssl.SslContextFactory class, and Drill should now use 
> the org.eclipse.jetty.util.ssl.SslContextFactory.Server class instead.
>  
> I was unable to find a Drill configuration file to change which class is used 
> (eg an instance of jetty-ssl.xml), and it looks like the specific 
> SslContextFactory class is hardcoded in 
> org.apache.drill.exec.server.rest.ssl.SslContextFactoryConfigurator:
> {code:java}
> public SslContextFactory configureNewSslContextFactory() throws Exception {   
>  
>   SSLConfig sslConf = new SSLConfigBuilder()
> .config(config)
> .mode(SSLConfig.Mode.SERVER)
> .initializeSSLContext(false)
> .validateKeyStore(true)
> .build();
>   final SslContextFactory sslContextFactory = new SslContextFactory();
>   if (sslConf.isSslValid()) {
> useOptionsConfiguredByUser(sslContextFactory, sslConf);
>   } else {
> useAutoGeneratedSelfSignedCertificate(sslContextFactory);
>   }
>   return sslContextFactory;  
> } {code}
> Relevant links for other products affected by the same issue:
> https://issues.apache.org/jira/browse/NIFI-7730 
> [https://opennms.discourse.group/t/jetty-with-ssl-throws-error-with-keystores-with-multiple-certificates-are-not-supported/1489]
> [https://community.microfocus.com/adtd/sws-qc/f/itrc-895/513861/java-lang-illegalstateexception-keystores-with-multiple-certificates-are-not-supported-on-the-base-class-org-eclipse-jetty-util-ssl-sslcontextfactory]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8450) Add Data Type Inference to XML Format Plugin

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8450.
---
Resolution: Fixed

> Add Data Type Inference to XML Format Plugin
> 
>
> Key: DRILL-8450
> URL: https://issues.apache.org/jira/browse/DRILL-8450
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Format - XML
>Affects Versions: 1.21.1
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 1.22.0
>
>
> This PR adds data type inference to the XML format plugin.  In similar 
> fashion to other plugins, it adds a new configuration parameter: allTextMode, 
> which when set to true, reads all data as strings.  The default is true.
> Note that the inference is limited to doubles, date, timestamps, boolean and 
> strings.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8452) Library upgrades

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8452:

Fix Version/s: 1.22.0
   (was: 1.21.2)

> Library upgrades
> 
>
> Key: DRILL-8452
> URL: https://issues.apache.org/jira/browse/DRILL-8452
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: library
>Affects Versions: 1.21.1
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.22.0
>
>
> * aircompressor.version -> 0.25
>  * antlr.version -> -4.13.0- 4.9.3
>  * asm.version -> 9.5
>  * avro.version -> 1.11.2
>  * commons.compress.version -> 1.23.0
>  * commons.validator.version -> 1.7
>  * hbase.version -> 2.5.5 (Hadoop 2 profile)
>  * hbase.version -> 2.5.5-hadoop3
>  * -hikari.version -> 5.0.1-
>  * httpclient.version -> 4.5.14
>  * httpdlog-parser.version -> 5.10.0
>  * jersey.version -> 2.40
>  * jetty -> 9.4.51.v20230217
>  * jna.version -> 5.13.0
>  * joda.version -> 2.12.5
>  * libthrift.version -> 0.18.1
>  * log4j.version -> 2.20.0
>  * -maven.version -> 3.9.4-
>  * metrics.version -> 4.2.19
>  * protostuff.version -> 1.8.0
>  * snakeyaml.version -> 2.1
>  * surefire.version -> 3.1.2
>  * testcontainers.version -> 1.18.3



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8453) Add XSD Support to XML Reader (Part 1)

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8453.
---
Resolution: Fixed

> Add XSD Support to XML Reader (Part 1)
> --
>
> Key: DRILL-8453
> URL: https://issues.apache.org/jira/browse/DRILL-8453
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Format - XML
>Affects Versions: 1.21.1
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 1.22.0
>
>
> This PR is a part of a series to add better support for reading XML data to 
> Drill.  One of the main challenges is that XML data does not have a way of 
> inferring data types, nor does it have a way of detecting arrays.  
> The only way to do this really well is to have a schema.  Some XML files link 
> a schema definition file to the data.  This PR adds the capability for Drill 
> to map XSD schema files into Drill schemas.  
> The current plan is as follows: Part 1 of this PR simply adds the reader but 
> adds no new user detectable functionality.  Part 2 will include the actual 
> integration with the XML reader.  Part 3 will include the ability to read 
> arrays.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8452) Library upgrades

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8452.
---
Resolution: Fixed

> Library upgrades
> 
>
> Key: DRILL-8452
> URL: https://issues.apache.org/jira/browse/DRILL-8452
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: library
>Affects Versions: 1.21.1
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.22.0
>
>
> * aircompressor.version -> 0.25
>  * antlr.version -> -4.13.0- 4.9.3
>  * asm.version -> 9.5
>  * avro.version -> 1.11.2
>  * commons.compress.version -> 1.23.0
>  * commons.validator.version -> 1.7
>  * hbase.version -> 2.5.5 (Hadoop 2 profile)
>  * hbase.version -> 2.5.5-hadoop3
>  * -hikari.version -> 5.0.1-
>  * httpclient.version -> 4.5.14
>  * httpdlog-parser.version -> 5.10.0
>  * jersey.version -> 2.40
>  * jetty -> 9.4.51.v20230217
>  * jna.version -> 5.13.0
>  * joda.version -> 2.12.5
>  * libthrift.version -> 0.18.1
>  * log4j.version -> 2.20.0
>  * -maven.version -> 3.9.4-
>  * metrics.version -> 4.2.19
>  * protostuff.version -> 1.8.0
>  * snakeyaml.version -> 2.1
>  * surefire.version -> 3.1.2
>  * testcontainers.version -> 1.18.3



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8453) Add XSD Support to XML Reader (Part 1)

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8453:

Fix Version/s: 1.22.0
   (was: 1.21.2)

> Add XSD Support to XML Reader (Part 1)
> --
>
> Key: DRILL-8453
> URL: https://issues.apache.org/jira/browse/DRILL-8453
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Format - XML
>Affects Versions: 1.21.1
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 1.22.0
>
>
> This PR is a part of a series to add better support for reading XML data to 
> Drill.  One of the main challenges is that XML data does not have a way of 
> inferring data types, nor does it have a way of detecting arrays.  
> The only way to do this really well is to have a schema.  Some XML files link 
> a schema definition file to the data.  This PR adds the capability for Drill 
> to map XSD schema files into Drill schemas.  
> The current plan is as follows: Part 1 of this PR simply adds the reader but 
> adds no new user detectable functionality.  Part 2 will include the actual 
> integration with the XML reader.  Part 3 will include the ability to read 
> arrays.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8456.
---
Resolution: Fixed

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reopened DRILL-8456:
-

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8454) Disable unsupported MapR profile and plugin

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8454.
---
Resolution: Fixed

> Disable unsupported MapR profile and plugin
> ---
>
> Key: DRILL-8454
> URL: https://issues.apache.org/jira/browse/DRILL-8454
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Tools, Build  Test
>Affects Versions: 1.21.1
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.22.0
>
>
> The MapR build profile and format plugin, which fell out of support in the 
> open source Drill codebase years ago, are disabled. Drill users needing 
> support for these components should contact HPE.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8456:

Affects Version/s: 1.21.1

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8456:

Fix Version/s: 1.22.0

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8459:

Affects Version/s: 1.21.1

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8460:

Affects Version/s: 1.21.1

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8459:

Fix Version/s: 1.22.0

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8460.
---
Resolution: Fixed

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reopened DRILL-8460:
-

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8460:

Fix Version/s: 1.22.0

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8459.
---
Resolution: Fixed

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Affects Versions: 1.21.1
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
> Fix For: 1.22.0
>
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reopened DRILL-8459:
-

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8459:

Summary: Bump avro to 1.11.3 due to CVE  (was: Bump avro to 1.11.3 due to 
cve)

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8459) Bump avro to 1.11.3 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8459.
---
  Assignee: PJ Fanning
Resolution: Fixed

> Bump avro to 1.11.3 due to CVE
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8460.
---
  Assignee: PJ Fanning
Resolution: Fixed

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8460) Bump zookeeper jar to 3.7.2 due to CVE

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8460:

Summary: Bump zookeeper jar to 3.7.2 due to CVE  (was: bump zookeeper jar 
to 3.7.2 due to cve)

> Bump zookeeper jar to 3.7.2 due to CVE
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8459) Bump avro to 1.11.3 due to cve

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8459:

Summary: Bump avro to 1.11.3 due to cve  (was: bump avro to 1.11.3 due to 
cve)

> Bump avro to 1.11.3 due to cve
> --
>
> Key: DRILL-8459
> URL: https://issues.apache.org/jira/browse/DRILL-8459
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/49



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton reassigned DRILL-8456:
---

Assignee: PJ Fanning

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton updated DRILL-8456:

Summary: Upgrade POI to 5.2.4  (was: uptake POI 5.2.4)

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8456) Upgrade POI to 5.2.4

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8456.
---
Resolution: Fixed

> Upgrade POI to 5.2.4
> 
>
> Key: DRILL-8456
> URL: https://issues.apache.org/jira/browse/DRILL-8456
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: PJ Fanning
>Assignee: PJ Fanning
>Priority: Major
>
> latest release with some transitive dependencies having security patches



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8436) Upgrade Hadoop 3.2.4 -> 3.3.6

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8436.
---

> Upgrade Hadoop 3.2.4 -> 3.3.6
> -
>
> Key: DRILL-8436
> URL: https://issues.apache.org/jira/browse/DRILL-8436
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: library
>Affects Versions: 1.21.1
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.22.0
>
>
> Hadoop is upgraded to 3.3.6. Jetty is upgraded to 9.4.51.v20230217.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (DRILL-8436) Upgrade Hadoop 3.2.4 -> 3.3.6

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton resolved DRILL-8436.
-
Resolution: Fixed

> Upgrade Hadoop 3.2.4 -> 3.3.6
> -
>
> Key: DRILL-8436
> URL: https://issues.apache.org/jira/browse/DRILL-8436
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: library
>Affects Versions: 1.21.1
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.22.0
>
>
> Hadoop is upgraded to 3.3.6. Jetty is upgraded to 9.4.51.v20230217.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Closed] (DRILL-8458) Reading Parquet v2 data page with repetition levels larger than column data throws IllegalArgumentException

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton closed DRILL-8458.
---

> Reading Parquet v2 data page with repetition levels larger than column data 
> throws IllegalArgumentException
> ---
>
> Key: DRILL-8458
> URL: https://issues.apache.org/jira/browse/DRILL-8458
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.21.1
>Reporter: Peter Franzen
>Assignee: James Turton
>Priority: Major
> Fix For: 1.22.0
>
>
> When the size of the repetition level bytes in a Parquet v2 data page is 
> larger than the size of the column data bytes, 
> {{org.apache.parquet.hadoop.ColumnChunkIncReadStore$ColumnChunkIncPageReader::readPage}}
>  throws an {{{}IllegalArgumentException{}}}. This is caused by trying to set 
> the limit of a ByteBuffer to a value large than its capacity.
>  
> The offending code is at line 226 in {{{}ColumnChunkIncReadStore.java{}}}:
>  
> {code:java}
> 217 int pageBufOffset = 0;
> 218 ByteBuffer bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 219 BytesInput repLevelBytes = BytesInput.from(
> 220   (ByteBuffer) bb.slice().limit(pageBufOffset + repLevelSize)
> 221 );
> 222 pageBufOffset += repLevelSize;
> 223
> 224 bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 225 final BytesInput defLevelBytes = BytesInput.from(
> 226   (ByteBuffer) bb.slice().limit(pageBufOffset + defLevelSize)
> 227 );
> 228 pageBufOffset += defLevelSize;  {code}
>  
> The buffer {{pageBuf}} contains the repetition level bytes followed by the 
> definition level bytes followed by the column data bytes.
>  
> The code at lines 217-221 reads the repetition level bytes, and then updates 
> the position of the {{pageBuf}} buffer to the start of the definition level 
> bytes (lines 222 and 224).
>  
> The code at lines 225-227 reads the definition level bytes, and when creating 
> a slice of the \{{pageBuf }}buffer containing the definition level bytes, the 
> slice's limit is set as if the position was at the beginning of the 
> repetition level bytes (line 226), i.e as if it not had been updated.
>  
> This means that if the capacity of the pageBuf buffer (which is the size of 
> the repetition level bytes + the size of the definition level bytes + the 
> size of the column data bytes) is less than (repLevelSize + repLevelSize + 
> defLevelSize), the call to limit() will throw.
>  
> The fix is to change line 226 to
> {code:java}
>   (ByteBuffer) bb.slice().limit(defLevelSize){code}
>  
> For symmetry, line 220 could also be changed to
> {code:java}
>   (ByteBuffer) bb.slice().limit(repLevelSize){code}
>  
> although {{pageBufOffset}} is always 0 there and will not cause the limit to 
> exceed the capacity.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (DRILL-8458) Reading Parquet v2 data page with repetition levels larger than column data throws IllegalArgumentException

2023-10-31 Thread James Turton (Jira)


 [ 
https://issues.apache.org/jira/browse/DRILL-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Turton resolved DRILL-8458.
-
Resolution: Fixed

> Reading Parquet v2 data page with repetition levels larger than column data 
> throws IllegalArgumentException
> ---
>
> Key: DRILL-8458
> URL: https://issues.apache.org/jira/browse/DRILL-8458
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.21.1
>Reporter: Peter Franzen
>Assignee: James Turton
>Priority: Major
> Fix For: 1.22.0
>
>
> When the size of the repetition level bytes in a Parquet v2 data page is 
> larger than the size of the column data bytes, 
> {{org.apache.parquet.hadoop.ColumnChunkIncReadStore$ColumnChunkIncPageReader::readPage}}
>  throws an {{{}IllegalArgumentException{}}}. This is caused by trying to set 
> the limit of a ByteBuffer to a value large than its capacity.
>  
> The offending code is at line 226 in {{{}ColumnChunkIncReadStore.java{}}}:
>  
> {code:java}
> 217 int pageBufOffset = 0;
> 218 ByteBuffer bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 219 BytesInput repLevelBytes = BytesInput.from(
> 220   (ByteBuffer) bb.slice().limit(pageBufOffset + repLevelSize)
> 221 );
> 222 pageBufOffset += repLevelSize;
> 223
> 224 bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 225 final BytesInput defLevelBytes = BytesInput.from(
> 226   (ByteBuffer) bb.slice().limit(pageBufOffset + defLevelSize)
> 227 );
> 228 pageBufOffset += defLevelSize;  {code}
>  
> The buffer {{pageBuf}} contains the repetition level bytes followed by the 
> definition level bytes followed by the column data bytes.
>  
> The code at lines 217-221 reads the repetition level bytes, and then updates 
> the position of the {{pageBuf}} buffer to the start of the definition level 
> bytes (lines 222 and 224).
>  
> The code at lines 225-227 reads the definition level bytes, and when creating 
> a slice of the \{{pageBuf }}buffer containing the definition level bytes, the 
> slice's limit is set as if the position was at the beginning of the 
> repetition level bytes (line 226), i.e as if it not had been updated.
>  
> This means that if the capacity of the pageBuf buffer (which is the size of 
> the repetition level bytes + the size of the definition level bytes + the 
> size of the column data bytes) is less than (repLevelSize + repLevelSize + 
> defLevelSize), the call to limit() will throw.
>  
> The fix is to change line 226 to
> {code:java}
>   (ByteBuffer) bb.slice().limit(defLevelSize){code}
>  
> For symmetry, line 220 could also be changed to
> {code:java}
>   (ByteBuffer) bb.slice().limit(repLevelSize){code}
>  
> although {{pageBufOffset}} is always 0 there and will not cause the limit to 
> exceed the capacity.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (DRILL-8458) Reading Parquet v2 data page with repetition levels larger than column data throws IllegalArgumentException

2023-10-31 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/DRILL-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781412#comment-17781412
 ] 

ASF GitHub Bot commented on DRILL-8458:
---

jnturton merged PR #2838:
URL: https://github.com/apache/drill/pull/2838




> Reading Parquet v2 data page with repetition levels larger than column data 
> throws IllegalArgumentException
> ---
>
> Key: DRILL-8458
> URL: https://issues.apache.org/jira/browse/DRILL-8458
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.21.1
>Reporter: Peter Franzen
>Assignee: James Turton
>Priority: Major
> Fix For: 1.22.0
>
>
> When the size of the repetition level bytes in a Parquet v2 data page is 
> larger than the size of the column data bytes, 
> {{org.apache.parquet.hadoop.ColumnChunkIncReadStore$ColumnChunkIncPageReader::readPage}}
>  throws an {{{}IllegalArgumentException{}}}. This is caused by trying to set 
> the limit of a ByteBuffer to a value large than its capacity.
>  
> The offending code is at line 226 in {{{}ColumnChunkIncReadStore.java{}}}:
>  
> {code:java}
> 217 int pageBufOffset = 0;
> 218 ByteBuffer bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 219 BytesInput repLevelBytes = BytesInput.from(
> 220   (ByteBuffer) bb.slice().limit(pageBufOffset + repLevelSize)
> 221 );
> 222 pageBufOffset += repLevelSize;
> 223
> 224 bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 225 final BytesInput defLevelBytes = BytesInput.from(
> 226   (ByteBuffer) bb.slice().limit(pageBufOffset + defLevelSize)
> 227 );
> 228 pageBufOffset += defLevelSize;  {code}
>  
> The buffer {{pageBuf}} contains the repetition level bytes followed by the 
> definition level bytes followed by the column data bytes.
>  
> The code at lines 217-221 reads the repetition level bytes, and then updates 
> the position of the {{pageBuf}} buffer to the start of the definition level 
> bytes (lines 222 and 224).
>  
> The code at lines 225-227 reads the definition level bytes, and when creating 
> a slice of the \{{pageBuf }}buffer containing the definition level bytes, the 
> slice's limit is set as if the position was at the beginning of the 
> repetition level bytes (line 226), i.e as if it not had been updated.
>  
> This means that if the capacity of the pageBuf buffer (which is the size of 
> the repetition level bytes + the size of the definition level bytes + the 
> size of the column data bytes) is less than (repLevelSize + repLevelSize + 
> defLevelSize), the call to limit() will throw.
>  
> The fix is to change line 226 to
> {code:java}
>   (ByteBuffer) bb.slice().limit(defLevelSize){code}
>  
> For symmetry, line 220 could also be changed to
> {code:java}
>   (ByteBuffer) bb.slice().limit(repLevelSize){code}
>  
> although {{pageBufOffset}} is always 0 there and will not cause the limit to 
> exceed the capacity.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (DRILL-8460) bump zookeeper jar to 3.7.2 due to cve

2023-10-31 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781264#comment-17781264
 ] 

ASF GitHub Bot commented on DRILL-8460:
---

pjfanning commented on PR #2842:
URL: https://github.com/apache/drill/pull/2842#issuecomment-1786721132

   looks like the zookeeper lib upgrade breaks tests in the Phoenix module when 
default Hadoop is used. Hadoop 2 tests pass.




> bump zookeeper jar to 3.7.2 due to cve
> --
>
> Key: DRILL-8460
> URL: https://issues.apache.org/jira/browse/DRILL-8460
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: PJ Fanning
>Priority: Major
>
> https://github.com/apache/drill/security/dependabot/51



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (DRILL-8458) Reading Parquet v2 data page with repetition levels larger than column data throws IllegalArgumentException

2023-10-31 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/DRILL-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781231#comment-17781231
 ] 

ASF GitHub Bot commented on DRILL-8458:
---

handmadecode commented on PR #2838:
URL: https://github.com/apache/drill/pull/2838#issuecomment-1786560246

   PR updated with refactored test code




> Reading Parquet v2 data page with repetition levels larger than column data 
> throws IllegalArgumentException
> ---
>
> Key: DRILL-8458
> URL: https://issues.apache.org/jira/browse/DRILL-8458
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Parquet
>Affects Versions: 1.21.1
>Reporter: Peter Franzen
>Assignee: James Turton
>Priority: Major
> Fix For: 1.22.0
>
>
> When the size of the repetition level bytes in a Parquet v2 data page is 
> larger than the size of the column data bytes, 
> {{org.apache.parquet.hadoop.ColumnChunkIncReadStore$ColumnChunkIncPageReader::readPage}}
>  throws an {{{}IllegalArgumentException{}}}. This is caused by trying to set 
> the limit of a ByteBuffer to a value large than its capacity.
>  
> The offending code is at line 226 in {{{}ColumnChunkIncReadStore.java{}}}:
>  
> {code:java}
> 217 int pageBufOffset = 0;
> 218 ByteBuffer bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 219 BytesInput repLevelBytes = BytesInput.from(
> 220   (ByteBuffer) bb.slice().limit(pageBufOffset + repLevelSize)
> 221 );
> 222 pageBufOffset += repLevelSize;
> 223
> 224 bb = (ByteBuffer) pageBuf.position(pageBufOffset);
> 225 final BytesInput defLevelBytes = BytesInput.from(
> 226   (ByteBuffer) bb.slice().limit(pageBufOffset + defLevelSize)
> 227 );
> 228 pageBufOffset += defLevelSize;  {code}
>  
> The buffer {{pageBuf}} contains the repetition level bytes followed by the 
> definition level bytes followed by the column data bytes.
>  
> The code at lines 217-221 reads the repetition level bytes, and then updates 
> the position of the {{pageBuf}} buffer to the start of the definition level 
> bytes (lines 222 and 224).
>  
> The code at lines 225-227 reads the definition level bytes, and when creating 
> a slice of the \{{pageBuf }}buffer containing the definition level bytes, the 
> slice's limit is set as if the position was at the beginning of the 
> repetition level bytes (line 226), i.e as if it not had been updated.
>  
> This means that if the capacity of the pageBuf buffer (which is the size of 
> the repetition level bytes + the size of the definition level bytes + the 
> size of the column data bytes) is less than (repLevelSize + repLevelSize + 
> defLevelSize), the call to limit() will throw.
>  
> The fix is to change line 226 to
> {code:java}
>   (ByteBuffer) bb.slice().limit(defLevelSize){code}
>  
> For symmetry, line 220 could also be changed to
> {code:java}
>   (ByteBuffer) bb.slice().limit(repLevelSize){code}
>  
> although {{pageBufOffset}} is always 0 there and will not cause the limit to 
> exceed the capacity.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)