[jira] [Created] (PARQUET-1503) Remove Ints Utility Class
BELUGA BEHR created PARQUET-1503:
Summary: Remove Ints Utility Class
Key: PARQUET-1503
URL: https://issues.apache.org/jira/browse/PARQUET-1503
Project: Parquet
Issue Type: Improvement
Affects Versions: 1.9.0
Reporter: BELUGA BEHR
{code:java}
public static int checkedCast(long value) {
int valueI = (int) value;
if (valueI != value) {
throw new IllegalArgumentException(String.format("Overflow casting %d to
an int", value));
}
return valueI;
}
{code}
[Link
Here|https://github.com/apache/parquet-mr/blob/dc61e510126aaa1a95a46fe39bf1529f394147e9/parquet-common/src/main/java/org/apache/parquet/Ints.java#L35]
Now that the Parquet-MR project uses Java 1.8, this method can now be replaced
with Java {{Math}} class and {{toIntExact}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PARQUET-1502) Convert FIXED_LEN_BYTE_ARRAY to arrow type in logicalTypeAnnotation if it is not null
Yongyan Wang created PARQUET-1502: - Summary: Convert FIXED_LEN_BYTE_ARRAY to arrow type in logicalTypeAnnotation if it is not null Key: PARQUET-1502 URL: https://issues.apache.org/jira/browse/PARQUET-1502 Project: Parquet Issue Type: Improvement Components: parquet-mr Affects Versions: 1.12.0 Reporter: Yongyan Wang Convert FIXED_LEN_BYTE_ARRAY to arrow type in logicalTypeAnnotation if it is not null in SchemaConverter -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PARQUET-1498) [Java] Add instructions to install thrift via homebrew
[
https://issues.apache.org/jira/browse/PARQUET-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748746#comment-16748746
]
ASF GitHub Bot commented on PARQUET-1498:
-
zivanfi commented on pull request #595: PARQUET-1498: Add instructions to
install thrift via homebrew
URL: https://github.com/apache/parquet-mr/pull/595
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [Java] Add instructions to install thrift via homebrew
> --
>
> Key: PARQUET-1498
> URL: https://issues.apache.org/jira/browse/PARQUET-1498
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
>Affects Versions: 1.10.0
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> Instead of manually building it, one can also install it via homebrew which
> much more comfortable. As this is not the latest thrift version, you need to
> explicitly include it in your {{PATH}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (PARQUET-1499) Add Java 11 build to the repository
[ https://issues.apache.org/jira/browse/PARQUET-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Szadovszky reassigned PARQUET-1499: - Assignee: Fokko Driesprong > Add Java 11 build to the repository > --- > > Key: PARQUET-1499 > URL: https://issues.apache.org/jira/browse/PARQUET-1499 > Project: Parquet > Issue Type: Improvement > Components: parquet-mr >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] Release Apache Parquet 1.11.0 RC3
Hello Gabor, you probably confused Wes and me ;) I'm going to give this a try on Linux and try to verify the release. The Java 9+ issues sadly makes it harder for me to verify it on my normal system. Uwe On Tue, Jan 22, 2019, at 9:44 AM, Gabor Szadovszky wrote: > Hi Wes, > > Thanks for checking the RC and voting. > I would like to highlight that the mentioned issues are also reproducible > on Linux with the parquet release 1.10.0 so these are not regressions. We > currently support java8 only so I cannot see any issue with the release. > I completely agree that these issues shall be fixed but I think we should > not stop the release because of them. > > Cheers, > Gabor > > On Mon, Jan 21, 2019 at 6:40 PM Uwe L. Korn wrote: > > > Hi, > > > > I'm sadly giving a +0 here. The signatures look good but I was unable to > > build with JDK 9/10/11 on OSX. > > https://issues.apache.org/jira/browse/PARQUET-1497 and > > https://issues.apache.org/jira/browse/PARQUET-1496 are the problems I'm > > running into. I've also opened a PR to document how to install thrift on > > OSX: https://github.com/apache/parquet-mr/pull/595 > > > > Cheers > > Uwe > > > > On Fri, Jan 18, 2019, at 1:55 PM, Anna Szonyi wrote: > > > Hi All, > > > > > > While not a binding one, it's a "+1 (non-binding), everything looks good" > > > from me! > > > > > > Best, > > > Anna > > > > > > On Thu, Jan 17, 2019 at 6:13 PM Zoltan Ivanfi > > > wrote: > > > > > > > Hi, > > > > > > > > Friendly reminder to please vote for the release. We need 2 more > > binding +1 > > > > votes. > > > > > > > > Thanks, > > > > > > > > Zoltan > > > > > > > > On Sat, Jan 12, 2019 at 3:07 AM 俊杰陈 wrote: > > > > > > > > > +1 (non-binding) > > > > > * contents looks good > > > > > * unit tests passed > > > > > > > > > > > > > > > Zoltan Ivanfi 于2019年1月11日周五 下午9:31写道： > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > * contents look good > > > > > > * unit tests pass > > > > > > * checksums match > > > > > > * signature matches > > > > > > > > > > > > Br, > > > > > > > > > > > > Zoltan > > > > > > > > > > > > On Thu, Jan 10, 2019 at 11:48 AM Gabor Szadovszky < > > ga...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > Checked tarball: checksum/signature are correct. Content is > > correct > > > > > based > > > > > > > on release tag. Unit tests pass. > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > Cheers, > > > > > > > Gabor > > > > > > > > > > > > > > On Wed, Jan 9, 2019 at 4:51 PM Zoltan Ivanfi > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Dear Parquet Users and Developers, > > > > > > > > > > > > > > > > I propose the following RC to be released as the official > > Apache > > > > > > > > Parquet 1.11.0 release: > > > > > > > > > > > > > > > > The commit id is 8be767d12cca295cf9858a521725fc440b0c6f93 > > > > > > > > * This corresponds to the tag: apache-parquet-1.11.0 > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/parquet-mr/tree/8be767d12cca295cf9858a521725fc440b0c6f93 > > > > > > > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.11.0-rc3/ > > > > > > > > > > > > > > > > You can find the KEYS file here: > > > > > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS > > > > > > > > > > > > > > > > Binary artifacts are staged in Nexus here: > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/1.11.0/ > > > > > > > > > > > > > > > > This release includes the following new features: > > > > > > > > > > > > > > > > - PARQUET-1201 - Column indexes > > > > > > > > - PARQUET-1253 - Support for new logical type representation > > > > > > > > - PARQUET-1381 - Add merge blocks command to parquet-tools > > > > > > > > - PARQUET-1388 - Nanosecond precision time and timestamp - > > > > parquet-mr > > > > > > > > > > > > > > > > The release also includes bug fixes, including: > > > > > > > > > > > > > > > > - PARQUET-1472: Dictionary filter fails on > > FIXED_LEN_BYTE_ARRAY. > > > > > > > > > > > > > > > > Please download, verify, and test. The vote will be open for at > > > > least > > > > > > 72 > > > > > > > > hours. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Zoltan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Thanks & Best Regards > > > > > > > > > > >
Re: [VOTE] Release Apache Parquet 1.11.0 RC3
Sorry, Uwe. I wanted to repy to you. :) On Tue, Jan 22, 2019 at 9:44 AM Gabor Szadovszky wrote: > Hi Wes, > > Thanks for checking the RC and voting. > I would like to highlight that the mentioned issues are also reproducible > on Linux with the parquet release 1.10.0 so these are not regressions. We > currently support java8 only so I cannot see any issue with the release. > I completely agree that these issues shall be fixed but I think we should > not stop the release because of them. > > Cheers, > Gabor > > On Mon, Jan 21, 2019 at 6:40 PM Uwe L. Korn wrote: > >> Hi, >> >> I'm sadly giving a +0 here. The signatures look good but I was unable to >> build with JDK 9/10/11 on OSX. >> https://issues.apache.org/jira/browse/PARQUET-1497 and >> https://issues.apache.org/jira/browse/PARQUET-1496 are the problems I'm >> running into. I've also opened a PR to document how to install thrift on >> OSX: https://github.com/apache/parquet-mr/pull/595 >> >> Cheers >> Uwe >> >> On Fri, Jan 18, 2019, at 1:55 PM, Anna Szonyi wrote: >> > Hi All, >> > >> > While not a binding one, it's a "+1 (non-binding), everything looks >> good" >> > from me! >> > >> > Best, >> > Anna >> > >> > On Thu, Jan 17, 2019 at 6:13 PM Zoltan Ivanfi >> > wrote: >> > >> > > Hi, >> > > >> > > Friendly reminder to please vote for the release. We need 2 more >> binding +1 >> > > votes. >> > > >> > > Thanks, >> > > >> > > Zoltan >> > > >> > > On Sat, Jan 12, 2019 at 3:07 AM 俊杰陈 wrote: >> > > >> > > > +1 (non-binding) >> > > > * contents looks good >> > > > * unit tests passed >> > > > >> > > > >> > > > Zoltan Ivanfi 于2019年1月11日周五 下午9:31写道： >> > > > >> > > > > +1 (binding) >> > > > > >> > > > > * contents look good >> > > > > * unit tests pass >> > > > > * checksums match >> > > > > * signature matches >> > > > > >> > > > > Br, >> > > > > >> > > > > Zoltan >> > > > > >> > > > > On Thu, Jan 10, 2019 at 11:48 AM Gabor Szadovszky < >> ga...@apache.org> >> > > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > Checked tarball: checksum/signature are correct. Content is >> correct >> > > > based >> > > > > > on release tag. Unit tests pass. >> > > > > > >> > > > > > +1 (non-binding) >> > > > > > >> > > > > > Cheers, >> > > > > > Gabor >> > > > > > >> > > > > > On Wed, Jan 9, 2019 at 4:51 PM Zoltan Ivanfi >> > > > > >> > > > > > wrote: >> > > > > > >> > > > > > > Dear Parquet Users and Developers, >> > > > > > > >> > > > > > > I propose the following RC to be released as the official >> Apache >> > > > > > > Parquet 1.11.0 release: >> > > > > > > >> > > > > > > The commit id is 8be767d12cca295cf9858a521725fc440b0c6f93 >> > > > > > > * This corresponds to the tag: apache-parquet-1.11.0 >> > > > > > > * >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> https://github.com/apache/parquet-mr/tree/8be767d12cca295cf9858a521725fc440b0c6f93 >> > > > > > > >> > > > > > > The release tarball, signature, and checksums are here: >> > > > > > > * >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.11.0-rc3/ >> > > > > > > >> > > > > > > You can find the KEYS file here: >> > > > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS >> > > > > > > >> > > > > > > Binary artifacts are staged in Nexus here: >> > > > > > > * >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/1.11.0/ >> > > > > > > >> > > > > > > This release includes the following new features: >> > > > > > > >> > > > > > > - PARQUET-1201 - Column indexes >> > > > > > > - PARQUET-1253 - Support for new logical type representation >> > > > > > > - PARQUET-1381 - Add merge blocks command to parquet-tools >> > > > > > > - PARQUET-1388 - Nanosecond precision time and timestamp - >> > > parquet-mr >> > > > > > > >> > > > > > > The release also includes bug fixes, including: >> > > > > > > >> > > > > > > - PARQUET-1472: Dictionary filter fails on >> FIXED_LEN_BYTE_ARRAY. >> > > > > > > >> > > > > > > Please download, verify, and test. The vote will be open for >> at >> > > least >> > > > > 72 >> > > > > > > hours. >> > > > > > > >> > > > > > > Thanks, >> > > > > > > >> > > > > > > Zoltan >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > >> > > > -- >> > > > Thanks & Best Regards >> > > > >> > > >> >
Re: [VOTE] Release Apache Parquet 1.11.0 RC3
Hi Wes, Thanks for checking the RC and voting. I would like to highlight that the mentioned issues are also reproducible on Linux with the parquet release 1.10.0 so these are not regressions. We currently support java8 only so I cannot see any issue with the release. I completely agree that these issues shall be fixed but I think we should not stop the release because of them. Cheers, Gabor On Mon, Jan 21, 2019 at 6:40 PM Uwe L. Korn wrote: > Hi, > > I'm sadly giving a +0 here. The signatures look good but I was unable to > build with JDK 9/10/11 on OSX. > https://issues.apache.org/jira/browse/PARQUET-1497 and > https://issues.apache.org/jira/browse/PARQUET-1496 are the problems I'm > running into. I've also opened a PR to document how to install thrift on > OSX: https://github.com/apache/parquet-mr/pull/595 > > Cheers > Uwe > > On Fri, Jan 18, 2019, at 1:55 PM, Anna Szonyi wrote: > > Hi All, > > > > While not a binding one, it's a "+1 (non-binding), everything looks good" > > from me! > > > > Best, > > Anna > > > > On Thu, Jan 17, 2019 at 6:13 PM Zoltan Ivanfi > > wrote: > > > > > Hi, > > > > > > Friendly reminder to please vote for the release. We need 2 more > binding +1 > > > votes. > > > > > > Thanks, > > > > > > Zoltan > > > > > > On Sat, Jan 12, 2019 at 3:07 AM 俊杰陈 wrote: > > > > > > > +1 (non-binding) > > > > * contents looks good > > > > * unit tests passed > > > > > > > > > > > > Zoltan Ivanfi 于2019年1月11日周五 下午9:31写道： > > > > > > > > > +1 (binding) > > > > > > > > > > * contents look good > > > > > * unit tests pass > > > > > * checksums match > > > > > * signature matches > > > > > > > > > > Br, > > > > > > > > > > Zoltan > > > > > > > > > > On Thu, Jan 10, 2019 at 11:48 AM Gabor Szadovszky < > ga...@apache.org> > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Checked tarball: checksum/signature are correct. Content is > correct > > > > based > > > > > > on release tag. Unit tests pass. > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > Cheers, > > > > > > Gabor > > > > > > > > > > > > On Wed, Jan 9, 2019 at 4:51 PM Zoltan Ivanfi > > > > > > > > > > wrote: > > > > > > > > > > > > > Dear Parquet Users and Developers, > > > > > > > > > > > > > > I propose the following RC to be released as the official > Apache > > > > > > > Parquet 1.11.0 release: > > > > > > > > > > > > > > The commit id is 8be767d12cca295cf9858a521725fc440b0c6f93 > > > > > > > * This corresponds to the tag: apache-parquet-1.11.0 > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/parquet-mr/tree/8be767d12cca295cf9858a521725fc440b0c6f93 > > > > > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.11.0-rc3/ > > > > > > > > > > > > > > You can find the KEYS file here: > > > > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS > > > > > > > > > > > > > > Binary artifacts are staged in Nexus here: > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/1.11.0/ > > > > > > > > > > > > > > This release includes the following new features: > > > > > > > > > > > > > > - PARQUET-1201 - Column indexes > > > > > > > - PARQUET-1253 - Support for new logical type representation > > > > > > > - PARQUET-1381 - Add merge blocks command to parquet-tools > > > > > > > - PARQUET-1388 - Nanosecond precision time and timestamp - > > > parquet-mr > > > > > > > > > > > > > > The release also includes bug fixes, including: > > > > > > > > > > > > > > - PARQUET-1472: Dictionary filter fails on > FIXED_LEN_BYTE_ARRAY. > > > > > > > > > > > > > > Please download, verify, and test. The vote will be open for at > > > least > > > > > 72 > > > > > > > hours. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Zoltan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Thanks & Best Regards > > > > > > > >
[jira] [Resolved] (PARQUET-1501) v1.8.x to be fixed with PARQUET-952 solution
[ https://issues.apache.org/jira/browse/PARQUET-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe L. Korn resolved PARQUET-1501. -- Resolution: Won't Fix > v1.8.x to be fixed with PARQUET-952 solution > > > Key: PARQUET-1501 > URL: https://issues.apache.org/jira/browse/PARQUET-1501 > Project: Parquet > Issue Type: Bug > Components: parquet-avro >Affects Versions: 1.8.1, 1.8.2, 1.8.3 >Reporter: Vijayakumar N >Priority: Major > > The following issue fixed in AVro parquet v1.11.0. PARQUET-952: Avro union > with single type fails with 'is not a group' > > But we require this fix in v1.8.x series also. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PARQUET-1501) v1.8.x to be fixed with PARQUET-952 solution
[ https://issues.apache.org/jira/browse/PARQUET-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe L. Korn updated PARQUET-1501: - Description: The following issue fixed in AVro parquet v1.11.0. PARQUET-952: Avro union with single type fails with 'is not a group' But we require this fix in v1.8.x series also. was: The following issue fixed in AVro parquet v1.11.0. h1. PARQUET-952: Avro union with single type fails with 'is not a group' But we require this fix in v1.8.x series also. > v1.8.x to be fixed with PARQUET-952 solution > > > Key: PARQUET-1501 > URL: https://issues.apache.org/jira/browse/PARQUET-1501 > Project: Parquet > Issue Type: Bug > Components: parquet-avro >Affects Versions: 1.8.1, 1.8.2, 1.8.3 >Reporter: Vijayakumar N >Priority: Major > > The following issue fixed in AVro parquet v1.11.0. PARQUET-952: Avro union > with single type fails with 'is not a group' > > But we require this fix in v1.8.x series also. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PARQUET-1501) v1.8.x to be fixed with PARQUET-952 solution
[ https://issues.apache.org/jira/browse/PARQUET-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748498#comment-16748498 ] Uwe L. Korn commented on PARQUET-1501: -- [~nvijayrech] We probably won't make a bugfix release for 1.8 as long as there is nobody taking care of this. Closing as we don't have sufficient volunteers. > v1.8.x to be fixed with PARQUET-952 solution > > > Key: PARQUET-1501 > URL: https://issues.apache.org/jira/browse/PARQUET-1501 > Project: Parquet > Issue Type: Bug > Components: parquet-avro >Affects Versions: 1.8.1, 1.8.2, 1.8.3 >Reporter: Vijayakumar N >Priority: Major > > The following issue fixed in AVro parquet v1.11.0. > h1. PARQUET-952: Avro union with single type fails with 'is not a group' > > But we require this fix in v1.8.x series also. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PARQUET-1501) v1.8.x to be fixed with PARQUET-952 solution
[ https://issues.apache.org/jira/browse/PARQUET-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe L. Korn updated PARQUET-1501: - Flags: Patch (was: Patch,Important) > v1.8.x to be fixed with PARQUET-952 solution > > > Key: PARQUET-1501 > URL: https://issues.apache.org/jira/browse/PARQUET-1501 > Project: Parquet > Issue Type: Bug > Components: parquet-avro >Affects Versions: 1.8.1, 1.8.2, 1.8.3 >Reporter: Vijayakumar N >Priority: Major > > The following issue fixed in AVro parquet v1.11.0. > h1. PARQUET-952: Avro union with single type fails with 'is not a group' > > But we require this fix in v1.8.x series also. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
