Re: [VOTE] Release Apache Parquet 1.11.0 RC7

2019-11-22 Thread Nandor Kollar
'_/ > > >/___/ .__/\_,_/_/ /_/\_\ version 2.4.4 > > > /_/ > > > > > > Using Scala version 2.11.12, Java HotSpot(TM) 64-Bit Server VM, > 1.8.0_191 > > > Branch > > > Compiled by user on 2019-08-27T21:21:38Z > >

Re: [VOTE] Release Apache Parquet 1.11.0 RC7

2019-11-21 Thread Nandor Kollar
I'm not sure that this is a binary compatibility issue. The missing builder method was recently added in 1.11.0 with the introduction of the new logical type API, while the original version (one with a single OriginalType input parameter called before by AvroSchemaConverter) of this method is kept

[jira] [Resolved] (PARQUET-1445) Remove Files.java

2019-09-05 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1445. Resolution: Fixed > Remove Files.java > - > >

[jira] [Assigned] (PARQUET-1445) Remove Files.java

2019-09-05 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1445: -- Assignee: David Mollitor > Remove Files.j

[jira] [Commented] (PARQUET-247) Add DATE mapping in ValidTypeMap of filter2

2019-09-05 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923149#comment-16923149 ] Nandor Kollar commented on PARQUET-247: --- Since PARQUET-201 was resolved by removing OriginalType

[jira] [Resolved] (PARQUET-1530) Remove Dependency on commons-codec

2019-09-05 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1530. Resolution: Fixed > Remove Dependency on commons-co

[jira] [Assigned] (PARQUET-1641) Parquet pages for different columns cannot be read in parallel

2019-08-29 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1641: -- Assignee: Samarth Jain > Parquet pages for different columns cannot be r

[jira] [Assigned] (PARQUET-1643) Use airlift non-native implementations for GZIP, LZ0 and LZ4 codecs

2019-08-29 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1643: -- Assignee: Samarth Jain > Use airlift non-native implementations for GZIP,

[jira] [Assigned] (PARQUET-1597) Fix parquet-cli's wrong or missing usage examples

2019-08-22 Thread Nandor Kollar (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1597: -- Assignee: Kengo Seki > Fix parquet-cli's wrong or missing usage examp

[jira] [Resolved] (PARQUET-1637) Builds are failing because default jdk changed to openjdk11 on Travis

2019-08-15 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1637. Resolution: Fixed > Builds are failing because default jdk changed to openjd

[jira] [Created] (PARQUET-1637) Builds are failing because default jdk changed to openjdk11 on Travis

2019-08-10 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1637: -- Summary: Builds are failing because default jdk changed to openjdk11 on Travis Key: PARQUET-1637 URL: https://issues.apache.org/jira/browse/PARQUET-1637 Project

[jira] [Updated] (PARQUET-1303) Avro reflect @Stringable field write error if field not instanceof CharSequence

2019-07-25 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1303: --- Fix Version/s: 1.12.0 > Avro reflect @Stringable field write error if field not instanc

[jira] [Resolved] (PARQUET-1303) Avro reflect @Stringable field write error if field not instanceof CharSequence

2019-07-25 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1303. Resolution: Fixed > Avro reflect @Stringable field write error if field not instanc

[jira] [Assigned] (PARQUET-1303) Avro reflect @Stringable field write error if field not instanceof CharSequence

2019-07-25 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1303: -- Assignee: Zack Behringer > Avro reflect @Stringable field write error if fi

[jira] [Resolved] (PARQUET-1605) Bump maven-javadoc-plugin from 2.9 to 3.1.0

2019-07-24 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1605. Resolution: Fixed > Bump maven-javadoc-plugin from 2.9 to 3.

[jira] [Resolved] (PARQUET-1606) Fix invalid tests scope

2019-07-23 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1606. Resolution: Fixed > Fix invalid tests sc

[jira] [Resolved] (PARQUET-1600) Fix shebang in parquet-benchmarks/run.sh

2019-07-23 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1600. Resolution: Fixed > Fix shebang in parquet-benchmarks/run

[jira] [Resolved] (PARQUET-1552) upgrade protoc-jar-maven-plugin to 3.8.0

2019-07-10 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1552. Resolution: Fixed > upgrade protoc-jar-maven-plugin to 3.

Re: New PMC member: Gabor Szadovszky

2019-06-28 Thread Nandor Kollar
Congratulations Gabor! On Fri, Jun 28, 2019 at 4:51 PM Zoltan Ivanfi wrote: > Hi, > > The Project Management Committee (PMC) for Apache Parquet has invited Gabor > Szadovszky to become a member of the PMC and we are pleased to announce > that he has accepted. > > Congratulations, Gabor! > > Br,

Re: New committer: Fokko Driesprong

2019-06-26 Thread Nandor Kollar
Congratulations Fokko! On Tue, Jun 25, 2019 at 7:32 PM Xinli shang wrote: > Congratulations Fokko! > > On Tue, Jun 25, 2019 at 8:55 AM Lars Volker > wrote: > > > Congratulations Fokko! > > > > On Tue, Jun 25, 2019 at 8:52 AM Tim Armstrong > > wrote: > > > > > Congratulations! > > > > > > On

[jira] [Comment Edited] (PARQUET-1496) [Java] Update Scala for JDK 11 compatibility

2019-05-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832559#comment-16832559 ] Nandor Kollar edited comment on PARQUET-1496 at 5/3/19 2:55 PM: Scrooge

[jira] [Comment Edited] (PARQUET-1496) [Java] Update Scala for JDK 11 compatibility

2019-05-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832559#comment-16832559 ] Nandor Kollar edited comment on PARQUET-1496 at 5/3/19 2:54 PM: Scrooge

[jira] [Commented] (PARQUET-1496) [Java] Update Scala for JDK 11 compatibility

2019-05-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832559#comment-16832559 ] Nandor Kollar commented on PARQUET-1496: Scrooge recently release 19.4.0 with a fix

[jira] [Commented] (PARQUET-1556) Instructions are missing for configuring twitter maven repo for hadoop-lzo dependency

2019-04-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808837#comment-16808837 ] Nandor Kollar commented on PARQUET-1556: This additional repository should be in the POM

[jira] [Created] (PARQUET-1554) Compilation error when upgrading Scrooge version

2019-04-02 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1554: -- Summary: Compilation error when upgrading Scrooge version Key: PARQUET-1554 URL: https://issues.apache.org/jira/browse/PARQUET-1554 Project: Parquet

[jira] [Commented] (PARQUET-1548) Meta data is lost when writing avro union types to parquet

2019-03-21 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798073#comment-16798073 ] Nandor Kollar commented on PARQUET-1548: Would you mind sharing more details how this happens

[jira] [Resolved] (PARQUET-1545) Logical Type for timezone-naive timestamps

2019-03-20 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1545. Resolution: Fixed > Logical Type for timezone-naive timesta

[jira] [Commented] (PARQUET-1545) Logical Type for timezone-naive timestamps

2019-03-20 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796990#comment-16796990 ] Nandor Kollar commented on PARQUET-1545: Great, closing this Jira as done then. By the way

[jira] [Commented] (PARQUET-1545) Logical Type for timezone-naive timestamps

2019-03-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795001#comment-16795001 ] Nandor Kollar commented on PARQUET-1545: [~swast] actually Parquet already supports both

Re: [VOTE] Release Apache Parquet 1.10.1 RC0

2019-01-29 Thread Nandor Kollar
+1 (non-binding) Verified signature and checksum, ran unit tests, all passed. Cheers, Nandor On Tue, Jan 29, 2019 at 1:21 PM Gabor Szadovszky wrote: > Hi Ryan, > > Checked the tarball: checksum/signature are correct. Content is correct > based on the release tag. Unit tests pass. > > +1

Re: [DISCUSS] Remove old modules?

2019-01-29 Thread Nandor Kollar
Removing parquet-hive-* is a great idea, the code in Parquet is not maintained any more, it is just a burden there. As of parquet-pig, I'd prefer moving it to Pig (if Pig community accepts it as it is) instead of dropping it or moving to a separate project. I know people who still use Pig with

[jira] [Updated] (PARQUET-409) InternalParquetRecordWriter doesn't use min/max row counts

2019-01-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-409: -- Fix Version/s: (was: 1.9.0) > InternalParquetRecordWriter doesn't use min/max row cou

[jira] [Updated] (PARQUET-182) FilteredRecordReader skips rows it shouldn't for schema with optional columns

2019-01-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-182: -- Fix Version/s: (was: 1.9.0) > FilteredRecordReader skips rows it shouldn't for sch

[jira] [Updated] (PARQUET-1006) ColumnChunkPageWriter uses only heap memory.

2019-01-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1006: --- Fix Version/s: (was: 1.9.0) > ColumnChunkPageWriter uses only heap mem

[jira] [Resolved] (PARQUET-1278) [Java] parquet-arrow is broken due to different JDK version

2019-01-11 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1278. Resolution: Duplicate [~andygrove] I resolved this Jira since it looked like

[jira] [Assigned] (PARQUET-1478) Can't read spec compliant, 3-level lists via parquet-proto

2019-01-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1478: -- Assignee: Nandor Kollar > Can't read spec compliant, 3-level lists via parq

[jira] [Created] (PARQUET-1478) Can't read spec compliant, 3-level lists via parquet-proto

2018-12-14 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1478: -- Summary: Can't read spec compliant, 3-level lists via parquet-proto Key: PARQUET-1478 URL: https://issues.apache.org/jira/browse/PARQUET-1478 Project: Parquet

[jira] [Created] (PARQUET-1476) Don't emit a warning message for files without new logical type

2018-12-13 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1476: -- Summary: Don't emit a warning message for files without new logical type Key: PARQUET-1476 URL: https://issues.apache.org/jira/browse/PARQUET-1476 Project

[jira] [Commented] (PARQUET-1434) Release parquet-mr 1.11.0

2018-12-12 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718669#comment-16718669 ] Nandor Kollar commented on PARQUET-1434: [~yumwang] the release is not yet finished, voting

Re: Regarding Apache Parquet Project

2018-12-11 Thread Nandor Kollar
Hi Arjit, I'd also recommend you to have a look at Parquet website: https://parquet.apache.org/ You can find a couple of old, but great presentations there, I recommend you to watch those to understand the basics (despite Parquet improved during the years with additional features, the basics

[jira] [Commented] (PARQUET-1470) Inputstream leakage in ParquetFileWriter.appendFile

2018-12-04 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708702#comment-16708702 ] Nandor Kollar commented on PARQUET-1470: Sounds like a reasonable improvement. Would you mind

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-28 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701851#comment-16701851 ] Nandor Kollar commented on PARQUET-1441: Ok, so looks like there is a more fundamental problem

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700913#comment-16700913 ] Nandor Kollar commented on PARQUET-1441: The compatibility check introduced by PARQUET-651

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700502#comment-16700502 ] Nandor Kollar commented on PARQUET-1441: Looks like this compatibility check broke

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700445#comment-16700445 ] Nandor Kollar commented on PARQUET-1441: After a bit more investigation, I think

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699061#comment-16699061 ] Nandor Kollar commented on PARQUET-1441: Well, I think the correct solution is setting

[jira] [Commented] (PARQUET-1441) SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter

2018-11-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698733#comment-16698733 ] Nandor Kollar commented on PARQUET-1441: The workaround proposed above should work, however

Re: [VOTE] Release Apache Parquet 1.11.0 RC0

2018-11-22 Thread Nandor Kollar
+1 (non-binding) Verified checksums, all matched. Executed unit tests, each passed. Regards, Nandor On Thu, Nov 22, 2018 at 9:07 AM Gabor Szadovszky wrote: > > Hi, > > Verified source tarball checksums and content. All are correct. Unit tests > pass. > > +1 (non-binding) > > Cheers, > Gabor > >

[jira] [Commented] (PARQUET-1407) Data loss on duplicate values with AvroParquetWriter/Reader

2018-11-19 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692014#comment-16692014 ] Nandor Kollar commented on PARQUET-1407: [~rdblue] [~scottcarey] [~jackytan] I added a unit

[jira] [Comment Edited] (PARQUET-1407) Data loss on duplicate values with AvroParquetWriter/Reader

2018-11-19 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692014#comment-16692014 ] Nandor Kollar edited comment on PARQUET-1407 at 11/19/18 5:09 PM

[jira] [Updated] (PARQUET-1451) Deprecate old logical types API

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1451: --- Fix Version/s: (was: 1.11.0) > Deprecate old logical types

[jira] [Updated] (PARQUET-1309) Parquet Java uses incorrect stats and dictionary filter properties

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1309: --- Fix Version/s: (was: 1.10.1) > Parquet Java uses incorrect stats and dictionary fil

[jira] [Updated] (PARQUET-1325) High-level flexible and fine-grained column level access control through encryption with pluggable key access

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1325: --- Fix Version/s: (was: 1.10.1) > High-level flexible and fine-grained column le

[jira] [Updated] (PARQUET-1341) Null count is suppressed when columns have no min or max and use unsigned sort order

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1341: --- Fix Version/s: (was: 1.10.1) > Null count is suppressed when columns have no

[jira] [Updated] (PARQUET-1435) Benchmark filtering column-indexes

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1435: --- Fix Version/s: (was: 1.11.0) > Benchmark filtering column-inde

[jira] [Updated] (PARQUET-1292) Add constructors to ProtoParquetWriter to write specs compliant Parquet

2018-11-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1292: --- Fix Version/s: (was: 1.11.0) > Add constructors to ProtoParquetWriter to write sp

Re: When the next release with page index release?

2018-11-05 Thread Nandor Kollar
Hi Aron Tao, We plan to have a release with page indexes and bunch of new features and bug fixes soon. Can't tell any ETA now, but we'll discuss it on the next Parquet sync tomorrow. Regards, Nandor On Thu, Nov 1, 2018 at 3:53 PM Tao JiaTao wrote: > > > Hi > > Recently I’m reading Parquet’s

[jira] [Updated] (PARQUET-1419) Enable old readers to access unencrypted columns in files with plaintext footer

2018-10-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1419: --- Fix Version/s: encryption-feature-branch > Enable old readers to access unencryp

[jira] [Updated] (PARQUET-1232) Document the modular encryption in parquet-format

2018-10-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1232: --- Fix Version/s: encryption-feature-branch > Document the modular encryption in parq

[jira] [Updated] (PARQUET-1401) RowGroup offset and total compressed size fields

2018-10-18 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1401: --- Fix Version/s: encryption-feature-branch > RowGroup offset and total compressed s

[jira] [Assigned] (PARQUET-1331) Use new logical types API in parquet-mr

2018-10-15 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1331: -- Assignee: (was: Nandor Kollar) > Use new logical types API in parquet

[jira] [Resolved] (PARQUET-1331) Use new logical types API in parquet-mr

2018-10-15 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1331. Resolution: Duplicate > Use new logical types API in parquet

[jira] [Updated] (PARQUET-1433) Parquet-format doesn't compile with Thrift 0.10.0

2018-10-15 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1433: --- Fix Version/s: format-2.7.0 > Parquet-format doesn't compile with Thrift 0.1

[jira] [Assigned] (PARQUET-1436) TimestampMicrosStringifier shows wrong microseconds for timestamps before 1970

2018-10-15 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1436: -- Assignee: Nandor Kollar > TimestampMicrosStringifier shows wrong microseco

[jira] [Updated] (PARQUET-1383) Parquet tools should indicate UTC parameter for time/timestamp types

2018-10-11 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1383: --- Fix Version/s: 1.11.0 > Parquet tools should indicate UTC parameter for time/timest

[jira] [Created] (PARQUET-1434) Release parquet-mr 1.11.0

2018-10-03 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1434: -- Summary: Release parquet-mr 1.11.0 Key: PARQUET-1434 URL: https://issues.apache.org/jira/browse/PARQUET-1434 Project: Parquet Issue Type: Task

[jira] [Updated] (PARQUET-1388) Nanosecond precision time and timestamp - parquet-mr

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1388: --- Fix Version/s: 1.11.0 > Nanosecond precision time and timestamp - parquet

[jira] [Updated] (PARQUET-1201) Column indexes

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1201: --- Fix Version/s: 1.11.0 > Column indexes > -- > >

[jira] [Updated] (PARQUET-1368) ParquetFileReader should close its input stream for the failure in constructor

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1368: --- Fix Version/s: 1.11.0 > ParquetFileReader should close its input stream for the fail

[jira] [Resolved] (PARQUET-1428) Move columnar encryption into its feature branch

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1428. Resolution: Fixed > Move columnar encryption into its feature bra

[jira] [Assigned] (PARQUET-1428) Move columnar encryption into its feature branch

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1428: -- Assignee: Nandor Kollar > Move columnar encryption into its feature bra

[jira] [Resolved] (PARQUET-1424) Release parquet-format 2.6.0

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1424. Resolution: Fixed > Release parquet-format 2.

[jira] [Assigned] (PARQUET-1424) Release parquet-format 2.6.0

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1424: -- Assignee: Nandor Kollar > Release parquet-format 2.

[jira] [Updated] (PARQUET-1399) Move parquet-mr related code from parquet-format

2018-10-02 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1399: --- Fix Version/s: 1.11.0 > Move parquet-mr related code from parquet-for

Re: [VOTE] Release Apache Parquet format 2.6.0 RC0

2018-10-01 Thread Nandor Kollar
ure > > > > > > Cheers, > > > Gabor > > > > > > On Thu, Sep 27, 2018 at 5:10 PM Zoltan Ivanfi > > > wrote: > > > > > > > +1 (binding) > > > > > > > > - contents look good > > > &g

[jira] [Assigned] (PARQUET-1433) Parquet-format doesn't compile with Thrift 0.10.0

2018-10-01 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar reassigned PARQUET-1433: -- Assignee: Nandor Kollar > Parquet-format doesn't compile with Thrift 0.1

Re: [VOTE] Release Apache Parquet format 2.6.0 RC0

2018-10-01 Thread Nandor Kollar
, Sep 27, 2018 at 5:10 PM Zoltan Ivanfi > > wrote: > > > > > +1 (binding) > > > > > > - contents look good > > > - units tests pass > > > - checksums match > > > - signature matches > > > > > > Thanks, >

[jira] [Created] (PARQUET-1433) Parquet-format doesn't compile with Thrift 0.10.0

2018-10-01 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1433: -- Summary: Parquet-format doesn't compile with Thrift 0.10.0 Key: PARQUET-1433 URL: https://issues.apache.org/jira/browse/PARQUET-1433 Project: Parquet

[VOTE] Release Apache Parquet format 2.6.0 RC0

2018-09-27 Thread Nandor Kollar
Hi everyone, I propose the following RC to be released as official Apache Parquet Format 2.6.0 release. The commit id is df6132b94f273521a418a74442085fdd5a0aa009 * This corresponds to the tag: apache-parquet-format-2.6.0 *

[jira] [Resolved] (PARQUET-1429) Turn off DocLint on parquet-format

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar resolved PARQUET-1429. Resolution: Fixed > Turn off DocLint on parquet-for

[jira] [Created] (PARQUET-1429) Turn off DocLint on parquet-format

2018-09-27 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1429: -- Summary: Turn off DocLint on parquet-format Key: PARQUET-1429 URL: https://issues.apache.org/jira/browse/PARQUET-1429 Project: Parquet Issue Type: Task

[jira] [Updated] (PARQUET-1227) Thrift crypto metadata structures

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1227: --- Fix Version/s: format-encryption-feature-branch > Thrift crypto metadata structu

[jira] [Updated] (PARQUET-1398) Separate iv_prefix for GCM and CTR modes

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1398: --- Fix Version/s: format-encryption-feature-branch > Separate iv_prefix for GCM and CTR mo

[jira] [Updated] (PARQUET-1424) Release parquet-format 2.6.0

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1424: --- Fix Version/s: (was: format-2.6.0) > Release parquet-format 2.

[jira] [Updated] (PARQUET-1227) Thrift crypto metadata structures

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1227: --- Fix Version/s: (was: format-2.6.0) > Thrift crypto metadata structu

[jira] [Updated] (PARQUET-1398) Separate iv_prefix for GCM and CTR modes

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1398: --- Fix Version/s: (was: format-2.6.0) > Separate iv_prefix for GCM and CTR mo

[jira] [Created] (PARQUET-1428) Move columnar encryption into its feature branch

2018-09-27 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1428: -- Summary: Move columnar encryption into its feature branch Key: PARQUET-1428 URL: https://issues.apache.org/jira/browse/PARQUET-1428 Project: Parquet

[jira] [Updated] (PARQUET-1425) [Format] Fix Thrift compiler warning

2018-09-27 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1425: --- Fix Version/s: (was: format-2.6.0) > [Format] Fix Thrift compiler warn

[jira] [Commented] (PARQUET-1425) [Format] Fix Thrift compiler warning

2018-09-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628950#comment-16628950 ] Nandor Kollar commented on PARQUET-1425: [~wesmckinn] would you like this to be fixed in 2.6.0

[jira] [Updated] (PARQUET-1387) Nanosecond precision time and timestamp - parquet-format

2018-09-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1387: --- Fix Version/s: format-2.6.0 > Nanosecond precision time and timestamp - parquet-for

[jira] [Updated] (PARQUET-1266) LogicalTypes union in parquet-format doesn't include UUID

2018-09-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1266: --- Fix Version/s: format-2.6.0 > LogicalTypes union in parquet-format doesn't include U

[jira] [Updated] (PARQUET-1290) Clarify maximum run lengths for RLE encoding

2018-09-26 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1290: --- Fix Version/s: format-2.6.0 > Clarify maximum run lengths for RLE encod

[jira] [Created] (PARQUET-1424) Release parquet-format 2.6.0

2018-09-25 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1424: -- Summary: Release parquet-format 2.6.0 Key: PARQUET-1424 URL: https://issues.apache.org/jira/browse/PARQUET-1424 Project: Parquet Issue Type: Task

Re: Date and time for next Parquet sync

2018-09-18 Thread Nandor Kollar
t be able to attend after all, sorry for the late > decline. > > Zoltan > > On Mon, Sep 10, 2018 at 7:21 PM Ryan Blue wrote: >> >> Sorry, looks like I was wrong on the dates. Thanks, Nandor. >> >> On Mon, Sep 10, 2018 at 5:15 AM Nandor Kollar >> wrote:

[jira] [Updated] (PARQUET-1383) Parquet tools should indicate UTC parameter for time/timestamp types

2018-09-12 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1383: --- Description: Parquet-tools should indicate if a time/timestamp is UTC adjusted or timezone

[jira] [Updated] (PARQUET-1383) Parquet tools should indicate UTC parameter for time/timestamp types

2018-09-12 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1383: --- Summary: Parquet tools should indicate UTC parameter for time/timestamp types

Re: Date and time for next Parquet sync

2018-09-10 Thread Nandor Kollar
emoving the duplicate part from the google doc >> (keeping the samples), with new comments to go to PRs (md and code). I'll >> send a detailed mail early next week. >> >> >> Cheers, Gidon. >> >> On Fri, Sep 7, 2018 at 3:42 PM Nandor Kollar > >

Date and time for next Parquet sync

2018-09-07 Thread Nandor Kollar
Hi All, I'd like propose to have a Parquet Sync next week Tuesday (September 18th) at 6pm CEST / 9 am PST. Some of the topics which would be nice to discuss: - review column indexes (PRs and feature branch) - move Java code from format to mr (PR #517) - Bloom filter spec - columnar encryption

[jira] [Updated] (PARQUET-1371) Time/Timestamp UTC normalization parameter doesn't work

2018-09-06 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated PARQUET-1371: --- Fix Version/s: 1.11.0 > Time/Timestamp UTC normalization parameter doesn't w

[jira] [Created] (PARQUET-1410) Refactor modules to use the new logical type API

2018-08-31 Thread Nandor Kollar (JIRA)
Nandor Kollar created PARQUET-1410: -- Summary: Refactor modules to use the new logical type API Key: PARQUET-1410 URL: https://issues.apache.org/jira/browse/PARQUET-1410 Project: Parquet

  1   2   3   >