[jira] [Resolved] (PARQUET-482) Organize src code file structure to have a very clear folder with public headers.
[ https://issues.apache.org/jira/browse/PARQUET-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Le Dem resolved PARQUET-482. --- Resolution: Fixed Fix Version/s: cpp-0.1 Issue resolved by pull request 70 [https://github.com/apache/parquet-cpp/pull/70] > Organize src code file structure to have a very clear folder with public > headers. > - > > Key: PARQUET-482 > URL: https://issues.apache.org/jira/browse/PARQUET-482 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp >Reporter: Nong Li >Assignee: Wes McKinney > Fix For: cpp-0.1 > > > We should organize the source code structure to have a folder where all the > public headers are and nothing else. This makes it easy to understand what is > the public API and which APIs needed to be looked at wrt to compatibility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PARQUET-519) Disable compiler warning supressions and fix all DEBUG build warnings
[ https://issues.apache.org/jira/browse/PARQUET-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Le Dem resolved PARQUET-519. --- Resolution: Fixed Fix Version/s: cpp-0.1 Issue resolved by pull request 69 [https://github.com/apache/parquet-cpp/pull/69] > Disable compiler warning supressions and fix all DEBUG build warnings > - > > Key: PARQUET-519 > URL: https://issues.apache.org/jira/browse/PARQUET-519 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: cpp-0.1 > > > Related to PARQUET-447 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PARQUET-537) LocalFileSource leaks resources
[ https://issues.apache.org/jira/browse/PARQUET-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Le Dem resolved PARQUET-537. --- Resolution: Fixed Fix Version/s: cpp-0.1 Issue resolved by pull request 68 [https://github.com/apache/parquet-cpp/pull/68] > LocalFileSource leaks resources > --- > > Key: PARQUET-537 > URL: https://issues.apache.org/jira/browse/PARQUET-537 > Project: Parquet > Issue Type: Bug > Components: parquet-cpp >Affects Versions: cpp-0.1 >Reporter: Aliaksei Sandryhaila >Assignee: Aliaksei Sandryhaila > Fix For: cpp-0.1 > > > As a result of modifications introduced in PARQUET-497, LocalFileSource never > gets deleted and the associated memory and file handle are leaked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PARQUET-542) Support memory allocation from external memory
[ https://issues.apache.org/jira/browse/PARQUET-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174613#comment-15174613 ] Wes McKinney commented on PARQUET-542: -- Can you describe what you are planning? Does it relate at all to the memory pool being used for dictionary encoding? > Support memory allocation from external memory > -- > > Key: PARQUET-542 > URL: https://issues.apache.org/jira/browse/PARQUET-542 > Project: Parquet > Issue Type: New Feature > Components: parquet-cpp >Reporter: Deepak Majeti >Assignee: Aliaksei Sandryhaila > > Implement a MemoryPool like class that provides memory allocation/management > for external provided memory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PARQUET-553) The README instructions for parquet-tools do not run
Jessica Kerr created PARQUET-553: Summary: The README instructions for parquet-tools do not run Key: PARQUET-553 URL: https://issues.apache.org/jira/browse/PARQUET-553 Project: Parquet Issue Type: Bug Components: parquet-mr Reporter: Jessica Kerr The README for parquet-mr/parquet-tools says to run locally: `java jar ./parquet-tools-.jar my_parquet_file.lzo.parquet` You need a dash before "jar" or it doesn't work. ``` $ java jar parquet-tools-1.6.0rc3-SNAPSHOT.jar Error: Could not find or load main class jar ``` Also, after the 'mvn clean package' instructions given, the jar is in ./target. That part is minor, but the "jar" vs "-jar" was hard to figure out. When this did not work, I went looking for other tools. Didn't find any, so came back and tried harder. Please, please add that one character. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PARQUET-542) Support memory allocation from external memory
[ https://issues.apache.org/jira/browse/PARQUET-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174523#comment-15174523 ] Aliaksei Sandryhaila commented on PARQUET-542: -- I'm going to poke at this. We can discuss and evolve this using a WIP pull request. > Support memory allocation from external memory > -- > > Key: PARQUET-542 > URL: https://issues.apache.org/jira/browse/PARQUET-542 > Project: Parquet > Issue Type: New Feature > Components: parquet-cpp >Reporter: Deepak Majeti > > Implement a MemoryPool like class that provides memory allocation/management > for external provided memory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-542) Support memory allocation from external memory
[ https://issues.apache.org/jira/browse/PARQUET-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aliaksei Sandryhaila reassigned PARQUET-542: Assignee: Aliaksei Sandryhaila > Support memory allocation from external memory > -- > > Key: PARQUET-542 > URL: https://issues.apache.org/jira/browse/PARQUET-542 > Project: Parquet > Issue Type: New Feature > Components: parquet-cpp >Reporter: Deepak Majeti >Assignee: Aliaksei Sandryhaila > > Implement a MemoryPool like class that provides memory allocation/management > for external provided memory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-519) Disable compiler warning supressions and fix all DEBUG build warnings
[ https://issues.apache.org/jira/browse/PARQUET-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-519: Assignee: Wes McKinney > Disable compiler warning supressions and fix all DEBUG build warnings > - > > Key: PARQUET-519 > URL: https://issues.apache.org/jira/browse/PARQUET-519 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney > > Related to PARQUET-447 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-446) Hide thrift dependency in parquet-cpp
[ https://issues.apache.org/jira/browse/PARQUET-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-446: Assignee: Wes McKinney > Hide thrift dependency in parquet-cpp > - > > Key: PARQUET-446 > URL: https://issues.apache.org/jira/browse/PARQUET-446 > Project: Parquet > Issue Type: Bug > Components: parquet-cpp >Reporter: Nong Li >Assignee: Wes McKinney > Fix For: cpp-0.1 > > > Pulling in thrift compiled headers tend to pull in a lot of things. It would > be nice to not expose them in the parquet library (the application should be > able to use a different version of thrift, etc). > We can also see if it is practical to not depend on thrift at all and > replicate the logic we need. Thrift is fairly stable at this point so this > might be feasible. This would allow us to do things like not rely on boost. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-433) Specialize ColumnReaders based on the column type
[ https://issues.apache.org/jira/browse/PARQUET-433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-433: Assignee: Wes McKinney > Specialize ColumnReaders based on the column type > - > > Key: PARQUET-433 > URL: https://issues.apache.org/jira/browse/PARQUET-433 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp >Reporter: Aliaksei Sandryhaila >Assignee: Wes McKinney > > ColumnReader class is used to read columns of all types. This leads to a lot > of type checking and 'switch' statements. ColumnReaders should be specialized > to different types, while sharing the same interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-463) Add DCHECK* macros for assertions in debug builds
[
https://issues.apache.org/jira/browse/PARQUET-463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney reassigned PARQUET-463:
Assignee: Wes McKinney
> Add DCHECK* macros for assertions in debug builds
> -
>
> Key: PARQUET-463
> URL: https://issues.apache.org/jira/browse/PARQUET-463
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-cpp
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Minor
> Fix For: cpp-0.1
>
>
> Some of these macros are already defined in {{parquet/util/logging.h}}, but
> they are no-ops. This will assist in "can't fail" assertions. See
> https://www.chromium.org/developers/coding-style#TOC-CHECK-DCHECK-and-NOTREACHED-
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PARQUET-469) Roll back Thrift bindings to 0.9.0
[ https://issues.apache.org/jira/browse/PARQUET-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-469: Assignee: Wes McKinney > Roll back Thrift bindings to 0.9.0 > -- > > Key: PARQUET-469 > URL: https://issues.apache.org/jira/browse/PARQUET-469 > Project: Parquet > Issue Type: Bug > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: cpp-0.1 > > > Thrift 0.9.3 conflicts with googletest in ugly ways on gcc 4.9. This is a > stopgap until PARQUET-468 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-451) Add a RowGroup reader interface class
[
https://issues.apache.org/jira/browse/PARQUET-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney reassigned PARQUET-451:
Assignee: Wes McKinney
> Add a RowGroup reader interface class
> -
>
> Key: PARQUET-451
> URL: https://issues.apache.org/jira/browse/PARQUET-451
> Project: Parquet
> Issue Type: New Feature
> Components: parquet-cpp
>Reporter: Wes McKinney
>Assignee: Wes McKinney
> Fix For: cpp-0.1
>
>
> Currently the logic for interacting with row group metadata and constructing
> column decoders is embedded in the parquet_reader.cc executable here:
> https://github.com/apache/parquet-cpp/blob/master/example/parquet_reader.cc
> With PARQUET-434, we have a file reader container, which can then provide a
> row group reader container, something like
> {code}
> RowGroupReader* group_reader = file_reader->row_group(i);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PARQUET-453) Refactor parquet_reader.cc into a ParquetFileReader::DebugPrint method
[ https://issues.apache.org/jira/browse/PARQUET-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-453: Assignee: Wes McKinney > Refactor parquet_reader.cc into a ParquetFileReader::DebugPrint method > -- > > Key: PARQUET-453 > URL: https://issues.apache.org/jira/browse/PARQUET-453 > Project: Parquet > Issue Type: New Feature > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney > > This is follow up work per discussion in PARQUET-418 and > https://github.com/apache/parquet-cpp/pull/18 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-525) Test coverage for malformed file failure modes on the read path
[ https://issues.apache.org/jira/browse/PARQUET-525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-525: Assignee: Wes McKinney > Test coverage for malformed file failure modes on the read path > --- > > Key: PARQUET-525 > URL: https://issues.apache.org/jira/browse/PARQUET-525 > Project: Parquet > Issue Type: Test > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: cpp-0.1 > > > These code paths do not have test coverage. We should construct test cases > that each possible kind of malformation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PARQUET-514) Automate coveralls.io updates in Travis CI
[ https://issues.apache.org/jira/browse/PARQUET-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned PARQUET-514: Assignee: Wes McKinney > Automate coveralls.io updates in Travis CI > -- > > Key: PARQUET-514 > URL: https://issues.apache.org/jira/browse/PARQUET-514 > Project: Parquet > Issue Type: Improvement > Components: parquet-cpp >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Minor > Fix For: cpp-0.1 > > > The repo has been enabled in INFRA-11273, so all that's left is to work on > the Travis CI build matrix and add coveralls to one of the builds (rather > than running it for all of them) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PARQUET-552) Add functionality to estimate dynamic memory usage prior to reading a Parquet file/rowgroup
Aliaksei Sandryhaila created PARQUET-552: Summary: Add functionality to estimate dynamic memory usage prior to reading a Parquet file/rowgroup Key: PARQUET-552 URL: https://issues.apache.org/jira/browse/PARQUET-552 Project: Parquet Issue Type: New Feature Components: parquet-cpp Reporter: Aliaksei Sandryhaila -- This message was sent by Atlassian JIRA (v6.3.4#6332)
