[
https://issues.apache.org/jira/browse/PARQUET-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17249775#comment-17249775
]
ASF GitHub Bot commented on PARQUET-1954:
-----------------------------------------
gszadovszky commented on a change in pull request #849:
URL: https://github.com/apache/parquet-mr/pull/849#discussion_r543485871
##########
File path:
parquet-tools/src/main/java/org/apache/parquet/tools/command/DumpCommand.java
##########
@@ -165,48 +165,47 @@ public static void dump(PrettyPrintWriter out,
ParquetMetadata meta, MessageType
ParquetFileReader freader = null;
if (showmd) {
+ long group = 0;
+ for (BlockMetaData block : blocks) {
try {
- long group = 0;
- for (BlockMetaData block : blocks) {
- if (group != 0) out.println();
- out.format("row group %d%n", group++);
- out.rule('-');
-
-
- List<ColumnChunkMetaData> ccmds = block.getColumns();
- if (showColumns != null) {
- ccmds = new ArrayList<ColumnChunkMetaData>();
- for (ColumnChunkMetaData ccmd : block.getColumns()) {
- String path =
Joiner.on('.').skipNulls().join(ccmd.getPath().toArray());
- if (showColumns.contains(path)) {
- ccmds.add(ccmd);
- }
- }
- }
-
- MetadataUtils.showDetails(out, ccmds);
+ if (group != 0)
+ out.println();
+ out.format("row group %d%n", group++);
+ out.rule('-');
+
+ List<ColumnChunkMetaData> ccmds = block.getColumns();
+ if (showColumns != null) {
+ ccmds = new ArrayList<ColumnChunkMetaData>();
+ for (ColumnChunkMetaData ccmd : block.getColumns()) {
+ String path =
Joiner.on('.').skipNulls().join(ccmd.getPath().toArray());
+ if (showColumns.contains(path)) {
+ ccmds.add(ccmd);
+ }
+ }
+ }
- List<BlockMetaData> rblocks =
Collections.singletonList(block);
- freader = new ParquetFileReader(
- conf, meta.getFileMetaData(), inpath, rblocks,
columns);
- PageReadStore store = freader.readNextRowGroup();
- while (store != null) {
- out.incrementTabLevel();
- for (ColumnDescriptor column : columns) {
- out.println();
- dump(out, store, column);
- }
- out.decrementTabLevel();
+ MetadataUtils.showDetails(out, ccmds);
- store = freader.readNextRowGroup();
- }
- out.flushColumns();
+ List<BlockMetaData> rblocks = Collections.singletonList(block);
+ freader = new ParquetFileReader(conf, meta.getFileMetaData(),
inpath, rblocks, columns);
+ PageReadStore store = freader.readNextRowGroup();
+ while (store != null) {
+ out.incrementTabLevel();
+ for (ColumnDescriptor column : columns) {
+ out.println();
+ dump(out, store, column);
}
+ out.decrementTabLevel();
+
+ store = freader.readNextRowGroup();
+ }
+ out.flushColumns();
} finally {
- if (freader != null) {
- freader.close();
- }
+ if (freader != null) {
+ freader.close();
Review comment:
If an I/O exception occurs in the `ParquetFileReader` constructor then
you might try to close the previous reader object. I would guess there is no
problem with invoking `close` multiple times on the same object but who knows?
It might be a good idea to reset the reference so you would not close the same
object again. Another idea would be to use try-with-resources for the code part
where the reader is really needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> TCP connection leak in parquet dump
> -----------------------------------
>
> Key: PARQUET-1954
> URL: https://issues.apache.org/jira/browse/PARQUET-1954
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Reporter: xiepengjie
> Priority: Major
>
> Hi,
> When i'm trying to dump a parquet file, i find the TCP connection leak in
> org.apache.parquet.tools.command.DumpCommand#dump :
>
> {code:java}
> ...
> ParquetFileReader freader = null;
> if (showmd) {
> try {
> long group = 0;
> for (BlockMetaData block : blocks) {
> ...
> freader = new ParquetFileReader(
> conf, meta.getFileMetaData(), inpath, rblocks, columns);
> ...
> out.flushColumns();
> }
> } finally {
> if (freader != null) {
> freader.close();
> }
> }
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
