date:20180327

[jira] [Commented] (PARQUET-1143) Update Java for format 2.4.0 changes

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416235#comment-16416235
 ] 

ASF GitHub Bot commented on PARQUET-1143:
-

scottcarey commented on issue #430: PARQUET-1143: Update to Parquet format 
2.4.0.
URL: https://github.com/apache/parquet-mr/pull/430#issuecomment-376672804
 
 
   Anything that I can help with to get 1.10.0 out?  I'll be happy to test out 
any RCs on my use case.  I'd rather spend time helping with 1.10.0 than testing 
a custom built version, but I may be forced to build and test a custom version 
if an official release with zstd available takes too long.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Update Java for format 2.4.0 changes
> 
>
> Key: PARQUET-1143
> URL: https://issues.apache.org/jira/browse/PARQUET-1143
> Project: Parquet
>  Issue Type: Task
>  Components: parquet-mr
>Affects Versions: 1.9.0, 1.8.2
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (PARQUET-1256) Please review and merge: added --print-key-value-metadata option to parquet_reader tool

2018-03-27 Thread Jacek Pliszka (JIRA)

Jacek Pliszka created PARQUET-1256:
--

 Summary: Please review and merge: added --print-key-value-metadata 
option to parquet_reader tool
 Key: PARQUET-1256
 URL: https://issues.apache.org/jira/browse/PARQUET-1256
 Project: Parquet
  Issue Type: Improvement
  Components: parquet-cpp
Reporter: Jacek Pliszka


Added --print-key-value-metadata option to parquet_reader tool

https://github.com/apache/parquet-cpp/pull/450

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415757#comment-16415757
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on a change in pull request #411: PARQUET-968 Add 
Hive/Presto support in ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#discussion_r177454755
 
 

 ##
 File path: 
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoMessageConverter.java
 ##
 @@ -345,4 +351,121 @@ public void addBinary(Binary binary) {
 }
 
   }
+
+  /**
+   * This class unwraps the additional LIST wrapper and makes it possible to 
read the underlying data and then convert
+   * it to protobuf.
+   * 
+   * Consider the following protobuf schema:
+   * message SimpleList {
+   *   repeated int64 first_array = 1;
+   * }
+   * 
+   * A LIST wrapper is created in parquet for the above mentioned protobuf 
schema:
+   * message SimpleList {
+   *   required group first_array (LIST) = 1 {
+   * repeated int32 element;
+   *   }
+   * }
+   * 
+   * The LIST wrappers are used by 3rd party tools, such as Hive, to read 
parquet arrays. The wrapper contains
+   * one only one field: either a primitive field (like in the example above, 
where we have an array of ints) or
+   * another group (array of messages).
+   */
+  final class ListConverter extends GroupConverter {
+private final Converter converter;
+private final boolean listOfMessage;
+
+public ListConverter(Message.Builder parentBuilder, 
Descriptors.FieldDescriptor fieldDescriptor, Type parquetType) {
+  OriginalType originalType = parquetType.getOriginalType();
+  if (originalType != OriginalType.LIST) {
+throw new ParquetDecodingException("Expected LIST wrapper. Found: " + 
originalType + " instead.");
+  }
+
+  listOfMessage = fieldDescriptor.getJavaType() == JavaType.MESSAGE;
 
 Review comment:
   done in https://github.com/costimuraru/parquet-mr/pull/2


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415758#comment-16415758
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on a change in pull request #411: PARQUET-968 Add 
Hive/Presto support in ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#discussion_r177454793
 
 

 ##
 File path: 
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoMessageConverter.java
 ##
 @@ -388,7 +388,7 @@ public ListConverter(Message.Builder parentBuilder, 
Descriptors.FieldDescriptor
   if (parquetType.asGroupType().containsField("list")) {
 parquetSchema = parquetType.asGroupType().getType("list");
 if (parquetSchema.asGroupType().containsField("element")) {
-  parquetSchema.asGroupType().getType("element");
+  parquetSchema = parquetSchema.asGroupType().getType("element");
 
 Review comment:
   done in https://github.com/costimuraru/parquet-mr/pull/2


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415756#comment-16415756
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on a change in pull request #411: PARQUET-968 Add 
Hive/Presto support in ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#discussion_r177454394
 
 

 ##
 File path: 
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoWriteSupport.java
 ##
 @@ -247,21 +282,45 @@ final void writeRawValue(Object value) {
 @Override
 final void writeField(Object value) {
   recordConsumer.startField(fieldName, index);
+  recordConsumer.startGroup();
   List list = (List) value;
 
+  recordConsumer.startField("list", 0); // This is the wrapper group for 
the array field
   for (Object listEntry: list) {
+recordConsumer.startGroup();
+
+recordConsumer.startField("element", 0); // This is the mandatory 
inner field
+
+if (!isPrimitive(listEntry)) {
+  recordConsumer.startGroup();
+}
+
 fieldWriter.writeRawValue(listEntry);
+
+if (!isPrimitive(listEntry)) {
+  recordConsumer.endGroup();
+}
+
+recordConsumer.endField("element", 0);
+
+recordConsumer.endGroup();
   }
+  recordConsumer.endField("list", 0);
 
+  recordConsumer.endGroup();
   recordConsumer.endField(fieldName, index);
 }
   }
 
+  private boolean isPrimitive(Object listEntry) {
+return !(listEntry instanceof Message);
 
 Review comment:
   done in https://github.com/costimuraru/parquet-mr/pull/2, I removed the 
isPrimitive method as wrapping with a group can be done in 
`MessageWriter.writeRawValue()`, removing the need to handle non-primitive 
types differently inside the ArrayWriter.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415751#comment-16415751
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on a change in pull request #411: PARQUET-968 Add 
Hive/Presto support in ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#discussion_r177453689
 
 

 ##
 File path: 
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoMessageConverter.java
 ##
 @@ -345,4 +351,121 @@ public void addBinary(Binary binary) {
 }
 
   }
+
+  /**
+   * This class unwraps the additional LIST wrapper and makes it possible to 
read the underlying data and then convert
+   * it to protobuf.
+   * 
+   * Consider the following protobuf schema:
+   * message SimpleList {
+   *   repeated int64 first_array = 1;
+   * }
+   * 
+   * A LIST wrapper is created in parquet for the above mentioned protobuf 
schema:
+   * message SimpleList {
+   *   required group first_array (LIST) = 1 {
+   * repeated int32 element;
+   *   }
+   * }
+   * 
+   * The LIST wrappers are used by 3rd party tools, such as Hive, to read 
parquet arrays. The wrapper contains
+   * one only one field: either a primitive field (like in the example above, 
where we have an array of ints) or
+   * another group (array of messages).
+   */
+  final class ListConverter extends GroupConverter {
+private final Converter converter;
+private final boolean listOfMessage;
+
+public ListConverter(Message.Builder parentBuilder, 
Descriptors.FieldDescriptor fieldDescriptor, Type parquetType) {
+  OriginalType originalType = parquetType.getOriginalType();
+  if (originalType != OriginalType.LIST) {
+throw new ParquetDecodingException("Expected LIST wrapper. Found: " + 
originalType + " instead.");
+  }
+
+  listOfMessage = fieldDescriptor.getJavaType() == JavaType.MESSAGE;
+
+  Type parquetSchema;
+  if (parquetType.asGroupType().containsField("list")) {
 
 Review comment:
   done in https://github.com/costimuraru/parquet-mr/pull/2


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415749#comment-16415749
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on a change in pull request #411: PARQUET-968 Add 
Hive/Presto support in ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#discussion_r177453607
 
 

 ##
 File path: 
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoMessageConverter.java
 ##
 @@ -129,10 +131,14 @@ public void add(Object value) {
   };
 }
 
-return newScalarConverter(parent, parentBuilder, fieldDescriptor, 
parquetType);
+OriginalType originalType = parquetType.getOriginalType() == null ? 
OriginalType.UTF8 : parquetType.getOriginalType();
 
 Review comment:
   I believe the reason for this is that if originalType is null, the swicth 
will throw an exception. 
   In https://github.com/costimuraru/parquet-mr/pull/2 I replaced the switch 
with an if.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415744#comment-16415744
 ] 

ASF GitHub Bot commented on PARQUET-968:


BenoitHanotte commented on issue #411: PARQUET-968 Add Hive/Presto support in 
ProtoParquet
URL: https://github.com/apache/parquet-mr/pull/411#issuecomment-376552693
 
 
   @costimuraru @lukasnalezenec @qinghui-xu I have updated my PR 
(https://github.com/costimuraru/parquet-mr/pull/2) on top of this one with a 
comment addressing the code review feedback. Could you please have a look?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Add Hive/Presto support in ProtoParquet
> ---
>
> Key: PARQUET-968
> URL: https://issues.apache.org/jira/browse/PARQUET-968
> Project: Parquet
>  Issue Type: Task
>Reporter: Constantin Muraru
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1021) [C++] Print more helpful failure message when PARQUET_TEST_DATA environment variable is not set

2018-03-27 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415660#comment-16415660
 ] 

Antoine Pitrou commented on PARQUET-1021:
-

This is the same issue as PARQUET-1255.

> [C++] Print more helpful failure message when PARQUET_TEST_DATA environment 
> variable is not set
> ---
>
> Key: PARQUET-1021
> URL: https://issues.apache.org/jira/browse/PARQUET-1021
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-cpp
>Affects Versions: cpp-1.1.0
>Reporter: Wes McKinney
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1071) [C++] parquet::arrow::FileWriter::Close is not idempotent

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415629#comment-16415629
 ] 

ASF GitHub Bot commented on PARQUET-1071:
-

pitrou opened a new pull request #449: PARQUET-1071: Check that 
arrow::FileWriter::Close() is idempotent
URL: https://github.com/apache/parquet-cpp/pull/449
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [C++] parquet::arrow::FileWriter::Close is not idempotent
> -
>
> Key: PARQUET-1071
> URL: https://issues.apache.org/jira/browse/PARQUET-1071
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-cpp
>Affects Versions: cpp-1.2.0
>Reporter: Wes McKinney
>Priority: Major
>
> Encountered a segfault when calling multiple times from Python



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1071) [C++] parquet::arrow::FileWriter::Close is not idempotent

2018-03-27 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415627#comment-16415627
 ] 

Antoine Pitrou commented on PARQUET-1071:
-

Adding an idempotence test doesn't seem to break things here.

> [C++] parquet::arrow::FileWriter::Close is not idempotent
> -
>
> Key: PARQUET-1071
> URL: https://issues.apache.org/jira/browse/PARQUET-1071
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-cpp
>Affects Versions: cpp-1.2.0
>Reporter: Wes McKinney
>Priority: Major
>
> Encountered a segfault when calling multiple times from Python



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

2018-03-27 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415615#comment-16415615
 ] 

ASF GitHub Bot commented on PARQUET-1255:
-

pitrou opened a new pull request #448: PARQUET-1255: Fix error message when 
PARQUET_TEST_DATA isn't defined
URL: https://github.com/apache/parquet-cpp/pull/448
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [C++] Exceptions thrown in some tests
> -
>
> Key: PARQUET-1255
> URL: https://issues.apache.org/jira/browse/PARQUET-1255
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-cpp
>Reporter: Antoine Pitrou
>Priority: Major
>
> Some tests (not all) throw a basic_string exception. Example:
> {code}
> $ ./debug/reader-test 
> Running main() from gtest_main.cc
> [==] Running 11 tests from 4 test cases.
> [--] Global test environment set-up.
> [--] 7 tests from TestAllTypesPlain
> [ RUN  ] TestAllTypesPlain.NoopConstructDestruct
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct (0 ms)
> [ RUN  ] TestAllTypesPlain.TestBatchRead
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestBatchRead (0 ms)
> [ RUN  ] TestAllTypesPlain.TestFlatScannerInt32
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32 (0 ms)
> [ RUN  ] TestAllTypesPlain.TestSetScannerBatchSize
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize (0 ms)
> [ RUN  ] TestAllTypesPlain.DebugPrintWorks
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelection
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelection (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange (0 ms)
> [--] 7 tests from TestAllTypesPlain (0 ms total)
> [--] 2 tests from TestLocalFile
> [ RUN  ] TestLocalFile.FileClosedOnDestruction
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction (0 ms)
> [ RUN  ] TestLocalFile.OpenWithMetadata
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.OpenWithMetadata (0 ms)
> [--] 2 tests from TestLocalFile (0 ms total)
> [--] 1 test from TestFileReaderAdHoc
> [ RUN  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage (1 ms)
> [--] 1 test from TestFileReaderAdHoc (1 ms total)
> [--] 1 test from TestJSONWithLocalFile
> [ RUN  ] TestJSONWithLocalFile.JSONOutput
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestJSONWithLocalFile.JSONOutput (0 ms)
> [--] 1 test from TestJSONWithLocalFile (0 ms total)
> [--] Global test environment tear-down
> [==] 11 tests from 4 test cases ran. (1 ms total)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 11 tests, listed below:
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct
> [  FAILED  ] TestAllTypesPlain.TestBatchRead
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks
> [  FAILED  ] TestAllTypesPlain.ColumnSelection
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction
> [  FAILED  ] TestLocalFile.OpenWithMetadata
> [  FAILED  ]

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

2018-03-27 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415588#comment-16415588
 ] 

Antoine Pitrou commented on PARQUET-1255:
-

Hmm, yes, of course, it works once I set PARQUET_TEST_DATA... :-/

 

> [C++] Exceptions thrown in some tests
> -
>
> Key: PARQUET-1255
> URL: https://issues.apache.org/jira/browse/PARQUET-1255
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-cpp
>Reporter: Antoine Pitrou
>Priority: Major
>
> Some tests (not all) throw a basic_string exception. Example:
> {code}
> $ ./debug/reader-test 
> Running main() from gtest_main.cc
> [==] Running 11 tests from 4 test cases.
> [--] Global test environment set-up.
> [--] 7 tests from TestAllTypesPlain
> [ RUN  ] TestAllTypesPlain.NoopConstructDestruct
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct (0 ms)
> [ RUN  ] TestAllTypesPlain.TestBatchRead
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestBatchRead (0 ms)
> [ RUN  ] TestAllTypesPlain.TestFlatScannerInt32
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32 (0 ms)
> [ RUN  ] TestAllTypesPlain.TestSetScannerBatchSize
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize (0 ms)
> [ RUN  ] TestAllTypesPlain.DebugPrintWorks
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelection
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelection (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange (0 ms)
> [--] 7 tests from TestAllTypesPlain (0 ms total)
> [--] 2 tests from TestLocalFile
> [ RUN  ] TestLocalFile.FileClosedOnDestruction
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction (0 ms)
> [ RUN  ] TestLocalFile.OpenWithMetadata
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.OpenWithMetadata (0 ms)
> [--] 2 tests from TestLocalFile (0 ms total)
> [--] 1 test from TestFileReaderAdHoc
> [ RUN  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage (1 ms)
> [--] 1 test from TestFileReaderAdHoc (1 ms total)
> [--] 1 test from TestJSONWithLocalFile
> [ RUN  ] TestJSONWithLocalFile.JSONOutput
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestJSONWithLocalFile.JSONOutput (0 ms)
> [--] 1 test from TestJSONWithLocalFile (0 ms total)
> [--] Global test environment tear-down
> [==] 11 tests from 4 test cases ran. (1 ms total)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 11 tests, listed below:
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct
> [  FAILED  ] TestAllTypesPlain.TestBatchRead
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks
> [  FAILED  ] TestAllTypesPlain.ColumnSelection
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction
> [  FAILED  ] TestLocalFile.OpenWithMetadata
> [  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
> [  FAILED  ] TestJSONWithLocalFile.JSONOutput
> 11 FAILED TESTS
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

2018-03-27 Thread Uwe L. Korn (JIRA)


[ 
https://issues.apache.org/jira/browse/PARQUET-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415585#comment-16415585
 ] 

Uwe L. Korn commented on PARQUET-1255:
--

This is the magic way of telling you to set {{PARQUET_TEST_DATA}} to the 
correct path to the {{data/}} folder than contains the reference files.

> [C++] Exceptions thrown in some tests
> -
>
> Key: PARQUET-1255
> URL: https://issues.apache.org/jira/browse/PARQUET-1255
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-cpp
>Reporter: Antoine Pitrou
>Priority: Major
>
> Some tests (not all) throw a basic_string exception. Example:
> {code}
> $ ./debug/reader-test 
> Running main() from gtest_main.cc
> [==] Running 11 tests from 4 test cases.
> [--] Global test environment set-up.
> [--] 7 tests from TestAllTypesPlain
> [ RUN  ] TestAllTypesPlain.NoopConstructDestruct
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct (0 ms)
> [ RUN  ] TestAllTypesPlain.TestBatchRead
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestBatchRead (0 ms)
> [ RUN  ] TestAllTypesPlain.TestFlatScannerInt32
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32 (0 ms)
> [ RUN  ] TestAllTypesPlain.TestSetScannerBatchSize
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize (0 ms)
> [ RUN  ] TestAllTypesPlain.DebugPrintWorks
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelection
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelection (0 ms)
> [ RUN  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange (0 ms)
> [--] 7 tests from TestAllTypesPlain (0 ms total)
> [--] 2 tests from TestLocalFile
> [ RUN  ] TestLocalFile.FileClosedOnDestruction
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction (0 ms)
> [ RUN  ] TestLocalFile.OpenWithMetadata
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in SetUp().
> [  FAILED  ] TestLocalFile.OpenWithMetadata (0 ms)
> [--] 2 tests from TestLocalFile (0 ms total)
> [--] 1 test from TestFileReaderAdHoc
> [ RUN  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage (1 ms)
> [--] 1 test from TestFileReaderAdHoc (1 ms total)
> [--] 1 test from TestJSONWithLocalFile
> [ RUN  ] TestJSONWithLocalFile.JSONOutput
> unknown file: Failure
> C++ exception with description "basic_string::_S_construct null not valid" 
> thrown in the test body.
> [  FAILED  ] TestJSONWithLocalFile.JSONOutput (0 ms)
> [--] 1 test from TestJSONWithLocalFile (0 ms total)
> [--] Global test environment tear-down
> [==] 11 tests from 4 test cases ran. (1 ms total)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 11 tests, listed below:
> [  FAILED  ] TestAllTypesPlain.NoopConstructDestruct
> [  FAILED  ] TestAllTypesPlain.TestBatchRead
> [  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32
> [  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize
> [  FAILED  ] TestAllTypesPlain.DebugPrintWorks
> [  FAILED  ] TestAllTypesPlain.ColumnSelection
> [  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange
> [  FAILED  ] TestLocalFile.FileClosedOnDestruction
> [  FAILED  ] TestLocalFile.OpenWithMetadata
> [  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
> [  FAILED  ] TestJSONWithLocalFile.JSONOutput
> 11 FAILED TESTS
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (PARQUET-1255) [C++] Exceptions thrown in some tests

2018-03-27 Thread Antoine Pitrou (JIRA)

Antoine Pitrou created PARQUET-1255:
---

 Summary: [C++] Exceptions thrown in some tests
 Key: PARQUET-1255
 URL: https://issues.apache.org/jira/browse/PARQUET-1255
 Project: Parquet
  Issue Type: Bug
  Components: parquet-cpp
Reporter: Antoine Pitrou


Some tests (not all) throw a basic_string exception. Example:

{code}
$ ./debug/reader-test 
Running main() from gtest_main.cc
[==] Running 11 tests from 4 test cases.
[--] Global test environment set-up.
[--] 7 tests from TestAllTypesPlain
[ RUN  ] TestAllTypesPlain.NoopConstructDestruct
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.NoopConstructDestruct (0 ms)
[ RUN  ] TestAllTypesPlain.TestBatchRead
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.TestBatchRead (0 ms)
[ RUN  ] TestAllTypesPlain.TestFlatScannerInt32
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32 (0 ms)
[ RUN  ] TestAllTypesPlain.TestSetScannerBatchSize
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize (0 ms)
[ RUN  ] TestAllTypesPlain.DebugPrintWorks
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.DebugPrintWorks (0 ms)
[ RUN  ] TestAllTypesPlain.ColumnSelection
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.ColumnSelection (0 ms)
[ RUN  ] TestAllTypesPlain.ColumnSelectionOutOfRange
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange (0 ms)
[--] 7 tests from TestAllTypesPlain (0 ms total)

[--] 2 tests from TestLocalFile
[ RUN  ] TestLocalFile.FileClosedOnDestruction
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestLocalFile.FileClosedOnDestruction (0 ms)
[ RUN  ] TestLocalFile.OpenWithMetadata
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in SetUp().
[  FAILED  ] TestLocalFile.OpenWithMetadata (0 ms)
[--] 2 tests from TestLocalFile (0 ms total)

[--] 1 test from TestFileReaderAdHoc
[ RUN  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in the test body.
[  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage (1 ms)
[--] 1 test from TestFileReaderAdHoc (1 ms total)

[--] 1 test from TestJSONWithLocalFile
[ RUN  ] TestJSONWithLocalFile.JSONOutput
unknown file: Failure
C++ exception with description "basic_string::_S_construct null not valid" 
thrown in the test body.
[  FAILED  ] TestJSONWithLocalFile.JSONOutput (0 ms)
[--] 1 test from TestJSONWithLocalFile (0 ms total)

[--] Global test environment tear-down
[==] 11 tests from 4 test cases ran. (1 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] TestAllTypesPlain.NoopConstructDestruct
[  FAILED  ] TestAllTypesPlain.TestBatchRead
[  FAILED  ] TestAllTypesPlain.TestFlatScannerInt32
[  FAILED  ] TestAllTypesPlain.TestSetScannerBatchSize
[  FAILED  ] TestAllTypesPlain.DebugPrintWorks
[  FAILED  ] TestAllTypesPlain.ColumnSelection
[  FAILED  ] TestAllTypesPlain.ColumnSelectionOutOfRange
[  FAILED  ] TestLocalFile.FileClosedOnDestruction
[  FAILED  ] TestLocalFile.OpenWithMetadata
[  FAILED  ] TestFileReaderAdHoc.NationDictTruncatedDataPage
[  FAILED  ] TestJSONWithLocalFile.JSONOutput

11 FAILED TESTS
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PARQUET-1143) Update Java for format 2.4.0 changes

[jira] [Created] (PARQUET-1256) Please review and merge: added --print-key-value-metadata option to parquet_reader tool

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-968) Add Hive/Presto support in ProtoParquet

[jira] [Commented] (PARQUET-1021) [C++] Print more helpful failure message when PARQUET_TEST_DATA environment variable is not set

[jira] [Commented] (PARQUET-1071) [C++] parquet::arrow::FileWriter::Close is not idempotent

[jira] [Commented] (PARQUET-1071) [C++] parquet::arrow::FileWriter::Close is not idempotent

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

[jira] [Commented] (PARQUET-1255) [C++] Exceptions thrown in some tests

[jira] [Created] (PARQUET-1255) [C++] Exceptions thrown in some tests

15 matches

Site Navigation

Mail list logo

Footer information