[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780620496 ## File path: c++/src/Vector.cc ## @@ -242,7 +242,6 @@ namespace orc { } return false; } - Review comment: Reverted. -- This is an

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780620362 ## File path: c++/src/Options.hh ## @@ -174,20 +175,36 @@ namespace orc { privateBits->selection = ColumnSelection_FIELD_IDS;

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780620218 ## File path: c++/src/ColumnReader.cc ## @@ -1006,9 +1009,12 @@ namespace orc { if (stream == nullptr) throw ParseError("LENGTH stream not found in

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780620049 ## File path: c++/src/Reader.cc ## @@ -290,6 +291,15 @@ namespace orc { return selectedColumns; } + ArrayReadIntent

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780619777 ## File path: c++/include/orc/Reader.hh ## @@ -565,6 +591,15 @@ namespace orc { */ virtual void seekToRow(uint64_t rowNumber) = 0; +/** + *

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780619681 ## File path: c++/include/orc/Reader.hh ## @@ -266,6 +286,12 @@ namespace orc { * Get desired timezone to return data of timestamp type */

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780619653 ## File path: c++/include/orc/Reader.hh ## @@ -149,6 +149,26 @@ namespace orc { */ RowReaderOptions& includeTypes(const std::list& types); +/**

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780619470 ## File path: c++/include/orc/Reader.hh ## @@ -149,6 +149,26 @@ namespace orc { */ RowReaderOptions& includeTypes(const std::list& types); +/**

[GitHub] [orc] rizaon commented on a change in pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on a change in pull request #990: URL: https://github.com/apache/orc/pull/990#discussion_r780619426 ## File path: c++/include/orc/Common.hh ## @@ -122,6 +122,11 @@ namespace orc { StreamKind_BLOOM_FILTER_UTF8 = 8 }; + enum ArrayReadIntent { Review

[GitHub] [orc] rizaon commented on pull request #990: ORC-450: Add option to avoid materialization of array elements

2022-01-07 Thread GitBox
rizaon commented on pull request #990: URL: https://github.com/apache/orc/pull/990#issuecomment-1007867103 I came to a realization that in order to avoid List materialization, it is sufficient to unselect its children Type in ColumnSelector. ListColumnReader will skip creating childReader

[GitHub] [orc] stiga-huang opened a new pull request #998: ORC-1081: Fix heap-use-after-free in SearchArgumentBuilderImpl::end()

2022-01-07 Thread GitBox
stiga-huang opened a new pull request #998: URL: https://github.com/apache/orc/pull/998 ### What changes were proposed in this pull request? This PR fixes the heap-use-after-free issue in `SearchArgumentBuilderImpl::end()`. `std::deque::pop_front` will delete the element.

[jira] [Created] (ORC-1081) heap-use-after-free in orc::SearchArgumentBuilderImpl::end()

2022-01-07 Thread Quanlong Huang (Jira)
Quanlong Huang created ORC-1081: --- Summary: heap-use-after-free in orc::SearchArgumentBuilderImpl::end() Key: ORC-1081 URL: https://issues.apache.org/jira/browse/ORC-1081 Project: ORC Issue

[GitHub] [orc] dongjoon-hyun commented on a change in pull request #996: ORC-1078: Row group end offset doesn't accommodate all the blocks

2022-01-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #996: URL: https://github.com/apache/orc/pull/996#discussion_r780506796 ## File path: java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java ## @@ -2449,4 +2449,23 @@ public void testSargApplier() throws Exception {

[GitHub] [orc] dongjoon-hyun commented on a change in pull request #996: ORC-1078: Row group end offset doesn't accommodate all the blocks

2022-01-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #996: URL: https://github.com/apache/orc/pull/996#discussion_r780506306 ## File path: java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java ## @@ -2449,4 +2449,23 @@ public void testSargApplier() throws Exception {

[GitHub] [orc] dongjoon-hyun commented on a change in pull request #996: ORC-1078: Row group end offset doesn't accommodate all the blocks

2022-01-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #996: URL: https://github.com/apache/orc/pull/996#discussion_r780504906 ## File path: java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java ## @@ -182,10 +182,13 @@ public static long estimateRgEndOffset(boolean

[GitHub] [orc] dongjoon-hyun commented on a change in pull request #996: ORC-1078: Row group end offset doesn't accommodate all the blocks

2022-01-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #996: URL: https://github.com/apache/orc/pull/996#discussion_r780503748 ## File path: java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java ## @@ -182,10 +182,13 @@ public static long estimateRgEndOffset(boolean

[GitHub] [orc] dongjoon-hyun commented on a change in pull request #996: ORC-1078: Row group end offset doesn't accommodate all the blocks

2022-01-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #996: URL: https://github.com/apache/orc/pull/996#discussion_r780502341 ## File path: java/core/src/test/org/apache/orc/impl/TestRecordReaderImpl.java ## @@ -1538,7 +1538,7 @@ public void testPartialPlanCompressed() throws

[GitHub] [orc] wgtmac commented on pull request #994: ORC-1004: Java ORC writer supports the selection vector

2022-01-07 Thread GitBox
wgtmac commented on pull request #994: URL: https://github.com/apache/orc/pull/994#issuecomment-1007228886 > > Thank you, @guiyanakuang . Now, this PR also fails. :) > > ``` > > Error: Failures: > > Error:TestSelectedVector.testWriteBaseTypeUseSelectedVector:209 expected: