[jira] [Created] (PARQUET-2183) Fix statistics issue of Column Encryptor

2022-09-02 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2183: Summary: Fix statistics issue of Column Encryptor Key: PARQUET-2183 URL: https://issues.apache.org/jira/browse/PARQUET-2183 Project: Parquet Issue Type:

[jira] [Commented] (PARQUET-1681) Avro's isElementType() change breaks the reading of some parquet(1.8.1) files

2022-04-08 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519686#comment-17519686 ] Xinli Shang commented on PARQUET-1681: -- [~theosib-amazon]It seems different. > Avro's

[jira] [Commented] (PARQUET-1595) Parquet proto writer de-nest Protobuf wrapper classes

2022-03-20 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509500#comment-17509500 ] Xinli Shang commented on PARQUET-1595: -- Is it a typo for Int32Value -> int64? > Parquet proto

[jira] [Updated] (PARQUET-2116) Cell Level Encryption

2022-03-12 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2116: - External issue URL:

[jira] [Comment Edited] (PARQUET-2127) Security risk in latest parquet-jackson-1.12.2.jar

2022-02-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494321#comment-17494321 ] Xinli Shang edited comment on PARQUET-2127 at 2/18/22, 2:23 AM: Thanks

[jira] [Commented] (PARQUET-2127) Security risk in latest parquet-jackson-1.12.2.jar

2022-02-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494321#comment-17494321 ] Xinli Shang commented on PARQUET-2127: -- Thanks for reporting [~phoebemaomao]! Will you be able to

[jira] [Comment Edited] (PARQUET-2122) Adding Bloom filter to small Parquet file bloats in size X1700

2022-02-14 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492099#comment-17492099 ] Xinli Shang edited comment on PARQUET-2122 at 2/14/22, 4:56 PM:

[jira] [Commented] (PARQUET-2122) Adding Bloom filter to small Parquet file bloats in size X1700

2022-02-14 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492099#comment-17492099 ] Xinli Shang commented on PARQUET-2122: -- [~junjie]Do you know why? > Adding Bloom filter to small

[jira] [Commented] (PARQUET-2117) Add rowPosition API in parquet record readers

2022-02-02 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485949#comment-17485949 ] Xinli Shang commented on PARQUET-2117: -- Thanks for opening this Jira! Look forward to the PR. >

[jira] [Updated] (PARQUET-2116) Cell Level Encryption

2022-01-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2116: - External issue URL:

[jira] [Created] (PARQUET-2116) Cell Level Encryption

2022-01-27 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2116: Summary: Cell Level Encryption Key: PARQUET-2116 URL: https://issues.apache.org/jira/browse/PARQUET-2116 Project: Parquet Issue Type: Improvement

[jira] [Resolved] (PARQUET-2091) Fix release build error introduced by PARQUET-2043

2022-01-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-2091. -- Resolution: Won't Fix > Fix release build error introduced by PARQUET-2043 >

[jira] [Commented] (PARQUET-2098) Add more methods into interface of BlockCipher

2022-01-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483225#comment-17483225 ] Xinli Shang commented on PARQUET-2098: -- [~gershinsky] Do you have time to work on it as we

[jira] [Resolved] (PARQUET-2112) Fix typo in MessageColumnIO

2022-01-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-2112. -- Resolution: Fixed > Fix typo in MessageColumnIO > --- > >

[jira] [Created] (PARQUET-2112) Fix typo in MessageColumnIO

2022-01-22 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2112: Summary: Fix typo in MessageColumnIO Key: PARQUET-2112 URL: https://issues.apache.org/jira/browse/PARQUET-2112 Project: Parquet Issue Type: Improvement

[jira] [Commented] (PARQUET-2111) Support limit push down and stop early for RecordReader

2022-01-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480128#comment-17480128 ] Xinli Shang commented on PARQUET-2111: -- Look forward to the PR > Support limit push down and stop

[jira] [Resolved] (PARQUET-2071) Encryption translation tool

2022-01-14 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-2071. -- Resolution: Fixed > Encryption translation tool > > >

[jira] [Resolved] (PARQUET-1872) Add TransCompression Feature

2022-01-14 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-1872. -- Resolution: Fixed > Add TransCompression Feature > - > >

[jira] [Resolved] (PARQUET-2105) Refactor the test code of creating the test file

2022-01-14 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-2105. -- Resolution: Fixed > Refactor the test code of creating the test file >

[jira] [Commented] (PARQUET-1889) Register a MIME type for the Parquet format.

2022-01-11 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17473147#comment-17473147 ] Xinli Shang commented on PARQUET-1889: -- +1 on [~westonpace]'s point > Register a MIME type for

[jira] [Commented] (PARQUET-1911) Add way to disables statistics on a per column basis

2022-01-04 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468759#comment-17468759 ] Xinli Shang commented on PARQUET-1911: -- [~panthony] Thanks for working on this! Just FYI that

[jira] [Resolved] (PARQUET-1874) Add to parquet-cli

2021-12-03 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-1874. -- Resolution: Fixed > Add to parquet-cli > -- > > Key:

[jira] [Resolved] (PARQUET-1873) Add to Parquet-tools

2021-12-03 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-1873. -- Resolution: Fixed > Add to Parquet-tools > - > > Key:

[jira] [Updated] (PARQUET-1396) EncryptionPropertiesFactory and DecryptionPropertiesFactory

2021-12-03 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1396: - Summary: EncryptionPropertiesFactory and DecryptionPropertiesFactory (was: Example of using

[jira] [Updated] (PARQUET-1872) Add TransCompression Feature

2021-12-03 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1872: - Summary: Add TransCompression Feature (was: Add TransCompression command ) > Add

[jira] [Created] (PARQUET-2105) Refactor the test code of creating the test file

2021-11-30 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2105: Summary: Refactor the test code of creating the test file Key: PARQUET-2105 URL: https://issues.apache.org/jira/browse/PARQUET-2105 Project: Parquet Issue

[jira] [Created] (PARQUET-2098) Add more methods into interface of BlockCipher

2021-09-29 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2098: Summary: Add more methods into interface of BlockCipher Key: PARQUET-2098 URL: https://issues.apache.org/jira/browse/PARQUET-2098 Project: Parquet Issue

[jira] [Closed] (PARQUET-2027) Merging parquet files created in 1.11.1 not possible using 1.12.0

2021-09-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang closed PARQUET-2027. > Merging parquet files created in 1.11.1 not possible using 1.12.0 >

[jira] [Closed] (PARQUET-2078) Failed to read parquet file after writing with the same parquet version

2021-09-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang closed PARQUET-2078. > Failed to read parquet file after writing with the same parquet version >

[jira] [Created] (PARQUET-2093) Add rewriter version to Parquet footer

2021-09-20 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2093: Summary: Add rewriter version to Parquet footer Key: PARQUET-2093 URL: https://issues.apache.org/jira/browse/PARQUET-2093 Project: Parquet Issue Type:

[jira] [Updated] (PARQUET-2075) Unified Rewriter Tool

2021-09-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2075: - External issue URL:

[jira] [Updated] (PARQUET-2075) Unified Rewriter Tool

2021-09-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2075: - Summary: Unified Rewriter Tool(was: Unified translation tool ) > Unified Rewriter Tool

[jira] [Resolved] (PARQUET-2087) Release parquet-mr 1.12.1

2021-09-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-2087. -- Resolution: Fixed > Release parquet-mr 1.12.1 > - > >

[jira] [Commented] (PARQUET-2091) Fix release build error introduced by PARQUET-2043

2021-09-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416806#comment-17416806 ] Xinli Shang commented on PARQUET-2091: -- No issues on build but when run the release command, it

[jira] [Created] (PARQUET-2091) Fix release build error introduced by PARQUET-2043

2021-09-13 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2091: Summary: Fix release build error introduced by PARQUET-2043 Key: PARQUET-2091 URL: https://issues.apache.org/jira/browse/PARQUET-2091 Project: Parquet Issue

[jira] [Created] (PARQUET-2087) Release parquet-mr 1.12.0

2021-09-09 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2087: Summary: Release parquet-mr 1.12.0 Key: PARQUET-2087 URL: https://issues.apache.org/jira/browse/PARQUET-2087 Project: Parquet Issue Type: Task

[jira] [Assigned] (PARQUET-2087) Release parquet-mr 1.12.1

2021-09-09 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang reassigned PARQUET-2087: Assignee: Xinli Shang Due Date: 18/Sep/21 > Release parquet-mr 1.12.1 >

[jira] [Updated] (PARQUET-2087) Release parquet-mr 1.12.1

2021-09-09 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2087: - Summary: Release parquet-mr 1.12.1 (was: Release parquet-mr 1.12.0) > Release parquet-mr

[jira] [Created] (PARQUET-2082) Encryption translation tool - Parquet-cli

2021-08-30 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2082: Summary: Encryption translation tool - Parquet-cli Key: PARQUET-2082 URL: https://issues.apache.org/jira/browse/PARQUET-2082 Project: Parquet Issue Type:

[jira] [Created] (PARQUET-2081) Encryption translation tool - Parquet-hadoop

2021-08-30 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2081: Summary: Encryption translation tool - Parquet-hadoop Key: PARQUET-2081 URL: https://issues.apache.org/jira/browse/PARQUET-2081 Project: Parquet Issue Type:

[jira] [Comment Edited] (PARQUET-2071) Encryption translation tool

2021-08-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402670#comment-17402670 ] Xinli Shang edited comment on PARQUET-2071 at 8/21/21, 5:40 PM: I just

[jira] [Commented] (PARQUET-2071) Encryption translation tool

2021-08-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402670#comment-17402670 ] Xinli Shang commented on PARQUET-2071: -- I just drafted the tool and had [~gershinsky] to have an

[jira] [Updated] (PARQUET-2071) Encryption translation tool

2021-08-05 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2071: - External issue ID: https://issues.apache.org/jira/browse/PARQUET-2075 > Encryption translation

[jira] [Updated] (PARQUET-2075) Unified translation tool

2021-08-05 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2075: - External issue ID: https://issues.apache.org/jira/browse/PARQUET-2071 > Unified translation

[jira] [Commented] (PARQUET-2071) Encryption translation tool

2021-08-05 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394098#comment-17394098 ] Xinli Shang commented on PARQUET-2071: -- Thanks, Gabor and Gidon! I think it is a good idea of

[jira] [Created] (PARQUET-2075) Unified translation tool

2021-08-05 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2075: Summary: Unified translation tool Key: PARQUET-2075 URL: https://issues.apache.org/jira/browse/PARQUET-2075 Project: Parquet Issue Type: New Feature

[jira] [Updated] (PARQUET-2071) Encryption translation tool

2021-08-04 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-2071: - External issue URL:

[jira] [Created] (PARQUET-2071) Encryption translation tool

2021-08-04 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2071: Summary: Encryption translation tool Key: PARQUET-2071 URL: https://issues.apache.org/jira/browse/PARQUET-2071 Project: Parquet Issue Type: New Feature

[jira] [Commented] (PARQUET-2064) Make Range public accessible in RowRanges

2021-07-12 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379217#comment-17379217 ] Xinli Shang commented on PARQUET-2064: -- [~gszadovszky], do you have some suggestions on how to

[jira] [Created] (PARQUET-2064) Make Range public accessible in RowRanges

2021-07-09 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2064: Summary: Make Range public accessible in RowRanges Key: PARQUET-2064 URL: https://issues.apache.org/jira/browse/PARQUET-2064 Project: Parquet Issue Type:

[jira] [Commented] (PARQUET-2062) Data masking(null) for column encryption

2021-07-05 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17374892#comment-17374892 ] Xinli Shang commented on PARQUET-2062: -- Great idea! On Mon, Jul 5, 2021 at 1:03 AM Gabor

[jira] [Updated] (PARQUET-1792) Add 'mask' command to parquet-tools/parquet-cli

2021-07-01 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1792: - Fix Version/s: 1.12.0 > Add 'mask' command to parquet-tools/parquet-cli >

[jira] [Commented] (PARQUET-1681) Avro's isElementType() change breaks the reading of some parquet(1.8.1) files

2021-07-01 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372862#comment-17372862 ] Xinli Shang commented on PARQUET-1681: -- We chose to revert the behavior back to 1.8.1. It runs

[jira] [Created] (PARQUET-2062) Data masking(null) for column encryption

2021-06-30 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2062: Summary: Data masking(null) for column encryption Key: PARQUET-2062 URL: https://issues.apache.org/jira/browse/PARQUET-2062 Project: Parquet Issue Type:

[jira] [Created] (PARQUET-2054) TCP connection leaking when calling appendFile()

2021-06-01 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2054: Summary: TCP connection leaking when calling appendFile() Key: PARQUET-2054 URL: https://issues.apache.org/jira/browse/PARQUET-2054 Project: Parquet Issue

[jira] [Commented] (PARQUET-1968) FilterApi support In predicate

2021-05-25 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351199#comment-17351199 ] Xinli Shang commented on PARQUET-1968: -- Go ahead to work on it. Thanks Huaxin! > FilterApi

[jira] [Commented] (PARQUET-1827) UUID type currently not supported by parquet-mr

2021-04-01 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313524#comment-17313524 ] Xinli Shang commented on PARQUET-1827: -- It seems the storage size is reduced by ~8% for the UUID

[jira] [Created] (PARQUET-2006) Column resolution by ID

2021-03-23 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-2006: Summary: Column resolution by ID Key: PARQUET-2006 URL: https://issues.apache.org/jira/browse/PARQUET-2006 Project: Parquet Issue Type: New Feature

[jira] [Commented] (PARQUET-1992) Cannot build from tarball because of git submodules

2021-03-05 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296124#comment-17296124 ] Xinli Shang commented on PARQUET-1992: -- I think we shouldn't let it fail when developers run 'mvn

[jira] [Commented] (PARQUET-1948) TransCompressionCommand Inoperable

2021-02-18 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286825#comment-17286825 ] Xinli Shang commented on PARQUET-1948: -- [~vanhooser], glad to see you have the interests of this

[jira] [Commented] (PARQUET-1968) FilterApi support In predicate

2021-02-01 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276664#comment-17276664 ] Xinli Shang commented on PARQUET-1968: -- Sure, will connect with you shortly. > FilterApi support

[jira] [Commented] (PARQUET-1968) FilterApi support In predicate

2021-02-01 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276533#comment-17276533 ] Xinli Shang commented on PARQUET-1968: -- Hi [~rdblue]. We didn't discuss it in last week's Parquet

[jira] [Updated] (PARQUET-1949) Mark Parquet-1872 with not support bloom filter yet

2021-01-10 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1949: - Summary: Mark Parquet-1872 with not support bloom filter yet (was: Mark Parquet-1872 with

[jira] [Commented] (PARQUET-1872) Add TransCompression command

2020-12-04 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244334#comment-17244334 ] Xinli Shang commented on PARQUET-1872: -- Thanks [~gszadovszky] for working on this! I just created

[jira] [Created] (PARQUET-1949) Mark Parquet-1872 with note support bloom filter yet

2020-12-04 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1949: Summary: Mark Parquet-1872 with note support bloom filter yet Key: PARQUET-1949 URL: https://issues.apache.org/jira/browse/PARQUET-1949 Project: Parquet

[jira] [Commented] (PARQUET-1901) Add filter null check for ColumnIndex

2020-12-02 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242634#comment-17242634 ] Xinli Shang commented on PARQUET-1901: -- For now, I think we can move it to the next release. >

[jira] [Comment Edited] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-12-02 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242631#comment-17242631 ] Xinli Shang edited comment on PARQUET-1927 at 12/2/20, 7:05 PM: It is

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-12-02 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242631#comment-17242631 ] Xinli Shang commented on PARQUET-1927: -- It is still not decided yet in the last Iceberg meeting.

[jira] [Commented] (PARQUET-1666) Remove Unused Modules

2020-12-02 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242625#comment-17242625 ] Xinli Shang commented on PARQUET-1666: -- I think adding "-deprecated" is a good idea.

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-11-04 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226089#comment-17226089 ] Xinli Shang commented on PARQUET-1927: -- [~gszadovszky], I just realized the RowGroupFilter only

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221481#comment-17221481 ] Xinli Shang commented on PARQUET-1927: -- Thanks [~gszadovszky] for the explanation. I see it now.

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-26 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221036#comment-17221036 ] Xinli Shang commented on PARQUET-1927: -- ParquetFileReader.getFilteredRecordCount() cannot be used

[jira] [Assigned] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-23 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang reassigned PARQUET-1927: Assignee: Xinli Shang > ColumnIndex should provide number of records skipped >

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-22 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219293#comment-17219293 ] Xinli Shang commented on PARQUET-1927: -- [~gszadovszky], the problem is when rowCount is 0(line 966

[jira] [Commented] (PARQUET-1396) Example of using EncryptionPropertiesFactory and DecryptionPropertiesFactory

2020-10-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218717#comment-17218717 ] Xinli Shang commented on PARQUET-1396: -- Most of the functionality of this Jira has been addressed

[jira] [Updated] (PARQUET-1396) Example of using EncryptionPropertiesFactory and DecryptionPropertiesFactory

2020-10-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1396: - Summary: Example of using EncryptionPropertiesFactory and DecryptionPropertiesFactory (was:

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218325#comment-17218325 ] Xinli Shang commented on PARQUET-1927: -- The workaround I can think of is to apply ColumnIndex to

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-20 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217827#comment-17217827 ] Xinli Shang commented on PARQUET-1927: -- Add [~rdblue],[~shardulm] as FYI** > ColumnIndex should

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-20 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217774#comment-17217774 ] Xinli Shang commented on PARQUET-1927: -- That is correct [~gszadovszky]! We need a finer-grained

[jira] [Commented] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-19 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216849#comment-17216849 ] Xinli Shang commented on PARQUET-1927: -- [~gszadovszky], the way that Iceberg Parquet reader

[jira] [Created] (PARQUET-1927) ColumnIndex should provide number of records skipped

2020-10-17 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1927: Summary: ColumnIndex should provide number of records skipped Key: PARQUET-1927 URL: https://issues.apache.org/jira/browse/PARQUET-1927 Project: Parquet

[jira] [Created] (PARQUET-1916) Add hash functionality

2020-09-23 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1916: Summary: Add hash functionality Key: PARQUET-1916 URL: https://issues.apache.org/jira/browse/PARQUET-1916 Project: Parquet Issue Type: Sub-task

[jira] [Assigned] (PARQUET-1915) Add null command

2020-09-23 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang reassigned PARQUET-1915: Assignee: Xinli Shang > Add null command > - > > Key:

[jira] [Created] (PARQUET-1915) Add null command

2020-09-23 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1915: Summary: Add null command Key: PARQUET-1915 URL: https://issues.apache.org/jira/browse/PARQUET-1915 Project: Parquet Issue Type: Sub-task

[jira] [Comment Edited] (PARQUET-1901) Add filter null check for ColumnIndex

2020-08-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186211#comment-17186211 ] Xinli Shang edited comment on PARQUET-1901 at 8/28/20, 2:23 AM: I have

[jira] [Commented] (PARQUET-1901) Add filter null check for ColumnIndex

2020-08-27 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186211#comment-17186211 ] Xinli Shang commented on PARQUET-1901: -- I have the initial version of Iceberg integration working

[jira] [Commented] (PARQUET-1901) Add filter null check for ColumnIndex

2020-08-24 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183352#comment-17183352 ] Xinli Shang commented on PARQUET-1901: -- Hi [~rdblue], please comment on this if you have different

[jira] [Created] (PARQUET-1901) Add filter null check for ColumnIndex

2020-08-22 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1901: Summary: Add filter null check for ColumnIndex Key: PARQUET-1901 URL: https://issues.apache.org/jira/browse/PARQUET-1901 Project: Parquet Issue Type: Bug

[jira] [Commented] (PARQUET-1801) Add column index support for 'prune' command in Parquet-tools/cli

2020-08-13 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176961#comment-17176961 ] Xinli Shang commented on PARQUET-1801: -- I will try to do it in 1.12.0. The feature works great!

[jira] [Commented] (PARQUET-1792) Add 'mask' command to parquet-tools/parquet-cli

2020-08-13 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176959#comment-17176959 ] Xinli Shang commented on PARQUET-1792: -- We might want to push it for next release. > Add 'mask'

[jira] [Created] (PARQUET-1893) H2SeekableInputStream readFully() doesn't respect start and len

2020-07-29 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1893: Summary: H2SeekableInputStream readFully() doesn't respect start and len Key: PARQUET-1893 URL: https://issues.apache.org/jira/browse/PARQUET-1893 Project: Parquet

[jira] [Commented] (PARQUET-1830) Vectorized API to support Column Index in Apache Spark

2020-07-20 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161573#comment-17161573 ] Xinli Shang commented on PARQUET-1830: -- [~FelixKJose]Do we have Spark task created for

[jira] [Commented] (PARQUET-1739) Make Spark SQL support Column indexes

2020-07-20 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161566#comment-17161566 ] Xinli Shang commented on PARQUET-1739: -- [~yumwang], Can you share is the implementation is done in

[jira] [Commented] (PARQUET-1883) int96 support in parquet-avro

2020-07-09 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154888#comment-17154888 ] Xinli Shang commented on PARQUET-1883: -- [~gszadovszky], Do you still have links for INT96 will be

[jira] [Commented] (PARQUET-1872) Add TransCompression command

2020-06-17 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17138607#comment-17138607 ] Xinli Shang commented on PARQUET-1872: -- That is correct understanding [~gszadovszky]. > Add

[jira] [Commented] (PARQUET-1872) Add TransCompression command

2020-06-16 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17137967#comment-17137967 ] Xinli Shang commented on PARQUET-1872: -- [~gszadovszky]Thanks for the reply! I just manually linked

[jira] [Assigned] (PARQUET-1874) Add to parquet-cli

2020-06-16 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang reassigned PARQUET-1874: Assignee: Xinli Shang > Add to parquet-cli > -- > >

[jira] [Created] (PARQUET-1876) Port ZSTD-JNI support to 1.10.x brach

2020-06-14 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1876: Summary: Port ZSTD-JNI support to 1.10.x brach Key: PARQUET-1876 URL: https://issues.apache.org/jira/browse/PARQUET-1876 Project: Parquet Issue Type: Bug

[jira] [Updated] (PARQUET-1872) Add TransCompression command

2020-06-12 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1872: - Description: When ZSTD becomes more popular, there is a need to translate existing data to

[jira] [Created] (PARQUET-1875) Add bloom filter support

2020-06-11 Thread Xinli Shang (Jira)
Xinli Shang created PARQUET-1875: Summary: Add bloom filter support Key: PARQUET-1875 URL: https://issues.apache.org/jira/browse/PARQUET-1875 Project: Parquet Issue Type: Sub-task

  1   2   >