[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-10-21 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-713498654 > I have some comments otherwise I am fine with this change. > > Meanwhile, the title and maybe the jira description do not fit this change. I think, they should be

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-12 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-672788803 > I don't agree we cannot have a meeting about this topic in terms of transparency. What we have to do is to document here about what we have discussed and what are the

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-12 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-672750035 @shangxinli Indeed, my understanding of this pull request is accurately described by "'how to transport the settings from the extended ParquetWriteSupport to

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-11 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-671793784 Yep, a couple of concerns related to encryption. The stated goal of the crypto factory design is to be "transparent to analytic frameworks, so they can leverage Parquet

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-10 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-671228812 @shangxinli thanks again for the additional details, it was interesting to have a glance at your usecase. It is indeed somewhat different from ours. Your existing

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-07 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-670350298 in the meantime, could you send an example of such extended ParquetWriteSupport? If CryptoHoodieAvroWriteSupport is the example, could you fix the link above, I get a

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-06 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-670157959 Thanks @shangxinli for the detailed response, much appreciated! I want to allocate ample time to get to the bottom of this subject, and with the weekend starting here, it

[GitHub] [parquet-mr] ggershinsky commented on pull request #808: Parquet-1396: Cryptodata Interface for Schema Activation of Parquet E…

2020-08-06 Thread GitBox
ggershinsky commented on pull request #808: URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-669725393 > extend ParquetWriteSupport which converts crypto setting in schema.. @shangxinli your previous pull request (already merged) added a very nice Encryption Factory