shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-714828022
I just pushed more comments and squashed into one. This is ready for review
now.
This is an automated
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-714205196
I did a push prematurely. That is not ready for review yet. I will rework on
it soon and let you know when it is ready for review.
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-713009156
@gszadovszky @ggershinsky Can you have another look? I just switched to us
Configuration.
This is an
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-672999165
Thanks @ggershinsky @gszadovszky for spending time on the discussion! If
both of you think adding 'metadata' field is not a good idea, I will just use
Configuration and see
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-672148979
@gszadovszky Thanks for the correction of PARQUET-1784! Regarding the
serialized/deserialized, it is not done. I was aware of that when I use
ExtType. But that is
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-672093542
@ggershinsky, from what we have discussed above I think now the different
opinions narrow to 'how to transport the settings from the extended
ParquetWriteSupport to
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-671465375
> But you have other options in the same WriteSupport init function. One of
them is to use the Hadoop configuration. It is not global anymore, but rather a
local copy,
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-670547099
> in the meantime, could you send an example of such extended
ParquetWriteSupport?
> If CryptoHoodieAvroWriteSupport is the example, could you fix the link
above, I get
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-670162874
Thanks Gidon! No problem. We have deployed completely, so no hurry for
reviewing. As long as it is merged to upstream, we have a path to go in the
future.
For
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-670094258
Hi @ggershinsky, Thanks for your comments!
I think the current difference is how to transport the crypto settings from
source to
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-669333155
The way it works is SchemaCryptoPropertiesFactory should have RPC call with
KMS to get the column key for keyId/metadata defined in metastore, and build up
encryption
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-668832445
I am fine with adding metadata to 'Type' directly. Hopefully, there are no
tools revoking it using reflection. Since this will be in a major release, it
should be fine.
shangxinli commented on pull request #808:
URL: https://github.com/apache/parquet-mr/pull/808#issuecomment-667344642
> I think, directly using `ExtType` is not really user friendly. As the
"metadata" for a type is currently used only for encryption I would not expose
it directly to the
13 matches
Mail list logo
