[
https://issues.apache.org/jira/browse/PARQUET-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gang Wu resolved PARQUET-2446.
------------------------------
Fix Version/s: 1.14.0
Assignee: fcyang
Resolution: Fixed
> ProtoParquetWriter Not Support DynamicMessage
> ---------------------------------------------
>
> Key: PARQUET-2446
> URL: https://issues.apache.org/jira/browse/PARQUET-2446
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-protobuf
> Affects Versions: 1.13.1
> Environment: Parquet-Protobuf 1.13.1
> Reporter: fcyang
> Assignee: fcyang
> Priority: Major
> Labels: Parquet, parquetWriter
> Fix For: 1.14.0
>
>
> when i write DynamicMessages to parquet by protoParquetWriter,
> {code:java}
> pbwriter = ProtoParquetWriter.<DynamicMessage>builder(s3ParquetOutputFile)
> .withMessage(DynamicMessage.class)
> .withConf(hadoopConf)
> .withWriteMode(ParquetFileWriter.Mode.OVERWRITE)
> .withDictionaryEncoding(true)
> .withCompressionCodec(storage.conf().parquetCompressionCodecName())
> .withPageSize(PAGE_SIZE)
> .build(); {code}
> I got an exception like this:
> {code:java}
> [ERROR] - com.twitter.elephantbird.util.Protobufs -Protobufs.java(126) -Could
> not find method getDescriptor in class
> com.google.protobuf.DynamicMessagecom.google.protobuf.DynamicMessage
> java.lang.NoSuchMethodException:
> com.google.protobuf.DynamicMessage.getDescriptor()
> at java.lang.Class.getMethod(Class.java:1786)
> at
> com.twitter.elephantbird.util.Protobufs.getMessageDescriptor(Protobufs.java:123)
> at
> org.apache.parquet.proto.ProtoWriteSupport.init(ProtoWriteSupport.java:136)
> at
> org.apache.parquet.hadoop.ParquetWriter.<init>(ParquetWriter.java:279)
> at
> org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:702)
> {code}
> I believe that ProtoParquetWriter should support DynamicMessage and be
> capable of creating ProtoWriteSupport using descriptors same with PBMessage
> class.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
