Hi All,
I am trying to write a list in parquet using below code, but something is going
wrong..
MessageType schema = MessageTypeParser.parseMessageType("message basket {
required group myList (LIST) { repeated group list { required float listfloat;}
} }");
ParquetWriter<Group> writer=new ParquetWriter<Group>(outDirPath,new
GroupWriteSupport() {
@Override
public WriteContext init(Configuration configuration) {
if (configuration.get(GroupWriteSupport.PARQUET_EXAMPLE_SCHEMA) == null) {
configuration.set(GroupWriteSupport.PARQUET_EXAMPLE_SCHEMA,
schema.toString());
}
return super.init(configuration);
}},CompressionCodecName.SNAPPY, 256*1024*1024, 100*1024);
GroupWriteSupport.setSchema(schema,config);
SimpleGroupFactory f=new SimpleGroupFactory(schema);
writer.write(f.newGroup().append("listfloat", ( float)2.8).append("listfloat",
3.3f));
Its not working....exception :
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration.deprecation).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Exception in thread "main" parquet.io.InvalidRecordException: listfloat not
found in message basket {
required group myList (LIST) {
repeated group list {
required float listfloat;
}
}
}
at parquet.schema.GroupType.getFieldIndex(GroupType.java:147)
at parquet.example.data.Group.add(Group.java:39)
at parquet.example.data.Group.append(Group.java:107)
at ParquetTestWriter.main(ParquetTestWriter.java:90)
Appreciate the response!!.
Manisha
________________________________