Hi all,
I'm running into a problem with BinaryEncoder.writeBytes. I'm trying to
wrap some documents in an Avro structure and then write them into an Avro
contain file.
My code looks roughly like this :
File file = new File(...);
ClassLoader cl = SimpleAvroExample.class.
getClassLoader();
InputStream isSimpleWrapperSchema=
cl.getResourceAsStream("simple_wrapper.avpr");
Schema.Parser parser = new Schema.Parser();
Schema schemaSimpleWrapper = parser.parse(isSimpleWrapperSchema);
GenericDatumWriter<GenericRecord> gdw = new
GenericDatumWriter<GenericRecord>(schemaSimpleWrapper);
DataFileWriter writer = new DataFileWriter(gdw);
File outputFile = new File(...);
outputFile.createNewFile();
writer.create(schemaSimpleWrapper, outputFile);
for(File documentFile : file.listFiles())
{
GenericRecord r = new GenericData.Record(schemaSimpleWrapper);
FileInputStream fis = new FileInputStream(documentFile);
FileChannel channel = fis.getChannel();
r.put("filename", documentFile.getName());
//Copying the whole file.
ByteBuffer byteBuffer =
channel.map(FileChannel.MapMode.READ_ONLY, 0, documentFile.length());
r.put("body", byteBuffer);
writer.append(r);
channel.close();
fis.close();
}
This code exceptions out :
Exception in thread "main"
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.UnsupportedOperationException
at
org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:261)
at org.iqt.cdl.SimpleAvroExample.main(SimpleAvroExample.java:107)
Caused by: java.lang.UnsupportedOperationException
at java.nio.ByteBuffer.arrayOffset(ByteBuffer.java:968)
at org.apache.avro.io.BinaryEncoder.writeBytes(BinaryEncoder.java:61)
at
org.apache.avro.generic.GenericDatumWriter.writeBytes(GenericDatumWriter.java:199)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:76)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
at
org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:255)
... 1 more
The problem seems to be that the writeBytes function on line 61 does the
following :
int start = bytes.arrayOffset() + pos.
Where byes is a java.nio.ByteBuffer.
According to the JavaDocs for ByteBuffer the arrayOffset fuctional is
optional.
As near as I can tell this OS X implementation of ByteBuffer does not
support his functionality.
--
Karthik Ramachandran
Mobile: 412-606-8981
