Yes, it appears as though GenericDatumWriter does not support native ByteBuffers. FileChannel returns native ByteBuffers, not HeapByteBuffer.
Some options: * Read the file contents as a byte[], and wrap a ByteBuffer around that * Create a new heap ByteBuffer and copy the contents of the file channel's byte buffer into it. Please file a bug in JIRA, Thanks! On 2/8/12 6:52 AM, "karthik ramachandran" <[email protected]> wrote: > Hi all, > > I'm running into a problem with BinaryEncoder.writeBytes. I'm trying to wrap > some documents in an Avro structure and then write them into an Avro contain > file. > > My code looks roughly like this : > > File file = new File(...); > ClassLoader cl = SimpleAvroExample.class. > getClassLoader(); > InputStream isSimpleWrapperSchema= > cl.getResourceAsStream("simple_wrapper.avpr"); > > Schema.Parser parser = new Schema.Parser(); > Schema schemaSimpleWrapper = parser.parse(isSimpleWrapperSchema); > GenericDatumWriter<GenericRecord> gdw = new > GenericDatumWriter<GenericRecord>(schemaSimpleWrapper); > DataFileWriter writer = new DataFileWriter(gdw); > > File outputFile = new File(...); > outputFile.createNewFile(); > > writer.create(schemaSimpleWrapper, outputFile); > > > for(File documentFile : file.listFiles()) > { > > GenericRecord r = new GenericData.Record(schemaSimpleWrapper); > FileInputStream fis = new FileInputStream(documentFile); > FileChannel channel = fis.getChannel(); > > r.put("filename", documentFile.getName()); > > //Copying the whole file. > ByteBuffer byteBuffer = channel.map(FileChannel.MapMode.READ_ONLY, > 0, documentFile.length()); > r.put("body", byteBuffer); > > > > > writer.append(r); > > channel.close(); > fis.close(); > > } > > This code exceptions out : > Exception in thread "main" > org.apache.avro.file.DataFileWriter$AppendWriteException: > java.lang.UnsupportedOperationException > at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:261) > at org.iqt.cdl.SimpleAvroExample.main(SimpleAvroExample.java:107) > Caused by: java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.arrayOffset(ByteBuffer.java:968) > at org.apache.avro.io.BinaryEncoder.writeBytes(BinaryEncoder.java:61) > at > org.apache.avro.generic.GenericDatumWriter.writeBytes(GenericDatumWriter.java: > 199) > at > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:76) > at > org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java > :105) > at > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65) > at > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) > at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:255) > ... 1 more > > > The problem seems to be that the writeBytes function on line 61 does the > following : > int start = bytes.arrayOffset() + pos. > > Where byes is a java.nio.ByteBuffer. > > According to the JavaDocs for ByteBuffer the arrayOffset fuctional is > optional. > > As near as I can tell this OS X implementation of ByteBuffer does not support > his functionality. > > > -- > Karthik Ramachandran > Mobile: 412-606-8981
