> I would like to write Avro files to HDFS using C as the programming > language. I have a simple queston: Can I use the avro-c API with > libhdfs? I tried the hdfs-nfs-proxy, but it doesn't support the append > operation.
I don't have any experience with libhdfs, so I'm not sure. The Avro C API lets you write Avro data files to a file on the local filesystem, identified by a `const char *` filename, or a `FILE *` instance that you've already opened, so my guess is that, out of the box, the only way it would work would be to create the file locally and then transfer the existing local file to HDFS. To get it to work directly with libhdfs, we'd probably need to add a new API for providing a callback function, which the Avro bindings would call whenever data needs to be written. (This is already what happens under the covers, so it would really just be exposing parts of the existing functionality in the public API.) cheers –doug
signature.asc
Description: OpenPGP digital signature
