Sorry. My bad. I had to use InflaterInputStream instead of DeflaterInputStream. What I was doing was simply recompressing the data that is already compressed. It works fine now.
Best regards, Ahmed Eldawy On Wed, Mar 11, 2015 at 5:36 PM, Ahmed Eldawy <[email protected]> wrote: > Hi all, > I'm working on a project that deals with HDF4 files provided by NASA. > This is part of a big project in University of Minnesota ( > http://spatialhadoop.cs.umn.edu/) that uses Hadoop as a distributed > environment to process large data including satellite data. I used to use > HDFJava wrappers to deal with HDF files and it worked OK. However, since it > depends on native libraries, I frequently have troubles distributing the > code over a heterogeneous cluster of different architectures and systems. > So, I decided to write my own Java-native code to open HDF files. > Currently, I'm just writing the minimal code to support the files I'm > working on. > > I followed the specifications provided on the website and I was > successfully able to get the structure of the HDF file and reach the data > blocks that contain the main data (temperature in my case). These blocks > are compressed using deflate with level 1 as indicated by the file. > However, when I try to decompress these blocks, they do not give me the > expected size. I feel that there's some kind of incompatible decompression > parameters that I need to fix. Here's the code I wrote to decompress the > file > > byte[] compressedData = readRawData(in); // Reads the raw data from > the block > DeflaterInputStream dis = new DeflaterInputStream(new > ByteArrayInputStream(compressedData), new Deflater(1, true)); // Creates a > decompressor > ByteArrayOutputStream baos = new ByteArrayOutputStream(); // A > temporary place to store decompressed data > // This loop reads the uncompressed data and caches it in memory > byte[] buffer = new byte[4096]; > int bufferLength; > while ((bufferLength = dis.read(buffer)) > 0) { > baos.write(buffer, 0, bufferLength); > } > dis.close(); > baos.close(); > // Retrieve the decompressed data > uncompressedData = baos.toByteArray(); > > The header of the compressed data indicate that the size after > decompression should be 2880000 bytes, but my code finds that it is > 1613097, which is totally different. When creating the deflater, I tried to > set the 'nowrap' parameter to true or false but both do not work. The same > file works with HDFJava libraries so it should be a bug in my code. > > Thanks in advance for your help. Please let me know if you need more > details about this issue. > > Best regards, > Ahmed Eldawy >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
