Hi Mark,
Thanks for the interesting results on FPC compression.
You probably already saw this, but FPC is licensed for academic
use only. So if you compress data using FPC and distribute
it to a commercial entity, conceivably they could not legally
read it. But then I'm not a lawyer! The FPC license that
accompanies the software reads ...
4. SOFTWARE is made available under this Agreement to allow
certain non-commercial research and teaching use. CRF reserves
all commercial rights to SOFTWARE and these rights may be
licensed by CRF to third parties.
Steve
On Wed, Jul 25, 2012 at 12:00:07PM -0400, [email protected] wrote:
> Send Hdf-forum mailing list submissions to
> [email protected]
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
> or, via email, send a message with subject or body 'help' to
> [email protected]
>
> You can reach the person managing the list at
> [email protected]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Hdf-forum digest..."
>
>
> Today's Topics:
>
> 1. Re: Experience with 'FPC' compressor in concert with HDF5
> (Mark Miller)
> 2. Re: help (Elena Pourmal)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 24 Jul 2012 18:22:53 -0700
> From: Mark Miller <[email protected]>
> To: HDF Users Discussion List <[email protected]>
> Subject: Re: [Hdf-forum] Experience with 'FPC' compressor in concert
> with HDF5
> Message-ID: <[email protected]>
> Content-Type: text/plain
>
> In case anyone winds up following 'FPC' to this archive, I thought I
> would just mention experience from a simple test I did with FPC (apart
> from HDF5 though).
>
> I created three binary files of 1,000,000 samples of 4 cycles of a sin
> wave. One with a magnitude of 1.0, one with a magnitude of 1e+15 and one
> with magnitude of 1e-15. So, its very smooth data!
>
> Test FPC compression ratio gzip --best ratio
> 1.0 mag 2.00 1.27
> 1e-5 mag 2.00 1.27
> 1e+5 mag 2.04 1.28
>
> So, FPC does better than gzip for this relatively smooth data and FPC
> appears to compress and decompress *much* faster (5-10x) than gzip too.
>
> Finally, I tried an integer example with a magnitude of 100000 (fitting
> well within the range of a 32-bit int) and compressed that with gzip and
> get a compression ratio of about 3.25.
>
>
> On Fri, 2012-07-13 at 10:09 -0700, Mark Miller wrote:
> > I searched the HDF5-Forum archives and didn't get any hits on 'FPC'
> >
> > I am curious if anyone has any experience adapting the 'FPC' floating
> > point compression code,
> > http://www.csl.cornell.edu/~burtscher/research/FPC/
> > for use as an internal HDF5 compression 'filter'?
> >
> > If so, what where your experiences? Would you be willing to share your
> > code? Do you know of anything 'better' for floating point data?
> >
> > Thanks for any info.
> >
> > Mark
> >
> >
> --
> Mark C. Miller, Lawrence Livermore National Laboratory
> <<<<<<<<<<<<<<<<!!LLNL BUSINESS ONLY!!>>>>>>>>>>>>>>>>
> [email protected] urgent: [email protected]
> T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 24 Jul 2012 21:33:38 -0500
> From: Elena Pourmal <[email protected]>
> To: HDF Users Discussion List <[email protected]>
> Subject: Re: [Hdf-forum] help
> Message-ID: <[email protected]>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi Andrea,
>
> Thank you for your report. This is definitely an oversight on our part. I
> will file a bug report (it is an easy fix).
>
> Elena
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Elena Pourmal The HDF Group http://hdfgroup.org
> 1800 So. Oak St., Suite 203, Champaign IL 61820
> 217.531.6112
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
> On Jul 23, 2012, at 10:40 AM, Andrea Negri wrote:
>
> > Hi, I'm using the last stable release of the gfortran compiler (4.7.1)
> > with the flags -Wconversion -Wconversion-extra.
> > I'm trying to build an extensible dataset, by using the command
> >
> > use hdf5
> > implicit none
> > integer(hsize_t), dimension(7), parameter:: dims_scalar =
> > INT((/1,0,0,0,0,0,0/),hsize_t)
> > integer, parameter :: rank = 1
> > integer(hid_t) :: dataspace_id
> > integer :: error
> > integer(hsize_t), dimension(1) :: maxdims
> >
> >
> > maxdims = H5S_UNLIMITED_F
> > CALL h5screate_simple_f(RANK, dims, dataspace_id, error, &
> > & maxdims)
> >
> >
> >
> > and the compiler returns the warning:
> >
> > maxdims = H5S_UNLIMITED_F
> > 1
> > Warning: Conversion from INTEGER(4) to INTEGER(8) at (1)
> >
> >
> >
> > I have checked in the file H5f90global.f90 and H5S_UNLIMITED_F is
> > declared as integer, whereas h5screate_simple_f actually requests an
> > integer(hsize_t) for the maxdims argument.
> >
> > I have "fixed" the warning by converting the H5S_UNLIMITED_F variable:
> > maxdims = INT(H5S_UNLIMITED_F, HSIZE_T)
> >
> >
> > but the inconsistency still remains.
> >
> > _______________________________________________
> > Hdf-forum is for HDF software users discussion.
> > [email protected]
> > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20120724/ccb27dfb/attachment-0001.html>
>
> ------------------------------
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
> End of Hdf-forum Digest, Vol 37, Issue 22
> *****************************************
--
Steve Sullivan [email protected] 303-497-2823
FL/2, Research Applications Laboratory
National Center for Atmospheric Research
PO Box 3000
Boulder CO 80307
USA
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
