On Sun, 20 Mar 2022 at 16:49, Sumanth Rajkumar <rajkumar.suma...@gmail.com> wrote:
> Thanks for the feedback Alex! > > As suggested, I reviewed the JTransforms and ComplexUtils class in the > complex streams package. > > The existing complex utils class has methods to convert to and from Array > data structures (the forms used in JTransform) to Complex class. > > I can come up with a Java 8/Streams based API for implementing complex > FFT algorithms of the types in JTransforms and support various methods in > ComplexUtils > The streams based complex operations API should allow for decoupling the > backing data structures. > This should make it possible to use single API to create an unit test > suite to benchmark/compare different backing data structures such as single > arrays, floats or even polar representations > Support for the output of Jtransforms ideally should not copy any data. This would be a wrapper to accept the array in place and allow further operations on the set of array complex data. At the end the same array can then be transformed back by JTransforms. However accepting a stream of Complex objects and performing a collect operation as a Stream termination operation to different backing data structures would be something to investigate: Complex[] c = ... ComplexList<Complex> list = Arrays.stream(c).collect(ComplexCollectors.toList()); double[] data = list.flatten(); Ideally the ComplexCollectors would create a collector to generate data in a suitable format for any other library, e.g. JTransforms, so that the flatten operation simply returns the underlying data. Both methods would require arguments to allow the user to specify the organisation of the underlying data structure. The flatten method would then have to support transformation if necessary. The types of structure could be for a 1D example: double[] real double[] imag // Packed as real = i*2, imag = i*2+1 double[] data // Packed as real = i, imag = len/2 + i double[] data Then support for 2, 3, ND equivalents. In this case the class Gilles mentioned is useful: org.apache.commons.numbers.arrays.MultidimensionCounter Data transformation this way via streams may not be the optimal way to perform this with regard to memory usage. A transform utility class may be better suited as it can validate the dimensions of the input size before any allocation is made for the output data structure. For example you cannot create a 512*512 2D array if you do not have the correct size input. > > As part of the project, I could implement a subset of the FFT operations > in JTransform using the new streams based Complex Numbers API and > benchmark it against JTransform implementation > This would be repeating a lot of work from JTransforms which already implements support for parallel processing during the transform. I am not sure what the gain would be. Note that a FFT operation must operate on a vector of data, typically a power of size 2. So that would limit the use in a stream to stripes of data. Is this your intention? If so how would it differ from a parallel striped implementation in JTransforms? > > > I understand that we are in the GSOC discussion phase. I am trying to > understand the background of this project and the requirements in order to > come up with my GSOC proposal > > Can you provide with more information on the envisaged usage of > Commons-Numbers (especially Complex Numbers), its current usage/users and > the vision/roadmap for future enhancements > I do not have metrics for the usage of numbers. This is something that requires an analysis of the dependencies of a large number of projects to create a histogram of the frequency for library usage. I am not aware of a recent meta analysis where we can obtain such data for Numbers which had its first release in July 2021. Typically there is no roadmap for Commons beyond accepting contributions that add functionality that has general applicability. Future additions to Numbers could be further refactoring of existing functionality in Commons Math. There are already many parts of Commons Math that now depend on Commons Numbers. Since this is a community supported project the direction is really depicted by the input efforts of the community. > > Are we expecting complex-numbers to be an efficient pure java library that > could be used by other java libraries such as commons-imaging for data > compression (DCT /JPEG lossy compression)? > Numbers should be seen as a toolbox to be used by other Java applications. The best location for routines is something to discuss on the mailing list. In the example of DCT, I am not aware if imaging currently has an encoder implementation for this. There is a decoder: org/apache/commons/imaging/formats/jpeg/decoder/Dct.jav > > Are there other Java/Non-Java (C/Python) libraries that provide similar > features that I can look into for design inspiration and also benchmark > Complex Numbers with? > Complex numbers are a requirement for a C library that conforms to ISO C99. So there are implementations of the functions, for example in GNU glibc. There you can use a complex number just as a real number simply by: #include <complex.h> complex double a = 3 + 4 * I; complex double b = -2 - 5 * I; complex double c = sin(a * b); #include <complex> complex<double> a = complex<double>(3, 4); ... But support for extended data structures would be either just simple arrays or lists, or require a matrix library that can be typed to complex. Matlab has complex number support for single and matrix operations. However implementation of a matrix library is a significant effort. One approach would be again to provide data structures that can interoperate with other existing libraries such as EJML [1]. I have not used the complex matrices in EJML. The overview indicates support for operators such as addition and multiplication. You could investigate if the extension to Complex to allow calling the functions as static methods on real and imaginary parts could be applied to EJML complex matrices allowing workflows to be created for example with the trigonomic operations in Complex. Alex [1] http://ejml.org/wiki/index.php?title=Main_Page
