Ichbinkiana commented on PR #2509: URL: https://github.com/apache/systemds/pull/2509#issuecomment-5160284448
## HDFS Comparison to external Benchmark - I extended the HDF5 benchmark with a local external Java baseline using io.jhdf:jhdf:0.11.0. - I picked jHDF because it is a pure-Java HDF5 library, resolves from Maven Central, and was easy to run locally on my system; it is only for local comparison and is not intended as a required SystemDS dependency. - The benchmark uses 1,000,000 x 200 FP64 matrices, giving a dense logical size of about 1.6 GB, with -Xmx12g, read/write parallelism 16, and medians over 3 non-warmup repetitions. - I ran two kinds of tests: dense-layout HDF5 comparisons and a separate sparse version with the optimized code of mine that was sparse-aware. - The dense-layout comparison is fair because SystemDS and jHDF write/read the same logical dense FP64 dataset with the same shape and dataset name. - For sparse-like input without spaarse-aware optimization flag, the comparison is also fair as a dense-layout comparison: both SystemDS and jHDF store the logical matrix as a dense HDF5 dataset, including zeros. - The test aginst sparse-aware optimization version is intentionally separate, because it measures the benefit of sparse-aware SystemDS storage. - For dense FP64, SystemDS sequential write was ~7.13 s, SystemDS parallel write was ~3.30 s, and jHDF write was ~10.03 s. - This means SystemDS sequential write was about 1.4× faster than jHDF, and SystemDS parallel write was about 3.0× faster than jHDF. - For dense reads from the SystemDS sequential file, SystemDS sequential read was ~883 ms and SystemDS parallel read was ~448 ms, giving about 2.0× read speedup. - An important issue appeared for dense parallel output: the writer produced 11 part files and wrote fast, but the current read path failed to materialize that directory output, so this should not be reported as a successful write/read roundtrip yet. - For sparse-like dense-layout HDF5, the input had only 1,000,000 nonzeros, having 0.005 sparsity, but the output was still ~1.6 GB, showing that sparse input alone does not necessarily cause sparse HDF5 storage. - In that dense-layout sparse mode, SystemDS write was ~7.35–7.42 s, while jHDF write was ~10.40 s; parallel write did not help because the output stayed as one file. - With SystemDS sparse-aware optimization enabled, the same sparse-like matrix dropped from ~1.6 GB to ~24 MB, a ~66.7× file-size reduction. - In this case write time improved from ~7.35 s dense-layout sparse write to ~112 ms sequential sprse-aware write, and from ~7.42 s dense-layout parallel write to ~106 ms sparse-aware parallel write. - Sparse-aware read also improved from ~695 ms dense-layout sparse sequential read to ~63 ms COO sequential read. jHDF read/materialization used much more heap, around 3.2–3.3 GB, because the benchmark reads into a dense Java array before materializing a MatrixBlock. - Important observation: write-only; read-back failed. Happened also with baseline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
