Ichbinkiana commented on PR #2509: URL: https://github.com/apache/systemds/pull/2509#issuecomment-5160292245
# Parquet Results and observations Summary ## Summary of Changes - The Parquet frame reader path was optimized, so instead of converting each vaalue to string first, the reader uses primitive type and reads values through integer, long, float, double, boolean, and etc. This impacted the read path for both sequential and parallel readers. - Improved write throughput by allowing multiple writer tasks to run concurrently instead of writing one large single file sequentially. (Parallel parquet writer) - The parallel writer now uses a target part-size configuration rather than writing baased on row/cell count. Runtime settings used in the benchmark were: ```text sysds.io.parquet.writer.target.part.size.mb = 128 sysds.io.parquet.writer.threads = 4 ``` - Limitation: Issue with sparse-like FP64, the writer can still create relatively small part files. --- ## Benchmark Setup ### Synthetic Benchmark My benchmark used three data profiles: - dense_fp64_only - mixed_schema - sparse_like_fp64 Main settings: - rows = 1,000,000 - cols = 100 - warmup = 2 - measured reps = 3 - writer target part size = 128 MB - writer thread limit = 4 - JVM heap = -Xmx12g ### External Real-Data Benchmark The external benchmark used, Data properties: - rows = 2,964,624 - cols = 19 - original external file size = 50 Mb - original input files = 1 --- ## Results and Baseline Comparison ## 1. Dense FP64 Synthetic Data ### Read | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential read | 1 | 7.74 s | | SystemDS parallel read, single-file input | 1 | 7.75 s | | Direct parquet-java read | 1 | 8.23 s | | SystemDS parallel multipart read-back | 6 | 3.17 s | | Direct parquet-java multipart scan | 6 | 8.33 s | ### Write | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential write | 1 | 12.74 s | | SystemDS parallel write | 6 | 6.20 s | | Direct parquet-java write | 1 | 12.74 s | ### Interpretation For dense FP64 data, the single-file SystemDS path and direct parquet-java baseline are close. The main improvement appears in the optimized parallel path: - SystemDS parallel write is about 2.1× faster than sequential/direct single-file write. - SystemDS multipart read-back is about 2.4–2.6× faster than the single-file read paths. This shows that the benefit comes from multipart parallel writing and parallel read-back. --- ## 2. Mixed-Schema Synthetic Data ### Read | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential read | 1 | 10.21 s | | SystemDS parallel read, single-file input | 1 | 10.21 s | | Direct parquet-java read | 1 | 10.41 s | | SystemDS parallel multipart read-back | 6 | 5.20 s | | Direct parquet-java multipart scan | 6 | 10.08 s | ### Write | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential write | 1 | 15.76 s | | SystemDS parallel write | 6 | 7.03 s | | Direct parquet-java write | 1 | 15.89 s | ### Interpretation The mixed-schema profile is slower than dense FP64 because it includes multiple value types, including strings and booleans imo. Similar to dense profile test: - SystemDS parallel write is about 2.2× faster than sequential/direct single-file write. - SystemDS multipart read-back is about 2× faster than single-file read. --- ## 3. Sparse-Like FP64 Synthetic Data ### Read | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential read | 1 | 5.90 s | | SystemDS parallel read, single-file input | 1 | 6.20 s | | Direct parquet-java read | 1 | distorted by one large outlier | | Direct parquet-java read median | 1 | 9.72 s | | SystemDS parallel multipart read-back | 6 | 2.56 s | | Direct parquet-java multipart scan | 6 | 5.94 s | ### Write | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential write | 1 | 9.09 s | | SystemDS parallel write | 6 | 3.65 s | | Direct parquet-java write | 1 | 9.21 s | ### Interpretation The optimized parallel path still shows a clear improvement: - SystemDS parallel write is about 2.5× faster than sequential/direct single-file write. - SystemDS multipart read-back is about 2.3× faster than SystemDS sequential read. --- ## 4. External Yellow Taxi Real Data ### External Single-File Read | Case | Files | Average Time | |---|---:|---:| | SystemDS sequential read | 1 | 4.07 s | | SystemDS parallel read, single-file input | 1 | 3.93 s | | Direct parquet-java read | 1 | 4.18 s | ### Multipart Read-Back | Case | Files | Average Time | |---|---:|---:| | SystemDS parallel multipart read-back | 4 | 1.69 s | | Direct parquet-java multipart scan | 4 | 4.67 s | ### Write from External FrameBlock | Case | Files | Output Size | Average Time | |---|---:|---:|---:| | SystemDS sequential write | 1 | 80 Mb | 5.71 s | | SystemDS parallel write | 4 | 77 Mb | 1.79 s | | Direct parquet-java write | 1 | 80 Mb | 5.41 s | ### Interpretation The external Yellow Taxi benchmark shows that the same behavior appears on real Parquet data. - SystemDS sequential read: ~4.07 s - SystemDS parallel read: ~3.93 s - Direct parquet-java read: ~4.18 s - SystemDS parallel write is about 3× faster than the single-file writers. - SystemDS parallel multipart read-back is about 2.4× faster than reading the original single file. --- ## Overall Interpretation - NOT that SystemDS is generally faster than Apache parquet-java. - SystemDS sequential and direct parquet-java single-file paths are usually close. - The major improvement comes from SystemDS parallel write and SystemDS parallel read-back of multipart output. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
