Hi guys,
I'm testing parquet write performance for optional fields with attached cc
file.
The throughput is 125MB/s, but the nvme disks limits is 570MB/s.
So, my question is that whether it is expected performance ?
Is there anything wrong with the test program ?
Thanks,
Lizhou
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$
./parquet_test
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0330 13:28:57.599980 17640 reader-writer-performance.cc:72] BEGIN
performance_test
I0330 13:29:13.640228 17640 reader-writer-performance.cc:112] END
performance_test
PROFILE: interrupts/evictions/bytes = 1551/857/61200
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ ls -lh
parquet_cpp_example.parquet
-rw-r--r-- 1 lizgao lizgao 2.0G Mar 30 13:29 parquet_cpp_example.parquet
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd
if=/dev/zero of=test bs=1M count=4096 oflag=direct
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 7.48397 s, 574 MB/s
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd
if=/dev/zero of=test bs=1M count=4096 oflag=direct
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 7.98757 s, 538 MB/s
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd
if=/dev/zero of=test bs=10M count=400 oflag=direct
400+0 records in
400+0 records out
4194304000 bytes (4.2 GB, 3.9 GiB) copied, 8.03284 s, 522 MB/s
lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd
if=/dev/zero of=test bs=1G count=4 oflag=direct
4+0 records in
4+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 8.30123 s, 517 MB/s
