Is there anybody able to help on this ?
Regards, Lizhou ------------------ Original ------------------ From: "高立周"<[email protected]>; Date: Fri, Mar 30, 2018 01:40 PM To: "dev"<[email protected]>; Subject: poor write performance with optional fields Hi guys, I'm testing parquet write performance for optional fields with attached cc file. The throughput is 125MB/s, but the nvme disks limits is 570MB/s. So, my question is that whether it is expected performance ? Is there anything wrong with the test program ? Thanks, Lizhou lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ ./parquet_test WARNING: Logging before InitGoogleLogging() is written to STDERR I0330 13:28:57.599980 17640 reader-writer-performance.cc:72] BEGIN performance_test I0330 13:29:13.640228 17640 reader-writer-performance.cc:112] END performance_test PROFILE: interrupts/evictions/bytes = 1551/857/61200 lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ ls -lh parquet_cpp_example.parquet -rw-r--r-- 1 lizgao lizgao 2.0G Mar 30 13:29 parquet_cpp_example.parquet lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd if=/dev/zero of=test bs=1M count=4096 oflag=direct 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 7.48397 s, 574 MB/s lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd if=/dev/zero of=test bs=1M count=4096 oflag=direct 4096+0 records in 4096+0 records out 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 7.98757 s, 538 MB/s lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd if=/dev/zero of=test bs=10M count=400 oflag=direct 400+0 records in 400+0 records out 4194304000 bytes (4.2 GB, 3.9 GiB) copied, 8.03284 s, 522 MB/s lizgao@lizgao-ubuntu:~/CLionProjects/parquet-test/cmake-build-debug$ dd if=/dev/zero of=test bs=1G count=4 oflag=direct 4+0 records in 4+0 records out 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 8.30123 s, 517 MB/s
