22.10.2015 22:00, [email protected] пишет: > I?m curious to see roughly how much query time is spent reading selected > column data vs evaluating the condition. Maybe try to select just a > single column with the same where clause. > Result follows: 1 column: albatros2 fastbit # echo 3 > /proc/sys/vm/drop_caches && time ibis -d tmp3.1 -q "SELECT SRCADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" -output res.txt -v
Constructed a part named tmp3.1 filter::sift2(SELECT SRCADDR FROM 1 data partition WHERE (0x848 ...) -- processing data partition tmp3.1 From tmp3.1 Where (0x84bdd0 OR 0x84af10) --> 6309 countQuery::evaluate -- duration: 1.33 sec(CPU), 1.80634 sec(elapsed) doQuaere -- "SELECT SRCADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" produced a table with 6309 rows and 1 column, took 1.33 CPU seconds, 1.88337 elapsed seconds ibis -- total CPU time 1.33 s, total elapsed time 2.0099 s 4 columns (2x times increase): albatros2 fastbit # echo 3 > /proc/sys/vm/drop_caches && time ibis -d tmp3.1 -q "SELECT FIRST,LAST,SRCADDR,DSTADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" -output res.txt -v Constructed a part named tmp3.1 filter::sift2(SELECT FIRST,LAST,SRCADDR,D ... FROM 1 data partition WHERE (0x84f ...) -- processing data partition tmp3.1 From tmp3.1 Where (0x84bc20 OR 0x84bd40) --> 6309 countQuery::evaluate -- duration: 1.33 sec(CPU), 1.84312 sec(elapsed) doQuaere -- "SELECT FIRST,LAST,SRCADDR,DSTADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" produced a table with 6309 rows and 4 columns, took 1.47 CPU seconds, 4.12235 elapsed seconds ibis -- total CPU time 1.47 s, total elapsed time 4.27049 s 8 columns(3x times increase): albatros2 fastbit # echo 3 > /proc/sys/vm/drop_caches && time ibis -d tmp3.1 -q "SELECT FIRST,LAST,PROTO,SRCADDR,SRCPORT,DSTADDR,DSTPORT,NETADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" -output res.txt -v Constructed a part named tmp3.1 filter::sift2(SELECT FIRST,LAST,PROTO,SRC ... FROM 1 data partition WHERE (0x848 ...) -- processing data partition tmp3.1 From tmp3.1 Where (0x84be20 OR 0x84aed0) --> 6309 countQuery::evaluate -- duration: 1.33 sec(CPU), 1.7951 sec(elapsed) doQuaere -- "SELECT FIRST,LAST,PROTO,SRCADDR,SRCPORT,DSTADDR,DSTPORT,NETADDR FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" produced a table with 6309 rows and 8 columns, took 2.29 CPU seconds, 6.2366 elapsed seconds ibis -- total CPU time 2.3 s, total elapsed time 6.37414 s 12 columns(5x times increase): albatros2 fastbit # echo 3 > /proc/sys/vm/drop_caches && time ibis -d tmp3.1 -q "SELECT FIRST,LAST,PROTO,SRCADDR,SRCPORT,DSTADDR,DSTPORT,NETADDR,NATPORT,DSTAS,DOCTETS,DPKTS FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" -output res.txt -v Constructed a part named tmp3.1 filter::sift2(SELECT FIRST,LAST,PROTO,SRC ... FROM 1 data partition WHERE (0x848 ...) -- processing data partition tmp3.1 From tmp3.1 Where (0x848440 OR 0x84b020) --> 6309 countQuery::evaluate -- duration: 1.33 sec(CPU), 1.86517 sec(elapsed) doQuaere -- "SELECT FIRST,LAST,PROTO,SRCADDR,SRCPORT,DSTADDR,DSTPORT,NETADDR,NATPORT,DSTAS,DOCTETS,DPKTS FROM tmp3.1 WHERE FIRST>1441054677 AND LAST<1441065957 AND SRCADDR=1481989497 or DSTADDR=1481989497" produced a table with 6309 rows and 12 columns, took 3.03 CPU seconds, 9.20098 elapsed seconds ibis -- total CPU time 3.04 s, total elapsed time 9.35995 s -- Signed, Oleg Gawriloff. _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
