HI John, I have program and data both in my local directory.
Also I tried building static library. but it is still taking more time alternatively [preetigupta25@dhalsim examples]$ time ./ibis -d s0 -v -q "select avg(charge) where zdc>90" Constructed a part named s0 filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z ...) -- processing data partition s0 countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc --> 11072 countQuery::evaluate -- duration: 0.008998 sec(CPU), 0.0116644 sec(elapsed) tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a table with 1 row and 1 column tableSelect -- the result table (1 x 1) for "SELECT avg(charge) FROM T-s0 WHERE zdc>90" 17.1549855491329 tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 WHERE zdc>90 took 0.013998 CPU seconds, 0.0502191 elapsed seconds /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis -- total CPU time 0.014998 s, total elapsed time 0.0509388 s real 0m0.071s user 0m0.013s sys 0m0.019s [preetigupta25@dhalsim examples]$ time ./ibis -d s0 -v -q "select avg(charge) where zdc>90" Constructed a part named s0 filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z ...) -- processing data partition s0 countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc --> 11072 countQuery::evaluate -- duration: 0.012999 sec(CPU), 0.0130408 sec(elapsed) tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a table with 1 row and 1 column tableSelect -- the result table (1 x 1) for "SELECT avg(charge) FROM T-s0 WHERE zdc>90" 17.1549855491329 tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 WHERE zdc>90 took 0.017998 CPU seconds, 0.0177805 elapsed seconds /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis -- total CPU time 0.017998 s, total elapsed time 0.0185473 s real 2m46.445s user 0m0.018s sys 0m0.014s On Mon, Nov 19, 2012 at 8:13 AM, K. John Wu <[email protected]> wrote: > Hi, Preeti, > > When you see a huge difference between the sum of user time and sys > time, and the real time, it usually means that your job is blocked for > something. For example, your file system might be busy. > > One thing you can try might be to move the program and the data to a > directory local to your test machine. Another thing to do is to > compile FastBit code with static option - it will produce larger > executables, but will remove the need to load libraries dynamically at > runtime. Please give these options a try. > > If anyone else has experience dealing with this sort of performance > fluctuations, please let us know how you have solved the problem. > > John > > > On 11/19/12 8:05 AM, preeti gupta wrote: > > Hey John, > > > > I ran the command and the output is here > > > > [preetigupta25@dhalsim examples]$ time ./ibis -d s0 -v -q "select > > avg(charge) wh > > ere zdc>90" > > > > Constructed a part named s0 > > filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z > > ...) -- proc > > essing data partition s0 > > countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc --> 5536 > > countQuery::evaluate -- duration: 0.002 sec(CPU), 0.00106263 sec(elapsed) > > tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a > > table with 1 > > row and 1 column > > tableSelect -- the result table (1 x 1) for "SELECT avg(charge) FROM > > T-s0 WHERE > > zdc>90" > > 17.1549855491329 > > > > tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 > > WHERE zdc>90 t > > ook 0.004 CPU seconds, 0.00380039 elapsed seconds > > /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis > > -- total CP > > U time 0.005 s, total elapsed time 0.00450349 s > > > > real 0m0.023s > > user 0m0.012s > > sys 0m0.007s > > [preetigupta25@dhalsim examples]$ time ./ibis -d s0 -v -q "select > > avg(charge) where zdc>90" > > > > > > Constructed a part named s0 > > filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z > > ...) -- processing data partition s0 > > countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc --> 5536 > > countQuery::evaluate -- duration: 0.001 sec(CPU), 0.0011282 sec(elapsed) > > tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a > > table with 1 row and 1 column > > tableSelect -- the result table (1 x 1) for "SELECT avg(charge) FROM > > T-s0 WHERE zdc>90" > > 17.1549855491329 > > > > tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 > > WHERE zdc>90 took 0.003999 CPU seconds, 0.00399041 elapsed seconds > > /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis > > -- total CPU time 0.004999 s, total elapsed time 0.00474072 s > > > > real 1m2.509s > > user 0m0.006s > > sys 0m0.013s > > > > > > On Sun, Nov 18, 2012 at 11:19 PM, K. John Wu <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi, Preeti, > > > > Would you mind to run > > > > time ./ibis -d s0 -v -q "select avg(charge) where zdc>90" > > > > and see what is reports? > > > > Which one of the STAR sample data you are using? > > > > John > > > > > > On 11/18/12 3:00 PM, preeti gupta wrote: > > > Hi, > > > > > > I am trying to setup Fastbit on linux. I have been able to > > load star > > > sample data file successfully. > > > > > > My query runs slower first time, and then faster next time and then > > > faster (sometimes) one more time but it really slows down next > time. > > > Though the query output shows CPU time taken is .004 secs but the > > > query actually returned in 125 secs. > > > It happens alternatively. > > > > > > [preetigupta25@dhalsim examples]$ ./fastbit.sh > > > > > > Constructed a part named s0 > > > filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z > > > ...) -- processing data partition s0 > > > countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc > > --> 5536 > > > countQuery::evaluate -- duration: 0.001 sec(CPU), 0.00105953 > > sec(elapsed) > > > tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a > > > table with 1 row and 1 column > > > tableSelect -- the result table (1 x 1) for "SELECT avg(charge) > FROM > > > T-s0 WHERE zdc>90" > > > 17.1549855491329 > > > > > > tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 > > > WHERE zdc>90 took 0.003 CPU seconds, 0.00379491 elapsed seconds > > > > /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis > > > -- total CPU time 0.004 s, total elapsed time 0.00450182 s > > > It took 0 seconds > > > [preetigupta25@dhalsim examples]$ ./fastbit.sh > > > > > > Constructed a part named s0 > > > filter::sift2(SELECT avg(charge) FROM 1 data partition WHERE 90 < z > > > ...) -- processing data partition s0 > > > countQuery::evaluate -- Select count(*) From s0 Where 90 < zdc > > --> 5536 > > > countQuery::evaluate -- duration: 0.001 sec(CPU), 0.00106502 > > sec(elapsed) > > > tableSelect -- select(avg(charge), zdc>90) on table T-s0 produced a > > > table with 1 row and 1 column > > > tableSelect -- the result table (1 x 1) for "SELECT avg(charge) > FROM > > > T-s0 WHERE zdc>90" > > > 17.1549855491329 > > > > > > tableSelect:: complete evaluation of SELECT avg(charge) FROM T-s0 > > > WHERE zdc>90 took 0.004 CPU seconds, 0.00384116 elapsed seconds > > > > /home/preetigupta25/FastBit/fastbit-ibis1.3.3/examples/.libs/lt-ibis > > > -- total CPU time 0.005 s, total elapsed time 0.0045433 s > > > It took 125 seconds > > > > > > > > > The script contains only one command > > > > > > #!/bin/bash > > > START=$(date +%s) > > > # do something > > > # start your script work here > > > # your logic ends here > > > test='./ibis -d s0 -v -q "select avg(charge) where zdc>90"' > > > eval $test > > > END=$(date +%s) > > > DIFF=$(( $END - $START )) > > > echo "It took $DIFF seconds" > > > > > > > > > > > > > > > > > > _______________________________________________ > > > FastBit-users mailing list > > > [email protected] <mailto:[email protected]> > > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > > > > > >
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
