Hi , I'm trying to use fastbit ibis 64bit. I've allocate 5G to fastbit (use memory mapped file) should be enough , but it still failed ,
What am i doing wrong ? see bellow for the output c:\GitHub\fastbit\win\Release64\ibis Options summary: batch mode, log level 3, skipping estimation resource::read -- parsing configuration file "C:\GitHub\pythia\tests\fastbit\scripts\tpch_queries\ibis_config.txt" resource::read -- parsing configuration file "C:\GitHub\pythia\tests\fastbit\scripts\tpch_queries\ibis_config.txt" fileManager initialization complete -- maxBytes=5242880000, maxOpenFiles=60 util::gatherParts -- examining C:\db\fastbit\xaa Constructed a part named xaa activeDir = "C:\db\fastbit\xaa" 5000000 rows and 15 columns util::gatherParts -- examining C:\db\fastbit\xab Constructed a part named xab activeDir = "C:\db\fastbit\xab" 5000000 rows and 15 columns Partitions[2]: xaa xab Query[1]: SELECT returnflag,linestatus,sum(quantity) as sum_qty,sum(price*(1-discount)) as sum_disc_price,sum(price*(1-discount)*(1+tax)) as sum_charge,avg antity) as avg_qty, avg(price) as avg_price, avg(discount) as avg_disc, count(*) as count_order WHERE shipdate <= 904608000 ; liga -- constructed table T-xaa (a simple list of partitions: xaa and xab) from a list of 2 data partitions, with 15 columns and 10000000 rows tableSelect -- processing "SELECT returnflag,linestatus,sum(quantity) as sum_qty,sum(price*(1-discount)) as sum_disc_price,sum(price*(1-discount)*( ax)) as sum_charge,avg(quantity) as avg_qty, avg(price) as avg_price, avg(discount) as avg_disc, count(*) as count_order FROM T-xaa WHERE shipdate 904608000 ;" selectClause::fillNames -- select clause internal details: low-level expressions (names_[8], aggr_[8], atms_[8]): 0: returnflag, returnflag 1: linestatus, linestatus 2: __2, SUM(quantity) 3: __3, SUM((price * (1 - discount))) 4: __4, SUM(((price * (1 - discount)) * (1 + tax))) 5: __5, COUNT(*) 6: __6, SUM(price) 7: __7, SUM(discount) high-level expressions (xnames_[9], xtms_[9]): 0: returnflag, returnflag 1: linestatus, linestatus 2: sum_qty, SUM(quantity) 3: sum_disc_price, SUM((price * (1 - discount))) 4: sum_charge, SUM(((price * (1 - discount)) * (1 + tax))) 5: avg_qty, (SUM(quantity) / COUNT(*)) 6: avg_price, (SUM(price) / COUNT(*)) 7: avg_disc, (SUM(discount) / COUNT(*)) 8: count_order, COUNT(*) countQuery::setWhereClause accepted new query conditions "shipdate <= 9.04608e+008" Constructed in-memory data partition hW04N1 -- filter::sift2S(SELECT returnflag,linestatu ... FROM 2 data partitions WHERE shipd ...) -- with 8 col s filter::sift2S(SELECT returnflag,linestatu ... FROM 2 data partitions WHERE shipd ...) -- processing data partition xaa countQuery assigned data partition xaa countQuery::evaluate -- start timer ... bin[xaa.shipdate]::ctor -- initialization completed with 2527 bins for 5000000 rows from a storage object @ 0000000001F2C5F0 offset 8 index::create -- the bin index for column xaa.shipdate was read from C:\db\fastbit\xaa in 0.002 sec(CPU), 0.00239134 sec(elapsed) countQuery::evaluate -- Select count(*) From xaa Where shipdate <= 9.04608e+008 --> 4927856 countQuery::evaluate -- duration: 0.009 sec(CPU), 0.00776298 sec(elapsed) part[xaa]::calculate - starting to evaluate "(price * (1 - discount))" with mask (4927856 out of 5000000) part[xaa]::calculate -- evaluating (price * (1 - discount)) on 4927856 records (total: 5000000) took 0.146679 sec elapsed time and produced 4927856 lues bord[hW04N1]::append -- adding 4927856 elements to column __3 from (price * (1 - discount)) part[xaa]::calculate - starting to evaluate "((price * (1 - discount)) * (1 + tax))" with mask (4927856 out of 5000000) part[xaa]::calculate -- evaluating ((price * (1 - discount)) * (1 + tax)) on 4927856 records (total: 5000000) took 0.222345 sec elapsed time and pr ced 4927856 values bord[hW04N1]::append -- adding 4927856 elements to column __4 from ((price * (1 - discount)) * (1 + tax)) text[xaa.returnflag]::selectStrings -- duration: 0.597 sec(CPU), 0.597355 sec(elapsed) bundles::ctor created a colValues for "returnflag" as cols[1] with size 4927856 bundles::ctor created a colValues for "linestatus" as cols[2] with size 4927856 bundles::ctor created a colValues for "quantity" as cols[3] with size 4927856 bundles::ctor created a colValues for "(price * (1 - discount))" as cols[4] with size 4927856 bundles::ctor created a colValues for "((price * (1 - discount)) * (1 + tax))" as cols[5] with size 4927856 bundles::ctor created a colValues for "price" as cols[6] with size 4927856 bundles::ctor created a colValues for "discount" as cols[7] with size 4927856 Constructed in-memory data partition ELH5k -- Select returnflag, linestatus, SUM(quantity), SUM((price * (1 - discount))), SUM(((price * (1 - disco )) * (1 + tax))), COUNT(*), SUM(price), SUM(discount) From hW04N1 -- with 4 rows and 8 columns filter::sift2S(SELECT returnflag,linestatu ... FROM 2 data partitions WHERE shipd ...) -- processing data partition xab countQuery changing data partition from xaa to xab countQuery::evaluate -- start timer ... column[xab.shipdate](INT)::index::mapValues -- generated histogram (2526 distinct values) fileManager::storage::ctor failed malloc(112), will attempt to free some memory fileManager::storage::ctor failed malloc(28672), will attempt to free some memory Warning -- column[shipdate]::bin::ctor encountered an exception, cleaning up ... Warning -- index::create(shipdate) received an std::exception -- bad allocation Warning -- index[xab.shipdate]::create failed to create an index of type default column[xab.shipdate]::evaluateRange(shipdate <= 9.04608e+008) received a std::exception -- bad allocation part::doScan -- evaluating shipdate <= 9.04608e+008 on 5000000 int values (total: 5000000) took 0.0192468 sec elapsed time and produced 4928205 hit countQuery::evaluate -- Select count(*) From xab Where shipdate <= 9.04608e+008 --> 4928205 countQuery::evaluate -- duration: 0.523 sec(CPU), 0.523351 sec(elapsed) part[xab]::calculate - starting to evaluate "(price * (1 - discount))" with mask (4928205 out of 5000000) clearing data partition ELH5k clearing data partition hW04N1 filter::sift2S(SELECT returnflag,linestatu ... FROM 2 data partitions WHERE shipd ...) -- duration: 3.897 sec(CPU), 3.89773 sec(elapsed) Warning -- table::select absorbed a std::exception (bad allocation), will return a nil pointer tableSelect:: select(returnflag,linestatus,sum(quantity) as sum_qty,sum(price*(1-discount)) as sum_disc_price,sum(price*(1-discount)*(1+tax)) as su harge,avg(quantity) as avg_qty, avg(price) as avg_price, avg(discount) as avg_disc, count(*) as count_order , shipdate <= 904608000 ;) failed on ta T-xaa c:\GitHub\fastbit\win\Release64\ibis -- total CPU time 3.913 s, total elapsed time 3.9138 s clearing data partition xaa clearing data partition xab Cleaning up the file manager Total pages accessed through read(unistd.h) is estimated to be 29351 fileManager::clear -- starting ... --- Mon Jun 10 16:04:29 2013 The number of memory mapped files is 1. (max = 60) file name: C:\db\fastbit\xab\shipdate storage @ 0000000001F2C410, 00000000027E0000, 1st 32 bits = 33345660, 1st 64 bits = 3399cbd033345660 file descriptor 00000000000005E0 fmap handle 00000000000005E4 base address 00000000027E0000 mapped y opened at Mon Jun 10 16:04:27 2013 last used at Mon Jun 10 16:04:27 2013 # of bytes 20000000 # of past acc 1 # of active acc 0 Size of all mapped files is 20,000,000 The number of files read into memory is 0 The total size of all files read into memory is 0 Size of all named storages is 20,000,000 Size of all unnamed storages is 0 The total size of all named and unnamed storages is 20,000,000 The prescribed maximum size is 5,242,880,000 Number of pages accessed (recorded so far) is 29351 (page size = 8192) --- Mon Jun 10 16:04:29 2013 The number of memory mapped files is 0. (max = 60) Size of all mapped files is 0 The number of files read into memory is 0 The total size of all files read into memory is 0 Size of all named storages is 0 Size of all unnamed storages is 0 The total size of all named and unnamed storages is 0 The prescribed maximum size is 5,242,880,000 Number of pages accessed (recorded so far) is 29351 (page size = 8192) fileManager decommissioned Started at 16:04:25.25 Ran for 4000 ms Press any key to continue . . .
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
