Hi, Amihay, Looks like there might be something strange on windows. You have a data set with 88 million rows, but index builder only built for 80 million rows. This is one. Is the data directory read-only? It is odd that ibis is not able to generate a .sp file required for working with text columns.
If you can let me have access to the test machine, I'd be happy to take a look. John On 4/8/13 4:42 PM, amihay gonen wrote: > Hi John, > I've compiled the version with the new updates , but it still fails . > > I'm attaching the new debug output. > > btw, if it's help, my machine is on the cloud + visual studio > installed , and I can provide you with an access to this machine. > > Thanks , > Amihay. > > > On Mon, Apr 8, 2013 at 5:48 PM, K. John Wu <[email protected] > <mailto:[email protected]>> wrote: > > Hi, Amihay, > > The problem is related to the parameter fileManaer.maxBytes. > Previously, this variable was using type 'unsigned long' and > therefore, there was a restriction on its size based on ULONG_MAX. > The type of maxByts has been changed to uint64_t a while back, but > this restriction remained in a few places in fileManager.cpp. Just > removed them and the updated code is checked in as SVN revision 624. > You may give it a try. > > John > > > On 4/8/13 4:09 AM, amihay gonen wrote: > > ok , new problem , last problem was seems to be related to > 64bit. I've > > compile a version on 64bit , but now I've encouter a new problem . > > > > I'm attaching debug output , I hope someone will be able to lead > me to > > a clue . > > > > thanks ,amihay > > > > > > On Mon, Apr 8, 2013 <tel:2013> at 9:11 AM, amihay gonen > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > Ok , I've found out that -part.txt was missing columns part (i > > don't understand way ) . I've fix this file , now I'm getting : > > memory malloc problem . I assume it is because of 32bit ? am i > > correct ? > > > > > > constructed a part named lineitem > > filter::sift2(SELECT returnflag,linestatu ... FROM 1 data > > partition WHERE shipda ...) -- processing data partition > lineitem > > index[lineitem.shipdate]::create created an index with 80000000 > > rows, but the data partition has 88000000 rows > > countQuery::evaluate -- Select count(*) From lineitem Where > > shipdate <= 9.04608e+008 --> 78850168 > > countQuery::evaluate -- duration: 14.793 sec(CPU), 14.7931 > > sec(elapsed) > > Error -- fileManager::storage failed to malloc 315,400,672 bytes > > of storage on retry > > Warning -- column[lineitem.quantity]::selectValuesT failed with > > error code -2 > > Error -- fileManager::storage failed to malloc 630,801,344 bytes > > of storage on retry > > tableSelect:: select(returnflag,linestatus,sum(quantity) as > > sum_qty,sum(price*(1-discount)) as > > sum_disc_price,sum(price*(1-discount)*(1+tax)) as > > sum_charge,avg(quantity) as avg_qty, avg(price) as avg_price, > > avg(discount) as avg_disc, count(*) as count_order , shipdate <= > > 904608000 ) failed on table T-lineitem > > D:\code\fastbit\fastbit-ibis1.3.6\win\Release\ibis.exe -- total > > CPU time 15.199 s, total elapsed time 15.1997 s > > > > > > attach is the debug output of ibis. > > > > Thanks Amihay > > > > > > On Mon, Apr 8, 2013 <tel:2013> <tel:2013 <tel:2013>> at 8:37 > AM, amihay gonen > > <[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > Hi , I'm trying to test fastbig , but got the following > Warning : > > Warning -- part::readMetaData found 0 columns, but 16 were > > expected > > > > > > what does it means ? > > > > Attached -part.txt file . > > > > thanks > > > > > > > > > > > > _______________________________________________ > > FastBit-users mailing list > > [email protected] <mailto:[email protected]> > > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
