Re: [FastBit-users] huge .sp files

s so they could be used to fetch the corresponding string. > Andrew > > On Jul 7, 2009, at 2:43 PM, K. John Wu wrote: > >> Hi, Andrew, >> >> You are right that FastBit should be able to remove .sp file for >> categorical values. It might be relatively easy to

Re: [FastBit-users] huge .sp files

ut.seqname, data.start, data.end from lut, data where > lut.seqid = data.seqid and data.score > 1000; > > Andrew > > On Jul 8, 2009, at 9:00 PM, K. John Wu wrote: > >> Hi, Andrew, >> >> I was looking at the possibility of removing .sp files for categorical &

Re: [FastBit-users] huge .sp files

would like to run could be something like this: > > select lut.seqname, data.start, data.end from lut, data where > lut.seqid = data.seqid and data.score > 1000; > > Andrew > > On Jul 8, 2009, at 9:00 PM, K. John Wu wrote: > >> Hi, Andrew, >> >> I

[FastBit-users] fixed intermittent errors

Hi, Jochen, Thanks for the patience in working with us through this set of intermittent errors. The test cases you've provided were extremely helpful. Since the test cases contains nearly 100 conditions in each query, it took me a while to sort through them and ping point the root cause of t

Re: [FastBit-users] get2DDistribution() segmentation fault

Hi, Andrew, Thanks for reporting the problem. We have a group of users making heavy use of the histograming functions, so we are very keen on fixing this problem. To do this, we would need some additional information from you. There are four different version of get2DDistribution in ibis::pa

Re: [FastBit-users] get2DDistribution() segmentation fault

Hi, Andrew, I am still interested in finding out what causes the program to crash. However, it might be helpful to keep the following in mind. None of the histogram function is supposed to work with strings. They should return a negative value and leave the arrays for histograms untouched.

Re: [FastBit-users] get2DDistribution() segmentation fault

t; > ./get2DDist -d test -b 20 -c1 start -c2 end -w "seqid=3 and strand='+'" > > Oddly it works with these options for -w > "seqid=2 and strand='+'" > "seqid IN (1,3) and strand='+'" > > > > > > > On

Re: [FastBit-users] fast bit on DFS

Hi, Avi, This is one of the things we are looking into right now. Hopefully, we will have something to report on this soon. In the mean time, if you have any suggestions and observations, please feel free to share with us. Thanks. John On 7/31/2009 9:47 AM, Avi Z wrote: > Is there a way t

Re: [FastBit-users] get2DDistribution() segmentation fault

;+'" Oddly it works with these options for -w "seqid=2 and strand='+'" "seqid IN (1,3) and strand='+'" On Jul 29, 2009, at 6:58 PM, K. John Wu wrote: Hi, Andrew, Thanks for reporting the problem. We have a group of users making heavy use

Re: [FastBit-users] huge .sp files

string terminators. Of course, the std::string data structure may not be the most appropriate for this, feel free to propose something else if you have one in mind.. John On 8/3/2009 12:11 PM, Matthias Vallentin wrote: > Hi John, > > On Tue, Jul 07, 2009 at 09:34:30AM -0700, K. John

Re: [FastBit-users] huge .sp files

now. What do you think? John On 8/4/2009 4:59 PM, Matthias Vallentin wrote: > On Mon, Aug 03, 2009 at 06:09:57PM -0700, K. John Wu wrote: >> The current design is based on C convention of string and user can >> input strings through an ASCII file (which can not contain nil

Re: [FastBit-users] bioinformatics

Hi, Andrew, I have just checked in an update with functions in ibis::table class that take ibis::qExpr* as query conditions. This should simplify the two step query processing considerably. Here is something that might work.. ibis::table *genome = ibis::table::create("path-to-genome-data");

Re: [FastBit-users] problem in segment method

Hi, Alexandre, Thanks for bring this issue to our attention. My first impression was that your input data might be missing a end-of-line or some such thing, but guess you suspect that ibis::colFloats::segment is in trouble. Would you mind check whether your input CSV file is indeed missing t

Re: [FastBit-users] problem in segment method

Hi, Alexandre, Guess I have completely missed the mark in my previous reply. You are absolutely right that ibis::colFloats::segment was in error and your fix was able to address the problem. This fix, along with others, has been checked into the SVN repository at codeforge.lbl.gov. You shoul

Re: [FastBit-users] problem in segment method

bl.gov> > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of FastBit-users digest..." > > > Today's Topics: > > 1. problem in segment method (Alexandre Maurel) > 2. problem in segment met

Re: [FastBit-users] boolean data

Hi, Andrew, It appears that the packed option might be a more reasonable one. You can pack up to 32 boolean flags in a 32-bit integer (ibis::INT or ibis::UINT). The parser understands "packedflags & 0100101001" as a bitwise expression (following C/C++ definition of bitwise operations betwee

Re: [FastBit-users] fast bit on DFS

. >> >> -Avi >> >> On Fri, Jul 31, 2009 at 8:02 PM, K. John Wu wrote: >>> Hi, Avi, >>> >>> This is one of the things we are looking into right now. Hopefully, >>> we will have something to report on this soon. In the mean time, if

Re: [FastBit-users] choosing optimal index spec

Hi, Andrew, You are right that DNA should be indexed with equality encoding and without binning. If you have not specified anything else, FastBit should be able to determine this one -- this is the easy case. If you want to make it very explicit, the index specification can be index= Both s

Re: [FastBit-users] choosing optimal index spec

Hi, Andrew, You can place index specification in -part.txt. For example, BEGIN Column name=end data_type=Unsigned index= END Column The problem with interval encoding () is that it produces very large indexes. FastBit will probably crash if you attempt to build an interval encoding index (wi

Re: [FastBit-users] choosing optimal index spec

binning works in Section 7.1 of this paper <http://crd.lbl.gov/~kewu/ps/LBNL-60891.pdf>. John On 9/15/2009 1:33 PM, Andrew Olson wrote: > On Sep 15, 2009, at 4:11 PM, K. John Wu wrote: > > >> BEGIN Column >> name=start >> data_type=Unsigned >> minimum=0 &

Re: [FastBit-users] Fastbit aggregate functions?

Dear Jan, Thanks for your interested in our work. We appreciate your suggestions and will put them on our list of things to do. Unfortunately, some of the items that takes a lot of programming effort might take a long time to come about. Regards, John On 10/1/2009 11:45 PM, Jan Steemann wr

Re: [FastBit-users] Fastbit aggregate functions?

r query string for each tuple. > > - A MEDIAN() aggregate function would be absolutely great, however, I think > will be much harder to implement than the previous two. > > - A PERCENTILE() aggregate function would probably be more generic than > MEDIAN() and would be absol

Re: [FastBit-users] Fastbit aggregate functions?

nd the patch to save you unnecessary work. > Is that ok? > > Best regards > Jan > > > -Original Message- > From: fastbit-users-boun...@hpcrdm.lbl.gov > [mailto:fastbit-users-boun...@hpcrdm.lbl.gov] On Behalf Of K. John Wu > Sent: Monday, October 05, 2

Re: [FastBit-users] FastBit performance

Dear Dr. Deri, Thanks for you interest in our software. I would say taking 30 seconds to read a subset from 10 million records is a little too long. Here is a longer explanation. Hope it helps. John PS: Longer explanation. The bitmap indexes are very good for counting the records satisfy

Re: [FastBit-users] Fastbit aggregate functions?

50M rows each and loading them > seperately worked fine. > However, is there a way to either cap the memory usage for data loading so > the size of the input file does not matter (with the tradeoff of the import > taking longer) or to predict the maximum size an import file can have so it >

Re: [FastBit-users] Fastbit aggregate functions?

:40 PM > To: FastBit Users > Subject: Re: [FastBit-users] Fastbit aggregate functions? > > Hi John, > > yes, I will try to clean it up as much as I can and then resend it. > It will probably take a week or so. > > Best regards > Jan > > -Original Message

Re: [FastBit-users] Querying in different kinds of indices

Hi, Thiago, I presume that the ith row of X is to be considered part of the ith row of the whole dataset/data table along with ith row of Y and Z. If this is the case, you need put all three of them in a single data partition (i.e., in the same directory). There is any tool to help you merge

Re: [FastBit-users] column order in table->select

Hi, Andrew, It is our intention to keep the output in the same order as the string string. From your question, we assume that we have not been able to keep this order. We will examine the code to see what is the problem. In the mean time, if you have a simple example for us, it would make

Re: [FastBit-users] column order in table->select

9912, "chr1", 3173252, 13102998 > 13557162, "chr1", 3202481, 13319912 > 17125658, "chr1", 3261794, 13607162 > 29878082, "chr1", 4141418, 17175658 > -- 28 skipped... > -- end printing -- > > On Oct 23, 2009, at 9:57 AM, K. John Wu wrote: >

Re: [FastBit-users] Fwd: Issue in configure stage of fastbit latest version

Hrishikesh, Would you mind try ./configure with sudo command? If you need to install the library to a directory that requires higher privilege, you can try to run the install command with sudo.. John On 10/23/2009 6:40 PM, hrishikesh pathak wrote: > > > -- Forwarded message

Re: [FastBit-users] Fwd: Fwd: Issue in configure stage of fastbit latest version

/23/2009 8:22 PM, hrishikesh pathak wrote: > > > Hi John > > I tried running the commnand both with and without using sudo > > Without sudo : it gave me permission denied message > and > with sudo : it gave me below error message > > Thanks > Hrishikesh >

Re: [FastBit-users] how many records at most can fastbit build bitmap indices on them?

Hi, Min, Thanks for your interest in FastBit. FastBit logically works with data tables, where each table can be divided into multiple partitions. Each data partition is stored in a directory on a file system and consists of a horizontal segment of a data table. FastBit builds an index for

Re: [FastBit-users] how many records at most can fastbit build bitmap indices on them?

Hi, Mian, Thanks for the information. The warning messages appear odd to me. For example, the 1st warning message says that the maximum allowed memory (maxBytes) is about 4GB, the amount of memory used currently (totalBytes) is only 33 MB, but it can not find enough space for 628 MB. From wh

Re: [FastBit-users] how many records at most can fastbit build bitmap indices on them?

Hi, Mian, By default, FastBit's memory manager (actually called ibis::fileManager) uses half of the physical memory available. The way to change this default is to provide a run-time parameter file. Including the following line fileManager::maxBytes = 6GB will increase the maximum amount of

Re: [FastBit-users] how many records at most can fastbit build bitmap indices on them?

Hi, Mian, We are able to confirm that the function ibis::fileManager::unload (which printed the warning message) was indeed performing a number of computations in 32-bit arithmetic. They will be changed to use size_t in the coming releases. This should address the problem you've seen. We are

Re: [FastBit-users] internals when fastbit executing select queries with bitmap index

Hi, Min, FastBit does not currently support the operator 'like'. It has been mostly used with numerical values. For string values, the only expression supported is equality test. Assuming you are still interested, here is a quick description of how FastBit handles the following statement se

Re: [FastBit-users] internals when fastbit executing select queries with bitmap index

Hope this helps. John On 10/29/2009 8:38 PM, Min Zhou wrote: > Thank you for your reply, John. > > I understanded that the bitmap is loading through bitwise OR , but I > do not know how to filter other columns besides col1. > Very appreciate for your help. > > &g

Re: [FastBit-users] Sorting fix

Hi, Luca, Thanks for the bug fix. Apparently the same error has been replicated once more in the same source file;-) Thanks for digging into this. It is very helpful to us. The corrections will be in the next release which is expected in a week or two. John On 11/1/2009 5:16 AM, Luca Deri

Re: [FastBit-users] internals when fastbit executing select queries with bitmap index

Hi, Min, In a bitmap index, each bit of a bitmap corresponds to a particular row. Typically, the 1st bit corresponds to the 1st row, the 2nd bit corresponds to the 2nd row, and so on. In resolve a query condition such as "col4 < 1.5", a bitmap is produce to represent the answer. For example,

[FastBit-users] SC09

Hi, Everyone, Just in case you plan to be at the upcoming SC09 conference in Portland, Please stop by LBNL booth. There is a schedule round-table for the people behind the FastBit work Wednesday 18th, 1:30-3:00PM. John ___ FastBit-users mailing list

Re: [FastBit-users] column order in table->select

368, "chr1", 51931, 317719 > 2634220, "chr1", 90344, 521368 > 3845268, "chr1", 618557, 2684220 > 13052998, "chr1", 908819, 3995268 > 13219912, "chr1", 3173252, 13102998 > 13557162, "chr1", 3202481, 13319912 > 17125658, "ch

Re: [FastBit-users] 64 bit Linux

Hi, Adam, FastBit should work fine on 64-bit machines. Please let us know if you spot any problem. Thanks. John On 11/8/2009 7:26 AM, Schwabacher, Adam wrote: > > > Has Fastbit been used ( or tested ) on 64 bit Linux kernels? > > > > Thanks in advance , > > Adam > > >

Re: [FastBit-users] internals when fastbit executing select queries with bitmap index

in > > On Sat, Nov 7, 2009 at 11:51 AM, K. John Wu wrote: >> Hi, Min, >> >> You are probably concerned about two things, the index size and query >> processing time. >> >> In terms of index size, one typically have enough disk space to store >>

Re: [FastBit-users] problem in segment method

t; fastbit-users@hpcrdm.lbl.gov >> >> To subscribe or unsubscribe via the World Wide Web, visit >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> or, via email, send a message with subject or body 'help' to >> fastbi

Re: [FastBit-users] problem in segment method

nelm) >res->push_back(nelm); >return res; > } // ibis::colUInts::segment > > Alexandre > > fastbit-users-requ...@hpcrdm.lbl.gov a écrit : >> Send FastBit-users mailing list submissions to >> fastbi

Re: [FastBit-users] problem in segment method

essage with subject or body 'help' to >> fastbit-users-requ...@hpcrdm.lbl.gov >> >> You can reach the person managing the list at >> fastbit-users-ow...@hpcrdm.lbl.gov >> >> When replying, please edit your Subject line so it is more specific

Re: [FastBit-users] Data Compression

Hi, Luca, You are right that FastBit compresses bitmaps but not the raw data. We are contemplating the option of compression the raw data as well, but have not started working on that yet. John On 11/12/2009 1:57 PM, Luca Deri wrote: > Dear all > it seems to me that fastbit compresses bitmaps

Re: [FastBit-users] bundle segment method reorder the raw data

Hi, Alexandre, Thanks for catching this problem. It seems that FastBit needs to mark certain array_t objects as read-only. We will look into this shortly. John On 11/16/2009 1:24 AM, Alexandre Maurel wrote: > Hi John, > > I think I found a problem in function ibis::column::selectValuesT. >

Re: [FastBit-users] bundle segment method reorder the raw data

Hi, Alexandre, I have implemented some changes that I think will address the problem you've encountered. The code has been checked in to SVN. If you get the chance, please give the nightly snapshot a try and let us know if it does address the problem. The nightly snapshot is available at

Re: [FastBit-users] query error

s] bundle segment method reorder the raw >>>> data >>>> To: fastbit-users@hpcrdm.lbl.gov >>>> Message-ID: <4b03b64d.8000...@gmail.com> >>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>>> >>>> Hi J

Re: [FastBit-users] selectParser bug

Hi, Andrew, Thanks for reporting the problem. A modification to the associativity of unary operators +, 1 and ! (not) has been made and it should address the problem you've noticed. Please give the nightly snapshot a try when you get the chance. John PS: The nightly snapshot is available at

Re: [FastBit-users] query error

>>>> >>>> Message: 1 >>>> Date: Wed, 18 Nov 2009 09:54:37 +0100 >>>> From: Alexandre Maurel >>>> Subject: Re: [FastBit-users] bundle segment method reorder the raw >>>> data >>>> To: fastbit-user

Re: [FastBit-users] query error

lbl.gov a écrit : >> Send FastBit-users mailing list submissions to >> fastbit-users@hpcrdm.lbl.gov >> >> Message: 2 >> Date: Mon, 23 Nov 2009 09:55:20 -0800 >> From: "K. John Wu" >> Subject: Re: [FastBit-users] query error >> To: FastBit

Re: [FastBit-users] query error

Hi, Alexandre, Thanks for the information. I have just successfully run the query that was giving you trouble. The code has been checked into the SVN repository. The nightly snapshot is available at and is generated daily around 2AM PST.

Re: [FastBit-users] test data (Array.txt)

Subject line so it is more specific >> than "Re: Contents of FastBit-users digest..." >> >> >> Today's Topics: >> >>1. Re: query error (Alexandre Maurel) >>2. Re: query error (K. John Wu) >> >> >> --

Re: [FastBit-users] query error

t; 9, A09, 12 > 9, B09, 12 > 9, C09, 12 > 9, E09, 12 > 9, F09, 12 > 9, G09, 12 > 9, H09, 12 > 10, A10, 12 > 10, B10, 12 > 10, C10, 12 > 10, D10, 12 > 10, E10, 12 > 10, F10, 12 > 10, G10, 12 > 10, H10,

Re: [FastBit-users] test data (Array.txt)

s-ow...@hpcrdm.lbl.gov >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of FastBit-users digest..." >> >> >> Today's Topics: >> >> 1. Re: query error (Alexandre Maurel) >> 2. Re: query erro

Re: [FastBit-users] Technical question

Dear Hyeongu Son, We are very glad that you are using FastBit. Regarding the use of FastBit in a parallel environment, we have done a number of tests with a relatively straightforward setup by having each processor working on a different data partition. This may not be ideal if you have a lar

Re: [FastBit-users] query error

27;OK'))" > > > > Regards, > > Alexandre > > > > fastbit-users-requ...@hpcrdm.lbl.gov a écrit : >> Send FastBit-users mailing list submissions to >> fastbit-users@hpcrdm.lbl.go

Re: [FastBit-users] Fastbit from Java, etc.?

Hi, Otis, Thanks for your interest in FastBit. Due to a lack of man-power, we have not put in any effort to develop interfaces for other languages. Since FastBit is open source, anyone is free to build on our work. In fact, we would encourage such work and would provide as much help as we ca

Re: [FastBit-users] Technical question

t; for massive computations. There may be some ideas on Vertica's website for > parallelizing FastBit in a more sophisticated manner. > > Andrew > > On Dec 16, 2009, at 11:00 AM, K. John Wu wrote: > >> Dear Hyeongu Son, >> >> We are very glad that you are us

[FastBit-users] FastBit version 1.1.5

Thanks to the help from Luca Deri, Jan Hapke, Alexandre Maurel, and Andrew Olson, we have fixed a number of serious bugs in the last release of FastBit. In addition, we also add a small number of features such as allowing strings to be used as group-by columns and allowing wild cards to be used in

Re: [FastBit-users] Technical question

matext -- http://sematext.com/ -- Solr - Lucene - Nutch > > > > - Original Message >> From: K. John Wu >> To: FastBit Users >> Sent: Thu, December 17, 2009 4:42:38 PM >> Subject: Re: [FastBit-users] Technical question >> >> Hi, Andrew, >> >> Th

Re: [FastBit-users] NULL values

Hi, Andreas, Thanks for your interest in FastBit. Regarding the Null values, the documentation about it needs some clarification. In particular, the fact a value is Null is recorded in a bit vector -- not recorded as a specific value. The particular note on null value left this most importa

Re: [FastBit-users] NULL values

Hi, Andreas, The code for treating blank fields in CSV files as NULL values have been checked into the SVN repository. For example, the last two fields of the first of the two following lines will be treated as NULL values when you tell ardea.cpp that they are numerical values, school, ,

Re: [FastBit-users] NULL values

y are right now, having your own null values might be a faster approach than having a standard conforming null values in FastBit. Thanks. John On 1/11/2010 4:52 AM, Andreas Streichardt wrote: > On Wednesday 23 December 2009 21:10:00 K. John Wu wrote: >> Hi, Andreas, > > Hi Joh

Re: [FastBit-users] NULL values

ntQuery.cpp: add calls to m_sel->getNullMask to exclude null values in the select clause On 1/11/2010 4:52 AM, Andreas Streichardt wrote: > On Wednesday 23 December 2009 21:10:00 K. John Wu wrote: >> Hi, Andreas, > > Hi John, > >> The code for treating blank fields in

Re: [FastBit-users] NULL values

Hi, Andreas, Thanks for the feedback. What I interpret MySQL is doing is that it is silently dropping the null values in the results. Internally, I presume MySQL is actually counting the row with null value in the result set, it only exclude the null value when computing the average. FastB

Re: [FastBit-users] Several Fastbit issues

Hi, Andreas, Thanks for the suggestions. We will look into them in the next day or so. John On 1/12/2010 11:02 AM, Andreas Streichardt wrote: > Hi, > > we are currently thinking about using fastbit for the analytic part of our > software. > > Currently i am testing using the ibis interface but

Re: [FastBit-users] Several Fastbit issues

Hi, Andreas, It turns out that the error handling problems you've observed are relatively easy to fix. An update has been checked into SVN repository. Please give it a try and see if it covers all the cases. Feel free to let us know if you spot any additional issues. On the aggregation fun

Re: [FastBit-users] NULL values

ow/record count, not a count on a specific column or expression (even on a > one-column table). > > I think all other databases I used in the past had about the same behaviour. > > Best regards > J > > -Original Message- > From: fastbit-users-boun...@hpcrdm.lbl.go

Re: [FastBit-users] Several Fastbit issues

Wednesday 13 January 2010 03:44:33 K. John Wu wrote: >> Hi, Andreas, > > Hi! > >> It turns out that the error handling problems you've observed are >> relatively easy to fix. An update has been checked into SVN >> repository. Please give it a try and see if it

Re: [FastBit-users] FastBit HDF5?

Hi, Sidney, The main stakeholders of HDF5_FastQuery are in the process of getting the software open-source. It might take a few months for the actual software to appear on the web, but it will be there. John On 1/15/2010 3:43 PM, Sidney Burks wrote: > Hi, > > For a few years now, there

Re: [FastBit-users] Several Fastbit issues

Hi, Andreas, Thanks for carefully examining all these cases for us. Obviously, we have neglected to check the consistency among the various cases you have examined. We appreciate the feedback you provided us. The error handing in a number of places have been updated to be more consistent now

Re: [FastBit-users] FastBit 1.1.5 segmentation fault

Hi, Valeria, Thanks for the bug report. I have a fix checked into the SVN repository now. Please download the nightly snapshot and give it a try when you get the chance. Feel free to let us know if you find any issues. The code just checked in also contains a fix to the problem revealed by

Re: [FastBit-users] Problem/Issue in configure stage of fastbit latest version

Dear Vilobh, Thanks for using FastBit. The particular error message indicates that you don't have a compatible version of FlexLexer.h which should be part of flex. There was one message to FastBit mailing list about configuration problems very similar to yours from Hrishikesh Pathak

Re: [FastBit-users] Creating a partition

Hi, Andreas, You have a point about read-only data partition. It would be useful to be able to create a data partition object representing read-only data or existing data. Let me suggest that we add another argument to the constructor to indicate what we want. Guess the argument has to be a

Re: [FastBit-users] Creating a partition

ers > Subject: Re: [FastBit-users] Creating a partition > > > Am 20.01.2010 um 22:34 schrieb K. John Wu: > > > Hi, Andreas, > > > > Hi John, > > > You have a point about read-only data partition. It would be useful > to be able

Re: [FastBit-users] Creating a partition

physical partitions. In many cases, the partition is not visible to the user. On 1/20/2010 2:02 PM, Andreas Streichardt wrote: > > Am 20.01.2010 um 22:34 schrieb K. John Wu: > >> Hi, Andreas, > > Hi John, > >> You have a point about read-only data partition. It wo

Re: [FastBit-users] joins with bitmap indexes

Hi, Min, There are some join code based on in-memory sort-merge join algorithm, but not based on the new bitmap cross products. There is a join class that represent some simply joins. You will need to use C++ to access it. John On 1/20/2010 9:28 PM, Min Zhou wrote: > Hi Kamesh& John, > >

Re: [FastBit-users] joins with bitmap indexes

> > Thanks, > Min > > On Thu, Jan 21, 2010 at 1:47 PM, K. John Wu wrote: >> Hi, Min, >> >> There are some join code based on in-memory sort-merge join algorithm, >> but not based on the new bitmap cross products. There is a join class >> that represent so

Re: [FastBit-users] joins with bitmap indexes

s, > Andrew > > On Jan 21, 2010, at 12:47 AM, K. John Wu wrote: > >> Hi, Min, >> >> There are some join code based on in-memory sort-merge join algorithm, >> but not based on the new bitmap cross products. There is a join class >> that represent so

[FastBit-users] Sorting error

Hi, Valeria, Thanks for the bug report regarding problems with order-by clauses. I checked in a set of fixes for the problems you've described. Apparently we have not test the code thoroughly for this functionality. I am afraid we have not done a thorough testing these changes either. When

Re: [FastBit-users] Infinite loop in mensa.cpp

Hi, Joel, Thanks for catching this problem. The fix has been checked into the SVN repository. Please feel free to let us know if you find any other issues with FastBit. If you are able to share with us, we would like to hear how you are using it as well. John On 1/28/2010 8:00 AM, Dolisy,

Re: [FastBit-users] File sharing violation during reorder on Windows

Hi, Joel, Thanks for the suggestion. In general, we should clear all cached content before reordering, because the reordering operation will invalid all the cached content of the data partition. Let me see how I might do this in a reasonable way and will get back to you shortly. John On 1

Re: [FastBit-users] File sharing violation during reorder on Windows

Hi, Joel, Thanks for the suggestion. Your observation on the need to empty the file cache is very perceptive. It is exactly what we need to do. The function that performs reordering already perform a number of operations to remove indexes and related files. I have just added more function

Re: [FastBit-users] Minor bug in ibis::part::init

Hi, Andreas, A fix for this issue has been checked into the SVN repository earlier this month. If you get the chance, please give it a try and let us know how it worked for you. Thanks. John On 2/9/2010 7:34 AM, Andreas Streichardt wrote: > Hi, > > referring to 1.1.6: > > if iname contains

Re: [FastBit-users] Memory leak in fileManager::storage::enlarge ?

Hi, Zeid, Thanks for the report. This particular issue should have been fixed in ibis1.1.6. If you get a chance, please give it a try. There is a note about the change from Jan 29 in ChangeLog, which reads * src/fileManager.cpp: storage::enlarge used copy-and-swap idiom incorrectly and caused

Re: [FastBit-users] Memory leak in fileManager::storage::enlarge ?

e that the enlarge method is still calling > the swap method which ends up replacing the reference count of the > temporary storage object with the existing one which results in the > buffer not getting freed up. > > Zeid > > > -Original Message- > From: K. John Wu

Re: [FastBit-users] Memory leak in fileManager::storage::enlarge ?

reproduce it with the test data. > > > -Original Message- > From: fastbit-users-boun...@hpcrdm.lbl.gov > [mailto:fastbit-users-boun...@hpcrdm.lbl.gov] On Behalf Of K. John Wu > Sent: Thursday, February 11, 2010 11:09 PM > To: FastBit Users > Subject: Re: [FastBit-users

Re: [FastBit-users] Memory leak in fileManager::storage::enlarge ?

7553) Line > 1214 >> C++ >> ibis.exe!ibis::util::addIncoreData >> ibis.exe!ibis::table::select >> ibis.exe!ibis::table::select >> ibis.exe!ibis::mensa::select(const char * sel=0x016c0930, const char * >> cond=0x0018f2f0) >> ibis.exe!tableSelect >> ibis.ex

Re: [FastBit-users] Bulk Data Loading

Hi, Teryl, Thanks for your interest in FastBit. If your data are already in the format usable by FastBit, then there is function called ibis::part::append that can put two data partitions together. Of course, you can always write your own function to extend an existing data partition if your

Re: [FastBit-users] Bulk Data Loading

ally (on the hard disk) into a single > set of columns or just in memory? > > Regards, > > Teryl > > > On Thu, Feb 25, 2010 at 3:46 PM, K. John Wu <mailto:k...@lbl.gov>> wrote: > > Hi, Teryl, > > Thanks for your interest in FastBit. > >

Re: [FastBit-users] python bindings for FastBit - in progress (and usable)

Hi, Jose, Thanks for taking the initiative to build a python interface for FastBit. Since the C API of FastBit has not been touched for a long while, there are definitely room for some updates and we would be happy to hear your suggestions. On the issue of handling string values, the main fun

Re: [FastBit-users] Missing files in Sourcetree

Hi, Andreas, Thanks for catching this one. I meant to add the files back 2/5 (according the notes in ChangeLog), but somehow neglected to actually to do it... John PS: The actual files are attached just in case you need them now. On 3/3/2010 10:00 AM, Andreas Streichardt wrote: Hi, it

Re: [FastBit-users] tabele::cursor reports wrong results

Hi, Andreas, Here is how I understand the SQL standard, select count(*) ... should return a table with one row and one column, and the content of the single value should be the value of "count(*)". So if you ask how many rows the table object has, it should return 1, similarly for the number

Re: [FastBit-users] tabele::cursor reports wrong results

rch 2010 20:12:51 K. John Wu wrote: >> Hi, Andreas, > > Hi John, > >> Here is how I understand the SQL standard, >> >> select count(*) ... >> >> should return a table with one row and one column, and the content of >> the single value should be the va

Re: [FastBit-users] tabele::cursor reports wrong results

om <https://codeforge.lbl.gov/snapshots.php?group_id=44>, and let us know if you find any additional problems. John PS: Please take a look at the file ChangeLog and see of you are ok with the description of you patch (it is dated 2010-03-03). On 3/3/2010 12:02 PM, K. John Wu wrote: > Hi, An

Re: [FastBit-users] Request to change nrows to COUNT(*)

Hi, Andreas, Guess you would like to see the column name as "count(*)" instead of "nrows", right? I have made this change in all the places where ibis::tabele is created. The change will go into the SVN repository as soon as I have a chance to test it a little bit. I am afraid that you might

Re: [FastBit-users] SELECT statements from a large set

Hi, Teryl, If you want, you could pass a long string to the query object. If you pass the string through a shell (command line or exec), then you may have a problem with how long a command line can be. I don't believe there is a limitation on the string length from Bison/flex generated parse

  1   2   3   4   5   6   7   8   >