I'm wondering if there is any practical limit on the number of columns in a FastBit "table"?
My company is looking at FastBit as a potential backend for an analytics engine. We would have one big table with about 100 columns - 30 of which need to be indexed for searching. We call these the "dimensions". The other 70 - the "metrics" - do not need to be indexed, but might need some aggregation like SUM, AVG, STDEV, etc. The data is going to be pretty sparse - typically 3-4 "dimensions" are populated and 8-12 "metrics" per row, with the rest of the columns set to NULL. Queries will be typically like: SELECT d1, d5, d18, m2, m19, m29 FROM TABLE where d1 between lowerBound and upperBound We are looking at about 100 columns and 100 MILLION rows. A few of the dimension columns (that need to be indexed) contain arbitrary length text values. We could partition the table and decrease the rows to about 10 MILLION if necessary. Does this sound like something that FastBit is well suited for? -- Mark -- Mark Hansen Founder & President Digital Brand Mine | 708 3rd Ave | New York, New York 10017 office: 212-961-7250 cell: 914-924-3398 http://digitalbrandmine.com/ | email: [email protected]
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
