Hi, We are using FastBit to meet one of our indexing need. We are running queries against a single table of 100G size. Further the table is partitioned into 20 parts of 5G each. Each of these tables consists of three rows say a, b, c of type int, float and short respectively. We are creating indices separately using the java API build_index. Using the build_query API, we are submitting the search queries. While retrieving the actual rows that meet the query condition, Fastbit responses are slow after the 6th partition is processed (i.e around 25GB data is processed) The average fetch time for the last partitions is approximately ten times more than the retrieval time of first 6 partitions. Once processing the query results, the query handle is destroyed.
We want to understand this behavior that why the search time shoots up after a certain amount of data is processed by FastBit. And any solution/workaround for the same. Appreciate the help. Many Thanks, Ghousia. _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
