On Wed, Jul 20, 2011 at 7:51 PM, Abhinav Upadhyay <[email protected]> wrote: > Hi, > > I have an FTS table with compress and uncompress options enabled. I am > using zlib(3) for doing the compression. The compression function > seems to be doing ok as I can see the size of the database coming down > drastically. But I the uncompress function is not working properly. > > For example if I do a query like: "select snippet(my_fts_table) from > my_fts_table where my_fts_table match 'my query'" , it should only > output the snippets of the individual matching documents. But instead > it is sending the complete document data to the output. I am not sure > if it is the fault of my uncompress function or a bug in sqlite so > showing my compress/uncompress code here. Hope to get some insight. > > Thanks > > Following are the functions that I have written for compress and uncompress: > > /* the compress function */ > void > zip(sqlite3_context *pctx, int nval, sqlite3_value **apval) > { > const Bytef *source = sqlite3_value_text(apval[0]); > uLong sourcelen = strlen((const char *)source); > uLong destlen = (sourcelen + 12) + (int)(sourcelen + 12) * .01/100; > Bytef *dest = (Bytef *) malloc(sizeof(Bytef) * destlen); > int ret_val = compress(dest, &destlen, source, sourcelen); > if (ret_val != Z_OK) { > sqlite3_result_error(pctx, "Error in compression", -1); > } > sqlite3_result_text(pctx, (const char *)dest, -1, NULL); > return; > } > > /* the uncompress function */ > > > void > unzip(sqlite3_context *pctx, int nval, sqlite3_value **apval) > { > int ret; > unsigned int have; > z_stream strm; > size_t bufsize; > unsigned char *in; > unsigned char out[CHUNK]; > unsigned char *dest = NULL; > strm.zalloc = Z_NULL; > strm.zfree = Z_NULL; > strm.opaque = Z_NULL; > strm.next_in = NULL; > strm.avail_in = 0; > > ret = inflateInit(&strm); > if (ret != Z_OK) { > fprintf(stderr, "Could not initiate decompression of > database\n"); > sqlite3_result_error(pctx, "Error in decompression", -1); > return; > } > > in = (unsigned char *) sqlite3_value_text(apval[0]); > bufsize = strlen((const char *) in); > strm.next_in = in; > strm.avail_in = bufsize; > > /* run inflate() on input until output buffer is not full */ > do { > strm.avail_out = CHUNK; > strm.next_out = out; > ret = inflate(&strm, Z_NO_FLUSH); > > have = CHUNK - strm.avail_out; > if (concat((char **) &dest, (const char *) strm.next_out) < 0) > { > fprintf(stderr, "Error in concatnating decompressed > stream\n"); > sqlite3_result_error(pctx, "Error in decompression", > -1); > return; > } > } while (strm.avail_out == 0); > > inflateEnd(&strm); > sqlite3_result_text(pctx, (const char *)dest, -1, NULL); > return; > } > > // concat is a utility function that concatenates its 2nd parameter at > the end of its Ist parameter.It returns a negative value on error. >
Just to add to what I said before. On executing a query from within my application it is sending the complete indexed data of the matched documents. So actually, first it sends this output to stderr, and then the same output to stdoutput. This I observed just a while ago. I am pretty sure I am doing something wrong in the decompression function but not sure what. :/ Thanks Abhinav _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
