Hi Janos, Thanks, that led me to what is causing the issue / confusion.
The -w switch is described as "Where condition to pass to sphinx, eg. "match('@subject: piler')" Which led me to believe the MATCH string was all that was supposed to be there/replaced, however a quick look at the code shows that if -w is used, it REPLACES the ENTIRE where clause. This distinction means that the use of -w negates the use of the a, b, and r switches as those parameters no longer go through your query builder. I was getting more results because it wasn't limited to a timeframe or to a recipient due to those flags being "skipped". The simplest workaround to this for others would be to note that -w allows you to build your own query and negates the use of other parameters. The ideal fix I think would be to still utilize the other parameters, but have -w content appended within the MATCH() portion of the query. Aside from that: I realize I'm behind on piler (1.3.8), and would like to update to get the latest pilerexport with zip capabilities, yet I see there is no upgrade information on https://www.mailpiler.org/wiki/current:upgrade . What is the process to the latest (1.3.11)? I'd also like to add a "--num-only" type flag to pilerexport to see the number of matches before exporting (would probably imply dryrun). I didn't see a way to do something like that already. If that's something that could be added, great, if not, I'll try my hand at it and submit a patch. When trying to just compile the latest, I get the error "error: cannot connect to 127.0.0.1:9306" so I'm not sure if that's an issue because not all the components are upgraded, or if I had a different configure flag/path configured during the original install. On Wed, Apr 7, 2021 at 5:19 PM Ryan Blenis <ryan.ble...@gmail.com> wrote: > Disregard that last email. Coffee is good, not re-running ./configure > after installing deps is bad. Following up shortly with more pertinent > info. Thank you. > > On Wed, Apr 7, 2021 at 3:58 PM Ryan Blenis <ryan.ble...@gmail.com> wrote: > >> Hi Janos, >> >> Thanks for the response, in trying to do this (I cloned the repo, >> ./configure --localstatedir=/var --with-database=mariadb , and ran make) >> and got this: >> >> Making all in src >> make: Entering directory '/tmp/piler/src/piler/src' >> gcc -std=c99 -O2 -fPIC -Wall -Wextra -Wimplicit-fallthrough=2 >> -Wuninitialized -Wno-format-truncation -g -I. -I.. -I/usr/include/mariadb >> -I/usr/include/mariadb/mysql -D_GNU_SOURCE -DHAVE_TRE -DNEED_MYSQL -o >> pilerexport pilerexport.c -lpiler -lz -lm -ldl -lcrypto -lssl -ltre >> -L/usr/lib/x86_64-linux-gnu/ -lmariadb -L. >> /usr/bin/ld: /tmp/ccU39C8h.o: in function `write_to_zip_file': >> /tmp/piler/src/piler/src/pilerexport.c:329: undefined reference to >> `zip_open' >> /usr/bin/ld: /tmp/piler/src/piler/src/pilerexport.c:335: undefined >> reference to `zip_source_file' >> /usr/bin/ld: /tmp/piler/src/piler/src/pilerexport.c:336: undefined >> reference to `zip_file_add' >> /usr/bin/ld: /tmp/piler/src/piler/src/pilerexport.c:342: undefined >> reference to `zip_close' >> /usr/bin/ld: /tmp/piler/src/piler/src/pilerexport.c:339: undefined >> reference to `zip_strerror' >> collect2: error: ld returned 1 exit status >> make: *** [Makefile:63: pilerexport] Error 1 >> make: Leaving directory '/tmp/piler/src/piler/src' >> make: *** [Makefile:41: all-recursive] Error 1 >> >> (Note that I originally got a zip.h not found error, which I ran apt >> install libzip-dev. Ubuntu 20.04.2 LTS >> >> I can't seem to get past this point to recompile. >> >> >> On Wed, Apr 7, 2021 at 2:50 PM <s...@acts.hu> wrote: >> >>> >>> Hello Ryan, >>> >>> please apply this patch to pilerexport.c, and recompile it. >>> >>> https://bitbucket.org/jsuto/piler/commits/e6607b0bf1d44562bcf2a08e3bfed94181b7b95d >>> >>> It syslogs the sphinx query. Then try the following. Enter the search >>> query >>> on the gui, and record the sphinx query syslogged. Then re-run the >>> pilerexport command, and record the new sphinx query, and compare it >>> with the previous value. >>> >>> Verify that even the single-quotes and double quotes are the same in >>> both queries. >>> >>> Janos SUTO >>> >>> >>> On 2021-04-07 18:18, Ryan Blenis wrote: >>> > Hi Janos, >>> > >>> > I have to export potentially a ton of emails and was looking to use >>> > pilerexport versus multiple batches of GUI searches. I saw the -w flag >>> > and thought "great, I can use this" but it doesn't seem to respond >>> > appropriately for my test case. I have 2 emails that match the >>> > following (generalized terms used vs actual), limiting with -m 3 for >>> > testing purposes (I should only get 2 back). >>> > >>> > pilerexport -a 2010.10.01 -b 2021.04.06 -r "j...@domain.com" -m 3 -w >>> > 'MATCH('"'"'searchterm NEAR/25 (MNF|(search term)|term|(test search >>> > term)|termin*)'"'"')' >>> > >>> > Now, that match is just the bash string escaped version of: >>> > MATCH('searchterm NEAR/25 (MNF|(search term)|term|(test search >>> > term)|termin*)') >>> > (That's just a fancy sphinx query for "searchterm" within 25 words of >>> > MNF OR "search term" OR "term" OR "test search term" or "termin*" for >>> > those unfamiliar with sphinx.) >>> > >>> > Which, when overloading the Advanced Search for the "body" field in >>> > the GUI with: >>> > searchterm NEAR/25 (MNF|(search term)|term|(test search term)|termin*) >>> > >>> > Seems to work just fine and as expected, however, in pilerexport with >>> > the aforementioned command I get tons of unrelated emails (not even >>> > scoped to the appropriate j...@domain.com recipient). Is using a MATCH >>> > term like this with -w possible, or am I looking to do too much here? >>> > >>> > Note that I saw you added the -o parameter in the source so I may be a >>> > version or 2 back (utility doesn't seem to have a -v or --version >>> > output), and my version doesn't appear to have that, so I don't really >>> > have any great diagnostic/output information to go off of other than >>> > the above description. >>> > >>> > Thank you in advance as always for any insight you can give! >>> >>