Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread Ryan Blenis
Perfect. Thank you so much again!

On Sun, May 2, 2021 at 3:04 PM  wrote:

>
>
> The main search query is equivalent in capabilities with the 'advanced'
> search.
> In fact I personally consider it the preferred way to search. See the
> examples
> at https://www.mailpiler.org/wiki/current:using-the-gui
>
> Also consider the below query how to use it:
>
> a:pdf raw: @subject test email >>> out of office date1:2015.11.18
> date2:2015.11.18
>
> Janos
>
>
> On 2021-05-02 20:44, Ryan Blenis wrote:
> > Ah I see, the "raw: " is in the main search query but not applicable
> > to advanced searches where you can combine the query with dates and /
> > or attachment preferences. Is there any way to utilize it in the
> > advanced search (I would imagine "Raw:" would replace To, From,
> > Subject, and Body- as those are all "MATCH" query participants. (I
> > don't use tags or notes currently, so I'm not sure if they are
> > contained in the MATCH phrase. Since dates and attachments aren't in
> > MATCH (but prior in the query) it would be great to utilize both of
> > them in tandem.
> >
> > On Sun, May 2, 2021 at 2:38 PM Ryan Blenis 
> > wrote:
> >
> >> Hi Janos,
> >>
> >> Thank you! The number of attachments seems to be working perfectly,
> >> the "raw" updates don't seem to work, but maybe I'm misunderstanding
> >> the implementation. There's no new "Raw" field in the advanced
> >> search, so I tried putting "raw: testing" in the Body search field,
> >> but the sphinx query showed "@body raw: testing" versus what was
> >> expected: "testing". Am I misunderstanding the implementation and/or
> >> where to utilize this? Thank you!
> >>
> >> On Sun, May 2, 2021 at 8:28 AM  wrote:
> >>
> >>> On 2021-05-02 09:15, Ryan Blenis wrote:
> >>>
>  Ideally, in the dry run, I'd like to have it print "attachment:
> >>> [id]"
>  much like the emails print "id: [id]" so I can get a count of
> >>> emails
>  and associated attachment numbers, which is often required in
> >>> legal
>  queries as a the "number of items" found for certain search
> >>> terms.
>  (I'm on a slightly older version that only prints "id: [id]" for
> >>> each
>  email found, though I believe you added an email found count at
> >>> the
>  end of the dry run in a newer version, ideally in that case
> >>> there
>  would be a count of emails, and a count of attachments as well).
> >>>
> >>> This commit gives you the total number of attachments at the
> >>> beginning
> >>> in case of dry run:
> >>>
> >>
> >
> https://bitbucket.org/jsuto/piler/commits/f2683962333a741c64920d959bc12e23ed797aa4
> >>>
> >>> Let me know if it meets your expectations, or you need the
> >>> attachments
> >>> for each
> >>> message instead.
> >>>
> >>> Janos
>
>


Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread sj




The main search query is equivalent in capabilities with the 'advanced' 
search.
In fact I personally consider it the preferred way to search. See the 
examples

at https://www.mailpiler.org/wiki/current:using-the-gui

Also consider the below query how to use it:

a:pdf raw: @subject test email >>> out of office date1:2015.11.18 
date2:2015.11.18


Janos


On 2021-05-02 20:44, Ryan Blenis wrote:

Ah I see, the "raw: " is in the main search query but not applicable
to advanced searches where you can combine the query with dates and /
or attachment preferences. Is there any way to utilize it in the
advanced search (I would imagine "Raw:" would replace To, From,
Subject, and Body- as those are all "MATCH" query participants. (I
don't use tags or notes currently, so I'm not sure if they are
contained in the MATCH phrase. Since dates and attachments aren't in
MATCH (but prior in the query) it would be great to utilize both of
them in tandem.

On Sun, May 2, 2021 at 2:38 PM Ryan Blenis 
wrote:


Hi Janos,

Thank you! The number of attachments seems to be working perfectly,
the "raw" updates don't seem to work, but maybe I'm misunderstanding
the implementation. There's no new "Raw" field in the advanced
search, so I tried putting "raw: testing" in the Body search field,
but the sphinx query showed "@body raw: testing" versus what was
expected: "testing". Am I misunderstanding the implementation and/or
where to utilize this? Thank you!

On Sun, May 2, 2021 at 8:28 AM  wrote:


On 2021-05-02 09:15, Ryan Blenis wrote:


Ideally, in the dry run, I'd like to have it print "attachment:

[id]"

much like the emails print "id: [id]" so I can get a count of

emails

and associated attachment numbers, which is often required in

legal

queries as a the "number of items" found for certain search

terms.

(I'm on a slightly older version that only prints "id: [id]" for

each

email found, though I believe you added an email found count at

the

end of the dry run in a newer version, ideally in that case

there

would be a count of emails, and a count of attachments as well).


This commit gives you the total number of attachments at the
beginning
in case of dry run:




https://bitbucket.org/jsuto/piler/commits/f2683962333a741c64920d959bc12e23ed797aa4


Let me know if it meets your expectations, or you need the
attachments
for each
message instead.

Janos




Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread Ryan Blenis
Ah I see, the "raw: " is in the main search query but not applicable to
advanced searches where you can combine the query with dates and / or
attachment preferences. Is there any way to utilize it in the advanced
search (I would imagine "Raw:" would replace To, From, Subject, and Body-
as those are all "MATCH" query participants. (I don't use tags or notes
currently, so I'm not sure if they are contained in the MATCH phrase. Since
dates and attachments aren't in MATCH (but prior in the query) it would be
great to utilize both of them in tandem.

On Sun, May 2, 2021 at 2:38 PM Ryan Blenis  wrote:

> Hi Janos,
>
> Thank you! The number of attachments seems to be working perfectly, the
> "raw" updates don't seem to work, but maybe I'm misunderstanding the
> implementation. There's no new "Raw" field in the advanced search, so I
> tried putting "raw: testing" in the Body search field, but the sphinx query
> showed "@body raw: testing" versus what was expected: "testing". Am I
> misunderstanding the implementation and/or where to utilize this? Thank you!
>
> On Sun, May 2, 2021 at 8:28 AM  wrote:
>
>>
>>
>> On 2021-05-02 09:15, Ryan Blenis wrote:
>>
>> > Ideally, in the dry run, I'd like to have it print "attachment: [id]"
>> > much like the emails print "id: [id]" so I can get a count of emails
>> > and associated attachment numbers, which is often required in legal
>> > queries as a the "number of items" found for certain search terms.
>> > (I'm on a slightly older version that only prints "id: [id]" for each
>> > email found, though I believe you added an email found count at the
>> > end of the dry run in a newer version, ideally in that case there
>> > would be a count of emails, and a count of attachments as well).
>>
>> This commit gives you the total number of attachments at the beginning
>> in case of dry run:
>>
>> https://bitbucket.org/jsuto/piler/commits/f2683962333a741c64920d959bc12e23ed797aa4
>>
>> Let me know if it meets your expectations, or you need the attachments
>> for each
>> message instead.
>>
>>
>> Janos
>>
>>


Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread Ryan Blenis
Hi Janos,

Thank you! The number of attachments seems to be working perfectly, the
"raw" updates don't seem to work, but maybe I'm misunderstanding the
implementation. There's no new "Raw" field in the advanced search, so I
tried putting "raw: testing" in the Body search field, but the sphinx query
showed "@body raw: testing" versus what was expected: "testing". Am I
misunderstanding the implementation and/or where to utilize this? Thank you!

On Sun, May 2, 2021 at 8:28 AM  wrote:

>
>
> On 2021-05-02 09:15, Ryan Blenis wrote:
>
> > Ideally, in the dry run, I'd like to have it print "attachment: [id]"
> > much like the emails print "id: [id]" so I can get a count of emails
> > and associated attachment numbers, which is often required in legal
> > queries as a the "number of items" found for certain search terms.
> > (I'm on a slightly older version that only prints "id: [id]" for each
> > email found, though I believe you added an email found count at the
> > end of the dry run in a newer version, ideally in that case there
> > would be a count of emails, and a count of attachments as well).
>
> This commit gives you the total number of attachments at the beginning
> in case of dry run:
>
> https://bitbucket.org/jsuto/piler/commits/f2683962333a741c64920d959bc12e23ed797aa4
>
> Let me know if it meets your expectations, or you need the attachments
> for each
> message instead.
>
>
> Janos
>
>


Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread sj




OK, this commit gives you the raw: label: 
https://bitbucket.org/jsuto/piler/commits/7b47ce3ca20f9f4ab3234afd6641fe364eaedbc6


Whatever you enter after 'raw:', it will be inside the MATCH('') clause.

Janos

On 2021-05-02 09:15, Ryan Blenis wrote:

Hi Janos,

On Sun, May 2, 2021 at 2:24 AM  wrote:


Hello Ryan,

On 2021-05-02 01:15, Ryan Blenis wrote:


1. Is there a way to have the pilerexport dry run output the

number of

attachments linked to the query?


Just to clarify. Let's say you have 10 matching emails with 15
attachments
total, and you want pilerexport to print 15?


Ideally, in the dry run, I'd like to have it print "attachment: [id]"
much like the emails print "id: [id]" so I can get a count of emails
and associated attachment numbers, which is often required in legal
queries as a the "number of items" found for certain search terms.
(I'm on a slightly older version that only prints "id: [id]" for each
email found, though I believe you added an email found count at the
end of the dry run in a newer version, ideally in that case there
would be a count of emails, and a count of attachments as well).
Thank you!


2. Would it be possible in the web UI to have an "expert"
setting/field where you can manually enter the full sphinx MATCH()
string for more precise results (while still utilizing the other
advanced search fields)? The sphinx query language allows quite a

bit

more flexibility and I'm finding myself using pilerexport's -w

flag to

get a read on match numbers versus querying in the web UI because

of

this limitation (even though you CAN overload the fields to an

extent)

It's possible with a little tweak, I'll add it soon, but it makes
sense
only for auditors. Otherwise it breaks security.


Yes, I completely agree. I'm usually in as an auditor so I almost
forgot about other roles, thank you!


2a. Overloading the web UI search fields with a more complex

query, or

even simply utilizing "quotes" in the field appears to break the
"sphinx" query viewing link. This appears to be just a simple need

for

some escaping on the front-end.


I've fixed the issue with this commit:


https://bitbucket.org/jsuto/piler/commits/dacb7c85aa7654c70f97c9bd4dc80a52913d842a

Awesome! Thank you!


Janos




Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread Ryan Blenis
Hi Janos,

On Sun, May 2, 2021 at 2:24 AM  wrote:

>
>
> Hello Ryan,
>
> On 2021-05-02 01:15, Ryan Blenis wrote:
>
> > 1. Is there a way to have the pilerexport dry run output the number of
> > attachments linked to the query?
>
> Just to clarify. Let's say you have 10 matching emails with 15
> attachments
> total, and you want pilerexport to print 15?
>

Ideally, in the dry run, I'd like to have it print "attachment: [id]" much
like the emails print "id: [id]" so I can get a count of emails and
associated attachment numbers, which is often required in legal queries as
a the "number of items" found for certain search terms. (I'm on a
slightly older version that only prints "id: [id]" for each email found,
though I believe you added an email found count at the end of the dry run
in a newer version, ideally in that case there would be a count of emails,
and a count of attachments as well).
Thank you!


> > 2. Would it be possible in the web UI to have an "expert"
> > setting/field where you can manually enter the full sphinx MATCH()
> > string for more precise results (while still utilizing the other
> > advanced search fields)? The sphinx query language allows quite a bit
> > more flexibility and I'm finding myself using pilerexport's -w flag to
> > get a read on match numbers versus querying in the web UI because of
> > this limitation (even though you CAN overload the fields to an extent)
>
> It's possible with a little tweak, I'll add it soon, but it makes sense
> only for auditors. Otherwise it breaks security.
>

Yes, I completely agree. I'm usually in as an auditor so I almost forgot
about other roles, thank you!

>
> > 2a. Overloading the web UI search fields with a more complex query, or
> > even simply utilizing "quotes" in the field appears to break the
> > "sphinx" query viewing link. This appears to be just a simple need for
> > some escaping on the front-end.
>
> I've fixed the issue with this commit:
>
> https://bitbucket.org/jsuto/piler/commits/dacb7c85aa7654c70f97c9bd4dc80a52913d842a


Awesome! Thank you!

>
>
> Janos
>
>


Re: Pilerexport attachment numbers plus advanced querying

2021-05-02 Thread sj




Hello Ryan,

On 2021-05-02 01:15, Ryan Blenis wrote:


1. Is there a way to have the pilerexport dry run output the number of
attachments linked to the query?


Just to clarify. Let's say you have 10 matching emails with 15 
attachments

total, and you want pilerexport to print 15?


2. Would it be possible in the web UI to have an "expert"
setting/field where you can manually enter the full sphinx MATCH()
string for more precise results (while still utilizing the other
advanced search fields)? The sphinx query language allows quite a bit
more flexibility and I'm finding myself using pilerexport's -w flag to
get a read on match numbers versus querying in the web UI because of
this limitation (even though you CAN overload the fields to an extent)


It's possible with a little tweak, I'll add it soon, but it makes sense
only for auditors. Otherwise it breaks security.


2a. Overloading the web UI search fields with a more complex query, or
even simply utilizing "quotes" in the field appears to break the
"sphinx" query viewing link. This appears to be just a simple need for
some escaping on the front-end.


I've fixed the issue with this commit:
https://bitbucket.org/jsuto/piler/commits/dacb7c85aa7654c70f97c9bd4dc80a52913d842a

Janos



Pilerexport attachment numbers plus advanced querying

2021-05-01 Thread Ryan Blenis
Hi again!

Loving the updates to pilerexport since our last chat. A couple questions
for upgrades based on requests I've had:

1. Is there a way to have the pilerexport dry run output the number of
attachments linked to the query?

2. Would it be possible in the web UI to have an "expert" setting/field
where you can manually enter the full sphinx MATCH() string for more
precise results (while still utilizing the other advanced search fields)?
The sphinx query language allows quite a bit more flexibility and I'm
finding myself using pilerexport's -w flag to get a read on match numbers
versus querying in the web UI because of this limitation (even though you
CAN overload the fields to an extent)
2a. Overloading the web UI search fields with a more complex query, or even
simply utilizing "quotes" in the field appears to break the "sphinx" query
viewing link. This appears to be just a simple need for some escaping on
the front-end.

Thank you as always!