D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd abandoned this revision.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  xattrs are no good for vaults because the xattr is ignored for the 
mountpoint, only restored when the volume is unmounted.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham added a comment.


  So the goal is to be able to turn off indexing for certain files?
  
  Baloo already has a global toggle to stop indexing anything, and you can also 
add folders to the exclude lists.
  
  But stepping back even farther: //why// do we want to turn off indexing for 
certain files? Is it because they might be dangerous? If that's the case, then 
we should work on sandboxing the file extractor, not break the functionality. 
We never intentionally break our software because it might be dangerous; we fix 
the danger, or offer a warning. Our users are adults and prefer to be treated 
as such.
  
  Perhaps we should get some more eyes on D8532: [WIP] Restrict file extractor 
with Seccomp .

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  > No, because even if the default umask is 002, or 022, POSIX says that the 
default permissions of a file on creation are going to be 0666, not 0777:
  
  Trial-and-error proven. :(
  
  In D15718#331449 , @ngraham wrote:
  
  > Let's back up a bit. What's the goal here? What are you trying to 
accomplish?
  
  
  Baloo currently has no method for do-not-index. Perhaps a better way exists 
that prevents new downloads from automatically being indexed and can also 
prevent Baloo vaults from being indexed?
  
  Tracker uses a .trackerignore file. This (or something similar) probably has 
little chance as a default distro safety feature and its usefulness is 
questionable (browsers let you save outside the downloads folder), so this is 
out for the first requirement. This may be a candidate for the second 
requirement.
  
  While this method is already proven, the xdg method of setting tags can fit 
well the second requirement. KDE is sadly lacking in xattr support (no 
'Attributes' tab in file property dialog, xattrs can be silently dropped by KIO 
when moved across filesystems, ...). A variation of the xdg recommendation, 
only recognizing false tags, would also work well.
  
  The first requirement could be solved by ignoring files with no execute bit 
set, though this is opposed. (extra work, etc)

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham added a comment.


  Let's back up a bit. What's the goal here? What are you trying to accomplish?

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  https://www.freedesktop.org/wiki/CommonExtendedAttributes suggests xattrs to 
use to store do-not-index information and also requires each file to be 
explicitly set to be indexed.
  
  > user.xdg.robots.index: On a file: "true" to index, "false" to not index. On 
a directory: "true" to traverse the into the directory for indexing, "false" 
for not traversing into it.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Luigi Toscano]

ltoscano requested changes to this revision.
ltoscano added a comment.


  I agree with all the objections raised so far, and I'm not sure about the 
rationale of this change. I'd like to add one point:
  
  In D15718#330844 , @smithjd wrote:
  
  > In D15718#330836 , @ngraham 
wrote:
  >
  > > Wouldn't this have the effect of un-indexing most files? A quick check of 
my documents (text, word processing, excel, etc) reveals that none of them have 
the execute bit set. As-is, I think this would render Baloo mostly useless.
  >
  >
  > A default mask of 0002 or more permissive looks fairly common across 
distros, and is permissive enough to index files by default.
  
  
  No, because even if the default umask is 002, or 022, POSIX says that the 
default permissions of a file on creation are going to be 0666, not 0777:
  
  
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag_17_01_01_04
  
  So not only file downloaded by browsers won't have the executable bit, but 
*every* file on the system. Let's leave the executable bit on files to scripts 
and other real programs.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo, ltoscano
Cc: ltoscano, marten, bruns, ngraham, kde-frameworks-devel, #baloo, 
ashaposhnikov, michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham added a comment.


  The point is, we don't make changes to our software that break users, and 
then demand that users alter their own files to keep using the software. It's 
not an option--especially if the breakage results in the software basically not 
working at all (i.e. search no longer works because nothing is being indexed 
anymore) or the fix is very invasive (i.e. you need to recursively change the 
permissions on all of your files, probably using terminal commands).
  
  Sorry, no.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: marten, bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, 
michaelh, astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  In D15718#330864 , @ngraham wrote:
  
  > Making files executable that don't need to be executable is a bad security 
habit. What if the contents get replaced with something malicious? Suddenly 
that now-malicious file has execute permissions.
  
  
  Replacing a file's contents requires write permissions on the file. I have 
plenty of executable shell scripts that aren't an immediate security risk, 
though I suppose if someone gained write privileges over my home statistically 
speaking a shell script is (currently) the most likely choice to gut and 
replace with malicious code. If an attacker already has write permission over 
your home you have bigger problems than a forgotten set-executable file in it 
somewhere anyway. The patched state of that machine's software packages 
dictates how devastating that payload was to your administrator, meanwhile your 
home has probably been wiped.
  
  > 
  > 
  >  ---
  > 
  > Conceptually, you are proposing that the rest of the world adapt to our 
software, rather than the other way around. That's simply not practical. Even 
if this were a good idea, the world will never adapt to us. We must adapt to 
the world. Our software does not exist in a perfect state of total control over 
the environment it inhabits; it exists to facilitate busy people with messy 
lives as they work to accomplish their tasks with a minimum of hassle. That 
goal is not enhanced by breaking KDE Plasma's search tool for them unless they 
give all of their files execute permissions.
  > 
  > Sorry, no go. We need to find a better way.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham added a comment.


  Making files executable that don't need to be executable is a bad security 
habit. What if the contents get replaced with something malicious? Suddenly 
that now-malicious file has execute permissions.
  
  ---
  
  Conceptually, you are proposing that the rest of the world adapt to our 
software, rather than the other way around. That's simply not practical. Even 
if this were a good idea, the world will never adapt to us. We must adapt to 
the world. Our software does not exist in a perfect state of total control over 
the environment it inhabits; it exists to facilitate busy people with messy 
lives as they work to accomplish their tasks with a minimum of hassle. That 
goal is not enhanced by breaking KDE Plasma's search tool for them unless they 
give all of their files execute permissions.
  
  Sorry, no go. We need to find a better way.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  In D15718#330845 , @ngraham wrote:
  
  > In D15718#330844 , @smithjd 
wrote:
  >
  > > In D15718#330836 , @ngraham 
wrote:
  > >
  > > > Wouldn't this have the effect of un-indexing most files? A quick check 
of my documents (text, word processing, excel, etc) reveals that none of them 
have the execute bit set. As-is, I think this would render Baloo mostly useless.
  > >
  > >
  > > A default mask of 0002 or more permissive looks fairly common across 
distros, and is permissive enough to index files by default.
  > >
  > > setfacl -d -m u::rwx ~ or umask 0022 will set default execute permissions 
on created files. You can set something less permissive on your downloads 
directory or plasma vault mount with setfacl -d u::rw or similar.
  > >
  > > chmod -R 755 ~ will recursively give every file in your home directory 
execute permisions.
  >
  >
  > You are proposing fundamentally breaking Baloo and then requiring that 
users or distros clean up our mess for us by making all their files 
executable--in the process reducing their own security. Sorry, but this is 
simply unacceptable.
  >
  > No part of this proposal makes any sense to me. We just can't do this.
  
  
  Folders with the execute bit off already won't allow their contents listed. 
How wouldn't this convention be useful extended to prevent a file from being 
listed in Baloo's database?
  
  How does setting files executable lead to reduced security? Some common 
folders may end up needing more restrictive defaults (downloads for instance) 
and the problem of potentially indexing sensitive information or feeding an 
extractor a specially crafted file designed to exploit a bug in it will have 
been mediated to some extent.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham added a comment.


  In D15718#330844 , @smithjd wrote:
  
  > In D15718#330836 , @ngraham 
wrote:
  >
  > > Wouldn't this have the effect of un-indexing most files? A quick check of 
my documents (text, word processing, excel, etc) reveals that none of them have 
the execute bit set. As-is, I think this would render Baloo mostly useless.
  >
  >
  > A default mask of 0002 or more permissive looks fairly common across 
distros, and is permissive enough to index files by default.
  >
  > setfacl -d -m u::rwx ~ or umask 0022 will set default execute permissions 
on created files. You can set something less permissive on your downloads 
directory or plasma vault mount with setfacl -d u::rw or similar.
  >
  > chmod -R 755 ~ will recursively give every file in your home directory 
execute permisions.
  
  
  You are proposing fundamentally breaking Baloo and then requiring that users 
or distros clean up our mess for us by making all their files executable--in 
the process reducing their own security. Sorry, but this is simply unacceptable.
  
  No part of this proposal makes any sense to me. We just can't do this.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  In D15718#330836 , @ngraham wrote:
  
  > Wouldn't this have the effect of un-indexing most files? A quick check of 
my documents (text, word processing, excel, etc) reveals that none of them have 
the execute bit set. As-is, I think this would render Baloo mostly useless.
  
  
  A default mask of 0002 or more permissive looks fairly common across distros, 
and is permissive enough to index files by default.
  
  setfacl -d -m u::rwx ~ or umask 0022 will set default execute permissions on 
created files. You can set something less permissive on your downloads 
directory or plasma vault mount with setfacl -d u::rw or similar.
  
  chmod -R 755 ~ will recursively give every file in your home directory 
execute permisions.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham requested changes to this revision.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Stefan Brüns]

bruns added a comment.


  This is completely wrong ...

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Stefan Brüns]

bruns added a reviewer: Baloo.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham, #baloo
Cc: bruns, ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, 
astippich, spoorun, abrahams

D15718: Do not index the path if the path has no execute permissions.

[Nathaniel Graham]

ngraham requested changes to this revision.
ngraham added a comment.
This revision now requires changes to proceed.


  Wouldn't this have the effect of un-indexing most files? A quick check of my 
documents (text, word processing, excel, etc) reveals that none of them have 
the execute bit set. As-is, I think this would render Baloo mostly useless.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd, ngraham
Cc: ngraham, kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, astippich, 
spoorun, bruns, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd added a comment.


  Files/folders should not be automatically indexed if the execute bit is 
unset. Downloads from most if not all popular browsers are not executable by 
default.
  
  Setting a file unexecutable would be used to prevent automatic indexing of 
files and most importantly prevent immediate indexing of newly downloaded files 
and indexing of plasma vaults .
  
  Inspiration:
  
  https://phabricator.kde.org/T8066
  https://phabricator.kde.org/D8532

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D15718

To: smithjd
Cc: kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, astippich, spoorun, 
ngraham, bruns, abrahams

D15718: Do not index the path if the path has no execute permissions.

[James Smith]

smithjd created this revision.
Herald added projects: Frameworks, Baloo.
Herald added subscribers: Baloo, kde-frameworks-devel.
smithjd requested review of this revision.

REVISION SUMMARY
  This will prevent automatic indexing of newly downloaded files and can be 
used to 
  prevent indexing of Plasma vaults.

REPOSITORY
  R293 Baloo

BRANCH
  master-donotindex (branched from master)

REVISION DETAIL
  https://phabricator.kde.org/D15718

AFFECTED FILES
  src/file/fileindexerconfig.cpp
  src/file/filtereddiriterator.cpp
  src/file/modifiedfileindexer.cpp
  src/file/newfileindexer.cpp
  src/file/unindexedfileiterator.cpp
  src/file/xattrindexer.cpp

To: smithjd
Cc: kde-frameworks-devel, #baloo, ashaposhnikov, michaelh, astippich, spoorun, 
ngraham, bruns, abrahams