D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

This revision was automatically updated to reflect the committed changes.
Closed by commit R293:e1d1b7e87ff1: Avoid crash when reading corrupt data from 
document terms db (authored by bruns).

REPOSITORY
  R293 Baloo

CHANGES SINCE LAST UPDATE
  https://phabricator.kde.org/D12047?vs=35069=35152

REVISION DETAIL
  https://phabricator.kde.org/D12047

AFFECTED FILES
  src/codecs/doctermscodec.cpp
  src/engine/documentdb.cpp

To: bruns, #baloo, michaelh, ngraham, #frameworks, dhaumann
Cc: dhaumann, kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, 
astippich, spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Dominik Haumann]

dhaumann added inline comments.

INLINE COMMENTS

> documentdb.cpp:101
> +if (result.isEmpty()) {
> +qDebug() << "Document Terms DB contains corrupt data for " << docId;
> +}

Ah, maybe this should be a qWarning()? Feel free to decide as you wish.

REPOSITORY
  R293 Baloo

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks, dhaumann
Cc: dhaumann, kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, 
astippich, spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Dominik Haumann]

dhaumann added a comment.


  Ok, then please commit.

REPOSITORY
  R293 Baloo

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks, dhaumann
Cc: dhaumann, kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, 
astippich, spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a comment.


  In D12047#269976 , @dhaumann wrote:
  
  > If the format is really such that a term must appear before any Suffix, 
then this patch is already better that before.
  >
  > Could it happen to have e.g.: a\0b1c1
  >
  > If so, this code would extend the Suffix b with Suffix c. Would that be 
correct? Or can that never happen? Or should c be a Suffix for a? If so, this 
code should be improved.
  
  
  a\x00b\x01c\x01 would be decoded as "a", "ab", "abc".
  
  "the", "their", "theirs", "there" is encoded as "the\x00ir\x01s\x01there\x00".

REPOSITORY
  R293 Baloo

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks, dhaumann
Cc: dhaumann, kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, 
astippich, spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Dominik Haumann]

dhaumann accepted this revision.
dhaumann added a comment.
This revision is now accepted and ready to land.


  If the format is really such that a term must appear before any Suffix, then 
this patch is already better that before.
  
  Could it happen to have e.g.: a\0b1c1
  
  If so, this code would extend the Suffix b with Suffix c. Would that be 
correct? Or can that never happen? Or should c be a Suffix for a? If so, this 
code should be improved.

REPOSITORY
  R293 Baloo

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks, dhaumann
Cc: dhaumann, kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, 
astippich, spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns updated this revision to Diff 35069.
bruns added a comment.


  Rebase

REPOSITORY
  R293 Baloo

CHANGES SINCE LAST UPDATE
  https://phabricator.kde.org/D12047?vs=31672=35069

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

AFFECTED FILES
  src/codecs/doctermscodec.cpp
  src/engine/documentdb.cpp

To: bruns, #baloo, michaelh, ngraham, #frameworks
Cc: kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, astippich, 
spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a comment.


  If there is noone willing to review, I will push this tomorrow

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks
Cc: kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, astippich, 
spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a comment.
Restricted Application added a subscriber: kde-frameworks-devel.


  Kind request to review ...

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks
Cc: kde-frameworks-devel, #frameworks, ashaposhnikov, michaelh, astippich, 
spoorun, ngraham, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a reviewer: Frameworks.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham, #frameworks
Cc: #frameworks, ashaposhnikov, michaelh, astippich, spoorun, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a comment.


  In D12047#246715 , @michaelh wrote:
  
  > > Corrupt the database
  >
  > As described in BUG: 392877?
  
  
  yes.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham
Cc: #frameworks, ashaposhnikov, michaelh, astippich, spoorun, bruns

D12047: Avoid crash when reading corrupt data from document terms db

[Michael Heidelbach]

michaelh added a comment.


  > Corrupt the database
  
  As described in BUG: 392877?

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham
Cc: #frameworks, ashaposhnikov, michaelh, astippich, spoorun, ngraham, bruns, 
alexeymin

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns added a reviewer: ngraham.

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D12047

To: bruns, #baloo, michaelh, ngraham
Cc: #frameworks, ashaposhnikov, michaelh, astippich, spoorun, ngraham, bruns, 
alexeymin

D12047: Avoid crash when reading corrupt data from document terms db

[Stefan Brüns]

bruns created this revision.
bruns added reviewers: Baloo, michaelh.
Restricted Application added projects: Frameworks, Baloo.
Restricted Application added a subscriber: Frameworks.
bruns requested review of this revision.

REVISION SUMMARY
  The terms db contains terms, where each terms is stored independently
  (terminated with 0), or as a suffix to the previous term (terminated with
  1).
  In case of corrupted data, the first terminator seen may be a 1, which
  leads to a crash when trying to access the previous term with
  QVector<>::last().
  Show a debug message, to give a hint about the bad data, which can be
  fixed by reindexing the relevant file.
  
  BUG: 392878
  CCBUG: 392877

TEST PLAN
  Corrupt the database
  Run balooshow -x 

REPOSITORY
  R293 Baloo

BRANCH
  b392878_avoid_docterms_decode_crash

REVISION DETAIL
  https://phabricator.kde.org/D12047

AFFECTED FILES
  src/codecs/doctermscodec.cpp
  src/engine/documentdb.cpp

To: bruns, #baloo, michaelh
Cc: #frameworks, ashaposhnikov, michaelh, astippich, spoorun, ngraham, bruns, 
alexeymin