I know I was supposed to commit this a long time back, but I just can't get
it to work! After some amount of debugging I realized my code was fine, and
the problem is that KDirWatch is buggy (I don't have the bandwidth to fix
it! Do I report it at bugs.kde?). There are 2 problems with KDirWatch -
1. It's slow - I block all the signals, and write to the config file, and
then unblock the signals. A little while after that the dirty() signal is
emitted! I decided to hack through this issue by ... look at strigi2.diff ->
StrigiService::slotConfigChanged(). And that's when I discovered the second
bug.
2. It doesn't always fire the dirty signal - Yes. I know. Weird! (If you
feel inclined to test it, apply the KDirWatchPatch to kdelibs/kdecore)
Possible fixes (in order of preference) -
1. Somebody fixes KDirWatch
2. We use a separate config file to store the "config changed" variable
3. We a use a QTimer::singleShot(...) to unblock the signals after a certain
interval. (Not fool proof. I've tried it)
- Vishesh Handa
On Tue, May 11, 2010 at 11:15 AM, Sebastian Trüg <[email protected]> wrote:
> Hi Vishesh,
>
> very nice. Please commit. Except if the blockSignals does not work. Does
> it work now?
>
> Only a minor hint: in general I would recommend to use
>
> Q_FOREACH( const QString& s, strings )
>
> since that avoids a copy of the string. Now in case of strings this is
> no big deal (shared) but it makes sense to remember it for other types.
>
> Cheers,
> Sebastian
>
> On 05/11/2010 01:08 AM, Vishesh Handa wrote:
> > Hi Sebastian
> >
> > How to block signals:
> > 1. Use QObject::blockSignals()
> >
> >
> > Didn't work! :-( I've seen the disconnect/reconnect approach being used
> > successfully as well. There must be something special about this case.
> > I'll check it out tom.
> >
> > * There is no real need to optimize the folder list in the filewatch
> > service. StrigiServiceConfig already does that before giving the list
> to
> > the IndexScheduler
> >
> >
> > I thought it's better to optimize it once before saving it rather than
> > optimize it every time while loading. Anyway, I've removed the
> > optimization code. It's now redundant.
> >
> >
> > * I think we need a better name than "restart". Maybe "ignore last
> > config change" or something like that.
> >
> >
> > I've changed it to "ignore changes". Is that okay?
> >
> >
> > * Any chance the config handling in IndexScheduler could be moved to
> > StrigiServiceConfig? Just so config handling is restricted to the
> > one class.
> >
> >
> > Done.
> >
> >
> > * Now for the only critical use case: imagine filewatch changes the
> > config, then the kcm changes the folders, and only then strigi gets
> to
> > the config. I am not sure if that can even happen but if so strigi
> would
> > ignore the new config. But I suppose this is very unlikely to happen
> or
> > even impossible.
> >
> >
> > The probability of this happening is very low. However I've edited KCM
> > to make sure it doesn't occur.
> >
> > I've modified the patch to remove auto adding of sub folders to indexed
> > list as that is what everyone wanted. (I don't!)
> >
> > - Vishesh Handa
> >
>
Index: services/strigi/indexscheduler.cpp
===================================================================
--- services/strigi/indexscheduler.cpp (revision 1125616)
+++ services/strigi/indexscheduler.cpp (working copy)
@@ -129,9 +129,6 @@ Nepomuk::IndexScheduler::IndexScheduler(
m_speed( FullSpeed )
{
m_analyzerConfig = new StoppableConfiguration;
-
- connect( StrigiServiceConfig::self(), SIGNAL( configChanged() ),
- this, SLOT( slotConfigChanged() ) );
}
@@ -487,14 +484,6 @@ void Nepomuk::IndexScheduler::queueAllFo
}
-void Nepomuk::IndexScheduler::slotConfigChanged()
-{
- // restart to make sure we update all folders and removeOldAndUnwantedEntries
- if ( isRunning() )
- restart();
-}
-
-
namespace {
class QDataStreamStrigiBufferedStream : public Strigi::BufferedStream<char>
{
Index: services/strigi/indexscheduler.h
===================================================================
--- services/strigi/indexscheduler.h (revision 1125616)
+++ services/strigi/indexscheduler.h (working copy)
@@ -172,9 +172,6 @@ namespace Nepomuk {
void indexingFolder( const QString& );
void indexingSuspended( bool suspended );
- private Q_SLOTS:
- void slotConfigChanged();
-
private:
void run();
Index: services/strigi/strigiservice.cpp
===================================================================
--- services/strigi/strigiservice.cpp (revision 1125616)
+++ services/strigi/strigiservice.cpp (working copy)
@@ -60,9 +60,9 @@ Nepomuk::StrigiService::StrigiService( Q
// monitor all kinds of events
( void )new EventMonitor( m_indexScheduler, this );
- // update the watches if the config changes
+ // watch for config changes
connect( StrigiServiceConfig::self(), SIGNAL( configChanged() ),
- this, SLOT( updateWatches() ) );
+ this, SLOT( slotConfigChanged() ) );
// export on dbus
( void )new StrigiServiceAdaptor( this );
@@ -129,6 +129,33 @@ void Nepomuk::StrigiService::finishIniti
}
+void Nepomuk::StrigiService::slotConfigChanged()
+{
+ KConfig config( "nepomukstrigirc" );
+ bool shouldRestart = !config.group( "General" ).readEntry( "ignore changes", false );
+
+ if( shouldRestart ) {
+ // restart the index scheduler to update all folders and removeOldAndUnwantedEntries
+ m_indexScheduler->restart();
+ updateWatches();
+ }
+ else {
+ // The metadata mover updated the strigiconfig file because some folders were moved.
+ // We donot want to restart, as the filewatcher is still updating the metadata and
+ // IndexScheduler::removeOldAndUnwantedEntries would cause certain not yet
+ // updated entries to be deleted.
+
+ StrigiServiceConfig::self()->blockSignals( true );
+
+ //Update the strigi config as don't want it ignore changes any longer.
+ config.group( "General" ).writeEntry( "ignore changes", false );
+ config.sync();
+
+ StrigiServiceConfig::self()->blockSignals( false );
+ }
+}
+
+
void Nepomuk::StrigiService::updateWatches()
{
org::kde::nepomuk::FileWatch filewatch( "org.kde.nepomuk.services.nepomukfilewatch",
Index: services/strigi/strigiservice.h
===================================================================
--- services/strigi/strigiservice.h (revision 1125616)
+++ services/strigi/strigiservice.h (working copy)
@@ -64,6 +64,7 @@ namespace Nepomuk {
private Q_SLOTS:
void finishInitialization();
void updateWatches();
+ void slotConfigChanged();
private:
void updateStrigiConfig();
Index: services/filewatch/nepomukfilewatch.cpp
===================================================================
--- services/filewatch/nepomukfilewatch.cpp (revision 1125616)
+++ services/filewatch/nepomukfilewatch.cpp (working copy)
@@ -28,11 +28,15 @@
#include <QtCore/QDir>
#include <QtCore/QThread>
+#include <QtCore/QRegExp>
#include <QtDBus/QDBusConnection>
#include <KDebug>
#include <KUrl>
#include <KPluginFactory>
+#include <KDirWatch>
+#include <KConfigGroup>
+#include <KStandardDirs>
#include <Nepomuk/ResourceManager>
@@ -140,6 +144,19 @@ void Nepomuk::FileWatch::watchFolder( co
KInotify::WatchEvents( KInotify::EventMove|KInotify::EventDelete|KInotify::EventDeleteSelf|KInotify::EventCreate ),
KInotify::WatchFlags() );
#endif
+
+ //Tracking Strigi indexed folder list
+
+ KDirWatch* dirWatch = KDirWatch::self();
+ connect( dirWatch, SIGNAL( dirty( const QString& ) ),
+ this, SLOT( slotStrigiConfigChanged() ) );
+ connect( dirWatch, SIGNAL( created( const QString& ) ),
+ this, SLOT( slotStrigiConfigChanged() ) );
+
+ KConfig config( "nepomukstrigirc" );
+ dirWatch->addFile( KStandardDirs::locateLocal( "config", config.name() ) );
+
+ slotStrigiConfigChanged();
}
@@ -151,6 +168,8 @@ void Nepomuk::FileWatch::slotFileMoved(
kDebug() << from << to;
m_metadataMover->moveFileMetadata( from, to );
+
+ handleStrigiIndexedFolder( urlFrom, urlTo );
}
@@ -202,4 +221,44 @@ void Nepomuk::FileWatch::slotInotifyWatc
}
#endif
+
+void Nepomuk::FileWatch::slotStrigiConfigChanged()
+{
+ m_strigiFoldersHash.clear();
+ KConfig config( "nepomukstrigirc" );
+
+ QStringList include = config.group( "General" ).readPathEntry( "folders", QStringList() );
+ foreach( QString path, include ) {
+ m_strigiFoldersHash.insert( path, true );
+ }
+
+ QStringList exclude = config.group( "General" ).readPathEntry( "exclude folders", QStringList() );
+ foreach( QString path, exclude ) {
+ m_strigiFoldersHash.insert( path, false );
+ }
+}
+
+
+void Nepomuk::FileWatch::handleStrigiIndexedFolder( const QString& urlFrom, const QString& urlTo )
+{
+ if( !m_strigiFoldersHash.contains( urlFrom ) )
+ return;
+
+ KConfig config( "nepomukstrigirc" );
+
+ // We want to select urlFrom and all UrlFrom/something but not UrlFromSomething.
+ // Therefore the trailing slash and everything after that is optional (/.*)?
+ QRegExp regex( urlFrom + "(/.*)?$" );
+
+ QStringList includeFolders = config.group( "General" ).readPathEntry( "folders", QStringList() );
+ includeFolders.replaceInStrings( regex, urlTo + "\\1" );
+
+ QStringList excludeFolders = config.group( "General" ).readPathEntry( "exclude folders", QStringList() );
+ excludeFolders.replaceInStrings( regex, urlTo + "\\1" );
+
+ config.group( "General" ).writePathEntry( "folders", includeFolders );
+ config.group( "General" ).writePathEntry( "exclude folders", excludeFolders );
+ config.group( "General" ).writeEntry( "ignore changes", true );
+}
+
#include "nepomukfilewatch.moc"
Index: services/filewatch/nepomukfilewatch.h
===================================================================
--- services/filewatch/nepomukfilewatch.h (revision 1125616)
+++ services/filewatch/nepomukfilewatch.h (working copy)
@@ -23,6 +23,7 @@
#include <QtCore/QUrl>
#include <QtCore/QVariant>
+#include <QtCore/QHash>
namespace Soprano {
class Model;
@@ -60,9 +61,12 @@ namespace Nepomuk {
#ifdef BUILD_KINOTIFY
void slotInotifyWatchUserLimitReached();
#endif
+ void slotStrigiConfigChanged();
private:
MetadataMover* m_metadataMover;
+ QHash<QString, bool> m_strigiFoldersHash;
+ void handleStrigiIndexedFolder( const QString& from, const QString& to );
#ifdef BUILD_KINOTIFY
KInotify* m_dirWatch;
Index: kcm/nepomukserverkcm.cpp
===================================================================
--- kcm/nepomukserverkcm.cpp (revision 1125616)
+++ kcm/nepomukserverkcm.cpp (working copy)
@@ -223,7 +223,7 @@ void Nepomuk::ServerConfigModule::save()
strigiConfig.group( "General" ).writeEntry( "exclude filters", m_editStrigiExcludeFilters->items() );
strigiConfig.group( "General" ).writeEntry( "index hidden folders", m_checkShowHiddenFolders->isChecked() );
strigiConfig.group( "General" ).writeEntry( "index newly mounted", m_checkIndexRemovableMedia->isChecked() );
-
+ strigiConfig.group( "General" ).writeEntry( "ignore changes", false );
// 3. update the current state of the nepomuk server
if ( m_serverInterface.isValid() ) {
Index: services/strigi/indexscheduler.cpp
===================================================================
--- services/strigi/indexscheduler.cpp (revision 1125616)
+++ services/strigi/indexscheduler.cpp (working copy)
@@ -129,9 +129,6 @@ Nepomuk::IndexScheduler::IndexScheduler(
m_speed( FullSpeed )
{
m_analyzerConfig = new StoppableConfiguration;
-
- connect( StrigiServiceConfig::self(), SIGNAL( configChanged() ),
- this, SLOT( slotConfigChanged() ) );
}
@@ -487,14 +484,6 @@ void Nepomuk::IndexScheduler::queueAllFo
}
-void Nepomuk::IndexScheduler::slotConfigChanged()
-{
- // restart to make sure we update all folders and removeOldAndUnwantedEntries
- if ( isRunning() )
- restart();
-}
-
-
namespace {
class QDataStreamStrigiBufferedStream : public Strigi::BufferedStream<char>
{
Index: services/strigi/indexscheduler.h
===================================================================
--- services/strigi/indexscheduler.h (revision 1125616)
+++ services/strigi/indexscheduler.h (working copy)
@@ -172,9 +172,6 @@ namespace Nepomuk {
void indexingFolder( const QString& );
void indexingSuspended( bool suspended );
- private Q_SLOTS:
- void slotConfigChanged();
-
private:
void run();
Index: services/strigi/strigiservice.cpp
===================================================================
--- services/strigi/strigiservice.cpp (revision 1125616)
+++ services/strigi/strigiservice.cpp (working copy)
@@ -60,9 +60,9 @@ Nepomuk::StrigiService::StrigiService( Q
// monitor all kinds of events
( void )new EventMonitor( m_indexScheduler, this );
- // update the watches if the config changes
+ // watch for config changes
connect( StrigiServiceConfig::self(), SIGNAL( configChanged() ),
- this, SLOT( updateWatches() ) );
+ this, SLOT( slotConfigChanged() ) );
// export on dbus
( void )new StrigiServiceAdaptor( this );
@@ -129,6 +129,57 @@ void Nepomuk::StrigiService::finishIniti
}
+void Nepomuk::StrigiService::slotConfigChanged()
+{
+ KConfig config("nepomukstrigirc");
+ bool shouldRestart = !config.group( "General" ).readEntry( "ignore changes", false );
+ kDebug() << "Should restart : " << shouldRestart;
+
+ if( shouldRestart ) {
+ // restart the index scheduler to update all folders and removeOldAndUnwantedEntries
+ m_indexScheduler->restart();
+ updateWatches();
+ }
+ else {
+ // The metadata mover updated the strigiconfig file because some folders were moved.
+ // We donot want to restart, as the filewatcher is still updating the metadata and
+ // IndexScheduler::removeOldAndUnwantedEntries would cause certain not yet
+ // updated entries to be deleted.
+
+ kDebug() << "Disconnecting signals";
+ disconnect( StrigiServiceConfig::self(), SIGNAL(configChanged()),
+ this, SLOT(slotConfigChanged()) );
+
+ // StrigiServiceConfig's configChanged signal is implemented via KDirWatch.
+ // KDirWatch is a little slow, and therefore the normal disconnect/connect signal
+ // approach doesn't work. This way the slot resetStrigiServiceConfigSignals()
+ // handles the unnecessary extra configChanged signal.
+ connect( StrigiServiceConfig::self(), SIGNAL(configChanged()),
+ this, SLOT(resetStrigiServiceConfigSignals()) );
+
+ // Update the strigi config as don't want it ignore changes any longer.
+ // This will trigger the resetStrigiServiceConfigSignals() slot.
+ config.group( "General" ).writeEntry( "ignore changes", false );
+ config.sync();
+ kDebug() << "Should emit dirty signal now!";
+ }
+}
+
+
+void Nepomuk::StrigiService::resetStrigiServiceConfigSignals()
+{
+ kDebug() << "Handling the extra signal";
+
+ // Ignore this configChanged signal, and forward all further configChanged
+ // signals to slotConfigChanged.
+ disconnect( StrigiServiceConfig::self(), SIGNAL(configChanged()),
+ this, SLOT(resetStrigiServiceConfigSignals()) );
+
+ connect( StrigiServiceConfig::self(), SIGNAL(configChanged()),
+ this, SLOT(slotConfigChanged()) );
+}
+
+
void Nepomuk::StrigiService::updateWatches()
{
org::kde::nepomuk::FileWatch filewatch( "org.kde.nepomuk.services.nepomukfilewatch",
Index: services/strigi/strigiservice.h
===================================================================
--- services/strigi/strigiservice.h (revision 1125616)
+++ services/strigi/strigiservice.h (working copy)
@@ -64,9 +64,10 @@ namespace Nepomuk {
private Q_SLOTS:
void finishInitialization();
void updateWatches();
+ void slotConfigChanged();
+ void resetStrigiServiceConfigSignals();
private:
- void updateStrigiConfig();
QString userStatusString( bool simple ) const;
Strigi::IndexManager* m_indexManager;
Index: services/filewatch/nepomukfilewatch.cpp
===================================================================
--- services/filewatch/nepomukfilewatch.cpp (revision 1125616)
+++ services/filewatch/nepomukfilewatch.cpp (working copy)
@@ -28,11 +28,15 @@
#include <QtCore/QDir>
#include <QtCore/QThread>
+#include <QtCore/QRegExp>
#include <QtDBus/QDBusConnection>
#include <KDebug>
#include <KUrl>
#include <KPluginFactory>
+#include <KDirWatch>
+#include <KConfigGroup>
+#include <KStandardDirs>
#include <Nepomuk/ResourceManager>
@@ -140,6 +144,19 @@ void Nepomuk::FileWatch::watchFolder( co
KInotify::WatchEvents( KInotify::EventMove|KInotify::EventDelete|KInotify::EventDeleteSelf|KInotify::EventCreate ),
KInotify::WatchFlags() );
#endif
+
+ //Tracking Strigi indexed folder list
+
+ KDirWatch* dirWatch = KDirWatch::self();
+ connect( dirWatch, SIGNAL( dirty( const QString& ) ),
+ this, SLOT( slotStrigiConfigChanged() ) );
+ connect( dirWatch, SIGNAL( created( const QString& ) ),
+ this, SLOT( slotStrigiConfigChanged() ) );
+
+ KConfig config("nepomukstrigirc");
+ dirWatch->addFile( KStandardDirs::locateLocal( "config", config.name() ) );
+
+ slotStrigiConfigChanged();
}
@@ -151,6 +168,8 @@ void Nepomuk::FileWatch::slotFileMoved(
kDebug() << from << to;
m_metadataMover->moveFileMetadata( from, to );
+
+ handleStrigiIndexedFolder( urlFrom, urlTo );
}
@@ -202,4 +221,44 @@ void Nepomuk::FileWatch::slotInotifyWatc
}
#endif
+
+void Nepomuk::FileWatch::slotStrigiConfigChanged()
+{
+ m_strigiFoldersHash.clear();
+ KConfig config("nepomukstrigirc");
+
+ QStringList include = config.group( "General" ).readPathEntry( "folders", QStringList());
+ foreach( QString path, include ) {
+ m_strigiFoldersHash.insert( path, true );
+ }
+
+ QStringList exclude = config.group( "General" ).readPathEntry( "exclude folders", QStringList());
+ foreach( QString path, exclude ) {
+ m_strigiFoldersHash.insert( path, false );
+ }
+}
+
+
+void Nepomuk::FileWatch::handleStrigiIndexedFolder( const QString& urlFrom, const QString& urlTo )
+{
+ if( !m_strigiFoldersHash.contains( urlFrom ) )
+ return;
+
+ KConfig config("nepomukstrigirc");
+
+ // We want to select urlFrom and all UrlFrom/something but not UrlFromSomething.
+ // Therefore the trailing slash and everything after that is optional (/.*)?
+ QRegExp regex( urlFrom + "(/.*)?$" );
+
+ QStringList includeFolders = config.group( "General" ).readPathEntry( "folders", QStringList());
+ includeFolders.replaceInStrings( regex, urlTo + "\\1" );
+
+ QStringList excludeFolders = config.group( "General" ).readPathEntry( "exclude folders", QStringList());
+ excludeFolders.replaceInStrings( regex, urlTo + "\\1" );
+
+ config.group( "General" ).writePathEntry( "folders", includeFolders );
+ config.group( "General" ).writePathEntry( "exclude folders", excludeFolders );
+ config.group( "General" ).writeEntry( "ignore changes", true );
+}
+
#include "nepomukfilewatch.moc"
Index: services/filewatch/metadatamover.cpp
===================================================================
--- services/filewatch/metadatamover.cpp (revision 1128117)
+++ services/filewatch/metadatamover.cpp (working copy)
@@ -246,13 +246,15 @@ void Nepomuk::MetadataMover::updateMetad
const QString newBasePath = to.path( KUrl::AddTrailingSlash );
Soprano::QueryResultIterator it = m_model->executeQuery( query, Soprano::Query::QueryLanguageSparql );
- while ( it.next() ) {
+ QList<Soprano::BindingSet> allBindings = it.allBindings();
+
+ foreach( const Soprano::BindingSet & bs, allBindings) {
// the resource URI of the resource to update
- const QUrl uri = it[0].uri();
+ const QUrl uri = bs[0].uri();
// the old URL of the resource to update
- const KUrl url = it[1].uri();
+ const KUrl url = bs[1].uri();
// now construct the new URL
QString oldRelativePath = url.path().mid( oldBasePath.length() );
Index: services/filewatch/nepomukfilewatch.h
===================================================================
--- services/filewatch/nepomukfilewatch.h (revision 1125616)
+++ services/filewatch/nepomukfilewatch.h (working copy)
@@ -23,6 +23,7 @@
#include <QtCore/QUrl>
#include <QtCore/QVariant>
+#include <QtCore/QHash>
namespace Soprano {
class Model;
@@ -60,9 +61,12 @@ namespace Nepomuk {
#ifdef BUILD_KINOTIFY
void slotInotifyWatchUserLimitReached();
#endif
+ void slotStrigiConfigChanged();
private:
MetadataMover* m_metadataMover;
+ QHash<QString, bool> m_strigiFoldersHash;
+ void handleStrigiIndexedFolder( const QString& from, const QString& to );
#ifdef BUILD_KINOTIFY
KInotify* m_dirWatch;
Index: services/filewatch/kinotify.cpp
===================================================================
--- services/filewatch/kinotify.cpp (revision 1125616)
+++ services/filewatch/kinotify.cpp (working copy)
@@ -56,7 +56,7 @@ class KInotify::Private
{
public:
Private( KInotify* parent )
- : watchHiddenFolders( true ),
+ : watchHiddenFolders( false ),
m_inotifyFd( -1 ),
m_notifier( 0 ),
q( parent) {
Index: kcm/nepomukserverkcm.cpp
===================================================================
--- kcm/nepomukserverkcm.cpp (revision 1125616)
+++ kcm/nepomukserverkcm.cpp (working copy)
@@ -223,7 +223,7 @@ void Nepomuk::ServerConfigModule::save()
strigiConfig.group( "General" ).writeEntry( "exclude filters", m_editStrigiExcludeFilters->items() );
strigiConfig.group( "General" ).writeEntry( "index hidden folders", m_checkShowHiddenFolders->isChecked() );
strigiConfig.group( "General" ).writeEntry( "index newly mounted", m_checkIndexRemovableMedia->isChecked() );
-
+ strigiConfig.group( "General" ).writeEntry( "ignore changes", false );
// 3. update the current state of the nepomuk server
if ( m_serverInterface.isValid() ) {
Index: io/kdirwatch.cpp
===================================================================
--- io/kdirwatch.cpp (revision 1128524)
+++ io/kdirwatch.cpp (working copy)
@@ -1858,6 +1858,7 @@ void KDirWatch::setCreated( const QStrin
void KDirWatch::setDirty( const QString & _file )
{
//kDebug(7001) << objectName() << "emitting dirty" << _file;
+ kDebug() << "emiting dirty " << _file;
emit dirty( _file );
}
_______________________________________________
Nepomuk mailing list
[email protected]
https://mail.kde.org/mailman/listinfo/nepomuk
