Hello all! I got a panicked call from a DJ/KJ friend of mine this weekend. He
knew I was handy with computers & needed help DJing a gig he had that evening.
I quickly found mixxx (in the rpmfusion repo for Fedora Core 17), put it
through its paces, called him back to tell him I thought I could fake it, spent
a few more hours load-testing mixxx and beating it to death...then went to the
gig. We pulled it off! So, just like that, I'm a DJ. :-)
Enclosed are some patches I've come up with so far. I doubt they're ready to
check in; I'm posting them mostly for feedback, and so you can point out any
problems with them. I'm building with the 1.11 branch.
The patch to src/library/basesqltablemodel.cpp is meant to fix bug #1090888,
i.e. https://bugs.launchpad.net/mixxx/+bug/1090888 . The problem is that, if
the database is locked, the code continues with an invalid SQL-query result.
My solution is to move the query to the top of the method, and return if the
query fails.
The rest of the patch makes it possible for library-rescan to run in parallel
with the rest of the GUI. Before, the database would stay locked during
rescan, although nothing prevented the user from trying to use the rest of the
GUI, which would lead to the GUI hanging and/or crashing relatively often. My
solution is to commit at the end of scanning each individual directory, so that
other database clients get a chance to do their work. I also added a pause
button to the library-rescan window.
I intend to spend part of this evening load-testing my changes.
My DJ friend has several hundred GB of music; I tried to scan it right before
the gig, but didn't finish. When I had time, I let it run, and it took about 4
hours. Being able to scan it incrementally, and allowing Mixxx to be usable
meanwhile, would have been a big help; these changes allow that.
I have more to say, but I figure I'll stop here for now...no need to talk your
ear off in my first e-mail :-)
Steven Boswell
--- mixxx/src/library/basesqltablemodel.cpp 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/basesqltablemodel.cpp 2013-04-09 13:00:07.983890678 -0700
@@ -177,15 +177,6 @@
QTime time;
time.start();
- // Remove all the rows from the table.
- // TODO(rryan) we could edit the table in place instead of clearing it?
- if (m_rowInfo.size() > 0) {
- beginRemoveRows(QModelIndex(), 0, m_rowInfo.size()-1);
- m_rowInfo.clear();
- m_trackIdToRows.clear();
- endRemoveRows();
- }
-
QString columns = m_tableColumnsJoined;
QString orderBy = orderByClause();
QString queryString = QString("SELECT %1 FROM %2 %3")
@@ -204,6 +195,16 @@
if (!query.exec()) {
qDebug() << this << "select() error:" << __FILE__ << __LINE__
<< query.executedQuery() << query.lastError();
+ return;
+ }
+
+ // Remove all the rows from the table.
+ // TODO(rryan) we could edit the table in place instead of clearing it?
+ if (m_rowInfo.size() > 0) {
+ beginRemoveRows(QModelIndex(), 0, m_rowInfo.size()-1);
+ m_rowInfo.clear();
+ m_trackIdToRows.clear();
+ endRemoveRows();
}
QSqlRecord record = query.record();
@@ -215,7 +216,7 @@
tableColumnIndices.push_back(record.indexOf(column));
}
- // sqlite does not set size and m_rowInfo was just cleared
+ // sqlite does not set size and m_rowInfo was just cleared
//int rows = query.size();
//if (sDebug) {
// qDebug() << "Rows returned" << rows << m_rowInfo.size();
--- mixxx/src/library/dao/trackdao.cpp 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/dao/trackdao.cpp 2013-04-09 13:43:04.586253057 -0700
@@ -14,6 +14,7 @@
#include "library/dao/cuedao.h"
#include "library/dao/playlistdao.h"
#include "library/dao/analysisdao.h"
+#include "util/sleepableqthread.h"
QHash<int, TrackWeakPointer> TrackDAO::m_sTracks;
QMutex TrackDAO::m_sTracksMutex;
@@ -376,6 +377,33 @@
"WHERE location = :location");
}
+// Allow added tracks to be flushed periodically, so as not to keep the database
+// locked.
+// Optionally pause between transactions.
+void TrackDAO::addTracksFlush (const volatile bool &a_rbPause,
+ const volatile bool &a_rbCancel)
+{
+ // Complete any pending transaction.
+ if (m_pTransaction) {
+ m_pTransaction->commit();
+ delete m_pTransaction;
+ m_pTransaction = NULL;
+ }
+
+ // Broadcast the list of added tracks to any interested subscribers.
+ emit (tracksAdded (m_tracksAddedSet));
+ m_tracksAddedSet.clear();
+
+ // If requested, pause.
+ // It'd be more efficient to use a QMutex/QWaitCondition, but that's
+ // probably overkill.
+ while (!a_rbCancel && a_rbPause)
+ SleepableQThread::sleep (1);
+
+ // Start a new transaction.
+ m_pTransaction = new ScopedTransaction(m_database);
+}
+
void TrackDAO::addTracksFinish() {
if (m_pTransaction) {
m_pTransaction->commit();
@@ -525,7 +553,7 @@
QList<int> TrackDAO::addTracks(QList<QFileInfo> fileInfoList, bool unremove) {
QList<int> trackIDs;
- TrackInfoObject* pTrack;
+ TrackInfoObject* pTrack;
addTracksPrepare();
--- mixxx/src/library/dao/trackdao.h 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/dao/trackdao.h 2013-04-09 13:36:53.563120091 -0700
@@ -82,6 +82,8 @@
int addTrack(const QFileInfo& fileInfo, bool unremove);
void addTracksPrepare();
bool addTracksAdd(TrackInfoObject* pTrack, bool unremove);
+ void addTracksFlush (const volatile bool &a_rbPause,
+ const volatile bool &a_rbCancel);
void addTracksFinish();
QList<int> addTracks(QList<QFileInfo> fileInfoList, bool unremove);
void hideTracks(QList<int> ids);
--- mixxx/src/library/libraryscanner.cpp 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/libraryscanner.cpp 2013-04-09 15:00:01.705245298 -0700
@@ -39,7 +39,7 @@
// Don't initialize m_database here, we need to do it in run() so the DB
// conn is in the right thread.
m_nameFilters(SoundSourceProxy::supportedFileExtensionsString().split(" ")),
- m_bCancelLibraryScan(0)
+ m_bPauseLibraryScan(false), m_bCancelLibraryScan(false)
{
qDebug() << "Constructed LibraryScanner";
@@ -289,10 +289,10 @@
emit(scanFinished());
}
-void LibraryScanner::scan(QString libraryPath, QWidget *parent)
+void LibraryScanner::scan(QString libraryPath, QWidget * /* parent */)
{
m_qLibraryPath = libraryPath;
- m_pProgress = new LibraryScannerDlg(parent);
+ m_pProgress = new LibraryScannerDlg();
m_pProgress->setAttribute(Qt::WA_DeleteOnClose);
// The important part here is that we need to use
@@ -310,6 +310,8 @@
//Qt::BlockingQueuedConnection);
connect(this, SIGNAL(scanFinished()),
m_pProgress, SLOT(slotScanFinished()));
+ connect(m_pProgress, SIGNAL(toggleScanPause()),
+ this, SLOT(togglePause()));
connect(m_pProgress, SIGNAL(scanCancelled()),
this, SLOT(cancel()));
connect(&m_trackDao, SIGNAL(progressVerifyTracksOutside(QString)),
@@ -318,14 +320,19 @@
}
//slot
+void LibraryScanner::togglePause()
+{
+ m_bPauseLibraryScan = !m_bPauseLibraryScan;
+}
+
void LibraryScanner::cancel()
{
- m_bCancelLibraryScan = 1;
+ m_bCancelLibraryScan = true;
}
void LibraryScanner::resetCancel()
{
- m_bCancelLibraryScan = 0;
+ m_bCancelLibraryScan = false;
}
void LibraryScanner::scan()
@@ -384,6 +391,12 @@
verifiedDirectories.append(dirPath);
}
+ // Flush the added tracks, so that the user can make use of incremental
+ // progress, and so that the track database doesn't stay locked during what
+ // could be a long operation (allowing the GUI a chance to access the DB).
+ // During the flush, allow library scanning to be paused.
+ m_trackDao.addTracksFlush (m_bPauseLibraryScan, m_bCancelLibraryScan);
+
// Let us break out of library directory hashing (the actual file scanning
// stuff is in TrackCollection::importDirectory)
if (m_bCancelLibraryScan) {
--- mixxx/src/library/libraryscannerdlg.cpp 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/libraryscannerdlg.cpp 2013-04-09 13:00:07.988890682 -0700
@@ -21,10 +21,10 @@
#include <QtGui>
#include "libraryscannerdlg.h"
-LibraryScannerDlg::LibraryScannerDlg(QWidget * parent, Qt::WindowFlags f) :
- QWidget(parent, f)
+LibraryScannerDlg::LibraryScannerDlg() :
+ QWidget(NULL, Qt::Window)
{
- m_bCancelled = false;
+ m_bPaused = m_bCancelled = false;
setWindowIcon(QIcon(":/images/ic_mixxx_window.png"));
@@ -34,10 +34,10 @@
QLabel* pLabel = new QLabel(tr("It's taking Mixxx a minute to scan your music library, please wait..."),this);
pLayout->addWidget(pLabel);
- QPushButton* pCancel = new QPushButton(tr("Cancel"), this);
- connect(pCancel, SIGNAL(clicked()),
- this, SLOT(slotCancel()));
- pLayout->addWidget(pCancel);
+ m_pPauseButton = new QPushButton(tr("Pause"), this);
+ connect(m_pPauseButton, SIGNAL(clicked()),
+ this, SLOT(slotTogglePause()));
+ pLayout->addWidget(m_pPauseButton);
QLabel* pCurrent = new QLabel(this);
pCurrent->setMaximumWidth(600);
@@ -45,6 +45,11 @@
connect(this, SIGNAL(progress(QString)),
pCurrent, SLOT(setText(QString)));
pLayout->addWidget(pCurrent);
+
+ QPushButton* pCancel = new QPushButton(tr("Cancel"), this);
+ connect(pCancel, SIGNAL(clicked()),
+ this, SLOT(slotCancel()));
+ pLayout->addWidget(pCancel);
setLayout(pLayout);
m_timer.start();
@@ -67,6 +72,16 @@
}
}
+void LibraryScannerDlg::slotTogglePause()
+{
+ if (m_bPaused)
+ m_pPauseButton->setText (tr ("Pause"));
+ else
+ m_pPauseButton->setText (tr ("Resume"));
+ emit(toggleScanPause());
+ m_bPaused = !m_bPaused;
+}
+
void LibraryScannerDlg::slotCancel()
{
qDebug() << "Cancelling library scan...";
--- mixxx/src/library/libraryscannerdlg.h 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/libraryscannerdlg.h 2013-04-09 13:00:07.988890682 -0700
@@ -27,15 +27,17 @@
class LibraryScannerDlg : public QWidget {
Q_OBJECT
public:
- LibraryScannerDlg(QWidget * parent = 0, Qt::WindowFlags f = Qt::Dialog);
+ LibraryScannerDlg();
~LibraryScannerDlg();
public slots:
void slotUpdate(QString path);
+ void slotTogglePause();
void slotCancel();
void slotScanFinished();
signals:
+ void toggleScanPause();
void scanCancelled();
void progress(QString);
@@ -43,7 +45,8 @@
// The path to the library on disk
QString m_qLibraryPath;
QTime m_timer;
- bool m_bCancelled;
+ QPushButton* m_pPauseButton;
+ bool m_bPaused, m_bCancelled;
};
#endif
--- mixxx/src/library/libraryscanner.h 2013-04-08 12:24:12.405359000 -0700
+++ mixxx-build/src/library/libraryscanner.h 2013-04-09 13:00:07.989890684 -0700
@@ -48,6 +48,7 @@
void scan();
bool recursiveScan(QString dirPath, QStringList& verifiedDirectories);
public slots:
+ void togglePause();
void cancel();
void resetCancel();
signals:
@@ -68,7 +69,7 @@
TrackDAO m_trackDao;
QStringList m_nameFilters;
- volatile bool m_bCancelLibraryScan;
+ volatile bool m_bPauseLibraryScan, m_bCancelLibraryScan;
QStringList m_directoriesBlacklist;
};
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Get Mixxx, the #1 Free MP3 DJ Mixing software Today
http://mixxx.org
Mixxx-devel mailing list
Mixxx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mixxx-devel
