Re: [Interest] Close DB connection when not actively using?

[Israel Brewster]

Ahh, thank you, that was most informative. So, basically, don't use 
QSqlTableModel/QSqlRelationalTableModel etc, and I should be fine. Would you 
say a good approach would be "writing my own" based on the lower-level 
QAbstractTableModel, and populating it with QSqlQueries? It seems that could 
address all the issues, as then I could open/close connections as needed, put 
in paging code (automatic or not), spin the queries off into separate threads, 
etc. It would also enable me to write SQL queries however worked best for the 
specific use, which I like. Hmmm

As for "why wouldn't your database be open?", there could be any number of 
reasons. This is a SQLITE database we're talking about, so perhaps the user 
doesn't have read/write permission to the proper directory. Maybe the location 
is on some removable media that isn't mounted. Etc. If we were talking about a 
"real" database, the server could be down, or not accepting connections. Lots 
of possibilities, so it behoves me to check. This is a public app, so I can't 
assume *anything* about the environment.

For what it's worth, I fully agree with your statement about each and every I/O 
going through the open/start/perform/commit/close cycle. That's essentially 
what I was envisioning my subclasses as doing - wrapping the *actual* db calls 
in that cycle. Also, yes, I/O should ideally be threaded off as you say. In my 
particular app, I highly doubt that the usage patterns would lend themselves to 
being able to notice these delays, even on a slower system with a lot of 
records, but it's certainly something to look at for future improvement. It's 
also an argument in favor of writing a custom "Table Model" class - it would be 
relatively easy at that point to spin the actual data I/O off to other threads.

Thanks again for the information and pointers! Lots to think about :-)!
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:31a341a7-a50f-46bf-bdc7-58d8fdcf82e4@flyravn.com]



[cid:5a6dfe8f-83f6-4356-9be9-5e594ccde583@flyravn.com]







On Aug 21, 2018, at 2:09 PM, Roland Hughes 
mailto:rol...@logikalsolutions.com>> wrote:


You need an architect level person to design your application. This is one of 
the far too numerous reasons AGILE is a false methodology. Developing even a 
simple application without a SAG (Systems Architecture Guide) opens an 8-lane 
highway for problems like these.

Why wouldn't your database be open?

You didn't write a custom table or replace any of the default I/O. As long as a 
QSqlTableModel is in existence it will keep the database open. You put it on a 
main window which won't go out of scope until the application is exited.

The "default" ease of use classes aren't for production systems connecting with 
a database. They all (as far as I can tell) take the lazy approach of opening 
the database once and keeping it open until the destructor is called. If you 
want to have an application which stays open, you have to write your own table 
class and provide all of your own I/O so it only connects to the database when 
data is needed. If you want a properly sized scroll bar, every I/O operation 
will need to perform a count(*) to get the maximum record count.

In a production quality system with a SAG, the SAG will state no database 
connection exists longer than a single transaction. Each I/O operation must

open

start transaction

perform I/O

commit transaction

close

Everyone of them. It's been a very long time since I looked at the code behind 
QSqlTableModel, but, it never used to do that. From your test it still doesn't. 
Part of this has to do with scrolling. They never wanted to figure out where 
they were in the database. One big select() creates a cursor in the database 
engine (or database itself) which lives as long as the transaction/connection. 
In order to dance around in that cursor, the connection must stay alive.

Putting scroll bars on a table necessitates either writing a lot of code 
yourself, or bad design. Most touch screen applications seem to rely on them 
rather than putting   buttons on the screen or enabling 
PageUp/PageDown. When your design uses this interface you still have to write a 
bit of code, but you can have a limited life connection. You only retrieve what 
fits on the screen bounded by the contents of either the first or last record.

Here's the real killer for you.

Almost every example you find on-line, including your test, is wrong. All I/O 
is performed in the GUI event loop instead of being threaded off. Oh sure, on a 
fast system with only a few thousand records in your database, like most 
examples, you don't notice the screen hang/stutter. If you are logging data 
points from sensors every few seconds, your database will have many hundreds of 
thousands of records after a few days. Something simple like changing the 

Re: [Interest] Close DB connection when not actively using?

[Roland Hughes]

  
  
You need an architect level person to design your application.
  This is one of the far too numerous reasons AGILE is a false
  methodology. Developing even a simple application without a SAG
  (Systems Architecture Guide) opens an 8-lane highway for problems
  like these.
Why wouldn't your database be open?
You didn't write a custom table or replace any of the default
  I/O. As long as a QSqlTableModel is in existence it will keep the
  database open. You put it on a main window which won't go out of
  scope until the application is exited.
The "default" ease of use classes aren't for production systems
  connecting with a database. They all (as far as I can tell) take
  the lazy approach of opening the database once and keeping it open
  until the destructor is called. If you want to have an application
  which stays open, you have to write your own table class and
  provide all of your own I/O so it only connects to the database
  when data is needed. If you want a properly sized scroll bar,
  every I/O operation will need to perform a count(*) to get the
  maximum record count.
In a production quality system with a SAG, the SAG will state no
  database connection exists longer than a single transaction. Each
  I/O operation must 

open
start transaction
perform I/O
commit transaction
close
Everyone of them. It's been a very long time since I looked at
  the code behind QSqlTableModel, but, it never used to do that.
  From your test it still doesn't. Part of this has to do with
  scrolling. They never wanted to figure out where they were in the
  database. One big select() creates a cursor in the database engine
  (or database itself) which lives as long as the
  transaction/connection. In order to dance around in that cursor,
  the connection must stay alive.
Putting scroll bars on a table necessitates either writing a lot
  of code yourself, or bad design. Most touch screen applications
  seem to rely on them rather than putting  
  buttons on the screen or enabling PageUp/PageDown. When your
  design uses this interface you still have to write a bit of code,
  but you can have a limited life connection. You only retrieve what
  fits on the screen bounded by the contents of either the first or
  last record.
Here's the real killer for you.
Almost every example you find on-line, including your test, is
  wrong. All I/O is performed in the GUI event loop instead of being
  threaded off. Oh sure, on a fast system with only a few thousand
  records in your database, like most examples, you don't notice the
  screen hang/stutter. If you are logging data points from sensors
  every few seconds, your database will have many hundreds of
  thousands of records after a few days. Something simple like
  changing the sort order of the display will cause the GUI to just
  hang until the I/O completes. If your database resides on a first
  generation (or something well below Class 10) MicroSD card, you
  won't even need a 100,000 records to see your GUI starting to
  hang.


On 08/21/2018 04:35 PM, Israel Brewster
  wrote:


  
  Ok, so as a test, I threw together
a basic project, consisting of just the default main window and
boiler plate code created by Qt creator. I then modified
main.cpp to be the following:


#include "mainwindow.h"
  #include 
  #include 
  
  bool setupTestConnection(){
      QSqlDatabase db=QSqlDatabase::addDatabase("QSQLITE");
      db.setDatabaseName("/tmp/testDb.db");
      bool result=db.open();
      db.close();
      return result;
  }
  
  int main(int argc, char *argv[])
  {
      QApplication a(argc, argv);
  
      bool canOpen=setupTestConnection();
      //TODO: if can't open, give appropriate error and
  terminate.

   
  qDebug("Call Result: %i, DB is STILL open: %i",canOpen,QSqlDatabase::database(QSqlDatabase::defaultConnection,false).isOpen());
  
      MainWindow w;
      w.show();
  
      return a.exec();
  }


And as expected (since I called close()), the
  debug line did show it was closed. Then in the main window I
  added a QSqlTableModel member (which I called testTable), and
  put the following in the QMainWindow constructor to set up the
  table model:


testTable.setTable("test");
testTable.select();


So keeping it as basic as I can. Note that I never
  need to call QSqlDatabase::database(), so that particular link
  you 

Re: [Interest] Close DB connection when not actively using?

[Israel Brewster]

Ok, so as a test, I threw together a basic project, consisting of just the 
default main window and boiler plate code created by Qt creator. I then 
modified main.cpp to be the following:

#include "mainwindow.h"
#include 
#include 

bool setupTestConnection(){
QSqlDatabase db=QSqlDatabase::addDatabase("QSQLITE");
db.setDatabaseName("/tmp/testDb.db");
bool result=db.open();
db.close();
return result;
}

int main(int argc, char *argv[])
{
QApplication a(argc, argv);

bool canOpen=setupTestConnection();
//TODO: if can't open, give appropriate error and terminate.

qDebug("Call Result: %i, DB is STILL open: 
%i",canOpen,QSqlDatabase::database(QSqlDatabase::defaultConnection,false).isOpen());

MainWindow w;
w.show();

return a.exec();
}

And as expected (since I called close()), the debug line did show it was 
closed. Then in the main window I added a QSqlTableModel member (which I called 
testTable), and put the following in the QMainWindow constructor to set up the 
table model:

testTable.setTable("test");
testTable.select();

So keeping it as basic as I can. Note that I never need to call 
QSqlDatabase::database(), so that particular link you sent doesn't help. When 
running this new application, lsof shows that the database is open at all times 
- so apparently the table model opened the connection and kept it open. So how 
is this one constructed wrong? I'm only opening the DB in main to make sure I 
can (which is kinda important), and I make sure to close it when I am done 
checking.

And BTW, I *have* read the manual. Repeatedly over the years. Obviously I have 
managed to miss the solutions you are seeing.
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:6793ac7e-ede7-41ad-8860-67d4cac2b49a@flyravn.com]



[cid:bdd42050-cfb6-4772-9280-b5284ed00f17@flyravn.com]







On Aug 21, 2018, at 1:08 PM, Roland Hughes 
mailto:rol...@logikalsolutions.com>> wrote:


You opened the database in main. The function you called is so small it most 
likely was placed in-line by the compiler. This would be especially true if you 
did nothing with dbOpen.

You have no reason to open it there. Again, read the fine manual.

http://doc.qt.io/qt-5/qsqldatabase.html#database

Every time you call QSqlDatabase::database( "someConnectionName") it will open 
the database if it is not currently open. You should never open the database in 
main. Just set up all of the values for it.

On 08/21/2018 03:54 PM, Israel Brewster wrote:
Ok, maybe you can give me a pointer on what I'm doing wrong then. My code is 
structured like the following:

bool openDatabase(){
QSqlDatabase db=QSqlDatabase::addDatabase('QSQLITE');
db.setDatabaseName();
return db.open();
}

 int main(int argc, char *argv[])
{
QApplication a(argc, argv);
...

...
bool dbOpen=openDatabase();
...

return a.exec();
}

And if I put a breakpoint at any point after the call to OpenDatabase, the 
database is open - it does not close, even though that QSqlDatabase object went 
out of scope. At no point in my application do I keep a copy of QSqlDatabase 
around in any of my classes - every call to QSqlDatabase is as a function local 
member, and as such goes out of scope (thereby theoretically calling the 
destructor) as soon as the function exits.

So what am I missing here?

---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---













On Aug 21, 2018, at 12:31 PM, Roland Hughes 
mailto:rol...@logikalsolutions.com>> wrote:


Why are you creating yet another class instead of properly structuring your 
application?

Take a gander at

http://www.logikalsolutions.com/wordpress/information-technology/qt-and-usb-pt-4/

for some ideas. You probably also want to find a copy of this book

http://www.theminimumyouneedtoknow.com/qt_book.html

Most importantly you need to read the fine manual.

http://doc.qt.io/qt-5/qsqldatabase.html#dtor.QSqlDatabase

QSqlDatabase::~QSqlDatabase()

Destroys the object and frees any allocated resources.

Note: When the last connection is destroyed, the destructor implicitly calls 
close() to release the database 
connection.

See also close().

If your database is open for the life of the application then the application 
has a failed architecture. Database connections aren't supposed to have actual 
life spans.

On 08/21/2018 03:08 PM, Israel Brewster wrote:

Ideally, what I'd have is a system where you set up the connection, and then it 
automatically opens/closes the connection when needed (i.e. when doing a 
SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) 
periodic updates or b) in direct 

Re: [Interest] Close DB connection when not actively using?

[Roland Hughes]

  
  
You opened the database in main. The function you called is so
  small it most likely was placed in-line by the compiler. This
  would be especially true if you did nothing with dbOpen.
You have no reason to open it there. Again, read the fine manual.
http://doc.qt.io/qt-5/qsqldatabase.html#database
Every time you call QSqlDatabase::database( "someConnectionName")
  it will open the database if it is not currently open. You should
  never open the database in main. Just set up all of the values for
  it.


On 08/21/2018 03:54 PM, Israel Brewster
  wrote:


  
  Ok, maybe you can give me a
pointer on what I'm doing wrong then. My code is structured like
the following:


bool openDatabase(){
QSqlDatabase
  db=QSqlDatabase::addDatabase('QSQLITE');
db.setDatabaseName();
return db.open();
}


 int main(int argc, char *argv[])
  {
QApplication
  a(argc, argv);
...

...
bool
  dbOpen=openDatabase();
...

return
  a.exec();
}


And if I put a breakpoint at any point after the
  call to OpenDatabase, the database is open - it does not
  close, even though that QSqlDatabase object went out of scope.
  At no point in my application do I keep a copy of QSqlDatabase
  around in any of my classes - every call to QSqlDatabase is as
  a function local member, and as such goes out of scope
  (thereby theoretically calling the destructor) as soon as the
  function exits. 


So what am I missing here? 


  

  

  
---


  Israel Brewster


  Systems Analyst II


  5245 Airport Industrial Rd


  Fairbanks, AK 99709


  (907) 450-7293


  
---

  
  
  
  

  
   
  

  

  

  
  
  
  

  
   
  

  


  
  
  
  
  
  
    
  
  
  
  
  
  

  
  

  


  
  
  
  
  

  
On Aug 21, 2018, at 12:31 PM, Roland
  Hughes 
  wrote:


  
Why are you creating yet another class
  instead of properly structuring your application?
Take a gander at 

http://www.logikalsolutions.com/wordpress/information-technology/qt-and-usb-pt-4/
for some ideas. You probably also want
  to find a copy of this book
http://www.theminimumyouneedtoknow.com/qt_book.html
Most importantly you need to read the
  fine manual.
http://doc.qt.io/qt-5/qsqldatabase.html#dtor.QSqlDatabase
QSqlDatabase::~QSqlDatabase()
Destroys the object and frees any
  allocated resources.
Note: When the 

Re: [Interest] Close DB connection when not actively using?

[Furkan Üzümcü]

I think you can use the following for this. I’m using a similar approach in my 
apps, and I have not run into any issues yet.

QSqlDatabase::addDatabase keeps the connection open If you do not explicitly 
close it. But in order to actually commit any transactions you have to call 
QSqlDatabase::open(). So If you open it in your main.cpp file, it will keep 
being open.

Instead, you can keep only the connection open. When you are using the 
connection, you just get the already open database connection using 
QSqlDatabase::database(), then open it, and do whatever you like. After the 
transaction, you can simply close it.

Regards,
Furkan Üzümcü
On Aug 21, 2018, 16:55 -0400, Israel Brewster , wrote:
> Ok, maybe you can give me a pointer on what I'm doing wrong then. My code is 
> structured like the following:
>
> bool openDatabase(){
> QSqlDatabase db=QSqlDatabase::addDatabase('QSQLITE');
> db.setDatabaseName();
> return db.open();
> }
>
>  int main(int argc, char *argv[])
> {
> QApplication a(argc, argv);
> ...
> 
> ...
> bool dbOpen=openDatabase();
> ...
> 
> return a.exec();
> }
>
> And if I put a breakpoint at any point after the call to OpenDatabase, the 
> database is open - it does not close, even though that QSqlDatabase object 
> went out of scope. At no point in my application do I keep a copy of 
> QSqlDatabase around in any of my classes - every call to QSqlDatabase is as a 
> function local member, and as such goes out of scope (thereby theoretically 
> calling the destructor) as soon as the function exits.
>
> So what am I missing here?
>
> ---
> Israel Brewster
> Systems Analyst II
> 5245 Airport Industrial Rd
> Fairbanks, AK 99709
> (907) 450-7293
> ---
>
>
>
>
>
>
>
>
>
>
>
> > On Aug 21, 2018, at 12:31 PM, Roland Hughes  
> > wrote:
> >
> > Why are you creating yet another class instead of properly structuring your 
> > application?
> > Take a gander at
> > http://www.logikalsolutions.com/wordpress/information-technology/qt-and-usb-pt-4/
> > for some ideas. You probably also want to find a copy of this book
> > http://www.theminimumyouneedtoknow.com/qt_book.html
> > Most importantly you need to read the fine manual.
> > http://doc.qt.io/qt-5/qsqldatabase.html#dtor.QSqlDatabase
> > QSqlDatabase::~QSqlDatabase()
> > Destroys the object and frees any allocated resources.
> > Note: When the last connection is destroyed, the destructor implicitly 
> > calls close() to release the database connection.
> > See also close().
> > If your database is open for the life of the application then the 
> > application has a failed architecture. Database connections aren't supposed 
> > to have actual life spans.
> >
> > On 08/21/2018 03:08 PM, Israel Brewster wrote:
> > > Ideally, what I'd have is a system where you set up the connection, and 
> > > then it automatically opens/closes the connection when needed (i.e. when 
> > > doing a SELECT, INSERT, UPDATE, etc). With my app, this would generally 
> > > only be for a) periodic updates or b) in direct response to user input, 
> > > so most of the time the connection could remain closed. To the best of my 
> > > knowledge (correct me if I am wrong), this is not possible with the 
> > > existing QSqlDatabase/ QSQLITE database driver classes.
> >
> > --
> > Roland Hughes, President
> > Logikal Solutions
> > (630) 205-1593
> >
> > http://www.theminimumyouneedtoknow.com
> > http://www.infiniteexposure.net
> > http://www.johnsmith-book.com
> > http://www.logikalblog.com
> > http://www.interestingauthors.com/blog
> > http://lesedi.us
> > ___
> > Interest mailing list
> > Interest@qt-project.org
> > http://lists.qt-project.org/mailman/listinfo/interest
>
> ___
> Interest mailing list
> Interest@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Close DB connection when not actively using?

[Israel Brewster]

Ok, maybe you can give me a pointer on what I'm doing wrong then. My code is 
structured like the following:

bool openDatabase(){
QSqlDatabase db=QSqlDatabase::addDatabase('QSQLITE');
db.setDatabaseName();
return db.open();
}

 int main(int argc, char *argv[])
{
QApplication a(argc, argv);
...

...
bool dbOpen=openDatabase();
...

return a.exec();
}

And if I put a breakpoint at any point after the call to OpenDatabase, the 
database is open - it does not close, even though that QSqlDatabase object went 
out of scope. At no point in my application do I keep a copy of QSqlDatabase 
around in any of my classes - every call to QSqlDatabase is as a function local 
member, and as such goes out of scope (thereby theoretically calling the 
destructor) as soon as the function exits.

So what am I missing here?

---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:8f9ccd5b-5a65-4794-b193-af4863e7f7aa@flyravn.com]



[cid:a01d4aae-c628-436a-a1af-b0b73bd6c7bd@flyravn.com]







On Aug 21, 2018, at 12:31 PM, Roland Hughes 
mailto:rol...@logikalsolutions.com>> wrote:


Why are you creating yet another class instead of properly structuring your 
application?

Take a gander at

http://www.logikalsolutions.com/wordpress/information-technology/qt-and-usb-pt-4/

for some ideas. You probably also want to find a copy of this book

http://www.theminimumyouneedtoknow.com/qt_book.html

Most importantly you need to read the fine manual.

http://doc.qt.io/qt-5/qsqldatabase.html#dtor.QSqlDatabase

QSqlDatabase::~QSqlDatabase()

Destroys the object and frees any allocated resources.

Note: When the last connection is destroyed, the destructor implicitly calls 
close() to release the database 
connection.

See also close().

If your database is open for the life of the application then the application 
has a failed architecture. Database connections aren't supposed to have actual 
life spans.

On 08/21/2018 03:08 PM, Israel Brewster wrote:

Ideally, what I'd have is a system where you set up the connection, and then it 
automatically opens/closes the connection when needed (i.e. when doing a 
SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) 
periodic updates or b) in direct response to user input, so most of the time 
the connection could remain closed. To the best of my knowledge (correct me if 
I am wrong), this is not possible with the existing QSqlDatabase/ QSQLITE 
database driver classes.


--
Roland Hughes, President
Logikal Solutions
(630) 205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog
http://lesedi.us

___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

BEGIN:VCARD
VERSION:3.0
N:Brewster;Israel;;;
FN:Israel Brewster
ORG:Frontier Flying Service;MIS
TITLE:PC Support Tech II
EMAIL;type=INTERNET;type=WORK;type=pref:isr...@frontierflying.com
TEL;type=WORK;type=pref:907-450-7293
item1.ADR;type=WORK;type=pref:;;5245 Airport Industrial Wy;Fairbanks;AK;99701;
item1.X-ABADR:us
CATEGORIES:General
X-ABUID:36305438-95EA-4410-91AB-45D16CABCDDC\:ABPerson
END:VCARD
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Close DB connection when not actively using?

[Roland Hughes]

  
  
Why are you creating yet another class instead of properly
  structuring your application?
Take a gander at 

http://www.logikalsolutions.com/wordpress/information-technology/qt-and-usb-pt-4/
for some ideas. You probably also want to find a copy of this
  book
http://www.theminimumyouneedtoknow.com/qt_book.html
Most importantly you need to read the fine manual.
http://doc.qt.io/qt-5/qsqldatabase.html#dtor.QSqlDatabase
QSqlDatabase::~QSqlDatabase()
Destroys the object and frees any allocated resources.
Note: When the last connection is destroyed, the
  destructor implicitly calls close()
  to release the database connection.
See also close().
If your database is open for the life of the application then the
  application has a failed architecture. Database connections aren't
  supposed to have actual life spans.


On 08/21/2018 03:08 PM, Israel Brewster
  wrote:


  Ideally, what I'd have is a system where you set up the connection, and then it automatically opens/closes the connection when needed (i.e. when doing a SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) periodic updates or b) in direct response to user input, so most of the time the connection could remain closed. To the best of my knowledge (correct me if I am wrong), this is not possible with the existing QSqlDatabase/ QSQLITE database driver classes.


-- 
Roland Hughes, President
Logikal Solutions
(630) 205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog
http://lesedi.us
  

___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Close DB connection when not actively using?

[Israel Brewster]

Ok, yeah, that could work. Then I'd simply have a member that is the original 
class, and call it for the data processing once I have verified connection 
state. Sounds like a plan, I'll give it a shot!
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:ea49bbc5-e539-4aee-9f26-34bcd6e3dea2@flyravn.com]



[cid:dea9973e-0608-4e10-bc95-ef7ca5e5e89a@flyravn.com]







On Aug 21, 2018, at 10:44 AM, Glen Mabey 
mailto:gma...@swri.org>> wrote:


I suggest you do not subclass any of those Qt classes, but instead create a new 
class that derives from QObject with the signals, slots and methods that you 
need.  Those methods could even have the same names as those of QSqlQuery, for 
example.

One of the methods of that class could be to set the timeout for a QTimer* data 
member that would close the database connection.

When your "exec()" method is invoked, it first checks whether the connection is 
open, and remedies that situation as needed.

Glen


From: Interest 
[interest-bounces+gmabey=swri@qt-project.org]
 on behalf of Israel Brewster 
[ibrews...@flyravn.com]
Sent: Tuesday, August 21, 2018 11:23 AM
To: interest@qt-project.org
Subject: [Interest] Close DB connection when not actively using?

I have an application that may be left running for extended periods of time. It 
uses a SQLite database via QSqlDatabase and various other QtSQL classes 
(QSQLTableModel, QSqlQuery, etc, depending on where in the program it is used). 
The problem is that once I call open() on the database, it remains open until I 
explicitly close it, which is not until the program exits. Since this means the 
SQLite file is held open, this causes problems with things like backup programs 
that don't back up open files, or potentially services like dropbox (haven't 
tested that one thoroughly) that try to sync files between different computers.

Ideally, what I'd have is a system where you set up the connection, and then it 
automatically opens/closes the connection when needed (i.e. when doing a 
SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) 
periodic updates or b) in direct response to user input, so most of the time 
the connection could remain closed. To the best of my knowledge (correct me if 
I am wrong), this is not possible with the existing QSqlDatabase/ QSQLITE 
database driver classes.

So, assuming that I need to subclass something to implement such functionality:

- Which class(es) will I need to subclass - QSqlDatabase? The SQLite Driver? 
Both?
- Which functions specifically should I be looking at modifying?
- How do classes like QSqlTableModel interact with the underlying QSqlDatabase 
as far as performing queries - would they know or care about the connection 
being closed at any point other than when it is explicitly doing a select or 
update/insert/delete?

I do realize this could result in multiple open/close cycles in a rapid period 
of time (maybe I could reduce that by putting some sort of timer on the 
automatic close), but for the purposes of my application I don't see this as 
being an issue. Thanks!
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---













___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

BEGIN:VCARD
VERSION:3.0
N:Brewster;Israel;;;
FN:Israel Brewster
ORG:Frontier Flying Service;MIS
TITLE:PC Support Tech II
EMAIL;type=INTERNET;type=WORK;type=pref:isr...@frontierflying.com
TEL;type=WORK;type=pref:907-450-7293
item1.ADR;type=WORK;type=pref:;;5245 Airport Industrial Wy;Fairbanks;AK;99701;
item1.X-ABADR:us
CATEGORIES:General
X-ABUID:36305438-95EA-4410-91AB-45D16CABCDDC\:ABPerson
END:VCARD
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Close DB connection when not actively using?

[Glen Mabey]

I suggest you do not subclass any of those Qt classes, but instead create a new 
class that derives from QObject with the signals, slots and methods that you 
need.  Those methods could even have the same names as those of QSqlQuery, for 
example.

One of the methods of that class could be to set the timeout for a QTimer* data 
member that would close the database connection.

When your "exec()" method is invoked, it first checks whether the connection is 
open, and remedies that situation as needed.

Glen


From: Interest [interest-bounces+gmabey=swri@qt-project.org] on behalf of 
Israel Brewster [ibrews...@flyravn.com]
Sent: Tuesday, August 21, 2018 11:23 AM
To: interest@qt-project.org
Subject: [Interest] Close DB connection when not actively using?

I have an application that may be left running for extended periods of time. It 
uses a SQLite database via QSqlDatabase and various other QtSQL classes 
(QSQLTableModel, QSqlQuery, etc, depending on where in the program it is used). 
The problem is that once I call open() on the database, it remains open until I 
explicitly close it, which is not until the program exits. Since this means the 
SQLite file is held open, this causes problems with things like backup programs 
that don't back up open files, or potentially services like dropbox (haven't 
tested that one thoroughly) that try to sync files between different computers.

Ideally, what I'd have is a system where you set up the connection, and then it 
automatically opens/closes the connection when needed (i.e. when doing a 
SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) 
periodic updates or b) in direct response to user input, so most of the time 
the connection could remain closed. To the best of my knowledge (correct me if 
I am wrong), this is not possible with the existing QSqlDatabase/ QSQLITE 
database driver classes.

So, assuming that I need to subclass something to implement such functionality:

- Which class(es) will I need to subclass - QSqlDatabase? The SQLite Driver? 
Both?
- Which functions specifically should I be looking at modifying?
- How do classes like QSqlTableModel interact with the underlying QSqlDatabase 
as far as performing queries - would they know or care about the connection 
being closed at any point other than when it is explicitly doing a select or 
update/insert/delete?

I do realize this could result in multiple open/close cycles in a rapid period 
of time (maybe I could reduce that by putting some sort of timer on the 
automatic close), but for the purposes of my application I don't see this as 
being an issue. Thanks!
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:e84a5d6b-1779-4d69-afc9-781595d38945@flyravn.com]



[cid:0c00c43a-c113-4a29-bfba-108a7d1636eb@flyravn.com]







___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Qt 5.11.1 inside of a Docker container

[Richard Weickelt]

> I recently had success installing Qt 5.9.2 in a Windows docker
> container using this link as a starting point:
> https://stackoverflow.com/a/34032216/991000

You can also see this approach in action here:
http://code.qt.io/cgit/qbs/qbs.git/tree/docker/stretch/Dockerfile

But pulling packages from https://launchpad.net/~beineri is far more simple
on Ubuntu.

Richard
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

[Interest] Close DB connection when not actively using?

[Israel Brewster]

I have an application that may be left running for extended periods of time. It 
uses a SQLite database via QSqlDatabase and various other QtSQL classes 
(QSQLTableModel, QSqlQuery, etc, depending on where in the program it is used). 
The problem is that once I call open() on the database, it remains open until I 
explicitly close it, which is not until the program exits. Since this means the 
SQLite file is held open, this causes problems with things like backup programs 
that don't back up open files, or potentially services like dropbox (haven't 
tested that one thoroughly) that try to sync files between different computers.

Ideally, what I'd have is a system where you set up the connection, and then it 
automatically opens/closes the connection when needed (i.e. when doing a 
SELECT, INSERT, UPDATE, etc). With my app, this would generally only be for a) 
periodic updates or b) in direct response to user input, so most of the time 
the connection could remain closed. To the best of my knowledge (correct me if 
I am wrong), this is not possible with the existing QSqlDatabase/ QSQLITE 
database driver classes.

So, assuming that I need to subclass something to implement such functionality:

- Which class(es) will I need to subclass - QSqlDatabase? The SQLite Driver? 
Both?
- Which functions specifically should I be looking at modifying?
- How do classes like QSqlTableModel interact with the underlying QSqlDatabase 
as far as performing queries - would they know or care about the connection 
being closed at any point other than when it is explicitly doing a select or 
update/insert/delete?

I do realize this could result in multiple open/close cycles in a rapid period 
of time (maybe I could reduce that by putting some sort of timer on the 
automatic close), but for the purposes of my application I don't see this as 
being an issue. Thanks!
---
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---

[cid:e84a5d6b-1779-4d69-afc9-781595d38945@flyravn.com]



[cid:0c00c43a-c113-4a29-bfba-108a7d1636eb@flyravn.com]







BEGIN:VCARD
VERSION:3.0
N:Brewster;Israel;;;
FN:Israel Brewster
ORG:Frontier Flying Service;MIS
TITLE:PC Support Tech II
EMAIL;type=INTERNET;type=WORK;type=pref:isr...@frontierflying.com
TEL;type=WORK;type=pref:907-450-7293
item1.ADR;type=WORK;type=pref:;;5245 Airport Industrial Wy;Fairbanks;AK;99701;
item1.X-ABADR:us
CATEGORIES:General
X-ABUID:36305438-95EA-4410-91AB-45D16CABCDDC\:ABPerson
END:VCARD
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

[Interest] Android: back key with frameless flag doesn't work

[Jason H]

I have an app that I want to be full-screen and close. It's a Window and a 
Rectangle. Somewhere along the line I googled and found:

flags: Qt.FramelessWindowHint

should be added to the root window. However when also using the back button, no 
back is received by the window or the rectangle inside. 
D ViewRootImpl@4d7c559[QtActivity]: ViewPostIme key 0
D ViewRootImpl@4d7c559[QtActivity]: ViewPostIme key 1

Is all that appears. Removing the flag, the rect will get it.

Is this a bug, or am I doing something wrong?
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Qt 5.11.1 inside of a Docker container

[Thiago Macieira]

On Tuesday, 21 August 2018 07:39:19 PDT Michael Jackson wrote:
> I’m a bit new to Docker but this will end up being a Qt question. I have
> been trying to figure out how to install Qt 5.11.1 into a Docker container
> based on Ubuntu 18.04. The first obvious way was to download the Qt
> installer and try running that until one realizes that the installer
> requires a GUI to run, which Docker does not seem to have. Searched around
> on the internet and about the only process seems to just be to outright
> build the version of Qt that you require if it is not provided by the
> normal packaging system. Qt 5.9.5 is provided for Ubuntu 18.04. For those
> that use Docker to host builds is that what is commonly done? I thought
> about just installing Qt 5.11.1 into my host environment and then use the
> COPY command to copy it into the Docker image that is being created but
> something just didn’t seem “right” about doing it that way. I have found
> some scripts that seem to think that they can strip out installer
> components but those scripts don’t seem to work. Has there been any
> movement in the ability to install Qt in a truly headless environment
> without having to build from source?

Add one of these PPA and install:
https://launchpad.net/~beineri

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center



___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Qt 5.11.1 inside of a Docker container

[Michael Jackson]

I actually use the --script in one of our "Superbuild" projects that is driven 
by CMake. The important bit is that when those scripts are run (macOS, Linux 
and Windows) there is *always* a GUI environment running. In a Docker container 
there is no GUI environment to run so the --script will not help unless 
something has changed lately with the Qt5 installers.
--
Michael Jackson | Owner, President
  BlueQuartz Software
[e] mike.jack...@bluequartz.net
[w] www.bluequartz.net 

On 8/21/18, 10:49 AM, "Carel Combrink"  wrote:

Hi,

I recently had success installing Qt 5.9.2 in a Windows docker
container using this link as a starting point:
https://stackoverflow.com/a/34032216/991000

I am on my way home at the moment, but if you do not get right I can
try to assist some more tomorrow.

Regards,
Carel
On Tue, Aug 21, 2018 at 4:39 PM Michael Jackson
 wrote:
>
> I’m a bit new to Docker but this will end up being a Qt question. I have 
been trying to figure out how to install Qt 5.11.1 into a Docker container 
based on Ubuntu 18.04. The first obvious way was to download the Qt installer 
and try running that until one realizes that the installer requires a GUI to 
run, which Docker does not seem to have. Searched around on the internet and 
about the only process seems to just be to outright build the version of Qt 
that you require if it is not provided by the normal packaging system. Qt 5.9.5 
is provided for Ubuntu 18.04. For those that use Docker to host builds is that 
what is commonly done? I thought about just installing Qt 5.11.1 into my host 
environment and then use the COPY command to copy it into the Docker image that 
is being created but something just didn’t seem “right” about doing it that 
way. I have found some scripts that seem to think that they can strip out 
installer components but those scripts don’t seem to work. Has there been any 
movement in the ability to install Qt in a truly headless environment without 
having to build from source?
> Thanks
> --
> Michael Jackson  


___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Qt 5.11.1 inside of a Docker container

[Carel Combrink]

Hi,

I recently had success installing Qt 5.9.2 in a Windows docker
container using this link as a starting point:
https://stackoverflow.com/a/34032216/991000

I am on my way home at the moment, but if you do not get right I can
try to assist some more tomorrow.

Regards,
Carel
On Tue, Aug 21, 2018 at 4:39 PM Michael Jackson
 wrote:
>
> I’m a bit new to Docker but this will end up being a Qt question. I have been 
> trying to figure out how to install Qt 5.11.1 into a Docker container based 
> on Ubuntu 18.04. The first obvious way was to download the Qt installer and 
> try running that until one realizes that the installer requires a GUI to run, 
> which Docker does not seem to have. Searched around on the internet and about 
> the only process seems to just be to outright build the version of Qt that 
> you require if it is not provided by the normal packaging system. Qt 5.9.5 is 
> provided for Ubuntu 18.04. For those that use Docker to host builds is that 
> what is commonly done? I thought about just installing Qt 5.11.1 into my host 
> environment and then use the COPY command to copy it into the Docker image 
> that is being created but something just didn’t seem “right” about doing it 
> that way. I have found some scripts that seem to think that they can strip 
> out installer components but those scripts don’t seem to work. Has there been 
> any movement in the ability to install Qt in a truly headless environment 
> without having to build from source?
>
>
>
> Thanks
>
> --
>
> Michael Jackson | Owner, President
>
>   BlueQuartz Software
>
> [e] mike.jack...@bluequartz.net
>
> [w] www.bluequartz.net
>
> ___
> Interest mailing list
> Interest@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

[Interest] Qt 5.11.1 inside of a Docker container

[Michael Jackson]

I’m a bit new to Docker but this will end up being a Qt question. I have been 
trying to figure out how to install Qt 5.11.1 into a Docker container based on 
Ubuntu 18.04. The first obvious way was to download the Qt installer and try 
running that until one realizes that the installer requires a GUI to run, which 
Docker does not seem to have. Searched around on the internet and about the 
only process seems to just be to outright build the version of Qt that you 
require if it is not provided by the normal packaging system. Qt 5.9.5 is 
provided for Ubuntu 18.04. For those that use Docker to host builds is that 
what is commonly done? I thought about just installing Qt 5.11.1 into my host 
environment and then use the COPY command to copy it into the Docker image that 
is being created but something just didn’t seem “right” about doing it that 
way. I have found some scripts that seem to think that they can strip out 
installer components but those scripts don’t seem to work. Has there been any 
movement in the ability to install Qt in a truly headless environment without 
having to build from source?

 

Thanks

--

Michael Jackson | Owner, President

  BlueQuartz Software

[e] mike.jack...@bluequartz.net

[w] www.bluequartz.net

___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] Qt3D Framegraphs

[Paul Lemire via Interest]

On 08/21/2018 01:54 PM, Andy wrote:
> Thank you so much Paul!
>
> That gives me something to start working on/pick apart. I see now how
> onscreen vs. offscreen works and can concentrate on getting the
> onscreen working the way I want first since they are very similar.
>
> 1) "I assume you want to fill the depth buffer with a simple shader
> right?"
>
> I think so? Ultimately I want to experiment with a cel-shaded scene,
> but for now I'd be happy with adding some black contours on my
> entities using depth - slightly thicker lines closer to the camera,
> thinner farther away. Is this the right setup for that?

Hmm that's not necessarily what I pictured. Usually a render pass where
the depth buffer is filled is used as an optimization technique so that
1) You draw your scene with a very simple shader to fill the depth
buffer 2) You draw you scene again using a more complex shader but you
then take advantage of the fact that the GPU will only execute the
fragment shader for fragment whose depth is equal to what is stored in
the depth buffer.

If you want to draw contours (which is usually referred as silhouetting)
the technique is different. Meshes are composed of triangles which are
specified in a given winding order (order in which the triangles
vertices are specified, either clockwise or counterclockwise). That
winding order can be used at draw time to distinguish between triangles
which are facing the camera and triangles which are backfacing the
camera. (Usually another optimization technique is to not draw
backfacing triangles a.k.a backface culling).

A possible silhouetting technique implementation can be to:
1) draw only the back faces of the mesh (slightly enlarged) and with
depth writing into the depth buffer disabled.
2) draw the front faces of the mesh (with depth writing enabled)

See http://sunandblackcat.com/tipFullView.php?l=eng=15 for a
more detailed explaination, there are other implementation with geometry
shaders as well (http://prideout.net/blog/?p=54)

In practice, you would play with render states to control back face /
front face culling, depth write ... e.g:
RenderStateSet {
        renderStates: [
                DepthTest { depthFunction: DepthTest.Equal } // Specify
which depth function to use to decide which fragments to key
                NoDepthWrite {} // Disable writing into the depth buffer
    CullFace { mode: CullFace.Front } // Cull Front faces
(usually you would do back face culling though)
            ]
}

Note that cell shading might yet be another technique (with a different
implementation than silhouetting). Usually it involves having steps of
colors that vary based on light position in your fragment shader. It
might even be simpler to implement than silhouetting actually.

The above link actually implements a combination of both techniques.
 
>
> 2) "Have you tried the rendercapture ones?"
>
> Yes I have. That's how I got my render capture working (once those
> examples worked).
>
> One thing that wasn't clear to me before was where to attach the
> RenderCapture node. In the rendercapture example, it's created and
> then the forward renderer is re-parented, which is what I did with
> mine. Your outline makes more sense.

I suppose it was made purely by convenience to avoid having to rewrite a
full FrameGraph, but I do agree that makes understanding a lot harder.

>
> ClearBuffers (and NoDraw!) now make sense too. In QForwardRenderer
> they are on the camera selector which seems strange.

That's a small optimization. If your FrameGraph results in a single
branch (which QForwardRenderer probably does), you can combine the
ClearBuffers and the CameraSelector as that translates to basically
clear then draw.

If your framegraph has more than a single branch:
RenderSurfaceSelector {
    Viewport {
  CameraSelector {
                ClearBuffers { ...
                    RenderPassFilter { ... } // Branch 1
                    RenderPassFilter { ...} // Branch 2
                }
     }
    }
}

What would happen in that case is:

1) clear buffers then draw branch 1
2) clear buffers then draw branch 2

So in the end you would only see the drawings from Branch 2 because the
back buffer was cleared.

In that case you should instead have it like:

RenderSurfaceSelector {
    Viewport {
  CameraSelector {
                ClearBuffers { ...
                    RenderPassFilter { ... } // Branch 1
                }
               RenderPassFilter { ...} // Branch 2
     }
    }
}

or (which is a bit easier to understand but adds one branch to the
FrameGraph)

RenderSurfaceSelector {
    Viewport {
  CameraSelector {
                ClearBuffers { ...
                    NoDraw {}
                } // Branch 1
                RenderPassFilter { ... } // Branch 2
                RenderPassFilter { ...} // Branch 3
     }
    }
}


>
> 3) If I want to use any of the "default materials" in extras - Phong,
> PhongAlpha, etc - 

Re: [Interest] Qt3D Framegraphs

[Andy]

Thank you so much Paul!

That gives me something to start working on/pick apart. I see now how
onscreen vs. offscreen works and can concentrate on getting the onscreen
working the way I want first since they are very similar.

1) "I assume you want to fill the depth buffer with a simple shader right?"

I think so? Ultimately I want to experiment with a cel-shaded scene, but
for now I'd be happy with adding some black contours on my entities using
depth - slightly thicker lines closer to the camera, thinner farther away.
Is this the right setup for that?

2) "Have you tried the rendercapture ones?"

Yes I have. That's how I got my render capture working (once those examples
worked).

One thing that wasn't clear to me before was where to attach the
RenderCapture node. In the rendercapture example, it's created and then the
forward renderer is re-parented, which is what I did with mine. Your
outline makes more sense.

ClearBuffers (and NoDraw!) now make sense too. In QForwardRenderer they are
on the camera selector which seems strange.

3) If I want to use any of the "default materials" in extras - Phong,
PhongAlpha, etc - then in (3) and (4.3) the filterkeys must be
"renderingStyle"/"forward", correct? Or can I even use them anymore if I'm
going this route?

4) I will use the offscreen to generate snapshot images and video - I
assume I can turn offscreen rendering on/off dynamically by simply
enabling/disabling the RenderTargetSelector?


Thanks again for your help. I finally feel like I'm in danger of
understanding something here!


On Mon, Aug 20, 2018 at 1:20 AM Paul Lemire  wrote:

> Hi Andy,
>
> Please see my reply below
>
> On 08/15/2018 02:59 PM, Andy wrote:
>
> I've been struggling with framegraphs for a very long time now and still
> don't feel like I understand  their structure - what goes where or what
> kind of nodes can be attached to what. I can throw a bunch of things
> together, but when it doesn't work I have no idea how to track down what's
> missing or what's in the wrong place.
>
> Can anyone give an outline of what a framegraph would look like to
> facilitate all of the following for a given scene:
>
> 1. rendering in a window onscreen
> 2. depth pass for shaders to use
>
> I assume you want to fill the depth buffer with a simple shader right?
>
> 3. render capture for taking "snapshots" of what the user is seeing
> onscreen
> 4. offscreen rendering of the current scene at a specified size (not the
> UI window size)
> 5. render capture of the offscreen scene to an image
>
>
> I've not tested but the I would image what you want would look like the
> frame Graph below:
>
> RenderSurfaceSelector { // Select window to render to
>
> Viewport {
>
> // 1 Clear Color and Depth buffers
> ClearBuffers {
> buffers: ClearBuffers.ColorDepthBuffer
> NoDraw {}
> }
>
>
> // Select Camera to Use to Render Scene
> CameraSelector {
> camera: id_of_scene_camera
>
> // 2 Fill Depth Buffer pass (for screen depth buffer)
> RenderPassFilter {
> filterKeys: [ FilterKey { name: "pass"; value: "depth_fill_pass"] //
> Requires a Material which defines such a RenderPass
> }
>
> // 3 Draw screen content and use depth compare == to benefit for z fill
> passs
> RenderPassFilter {
>filterKeys: [ FilterKey { name: "pass"; value: "color_pass"] //
> Requires a Material which defines such a RenderPass
>RenderStateSet {
> renderStates: DepthTest { depthFunction: DepthTest.Equal }
> RenderCapture { // Use this to capture screen frame buffer
> id: onScreenCapture
> }
>}
> }
>
> // 4 Create FBO for offscreen rendering
> RenderTargetSelector {
> target: RenderTarget {
>   attachments: [
> RenderTargetOutput {
> attachmentPoint: RenderTargetOutput.Color0
> texture: Texture2D { width: width_of_offscreen_area;
> height: height_of_offscreen_area;  }
> },
>RenderTargetOutput {
> attachmentPoint: RenderTargetOutput.Depth
> texture: Texture2D { width: width_of_offscreen_area;
> height: height_of_offscreen_area;  }
> } ]
>} // RenderTarget
>
> // Note: ideally 4.1, 4.2 and 4.3 and 1, 2, 3 could be factored
> out as a reusable subtree (if using QML)
>
> // 4.1 Clear FBO
> ClearBuffers {
>   buffers: ClearBuffers.ColorDepthBuffer
>   NoDraw {}
>}
>
>// 4.2 Fill Depth Buffer pass (for offscreen depth buffer)
> RenderPassFilter {
> filterKeys: [ FilterKey { name: "pass"; value: "depth_fill_pass"]
> // Requires a Material which defines such a RenderPass
> }
>
> // 4.3 Draw content into offscreen color buffer and use depth compare
> == to benefit for z fill pass
> RenderPassFilter {
>filterKeys: [ FilterKey { name: "pass"; value: "color_pass"] //
> Requires a Material which defines such a RenderPass
>RenderStateSet {
>