Re: [sqlite] progressbar

[Patrik Nilsson]

Thank you for your suggestion. I will consider it.

Patrik

On 05/05/2012 10:31 PM, Roger Binns wrote:
> On 05/05/12 12:52, Patrik Nilsson wrote:
>> I use Sqlite as a document-file. Saving is when user requests to or
>> when program quits.
> 
> Even less reason to couple the user interface to database operations.  Are
> you really sure your users want to micromanage moving of data between
> transient (RAM) and persistent (disk) storage?
> 
> I got rid of the 'Save' menu from my programs many years ago because users
> really should not have to care about that, or worry if they have saved
> their work.  I made sure there was a very robust undo mechanism as well as
> a way of seeing what things looked like at any point in the past.
> 
> These days requiring users to tell the program when to copy data from
> transient storage to persistent is very anachronistic.  As an example
> you'll notice that no mobile apps do that, and most web ones don't either.
> 
> Roger
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] progressbar

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 05/05/12 12:52, Patrik Nilsson wrote:
> I use Sqlite as a document-file. Saving is when user requests to or
> when program quits.

Even less reason to couple the user interface to database operations.  Are
you really sure your users want to micromanage moving of data between
transient (RAM) and persistent (disk) storage?

I got rid of the 'Save' menu from my programs many years ago because users
really should not have to care about that, or worry if they have saved
their work.  I made sure there was a very robust undo mechanism as well as
a way of seeing what things looked like at any point in the past.

These days requiring users to tell the program when to copy data from
transient storage to persistent is very anachronistic.  As an example
you'll notice that no mobile apps do that, and most web ones don't either.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk+ljgwACgkQmOOfHg372QS+ZgCdGtSGEAZdyOt8el9tiA29cTpp
O30AoInBPhkKlDlEepiV72hybGF4f962
=2eL9
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] progressbar

[Patrik Nilsson]

On 05/05/2012 08:25 PM, Roger Binns wrote:

> Something else you can consider is changing how your program works so that
> the user interface isn't slaved to database operations.  You can let the
> UI queue up work to be done, and then have a background worker thread
> actually do the work in the queue.  Save the queue in a separate database
> so that work can be resumed even if the application crashes or is
> terminated.  (BTW this is how many mobile apps operate because db
> operations can take quite a while and they also have to be synced to a
> server over uncertain network connections.)

I use Sqlite as a document-file. Saving is when user requests to or when
program quits.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] progressbar

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 05/05/12 10:18, Patrik Nilsson wrote:
> Is it possible to get the total number of virtual machine instructions 
> an operation is requiring?

You know how many there are in the program via explain, but you will not
know how many will be executed.  For example it is only a few instructions
to loop over every row in a table, but that loop will be executed
repeatedly for every row.

> I want to use "sqlite3_progress_handler" during "vacuum" to inform the 
> user of the relative progress, i.e. sending vacuum to a 3 gigabyte 
> database takes a while. Possible?

vacuum is a single virtual instruction.  If you wanted to show progress of
a vacuum you could monitor the size of the database file or journal as a
proportion of the size of the original file although this may not be that
useful depending on how the vacuum is being done.  (The final file size is
unlikely to be the same as the original, but any progress calculations
will be good enough.)

> Is it possible to get the total number of virtual machine instructions
> a "begin... commit"-statement will need? I.e. progress of saving data
> to the database.

You already know the answer since you are providing the insert statements
and executing them one at a time.  Divide how many have already been
submitted versus how many there were in total.

If you do lengthy operations at the end such as indexing then the usual
approach to make the data insertion add up to 90% progress and the rest be
10%.  Again this won't be accurate, but it will be good enough.

Something else you can consider is changing how your program works so that
the user interface isn't slaved to database operations.  You can let the
UI queue up work to be done, and then have a background worker thread
actually do the work in the queue.  Save the queue in a separate database
so that work can be resumed even if the application crashes or is
terminated.  (BTW this is how many mobile apps operate because db
operations can take quite a while and they also have to be synced to a
server over uncertain network connections.)

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk+lcIEACgkQmOOfHg372QRiAwCfXZ4KuBNOppllW+HY2os8+wzw
IdsAoI3UKPIKFhQVGJZC1tJdbPD6qyf6
=uaYR
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] progressbar

[Patrik Nilsson]

It will not relate to the number of bytes/pages to move.

An example:

If I "explain vacuum"...
0|Trace|0|0|0||00|
1|Vacuum|0|0|0||00|
2|Halt|0|0|0||00|

I count these rows to two (excluding halt). When running a lengthy
operation the best resolution I can get is 50 percent.

Unfortunately, this will not be informative to the user.

Patrik

On 05/05/2012 07:45 PM, Simon Slavin wrote:
> 
> On 5 May 2012, at 6:18pm, Patrik Nilsson  wrote:
> 
>> Is it possible to get the total number of virtual machine instructions a
>> "begin... commit"-statement will need? I.e. progress of saving data to
>> the database.
> 
> Using 'EXPLAIN ...' for each of your instructions, and adding up the number 
> of rows in each answer might give you a useful number.
> 
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] progressbar

[Simon Slavin]

On 5 May 2012, at 6:18pm, Patrik Nilsson  wrote:

> Is it possible to get the total number of virtual machine instructions a
> "begin... commit"-statement will need? I.e. progress of saving data to
> the database.

Using 'EXPLAIN ...' for each of your instructions, and adding up the number of 
rows in each answer might give you a useful number.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users