Re: [sqlite] Protecting databases

[Teg]

Hello Damien,

I  use  the  "encrypt  specific  fields"  method of encrypting data in
sqlite.   For  you  to use that with Basic you'd probably have to find
or  write  a  wrapper  DLL which handled the field encryption. So your
basic  would only see unencrypted data after it sent a password to the
DLL.

I know this probably makes it a non-starter for you.

C


Saturday, October 8, 2016, 1:46:02 AM, you wrote:

DSL> Hi there,
DSL> My name is Damien Lindley, and I am, among other things, an
DSL> independent, hobbiest programmer. I have been blind since birth
DSL> and thus all my computer work relies on screenreader software and keyboard.
DSL> I have only just come through the brink of scripting into
DSL> compiled programming and so I guess I am still a beginner in many
DSL> respects. However I don’t work in C or C++, so most of my
DSL> programming, if using a library, relies on precompiled static or
DSL> dynamic libraries. Or of course libraries that are written or
DSL> converted specifically for the language I work in (FreeBASIC).
DSL> Recently, I decided I needed to create a piece of software that
DSL> could manage family trees, since there seems to be a lack of
DSL> screenreader accessible genealogy managers out there. I was
DSL> advised the best way to do this is to use a database engine. I
DSL> was also informed that SQLite is always a good choice for databases.
DSL> I must admit, I have never worked with databases before and so
DSL> now I am in the process of learning SQL. However looking at the
DSL> programming API for SQLite I cannot see any means of password
DSL> protecting the database without either buying a commercial
DSL> extension to do this, or recompiling SQLite with the
DSL> authentication extension. Due to financial constraints and
DSL> unfamiliarity with compiling in C both of these are not an option
DSL> for me. Also I need a secure way to do this, as I think I read
DSL> that the SQLite version simply uses a table to store the user
DSL> data, which of course can be read and accessed elsewhere.
DSL> Are there any other options available for doing this?
DSL> Any help appreciated.
DSL> Thanks.
DSL> Damien.
DSL> ___
DSL> sqlite-users mailing list
DSL> sqlite-users@mailinglists.sqlite.org
DSL> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



-- 
 Tegmailto:t...@djii.com

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Backward compatibility of indexes with "WHERE function()"

[Richard Hipp]

On 10/8/16, Jens Alfke  wrote:
>
> SQLite is only about 1MB of
> code(?)

See http://sqlite.org/graphs/size-20161009.jpg for a graph of
compiled-binary size using gcc 4.8.4 and -Os on x64 Linux.  Less than
0.5 MB, though to be fair this is without extensions such as FTS5 or
RTREE or JSON.

If you are getting 1MB binaries, then you are probably using -O3 which
does lots of loop unrolling and function inlining and thereby runs up
the library size.  If the library size is really important to you,
maybe you should consider changing to -Os, even if you continue to
using the system SQLite.dylib.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Backward compatibility of indexes with "WHERE function()"

[Jens Alfke]

> On Oct 8, 2016, at 12:56 PM, Richard Hipp  wrote:
> 
> Databases that use functions in the WHERE clause of a partial index
> will be unreadable by any version of SQLite prior to 3.15.0.

Yikes! This would be good to call out in the docs.

> Why can't you statically link against the SQLite 3.15.0?  What compels
> you to use whatever version of SQLite that iOS has built-in?

Code size of mobile apps. (To be clear, these are apps shipped by developers 
using our library, not our own apps.) Yes, SQLite is only about 1MB of code(?), 
but those megabytes add up, and mobile developers have told us they’re mindful 
of app size because it can act as a deterrent to people installing an app over 
a cell connection. So if we can tell them “our library is only 1MB of code!” 
that sounds a lot better than “…2MB of code!”.

That said, using the system SQLite library is apparently no longer an option in 
Android N. And for developers wanting extra/redundant encryption we support 
SQLCipher, which gets statically linked of course. But I think most of our 
developers are still using the system SQLite library.

I’m not saying this is a blocker for our using this feature; but it’s a factor 
to consider. Thanks for the info.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Backward compatibility of indexes with "WHERE function()"

[Richard Hipp]

On 10/8/16, Jens Alfke  wrote:
> What’s the compatibility of the new partial indexes* with deterministic
> functions in their WHERE clauses?
> I.e. if I create such an index, and then later the database is opened by an
> older version of SQLite, what happens? (Let’s say the function used in the
> WHERE clause is still properly registered.)

Databases that use functions in the WHERE clause of a partial index
will be unreadable by any version of SQLite prior to 3.15.0.

>
> I’m just trying to decide whether I can safely make use of this feature yet,
> since some places my code will run will have only the built-in iOS version
> of SQLite, which is currently 3.14.0 at best.

Why can't you statically link against the SQLite 3.15.0?  What compels
you to use whatever version of SQLite that iOS has built-in?


>
> (But I suppose that even if it’s not version-compatible, I can at least
> detect the situation when opening the db and issue a REINDEX command.)

That won't work.  Once SQLite sees that it cannot parse the schema, it
gives up and won't let you do anything.  You can work around it using
"PRAGMA writable_schema=ON" but that comes with its own peril.


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Backward compatibility of indexes with "WHERE function()"

[Jens Alfke]

What’s the compatibility of the new partial indexes* with deterministic 
functions in their WHERE clauses?
I.e. if I create such an index, and then later the database is opened by an 
older version of SQLite, what happens? (Let’s say the function used in the 
WHERE clause is still properly registered.)
If that’s ok, then what happens if the older SQLite modifies the database and 
updates the index? And what happens if the database is then re-opened by an 
instance of SQLite with the new functionality?

I’m just trying to decide whether I can safely make use of this feature yet, 
since some places my code will run will have only the built-in iOS version of 
SQLite, which is currently 3.14.0 at best.

(But I suppose that even if it’s not version-compatible, I can at least detect 
the situation when opening the db and issue a REINDEX command.)

—Jens

* is it ‘indexes’ or ‘indices’, in computer science?
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Jens Alfke]

> On Oct 7, 2016, at 10:46 PM, Damien Sykes-Lindley 
>  wrote:
> 
> I cannot see any means of password protecting the database without either 
> buying a commercial extension to do this, or recompiling SQLite with the 
> authentication extension.

I’m surprised no one has mentioned SQLCipher 
(https://www.zetetic.net/sqlcipher/ ) — 
it’s a free/OSS plugin that adds transparent file encryption. Since it encrypts 
individual pages of the file as they’re transferred between disk and SQLite’s 
page cache, it doesn’t affect the functionality of SQLite at all. Once you 
issue a ‘pragma’ statement to provide the key, the database behaves exactly as 
though it were unencrypted.

Despite being technically a plugin, it’s distributed as a modified copy of the 
SQLite ‘amalgamation’ source file, I think to make it easier to build. They 
also have pre-built binaries for popular OSs.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Parallel access to read only in memory database

[Jens Alfke]

> On Oct 7, 2016, at 1:45 PM, Joe Mistachkin  wrote:
> 
> Have you tried using the URI "file::memory:?cache=shared”

Shared cache will definitely help when using multiple read-only connections, 
but I have a feeling memory-mapping would help even more since it would 
hopefully eliminate redundant memcpy calls between the database and the “file” 
cache.

Daniel, try using “pragma mmap_size=N” where N is basically the minimum of the 
size of the database file and the amount of free RAM.

> How can we allow many reader threads on an in memory, write once read many
> times database and achieve multi-core performance?

This part’s easy: just open a database connection per thread. Using the 
shared-cache mode as above means the threads won’t each allocate their own 
cache, which will save memory and improve locality of reference.

—Jens

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[David Bicking]

If you are trying to protect against casual snooping, you could probably 
zip the sqlite data with a password then have your application unzip the 
data to a temp location on open, then re-zip at the end, deleting the 
unzipped file. Your application then would be able to use the normal 
sqlite calls to use the data.


Or for field by field encryption, you could create an encryt/decrypt 
function so that only encrypted data is stored in the db.


INSERT INTO myTable (mySensitiveField) VALUES( encrypt(@ClearText, 
@password) );


SELECT * FROM myTable where decrypt(mySensitiveField, @password) LIKE 
@SearchPattern;


Hope those ideas help.

I've written plenty of Freebasic programs using sqlite, but never had 
the need to encrypt or control access.


David


On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an 
independent, hobbiest programmer. I have been blind since birth and 
thus all my computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. 
However I don’t work in C or C++, so most of my programming, if 
using a library, relies on precompiled static or dynamic libraries. 
Or of course libraries that are written or converted specifically 
for the language I work in (FreeBASIC).
Recently, I decided I needed to create a piece of software that 
could manage family trees, since there seems to be a lack of 
screenreader accessible genealogy managers out there. I was advised 
the best way to do this is to use a database engine. I was also 
informed that SQLite is always a good choice for databases.
I must admit, I have never worked with databases before and so now I 
am in the process of learning SQL. However looking at the 
programming API for SQLite I cannot see any means of password 
protecting the database without either buying a commercial extension 
to do this, or recompiling SQLite with the authentication extension. 
Due to financial constraints and unfamiliarity with compiling in C 
both of these are not an option for me. Also I need a secure way to 
do this, as I think I read that the SQLite version simply uses a 
table to store the user data, which of course can be read and 
accessed elsewhere.

Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: Lemon: Simple recursive rule causes assertion failed: stateno <= YY_SHIFT_COUNT

[Conor O]

> Absolutely! I've simplified some source code and attached two files testlex.c 
> and testpar.c.
> I'm not sure if the mailing list strips out attachments 

I guess it does strip attachments. For the benefits of the mailing
list, I'll inline the code (assuming gmail doesn't mess it up more
than it is already)...

- testlex.c -
/* Test Lexer for Lemon Parser */

#include 
#include 

#include "testpar.h"

extern void *ParseAlloc(void *(*mallocProc)(size_t));
extern void Parse(void *, int, void *);
extern void ParseFree(void *, void (*freeProc)(void *));
extern void ParseTrace(FILE *stream, char *zPrefix);

int main()
{
void *pParser;

pParser = ParseAlloc(malloc);
if (!pParser)
{
printf("out of memory\n");
exit(1);
}

ParseTrace(stderr, "Debug: ");

Parse(pParser, LPC_CMDNAME, 0);
Parse(pParser, LPC_INTEGER, 0);
Parse(pParser, LPC_TEXT, 0);
Parse(pParser, LPC_EOL, 0);

Parse(pParser, LPC_CMDNAME, 0);
Parse(pParser, LPC_INTEGER, 0);
Parse(pParser, LPC_TEXT, 0);
Parse(pParser, LPC_EOL, 0);

Parse(pParser, 0, 0);
ParseFree(pParser, free);
return 0;
}

- testpar.y -

%include {
#include 
}

%token_prefix LPC_

%parse_accept {
printf("Parsing complete\n");
}

%parse_failure {
printf("Giving up...\n");
}

%syntax_error {
printf("Error in input syntax\n");
}

%start_symbol database

/* Start symbol */

database ::= entrylist. { printf("Constructed the root\n"); }

entrylist ::= command.
entrylist ::= entrylist command.

command ::= CMDNAME cmdargs EOL. { printf("Reducing to command state\n"); }

cmdargs ::= .
cmdargs ::= cmdargs cmdarg.

cmdarg ::= INTEGER.
cmdarg ::= TEXT.

- end -
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Stephen Chrzanowski]

If you're developing in Windows, you could leave it to the file system, but
could be dangerous, and then binds you to using an NTFS based OS.

NTFS has the capability of doing file level encryption, and you can flip
the flag on the file to have the filesystem encrypt the file.  The catch
is, the encryption mechanism includes information about the user, based on
their SID (System ID).  So if you end up losing the login, or, reloading
your OS, unless you log in with that EXACT same user (Not just recreate a
new user account with the same name) you will lose the data.

I don't recall if the encryption is based on the owner of the directory or
file, or, if it is the active user who performs the encryption.

Another option is, if your database is small enough, you could compress the
file with any number of compression algorithms.  Depending on how sensitive
you feel you need to be about longevity of unencrypted data on your system,
you could read in a password protected ZIP file, extract it to a scratch
file, use the SQLite backup API to pull the database into memory, then
delete the scratch file.  Do your work on the in-memory database, then when
done, use the backup API again to dump it to a scratch file, then zip the
scratch file with a password.

On Sat, Oct 8, 2016 at 7:12 AM, Damien Sykes-Lindley <
dam...@dcpendleton.plus.com> wrote:

> Hi Ryan,
> I think there may have been some misunderstanding here. The application is
> a local application and thus needs no server, at least on the data side.
> But multiple people may have access to the same machine. So while user
> control is needed elsewhere, that certainly isn't on the local side. Just a
> simple password would do.
> The network/user control scenario is much more important in the
> collaboration features of the software. I am still trying to work out
> ultimately how best to do this (client/server, peer-to-peer possibly with
> relay server etc). However the specific data that is being collaborated on
> doesn't need to be stored remotely, unless of course I use a
> check-in/check-out style system. Again, collaboration is way into the
> future and so I'm not thinking along those lines just yet.
> I have looked into the SEE extension again, just to make sure I am correct
> in my assumptions, and it appears I am. I doubt I even get $2000 disposable
> income in one year, let alone a month so that option would be impossible
> for me at this time. Also it appears that you are responsible for compiling
> SEE, again which I would find it very hard to do, never having compiled
> anything in C.
> On the other hand, searching is a necessity so it appears that encrypting
> the data from the application itself is also not an option. So I think at
> this point I am stuck
> Cheers.
> Damien.
> -Original Message- From: R Smith
> Sent: Saturday, October 08, 2016 10:02 AM
> To: sqlite-users@mailinglists.sqlite.org
>
> Subject: Re: [sqlite] Protecting databases
>
> Hi Damien,
>
> We are obviously all fans of SQLite and keen to involve more people in
> what is one of the best DB systems in existence and most widely used DB
> in the World.
>
> But...
>
> SQLite answers a need that is quite specific (though widely used) in
> that it is very small, very fast and, as a bonus, can be compiled right
> into most software projects so distribution is not an issue - which
> further means it is also server-less. This last feature makes it
> specifically not well suited for large network-server and service type
> applications and not well suited for user control. Encryption is easy
> (as others explained) but your replies indicate you are more interested
> in user control. While some extensions do exist to allow a form of user
> control, I think what you really want is called PostGres or Maria-DB
> (previously MySQL). They are equally free, open-source, they support
> client-server architecture and both storage-encryption and user-control
> are inherent to these systems, and they play very well with web-script
> engines (such as PHP and its ilk).
>
> Use SQLite for local storage that need not be protected from users with
> access to the local file system. Store all your application settings in
> SQLite, etc. It is not uncommon to use multiple database engines in a
> single project. Right tool for the job and such.
>
> Of the two alternate engines above, my preferred is PostGres, but there
> are situations where Maria DB works a treat (such as the usual
> LAMP-stack quick web-server solutions and the like) and it has some easy
> and strong scripting capabilities.
>
> Links:
> https://mariadb.org/
> https://www.postgresql.org/
>
> Good luck,
> Ryan
>
>
> On 2016/10/08 9:18 AM, Damien Sykes-Lindley wrote:
>
>> Hi Darren,
>> You are correct in that genealogy is generally public. However more often
>> than not the information you want to publish may very well differ from what
>> is in your private database. You may have private notes telling you what
>> you need to do. You may 

Re: [sqlite] Protecting databases

[R Smith]

On 2016/10/08 1:12 PM, Damien Sykes-Lindley wrote:

Hi Ryan,
I think there may have been some misunderstanding here. The 
application is a local application and thus needs no server, at least 
on the data side. But multiple people may have access to the same 
machine. So while user control is needed elsewhere, that certainly 
isn't on the local side. Just a simple password would do.


Sorry, I think there are some misunderstandings, perhaps our word 
choices are unfortunate.
Saying  "We don't need user control - A simple password will do" is much 
like saying "We don't need a lock, a simple key-and-keyhole mechanism 
would do". A password is the very essence of user access control. Unless 
you lock the file (and in so doing impose user-access-control via the 
O.S.) but that is just removing the issue one step. You need user access 
control, even if very simple. There is no reason to not also use a 
normal DB system locally - especially since you are already opting that 
way for the other more complex stuff (if I understand you correct).


To go one further, SQLite is very geared towards programmed systems, you 
seem to need more a script connector DB system, these other DB's were 
made for your use case. (Note: SQlite can do very well here too, just 
that it isn't necessarily better than the others for that interface, add 
to that the user control issue and I really think you should consider 
another solution).


Hope that was less misunderstandy :)
Ryan




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: Lemon: Simple recursive rule causes assertion failed: stateno <= YY_SHIFT_COUNT

[Conor O]

On 8 October 2016 at 14:31, Richard Hipp  wrote:

> On 10/8/16, Conor O  wrote:
> > Assertion failed: stateno <= YY_SHIFT_COUNT, file testpar.c, line 512
> >
> > I'm trying to figure out if this is a bug or something I'm just not
> > understanding correctly.
>
> An assertion fault is always a bug.  This is true of any application.
>

I was thinking that I have asserts in my own code which would break if
someone else called it badly, so it could always have be me :-)


> Can you send in sufficient information for me to reproduce the problem?
>

Absolutely! I've simplified some source code and attached two files
testlex.c and testpar.c. I'm not sure if the mailing list strips out
attachments so if it's ok I'll CC to your personal email as well. I was
compiling with simply:
gcc -Wall -mconsole -o testlex.exe testlex.c testpar.c

By the way, Lemon itself will assert if you place the start symbol on the
right hand side of a rule. It does tell you that as it going about the
assertion though, so I think that's ok!

Thanks,

Conor.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Lemon documention (small error)

[Richard Hipp]

On 10/8/16, Conor O'Rourke  wrote:
> I believe the Lemon documentation at:
> http://www.hwaci.com/sw/lemon/lemon.html is out of date with respect to the
> default token type. If I create a parser with no token_type override, the
> default type is:
>
> #define ParseTOKENTYPE void*
>
> not int as stated.

Fix checked in.  Thanks.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: Lemon: Simple recursive rule causes assertion failed: stateno <= YY_SHIFT_COUNT

[Richard Hipp]

On 10/8/16, Conor O  wrote:
> Assertion failed: stateno <= YY_SHIFT_COUNT, file testpar.c, line 512
>
> I'm trying to figure out if this is a bug or something I'm just not
> understanding correctly.

An assertion fault is always a bug.  This is true of any application.

Can you send in sufficient information for me to reproduce the problem?
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Fwd: Lemon: Simple recursive rule causes assertion failed: stateno <= YY_SHIFT_COUNT

[Conor O]

I've been experimenting with some simple rules as I get used to the Lemon
parser. This very simple parser file works correctly. A database consists
of an entrylist which consists of any number of commands:

database ::= entrylist. { printf("Constructed the root\n"); }

entrylist ::= command.
entrylist ::= entrylist command.

command ::= CMDNAME INTEGER TEXT EOL. { printf("Reducing to command
state\n"); }

For reference, I'm feeding the parser with:
Parse(pParser, CMDNAME, 0);
Parse(pParser, INTEGER, 0);
Parse(pParser, TEXT, 0);
Parse(pParser, EOL, 0);

However, if I enhance the command slightly to accept any number of optional
arguments:

database ::= entrylist. { printf("Constructed the root\n"); }

entrylist ::= command.
entrylist ::= entrylist command.

command ::= CMDNAME cmdargs EOL. { printf("Reducing to command
state\n"); }
cmdargs ::= .
cmdargs ::= cmdargs cmdarg.

cmdarg ::= INTEGER.
cmdarg ::= TEXT.

This causes an assertion failure:

> test.exe
Debug: Input 'CMDNAME'
Debug: Shift 'CMDNAME', go to state 3
Debug: Return. Stack=[CMDNAME]
Debug: Input 'INTEGER'
Assertion failed: stateno <= YY_SHIFT_COUNT, file testpar.c, line 512

I'm trying to figure out if this is a bug or something I'm just not
understanding correctly.

Thanks, Conor.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Error trying to make sqlite3 documentation

[Richard Hipp]

On 10/8/16, Domingo Alvarez Duarte  wrote:
> Hello !
>
> I'm trying to make sqlite3 documentation to use locally and got this error:
>
> Has someone build the latest documentation ?
>

Yes.  https://sqlite.org/draft/index.html was built by typing "make
all" (or "make fast") and then using rsync to move the result to the
website.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Error trying to make sqlite3 documentation

[Domingo Alvarez Duarte]

Hello !

I'm trying to make sqlite3 documentation to use locally and got this error:



make all

...

Processing ./pages/btreemodule.in
./tclsh: missing close-brace
while executing
"hd_puts {"
("eval" body line 590)
invoked from within
"eval "hd_puts \173$text\175""
(procedure "hd_resolve" line 5)
invoked from within
"hd_resolve [subst {

$PREAMBLE


style="font-size:2em;text-align:center;color:#80a796">$zTitle


This docu..."
("eval" body line 38)
invoked from within
"eval {

hd_keywords {btree design}
source [file join $::DOC pages fancyformat.tcl]

foreach header {btree.h sqliteInt.h} {
  set fd [open [file join $..."
("eval" body line 2)
invoked from within
"eval "hd_resolve \173$in\175""
("foreach" body line 16)
invoked from within
"foreach infile [lrange $argv 3 end] {
  cd $HOMEDIR
  puts "Processing $infile"
  set fd [open $infile r]
  set in [read $fd]
  close $fd
  if {[strin..."
(file "./wrap.tcl" line 783)
make: *** [base] Error 1



Has someone build the latest documentation ?

Cheers !

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Damien Sykes-Lindley]

Hi Ryan,
I think there may have been some misunderstanding here. The application is a 
local application and thus needs no server, at least on the data side. But 
multiple people may have access to the same machine. So while user control 
is needed elsewhere, that certainly isn't on the local side. Just a simple 
password would do.
The network/user control scenario is much more important in the 
collaboration features of the software. I am still trying to work out 
ultimately how best to do this (client/server, peer-to-peer possibly with 
relay server etc). However the specific data that is being collaborated on 
doesn't need to be stored remotely, unless of course I use a 
check-in/check-out style system. Again, collaboration is way into the future 
and so I'm not thinking along those lines just yet.
I have looked into the SEE extension again, just to make sure I am correct 
in my assumptions, and it appears I am. I doubt I even get $2000 disposable 
income in one year, let alone a month so that option would be impossible for 
me at this time. Also it appears that you are responsible for compiling SEE, 
again which I would find it very hard to do, never having compiled anything 
in C.
On the other hand, searching is a necessity so it appears that encrypting 
the data from the application itself is also not an option. So I think at 
this point I am stuck

Cheers.
Damien.
-Original Message- 
From: R Smith

Sent: Saturday, October 08, 2016 10:02 AM
To: sqlite-users@mailinglists.sqlite.org
Subject: Re: [sqlite] Protecting databases

Hi Damien,

We are obviously all fans of SQLite and keen to involve more people in
what is one of the best DB systems in existence and most widely used DB
in the World.

But...

SQLite answers a need that is quite specific (though widely used) in
that it is very small, very fast and, as a bonus, can be compiled right
into most software projects so distribution is not an issue - which
further means it is also server-less. This last feature makes it
specifically not well suited for large network-server and service type
applications and not well suited for user control. Encryption is easy
(as others explained) but your replies indicate you are more interested
in user control. While some extensions do exist to allow a form of user
control, I think what you really want is called PostGres or Maria-DB
(previously MySQL). They are equally free, open-source, they support
client-server architecture and both storage-encryption and user-control
are inherent to these systems, and they play very well with web-script
engines (such as PHP and its ilk).

Use SQLite for local storage that need not be protected from users with
access to the local file system. Store all your application settings in
SQLite, etc. It is not uncommon to use multiple database engines in a
single project. Right tool for the job and such.

Of the two alternate engines above, my preferred is PostGres, but there
are situations where Maria DB works a treat (such as the usual
LAMP-stack quick web-server solutions and the like) and it has some easy
and strong scripting capabilities.

Links:
https://mariadb.org/
https://www.postgresql.org/

Good luck,
Ryan


On 2016/10/08 9:18 AM, Damien Sykes-Lindley wrote:

Hi Darren,
You are correct in that genealogy is generally public. However more often 
than not the information you want to publish may very well differ from 
what is in your private database. You may have private notes telling you 
what you need to do. You may have anecdotes shared by many family members 
that may need to be kept private, at least until the involved parties are 
deceased or otherwise choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in 
there so you can easily group and contact appropriate family members for 
whatever reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to 
be used online then it is needless to say that authentication would be 
required for various people to view it. Even if I decide to make it local 
only, there is the possibility that anyone sharing the computer or network 
may peruse the database when you don't want them to.

Kind regards,
Damien.
-Original Message- From: Darren Duncan
Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent, 
hobbiest programmer. I have been blind since birth and thus all my 
computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. 
However I don’t work in C or C++, so most of my programming, if using a 
library, relies on precompiled static or dynamic libraries. Or of 

Re: [sqlite] Protecting databases

[R Smith]

Hi Damien,

We are obviously all fans of SQLite and keen to involve more people in 
what is one of the best DB systems in existence and most widely used DB 
in the World.


But...

SQLite answers a need that is quite specific (though widely used) in 
that it is very small, very fast and, as a bonus, can be compiled right 
into most software projects so distribution is not an issue - which 
further means it is also server-less. This last feature makes it 
specifically not well suited for large network-server and service type 
applications and not well suited for user control. Encryption is easy 
(as others explained) but your replies indicate you are more interested 
in user control. While some extensions do exist to allow a form of user 
control, I think what you really want is called PostGres or Maria-DB 
(previously MySQL). They are equally free, open-source, they support 
client-server architecture and both storage-encryption and user-control 
are inherent to these systems, and they play very well with web-script 
engines (such as PHP and its ilk).


Use SQLite for local storage that need not be protected from users with 
access to the local file system. Store all your application settings in 
SQLite, etc. It is not uncommon to use multiple database engines in a 
single project. Right tool for the job and such.


Of the two alternate engines above, my preferred is PostGres, but there 
are situations where Maria DB works a treat (such as the usual 
LAMP-stack quick web-server solutions and the like) and it has some easy 
and strong scripting capabilities.


Links:
https://mariadb.org/
https://www.postgresql.org/

Good luck,
Ryan


On 2016/10/08 9:18 AM, Damien Sykes-Lindley wrote:

Hi Darren,
You are correct in that genealogy is generally public. However more 
often than not the information you want to publish may very well 
differ from what is in your private database. You may have private 
notes telling you what you need to do. You may have anecdotes shared 
by many family members that may need to be kept private, at least 
until the involved parties are deceased or otherwise choose to divulge 
it publicly themselves.
Even more importantly I may choose to add an address-book style 
feature in there so you can easily group and contact appropriate 
family members for whatever reason (special occasions etc). Of course 
that will be private.
Password protecting it is also good on many levels - if the database 
is to be used online then it is needless to say that authentication 
would be required for various people to view it. Even if I decide to 
make it local only, there is the possibility that anyone sharing the 
computer or network may peruse the database when you don't want them to.

Kind regards,
Damien.
-Original Message- From: Darren Duncan
Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an 
independent, hobbiest programmer. I have been blind since birth and 
thus all my computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. 
However I don’t work in C or C++, so most of my programming, if using 
a library, relies on precompiled static or dynamic libraries. Or of 
course libraries that are written or converted specifically for the 
language I work in (FreeBASIC).
Recently, I decided I needed to create a piece of software that could 
manage family trees, since there seems to be a lack of screenreader 
accessible genealogy managers out there. I was advised the best way 
to do this is to use a database engine. I was also informed that 
SQLite is always a good choice for databases.
I must admit, I have never worked with databases before and so now I 
am in the process of learning SQL. However looking at the programming 
API for SQLite I cannot see any means of password protecting the 
database without either buying a commercial extension to do this, or 
recompiling SQLite with the authentication extension. Due to 
financial constraints and unfamiliarity with compiling in C both of 
these are not an option for me. Also I need a secure way to do this, 
as I think I read that the SQLite version simply uses a table to 
store the user data, which of course can be read and accessed elsewhere.

Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.


Damien,

Why do you need to password protect the database?

Genealogy information is generally of the public record variety so 
there is
nothing sensitive to protect.  I am making genealogy software myself 
and so am

familiar with many of the relevant issues.

I would say please explain why you think you need password protection 
for this

project and then the real issue at hand can be addressed.

If 

Re: [sqlite] Protecting databases

[Jean-Christophe Deschamps]

Damien,

At 09:18 08/10/2016, you wrote:
Password protecting it is also good on many levels - if the database 
is to be used online then it is needless to say that authentication 
would be required for various people to view it.


SQLite can't be put "online" per se. It will then be the duty of the 
host software layer (website software or equivalent using some other 
protocol) to accept only users having been granted access.


Even if I decide to make it local only, there is the possibility that 
anyone sharing the computer or network may peruse the database when 
you don't want them to.


That's the reason why OSes provide user login and file access rights. 
Again this is external to SQLite and its DB file(s). Also please note 
that SQLite is not a client-server engine, so access thru a LAN is 
prone to fail due to network file locking being poorly implemented.


Alternatively you can also use strong DB encryption with or without 
adding the authentication function. Google SQLite encryption and you'll 
find a number of answers, some of them correct.


JcD

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Darren Duncan]

Replying to myself.  This being said, going for the application-level encryption 
option would prevent you from using SQLite to do some useful things, such as 
being able to do a substring search for text in encrypted fields, since 
encrypted data is just a black box to it.  Typically the application-level 
solution is just encrypting a minimum number of fields, such as credit card 
numbers or SINs or passwords etc, that wouldn't be searched except for a whole 
value match.  To use SQLite normally as if it weren't encrypted but with it 
actually encrypted, you need the SEE or similar for that. -- Darren Duncan


On 2016-10-08 12:31 AM, Darren Duncan wrote:

So, if you don't want to pay the one-time fee for the SQLite Encryption
Extension et al to get database-level security, your only option really is to
encrypt individual fields at the application level that you want to protect, and
there are various free encryption libraries you can use for that, the specific
options depending on your choice of programming language.  But using those has
nothing to do with SQLite specifically, so your answer wouldn't be found on this
SQLite forum, but rather forums for your programming language. -- Darren Duncan

On 2016-10-08 12:18 AM, Damien Sykes-Lindley wrote:

Hi Darren,
You are correct in that genealogy is generally public. However more often than
not the information you want to publish may very well differ from what is in
your private database. You may have private notes telling you what you need to
do. You may have anecdotes shared by many family members that may need to be
kept private, at least until the involved parties are deceased or otherwise
choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in there
so you can easily group and contact appropriate family members for whatever
reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to be
used online then it is needless to say that authentication would be required for
various people to view it. Even if I decide to make it local only, there is the
possibility that anyone sharing the computer or network may peruse the database
when you don't want them to.
Kind regards,
Damien.
-Original Message- From: Darren Duncan
Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent,
hobbiest programmer. I have been blind since birth and thus all my computer
work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled programming
and so I guess I am still a beginner in many respects. However I don’t work in
C or C++, so most of my programming, if using a library, relies on precompiled
static or dynamic libraries. Or of course libraries that are written or
converted specifically for the language I work in (FreeBASIC).
Recently, I decided I needed to create a piece of software that could manage
family trees, since there seems to be a lack of screenreader accessible
genealogy managers out there. I was advised the best way to do this is to use
a database engine. I was also informed that SQLite is always a good choice for
databases.
I must admit, I have never worked with databases before and so now I am in the
process of learning SQL. However looking at the programming API for SQLite I
cannot see any means of password protecting the database without either buying
a commercial extension to do this, or recompiling SQLite with the
authentication extension. Due to financial constraints and unfamiliarity with
compiling in C both of these are not an option for me. Also I need a secure
way to do this, as I think I read that the SQLite version simply uses a table
to store the user data, which of course can be read and accessed elsewhere.
Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.


Damien,

Why do you need to password protect the database?

Genealogy information is generally of the public record variety so there is
nothing sensitive to protect.  I am making genealogy software myself and so am
familiar with many of the relevant issues.

I would say please explain why you think you need password protection for this
project and then the real issue at hand can be addressed.

If yours is a network application and you don't want people on the open internet
from accessing the database, fair enough, but that's an application-level
solution; what you're asking for here is that people who have direct access to
the SQLite database file are blocked by a password, and this I question.

-- Darren Duncan


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Protecting databases

[Darren Duncan]

So, if you don't want to pay the one-time fee for the SQLite Encryption 
Extension et al to get database-level security, your only option really is to 
encrypt individual fields at the application level that you want to protect, and 
there are various free encryption libraries you can use for that, the specific 
options depending on your choice of programming language.  But using those has 
nothing to do with SQLite specifically, so your answer wouldn't be found on this 
SQLite forum, but rather forums for your programming language. -- Darren Duncan


On 2016-10-08 12:18 AM, Damien Sykes-Lindley wrote:

Hi Darren,
You are correct in that genealogy is generally public. However more often than
not the information you want to publish may very well differ from what is in
your private database. You may have private notes telling you what you need to
do. You may have anecdotes shared by many family members that may need to be
kept private, at least until the involved parties are deceased or otherwise
choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in there
so you can easily group and contact appropriate family members for whatever
reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to be
used online then it is needless to say that authentication would be required for
various people to view it. Even if I decide to make it local only, there is the
possibility that anyone sharing the computer or network may peruse the database
when you don't want them to.
Kind regards,
Damien.
-Original Message- From: Darren Duncan
Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent,
hobbiest programmer. I have been blind since birth and thus all my computer
work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled programming
and so I guess I am still a beginner in many respects. However I don’t work in
C or C++, so most of my programming, if using a library, relies on precompiled
static or dynamic libraries. Or of course libraries that are written or
converted specifically for the language I work in (FreeBASIC).
Recently, I decided I needed to create a piece of software that could manage
family trees, since there seems to be a lack of screenreader accessible
genealogy managers out there. I was advised the best way to do this is to use
a database engine. I was also informed that SQLite is always a good choice for
databases.
I must admit, I have never worked with databases before and so now I am in the
process of learning SQL. However looking at the programming API for SQLite I
cannot see any means of password protecting the database without either buying
a commercial extension to do this, or recompiling SQLite with the
authentication extension. Due to financial constraints and unfamiliarity with
compiling in C both of these are not an option for me. Also I need a secure
way to do this, as I think I read that the SQLite version simply uses a table
to store the user data, which of course can be read and accessed elsewhere.
Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.


Damien,

Why do you need to password protect the database?

Genealogy information is generally of the public record variety so there is
nothing sensitive to protect.  I am making genealogy software myself and so am
familiar with many of the relevant issues.

I would say please explain why you think you need password protection for this
project and then the real issue at hand can be addressed.

If yours is a network application and you don't want people on the open internet
from accessing the database, fair enough, but that's an application-level
solution; what you're asking for here is that people who have direct access to
the SQLite database file are blocked by a password, and this I question.

-- Darren Duncan


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Damien Sykes-Lindley]

Hi Darren,
You are correct in that genealogy is generally public. However more often 
than not the information you want to publish may very well differ from what 
is in your private database. You may have private notes telling you what you 
need to do. You may have anecdotes shared by many family members that may 
need to be kept private, at least until the involved parties are deceased or 
otherwise choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in 
there so you can easily group and contact appropriate family members for 
whatever reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to 
be used online then it is needless to say that authentication would be 
required for various people to view it. Even if I decide to make it local 
only, there is the possibility that anyone sharing the computer or network 
may peruse the database when you don't want them to.

Kind regards,
Damien.
-Original Message- 
From: Darren Duncan

Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent, 
hobbiest programmer. I have been blind since birth and thus all my 
computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. However 
I don’t work in C or C++, so most of my programming, if using a library, 
relies on precompiled static or dynamic libraries. Or of course libraries 
that are written or converted specifically for the language I work in 
(FreeBASIC).
Recently, I decided I needed to create a piece of software that could 
manage family trees, since there seems to be a lack of screenreader 
accessible genealogy managers out there. I was advised the best way to do 
this is to use a database engine. I was also informed that SQLite is 
always a good choice for databases.
I must admit, I have never worked with databases before and so now I am in 
the process of learning SQL. However looking at the programming API for 
SQLite I cannot see any means of password protecting the database without 
either buying a commercial extension to do this, or recompiling SQLite 
with the authentication extension. Due to financial constraints and 
unfamiliarity with compiling in C both of these are not an option for me. 
Also I need a secure way to do this, as I think I read that the SQLite 
version simply uses a table to store the user data, which of course can be 
read and accessed elsewhere.

Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.


Damien,

Why do you need to password protect the database?

Genealogy information is generally of the public record variety so there is
nothing sensitive to protect.  I am making genealogy software myself and so 
am

familiar with many of the relevant issues.

I would say please explain why you think you need password protection for 
this

project and then the real issue at hand can be addressed.

If yours is a network application and you don't want people on the open 
internet

from accessing the database, fair enough, but that's an application-level
solution; what you're asking for here is that people who have direct access 
to

the SQLite database file are blocked by a password, and this I question.

-- Darren Duncan

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users