Re: [sqlite] New draft document on the new pointer-passing interfaces

[Peter Da Silva]

If you’re in a position to perform an attack that address space randomization 
can defend against, then you have already successfully performed a code 
execution attack.

And so far as I can tell... *any* constant strings in the executable, including 
things like elements of the SQL language itself, provide a mechanism for 
locating code.

Meanwhile, what this design does is remove an attack surface for promoting an 
SQL injection attack to a code execution attack. The comments about randomizing 
strings in the previous 
threads were about possible alternate ways of mitigating that attack, and have 
nothing to do with ASLR.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

>...the attacker is already able to read the process’s address space. The
rest of us here are saying that, in that case, as far as we’re concerned
the attacker has already won.

I understand that.  What I'm saying is your standard is not nuanced.
Applying a security standard that amounts to best practice web server
administration is insufficient for all environments.  Extra care is
warranted where compromised apps can be installed alongside yours.  Maybe
there is no exploit to take over the app or host but there could be very
good ones for your app to assist in eavesdropping and boosting social
engineering attacks.  The goal should be "secure in the wild" rather than
merely secure on the server.

A bit of reading reveals how the most spectacular SQLite exploits so far
occurred in desktop and portable environments marketed as a platforms for
users to commingle apps of their choice and quality.  Therefore, a more
realistic security threat standard comes from imagining an environment
where unknown threats and partially effective countermeasures are presently
in a precarious stalemate equilibrium.  Introducing an app with secure in
isolation but unorthodox implementation may not have the secure in the wild
guarantee that is expected.

At this point I've got no dog in this fight.  The new pointer API is now
fairly complete and can be mechanically patched to use a more sensible key
type.  Serious implementors anticipating widespread desktop or mobile
deployment can do this easily for themselves.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Jens Alfke]

> On Jul 28, 2017, at 2:46 PM, petern  wrote:
> 
> The attack vector is described well enough.  A penetration search harness
> would work directly from predictable accesses to the published key
> constants as mapped into process address space.  [Full ASLR decoding, see
> the paper, isn't needed for this.]   These well advertised accesses "light
> up" otherwise randomized program and stack layouts very well since there is
> no access for any other purpose.   CPU security might prevent directly
> forging pseudo-null pointers but they are as observable as subtype pointers
> and leak more address space layout information.

You’re getting _closer_ to something a non-expert could understand, but you’re 
not there yet. Keep trying, please.

It would help if you described what the attack requires, and what it would 
allow the attacker to do. Reading between the lines, I think you’re assuming 
the attacker is already able to read the process’s address space. The rest of 
us here are saying that, in that case, as far as we’re concerned the attacker 
has already won. And I don’t understand how this pointer feature would help 
such an attacker get any further.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Here I'm replying to Richard and subsequent comments on Richard's question.

The attack vector is described well enough.  A penetration search harness
would work directly from predictable accesses to the published key
constants as mapped into process address space.  [Full ASLR decoding, see
the paper, isn't needed for this.]   These well advertised accesses "light
up" otherwise randomized program and stack layouts very well since there is
no access for any other purpose.   CPU security might prevent directly
forging pseudo-null pointers but they are as observable as subtype pointers
and leak more address space layout information.

I suppose what you're really after is to see a demonstrable exploit to know
whether or not is a SQLite component or side channel.  For that, you'll
have to wait.  I don't have one at my fingertips but can imagine
possibilities and incentives.

About that 2017 paper.  It details a practical universal decoder against
the randomized memory space countermeasures on every major CPU
architecture.  It's not really that technical.  In fact, pointer leaks to
randomized memory were cited in Richard's essay where, in a jarring leap,
he went from the goal of preserving randomization to a new API with new
countermeasures. This reader was left wondering how he got there rather
than to a plan that finished up the deliberately incomplete but proven
subtype pointer API.

By introducing the paper, I was hoping for an appreciation of irony.  For
it is ironic how the new pseudo-null API leans even more heavily on a
waning shield of address space randomization than the subtype API did.







On Thu, Jul 27, 2017 at 11:41 AM, Jens Alfke  wrote:

>
> > On Jul 27, 2017, at 10:02 AM, petern 
> wrote:
> >
> > Are you able to put two facts together?
> >
> > What prevents stack busting or other code injection attacks on an
> otherwise
> > valid pseudo-null pointer by simply decoding the address space and
> > observing where strcmp() loads a register to one of the pointer "keys"
> > you've insisted be conveniently published for hackers in the data
> segment?
>
> Peter, you’re
> (a) assuming a lot of specialized domain knowledge, by using jargon and
> referring to highly technical papers;
> (b) referring vaguely to threats instead of describing a clear problem or
> attack vector;
> (c) speaking very condescendingly to the people you’re trying to convince.
>
> None of these help your case at all. I’m interested in the security
> implications of this API, and I’ve got some security knowledge, but not
> apparently enough to follow along. Richard is right: please describe
> clearly a situation where this API results in an attack vector.
>
> —Jens
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Keith Medcalf]

On Thursday, 27 July, 2017 11:03, petern  wrote:

>Richard. Were you aware of this paper?

>http://www.cs.vu.nl/~herbertb/download/papers/anc_ndss17.pdf

>Are you able to put two facts together?

>What prevents stack busting or other code injection attacks on an
>otherwise valid pseudo-null pointer by simply decoding the address 
>space and observing where strcmp() loads a register to one of the 
>pointer "keys" you've insisted be conveniently published for 
>hackers in the data segment?

Absolutely nothing prevents it.  However, this would be an application problem 
and not an SQLite problem.  The purpose of using the pointer interface is to 
disable the ability of "unsanitized user created SQL" (aka Johhny Tables) from 
being able to pass pointers into and out of the SQLite3 library via crafted 
SQL.  The application developer must specifically code the ability to send and 
receive pointers.  This can no longer be done and will no longer return 
meaningful results.  It does nothing to secure the application code from 
exploitation by "other means".

In other words, if a "badly written application" were subject to any sort of 
machine code injection attack, there is nothing to whatsoever to prevent the 
injected code from accessing the SQLite engine functions (or the OS kernel or 
anything else), binding pointers using the new interface, and extracting 
pointers.  Nothing except that if the application already has this 
vulnerability (code injection via whatever means) there are much easier (not to 
mention smaller and simpler) methods to retrieve the same data which does not 
depend on using the database engine.




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Jens Alfke]

> On Jul 27, 2017, at 10:02 AM, petern  wrote:
> 
> Are you able to put two facts together?
> 
> What prevents stack busting or other code injection attacks on an otherwise
> valid pseudo-null pointer by simply decoding the address space and
> observing where strcmp() loads a register to one of the pointer "keys"
> you've insisted be conveniently published for hackers in the data segment?

Peter, you’re 
(a) assuming a lot of specialized domain knowledge, by using jargon and 
referring to highly technical papers;
(b) referring vaguely to threats instead of describing a clear problem or 
attack vector;
(c) speaking very condescendingly to the people you’re trying to convince.

None of these help your case at all. I’m interested in the security 
implications of this API, and I’ve got some security knowledge, but not 
apparently enough to follow along. Richard is right: please describe clearly a 
situation where this API results in an attack vector.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Richard Hipp]

On 7/27/17, petern  wrote:
>
> What prevents stack busting or other code injection attacks on an otherwise
> valid pseudo-null pointer by simply decoding the address space and
> observing where strcmp() loads a register to one of the pointer "keys"
> you've insisted be conveniently published for hackers in the data segment?
>

I do not understand what this sentence means.  Can you explain it
again in simpler terms?

Refresh my memory please:  What exactly (and succinctly) is your
complain with the current sqlite3_bind_pointer(),
sqlite3_result_pointer(), and sqlite3_value_pointer() design?  Are
there multiple complains?  Can you enumerate them?  Please be as
specific as possible.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Richard. Were you aware of this paper?

http://www.cs.vu.nl/~herbertb/download/papers/anc_ndss17.pdf

Are you able to put two facts together?

What prevents stack busting or other code injection attacks on an otherwise
valid pseudo-null pointer by simply decoding the address space and
observing where strcmp() loads a register to one of the pointer "keys"
you've insisted be conveniently published for hackers in the data segment?








On Tue, Jul 25, 2017 at 10:43 AM, Richard Hipp  wrote:

> On 7/24/17, petern  wrote:
> > Great.  But, if this is an ultimate replacement for BLOB'ed pointers,
> these
> > new pseudo-null pointers must support SQLITE_STATIC and destructor
> function
> > pointer lifetime disposition for those migrating their code.
>
> Nobody is forcing you to migrate your legacy code to the new API.
> Anything that worked for you in 3.19.3 (or earlier) will continue to
> work just as well in 3.20.0.  If what you are doing now works for you,
> then you are welcomed to keep doing exactly the same in the future.
>
> >
> > Why can't the producer destructor disposition be preserved within a chain
> > of application functions by subsequent consumers passing SQLITE_STATIC
> > disposition as they do now?
>
> I cannot say with precision because your proposal is vague.  But what
> you want to do will very likely use extra memory and extra CPU cycles
> for the overwhelmingly common case where pointers are not being
> passed.  We do not want to burden the common case for the convenience
> of an outlier.
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Richard Hipp]

On 7/24/17, petern  wrote:
> Great.  But, if this is an ultimate replacement for BLOB'ed pointers, these
> new pseudo-null pointers must support SQLITE_STATIC and destructor function
> pointer lifetime disposition for those migrating their code.

Nobody is forcing you to migrate your legacy code to the new API.
Anything that worked for you in 3.19.3 (or earlier) will continue to
work just as well in 3.20.0.  If what you are doing now works for you,
then you are welcomed to keep doing exactly the same in the future.

>
> Why can't the producer destructor disposition be preserved within a chain
> of application functions by subsequent consumers passing SQLITE_STATIC
> disposition as they do now?

I cannot say with precision because your proposal is vague.  But what
you want to do will very likely use extra memory and extra CPU cycles
for the overwhelmingly common case where pointers are not being
passed.  We do not want to burden the common case for the convenience
of an outlier.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Stephen Chrzanowski]

My tone isn't about the technical development discussion.  Its about my
subscribing to this forum and seeing my 11 year olds mentality shine
through with his "I'm not getting the attention I want, so I'm going to
yell and scream and pout until I get what I want".  Perhaps it is a
language barrier, or a cultural difference I don't know about, but, in the
last SEVERAL notes (Pretty much the post right after Dr Hipps notice of the
draft in this particular thread), this hasn't been levied to the technical
side of things, but an individual who is screaming and making demands.
Making demands isn't a discussion.

No one is perfect, I get that.  And please, do discuss the possible or
impossibility of the security faults of the library.  I'm interested in
that.  I'm not interested in seeing the demands.


On Tue, Jul 25, 2017 at 1:00 PM, Jens Alfke  wrote:

>
> > On Jul 25, 2017, at 9:39 AM, Stephen Chrzanowski 
> wrote:
> >
> > Your attitude towards a public forum and bully attempts isn't required
> > here.  I'd ask YOU to leave based on the fact that your behavior is
> > anything but professional, as I'm not interested in your self proclaimed
> > expertise.
>
> Stephen, I don’t think this is called for. Peter is being somewhat
> vehement (as are you) but I don’t think anything he’s said counts as
> bullying or is otherwise out of bounds in a technical discussion.
>
> > My expertise extends to the knowledge that what Dr Hipp has written is of
> > the best interest of all the BILLIONS of devices that are out in the
> field
>
> Argument by appeal to authority isn’t a good idea here. I’m sure Dr Hipp
> doesn’t consider himself infallible (even if he is a doctor), and part of
> the reason for open development is to catch the mistakes and incorrect
> assumptions that all of us are prone to.
>
> I have some modest knowledge of computer security, and I know it’s
> vulnerable to very, very subtle design errors that can leave openings for
> attack. (Security bugs are unique in that they are not triggered randomly,
> but are actively hunted down and exploited by human-level intelligences.)
> It’s in everyone’s best interest that any security related issues in the
> design be thoroughly debated before this is released.
>
> > If there is an aspect of this software you do not like, fork it, and
> write
> > your own code to remove what Dr Hipp and his team has put into the code.
>
> Again, this is unhelpful. We all understand the basics of open source
> development. Forking isn’t a solution, it’s an admission of failure.
>
> —Jens
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Jens Alfke]

> On Jul 25, 2017, at 9:39 AM, Stephen Chrzanowski  wrote:
> 
> Your attitude towards a public forum and bully attempts isn't required
> here.  I'd ask YOU to leave based on the fact that your behavior is
> anything but professional, as I'm not interested in your self proclaimed
> expertise.

Stephen, I don’t think this is called for. Peter is being somewhat vehement (as 
are you) but I don’t think anything he’s said counts as bullying or is 
otherwise out of bounds in a technical discussion.

> My expertise extends to the knowledge that what Dr Hipp has written is of
> the best interest of all the BILLIONS of devices that are out in the field

Argument by appeal to authority isn’t a good idea here. I’m sure Dr Hipp 
doesn’t consider himself infallible (even if he is a doctor), and part of the 
reason for open development is to catch the mistakes and incorrect assumptions 
that all of us are prone to.

I have some modest knowledge of computer security, and I know it’s vulnerable 
to very, very subtle design errors that can leave openings for attack. 
(Security bugs are unique in that they are not triggered randomly, but are 
actively hunted down and exploited by human-level intelligences.) It’s in 
everyone’s best interest that any security related issues in the design be 
thoroughly debated before this is released.

> If there is an aspect of this software you do not like, fork it, and write
> your own code to remove what Dr Hipp and his team has put into the code.

Again, this is unhelpful. We all understand the basics of open source 
development. Forking isn’t a solution, it’s an admission of failure.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Stephen Chrzanowski]

On Tue, Jul 25, 2017 at 12:25 PM, petern 
wrote:

> You're trying to change the topic to the security model. This thread is
> supposed to be about a lengthy beyond the pale proposal that named all
> manner of hypothetical boogie men before concluding the only way is a
> "nuclear solution" as in: "Let's just nuke it, that's only way to be
> safe".   I'll ask again. Did you read the proposal itself? If not, why are
> you responding on this thread?  What is your expertise exactly?
>
>
My expertise extends to the knowledge that what Dr Hipp has written is of
the best interest of all the BILLIONS of devices that are out in the field,
actively used knowingly, or unknowingly by the users who use the library.

Your attitude towards a public forum and bully attempts isn't required
here.  I'd ask YOU to leave based on the fact that your behavior is
anything but professional, as I'm not interested in your self proclaimed
expertise.  Perhaps in the software development area, you are an expert in
your own products you deploy, but absolutely NOT in the area of what this
public domain library is being used for.

If there is an aspect of this software you do not like, fork it, and write
your own code to remove what Dr Hipp and his team has put into the code.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Peter Da Silva]

On 7/25/17, 11:25 AM, "sqlite-users on behalf of petern" 
 wrote:
> You're trying to change the topic to the security model.

All I was doing was pointing out that hiding the type information from 
attackers is not a requirement, so the fact that it’s visible if you examine 
the binary or source is not relevant.

Now you’ve acknowledged that you’re not pointing out any actual security flaw, 
we’re done, right?

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Keith Medcalf]

Richard,

Your description of the rational for the new pointer values is sound and quite 
reasonable.  I believe that it meets all the requirements you have set out to 
prevent SQL from being used to either retrieve or set arbitrary internal use 
(that is internal to both the application and SQLite3) pointers that was not 
directly intended (and pre-coded) by the application developer.  How much of a 
security risk being able to do so would present is not particularly relevant to 
the implementation and is difficult to estimate because it certainly varies 
considerably by host OS.  Simply eliminating the possibility of vulnerability 
via this method is adequate and commendable.

It should be noted however that this does not in any way prevent or mitigate 
that a "malicious application" will in fact be able to access or set pointer 
values, however, this is not within the scope of the change.  The scope of the 
change is to protect existing and future applications from being used to 
perform side channel pointer attacks via crafted SQL injections that were not 
intended by the application author.  At this it will succeed brilliantly.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.

>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp
>Sent: Monday, 24 July, 2017 05:54
>To: General Discussion of SQLite Database
>Subject: [sqlite] New draft document on the new pointer-passing
>interfaces
>
>https://www.sqlite.org/draft/bindptr.html
>
>--
>D. Richard Hipp
>d...@sqlite.org
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

You're trying to change the topic to the security model. This thread is
supposed to be about a lengthy beyond the pale proposal that named all
manner of hypothetical boogie men before concluding the only way is a
"nuclear solution" as in: "Let's just nuke it, that's only way to be
safe".   I'll ask again. Did you read the proposal itself? If not, why are
you responding on this thread?  What is your expertise exactly?

Also, where is your critique of my suggestion to simply and logically
expand the well proven and secure subtype mechanism?  Why do you presume
that Richard's idea to suddenly create a parallel API is automatically the
best plan?  What is your expertise on the main issue of how to pass around
runtime object pointers?  If you reply to this thread again, please
demonstrate some expertise on the issue at hand.

Now I'll address the question of remote attack path in case somebody
mistakes a superficial question for a valid point.  I don't know any exact
attack path, local or remote, and neither does Richard.  That's the point.
However, all unknowns being equal, if there is a local way to inject
pointers then there could be a remote way.   Whatever the method, the
attacker can then use the supposedly secure constant space names everywhere
the application is deployed without further opposition.   If you recall, I
pointed out that the programmer is free to rotate or randomize integer
subtypes of the current API at runtime if it's keeping them from sleep.  So
in this vein of these hypothetical threats, constant space names are no
magic bullet.  They have drawbacks too.

That being said, let me be very clear once again for short attention
spans.  There is no attack path in the current API.  It works fine and that
was proven in the field!  So please, BEFORE RESPONDING TO THIS POST, DO
READ THE PROPOSAL WHERE THIS IS CLEARLY STATED!



On Tue, Jul 25, 2017 at 6:11 AM, Peter Da Silva <
peter.dasi...@flightaware.com> wrote:

> On 7/24/17, 7:20 PM, "sqlite-users on behalf of petern" <
> sqlite-users-boun...@mailinglists.sqlite.org on behalf of
> peter.nichvolo...@gmail.com> wrote:
> > Justifications presented in the proposal claim hardwired constants for
> mandatory lock and key style pointer value receiving are a great idea
> because SQL can't generate constant space strings.
>
> This has nothing to do with the secrecy or otherwise of these strings, but
> to prevent developers from _even in principle_ implementing a mechanism for
> passing these strings in from SQL. If it is not possible to inject the
> string, then it doesn’t matter if they’re secret or not. Think of it as
> preventing the creation of a “cross domain” attack from an insecure module.
>
> You might as well complain that the names of internal functions in SQLite
> are known. There’s no way from SQL to call them, so it’s not an attack
> surface.
>
> > On the local device the hacker attack space would be immediately
> narrowed to constants listed in the executable which, I might add, are
> guaranteed to work on remote copy of the same application!
>
> How do you figure? What is the attack path you’re postulating?
>
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Justifications presented in the proposal claim hardwired constants for
mandatory lock and key style pointer value receiving are a great idea
because SQL can't generate constant space strings.  And, this is true
provided the executable is secret and remote.  I know there are a lot of
web server jockeys on this forum that are quick with the PHP answers so,
perhaps, this assertion makes sense to many.

What I am pointing out is how those same mandatory hardwired secret
constants work against security in the domain of local DB on a portable
device. On the local device the hacker attack space would be immediately
narrowed to constants listed in the executable which, I might add, are
guaranteed to work on remote copy of the same application!

As well, this particular justification apparently is the reason to make
something completely new and utterly parallel with the existing subtype
solution which works fine and could be extended to do the job.  Did you
read the part of the proposal essay where the existing API is mentioned?

Now, what about the first part of my reply?  No comment on that?  You
accept what I said there?  I'm glad to learn people are coming to their
senses. :-)



On Mon, Jul 24, 2017 at 1:52 PM, Peter Da Silva <
peter.dasi...@flightaware.com> wrote:

> On 7/24/17, 3:50 PM, "sqlite-users on behalf of petern" <
> sqlite-users-boun...@mailinglists.sqlite.org on behalf of
> peter.nichvolo...@gmail.com> wrote:
> > BTW, if the hypothetical attacker has a copy of the application, aren't
> the constant space pointer access keys' string addresses all there in clear
> text
>
> But that’s not part of the security model, so what’s the problem?
>
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Peter Da Silva]

On 7/24/17, 3:50 PM, "sqlite-users on behalf of petern" 
 wrote:
> BTW, if the hypothetical attacker has a copy of the application, aren't the 
> constant space pointer access keys' string addresses all there in clear text

But that’s not part of the security model, so what’s the problem?
 

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Your proposal does not walk through the alternative of sticking with
subtypes to add non-persistent sqlite3_bind_subtype() and corresponding
sqlite3_column_subtype() methods.  With a few extra lines and some
imagination can't this more straightforward alternative be combined with
the existing BLOB pointer interface to reach the desired outcome in FTS and
carray?

BTW, if the hypothetical attacker has a copy of the application, aren't the
constant space pointer access keys' string addresses all there in clear
text.  The castle walls will be no higher than those of a discretionary
pointer access protocol with subtypes.  In fact, subtypes could afford
greater security at runtime if the programmer rotates or otherwise
randomizes the type id's.






On Mon, Jul 24, 2017 at 10:05 AM, Richard Hipp  wrote:

> On 7/24/17, petern  wrote:
> >
> > Are sqlite3_result_subtype() and sqlite3_value_subtype() being deprecated
> > in light of the duplicate functionality?
> >
>
> No.  The subtype() interfaces were originally created for completely
> unrelated purposes (specifically to identify validated JSON text in
> the JSON1 extension) and will continue to live on to serve those
> unrelated purposes.
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Gwendal.

Yes.  You've missed something.  My application is working code not a
hypothetical feature request.  BLOB application object pointer lifetime
presently works precisely as I've described and without memory leak.  My
point (and Dominique's point) was that this proposal as it stands isn't a
drop in replacement for that BLOB'ed application object pointer protocol.

Your reply also suggests superficial understanding.  For example, regarding
the SQLITE_TRANSIENT BLOB disposition you mentioned.   That one is
generally not useful here unless the particular application object can be
completely materialized elsewhere in memory from shallow copy without loss
of fidelity or introduction of memory leak.  I'm not talking about copying
dumb strings.  These are highly stateful and possibly third party objects
containing still more pointers and reference counters.  They cannot be
memcopied around willy-nilly.

Take a look at my earlier email that really applied the BLOB object
technique to functions if you have genuine interest.  This isn't some half
baked idea that came about in a flight of fancy.  There is a thread of
emails leading up to this point.





On Mon, Jul 24, 2017 at 11:12 AM, Gwendal Roué 
wrote:

>
> > Le 24 juil. 2017 à 19:02, petern  a écrit :
> >
> > Great.  But, if this is an ultimate replacement for BLOB'ed pointers,
> these
> > new pseudo-null pointers must support SQLITE_STATIC and destructor
> function
> > pointer lifetime disposition for those migrating their code.
>
> You're right that the new APIs make pointer values unsuitable for
> *intermediate* values.
>
> For example, `SELECT free(use(initialize(alloc(123`, while possible,
> is out of scope. One can define such functions, but it's very risky to
> misuse them, leak memory, or double-free.
>
> The new pointer values indeed look *designed* so that the  lifetime of
> pointed values is managed *outside* of any statement execution.
>
> I fail to see how blob'ed pointers did not have the exact same issue.
> SQLITE_STATIC and SQLITE_TRANSIENT manage the lifetime of the blob content,
> not of the content pointed by a blob'ed pointer. Or did I miss something?
>
> Gwendal Roué
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Gwendal Roué]

> Le 24 juil. 2017 à 19:02, petern  a écrit :
> 
> Great.  But, if this is an ultimate replacement for BLOB'ed pointers, these
> new pseudo-null pointers must support SQLITE_STATIC and destructor function
> pointer lifetime disposition for those migrating their code.

You're right that the new APIs make pointer values unsuitable for 
*intermediate* values.

For example, `SELECT free(use(initialize(alloc(123`, while possible, is out 
of scope. One can define such functions, but it's very risky to misuse them, 
leak memory, or double-free.

The new pointer values indeed look *designed* so that the  lifetime of pointed 
values is managed *outside* of any statement execution.

I fail to see how blob'ed pointers did not have the exact same issue. 
SQLITE_STATIC and SQLITE_TRANSIENT manage the lifetime of the blob content, not 
of the content pointed by a blob'ed pointer. Or did I miss something?

Gwendal Roué

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Gwendal Roué]

> Le 24 juil. 2017 à 19:02, petern  a écrit :
> 
> To those posting low information congratulatory notes on this thread, you'd
> better hold off on popping those champagne corks.  The current API already
> contains irreversible additions to solve this problem that fell short.

Congrats can also go to clear documentation, scope, and rationale, even if the 
feature can be discussed :-)

Gwendal

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Richard Hipp]

On 7/24/17, petern  wrote:
>
> Are sqlite3_result_subtype() and sqlite3_value_subtype() being deprecated
> in light of the duplicate functionality?
>

No.  The subtype() interfaces were originally created for completely
unrelated purposes (specifically to identify validated JSON text in
the JSON1 extension) and will continue to live on to serve those
unrelated purposes.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[petern]

Great.  But, if this is an ultimate replacement for BLOB'ed pointers, these
new pseudo-null pointers must support SQLITE_STATIC and destructor function
pointer lifetime disposition for those migrating their code.

Why can't the producer destructor disposition be preserved within a chain
of application functions by subsequent consumers passing SQLITE_STATIC
disposition as they do now?   Isn't this feature just an accident of
statement scope controlled destruction that will continue to work with
tracked lifetime pseudo-null pointers?

BTW, let's call them what they are.  These are explicit pseudo-nulls for
the purpose of keeping pointer bits out of band from hacker SQL.

Also.

What is to stop black budget funded developers from creating popular
applications in the wild which preserve penetration channels of BLOB
pointers the original way?  Total security improvement justifications for
the pseudo-null pointer API are specious if the API is merely another
alternative.

Are sqlite3_result_subtype() and sqlite3_value_subtype() being deprecated
in light of the duplicate functionality?

Supplementing/deprecating the already secure sqlite3_X_subtype() API with a
more complete and pointer leak opaque replacement sqlite3_X_pointer() API
seems a worthy goal.  But, if that's the plan, where is the rest to the
proposal?  Honestly, it appears all you've proposed so far is yet another
way to pass pointers more aligned with the whims of your present tastes for
FTS3 MATCH, FTS5 extensions, and one code sample.

To those posting low information congratulatory notes on this thread, you'd
better hold off on popping those champagne corks.  The current API already
contains irreversible additions to solve this problem that fell short.


On Mon, Jul 24, 2017 at 4:54 AM, Richard Hipp  wrote:

> https://www.sqlite.org/draft/bindptr.html
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Keith Medcalf]

Minor typo's and grammo's:

2.1) suppose -> supposed
  For example, the first argument to the snippet() function is suppose to be a 
special column
  space instead of the fts3cursor object it was suppose to be operating on

2.2) forge pointer -> forge a pointer
  forge pointer without the help of SQLite.

3)  forge a pointers -> forge pointers
  Hence, it is not possible for SQL to forge a pointers.

3.1) and -> an
  been passed and invalid pointer.

4)  operating -> operation
  insert a sort operating prior

4)  insert -> inserted
  pointer value insert into a query



---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.

>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp
>Sent: Monday, 24 July, 2017 05:54
>To: General Discussion of SQLite Database
>Subject: [sqlite] New draft document on the new pointer-passing
>interfaces
>
>https://www.sqlite.org/draft/bindptr.html
>
>--
>D. Richard Hipp
>d...@sqlite.org
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Dominique Devienne]

On Mon, Jul 24, 2017 at 1:54 PM, Richard Hipp  wrote:

> https://www.sqlite.org/draft/bindptr.html


Thanks. Very helpful. Still unsure whether not having a destructor D for
pointer P is a good thing though.
The text explicitly says the pointer is "destroyed" when not flowing
directly from producer to consumer,
which could mean calling the D(P) in that case. The current semantic seem
to imply the pointer producer
is longer lived than the pointer consumer, and thus no lifetime management
is necessary, but will that always be the case?

Also, the new pointer APIs are said to be an enhanced version of
*_subtype(), so there's some overlap,
but I guess one can use both at the same time, no? Or is _subtype()
deprecated? JSON1 uses _subtype
I believe, so _subtype() has more uses than the one _pointer() APIs replace
it for? Maybe an extra paragraph
regarding _pointer vs _subtype() would be helpful? At least I'd like your
perspective on it.

Thanks, --DD

Minor fixes:
s/Upping The Thread Level/Upping The Threat Level/
s/Consideration was giving to using integer values as the pointer
type/Consideration was giveb to using integer values as the pointer type/
s/if a pointer value insert into a query/if a pointer value inserted into a
query/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Peter Da Silva]

If you’re concerned about people faking pointers, you could always go with 
something like “org.sqlite.fts3.01Yzg5x.snippet”.

But given this design, you need to have already injected a C extension to even 
access the pointers, so I’m not sure what the issue is.

On 7/24/17, 9:19 AM, "sqlite-users on behalf of Hick Gunter" 
<sqlite-users-boun...@mailinglists.sqlite.org on behalf of h...@scigames.at> 
wrote:

Naming conventions (if strictly adhered to) are moderately good at avoiding 
conflicts, but take the guesswork out of "faking a pointer". It also assumes 
that pointers for "whatever" are interchangeable between different queries in a 
process, i.e. passing a "whatever" pointer from statement A to a function in 
statement B does not pose a risk.

-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Peter Da Silva
Gesendet: Montag, 24. Juli 2017 15:37
An: SQLite mailing list <sqlite-users@mailinglists.sqlite.org>
    Betreff: Re: [sqlite] New draft document on the new pointer-passing 
interfaces

What about imposing some structure on the pointer type strings that uses a 
guaranteed unique substring, for example “org.sqlite.fts3.snippet”,  to ensure 
there wouldn’t be accidental conflicts?

On 7/24/17, 6:54 AM, "sqlite-users on behalf of Richard Hipp" 
<sqlite-users-boun...@mailinglists.sqlite.org on behalf of d...@sqlite.org> 
wrote:

https://www.sqlite.org/draft/bindptr.html

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: h...@scigames.at

This communication (including any attachments) is intended for the use of 
the intended recipient(s) only and may contain information that is 
confidential, privileged or legally protected. Any unauthorized use or 
dissemination of this communication is strictly prohibited. If you have 
received this communication in error, please immediately notify the sender by 
return e-mail message and delete all copies of the original communication. 
Thank you for your cooperation.


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Hick Gunter]

Naming conventions (if strictly adhered to) are moderately good at avoiding 
conflicts, but take the guesswork out of "faking a pointer". It also assumes 
that pointers for "whatever" are interchangeable between different queries in a 
process, i.e. passing a "whatever" pointer from statement A to a function in 
statement B does not pose a risk.

-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Peter Da Silva
Gesendet: Montag, 24. Juli 2017 15:37
An: SQLite mailing list <sqlite-users@mailinglists.sqlite.org>
Betreff: Re: [sqlite] New draft document on the new pointer-passing interfaces

What about imposing some structure on the pointer type strings that uses a 
guaranteed unique substring, for example “org.sqlite.fts3.snippet”,  to ensure 
there wouldn’t be accidental conflicts?

On 7/24/17, 6:54 AM, "sqlite-users on behalf of Richard Hipp" 
<sqlite-users-boun...@mailinglists.sqlite.org on behalf of d...@sqlite.org> 
wrote:

https://www.sqlite.org/draft/bindptr.html

--
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: h...@scigames.at

This communication (including any attachments) is intended for the use of the 
intended recipient(s) only and may contain information that is confidential, 
privileged or legally protected. Any unauthorized use or dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please immediately notify the sender by return e-mail message and 
delete all copies of the original communication. Thank you for your cooperation.


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Peter Da Silva]

What about imposing some structure on the pointer type strings that uses a 
guaranteed unique substring, for example “org.sqlite.fts3.snippet”,  to ensure 
there wouldn’t be accidental conflicts?

On 7/24/17, 6:54 AM, "sqlite-users on behalf of Richard Hipp" 
 
wrote:

https://www.sqlite.org/draft/bindptr.html

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Eric Minbiole]

>https://www.sqlite.org/draft/bindptr.html

Thank you very much for this, detailed, informative write-up, Dr Hipp. It's
very helpful to see the sensible rationale behind the new interfaces.
Thanks for continuing to enhance the API!
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] New draft document on the new pointer-passing interfaces

[Gwendal Roué]

> Le 24 juil. 2017 à 13:54, Richard Hipp  a écrit :
> 
> https://www.sqlite.org/draft/bindptr.html

Thank you very much for this detailed rationale!

Gwendal Roué

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users