Re: [GENERAL] Shared system resources

[Andres Freund]

On 2015-12-23 11:55:36 -0500, George Neuner wrote:
> With sufficient privileges, a debugger-like process can attach and
> examine the memory of a running - or just terminated - process, but it
> won't have access to discarded (unmapped) memory.

You just have to load a kernel module to do so - it's just a few dozen
lines of C.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[Jim Nasby]

On 12/23/15 12:05 PM, Melvin Davidson wrote:

As others have pointed out, worrying about someone accessing database
shared memory is like worrying about an asteroid striking the earth and
wiping out all life.
It's a one in a billion chance compared to other security violations
that can occur.
You are better off concentrating on proper O/S security and user/table
permissions. That is how to implement database security!


True, but in my experience security audits have nothing to do with 
security and everything to do with marking off checkboxes and 
complicating lawsuits. ;)

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[Jim Nasby]

On 12/23/15 7:55 AM, oleg yusim wrote:

Sure David. For simplicity of modeling here, let's assume raw database
data was encrypted and the only possibility for attacker to get
something from raw data is to go and dig into sessions leftovers. Now,
with that has been said, do you happen to know what information actually
gets stored during the session into memory, reserved by session process?
I'm trying to determine, basically, does it even worth a talk - maybe
there is nothing at all valuable.


There's tons of raw data stored in the shared memory segment, and some 
of that can be copied to process local memory at any time. If they OS 
doesn't secure that adequately there's certainly nothing that Postgres 
or any other database can do about it.


As David said, by the time you're concerned about someone getting access 
to raw memory it's already way too late.


As for memory pages being zero'd after they are returned to the OS, 
that's entirely up to the OS. The only thing you could do on the 
Postgres side is to compile with memory debugging enabled, which will 
over-write any memory that's freed with a magic value. That's done to 
help hunt down memory access bugs, but would have the obvious side 
effect of obliterating any data that was in the page.


Uh, only thing is, I don't know if this is done if we're going to be 
returning the memory to the OS.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[David Wilson]

On Wed, Dec 23, 2015 at 07:07:31AM -0600, oleg yusim wrote:

> May we run into situation, when attacker dumps memory and analyses it
> for valuable content, instead of reserving it for own process, where
> it would be zeroed? My understanding, it is a possibility. Does kernel
> have any safeguard against it?

Sure it might be possible, but they would not have much useful
information about which old processes the pages belonged to, and
besides, they could most likely simply dump memory of a connected client
in this case, or indeed just examine the filesystem or cache to get at
the raw PG database files.

Once someone has this level of access to the system it's not really
useful to model threats much further.

One minor correction from my first mail: MAP_UNINITIALIZED is indeed
accessible to non-root, but as George mentions only when a non-default
kernel parameter has been enabled.


David


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[oleg yusim]

HI George,

Thanks, this information clears the situation. Now, question to you and
David.

May we run into situation, when attacker dumps memory and analyses it for
valuable content, instead of reserving it for own process, where it would
be zeroed? My understanding, it is a possibility. Does kernel have any
safeguard against it?

Thanks,

Oleg

On Wed, Dec 23, 2015 at 2:13 AM, George Neuner  wrote:

> On Tue, 22 Dec 2015 23:21:27 +, David Wilson 
> wrote:
>
> >On Linux the memory pages of an exiting process aren't sanitized at
> >exit, however it is impossible(?) for userspace to reallocate them
> >without the kernel first zeroing their contents.
>
> Not impossible, but it requires a non-standard kernel.
>
> Since 2.6.33, mmap() accepts the flag MAP_UNINITIALIZED which allows
> pages to be mapped without being cleared.  The flag has no effect
> unless the kernel was built with CONFIG_MMAP_ALLOW_UNINITIALIZED.
>
>
> No mainstream distro enables this.  AFAIK, there is NO distro at all
> that enables it ... it's too big a security risk for a general purpose
> system.  It's intended to support embedded systems where the set of
> programs is known.
>
> George
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

Re: [GENERAL] Shared system resources

[oleg yusim]

Jim,

Help me out with this statement:

"There's tons of raw data stored in the shared memory segment, and some of
that can be copied to process local memory at any time. If they OS doesn't
secure that adequately there's certainly nothing that Postgres or any other
database can do about it."

To my knowledge, many databases are using what called TDE to encrypt data
at rest and protect data from being accessed by attacker on host this way.
Here is the reference to quick guide on it:
https://www.simple-talk.com/sql/database-administration/transparent-data-encryption/

Now, when you are saying " tons of raw data stored in the shared memory
segment, and some of that can be copied to process local memory at any time"
what kind of memory you are referring too? Is it that files, which
generally end up being protected with TDE, or is it a buffer memory, which
get's used by database processes, but doesn't belong to database permanent
storage?

Can you give me more details here, so I would understand the actual mapping
and scale of the issue?

Thanks,

Oleg

On Wed, Dec 23, 2015 at 9:55 AM, Jim Nasby  wrote:

> On 12/23/15 7:55 AM, oleg yusim wrote:
>
>> Sure David. For simplicity of modeling here, let's assume raw database
>> data was encrypted and the only possibility for attacker to get
>> something from raw data is to go and dig into sessions leftovers. Now,
>> with that has been said, do you happen to know what information actually
>> gets stored during the session into memory, reserved by session process?
>> I'm trying to determine, basically, does it even worth a talk - maybe
>> there is nothing at all valuable.
>>
>
> There's tons of raw data stored in the shared memory segment, and some of
> that can be copied to process local memory at any time. If they OS doesn't
> secure that adequately there's certainly nothing that Postgres or any other
> database can do about it.
>
> As David said, by the time you're concerned about someone getting access
> to raw memory it's already way too late.
>
> As for memory pages being zero'd after they are returned to the OS, that's
> entirely up to the OS. The only thing you could do on the Postgres side is
> to compile with memory debugging enabled, which will over-write any memory
> that's freed with a magic value. That's done to help hunt down memory
> access bugs, but would have the obvious side effect of obliterating any
> data that was in the page.
>
> Uh, only thing is, I don't know if this is done if we're going to be
> returning the memory to the OS.
>
> --
> Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
> Experts in Analytics, Data Architecture and PostgreSQL
> Data in Trouble? Get it in Treble! http://BlueTreble.com
>

Re: [GENERAL] Shared system resources

[oleg yusim]

Sure David. For simplicity of modeling here, let's assume raw database data
was encrypted and the only possibility for attacker to get something from
raw data is to go and dig into sessions leftovers. Now, with that has been
said, do you happen to know what information actually gets stored during
the session into memory, reserved by session process? I'm trying to
determine, basically, does it even worth a talk - maybe there is nothing at
all valuable.

Thanks,

Oleg

On Wed, Dec 23, 2015 at 7:41 AM, David Wilson  wrote:

> On Wed, Dec 23, 2015 at 07:07:31AM -0600, oleg yusim wrote:
>
> > May we run into situation, when attacker dumps memory and analyses it
> > for valuable content, instead of reserving it for own process, where
> > it would be zeroed? My understanding, it is a possibility. Does kernel
> > have any safeguard against it?
>
> Sure it might be possible, but they would not have much useful
> information about which old processes the pages belonged to, and
> besides, they could most likely simply dump memory of a connected client
> in this case, or indeed just examine the filesystem or cache to get at
> the raw PG database files.
>
> Once someone has this level of access to the system it's not really
> useful to model threats much further.
>
> One minor correction from my first mail: MAP_UNINITIALIZED is indeed
> accessible to non-root, but as George mentions only when a non-default
> kernel parameter has been enabled.
>
>
> David
>

Re: [GENERAL] Shared system resources

[oleg yusim]

John,

To my knowledge, TDE is employed not only by Microsoft, but by Oracle too.
I recall it also has a mechanism of protecting keys. Here are references:

https://msdn.microsoft.com/en-us/library/bb934049.aspx
http://www.oracle.com/technetwork/database/options/advanced-security/index-099011.html

Thank you very much for that piece:

"In PostgreSQL 'shared memory' has a quite specific meaning, its referring
to the pool of buffer memory (ram) shared by all postgres server
processes.   this is primarily used as the buffer cache. In a properly
secured operating system, ONLY the postgres server processes have access to
this shared memory segment"

It helped me to understand terminology used by other reponders better.

Thanks,

Oleg

On Wed, Dec 23, 2015 at 10:48 AM, John R Pierce  wrote:

> On 12/23/2015 8:16 AM, oleg yusim wrote:
>
>>
>> To my knowledge, many databases are using what called TDE to encrypt data
>> at rest and protect data from being accessed by attacker on host this way.
>> Here is the reference to quick guide on it:
>> https://www.simple-talk.com/sql/database-administration/transparent-data-encryption/
>>
>
> that article is talking about a specific feature of Microsoft SQL Server
> Enterprise Edition, which upon a quick skim sounds to me to be smoke and
> mirrors 'security-by-checklist' protection.   If the encryption keys are
> stored on the system, then anyone with access to the raw data can decrypt
> it, no matter how much smoke and mirrors you wave around to obfuscate this
> fact.
>
> In PostgreSQL 'shared memory' has a quite specific meaning, its referring
> to the pool of buffer memory (ram) shared by all postgres server
> processes.   this is primarily used as the buffer cache. In a properly
> secured operating system, ONLY the postgres server processes have access to
> this shared memory segment, but the details of OS level memory management
> are outide postgres's scope, since its portable and designed to be able to
> run on most any OS that provides basic memory management, multiple
> processes, and a reliable/robust file system, with tcp/ip socket support.
>
>
>
> --
> john r pierce, recycling bits in santa cruz
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

Re: [GENERAL] Shared system resources

[John R Pierce]

On 12/23/2015 8:16 AM, oleg yusim wrote:


To my knowledge, many databases are using what called TDE to encrypt 
data at rest and protect data from being accessed by attacker on host 
this way. Here is the reference to quick guide on it: 
https://www.simple-talk.com/sql/database-administration/transparent-data-encryption/


that article is talking about a specific feature of Microsoft SQL Server 
Enterprise Edition, which upon a quick skim sounds to me to be smoke and 
mirrors 'security-by-checklist' protection.   If the encryption keys are 
stored on the system, then anyone with access to the raw data can 
decrypt it, no matter how much smoke and mirrors you wave around to 
obfuscate this fact.


In PostgreSQL 'shared memory' has a quite specific meaning, its 
referring to the pool of buffer memory (ram) shared by all postgres 
server processes.   this is primarily used as the buffer cache. In a 
properly secured operating system, ONLY the postgres server processes 
have access to this shared memory segment, but the details of OS level 
memory management are outide postgres's scope, since its portable and 
designed to be able to run on most any OS that provides basic memory 
management, multiple processes, and a reliable/robust file system, with 
tcp/ip socket support.




--
john r pierce, recycling bits in santa cruz



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[oleg yusim]

Thank you very much George, that is exactly the piece of information I was
missing.

Oleg

On Wed, Dec 23, 2015 at 10:55 AM, George Neuner 
wrote:

> Hi Oleg,
>
> On Wed, 23 Dec 2015 07:07:31 -0600, oleg yusim 
> wrote:
>
> >May we run into situation, when attacker dumps memory and analyses it for
> >valuable content, instead of reserving it for own process, where it would
> >be zeroed? My understanding, it is a possibility. Does kernel have any
> >safeguard against it?
>
> With recent kernels, by default there is no way for a userspace
> process (even root) to dump memory.  Older kernels by default
> permitted a root process unrestricted access to /dev/mem and
> /dev/kmem, however in general that isn't needed and has long been
> disabled by the mahor distros.  [see CONFIG_STRICT_DEVMEM].  IIRC, the
> default setting was changed in 2011.
>
> With sufficient privileges, a debugger-like process can attach and
> examine the memory of a running - or just terminated - process, but it
> won't have access to discarded (unmapped) memory.
>
> The MAP_UNINITIALIZED trick, even if it works, is not a predictable
> attack vector.  There is no way to ask for any *particular* VMM page -
> mmap() just gives you a set of pages sufficient to cover the requested
> address range ... you don't know what process those pages previously
> belonged to.  Obviously there is a known algorithm for satisfying the
> page requests, but the set of free pages includes both code and data
> and depends on the history of system activity.  There's no guarantee
> to get anything useful.
>
> I'm not sure any of this really answers your question.
> George
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

Re: [GENERAL] Shared system resources

[Melvin Davidson]

Oleg,

As others have pointed out, worrying about someone accessing database
shared memory is like worrying about an asteroid striking the earth and
wiping out all life.
It's a one in a billion chance compared to other security violations that
can occur.
You are better off concentrating on proper O/S security and user/table
permissions. That is how to implement database security!

On Wed, Dec 23, 2015 at 12:10 PM, oleg yusim  wrote:

> Thank you very much George, that is exactly the piece of information I was
> missing.
>
> Oleg
>
> On Wed, Dec 23, 2015 at 10:55 AM, George Neuner 
> wrote:
>
>> Hi Oleg,
>>
>> On Wed, 23 Dec 2015 07:07:31 -0600, oleg yusim 
>> wrote:
>>
>> >May we run into situation, when attacker dumps memory and analyses it for
>> >valuable content, instead of reserving it for own process, where it would
>> >be zeroed? My understanding, it is a possibility. Does kernel have any
>> >safeguard against it?
>>
>> With recent kernels, by default there is no way for a userspace
>> process (even root) to dump memory.  Older kernels by default
>> permitted a root process unrestricted access to /dev/mem and
>> /dev/kmem, however in general that isn't needed and has long been
>> disabled by the mahor distros.  [see CONFIG_STRICT_DEVMEM].  IIRC, the
>> default setting was changed in 2011.
>>
>> With sufficient privileges, a debugger-like process can attach and
>> examine the memory of a running - or just terminated - process, but it
>> won't have access to discarded (unmapped) memory.
>>
>> The MAP_UNINITIALIZED trick, even if it works, is not a predictable
>> attack vector.  There is no way to ask for any *particular* VMM page -
>> mmap() just gives you a set of pages sufficient to cover the requested
>> address range ... you don't know what process those pages previously
>> belonged to.  Obviously there is a known algorithm for satisfying the
>> page requests, but the set of free pages includes both code and data
>> and depends on the history of system activity.  There's no guarantee
>> to get anything useful.
>>
>> I'm not sure any of this really answers your question.
>> George
>>
>>
>>
>> --
>> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-general
>>
>
>


-- 
*Melvin Davidson*
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

Re: [GENERAL] Shared system resources

[George Neuner]

Hi Oleg,

On Wed, 23 Dec 2015 07:07:31 -0600, oleg yusim 
wrote:

>May we run into situation, when attacker dumps memory and analyses it for
>valuable content, instead of reserving it for own process, where it would
>be zeroed? My understanding, it is a possibility. Does kernel have any
>safeguard against it?

With recent kernels, by default there is no way for a userspace
process (even root) to dump memory.  Older kernels by default
permitted a root process unrestricted access to /dev/mem and
/dev/kmem, however in general that isn't needed and has long been
disabled by the mahor distros.  [see CONFIG_STRICT_DEVMEM].  IIRC, the
default setting was changed in 2011. 

With sufficient privileges, a debugger-like process can attach and
examine the memory of a running - or just terminated - process, but it
won't have access to discarded (unmapped) memory.

The MAP_UNINITIALIZED trick, even if it works, is not a predictable
attack vector.  There is no way to ask for any *particular* VMM page -
mmap() just gives you a set of pages sufficient to cover the requested
address range ... you don't know what process those pages previously
belonged to.  Obviously there is a known algorithm for satisfying the
page requests, but the set of free pages includes both code and data
and depends on the history of system activity.  There's no guarantee
to get anything useful.

I'm not sure any of this really answers your question.
George



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[George Neuner]

On Tue, 22 Dec 2015 23:21:27 +, David Wilson 
wrote:

>On Linux the memory pages of an exiting process aren't sanitized at
>exit, however it is impossible(?) for userspace to reallocate them
>without the kernel first zeroing their contents.

Not impossible, but it requires a non-standard kernel.

Since 2.6.33, mmap() accepts the flag MAP_UNINITIALIZED which allows
pages to be mapped without being cleared.  The flag has no effect
unless the kernel was built with CONFIG_MMAP_ALLOW_UNINITIALIZED.


No mainstream distro enables this.  AFAIK, there is NO distro at all
that enables it ... it's too big a security risk for a general purpose
system.  It's intended to support embedded systems where the set of
programs is known.

George



-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

[GENERAL] Shared system resources

[oleg yusim]

Greetings,

I'm looking at the following security control right now:

*The DBMS must prevent unauthorized and unintended information transfer via
shared system resources.*

The purpose of this control is to prevent information, including encrypted
representations of information, produced by the actions of a prior
user/role (or the actions of a process acting on behalf of a prior
user/role) from being available to any current user/role (or current
process) that obtains access to a shared system resource (e.g., registers,
main memory, secondary storage) after the resource has been released back
to the information system. Control of information in shared resources is
also referred to as object reuse.

>From previous discussions, I understood that session in PostgreSQL is
closely associated with process, and it is essentially new process for
every user connection. In regards to that, my question would be:

When user disconnects, process is terminated and all resources are
released, does memory, session/process stored information at gets
"sanitized" or just released as is?

Thanks,

Oleg

Re: [GENERAL] Shared system resources

[Jim Nasby]

On 12/22/15 6:03 PM, oleg yusim wrote:

Absolutely. But we are not talking about that type of data leakage here.
We are talking about potential situation when user, who doesn't have
access to database, but has (or gained) access to the Linux box DB is
installed one and gets his hands on data, database processes stored in
memory (memory would be a common resource here).


Of far larger concern at that point is unauthorized access to the 
database files.


Basically, if someone gains access to the OS user that Postgres is 
running as, or to root, it's game-over.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[oleg yusim]

Jim,

Yes, you are right. Generally the security control here is encryption of
data at rest (TDE), but PostgreSQL doesn't support it, to my knowledge. I
know about that vulnerability, but here I posed the question on different
one. I agree it is smaller one, compare to the absence of TDE, but I would
like to find out if this gates are opened too or not.

Thanks,

Oleg

On Tue, Dec 22, 2015 at 8:48 PM, Jim Nasby  wrote:

> On 12/22/15 6:03 PM, oleg yusim wrote:
>
>> Absolutely. But we are not talking about that type of data leakage here.
>> We are talking about potential situation when user, who doesn't have
>> access to database, but has (or gained) access to the Linux box DB is
>> installed one and gets his hands on data, database processes stored in
>> memory (memory would be a common resource here).
>>
>
> Of far larger concern at that point is unauthorized access to the database
> files.
>
> Basically, if someone gains access to the OS user that Postgres is running
> as, or to root, it's game-over.
> --
> Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
> Experts in Analytics, Data Architecture and PostgreSQL
> Data in Trouble? Get it in Treble! http://BlueTreble.com
>

Re: [GENERAL] Shared system resources

[oleg yusim]

John,

Absolutely. But we are not talking about that type of data leakage here. We
are talking about potential situation when user, who doesn't have access to
database, but has (or gained) access to the Linux box DB is installed one
and gets his hands on data, database processes stored in memory (memory
would be a common resource here).

Thanks,

Oleg

On Tue, Dec 22, 2015 at 5:28 PM, John R Pierce  wrote:

> On 12/22/2015 2:52 PM, oleg yusim wrote:
>
>
> *The DBMS must prevent unauthorized and unintended information transfer
> via shared system resources.*
>
>
>
> you realize the database *itself* is a shared system resource and of
> and by itself has no idea about unauthorized/unintended information
> transfer, eg, any user with the proper credentials to connect to the
> database can query any tables those credentials are allowed to?
>
> --
> john r pierce, recycling bits in santa cruz
>
>

Re: [GENERAL] Shared system resources

[David Wilson]

On Tue, Dec 22, 2015 at 04:52:23PM -0600, oleg yusim wrote:
> Greetings,
> 
> I'm looking at the following security control right now:
> 
> The DBMS must prevent unauthorized and unintended information transfer via
> shared system resources.
> 
> The purpose of this control is to prevent information, including encrypted
> representations of information, produced by the actions of a prior user/role
> (or the actions of a process acting on behalf of a prior user/role) from being
> available to any current user/role (or current process) that obtains access to
> a shared system resource (e.g., registers, main memory, secondary storage)
> after the resource has been released back to the information system. Control 
> of
> information in shared resources is also referred to as object reuse.
> 
> From previous discussions, I understood that session in PostgreSQL is closely
> associated with process, and it is essentially new process for every user
> connection. In regards to that, my question would be:
> 
> When user disconnects, process is terminated and all resources are released,
> does memory, session/process stored information at gets "sanitized" or just
> released as is?

On Linux the memory pages of an exiting process aren't sanitized at
exit, however it is impossible(?) for userspace to reallocate them
without the kernel first zeroing their contents.

It might be possible for root to use some debugging mechanisms to access
the freed memory, but certainly there is no mechanism for a non-root
user to do so.

Regarding PG in particular, I can't speak for any shared internal state
that might be maintained after a session has exitted (e.g. in the SHM
regions PG maintains). Since that state lives longer than an individual
process, it's possible some information leakage could occur that way,
but "object reuse", it seems doubtful.


David


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Shared system resources

[John R Pierce]

On 12/22/2015 2:52 PM, oleg yusim wrote:


/The DBMS must prevent unauthorized and unintended information 
transfer via shared system resources./



you realize the database *itself* is a shared system resource and of 
and by itself has no idea about unauthorized/unintended information 
transfer, eg, any user with the proper credentials to connect to the 
database can query any tables those credentials are allowed to?


--
john r pierce, recycling bits in santa cruz