Some performance figures, atomic de-serialization using loopback local
network sockets, codebase downloads, dynamic class loading, with
security manager and policies in force and Jini extensible remote method
invocation, there's communication going on between 4 different jvm
instances, the one
A simpler example of production code modified to support @AtomicSerial
that satisfies invariants follows. Note how we can't type check
Generics with the static validator method, but the compiler does it for
us with the constructor :)
I'm wondering if there's scope for secure alternatives of
Hi Peter,
On 2 Feb 2015, at 11:16, Peter Firmstone peter.firmst...@zeus.net.au wrote:
As mentioned I've been experimenting with an invariant validating
ObjectInputStream, that's backward and forward compatible with Java's Object
Serialization stream protocol.
No changes have been made to
As mentioned I've been experimenting with an invariant validating
ObjectInputStream, that's backward and forward compatible with Java's
Object Serialization stream protocol.
No changes have been made to ObjectOutputStream.
ObjectInputStream has been overridden and reading from the stream has
Logging output from failed validation:
NonActGrp-out: Jan 29, 2015 7:36:16 PM
net.jini.jeri.BasicInvocationHandler invoke
NonActGrp-out: Jan 29, 2015 7:36:16 PM
net.jini.jeri.BasicInvocationHandler invoke
NonActGrp-out: FAILED: outbound call
net.jini.core.event.RemoteEventListener.notify
Although not directly relevant to this discussion, here are some
functional examples of deserialization constructors, I now have a fully
functional validating ObjectInputStream.
Unfortunately in our project we have intra object dependencies as
demonstrated by this example; a static validator,
Just a quick note, to avoid stream corruption, object that aren't
instantiated are replaced by an enum marker, Reference.DISCARDED, and
null is returned in their place, fields are read and any trailling
writeObject data is discarded. Object that are not allowed to be
deserialized are still
For the sharper eyed, you'll have noticed the readObjectNoData() method
I forgot to check:
private static boolean check(GetArg arg) throws IOException{
// If this class is not in the heirarchy of classes the stream
// may have been tampered with, see the Serialization Specification
Two examples:
1. of a child class with super class invariants and
2. Checking List and Map contents for type correctness.
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for
Another example of intra dependencies, turns out we have a lot of these
intra class dependencies in our project.
Cheers,
Peter.
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for
I decided to sample cpu load (see attached), with debugging enabled for
the validating ObjectInputStream and JERI, so heaps of output to the
console.
There are no performance optimisations with stream validation, I've just
focused on correctness and security.
Thank you HotSpot developers,
I have attempted to capture some of the ideas that we have discussed so far.
https://bugs.openjdk.java.net/browse/JDK-8071471
-Chris.
On 21/01/15 21:43, David M. Lloyd wrote:
At some point, the responsibility *must* fall on the author of the
serializable class in question to avoid constructs
And that is the point of this discussion, to determine how to do just that.
But considering it's such a difficult problem, we haven't figured out how to
yet.
Peter.
- Original message -
Message: 2
Date: Wed, 21 Jan 2015 15:43:13 -0600
From: David M. Lloyd david.ll...@redhat.com
On 20/01/15 20:22, Peter Levart wrote:
Hi Chris and Peter,
It just occurred to me that the following scheme would provide failure
atomicity for the whole Object regardless of whether readObject methods
are used or not for various classes in hierarchy:
- allocate uninitialized instance
- call
Don't forget that null may also be an invalid state.
What I have learnt so far:
1. An attacker can craft a stream to obtain a reference to any object
created during deserialization, using finalizers or circular links
(there may be yet other undiscovered methods).
2. An attacker
At some point, the responsibility *must* fall on the author of the
serializable class in question to avoid constructs which are exploitable
- like finalizers, and classes that can have side-effects before
validation can be completed.
On 01/21/2015 02:26 PM, Peter Firmstone wrote:
Don't
Hi Chris and Peter,
It just occurred to me that the following scheme would provide failure
atomicity for the whole Object regardless of whether readObject methods
are used or not for various classes in hierarchy:
- allocate uninitialized instance
- call default accessible constructor of the
- Original message -
On 15/01/15 20:33, Peter Firmstone wrote:
Thanks Chris,
WRT circular references, is it possible to detect and delay setting
these until after all verifiers run?
It is possible to detect the circular reference.
Currently you can retrieve a circular
On 15/01/15 20:33, Peter Firmstone wrote:
Thanks Chris,
WRT circular references, is it possible to detect and delay setting
these until after all verifiers run?
It is possible to detect the circular reference.
Currently you can retrieve a circular reference ( in readObject ) from
On 15 Jan 2015, at 12:25, Peter Firmstone peter.firmst...@zeus.net.au wrote:
Chris,
Can you explain some of the detail in your failure atomicity work?
Certainly.
The current implementation of defaultReadObject sets non-primitive field values
one at a time, without first checking that all
Thanks Chris,
WRT circular references, is it possible to detect and delay setting these until
after all verifiers run?
An option might be to provide a protected method in ObjectInputStream to opt
out or switch off support for circular references.
Apart from Serialization, another common use
Chris,
Can you explain some of the detail in your failure atomicity work?
Thanks,
Peter.
On 13/01/2015 1:56 AM, Chris Hegarty wrote:
On 10/01/15 07:00, Peter Firmstone wrote:
As shown in my earlier example, intra class invariants can be enforced
using Serialization constructors, from which static methods are called
to check invariants prior to super class constructors being
On 15 January 2015 at 13:01, Peter Firmstone
peter.firmst...@zeus.net.au wrote:
A third option that hasn't yet been considered would be to investigate an
api that various already existing frameworks can implement providers for, so
they no longer need to find creative ways to construct instances
It's only top down if there are intra class invariants, in this case invariants
are checked more than once.
This done simply because we can't move up the constructor call chain untill all
child class invariants have been satisfied.
For simpler class relationships, invariant check order is
On 01/14/2015 12:37 PM, Chris Hegarty wrote:
What do you think?
I agree completely.
An API at the level that you are proposing will provide the necessary
functionality and flexibility that is required to do validation in
readObject. As you clearly stated, and is already the case,
The following is a summary of our exploration so far:
We have narrowed down to two options:
A. constructors with static invariant check methods.
B. Continue to use readObject(), we have a proposed solution to the
final field problem and could modify the jvm to register for
finalization
On 13/01/15 10:00, Peter Levart wrote:
Hi Chris,
I stepped back and asked myself what problem(s) we are searching
solution(s) for and tried to see how existing API solves or doesn't
solve them. I think that existing readObject() + GetFields API already
solves most of the problems discussed so
On 13/01/15 00:51, Peter Firmstone wrote:
- Original message -
On 10/01/15 07:00, Peter Firmstone wrote:
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with
Hi Chris,
Sorry, no.
Currently when an ObjectStreamClass is read in from the stream, the framework
searches for the first zero arg constructor of a non serializable class and
creates and instance of the class read and resolved from the stream, however it
does so using a super class
On 14/01/15 12:58, Peter Firmstone wrote:
Hi Chris,
Sorry, no.
Currently when an ObjectStreamClass is read in from the stream, the
framework searches for the first zero arg constructor of a non
serializable class and creates and instance of the class read and
resolved from the stream, however
Hi Chris,
I stepped back and asked myself what problem(s) we are searching
solution(s) for and tried to see how existing API solves or doesn't
solve them. I think that existing readObject() + GetFields API already
solves most of the problems discussed so far as it:
- provides encapsulation
Could we use a static validator method and generate bytecode for constructors
dynamically?
The developer can optionally implement the constructors.
static GetField invariantCheck(GetField f);
Create a caller sensitive GetField implementation and add a two new methods to
GetField:
abstract
On 08/01/15 20:10, Brian Goetz wrote:
1) Validate invariants
A clear and easy to understand mechanism that can validate the
deserialized
fields. Does not prevent the use of final fields, as the
serialization framework
will be responsible for setting them. Something along the
On 08/01/15 22:03, David M. Lloyd wrote:
private static void validate(GetField fields) {
if (fields.getInt(lo) fields.getInt(hi)) { ... }
}
...
In fact you cannot validate invariants across multiple objects at all
using this method *or* readObject() (existing
Am 12.01.2015 um 17:15 schrieb Stephen Colebourne scolebou...@joda.org:
On 12 January 2015 at 11:37, Chris Hegarty chris.hega...@oracle.com wrote:
For clarity, I would like to describe how things currently work.
1) Allocate a new instance of the deserialized type.
2) Call the first
- Original message -
On 10/01/15 07:00, Peter Firmstone wrote:
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with readObject(), the examples I provide can do
On 10/01/15 07:00, Peter Firmstone wrote:
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with readObject(), the examples I provide can do this.
I have replied to Davids mail with a
On 01/12/2015 05:51 AM, Chris Hegarty wrote:
On 08/01/15 22:03, David M. Lloyd wrote:
private static void validate(GetField fields) {
if (fields.getInt(lo) fields.getInt(hi)) { ... }
}
...
In fact you cannot validate invariants across multiple objects at all
On 12 January 2015 at 11:37, Chris Hegarty chris.hega...@oracle.com wrote:
For clarity, I would like to describe how things currently work.
1) Allocate a new instance of the deserialized type.
2) Call the first non-Serializable types no-arg constructor
( may be j.l.Object ).
3) For
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with readObject(), the examples I provide can do this.
ObjectInputValidation is sufficient for enforcing business rules,
however it
On 01/08/2015 02:10 PM, Brian Goetz wrote:
1) Validate invariants
A clear and easy to understand mechanism that can validate the
deserialized
fields. Does not prevent the use of final fields, as the
serialization framework
will be responsible for setting them. Something along the
Thank you all for participating in this discussion.
Initially a constructor signature for deserialization was proposed to
enforce invariants and encapsulation, however there appears to be
interest in using alternative methods, although they appear to be
improvements over the status quo, I'm
On 01/08/2015 02:32 AM, Peter Firmstone wrote:
Thank you all for participating in this discussion.
Initially a constructor signature for deserialization was proposed to
enforce invariants and encapsulation, however there appears to be
interest in using alternative methods, although they appear
Peter, David
I would like to see how far we can push the existing Serialization mechanism,
before embarking on a road that may lead us to substantially different one.
Whether that be constructor based, to otherwise ( we know we will need a
replacement for Unsafe.allocateInstance, for
1) Validate invariants
A clear and easy to understand mechanism that can validate the deserialized
fields. Does not prevent the use of final fields, as the serialization
framework
will be responsible for setting them. Something along the lines of what
David
suggested:
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart peter.lev...@gmail.com wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
On 01/07/2015 03:54 PM, Chris Hegarty wrote:
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart peter.lev...@gmail.com wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in)
On 01/07/2015 10:02 AM, Peter Levart wrote:
On 01/07/2015 03:54 PM, Chris Hegarty wrote:
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart peter.lev...@gmail.com wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
On 6 Jan 2015, at 08:31, Stephen Colebourne scolebou...@joda.org wrote:
On 6 January 2015 at 07:27, Peter Levart peter.lev...@gmail.com wrote:
readObject() can be used on both ends of the spectrum or anywhere
in-between. It can be used for explicit deserialization or just for
validation of
On 6 January 2015 at 07:27, Peter Levart peter.lev...@gmail.com wrote:
readObject() can be used on both ends of the spectrum or anywhere
in-between. It can be used for explicit deserialization or just for
validation of default deserialized state. Would separate validation method
give us
On 6 Jan 2015, at 07:27, Peter Levart peter.lev...@gmail.com wrote:
Hi Brian,
On 01/05/2015 09:51 PM, Brian Goetz wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals, since
there are an infinitude of issues with serialization, only some of which we
can address.
On 6 January 2015 at 10:25, Chris Hegarty chris.hega...@oracle.com wrote:
On 6 Jan 2015, at 08:31, Stephen Colebourne scolebou...@joda.org wrote:
I've thought on a number of occasions that what I wanted from
serializable was a merger of readObject and readResolve
private Object
- Original message -
On 5/01/2015 6:51 PM, Peter Firmstone wrote:
Thanks Dave,
That's another way of checking invariants, you could expose A's
check method but you'd also need a couple of additional static
methods to obtain integers upper and lower from ReadSerial,
On 5 Jan 2015, at 20:51, Brian Goetz brian.go...@oracle.com wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals, since
there are an infinitude of issues with serialization, only some of which we
can address.
To that end, let me toss out some of the results of our
On 6 January 2015 at 11:46, Chris Hegarty chris.hega...@oracle.com wrote:
With shallow/no hierarchies, and others, the serialization proxy pattern
works quite well. The overhead of course being the creation of the proxy
itself, but this is typically a minimal container with just the state
On 6 Jan 2015, at 11:13, Stephen Colebourne scolebou...@joda.org wrote:
On 6 January 2015 at 10:25, Chris Hegarty chris.hega...@oracle.com wrote:
On 6 Jan 2015, at 08:31, Stephen Colebourne scolebou...@joda.org wrote:
I've thought on a number of occasions that what I wanted from
serializable
Hi Chris,
On 01/06/2015 12:10 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 07:27, Peter Levart peter.lev...@gmail.com wrote:
Hi Brian,
On 01/05/2015 09:51 PM, Brian Goetz wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals, since
there are an infinitude of issues with
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.GetField fields = in.readFields(); // this
already validates the types
Well, not true currently. But type validation could be added
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart peter.lev...@gmail.com wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.GetField fields =
On 6 Jan 2015, at 15:06, Peter Levart peter.lev...@gmail.com wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.GetField fields = in.readFields(); // this already
validates
Hi Brian,
On 01/05/2015 09:51 PM, Brian Goetz wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals,
since there are an infinitude of issues with serialization, only some
of which we can address.
To that end, let me toss out some of the results of our recent
My mistake, thank you.
Peter.
On 5/01/2015 9:57 PM, David Holmes wrote:
Hi Peter,
Did you send this to the list? I haven't seen it turn up yet.
David
On 5/01/2015 6:51 PM, Peter Firmstone wrote:
Thanks Dave,
That's another way of checking invariants, you could expose A's check
method but
Hi Peter and others,
A assume your approach described here (to use constructors for
deserialization) is motivated mainly by the fact that only constructors
are allowed to set the final fields. Otherwise the explicit features of
your ReadSerial API are more or less accessible even now by using
On 01/03/2015 09:36 AM, Peter Levart wrote:
On 01/03/2015 01:38 PM, Peter Firmstone wrote:
Hi,
I would like to know what are the potential issues with simple
constructor chaining where each constructor checks the invariant of its
class state (with the already initialized state of
On 01/03/2015 02:29 PM, Peter Firmstone wrote:
- Original message -
Wouldn't it be better to register for finalization just those instances that
complete their
construction or deserialization normally? I'm just trying to understand
why it is the way it is.
Perhaps that might be an
On 01/05/2015 03:17 PM, David M. Lloyd wrote:
Would something like this prevent Finalizer attacks?
- leave finalization registration the way it is (at object allocation
time).
This was written incorrectly: after Object default constructor completes
- provide internal API with which a
On 5/01/2015 6:51 PM, Peter Firmstone wrote:
Thanks Dave,
That's another way of checking invariants, you could expose A's check
method but you'd also need a couple of additional static methods to
obtain integers upper and lower from ReadSerial, so that B can ensure
its invariant.
Not sure I
Thanks Peter(s): I think it's just about time to re-sync on the goals,
since there are an infinitude of issues with serialization, only some of
which we can address.
To that end, let me toss out some of the results of our recent
explorations with improving serialization, where we started out
- Original message -
On 01/04/2015 02:48 AM, Peter Firmstone wrote:
class B extends A {
public final int cur;
private static ReadSerial check(ReadSerial rs) {
A a = new A(rs);
What to do if A is abstract?
:)
Create an anonymous instance, or create a static private
On 01/04/2015 02:48 AM, Peter Firmstone wrote:
class B extends A {
public final int cur;
private static ReadSerial check(ReadSerial rs) {
A a = new A(rs);
What to do if A is abstract?
Regards, Peter
int cur = rs.getInt(cur);
if ( a.lower cur || cur a.upper )
On 3/01/2015 9:24 PM, Peter Firmstone wrote:
Thanks Brian,
Those are good questions, some thoughts and examples inline:
- Original message -
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a view
of
Hi,
I would like to know what are the potential issues with simple
constructor chaining where each constructor checks the invariant of its
class state (with the already initialized state of superclass(es)).
Finalizer attack; a subclass can override the finalize method and
receive a
On 01/02/2015 11:53 PM, Brian Goetz wrote:
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a
view of its own fields.
- Invariant checking is separate from deserialization, and does not
seem entirely built-in --
- Original message -
As Brian points out, this scheme can only validate intra-class
invariants. It can't validate class-against-subclass state.
Did he say that?
It's true that a superclass can't validate subclass state, it can't be expected
to know much about it, but it can validate
Just a quick comment about the finalization aspect ( as I have been thinking
about this too ).
On 3 Jan 2015, at 15:36, Peter Levart peter.lev...@gmail.com wrote:
On 01/03/2015 01:38 PM, Peter Firmstone wrote:
Hi,
I would like to know what are the potential issues with simple
Thanks Brian,
Those are good questions, some thoughts and examples inline:
- Original message -
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a view
of its own fields.
- Invariant checking is
On 01/03/2015 01:38 PM, Peter Firmstone wrote:
Hi,
I would like to know what are the potential issues with simple
constructor chaining where each constructor checks the invariant of its
class state (with the already initialized state of superclass(es)).
Finalizer attack; a subclass can
P.S. Thanks for engaging this difficult subject.
It's worth remembering the finalizer attack isn't the only issue, a subclass
will have a reference after construction completes, it has a thread of
execution and if the superclass hasn't checked invarients, because circular
links haven't been
On 01/03/2015 09:29 PM, Peter Firmstone wrote:
- Original message -
As Brian points out, this scheme can only validate intra-class
invariants. It can't validate class-against-subclass state.
Sorry, I meant it can't validate class -against-superclass state.
Did he say that?
Not
On 4/01/2015 9:55 AM, Peter Levart wrote:
- Original message -
As Brian points out, this scheme can only validate intra-class
invariants. It can't validate class-against-subclass state.
Sorry, I meant it can't validate class -against-superclass state.
Did he say that?
Not
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a view
of its own fields.
- Invariant checking is separate from deserialization, and does not
seem entirely built-in -- subclass constructors seem responsible for
Subclass example:
class SubFoo extends BaseFoo {
public static ReadSerial check(ReadSerial rs){
if (rs.getInt(y) 128) throw Exception(too big);
return rs;
}
private final int y;
public SubFoo( int x , int y) {
super(x);
this.y = y;
}
Not quite, the constructor signature is the same for super and child classes.
ReadSerial is a container for each serialized Object in an ObjectInputStream
that provides and controlls access to serial fields, identified by name, type
and calling class, such that each class has it's own
Just to note that there is some overlap between improving
serialization and the meta-model for Java (Beans v2.0) work I'm
looking at [1][2]. The key tool that a meta-model provides is to
enable class authors to select which pieces of their state form the
published properties. In most cases, the
So, if I understand this correctly, the way this would get used is:
class BaseFoo implements Serializable {
private final int x;
public BaseFoo(ReadSerial rs) {
this(rs.getInt(x));
}
public BaseFoo(int x) {
this.x = x;
}
}
Right?
What happens with
86 matches
Mail list logo
