@yuyichao made the following comments in #16999:
I think Scott has a point
No he doesn't and never did.
who needs to understand the precise rules for deepcopy.
I'll never make the mistake of wasting my time arguing with SPJ (edit:
> sorry, mistyped as SGJ just now....) again but I certainly agree that the
> doc should be improved and I'm happy to explain the doc to you here.
I'd respectfully ask @yuyichao to tone it down a bit.
I have only been trying to get a fix in for a serious problem that I helped
debug for @chrisraukaukus.
He's told me that he wants to fit as many BigFloats as can fit in the 128GB
he has available, so @yuyichao's solution is much less useful for his use
case.
He used deepcopy because he has a structure with Vector of Vector of
Numbers.
Performance of course is also important for his use case (although BigFloat
is pretty horrible for performance in general, I think it would be better
to move BigInt and BigFloat into packages and come up with something better
for Julians to use i.e. using the immutable type to prevent all of this
hassle)
One of my points is this: The Number abstract type has always been treated
as something immutable and scalar in Julia, no matter how they are
implemented (Rational and Complex numbers are not treated as collections,
for example).
@yuyichao goes on to say:
Multiple logical fallacy here,
>
> 1. This (what can be more identical to the original than the original
> itself) is basically claiming that we should define deepcopy(x) = x,
> which is clearly wrong.
>
>
> 1. as identical as possible is the basic requirement for any kind of
> copy, and has nothing to do with the schematics of deepcopy or copy (in
> that it is not at all the factor to decide how each of them should be
> implemented). This should be clear since we don't implement either of them
> as identity for all objects.
>
>
> 1. The important part of same effect as serializing and deserializing is
> that they should be totally independent. "Serializing and deserializing"
> is
> roughly equal to save an object to disc, restart the julia session, and
> load it from disk again. The new object shouldn't share any information
> with the old one that depends on the environment the old object is defined
> that is not present in the new session. "Same effect" should at least mean
> that you shouldn't be able to tell the difference by doing any well
> defined
> operation on the two objects to notice that it is not actually serialized
> and deserialized. Bothis and finalize are documented and well defined
> operations on these objects so you shouldn't be able to use them to
> observe
> the difference between the two approaches (serialize + deserialize v.s.
> deepcopy)
>
> (for some reason the quoting messed up the numbering).
I would say that there are multiple logical falacies in the above.
In the first part, saying that "`deepcopy(x) = x` is clearly wrong", is
simply wrong.
Even @yuyichao's code will frequently NOT make a totally independent copy
of a BigInt or BigFloat.
It's very easy to prove that his "fix" is not complete or consistent with
the behavior of 'copy', and doesn't solve at all the problem of manually
calling `finalize` on a `BigFloat` or `BigInt`.
For example:
> *julia> *
> *a = big"1.5"*
> *1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *julia> *
> *v = [a, copy(a)]*
> *2-element Array{BigFloat,1}:*
>
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *julia> *
> *v[1] === v[2]**true*
> *julia> *
> *x = copy(v)*
> *2-element Array{BigFloat,1}:*
>
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *julia> *
> *v[1] === v[2]**true*
> *julia> *
> *x[1] === x[2]**true*
> *julia> *
> *y = deepcopy(v)*
> *2-element Array{BigFloat,1}:*
>
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *
> 1.500000000000000000000000000000000000000000000000000000000000000000000000000000*
> *julia> *
> *y[1] === y[2]**true*
As you can see, `copy` (defined for Number types in number.jl) assumes that
ALL subtypes of Number are treated as immutable, and `copy(x::Number) = x`.
This means that @yuyichao's change to `deepcopy` does absolutely nothing to
really prevent the problem of being able to manually call `finalize` on a
`BigInt` or `BigFloat`,
so his fix means accepting a >10x drop in performance, 3x memory usage,
without even really fixing the "problem" that he sees.
In the third part, he states that "same effect" requires that `finalize` be
callable manually on the deepcopy of a BigFloat, just because it is a
documented public function.
That is a fallacy - I could also call `setfield!(x,:prec,128)` on a
BigFloat, and get a core dump. Just because something is *doable* with a
documented public function on an object, doesn't guarantee it is safe!
(Julia's old "CA" philosophy has gotten in the way of any mechanism that
would allow one to make those sorts of guarantees)
As a fun example (this time without any finalizers involved):
*julia> *
> *s = "foo"[3:2]**""*
> *julia> *
> *p = "bar"[3:2]**""*
> *julia> *
> *s === p**true*
> *julia> *
> *push!(s.data,123)*
> *1-element Array{UInt8,1}:** {0x7b{*
> *julia> *
> *p**"{"*
This shows clearly why Julia is in general NOT safe in many areas, if you
start messing around with their implementation details (and whether or not
a type has a finalizer is really an implementation detail)
It would be nice if any people actually using lots of BigInts or BigFloats
(like @chrisraukaukas is) would chime in.
Do they really want a "fix" that isn't even complete, for either BigFloats
and BigInts, that takes >10x more time and ~3x more memory, that doesn't
really prevent any potential problems that might occur because two
BigFloat/BigInt values are === and finalizer is called manually on one of
them?
