Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
For what it is worth. On 2/10/22 11:49 AM, Harald Anlauf via Fortran wrote: Hi Paul, Am 10.02.22 um 13:25 schrieb Paul Richard Thomas via Fortran: Conclusions on ifort: (i) The agreement between gfortran, with the patch applied, and ifort is strongest of all the other brands; (ii) The disagreements are all down to the treatment of the parent component of arrays of extended types: gfortran finalizes the parent component as an array, whereas ifort does a scalarization. I have a patch ready to do likewise. Overall conclusions: (i) Sort out whether or not derived type constructors are considered to be functions; (ii) Come to a conclusion about scalarization of parent components of extended type arrays; (iii) Check and, if necessary, correct the ordering of finalization in intrinsic assignment of class arrays. (iv) Finalization is difficult to graft on to existing pre-F2003 compilers, as witnessed by the range of implementations. I would be really grateful for thoughts on (i) and (ii). My gut feeling, as remarked in the submission, is that we should aim to be as close as possible, if not identical to, ifort. Happily, that is already the case. I am really sorry to be such a bother, but before we think we should do the same as Intel, we need to understand what Intel does and whether that is actually correct. Or not inconsistent with the standard. And I would really like to understand even the most simple, stupid case. I did reduce testcase finalize_38.f90 to an almost bare minimum, see attached, and changed the main to type(simple), parameter :: ThyType = simple(21) type(simple) :: ThyType2 = simple(22) type(simple), allocatable :: MyType, MyType2 print *, "At start of program: ", final_count MyType = ThyType print *, "After 1st allocation:", final_count MyType2 = ThyType2 print *, "After 2nd allocation:", final_count Note that "ThyType" is now a parameter. - snip Ignore whether Thytype is a Parameter. Regardless Mytype and Mytype2 are allocated upon the assignment. Now if these are never used anywhere, it seems to me the deallocation can be done by the compiler anywhere after the last time it is used. So it can be either after the PRINT statement before the end if the program or right after the assignment before your PRINT statements that examine the value of final_count. I think the result is arbitrary/undefined in your reduced test case I do not have the Intel compiler yet, so I was going to suggest see what it does if your test program prints something from within MyType and MyType2 after all your current print statements at the end. Try this variation of the main program. program test_final use testmode implicit none type(simple), parameter :: ThyType = simple(21) type(simple) :: ThyType2 = simple(22) type(simple), allocatable :: MyType, MyType2 print *, "At start of program: ", final_count MyType = ThyType print *, "After 1st allocation:", final_count MyType2 = ThyType2 print *, "After 2nd allocation:", final_count print *, MyType%ind, MyType2%ind, final_count deallocate(Mytype) print *, MyType%ind, MyType2%ind, final_count deallocate(Mytype2) print *, MyType%ind, MyType2%ind, final_count end program test_final I get with trunk: $ ./a.out At start of program: 0 After 1st allocation: 0 After 2nd allocation: 0 21 22 0 0 22 1 0 0 2 Which makes sense to me. Regards, Jerry
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Hi Paul, Am 10.02.22 um 13:25 schrieb Paul Richard Thomas via Fortran: Conclusions on ifort: (i) The agreement between gfortran, with the patch applied, and ifort is strongest of all the other brands; (ii) The disagreements are all down to the treatment of the parent component of arrays of extended types: gfortran finalizes the parent component as an array, whereas ifort does a scalarization. I have a patch ready to do likewise. Overall conclusions: (i) Sort out whether or not derived type constructors are considered to be functions; (ii) Come to a conclusion about scalarization of parent components of extended type arrays; (iii) Check and, if necessary, correct the ordering of finalization in intrinsic assignment of class arrays. (iv) Finalization is difficult to graft on to existing pre-F2003 compilers, as witnessed by the range of implementations. I would be really grateful for thoughts on (i) and (ii). My gut feeling, as remarked in the submission, is that we should aim to be as close as possible, if not identical to, ifort. Happily, that is already the case. I am really sorry to be such a bother, but before we think we should do the same as Intel, we need to understand what Intel does and whether that is actually correct. Or not inconsistent with the standard. And I would really like to understand even the most simple, stupid case. I did reduce testcase finalize_38.f90 to an almost bare minimum, see attached, and changed the main to type(simple), parameter :: ThyType = simple(21) type(simple) :: ThyType2 = simple(22) type(simple), allocatable :: MyType, MyType2 print *, "At start of program: ", final_count MyType = ThyType print *, "After 1st allocation:", final_count MyType2 = ThyType2 print *, "After 2nd allocation:", final_count Note that "ThyType" is now a parameter. I tested the above and found: Intel: At start of program:0 After 1st allocation: 1 After 2nd allocation: 2 NAG 7.0: At start of program: 0 After 1st allocation: 0 After 2nd allocation: 0 Crayftn 12.0.2: At start of program: 2 After 1st allocation: 2 After 2nd allocation: 2 Nvidia 22.1: At start of program: 0 After 1st allocation:0 After 2nd allocation:0 So my stupid questions are: - is ThyType invoking a constructor? It is a parameter, after all. Should using it in an assignment invoke a destructor? If so why? And why does Intel then increment the final_count? - is the initialization of ThyType2 invoking a constructor? It might, if that is the implementation in the compiler, but should there be a finalization? Then ThyType2 is used in an intrinsic assignment, basically the same as the other one before. Now what is the difference? Are all compilers correct, but I do not see it? Someone please help! Best regards Paul Cheers, Harald module testmode implicit none type :: simple integer :: ind contains final :: destructor1 end type simple integer :: final_count = 0 contains subroutine destructor1(self) type(simple), intent(inout) :: self final_count = final_count + 1 end subroutine destructor1 end module testmode program test_final use testmode implicit none type(simple), parameter :: ThyType = simple(21) type(simple) :: ThyType2 = simple(22) type(simple), allocatable :: MyType, MyType2 print *, "At start of program: ", final_count MyType = ThyType print *, "After 1st allocation:", final_count MyType2 = ThyType2 print *, "After 2nd allocation:", final_count end program test_final
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Hi Harald, I have run your modified version of finalize_38.f90, and now I see > that you can get a bloody head just from scratching too much... > > crayftn 12.0.2: > > 1, 3, 1 > It appears that Cray interpret a derived type constructor as being a function call and so "6 If a specification expression in a scoping unit references a function, the result is finalized before execution of the executable constructs in the scoping unit." A call to 'test' as the first statement might be useful to diagnose: call test(2, 0, [0,0], -10) > 2, 21, 0 > 21 is presumably the value left over from simple(21) but quite why it should happen in this order is not apparent to me. > 11, 3, 2 > I am mystified as to why the finalization of 'var' is not occurring because "1 When an intrinsic assignment statement is executed (10.2.1.3), if the variable is not an unallocated allocatable variable, it is finalized after evaluation of expr and before the definition of the variable." Note the double negative! 'var' has been allocated and should return 1 to 'scalar' > 12, 21, 1 > 21, 4, 3 > This is a residue of earlier differences in the final count. > 23, 21, 22 | 42, 43 > The value is inexplicable to me. 31, 6, 4 > 41, 7, 5 > 51, 9, 7 > 61, 10, 8 > 71, 13, 10 > 101, 2, 1 > One again, a function 'expr' finalization has been added after intrinsic assignment; ie. derived type constructor == function. > 102, 4, 3 > > 111, 3, 2 > 121, 4, 2 > 122, 0, 4 > 123, 5, 6 | 2*0 > >From the value of 'array', I would devine that the source in the allocation is being finalized as an array, whereas I would expect each invocation of 'simple' to generate a scalar final call. > 131, 5, 2 > 132, 0, 4 > 133, 7, 8 | 2*0 > The final count has increased by 1, as expected. The value of 'scalar' is stuck at 0, so the second line is explicable. The array value is explicable if the finalization is of 'expr' and that 'var' is not finalized or the finalization of 'var' is occuring after assignment; ie. wrong order. ***I notice from the code that even with the patch, gfortran is finalizing before evaluation of 'expr', which is incorrect. It should be after evaluation of 'expr' and before the assignment.*** 141, 6, 3 > Final count offset - OK 151, 10, 5 > The two extra calls come, I presume from the source in the allocation. Since the type is extended, we see two finalizations each for the allocation and the deallocation. 161, 16, 9 > I think that the extra two finalizations come from the evaluation of 'src' in 'constructor2'. 171, 18, 11 > Final count offset - OK 175, 0., 20. | 10., 20. > The value of 'rarray' is mystifying. Conclusions from Cray: (i) Determine if derived type constructors should be interpreted as function calls. (ii) The order of finalization in class array assignment needs to be checked and fixed if necessary. > > nagfor 7.0: > > 1 0 1 > "1 When an intrinsic assignment statement is executed (10.2.1.3), if the variable is not an unallocated allocatable variable, it is finalized after evaluation of expr and before the definition of the variable." So I think that NAG has this wrong, either because the timing is right and an unallocatable allocatable is being finalized or because the timing is wrong. 11 1 2 > 23 21 22 | 42 43 > It seems that the finalization is occurring after assignment. 71 9 10 > 72 11 99 > It seems that the finalization of the function 'expr' after assignment is not happening. 131 3 2 > 132 5 4 > I am not sure that I know where the extra final call is nor where the scalar value of 5 comes from. 141 4 3 > 151 6 5 > 161 10 9 > 171 12 11 > The above are OK since there is an offset in the final count, starting at 131. Conclusions from NAG: (i) Some minor nits but pretty close to my interpretation. Intel 2021.5.0: > > 131 3 2 > 132 0 4 > 133 5 6 | 0 0 > 141 4 3 > 151 7 5 > 152 3 0 > 153 0 0 | 1 3 > forrtl: severe (174): SIGSEGV, segmentation fault occurred > [...] > ifort (IFORT) 2021.1 Beta 20201112 manages to carry on to the end. 161 13 9 162 20 0 163 0 0 | 10 20 171 14 11 Conclusions on ifort: (i) The agreement between gfortran, with the patch applied, and ifort is strongest of all the other brands; (ii) The disagreements are all down to the treatment of the parent component of arrays of extended types: gfortran finalizes the parent component as an array, whereas ifort does a scalarization. I have a patch ready to do likewise. Overall conclusions: (i) Sort out whether or not derived type constructors are
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Remember the days when reading very old cryptic Fortran code? Remember the fixed line lengths and cryptic variable names! I fear the Standards committee has achieved history with the Standard itself it is so difficult to understand sometimes. Cheers to Paul and Harald for digging on this. Jerry On 2/8/22 11:29 AM, Harald Anlauf via Fortran wrote: Hi Paul, Am 08.02.22 um 12:22 schrieb Paul Richard Thomas via Fortran: Hi Harald, Thanks for giving the patch a whirl. the parent components as an array. I strongly suspect that, from reading 7.5.6.2 paragraphs 2 and 3 closely, that ifort has it right. However, this is another issue to come back to in the future. Could you specify which version of Intel you tried? ifort (IFORT) 2021.1 Beta 20201112 ok, that's good to know. Testcase finalize_38.f90 fails for me with ifort 2021.5.0 with: 131 This test also fails with crayftn 11 & 12 and nagfor 7.0, but in a different place. I have run your modified version of finalize_38.f90, and now I see that you can get a bloody head just from scratching too much... crayftn 12.0.2: 1, 3, 1 2, 21, 0 11, 3, 2 12, 21, 1 21, 4, 3 23, 21, 22 | 42, 43 31, 6, 4 41, 7, 5 51, 9, 7 61, 10, 8 71, 13, 10 101, 2, 1 102, 4, 3 111, 3, 2 121, 4, 2 122, 0, 4 123, 5, 6 | 2*0 131, 5, 2 132, 0, 4 133, 7, 8 | 2*0 141, 6, 3 151, 10, 5 161, 16, 9 171, 18, 11 175, 0., 20. | 10., 20. nagfor 7.0: 1 0 1 11 1 2 23 21 22 | 42 43 71 9 10 72 11 99 131 3 2 132 5 4 141 4 3 151 6 5 161 10 9 171 12 11 Intel 2021.5.0: 131 3 2 132 0 4 133 5 6 | 0 0 141 4 3 151 7 5 152 3 0 153 0 0 | 1 3 forrtl: severe (174): SIGSEGV, segmentation fault occurred [...] That got me reading 7.5.6.3, where is says in paragraph 1: "When an intrinsic assignment statement is executed (10.2.1.3), if the variable is not an unallocated allocatable variable, it is finalized after evaluation of expr and before the definition of the variable. ..." Looking at the beginning of the testcase code (abridged): type(simple), allocatable :: MyType, MyType2 type(simple) :: ThyType = simple(21), ThyType2 = simple(22) ! The original PR - one finalization of 'var' before (re)allocation. MyType = ThyType call test(1, 0, [0,0], 0) This is an intrinsic assignment. Naively I would expect MyType to be initially unallocated. ThyType is not allocatable and non-pointer and cannot become undefined here and would not play any role in finalization. I am probably too blind-sighted to see why there should be a finalization here. What am I missing? Could you use the attached version of finalize_38.f90 with crayftn and NAG? All the stop statements are replaced with prints. Ifort gives: 131 3 2 132 0 4 133 5 6 | 0 0 141 4 3 151 7 5 152 3 0 153 0 0 | 1 3 161 13 9 162 20 0 163 0 0 | 10 20 171 14 11 I think it is a good idea to have these prints in the testcase whenever there is a departure from expectations. So print? Furthermore, for the sake of health of people reading the testcases later, I think it would not harm to add more explanations why we expect a certain behavior... ;-) Best regards Paul Best regards, Harald
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Hi Paul, Am 08.02.22 um 12:22 schrieb Paul Richard Thomas via Fortran: Hi Harald, Thanks for giving the patch a whirl. the parent components as an array. I strongly suspect that, from reading 7.5.6.2 paragraphs 2 and 3 closely, that ifort has it right. However, this is another issue to come back to in the future. Could you specify which version of Intel you tried? ifort (IFORT) 2021.1 Beta 20201112 ok, that's good to know. Testcase finalize_38.f90 fails for me with ifort 2021.5.0 with: 131 This test also fails with crayftn 11 & 12 and nagfor 7.0, but in a different place. I have run your modified version of finalize_38.f90, and now I see that you can get a bloody head just from scratching too much... crayftn 12.0.2: 1, 3, 1 2, 21, 0 11, 3, 2 12, 21, 1 21, 4, 3 23, 21, 22 | 42, 43 31, 6, 4 41, 7, 5 51, 9, 7 61, 10, 8 71, 13, 10 101, 2, 1 102, 4, 3 111, 3, 2 121, 4, 2 122, 0, 4 123, 5, 6 | 2*0 131, 5, 2 132, 0, 4 133, 7, 8 | 2*0 141, 6, 3 151, 10, 5 161, 16, 9 171, 18, 11 175, 0., 20. | 10., 20. nagfor 7.0: 1 0 1 11 1 2 23 21 22 | 42 43 71 9 10 72 11 99 131 3 2 132 5 4 141 4 3 151 6 5 161 10 9 171 12 11 Intel 2021.5.0: 131 3 2 132 0 4 133 5 6 | 0 0 141 4 3 151 7 5 152 3 0 153 0 0 | 1 3 forrtl: severe (174): SIGSEGV, segmentation fault occurred [...] That got me reading 7.5.6.3, where is says in paragraph 1: "When an intrinsic assignment statement is executed (10.2.1.3), if the variable is not an unallocated allocatable variable, it is finalized after evaluation of expr and before the definition of the variable. ..." Looking at the beginning of the testcase code (abridged): type(simple), allocatable :: MyType, MyType2 type(simple) :: ThyType = simple(21), ThyType2 = simple(22) ! The original PR - one finalization of 'var' before (re)allocation. MyType = ThyType call test(1, 0, [0,0], 0) This is an intrinsic assignment. Naively I would expect MyType to be initially unallocated. ThyType is not allocatable and non-pointer and cannot become undefined here and would not play any role in finalization. I am probably too blind-sighted to see why there should be a finalization here. What am I missing? Could you use the attached version of finalize_38.f90 with crayftn and NAG? All the stop statements are replaced with prints. Ifort gives: 131 3 2 132 0 4 133 5 6 | 0 0 141 4 3 151 7 5 152 3 0 153 0 0 | 1 3 161 13 9 162 20 0 163 0 0 | 10 20 171 14 11 I think it is a good idea to have these prints in the testcase whenever there is a departure from expectations. So print? Furthermore, for the sake of health of people reading the testcases later, I think it would not harm to add more explanations why we expect a certain behavior... ;-) Best regards Paul Best regards, Harald
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Hi Harald,
Thanks for giving the patch a whirl.
> the parent components as an array. I strongly suspect that, from reading
> > 7.5.6.2 paragraphs 2 and 3 closely, that ifort has it right. However,
> this
> > is another issue to come back to in the future.
>
> Could you specify which version of Intel you tried?
>
ifort (IFORT) 2021.1 Beta 20201112
>
> Testcase finalize_38.f90 fails for me with ifort 2021.5.0 with:
>
> 131
>
That's the point where the interpretation of the standard diverges. Ifort
uses the scalar finalization for the parent component, whereas gfortran
uses the rank 1. Thus the final count is different by one. I have a version
of the patch, where gfortran behaves in the same way as ifort.
> This test also fails with crayftn 11 & 12 and nagfor 7.0,
> but in a different place.
>
>
> (Also finalize_45.f90 fails with that version with something that
> looks like memory corruption, but that might be just a compiler bug.)
>
I take it 'that version' is of ifort? Mine does the same. I suspect that it
is one of the perils of using pointer components in such circumstances! You
will notice that I had to nullify pointer components when doing the copy.
>
> Thanks,
> Harald
>
Could you use the attached version of finalize_38.f90 with crayftn and NAG?
All the stop statements are replaced with prints. Ifort gives:
131 3 2
132 0 4
133 5 6 | 0 0
141 4 3
151 7 5
152 3 0
153 0 0 | 1 3
161 13 9
162 20 0
163 0 0 | 10 20
171 14 11
Best regards
Paul
! { dg-do run }
!
! Test finalization on intrinsic assignment (F2018 (7.5.6.3))
!
module testmode
implicit none
type :: simple
integer :: ind
contains
final :: destructor1, destructor2
end type simple
type, extends(simple) :: complicated
real :: rind
contains
final :: destructor3, destructor4
end type complicated
integer :: check_scalar
integer :: check_array(4)
real :: check_real
real :: check_rarray(4)
integer :: final_count = 0
contains
subroutine destructor1(self)
type(simple), intent(inout) :: self
check_scalar = self%ind
check_array = 0
final_count = final_count + 1
end subroutine destructor1
subroutine destructor2(self)
type(simple), intent(inout) :: self(:)
check_scalar = 0
check_array(1:size(self, 1)) = self%ind
final_count = final_count + 1
end subroutine destructor2
subroutine destructor3(self)
type(complicated), intent(inout) :: self
check_real = self%rind
check_array = 0.0
final_count = final_count + 1
end subroutine destructor3
subroutine destructor4(self)
type(complicated), intent(inout) :: self(:)
check_real = 0.0
check_rarray(1:size(self, 1)) = self%rind
final_count = final_count + 1
end subroutine destructor4
function constructor1(ind) result(res)
type(simple), allocatable :: res
integer, intent(in) :: ind
allocate (res, source = simple (ind))
end function constructor1
function constructor2(ind, rind) result(res)
class(simple), allocatable :: res(:)
integer, intent(in) :: ind(:)
real, intent(in), optional :: rind(:)
type(complicated), allocatable :: src(:)
integer :: sz
integer :: i
if (present (rind)) then
sz = min (size (ind, 1), size (rind, 1))
src = [(complicated (ind(i), rind(i)), i = 1, sz)]
allocate (res, source = src)
else
sz = size (ind, 1)
allocate (res, source = [(simple (ind(i)), i = 1, sz)])
end if
end function constructor2
subroutine test (cnt, scalar, array, off, rind, rarray)
integer :: cnt
integer :: scalar
integer :: array(:)
integer :: off
real, optional :: rind
real, optional :: rarray(:)
if (final_count .ne. cnt) print *, 1 + off, final_count, cnt
if (check_scalar .ne. scalar) print *, 2 + off, check_scalar, scalar
if (any (check_array(1:size (array, 1)) .ne. array)) print *, 3 + off, &
check_array(1:size (array, 1)), "|", array
if (present (rind)) then
if (check_real .ne. rind) print *, 4+off, check_real, rind
end if
if (present (rarray)) then
if (any (check_rarray(1:size (rarray, 1)) .ne. rarray)) print *, 5 + off, &
check_rarray(1:size (rarray, 1)), "|", rarray
end if
end subroutine test
end module testmode
program test_final
use testmode
implicit none
type(simple), allocatable :: MyType, MyType2
type(simple), allocatable :: MyTypeArray(:)
type(simple) :: ThyType = simple(21), ThyType2 = simple(22)
class(simple), allocatable :: MyClass
class(simple), allocatable ::
Re: [Patch, fortran] PR37336 (Finalization) - [F03] Finish derived-type finalization
Hi Paul, thanks for attacking this. I haven't looked at the actual patch, only tried to check the new testcases with other compilers. Am 03.02.22 um 18:14 schrieb Paul Richard Thomas via Fortran: I have tried to interpret F2018 7.5.6.2 and 7.5.6.3 as well as possible. This is not always straightforward and has involved a lot of head scratching! I have used the Intel compiler as a litmus test for the outcomes. This was largely motivated by the observation that, in the user survey conducted by Steve Lionel, gfortran and ifort are often used together . Therefore, quite aside from wishing to comply with the standard as far as possible, it is more than reasonable that the two compilers comply. On application of this patch, only exception to this is the treatment of finalization of arrays of extended types, where the Intel takes "If the entity is of extended type and the parent type is finalizable, the parent component is finalized" such that the parent component is finalized one element at a time, whereas gfortran finalises the parent components as an array. I strongly suspect that, from reading 7.5.6.2 paragraphs 2 and 3 closely, that ifort has it right. However, this is another issue to come back to in the future. Could you specify which version of Intel you tried? Testcase finalize_38.f90 fails for me with ifort 2021.5.0 with: 131 This test also fails with crayftn 11 & 12 and nagfor 7.0, but in a different place. (Also finalize_45.f90 fails with that version with something that looks like memory corruption, but that might be just a compiler bug.) Thanks, Harald
