Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Paul Beach]

On 17/02/2017 09:39, Stephan Bergmann wrote:
> On 02/17/2017 12:24 PM, Alex Peshkoff wrote:
>> On 02/17/17 14:15, Stephan Bergmann wrote:
>>
>> This idea seems to be much better than DEBUG_GDS_ALLOC:
>>
>>> Since LibreOffice builds intended for widespread distribution are on
>>> Linux usually done with GCC not Clang, this shouldn't have a performance
>>> impact.)
>> But what makes me surprised - I know that at least one of FB developers
>> often does build firebird using clang (Adriano - is it so?). And we
>> never used to meet such issues before. What changed? New version? Too
>> good optimization flags?
> I think this aggressive alignment-assumption behavior is new with 
> upcoming Clang 4.  (And you need at least -O optimization for such 
> problematic MOVAPS code to be produced.)

Clang is also the default compiler on MacOSX nowadays 

Paul 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Adriano dos Santos Fernandes]

On 17/02/2017 09:39, Stephan Bergmann wrote:
> On 02/17/2017 12:24 PM, Alex Peshkoff wrote:
>> On 02/17/17 14:15, Stephan Bergmann wrote:
>>
>> This idea seems to be much better than DEBUG_GDS_ALLOC:
>>
>>> Since LibreOffice builds intended for widespread distribution are on
>>> Linux usually done with GCC not Clang, this shouldn't have a performance
>>> impact.)
>> But what makes me surprised - I know that at least one of FB developers
>> often does build firebird using clang (Adriano - is it so?). And we
>> never used to meet such issues before. What changed? New version? Too
>> good optimization flags?
> I think this aggressive alignment-assumption behavior is new with 
> upcoming Clang 4.  (And you need at least -O optimization for such 
> problematic MOVAPS code to be produced.)
>
>
I'm using clang 3.8 (from Ubuntu 16.10) and always building FB in debug
mode.


Adriano


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Stephan Bergmann]

On 02/17/2017 12:24 PM, Alex Peshkoff wrote:
> On 02/17/17 14:15, Stephan Bergmann wrote:
>
> This idea seems to be much better than DEBUG_GDS_ALLOC:
>
>> Since LibreOffice builds intended for widespread distribution are on
>> Linux usually done with GCC not Clang, this shouldn't have a performance
>> impact.)
>
> But what makes me surprised - I know that at least one of FB developers
> often does build firebird using clang (Adriano - is it so?). And we
> never used to meet such issues before. What changed? New version? Too
> good optimization flags?

I think this aggressive alignment-assumption behavior is new with 
upcoming Clang 4.  (And you need at least -O optimization for such 
problematic MOVAPS code to be produced.)


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Stephan Bergmann]

On 02/17/2017 12:21 PM, Alex Peshkoff wrote:
> On 02/17/17 14:15, Stephan Bergmann wrote:
>> On 02/17/2017 11:48 AM, Alex Peshkoff wrote:
>>> On 02/16/17 15:52, Stephan Bergmann wrote:
 Forgive me if this has already been discussed or even fixed in later
 versions:  At least the Firebird 3.0 we build as part of LibreOffice
 defines global operator new replacement functions in
 src/common/classes/alloc.h (forwarding to MemoryPool) that do not in
 general fulfil the alignment requirements for such functions.

 Came across this when Firebird compiled with a recent trunk Clang (with
 -O, and DEBUG_GDS_ALLOC being undefined) on x86_64-unknown-linux-gnu
 causes SEGV from misaligned MOVAPS instructions.

>>> Yes - allocated memory is aligned at 8 bytes boundary now.
>>> I've tried to set alignment to 16 but looks like that's far not 5 lines
>>> patch - sometimes we were choosing between 4/8 bytes alignment, but last
>>> years only 8 bytes alignment was used.
>>> May be finding specific compiler flag to avoid this instruction is
>>> simpler choice for today?
>> (The way I work around this for now in LibreOffice is by always defining
>> DEBUG_GDS_ALLOC when building with Clang on Linux X86-64,
>> 
>> "external/firebird: Work around operator new alignment violations".
>> Since LibreOffice builds intended for widespread distribution are on
>> Linux usually done with GCC not Clang, this shouldn't have a performance
>> impact.)
>
> Afraid that this work around is not too stable (not to say more).
> Certainly that macro does change details of memory allocation and may
> provide 16-bit alignment in some particular case but be sure it's not
> more than side effect and does not gurantee such alignment.

Note that I'm not relying on any 16-byte alignment there, but on the 
fact that Clang happens to make those aggressive alignment assumptions 
only for the replaceable global allocation functions.  And with 
DEBUG_GDS_ALLOC defined, the global allocation functions defined in 
alloc.h are no longer replacements of such replaceable allocation functions.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Alex Peshkoff]

On 02/17/17 14:15, Stephan Bergmann wrote:

This idea seems to be much better than DEBUG_GDS_ALLOC:

> Since LibreOffice builds intended for widespread distribution are on
> Linux usually done with GCC not Clang, this shouldn't have a performance
> impact.)

But what makes me surprised - I know that at least one of FB developers 
often does build firebird using clang (Adriano - is it so?). And we 
never used to meet such issues before. What changed? New version? Too 
good optimization flags?



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Alex Peshkoff]

On 02/17/17 14:15, Stephan Bergmann wrote:
> On 02/17/2017 11:48 AM, Alex Peshkoff wrote:
>> On 02/16/17 15:52, Stephan Bergmann wrote:
>>> Forgive me if this has already been discussed or even fixed in later
>>> versions:  At least the Firebird 3.0 we build as part of LibreOffice
>>> defines global operator new replacement functions in
>>> src/common/classes/alloc.h (forwarding to MemoryPool) that do not in
>>> general fulfil the alignment requirements for such functions.
>>>
>>> Came across this when Firebird compiled with a recent trunk Clang (with
>>> -O, and DEBUG_GDS_ALLOC being undefined) on x86_64-unknown-linux-gnu
>>> causes SEGV from misaligned MOVAPS instructions.
>>>
>> Yes - allocated memory is aligned at 8 bytes boundary now.
>> I've tried to set alignment to 16 but looks like that's far not 5 lines
>> patch - sometimes we were choosing between 4/8 bytes alignment, but last
>> years only 8 bytes alignment was used.
>> May be finding specific compiler flag to avoid this instruction is
>> simpler choice for today?
> (The way I work around this for now in LibreOffice is by always defining
> DEBUG_GDS_ALLOC when building with Clang on Linux X86-64,
> 
> "external/firebird: Work around operator new alignment violations".
> Since LibreOffice builds intended for widespread distribution are on
> Linux usually done with GCC not Clang, this shouldn't have a performance
> impact.)

Afraid that this work around is not too stable (not to say more). 
Certainly that macro does change details of memory allocation and may 
provide 16-bit alignment in some particular case but be sure it's not 
more than side effect and does not gurantee such alignment.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Stephan Bergmann]

On 02/17/2017 11:48 AM, Alex Peshkoff wrote:
> On 02/16/17 15:52, Stephan Bergmann wrote:
>> Forgive me if this has already been discussed or even fixed in later
>> versions:  At least the Firebird 3.0 we build as part of LibreOffice
>> defines global operator new replacement functions in
>> src/common/classes/alloc.h (forwarding to MemoryPool) that do not in
>> general fulfil the alignment requirements for such functions.
>>
>> Came across this when Firebird compiled with a recent trunk Clang (with
>> -O, and DEBUG_GDS_ALLOC being undefined) on x86_64-unknown-linux-gnu
>> causes SEGV from misaligned MOVAPS instructions.
>>
>
> Yes - allocated memory is aligned at 8 bytes boundary now.
> I've tried to set alignment to 16 but looks like that's far not 5 lines
> patch - sometimes we were choosing between 4/8 bytes alignment, but last
> years only 8 bytes alignment was used.
> May be finding specific compiler flag to avoid this instruction is
> simpler choice for today?

(The way I work around this for now in LibreOffice is by always defining 
DEBUG_GDS_ALLOC when building with Clang on Linux X86-64, 

 
"external/firebird: Work around operator new alignment violations". 
Since LibreOffice builds intended for widespread distribution are on 
Linux usually done with GCC not Clang, this shouldn't have a performance 
impact.)

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] alloc.h global operator new replacement violating alignment requirements

[Alex Peshkoff]

On 02/16/17 15:52, Stephan Bergmann wrote:
> Forgive me if this has already been discussed or even fixed in later
> versions:  At least the Firebird 3.0 we build as part of LibreOffice
> defines global operator new replacement functions in
> src/common/classes/alloc.h (forwarding to MemoryPool) that do not in
> general fulfil the alignment requirements for such functions.
>
> Came across this when Firebird compiled with a recent trunk Clang (with
> -O, and DEBUG_GDS_ALLOC being undefined) on x86_64-unknown-linux-gnu
> causes SEGV from misaligned MOVAPS instructions.
>

Yes - allocated memory is aligned at 8 bytes boundary now.
I've tried to set alignment to 16 but looks like that's far not 5 lines 
patch - sometimes we were choosing between 4/8 bytes alignment, but last 
years only 8 bytes alignment was used.
May be finding specific compiler flag to avoid this instruction is 
simpler choice for today?



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5486) Using GBAK to restore a 3GB+ database infinite "restoring privilege for user SYSDBA gbak"

[Chau Chee Yang (JIRA)]

Using GBAK to restore a 3GB+ database infinite "restoring privilege for user 
SYSDBA gbak"
-

 Key: CORE-5486
 URL: http://tracker.firebirdsql.org/browse/CORE-5486
 Project: Firebird Core
  Issue Type: Bug
  Components: GBAK
Affects Versions: 3.0.1
 Environment: Windows 10 x64
Reporter: Chau Chee Yang


I am trying to restore a firebird 3 backup file of size 3GB++,  the restore 
goes smooth by restoring table, create index.  When i reach "restoring 
privilege" stage, it keep showing unstop

gbak:restoring privilege for user SYSDBA
gbak:restoring privilege for user SYSDBA
gbak:restoring privilege for user SYSDBA
gbak:restoring privilege for user SYSDBA
gbak:restoring privilege for user SYSDBA


I don't know when it will end.  It have run for more than 1 hour.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel