Re: [firebird-support] Very slow response after massive delete on table. (FB3 on Win 64)

[fabia...@itbizolutions.com.au [firebird-support]]

Thank you, much appreciated. 

Re: [firebird-support] Choosing best server / processor for Firebird 3.03 64 bits database on Windows server 2016 64 bits

[fabia...@itbizolutions.com.au [firebird-support]]

Thank you Mark!!

So the key here would be RAM and SSD disks correct?


Cheers,
Fabian


- Original Message - 
From: "Mark Rotteveel m...@lawinegevaar.nl [firebird-support]" 
<firebird-support@yahoogroups.com>
To: <firebird-support@yahoogroups.com>
Sent: Sunday, May 20, 2018 6:31 PM
Subject: Re: [firebird-support] Choosing best server / processor for 
Firebird 3.03 64 bits database on Windows server 2016 64 bits


> On 19-5-2018 15:00, fabia...@itbizolutions.com.au [firebird-support] 
> wrote:
>> We are trying to define the best specs for a windows based Firebird 3 64
>> bits instance.
>>
>> Based on our knowledge, and the type of applications running we know we 
>> will
>> need the best possible performance for 1 to 10 concurrent users demanding
>> queries (reading not writting) from just a few relational tables. We will
>> organize indexes and the question here is "what would be the best server
>> configuration". We believe the following:
>>
>> 1) We need the best processor, that can process the fastest per core, not
>> combined, as FB will run each request (sql select statement) on a 
>> separate
>> core, and because we will have up to 10 concurrent users we only need a
>> processor that has 10+ cores (all available new processors support that,
>> either on its own or on a dual processor configuration). The benchmarks
>> available on internet compare "the processsing capacity of the hole
>> processor, including all cores", so we don't have a "comparisson one core
>> against one core of another processor".
>> We believe the following processor would be the best:
>> Platinum 8156 Processor
>> Anyone has a better suggestion?
>
> To be honest, with only 1-10 concurrent users, the processor is hardly
> going to matter for performance. Database work is - usually - not very
> CPU-intensive and mostly IO-bound.
>
> Mark
> -- 
> Mark Rotteveel
>
>
> 
> Posted by: Mark Rotteveel <m...@lawinegevaar.nl>
> 
>
> ++
>
> Visit http://www.firebirdsql.org and click the Documentation item
> on the main (top) menu.  Try FAQ and other links from the left-side menu 
> there.
>
> Also search the knowledgebases at 
> http://www.ibphoenix.com/resources/documents/
>
> ++
> 
>
> Yahoo Groups Links
>
>
>

[firebird-support] Choosing best server / processor for Firebird 3.03 64 bits database on Windows server 2016 64 bits

[fabia...@itbizolutions.com.au [firebird-support]]

We are trying to define the best specs for a windows based Firebird 3 64 
bits instance.

Based on our knowledge, and the type of applications running we know we will 
need the best possible performance for 1 to 10 concurrent users demanding 
queries (reading not writting) from just a few relational tables. We will 
organize indexes and the question here is "what would be the best server 
configuration". We believe the following:

1) We need the best processor, that can process the fastest per core, not 
combined, as FB will run each request (sql select statement) on a separate 
core, and because we will have up to 10 concurrent users we only need a 
processor that has 10+ cores (all available new processors support that, 
either on its own or on a dual processor configuration). The benchmarks 
available on internet compare "the processsing capacity of the hole 
processor, including all cores", so we don't have a "comparisson one core 
against one core of another processor".
We believe the following processor would be the best:
Platinum 8156 Processor
Anyone has a better suggestion?

2) Our database will be about 5 TB, so having SSD disks will be essential to 
read fast.
3) RAM Memory, if the drives are SSD we probably don't need more than 128 GB 
ram, as the "swapping into ssd drives" will be very fast anyways.

We are assuming (and will check it) that buying SD disks is much cheaper 
than buying RAM for a server, hence option 2) and 3) are defined with an eye 
on the budget. If RAM is not as expensive we could buy magnetic disks and 
expand ram to 5TB, and end up placing the hole DB on ram via the FB cache 
(after the first select of each table with no filters reads the data, most 
of the database will be on RAM via cache correct? So we could run a dummy 
select on server starts and after say 1/2 hour the server will have the hole 
DB on ram.


Any suggestions?

Cheers,
Fabian







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Bulk upload options

[fabia...@itbizolutions.com.au [firebird-support]]

Hi

This article explains the solutions to Bulk upload,

http://www.firebirdfaq.org/faq209/

Like Firebird FAQ already thought of many issues

Cheers,
Fabian


- Original Message - 
From: "Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]" 
<firebird-support@yahoogroups.com>
To: <firebird-support@yahoogroups.com>
Sent: Wednesday, May 16, 2018 7:10 PM
Subject: Re: [firebird-support] Bulk upload options


> 16.05.2018 4:48, fabia...@itbizolutions.com.au [firebird-support] wrote:
>> Does anyone have experience with bulk updates via isql? Would it be X 
>> times faster than using a normal application inserting the records inside 
>> a begin transaction and commit at the end?
>
>   No, because it is exactly what isql does: execute a prepared statement 
> for each line of
> data. No magic inside.
>
>> I think this solutions will do
>
>   External tables has a definite limitations, for example you cannot put 
> NULL into it. If
> it is ok for you - fine. Otherwise the fastest way is export data into a 
> file on one side,
> compress, transfer to other side, uncompress and then parse the file and 
> insert records
> one-by-one in a single transaction.
>   Firebird 4 has some kind of batch interface, but I'm not sure if it can 
> speed up bulk load.
>
> -- 
>   WBR, SD.
>
>
> 
>
> 
>
> ++
>
> Visit http://www.firebirdsql.org and click the Documentation item
> on the main (top) menu.  Try FAQ and other links from the left-side menu 
> there.
>
> Also search the knowledgebases at 
> http://www.ibphoenix.com/resources/documents/
>
> ++
> 
>
> Yahoo Groups Links
>
>
>







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Bulk upload options

[fabia...@itbizolutions.com.au [firebird-support]]

Well, let's think this through

When firebird executes the following command it does it extremelly quickly:

Insert into Table
Select * from Table

That probably is because the sentence insert into table expects multiple 
results from the Select statement.

isql on the other hand will have to execute an insert per record, unless I 
could say:

Insert into Table
Values (a,b,c)
+ (next record)
Values (a2,b2,c2)
+ etc

Is there a sentence in Firebird that would allow inserting multiple records 
from a TXT file?
Perhaps defining an external file table, with the contents of what I want to 
upload, and executing:

Insert into Target Table
Select * from External TXT File, and if the TXT file is on an SSD HDD one 
would assume this would fly, right??

Cheers,
Fabian



- Original Message - 
From: "Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]" 
<firebird-support@yahoogroups.com>
To: <firebird-support@yahoogroups.com>
Sent: Wednesday, May 16, 2018 7:10 PM
Subject: Re: [firebird-support] Bulk upload options


> 16.05.2018 4:48, fabia...@itbizolutions.com.au [firebird-support] wrote:
>> Does anyone have experience with bulk updates via isql? Would it be X 
>> times faster than using a normal application inserting the records inside 
>> a begin transaction and commit at the end?
>
>   No, because it is exactly what isql does: execute a prepared statement 
> for each line of
> data. No magic inside.
>
>> I think this solutions will do
>
>   External tables has a definite limitations, for example you cannot put 
> NULL into it. If
> it is ok for you - fine. Otherwise the fastest way is export data into a 
> file on one side,
> compress, transfer to other side, uncompress and then parse the file and 
> insert records
> one-by-one in a single transaction.
>   Firebird 4 has some kind of batch interface, but I'm not sure if it can 
> speed up bulk load.
>
> -- 
>   WBR, SD.
>
>
> 
>
> 
>
> ++
>
> Visit http://www.firebirdsql.org and click the Documentation item
> on the main (top) menu.  Try FAQ and other links from the left-side menu 
> there.
>
> Also search the knowledgebases at 
> http://www.ibphoenix.com/resources/documents/
>
> ++
> 
>
> Yahoo Groups Links
>
>
>







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Bulk upload options

[fabia...@itbizolutions.com.au [firebird-support]]

Just found this article https://www.arbinada.com/en/node/1425


Firebird: bulk insert and bulk export
Submitted by Serguei_Tarassov on Fri, 02/08/2013 - 16:50
Firebird doesn't support INSERT BULK command or bcp.exe utility. But the 
external tables can be a good replacement. Simple example is below.

Check that firebird.conf file allow to create external tables. Write or 
uncomment line like

ExternalFileAccess = Full

Create external table


CREATE TABLE BCP_DATA EXTERNAL FILE 'с:\temp\temp.txt' (
  ID CHAR(10) NOT NULL,
  NAME CHAR(100),
  CRLF CHAR(2)  -- CHAR(1) for Linux
);
COMMIT;
Bulk export from some Firebird table (ensure that temp.txt file is empty):


INSERT INTO BCP_DATA(ID, NAME, CRLF)
SELECT ID, NAME, ascii_char(13) || ascii_char(10) FROM MY_REAL_TABLE;
As result, you have temp.txt file in "fixed length columns" format

Bulk import (ensure that temp.txt has data in fixed length format):


INSERT INTO MY_REAL_TABLE(ID, NAME)
SELECT ID, NAME FROM BCP_DATA;
I think this solutions will do




- Original Message - 
From: "fabia...@itbizolutions.com.au [firebird-support]" 
<firebird-support@yahoogroups.com>
To: <firebird-support@yahoogroups.com>
Sent: Wednesday, May 16, 2018 12:43 PM
Subject: [firebird-support] Bulk upload options


>I am looking for the fastest way to run a bulk insert.
> I believe one way would be using isql.exe , I need to plan a solution to 
> upload into a single table around 500 million records. Does anyone have 
> experience with bulk updates via isql? Would it be X times faster than 
> using a normal application inserting the records inside a begin 
> transaction and commit at the end?
> The issue is I have a remote database (not a FB database) and my options 
> are reading the remote DB and writing into FB one record at the time with 
> a loop, or asking the remote DB to dump the table into a txt file, then 
> compressing the file with winrar or whatever compression, and then using 
> that file to execute a bulk upload to FB," This is an ongoing issue, is 
> not a one of, so I am trying to design the best performing solution, not a 
> one of trick.
> I know the volume of data will be around 250 Gb per run, hence that could 
> be compressed at the server non-FB and sent to the FB server via internet, 
> the compressed size would be probably just 5% of the uncompressed size as 
> it is all text, so we are talking about 12 Gb aprox, the only question is 
> "would then uploading the txt file into a table  be quicker than a line by 
> line insert into table?
>
>
>
>
> 

[firebird-support] Bulk upload options

[fabia...@itbizolutions.com.au [firebird-support]]

I am looking for the fastest way to run a bulk insert. 
 I believe one way would be using isql.exe , I need to plan a solution to 
upload into a single table around 500 million records. Does anyone have 
experience with bulk updates via isql? Would it be X times faster than using a 
normal application inserting the records inside a begin transaction and commit 
at the end?
 The issue is I have a remote database (not a FB database) and my options are 
reading the remote DB and writing into FB one record at the time with a loop, 
or asking the remote DB to dump the table into a txt file, then compressing the 
file with winrar or whatever compression, and then using that file to execute a 
bulk upload to FB," This is an ongoing issue, is not a one of, so I am trying 
to design the best performing solution, not a one of trick.
 I know the volume of data will be around 250 Gb per run, hence that could be 
compressed at the server non-FB and sent to the FB server via internet, the 
compressed size would be probably just 5% of the uncompressed size as it is all 
text, so we are talking about 12 Gb aprox, the only question is "would then 
uploading the txt file into a table  be quicker than a line by line insert into 
table?
 

 

[firebird-support] Attempt to set in database number of buffers which is out of acceptable range ...

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All

Our production server is a Win 64 massive ram memory Firbierd 3 64bits, and 
our development server is a VM on Win 32 with Fb 3 (32 bits)

When we backup the database from the production x64 server, and try to 
retore it on the win32 server we usually get the following message:

Attempt to set in database number of buffers which is out of acceptable 
range ...


We know why it happens, because the production database has set the cache 
databae pages to half a million pages. However on Win 32 the limit is 132K 
cache pages.
Is there any   way to "force" the restore anyways and then set the DB cache 
pages to a reasonable number on the Win32 server? Else we need to modify in 
production the cache pages before running the backup, something we are 
trying to avoid.

Thanks
Fabian



- Original Message - 
From: "Jorge Andres Brugger jorge.brug...@gmail.com [firebird-support]" 

To: 
Sent: Thursday, October 19, 2017 1:09 AM
Subject: Re: [firebird-support] "Invalid Cursor State"


Sean:
It´s an executable procedure which returns a parameter. It´s not a
selectable procedure.
Its the kind of SP you invoke like "execute procedure ... returning values
..." while selectable ones are usually called as "select ... from
stored_procedure_name"
I´ve managed to fix my issue, anyway.
Thanks!

2017-10-17 17:47 GMT-03:00 'Leyne, Sean' s...@broadviewsoftware.com
[firebird-support] :

>
>
>
>
> Dimitry:
> I do it all the time ... (read output parameters from executable
> procedures, using "call executable_sp" by ODBC). No issues with other SPs.
>
>
>
>  Unless I missed something, your SP is missing the required SUSPEND to
> make the SP selectable.
>
>
>
>
>
> Regards
>
>
>
>
>
>
>
> 2017-10-17 15:38 GMT-03:00 Dimitry Sibiryakov s...@ibphoenix.com
> [firebird-support] :
>
> 17.10.2017 20:12, Jorge Andres Brugger jorge.brug...@gmail.com
> [firebird-support] wrote:
> > What am I doing wrong?
>
>You cannot fetch from non-select query.
>
>
> --
>WBR, SD.
>
>
> 
>
> 
>
> ++
>
> Visit http://www.firebirdsql.org and click the Documentation item
> on the main (top) menu.  Try FAQ and other links from the left-side menu
> there.
>
> Also search the knowledgebases at http://www.ibphoenix.com/
> resources/documents/
>
> ++
> 
>
> Yahoo Groups Links
>
>
>
>
>
>
>
>
>
>
>
>

Re: [firebird-support] SS FB3.01 - Is it possible to set CPU utilization to max?

[fabia...@itbizolutions.com.au [firebird-support]]

Anto

Each "request" to the database is managed by one CPU, and 6 CPU will be used 
in full when 6 concurrent requests are passed to Firebird, however to get 
Firebird to process each request with all available CPUs is not possible at 
the moment, it would require a re-write of Firebird.

Regards
Fabian


- Original Message - 
From: "trsk...@yahoo.com [firebird-support]" 

To: 
Sent: Tuesday, December 06, 2016 1:59 PM
Subject: [firebird-support] SS FB3.01 - Is it possible to set CPU 
utilization to max?


> Hi all,
>
> During project development/testing or database maintenance, such as 
> backup/restore, mostly, there is only 1 connection to FB Server.
>
> I have 6 core processors and during those process, cpu's utilization is 
> about  16%. I knew, this is because there is only 1 connection to FB 
> server, 2 connections will raised cpu's utilization to 32% and so on.
>
> Is it possible to force Firebird to use max. cpu utilization when there is 
> only 1 connection?
>
> Thanks & regards,
> Anto
>
>
>
>
> 

Re: [firebird-support] Overflow problem

[fabia...@itbizolutions.com.au [firebird-support]]

Are you trying to store details of the American public debt? or the world's 
public debt? Just kiding Not

How about assigning another integer field to each record with a value 
representing the "segment" for the value you are trying to store, and then 
use an index on that value, I'll explain with an example:

Say your table contains values from 0 to 1,000,000,,000,000

Add another field called "segment" and convert your values as follows:

where the original record value was 1:
Value = 1
Segment = 0

Where the original record was 65534
Value = 65534
Segment = 0


Where the original record was 65537
Value = 1
Segment = 1 (Segment 1 = 2 bytes segment, representing 256*256 = 65536)
So Value 1 plus segment 1 = 1+65536 = 65537

Where the original record was 131073
Value = 1
Segment = 2 (Segment 2 = 2 bytes segment, representing 256*256 * 2 = 131072)
So Value 1 plus segment 2 = 1+131072 = 131073

If you use 4 bytes for the segment then the segment 1 = 256*256*256*256 = 
4228250625 = 4GB


Now the important part, when running the select say I want to get all values 
between 2GB and 3GB, then :

Select  value from my_table where segment between  2GB / 65536 and 3GB / 
65536 (if you choose to use 2 bytes for the segment)

This will fly compared to any other solution asuming you have an index on 
segment.

Are you storing prime number in your DB? is that what you are trying to 
resolve? If yes, I believe using the solution I am mentioning will work 
fine, perhaps you may need 2 or more segment fields to get the best 
performance,

Cheers,
Fabian

[firebird-support] Explicit Transaction + Suspend

[fabia...@itbizolutions.com.au [firebird-support]]

Just wondering can my app start a transaction with Begin Transaction and 
then call a stored procedure that contains the SUSPEND sentence at the end 
while the transaction is still opened? Is there any special consideration 
when comencing a transaction other than to ensure the process finished as 
soon as posible with a commit or rollback, and no human intervention is 
needed in between Begin Transaction and Commit , such as messages on screen 
or buttons needed to be pressed, etc?







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Safe Thread - ODBC

[fabia...@itbizolutions.com.au [firebird-support]]

SOLVED - Changed code to use explicit transactions and the problem went 
away, so I guess implicit transactions was "keeping the transaction opened" 
for a few extra milliseconds... while the other "process / thread / or ODBC 
object" was trying to read the data added by the last transaction, and as it 
was not commited yet, the data was not visible. Many many years ago, like 15 
years ago, a friend who advised me to use Firebird told me "dont use 
explicit transations with Firebird; you'll be better of leaving Firebird 
manage it implicit..."; it did not sound correct then and it came back 
to bite me now...

Cheers,
Fabian

[firebird-support] Safe Thread - ODBC

[fabia...@itbizolutions.com.au [firebird-support]]

We have a multithreaded app written in Clarion 9 (win32), running on several 
win 2012 64 bits r2, connecting to Firebird 3 (latest build) via ODBC 32 
bits (latest ODBC). We are experiencing a rare propblem, that happens not 
very often. The issue is as follows:

The application does not use "begin transaction and commit", so we let the 
database commit automatically
after a write process, we read the table from another thread inside the app 
or from another app inside the same OS session, and the data is not there. 
Of course after a few seconds (or milliseconds) the data is available. Our 
concern is that we have a sequence of code that assume whatever was written 
5 lines above is already available for reading, but it is failing every so 
often. One solution we are considering is to use "explicit" begin 
transaction and commit instead of implicit, any thoughts? (in the past, when 
firebird was on version 1.5 or 2) we tried to use explicit transactions and 
the system would lock up every 20 minutes... probably deadly embrace, but we 
believe it was not related to the application itself, we attributed at the 
time to a possible bug somewhere betweeen the ODBC or the Firebird engine 
because the same app running against another DB engine would not lock up. 
Any suggestions?


Cheers,
Fabian

[firebird-support] How to "embedded mode" configure and shipping with an App

[fabia...@itbizolutions.com.au [firebird-support]]

We are trying to configure and package Firebird 3 as an embedded database 
with our App. Could someone provide an example of how to do it please? We 
have configured Firebird to work as embedded locally but we need to ship all 
files into one directory with our app, and don't know what files are needed 
just for an embedded database server.

[firebird-support] Encryption on FB3

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Guys

We are considering encrypting the DB, we are on FB 3 on top of Windows 2012 
R2 64 bits, the options we have are:

1) Do what we did with FB2, that is encrypt the DB folder with the same user 
account as the firebird service was running on, and nevel login with that 
user. That is an OS file encryption, and it worked very well for us.

2) Use the new FB 3 encrytion, we do not have any idea how to, is there any 
document we can read to start testing?

3) Use TrueCrypt on the hole drive , or something similar, placing the 
encryption at the OS level.

What would be better? Anybody already tried and would like to share ?

Alternativelly we can wait for FB4, where I believe encryption will be 
"easier" to configure, am I right?

Cheers
Fabian

Re: [firebird-support] Re: Nbackup restored database size is 50 Mega shorter than original DB

[fabia...@itbizolutions.com.au [firebird-support]]

Thanks Vlad

- Original Message - 
From: "hv...@users.sourceforge.net [firebird-support]" 

To: 
Sent: Wednesday, September 21, 2016 6:00 AM
Subject: [firebird-support] Re: Nbackup restored database size is 50 Mega 
shorter than original DB


>> ---In firebird-support@yahoogroups.com,  wrote :
> >
> > We noticed the results of restoring with NBackup a 7 Gb database is 50
>> megabytes shorter in size than the original DB. Is this a problem? Or it
>> does not matter? Both Db are equal?
>
>  Engine expands database file by relatively big chunks, in advance, 
> therefore some pages at the
> tail of the database file could be never used at backup moment. nbackup 
> doesn't include such
> pages into backup file.
>
>  If you want to be sure, run gfix -v on restored database.
>
> Regards,
> Vlad
>
>
> 

[firebird-support] FB 3.0 + ODBC

[fabia...@itbizolutions.com.au [firebird-support]]

This is a message to the development team, including all testers, 
documenters, project management, etc

After almost 14 years using Firebird (from version 1 candidate release 3) 
until today, with FB 3 migration just finalized on Tuesday night this week, 
with 3 databases (100Mb, 7GB and 70Gb) serving multiple Apps (7+) with 
mutliple users (30+ concurrent); and after been warned against jumping on 
FB3, with some reasonable concerns, we are ABSOLUTELLY delighted to see the 
results on FB3, in production. We can only be gratefull to the FB 
development team, the product ROCKS! We have tried to "disturb" the server 
by runnning backup and restore of other databases in parallel while users 
are online, we have even started in parallel multiple restore of mutliple 
databases on the side while the system is being accessed by 20 users, and 
the performance and response is outstanding. For those who want to know our 
config is:

DL 380 G7 Dual 12 core processor server, with 192 GB ram, 3.5 TB Raid 6; 512 
cache on raid.
Windows standard 2012 R2 64 bits.
ODBC 2.04 (firebird)
Firebird 3.0 64 bits (snapshot 9/9/16)
All apps running compiled with Clarion 9.1

I would not consider MS SQL even if MS pays me to change.


Well done Dev team
Congrat

Fabian

Re: [firebird-support] Gbak performance issues on FB3

[fabia...@itbizolutions.com.au [firebird-support]]

Ahhh, OK, thanks!!

Re: [firebird-support] "nbackup needs local access to database file"

[fabia...@itbizolutions.com.au [firebird-support]]

Ok, but something is not right, in some instances I am getting about 1 or 2 
megabytes throughput with gbak fb3, and I am getting 150 MB throughput with 
gbak fb 2.5; the problem is so evident it cannot be just different access. It 
seems there is a bottle neck somewhere; perhaps the small changes I've made to 
the firebird.conf are affecting it substancially??? Simple changes like tcpip 
packet size to 16384 instead of 8192, more memory assigned to some resources, 
nothing creazy, like multiply by 4 changes...
  - Original Message - 
  From: Mark Rotteveel m...@lawinegevaar.nl [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Sunday, September 11, 2016 5:11 PM
  Subject: Re: [firebird-support] "nbackup needs local access to database file"


  On 10-9-2016 23:34, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > I am getting this error while running nbackup:
  >
  > "nbackup needs local access to database file"
  >
  > The nbackup sentence is:
  >
  > Nbackup.exe -user SYSDBA -pas HereMyPass   -B  0
  >  localhost:MyDB_FB3.fdbC:\temp\MyDB_Level_0.nbk
  >
  > If I remove localhost: then it works, but that would be running as
  > embebed and it is a superserver with active network connections.
  >
  > Any ideas?

  Gbak and nbackup are different tools that work in a different way. Gbak 
  works through a Firebird server (real or embedded), while nbackup works 
  with the physical backup file.

  Mark
  -- 
  Mark Rotteveel


  
  Posted by: Mark Rotteveel <m...@lawinegevaar.nl>
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Gbak performance issues on FB3

[fabia...@itbizolutions.com.au [firebird-support]]

Mark

Ok, I understand, but on the same server I just reverted to Firebird 2.56 and 
the gbak flies, like it works several times faster than on FB3. One more thing 
I've noticed that may help understand what is happening, on FB3 gbak uses 15x 
(15 times) aprox. more network resources that hdd resources while doing a local 
backup or restore. Obviously in FB 2.5 the network is at zero because there is 
no need to go through the localhost or ip to avoid launching the embebed 
connection (the fbk files are local on the hdd, and the database obviously is 
also local to the same server). My theory is that the issue is somewhere around 
there. Pehaps there is a way to talk to the super server without involving the 
network card? Perhaps there is a way to finetune the data flow between gbak and 
the database and the fbk creation, as it seems somewhere in the chain there is 
a delay, it seems somewhere there is a bottleneck.

I hope my long hours of debbuging are helping identify a potential issue. 

And to finish on a good note, so far with my application FB3 has not shown any 
negative behaviour when compared with FB 2.5, I am connecting via ODBC 32 bits.

Cheers,
Fabian




- Original Message - 
  From: Mark Rotteveel m...@lawinegevaar.nl [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Sunday, September 11, 2016 5:09 PM
  Subject: Re: [firebird-support] Gbak performance issues on FB3


  On 10-9-2016 22:11, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > I am running both gbak.exe and nbackup.exe on an FB3 server with Windows
  > 2012 R2 64 bits, 24 processor, 192 GB ram, the nbackup.exe level zero is
  > much faster than the gbak.exe (does not make any sense, it should take
  > about the same time as gbak), both running locally but connecting using
  > the super server (localhost:database_name); any ideas why gbak.exe is so
  > slow?

  Your assumption is wrong, nbackup is expected to be a lot faster than 
  gbak as it makes a physical backup of the database pages instead of a 
  logical backup of the data in the database.

  Mark

  -- 
  Mark Rotteveel


  
  Posted by: Mark Rotteveel <m...@lawinegevaar.nl>
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

[firebird-support] "nbackup needs local access to database file"

[fabia...@itbizolutions.com.au [firebird-support]]

I am getting this error while running nbackup:

"nbackup needs local access to database file"

The nbackup sentence is:

Nbackup.exe -user SYSDBA -pas HereMyPass   -B  0  localhost:MyDB_FB3.fdb
C:\temp\MyDB_Level_0.nbk

If I remove localhost: then it works, but that would be running as embebed and 
it is a superserver with active network connections.

Any ideas?

  - Original Message - 
  From: fabia...@itbizolutions.com.au [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Sunday, September 11, 2016 6:11 AM
  Subject: [firebird-support] Gbak performance issues on FB3





  I am running both gbak.exe and nbackup.exe on an FB3 server with Windows 2012 
R2 64 bits, 24 processor, 192 GB ram, the nbackup.exe level zero is much faster 
than the gbak.exe (does not make any sense, it should take about the same time 
as gbak), both running locally but connecting using the super server 
(localhost:database_name); any ideas why gbak.exe is so slow?

  The test database size is 70GB, 
  gbak takes 01:07:00 to complete,
  nbackup.exe level zero takes 13 minutes, that is 5 times faster than gbak... 
something is not right, any ideas



  

[firebird-support] Gbak performance issues on FB3

[fabia...@itbizolutions.com.au [firebird-support]]

I am running both gbak.exe and nbackup.exe on an FB3 server with Windows 2012 
R2 64 bits, 24 processor, 192 GB ram, the nbackup.exe level zero is much faster 
than the gbak.exe (does not make any sense, it should take about the same time 
as gbak), both running locally but connecting using the super server 
(localhost:database_name); any ideas why gbak.exe is so slow?

The test database size is 70GB, 
gbak takes 01:07:00 to complete,
nbackup.exe level zero takes 13 minutes, that is 5 times faster than gbak... 
something is not right, any ideas

Re: [firebird-support] "File Database is not found" error appears while the database is been backup with gbak

[fabia...@itbizolutions.com.au [firebird-support]]

To Make it perfectly clear, the following statment works:
gbak -b -v -z -user SYSDBA -pas Hello-2 -T  localhost:DB_FB3.fdb c:\Temp\DB.Fbk

And the following DOES NOT work:

gbak -b -v -z -user SYSDBA -pas 'Hello-2 -T'  localhost:DB_FB3.fdb 
c:\Temp\DB.Fbk   (note the quotes enclosing the password)


However the following does work:

gbak -b -v -z -user SYSDBA -pas 'Hello-2 -T'  DB_FB3.fdb c:\Temp\DB.Fbk   (note 
I removed localhost: and the quotes enclosing the password are still there but 
the server is working in embebed mode instead of super server)

Re: [firebird-support] "File Database is not found" error appears while the database is been backup with gbak

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Guys

Just to let you know the gbak does not like the password eclosed with single 
quotes in this context (executed from the server where FB3 is actually 
installed, with localhost: before the name of the DB, so it likes it if you try 
to use the embedded mode but does not if trying the super server mode,

Cheers,
Fabian

  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Sunday, September 11, 2016 1:19 AM
  Subject: Re: [firebird-support] "File Database is not found" error appears 
while the database is been backup with gbak


  10.09.2016 17:10, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > what can I do to run a 24x7 system and have backups every so often without 
shutting down
  > the system for a few minutes?

 Don't ever use embedded connection with superserver mode in firebird.conf.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] "File Database is not found" error appears while the database is been backup with gbak

[fabia...@itbizolutions.com.au [firebird-support]]

Thanks Dimitry, I forgot,

So I just replace my database name on the gbak statment with a 
127.0.0.1/3050:database name and happy days?

Cheers,
Fabian
  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Sunday, September 11, 2016 1:19 AM
  Subject: Re: [firebird-support] "File Database is not found" error appears 
while the database is been backup with gbak


  10.09.2016 17:10, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > what can I do to run a 24x7 system and have backups every so often without 
shutting down
  > the system for a few minutes?

 Don't ever use embedded connection with superserver mode in firebird.conf.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

[firebird-support] "File Database is not found" error appears while the database is been backup with gbak

[fabia...@itbizolutions.com.au [firebird-support]]

Hi 

I just upgraded to FB 3 and the server has a task scheduled after midnight to 
execute gbak for all 3 databases on the server.
I've noticed the database connection attempts fail during the gbak execution. 
Is this a new "restriction" on gbak FB3?

The specifics:
gbak with version WI-V3.0.1.32591 Firebird 3.0, server taken from snapshots two 
days ago.
the gbak sentence is:
gbak -b -v -z -user SYSDBA -pas 'mypasshere' -T -Y 
c:\Database\FB-Backups\Backup_Main_DB.log  Main_DB_FB3.fdb 
c:\Database\FB-Backups\Main_DB.Fbk

the ODBC test feature responds "file database is not found" while gbak is 
running, then it goes back to working fine.

And if gbak requires full exclusive access then what can I do to run a 24x7 
system and have backups every so often without shutting down the system for a 
few minutes?

Cheers
Fabian

Re: [firebird-support] Re: Problems after FB3 upgrade

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Tony

Yes I had a working 3.0 environment, I stopped the service, replaced all files 
and folders and kept the cfg files only as the config was already operational, 
and then restarted the service. Run a restore with no issues, but the when 
trying to access the DB from another machine's IBExpert that was previosuly 
connecting fine with this server, the error message appeared,

Cheers,
Fabian
  - Original Message - 
  From: Tony Christiansen t...@adegroup.com.au [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, September 08, 2016 9:45 AM
  Subject: Re: [firebird-support] Re: Problems after FB3 upgrade




  Hi Fabian

  Did you have v3.0.0 installed before copying over the 3.0.1 snapshot. If not 
you may have to reinstall v3.0.0 as it uses a new security design and then copy 
the latest snapshot.

  (I updated v3.0.0 the the latest snapshot last night as Dimitry explained and 
FB seems fine)

  Regards
  Tony


  On 08/09/2016 9:23 AM, fabia...@itbizolutions.com.au [firebird-support] wrote:

  

After replacing the existing install with the snapshot I am getting the 
following error on connection:


Connecting... Failed!

Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
Install incomplete, please read the Compatibility chapter in the release 
notes for this version.
 
Attempting to connect to services manager... Failed!


Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
Install incomplete, please read the Compatibility chapter in the release 
notes for this version.



And the Firebird.log file is not present (I deleted the old file before 
replacing the FB installation with the snapshot files, and after restarting the 
service the log file is not being created)

Any idea what's going on??




  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, September 07, 2016 7:45 PM
  Subject: Re: [firebird-support] Re: Problems after FB3 upgrade


  07.09.2016 11:38, Tony Christiansen t...@adegroup.com.au 
[firebird-support] wrote:
  > I cannot find any info on doing a manual install to upgrade to the 
latest snapshot of FB3
  > on Windows.

 Just stop Firebird and unpack the archive over existing installation 
overwriting 
  everything except configuration files and security database.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links







  

Re: [firebird-support] Re: Problems after FB3 upgrade

[fabia...@itbizolutions.com.au [firebird-support]]

After replacing the existing install with the snapshot I am getting the 
following error on connection:

 
Connecting... Failed!

Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
Install incomplete, please read the Compatibility chapter in the release notes 
for this version.
 
Attempting to connect to services manager... Failed!


Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
Install incomplete, please read the Compatibility chapter in the release notes 
for this version.



And the Firebird.log file is not present (I deleted the old file before 
replacing the FB installation with the snapshot files, and after restarting the 
service the log file is not being created)

Any idea what's going on??




  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, September 07, 2016 7:45 PM
  Subject: Re: [firebird-support] Re: Problems after FB3 upgrade


  07.09.2016 11:38, Tony Christiansen t...@adegroup.com.au [firebird-support] 
wrote:
  > I cannot find any info on doing a manual install to upgrade to the latest 
snapshot of FB3
  > on Windows.

 Just stop Firebird and unpack the archive over existing installation 
overwriting 
  everything except configuration files and security database.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Problems after FB3 upgrade

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Dimitry

> Why to jump to a new version at once then? Run two systems in parallel, 
> synchronizing data with replication.

FB 3 brings in many new features we are needing, hence the plan to upgrade. I 
am hoping the issues with FB3 will not compromise data integrity and will not 
make the system unavailable. I understand there may be one "error message" here 
and there on select statments, but no data would be compromise. Am I wrong? 

Fabian






  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, September 07, 2016 11:06 PM
  Subject: Re: [firebird-support] Re: Problems after FB3 upgrade


  07.09.2016 14:23, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > Would you recommend installing the latest snapshot, or stay with the 
released FB3 version?

 Yes, use snapshot.

  >  would like to avoid getting my customer into shaky watters

 Why to jump to a new version at once then? Run two systems in parallel, 
synchronizing 
  data with replication.

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Problems after FB3 upgrade

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All

We are going live with the upgrade to FB3 this comming Friday night. Would you 
recommend installing the latest snapshot, or stay with the released FB3 
version? I have read some posts mentioning an issue with "invalid transaction 
handle" and would like to avoid getting my customer into shaky watters, as FB 
2.5 has been rock solid and most users have a high opinion of the system ; and 
would hate to see that damaged unnecesarelly.

Cheers,
Fabian



- Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, September 07, 2016 7:45 PM
  Subject: Re: [firebird-support] Re: Problems after FB3 upgrade


  07.09.2016 11:38, Tony Christiansen t...@adegroup.com.au [firebird-support] 
wrote:
  > I cannot find any info on doing a manual install to upgrade to the latest 
snapshot of FB3
  > on Windows.

 Just stop Firebird and unpack the archive over existing installation 
overwriting 
  everything except configuration files and security database.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] After update trigger: the same values for new.somefiled and old.somefield

[fabia...@itbizolutions.com.au [firebird-support]]

Would it be possible that the original update triggered a second update on the 
same record, hence the value of new and old are the same? For instance a 
trigger after update calls a procedure to update the same record, the first run 
of the trigger will have old <> new, but perhaps on the second run, triggered 
by the SP, the value of old would be the new value on the first execution of 
the trigger, and the value of new on the second execution may be the same as 
old if that field has not changed on the SP. Is that possible?


  - Original Message - 
  From: patrick_mar...@yahoo.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Saturday, August 27, 2016 8:04 PM
  Subject: [firebird-support] After update trigger: the same values for 
new.somefiled and old.somefield




  Hello,

  shouldn't the values of new.somefield and old.somefield be different in an 
after update trigger, when the value of that field has definitely changed?




  For some reason they both have the new value and I can't figure out why. If I 
didn't misread all the articles, I've found, completely, then they should be 
different, shouldn't they?





  I'm using Firebird 2.5.2.26539.




  Best regards,

  Patrick





  

Re: [firebird-support] Firebird 3 performance

[fabia...@itbizolutions.com.au [firebird-support]]

Hello

It is my understanding that the difference in performance is not as in previous 
versions because the server model has been changed to have multiple processors 
to attend to multiple requests, however each request is only served by one 
processor. In other words the FB3 server can assign each request to a different 
processor on demand, but the bottle neck remains being the hard drive, the 
memory, the semaphores when a parallel process is locking up some resource, 
etc. Having said all that, it is true that in theory one would imagine that 
comparing clasic server 2.5 (a model that assigns an independent process per 
connection) should compare similar in performance to FB3 with a upper hand 
towards FB3 because it shares the memory (cache) and fb2.5 classic does not 
share cache and hence each instance ends up re-reading the same data when 
reading the same data.

Cheers,
Fabian

  - Original Message - 
  From: 'Gert Scholten | Syntri ERP B.V.' e.g.schol...@syntri.nl 
[firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, July 28, 2016 6:31 PM
  Subject: [firebird-support] Firebird 3 performance


  Hello,


   


  I've got a question about the Firebird 3 performance. I've created a test
  application which does the following:


   




   


  FB 2.5.5 (Superclassic)


  FB 3.0 (Superserver)




  1.000.000 inserts


  27 sec


  47 sec




  # of connections in 30 sec


  ~1800 connections


  ~1300 connections




  Concurent reads 50 threads 100/queries / thread - average query time


  1,35 ms


  2,45 ms


   


   


  Firbebird is installed local as 64-bit service. The database is on an SSD.


   


  I was under the impression that the results should be the other way around.
  Ergo, Firebird 3.0 should be a lot faster.


   


  Am I doing something wrong?


   


   




  Met vriendelijke groet,
   




  E.G. (Gert) Scholten
  Technisch Manager


  Projectleider





  + 31 (0)528 82 00 55 (T)
  + 31 (0)528 82 00 59 (F)
  04086914 (KvK) 








  Elbe 2
  7908 HB Hoogeveen 





   


    e.g.schol...@syntri.nl (E)
    www.syntri.nl (I) 





   


   






  [Non-text portions of this message have been removed]




  
  Posted by: "Gert Scholten | Syntri ERP B.V." 
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

[firebird-support] How to make available a Firebird tool I prepared

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All

This may not be the right place to ask, I have prepared a tool that optimazes 
and automates the backup for Firebird Databases, providing a backup almost up 
to the last minute for large production databases (That was the aim when we 
planned it). I would like tomake it available to other Firebird users.Is there 
a process in place for vetting and considering tools?
Cheers
Fabian

Re: [firebird-support] Unable to backup production database with gbak and SuperServer 3

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Martin

Now the error is regarding the user and password, that is good news, the engine 
trying to grab the DB is the superserver, not the local embedded.
Now regarding the user and password, there are big changes in FB3 with 
security, can I recommend you read the release notes where it explains how to 
setup security and then decide the security model and then based on that sort 
out the backup? That is what I did and it worked for me, I got rid of the 
legacy security, I am using the new model, and used ISQL to create the users 
and change the sysdba password, and things are working, but if you choose to 
use legacy security I do not know how to assist because I did not go that way,

Cheers,
Fabian


- Original Message - 
  From: Köditz, Martin martin.koed...@it-syn.de [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Monday, July 04, 2016 11:30 PM
  Subject: Re: [firebird-support] Unable to backup production database with 
gbak and SuperServer 3





  Hi Fabian,

   

  I changed my command, but now I'm getting that error:

  gbak -v -backup 127.0.0.1:prod.fdb backup/prod.fbk -user sysdba -password 
masterkey

  gbak: ERROR:Your user name and password are not defined. Ask your database 
administrator to set up a Firebird login.

  gbak:Exiting before completion due to errors

   

  Am I doing something fundamental wrong? Password and user name are correct. 
No problems when connecting with Flamerobin.

  Is gbak running in embedded mode per default? I thought it communicates with 
the running server engine.

   

  Regards

  Martin




  

Re: [firebird-support] Engine12 mistery

[fabia...@itbizolutions.com.au [firebird-support]]

Dimitry

> No. Y-valve is fbclient.dll. It loads providers as configured. If you don't 
> need to 
> work with databases directly, you can omit Engine from list of providers.
>  Firebird server (fbserver.exe) uses fbclient.dll the same way as any other 
> application

mmm, now we are back to my original question / problem. When I omit Engine12 
from the provider's list at the server's config, to ensure the server always 
acts as a super server, and never acts as an embedded server, the clients from 
other computers cannot connect via tcp-ip (ip.ip.ip.ip:port/databasename); so 
why is engine12 needed as a provider at the server's config? (Everybody already 
answered that question saying because it is the engine of the database, but you 
are saying it is only the embedded engine of the DB, correct?

Cheers,
Fabian



  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 15, 2016 6:56 PM
  Subject: Re: [firebird-support] Engine12 mistery


  14.06.2016 22:46, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > It seems Engine12 is not the same "type of component" as the other 
Providers. In the
  > documentation it is refered as a Y valve, if I understand it correctly 
Engine12 is the
  > base of the Y, while the other providers are the top left and top right of 
the Y, you can
  > have any top you want, BUT YOU MUST HAVE the base always, else it does not 
work, correct?

 No. Y-valve is fbclient.dll. It loads providers as configured. If you 
don't need to 
  work with databases directly, you can omit Engine from list of providers.
 Firebird server (fbserver.exe) uses fbclient.dll the same way as any other 
application.

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Engine12 mistery

[fabia...@itbizolutions.com.au [firebird-support]]

Mark

Thank you, I think I am beggining to understand the concept, so the provider's 
have to be combined including ALWAYS Engine12 (the core FB server 
procedure/method that really goes to the file on disk and does the reading and 
writting to the DB). So the following combinations are valid:
Providers = Remote + Engine12 
Providers = Loopback + Engine12 
Providers = Remote  + Loopback + Engine12 (the first two in any orther)

And any Providers combination excluding Engine12 would not work because it is 
not including the component that goes to the DB file and reads/writes from /to 
it.

It seems Engine12 is not the same "type of component" as the other Providers. 
In the documentation it is refered as a Y valve, if I understand it correctly 
Engine12 is the base of the Y, while the other providers are the top left and 
top right of the Y, you can have any top you want, BUT YOU MUST HAVE the base 
always, else it does not work, correct?


Cheers,
Fabian


  - Original Message - 
  From: Mark Rotteveel m...@lawinegevaar.nl [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 15, 2016 5:32 AM
  Subject: Re: [firebird-support] Engine12 mistery


  On 2016-06-14 19:44, fabia...@itbizolutions.com.au [firebird-support] 
  wrote:
  > My understanding from reading the documentation is:
  >
  > Provider = Remote means the client is on the network, excluding
  > 127.0.0.1
  > Provider = Loopback means the client is actually on the same OS
  > instance as the FB3 engine, and it is using the 127.0.0.1 to access
  > the database to avoid the "embedded server concept" from answering 
  > the
  > request, as it would capture the DB file and will not allow any other
  > clients from remote /network source.
  > Engine12 = The local server takes control of the database as if it 
  > was
  > an embedded server, killing all future chance of accessing the DB 
  > from
  > the network, so Engine12 is if I understand correctly the way to talk
  > to the DB engine in "exclusive" mode when you want to perform
  > maintenance or work on the security database.
  >
  > Is the above wrong?

  Yes, your understanding is wrong. Engine12 **is** the component that 
  actually does the work with the database file. The rest is just plumbing 
  to connect to servers (or make fbclient connect to a local server), 
  select the right provider, etc.

  As the release notes say: 
  
http://www.firebirdsql.org/file/documentation/release_notes/html/en/3_0/rnfb30-engine.html

  * Remote (establish network connection)
  * Engine12 (main database engine)
  * Loopback (force network connection to the local server for  without an explicit network protocol being supplied).

  There was also talk of providing an 'Engine11' which would essentially 
  be a Firebird 2.5 you could use to connect to ODS 11.2 and older 
  database files.

  Your confusion likely stems from the fact that Firebird embedded now is 
  fbclient + Engine12, as opposed to Firebird server + Engine12 for a 
  normal database server, while Firebird embedded used to be a separate 
  DLL/SO. BTW: In normal situations fbclient **does not** have access to 
  Engine12, so it only behaves as a client library.

  Mark



  
  Posted by: Mark Rotteveel <m...@lawinegevaar.nl>
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Engine12 mistery

[fabia...@itbizolutions.com.au [firebird-support]]

My understanding from reading the documentation is:

Provider = Remote means the client is on the network, excluding 127.0.0.1
Provider = Loopback means the client is actually on the same OS instance as the 
FB3 engine, and it is using the 127.0.0.1 to access the database to avoid the 
"embedded server concept" from answering the request, as it would capture the 
DB file and will not allow any other clients from remote /network source. 
Engine12 = The local server takes control of the database as if it was an 
embedded server, killing all future chance of accessing the DB from the 
network, so Engine12 is if I understand correctly the way to talk to the DB 
engine in "exclusive" mode when you want to perform maintenance or work on the 
security database.

Is the above wrong?


Thanks
Fabian

  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 15, 2016 4:19 AM
  Subject: Re: [firebird-support] Engine12 mistery


  14.06.2016 19:56, 'Mark Rotteveel' m...@lawinegevaar.nl [firebird-support] 
wrote:
  > Is there a reason why the FB 3 server configured as follows would require 
Engine12 as a
  > provider?

 Because it is server. The configuration is enough for Firebird client, but 
server needs 
  engine to work with databases.


  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

[firebird-support] Engine12 mistery

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All,

Is there a reason why the FB 3 server configured as follows would require 
Engine12 as a provider?

DatabaseAccess  = None
AuthServer  = Srp
AuthClient  = Srp
UserManager  = Srp
Providers  = Remote
WireCrypt  = Enabled 
WireCompression = false
ServerMode  = Super


I've noticed the network applications that access the FB3 db failed to connect 
unless I add the Engine12 to the providers list, would that be because FB3 
engine needs Engine12 to access the security DB at the handshake?

The way the network apps try to talk to the FB3 server is via ODBC and the 
database field is defined as 192.168.xxx.xxx:MyDbName , I also tried with 
192.168.xxx.xxx/3050:MyDbName and made no difference, any ide why Engine12 is 
required?


Cheers,
Fabian

Re: [firebird-support] FB3 - Is RemoteBindAddress a Serverwide variable or a Database linked variable?

[fabia...@itbizolutions.com.au [firebird-support]]

I'll answer myself for 2)

2) page 42 of 
http://www.firebirdsql.org/file/documentation/release_notes/Firebird-3.0.0-ReleaseNotes.pdf
 

1) and 3) remain a question

Cheers
Fabian

  - Original Message - 
  From: fabia...@itbizolutions.com.au [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Tuesday, June 14, 2016 10:52 PM
  Subject: [firebird-support] FB3 - Is RemoteBindAddress a Serverwide variable 
or a Database linked variable?





  Now that I have the production candidate FB 3 server up and running I am 
doing some fine tunning, and there a few questions I'd like to ask:

  1) The variable "RemoteBindAddress" in firebird.conf is a Server Wide value, 
or it can be set per database in databases.conf?
  2) The "WireCompression" variable at the server's firebird.conf file 
specifies it depends on the client's config. How can I config the client to 
ensure wire compression is enabled?
  3) After installing the client on Win 64 bits I've noticed there is a 
databases.conf file , where can I find info regarding understanding the client 
side features and ways to configure it to fine tune Firebird 3 from the client 
side? 


  For those who are involved in the Firebird Development Project, let me send 
you a MASSIVE CONGRATULATIONS on this product (FB3), it 
ROCKS

  Cheers,
  Fabian



  

[firebird-support] FB3 - Is RemoteBindAddress a Serverwide variable or a Database linked variable?

[fabia...@itbizolutions.com.au [firebird-support]]

Now that I have the production candidate FB 3 server up and running I am doing 
some fine tunning, and there a few questions I'd like to ask:

1) The variable "RemoteBindAddress" in firebird.conf is a Server Wide value, or 
it can be set per database in databases.conf?
2) The "WireCompression" variable at the server's firebird.conf file specifies 
it depends on the client's config. How can I config the client to ensure wire 
compression is enabled?
3) After installing the client on Win 64 bits I've noticed there is a 
databases.conf file , where can I find info regarding understanding the client 
side features and ways to configure it to fine tune Firebird 3 from the client 
side? 


For those who are involved in the Firebird Development Project, let me send you 
a MASSIVE CONGRATULATIONS on this product (FB3), it 
ROCKS

Cheers,
Fabian

Re: [firebird-support] FB 3 - Adding anew User

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Thomas,

I understand, I think I've done it 3 times before adding the password at the 
end, I will try the hole thing again in a new environment and come back with 
the log so we are sure we got this on nailed down. 

Cheers,
Fabian


  - Original Message - 
  From: 'Thomas Steinmaurer' t...@iblogmanager.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Tuesday, June 14, 2016 4:17 PM
  Subject: Re: [firebird-support] FB 3 - Adding anew User


  Hello Fabian,

  > Sorted after reading "Initializing the Security Database" from
  > 
http://www.firebirdsql.org/file/documentation/release_notes/Firebird-3.0.0-ReleaseNotes.pdf
  > at page 116. I think there is a small error on the documentation, where it 
says
  > " isql -user sysdba employee", the sentence works probably on Linux but on 
Ms
  > WIn 2012 Server R2 it requires the password to run.

  You need to shutdown every running regular Firebird server process, otherwise 
you won't end up in an Firebird 3.0 Embedded connection, which is needed if you 
want to initialize your 3.0 security database in SRP mode.

  --
  With regards,
  Thomas Steinmaurer
  http://www.upscene.com

  Professional Tools and Services for Firebird
  FB TraceManager, IB LogManager, Database Health Check, Tuning etc.



  > 
  > Cheers,
  > Fabian
  > 
  >  - Original Message - 
  >  From: fabia...@itbizolutions.com.au [firebird-support] 
  >  To: firebird-support@yahoogroups.com 
  >  Sent: Monday, June 13, 2016 4:34 AM
  >  Subject: [firebird-support] FB 3 - Adding anew User
  > 
  > 
  > 
  > 
  > 
  >  Hi All,
  > 
  >  I have configured the FB 3 server to only accept SRP connections and need 
to
  >  add a new User to the security DB, I read GSEC is not the way, and I have
  >  tried with isql but for some reason I am not having luck, I am trying the
  >  following:
  > 
  > 
  > 
  >  I created a script into a file as follows:
  > 
  >  SET SQL DIALECT 3;
  >  SET NAMES UTF8;
  >  SET CLIENTLIB 'C:\Temp\FB32Bits\fbclient.dll';   (this dll is 32 bits dll, 
but
  >  the server is 64 bits, and I use the 32 bits dll because IBexpert is 32 
bits)
  > 
  >  CONNECT '127.0.0.1/3050:My_New_DB_Configured_Properly_in_Databases.conf' 
USER
  >  'SYSDBA' PASSWORD 'NewPassNotMasterkey';
  > 
  >  CREATE USER NUCLEUSCRM PASSWORD 'WhateverPassword'   ;
  > 
  >  COMMIT work ;
  > 
  > 
  >  The above fails when executed from IBExpert script executive, the error is:
  > 
  >  "Unsuccessful execution caused by a system error that precludes successful
  >  execution of subsequent statements.
  >  Your user name and password are not defined. Ask your database 
administrator
  >  to set up a Firebird login."
  > 
  >  Its funny because IBexpert is connecting fine to the DB, and the following
  >  config is in place:
  > 
  >  AuthServer = Srp
  >  AuthClient = Srp
  >  UserManager = Srp
  > 
  > 
  >  So the only way to connect is with the new security model, and IBExpert is
  >  connecting when restoring the DB, but when running script into the DB
  >  something does not work. Any ideas?
  > 
  > 
  >   
  >  Cheers,
  >  Fabian
  > 
  > 
  > 
  > 
  > 
  > 
  >  



  
  Posted by: "Thomas Steinmaurer" <t...@iblogmanager.com>
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] FB 3 - Adding anew User

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Helen


I understand, I have read the comments, we are reffering to two different 
things, I mean the sentence in step 2) not further ahead, the sentence as 
placed on the Release Notes to execute the isql program fails to execute unless 
it contains the passowrd. What I am referring to is perhaps the sentence should 
be changed to include the password , let me be specific:

The following statment taken from 
http://www.firebirdsql.org/file/documentation/release_notes/Firebird-3.0.0-ReleaseNotes.pdf
 , page 116, "In a suitable shell, start an isql interactive session, opening 
the employee database via its alias: > isql -user sysdba employee" fails as 
follows:
isql -user sysdba employee

Statment Failed, SQLSTATE = 28000
Your User Name and PAssword are not defined

At this stage the connection was in embedded mode, and the service has just 
been started before running isql.

Now if we change the sentence and add the password the hole things works. That 
is what I am referrring to.
So the following sentence works fine:

.\isql -user sysdba employee -password 'whatever is your password for SYSDBA'


Other that that little issue, the hole explanation placed under title 
"Initialization Steps" is very clear, and has helped me understand the way 
secuity works on F3 3 and why it cannot be backwards compatible hence many 
security tools from the past dont work.



Cheers,
Fabian

  - Original Message - 
  From: Helen Borrie hele...@iinet.net.au [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Tuesday, June 14, 2016 1:04 PM
  Subject: Re: [firebird-support] FB 3 - Adding anew User


  Hello fabia...@itbizolutions.com.au,

  Tuesday, June 14, 2016, 1:49:30 PM, you wrote:


  > Sorted after reading "Initializing the Security Database" from
  > 
http://www.firebirdsql.org/file/documentation/release_notes/Firebird-3.0.0-ReleaseNotes.pdf
  > at  page 116. I think there is a small error on the documentation,
  > where it says "  isql -user sysdba employee", the sentence works
  > probably on Linux but on Ms WIn  2012 Server R2 it requires the password to 
run.

  No, it does not.  At the start of that very section, we state:

  "Initialization is performed in embedded mode using the isql utility.
  For an embedded connection, an authentication password is not required
  and will be ignored if you provide one"and so on.  It goes on to
  explain that you DO need to supply SYSDBA as the session user:  that's
  because maintaining users is a privileged task in the security
  database, even though you are connected to a regular database.

  From your Original Message -

  > CONNECT
  > '127.0.0.1/3050:My_New_DB_Configured_Properly_in_Databases.conf'   
  > USER 'SYSDBA' PASSWORD 'NewPassNotMasterkey';  
  >
  Here, you are using a client/server connection, not an embedded one.
  You certainly need to supply full credentials for that.

  So - time to read that chapter again.

  Helen



  
  Posted by: Helen Borrie 
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] FB 3 - Adding anew User

[fabia...@itbizolutions.com.au [firebird-support]]

Sorted after reading "Initializing the Security Database" from 
http://www.firebirdsql.org/file/documentation/release_notes/Firebird-3.0.0-ReleaseNotes.pdf
 at page 116. I think there is a small error on the documentation, where it 
says " isql -user sysdba employee", the sentence works probably on Linux but on 
Ms WIn 2012 Server R2 it requires the password to run.

Cheers,
Fabian

  - Original Message - 
  From: fabia...@itbizolutions.com.au [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Monday, June 13, 2016 4:34 AM
  Subject: [firebird-support] FB 3 - Adding anew User





  Hi All,

  I have configured the FB 3 server to only accept SRP connections and need to 
add a new User to the security DB, I read GSEC is not the way, and I have tried 
with isql but for some reason I am not having luck, I am trying the following:



  I created a script into a file as follows:

  SET SQL DIALECT 3;
  SET NAMES UTF8;
  SET CLIENTLIB 'C:\Temp\FB32Bits\fbclient.dll';   (this dll is 32 bits dll, 
but the server is 64 bits, and I use the 32 bits dll because IBexpert is 32 
bits)

  CONNECT '127.0.0.1/3050:My_New_DB_Configured_Properly_in_Databases.conf' USER 
'SYSDBA' PASSWORD 'NewPassNotMasterkey';

  CREATE USER NUCLEUSCRM PASSWORD 'WhateverPassword'   ;

  COMMIT work ;


  The above fails when executed from IBExpert script executive, the error is:

  "Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
  Your user name and password are not defined. Ask your database administrator 
to set up a Firebird login."

  Its funny because IBexpert is connecting fine to the DB, and the following 
config is in place:

  AuthServer = Srp
  AuthClient = Srp
  UserManager = Srp


  So the only way to connect is with the new security model, and IBExpert is 
connecting when restoring the DB, but when running script into the DB something 
does not work. Any ideas?


   
  Cheers,
  Fabian






  

[firebird-support] FB 3 - Adding anew User

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All,

I have configured the FB 3 server to only accept SRP connections and need to 
add a new User to the security DB, I read GSEC is not the way, and I have tried 
with isql but for some reason I am not having luck, I am trying the following:



I created a script into a file as follows:

SET SQL DIALECT 3;
SET NAMES UTF8;
SET CLIENTLIB 'C:\Temp\FB32Bits\fbclient.dll';   (this dll is 32 bits dll, but 
the server is 64 bits, and I use the 32 bits dll because IBexpert is 32 bits)

CONNECT '127.0.0.1/3050:My_New_DB_Configured_Properly_in_Databases.conf' USER 
'SYSDBA' PASSWORD 'NewPassNotMasterkey';

CREATE USER NUCLEUSCRM PASSWORD 'WhateverPassword'   ;

COMMIT work ;


The above fails when executed from IBExpert script executive, the error is:

"Unsuccessful execution caused by a system error that precludes successful 
execution of subsequent statements.
Your user name and password are not defined. Ask your database administrator to 
set up a Firebird login."

Its funny because IBexpert is connecting fine to the DB, and the following 
config is in place:

AuthServer = Srp
AuthClient = Srp
UserManager = Srp


So the only way to connect is with the new security model, and IBExpert is 
connecting when restoring the DB, but when running script into the DB something 
does not work. Any ideas?



Cheers,
Fabian

Re: [firebird-support] Re: Max RAM for Database Cache in SS FB 3.0 64bit is to small?

[fabia...@itbizolutions.com.au [firebird-support]]

Dimitry

Thanks, it works 100%,

Cheers,
Fabian

- Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, June 02, 2016 11:17 PM
  Subject: Re: [firebird-support] Re: Max RAM for Database Cache in SS FB 3.0 
64bit is to small?


  02.06.2016 15:12, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > the limit remains as in FB 2.5, meaning you cannot enter a value above 
131071 pages , now
  > I don't know if the bug is on IBExpert, or if Firebird 3.0 rejects the 
value.

 Try gfix and you'll see.

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Max RAM for Database Cache in SS FB 3.0 64bit is to small?

[fabia...@itbizolutions.com.au [firebird-support]]

Anto


Could you let me know how you setup the DB cache value? I mean what tool or 
sentence are you using to set it and then check it, because I am having trouble 
setting the value above 131071 pages with IBexpert, probably an IBExpert bug, 
but I just want to be sure it is not something else,

Thanks
Fabian



  - Original Message - 
  From: trsk...@yahoo.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, June 02, 2016 6:03 PM
  Subject: [firebird-support] Re: Max RAM for Database Cache in SS FB 3.0 64bit 
is to small?




  Dear all,

  Sorry, I have miscalculated the setting in DefaultDbCachePages, it was set 
more than available RAM on my Computer, no wonder it crashed.

  Now, everything is OK, tested on SS FB 3.0 in Win64.

  Thanks for your helps,

  Best regards,
  Anto.



  

Re: [firebird-support] Re: Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Robert

Here is the link to both files as the yahoo group may not allow me to attach 
the files
http://document.nucleuscrm.com.au:7770/pdfs/TMPFiles/TaskManager_Performance_ALL_RAM_Consumed.jpg
http://document.nucleuscrm.com.au:7770/pdfs/TMPFiles/TaskManager_Process_Firebird_Only_Consumed_2GB.jpg

The firebird is definetly 64 bits, however the ODBC that access the server is 
32 bits, but the client should have no effect on the way the server handles the 
DB (in theory).

Cheers,
Fabian

  - Original Message - 
  From: Robert martin r...@chreos.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, June 02, 2016 9:22 AM
  Subject: Re: [firebird-support] Re: Issue with Database Cache Size on FB 3.0




  Hi Fabian


  Can't see your screen shot but FB usage at 2GB sounds like the limit for a 
32Bit app.  Have you installed the 32Bit version of FB accidentally?

  Thanks
  Rob

  On 2/06/2016 11:12 a.m., fabia...@itbizolutions.com.au [firebird-support] 
wrote:

  
Dmitry

Something funny is going on regarding the memory consumption, I have 
attached screenshots showing the memory counters. If you look at the process 
itself (file TaskManager_Process_Firebird_Only_Consumed_2GB.jpg), it only 
consumed 2 GB. If you look at Windows Task Manager (file 
TaskManager_Performance_ALL_RAM_Consumed.jpg), where it says "Physical Memory, 
just under the memory graph where it shows 3.13 GB used, the cached memory is 
29963 Mb. I know Firebird is who consumed that CACHED memory because I am 
forcing the DB to read 70GB on purpouse, but why is the memory not showing 
under the process Firebird ? (it only shows 2GB used there).

My aim is to set the config to only use RAM (no file system cache), and to 
place the hole DB in RAM for caching when reading, I am struggling to find a 
config mix that will do that. The config variables are:

FileSystemCacheThreshold = 0
FileSystemCacheSize = 80
TempCacheLimit = 24576M (I am not sure what this value is for, it seems to 
mean the maximum amount of temporary things to have in memory, like DB read 
cache)

The above 3 variables are the ones I am playing with, plus the 
DefaultDBCachePages = 9 in Databases.conf

Cheers,
Fabian

- Original Message - 
From: Dmitry Yemanov dim...@users.sourceforge.net [firebird-support] 
To: firebird-support@yahoogroups.com 
Sent: Wednesday, June 01, 2016 3:49 PM
Subject: [firebird-support] Re: Issue with Database Cache Size on FB 3.0

01.06.2016 04:16, Fabian Ernesto Chocron wrote:
>
> We are having trouble setting up the database cache size on FB 3.0
> running on Windows 2008 R2 64 bits with 32 GB ram.
>
> The problem we have is we cannot get the server to allocate the ram for
> the cache as we intend. With FB 2.54 we had the DB cache set very high,
> close to 1 GB per database, all running in RAM memory. With FB 3.0 we
> read it can allocate much more RAM to the cache, but it appears the
> server is allocating very small amount of Ram when the first user
> connects to the DB, and as we connect more users to the DB the ram
> consumption increases slowly.

What memory counters do you look at? Virtual (reserved) memory usage 
should grow high with the first connect, physical (committed) memory 
usage should grow together with your load, while pages are being read 
from disk.

> On firebird.conf
>
> FileSystemCacheThreshold = 0
> FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB
> ram.)

These changes conflict with each other. You disable filesystem cache 
with the first line and still want to use 16GB with the second line.

> On databases.conf
>
> MyTestDB = c:\Temp\MyDb.fdb
> {
> DefaultDbCachePages = 458752
> }

First of all, I'd suggest to reset FileSystemCacheSize back to zero and 
set FileSystemCacheThreshold to something higher than 458752 pages (e.g. 
50). Then re-test your memory usage and only then start tweaking the 
filesystem cache.

BTW, don't you occasionally have page cache override at the database 
level? What is gstat -h output?

Dmitry





++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu. Try FAQ and other links from the left-side menu 
there.

Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

[Non-text portions of this message have been removed]



No virus found in this message.
Checked by AVG - www.avg.com

Re: [firebird-support] Re: Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Dmitry

Something funny is going on regarding the memory consumption, I have attached 
screenshots showing the memory counters. If you look at the process itself 
(file TaskManager_Process_Firebird_Only_Consumed_2GB.jpg), it only consumed 2 
GB. If you look at Windows Task Manager (file 
TaskManager_Performance_ALL_RAM_Consumed.jpg), where it says "Physical Memory, 
just under the memory graph where it shows 3.13 GB used, the cached memory is 
29963 Mb. I know Firebird is who consumed that CACHED memory because I am 
forcing the DB to read 70GB on purpouse, but why is the memory not showing 
under the process Firebird ? (it only shows 2GB used there).

My aim is to set the config to only use RAM (no file system cache), and to 
place the hole DB in RAM for caching when reading, I am struggling to find a 
config mix that will do that. The config variables are:

FileSystemCacheThreshold = 0
FileSystemCacheSize = 80
TempCacheLimit = 24576M  (I am not sure what this value is for, it seems to 
mean the maximum amount of temporary things to have in memory, like DB read 
cache)

The above 3 variables are the ones I am playing with, plus the 
DefaultDBCachePages = 9 in Databases.conf

Cheers,
Fabian


  - Original Message - 
  From: Dmitry Yemanov dim...@users.sourceforge.net [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 01, 2016 3:49 PM
  Subject: [firebird-support] Re: Issue with Database Cache Size on FB 3.0


  01.06.2016 04:16, Fabian Ernesto Chocron wrote:
  >
  > We are having trouble setting up the database cache size on FB 3.0
  > running on Windows 2008 R2 64 bits with 32 GB ram.
  >
  > The problem we have is we cannot get the server to allocate the ram for
  > the cache as we intend. With FB 2.54 we had the DB cache set very high,
  > close to 1 GB per database, all running in RAM memory. With FB 3.0 we
  > read it can allocate much more RAM to the cache, but it appears the
  > server is allocating very small amount of Ram when the first user
  > connects to the DB, and as we connect more users to the DB the ram
  > consumption increases slowly.

  What memory counters do you look at? Virtual (reserved) memory usage 
  should grow high with the first connect, physical (committed) memory 
  usage should grow together with your load, while pages are being read 
  from disk.

  > On firebird.conf
  >
  > FileSystemCacheThreshold = 0
  > FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB
  > ram.)

  These changes conflict with each other. You disable filesystem cache 
  with the first line and still want to use 16GB with the second line.

  > On databases.conf
  >
  > MyTestDB = c:\Temp\MyDb.fdb
  > {
  > DefaultDbCachePages = 458752
  > }

  First of all, I'd suggest to reset FileSystemCacheSize back to zero and 
  set FileSystemCacheThreshold to something higher than 458752 pages (e.g. 
  50). Then re-test your memory usage and only then start tweaking the 
  filesystem cache.

  BTW, don't you occasionally have page cache override at the database 
  level? What is gstat -h output?


  Dmitry




  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links




[Non-text portions of this message have been removed]

Re: [firebird-support] Re: Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Dmitry

I am progressing with solving this one, the first issue was related with a 
config variable reserving most of the ram upfront, hence there was no ram 
available for the DB cache. The second issue was misinterpreting what the 
following meant:

# File system cache size
# Note that the lowest number presently supported is 10%, and the highest number
# is 95%; numbers outside these limits will be set to the default of 30%.

First interpretation on the above was the variable must be between 0 and 1, as 
50% in math is usually written as 0.5, but then the following made it confusing:
# Type: integer, measured in % of total physical RAM

Where it says "integer" it complicated things, as it would not make sense to 
have a percentage forced to be integer, as there would be no way to represent 
40.5% for instance. So I assume the value is an integer because it is 
representing bytes, and bytes cannot have decimals.

Anyways, as I play along with different configurations I am finding the FB 3 
server VERY responsive interms of multiusers, I am glad the SMP efforts are 
showing very interesting performance, so congrats to all the development 
team!!

As a test I am using IBexpert to compare the data between the FB 2.54 DB and 
the 3.0, it is a 80 GB database, and at the same time I am loading users on the 
FB 3 database, and the performance is almost as good as running without any 
other parallel process, so I am very impressed.


Cheers
Fabian




  - Original Message - 
  From: Dmitry Yemanov dim...@users.sourceforge.net [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 01, 2016 3:49 PM
  Subject: [firebird-support] Re: Issue with Database Cache Size on FB 3.0


  01.06.2016 04:16, Fabian Ernesto Chocron wrote:
  >
  > We are having trouble setting up the database cache size on FB 3.0
  > running on Windows 2008 R2 64 bits with 32 GB ram.
  >
  > The problem we have is we cannot get the server to allocate the ram for
  > the cache as we intend. With FB 2.54 we had the DB cache set very high,
  > close to 1 GB per database, all running in RAM memory. With FB 3.0 we
  > read it can allocate much more RAM to the cache, but it appears the
  > server is allocating very small amount of Ram when the first user
  > connects to the DB, and as we connect more users to the DB the ram
  > consumption increases slowly.

  What memory counters do you look at? Virtual (reserved) memory usage 
  should grow high with the first connect, physical (committed) memory 
  usage should grow together with your load, while pages are being read 
  from disk.

  > On firebird.conf
  >
  > FileSystemCacheThreshold = 0
  > FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB
  > ram.)

  These changes conflict with each other. You disable filesystem cache 
  with the first line and still want to use 16GB with the second line.

  > On databases.conf
  >
  > MyTestDB = c:\Temp\MyDb.fdb
  > {
  > DefaultDbCachePages = 458752
  > }

  First of all, I'd suggest to reset FileSystemCacheSize back to zero and 
  set FileSystemCacheThreshold to something higher than 458752 pages (e.g. 
  50). Then re-test your memory usage and only then start tweaking the 
  filesystem cache.

  BTW, don't you occasionally have page cache override at the database 
  level? What is gstat -h output?


  Dmitry




  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Vlad

Thank you, english is not my first language, I interpreted the sentence as "go 
calculate the total amount of ram, then set the value as a number within such 
percentage of your ram...", but thanks for the clarification. Perhaps to avoid 
other users making the same mistake each configurable variable on firebird .cfg 
files could have a "domain" explaining what type of variable it is, like 
"Domain = From 0 to 100, up to 2 decimals allowed, decimal separator is DOT". 
Furthermore, in this case I first understood what was required was actually a 
percentage number, but then changed my mind as it would be too hard for most 
users to think that way, as percentage mathematically speaking the way to 
represent 20% is actually 0.2, so it goes from 0 to 1.

I hope this helps other users,

Cheers,
Fabian

  - Original Message - 
  From: hv...@users.sourceforge.net [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 01, 2016 9:32 PM
  Subject: [firebird-support] Re: Issue with Database Cache Size on FB 3.0




  ---In firebird-support@yahoogroups.com,  wrote :



  ...

  > The setting we are playing with are:
  > 
  > On firebird.conf
  > 
  > FileSystemCacheThreshold = 0
  > FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB 
  > ram.)

Does you ever read comments describing this setting ? I can do it for you : 

  "measured in % of total physical RAM". 


  I.e. any value not in range [0..100] is wrong by definition. Read 
firebird.log, it should contain 
  error message:

  "Incorrect FileSystemCacheSize setting %d. Using default (30 percent)."

  If you really want to limit file system cache by 50% of avalable RAM - set 
FileSystemCacheSize 
  value to 50 (and restart Firebird), no need to calc in bytes...

  Regards,
  Vlad





  

Re: [firebird-support] Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Dimitry

That is my concern, on FB 2.5 I was able to see the memory allocated as soon as 
the first connection is granted to the DB, but on FB 3 I am not seeing almost 
any memory been allocated what makes me believe that something is not right. 
However as more users connect and use the DB, the memory usage / allocation is 
growing, so something has either changed, or I have not configured properly the 
cfg files.

Regards,
Fabian

  - Original Message - 
  From: Dmitry Yemanov dim...@users.sourceforge.net [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 01, 2016 3:38 PM
  Subject: Re: [firebird-support] Issue with Database Cache Size on FB 3.0


  01.06.2016 05:30, Alexey Kovyazin wrote:
  >
  > Firebird caches only actually used pages.

  But it allocates all the configured memory immediately, so the memory 
  usage (virtual/reserved bytes) should be visible anyway.


  Dmitry




  
  Posted by: Dmitry Yemanov 
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Issue with Database Cache Size on FB 3.0

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Alexey

Thank you, I've noticed as the App was being used the DB cache was eating more 
memory, so I guess the config may be ok, it is a matter of loading more users 
online and check the memory

Cheers
Fabian

  - Original Message - 
  From: Alexey Kovyazin (ak) a...@ib-aid.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Wednesday, June 01, 2016 12:30 PM
  Subject: Re: [firebird-support] Issue with Database Cache Size on FB 3.0





  Hi Fabian,

  Firebird caches only actually used pages.

  The small cache means that your application touches the small part of the 
database.


  --
  Regards,
  Alexey Kovyazin
  IBSurgeon
  PS It is difficult and wrong to give you any direct advice without all 
details in hands - it can lead to worse performance than with default 
parameters.


  среда, 01 июня 2016г., 04:16 +03:00 от Fabian Ernesto Chocron 
fabia...@itbizolutions.com.au [firebird-support] 
firebird-support@yahoogroups.com:


  
Hi All

We are having trouble setting up the database cache size on FB 3.0 
running on Windows 2008 R2 64 bits with 32 GB ram.

The problem we have is we cannot get the server to allocate the ram for 
the cache as we intend. With FB 2.54 we had the DB cache set very high, 
close to 1 GB per database, all running in RAM memory. With FB 3.0 we 
read it can allocate much more RAM to the cache, but it appears the 
server is allocating very small amount of Ram when the first user 
connects to the DB, and as we connect more users to the DB the ram 
consumption increases slowly. The setting we are playing with are:

On firebird.conf

FileSystemCacheThreshold = 0
FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB 
ram.)

On databases.conf

MyTestDB = c:\Temp\MyDb.fdb
{
DefaultDbCachePages = 458752
}

Any ideas what could be wrong? Or what settings would give us maximum 
RAM usage for the DB cache (we dont want file system cache meaning HDD 
cache, we want to have the DB in RAM for the purpose of reading the DB)

Cheers,
Fabian







  

[firebird-support] Issue with Database Cache Size on FB 3.0

[Fabian Ernesto Chocron fabia...@itbizolutions.com.au [firebird-support]]

Hi All

We are having trouble setting up the database cache size on FB 3.0 
running on Windows 2008 R2 64 bits with 32 GB ram.

The problem we have is we cannot get the server to allocate the ram for 
the cache as we intend. With FB 2.54 we had the DB cache set very high, 
close to 1 GB per database, all running in RAM memory. With FB 3.0 we 
read it can allocate much more RAM to the cache, but it appears the 
server is allocating very small amount of Ram when the first user 
connects to the DB, and as we connect more users to the DB the ram 
consumption increases slowly. The setting we are playing with are:

On firebird.conf

FileSystemCacheThreshold = 0
FileSystemCacheSize = 17179869184 (this is 16 GB - the server has 32 GB 
ram.)


On databases.conf

MyTestDB = c:\Temp\MyDb.fdb
{
DefaultDbCachePages = 458752
}

Any ideas what could be wrong? Or what settings would give us maximum 
RAM usage for the DB cache (we dont want file system cache meaning HDD 
cache, we want to have the DB in RAM for the purpose of reading the DB)

Cheers,
Fabian

 

[firebird-support] Risks - Experience and doubts regarding upgrading to FB 3.0 from 2.54?

[Fabian Ernesto Chocron fabia...@itbizolutions.com.au [firebird-support]]

Hi Again

We are about to upgrade from Windows 2008 FB 2.54 64 bits into Windows 
2012 R2 FB 3.0 64 bits, and would appreciate comments from experienced 
DB administrators. We use FB as the main DB, and we access it via 32 bit 
native ODBC (the latest version of Firebird ODBC). 

I would appreciate if someone can give us a list of risks associated 
with the upgrade upfront, I have read other posts where some issues were 
experienced such as having to convert Stored Procedures and triggers and 
views, but I am not sure why would that be, in our case we did a test 
conversion to 3.0 and all SPs and Triggers and Views recompiled with no 
issues, so I am not really sure if we are missing something or in our 
case the problem does not apply because we have always used standard SQL 
92 when writting SPs and Triggers.

Also any upfront advise would be highly appreciated.

>From our point of view what we are after with the upgrade is:

1) Get parallel requests to be assigned to different CPUs, improving 
performance when a big process is being executed by another user. 

2) Be able to execute backups excluding some tables ( we read this is a 
new feature)

3) Be able to use Nbackup very often, like every 20 minutes, without 
killing the performance of the database, as it would run on an isolated 
processor, we understand the HDD would still be under the pump, but we 
estimate other system users will not be as affected as on FB 2.54 
because of the SMP feature.

4) DB encryption, we are planning to use this feature

5) Communications encryption, we are also planning to use this feature.

6) More DB pages into ram cache under 64 bit server, we would like to 
dedicate 96 GB ram to the DB cache, under FB 2.54 the limit was 131071 
DB pages at 16KB each page, meaning 2GB ram per database.


Any comments are highly appreciated!

Cheers
Fabian




 

Re: [firebird-support] Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Dimitry

I just tested it and seems pretty good, thanks for the hint

Cheers,
Fabian

  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Friday, May 27, 2016 10:12 PM
  Subject: Re: [firebird-support] Is Firebird 3 ready for Production?


  24.05.2016 18:38, Fabian Ernesto Chocron fabia...@itbizolutions.com.au 
[firebird-support] 
  wrote:
  > And regarding the replication, any advise on what product may be the right 
choice?

 I would recommend to look at IBPhoenix Replicator.
 http://www.ibphoenix.com/products/software/ibreplicator
 Especially at version 5 which is on pre-release stage now in accompany 
with Avalerion.
 http://www.ibphoenix.com/ibpr_devel/

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Ann

> I might also have mentioned that sharing caches and other internal structures 
> between simultaneous threads is one challenge. 

So the above mentioned changes have been implemented in FB3. And the was 
referred as true SMP in the white papers about FB3. Correct?


> Distributing queries across processors is another and totally different 
> challenge. The Firebird developers were wise (in my opinion) to take the 
> challenges one at a time. 

And the above mentioned task was not implemented in FB3.0, Correct? Do you know 
if it planned to be implemented in FB 4.0?


Don't take me wrong, I am very happy about Firebird's quality and performance, 
I was just expecting the hole SMP issue to be resolved or enhanced in FB 3.0, 
hence my surprise when I did not see the hole CPUs jumping on a query.


Cheers,
Fabian





  - Original Message - 
  From: Ann Harrison aharri...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 5:41 AM
  Subject: Re: [firebird-support] Re: Is Firebird 3 ready for Production?





  On Wed, May 25, 2016 at 2:53 PM, Ann Harrison <aharri...@ibphoenix.com> wrote:

On Wed, May 25, 2016 at 1:11 PM, fabia...@itbizolutions.com.au 
[firebird-support] <firebird-support@yahoogroups.com> wrote:



   Now on the flip side, the performance sucks, it is worst than with FB 
2.54, and when looking at the task manager on windows it appears only one 
processor it doing the job, as if the code was not SMP enabled very 
strange. 


One possibility is that you're testing V3.0 SuperServer single user.  In 
V3.0, Firebird is multi-threaded at the client statement level.  It does not 
decompose queries and schedule the pieces on different processors.  That means 
that a full-table scan runs on only one processor.  Two simultaneous full-table 
scans will run on two processors.  


  I should have continued to say that two full-table scans probably won't be 
any faster in 3.0 than they were in 2.5 because you're measuring disk transfers 
and adding processors doesn't make the disk go faster.  I might also have 
mentioned that sharing caches and other internal structures between 
simultaneous threads is one challenge.  Distributing queries across processors 
is another and totally different challenge.  The Firebird developers were wise 
(in my opinion) to take the challenges one at a time.  


  Good luck,


  Ann 





  

Re: [firebird-support] FB 3 - SMP switched off ?

[fabia...@itbizolutions.com.au [firebird-support]]

Thanks Dimitry

Cheers
Fabian

  - Original Message - 
  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 10:44 PM
  Subject: Re: [firebird-support] FB 3 - SMP switched off ?


  26.05.2016 14:42, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > Do you know if it is planned for Firebird to distribute each client request 
across all
  > CPUs? When?

 Parallel execution of a single request is not planned at all yet.

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Re: [firebird-support] Is Firebird 3 ready for Production?I see, will read it

Cheers
Fabian

  - Original Message - 
  From: 'Carlos H. Cantu' lis...@warmboot.com.br [firebird-support] 
  To: fabia...@itbizolutions.com.au [firebird-support] 
  Sent: Thursday, May 26, 2016 6:01 AM
  Subject: Re: [firebird-support] Is Firebird 3 ready for Production?




  Fabian

  I suggest you to read the release notes carefully. FB 3 can use different 
user authentication plugins. This means that you can have different users with 
the same name, in different plugins.

  There are many other conceptual changes in FB 3 that needs caution. That's 
why I decided to write a Migration Guide to Firebird 3.

  Carlos
  Migration Guide to Firebird 3 eBook - 
http://www.firebirdnews.org/migration-guide-to-firebird-3/ 
  www.firebirdnews.org - www.FireBase.com.br


   


Hi Alexey

Thank you for the comments. So far everything has gone ok, I fixed all 
the fields with ASCII charset at the FB 2.54 DB (changed the domain to charset 
NONE), and now the backup and restore workes fine, with no errors.

I've noticed the new FB3 DB has now two SYSDBA users, one seems to be 
disabled, and the other has the masterkey password even though the install 
asked me for the password, but it appears it ignored my answer, or it used the 
password I entered for the disabled SYSDBA users... no big deal.

Other than that I have not encountered any other issues so far, I will 
recompile all stored procedures and views and triggers shortly, but I don't 
expect any issues there because the DB is operational with the Application and 
no errors have appeared on the APP side using the FB3 DB.

Cheers,
Fabian 




  

Re: [firebird-support] FB 3 - SMP switched off ?

[fabia...@itbizolutions.com.au [firebird-support]]

Dimitry 

I see, I will shut the FB server many requests at once from different instances 
of client APP and see how it performs, I guess this test will show many CPUs 
going high.



Do you know if it is planned for Firebird to distribute each client request 
across all CPUs? When?



Cheers,

Fabian





- Original Message - 

  From: Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 6:01 PM
  Subject: Re: [firebird-support] FB 3 - SMP switched off ?


  25.05.2016 19:34, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > Any ideas why the SMP is not activated?

 Single connection uses only one execution thread.

  -- 
 WBR, SD.


  

  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] Re: Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Ann

I understand, I thought true SMP was referring to assigning each client's 
request to all CPU at once, instead of balacing the clients requests across 
processors. So the current version (FB 3) is probably going to outperform the 
FB 2.54 when there are concurrent requests passed to the server engine because 
each request will go to a different processor, correct? The more concurrent 
requests we have the more we are going to notice the difference, right?

Cheers
Fabian



- Original Message - 
  From: Ann Harrison aharri...@ibphoenix.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 4:53 AM
  Subject: Re: [firebird-support] Re: Is Firebird 3 ready for Production?





  On Wed, May 25, 2016 at 1:11 PM, fabia...@itbizolutions.com.au 
[firebird-support] <firebird-support@yahoogroups.com> wrote:



 Now on the flip side, the performance sucks, it is worst than with FB 
2.54, and when looking at the task manager on windows it appears only one 
processor it doing the job, as if the code was not SMP enabled very 
strange. 


  One possibility is that you're testing V3.0 SuperServer single user.  In 
V3.0, Firebird is multi-threaded at the client statement level.  It does not 
decompose queries and schedule the pieces on different processors.  That means 
that a full-table scan runs on only one processor.  Two simultaneous full-table 
scans will run on two processors.  


  Good luck,


  Ann








  

[firebird-support] FB 3 - SMP switched off ?

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All
 

 First of all CONGRATS to all developers for the 3.0 project, the Firebird DB 
engine has been an excelent tool for us for more than a decade, and we do 
appreciate the efforts and great product the developing team has created!
 

 Specifically to the SMP capable feature, we are not sure what is going on, but 
at least on our server it appears to be switched off, windows task manager 
shows only one processor doing the job, off course we set affinity to zero, so 
at least on the conf file it is correctly setup. Any ideas why the SMP is not 
activated?
 

 We are using a HP DL 380 G5 2 x Xeon x5470, 32 GB ram, Windows 2008 server 64 
bit, FB 3.0.0.32483 64 bits, and the client side is still using the FB 2.54 32 
bit client library and the latest firebird ODBC to connect to the database, 
would that have anything to do with it?
 

 Cheers,
 Fabian
 

Re: [firebird-support] Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Thanks Alexey, will try it and post back with comments


  - Original Message - 
  From: Alexey Kovyazin a...@ib-aid.com [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 6:58 AM
  Subject: Re: [firebird-support] Is Firebird 3 ready for Production?





  Hi Fabian,

  Also I recommend you to try our optimized configuration file for FB3:
  http://ib-aid.com/en/optimized-firebird-configuration/

  Regards,
  Alexey


  
Hi Alexey



Thank you for the comments. So far everything has gone ok, I fixed all the 
fields with ASCII charset at the FB 2.54 DB (changed the domain to charset 
NONE), and now the backup and restore workes fine, with no errors.


I've noticed the new FB3 DB has now two SYSDBA users, one seems to be 
disabled, and the other has the masterkey password even though the install 
asked me for the password, but it appears it ignored my answer, or it used the 
password I entered for the disabled SYSDBA users... no big deal.


Other than that I have not encountered any other issues so far, I will 
recompile all stored procedures and views and triggers shortly, but I don't 
expect any issues there because the DB is operational with the Application and 
no errors have appeared on the APP side using the FB3 DB.


Cheers,
Fabian








  

Re: [firebird-support] Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Alexey 

 Thank you for the comments. So far everything has gone ok, I fixed all the 
fields with ASCII charset at the FB 2.54 DB (changed the domain to charset 
NONE), and now the backup and restore workes fine, with no errors.
 

 I've noticed the new FB3 DB has now two SYSDBA users, one seems to be 
disabled, and the other has the masterkey password even though the install 
asked me for the password, but it appears it ignored my answer, or it used the 
password I entered for the disabled SYSDBA users... no big deal.
 

 Other than that I have not encountered any other issues so far, I will 
recompile all stored procedures and views and triggers shortly, but I don't 
expect any issues there because the DB is operational with the Application and 
no errors have appeared on the APP side using the FB3 DB.
 

 Cheers,
 Fabian
 

Re: [firebird-support] FB 3 issues with String from FB 2.54

[fabia...@itbizolutions.com.au [firebird-support]]

Mark

Are you saying that the filter I placed on FB 2.5 with charset ASCII was 
correct from the theory, but there was a bug that made the filter be ignored by 
the FB engine? and the ugly characters got inside the DB because of the bug?

And for me to be able to move onto FB 3 I had to remove the ASCII filter 
(charset) because otherwise the restore onto 3.0 would fail, so now I have lots 
of records with unwanted characters and cannot put the filter back onto 3.0 
because if I do so I would end up having a table that does not allow such 
characters however the characters are already there, so for intance a user 
trying to change a dot, or a comma on one of those records would get a failed 
message as the DB cannot write back into the engine the same text that is 
alraedy there becuase of the offending character. Is that correct?  How do I 
solve the issue? At the moment I set charset to NONE and it allows garbage in 
and out with no issues, but I don't want the bad characters there..

Cheers,
Fabian

  - Original Message - 
  From: Mark Rotteveel m...@lawinegevaar.nl [firebird-support] 
  To: firebird-support@yahoogroups.com 
  Sent: Thursday, May 26, 2016 4:08 AM
  Subject: Re: [firebird-support] FB 3 issues with String from FB 2.54


  On 23-5-2016 03:45, fabia...@itbizolutions.com.au [firebird-support] wrote:
  > I have been trying to migrate from FB2.54 into FB 3 for a few weeks, and
  > after hitting a string related error for some time i have got to the
  > point where I do understand the issue, but I don't know how to solve it.
  > The issue is pretty simple, the FB 2.54 DB contains a few characters
  > that are not allowed into the FB 3 database, one example of a character
  > causing an error during the restore was "Mcgarrity’s" (note the ’) as it
  > appears to be outside the scope of the FB3 string domain, I have trying
  > creating a new FB3 DB with many different charsets but none works. The
  > other string causing issues is for example "΢ÈíÑźÚ", I have many
  > records with this type of strings because the DB contains raw emails
  > received by the system, stored into Varchars, and apparently some emails
  > contain very weird characters, all were handled by FB2.54 but FB3
  > rejects the records. I have been able to isolate all recrods with issues
  > using IBExpert's table data comparer function, as it created a script
  > with all recrods from all tables from FB2.54 and when running the script
  > against FB3.0 it singles out all the offending records.
  >
  > Can anyone advise what options I have available to force FB3.0 to accept
  > any stuff into string fields?

  In your other e-mail you indicate you solved this by changing the 
  character set from ASCII to NONE. The fact it worked before was a bug, 
  see http://tracker.firebirdsql.org/browse/CORE-3416. ASCII only supports 
  characters 0-127, characters outside that range are 'extended ascii', eg 
  one of the other singly by character sets like WIN1252 or ISO8859_1. The 
  characters shown (΢ÈíÑÅºÚ and ’) are all outside the ASCII range.

  The last (’) is particularly nasty, because it should have been a ' 
  (u+0027 Apostrophe, ascii 39), instead u+2019 Right single quotation 
  mark (character 146 in Windows-1252) was used.

  Given the context of e-mails either NONE or OCTETS is the only real 
  option, as e-mails can have multiple parts with each their own character 
  set, and can also have binary parts (although usually those are encoded 
  with something like base64).

  Mark
  -- 
  Mark Rotteveel


  
  Posted by: Mark Rotteveel <m...@lawinegevaar.nl>
  

  ++

  Visit http://www.firebirdsql.org and click the Documentation item
  on the main (top) menu.  Try FAQ and other links from the left-side menu 
there.

  Also search the knowledgebases at 
http://www.ibphoenix.com/resources/documents/ 

  ++
  

  Yahoo Groups Links

Re: [firebird-support] FB 3 issues with String from FB 2.54

[fabia...@itbizolutions.com.au [firebird-support]]

Hi Kjell 

 We had a FB 2.54 DB with a few varchars defined as charset ASCII, the 
intention was to stop users from entering unreadable characters, so the charset 
was there to attempt to filter garbage, at least that was the idea), anyways we 
converted all domains from charset ASCII to NONE, and then backup and restore 
onto FB 3.0, and it all worked fine. So I guess we were lucky, and the 
migration process was not too painfull. On the flip side, the performance we 
are getting on 3.0 is worst than on 2.54 it seems the SMP is not active as 
only one processor is doing all the job, and CPU affinity is set to zero, so we 
are not sure what's going on.
 

 Cheers,
 Fabian
 

 

[firebird-support] Re: Is Firebird 3 ready for Production?

[fabia...@itbizolutions.com.au [firebird-support]]

Pete 

 Thanks for your input.
 

 We managed to get the DB upgraded to 3.0 (removed all charset ASCII 
definitions on fields while on 2.54, then backup the DB and restored it on 3.0, 
and it all worked). Now on the flip side, the performance sucks, it is worst 
than with FB 2.54, and when looking at the task manager on windows it appears 
only one processor it doing the job, as if the code was not SMP enabled 
very strange. We will leave 3.0 for testing at the moment, and continue to use 
2.54 in production, in the meantime we will play with 3.0 and try to figure out 
what is it we are doing wrong.
 

 Cheers,
 Fabian
 

[firebird-support] Is Firebird 3 ready for Production?

[Fabian Ernesto Chocron fabia...@itbizolutions.com.au [firebird-support]]

Hi All

We have a multi-user, multi-company, cloud based system running with 
Firebird 2.54 as the main database, with about 100 concurrent connections.  
We are considering migrating to FB 3.0 to take advantaje of the SMP 
architecture, we have a 32 core / processors server, so we are hopping to 
see an increased performance. We also have the need to set up online 
replication. 
Would anyone be able to advise if it it too soon to jump into FB 3.0 for 
production? And if the performance increase would be enough to make the 
upgrade worth considering 3.0 has not been around for long yet, and may be 
"taking too much risk" to move into it so soon? And regarding the 
replication, any advise on what product may be the right choice?

Cheers
Fabian

[firebird-support] Re: ODP: FB 3 issues with String from FB 2.54

[Fabian Ernesto Chocron fabia...@itbizolutions.com.au [firebird-support]]

liviuslivius liviuslivius  poczta.onet.pl [firebird-support] 
 yahoogroups.com> writes:

> 
> 
> 
> 
> 
> 
> hi,
> 
> will be good to know which charsets did you tried. utf8, none, 
octets...
> 
> but remember to connet to db also with this charset settings
> 
> regards,
> Karol Bieniaszewski
>  Oryginalna wiadomość Od: "fabianch  
itbizolutions.com.au [firebird-support]"  
yahoogroups.com> Data: 23.05.2016  03:45  (GMT+01:00) Do: firebird-
support  yahoogroups.com Temat: [firebird-support] FB 3 issues with 
String from FB 2.54  
> 
> 
> 
> 
>   
>   
>   
> Hi All
> 
> I have been trying to migrate from FB2.54 into FB 3 for a few weeks, 
and after hitting a string related error for some time i have got to the 
point where I do understand the issue, but I don't know how to solve it. 
The issue is pretty simple, the FB 2.54 DB contains a few characters 
that are not allowed into the FB 3 database, one example of a character 
causing an error during the restore was "Mcgarrity’s" (note the ’) as it 
appears to be outside the scope of the FB3 string domain, I have trying 
creating a new FB3 DB with many different charsets but none works. The 
other string causing issues is for example "΢ÈíÑźÚ", I have many 
records with this type of strings because the DB contains raw emails 
received by the system, stored into Varchars, and apparently some emails 
contain very weird characters, all were handled by FB2.54 but FB3 
rejects the records. I have been able to isolate all recrods with issues 
using IBExpert's table data comparer function, as it created a script 
with all recrods from all tables from FB2.54 and when running the script 
against FB3.0 it singles out all the offending records. 
> 
> Can anyone advise what options I have available to force FB3.0 to 
accept any stuff into string fields?
> 
> 
> Thanks in advanced
> 
> 
> 
> 
>  
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 


Hi Karol

After playing with the problem for a while I noticed the definition of 
the field was Varchar (8192) charset ASCII collate ASCII, so that was 
probably causing the problem. I changed the domain settings to be 
Charset NONE, Collate NONE, and that fixed the issue!  

Cheers,
Fabian

[firebird-support] FB 3 issues with String from FB 2.54

[fabia...@itbizolutions.com.au [firebird-support]]

Hi All
 

 I have been trying to migrate from FB2.54 into FB 3 for a few weeks, and after 
hitting a string related error for some time i have got to the point where I do 
understand the issue, but I don't know how to solve it. The issue is pretty 
simple, the FB 2.54 DB contains a few characters that are not allowed into the 
FB 3 database, one example of a character causing an error during the restore 
was "Mcgarrity’s" (note the ’) as it appears to be outside the scope of the FB3 
string domain, I have trying creating a new FB3 DB with many different charsets 
but none works. The other string causing issues is for example "΢ÈíÑźÚ", I 
have many records with this type of strings because the DB contains raw emails 
received by the system, stored into Varchars, and apparently some emails 
contain very weird characters, all were handled by FB2.54 but FB3 rejects the 
records. I have been able to isolate all recrods with issues using IBExpert's 
table data comparer function, as it created a script with all recrods from all 
tables from FB2.54 and when running the script against FB3.0 it singles out all 
the offending records. 
 

 Can anyone advise what options I have available to force FB3.0 to accept any 
stuff into string fields?
 

 
 Thanks in advanced