RE: [firebird-support] Restoring Backup

['Leyne, Sean' s...@broadviewsoftware.com [firebird-support]]

Dimitry,

> > When it is restoring data, it
> > fills a data page and goes on to the next one.  A large cache will
> > fill with pages that will not be referenced again until the indexes
> > are built.  To build indexes, Firebird reads records and sorts by
> > keys.  That might suggest that keeping millions of pages in cache
> > would improve performance by eliminating disk reads.
> 
>Putting restored data to data pages and sorting streams at the same time
> also could eliminate reads.

While I agree, that would be ideal.

It, however, would require the engine to have special/specific functions to 
support gbak restores/operations.

Currently, gbak uses standard/public data access/write functions which do not 
have intelligence to process the data writes and build the sorting streams in 
one operation.

An intermediate step/compromise would be to implement a multi-index rebuild, 
like Interbase v6.5(?) introduced.  This would have all data writes occur as 
they do now, but the all the indexes for a table would be built via a single 
read pass of the table (versus the separate read pass for each index as is the 
case today).

The first step, would be to ensure that gbak would rebuilds indexes in an 
ordered approach -- ie. rebuild indexes for Table A before proceeding to 
indexes for Table B.  I have seen cases where indexes are rebuilt in an 
"intermixed" fashion.  The ordered approach would provide at least some hope 
that the OS cache could have the data pages cached from the 1st index rebuild 
when rebuilding subsequent indexes.


Sean

Re: [firebird-support] Restoring Backup

[Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]]

24.02.2016 21:30, Hugo Eyng hugoe...@msn.com [firebird-support] wrote:
> I am not sure that is the best way

   Yes, gfix is better. At least faster.

-- 
   WBR, SD.






++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Restoring Backup

[Hugo Eyng hugoe...@msn.com [firebird-support]]

Hello Ann.

Thank you very much.

The two hundred pages are not intended to be used for backup or restore 
purposes.


In fact, I use the switche '-BU 20', in gbak -REP -c,  to set the 
database cache pages option. This way I don´t need change

the DefaultDbCachePages switche in the firebird.conf file.

I am not sure that is the best way, but I did some tests and it worked fine.

Hugo


Em 24/02/2016 16:37, Ann Harrison aharri...@ibphoenix.com 
[firebird-support] escreveu:
On Wed, Feb 24, 2016 at 12:01 PM, Hugo Eyng hugoe...@msn.com 
 [firebird-support] 
> wrote:


Referring to the buffers requested for gbak:



Why 12? Why not? :)


There's a cost to managing a large buffer cache. Firebird will use the 
entire

cache even if there's no particular reason to do so.

A gbak backup needs a few dozen pages for system table stuff that's
referenced frequently and for each table backed up another few pages
that help locate data.  The data pages are read once and then are of no
more interest.   So the default cache size is fine for a backup.   Two
hundred pages is probably enough to backup almost any database.  If
you use a huge cache, it will be filled with useless data pages.

A gbak restore needs the same few dozen pages of system tables and
some pages for internal table management.  When it is restoring data, it
fills a data page and goes on to the next one.  A large cache will 
fill with

pages that will not be referenced again until the indexes are built.  To
build indexes, Firebird reads records and sorts by keys.  That might
suggest that keeping millions of pages in cache would improve performance
by eliminating disk reads.   However, unless you've got a huge amount
of memory, those data pages will reduce the amount of memory available
for the sort.  The sort algorithm produces chunks of sorted key/record id
pairs which are subsequently merged.  The more chunks that can be kept
in memory, the faster the sort.   If you've used all your machine's 
 memory

for the cache, the sort will be slow and may start paging.

Good luck,

Ann





--


Atenciosamente,

Hugo Eyng

Re: [firebird-support] Restoring Backup

[Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]]

24.02.2016 20:37, Ann Harrison aharri...@ibphoenix.com [firebird-support] wrote:
> When it is restoring data, it
> fills a data page and goes on to the next one.  A large cache will fill with
> pages that will not be referenced again until the indexes are built.  To
> build indexes, Firebird reads records and sorts by keys.  That might
> suggest that keeping millions of pages in cache would improve performance
> by eliminating disk reads.

   Putting restored data to data pages and sorting streams at the same time 
also could 
eliminate reads.

-- 
   WBR, SD.






++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Restoring Backup

[Ann Harrison aharri...@ibphoenix.com [firebird-support]]

On Wed, Feb 24, 2016 at 12:01 PM, Hugo Eyng hugoe...@msn.com
[firebird-support]  wrote:

Referring to the buffers requested for gbak:

>
>
> Why 12? Why not? :)
>

There's a cost to managing a large buffer cache.  Firebird will use the
entire
cache even if there's no particular reason to do so.

A gbak backup needs a few dozen pages for system table stuff that's
referenced frequently and for each table backed up another few pages
that help locate data.  The data pages are read once and then are of no
more interest.   So the default cache size is fine for a backup.   Two
hundred pages is probably enough to backup almost any database.  If
you use a huge cache, it will be filled with useless data pages.

A gbak restore needs the same few dozen pages of system tables and
some pages for internal table management.  When it is restoring data, it
fills a data page and goes on to the next one.  A large cache will fill with
pages that will not be referenced again until the indexes are built.  To
build indexes, Firebird reads records and sorts by keys.  That might
suggest that keeping millions of pages in cache would improve performance
by eliminating disk reads.   However, unless you've got a huge amount
of memory, those data pages will reduce the amount of memory available
for the sort.  The sort algorithm produces chunks of sorted key/record id
pairs which are subsequently merged.  The more chunks that can be kept
in memory, the faster the sort.   If you've used all your machine's  memory
for the cache, the sort will be slow and may start paging.

Good luck,

Ann

Re: [firebird-support] Restoring Backup

[Hugo Eyng hugoe...@msn.com [firebird-support]]

Hello HB.

It happens with 2.5.5 32 bit. I believe that the limitation is beacuse 
20 x 16384 (page size) = 3GB greater than the avaliable amount of 
memory or greater than the 32bit engine is capable to manage.


I am not sure.

Why 12? Why not? :)


Hugo

Em 24/02/2016 00:09, Helen Borrie hele...@iinet.net.au 
[firebird-support] escreveu:


Wednesday, February 24, 2016, 12:22:31 PM, Hugo Eyng wrote:

> Hello.

> I tried to restore a backup using the -BU 20 switch and gbak
> returned an error message: invalid parameter.

That is not a gbak message. If you are using a third-party tool, it
is likely to be some limit imposed by that - or a bug?

> Then I change the value 20 by 12 and gbak worked.

> Is it a limitation?

With Firebird 2.5.5 64-bit, I am able to execute this with no error:

gbak -c -bu 80 h:\data\backups\backup20160131.fbk
h:\data\restores\restored20160224.fdb -user sysdba -password xyxyxyxy

(Although I cannot think why I would want such a huge number of pages
in the cache, not even 12!!)

HB




--


Atenciosamente,

Hugo Eyng

Re: [firebird-support] Restoring Backup

[Helen Borrie hele...@iinet.net.au [firebird-support]]

Wednesday, February 24, 2016, 12:22:31 PM, Hugo Eyng wrote:

> Hello.

> I tried to restore a backup using the -BU 20 switch and gbak 
> returned an error message: invalid parameter.

That is not a gbak message.  If you are using a third-party tool, it
is likely to be some limit imposed by that - or a bug?

> Then I change the value 20 by 12 and gbak worked.

> Is it a limitation?

With Firebird 2.5.5 64-bit, I am able to execute this with no error:

gbak -c -bu 80 h:\data\backups\backup20160131.fbk
h:\data\restores\restored20160224.fdb -user sysdba -password xyxyxyxy

(Although I cannot think why I would want such a huge number of pages
in the cache, not even 12!!)

HB

Re: [firebird-support] Restoring backup from 2.5 to 2.1

[Milan Babuskov]

Aldo Caruso wrote:
When I try to restore a backup done in a 2.5 firebird server to a 2.1 
 firebird server, I get the following error message:
 
 gbak:transportable backup -- data in XDR format
 gbak: ERROR:Expected backup version 1..8.  Found 9
 gbak:Exiting before completion due to errors
 
Isn't it possible to restore a backup database from a 2.5 to a 2.1 
 firebird server ?

It is not possible since 2.5 introduces incompatible changes. If you 
need to move from 2.5 back to 2.1, the only way would be to copy data 
from one database into another. I tried this in the past and met with 
another problem: which version to use to access the databases for copy. 
IIRC, 2.5 did alter the database structure so I could not use it with 
2.1. The final solution was to start up the source database on 2.5 
server and prepare a clean database on a different machine with 2.1 
server. And then I copied the data using FBCopy tool.

To prepare a clean database, you can use something like isql -x to 
dump the database schema from 2.5, and then run that script on an empty 
database with Firebird 2.1.

Good luck.

-- 
Milan Babuskov

==
The easiest way to import XML, CSV
and textual files into Firebird:
http://www.guacosoft.com/xmlwizard
==

Re: [firebird-support] Restoring backup from 2.5 to 2.1

[Thomas Steinmaurer]

 When I try to restore a backup done in a 2.5 firebird server to a 2.1
 firebird server, I get the following error message:

 gbak:transportable backup -- data in XDR format
 gbak: ERROR:Expected backup version 1..8.  Found 9
 gbak:Exiting before completion due to errors

 Isn't it possible to restore a backup database from a 2.5 to a 2.1
 firebird server ?

Are you using gbak from 2.1 against a 2.5 server and I hope you don't 
use any new/specific things from 2.5?

-- 
With regards,
Thomas Steinmaurer
http://www.upscene.com/

Re: [firebird-support] Restoring backup from 2.5 to 2.1

[Alexandre Benson Smith]

Did you tried to use 2.1 gbak against 2.5 server to produce the gbk ?

see you !

Em 4/7/2012 15:59, Aldo Caruso escreveu:
 To restore, I am using gbak 2.1 against a 2.1 server, but reading a
 *.fbk backup file that was created using a 2.5 server.
 AFAIK I'm not using specific things from 2.5

 Thanks,
 Aldo

 El 04/07/12 13:56, Thomas Steinmaurer escribió:
 When I try to restore a backup done in a 2.5 firebird server to a 2.1
 firebird server, I get the following error message:

 gbak:transportable backup -- data in XDR format
 gbak: ERROR:Expected backup version 1..8. Found 9
 gbak:Exiting before completion due to errors

 Isn't it possible to restore a backup database from a 2.5 to a 2.1
 firebird server ?
 Are you using gbak from 2.1 against a 2.5 server and I hope you don't
 use any new/specific things from 2.5?

 -- 
 With regards,
 Thomas Steinmaurer
 http://www.upscene.com/





 [Non-text portions of this message have been removed]



 

 ++

 Visit http://www.firebirdsql.org and click the Resources item
 on the main (top) menu.  Try Knowledgebase and FAQ links !

 Also search the knowledgebases at http://www.ibphoenix.com

 ++
 Yahoo! Groups Links

Re: [firebird-support] Restoring backup from 2.5 to 2.1

[Aldo Caruso]

Good idea, but it is unfeasible for me because the server is remote on 
Internet (with very high latency), so I have to create the back ups on 
the server, download the file and apply it to my local machine (which 
has firebird 2.1)

Thanks,
Aldo

El 04/07/12 16:04, Alexandre Benson Smith escribió:

 Did you tried to use 2.1 gbak against 2.5 server to produce the gbk ?

 see you !

 Em 4/7/2012 15:59, Aldo Caruso escreveu:
  To restore, I am using gbak 2.1 against a 2.1 server, but reading a
  *.fbk backup file that was created using a 2.5 server.
  AFAIK I'm not using specific things from 2.5
 
  Thanks,
  Aldo
 
  El 04/07/12 13:56, Thomas Steinmaurer escribió:
  When I try to restore a backup done in a 2.5 firebird server to a 2.1
  firebird server, I get the following error message:
 
  gbak:transportable backup -- data in XDR format
  gbak: ERROR:Expected backup version 1..8. Found 9
  gbak:Exiting before completion due to errors
 
  Isn't it possible to restore a backup database from a 2.5 to a 2.1
  firebird server ?
  Are you using gbak from 2.1 against a 2.5 server and I hope you don't
  use any new/specific things from 2.5?
 
  --
  With regards,
  Thomas Steinmaurer
  http://www.upscene.com/
 
 
 
 
 
  [Non-text portions of this message have been removed]
 
 
 
  
 
  ++
 
  Visit http://www.firebirdsql.org and click the Resources item
  on the main (top) menu. Try Knowledgebase and FAQ links !
 
  Also search the knowledgebases at http://www.ibphoenix.com
 
  ++
  Yahoo! Groups Links
 
 
 
 

 




[Non-text portions of this message have been removed]

Re: [firebird-support] Restoring backup from 2.5 to 2.1

[Helen Borrie]

El 04/07/12 16:04, Alexandre Benson Smith escribió:

 Did you tried to use 2.1 gbak against 2.5 server to produce the gbk ?

At 07:11 AM 5/07/2012, Aldo Caruso wrote:
Good idea, but it is unfeasible for me because the server is remote on 
Internet (with very high latency), so I have to create the back ups on 
the server, download the file and apply it to my local machine (which 
has firebird 2.1)

Then it cannot possibly work.  Firebird does *not* support back-porting 
databases.  It was found to be possible (sometimes) to backup and restore 
between 2.1 and 2.0, only if the v.2.0 gbak was running in the v.2.1 server to 
make the backup.

If you have a copy of The Firebird Book Second Edition, you can read up on this 
in Chapter 5, Migration Issues.

./heLen

RE: [firebird-support] Restoring backup from 2.5 to 2.1 - Email found in subject

[Leyne, Sean]

 Good idea, but it is unfeasible for me because the server is remote on
 Internet (with very high latency), so I have to create the back ups on the
 server, download the file and apply it to my local machine (which has firebird
 2.1)

You could copy the v2.1 GBAK executable onto the remote machine, into a 
separate folder from the existing FB install.  

Then create a local backup using that executable.  That would create a v2.1 
backup of the database.


Sean

Re: [firebird-support] Restoring backup from 2.5 to 2.1 - Email found in subject

[Aldo Caruso]

Thanks for your idea.

Aldo

El 04/07/12 16:49, Leyne, Sean escribió:

  Good idea, but it is unfeasible for me because the server is remote on
  Internet (with very high latency), so I have to create the back ups 
 on the
  server, download the file and apply it to my local machine (which 
 has firebird
  2.1)

 You could copy the v2.1 GBAK executable onto the remote machine, into 
 a separate folder from the existing FB install.

 Then create a local backup using that executable. That would create a 
 v2.1 backup of the database.

 Sean

 




[Non-text portions of this message have been removed]