It seems Oracle has a related issue with their SCN values:
http://www.infoworld.com/d/security/revealed-fundamental-oracle-flaw-184163?source=IFWNLE_nlt_daily_2012-01-17
--
Keep Your Developer Skills Current with LearnDe
Excellent Ann, you had explained it very clearly (as usual).
Greetings.
Walter.
On Mon, Jan 16, 2012 at 12:51 PM, Ann Harrison wrote:
> On Mon, Jan 16, 2012 at 8:24 AM, u.maso...@libero.it
> wrote:
> >
> > I remember that in 8086 era, when addressable memory became cheaper and
> bigger
> >
On Mon, Jan 16, 2012 at 8:24 AM, u.maso...@libero.it
wrote:
>
> I remember that in 8086 era, when addressable memory became cheaper and bigger
> than dimension reserved in opcode space (24 bits IIRC), hardware architecture
> changed a bit, implementing relocation tables, where low level bits where
Hi all,
i've read many comments on this argument, but seems that ideas to solve this
problem (if it is a true real problem in near future) are not many.
I remember that in 8086 era, when addressable memory became cheaper and bigger
than dimension reserved in opcode space (24 bits IIRC), hardwar
On 01/13/12 23:57, Yi Lu wrote:
> Hi All,
> When we are trying to change structure “header page” in file ods.h
> (For example hdr_next_transaction from SLONG to SINT64)
> We have the following error
> C:\Program Files\Firebird\Firebird_2_5\bin>gbak -c -r -user sysdba
> -password masterkey -se
Hi All,
When we are trying to change structure “header page” in file ods.h
(For example hdr_next_transaction from SLONG to SINT64)
We have the following error
C:\Program Files\Firebird\Firebird_2_5\bin>gbak -c -r -user sysdba
-password masterkey -service localhost:service_mgr "C:\Program
File
> How do I debug if firebird server running with code changes that we made?
Are you asking how to debug applications ? Using debugger ;)
> I am running fbserver.exe with "-a -p 3050 -database
> "localhost:..\myDatabase.fdb"". Firebird tray icon is showing me that
> nothing has been attached
How do I debug if firebird server running with code changes that we made? I
am running fbserver.exe with "-a -p 3050 -database
"localhost:..\myDatabase.fdb"". Firebird tray icon is showing me that
nothing has been attached and nothing is happening outside of the windows
message loop. Also, when par
On Tue, Jan 3, 2012 at 3:06 PM, Dmitry Yemanov wrote:
>
> In the past, I liked the idea to wrap the txn IDs, but now I'm more and
> more keen to consider other solutions instead.
>
Completely agree - including having changed my mind. I agree with
Dimitry S. that his replicate/backup/restore/rev
Den 2012-01-03 21:11 skrev Dimitry Sibiryakov såhär:
> 03.01.2012 21:04, Kjell Rilbe wrote:
>> What's needed is, in principle, a task that reads through ALL record
>> versions, and for each one with transaction id< OIT, change it to OIT -
>> 1. When it has done that for the entire database, it ca
04.01.2012 0:04, Kjell Rilbe wrote:
> To make this a bit less work intensive, would it be possible and a good
> idea to mark each database page with the lowest transaction id in use on
> that page? In that case, the task could skip all pages where this value
> is>= OIT - 1.
It could avoid page wr
03.01.2012 21:04, Kjell Rilbe wrote:
> What's needed is, in principle, a task that reads through ALL record
> versions, and for each one with transaction id< OIT, change it to OIT -
> 1. When it has done that for the entire database, it can move the max
> useable transaction id to OIT - 2.
It
Thomas,
> While the restore process is pretty much I/O bound,
While that is true for desktop PCs with HDDs (not SSDs) or server without a
cached RAID controllers, but it is certainly not true as a blanket statement.
There is plenty of room for more throughput.
> Another option might be to res
03.01.2012 23:49, Kjell Rilbe wrote:
> As far as I understand, with very limited knowledge about the ods, each
> version of each record contains the id of the transaction that created
> that record version. So, if transaction with id 5 created the record
> version it will say "5" in there, always,
Den 2012-01-01 21:31 skrev Kjell Rilbe såhär:
> 3. Stay at 32 bit id:s but somehow "compact" all id:s older than OIT(?)
> into OIT-1. Allow id:s to wrap around.
> Pros:
> - No extra disk space.
> - Probably rather simple code changes for id usage.
> Cons:
> - May be difficult to find an effective w
03.01.2012 20:37, Leyne, Sean wrote:
> As for the FKs, I see the tool as being a some which needs to have the
> maximum performance.
If you are going to move whole database at once - yes. Fortunately, in most
cases it is
not necessary.
--
SY, SD.
---
>>> The problem is not downtime is how much downtime. Backup and restore is
>>> so much downtime.
>>
>> There are a couple of possible solutions which would reduce the downtime;
>> - a new backup/restore tool which would use multiple readers/writers to
>> minimize execution time,
>
> Here we're ta
Den 2012-01-03 20:14 skrev Woody såhär:
> Maybe I'm a little dense, (probably :), but doesn't FB already know what the
> oldest interesting transaction id is? Why couldn't transaction numbers be
> allowed to wrap back around up to that point? As long as transactions are
> committed at some point, t
Woody,
> Maybe I'm a little dense, (probably :), but doesn't FB already know what the
> oldest interesting transaction id is? Why couldn't transaction numbers be
> allowed to wrap back around up to that point? As long as transactions are
> committed at some point, the oldest transaction would move
> 03.01.2012 18:51, Leyne, Sean wrote:
> > - a "data port" utility which would allow for data to be ported from a live
> database to a new database while live is active but would need a finalization
> step where the live database is shutdown to apply the final data changes and
> add FK constraint
Ann,
> > - a new backup/restore tool which would use multiple readers/writers
> > to minimize execution time,
>
> Here we're talking about a logical backup that can be used to restart
> transaction numbers. Record numbers are based loosely on record storage
> location. Since a logical backup/re
From: "Ann Harrison"
> Dimitry,
>
>> And exactly this utility is called "replicator". If made right, it
>> doesn't need FK
>> deactivation and can do "finalization step" when new database is already
>> in use.
>> Aren't you tired inventing a wheel?..
>
> Different vehicles need different whe
03.01.2012 19:44, Ann Harrison wrote:
> I'm not sure that either replication or the
> mechanism Sean is proposing (similar to either the start of a shadow
> database or nbackup) can solve the overflowing transaction id problem.
Simply: let's say we have two synchronous database. One is primary
Dimitry,
>> - a "data port" utility which would allow for data to be ported from a live
>> database to a new database while live is active but would need a
>> finalization step where the live database is shutdown to apply the final
>> data changes and add FK constraints.
>
> And exactly this
Sean,
>
>> The problem is not downtime is how much downtime. Backup and restore is
>> so much downtime.
>
> There are a couple of possible solutions which would reduce the downtime;
> - a new backup/restore tool which would use multiple readers/writers to
> minimize execution time,
Here we're tal
03.01.2012 18:51, Leyne, Sean wrote:
> - a "data port" utility which would allow for data to be ported from a live
> database to a new database while live is active but would need a finalization
> step where the live database is shutdown to apply the final data changes and
> add FK constraints.
Jesus
> > Single server without downtime is a myth anyway.
> The problem is not downtime is how much downtime. Backup and restore is
> so much downtime.
If that is the case, how much downtime is acceptable?
There are a couple of possible solutions which would reduce the downtime;
- a new backu
We have read that negative transaction IDs are used to indicate an invalid
transaction. Thus, we have decided to continue with the signed approach to
prevent logic problems.
--
View this message in context:
http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-solution-tp4230210p425
Yi Lu [2012-01-03 02:02] :
> Also, we would be willing to contribute to the Firebird foundation in order
> to help us to develop this feature as we can't wait until Firebird 3.0.
Please contact me directly, to see what are your need and willing so we
can find the best solution
--
Philippe Makow
>
> 1. hdr_oldest_transaction, hdr)oldest_active, hdr_next_transaction on
> header_page structure should be SINT64 on file ods.h.
> 2. fld_trans_id should have dtype_int64 and size should be sizeof(SINT64) on
Why signed?
Ann
>
--
We are trying to take the approach #1, involving ODS change and data type
changes for transaction id to use signed 64 bit integer. We just start
looking at the code from today and following changes should be applied on
code changes so far,
1. hdr_oldest_transaction, hdr)oldest_active, hdr_next_tran
Dimitry Sibiryakov wrote:
>> In my own cases, ANY downtime during office hours is unacceptable, so any
>> > failure that brings the whole system down would result in penalties!
>> > Organised
>> > downtime is possible on many sites, but some sites are running 24/7, So we
>> > maintain data in a
On Mon, Jan 2, 2012 at 2:32 PM, Yi Lu wrote:
> Approach 1 seems to be least risky. Disk space should not be a big issue with
> today's hardware.
>
The problem is not disk space, but locality of reference. With small
records, adding
four bytes could decrease the number of rows per page by 10-15%,
We turn off the automatical sweep option, and manually perform sweep (as well
as recompute index selectivity) once a day at low hour (2am). The
performance is actually not impacted so much.
--
View this message in context:
http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-soluti
Approach 1 seems to be least risky. Disk space should not be a big issue with
today's hardware.
--
View this message in context:
http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-solution-tp4230210p4254287.html
Sent from the firebird-devel mailing list archive at Nabble.com.
--
02.01.2012 17:40, Lester Caine wrote:
> In my own cases, ANY downtime during office hours is unacceptable, so any
> failure that brings the whole system down would result in penalties! Organised
> downtime is possible on many sites, but some sites are running 24/7, So we
> maintain data in a manor
Dimitry Sibiryakov wrote:
>> Could you stop with the absurd comparisons? One is normal maintenance
>> > and the other is (extreme) disaster recovery which are in no way
>> > comparable.
> But downtime is downtime. Customers don't care about its reason, do
> they?..
In my own cases, ANY down
On 2-1-2012 16:43, Dimitry Sibiryakov wrote:
> 02.01.2012 16:10, Mark Rotteveel wrote:
>> Could you stop with the absurd comparisons? One is normal maintenance
>> and the other is (extreme) disaster recovery which are in no way comparable.
>
> But downtime is downtime. Customers don't care abou
Mark Rotteveel wrote:
>> 02.01.2012 14:46, Jesus Garcia wrote:
>>> >> he problem is not downtime is how much downtime. Backup and restore is
>>> >> so much downtime.
>> >
>> > Not more downtime that simple restore after complete server
>> > destruction by flood or
>> > plane crash.
> Coul
02.01.2012 16:10, Mark Rotteveel wrote:
> Could you stop with the absurd comparisons? One is normal maintenance
> and the other is (extreme) disaster recovery which are in no way comparable.
But downtime is downtime. Customers don't care about its reason, do they?..
--
SY, SD.
---
On 2-1-2012 15:50, Dimitry Sibiryakov wrote:
> 02.01.2012 14:46, Jesus Garcia wrote:
>> he problem is not downtime is how much downtime. Backup and restore is so
>> much downtime.
>
> Not more downtime that simple restore after complete server destruction
> by flood or
> plane crash.
Could y
02.01.2012 14:46, Jesus Garcia wrote:
> he problem is not downtime is how much downtime. Backup and restore is so
> much downtime.
Not more downtime that simple restore after complete server destruction by
flood or
plane crash.
--
SY, SD.
---
>
> Single server without downtime is a myth anyway.
The problem is not downtime is how much downtime. Backup and restore is so much
downtime.
Jesus
--
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a
01.01.2012 23:43, Kjell Rilbe wrote:
> Den 2012-01-01 23:25 skrev Dimitry Sibiryakov såhär:
>> 01.01.2012 21:31, Kjell Rilbe wrote:
>>> So far, these are the suggestions I've seen
>> Hey, how about my suggestion leave everything as is and perform
>> backup-restore cycle on
>> demand?..
>>
>>
Den 2012-01-01 23:47 skrev Lester Caine såhär:
> Dimitry Sibiryakov wrote:
>> 01.01.2012 21:31, Kjell Rilbe wrote:
So far, these are the suggestions I've seen
>> Hey, how about my suggestion leave everything as is and perform
>> backup-restore cycle on
>> demand?..
>>
>> Pros:
>> - No
Dimitry Sibiryakov wrote:
> 01.01.2012 21:31, Kjell Rilbe wrote:
>> > So far, these are the suggestions I've seen
> Hey, how about my suggestion leave everything as is and perform
> backup-restore cycle on
> demand?..
>
> Pros:
> - No overhead.
> - No additional disk space.
> - No code change
Den 2012-01-01 23:25 skrev Dimitry Sibiryakov såhär:
> 01.01.2012 21:31, Kjell Rilbe wrote:
>> So far, these are the suggestions I've seen
> Hey, how about my suggestion leave everything as is and perform
> backup-restore cycle on
> demand?..
>
> Pros:
> - No overhead.
> - No additional disk s
01.01.2012 21:31, Kjell Rilbe wrote:
> So far, these are the suggestions I've seen
Hey, how about my suggestion leave everything as is and perform
backup-restore cycle on
demand?..
Pros:
- No overhead.
- No additional disk space.
- No code change.
Cons:
- DBA with brain is required.
--
So far, these are the suggestions I've seen (I'm adding my own
suggestion at the bottom):
1. Implement 64 bit id:s and use them everywhere always.
Pros:
- Straightforward once all code has been changed to use 64 bit id:s.
- No overhead for various special flags and stuff.
Cons:
- Will increase di
>Nobody needs more than 4 billions active transactions concurrently.
But that's not what you'll get, because you can complete transactions
and get a hol, as it were. Can a process grab a transaction id and hold
onto it 'for ever' without messing up the system in other ways that
preventing the
01.01.2012 18:22, Adriano dos Santos Fernandes wrote:
> Let something like sweep consolidate old transactions in only one,
> concurrently with user operations.
Sweep is mostly read-only operation which is known to cause intolerable
slowdown.
Everybody turn it off because of that. You are sugg
On 01-01-2012 15:11, Jesús García wrote:
>
> I think the question is if it is neccesary and good for Firebird, and if the
> pros of having transactions id of 64bits are better that the cons of not
> having it. From my POV using 32 bits id is an important restriction in the
> engine, and i would
>
> Disk speed was always the issue, it doesn't increase that fast as
> CPU/memory speed. So the on-disk size always matters quite a lot.
>
>
> Dmitry
I think the question is if it is neccesary and good for Firebird, and if the
pros of having transactions id of 64bits are better that the cons
>
> CPU speed also has stopped growing.
>
You are right, speed has stopped but performance has increased.
Jesus
--
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT r
01.01.2012 14:06, Dmitry Yemanov wrote:
> Disk speed was always the issue, it doesn't increase that fast as
> CPU/memory speed. So the on-disk size always matters quite a lot.
CPU speed also has stopped growing.
--
SY, SD.
--
01.01.2012 16:48, Jesus Garcia wrote:
>
> Is not better use 64 bits integer?. In the near future computers speed, hdd
> speeds, etc. will increase, and transaction id of 64 bits overhead will
> affect less and less to performance.
Disk speed was always the issue, it doesn't increase that fast as
>
> We have 7 or 8 bits (out of 16) currently available.
>
>
> Dmitry
Is not better use 64 bits integer?. In the near future computers speed, hdd
speeds, etc. will increase, and transaction id of 64 bits overhead will affect
less and less to performance.
Jesus
---
01.01.2012 14:34, Kjell Rilbe wrote:
> Are there unused flag bits available in the current record format, that
> could be used for this purpose? Or how are such flag bits encoded?
We have 7 or 8 bits (out of 16) currently available.
Dmitry
--
Den 2011-12-31 15:08 skrev Dmitry Yemanov såhär:
> 31.12.2011 17:57, Alexander Peshkov wrote:
>
>> This will make each version of the record (not the record- but EACH
>> version of it) 4 bytes longer.
> Not strictly necessary. We could use a variable-length encoding for txn
> ids longer than 32 bit
В Сб., 31/12/2011 в 18:08 +0400, Dmitry Yemanov пишет:
> 31.12.2011 17:57, Alexander Peshkov wrote:
>
> > This will make each version of the record (not the record- but EACH
> > version of it) 4 bytes longer.
>
> Not strictly necessary. We could use a variable-length encoding for txn
> ids longe
31.12.2011 17:57, Alexander Peshkov wrote:
> This will make each version of the record (not the record- but EACH
> version of it) 4 bytes longer.
Not strictly necessary. We could use a variable-length encoding for txn
ids longer than 32 bits and mark such records with a new flag. It would
add z
Dmitry,
> Trunk is the ongoing development branch (formerly known as HEAD in CVS),
> i.e. transaction IDs are already unsigned long in FB 3.0.
Although this would mean a further ODS change as well as an increase in
overhead associated with all rows, perhaps the ODS size should be increased
from
On 12/29/11 22:05, Leyne, Sean wrote:
>> The reason for signed type here was (at least visible reason) very simple
>> - into some functions using same parameter might
>> be passed transaction number (positive value) and something else (negative
>> value). I.e. negative sign meant 'this is not tr
29.12.2011 22:05, Leyne, Sean wrote:
>
>> This change was done in trunk.
>
> To be clear, you have a code-branch that uses ULONG for transaction ID?
Trunk is the ongoing development branch (formerly known as HEAD in CVS),
i.e. transaction IDs are already unsigned long in FB 3.0.
Dmitry
---
> Actually, there is at least one other "special" value for transaction ids.
> Zero is
> always the system transaction.
The "system transaction"
I seem to have forgotten about that item.
> Maybe it's time to look at all the small integers as well.
Agreed.
Mike Nordell and I had tried
Sean,
> Alex wrote:
>> - into some functions using same parameter might
>> be passed transaction number (positive value) and something else (negative
>> value). I.e. negative sign meant 'this is not transaction' and function
>> behaved
>> according to it.
>
> And some people have complained ab
>There is transaction number zero.
Actually, all databases are initialized with transaction #1 as the starting
value, so there should not be any transaction 0.
Sean
--
Ridiculously easy VDI. With Citrix VDI-in-a
> Heavy loaded systems should use clusters. That's all. While one node is on
> maintenance,
> others do all work. These systems need clusters anyway for high availability
> and/or load
> balancing. One can't be serious running critical systems on a single server.
>
I don't agree with you. Y
29.12.2011 20:41, Jesus Garcia wrote:
> Would not be better, instead of that, If transaction id is equal To 0, no
> transaction, else transaction.
There is transaction number zero.
> As now there is a problem with transactionid and heavy loaded systems, that
> could solve in a little the pro
> This change was done in trunk. The reason for signed type here was (at
> least visible reason) very simple - into some functions using same
> parameter might be passed transaction number (positive value) and
> something else (negative value). I.e. negative sign meant 'this is not
> transaction'
Alex,
> > The "simple" solution is to change the datatype of Transaction related
> variables from SLONG* to ULONG. The reality of this solution is,
> unfortunately, far uglier given the testing required to confirm that all
> references have been changed.
> >
> >
> > Sean
> >
> >
> > * for the life
On 12/28/11 22:51, Leyne, Sean wrote:
>> It is feasible to roll over the transaction ID without putting the database
>> offline? i.e. when ID is close to limit, reset it to 0 from the code?
> In theory there is a "simple" code change which would provide you another 6
> months breathing room.
>
>
> It is feasible to roll over the transaction ID without putting the database
> offline? i.e. when ID is close to limit, reset it to 0 from the code?
In theory there is a "simple" code change which would provide you another 6
months breathing room.
But there is no permanent solution which is cu
On Wed, Dec 28, 2011 at 5:34 PM, Yi Lu wrote:
> It is feasible to roll over the transaction ID without putting the database
> offline? i.e. when ID is close to limit, reset it to 0 from the code?
Seems that PostgreSQL AutoVacuum daemon do that for Postgresql databases :
http://www.postgresql.org
28.12.2011 17:34, Yi Lu wrote:
> It is feasible to roll over the transaction ID without putting the database
> offline? i.e. when ID is close to limit, reset it to 0 from the code?
It will result in comple data loss. "DROP DATABASE" has the same result, but
easier to do.
--
SY, SD.
-
It is feasible to roll over the transaction ID without putting the database
offline? i.e. when ID is close to limit, reset it to 0 from the code?
--
View this message in context:
http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-solution-tp4230210p4240359.html
Sent from the fire
One of our busy sites has more than 120 TPS and it cannot last longer than 6
month. The database is online 24-7 and number of simultaneous users ranging
from 0 to 50. Currently, its transaction number is at 2,089,589,687
By calculation, an average of 39 transactions per second (TPS) corresponds
to
27.12.2011 22:17, Leyne, Sean wrote:
> I should have said:
>
> "That type of solution is not what immediately comes to mind for me, since I
> see a shared disk solution (using redundant SAN storage) to be much easier to
> implement for FB."
Unfortunately, shared-storage cluster doesn't solve
Dimitry,
> 27.12.2011 19:17, Leyne, Sean wrote:
> > That type of solution is not what I would define as a cluster.
>
>As you wish. But the rest of world consider this kind of system to be
> called
> "a shared-nothing cluster".
You are correct! "a shared-nothing cluster" is a type of cluste
27.12.2011 19:17, Leyne, Sean wrote:
> That type of solution is not what I would define as a cluster.
As you wish. But the rest of world consider this kind of system to be called
"a
shared-nothing cluster".
--
SY, SD.
-
Dimitry,
> 25.12.2011 1:00, Leyne, Sean wrote:
> > A cluster won't help, since all databases would see the same number of
> transactions, the number of which is the problem.
>
>No, if several transactions during transfer between nodes are merged into
> one.
>Besides, there is no need for
25.12.2011 1:00, Leyne, Sean wrote:
> A cluster won't help, since all databases would see the same number of
> transactions, the number of which is the problem.
No, if several transactions during transfer between nodes are merged into
one.
Besides, there is no need for all nodes to start f
among Firebird Developers
Subject: Re: [Firebird-devel] Firebird Transaction ID limit solution - Email
found in subject
24.12.2011 1:37, Yi Lu wrote:
> 1.I wonder if anyone now has a proper solution to this issue, without
> requireing shutting down the system.
Set up cluster. While one node
Hello, Yi!
Saturday, December 24, 2011, 4:37:08 AM, you wrote:
YL> In the past week, we have a few databases from several customers run out of
YL> the 2^31-1 transaction ID limit. After some research I found this problem to
YL> have been around since five years ago. Some suggestions were given su
24.12.2011 1:37, Yi Lu wrote:
> 1.I wonder if anyone now has a proper solution to this issue, without
> requireing shutting down the system.
Set up cluster. While one node is down for maintenance, users can work with
other(s).
--
SY, SD.
--
In the past week, we have a few databases from several customers run out of
the 2^31-1 transaction ID limit. After some research I found this problem to
have been around since five years ago. Some suggestions were given such as
this thread,
http://firebird.1100200.n4.nabble.com/discussion-about-rea
86 matches
Mail list logo
