Nikolay Samofatov nikolay.samofa...@red-soft.biz wrote Thu, 03 Jul 2014
01:49:11 +0400:
With these changes, I expect to see no scenarios when request-level
snapshots would impose
significant performance impact (long-running transactions would not
inhibit GC anymore),
and it would
You're on the right track. But let me sketch how Falcon handled
transactions as food for thought.
Falcon kept a linked list of transactions in transaction start order (batch
inserts of unassigned transaction objects that could be grabbed with a
compare and swap). When a transaction was started,
All,
I have completed my investigation, and it seems possible to implement
intermediate GC in a way that makes everything faster, not slower.
Here is what I am going to do:
1) Move TPC into shared memory. For each transaction store its Commit # in
memory.
Transactions committed before the
On 27-6-2014 19:25, Ann Harrison wrote:
On Jun 23, 2014, at 3:57 PM, Nikolay Samofatov
nikolay.samofa...@red-soft.biz wrote:
Some records for a join can be read before a transaction is committed, and
some after. Same with EXISTS. It can see different set of commits from the
one when
On 24-6-2014 17:53, Paul Beach wrote:
But, after reading MS-SQL's read committed isolation level, their
default behavior is the same as Firebird's. And, they introduced the
READ COMMITTED SNAPSHOT isolation level, which is at the statement
level, and that is what Nickolay is looking for. And,
On 24-6-2014 18:05, Dimitry Sibiryakov wrote:
24.06.2014 17:53, Paul Beach wrote:
we need to keep our default behaviour as is and
create a new isolation level that supports the cursor stability
in Read Committed that Nikolay wants.
I would rather vote for a TPB flag like
On 25-6-2014 15:21, Dimitry Sibiryakov wrote:
This is right only for those who set up READ COMMITTED transaction
isolation level
which is not default.
That is not true, READ_COMMITTED is the default in a lot of the Firebird
drivers (eg in Jaybird, Firebird .NET provider, fdb).
Mark
--
28.06.2014 11:01, Mark Rotteveel wrote:
If a cursor is open, and the SQL-transaction in which the cursor was
opened makes a significant change to SQL-data
At the first glance, this is what we called cursor stability and have
fixed in FB3. But in fact it covers other cases as well and I don't
On 28-6-2014 09:51, Dmitry Yemanov wrote:
28.06.2014 11:34, Mark Rotteveel wrote:
Looking at the SQL standard, sensitivity should really be a property of
the statement (or more correctly: its cursor), not of the transaction.
True, but having all cursors insensitive by default per the given
On 28-6-2014 09:49, Dmitry Yemanov wrote:
28.06.2014 11:01, Mark Rotteveel wrote:
If a cursor is open, and the SQL-transaction in which the cursor was
opened makes a significant change to SQL-data
At the first glance, this is what we called cursor stability and have
fixed in FB3. But in
28.06.2014 11:54, Mark Rotteveel wrote:
As far as I understood, the cursor stability fix was about changes made
by the statement to its own data (eg an UPDATE or DELETE).
The fix was about changes made by statements located underneath the
cursor (e.g. for select + insert/update/delete inside,
28.06.2014 9:46, Mark Rotteveel wrote:
On 25-6-2014 15:21, Dimitry Sibiryakov wrote:
This is right only for those who set up READ COMMITTED transaction
isolation level
which is not default.
That is not true, READ_COMMITTED is the default in a lot of the Firebird
drivers (eg in
On 28-6-2014 10:36, Dimitry Sibiryakov wrote:
28.06.2014 9:46, Mark Rotteveel wrote:
On 25-6-2014 15:21, Dimitry Sibiryakov wrote:
This is right only for those who set up READ COMMITTED transaction
isolation level
which is not default.
That is not true, READ_COMMITTED is the default
28.06.2014 12:47, Mark Rotteveel wrote:
But as a datapoint, PostgreSQL does this:
Read Committed is the default isolation level in PostgreSQL. When a
transaction uses this isolation level, a SELECT query (without a FOR
UPDATE/SHARE clause) sees only data committed before the query began; it
28.06.2014 10:43, Mark Rotteveel wrote:
How about no. READ COMITTED is the default for a lot of other database
drivers for other DBMS as well, and usually matches the expectations of
users (which includes the occurrence of non-repeatable reads and phantom
reads).
Who cares about other
On 06/26/14 22:52, Nikolay Samofatov wrote:
You know that it is used to store intermediate RecordBitmap's during index
retrieval operations.
This is very performance sensitive code, and this implementation outperformed
and used less memory
than older SBM-based implementation.
I remember
On Jun 23, 2014, at 3:57 PM, Nikolay Samofatov
nikolay.samofa...@red-soft.biz wrote:
Some records for a join can be read before a transaction is committed, and
some after. Same with EXISTS. It can see different set of commits from the
one when main row was read.
You can see partial
25.06.2014 12:52, Dimitry Sibiryakov wrote:
25.06.2014 11:46, Roman Simakov wrote:
2014-06-25 12:09 GMT+04:00 Dimitry Sibiryakov s...@ibphoenix.com:
In this case why to use READ COMMITED transaction at all?..
Because opening cursor is not opening transaction. You can have long
transaction
23.06.2014 20:41, Nikolay Samofatov wrote:
Hello, Dmitry!
On 06/23/2014 04:15 AM, Dmitry Yemanov wrote:
21.06.2014 01:52, Nikolay Samofatov wrote:
I attach patch for this functionality to give you an idea of
implementation. It depends on a couple other changes so it doesn't apply
to FB2.5
Vlad, All,
On 06/25/2014 05:58 PM, Vlad Khorsun wrote:
AFAIK, concurrent GC in pre-committed transactions without transaction lock
was unsafe, and created
permanently broken databases.
I believe corruption has become especially apparent after commit intended to
fix CORE-3515. I don't
know
On 06/24/14 22:22, Dmitry Yemanov wrote:
24.06.2014 22:12, Dmitry Yemanov wrote:
(*) AFAIU, there's no other practical benefit in the sensitive mode
except the performance.
And the GC blockage in RC RO txns.
This GC blockage will definitely cause performance problems. It's
unavoidable.
Hi, all!
My 2 cents. Current behavior is really a BUG and I cannot imagine who
wants to have such logically incorrect data. It's enough for me to
make a conclusion to have new behavior as default. Some developers who
really understand that such situation cannot happen in his system can
EXPLICITLY
On 06/25/14 10:49, Roman Simakov wrote:
Hi, all!
My 2 cents. Current behavior is really a BUG and I cannot imagine who
wants to have such logically incorrect data.
A sample.
Warehouse has a number of workplaces with computers for shipping goods.
At some workplace in the beginning of a day
2014-06-25 11:24 GMT+04:00 Alex Peshkoff peshk...@mail.ru:
A sample.
Warehouse has a number of workplaces with computers for shipping goods.
At some workplace in the beginning of a day were shipped some antennas,
books and computers. I.e. we are at the top of the grid, cursor is
opened but
Roman Simakov roman.sima...@gmail.com wrote Wed, 25 Jun 2014 10:49:01
+0400:
Hi, all!
My 2 cents. Current behavior is really a BUG and I cannot imagine who
wants to have such logically incorrect data. It's enough for me to
make a conclusion to have new behavior as default. Some developers
On 06/25/14 11:52, Roman Simakov wrote:
2014-06-25 11:24 GMT+04:00 Alex Peshkoff peshk...@mail.ru:
A sample.
Warehouse has a number of workplaces with computers for shipping goods.
At some workplace in the beginning of a day were shipped some antennas,
books and computers. I.e. we are at the
25.06.2014 11:52, Roman Simakov wrote:
present at the opening cursor. IMO it's bad style to open cursor. It
may keep open transaction (it's also not good but RO+RC possible) but
it should not keep open cursor. I do not understand why to open cursor
at all if you are not going to fetch right
25.06.2014 9:52, Roman Simakov wrote:
2014-06-25 11:24 GMT+04:00 Alex Peshkoff peshk...@mail.ru:
please answer what quantity of them do we prefer to see - current or
present at the beginning of a day?
present at the opening cursor.
In this case why to use READ COMMITED transaction at
2014-06-25 12:09 GMT+04:00 Dimitry Sibiryakov s...@ibphoenix.com:
In this case why to use READ COMMITED transaction at all?..
Because opening cursor is not opening transaction. You can have long
transaction but not opened cursors. Do not open if you are not going
to fetch.
--
Roman Simakov
25.06.2014 11:46, Roman Simakov wrote:
2014-06-25 12:09 GMT+04:00 Dimitry Sibiryakov s...@ibphoenix.com:
In this case why to use READ COMMITED transaction at all?..
Because opening cursor is not opening transaction. You can have long
transaction but not opened cursors. Do not open if you
2014-06-25 13:52 GMT+04:00 Dimitry Sibiryakov s...@ibphoenix.com:
I.e. a strange application developer start transaction, then, after some
time, start a
query and want to see data consistent to the moment of query start instead of
transaction
start?
Exactly. The same prepared query can
25.06.2014 12:41, Roman Simakov wrote:
The same prepared query can re-read fresh data in the same
transaction, for instance refresh number of books.
You'll be wondered, perhaps, but a prepared query can re-read fresh data in
any
transaction, not only in one where it was prepared.
But
On 25/06/2014 05:10, Dmitry Yemanov wrote:
25.06.2014 11:52, Roman Simakov wrote:
present at the opening cursor. IMO it's bad style to open cursor. It
may keep open transaction (it's also not good but RO+RC possible) but
it should not keep open cursor. I do not understand why to open cursor
2014-06-25 14:49 GMT+04:00 Dimitry Sibiryakov s...@ibphoenix.com:
25.06.2014 12:41, Roman Simakov wrote:
The same prepared query can re-read fresh data in the same
transaction, for instance refresh number of books.
You'll be wondered, perhaps, but a prepared query can re-read fresh data
On 06/25/14 15:14, Adriano dos Santos Fernandes wrote:
On 25/06/2014 05:10, Dmitry Yemanov wrote:
25.06.2014 11:52, Roman Simakov wrote:
present at the opening cursor. IMO it's bad style to open cursor. It
may keep open transaction (it's also not good but RO+RC possible) but
it should not
25.06.2014 15:28, Alex Peshkoff wrote:
The problem with RC+RO consistent transaction blocking GC may be gone if
the engine runs the query till the end and store it in temporary buffer.
Dmitry, is this (more or less) what is already done to support
bidirectional cursors?
Yes, but this is
25.06.2014 13:13, Roman Simakov wrote:
Explain it to strange developers who start transaction and open
cursor at the begin of day in order to fetch at the end :)
And one more about logic. Currently it's possible:
1) We have a table X(c char) with 2 records {A, B}
2) transaction T1 opens
I suggest you investigate garbage collecting intermediate
unreachable record versions, which would completely take the sting out of
long running transactions. It's not difficult for superserver, but would
be problematic for classic. But I don't think that in itself is a problem.
I have never
All,
On 06/25/2014 09:41 AM, James Starkey wrote:
I suggest you investigate garbage collecting intermediate unreachable record
versions, which would
completely take the sting out of long running transactions. It's not
difficult for superserver,
but would be problematic for classic. But
AFAIK, concurrent GC in pre-committed transactions without transaction lock
was unsafe, and created
permanently broken databases.
I believe corruption has become especially apparent after commit intended to
fix CORE-3515. I don't
know who fixed it, but Red Soft builds include a patch for
Hi,
i try to follow this discussion and try to understand it.
Is this discussion only about Read-only RC or about all RC transactions?
If i understand correctly this bug it look like (correct me if i am wrong):
SELECT
(SELECT COUNT(*) FROM TEST T2 WHERE T2.SOME_FIELD=1) /* Query 2 */
FROM
TEST
i try to follow this discussion and try to understand it.
Is this discussion only about Read-only RC or about all RC transactions?
It is only about READ-ONLY + READ COMMITTED.
Sean
--
Open source business process
Hi,
At June 23, 2014, 4:19 PM, Carlos H. Cantu wrote:
NS Yes, for READ_ONLY + READ COMMITTED transactions you will now
NS inhibit GC if you keep a cursor open
NS for a long time.
This is serious change, and I'm probably against it being the default
behavior. So far, it is well advertised
23.06.2014 21:57, Nikolay Samofatov wrote:
You can see partial commits in results, even inside a single row returned by
the query.
Nobody is ready for this, this is CRAZY, nobody expects this. If this data is
used for any remotely
important purpose, you will get whammed.
Shouldn't data
24.06.2014 17:35, Daniel Rail wrote:
I don't think it takes out the non-blocking behavior, because the
blocking is only performed at the statement level, not the transaction
level. So, for a lookup dataset, it would not change a thing, since
the query results will always pick up committed
But, after reading MS-SQL's read committed isolation level, their
default behavior is the same as Firebird's. And, they introduced the
READ COMMITTED SNAPSHOT isolation level, which is at the statement
level, and that is what Nickolay is looking for. And, both READ
COMMITTED isolation levels fall
Hi,
At June 24, 2014, 12:45 PM, Dmitry Yemanov wrote:
24.06.2014 17:35, Daniel Rail wrote:
I don't think it takes out the non-blocking behavior, because the
blocking is only performed at the statement level, not the transaction
level. So, for a lookup dataset, it would not change a thing,
24.06.2014 17:53, Paul Beach wrote:
we need to keep our default behaviour as is and
create a new isolation level that supports the cursor stability
in Read Committed that Nikolay wants.
I would rather vote for a TPB flag like
isc_tpb_insensitive_cursor/sensitive_cursor. It
would fit
24.06.2014 01:33, Nikolay Samofatov wrote:
If this is what people want, it is possible to add new TPB parameter -
isc_tpb_inconsistency, and
permit it for READ_ONLY + READ COMMITTED transactions only. For as long you
don't use returned data
for anything important it is somewhat safe.
24.06.2014 22:12, Dmitry Yemanov wrote:
(*) AFAIU, there's no other practical benefit in the sensitive mode
except the performance.
And the GC blockage in RC RO txns.
Dmitry
--
Open source business process
21.06.2014 0:52, Nikolay Samofatov wrote:
Hello, All!
We have encountered subtle errors and data corruptions when using complex
triggers/stored procedures in READ COMMITTED
transactions.
I assume you tell about logical data corruptions, not physical, correct ?
Most applied programmers
21.06.2014 01:52, Nikolay Samofatov wrote:
I attach patch for this functionality to give you an idea of
implementation. It depends on a couple other changes so it doesn't apply
to FB2.5 cleanly.
The first thing I'm interested in is the performance impact in OLTP
tests similar to TPCC (with
Vlad,
On 06/23/2014 03:11 AM, Vlad Khorsun wrote:
21.06.2014 0:52, Nikolay Samofatov wrote:
Hello, All!
We have encountered subtle errors and data corruptions when using complex
triggers/stored procedures in READ COMMITTED
transactions.
I assume you tell about logical data corruptions,
Hello, Dmitry!
On 06/23/2014 04:15 AM, Dmitry Yemanov wrote:
21.06.2014 01:52, Nikolay Samofatov wrote:
I attach patch for this functionality to give you an idea of
implementation. It depends on a couple other changes so it doesn't apply
to FB2.5 cleanly.
The first thing I'm interested in is
Здравствуйте!
Monday, June 23, 2014, 10:31:57 PM, you wrote:
NS Denis,
NS On 06/21/2014 08:10 AM, Simonov Denis wrote:
A long-running transaction READ READ_COMMITED will not keep the garbage?
Maybe we should introduce a new key TPB, if you want to maintain
compatibility.
NS Yes, for
NS Yes, for READ_ONLY + READ COMMITTED transactions you will now
NS inhibit GC if you keep a cursor open for a long time.
nice (sarcazm). So, dilemma is to enable your behavior, and get versions
stall,
or get versions not stalled by GS by usual RC Read?
Actually, without the new
23.06.2014 23:15, Leyne, Sean wrote:
Actually, without the new behaviour, the engine results are not guaranteed to
be correct.
But the point is that their correctness depends on the application
logic. Lots of applications using RC RO transactions will never be
affected by the cursor
Nikolay Samofatov nikolay.samofa...@red-soft.biz писал(а) в своём письме
Mon, 23 Jun 2014 22:31:57 +0400:
Denis,
On 06/21/2014 08:10 AM, Simonov Denis wrote:
A long-running transaction READ READ_COMMITED will not keep the garbage?
Maybe we should introduce a new key TPB, if you want to
NS Yes, for READ_ONLY + READ COMMITTED transactions you will now
NS inhibit GC if you keep a cursor open
NS for a long time.
This is serious change, and I'm probably against it being the default
behavior. So far, it is well advertised that transactions using RC+RO
does NOT block GC, and several
All,
On 06/23/2014 03:26 PM, Dmitry Yemanov wrote:
23.06.2014 23:15, Leyne, Sean wrote:
Actually, without the new behaviour, the engine results are not guaranteed
to be correct.
But the point is that their correctness depends on the application
logic. Lots of applications using RC RO
Hello, Carlos!
On 06/23/2014 03:19 PM, Carlos H. Cantu wrote:
NS Yes, for READ_ONLY + READ COMMITTED transactions you will now
NS inhibit GC if you keep a cursor open
NS for a long time.
This is serious change, and I'm probably against it being the default
behavior. So far, it is well
20.06.2014 23:52, Nikolay Samofatov wrote:
Any objections if I start porting this functionality to FB3?
Isn't cursor stability already implemented in FB3?..
--
WBR, SD.
--
HPCC Systems Open Source Big Data
Nikolay Samofatov nikolay.samofa...@red-soft.biz писал(а) в своём письме
Sat, 21 Jun 2014 01:52:46 +0400:
Hello, All!
We have encountered subtle errors and data corruptions when using
complex triggers/stored procedures
in READ COMMITTED transactions.
Most applied programmers don't think
Hello, All!
We have encountered subtle errors and data corruptions when using complex triggers/stored procedures
in READ COMMITTED transactions.
Most applied programmers don't think that while their SP/trigger is executing the world is changing
underneath them.
So lost updates and
Hello!
Saturday, June 21, 2014, 1:52:46 AM, you wrote:
NS To address this problem, in our engine builds we changed behavior of READ
COMMITTED + REC_VERSION
NS transactions to ensure cursor stability.
NS Each request is executed in its own snapshot, that is released when
execution of request
65 matches
Mail list logo
