RE: [firebird-support] performance issue with firebird 3.0 embedded on linux

[chen hsu chen...@outlook.com [firebird-support]]

I see. That might be the cause that I did not see throughput increase by adding 
more threads. let me try to have each thread create its own dedicated 
connection.  

To: firebird-support@yahoogroups.com
From: firebird-support@yahoogroups.com
Date: Fri, 17 Jun 2016 12:11:03 -0700
Subject: RE: [firebird-support] performance issue with firebird 3.0 embedded on 
linux














 

 



  



  
  
  ---In firebird-support@yahoogroups.com,  wrote :


> yes, one connection per database, any thread needs to access this database 
> would 
> create its own transaction. 

  Then you have serialization at connection level. Both engine and remote layer 
doesn't allow to run more than one thread per connection at the same time.

Regards,
Vlad




 









  

Re: [firebird-support] Re: Column Name Size

[Hugo Eyng hugoe...@msn.com [firebird-support]]

Thank you (all) for the answer.




Em 17/06/2016 10:31, Dmitry Yemanov dim...@users.sourceforge.net 
[firebird-support] escreveu:
> 17.06.2016 10:56, Dimitry Sibiryakov s...@ibphoenix.com wrote:
 Is there the limitation of 31 bytes for column names in the new FB 3?
>>> 31 CHARACTERS - as before
>> No, it is 31 byte.
> Correct, the limit is 31 *bytes*. The legacy UNICODE_FSS charset is
> multi-byte (up to three bytes per character) in user-defined tables, but
> it's treated specially for system tables, limiting the length to 31
> bytes, not characters (like it was a single-byte charset). The reasons
> are mainly historical (thanks to Borland) and related to backward
> compatibility of API and GPRE-generated code.
>
> v3 preserves this legacy behaviour. v4 resolves this issue from multiple
> angles:
>
> - switching to UTF8 instead of UNICODE_FSS
> - counting the length in characters instead of bytes
> - increasing the limit to 63 characters (at the moment, may change
> before the final release)
>
>
> Dmitry
>
>
>
>
> 
>
> 
>
> ++
>
> Visit http://www.firebirdsql.org and click the Documentation item
> on the main (top) menu.  Try FAQ and other links from the left-side menu 
> there.
>
> Also search the knowledgebases at 
> http://www.ibphoenix.com/resources/documents/
>
> ++
> 
>
> Yahoo Groups Links
>
>
>
>
>

-- 


Atenciosamente,

Hugo Eyng







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

RE: [firebird-support] performance issue with firebird 3.0 embedded on linux

[hv...@users.sourceforge.net [firebird-support]]

---In firebird-support@yahoogroups.com,  wrote :
 
 > yes, one connection per database, any thread needs to access this database 
 > would 
> create its own transaction. 

  Then you have serialization at connection level. Both engine and remote layer 
doesn't allow to run more than one thread per connection at the same time.

Regards,
Vlad
 

RE: [firebird-support] Re: performance issue with firebird 3.0 embedded on linux

[chen hsu chen...@outlook.com [firebird-support]]

Thanks, these tips are very helpful. 
#1: has very good improvement, please see the new result below.  but will this 
turning off of write-barrier cause data loss in case of power loss? 
Write-barrier looks about the ordering of journaling transaction content and 
commit record, how come it impacts firebird so dramatically? 
#2: totally agree. 
#3: yes, I am using classic or superclassic. 
The other thing I am still wondering is, it looks that increasing number of 
threads did not improve the performance much? In my specific experiment, it 
starts with 5 tx/sec with 1 thread, and increases only to 8-9 tx/sec with 6 
threads. 
with one cpu, 200 insertion per transaction, 1 transaction in total. each 
record is 128 bytes. * Finish Insert: total=25600 bytes elapsed=00.193 
throughput=132642.487047 bytes/sec, 1036.269430 ops/sec, 5.181347 tx/sec 
succeeded=1 
with 2 cpus, 200 insertion per transaction, 2 transaction in total. each record 
is 128 bytes. * Finish Insert: total=51200 bytes elapsed=00.292 
throughput=175342.465753 bytes/sec, 1369.863014 ops/sec, 6.849315 tx/sec 
succeeded=2
with 3 cpus, 200 insertion per transaction, 3 transaction in total. each record 
is 128 bytes. * Finish Insert: total=76800 bytes elapsed=00.375 
throughput=204800.00 bytes/sec, 1600.00 ops/sec, 8.00 tx/sec 
succeeded=3
with 4 cpus, 200 insertion per transaction, 4 transaction in total. each record 
is 128 bytes. * Finish Insert: total=102400 bytes elapsed=00.473 
throughput=216490.486258 bytes/sec, 1691.331924 ops/sec, 8.456660 tx/sec 
succeeded=4
with 5 cpus, 200 insertion per transaction, 5 transaction in total. each record 
is 128 bytes. * Finish Insert: total=128000 bytes elapsed=00.590 
throughput=216949.152542 bytes/sec, 1694.915254 ops/sec, 8.474576 tx/sec 
succeeded=5
with 6 cpus, 200 insertion per transaction, 6 transaction in total. each record 
is 128 bytes. * Finish Insert: total=153600 bytes elapsed=00.717 
throughput=214225.941423 bytes/sec, 1673.640167 ops/sec, 8.368201 tx/sec 
succeeded=6

To: firebird-support@yahoogroups.com
From: firebird-support@yahoogroups.com
Date: Thu, 16 Jun 2016 23:06:12 -0700
Subject: [firebird-support] Re: performance issue with firebird 3.0 embedded on 
linux














 

 



  



  
  
  Hi!

#1 : FB does not like ext4 : Firebird News » Forced Writes Performance impact 
on #Ubuntu with ext4 no barrier
 Firebird News » Forced Writes Performance impact 
on #Ub...   Tweet of the day comes from Carlos H. Cantu about forced writes 
effects for running firebird scripts Script executing time – Linux Ubuntu 
10.04.3 Ext4 – ...  View on www.firebirdnews.org  Preview 
by Yahoo  


#2 : HT is no necessary a good idea in case of data intensive applications (eg. 
databases) : Be aware: To Hyper or not to Hyper Be 
aware: To Hyper or not to Hyper   Our customers observed very interesting 
behavior on high end Hyperthreading (HT) enabled hardware. They noticed that in 
some cases when high load is applied... View on 
blogs.msdn.microsof...Preview by Yahoo
 
#3 : Use classic server, or superclassic (superserver does not like concurrency 
is a single database).



#4 : I think the bottleneck here is the disk, not the engine itself. We made a 
test for bulk insert and best method was to run execute blocks with  as many 
inserts as possible in it in a single thread. We achieved 
8-9K records / sec for small tables and 3-4K records / sec for bigger tables. 
(tables were not indexed, and it was a simple HDD)







 









  

[firebird-support] Re: Column Name Size

[Dmitry Yemanov dim...@users.sourceforge.net [firebird-support]]

17.06.2016 10:56, Dimitry Sibiryakov s...@ibphoenix.com wrote:
>
>>> Is there the limitation of 31 bytes for column names in the new FB 3?
>>
>> 31 CHARACTERS - as before
>
> No, it is 31 byte.

Correct, the limit is 31 *bytes*. The legacy UNICODE_FSS charset is 
multi-byte (up to three bytes per character) in user-defined tables, but 
it's treated specially for system tables, limiting the length to 31 
bytes, not characters (like it was a single-byte charset). The reasons 
are mainly historical (thanks to Borland) and related to backward 
compatibility of API and GPRE-generated code.

v3 preserves this legacy behaviour. v4 resolves this issue from multiple 
angles:

- switching to UTF8 instead of UNICODE_FSS
- counting the length in characters instead of bytes
- increasing the limit to 63 characters (at the moment, may change 
before the final release)


Dmitry








++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Slavomir Skopalik skopa...@elektlabs.cz [firebird-support]]

Hi Roland,

in some cases we did this to log all transaction starts commits (see 
script below).

Set GenMonTransactions to value less 1 will activate log function.

You can simply to change to log transaction commits or rollbacks as well.

And in application you can easy log transaction id that was requested.


Slavek

Create Table MON_Transactions  (
 tDateUTC TimeDateUTC NOT NULL,
 sys_NETWORK_PROTOCOL Varchar(255),
 sys_CLIENT_ADDRESS Varchar(255),
 sys_CURRENT_USER Varchar(255),
 sys_CURRENT_ROLE Varchar(255),
 sys_SESSION_ID Varchar(255),
 sys_TRANSACTION_ID Varchar(255),
 sys_ISOLATION_LEVEL Varchar(255),
 mon_REMOTE_PROCESS Varchar(255),
 data TMemo
);

CREATE TRIGGER TRGMON_TransactionStart ON TRANSACTION START
AS
   DECLARE i INTEGER;
   DECLARE transactionID INTEGER;
   DECLARE remoteProcess VARCHAR(255);
BEGIN
   IF (1 <= GEN_ID(GenMonTransactions, 0)) THEN BEGIN
 EXIT;
   END
   i = GEN_ID(GenMonTransactions, 1);
   transactionID = RDB$GET_CONTEXT('SYSTEM', 'TRANSACTION_ID');

   SELECT FIRST 1 mon$remote_process
   FROM mon$transactions T
   JOIN mon$attachments A ON A.mon$attachment_id = T.mon$attachment_id
   WHERE T.mon$transaction_id = :transactionID
   INTO :remoteProcess;

   INSERT INTO MON_Transactions(
 sys_NETWORK_PROTOCOL,
 sys_CLIENT_ADDRESS,
 sys_CURRENT_USER,
 sys_CURRENT_ROLE,
 sys_SESSION_ID,
 sys_TRANSACTION_ID,
 sys_ISOLATION_LEVEL,
 mon_REMOTE_PROCESS
   ) VALUES(
 RDB$GET_CONTEXT('SYSTEM', 'NETWORK_PROTOCOL'),
 RDB$GET_CONTEXT('SYSTEM', 'CLIENT_ADDRESS'),
 RDB$GET_CONTEXT('SYSTEM', 'CURRENT_USER'),
 RDB$GET_CONTEXT('SYSTEM', 'CURRENT_ROLE'),
 RDB$GET_CONTEXT('SYSTEM', 'SESSION_ID'),
 :transactionID,
 RDB$GET_CONTEXT('SYSTEM', 'ISOLATION_LEVEL'),
 :remoteProcess
   );
END^

Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o.
Chaloupky 158
783 72 Velky Tynec
Czech Republic
---
Mobile: +420 724 207 851
icq:199 118 333
skype:skopaliks
e-mail:skopa...@elektlabs.cz
http://www.elektlabs.cz

On 17.6.2016 13:22, Roland Turcan k...@rotursoft.sk [firebird-support] 
wrote:
> <<< 17.06.2016 13:20 - Dimitry Sibiryakov s...@ibphoenix.com 
> [firebird-support] "firebird-support@yahoogroups.com" >>>
> DSsicfs> 17.06.2016 13:17, Roland Turcan k...@rotursoft.sk [firebird-support] 
> wrote:
>>> I don't want to post this issue into DEVEL, because it is a support
>>> question, but where are the guys able to answer or at least to point
>>> us to some hints.
> DSsicfs>Every time I saw this error, it was a bug in my
> DSsicfs> application. No exceptions so far.
>
> Dmitry,  thanks  for your reply, could you please tell me what was the
> problem in your case?
>
> PS: I have never had this issue with FB2.5 or older.
>








++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]]

17.06.2016 13:22, Roland Turcan k...@rotursoft.sk [firebird-support] wrote:
> Dmitry, thanks for your reply, could you please tell me what was the
> problem in your case?

   Bad synchronization, global variables, garbage values and so on.


-- 
   WBR, SD.






++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Roland Turcan k...@rotursoft.sk [firebird-support]]

<<< 17.06.2016 13:20 - Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] 
"firebird-support@yahoogroups.com" >>>
DSsicfs> 17.06.2016 13:17, Roland Turcan k...@rotursoft.sk [firebird-support] 
wrote:
>> I don't want to post this issue into DEVEL, because it is a support
>> question, but where are the guys able to answer or at least to point
>> us to some hints.

DSsicfs>Every time I saw this error, it was a bug in my
DSsicfs> application. No exceptions so far.

Dmitry,  thanks  for your reply, could you please tell me what was the
problem in your case?

PS: I have never had this issue with FB2.5 or older.

-- 
Best regards, TRoland
http://www.rotursoft.sk
http://exekutor.rotursoft.sk

Re: [firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]]

17.06.2016 13:17, Roland Turcan k...@rotursoft.sk [firebird-support] wrote:
> I don't want to post this issue into DEVEL, because it is a support
> question, but where are the guys able to answer or at least to point
> us to some hints.

   Every time I saw this error, it was a bug in my application. No exceptions 
so far.


-- 
   WBR, SD.






++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Roland Turcan k...@rotursoft.sk [firebird-support]]

Hello Norbert,

Thanks for your confimation, that I am not alone on this planet having
this problem.

I  can't  say if customers had it during concurrent access to the same
record,  but  in  my  case  it was definitely not caused by concurrent
access,  because  I have got it on my developer test database with the
only connection.

I  don't  want  to post this issue into DEVEL, because it is a support
question,  but  where are the guys able to answer or at least to point
us to some hints.

Thanks.

RT;

<<< 17.06.2016 12:29 - Norbert Saint Georges n...@tetrasys.eu 
[firebird-support] "firebird-support@yahoogroups.com" >>>
NSGntefs> I have the same problem without being able to identify or reproduce.

NSGntefs> My background, dot net WebForm with Firebird 3.0, when multiple update
NSGntefs> via multiple threads on the same record.

NSGntefs> See if this is the same context for you?





-- 
Best regards, TRoland
http://www.rotursoft.sk
http://exekutor.rotursoft.sk

[firebird-support] Re: Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Norbert Saint Georges n...@tetrasys.eu [firebird-support]]

I have the same problem without being able to identify or reproduce.

My background, dot net WebForm with Firebird 3.0, when multiple update 
via multiple threads on the same record.

See if this is the same context for you?

-- 
Norbert Saint Georges
http://tetrasys.fi







++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Re: [firebird-support] Firebird 3.0 - invalid transaction handle (expecting explicit transaction start).

[Roland Turcan k...@rotursoft.sk [firebird-support]]

Re: [firebird-support] Column Name Size

[Dimitry Sibiryakov s...@ibphoenix.com [firebird-support]]

17.06.2016 0:01, Helen Borrie hele...@iinet.net.au [firebird-support] wrote:
>> Is there the limitation of 31 bytes for column names in the new FB 3?
>
> 31 CHARACTERS - as before

   No, it is 31 byte. And in 3.0 the limitation is more restrictive than 
before, look at 
CORE-5282 for example.


-- 
   WBR, SD.






++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ 

++


Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/firebird-support/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/firebird-support/join
(Yahoo! ID required)

<*> To change settings via email:
firebird-support-dig...@yahoogroups.com 
firebird-support-fullfeatu...@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
firebird-support-unsubscr...@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

[firebird-support] Re: performance issue with firebird 3.0 embedded on linux

[ma_go...@yahoo.com [firebird-support]]

Hi!

#1 : FB does not like ext4 : Firebird News » Forced Writes Performance impact 
on #Ubuntu with ext4 no barrier 
http://www.firebirdnews.org/forced-writes-performance-impact-on-ubuntu/ 
 
 http://www.firebirdnews.org/forced-writes-performance-impact-on-ubuntu/ 
 
 Firebird News » Forced Writes Performance impact on #Ub... 
http://www.firebirdnews.org/forced-writes-performance-impact-on-ubuntu/ Tweet 
of the day comes from Carlos H. Cantu about forced writes effects for running 
firebird scripts Script executing time – Linux Ubuntu 10.04.3 Ext4 – ...
 
 
 
 View on www.firebirdnews.org 
http://www.firebirdnews.org/forced-writes-performance-impact-on-ubuntu/ 
 Preview by Yahoo 
 
 
  


#2 : HT is no necessary a good idea in case of data intensive applications (eg. 
databases) : Be aware: To Hyper or not to Hyper 
https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-to-hyper-or-not-to-hyper/
 
 
 
https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-to-hyper-or-not-to-hyper/
 
 
 Be aware: To Hyper or not to Hyper 
https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-to-hyper-or-not-to-hyper/
 Our customers observed very interesting behavior on high end Hyperthreading 
(HT) enabled hardware. They noticed that in some cases when high load is 
applied...
 
 
 
 View on blogs.msdn.microsof... 
https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-to-hyper-or-not-to-hyper/
 
 Preview by Yahoo 
 
 
 
 
 #3 : Use classic server, or superclassic (superserver does not like 
concurrency is a single database).

 

 #4 : I think the bottleneck here is the disk, not the engine itself. We made a 
test for bulk insert and best method was to run execute blocks with as many 
inserts as possible in it in a single thread. We achieved 
8-9K records / sec for small tables and 3-4K records / sec for bigger tables. 
(tables were not indexed, and it was a simple HDD)