Re: [Firebird-devel] (possible) Database header
gstat -h вт, 23 авг. 2022 г. в 07:15, Tommi Prami : > > Hello, > > > > I was thinking that if there is some kind of header (simple enough) in > Firebird Databases and/or backup files, maybe some one could point me into > the right direction. > > I would preferably need (mainly for installer) small tool that would check > the ODS version of the DB before I try to connect into it. And depending on > that, do some preprocessing if needed or make some other choices based upon > that. > > Currently I open the database and detect ODS version that way, but it makes > life bit too complicated. > > > > -Tee- > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Refresh roles
ср, 11 мая 2022 г. в 21:10, Adriano dos Santos Fernandes : > > On 11/05/2022 14:47, Roman Simakov wrote: > > > > I was thinking the profiler should not touch profiled attachments at > > all. It might influence its logic. > > > > The profiler should work on its initial usage when it creates its metadata. > > This is priority for me than theoretical problems due to roles being > refreshed. SQL SECURITY replaces an effective user for some time. Can you use the same trick while profiler initializing? -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Refresh roles
ср, 11 мая 2022 г. в 19:40, Adriano dos Santos Fernandes : > > On 11/05/2022 11:55, Roman Simakov wrote: > > Can you make up or describe a potential case of the situation? > > Undoubtedly the feature would be useful but we've lived without it for > > all the time. > > > > In the profiler, I will create a role, the tables and views in another > connection. I'll grant privileges to the role and default grant the role > to public (currently not working due to #7178, so I'm testing with > another user instead of public). > > But the user connection is already made and is going to use the profiler. > > Now I need to see what is its current_role, then "set role plg$profiler" > (or any other role different than current), then "set role > old_current_role" to refresh the current roles. I was thinking the profiler should not touch profiled attachments at all. It might influence its logic. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Refresh roles
Can you make up or describe a potential case of the situation? Undoubtedly the feature would be useful but we've lived without it for all the time. If a role is revoked from a user, it will affect the next attachment only, won't it? ср, 11 мая 2022 г. в 16:36, Adriano dos Santos Fernandes : > > Hi! > > When roles are default granted to an user, they are not immediately > reflected in the attachments. > > Attachments need to issue a SET ROLE which should be different than > currently used, then another SET ROLE to the correct one to refresh the > roles. Or re-attach. > > I think we should improve that automatically refreshing roles via AST or > creating some command like ALTER SESSION REFRESH ROLES. > > > Adriano > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] DEFAULT grants to PUBLIC
It makes sense. Could you create a ticket? вт, 10 мая 2022 г. в 04:31, Adriano dos Santos Fernandes : > > Hi! > > A role can be default granted to PUBLIC. > > create role r1; > grant default r1 to public; > > In this case I think the role should be act as a defaulted role to every > user. > > But it's not working. > > The role privileges only works if user uses it explicitly. > > > Adriano > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Fw: [firebird-support] ARM64 Support
Don't general instructions work? вт, 19 апр. 2022 г. в 20:07, Hugo Larson via Firebird-devel : > > Hi, > > I want to build FB4 for Linux (Ubuntu) arm64 and wonder if there is > documentation with instructions? > > Thanks, > Hugo. > > - Forwarded Message - > From: Mark Rotteveel > To: "firebird-supp...@googlegroups.com" > Sent: Friday, April 15, 2022, 02:54:42 PM GMT+7 > Subject: Re: [firebird-support] ARM64 Support > > On 15-04-2022 07:03, 'Hugo Larson' via firebird-support wrote: > > I would like to build FB4 for ARM64 Linux. > > Is there a documentation with instructions on how to do this? > > I recommend asking on firebird-devel. > > > Mark > -- > Mark Rotteveel > > -- > You received this message because you are subscribed to the Google Groups > "firebird-support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to firebird-support+unsubscr...@googlegroups.com. > > To view this discussion on the web, visit > https://groups.google.com/d/msgid/firebird-support/a07d964a-e725-c7ac-5b76-3252edaf97be%40lawinegevaar.nl. > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Handling exception from external plugins
Alex, Is it documented somewhere? пн, 11 апр. 2022 г. в 18:45, Alex Peshkoff via Firebird-devel : > > On 4/11/22 17:56, Dimitry Sibiryakov wrote: > > Vasiliy Yashkov wrote 11.04.2022 16:52: > >> What is the right way to throw an exception? > > > > Plugins (as any other DLL) is not supposed to throw exceptions ever. > > > > Yes, throwing directly (like in your code) is impossible. But there is a > way to return an exception information from plugin using Status > interface, which is passed as first parameter in that case. > Pay attention - FB_PLUGIN_ENTRY_POINT has no such parameter and > therefore is not expected to return exception information. (there are > more reasons why exceptions not to be used in that call but let me not > dive too deep here) > > But if some plugin entry has first Status parameter you can throw > FbException in it, it will be caught by thin API layer which actually > invoked your plugin entry and delivered to the code which called plugin. > > > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Dropping database default SQL SECURITY
I don't remember exactly why we decided to make it nullable. I suppose for more backward compatibility. If a client doesn't use it it will be NULL everywhere. So in fact it can be NULL physically. But logically only two options make sense. NULL is considered as legacy, i.e. INVOKER. Thus you are right. For INVOKER we have two values to read and one to write. пн, 14 февр. 2022 г. в 19:43, Jiří Činčura : > > You say it cannot be NULL, yet you say the "initial NULL". Looks very > inconsist to me. Now for INVOKER there's 2 possible values. Yet one can't > ever be set by user. > > -- > Mgr. Jiří Činčura > https://www.tabsoverspaces.com/ > > On Mon, Feb 14, 2022, at 15:22, Roman Simakov wrote: > > It's the default for a database. It cannot be NULL. Initial NULL means > > INVOKER. > > > > пн, 14 февр. 2022 г. в 16:43, Jiří Činčura : > >> > >> Hi *, > >> > >> How can I drop SQL SECURITY on database, aka setting the RDB$SQL_SECURITY > >> column back to "null"? > >> > >> -- > >> Mgr. Jiří Činčura > >> https://www.tabsoverspaces.com/ > >> > >> > >> Firebird-Devel mailing list, web interface at > >> https://lists.sourceforge.net/lists/listinfo/firebird-devel > > > > > > > > -- > > Роман Симаков > > > > > > Firebird-Devel mailing list, web interface at > > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] DROP SQL SECURITY for trigger
It's possible to alter a trigger partly, alter an option of a trigger. SQL SECURITY is one of such options. For procedures you must specify "complete definition" every time. пн, 14 февр. 2022 г. в 12:35, Jiří Činčura : > > Hi *, > > I'm wondering why is there explicit "DROP SQL SECURITY" for triggers, while > i.e. procedures use regular "alter" statement? > > -- > Mgr. Jiří Činčura > https://www.tabsoverspaces.com/ > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Dropping database default SQL SECURITY
It's the default for a database. It cannot be NULL. Initial NULL means INVOKER. пн, 14 февр. 2022 г. в 16:43, Jiří Činčura : > > Hi *, > > How can I drop SQL SECURITY on database, aka setting the RDB$SQL_SECURITY > column back to "null"? > > -- > Mgr. Jiří Činčura > https://www.tabsoverspaces.com/ > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] obj_database value in FB3 and FB4
I hope it can be handled while restoring. I have no good idea how to deal with differences between 3 and 4. It's possible to make a gap in 5 and handle it while restoring as well. ср, 26 янв. 2022 г. в 17:03, Jiří Činčura : > > > They are also reinserted without changes in backup/restore... > > Even better, not! > > This looks like a bug to me. Should I create an issue? > > -- > Mgr. Jiří Činčura > https://www.tabsoverspaces.com/ > > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] obj_database value in FB3 and FB4
It's the first constant for DDL privileges and other constants must be before it. See obj.h. Adding objects for DML privileges moved it further. Perhaps we should have reserved several items but ... Maybe to leave a comment in code for the next move to make a gap? ср, 26 янв. 2022 г. в 16:01, Jiří Činčura : > > Hi *, > > Is there a reason why the obj_database (and items following) changed values > between FB3 and FB4? This sucks a lot. Now when reading i.e. from > RDB$USER_PRIVILEGES.RDB$OBJECT_TYPE one has to distinguish between FB3 and > FB4 when handling what object it is. > > -- > Mgr. Jiří Činčura > https://www.tabsoverspaces.com/ > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Maximum number of HashSlots
пн, 6 дек. 2021 г. в 23:07, Dimitry Sibiryakov : > > Roman Simakov wrote 06.12.2021 18:50: > > It looks so and we've returned to my original question) > >Actually no. It is still unknown which version of Firebird 3.0 you watched > and which hash was used. Actually, yes. I use the up-to-date version with backported implementation identical to master. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Maximum number of HashSlots
пн, 6 дек. 2021 г. в 15:26, Dimitry Sibiryakov : > > Roman Simakov wrote 06.12.2021 13:16: > > Can you point me to the commit? I see no difference between 3 and 5 > > versions of InternalHash::hash implementation. > >It looks like using of CRC32C was backported into some 3.0.X. Original > commit > was > https://github.com/FirebirdSQL/firebird/commit/a1d55c3e6e07714a5fcbc5034311ae35c6d2f95f Yep. It looks so and we've returned to my original question) -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Maximum number of HashSlots
Can you point me to the commit? I see no difference between 3 and 5 versions of InternalHash::hash implementation. пн, 6 дек. 2021 г. в 14:35, Dimitry Sibiryakov : > > Dimitry Sibiryakov wrote 06.12.2021 12:26: > > Hashing function has been changed to one with supposedly "more even" > > distribution of values. > >BTW a side effect of this change is that number of slots doesn't have to be > prime anymore. > > -- >WBR, SD. > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Роман Симаков Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Maximum number of HashSlots
пн, 6 дек. 2021 г. в 14:08, Dimitry Sibiryakov : > > Roman Simakov wrote 06.12.2021 10:24: > > We can see: > > Hash slots: 65521, Hash lengths (min/avg/max):0/ 13/ 30 > > Under heavy load. > >On which version do you see it? 3.0. Were there any changes in 4 or 5? >As long as the first number is zero increasing number of slots won't help. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Maximum number of HashSlots
пн, 6 дек. 2021 г. в 12:47, Alex Peshkoff via Firebird-devel : > > On 12/6/21 12:24, Roman Simakov wrote: > > Hello! > > > > Isn't it time to increase the maximum number of HashSlots and make it > > ULONG instead of USHORT? We can see: > > Hash slots: 65521, Hash lengths (min/avg/max):0/ 13/ 30 > > Under heavy load. > > > > In FB5? I doubt it's acceptable in FB3 but would be nice in FB4 as well. In FB5 definitely. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Maximum number of HashSlots
Hello! Isn't it time to increase the maximum number of HashSlots and make it ULONG instead of USHORT? We can see: Hash slots: 65521, Hash lengths (min/avg/max):0/ 13/ 30 Under heavy load. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] INF_database_info
Hello! isql after set stat returns some statistics of a query execution like Current memory = 10647136 Delta memory = 148432 Max memory = 10728544 Elapsed time = 162.476 sec Cpu = 0.000 sec Buffers = 1024 Reads = 332280 Writes = 0 Fetches = 70584692 Actually the server gets it from the function INF_database_info using dbb_stats. For SuperServer architecture it includes concurrent changes but for Classic it does not. At least it's inconsistent. Also note the statistics is shown after query execution and I guess a user expects it to be related to the query. I think it's more correct to use att_stats here. Even taking into account legacy clients. att_stats like dbb_stats for Classic mode. Also note that for information like isc_info_update_count we do use att_stats. Another option is to add att_stats as well and show them in isql if stat ON. Or even add new option into isql like ATT_STAT ON/OFF. Opinions? -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ODP: SQL profiler
вт, 9 нояб. 2021 г. в 14:30, Adriano dos Santos Fernandes : > > Currently it's only possible via database triggers making others > attachments start they own sessions. > > > > It would be nice to have it implemented. > > > > Agree. Are you going to do it? > Please note I'm refactoring the profiler branch, making it via a plugin. OK. > But a thing didn't changed. There could only be a single active profile > session per attachment. > > When profiling others attachment it's not clear for me if multiple > active sessions per attachment should be allowed or if the remote > session should count as active session for the initiator or the being > profiled attachment. I think it's not critical to have multiple active sessions. I hardly can imagine multiple developers profiling the same session. So an attached connection can be the owner of the profile data. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ODP: SQL profiler
Hello! There are 2 threads: PSQL profiler and SQL profiler. I've decided to continue here) It seems to me that profiling only the same attachment is not as useful as it could be. Imagine a working application. It's ready but you want to know where it spends time. You attach to the server by profiler and run profiling session in *your* attachment but specifying the attachment ID of the target. Then you run a command in the application and see collected data in the profiling attachment. Like the trace does where you can trace any activity. It would be nice to have it implemented. -- Best, Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
Hello!
PROPOSAL v.3===
GOALS
==
1) Extend the current limits on database size
2) Keep non active parts of a database on slow disks (having big volume)
3) Split indices from the database
etc
SYNTAX
===
Note: MAIN - is a name of the basic database file.
1. CREATE TABLESPACE FILE '/path/to/file'
2. ALTER TABLESPACE SET FILE TO '/path/to/file'
FILE can contain either an absolute path or a path relative to the
primary database file.
3. DROP TABLESPACE [INCLUDING CONTENTS]
If the tablespace contains some database objects the behaviour depends
on INCLUDING CONTENTS clause. if it is specified all database objects
in the tablespace will be dropped as well. Otherwise there will be an
error.
4. CREATE TABLE ...
(
...
FIELD TYPE CONSTRAINT ... USING INDEX ... [ [IN] TABLESPACE {
| PRIMARY} ] -- field constraint tablespace
...
CONSTRAINT ... USING INDEX ... [ [IN] TABLESPACE { | PRIMARY}
] -- table constraint tablespace
...
)
[ [IN] TABLESPACE ]
if the tablespace name is not specified PRIMARY will be used as
default tablespace for the table data.
A constraint index will use the tablespace of the table if TABLESPACE
is omitted.
5. ALTER TABLE SET TABLESPACE [TO] { | PRIMARY}
Data of the table will be moved to the specified tablespace or the
primary database.
The indices of the table without explicit TABLESPACE will be moved as
well (as if they were created now).
(And I cannot think up a good option for it)
6. CREATE INDEX … [ [IN] TABLESPACE { | PRIMARY} ]
The index will be created in the specified tablespace or the primary database.
If tablespace is omitted the index will be created in the tablespace
of the table.
7. ALTER INDEX SET TABLESPACE [TO] { | PRIMARY}
Data of the index will be moved to the specified tablespace or the
primary database.
ODS CHANGES
=
A new table RDB$TABLESPACES:
RDB$TABLESPACE_ID - SMALLINT
RDB$TABLESPACE_NAME - CHAR (63)
RDB$SECURITY_CLASS - CHAR (63)
RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80
RDB$OWNER_NAME - CHAR (63)
RDB$FILE_NAME - VARCHAR (255)
A new field in RDB$INDICES:
RDB$TABLESPACE_NAME - CHAR (63)
A new field in RDB$RELATION_FIELDS:
RDB$TABLESPACE_NAME - CHAR (63)
New fields in RDB$RELATIONS:
RDB$TABLESPACE_NAME - CHAR (63)
RDB$POINTER_PAGE - INTEGER
RDB$ROOT_PAGE - INTEGER
Add page space id to page number in ods.h:index_root_page.
Data type of PageSpaceID changes from USHORT to ULONG.
UTILITIES
Logical backup
gbak -b works as usual for now. It gets data from a database
transparently working with tablespaces.
Logical restore
gbak -c -TABLESPACE_MAP(PING_FILE) /path/to/file/map/tablespaces
The option TABLESPACE_MAP(PING_FILE) specifies a path to the file
which maps tablespace names on file names. For example,
TS1 /path/to/tablespace1
TS2 /path/to/tablespace2
or directly in the command line
gbak -c -ts =/path/to/tablespace1 -ts =...
It allows you to restore tablespace contents to new places.
If the option is not specified gbak will use old locations for every tablespace.
The initial implementation requires explicitly specifying all
tablespace files. Later we can add merging tablespaces or default
original paths.
SOME DETAILS
=
pag_header in every tablespace is reserved and may be replaced by a
new page type.
pag_scns and pag_pip are located in every tablespace
pag_root is located in the tablespace where a table is located
It's possible to create up to 253 tablespaces.
nbackup support is postponed.
Location of BLOB fields is postponed.
Page size is identical for every tablespace and the primary database.
Tablespaces are excluded from replication because it's physical layout
of a database and a replica may have another one.
==END=========
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 13 окт. 2021 г. в 15:25, Dimitry Sibiryakov : > Roman Simakov wrote 13.10.2021 14:20: > > I suppose you mean async replication. I guess like other errors, no? > >No, I meant exactly synch replication. "Other errors" in such cases > leads to > transaction rollback on both sides. Can tablespace moving to be rolled > back? > For moving relation data pages I think yes. The transaction will be rolled back and the old data pages continue using. I'm not sure about index data. We'll check it. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 13 окт. 2021 г. в 13:15, Dimitry Sibiryakov : >About moving of tablespaces and objects between tablespaces: on primary > site > it succeeded, on synchronous replica it failed (and vice versa). How such > situation is going to be handled? > I suppose you mean async replication. I guess like other errors, no? -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 18:27, Dimitry Sibiryakov : > > Roman Simakov wrote 12.10.2021 17:16: > > I'm not sure that keeping page numbers in a transactional relation is > > a really excellent idea. > >But you added it into RDB$RELATIONS table, no?.. It does not make sense to discuss it without code. Shortly, IRT changes after moving index data pages. RDB$PAGES changes after moving relation data pages. RDB$PAGES is the only system table handling in the system transaction. But it was necessary to have a way for transactional reading where the moved data is located. RDB$RELATION is that transactional storage. More details you will find in PR and then we will be able to discuss it. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 18:03, Dimitry Sibiryakov : > > Roman Simakov wrote 12.10.2021 16:55: > >> If this field is needed at all. Still there is no answer why tablespace > >> cannot be identified by name only except "it was done this way 40 years > >> ago". > > Why not to suggest PR for this refactoring? > >I understand that you insist on merging your existing code "as is" but I > think that it is better to write a right code from the beginning than refactor > it later. I do not insist on including "as is" and we will adjust our code after this discussion to be as close as possible (ideally equal) in syntax, ODS and other. That's why we are discussing it. But I see no problem with the current implementation. You see. I assume that's historically but I have no answer why it was done in such way. I'm sure such experiments quite reasonable but let's not mix everything in one PR. I'm not sure that keeping page numbers in a transactional relation is a really excellent idea. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 18:07, Vlad Khorsun : > > 12.10.2021 17:53, Roman Simakov wrote: > > вт, 12 окт. 2021 г. в 13:11, Vlad Khorsun : > >> > >> 12.10.2021 9:09, Roman Simakov wrote: > >>> пн, 11 окт. 2021 г. в 23:03, Vlad Khorsun >>> <mailto:hv...@optima.com.ua>>: > > ... > > >>> > But MAIN exactly specifies the database itself. We especially > >>> have removed DEFAULT from the new version of the proposal > >>> because it's > >>> > better to explicitly require a tablespace name in the beginning. > >>> Later we can add defaults. > >>> > >>> I hope you don't require to use TABLESPACE clause every time ? > >>> If yes, you > >>> should define defaults anyway ;) > >>> > >>> > >>> Definitely not. > >> > >> Hmm... when object is creating and tablespace was not specified, we > >> must use something > >> (by default). Obvious choice is to use 'SYSTEM' tablespace, correct ? > > > > For tablespace yes. > >For tables, perhaps ? Yes. Sorry. > > For indices the default tablespace is a tablespace > > of its table. > >Sure. I meant tables (and other "independent" objects, if any). > > ... > > >>> After DY's statement re. tablespace per partition, we should > >>> consider > >>> ability to create much more tablespaces. > >>> > >>> > >>> I see no problem with increasing the limit. I see problems with reducing > >>> it (someone may use them). So let's start from a small > >>> number 63. When we implement partitions we increase it more consciously. > >> > >> I speak about data type used in ODS for tablespace ID. It seems INT > >> should be used, > >> not SMALLINT. > > > > You suggest extending it in the PR or we can put it off? > >In the PR. It costs nothing but allows to avoid additional ODS changes. OK. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 18:01, Adriano dos Santos Fernandes : > > In the context of Firebird, IMO it seems weird to have a SYSTEM > tablespace where user objects are put. > > It makes it appear that it would be related to system objects. > > I prefer PRIMARY. I see that most people prefer PRIMARY so I think we include it in the next version of the proposal. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 13:17, Dimitry Sibiryakov : > > Vlad Khorsun wrote 12.10.2021 12:10: > >> I see no problem with increasing the limit. I see problems with reducing it > >> (someone may use them). So let's start from a small number 63. When we > >> implement partitions we increase it more consciously. > > > >I speak about data type used in ODS for tablespace ID. It seems INT > > should be > > used, not SMALLINT. > >If this field is needed at all. Still there is no answer why tablespace > cannot be identified by name only except "it was done this way 40 years ago". Why not to suggest PR for this refactoring? -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 13:11, Vlad Khorsun :
>
> 12.10.2021 9:09, Roman Simakov wrote:
> > пн, 11 окт. 2021 г. в 23:03, Vlad Khorsun > <mailto:hv...@optima.com.ua>>:
> >
> > 11.10.2021 21:23, Roman Simakov wrote:
> > > I'd be happy to agree. Actually we took a look at Oracle syntax. The
> > fact is that DEFAULT means different things. For example,
> > > DEFAULT tablespace for indices is the tablespace of its table.
> > That's why DEFAULT is not such an obvious name as we want it
> > to be.
> >
> > This is matter of documentation, IMHO. BTW, why you don't like
> > ORACLE's way ?
> > It looks logical for me. If you want to avoid ambiguity we could
> > introduce
> > special syntax for the table's sub-objects (blob fields, indices,
> > constraints),
> > say use keyword TABLE or PARENT as tablespace name, for ex:
> >
> > CREATE INDEX … AT TABLESPACE { | DEFAULT | TABLE}, or more
> > natural
> >
> > CREATE INDEX … AT DEFAULT | TABLE TABLESPACE
> > CREATE INDEX … AT TABLESPACE
> >
> >
> > I had such an idea but didn't want to make up our own way.
> > If we go Oracle way and use DEFAULT we won't be able to move index data to
> > the main database for indices for a table at;) a
> > tablespace. I.e. we can move either to a named tablespace or to a default
> > (table's) tablespace.
>
>Now I understand you better, thanks. But I still against word MAIN :)
OK) It won't be MAIN)
> > It seems Oracle uses the name SYSTEM for the main database. Do you like it?
> > Anyway the main database tablespace has to have a name.
>
>'SYSTEM' is good choice. All system relations is here. So, engine will
> always create
> tablespace with name 'SYSTEM', and put all system relations and TIP here,
> correct ?
> 'SYSTEM' tablespace can't be renamed and could (should?) be marked as system
> one.
>
> > The question is what name?
> > MAIN
> > PRIMARY
> > SYSTEM
> > DATABASE TABLESPACE
> > DATABASE
>
>SYSTEM (best) or PRIMARY, imho.
Oracle's SYSTEM tablespace contains server-wide objects but not only
database ones. But for now I agree we have two the most suitable
options: SYSTEM, PRIMARY.
> > in this case, when table's table space is changed, all dependent object
> > should
> > be changed accordingly
> >
> >
> > What do you mean saying "changed"? Now we explicitly set the tablespace
> > name for an index and when a table is moving leave the index
> > where it was. So subobjects are not bind to the parent. So does Oracle. Do
> > you suggest moving all dependent objects implicitly? So
> > the question is to bind or not to bind?
>
>Yes. I assumed sub-objects placed in the same tablespace as object itself
> should be
> moved all together (i.e. bound). But now I think there should be option to
> [not]move
> sub-objects when object moved into new tablespace.
I think the option is good. We'll add in in the 3th version of the proposal.
> > > But MAIN exactly specifies the database itself. We especially have
> > removed DEFAULT from the new version of the proposal
> > because it's
> > > better to explicitly require a tablespace name in the beginning.
> > Later we can add defaults.
> >
> > I hope you don't require to use TABLESPACE clause every time ? If
> > yes, you
> > should define defaults anyway ;)
> >
> >
> > Definitely not.
>
>Hmm... when object is creating and tablespace was not specified, we must
> use something
> (by default). Obvious choice is to use 'SYSTEM' tablespace, correct ?
For tablespace yes. For indices the default tablespace is a tablespace
of its table.
(Definitely not related to "I hope you don't require to use TABLESPACE
clause every time")
> > The point is that we cannot use DEFAULT as a name for the main database. If
> > so I decided not to introduce DEFAULT
> > keyword at all. We can add it when we understand how it works and what
> > defaults are useful.
>
>Ok. So, we should remove all mentions of MAIN in your next version of
> proposal, correct ?
> If one need to place\move object into main database file (tablespace) name
> 'SYSTEM' should
> be used explicitly (so far).
Exactly!
> > After DY's statement re. tablespace per partition, we should
> > consider
> > ability to create much more tablespaces.
> >
> >
> > I see no problem with increasing the limit. I see problems with reducing it
> > (someone may use them). So let's start from a small
> > number 63. When we implement partitions we increase it more consciously.
>
>I speak about data type used in ODS for tablespace ID. It seems INT should
> be used,
> not SMALLINT.
You suggest extending it in the PR or we can put it off?
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
вт, 12 окт. 2021 г. в 11:56, Mark Rotteveel :
>
> On 11-10-2021 14:17, Roman Simakov wrote:
> [..]
>
> > SYNTAX
> > ===
> [..]
> > 3. *DROP TABLESPACE [INCLUDING CONTENTS]*
> >
> > If the tablespace contains some database objects the behaviour depends
> > on INCLUDING CONTENTS clause. if it is specified all database objects
> > in the tablespace will be dropped as well. Otherwise there will be an
> > error.
>
> I want to repeat my objection against using [INCLUDING CONTENTS] instead
> of {CASCADE | RESTRICT}:
>
> """
> I think using {CASCADE | RESTRICT} instead of [INCLUDING CONTENTS] would
> be better, as that fits with similar definitions in the SQL standard
> (e.g. 11.2 , and Feature F032 "CASCADE drop
> behaviour").
>
> That is, you always have to specify either CASCADE (delete everything
> contained in it), or RESTRICT (only delete if empty).
>
> However, the SQL standard also specifies that all objects then need to
> be dropped with CASCADE (so dependents are dropped as well), and
> Firebird doesn't provide such feature (Feature F032 "CASCADE drop
> behaviour") at the moment, and it is probably necessary for this to work
> correctly, otherwise objects with dependents in other tablespaces will
> result in the drop failing anyway.
>
> Personally, I could live with not providing such feature for now, and
> instead requiring the tablespace to be empty before dropping it.
> """
Sorry, but I think I've answered that question and explained.
> > 4. CREATE TABLE ...
> > (
> > ...
> > FIELD TYPE CONSTRAINT ... USING INDEX ... *TABLESPACE { |
> > MAIN}* -- field constraint tablespace
> > ...
> > CONSTRAINT ... USING INDEX ... *TABLESPACE { | MAIN}* -- table
> > constraint tablespace
> > ...
> > )
> > *TABLESPACE *
> >
> > if the tablespace name is not specified *MAIN* will be used as default
> > tablespace for the table data.
> > A constraint will use the tablespace of the table if *TABLESPACE* is
> > omitted.
>
> I assume you mean the entire clause is option, so: [TABLESPACE
> { | MAIN}]
Sure.
> > 6. CREATE INDEX … *TABLESPACE { | MAIN}*
> >
> > The index will be created in the specified tablespace or the main database.
> > If tablespace is omitted the index will be created in the tablespace of
> > the table.
>
> I assume you mean the entire clause is option, so: [TABLESPACE
> { | MAIN}]
Sure.
> > 7. ALTER INDEX *ALTER TABLESPACE { | MAIN}*
> >
> > Data of the index will be moved to the specified tablespace or the main
> > database.
> >
> > ODS CHANGES
> > =
> >
> > A new table RDB$TABLESPACES:
> >
> >RDB$TABLESPACE_ID - SMALLINT
> >RDB$TABLESPACE_NAME - CHAR (63)
> >RDB$SECURITY_CLASS - CHAR (63)
> >RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80
> >RDB$OWNER_NAME - CHAR (63)
> >RDB$FILE_NAME - VARCHAR (255)
> >
> > A new field in RDB$INDICES:
> >RDB$TABLESPACE_NAME - CHAR (63)
> >
> > A new field in RDB$RELATION_FIELDS:
> >RDB$TABLESPACE_NAME - CHAR (63)
> >
> > New fields in RDB$RELATIONS:
> >RDB$TABLESPACE_NAME - CHAR (63)
> >RDB$POINTER_PAGE - INTEGER
> >RDB$ROOT_PAGE - INTEGER
> >
> > Add page space id to page number in ods.h:index_root_page.
> >
> > UTILITIES
> > ====
> [..]
> > Logical restore
> >
> [..]
> > It allows you to restore tablespace contents to new places.
> > If the option is not specified gbak will use old locations for every
> > tablespace.
>
> I still think this should be an explicit option as well, otherwise
> surprising things - for the user - can happen when restoring a backup of
> a database you didn't know had tablespaces.
In other words due restore we require file names for the database and
for every tablespace. Sounds reasonable.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
пн, 11 окт. 2021 г. в 23:03, Vlad Khorsun :
> 11.10.2021 21:23, Roman Simakov wrote:
> > пн, 11 окт. 2021 г. в 18:42, Vlad Khorsun hv...@optima.com.ua>>:
> >
> > 11.10.2021 15:17, Roman Simakov wrote:
> > > SYNTAX
> > > ===
> > >
> > > Note: *MAIN* - is a name of the basic database file.
> >
> > Please, use *DEFAULT* for default (main) tablespace at "main"
> database file.
> > It is much more consistent with SQL and allows to avoid new
> unnecessary keyword.
> >
> >
> > I'd be happy to agree. Actually we took a look at Oracle syntax. The
> fact is that DEFAULT means different things. For example,
> > DEFAULT tablespace for indices is the tablespace of its table. That's
> why DEFAULT is not such an obvious name as we want it to be.
>
>This is matter of documentation, IMHO. BTW, why you don't like ORACLE's
> way ?
> It looks logical for me. If you want to avoid ambiguity we could introduce
> special syntax for the table's sub-objects (blob fields, indices,
> constraints),
> say use keyword TABLE or PARENT as tablespace name, for ex:
>
>CREATE INDEX … AT TABLESPACE { | DEFAULT | TABLE}, or more
> natural
>
>CREATE INDEX … AT DEFAULT | TABLE TABLESPACE
>CREATE INDEX … AT TABLESPACE
>
I had such an idea but didn't want to make up our own way.
If we go Oracle way and use DEFAULT we won't be able to move index data to
the main database for indices for a table at;) a tablespace. I.e. we can
move either to a named tablespace or to a default (table's) tablespace.
It seems Oracle uses the name SYSTEM for the main database. Do you like it?
Anyway the main database tablespace has to have a name. The question is
what name?
MAIN
PRIMARY
SYSTEM
DATABASE TABLESPACE
DATABASE
but definitely it could not be DEFAULT because DEFAULT meaning depends on
the context.
>
> in this case, when table's table space is changed, all dependent object
> should
> be changed accordingly
>
What do you mean saying "changed"? Now we explicitly set the tablespace
name for an index and when a table is moving leave the index where it was.
So subobjects are not bind to the parent. So does Oracle. Do you suggest
moving all dependent objects implicitly? So the question is to bind or not
to bind?
>
> // let me use AT until we agreed to use IN ;)
>
I'd like to get an answer from native speakers, but I think it's like a
database or file (in a database, in a file).
>
> > But MAIN exactly specifies the database itself. We especially have
> removed DEFAULT from the new version of the proposal because it's
> > better to explicitly require a tablespace name in the beginning. Later
> we can add defaults.
>
>I hope you don't require to use TABLESPACE clause every time ? If yes,
> you
> should define defaults anyway ;)
>
Definitely not. The point is that we cannot use DEFAULT as a name for the
main database. If so I decided not to introduce DEFAULT keyword at all. We
can add it when we understand how it works and what defaults are useful.
>
> > But if you have a good idea how to resolve this issue we will be happy
> to use it in the proposal.
>
>See above.
>
> > > 1. *CREATE TABLESPACE FILE '/path/to/file'*
> > >
> > > 2. *ALTER TABLESPACE FILE '/path/to/file'*
> >
> > In DDL, ALTER usually combined with ADD | SET | DROP, so let
> follow this
> > convention. I.e. ALTER TABLESPACE SET FILE '/path/to/file'
> >
> >
> > Oracle syntax:
> >
> > ALTER TABLESPACE users
> > RENAME DATAFILE '/u02/oracle/rbdb1/user1.dbf',
> > '/u02/oracle/rbdb1/user2.dbf'
> > TO '/u02/oracle/rbdb1/users01.dbf',
> > '/u02/oracle/rbdb1/users02.dbf';
> >
> > I don't like it and we have only one data file actually and have no
> preferences here. We can make [SET] optional.
>
>ALTER somethings SET property TO value - looks as natural way to speak.
>
I don't mind but native speakers - say your words :)
>
>Yes, we may introduce RENAME and so on for every property of every
> altering
> object, but it looks too noisy for me.
>
> > > 4. CREATE TABLE ...
> > > (
> > > ...
> > > FIELD TYPE CONSTRAINT ... USING INDEX ... *TABLESPACE {
> | MAIN}* -- field constraint tablespace
> > > ...
> > > CONSTRAINT ... USING INDEX ... *TABLESPACE { | MAIN}*
> -- table constraint tablespace
> >
Re: [Firebird-devel] Tablespaces proposal
> > > >Two ID's is reserved for DEFAULT and TEMPORARY tablespaces, correct ? > > I would reserve some more ID's for future system usage. I don't see it as > > limitation for end users. > > For regular tablespaces (created explicitly) - sure. But if we think > about automatically created partitions, even 253 tablespaces may become > a sad limit. > We can wait partitions) > > >> nbackup support is postponed. > > > >At this stage I agree, but this must be 1st goal after initial > > implementation, IMO. > > The first PR may come without it, but I'm against releasing it until > nbackup is supported. > Good! -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
пн, 11 окт. 2021 г. в 18:42, Vlad Khorsun :
> 11.10.2021 15:17, Roman Simakov wrote:
> > SYNTAX
> > ===
> >
> > Note: *MAIN* - is a name of the basic database file.
>
>Please, use *DEFAULT* for default (main) tablespace at "main" database
> file.
> It is much more consistent with SQL and allows to avoid new unnecessary
> keyword.
>
I'd be happy to agree. Actually we took a look at Oracle syntax. The fact
is that DEFAULT means different things. For example, DEFAULT tablespace for
indices is the tablespace of its table. That's why DEFAULT is not such an
obvious name as we want it to be. But MAIN exactly specifies the database
itself. We especially have removed DEFAULT from the new version of the
proposal because it's better to explicitly require a tablespace name in the
beginning. Later we can add defaults.
But if you have a good idea how to resolve this issue we will be happy to
use it in the proposal.
>
> > 1. *CREATE TABLESPACE FILE '/path/to/file'*
> >
> > 2. *ALTER TABLESPACE FILE '/path/to/file'*
>
>In DDL, ALTER usually combined with ADD | SET | DROP, so let follow this
> convention. I.e. ALTER TABLESPACE SET FILE '/path/to/file'
>
Oracle syntax:
ALTER TABLESPACE users
RENAME DATAFILE '/u02/oracle/rbdb1/user1.dbf',
'/u02/oracle/rbdb1/user2.dbf'
TO '/u02/oracle/rbdb1/users01.dbf',
'/u02/oracle/rbdb1/users02.dbf';
I don't like it and we have only one data file actually and have no
preferences here. We can make [SET] optional.
> > 4. CREATE TABLE ...
> > (
> > ...
> > FIELD TYPE CONSTRAINT ... USING INDEX ... *TABLESPACE { |
> MAIN}* -- field constraint tablespace
> > ...
> > CONSTRAINT ... USING INDEX ... *TABLESPACE { | MAIN}* -- table
> constraint tablespace
> > ...
> > )
> > *TABLESPACE *
>
>Should we consider optional AT before TABLESPACE in all CREATE
> [AT] TABLESPACE
> statements ? And, perhaps, optional TO in ALTER SET
> TABLESPACE [TO] .
> Below I'll use both AT and TO, but not insist on it.
>
Oracle doesn't use prepositions here.
> > if the tablespace name is not specified *MAIN* will be used as default
> tablespace for the table data.
>
>DEFAULT, not MAIN, please.
>
See above.
> > A constraint will use the tablespace of the table if *TABLESPACE* is
> omitted.
> >
> > 5. ALTER TABLE *ALTER TABLESPACE { | MAIN}*
> >
> > Data of the table will be moved to the specified tablespace or the main
> database.
>
>This statement not alters tablespace itself, i.e. there should be SET
> (and DEFAULT):
>
>ALTER TABLE SET TABLESPACE [TO] | DEFAULT
>
Oracle uses ALTER TABLE MOVE TABLESPACE and without
any prepositions as well.
However I agree. SET looks more appropriate here than ALTER. But I still
see no point in prepositions.
Regarding DEFAULT see above.
>
> > 6. CREATE INDEX … *TABLESPACE { | MAIN}*
> >
> > The index will be created in the specified tablespace or the main
> database.
> > If tablespace is omitted the index will be created in the tablespace of
> the table.
>
>CREATE INDEX … [AT] TABLESPACE { | DEFAULT}
>
The same reference to Oracle.
> > 7. ALTER INDEX *ALTER TABLESPACE { | MAIN}*
> >
> > Data of the index will be moved to the specified tablespace or the main
> database.
>
>ALTER INDEX SET TABLESPACE [TO] { | DEFAULT}
>
Agreed about SET.
Prepositions and DEFAULT see above.
>
> It seems we missed
>
>ALTER TABLE ADD CONSTRAINT ... [AT] TABLESPACE { | DEFAULT},
> and
>ALTER TABLE ALTER COLUMN ... SET TABLESPACE [TO] { | DEFAULT}
>
Thank you. I agree with the same remarks.
>
> > SOME DETAILS
> > =
> >
> > pag_header in every tablespace is reserved and may be replaced by a
> > new page type.
>
>You mean page zero, which is currently always pag_header. I see no
> reason
> to change this, so far. Header page uses to describe properties of database
> and could be extended to describe pagespace when necessary.
>
I have no objections. I suppose it will be used for nbackup implementation
at least.
> > pag_scns and pag_pip are located in every tablespace
> > pag_root is located in the tablespace where a table is located
> >
> > It's possible to create up to 253 tablespaces.
>
>Two ID's is reserved for DEFAULT and TEMPORARY tablespaces, correct ?
>
Currently it's like
const USHORT DB_PAGE_SPACE = 1;
// .. here all tablespace IDs. Keep TRANS_PAGE_SPACE right after DB_PAGE_SPACE
const USHORT TRANS_PAGE_SPACE = 255; // is not used for tablespace id
I would reserve some more ID's for future system usage. I don't see it as
> limitation for end users.
>
I see no problem with it. At least for the first version we can use 63
tablespaces for example (64 including the main database) .
We add another constant for the border of the available IDs.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
пн, 11 окт. 2021 г. в 16:14, Dimitry Sibiryakov : > > Roman Simakov wrote 11.10.2021 14:52: > >>> Add page space id to page number in ods.h:index_root_page. > >>> pag_root is located in the tablespace where a table is located > >> Why is this difference? What does prevent you from putting irp into the > >> tablespace where index is located? > > IRP describes every index of a table and belongs to the table. > >Cannot be tablespace identification loaded from RDB$INDEXES? >Adding it into irp will increase size of irt_repeat by 1/3 decreasing limit > of indexes per table, no? I'm sure the answer is the same as why the root page cannot be loaded from RDB$INDEXES (idx_root). -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
пн, 11 окт. 2021 г. в 15:27, Dimitry Sibiryakov :
>
> Roman Simakov wrote 11.10.2021 14:17:
> > FIELD TYPE CONSTRAINT ... USING INDEX ... TABLESPACE { | MAIN} --
> > field constraint tablespace
>
>What's the point of using "MAIN" here? Whole TABLESPACE clause cannot be
> omitted?
>
> > Add page space id to page number in ods.h:index_root_page.
>
> > pag_root is located in the tablespace where a table is located
>Why is this difference? What does prevent you from putting irp into the
> tablespace where index is located?
IRP describes every index of a table and belongs to the table.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
Here is the second version of the proposal. It's taken into account all
agreements we made during discussion and we'll do it in this way if there
are no objections.
PROPOSAL==
GOALS
==
1) Extend the current limits on database size
2) Keep non active parts of a database on slow disks (having big volume)
3) Split indices from the database
etc
SYNTAX
===
Note: *MAIN* - is a name of the basic database file.
1. *CREATE TABLESPACE FILE '/path/to/file'*
2. *ALTER TABLESPACE FILE '/path/to/file'*
FILE can contain either an absolute path or a relative to the main database
file.
3. *DROP TABLESPACE [INCLUDING CONTENTS]*
If the tablespace contains some database objects the behaviour depends
on INCLUDING CONTENTS clause. if it is specified all database objects
in the tablespace will be dropped as well. Otherwise there will be an
error.
4. CREATE TABLE ...
(
...
FIELD TYPE CONSTRAINT ... USING INDEX ... *TABLESPACE { | MAIN}*
-- field constraint tablespace
...
CONSTRAINT ... USING INDEX ... *TABLESPACE { | MAIN}* -- table
constraint tablespace
...
)
*TABLESPACE *
if the tablespace name is not specified *MAIN* will be used as default
tablespace for the table data.
A constraint will use the tablespace of the table if *TABLESPACE* is
omitted.
5. ALTER TABLE *ALTER TABLESPACE { | MAIN}*
Data of the table will be moved to the specified tablespace or the main
database.
6. CREATE INDEX … *TABLESPACE { | MAIN}*
The index will be created in the specified tablespace or the main database.
If tablespace is omitted the index will be created in the tablespace of the
table.
7. ALTER INDEX * ALTER TABLESPACE { | MAIN}*
Data of the index will be moved to the specified tablespace or the main
database.
ODS CHANGES
=
A new table RDB$TABLESPACES:
RDB$TABLESPACE_ID - SMALLINT
RDB$TABLESPACE_NAME - CHAR (63)
RDB$SECURITY_CLASS - CHAR (63)
RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80
RDB$OWNER_NAME - CHAR (63)
RDB$FILE_NAME - VARCHAR (255)
A new field in RDB$INDICES:
RDB$TABLESPACE_NAME - CHAR (63)
A new field in RDB$RELATION_FIELDS:
RDB$TABLESPACE_NAME - CHAR (63)
New fields in RDB$RELATIONS:
RDB$TABLESPACE_NAME - CHAR (63)
RDB$POINTER_PAGE - INTEGER
RDB$ROOT_PAGE - INTEGER
Add page space id to page number in ods.h:index_root_page.
UTILITIES
Logical backup
gbak -b works as usual for now. It gets data from a database
transparently working with tablespaces.
Logical restore
gbak -c -TABLESPACE_MAP(PING_FILE) /path/to/file/map/tablespaces
The option TABLESPACE_MAP(PING_FILE) specifies a path to the file
which maps tablespace names on file names. For example,
TS1 /path/to/tablespace1
TS2 /path/to/tablespace2
or directly in the command line
gbak -c -ts =/path/to/tablespace1 -ts =...
It allows you to restore tablespace contents to new places.
If the option is not specified gbak will use old locations for every
tablespace.
The initial implementation requires explicitly specifying all tablespace
files. Later we can add merging tablespaces or default original paths.
SOME DETAILS
=
pag_header in every tablespace is reserved and may be replaced by a
new page type.
pag_scns and pag_pip are located in every tablespace
pag_root is located in the tablespace where a table is located
It's possible to create up to 253 tablespaces.
nbackup support is postponed.
Location of BLOB fields is postponed.
Page size is identical for every tablespace and the main database.
==END=====
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 17:12, Mark Rotteveel :
>
> On 07-10-2021 15:54, Roman Simakov wrote:
> > чт, 7 окт. 2021 г. в 11:14, Mark Rotteveel :
> >> I think it will save a lot of headaches if ALTER DATABASE {BEGIN|END}
> >> BACKUP can do that for all tablespaces at once. It would be a lot
> >> simpler than having to arrange that per tablespace. That is not to say
> >> there might be a use case to do it per tablespace file, but I think
> >> simple options should be preferred initially.
> >
> > Perhaps. It depends of what we will have to backup up: the whole set
> > of tablespaces or not. I think there must be options.
>
> Backing up only one tablespace could lead to inconsistent restores
> though. To me, that sounds like a power-feature that doesn't belong in a
> first version.
Restoring one tablespace makes no sense. But it might be useful to
skip some tablespaces: indices, logs, unimportant tables.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 11:47, Dimitry Sibiryakov : > > Roman Simakov wrote 06.10.2021 21:49: > >> How are data moved? > >> > >> Under transaction control? > >> > >> In background (and interruptible) like db crypt? > > it's a dfw operation with EX database lock. I hope to relax this > > limitation later. So there are no concurrent changes. > > 1) copy all data pages > > 2) switch RDB$POINTER_PAGE and RDB$ROOT_PAGE transactionally > > 3) Rebuild RDB$PAGES > > 4) clear old data pages (as post-dfw operation) > > > > It can be interrupted but not resumed. I'm afraid it starts from the > > beginning. > > > > I guess it's better to discuss in a context of PR. > >Tablespaces has meaning for large databases only that don't fit into single > storage (terrabytes). For such volumes this algorithm is no-go. Better not to > have move option in the initial implementation at all. Let's eat an elephant one bite at a time -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 11:45, Dimitry Sibiryakov : > > Roman Simakov wrote 07.10.2021 8:14: > > ср, 6 окт. 2021 г. в 23:57, Dimitry Sibiryakov: > >> Roman Simakov wrote 06.10.2021 22:44: > >>> Sorry I was wrong. TABLESPACE_ID is used as pagespace ID internally > >>> but this id presents at index root page for example. > >> I don't see it in ods.h:index_root_page. > > Maybe because I haven't made a PR yet) > >So you didn't describe all ODS changes you are going to make. What else did > you miss? I hope that's it. > > RDB$SYSTEM_FLAG - SMALLINT # reserved > > RDB$OFFLINE - SMALLINT # reserved for future > > implementation offline tablespaces > > RDB$READ_ONLY - SMALLINT# reserved for future > > implementation read only tablespaces > >These fields can be added later when (if) implementation require them. I'm > sure that "read only" flag belongs to the file header, not TS description > (unless you are going to create them in already RO state which is pointless). It's hard to predict the full set of fields for future use. I try to reduce a number of ODS changes. But I do not insist on these fields. Let's remove them from the initial implementation. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 11:14, Mark Rotteveel :
>
> On 2021-10-06 21:37, Roman Simakov wrote:
> > ср, 6 окт. 2021 г. в 19:29, Adriano dos Santos Fernandes
> > :
>
> >> And what about nbackup? Will it create a .delta file per tablespace?
> >
> > Our implementation does not support nbackup yet. At first glance it
> > might be a special DDL operation like
> > ALTER TABLESPACE {BEGIN|END} BACKUP
>
> I think it will save a lot of headaches if ALTER DATABASE {BEGIN|END}
> BACKUP can do that for all tablespaces at once. It would be a lot
> simpler than having to arrange that per tablespace. That is not to say
> there might be a use case to do it per tablespace file, but I think
> simple options should be preferred initially.
Perhaps. It depends of what we will have to backup up: the whole set
of tablespaces or not. I think there must be options.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 11:11, Mark Rotteveel : > > On 2021-10-06 21:27, Roman Simakov wrote: > > ср, 6 окт. 2021 г. в 19:13, Mark Rotteveel : > >> On 06-10-2021 17:32, Roman Simakov wrote: > >> > 9. ALTER INDEX DROP TABLESPACE > >> > > >> > Data of the index will be moved to the main database. > >> > >> How will this work for indexes backing constraints? > > > > I see no problems with it. Could you explain what you mean? > > Sorry, I think I should have been more explicit. Is there an option to > specify the tablespace for the constraint when defining constraints, > instead of having to alter this after the fact (with potential issues of > having to either find the generated index name, or making sure you name > things explicitly). Now it's clear. Alex mentioned that. Now there is no way but it should be added. > >> I also think that using the old locations should be an explicit > >> option, > >> so if a database backup has tablespaces, and no option is specified to > >> define the tablespace mapping, or the mapping is incomplete, restore > >> should fail *before it started to write anything*. > > > > or we can restore the whole backup in the single database file without > > tablespaces at all. I.e. merge every unmapped TS into the main > > database. > > That is an interesting option as well, but I think that should be an > explicit option as well. That is, I think trying to restore a > tablespaced database should fail unless explicit configuration is in > place that specifies how to handle it (restore with filenames in backup, > explicit mapping, or 'restore into single file'). I agree. More strict rules are better in the beginning as a rule) It's easier to add defaults later than remove them) -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
чт, 7 окт. 2021 г. в 09:19, Omacht András : > > Hi Roman! > > First of all, thanks for the development and offering it to Firebird. Thanks) > Do you plan to partition the data depending on field value? Yes. We do. I hope we'll do something in the next version of RedDatabase and then share with Firebird as well. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 23:57, Dimitry Sibiryakov : > > Roman Simakov wrote 06.10.2021 22:44: > > Sorry I was wrong. TABLESPACE_ID is used as pagespace ID internally > > but this id presents at index root page for example. > >I don't see it in ods.h:index_root_page. Maybe because I haven't made a PR yet) -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
> >ID is not referenced anywhere and seems to have no purpose at all. > > Probably it could be removed and pagespace IDs will be generated internally. Sorry I was wrong. TABLESPACE_ID is used as pagespace ID internally but this id presents at index root page for example. Now I'm not sure if it's easy to break the relationship between TABLESPACE_ID and TABLESPACE_NAME. It's an effective way to refer to a tablespace. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 22:17, Dimitry Sibiryakov : > > Roman Simakov wrote 06.10.2021 20:43: > > Let me not describe every field. I hope most of them are obvious. > >No, they aren't. ok A new table RDB$TABLESPACES # keeps metadata of tablespaces. RDB$TABLESPACE_ID - SMALLINT # internally it will be pagespaceid. probably can be avoided. RDB$TABLESPACE_NAME - CHAR (63) # name of a tablespace RDB$SECURITY_CLASS - CHAR (63) # security class for tablespace RDB$SYSTEM_FLAG - SMALLINT # reserved RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80 #description of a tablespace RDB$OWNER_NAME - CHAR (63) # owner of a tablespace RDB$FILE_NAME - VARCHAR (255)# file where a tablespace data are located RDB$OFFLINE - SMALLINT # reserved for future implementation offline tablespaces RDB$READ_ONLY - SMALLINT# reserved for future implementation read only tablespaces A new field in RDB$INDICES: RDB$TABLESPACE_NAME - CHAR (63) # name of a tablespace A new field in RDB$RELATION_FIELDS: RDB$TABLESPACE_NAME - CHAR (63) # name of a tablespace New fields in RDB$RELATIONS: RDB$TABLESPACE_NAME - CHAR (63) # name of a tablespace RDB$POINTER_PAGE - INTEGER # a number of the first pointer page of a relation RDB$ROOT_PAGE - INTEGER # a number of the root page of a relation >ID is not referenced anywhere and seems to have no purpose at all. Probably it could be removed and pagespace IDs will be generated internally. >TABLESPACE_NAME being CHAR is pointless. CHAR was used in ancient times but > new fields should prefer VARCHAR. It's equal to other SQL_IDENTIFIERs >FILE_NAME limited to 255 characters is strange in modern world at least. It refers to the existing fld_file_name. I don't mind increasing it but think it's not a topic for tablespaces. > > RDB$SYSTEM_FLAG is not currently used but I suppose it would be useful one > > day. > >What for? There isn't going to be an automatically created tablespace for > internal purposes, right? Where in host filesystem would it be created?.. I've not designed it and am not designing it now. > > OFFLINE and READ_ONLY are a kind of physical properties and supposed > > to be used but not in this merge request anycase. > >Mostly I wanted to know the reason why they are SMALLINT instead of > BOOLEAN. I think you are right and it must be BOOLEAN. > > These fields are necessary for reliable implementation of moving data > > pages to another tablespace. You'll be able to understand their > > meaning in code. > >Documentation also must contain the code?.. After documentation you will ask me why they are needed) For documentation purposes I've provided a short description above. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 22:48, Dimitry Sibiryakov : > > Roman Simakov wrote 06.10.2021 21:27: > > So am I. Currently tablespace headers are just not used. > >In this case how do you determine page size inside of tablespace file? > Having > it different from main one is an essential feature for indexes. All pages share the same bdb cache. That's why they must have the same page size. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 22:01, Adriano dos Santos Fernandes : > > On 06/10/2021 15:43, Roman Simakov wrote: > > > >>> RDB$POINTER_PAGE - INTEGER > >>> RDB$ROOT_PAGE - INTEGER > >> > >>What these are for? > > > > These fields are necessary for reliable implementation of moving data > > pages to another tablespace. You'll be able to understand their > > meaning in code. > > > > How are data moved? > > Under transaction control? > > In background (and interruptible) like db crypt? it's a dfw operation with EX database lock. I hope to relax this limitation later. So there are no concurrent changes. 1) copy all data pages 2) switch RDB$POINTER_PAGE and RDB$ROOT_PAGE transactionally 3) Rebuild RDB$PAGES 4) clear old data pages (as post-dfw operation) It can be interrupted but not resumed. I'm afraid it starts from the beginning. I guess it's better to discuss in a context of PR. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 19:29, Alex Peshkoff via Firebird-devel : > > On 10/6/21 6:32 PM, Roman Simakov wrote: > > > 4. CREATE TABLE … > > TABLESPACE > > Given syntax does not let specify tablespace for constraints-related > indexes and separate fields. That's serious limitation. At least for > blobs that's IMHO needed. Originally we thought about specifying tablespaces for BLOB fields. It will be implemented later. But constraints-related fields were missed. We'll try to add it to the syntax. Thanks. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 19:29, Adriano dos Santos Fernandes :
>
> On 06/10/2021 13:12, Mark Rotteveel wrote:
> >>
> >> Logical restore
> >>
> >> gbak -c -TABLESPACE_MAP(PING_FILE) /path/to/file/map/tablespaces
> >>
> >> The option TABLESPACE_MAP(PING_FILE) specifies a path to the file
> >> which maps tablespace names on file names. For example,
> >> TS1 /path/to/tablespace1.dat
> >> TS2 /path/to/tablespace2.dat
> >>
>
> It would be good to also have relative paths support, in relation to
> main database file.
>
> In this case, restore will also restore the relative paths to the new
> location.
Good point. We'll do it such way.
> And what about nbackup? Will it create a .delta file per tablespace?
Our implementation does not support nbackup yet. At first glance it
might be a special DDL operation like
ALTER TABLESPACE {BEGIN|END} BACKUP
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 19:13, Mark Rotteveel :
>
> On 06-10-2021 17:32, Roman Simakov wrote:
> > 3. DROP TABLESPACE [INCLUDING CONTENTS]
>
> I think using {CASCADE | RESTRICT} instead of [INCLUDING CONTENTS] would
> be better, as that fits with similar definitions in the SQL standard
> (e.g. 11.2 , and Feature F032 "CASCADE drop
> behaviour").
>
> That is, you always have to specify either CASCADE (delete everything
> contained in it), or RESTRICT (only delete if empty).
>
> However, the SQL standard also specifies that all objects then need to
> be dropped with CASCADE (so dependents are dropped as well), and
> Firebird doesn't provide such feature (Feature F032 "CASCADE drop
> behaviour") at the moment, and it is probably necessary for this to work
> correctly, otherwise objects with dependents in other tablespaces will
> result in the drop failing anyway.
>
> Personally, I could live with not providing such feature for now, and
> instead requiring the tablespace to be empty before dropping it.
Original syntax and semantics have been taken from Oracle
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/sqlrf/DROP-TABLESPACE.html
It means that CASCADE handles objects dependent on objects in the
dropping tablespace or such drop can be failed as well, in spite of
INCLUDING CONTENT.
I think it would be implemented as you described like DROP ... CASCADE
but later. I see no problem with proposed syntax.
> > 6. ALTER TABLE DROP TABLESPACE
> >
> > Data of the table will be moved to the main database.
>
> Does this require a separate DROP TABLESPACE option? The same could be
> achieved with ALTER TABLE ALTER TABLESPACE ...
>
> where ... might be DEFAULT, MAIN or the explicit name of the main
> tablespace (assuming it has a name).
>
> Using ALTER TABLESPACE also makes sense, because that is what's actually
> happening.
I have no objections but it would be useful to know other opinions.
> > 9. ALTER INDEX DROP TABLESPACE
> >
> > Data of the index will be moved to the main database.
>
> How will this work for indexes backing constraints?
I see no problems with it. Could you explain what you mean?
> > UTILITIES
> >
> >
> > Logical backup
> >
> > gbak -b works as usual for now. It gets data from a database
> > transparently working with tablespaces.
> >
> > Logical restore
> >
> > gbak -c -TABLESPACE_MAP(PING_FILE) /path/to/file/map/tablespaces
> >
> > The option TABLESPACE_MAP(PING_FILE) specifies a path to the file
> > which maps tablespace names on file names. For example,
> > TS1 /path/to/tablespace1.dat
> > TS2 /path/to/tablespace2.dat
> >
> > It allows you to restore tablespace contents to new places.
> > If the option is not specified gbak will use old locations for every
> > tablespace.
>
> I'd prefer if there is (also) an option to define the mapping on the
> commandline without requiring users to first create the mapping file.
Good point. I think it would be like
gbak ... -ts TS1=/path/to/tablespace1.dat -ts TS2=/path/to/tablespace2.dat
> I also think that using the old locations should be an explicit option,
> so if a database backup has tablespaces, and no option is specified to
> define the tablespace mapping, or the mapping is incomplete, restore
> should fail *before it started to write anything*.
or we can restore the whole backup in the single database file without
tablespaces at all. I.e. merge every unmapped TS into the main
database.
> As an aside, we really need to look at how we handle commandline
> options, because having really long commandline options with no simple
> one or two character abbreviations is not very friendly IMHO. Maybe we
> should consider switching to `-v` vs `--verbose-names` convention (with
> backwards compatibility for existing options).
>
> > SOME DETAILS
> > =
> >
> > pag_header in every tablespace is reserved and may be replaced by a
> > new page type.
>
> I think it should be a separate page type.
So am I. Currently tablespace headers are just not used.
--
Roman Simakov
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
ср, 6 окт. 2021 г. в 18:56, Dimitry Sibiryakov : > > Roman Simakov wrote 06.10.2021 17:32: > > A new table RDB$TABLESPACES: > > > > RDB$TABLESPACE_ID - SMALLINT > > RDB$TABLESPACE_NAME - CHAR (63) > > RDB$SECURITY_CLASS - CHAR (63) > > RDB$SYSTEM_FLAG - SMALLINT > > RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80 > > RDB$OWNER_NAME - CHAR (63) > > RDB$FILE_NAME - VARCHAR (255) > > RDB$OFFLINE - SMALLINT > > RDB$READ_ONLY - SMALLINT > >Could you briefly describe purpose of each field? Necessity of > RDB$SYSTEM_FLAG and RDB$TABLESPACE_ID at least is questionable IMHO and I see > no > support for OFFLINE/READ ONLY options in DDL. Let me not describe every field. I hope most of them are obvious. RDB$SYSTEM_FLAG is not currently used but I suppose it would be useful one day. OFFLINE and READ_ONLY are a kind of physical properties and supposed to be used but not in this merge request anycase. > > A new field in RDB$RELATION_FIELDS: > >RDB$TABLESPACE_NAME - CHAR (63) > >Is it possible to have in a separate tablespace some FIELDS of a table? I > see > no support for it in DDLs. It's supposed to locate BLOBs in tablespaces but not in this merge request as well. It's not implemented yet because of impossibility to understand what the table BLOBs belongs to. There are some ideas to solve it but not in the initial implementation. > > New fields in RDB$RELATIONS: > > RDB$TABLESPACE_NAME - CHAR (63) > >Cannot RDB$EXTERNAL_FILE be reused? Why? To surprise users?) > > RDB$POINTER_PAGE - INTEGER > > RDB$ROOT_PAGE - INTEGER > >What these are for? These fields are necessary for reliable implementation of moving data pages to another tablespace. You'll be able to understand their meaning in code. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Tablespaces proposal
Hello, team!
As you might know Red Soft has implemented Tablespace support for
RedDatabase 4 which is based on Firebird 4 code base.
I hope we start working on a merge request for Firebird 5 or later but
previously I would like to get agreement about basic user visible
things like the syntax and ODS.
PROPOSAL==
GOALS
==
1) Extend the current limits on database size
2) Keep non active parts of a database on slow disks (having big volume)
3) Split indices from the database
etc
SYNTAX
===
1. CREATE TABLESPACE FILE '/path/to/file'
2. ALTER TABLESPACE FILE '/path/to/file'
3. DROP TABLESPACE [INCLUDING CONTENTS]
If the tablespace contains some database objects the behaviour depends
on INCLUDING CONTENTS clause. if it is specified all database objects
in the tablespace will be dropped as well. Otherwise there will be an
error.
4. CREATE TABLE …
TABLESPACE
5. ALTER TABLE ALTER TABLESPACE
Data of the table will be moved to the specified tablespace.
6. ALTER TABLE DROP TABLESPACE
Data of the table will be moved to the main database.
7. CREATE INDEX … TABLESPACE { | DEFAULT}
The index will be created:
- in the main database file if TABLESPACE is omitted.
- in the table tablespace for TABLESPACE DEFAULT.
- or in the specified tablespace.
8. ALTER INDEX ALTER TABLESPACE
Data of the index will be moved to the specified tablespace.
9. ALTER INDEX DROP TABLESPACE
Data of the index will be moved to the main database.
ODS CHANGES
=
A new table RDB$TABLESPACES:
RDB$TABLESPACE_ID - SMALLINT
RDB$TABLESPACE_NAME - CHAR (63)
RDB$SECURITY_CLASS - CHAR (63)
RDB$SYSTEM_FLAG - SMALLINT
RDB$DESCRIPTION - BLOB SUBTYPE TEXT SEGMENT SIZE 80
RDB$OWNER_NAME - CHAR (63)
RDB$FILE_NAME - VARCHAR (255)
RDB$OFFLINE - SMALLINT
RDB$READ_ONLY - SMALLINT
A new field in RDB$INDICES:
RDB$TABLESPACE_NAME - CHAR (63)
A new field in RDB$RELATION_FIELDS:
RDB$TABLESPACE_NAME - CHAR (63)
New fields in RDB$RELATIONS:
RDB$TABLESPACE_NAME - CHAR (63)
RDB$POINTER_PAGE - INTEGER
RDB$ROOT_PAGE - INTEGER
UTILITIES
Logical backup
gbak -b works as usual for now. It gets data from a database
transparently working with tablespaces.
Logical restore
gbak -c -TABLESPACE_MAP(PING_FILE) /path/to/file/map/tablespaces
The option TABLESPACE_MAP(PING_FILE) specifies a path to the file
which maps tablespace names on file names. For example,
TS1 /path/to/tablespace1.dat
TS2 /path/to/tablespace2.dat
It allows you to restore tablespace contents to new places.
If the option is not specified gbak will use old locations for every tablespace.
SOME DETAILS
=
pag_header in every tablespace is reserved and may be replaced by a
new page type.
pag_scns and pag_pip are located in every tablespace
pag_root is located in the tablespace where a table is located
It's possible to create up to 253 tablespaces.
==END=
First of all, please let me know whether you agree or not with SYNTAX
and ODS parts. Other opinions and suggestions are welcome as well.
--
Roman Simakov
https://reddatabase.ru
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] File and table sizes in Firebird 4.0
I think it's a good idea to keep not only limits but the formulas for their calculations. сб, 14 авг. 2021 г. в 10:32, Mark Rotteveel : > > On 14-08-2021 09:12, Dmitry Yemanov wrote: > > 14.08.2021 10:02, Mark Rotteveel пишет: > >> > >>> Now I see you mentioned 32 TB as the database size limit. How was it > >>> calculated? It should be the same 128 TB, AFAIK. > >> > >> The 32 TB is the old value on the page, when the entire page was for > >> Firebird 2.5. I have no idea who wrote the previous version of that > >> page or if that value was correct for Firebird 2.5. > > > > It was correct: 2^31 (page number was signed) * 16 KB = 32 TB. > > > > For FB 3.0 it should be: 2^32 * 16 KB = 64 TB. > > Ok, I have made what I think are the final updates to the page: > https://firebirdsql.org/en/firebird-technical-specifications/ > > Mark > > -- > Mark Rotteveel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Contradiction in SQL SECURITY documentation in Firebird 4 release notes
Great) So we just need to adjust documentation? вс, 9 мая 2021 г. в 20:05, Mark Rotteveel : > > On 09-05-2021 18:38, Roman Simakov wrote: > > Create a ticket, please: drop clauses for procedures, functions and > > packages and inheriting a clause for packages. > > Currently, an alter, create or alter or recreate without a SQL SECURITY > clause will implicitly drop it for procedures, functions and packages, > so I'm not sure those need an explicit drop clause. > > I have also tested it, and packages actually do inherit the SQL SECURITY > from the database. > > Mark > -- > Mark Rotteveel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Contradiction in SQL SECURITY documentation in Firebird 4 release notes
Create a ticket, please: drop clauses for procedures, functions and packages and inheriting a clause for packages. вс, 9 мая 2021 г. в 19:35, Mark Rotteveel : > > On 09-05-2021 18:24, Roman Simakov wrote: > > вс, 9 мая 2021 г. в 19:04, Mark Rotteveel : > >> Does that mean that the ALTER DATABASE SET DEFAULT SQL SECURITY > >> influences all objects without an explicit SQL SECURITY property? > > > > I've checked. It influences tables, functions and procedures. Triggers > > inherit it from tables (which already have it inherited from DB). I'm > > afraid packages do not inherit and it's probably a fault as well. > > Thanks. Will you fix it or create a ticket for it? > > >> I had assumed it would specify the default for creating a new object > >> (similar as SET DEFAULT CHARACTER SET). Can you confirm that assumption > >> was wrong? > > > > Yes. The assumption was wrong. SQL SECURITY DEFINER is default in SQL > > STANDARD. > > ALTER DATABASE SET DEFAULT SQL SECURITY DEFINER makes behaviour standard. > > Clear, thanks. I've updated it in the fblangref40 I'm working on. > > Mark > -- > Mark Rotteveel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Contradiction in SQL SECURITY documentation in Firebird 4 release notes
вс, 9 мая 2021 г. в 19:04, Mark Rotteveel : > > On 09-05-2021 17:58, Roman Simakov wrote: > > I guess this was the reason not to implement it. But objects inherit > > SQL SECURITY from the database and it might be useful to drop sql > > security in order to use database default. > > Does that mean that the ALTER DATABASE SET DEFAULT SQL SECURITY > influences all objects without an explicit SQL SECURITY property? I've checked. It influences tables, functions and procedures. Triggers inherit it from tables (which already have it inherited from DB). I'm afraid packages do not inherit and it's probably a fault as well. > I had assumed it would specify the default for creating a new object > (similar as SET DEFAULT CHARACTER SET). Can you confirm that assumption > was wrong? Yes. The assumption was wrong. SQL SECURITY DEFINER is default in SQL STANDARD. ALTER DATABASE SET DEFAULT SQL SECURITY DEFINER makes behaviour standard. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Contradiction in SQL SECURITY documentation in Firebird 4 release notes
вс, 9 мая 2021 г. в 18:46, Dmitry Yemanov : > What is the point in dropping, if AFAIU the legacy (lacking sql > security) mode is actually the same as "definer"? I guess this was the reason not to implement it. But objects inherit SQL SECURITY from the database and it might be useful to drop sql security in order to use database default. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Contradiction in SQL SECURITY documentation in Firebird 4 release notes
I don't remember why I did so but maybe it's better to allow drop sql security for any object type and remove the wrong statement at all. вс, 9 мая 2021 г. в 17:07, Mark Rotteveel : > > The 'SQL SECURITY Feature' section of the Firebird 4.0 Release Notes has > a contradictory statement. It says: > > """ > The property cannot be dropped but it can be changed from INVOKER to > DEFINER and vice versa. > """ > > However, the syntax explicitly includes `DROP SQL SECURITY` for `ALTER > TABLE` and `ALTER TRIGGER`. > > Would it be sufficient to say something like > > """ > The property cannot be dropped for procedures, functions and packages, > but it can be changed from INVOKER to DEFINER and vice versa. > """ > > Mark > -- > Mark Rotteveel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-6502) Stored procedure isn't able to execute statement 'GRANT'
Stored procedure isn't able to execute statement 'GRANT'
Key: CORE-6502
URL: http://tracker.firebirdsql.org/browse/CORE-6502
Project: Firebird Core
Issue Type: Bug
Affects Versions: 4.0 RC 1
Reporter: Roman Simakov
The engine executing GRANT from a stored procedure with SQL SECURITY DEFINER
runned from another user with EXECUTE privilege raises the error "User cannot
write to RDB$USER_PRIVILEGES"
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-6437) gfix cannot set buffers over 64000
gfix cannot set buffers over 64000 -- Key: CORE-6437 URL: http://tracker.firebirdsql.org/browse/CORE-6437 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 4.0 Beta 2 Reporter: Roman Simakov roman@roman-home % ./isql firebird/gen/Debug/firebird/bin Use CONNECT or CREATE DATABASE to specify a database SQL> create database '/tmp/buffb.fdb'; SQL> ^D% roman@roman-home % roman@roman-home % ./gfix -buffers 100 /tmp/buffb.fdb firebird/gen/Debug/firebird/bin I/O error during "read" operation for file "/tmp/buffb.fdb" -Error while trying to read from file -Bad address Firebird 3.0 works correctly -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Conversion error from string to date in Firebird 4
вт, 20 окт. 2020 г. в 14:43, Dmitry Yemanov : > > 20.10.2020 13:58, Vlad Khorsun via Firebird-devel wrote: > > >> AFAIU, it was discussed here in February 2018, thread "Valid date or > >> not". > > > > I've re-read that thread quickly and I saw nor final decision, nor > > proposition > > to change (or break) rules for traditional (legacy) date\time types > > (without TZ). > > The tracker ticket (CORE-5750) mentions the problem with spaces inside > the literal. If time parts are separated by spaces, it's hard to guess > whether the final part is TZ or milliseconds, AFAIU. The final decision > is also documented in that ticket. > > While I agree that breaking things is usually bad, time '10 20 30' or > time '10,20,30' look so terribly wrong to me, so personally I support > breaking them ;-) The same for date with commas. The only "broken" > format I agree to consider useful is '20 Oct 2020'. Given that > space-parsing problems appear inside the time part, perhaps it could be > re-allowed for dates (only). Yes! I agree. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Conversion error from string to date in Firebird 4
вт, 20 окт. 2020 г. в 14:15, Vlad Khorsun via Firebird-devel : >I've re-read that thread quickly and I saw nor final decision, nor > proposition > to change (or break) rules for traditional (legacy) date\time types (without > TZ). Me too. Generally that discussion was focused on other formats. Here I'd focus on the broken backward compatibility. I see no problem to keep it possible since we have had it already. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Conversion error from string to date in Firebird 4
вт, 20 окт. 2020 г. в 13:28, Mark Rotteveel : > > Because documentantion here > > (https://firebirdsql.org/en/firebird-date-literals/) says they are valid > > separators. > > Keep in mind, that link is an excerpt from Helen Borrie's Firebird Book > from 2004 (Firebird 1.5 era). It is descriptive of what worked at the > time, it is not prescriptive as to how Firebird is supposed to behave. Why not? For me and lots of other developers, who have been using such code, it's exactly how Firebird is supposed to behave :) In other words, what are the reasons to break the backward compatibility? -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] OO API examples for Object Pascal
пн, 17 авг. 2020 г. в 20:08, Adriano dos Santos Fernandes : > I'd say its correct place would be a new project under the FirebirdSQL > organization. > > There is no Java, .Net, etc in core, so should not be Pascal examples > there, specially, as you have figured out, the API is not easy to be > directly used by application developers. I agree -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] FB 3 & 4, new API and the ARRAY type
ср, 22 апр. 2020 г. в 21:04, Mark Rotteveel : > > On 2020-04-22 19:02, Jiří Činčura wrote: > >> Why to remove the Array type ? , I see that Postgresql still provides > >> it https://www.postgresql.org/docs/current/arrays.html > > > > Arrays in PG have better performance and the support in PL/pgSQL is > > good too. > > That doesn't mean we should just remove them, we could also improve > support for arrays in Firebird, both in PSQL, DSQL and in the > API/protocol I agree. > With regard to performance, the problem with arrays in Firebird is that > - just like blobs - they are out-of-band: you need separate requests to > get them. > > That said, I've never had the need to use arrays, and I won't miss them > if they are gone. They probably havn't been used because of weak support at SQL/PSQL level. Recently I've analyzed requirements of the Federal Tax Service of Russia to Database Server. They declared that DBMS like RedDatabase (Firebird), PostgresPro (Postgres), etc. can't handle their needs. I could agree with them about performance and maybe scalability. But regarding functional features Firebird almost fits the requirements. Supporting of ARRAYs also required. Maybe it's must be implemented in another way but before we understand it we should not neither remove it nor depricate it. At least if there are no strong reasons that block other good aims. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] TDBB_trusted_ddl flag
Hello! I've made PR https://github.com/FirebirdSQL/firebird/pull/247 I left my comments there too. Please, take a look. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] [Firebird-checkins] [FirebirdSQL/firebird] 74f481: Backport of the fix from the master branch: Now CR...
:) Certainly they are. I just tested you) ср, 27 нояб. 2019 г. в 21:50, Gabor Boros : > > 2019. 11. 27. 15:52 keltezéssel, Roman Simakov írta: > >Backport of the fix from the master branch: Now CREATE > > FUNCTION/PROCEDURE inside CREATE PACKAGE does not require CREATE > > FUNCTION/PROCEDURE privilege > > It's not really necessary since there are no packages in 3.0 but the > > patch fixes TDBB_trusted_ddl flag reset. So let it be. > > Are you sure? I used the packages feature already with 3.0. ;-) > > Gabor > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Power efficient userspace waiting with the umwait x86 instructions
Exactly. First of all it's interesting to understand an effect of implementation and then we can at least to keep it in mind. вт, 17 сент. 2019 г. в 13:58, Alex Peshkoff via Firebird-devel : > > On 16.09.2019 20:07, Leyne, Sean wrote: > > Roman, > > > >> From: Roman Simakov > >> Sent: Monday, September 16, 2019 7:51 AM > > > >> I guess it would be interesting alternative for spinlock > >> (https://kernelnewbies.org/Linux_5.3) > >> 1.6. Power efficient userspace waiting with the umwait x86 instructions > >> > >> More description is here: > >> https://lwn.net/Articles/790920/ > > It is a shame that: > > > > - will only be available on new Intel Tremont micro-architecture CPUs, > > whenever they ship > > - will not be available for AMD CPUs, the new ROME/EPYC 2 CPUs are kicking > > Intel's offering > > - will not be available via Windows API for some time > > > > I think it is worthy to create a JIRA ticket for this issue, but believe > > that it should not be implemented until Windows API support is available at > > the very least. > > That's not completely true - we can provide OS-specific implementations > for various classes, particular for syncs we always do it - critical > section in windows and mutex in posix are 2 different things from API > calls POV. Cumrently we have: > > class Spinlock : public Mutex > > and nothing prevents to keep this current (IMHO not efficient) way for > windows in case of no API. > > Need to detect CPU type on the fly and select one or other way to work > with spinlock may be a bit more problematic (we need to do it in > runtime) but must say I also do not see something tragic with it. > > > > > The articles don't mention what if applications would need to detect if > > umwait is implemented/active, and implement alternate logic if not, or will > > OSs need to have a default implementation for incompatible CPUs. > > IMHO application - doing that in OS kills performance benefits of > umwait. May be C-runtime library may help us with it... > > > > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Power efficient userspace waiting with the umwait x86 instructions
Hi, I guess it would be interesting alternative for spinlock (https://kernelnewbies.org/Linux_5.3) 1.6. Power efficient userspace waiting with the umwait x86 instructions More description is here: https://lwn.net/Articles/790920/ -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] IDE or code editor on Linux
I use QtCreator пт, 13 сент. 2019 г. в 15:33, Dimitry Sibiryakov : > >Hello, All. > >What would you recommend for editing Firebird sources on Linux? > > -- >WBR, SD. > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Difference in performance between current_timestamp and localtimestamp
OK. Thanks. I forgot that we carry ICU with server on Windows. Sorry. ср, 3 апр. 2019 г. в 18:27, Mark Rotteveel : > > On 3-4-2019 17:03, Roman Simakov wrote: > > чт, 3 янв. 2019 г. в 14:15, Adriano dos Santos Fernandes > > : > >> > >> On 30/12/2018 07:22, Mark Rotteveel wrote: > >>> > >>> The error is now: > >>> > >>> """ > >>> RAMONASun Dec 30 09:59:28 2018 > >>> ICU error (0) retrieving the system time zone (W. Europe Standard > >>> Time). Falling back to displacement. > >>> """ > >>> > >> > >> That's because Windows build is still with old ICU without time zone data. > > > > Is there any chance to fix it? > > > > This is already fixed (since before Firebird 4 Beta 1). > > Mark > -- > Mark Rotteveel > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Difference in performance between current_timestamp and localtimestamp
чт, 3 янв. 2019 г. в 14:15, Adriano dos Santos Fernandes : > > On 30/12/2018 07:22, Mark Rotteveel wrote: > > > > The error is now: > > > > """ > > RAMONASun Dec 30 09:59:28 2018 > > ICU error (0) retrieving the system time zone (W. Europe Standard > > Time). Falling back to displacement. > > """ > > > > That's because Windows build is still with old ICU without time zone data. Is there any chance to fix it? -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
Have you tried to profile? :) чт, 7 февр. 2019 г. в 18:15, Dimitry Sibiryakov : > >Hello, All. > >I somehow understand why build of current HEAD is failing with message > "Could not find > acceptable ICU library", but I wonder why it takes so much time to get this > error? More > than 15 second between running of isql and this failure on my notebook. > > -- >WBR, SD. > > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Performance - 2.5 vs 3.0 vs 4.0
чт, 24 янв. 2019 г. в 10:25, liviuslivius : > > Hi. > > I have thinked about oltp comparision, and i think that comparing e.g. > classic vs superserver on RAM disc is wrong. Why? Because read from "disc" is > as fast as read from cache. This case eliminates disc problems and allow to highlight other ones. In other words RAM simulates the very good system IO like a PCIe-SSD. > The comparision can show some problems but results must be analysed carefully. The real tuning exactly must take into account all layers including disk IO. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5955) Unable to init binreloc with ld >= 2.31
Unable to init binreloc with ld >= 2.31 --- Key: CORE-5955 URL: http://tracker.firebirdsql.org/browse/CORE-5955 Project: Firebird Core Issue Type: Bug Environment: Libux binaried built with ld >=2.31 Reporter: Roman Simakov See https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob_plain;f=ld/NEWS;hb=refs/tags/binutils-2_31 for 2.31 *Add a configure option --enable-separate-code to decide whether -z separate-code should be enabled in ELF linker by default. Default to yes for Linux/x86 targets. Note that -z separate-code can increase disk and memory size. Now -z separate-code linker option is default and in /proc/self/maps we can find 6 sections for libfbclient.so 77b8d000-77c0f000 r--p 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 77c0f000-77da8000 r-xp 00082000 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 77da8000-77f83000 r--p 0021b000 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 77f83000-77f84000 ---p 003f6000 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 77f84000-77fbc000 r--p 003f6000 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 77fbc000-77fbe000 rw-p 0042e000 08:15 52957347 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 Previosly we could find 4. Now "r-xp" pages contains ONLY instructions. Any data moved to separate pages "a--p". 7f8ef3f04000-7f8ef42f9000 r-xp 08:15 52957337 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 7f8ef42f9000-7f8ef42fa000 ---p 003f5000 08:15 52957337 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 7f8ef42fa000-7f8ef4332000 r--p 003f5000 08:15 52957337 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 7f8ef4332000-7f8ef4334000 rw-p 0042d000 08:15 52957337 /home/roman/prj/RedDatabase/red-database/gen/Debug/firebird/lib/libfbclient.so.4.0.0 binreloc library skip any lines of /proc/self/maps except "a-xp" and it's a bug. It tries to find an address of empty constant string "" and supposes it will be allocated at the same page as instructions. Starting from ld 2.31 it's not so by default. We need to conider r--p sections as well. A simpthom is "Missing master config file firebird.conf" if you try to run server without installation. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] FB4 vs CORE-5222
ср, 19 сент. 2018 г. в 19:57, Gabor Boros : > Default value of ReadConsistency is illogical for me. The > documentation say "To help test existing applications...". If I want to > test a new feature I turn it on. Currently turn off to eliminate "update > conflicts...". (My own test application execute 1 statement/second which > is not a high load and got "update conflicts..." instantly if execute > two instances in parallel from it. > > A defaulted non properly working new feature is not a good message (for > me). Develop it, test it and if everything works with it as before, set > as default. But I am just a user. :D This value is default because of this behavior is consistent and correct. Previous one could lead to logical errors in the results of queries. I'm totally support this value as default and the need to change it only if you have a problem and consciously understand what are you doing. -- Roman Simakov Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5910) Server crash after kill in EDS manager destructor
Server crash after kill in EDS manager destructor - Key: CORE-5910 URL: http://tracker.firebirdsql.org/browse/CORE-5910 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 2.5.8 Environment: Ubuntu 18.04, superclassic, classic Reporter: Roman Simakov To reproduce: in terminal 1 run: ./isql -u sysdba -p masterkey create database 'localhost:/tmp/test'; set term ^; execute block as begin execute statement 'execute procedure a;'; end^ after in terminal 2 run: sudo killall fb_smp_server stack is: 1 Firebird::RefPtr::operator-> RefCounted.h 136 0x771859a0 2 EDS::EngineCallbackGuard::init ExtDS.cpp 1654 0x773fd6fb 3 EDS::EngineCallbackGuard::EngineCallbackGuard ExtDS.h467 0x7740492c 4 EDS::InternalStatement::doClose InternalDS.cpp 509 0x77403688 5 EDS::Statement::deallocate ExtDS.cpp 1046 0x773fafde 6 EDS::Statement::deleteStatement ExtDS.cpp 865 0x773fa4a4 7 EDS::Connection::clearStatements ExtDS.cpp 504 0x773f8f42 8 EDS::Connection::detach ExtDS.cpp 522 0x773f8fcf 9 EDS::Connection::deleteConnection ExtDS.cpp 318 0x773f82d6 10 EDS::Provider::clearConnections ExtDS.cpp 272 0x773f7f1a 11 EDS::Manager::~Manager ExtDS.cpp 72 0x773f7110 12 Firebird::GlobalPtr::dtor init.h 121 0x77401751 13 Firebird::InstanceControl::InstanceLink, (Firebird::InstanceControl::DtorPriority)2>::dtor init.h 97 0x77401721 14 Firebird::InstanceControl::InstanceList::destructors init.cpp 199 0x775efc02 15 Firebird::InstanceControl::destructors init.cpp 180 0x775efb53 16 (anonymous namespace)::allClean init.cpp 61 0x775ef96c 17 (anonymous namespace)::Cleanup::~Cleanup init.cpp 89 0x775ef9bd 18 __cxa_finalize cxa_finalize.c 83 0x75780615 19 __do_global_dtors_aux 0x77123e63 20 ?? 0x7fffe300 21 _dl_fini dl-fini.c 138 0x77de5b73 In frame 11 in ~Manager we create tdbb with data
Re: [Firebird-devel] Read consistency patch
ср, 25 июл. 2018 г. в 13:08, liviuslivius : > > > > Note: this SELECT may have greater CN then you have stored and it > > means on the next step you may receive records already selected here. > > You merge algorithm should take it into account. > > > > Roman Simakov > > > If transaction is in snapshot isolation mode then problem should not exists > Firebird should provide a way to got last CN from transaction start point > But merge is simple, based on Primary Key ID of the record. In this case exactly. You wrote about read committed transactions so I made such notation) One more note about your idea. Engine will have to read every record in anycase. But maybe expression index could help. -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Read consistency patch
ср, 25 июл. 2018 г. в 10:38, liviuslivius : > but from below scenario you can bring the idea > > usage case: > 1. > TR1 insert and update data on TableX and is still active > > 2. > Application do > TR2 start and we get recentCN and remember it as interestingCN > and we do > SELECT * FROM TABLEX > and cache above data in some memory dataset > TR2 commit Note: this SELECT may have greater CN then you have stored and it means on the next step you may receive records already selected here. You merge algorithm should take it into account. > 3. > TR1 commit and some other transactions commited. > > 4. > e.g. TR55 start and we get recentCN and remember it > and we got only "delta" > SELECT * FROM TABLEX WHERE CN>interestingCN > and we merge (which is really simple) this dataset with previously cached > dataset > now we remember current CN as interesting > interestingCN:=recentCN > TR55 commit -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Read consistency patch
вт, 24 июл. 2018 г. в 17:06, liviuslivius : > Is this possible that this CN(commit number) of transactions (pair > transaction id + CN) can be stored into some table? CN is not stored at all. A list of CommitNumbers is generated looking into TIP on database init in memory. Then it's mainainted in memory. > Is it possible to retrive most recent CN from transaction start point (call > this recentCN). I guess it makes more sence to think about either request CN or snapshot transaction CN. In your example olny request CN in read committed transaction make sense. Snapshot transaction has no changes to see any new table updates. If provide a system function like RDB$REQUEST_CN and RDB$RECORD_CN which will get RDB$RECORD_VERSION, get related CN and return so I guess it's possible to implement. But note. All these allow you to select records from a table and remember REQUEST_CN. You have to select RECUEST_CN in the same request as select) Otherwise the next request will have its own new CN. > It can provide simple way to retrive new records/changes in tables (new > feature). -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Read consistency patch
вт, 24 июл. 2018 г. в 13:15, liviuslivius : > > Hi, > > this is not the same > consider: > > transaction id=1 exists and is not commited but have modified some records in > tableX. > you start new transaction id=2 (read commited isolation mode) > you commit transaction id=1 > > > you do > SELECT * from tableX where RDB$RECORD_VERSION>2 > > above select will not see data commited by transaction id=1 because > it modify records before tr2 start > and tr1 is commited after tr2 start :) tr1 committed BEFORE select in tr2 and in read committed mode MUST see new versions. > > regards, > Karol Bieniaszewski > > > W dniu 2018-07-24 11:27:54 użytkownik Dimitry Sibiryakov > napisał: > > 24.07.2018 10:13, liviuslivius wrote: > > > It can provide simple way to retrive new records/changes in tables (new > > > feature). > > > E.g. whe can then do > > > SELECT * FROM TABLEX WHERE CN>:SCN; > > > >You already can do it with RDB$RECORD_VERSION. > > > > > > -- > >WBR, SD. > > > > -- > > Check out the vibrant tech community on one of the world's most > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > Firebird-Devel mailing list, web interface at > > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5861) GRANT OPTION is not checked for new object
GRANT OPTION is not checked for new object -- Key: CORE-5861 URL: http://tracker.firebirdsql.org/browse/CORE-5861 Project: Firebird Core Issue Type: Bug Affects Versions: 3.0.3, 4.0 Alpha 1 Reporter: Roman Simakov New objects and some old object of database cannot be granted via role privileges. Suggestion is the same as for check of existing objects: move checks to code like it's already done for generators and exceptions. To test a problem you can use the script that create users, roles, objects and step-by-step grants a privilege on every possible object 1) with grant option, 2) without grant option, 3) from role having it without grant option to prevent delegating: set echo on; create database 'localhost:/tmp/23976.fdb'; create or alter user user1 password 'pass'; create or alter user user2 password 'pass'; create or alter user user3 password 'pass'; create role role1; -- Has privileges with grant option create role role2; -- Has privileges without errors and without grant option create role role3; -- Must get errors in granting privileges from role2 grant role1 to user1; grant role2 to user2; grant role3 to user3; create procedure p as begin end; create function f returns int as begin end; create generator g; create exception e 'ex'; create table tab(id int); create package pak as begin end; grant create table to role1 with grant option; grant create procedure to role1 with grant option; grant execute on procedure p to role1 with grant option; grant execute on function f to role1 with grant option; grant usage on generator g to role1 with grant option; grant usage on exception e to role1 with grant option; grant select on tab to role1 with grant option; grant update(id) on tab to role1 with grant option; grant execute on package pak to role1 with grant option; commit; connect 'localhost:/tmp/23976.fdb' user 'user1' password 'pass' role 'role1'; select rdb$role_name from rdb$roles where rdb$role_in_use(rdb$role_name); grant create table to role2; grant execute on procedure p to role2; grant execute on function f to role2; grant usage on generator g to role2; grant usage on exception e to role2; grant select on tab to role2; grant update(id) on tab to role2; grant execute on package pak to role2; commit; -- create own objects create table tab_of_user1(i int); create procedure proc_of_user1 as begin end; commit; -- try to grant privileges for owned objects grant select on table tab_of_user1 to role2; grant execute on procedure proc_of_user1 to role2; commit; connect 'localhost:/tmp/23976.fdb' user 'user2' password 'pass' role 'role2'; -- check every privilege create table t(i integer); execute procedure p; select f() from rdb$database; select gen_id(g, 1) from rdb$database; select * from tab; -- try to grant every privilege to role3 and sure this causes an error grant create table to role3; grant execute on procedure p to role3; grant execute on function f to role3; grant usage on generator g to role3; grant usage on exception e to role3; grant select on tab to role3; grant update(id) on tab to role3; grant execute on package pak to role3; Note a couple of check for granting privileges on owned object. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5852) There is no check of existance generator and exception when privileges are granted
There is no check of existance generator and exception when privileges are granted -- Key: CORE-5852 URL: http://tracker.firebirdsql.org/browse/CORE-5852 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 3.0.3, 4.0 Alpha 1 Reporter: Roman Simakov RDB$TRIGGER9 checks that object exists. After adding USAGE privilege on generator and exception related checks were not added to RDB$TRIGGER9. As result we can grant privilege to non existing object. In the same time it's reaaly hard to maintain system triggers in clean BLR code and after protecting system tables from modifications we may move such checks to engine. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Services version 1 cleanup
вт, 19 июн. 2018 г. в 16:55, Dimitry Sibiryakov : > > 19.06.2018 15:42, Alex Peshkoff via Firebird-devel wrote: > > I plan to clean up support of them except anonymous (it may be used to ping > > server) in > > HEAD. Any objections? > >I would vote to clean out "service_mgr" as well and either to ignore this > part of > connection string completely or threat it as "expected db" item for > determining security > db, etc. Second option would allow explicit control over access of users to > services by > server configuration. I agree. "service_mgr" really seems very ugly and not so obvious why it needs. -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] System procedures
2018-05-28 8:02 GMT+03:00 Dmitry Yemanov : > 28.05.2018 04:04, Adriano dos Santos Fernandes wrote: >> >> >> I'm adding system procedure support, coded in C++, initially for list >> time zone rule transition, as it can't be done with virtual table (needs >> parameters). >> >> I see others great potentials with this. We can also later code system >> triggers with them, as they are currently coded directly in BLR and are >> unmaintainable. > > > I'd rather get rid of them at all, replacing with built-in VIO-level > processing. We already did that for some. I'm trying to do it for trigger1 right now because it do not cover some new objects privileges and it's really hard to maintain. -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5804) Multiple error in REVOKE operator
Multiple error in REVOKE operator - Key: CORE-5804 URL: http://tracker.firebirdsql.org/browse/CORE-5804 Project: Firebird Core Issue Type: Bug Affects Versions: 4.0 Alpha 1 Environment: All platforms Reporter: Roman Simakov A logic of rvoking options, especially for field permissions is very complicated and wrong. Several examples: EXAMPLE 1 grant update(f1, f2) on table t to u with grant option; commit;show grants; /* Grant permissions for this database */ GRANT UPDATE (F1) ON T TO USER U WITH GRANT OPTION GRANT UPDATE (F2) ON T TO USER U WITH GRANT OPTION revoke grant option for update on table t from u; commit;show grants; /* Grant permissions for this database */ GRANT UPDATE (F2) ON T TO USER U But should be: /* Grant permissions for this database */ GRANT UPDATE (F1) ON T TO USER U GRANT UPDATE (F2) ON T TO USER U EXAMPLE 2 grant update(f1, f2) on table t to u with grant option; commit;show grants; /* Grant permissions for this database */ GRANT UPDATE (F1) ON T TO USER U WITH GRANT OPTION GRANT UPDATE (F2) ON T TO USER U WITH GRANT OPTION revoke grant option for update(f1) on table t from u; commit;show grants; /* Grant permissions for this database */ GRANT UPDATE ON T TO USER U GRANT UPDATE (F2) ON T TO USER U WITH GRANT OPTION But should be /* Grant permissions for this database */ GRANT UPDATE (F1) ON T TO USER U GRANT UPDATE (F2) ON T TO USER U WITH GRANT OPTION =EXAMPLE 3 grant default r1 to role r2; commit; show grants; /* Grant permissions for this database */ GRANT DEFAULT R1 TO R2 revoke default r1 from role r2;-- revoke only default option commit; show grants; /* Grant permissions for this database */ GRANT DEFAULT R1 TO R2 But should be: /* Grant permissions for this database */ GRANT R1 TO R2 ETC. It's necessary to fix a logic and make a code more readable in this place. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5790) User with DROP DATABASE privilege can't drop database
User with DROP DATABASE privilege can't drop database - Key: CORE-5790 URL: http://tracker.firebirdsql.org/browse/CORE-5790 Project: Firebird Core Issue Type: Bug Affects Versions: 3.0.3, 4.0 Alpha 1, 3.0.2, 3.0.1, 3.0.0, 4.0 Initial Reporter: Roman Simakov isql -u sysdba -p masterkey create database 'localhost:d:\db\test.fdb'; create user user1 password 'pass'; grant drop database to user1; isql -u user1 -p pass connect 'localhost:d:\db\test.fdb'; drop database; Statement failed, SQLSTATE = 28000 no permission for drop access to database D:\DB\WWW.FDB -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5754) ALTER TRIGGER check privilege for alter database instead of table
ALTER TRIGGER check privilege for alter database instead of table - Key: CORE-5754 URL: http://tracker.firebirdsql.org/browse/CORE-5754 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 3.0.3, 4.0 Alpha 1 Reporter: Roman Simakov That's because engine considers trigger as DDL. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5755) Existing a subject of privileges
Existing a subject of privileges Key: CORE-5755 URL: http://tracker.firebirdsql.org/browse/CORE-5755 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 3.0.3, 4.0 Alpha 1 Reporter: Roman Simakov We have no check of existance a subjects of privileges. I.e. roman:bin$ ./isql Use CONNECT or CREATE DATABASE to specify a database SQL> create database 'a'; SQL> create table t(i integer); SQL> grant select on t to function wrong_func; SQL> show function wrong_func; There is no user-defined function WRONG_FUNC in this database We cannot check of existance user in such case but when we grant privilege to the database object we need to do it. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Existing a subject of privileges and refactoring
2018-02-20 15:43 GMT+03:00 Alex Peshkoff via Firebird-devel : > On 02/20/18 15:39, Roman Simakov wrote: >> In this case it's a bug right? >> > > yes http://tracker.firebirdsql.org/browse/CORE-5755 -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Existing a subject of privileges and refactoring
2018-02-20 15:27 GMT+03:00 Alex Peshkoff via Firebird-devel : > Always requiring object type is best of possible solutions but I'm afraid > that's unreal. > > I've supposed missing keyword, i.e. real failing statement is: > > grant select on t to FUNCTION wrong_func; > > Roman - am I wrong? Right. I meant explicit keyword. See this: roman:bin$ ./isql Use CONNECT or CREATE DATABASE to specify a database SQL> create database 'a'; SQL> create table t(i integer); SQL> grant select on t to function wrong_func; SQL> show function wrong_func; There is no user-defined function WRONG_FUNC in this database In this case it's a bug right? -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Existing a subject of privileges and refactoring
Hello! I fixed several bugs related to GRANT operator and investigated a couple of issues I'd like to discuss: 1) Replace RDB$TRIGGER_9 (trigger1 in trig.h) by code in DdlNodes.epp with the same functions. I already did it in CORE-5747 to check grant option. We can do it since we remove direct modifying system tables. It's more obvious place and we can use at least assert to check that all object types are verifyied. That could avoid errors like CORE-5747 in future. BLR of trigger is hard for support IMO. Also note I'm checking GRANT OPTION of roles which current user use. Now he can have several such roles. RDB$TRIGGER_9 do not check them I think. 2) We have no check of existance a subjects of privileges. I.e. SQL> create table t(i integer); SQL> grant select on t to wrong_func; SQL> show function wrong_func; There is no user-defined function WRONG_FUNC in this database I tend to consider it as a bug to be fixed. Am I right? -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5753) Parser allows to use GRANT OPTION for FUNCTION and PACKAGE
Parser allows to use GRANT OPTION for FUNCTION and PACKAGE -- Key: CORE-5753 URL: http://tracker.firebirdsql.org/browse/CORE-5753 Project: Firebird Core Issue Type: Bug Reporter: Roman Simakov For example: GRANT EXECUTE ON PROCEDURE TEST_PROCEDURE TO FUNCTION TEST_FUNCTION WITH GRANT OPTION; GRANT EXECUTE ON PROCEDURE TEST_PROCEDURE TO PACKAGE TEST_PACKAGE WITH GRANT OPTION; -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird 3 and ICU
Sounds resonable
2018-01-19 20:11 GMT+03:00 Alex Peshkoff via Firebird-devel
:
> On 01/19/18 14:03, Roman Simakov wrote:
>>
>> 2018-01-19 12:54 GMT+03:00 Vlad Khorsun via Firebird-devel
>> :
>>>
>>> 19.01.2018 11:41, Adriano dos Santos Fernandes wrote:
>>>>
>>>> On 19/01/2018 07:14, Roman Simakov wrote:
>>>
>>> it search for ICU library using versions numbers such as 5.19 ... 5.0,
>>> 4.19
>>> ... 4.0 etc.
>>>
>>>But, IIRC, current ICU versions is 51.x, 52.x, 60.x etc. I.e. they
>>> changed
>>> version
>>> numbering from N.xy to Nx.y. Is it correct ?
>>
>> Also take a look into formatFilename (unicode_util.cpp:346)
>>
>> static void formatFilename(PathName& filename, const char* templateName,
>> int majorVersion, int minorVersion)
>> {
>>string s;
>>if (majorVersion >= ICU_NEW_VERSION_MEANING)
>> s.printf("%d", majorVersion);
>>else
>> s.printf("%d%d", majorVersion, minorVersion);
>>
>>filename.printf(templateName, s.c_str());
>> }
>>
>> I see two ways to fix:
>> 1) enumerate additional suffix as well
>> 2) implement direct mapping both module and function names via config
>> and every Linux packages could provide its own specific format. Maybe
>> even implement template with variables in config. Something like this:
>> For Suse: libicuuc = " libicuuc.so.${Major}{$Minor}.{$Release}"
>> For Ubuntu: libicuuc = " libicuuc.so.${Major}{$Minor}"
>> Some other: libicuuc = " libicuuc.so.${Major}.{$Minor}"
>>
>> For functions can do the same.
>>
>> I loading code we will try Majoir in {5,4,3,6,0}, Minor in {1..20},
>> Release in {1..3} on like that.
>>
>
> This is almost funny case. The most traditional schema of versioning linux
> dynamic libraries is use of $Major in soname and ignoring $Minor (i.e. no
> API/ABI changes are expected in minor releases, only bugfixes). But ICU used
> to generate different sort keys in different minor releases (for us that
> would be a kind of incompatible between for example 3.0.2 / 3.0.3 db
> structure). Therefore $Minor became major-ilke part of ICU version. People
> even stopped to separate them with '.', i.e. instead ICU with major==5 and
> minor==3 we do have ICU with major==53. Well - in addition to minors
> $Release was invented. Now people try to add it to already big major too :)
>
> Well, it's really hard to predict how will develop ICU versioning. I'll be
> not surprised if _subrelease_ will arrive. Worse is that we can not limit
> ourself with loading ICU using libicuuc.so symlink - sometimes we need old
> versions to support old databases. It starts to seem to me that (at least
> for posix) it will be better to try to load all libraries having
> libicuuc.so.* names appending what we find after dot to functio names. Will
> try with it next week.
>
>
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
--
Roman Simakov
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird 3 and ICU
2018-01-19 12:54 GMT+03:00 Vlad Khorsun via Firebird-devel
:
> 19.01.2018 11:41, Adriano dos Santos Fernandes wrote:
>>
>> On 19/01/2018 07:14, Roman Simakov wrote:
> it search for ICU library using versions numbers such as 5.19 ... 5.0, 4.19
> ... 4.0 etc.
>
> But, IIRC, current ICU versions is 51.x, 52.x, 60.x etc. I.e. they changed
> version
> numbering from N.xy to Nx.y. Is it correct ?
Also take a look into formatFilename (unicode_util.cpp:346)
static void formatFilename(PathName& filename, const char* templateName,
int majorVersion, int minorVersion)
{
string s;
if (majorVersion >= ICU_NEW_VERSION_MEANING)
s.printf("%d", majorVersion);
else
s.printf("%d%d", majorVersion, minorVersion);
filename.printf(templateName, s.c_str());
}
I see two ways to fix:
1) enumerate additional suffix as well
2) implement direct mapping both module and function names via config
and every Linux packages could provide its own specific format. Maybe
even implement template with variables in config. Something like this:
For Suse: libicuuc = " libicuuc.so.${Major}{$Minor}.{$Release}"
For Ubuntu: libicuuc = " libicuuc.so.${Major}{$Minor}"
Some other: libicuuc = " libicuuc.so.${Major}.{$Minor}"
For functions can do the same.
I loading code we will try Majoir in {5,4,3,6,0}, Minor in {1..20},
Release in {1..3} on like that.
--
Roman Simakov
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Firebird 3 and ICU
Hello! It seems that we have a problem with installation of Firebird 3 on OpenSuse. I get "Could not find acceptable ICU library". I took a look into sources. FB tries to search ICU modules in some magic manner combining major and minor versions. But it looks in Suse icu libraries has additional prefix like ".1" or "_1". I tries to make symlink and server could find .so! However it could not find functions because of the same problem. They also use such suffixes. Anybody can fix a magic of library and functions looking up or at least explain me an idea of naming? -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5677) Dirty RDB$PAGES after an error after phase 3 of create_relation
Dirty RDB$PAGES after an error after phase 3 of create_relation --- Key: CORE-5677 URL: http://tracker.firebirdsql.org/browse/CORE-5677 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 4.0 Alpha 1, 3.0.2, 2.5.7, 3.0.1, 2.5.6, 3.0.0, 4.0 Initial Environment: Ubuntu x86_64 Reporter: Roman Simakov In case of error after phase 3 of create_relation in dfw the engine have filled RDB$PAGES in system transaction. Changes of RDB$RELATIONS and RDB$DATABASE are doing in user transaction and they rolled back. As result we have a couple of records in RDB$PAGES. Later if some attachment will scan pages it creates an object jrd_rel in att_relations and marks it by flag REL_check_existance and later REL_deleted. If we will try to create a table in this attachment we can do it. But at least we will have duplicated records in RDB$PAGES for the relation. Then if we will try to create an index for that table MET_lookup_relation_id will check REL_deleted and returns NULL. Index creation will fails. Reconnect can helps but duplicates of RDB$PAGES is not good in any case. Here is a scenario to reproduce it by 2 terminals (1: and 2:) on Classic arcitecture: 2: create database '/tmp/d.fdb'; 2: set transaction read committed; 1: gdb --args ./isql /tmp/d.fdb 1: r 1: Ctrl+C 1: break check_not_null if (phase==3) 1: c 1: create table tn(i integer); 1: insert into tn values(NULL); 1: commit; 1: set autoddl off; 1: set transaction read committed; 1: create table terr(i integer); 1: alter table tn alter i set not null; 1: commit; 1: will break in breakpoint 2: select * from rdb$pages; // to force write RDB$PAGES via AST 1: c 1: ERROR: Cannot make field I of table TN NOT NULL because there are NULLs present 1+2: Ctrl+D to exit Now we can connect to the database and check: SQL> select * from rdb$pages where rdb$relation_id=129; RDB$PAGE_NUMBER RDB$RELATION_ID RDB$PAGE_SEQUENCE RDB$PAGE_TYPE === === = = 186 129 0 4 187 129 0 6 SQL> select * from rdb$relations where rdb$relation_id=129; SQL> select rdb$relation_id from rdb$database; RDB$RELATION_ID === 129 The idea of fix is to cleanup RDB$PAGES and release physically located pages at phase 0 of create_relation. I'll prepare PR soon. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] cmake build error
You sould not strip out TOK_ https://github.com/FirebirdSQL/firebird/pull/133 fixed errors you see 2017-12-02 23:58 GMT+03:00 William L. Thomson Jr. : > On Sat, 2 Dec 2017 23:45:09 +0300 > Roman Simakov wrote: >> >> Hmmm. I do not understand. Did ypu strip out TOK_ prefix from src > > Yes I strip it out via sed, just removing TCK_, replacing with nothing. > > Before removing TOK_ I got a different error, I think about token not > found or something. It was not a scope issue. Assuming maybe something > translates the TOK_ to something else. Something the cmake build system > was failing to do or something. > >> and wander why it's not found? > > Seems it was something I did wrong. I just went to comment it out and > replicate the original error. It does not seem to be failing anymore. > Maybe due to the changes to cmake via the patch/PR133. > > Can disregard sorry for the noise. I should have commented out my sed > and tested cmake changes before posting. My bad, sorry :P > > -- > William L. Thomson Jr. > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > -- Roman Simakov -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] cmake build error
Hmmm. I do not understand. Did ypu strip out TOK_ prefix from src and
wander why it's not found?
2017-12-02 23:30 GMT+03:00 William L. Thomson Jr. :
> I ran into this before and seems same even with updated cmake patches.
> Not sure if its a gcc 6 issue, some other scope issue, or something I
> did to cmake build, or env.
>
> I am removing some tokens, maybe btyacc issue. Here I strip out TOK_
>
> for f in yvalve/keywords dsql/Parser; do
> sed -i -e "s|TOK_||g" "src/${f}.cpp" \
> || die "Failed to correct ${f}cpp"
> done
>
>
> FAILED: src/CMakeFiles/yvalve_common.dir/yvalve/keywords.cpp.o
> /usr/bin/x86_64-pc-linux-gnu-g++ -DDEV_BUILD
> -I/tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/extern/decNumber
> -I/tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/extern/icu/include
> -I/tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/extern/zlib
> -I/tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/src/include
> -I/tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/src/include/gen
> -Isrc/include -Isrc/include/gen -DNDEBUG -O2 -pipe -march=amdfam10
> -mcx16 -msahf -mabm -mlzcnt -fpermissive -msse4 -std=c++11 -fPIC -msse4
> -std=c++11 -MD -MT
> src/CMakeFiles/yvalve_common.dir/yvalve/keywords.cpp.o -MF
> src/CMakeFiles/yvalve_common.dir/yvalve/keywords.cpp.o.d -o
> src/CMakeFiles/yvalve_common.dir/yvalve/keywords.cpp.o
> -c
> /tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/src/yvalve/keywords.cpp
> /tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/src/yvalve/keywords.cpp:44:3:
> error: ‘NOT_LSS’ was not declared in this scope
> {NOT_LSS, "!<", false},
> ^~~
> /tmp/portage/dev-db/firebird-4.0.0_alpha1/work/firebird-T4_0_0_Alpha1/src/yvalve/keywords.cpp:45:3:
> error: ‘NEQ’ was not declared in this scope
> {NEQ, "!=", false},
> ^~~
>
> And it repeats for several lines, many things not declared.
>
> --
> William L. Thomson Jr.
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>
--
Roman Simakov
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
