in Firebird to rip out compresor from
storage engine (and put new RLE in second step, encoding in third step),
but it was rejected by comunity :)
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http
data for research.
If you are interesting in record encoding/compresion, I'm ready to help.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
Hi Alex,
please can you take your tpcc.fdb, backup, 7zip and send to me?
I will use as reference database.
Thank you.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http
not sure that 4kB is good trashhold, but I belive, that
some trashold will be needed.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
Hi Karol,
in current simplified stack you have:
execution engine
record level compresion
page storage
cache
hdd
On small records, you have CPU problem, but on large records you can
simply utilize HDD.
To utilize HDD tou have to meet at least ~200MB/s on common CPU.
Target is: be able utilize
blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Ing. Slavomir Skopalik
Executive Head
Elekt Labs
;
}
}
As I wrote, it is imposible change encoding without refactoring current
code base.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz
tr_id
uint8format
data
Any comments?
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
330 running a 68010, approximately the
norm for workstations circa 1984. Machines have changed since then, and
with them, the tradeoffs.
On Friday, February 27, 2015, Slavomir Skopalik skopa...@elektlabs.cz
wrote:
Hi Jim,
I don't tell your scheme hack, this is misunderstanding.
I tell
-describing, value driven encoding I
described earlier.
Another area where this is much room for improvement is the encoding of
multi-column indexes. There is a much more clever scheme that doesn't
waste everything fifth byte.
On Friday, February 27, 2015, Slavomir Skopalik skopa...@elektlabs.cz
(char 0x0 is in use).
This wastage CPU and disk space.
Do anybody have a idea, why firebird did this this way?
Thanks Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz
Hi Vlad,
as I see, in some situation (that really happen), packing into small
area is padded by zeroes
(uncomress prefix with zero length).
And new control char added at begining next fragment (you will lost 2
bytes).
The differencies in current compression is not so much, but with better
one
Compressor : public Firebird::AutoStorage
If will be posible to have access in record format, it can be easy to create
self-described encoding.
I have in my mind a idea of this schema that I woudlike to test it.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation
into SQZ ?
History info: I was designed my RLE for Zilog Z80 CPU on ZX Spectrum in 80'.
It is normally operate during commpression/decompression in same buffer.
Is it clear ?
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines
).
Also I was made some speed optimization, this version is faster, then
previous one.
If somebody else is interesting in this, I can put my private buid for
Win64 on my web site.
Best regards Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from
Hi Jim,
can you explain more about your algorithm for self-describing value
encoding ?
I'm interesting in this.
Thank you Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http
into one byte,
and in worst case add max 3 bytes for 64kb record.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
On 23.2.2015 20:36, James Starkey wrote:
Encode null as a value type and skip the null flags altogether -- saves a
couple of bytes for every record.
I think to use flags only for nullable fields.
In this case, you will lost one byte per each NULL field, but only when
is it NULL, I will be lost
in future.
Best regards Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o
Hi,
windows file compression using LZNT1
https://msdn.microsoft.com/en-us/library/jj711990.aspx
that is dictionary based compression like LZ4.
It is work in 64kB block that compress into smaller are with some free space
for update.
But any update is problematic and for MSSQL, Hyper-V and more is
> SS> Yes, but now you have to told to FB explicitly which part of number have
> SS> to be discarded.
>
> And you still could do this, as you wish :) You remain with full
> control, if/when you prefer to have it. I guess the proposed
> enhancement will be useful for 99% of the cases, when user
> SS> 1. Decimal or Numeric is used for to keep exact accuracy.
> SS>If you don't need this accuracy use float point data types.
>
> You didn't get the point. There is no loss of accuracy. The data types
> will continue working as is. Remember, as I said, today you already have to
> cast
>
.
There will lot of risk in financial calculation because real result
will depend
on server config.
From my point of view, this auto casting will generate more problems
than helps.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines
. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o.
Chaloupky 158
783 72 Velky Tynec
Hi Dmitry,
if you help me with integration, will do it.
Slavek
> I have nothing against a value-based encoding. But our v4 development
> tasks currently do not include inventing one. If someone jumps in with a
> prototype implementation, I'd be definitely willing to review/consider it.
>
>
>
).
Finally you will can chose between RLE and value encoding.
Is it OK for you?
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
Hi,
this is about record encoding .
My private test with RLE + LZ4 shows, that from combination RLE, LZ4,
RLE + LZ4 the last is the best.
I believe, that better record encoding will help wire transfer in any case.
From other side, record encoding can be used in all situation (in
average speed
Hi Jim,
Elekt Labs RLE is ready to use and is better then current one.
The best one have to be developed and I will help if I can.
The question is still same: Is it right time or will be postponed to V5
(2024+) ?
Slavek
>
> Excuse me, but faster than the current one is not the question. The
.
Slavek
> 03.03.2016 18:34, Slavomir Skopalik wrote:
>> But value encoding cannot be implemented until we switch from fragment
>> compression to true record level compression.
> I was not speaking about any encoding, just about compacting the record.
>
> Fragments are not
Hi Dmitry,
I will send latest version, that have some improvements.
In any case, feel free to contact privately (E-mail or skype).
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http
==
T1 _SUFIX
T1234567890 _SUFIX
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
= rpb->rpb_format_number;
header->rhd_b_page = rpb->rpb_b_page;
header->rhd_b_line = rpb->rpb_b_line;
UCHAR* const data = (UCHAR*) header + header_size;
dcc.pack(rpb->rpb_address, data);
if (fill)
memset(data + size, 0, fill);
--
Ing. Slavom
?
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o.
Chaloupky 158
783
ith "default"
> or
> 2. add field without not nullupdate table and set that field valueafter that
> change that field to not null
> regards,Karol Bieniaszewski
>
> ---- Oryginalna wiadomość
> Od: Slavomir Skopalik <skopa...@elektlabs.cz>
> Data: 12.10
Big disadvantages of default is:
1. Immediately update all rows in table.
Interesting only if all or most already existed rows will use this value.
2. Less readable, this is important for me.
I prefer this scenario:
ALTER TABLE ... ADD null able;
COMMIT;
UPDATE ...
COMMIT;
ALTER TABLE ...
Hi,
I was tested FB3.0.1 Windows 64 with different result.
DaviceData is table with several millions of rows.
alter table DeviceData add a integer default 7 not null;
This is fast, just few ms.
But commit was taken several minutes with high disk activities.
Also I made another test.
1.
Hi Alex,
I'm not worry about 2.5. I was just put here for info.
Slavek
PS: Spaces was deleted by HTML in E-mail :(
> T2
> ==
> T1 _SUFIX
> T1234567890_SUFIX
>
> where _SUFIX starts exactly at position == 31
Hi,
is it correct that empty string '' in comparison with one space string '
' is evaluated as true?
SELECT * FROM rdb$database WHERE ''=' '
FB 2.5.6, database dialect 1
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines
Hi Jiri,
you must decode input UTF8 buffer into string(probably unicode) and next
you have to trim trailing spaces.
For text, just put some non US characters into text.
Slavek
> The length from metadata is 80. The memory dump where the pointer points
> is:
> { 80, 0, 49, 50, 51, 52, 53, 54, 55,
around for this?
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o
Thank you, good to know.
But compatibility with 2.5 must be (we cannot use 3.0.2 because lack of
stability).
How to solve in both version?
Or I can prepare windows bat file that will call isql.
Slavek
Em 13/08/2017 14:49, Slavomir Skopalik escreveu:
3. The best will be, if each SP
Hi Adriano, we have serious problems with FB3 with stability.
And I have info, that also other companies has similar problems.
That is the reason to develop library for 2.5 with compatibility with FB3.
Slavek
My opinion is to forget FB 2.5 and create a package instead of prefix
every object.
to drop
support for FB2.5.
But library can contain FB3 specific things, contributors are welcome.
Slavek
On 14.8.2017 14:29, Alex via Firebird-devel wrote:
On 14.08.2017 15:21, Slavomir Skopalik wrote:
Hi Adriano, we have serious problems with FB3 with stability.
What problems
BTW - why not use snapshots? Currently they pass same tests control as
releases. Review tests result and if nothing critical present - fill
free to work with snapshot. For stable releases snapshots are pretty
stable.
I'm not so brave to install snapshot into production environment.
Now
The situation is clear.
We have to support FB2.5 because of customer, we don't have a choice ->
our code must be compatible with 2.5.
And I know about other companies that are is same situation.
If somebody will create FB3 like version, it will be welcome.
Will someone do that?
Slavek
||';';
EXECUTE STATEMENT SQL;
END
END
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt
Hi all,
I'm testing Firebird-3.0.3.32798-0_Win32 client and I found strange
behavior when I canceling event
by isc_cancel_events.
I supposed that after that call, no call back happen, but in reality it
is happen.
Is it correct?
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs
Should I create ticket for this?
From programing theory, no call back after successful cancel event is
allowed.
Slavek
Hi all,
I'm testing Firebird-3.0.3.32798-0_Win32 client and I found strange
behavior when I canceling event
by isc_cancel_events.
I supposed that after that call, no
On 28.08.2017 04:15, Slavomir Skopalik wrote:
Should I create ticket for this?
No. Removing that call will break old code.
You mean, old code will not crash so often.
How much time I have to wait to be sure that no more call back?
Or is there other technique how to solve it?
From
But this mean to have some global dispatcher that will die as last
object during termination process.
This is far from clean programing.
And by the way, Embarcadero in delphi IBX code don't counted with that.
Slavek
28.08.2017 11:09, Paul Reeves wrote:
The event listener should really be
Thread is created inside gds32.dll.
Slavek
Maybe I'm missing something, but I seem to recall from the code that
you posted that your listener is running on a timer in the main thread.
The event listener should really be running in its own thread.
Have you tried that?
Paul
canceling some async operation,
I in critical section invalidate call backs, wait for thread finish and
that's all.
From my point of view, I use fbclient wrong way or there is bug in
fbclient.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from
Not exactly,
procedure EventCallback(P: Pointer; Length: Short; Updated: PByte); cdecl;
var e:TFBEvents;
begin
if (Assigned(P) and Assigned(Updated)) then begin
e:=TFBEvents(p);
try
e.FSO.Enter;
e.EventsReceived := true;
if e.ResultBuffer<>nil then
2.5.8 snapshot and I will share result.
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo
}
SetLength(Result, Count);
if GetComputerName(PChar(Result), Count) then
StrResetLength(Result)
else
Result := '';
end;
Slavek
--
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
Collection and evaluation of data from machines and laboratories
by means of system MASA (http://www.elektlabs.cz/m2demo)
-
Address:
Elekt Labs s.r.o.
Chaloupky 158
783 72 Velky
V3 super server should be compared with V2.5 super classic.
Slavek
2.5 - 10:55
3.0 - 8:11
4.0 - 8:41
Same script with numbered(1..5) table names into one database
concurrently:
2.5 - 10:51 - 336.72 MB
3.0 - 8:09 - 329.79 MB
4.0 - 8:45 - 329.79 MB
i.e. v3 finally becomes faster than
In case, that you use transaction ID for business logic and needs to
only going up.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
MASA - Collection and evaluation of data from machines and laboratories
http://eng.elektlabs.cz/products-and-services/masa
But this is automated tests, not a real experiences.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
MASA - Collection and evaluation of data from machines and laboratories
http://eng.elektlabs.com/products-and-services/masa
are test by large number of users, but snapshots?
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
MASA - Collection and evaluation of data from machines and laboratories
http://eng.elektlabs.com/products-and-services/masa
Sorry to say,
if you will put snapshots into production, you will be responsible for
everything.
The another point of view is, that snapshot is not tested and verified
by public.
For my customers, no way to go.
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
MASA
quality of release, why is not released more often
officially?
Slavek
Ing. Slavomir Skopalik
Executive Head
Elekt Labs s.r.o.
MASA - Collection and evaluation of data from machines and laboratories
http://eng.elektlabs.com/products-and-services/masa
-5392
Project: Firebird Core
Issue Type: Bug
Components: Engine
Affects Versions: 3.0.1
Environment: Windows 10 64bit, Firebird 3.0.1.32609 official build
super server, connection to DB by TCP localhost looback.
Reporter: Slavomir Skopalik
Components: Engine
Affects Versions: 2.5.7
Environment: FB 2.5.7 x64
Reporter: Slavomir Skopalik
Priority: Minor
Simple test case:
SET TERM ^;
CREATE PROCEDURE A(A RDB$Field_Name)
AS
BEGIN
END
^
SET TERM ;^
COMMIT;
DROP PROCEDURE A;
COMMIT;
Cannot
Environment: Any
Reporter: Slavomir Skopalik
Priority: Trivial
Add Sleep function as common fbudf function.
This is helpful during obtaining lock or testing.
Declaration like:
DECLARE EXTERNAL FUNCTION Sleep
INTEGER NULL
RETURNS INTEGER BY VALUE
: Firebird Core
Issue Type: Bug
Components: API / Client Library
Affects Versions: 2.5.7
Environment: Windows, DelphiXE7, simplified (no thread) implementation
of IBEvent
Reporter: Slavomir Skopalik
Priority: Minor
32 bits gds32.dll from
bits
Reporter: Slavomir Skopalik
Firebird hangs (stop responding to any client), utilize one CPU core.
Dump file created, ask me privately .
Call stack of thread that consume one core:
ntoskrnl.exe!IoAcquireRemoveLockEx+0xe7
ntoskrnl.exe!memset+0x22a
ntoskrnl.exe!KeWaitForMutexObject
67 matches
Mail list logo
