Any suggestions, guys? Can anyone please point us to the right place in the
source code so we can have a closer look?
Also, it appears that external table doesn't like a row being over 32k
(including padding) in the external file. When we have rows over 32k,
select from the external table just returns nothing. Is this a known
limitation? I couldn't find this mentioned in any documentation.
We are running Firebird 2.1.2 64bit Classic Server on Centos 6.
On 14 September 2013 19:24, Huan Ruan <leohuanr...@gmail.com> wrote:
>
>
> Hi All
>>
>> We are trying to implement a bulk data importer by writing our own
>> external table files. We've tried storing all fields as text, but due to
>> the volume of data, this put us well over the 2GB external file size limit
>> in Firebird 2.1. So we've resorted to saving the files in Firebird's own
>> binary format.
>>
>> To simplify things a little, we only need to deal with CHAR, NUMERIC and
>> TIMESTAMP fields.
>>
>> The data formats were easy enough to figure out. However, the byte
>> alignment rules for some field types are a little baffling. From what I've
>> seen:
>> - CHAR fields can start at any position within a row
>> - 2-byte NUMERIC fields can start at bytes 0, 2, 4...
>> - 4-byte NUMERIC fields can start at bytes 0, 4, 8...
>> - 8-byte NUMERIC / TIMESTAMP fields:
>> -- Can start at bytes 0, 8, 16... as long as either:
>> --- The first field in the table is an 8-byte NUMERIC / TIMESTAMP, or
>> --- The table contains more than 32 fields in total
>> -- Otherwise, can start at bytes 4, 12, 20...
>>
>> Is this right? Is there anything I'm missing? We didn't get very far
>> digging around in the source code.
>>
>> Thanks
>> Huan
>>
>
> After a bit more testing, it looks like there's a bit more to the 8-byte
> NUMERIC / TIMESTAMP rule. The fields will start at bytes 0, 8, 16... if the
> total number of fields is between 33 and 64, between 97 and 128, etc. In
> other words, if floor((FieldCount - 1)/32) is odd.
>
>
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
