Re: [Maria-developers] d282f5c5560: MDEV-10963 Fragmented BINLOG query

2018-12-18 Thread andrei . elkin
Sergei,

One correction to just sent reply,
in fact I've removed this:

>> --- a/sql/log_event.h
>> +++ b/sql/log_event.h
>> @@ -749,6 +749,7 @@ typedef struct st_print_event_info
>>  that was printed.  We cache these so that we don't have to print
>>  them if they are unchanged.
>>*/
>> +  static const uint max_delimiter_len= 16;
>
> why did you introduce this max_delimiter_len, if all you use
> is sizeof(delimiter) anyway? (and even that is not needed)

through following your simplification idea.


Andrei.

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp


Re: [Maria-developers] d282f5c5560: MDEV-10963 Fragmented BINLOG query

2018-12-18 Thread andrei . elkin
Sergei, hallo.

> Hi, Andrei!
>
> Looks better!

There's never a limit to improvements though :-).
Thanks for checking and insightful comments as usual!

I am publishing an updated patch to cover all of them.
To your notes, I am replying and commenting on a few of your questions below.

Could you also please clarify on one point, which is

   >> -my_b_copy_to_file(IO_CACHE *cache, FILE *file)
   >> +my_b_copy_to_file(IO_CACHE *cache, FILE *file,
   >> +  my_bool do_reinit,
   >> +  size_t count)
   >>  {
   >> -  size_t bytes_in_cache;
   >> +  size_t curr_write, bytes_in_cache;
   >>DBUG_ENTER("my_b_copy_to_file");
   >>  
   >>/* Reinit the cache to read from the beginning of the cache */
   >> -  if (reinit_io_cache(cache, READ_CACHE, 0L, FALSE, FALSE))
   >> +  if (do_reinit && reinit_io_cache(cache, READ_CACHE, 0L, FALSE, FALSE))
   >
   > generally, when there's a function that is always called with a
   > constant (compile-time) argument, I prefer to split the code
   > compile-time too, if it isn't too much trouble. In this case it would
   > mean a new function like
   >
   >   int my_b_copy_all_to_file(IO_CACHE *cache, FILE *file)
   >   {
   > if (reinit_io_cache(cache, READ_CACHE, 0L, FALSE, FALSE)
   >   return 1;
   > return my_b_copy_to_file(cache, file, SIZE_T_MAX);
   >   }
   >

why is the preference specifically?
I looked around to find something like
 https://en.wikipedia.org/wiki/Compile_time_function_execution
... but I somewhat doubt you meant that. Or did you really?


Now to my acknowledgments and comments...

>
> There are no major problems, but see comments below. There're few
> suggestions how to simplify the code.
>
> On Nov 05, Andrei Elkin wrote:
>> revision-id: d282f5c55609469cd74d7390f70c7d922c778711 
>> (mariadb-10.1.35-93-gd282f5c5560)
>> parent(s): 2a576f71c5d3c7aacef564e5b1251f83bde48f51
>> author: Andrei Elkin 
>> committer: Andrei Elkin 
>> timestamp: 2018-10-21 23:42:00 +0300
>> message:
>> 
>> MDEV-10963 Fragmented BINLOG query
>> 
>> diff --git
>> a/mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_frag.test
>> b/mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_frag.test
>> new file mode 100644
>> index 000..bdf41c94c76
>> --- /dev/null
>> +++ b/mysql-test/suite/binlog/t/binlog_mysqlbinlog_row_frag.test
>> @@ -0,0 +1,50 @@
>> +--source include/have_debug.inc
>> +--source include/have_log_bin.inc
>> +--source include/have_binlog_format_row.inc
>
> you don't need to include have_log_bin, if you include
> have_binlog_format_row.

ack

>
>> +
>> +--let $MYSQLD_DATADIR= `select @@datadir`
>> +--let $max_size=1024
>> +
>> +CREATE TABLE t (a TEXT);
>> +# events of interest are guaranteed to stay in 01 log
>> +RESET MASTER;
>> +--eval INSERT INTO t SET a=repeat('a', $max_size)
>
> eh? why did you do it with let/eval instead of a simple sql statement?
> you don't use $max_size anywhere else.

left inadvertently from a previous patch.

>
>> +SELECT a from t into @a;
>> +FLUSH LOGS;
>> +DELETE FROM t;
>> +
>> +--exec $MYSQL_BINLOG --debug-binlog-row-event-max-encoded-size=256
>> $MYSQLD_DATADIR/master-bin.01 >
>> $MYSQLTEST_VARDIR/tmp/mysqlbinlog.sql
>> +
>> +--let $assert_text= BINLOG is fragmented
>> +--let $assert_select= BINLOG @binlog_fragment_0, @binlog_fragment_1
>> +--let $assert_count= 1
>> +--let $assert_file= $MYSQLTEST_VARDIR/tmp/mysqlbinlog.sql
>> +--source include/assert_grep.inc
>
> no, please, use search_pattern_in_file.inc instead.

ack

>
>> +
>> +--exec $MYSQL test < $MYSQLTEST_VARDIR/tmp/mysqlbinlog.sql
>> +
>> +SELECT a LIKE @a as 'true' FROM t;
>> +SELECT @binlog_fragment_0, @binlog_fragment_1 as 'NULL';
>
> that makes no sense, @binlog_fragment_0 and _1 were set in a separate
> client session. You cannot test whether they were cleared or not there,
> by looking at the values here

You caught me here! :-).
In the new patch I relocate the check in the other test file.

>
>> +
>> +# improper syntax error
>> +--echo BINLOG number-of-fragments must be exactly two
>> +--error ER_PARSE_ERROR
>> +BINLOG @binlog_fragment;
>> +--error ER_PARSE_ERROR
>> +BINLOG @binlog_fragment, @binlog_fragment, @binlog_fragment;
>> +
>> +# corrupted fragments error check (to the expected error code notice,
>> +# the same error code occurs in a similar unfragmented case)
>> +SET @binlog_fragment_0='012345';
>> +SET @binlog_fragment_1='012345';
>> +--error ER_SYNTAX_ERROR
>> +BINLOG @binlog_fragment_0, @binlog_fragment_1;
>> +
>> +# Not existing fragment is not allowed
>> +SET @binlog_fragment_0='012345';
>> +--error ER_WRONG_TYPE_FOR_VAR
>> +BINLOG @binlog_fragment_0, @binlog_fragment_not_exist;
>> +
>> +--echo # Cleanup
>> +--remove_file $MYSQLTEST_VARDIR/tmp/mysqlbinlog.sql
>> +DROP TABLE t;
>> diff --git a/mysys/mf_iocache2.c b/mysys/mf_iocache2.c
>> --- a/mysys/mf_iocache2.c
>> +++ b/mysys/mf_iocache2.c
>> @@ -22,52 +22,53 @@
>>  #include 
>>  #include 
>>  
>> -/*
>> -  Copy contents of an IO_CACHE to a 

[Maria-developers] New Foundation Buildbot and Github build status push

2018-12-18 Thread Teodor Mircea Ionita
Hi all,

For the past 6 months we have been working on a new Buildbot deployment at:

https://buildbot.mariadb.org

Currently we have docker x64 builders for all supported Linux platforms (aptly 
named quick builders, since they all finish a build + default MTR under 40 
minutes):
https://buildbot.mariadb.org/#/builders

We are working on adding builders for all the other supported targets in the 
near future as well as acquiring more building power to accommodate for --big 
and more extensive/complex tests. We are open to donations in that regard.

Saved package artifacts can be accessed from any individual build log via the 
"Download" link or directly from here:
https://ci.mariadb.org

The configuration and associated documentation can be found in the tools repo:
https://github.com/MariaDB/mariadb.org-tools/tree/master/buildbot.mariadb.org

We encourage collaboration and are open to pull requests, you can do so for 
adding new builders, fixing build steps or adjusting the docker images 
available in the dockerfiles/ sub-directory, etc..

Currently we consider this deployment in beta phase and have enabled Github 
status push for the main repo for direct code pushes as well as pull requests. 
The build status shows up in the same Github widget as Travis does and you can 
follow any individual report to the actual buildbot log via the links. Sample 
report for a push:

https://github.com/MariaDB/server/commits/bb-10.4-statustest

Sample PR:
https://github.com/MariaDB/server/pull/1034

Reproducing Linux failures are facilitated by the Dockerfiles available in the 
tool's repo, which are used by the Buildbot master to create the build 
environment on the docker enabled workers:

https://github.com/MariaDB/mariadb.org-tools/tree/master/buildbot.mariadb.org#debugging-build-failures

Ideally, any developer should pay attention to these build failures on Github 
and attempt to fix any issues before merging into the main branches. While in 
beta, we are trying to weed out environmental or configuration failures too, so 
please report those as you come by them so we can fix them.

The end goal is to enable protected branches, where any individual change has 
to pass a certain selection of robust tests before merging, which ultimately, 
would ensure that the MariaDB code is in a releasable state after any 
individual commit.

We welcome feedback, improvement suggestions and any particular questions you 
might have about the current setup and future plans.

We will check-in from time to time as we have more goodness to show.

Best regards,
Teodor

-- 
Developer, Infra
MariaDB Foundation
https://mariadb.org
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp