Re: [sqlite] Using .testcase and .check in continuous integration test

[Dominique Devienne]

On Tue, Oct 17, 2017 at 3:52 PM Dominique Devienne 
wrote:

> On Tue, Oct 17, 2017 at 3:16 PM, Lodewijk Duymaer van Twist <
> lodew...@adesys.nl> wrote:
>
>> Thank you for investigating. You're fix works. Should I repost this as a
>> bug with your fix, or will this be picked up as is right now?
>>
>
> Glad it did. Just sit tight and again wait and see if Dr Hipp agrees the
> behavior should change or not.
> I'm hopeful he might, but if he doesn't, there's little you can do about
> it. I don't think reposting is necessary for now. --DD
>

FYI, Richard checked the change in the day of that post.
Only noticed it today, thus this late answer.  Thank you Richard.
http://www.sqlite.org/src/info/e2af0cc6ef5fafc7
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using .testcase and .check in continuous integration test

[Dominique Devienne]

On Tue, Oct 17, 2017 at 3:16 PM, Lodewijk Duymaer van Twist <
lodew...@adesys.nl> wrote:

> Thank you for investigating. You're fix works. Should I repost this as a
> bug with your fix, or will this be picked up as is right now?
>

Glad it did. Just sit tight and again wait and see if Dr Hipp agrees the
behavior should change or not.
I'm hopeful he might, but if he doesn't, there's little you can do about
it. I don't think reposting is necessary for now. --DD
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using .testcase and .check in continuous integration test

[Lodewijk Duymaer van Twist]

Hi Dominique, 

Thank you for investigating. You're fix works. Should I repost this as a bug 
with your fix, or will this be picked up as is right now? 

Kind regard, 

Lodewijk 


Van: "Dominique Devienne" <ddevie...@gmail.com> 
Aan: "sqlite-users" <sqlite-users@mailinglists.sqlite.org> 
Verzonden: Dinsdag 17 oktober 2017 13:38:13 
Onderwerp: Re: [sqlite] Using .testcase and .check in continuous integration 
test 

On Tue, Oct 17, 2017 at 12:30 PM, Lodewijk Duymaer van Twist < 
lodew...@adesys.nl> wrote: 

> That would be an other way, but what I'm looking for is using the Command 
> Line Shell ".testcase" and ".check" method. 
> 

OK. That's new information :) 


> Create an example test file: 
> echo ".testcase 100 
> SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1); 
> .check 1||1 
> 
> .testcase 110 
> SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1); 
> .check 1|5|1" > test-script.sql 
> 
> Execute the test: 
> sqlite3 < test-script.sql 
> 
> Bail will not work for that. 
> 

shell.c needs to change for that, by 1 character I believe, in 
do_meta_command. 
In case of a failure, set rc to 1 instead of 2. That way the shell no 
longer exits 
with 0, but 1, i.e. an error. 

I'm actually surprised the result of do_meta_command() is not subject to 
.bail, 
and always exits if non-zero. 

Either do those changes yourself in your locally compiled shell, 
or hope Richard agrees the behavior should change. --DD 

rc = do_meta_command(azCmd[i], ); 
if( rc ) return rc==2 ? 0 : rc; 

static int do_meta_command(char *zLine, ShellState *p){ 
... 
/* Cancel output redirection, if it is currently set (by .testcase) 
** Then read the content of the testcase-out.txt file and compare against 
** azArg[1]. If there are differences, report an error and exit. 
*/ 
if( c=='c' && n>=3 && strncmp(azArg[0], "check", n)==0 ){ 
char *zRes = 0; 
output_reset(p); 
if( nArg!=2 ){ 
raw_printf(stderr, "Usage: .check GLOB-PATTERN\n"); 
rc = 2; 
}else if( (zRes = readFile("testcase-out.txt", 0))==0 ){ 
raw_printf(stderr, "Error: cannot read 'testcase-out.txt'\n"); 
rc = 2; 
}else if( testcase_glob(azArg[1],zRes)==0 ){ 
utf8_printf(stderr, 
"testcase-%s FAILED\n Expected: [%s]\n Got: [%s]\n", 
p->zTestcase, azArg[1], zRes); 
rc = 2; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< change to 1 
}else{ 
utf8_printf(stdout, "testcase-%s ok\n", p->zTestcase); 
p->nCheck++; 
} 
sqlite3_free(zRes); 
}else 
... 

meta_command_exit: 
if( p->outCount ){ 
p->outCount--; 
if( p->outCount==0 ) output_reset(p); 
} 
return rc; 
} 
___ 
sqlite-users mailing list 
sqlite-users@mailinglists.sqlite.org 
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using .testcase and .check in continuous integration test

[Dominique Devienne]

On Tue, Oct 17, 2017 at 12:30 PM, Lodewijk Duymaer van Twist <
lodew...@adesys.nl> wrote:

> That would be an other way, but what I'm looking for is using the Command
> Line Shell ".testcase" and ".check" method.
>

OK. That's new information :)


> Create an example test file:
> echo ".testcase 100
> SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1);
> .check 1||1
>
> .testcase 110
> SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1);
> .check 1|5|1" > test-script.sql
>
> Execute the test:
> sqlite3 < test-script.sql
>
> Bail will not work for that.
>

shell.c needs to change for that, by 1 character I believe, in
do_meta_command.
In case of a failure, set rc to 1 instead of 2. That way the shell no
longer exits
with 0, but 1, i.e. an error.

I'm actually surprised the result of do_meta_command() is not subject to
.bail,
and always exits if non-zero.

Either do those changes yourself in your locally compiled shell,
or hope Richard agrees the behavior should change. --DD

rc = do_meta_command(azCmd[i], );
if( rc ) return rc==2 ? 0 : rc;

static int do_meta_command(char *zLine, ShellState *p){
...
  /* Cancel output redirection, if it is currently set (by .testcase)
  ** Then read the content of the testcase-out.txt file and compare against
  ** azArg[1].  If there are differences, report an error and exit.
  */
  if( c=='c' && n>=3 && strncmp(azArg[0], "check", n)==0 ){
char *zRes = 0;
output_reset(p);
if( nArg!=2 ){
  raw_printf(stderr, "Usage: .check GLOB-PATTERN\n");
  rc = 2;
}else if( (zRes = readFile("testcase-out.txt", 0))==0 ){
  raw_printf(stderr, "Error: cannot read 'testcase-out.txt'\n");
  rc = 2;
}else if( testcase_glob(azArg[1],zRes)==0 ){
  utf8_printf(stderr,
 "testcase-%s FAILED\n Expected: [%s]\n  Got: [%s]\n",
 p->zTestcase, azArg[1], zRes);
  rc = 2; < change to 1
}else{
  utf8_printf(stdout, "testcase-%s ok\n", p->zTestcase);
  p->nCheck++;
}
sqlite3_free(zRes);
  }else
...

meta_command_exit:
  if( p->outCount ){
p->outCount--;
if( p->outCount==0 ) output_reset(p);
  }
  return rc;
}
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using .testcase and .check in continuous integration test

[Lodewijk Duymaer van Twist]

Hi Dominique, 

That would be an other way, but what I'm looking for is using the Command Line 
Shell ".testcase" and ".check" method. 

Create an example test file: 
echo ".testcase 100 
SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1); 
.check 1||1 

.testcase 110 
SELECT (1,2,3)<(1,2,4), (1,2,3)<(1,NULL,4), (1,2,3)<(2,NULL,1); 
.check 1|5|1" > test-script.sql 

Execute the test: 
sqlite3 < test-script.sql 

Bail will not work for that. 

Kind regards, 

Lodewijk 


Van: "Dominique Devienne" <ddevie...@gmail.com> 
Aan: "sqlite-users" <sqlite-users@mailinglists.sqlite.org> 
Verzonden: Maandag 16 oktober 2017 13:49:18 
Onderwerp: Re: [sqlite] Using .testcase and .check in continuous integration 
test 

On Mon, Oct 16, 2017 at 12:32 PM, Lodewijk Duymaer van Twist < 
lodew...@adesys.nl> wrote: 

> I would like use .testcase and .check in our GitLab Continuous Integration 
> test. 
> 
> GitLab pipelines will check process return code for success or fail. 
> 
> Consider a simple test: 
> lodewijk@DebianDev:~$ sqlite3 database.db3 < test.sql 
> testcase-100 ok 
> testcase-110 ok 
> lodewijk@DebianDev:~$ echo $? 
> 0 
> 
> Now if I would have a failure the return value of the sqlite3 process will 
> also be 0: 
> lodewijk@DebianDev:~$ sqlite3 octalarm.db3 < test/test-languages.sql 
> testcase-100 FAILED 
> Expected: [66] 
> Got: [67 
> ] 
> lodewijk@DebianDev:~$ echo $? 
> 0 
> 
> Is there a nice elegant way of making my CI stop on a failure? 
> 

C:\Users\ddevienne>sqlite3 -bail bad.db "create table foo(id)" && echo OK 
OK 

C:\Users\ddevienne>sqlite3 -bail bad.db "create table bar" && echo OK 
Error: near "bar": syntax error 

C:\Users\ddevienne> 
___ 
sqlite-users mailing list 
sqlite-users@mailinglists.sqlite.org 
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using .testcase and .check in continuous integration test

[Dominique Devienne]

On Mon, Oct 16, 2017 at 12:32 PM, Lodewijk Duymaer van Twist <
lodew...@adesys.nl> wrote:

> I would like use .testcase and .check in our GitLab Continuous Integration
> test.
>
> GitLab pipelines will check process return code for success or fail.
>
> Consider a simple test:
> lodewijk@DebianDev:~$ sqlite3 database.db3 < test.sql
> testcase-100 ok
> testcase-110 ok
> lodewijk@DebianDev:~$ echo $?
> 0
>
> Now if I would have a failure the return value of the sqlite3 process will
> also be 0:
> lodewijk@DebianDev:~$ sqlite3 octalarm.db3 < test/test-languages.sql
> testcase-100 FAILED
> Expected: [66]
> Got: [67
> ]
> lodewijk@DebianDev:~$ echo $?
> 0
>
> Is there a nice elegant way of making my CI stop on a failure?
>

C:\Users\ddevienne>sqlite3 -bail bad.db "create table foo(id)" && echo OK
OK

C:\Users\ddevienne>sqlite3 -bail bad.db "create table bar" && echo OK
Error: near "bar": syntax error

C:\Users\ddevienne>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Using .testcase and .check in continuous integration test

[Lodewijk Duymaer van Twist]

I would like use .testcase and .check in our GitLab Continuous Integration 
test. 

GitLab pipelines will check process return code for success or fail. 

Consider a simple test: 
lodewijk@DebianDev:~$ sqlite3 database.db3 < test.sql 
testcase-100 ok 
testcase-110 ok 
lodewijk@DebianDev:~$ echo $? 
0 

Now if I would have a failure the return value of the sqlite3 process will also 
be 0: 
lodewijk@DebianDev:~$ sqlite3 octalarm.db3 < test/test-languages.sql 
testcase-100 FAILED 
Expected: [66] 
Got: [67 
] 
lodewijk@DebianDev:~$ echo $? 
0 

Is there a nice elegant way of making my CI stop on a failure? 

Kind regards, 

Lodewijk 

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users