(English is not my native language; please excuse typing errors.) Greetings!
I have a fun with my own simple project Lemon Server. It is a simple HTTP Server where HTTP protocol (RFC7230) was implemented with a Lemon Parser. I got a result grammar file but a fresh revision of the Lemon Parser can not parse incoming requests. Lemon Server repo: https://github.com/devAarno/lemon-server/tree/feature/LS-16-Lemon-update Grammar file: https://github.com/devAarno/lemon-server/blob/feature/LS-16-Lemon-update/src/lemonHttp/http11.y As I see in a Fossil repository, on 2017-12-27 18:19:06 a developer 'drh' has added a line "tool/lempar.c:514": assert( i>=0 && i+YYNTOKEN<=sizeof(yy_lookahead)/sizeof(yy_lookahead[0]) ); (SHA3-256: 1b22b42e59793af19c69a2e5f6822883cc2687d4a0d9b9280bbff885276c6baa, https://www.sqlite.org/src/info/1b22b42e59793af1) Last commit (drh on 2018-02-09 15:04:51) says that this line was split into: assert( i>=0 ); assert( i+YYNTOKEN<=(int)sizeof(yy_lookahead)/sizeof(yy_lookahead[0]) ); (SHA3-256: a6c3115483d597fc77ab19fdcfd1d3437cad7e467081ad8c5315fb98c115eed9, https://www.sqlite.org/src/info/a6c3115483d597fc) So, I have some problems with a second assert (i+YYNTOKEN<=(int)sizeof(yy_lookahead)/sizeof(yy_lookahead[0])). My tests say that parser works well without this assert. For example, test "test_byRawRequest1" says: (test in https://github.com/devAarno/lemon-server/blob/feature/LS-16-Lemon-update/tests/t_1.c) parser >>Reduce 35 [http_version ::= H T T P SLASH ZERO|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE DOT ZERO|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE], go to state 19. parser >>... then shift 'http_version', go to state 18 parser >>Shift 'CR', go to state 40 parser >>Return. Stack=[method SP request_target SP http_version CR] parser >>Input 'CLF' in state 40 parser >>Shift 'CLF', pending reduce 54 parser >>Return. Stack=[method SP request_target SP http_version CR CLF] parser >>Input 'CR' with pending reduce 54 parser >>Reduce 54 [crlf ::= CR CLF], go to state 18. parser >>... then shift 'crlf', pending reduce 42 parser >>Reduce 42 [request_line ::= method SP request_target SP http_version crlf], go to state 0. parser >>... then shift 'request_line', go to state 22 parser >>Reduce 48 [http_headers ::=]. parser >>... then shift 'http_headers', go to state 8 parser >>Shift 'CR', go to state 39 parser >>Return. Stack=[request_line http_headers CR] parser >>Input 'CLF' in state 39 parser >>Shift 'CLF', pending reduce 40 parser >>Return. Stack=[request_line http_headers CR CLF] parser >>Input 'CONTROL' with pending reduce 40 parser >>Reduce 40 [finalcrlf ::= CR CLF], go to state 8. DONE parser >>... then shift 'finalcrlf', go to state 42 parser >>WILDCARD CONTROL => ANY parser >>Shift 'CONTROL', go to state 41 parser >>Return. Stack=[request_line http_headers finalcrlf CONTROL] parser >>Input '$' in state 41 parser >>Reduce 41 [http_message ::= request_line http_headers finalcrlf ANY], go to state 0. parser >>... then shift 'http_message', pending reduce -2 parser >>Accept! /home/aarno/NetBeansProjects/lemon-server/tests/t_1.c:165:test_byRawRequest1:PASS ----------------------- 1 Tests 0 Failures 0 Ignored If I uncomment this assert, I shall get: parser >>Reduce 35 [http_version ::= H T T P SLASH ZERO|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE DOT ZERO|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE], go to state 19. parser >>... then shift 'http_version', go to state 18 parser >>Shift 'CR', go to state 40 parser >>Return. Stack=[method SP request_target SP http_version CR] parser >>Input 'CLF' in state 40 parser >>Shift 'CLF', pending reduce 54 parser >>Return. Stack=[method SP request_target SP http_version CR CLF] parser >>Input 'CR' with pending reduce 54 parser >>Reduce 54 [crlf ::= CR CLF], go to state 18. parser >>... then shift 'crlf', pending reduce 42 parser >>Reduce 42 [request_line ::= method SP request_target SP http_version crlf], go to state 0. parser >>... then shift 'request_line', go to state 22 t_1: /home/aarno/NetBeansProjects/lemon-server/src/lemonHttp/http11.c:1245: yy_find_shift_action: Assertion `i+YYNTOKEN<=(int)sizeof(yy_lookahead)/sizeof(yy_lookahead[0])' failed. gdb says: (gdb) print i $2 = 1019 (gdb) print YYNTOKEN $3 = 104 (gdb) print sizeof(yy_lookahead) $4 = 1390 (gdb) print sizeof(yy_lookahead[0]) $5 = 1 (Linux x86-64, Ubuntu 17.10, gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0) I'm not an expert in a Lemon Parser's source code, so I would like to have some explanation about this assert. What is exactly it check? What shall I do to avoid this assert? Is it bug in my code? If you are interested to build my source (branch feature/LS-16-Lemon-update), please, follow: 1. "git clone -b feature/LS-16-Lemon-update https://github.com/devAarno/lemon-server.git"; 2. "cd lemon-server/" 3. "cmake -DWITHTESTS=1 ." (git and cmake are required) 4. "make clean all" (use after ./src/lemonHttp/lempar.c changing) 5. "./tests/t_1" Additional question: how properly use "ParseHTTP11Init()" with "Parse_ENGINEALWAYSONSTACK" definition? I use hack like https://github.com/devAarno/lemon-server/blob/feature/LS-16-Lemon-update/src/lemonHttp/lempar.c:362 https://github.com/devAarno/lemon-server/blob/feature/LS-16-Lemon-update/src/lemonHttp/parser.c:135 Can you show me an example where Lemon Parser works with "Parse_ENGINEALWAYSONSTACK" and unmodified "lempar.c"? Thanks for your attention. Sincerely, Stanislav. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
