Here's a patch that makes several changes to the token reader, and fixes a few other bugs. Please apply.
The token reader can now increase the buffer size as needed, to read long strings and comments. This makes the comment 'continuation' flag unnecessary, so it was removed. Also: * Don't store a leading '%' for comments. * Ensure a non-null character follows a token's data buffer, unless the token is null terminated. * Call pdf_realloc correctly in pdf_buffer_resize. * Ensure pdf_read_char can read more than 4096 bytes of data (this fixes the bug I reported in a previous message, and includes the test case I posted with the report). -- Michael
# Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: [email protected] # target_branch: file:///home/michael/src/%2Blocal/gnupdf/trunk/ # testament_sha1: 3a6fa8159c5a22b9645e5c1135b757b079d28a16 # timestamp: 2009-10-21 03:10:04 -0400 # base_revision_id: [email protected] # # Begin patch === modified file 'ChangeLog' --- ChangeLog 2009-09-20 14:34:41 +0000 +++ ChangeLog 2009-10-21 07:08:09 +0000 @@ -1,3 +1,30 @@ +2009-10-20 Michael Gold <[email protected]> + + * src/base/pdf-types.c (pdf_buffer_resize): Call pdf_realloc correctly. + + * src/base/pdf-token-reader.c: Don't store a leading '%' for comments. + Reported by Pierre Filot. + + * src/base/pdf-token-reader.c: Support strings longer than 32767 + octets. Suggested by Pierre Filot. + + * src/base/pdf-token-reader.h: Likewise. + + * doc/gnupdf.texi: Remove the 'continuation' flag for comment tokens, + and update the implementation limits. + + * src/base/pdf-token.c: Remove the 'continuation' flag. + + * src/base/pdf-token.h: Likewise. + + * src/base/pdf-stm.c (pdf_stm_read_peek_char): If the cache is empty, + rewind the buffer before reading more data from the filter. + + * torture/unit/base/stm/pdf-stm-read-char.c: Add a test case that + reads more than PDF_STM_DEFAULT_CACHE_SIZE bytes from a stream. + + * torture/unit/base/token/pdf-token-read.c: Test long string tokens. + 2009-09-20 Jose E. Marchesi <[email protected]> * configure.ac: Call AC_CONFIG_AUX_DIR. === modified file 'doc/gnupdf.texi' --- doc/gnupdf.texi 2009-09-08 20:44:16 +0000 +++ doc/gnupdf.texi 2009-10-21 07:08:09 +0000 @@ -10380,7 +10380,7 @@ @end deftypefun -...@deftypefun pdf_status_t pdf_token_comment_new (const pdf_char_t *...@var{value}, pdf_size_t @var{size}, pdf_bool_t continuation, pdf_token_t *...@var{token}) +...@deftypefun pdf_status_t pdf_token_comment_new (const pdf_char_t *...@var{value}, pdf_size_t @var{size}, pdf_token_t *...@var{token}) Create a comment token containing a copy of the given data. @@ -10391,8 +10391,6 @@ A pointer to the binary data that will make up the keyword. @item size The amount of data to copy, in octets. -...@item continuation -A boolean value indicating whether this is a continuation of a previous comment token (if false, it's a new comment). @item token A pointer to the newly created token. @end table @@ -10707,30 +10705,6 @@ @end deftypefun -...@deftypefun pdf_bool_t pdf_token_get_comment_continued (const pdf_token_t @var{token}) - -Returns a boolean indicating whether the comment token is a continuation -of a previous token. If the token reader breaks a comment into multiple -tokens, each token except the last will be at least 32767 bytes long. - -...@table @strong -...@item Parameters -...@table @var -...@item token -A token of type PDF_TOKEN_COMMENT. -...@end table -...@item Returns -A boolean value: -...@table @code -...@item PDF_TRUE -The given comment token is a continuation of another. -...@item PDF_FALSE -The token is a complete comment, or the beginning of a comment. -...@end table -...@end table - -...@end deftypefun - @node Encryption @section Encryption @@ -14840,11 +14814,11 @@ @tab -FLT_MAX @tab Smallest real value. @item string in content stream -...@tab 32767 +...@tab SIZE_MAX/2 @tab Maximum length of a string contained in a content stream, in bytes. @item name -...@tab 32767 +...@tab 32768 @tab Maximum length of a name, in bytes. @item indirect object @tab ?? === modified file 'src/base/pdf-stm.c' --- src/base/pdf-stm.c 2009-09-08 21:06:54 +0000 +++ src/base/pdf-stm.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/08 23:05:51 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-21 04:48:36 mgold" * * File: pdf-stm.c * Date: Fri Jul 6 18:43:15 2007 @@ -577,6 +577,7 @@ ret = PDF_OK; if (pdf_buffer_eob_p (stm->cache)) { + pdf_buffer_rewind (stm->cache); ret = pdf_stm_filter_apply (stm->filter, PDF_FALSE); } === modified file 'src/base/pdf-token-reader.c' --- src/base/pdf-token-reader.c 2009-09-05 14:47:16 +0000 +++ src/base/pdf-token-reader.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/05 16:00:57 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-21 04:58:12 mgold" * * File: pdf-token-reader.c * Date: Mon Dec 29 00:45:09 2008 @@ -32,9 +32,10 @@ #include <pdf-token-reader.h> -static INLINE int can_store_char (pdf_token_reader_t reader); static INLINE pdf_status_t store_char (pdf_token_reader_t reader, pdf_char_t ch); +static INLINE pdf_status_t store_char_grow (pdf_token_reader_t reader, + pdf_char_t ch); static pdf_status_t exit_state (pdf_token_reader_t reader, pdf_u32_t flags, pdf_token_t *token); static INLINE pdf_status_t enter_state (pdf_token_reader_t reader, @@ -83,7 +84,7 @@ err = PDF_ERROR; len = snprintf (decpt, sizeof (decpt), "%#.0f", 1.0); - if (len <= 0 || len >= sizeof (decpt)) /* shouldn't happen */ + if (len <= 0 || (pdf_size_t)len >= sizeof (decpt)) /* shouldn't happen */ goto fail; err = PDF_ENOMEM; @@ -95,8 +96,11 @@ memcpy (new_tokr->decimal_point, &decpt[1], len); } - /* max string size 32767 + terminating '\0' */ - new_tokr->buffer = pdf_buffer_new (32768); + /* buffer_size_min is the default buffer size, which is also the maximum + * size for keywords, names, numbers, etc.; strings and comments will + * enlarge the buffer to whatever size is needed. */ + new_tokr->buffer_size_min = 32768; + new_tokr->buffer = pdf_buffer_new (new_tokr->buffer_size_min); if (!new_tokr->buffer) goto fail; @@ -117,11 +121,23 @@ return err; } +static void +reset_buffer (pdf_token_reader_t reader) +{ + reader->buffer->wp = 0; + if (reader->buffer->size > reader->buffer_size_min) + { + /* Try to shrink the buffer, but don't worry if it fails. */ + pdf_buffer_resize (reader->buffer, reader->buffer_size_min); + } +} + pdf_status_t pdf_token_reader_reset (pdf_token_reader_t reader) { enter_state (reader, PDF_TOKR_STATE_NONE); reader->substate = 0; + reset_buffer (reader); return PDF_OK; } @@ -196,18 +212,12 @@ return PDF_EAGAIN; } - if (store_char (reader, ch) != PDF_OK) - { - /* the comment buffer is full, so split the token */ - rv = flush_token (reader, flags, token); - if (rv != PDF_OK) - return rv; - - reader->intparam = 1; /* mark the next token as a continuation */ - return store_char (reader, ch); /* can't fail this time */ - } - - return PDF_OK; + if (!(flags & PDF_TOKEN_RET_COMMENTS)) + reader->substate = 1; + if (reader->substate == 1) + return PDF_OK; /* we don't care about this comment */ + + return store_char_grow (reader, ch); default: ; } @@ -255,8 +265,7 @@ { case 37: /* '%' */ enter_state (reader, PDF_TOKR_STATE_COMMENT); - reader->intparam = 0; - return store_char (reader, ch); + return PDF_OK; case 40: /* '(' */ enter_state (reader, PDF_TOKR_STATE_STRING); reader->intparam = 0; @@ -306,6 +315,7 @@ /* fall through */ case PDF_TOKR_STATE_KEYWORD: + /* Note: numbers are treated as keywords until flush_token is called. */ return store_char (reader, ch); case PDF_TOKR_STATE_NAME: @@ -339,16 +349,25 @@ default: assert (0); + return PDF_ERROR; } - - return store_char (reader, ch); } static INLINE int can_store_char (const pdf_token_reader_t reader) { - return reader->buffer->wp < (reader->buffer->size - 1); + return reader->buffer->wp < reader->buffer->size; +} + +static pdf_status_t +enlarge_buffer (pdf_token_reader_t reader) +{ + pdf_size_t size = reader->buffer->size, newsize = size * 2; + if (newsize < size) + return PDF_EIMPLLIMIT; + + return pdf_buffer_resize (reader->buffer, newsize); } static INLINE pdf_status_t @@ -361,6 +380,19 @@ } static INLINE pdf_status_t +store_char_grow (pdf_token_reader_t reader, pdf_char_t ch) +{ + if (!can_store_char (reader)) + { + pdf_status_t rv = enlarge_buffer(reader); + if (rv != PDF_OK) + return rv; + } + reader->buffer->data[reader->buffer->wp++] = ch; + return PDF_OK; +} + +static INLINE pdf_status_t enter_state (pdf_token_reader_t reader, enum pdf_token_reader_state_e state) { @@ -387,11 +419,10 @@ return PDF_EEOF; /* can't continue parsing after EOF */ case PDF_TOKR_STATE_COMMENT: - if (!(flags & PDF_TOKEN_RET_COMMENTS)) - goto finish; + if ((reader->substate == 1) || !(flags & PDF_TOKEN_RET_COMMENTS)) + goto finish; /* don't return a token */ - rv = pdf_token_comment_new (data, datasize, - reader->intparam /*continued*/, &new_tok); + rv = pdf_token_comment_new (data, datasize, &new_tok); break; case PDF_TOKR_STATE_KEYWORD: @@ -481,7 +512,7 @@ reader->beg_pos = reader->state_pos; finish: - reader->buffer->wp = 0; + reset_buffer (reader); return PDF_OK; } @@ -516,30 +547,35 @@ /* fall through */ case 0: /* no special state */ - if (ch == 92) /* '\\' */ - { - reader->substate = 2; /* start an escape sequence */ - return PDF_OK; - } - else if (ch == 41 && reader->intparam <= 0) /* ')'; end of string */ - { - reader->intparam = -1; - return exit_state (reader, flags, token); - } + { + if (ch == 92) /* '\\' */ + { + reader->substate = 2; /* start an escape sequence */ + return PDF_OK; + } + else if (ch == 41 && reader->intparam <= 0) /* ')'; end of string */ + { + reader->intparam = -1; + return exit_state (reader, flags, token); + } - if (!can_store_char (reader)) - return PDF_EIMPLLIMIT; - else if (ch == 40) /* '(' */ - ++reader->intparam; - else if (ch == 41) /* ')' */ - --reader->intparam; - else if (ch == 13) /* '\r' */ - { + pdf_bool_t wasCR = (ch == 13); + if (wasCR) ch = 10; /* treat as LF */ - reader->substate = 1; /* ignore the next char if it's LF */ - } - - return store_char (reader, ch); + rv = store_char_grow (reader, ch); + + if (rv == PDF_OK) + { + if (wasCR) /* '\r' */ + reader->substate = 1; /* ignore the next char if it's LF */ + else if (ch == 40) /* '(' */ + ++reader->intparam; + else if (ch == 41) /* ')' */ + --reader->intparam; + } + + return rv; + } case 2: /* just saw a '\\' (starting an escape sequence) */ reader->substate = 0; @@ -573,14 +609,14 @@ /* for any other character, including '(', ')', and '\\', * store the same character (dropping the leading backslash) */ - return store_char (reader, ch); + return store_char_grow (reader, ch); case 3: /* saw 1 digit of an octal escape */ /* fall through */ case 4: /* saw 2 digits of an octal escape */ if (ch < 48 || ch > 48+7) /* not digits '0'--'7' */ { - rv = store_char (reader, reader->charparam); + rv = store_char_grow (reader, reader->charparam); if (rv != PDF_OK) return rv; /* ch isn't part of the escape sequence, so retry */ @@ -592,7 +628,7 @@ reader->charparam = ((reader->charparam & 0x1f) << 3) | (ch - 48); if (reader->substate == 4) /* this was the final digit */ { - rv = store_char (reader, reader->charparam); + rv = store_char_grow (reader, reader->charparam); if (rv != PDF_OK) return rv; reader->substate = 0; @@ -638,7 +674,7 @@ if (reader->substate == 2) { /* the last digit is missing; assume it's '0' */ - rv = store_char (reader, reader->charparam << 4); + rv = store_char_grow (reader, reader->charparam << 4); if (rv != PDF_OK) return rv; } @@ -656,7 +692,7 @@ return PDF_OK; } - rv = store_char (reader, (reader->charparam << 4) | ch); + rv = store_char_grow (reader, (reader->charparam << 4) | ch); if (rv == PDF_OK) reader->substate = 1; return rv; === modified file 'src/base/pdf-token-reader.h' --- src/base/pdf-token-reader.h 2009-09-05 20:59:39 +0000 +++ src/base/pdf-token-reader.h 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/05 22:56:21 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-20 21:44:20 mgold" * * File: pdf-token-reader.h * Date: Mon Dec 29 00:45:09 2008 @@ -72,9 +72,10 @@ /* Token reader states (from pdf_token_reader_state_e): * NONE - Initial state; not reading a token. - * COMMENT - Reading a comment. buffer collects the comment bytes, excluding - * intparam is 1 if this token is continued from a previous token, - * and 0 otherwise. + * COMMENT - Reading a comment. buffer collects the comment bytes. + * Substates: + * 0 - normal state + * 1 - don't produce a token * KEYWORD - Reading some regular characters into buffer; this could result * in a symbol like "null", or a number. * NAME - Reading a name (which starts with '/'). @@ -125,6 +126,7 @@ int intparam; pdf_buffer_t buffer; /***/ + pdf_size_t buffer_size_min; }; #endif === modified file 'src/base/pdf-token.c' --- src/base/pdf-token.c 2009-09-05 14:47:16 +0000 +++ src/base/pdf-token.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/05 16:01:21 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-20 22:09:19 mgold" * * File: pdf-token.c * Date: Sat Jul 7 03:04:30 2007 @@ -87,14 +87,16 @@ goto fail; rv = PDF_ENOMEM; - new_obj->value.buffer.data = pdf_alloc (nullterm ? size + 1 : size); + new_obj->value.buffer.data = pdf_alloc (size + 1); if (!new_obj->value.buffer.data) goto fail; new_obj->value.buffer.size = size; memcpy (new_obj->value.buffer.data, value, size); - if (nullterm) - new_obj->value.buffer.data[size] = 0; + + /* If the value isn't null terminated, append a non-null character + * to catch bugs. */ + new_obj->value.buffer.data[size] = nullterm ? 0 : 'X'; *token = new_obj; return PDF_OK; @@ -138,11 +140,7 @@ case PDF_TOKEN_REAL: return token1->value.real == token2->value.real; - case PDF_TOKEN_COMMENT: - if (token1->value.comment.continued != token2->value.comment.continued) - return PDF_FALSE; - - /* fall through */ + case PDF_TOKEN_COMMENT: /* fall through */ case PDF_TOKEN_STRING: /* fall through */ case PDF_TOKEN_NAME: /* fall through */ case PDF_TOKEN_KEYWORD: @@ -193,9 +191,8 @@ token->value.buffer.size, new); case PDF_TOKEN_COMMENT: - return pdf_token_comment_new (token->value.comment.data, - token->value.comment.size, - token->value.comment.continued, + return pdf_token_comment_new (token->value.buffer.data, + token->value.buffer.size, new); default: /* Should not be reached: make the compiler happy */ @@ -329,10 +326,8 @@ pdf_status_t pdf_token_comment_new (const pdf_char_t *value, pdf_size_t size, - pdf_bool_t continued, pdf_token_t *token) { - pdf_status_t rv; pdf_size_t i; for (i = 0; i < size; ++i) { @@ -341,11 +336,7 @@ return PDF_EBADDATA; } - rv = pdf_token_buffer_new (PDF_TOKEN_COMMENT, value, size, 0, token); - if (rv == PDF_OK) - (*token)->value.comment.continued = !!continued; - - return rv; + return pdf_token_buffer_new (PDF_TOKEN_COMMENT, value, size, 0, token); } pdf_size_t @@ -362,13 +353,6 @@ return comment->value.buffer.data; } -pdf_bool_t -pdf_token_get_comment_continued (const pdf_token_t comment) -{ - assert (comment && comment->type == PDF_TOKEN_COMMENT); - return comment->value.comment.continued; -} - /** keywords *****/ === modified file 'src/base/pdf-token.h' --- src/base/pdf-token.h 2009-09-05 20:59:39 +0000 +++ src/base/pdf-token.h 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/05 22:56:05 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-20 21:38:20 mgold" * * File: pdf-token.h * Date: Sat Jul 7 01:10:11 2007 @@ -80,7 +80,7 @@ pdf_status_t pdf_token_keyword_new (const pdf_char_t *value, pdf_size_t size, pdf_token_t *token); pdf_status_t pdf_token_comment_new (const pdf_char_t *value, pdf_size_t size, - pdf_bool_t continued, pdf_token_t *token); + pdf_token_t *token); pdf_status_t pdf_token_valueless_new (enum pdf_token_type_e type, pdf_token_t *token); pdf_status_t pdf_token_dup (const pdf_token_t token, pdf_token_t *new); @@ -112,7 +112,6 @@ /* Managing comments */ pdf_size_t pdf_token_get_comment_size (const pdf_token_t token); const pdf_char_t *pdf_token_get_comment_data (const pdf_token_t token); -pdf_bool_t pdf_token_get_comment_continued (const pdf_token_t token); /* END PUBLIC */ @@ -177,16 +176,6 @@ pdf_size_t size; }; -struct pdf_comment_s -{ - pdf_char_t *data; - pdf_size_t size; - /* This structure shares a common initial sequence with pdf_token_buffer_s, - * so the above fields can be accessed via token->value.buffer. */ - - pdf_bool_t continued; /* is data continued from a previous token? */ -}; - /* A `pdf_token_s' structure stores a PDF object. The object may be of any type (including NULL). */ @@ -201,7 +190,6 @@ struct pdf_token_buffer_s buffer; pdf_i32_t integer; pdf_real_t real; - struct pdf_comment_s comment; } value; }; === modified file 'src/base/pdf-types.c' --- src/base/pdf-types.c 2009-09-08 21:06:54 +0000 +++ src/base/pdf-types.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/08 23:03:23 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-21 05:18:24 mgold" * * File: pdf-types.c * Date: Sun Feb 10 21:33:44 2008 @@ -1266,9 +1266,11 @@ pdf_status_t pdf_buffer_resize (pdf_buffer_t buffer, pdf_size_t newsize) { - if (pdf_realloc (buffer->data, newsize) != PDF_OK) + pdf_char_t *newdata = pdf_realloc (buffer->data, newsize); + if (!newdata) return PDF_ENOMEM; + buffer->data = newdata; buffer->size = newsize; buffer->rp = PDF_MIN (buffer->rp, newsize); buffer->wp = PDF_MIN (buffer->wp, newsize); === modified file 'torture/testdata/Makefile.am' --- torture/testdata/Makefile.am 2009-09-15 20:52:06 +0000 +++ torture/testdata/Makefile.am 2009-10-21 07:08:09 +0000 @@ -17,6 +17,7 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. check-local: +mg-disabled: @CHMOD@ a-r $(srcdir)/TD00004 @CHMOD@ a-w $(srcdir)/TD00005 === modified file 'torture/unit/base/stm/pdf-stm-read-char.c' --- torture/unit/base/stm/pdf-stm-read-char.c 2009-08-05 20:32:56 +0000 +++ torture/unit/base/stm/pdf-stm-read-char.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "2009-08-05 21:58:33 davazp" +/* -*- mode: C -*- Time-stamp: "2009-10-21 02:37:22 mgold" * * File: pdf-stm-read-char.c * Date: Sat Sep 20 16:59:27 2008 @@ -182,6 +182,61 @@ /* + * Test: pdf_stm_read_char_004 + * Description: + * Read more than PDF_STM_DEFAULT_CACHE_SIZE bytes from a stream. + * Success condition: + * The entire stream should be read successfully. + */ +START_TEST (pdf_stm_read_char_004) +{ + pdf_status_t ret; + pdf_stm_t stm; + pdf_char_t *buf; + pdf_char_t ret_char, ch; + pdf_size_t buf_size; + pdf_size_t buf_pos; + pdf_size_t i; + + /* Create a buffer with some contents */ + buf_size = 42000; + pdf_init(); + + buf = pdf_alloc (buf_size); + fail_if(buf == NULL); + for (i = 0, ch = 0; i < buf_size; ++i, ch = (ch+1) % 251) + buf[i] = ch; + + /* Create the stream */ + ret = pdf_stm_mem_new (buf, + buf_size, + 0, /* Use the default cache size */ + PDF_STM_READ, + &stm); + fail_if(ret != PDF_OK); + + /* Read all characters from the stream */ + for (buf_pos = 0, ch = 0; buf_pos < buf_size; ++buf_pos, ch = (ch+1) % 251) + { + ret = pdf_stm_read_char (stm, &ret_char); + if (ret != PDF_OK) + printf("pdf_stm_read_char_004 failed at %d bytes, ret=%d\n", buf_pos,ret); + fail_if(ret != PDF_OK); + fail_if(ret_char != ch); + } + + /* Try to read a character from the stream */ + ret = pdf_stm_read_char (stm, &ret_char); + fail_if(ret != PDF_EEOF); + + /* Destroy data */ + pdf_dealloc (buf); + pdf_stm_destroy (stm); +} +END_TEST + + +/* * Test case creation function */ TCase * @@ -192,6 +247,7 @@ tcase_add_test(tc, pdf_stm_read_char_001); tcase_add_test(tc, pdf_stm_read_char_002); tcase_add_test(tc, pdf_stm_read_char_003); + tcase_add_test(tc, pdf_stm_read_char_004); return tc; } === modified file 'torture/unit/base/token/pdf-token-read.c' --- torture/unit/base/token/pdf-token-read.c 2009-09-05 14:47:16 +0000 +++ torture/unit/base/token/pdf-token-read.c 2009-10-21 07:08:09 +0000 @@ -1,4 +1,4 @@ -/* -*- mode: C -*- Time-stamp: "09/09/05 16:06:57 jemarch" +/* -*- mode: C -*- Time-stamp: "2009-10-21 06:43:46 mgold" * * File: pdf-token-read.c * Date: Wed Jan 14 05:44:48 2009 @@ -15,7 +15,7 @@ #include <pdf.h> -#define STR_AND_LEN(s) (s),(sizeof(s)-1) +#define STR_AND_LEN(s) (pdf_char_t*)(s),(sizeof(s)-1) /* Initialize an in-memory reader stream (pdf_stm_t *stm) * with the given string (a char* constant). */ @@ -78,7 +78,7 @@ } /* - * Test: pdf_token_read_001 + * Test: pdf_token_read_toktypes * Description: * Read various tokens from an in-memory stream, and check whether they * match the expected values. @@ -86,7 +86,7 @@ * Each token matches the expected one (according to pdf_token_equal_p), * and no tokens remain afterwards. */ -START_TEST(pdf_token_read_001) +START_TEST(pdf_token_read_toktypes) { pdf_stm_t stm; pdf_token_reader_t tokr; @@ -122,14 +122,14 @@ END_TEST /* - * Test: pdf_token_read_002 + * Test: pdf_token_read_eos * Description: * Test the PDF_TOKEN_END_AT_STREAM flag. * Success condition: * The stream should be positioned after the '\n' character (at '<'), and * the token reader should act as if it reached the end of the input file. */ -START_TEST(pdf_token_read_002) +START_TEST(pdf_token_read_eos) { pdf_stm_t stm; pdf_token_reader_t tokr; @@ -152,14 +152,58 @@ END_TEST /* + * Test: pdf_token_read_longstring + * Description: + * Try to read a string longer than the normal buffer size. + * Success condition: + * A token containing the string should be produced. + */ +START_TEST(pdf_token_read_longstring) +{ + const pdf_size_t filesize = 42000; + pdf_char_t *file; + pdf_stm_t stm; + pdf_token_reader_t tokr; + pdf_token_t token = NULL; + pdf_status_t rv; + pdf_size_t i, j; + + pdf_init(); + + /* make long string '(XX'...'XX)' */ + file = nonnull(pdf_alloc(filesize)); + memset(file, 'X', filesize); + file[0] = '('; + file[filesize-1] = ')'; + + fail_unless(PDF_OK == pdf_stm_mem_new( file, filesize, + 0 /*cache_size*/, PDF_STM_READ /*mode*/, &stm )); + INIT_TOKR(&tokr, stm); + + rv = pdf_token_read(tokr, 0, &token); + fail_unless(rv == PDF_OK); + if (token) + { + fail_unless(pdf_token_get_type(token) == PDF_TOKEN_STRING); + fail_unless(pdf_token_get_string_size(token) == filesize-2); + pdf_token_destroy(token); + } + + fail_unless( tokr_eof(tokr, 0) ); + pdf_dealloc(file); +} + +END_TEST +/* * Test case creation function */ TCase * test_pdf_token_read (void) { TCase *tc = tcase_create("pdf_token_read"); - tcase_add_test(tc, pdf_token_read_001); - tcase_add_test(tc, pdf_token_read_002); + tcase_add_test(tc, pdf_token_read_toktypes); + tcase_add_test(tc, pdf_token_read_eos); + tcase_add_test(tc, pdf_token_read_longstring); return tc; } # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWZfJZyUADl9/gHV0AgB7//// /+/eXr////5gF+x2u7fd7Pcu8c9PT3Xpt7ue9lbMlhJWg0CQa7eg8OwYrWtKU1pWjQLYK9ZHTNtJ PWmzAVhKahACj0TZCYEyZTxJj1KGmI9IaGnqAD1D0mg9IJQmjRMgCp6mFJ+TVHqBtQAMg0AAAAAA aNBCQ0FDyQxD0TIyAADQA0AA0AAJCQiaJkyFNqninpqep+Snk0xT02pPTTKekeTKaeo9EAHqAODI NGgDJpiGmgyDEMIGgNGJiNAAAJEggmRoAjQBMTImp4Un6kfpR+hlT9U9QfqnpD1NA08ScGZpWBsN unwc9m3I7V7825ybWnISCfW/UPxE6E12p9mIWLtnb0zJ5rZakVVDmLKwVb6qwDKPnhMyU4Ml+XMV 3XHrCu8AmG0boxjBUKgwAH+5mQ8/CyY5hlL6DVAoi0w0w15SlRKKtEJVLgqwo4Pp97gl61eeeqys OzG6aLVGzY47tWZcYuKUXLXoaHFeL9UZkXjveVSyJ4kGid4FkMjPYQpHDk0Ti7VsH0Q53fZz3z5N OGTcQr5VSSjBtbeWWdMLqIgeh55hoolNfFWss52cypidZQ7zi4FJuuIh7rVgy1JuZjL1i6zzc1sM iCNVRNqrFcqDJoAqggSwC7Y0klE1EQV0UlbrOVDXe3HDmZIhpCIaAf7bD+bObMBmJdZlh1I5jL3x EOV0HQfzD/f9F0Yq4KOM92ks99d/R6kCooZKilc1CfjGVwkRz0WM06YKmwe3DuFb5/qmcyxupjPB xSUDSZi3hz/IDmA5taRxYFPn6gXOSGIbBDaTYIPvsLBj35kSX/iQju7i6WqEFUFT7q+kpD8X4yi4 RzjLhFsRCmIs+ClItuzaTdcUrBZnd6EgUVi2RYHk8qSIXJUya0iD4hp0rKH9V54yi5jX+9bxMvUZ hkXDD6LSSMt1KXbAv4S+akYnIteuJE6Krjow8qIwMxVLUgtXEaqJbFWlJue03QJusQ1FsrxPRM+W SDUIkWqn/M+krWX+jv3EJppoEzTIUp2bIP6K8T7SwYGhmaMUfOfC+m3Pj+hB8o2hsweYt0l78kLd Ys9BiYujV9pbymMvdNR053sN2GEuBIy3V5YTeMnE5t3s3SUHPp48Yp94zquWuu7bb9fPWe86WnVd NGPt1BlFX4O4ZZa6rFbMPkMcTNmpXwO+Xlmxyvc0ZdpbUvVcCe/aJfmUhEHgrLr7A/UEeMyP9fU5 3N73Xa2Zi5NWU0z3ztv47aje81Vvzla431040z7uX1D8T3eIFurr0ZqgcPgPCSacSbLeXpSt+riF E6K4SqoXn4Duk1Tz8qyJLAZxInE2Wc1VVmqk7fGIaQPTkw29UDZesAVhSqpeiDVUMwkGc+xrX3xL 8LHiInXmB1OEJmBUsHcZ7NKR5x633C8cCIazRVSi+y/eS6XpdpMusqaZ4OPFNNSn6cdHD55t8i5G /LW96KcvGk/IaeKlDIpAuu5zX17eV0H7c90pCKVTPg9WNzJdbtKmkrxhQMtnoSMfZJuWXp09My/o 8yNwDEE9/mkgRwTMeiM8ono3KhMdyk9yz6VxHmIwPaQUbnnag1yAerpsuf6uBWReVeJLt3fvKR42 LI9SiW9oW7VGkRfvXCHQgpBdRc6LO49fLRpwOdNaVPXpTnGzf5cHBbJFwqngVRUksMuhHZR4GKzk pFFX9ODKdUVlcl6Z3CNz5CN8d9QzPm/2uN/srvwphOX5u6uvo2FLcfoVSaEz1QQOBpw+thQfBxg4 lJCX9QBYBZjVm2vWz7+Y8oGLbqN9cm1nbMMWyzlQW5ngfu4aBgLNtWeXZoY7nSATgfE6V/VDI+ze AkAmAt7f917hO2+z6KJGCyeqSWoiJaAwNWwYTFwg2s5zvgILEFBooIVqM5qGeCkSHqHAr/FaA4CW ZpB//RuFzFcfvU8RCXNawXBy457VQ1EMuAPYgDVBeznx/kkttsUYmN9i1+as7wf1Hc38vPolls4P ZT0oooKLEG5pBKFNzYxv9BKJh1gLyOhDAYfkxtIke6AWKWowIf0w25JEsomxUAmoD58vLD6VD/LL OjstJey8SCSJmTiAbby5/PSR23rYWYYHxpC/Bq5BMCJ5wmQVMD9AhTENHKYEAkFigYQlwaDAOp2K GpkIJZEIKGjsiCpIZh1I0aFXOhQzEt5gRqotkzLvfdRoW1KakRQQtpFIXsG43GlJGvMQoothq9w7 IKEKahIY2mgQ0oNSxYKmNRn6bDWaypLDEeRcYB+/0N3EjwDIQrQ8d6pplgzSloaQ2ki7LMqoRu87 G5EEarRPdJXFs0yxllpQ3WtYa5nqL5dRDuqg2gwo15IMi4jYIV0tWCMQnFnJiTRYFRGKJzW05CgS u4RUdrHy3EhcpcbS3M8ZTjDdpP3zK5If1VUQwK3ZdV0LgjJQOs0khqRLpOk4zSREGeQLIaaRmSHa uhmPtrBKowe6RJbM3uoMdhe3zWflnXkN2BzUiXS7AjdjMwKdhBLmR6heSwyQV+cIJ9a6WwJvrfJ5 LPmiq3orwsitOF602RyHQItuwoxpxuMDBUSD0cQIQhSzAwY0K3KEQ7tBVFKM4gg9IZZxlA0U48I4 H7GfGpVNgqTcYUjF5I0EMVok5TWRuyMqqqp3jpuSHlS5o+Q1COGZrEZzESp3liC0sNJtDkNZefqH kGk7Fl88FY+TPn07rxsjjW2tkisU6fmk47TZa3ydLCF53cKnacirXZpkNYaip2NLhoYimU6zibHx l4PUQ2g4OhDek5hhX1nd5ubrBVGwpwQ2qYMDD3HtyRRVVO41mQHRcwTRzzSBhxRzvPfT3ChfuNM+ H1VtVk/r3lo+9tFlKtIWdwGuquFikrMWJMCdMxLEpgLVGRhiEYS8Ic+iSLOoUCLjpFCsqD+vQoFB bp5HiCeJPO4yly8rTHjgqZaHYE7xyDxNs9CRkULy9YSNY9WmGsmxFkQ6RbfZFWpzdkqN4bXw+ST3 O8LHArmpvW0ksoVMVq6KqAZoqUNQkDT3OSc0zKKMkjA+QdGZKgxIu95oOQYmqzncwOJkaRilxqdH JuTkMTNzM3T5QSIMzgmewQTHHDKpstdq02zrsoYqrFSx1yVaj1gtkOSgVxoOeLPG4p5Hf2NyNynC U2OhTkyEFrTji3JSdBkbYgpyZ6oJgyINzY55zKG5sKQdjMmaTMHIIXrOs2rk+wqC337sYwynXXZW lbN1I30ZRynRJB3BEp8lFar81ke4pPiQVHsZYDHeVteBRVZjxJQanh4WuCQgmRgKlRzuLhBmd5yM eB5CD0wsRlu9pM69PnJ4bWz0fXRuHLAmwoIlyZUmZoxOZW2IMUkpKzlrrxxqUsVINS49cj3ipQvg mNcYb1WNClNyllJdDmmVLO17Ge25kWFLk5yPJODFLXtk11lNs2nXl3nab2QJFRhBMbjkKSA5xCvE JUar0KBycRRjQuScxcqyjK/qDczrqcuWxt0SYyLky2TmQpgqObkxSgecgfBVo0h5swHHg5QPictR zPaGQLMN+3LyPsRFbhfPjVUgGbMsZgmCaFVwWYlfdt3QOructHN2Snc8RQJRU2FAzKhXaWS7BIUX 5CMctpc6mTBq708abd/O63Xlbtt7mzc/m1n0nyh4VxuA0Js9LIKw0ftTBpjH7VGMcUYDYrtGvb0Q U0IoDY1w6RVUjZIuwkMgDSL5IXY7/kA8m+BWdGrKqnzCAu0pw286FxAlRb6x/wPgefoD7T443ahN oaBsbyoDI+8kDjUONlix/tJHvVkf9IeRmr9B+eRnR3Iz00itL/5mF/6WmCSJmLNhWjLprGv1Ui5A VMZRQ/U0smba3I/W5D0htsefaygmiY0N2pFRZIyovyaF+MQGDKbmY3BddfgErRfWCpKguASRJLat jTMoCYmDEyRtLDMEcjARFA7p+9J998bWHw7Tvi5ldw8/2zdIXsJfEZTY3c+SJFFyigli4Uh5hvB+ FPwUKq3r8KUCkFgLpn6JtuoRBgWG6bGBF7UZFCWFkQWUwRz+JfCM7EYQTYETwjweHM8jQY/Y9bCk jIUQRpBnMyWB3CDVQG2TUS1tl5rDZs0QKKlpkhh202gz9lIYhPIKzBWDBfccp8z5H4HyIKEj5ky4 PoQYYsqft/pIqbdNxaHw2qSFeNAeAz1A0JL5GRmNhkbzOswlC3gsuXtYd+v3FdmWltp0/vSksAfz gZBB0oWKDeGWqdcJix1XVnXvgudgB1wLcELEpAuVXVDNoVetWnBJMKVpeutJUolpkzTKxo500tpW ubX93HA5sx0jrxoJyIjLXHcbAmHhqJyYsKzKQJmAu43n2i+SCZQ2Kf+eBxOUQrCOBiZDPA6Oo4fl fFhdoWn0r1/Gq6op9kWge2Cjg7RLvhZKWaaOdNHr0Ko/ikIPsIGSFB5wcAo1pS2nMdJ3nMRmMcaz UZBxykjTEQ41jzopONgDkLMjjuudme0H5z6rOdSwPsjyO46SQ1MEtheHWjDoklIpC5kKa8u7tRRh +OI4QiS3h0B7k7q0FVNLBh9ojj2mNiZtkG1UBaLQYJErykICP4iHQHJEAJEi3fBSrGmEJEzmOU8J k7O49siwZ5i48DFC4+rBCzlwzOVZGoBZAlpzCWoGvE38/hrk44z64onxPuv9ZhLSkxGIiHnzoMtY gNJet4jx98oDNGcmlCHkQkkGIwBgwVA7eJnXL0Yor3nqWpcfaPwXa1CZeyB2HdjORV2I9sQdTuSH dcoI8vPSy8A+Wgj6os4eebbO8eNERrX3YiNvY1mLaAyCamDSJILTcHQU9ugQvI9JsPMcDYHOaSQv 9dCWN2gxPdQUhGFJlKAMeHCmnTMYgakmsQixysK8AkcBE0iBKYtLQYm4yOBafc20JBY0XEoMmm0H gVNR1eNaluls2txOw7SWzI34c9ECYyGhnWKN+cnqMR1eKzH6MNextshQUxOSZmG1yoWuaCqNQhhL tXCQTSVOk8PBNY9mF3ak80Goa+QBsG2xA0waaGrQUTBvSqQqRZI9Zwh3IhM0QQHjHfU1fca0bhsH 1kIvIt+LBhKNOi12paH5DITChmskANiDrAGcCAkE4hJEi/l25AehiJEQdMfM68r6ooWE0ByAzIIz eXD4zIUmQiNob99BQcC8dJMPKjeQMiQkNDBiaELEN6/MYxhceReZlYQaUIWBsBZkCLS7+KR7CJLy SPhISmz82UfpZMZ0+yEqNesaMhKTOHml2iLLRbWB0M7CCglE/jzBWk1EECUB3fuNxbv9/2IXZ8G6 Ag/X9xrDOZkGczp4wv/ewy70jy0kIGDziFtXYIUyazsSaBMqj5wczEIl6v4CFCD2Mm02J8j6M+Ns fkKQdpDazMzVa+X1vcTyekYuD0EUe/RIG131mfsCnuAL8DM/PrG16P1wpVHywyRzAp4f3GrvCOuk IbFx/ElLX1gVPtv4PwGNF58BK7K/QMbb3tLGmkkfgHVoXvvz9kwbBgFhEjnax0Gf4iCagNU6H9es 04CANjmWLwXvXadQi9Y+dB7mvQdJs+IZtbvo6HMHjIEB6TA6BKSDwEJrbcWwWMhrDahbOQRVFrRv RkvXM9owQOiQjQZofJh1fnQV6Mt1aom3pEEWgHGYRwWSpYweW2RZLdfmu7+YJyql14ENOTGhjTGk xovGgw9/NYUQmnqVuBYE0J+wCqg+XtgyPb6WA/zBogiohX7QhYiuvwzoXefbgCyXSGlB5gV09g4x OYkpVIhnEKh7Kqpa1Tmhd3YS1/3td3ZEWyQ22vSymkm/sWYUF2Csk00k6i3IVp+kErVUHIYlQesi uxJnQzr2/jKTC0RYmi5QQmtkBrHgoROp8Gegog8QofX7yYgk3ddCl2jU0MEfyMwlJYYeDUwmuCs8 OhEnkrie48AOY+vKg6Vm4hGA+OUe113WzFfOgBjF7i0GCSpixzDWsX2+wfCSETNOQE5Jeu5G9LOJ fvGbpNWSEfISgmrSupFUbYGNZj8tmvH0rRoUgWZUJBuaDTyVM4xkEeLxeJC6ejx3GNsbbbHAwd70 Q9NSNEbYA2EYF6jYqhBsbG2DDFm220kkkqgbhdtWM3YgezWBvl0FkrZlOxQXw9lZXaPp7DHyDiHN Qzdho1jJbSCGMY3AOIcJe1hGNtnoGEaUGgbbSQ2omNgvCqlBDQQhMTBthQSYHZ1x4pZyEKhwGQ9z 1MJSIEbQ32LkHCA2kzoBtDaG2H4Mhh5sezs/4tgxjQaY9EV5IuAvaPgVvQLTIoNAnS/iyvMT2X7+ dt7jLOrVOi90joarXx7xHWEty/Uc9ZawXq8O4rajzkdABSx96L4C7vQUWYO64zJqUBBnhQcmrcze fsTJKvN81BendiYyP490y2DvHYGpWevMwWNEFowop2Fuo4s+gWFIJXrS6BwNwIFFjKjxUWpY7c5V eSswT/6YrBoWv3em1XesLMKgvxgswsIMiHN1Ve1e2FOxapJ5qnFfDsN2poXraSnb+2xC0Bh3LXYB mEaQZwGB1gOZhuF5BCUz7KPQve7Q8D01LQUjgJSJO9oXS0MkUwJnQETemiRIBZghZDygozLB053G /GtqPWhFCyRibQx1eYZbC1ZB35c7hlgse4Qsgm/i2lAZIW0SM6LCXrL5iKYKNLrBfgF7MRhdGmcW 8xhzu6XOistIcaT6goJFl2mQJZ6hiUmNSCmxuVGXGdJf2DALbmGLBvUn8ojpylLpA6BjH02oFIU1 11LaLpoQsBCg0JapCkwUmmZMm+RmpoUoJAGxL6qW136POtRBeAeKlCxR/JOa5Bc/rjvEJV4w1aF5 kdHjMnxMkbyYX6GzqEGcaWfPn+mxI5NBCwOUOfsWy5cgaVxSDrEXbhGVvxWpWIvQMCzqPrJS7hqA fEelSD3BuPGvUujsmA43ZcIAq3dgf4jlOSKpRX3BU+fC7XJdB5g94URGBGHFaQQHSZw2F5bZ1B8A +HvXtRrX8GsjPIJkgsVx3D/i7kinChIS+SzkoA==
signature.asc
Description: Digital signature
