Reviewers: dcarney, ulan,
Message:
Committed patchset #1 manually as r18098 (presubmit successful).
Description:
Experimental scanner: fix two-byte handling in lexer-shell.
In many cases (e.g., when the utf8 file contained an รค), it's a legit utf8
file (not ascii), but it'll still be 1-byte representation (latin1) inside
v8.
[email protected], [email protected]
BUG=
Committed: https://code.google.com/p/v8/source/detail?r=18098
Please review this at https://codereview.chromium.org/91233002/
SVN Base: https://v8.googlecode.com/svn/branches/experimental/parser
Affected files (+9, -4 lines):
M src/lexer/lexer-shell.cc
Index: src/lexer/lexer-shell.cc
diff --git a/src/lexer/lexer-shell.cc b/src/lexer/lexer-shell.cc
index
266727ff3ffaa9bec51d889c907e15d0e38b92b6..28722f5c8bc093d493ece96482e2e9c81715bb34
100644
--- a/src/lexer/lexer-shell.cc
+++ b/src/lexer/lexer-shell.cc
@@ -295,10 +295,15 @@ std::pair<TimeDelta, TimeDelta> ProcessFile(
Vector<const uc16>(buffer_16, buffer_end_16 - buffer_16));
// If the string was just an expaneded one byte string, V8 detects
it
// and doesn't store it as two byte.
- CHECK(source->IsTwoByteRepresentation());
- experimental_time = RunExperimentalScanner<uint16_t>(
- source, isolate, encoding, print_tokens || check_tokens,
- &experimental_tokens, repeat, harmony_settings);
+ if (!source->IsTwoByteRepresentation()) {
+ experimental_time = RunExperimentalScanner<uint8_t>(
+ source, isolate, encoding, print_tokens || check_tokens,
+ &experimental_tokens, repeat, harmony_settings);
+ } else {
+ experimental_time = RunExperimentalScanner<uint16_t>(
+ source, isolate, encoding, print_tokens || check_tokens,
+ &experimental_tokens, repeat, harmony_settings);
+ }
break;
}
default:
--
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
