Author: daijy
Date: Thu Oct 27 20:18:02 2016
New Revision: 1766879
URL: http://svn.apache.org/viewvc?rev=1766879&view=rev
Log:
PIG-4798: big integer literals fail to parse
Modified:
pig/trunk/CHANGES.txt
pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml
pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java
pig/trunk/src/org/apache/pig/parser/QueryParser.g
pig/trunk/test/org/apache/pig/parser/TestQueryParser.java
Modified: pig/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?rev=1766879&r1=1766878&r2=1766879&view=diff
==============================================================================
--- pig/trunk/CHANGES.txt (original)
+++ pig/trunk/CHANGES.txt Thu Oct 27 20:18:02 2016
@@ -50,6 +50,8 @@ OPTIMIZATIONS
Â
BUG FIXES
+PIG-4798: big integer literals fail to parse (szita via daijy)
+
PIG-5045: CSVExcelStorage Load: A Quoted Field with a Single Escaped Quote
"""" Becomes "" This should become " instead
(szita via daijy)
Modified: pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml
URL:
http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml?rev=1766879&r1=1766878&r2=1766879&view=diff
==============================================================================
--- pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml (original)
+++ pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml Thu Oct 27
20:18:02 2016
@@ -1335,6 +1335,26 @@ dump X;
</tr>
<tr>
<td>
+ <p>biginteger</p>
+ </td>
+ <td>
+ <p>19211921192119211921BI</p>
+ </td>
+ <td>
+ </td>
+ </tr>
+ <tr>
+ <td>
+ <p>bigdecimal</p>
+ </td>
+ <td>
+ <p>192119211921.192119211921BD</p>
+ </td>
+ <td>
+ </td>
+ </tr>
+ <tr>
+ <td>
<p><strong>Complex Data Types</strong></p>
</td>
<td>
@@ -1388,7 +1408,18 @@ dump X;
<p>To specify a long constant, l or L must be appended to the number
(for example, 12345678L). If the l or L is not specified, but the number is too
large to fit into an int, the problem will be detected at parse time and the
processing is terminated. </p>
</li>
<li>
- <p>Any numeric constant with decimal point (for example, 1.5) and/or
exponent (for example, 5e+1) is treated as double unless it ends with f or F in
which case it is assigned type float (for example, Â 1.5f). </p>
+ <p>Any numeric constant with decimal point (for example, 1.5) and/or
exponent (for example, 5e+1) is treated as double unless it ends with the
following characters:</p>
+ <ul>
+ <li>
+ <p>f or F in which case it is assigned type float (for example,
 1.5f)</p>
+ </li>
+ <li>
+ <p>BD or bd in which case it is assigned type BigDecimal (for
example, Â 12345678.12345678BD)</p>
+ </li>
+ </ul>
+ </li>
+ <li>
+ <p>BigIntegers can be specified by supplying BI or bi at the end of
the number (for example, 123456789123456BI)</p>
</li>
<li>
<p>There is no native constant type for datetime field. You can use a
ToDate udf with chararray constant as argument to generate a datetime value.
</p>
Modified: pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java
URL:
http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java?rev=1766879&r1=1766878&r2=1766879&view=diff
==============================================================================
--- pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java (original)
+++ pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java Thu Oct 27
20:18:02 2016
@@ -1357,13 +1357,19 @@ public class LogicalPlanBuilder {
return Long.parseLong( num );
}
+ /**
+ * Parse big integer formatted string (e.g. "1234567890123BI") into
BigInteger object
+ */
static BigInteger parseBigInteger(String s) {
- String num = s.substring( 0, s.length() - 1 );
+ String num = s.substring( 0, s.length() - 2 );
return new BigInteger( num );
}
+ /**
+ * Parse big decimal formatted string (e.g. "123456.7890123BD") into
BigDecimal object
+ */
static BigDecimal parseBigDecimal(String s) {
- String num = s.substring( 0, s.length() - 1 );
+ String num = s.substring( 0, s.length() - 2 );
return new BigDecimal( num );
}
Modified: pig/trunk/src/org/apache/pig/parser/QueryParser.g
URL:
http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/parser/QueryParser.g?rev=1766879&r1=1766878&r2=1766879&view=diff
==============================================================================
--- pig/trunk/src/org/apache/pig/parser/QueryParser.g (original)
+++ pig/trunk/src/org/apache/pig/parser/QueryParser.g Thu Oct 27 20:18:02 2016
@@ -889,6 +889,8 @@ scalar : INTEGER
| LONGINTEGER
| FLOATNUMBER
| DOUBLENUMBER
+ | BIGINTEGERNUMBER
+ | BIGDECIMALNUMBER
| QUOTEDSTRING
| NULL
| TRUE
Modified: pig/trunk/test/org/apache/pig/parser/TestQueryParser.java
URL:
http://svn.apache.org/viewvc/pig/trunk/test/org/apache/pig/parser/TestQueryParser.java?rev=1766879&r1=1766878&r2=1766879&view=diff
==============================================================================
--- pig/trunk/test/org/apache/pig/parser/TestQueryParser.java (original)
+++ pig/trunk/test/org/apache/pig/parser/TestQueryParser.java Thu Oct 27
20:18:02 2016
@@ -652,4 +652,14 @@ public class TestQueryParser {
public void testSplit2() throws Exception {
shouldPass("SPLIT logs INTO logins IF command == 'login', all_quits IF
command == 'quit';");
}
+
+ @Test
+ public void testBigDecimalParsing() throws Exception {
+ shouldPass("B = FILTER A BY $1 < 1234567890.123456789BD;");
+ }
+
+ @Test
+ public void testBigIntegerParsing() throws Exception {
+ shouldPass("B = FILTER A BY $1 < 1234567890123456789BI;");
+ }
}
