Kazuaki Ishizaki created SPARK-13432:
----------------------------------------
Summary: Add the origin of the source code into a generated Java
code
Key: SPARK-13432
URL: https://issues.apache.org/jira/browse/SPARK-13432
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Kazuaki Ishizaki
Priority: Minor
Based on a discussion with [~sarutak], it would be good to add the function and
line of the original Spark program into a generated Java file by Catalyst. This
feature helps identify an reported issue in a generated code quickly.
Here is an example.
Original Spark program.
{code:java}
object Test {
...
df.filter("v <= 3")
.filter("v % 2 == 0")
.show()
...
}
{code}
Generated Java code
{code:java}
/* 031 */ protected void processNext() throws java.io.IOException {
/* 032 */ while (input.hasNext()) {
/* 033 */ InternalRow inputadapter_row = (InternalRow) input.next();
/* 034 */ /* input[0, string] @ filter at Test.scala:23 */
/* 035 */ boolean inputadapter_isNull = inputadapter_row.isNullAt(0);
/* 036 */ UTF8String inputadapter_value = inputadapter_isNull ? null :
(inputadapter_row.getUTF8String(0));
/* 037 */ /* input[1, int] @ filter at Test.scala:23 */
/* 038 */ boolean inputadapter_isNull1 = inputadapter_row.isNullAt(1);
/* 039 */ int inputadapter_value1 = inputadapter_isNull1 ? -1 :
(inputadapter_row.getInt(1));
/* 040 */ /* ((input[1, int] <= 3) && ((input[1, int] % 2) = 0)) @ filter
at Test.scala:23 */
/* 041 */ /* (input[1, int] <= 3) @ filter at Dataset1.scala:22 */
...
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
