GitHub user kiszk opened a pull request:
https://github.com/apache/spark/pull/11301
[SPARK-13432][SQL] add the source file name and line into a generated Java
code
## What changes were proposed in this pull request?
This PR adds the source file name and line into a comment of a Java code
generated by Catalyst. It would be helpful to quickly identify the original
source file from a position where an error occurs during a problem
determination of a customer. It supports only DataFrame and SQL.
This PR adds the information to a comment for existing operations. Other
PRs will address the followings:
- Add a comment for Dataset
- Insert a comment for other places
Here is an example. The original Java program.
object Test {
...
df.filter("v <= 3")
.filter("v % 2 == 0")
.show()
...
}
Generated Java code
...
/* 031 */ protected void processNext() throws java.io.IOException {
/* 032 */ while (input.hasNext()) {
/* 033 */ InternalRow inputadapter_row = (InternalRow)
input.next();
/* 034 */ /* input[0, string] @ filter at Test.scala:23 */
/* 035 */ boolean inputadapter_isNull =
inputadapter_row.isNullAt(0);
/* 036 */ UTF8String inputadapter_value = inputadapter_isNull ?
null : (inputadapter_row.getUTF8String(0));
/* 037 */ /* input[1, int] @ filter at Test.scala:23 */
/* 038 */ boolean inputadapter_isNull1 =
inputadapter_row.isNullAt(1);
/* 039 */ int inputadapter_value1 = inputadapter_isNull1 ? -1 :
(inputadapter_row.getInt(1));
/* 040 */ /* ((input[1, int] <= 3) && ((input[1, int] % 2) = 0))
@ filter at Test.scala:23 */
/* 041 */ /* (input[1, int] <= 3) @ filter at Dataset1.scala:22 */
...
## How was the this patch tested?
Unit test (add a test to keep Origin during SerDe)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kiszk/spark SPARK-13432
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11301.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11301
----
commit ad7cb33ba6678960ddaf8d7886724009190d48aa
Author: Kazuaki Ishizaki <[email protected]>
Date: 2016-02-22T09:54:04Z
add the source file name and line into a generated Java code
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
