Github user kiszk commented on the pull request:
https://github.com/apache/spark/pull/11301#issuecomment-188873899
I also add one feature. This is to show a message points out the origin of
a generated method when an exception occurs in the generated method at runtime.
An example of a message (the first line is newly added)
07:49:29.525 ERROR
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator: The
method GeneratedIterator.processNext() is generated for filter at Test.scala:23
07:49:29.526 ERROR org.apache.spark.executor.Executor: Exception in
task 1.0 in stage 2.0 (TID 4)
java.lang.NullPointerException:
at ...
Here is a part of generated code.
...
/* 031 */ protected void processNext() throws java.io.IOException {
/* 032 */ try {
/* 033 */ while (input.hasNext()) {
/* 034 */ InternalRow inputadapter_row = (InternalRow)
input.next();
/* 035 */ /* input[0, string] @ filter at Test.scala:23 */
/* 036 */ boolean inputadapter_isNull =
inputadapter_row.isNullAt(0);
/* 037 */ UTF8String inputadapter_value = inputadapter_isNull ?
null : (inputadapter_row.getUTF8String(0));
/* 038 */ /* input[1, int] @ filter at Test.scala:23 */
/* 039 */ boolean inputadapter_isNull1 =
inputadapter_row.isNullAt(1);
/* 040 */ int inputadapter_value1 = inputadapter_isNull1 ? -1 :
(inputadapter_row.getInt(1));
/* 041 */ /* ((input[1, int] <= 3) && ((input[1, int] % 2) =
0)) @ filter at Test.scala:23 */
/* 042 */ /* (input[1, int] <= 3) @ filter at Dataset1.scala:22
*/
...
/* 068 */ if (shouldStop()) {
/* 069 */ return;
/* 070 */ }
/* 071 */ }
/* 072 */ } catch (final Throwable e) {
/* 073 */ org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(this.getClass());
/* 074 */ logger.error("The method processNext() is generated for
" +
/* 075 */ "filter at Test.scala:22");
/* 076 */ throw e;
/* 077 */ }
...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
