This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new a8af23d [SPARK-27193][SQL] CodeFormatter should format multiple
comment lines correctly
a8af23d is described below
commit a8af23d7aba62d22b257c94d38a50cba05d0fcfd
Author: wuyi <[email protected]>
AuthorDate: Tue Mar 19 14:47:51 2019 +0800
[SPARK-27193][SQL] CodeFormatter should format multiple comment lines
correctly
## What changes were proposed in this pull request?
when enable `spark.sql.codegen.comments`, there will be multiple comment
lines. However, CodeFormatter can not handle multiple comment lines currently:
```
/* 001 */ public Object generate(Object[] references) {
/* 002 */ return new GeneratedIteratorForCodegenStage1(references);
/* 003 */ }
/* 004 */
/* 005 */ /**
* Codegend pipeline for stage (id=1)
* *(1) Project [(id#0L + 1) AS (id + 1)#3L]
* +- *(1) Filter (id#0L = 1)
* +- *(1) Range (0, 10, step=1, splits=4)
*/
/* 006 */ // codegenStageId=1
/* 007 */ final class GeneratedIteratorForCodegenStage1 extends
org.apache.spark.sql.execution.BufferedRowIterator {
```
After applying this pr:
```
/* 001 */ public Object generate(Object[] references) {
/* 002 */ return new GeneratedIteratorForCodegenStage1(references);
/* 003 */ }
/* 004 */
/* 005 */ /**
/* 006 */ * Codegend pipeline for stage (id=1)
/* 007 */ * *(1) Project [(id#0L + 1) AS (id + 1)#4L]
/* 008 */ * +- *(1) Filter (id#0L = 1)
/* 009 */ * +- *(1) Range (0, 10, step=1, splits=2)
/* 010 */ */
/* 011 */ // codegenStageId=1
/* 012 */ final class GeneratedIteratorForCodegenStage1 extends
org.apache.spark.sql.execution.BufferedRowIterator {
```
## How was this patch tested?
Tested Manually.
Closes #24133 from Ngone51/fix-codeformatter-for-multi-comment-lines.
Authored-by: wuyi <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
---
.../apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala
index ea1bb87..2ec3145 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala
@@ -41,7 +41,8 @@ object CodeFormatter {
val commentReplaced = commentHolder.replaceAllIn(
line.trim,
m =>
code.comment.get(m.group(1)).map(Matcher.quoteReplacement).getOrElse(m.group(0)))
- formatter.addLine(commentReplaced)
+ val comments = commentReplaced.split("\n")
+ comments.foreach(formatter.addLine)
}
if (needToTruncate) {
formatter.addLine(s"[truncated to $maxLines lines (total lines is
${lines.length})]")
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
