subject:"Re\: \[PR\] ZOOKEEPER\-4880\: Generate comments from zookeeper.jute into code. \[zookeeper\]"

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-12-06 Thread via GitHub



kezhuw commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2524956933

   @luozongle01 Thank you for your contribution! Merged!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-12-06 Thread via GitHub



kezhuw merged PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-12-02 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2513496556

   > LGTM
   > 
   > Could you squash this pr to one commit ?
   
   Thanks, I have already squash it to one commit.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-12-01 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2510630046

   > ```
   > int field; /* A multi-line comment. */ /* Another multi-line comment. */
   > ```
   > 
   > I find that we are not handing above case. I left comment about this.
   
   Your testing is incredibly thorough! I hadn’t considered these cases before. 
I’m sorry that my code had so many issues and caused you so much extra work. 😂😂😂
   I added a few more test cases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-12-01 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1865098118


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -208,8 +217,18 @@ public void genCCode(FileWriter h, FileWriter c) throws 
IOException {
 }
 }
 String recName = getName();
+
+String recordComments = getRecordComments();
+if (recordComments != null && !recordComments.isEmpty()) {
+h.write(recordComments);
+}
 h.write("struct " + recName + " {\n");
 for (JField f : mFields) {
+

Review Comment:
   ```suggestion
   ```



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +794,122 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+Token beforeTheLineCommentToken = 
getCommentToken(jField.getTypeToken(), jField.getPreviousToken());
+List comments = extractComments(beforeTheLineCommentToken, 
null);

Review Comment:
   ```suggestion
   List comments = extractComments(beforeTheLineCommentToken, 
Integer.MAX_VALUE);
   ```



##
zookeeper-jute/src/test/java/org/apache/jute/compiler/JRecordTest.java:
##
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jute.compiler;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import java.io.StringReader;
+import java.lang.reflect.Field;
+import java.util.List;
+import org.apache.jute.compiler.generated.ParseException;
+import org.apache.jute.compiler.generated.Rcc;
+import org.junit.jupiter.api.Test;
+
+@SuppressWarnings({"unchecked", "SameParameterValue"})
+public class JRecordTest {
+
+@Test
+public void testEndOfLineComments() throws ParseException, 
NoSuchFieldException, IllegalAccessException {
+String juteStr = "module org.apache.zookeeper.data {\n"
++ "// information explicitly stored by the server 
persistently\n"
++ "class StatPersisted {\n"
++ "long czxid;  // created zxid\n"
++ "long mzxid;  // last modified zxid\n"
++ "long ctime;  // created\n"
++ "long mtime;  // last modified\n"
++ "}\n"
++ "}";
+
+try (StringReader stringReader = new StringReader(juteStr)) {
+Rcc parser = new Rcc(stringReader);
+JFile jFile = parser.Input();
+List mRecords = getField(jFile, "mRecords", List.class);
+assertEquals(1, mRecords.size());
+
+JRecord jRecord = mRecords.get(0);
+assertEquals("StatPersisted", jRecord.getName());
+List fields = jRecord.getFields();
+assertFiled(fields);
+
+assertEquals("// information explicitly stored by the server 
persistently\n", jRecord.getRecordComments());
+assertEquals("  // created zxid\n", 
jRecord.getJavaFieldComments(fields.get(0)));
+assertEquals("  // last modified zxid\n", 
jRecord.getJavaFieldComments(fields.get(1)));
+assertEquals("  // created\n", 
jRecord.getJavaFieldComments(fields.get(2)));
+assertEquals("  // last modified\n", 
jRecord.getJavaFieldComments(fields.get(3)));
+}
+}
+
+@Test
+public void testCommentBeforeLineAndEndOfLine() throws ParseException, 
NoSuchFieldException, IllegalAccessException {
+String juteStr = "module org.apache.zookeeper.data {\n"
++ "// information explicitly stored by the server 
persistently\n"
++ "class Stat

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-29 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1863136846


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,135 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+Token beforeTheLineCommentToken = 
getCommentToken(jField.getPreviousToken(), jField.getTypeToken());
+List comments = 
extractLeadingComments(beforeTheLineCommentToken, null);
+
+Token endOfLineCommentToken = getCommentToken(null, 
jField.getNextToken());
+if (endOfLineCommentToken != null && jField.getTypeToken().beginLine 
== endOfLineCommentToken.beginLine) {
+
+comments.addAll(extractLeadingComments(endOfLineCommentToken, 
endOfLineCommentToken.next));
+}
+
+return formatComments(indent, comments);
+}
+
+private Token getCommentToken(Token previousToken, Token token) {
+if (token == null || token.specialToken == null || 
belongsToThePreviousToken(previousToken, token.specialToken)) {
+return null;
+}
+
+Token tmpToken = token.specialToken;
+while (tmpToken.specialToken != null) {
+
+if (belongsToThePreviousToken(previousToken, 
tmpToken.specialToken)) {
+return tmpToken;
+}
+tmpToken = tmpToken.specialToken;
+}
+return tmpToken;
+}
+
+/**
+ * Determine whether the current commentToken belongs to the previousToken.
+ *
+ * @return true: If the current commentToken should belong to the 
previousToken.
+ */
+private boolean belongsToThePreviousToken(Token previousToken, Token 
commentToken) {
+if (previousToken == null || commentToken == null) {
+return false;
+}
+
+return previousToken.beginLine == commentToken.beginLine;
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+Token beforeTheClassToken = getCommentToken(null, getRecordToken());

Review Comment:
   > I think `commentToken` is good enough. Also I suggest renaming 
`extractLeadingComments` to simple `extractComments`.
   
   Oh, that's true



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-29 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1863085617


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,135 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+Token beforeTheLineCommentToken = 
getCommentToken(jField.getPreviousToken(), jField.getTypeToken());
+List comments = 
extractLeadingComments(beforeTheLineCommentToken, null);
+
+Token endOfLineCommentToken = getCommentToken(null, 
jField.getNextToken());
+if (endOfLineCommentToken != null && jField.getTypeToken().beginLine 
== endOfLineCommentToken.beginLine) {
+
+comments.addAll(extractLeadingComments(endOfLineCommentToken, 
endOfLineCommentToken.next));
+}
+
+return formatComments(indent, comments);
+}
+
+private Token getCommentToken(Token previousToken, Token token) {
+if (token == null || token.specialToken == null || 
belongsToThePreviousToken(previousToken, token.specialToken)) {
+return null;
+}
+
+Token tmpToken = token.specialToken;
+while (tmpToken.specialToken != null) {
+
+if (belongsToThePreviousToken(previousToken, 
tmpToken.specialToken)) {

Review Comment:
   ```java
   Token commentToken = token.specialToken;
   while (commentToken.specialToken != null) {
   commentToken = commentToken.specialToken;
   }
   // Skip end of line comment belong to previous token.
   if (previousToken != null && commentToken.beginLine == 
previousToken.endLine) {
   commentToken = commentToken.next;
   }
   ```
   
   Is this sufficient ?



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,135 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+Token beforeTheLineCommentToken = 
getCommentToken(jField.getPreviousToken(), jField.getTypeToken());
+List comments = 
extractLeadingComments(beforeTheLineCommentToken, null);
+
+Token endOfLineCommentToken = getCommentToken(null, 
jField.getNextToken());
+if (endOfLineCommentToken != null && jField.getTypeToken().beginLine 
== endOfLineCommentToken.beginLine) {
+
+comments.addAll(extractLeadingComments(endOfLineCommentToken, 
endOfLineCommentToken.next));
+}
+
+return formatComments(indent, comments);
+}
+
+private Token getCommentToken(Token previousToken, Token token) {
+if (token == null || token.specialToken == null || 
belongsToThePreviousToken(previousToken, token.specialToken)) {
+return null;
+}
+
+Token tmpToken = token.specialToken;
+while (tmpToken.specialToken != null) {
+
+if (belongsToThePreviousToken(previousToken, 
tmpToken.specialToken)) {
+return tmpToken;
+}
+tmpToken = tmpToken.specialToken;
+}
+return tmpToken;
+}
+
+/**
+ * Determine whether the current commentToken belongs to the previousToken.
+ *
+ * @return true: If the current commentToken should belong to the 
previousToken.
+ */
+private boolean belongsToThePreviousToken(Token previousToken, Token 
commentToken) {
+if (previousToken == null || commentToken == null) {
+return false;
+}
+
+return previousToken.beginLine == commentToken.beginLine;
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+Token beforeTheClassToken = getCommentToken(null, getRecordToken());

Review Comment:
   ```suggestion
   Token commentToken = getCommentToken(getRecordToken(), null);
   ```
   
   I think `commentToken` is good enough. Also I suggest renaming 
`extractLeadingComments` to si

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-28 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2506432669

   Hi @kezhuw , I have finished the revision. Thank you very much for your 
review. Since I may not have much work experience, there may be many small 
problems in my code. Thank you for pointing out where I can improve.😆


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-28 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1862113457


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,127 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(getCommentToken(jField.getTypeToken().specialToken));
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token commentToken = 
getCommentToken(jField.getNextToken().specialToken);
+if (commentToken != null && jField.getTypeToken().beginLine == 
commentToken.beginLine) {
+
+if (jField.getNextToken().specialToken == commentToken) {
+jField.getNextToken().specialToken = null;
+}
+
+if (commentToken.next != null) {
+commentToken.next.specialToken = null;
+commentToken.next = null;

Review Comment:
   > I am thinking whether we should have a `previousToken`. This way 
`getCommentToken` can skip end-line comment for previous entry without 
modifying things in code generation. I expect code generation is reproducible.
   
   That makes sense, I understand, I'll change it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-28 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1862093585


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -274,22 +257,38 @@ JRecord Record() :
 ArrayList flist = new ArrayList();
 Token t;
 JField f;
+// Get the comments on the class token
+Token recordTkn;
+Token rbraceTkn;
+Token typeTkn;
 }
 {
-
+recordTkn = 
 t = 
 { rname = t.image; }
 
 (
+{typeTkn = getToken(1);}
 f = Field()
-{ flist.add(f); }
+{
+flist.add(f);
+}
 
+{
+f.setTypeToken(typeTkn);
+prevFieldSetNextTkn(typeTkn);
+prevField = f;

Review Comment:
   > How about replace `nextToken` with `semicolonToken` ? I think it is 
suitable:
   > 
   > 1. Semicolon is the end of field declaration.
   > 2. `semicolonToken.next` is the next token. We can extract possible end of 
line comments from there.
   > 
   > This way we don't need `prevField` anymore.
   
   Ah, I see. Sorry I didn't think of writing this before.🤣🤣



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-28 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1862056149


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,127 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(getCommentToken(jField.getTypeToken().specialToken));
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token commentToken = 
getCommentToken(jField.getNextToken().specialToken);
+if (commentToken != null && jField.getTypeToken().beginLine == 
commentToken.beginLine) {
+
+if (jField.getNextToken().specialToken == commentToken) {
+jField.getNextToken().specialToken = null;
+}
+
+if (commentToken.next != null) {
+commentToken.next.specialToken = null;
+commentToken.next = null;
+}
+comments.addAll(extractLeadingComments(commentToken));
+}
+
+return formatComments(indent, comments);
+}
+
+private Token getCommentToken(Token token) {

Review Comment:
   > How about accept normal token as input ? I saw pattern 
`getCommentToken(xzyToken.specialToken)`.
   
   Sure, I'll make some changes here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-28 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1861950607


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -274,22 +257,38 @@ JRecord Record() :
 ArrayList flist = new ArrayList();
 Token t;
 JField f;
+// Get the comments on the class token
+Token recordTkn;
+Token rbraceTkn;
+Token typeTkn;
 }
 {
-
+recordTkn = 
 t = 
 { rname = t.image; }
 
 (
+{typeTkn = getToken(1);}
 f = Field()
-{ flist.add(f); }
+{
+flist.add(f);
+}
 
+{
+f.setTypeToken(typeTkn);
+prevFieldSetNextTkn(typeTkn);
+prevField = f;

Review Comment:
   `prevField` is not used outside `Record`, it should be a local also.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -274,22 +257,38 @@ JRecord Record() :
 ArrayList flist = new ArrayList();
 Token t;
 JField f;
+// Get the comments on the class token
+Token recordTkn;
+Token rbraceTkn;
+Token typeTkn;
 }
 {
-
+recordTkn = 
 t = 
 { rname = t.image; }
 
 (
+{typeTkn = getToken(1);}
 f = Field()
-{ flist.add(f); }
+{
+flist.add(f);
+}
 
+{
+f.setTypeToken(typeTkn);
+prevFieldSetNextTkn(typeTkn);
+prevField = f;

Review Comment:
   How about replace `nextToken` with `semicolonToken` ? I think it is suitable:
   1. Semicolon is the end of field declaration.
   2. `semicolonToken.next` is the next token. We can extract possible end of 
line comments from there.
   
   This way we don't need `prevField` anymore.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,127 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(getCommentToken(jField.getTypeToken().specialToken));
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token commentToken = 
getCommentToken(jField.getNextToken().specialToken);
+if (commentToken != null && jField.getTypeToken().beginLine == 
commentToken.beginLine) {
+
+if (jField.getNextToken().specialToken == commentToken) {
+jField.getNextToken().specialToken = null;
+}
+
+if (commentToken.next != null) {
+commentToken.next.specialToken = null;
+commentToken.next = null;
+}
+comments.addAll(extractLeadingComments(commentToken));
+}
+
+return formatComments(indent, comments);
+}
+
+private Token getCommentToken(Token token) {

Review Comment:
   How about accept normal token as input ? I saw pattern 
`getCommentToken(xzyToken.specialToken)`.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,127 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String indent) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(getCommentToken(jField.getTypeToken().specialToken));
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token commentToken = 
getCommentToken(jField.getNextToken().specialToken);
+if (commentToken != null && jField.getTypeToken().beginLine == 
commentToken.beginLine) {
+
+if (jField.getNextToken().specialToken == commentToken) {
+jField.getNextToken().specialToken = null;
+}
+
+if (commentToken.next != null)

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-27 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2505151135

   Hi @kezhuw , I have completed the modifications and testing, can you help me 
take a look again


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-27 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1860645336


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,122 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(extractLeadingComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", 
extractLeadingComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String indent, List 
commentLines) {
+if (commentLines == null || commentLines.isEmpty()) {
+return "";
+}
+
+StringBuilder builder = new StringBuilder();
+for (String line : commentLines) {
+if (!line.isEmpty()) {
+builder.append(indent).append(line);
+}
+builder.append(System.lineSeparator());
+}
+
+return builder.toString();
+}
+
+/**
+ * Extracts leading comments with indentation and line separator trimmed.
+ *
+ * Empty line is represented as empty string.
+ */
+private static List extractLeadingComments(Token token) {
+List comments = new ArrayList<>();
+
+if (token == null) {
+return comments;
+}
+
+Token tmpToken = token;
+while (tmpToken.specialToken != null) {
+tmpToken = tmpToken.specialToken;
+}

Review Comment:
   > Add a method `getCommentToken` ? There is a similar loop in 
`getFieldComments`.
   
   Okay, I'll make some changes here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-27 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1860644349


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {

Review Comment:
   > Sorry for the confusion! I means it should be better to rename variable 
`space` to `indent`.
   
   Okay, I will make some modifications here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-27 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1860637296


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -274,22 +261,37 @@ JRecord Record() :
 ArrayList flist = new ArrayList();
 Token t;
 JField f;
+// Get the comments on the class token
+Token recordTkn;
+Token rbraceTkn;
 }
 {
-
+recordTkn = 
 t = 
 { rname = t.image; }
 
 (
 f = Field()

Review Comment:
   > I think 
[`getToken(1)`](https://www.cs.purdue.edu/homes/hosking/javacc/doc/apiroutines.html#getToken)
 could help you eliminating most intrusive code.
   > 
   > ```
   > { typeToken = getToken(1) }
   > ```
   > 
   > Then most changes should resides in `Record()`.
   
   Oh, I've been searching for a long time before. I was looking for an API 
that could traverse tokens, but I couldn't find it. I didn't expect to find 
such an API. I'll give it a try. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-27 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1860576159


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +795,122 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
extractLeadingComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(extractLeadingComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", 
extractLeadingComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String indent, List 
commentLines) {
+if (commentLines == null || commentLines.isEmpty()) {
+return "";
+}
+
+StringBuilder builder = new StringBuilder();
+for (String line : commentLines) {
+if (!line.isEmpty()) {
+builder.append(indent).append(line);
+}
+builder.append(System.lineSeparator());
+}
+
+return builder.toString();
+}
+
+/**
+ * Extracts leading comments with indentation and line separator trimmed.
+ *
+ * Empty line is represented as empty string.
+ */
+private static List extractLeadingComments(Token token) {
+List comments = new ArrayList<>();
+
+if (token == null) {
+return comments;
+}
+
+Token tmpToken = token;
+while (tmpToken.specialToken != null) {
+tmpToken = tmpToken.specialToken;
+}

Review Comment:
   Add a method `getCommentToken` ? There is a similar loop in 
`getFieldComments`.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -274,22 +261,37 @@ JRecord Record() :
 ArrayList flist = new ArrayList();
 Token t;
 JField f;
+// Get the comments on the class token
+Token recordTkn;
+Token rbraceTkn;
 }
 {
-
+recordTkn = 
 t = 
 { rname = t.image; }
 
 (
 f = Field()

Review Comment:
   I think 
[`getToken(1)`](https://www.cs.purdue.edu/homes/hosking/javacc/doc/apiroutines.html#getToken)
 could help you eliminating most intrusive code.
   
   ```jj
   { typeToken = getToken(1) }
   ```
   
   Then most changes should resides in `Record()`.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {

Review Comment:
   Sorry for the confusion! I means it should be better to rename variable 
`space`  to `indent`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-26 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2503012842

   Hi @kezhuw , Thank you for providing the example. 
   I didn't expect this to reconstruct the style of the judge. 😂  This is much 
better than before. I have made modifications based on your example and tested 
it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857886486


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   > Currently, all leading spaces in mult-line comment are dropped in 
generation. Can we reconstruct these for multi-line comment ?
   > 
   > Or how about `` ? 
(https://stackoverflow.com/questions/34550579/multiline-comments-in-javacc)
   > 
   > I find no hole despite there is one comment say that "not correct with 
respect to the language definition".
   > 
   > I think it is easy to reconstruct original format if multi-line comment is 
captured into one token.
   
   
   This way, multiple lines of comments can be obtained into one token.
   ```
   MORE :
   {
  { input_stream.backup(1); } : 
IN_JAVADOC_COMMENT
   |
  : IN_MULTI_LINE_COMMENT
   }
   
   
   SPECIAL_TOKEN :
   {
  : DEFAULT
   }
   
   
   SPECIAL_TOKEN :
   {
  : DEFAULT
   }
   
   
   MORE :
   {
 
   }
   ```
   
   But there is still a small issue, it seems that there is no way to remove 
the space at the beginning of each middle line, and there is no space before 
the first line of multi line comments. 😂😂
   https://github.com/user-attachments/assets/6c47a123-e595-4608-9146-b8d196e876e6";>
   https://github.com/user-attachments/assets/9eee459f-f946-47b8-b119-701fe0b72b9f";>
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857771645


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -111,17 +119,7 @@ SKIP :
 
 SPECIAL_TOKEN :
 {
-  "//" : WithinOneLineComment
-}
-
- SPECIAL_TOKEN :
-{
-  <("\n" | "\r" | "\r\n" )> : DEFAULT
-}
-
- MORE :
-{
-  <~[]>
+  

Review Comment:
   > Should be "ONE_LINE_COMMENT" ? It is as a "Token.kind" constant in 
generated in "RccConstants".
   
   Yes, I'll make some modifications here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857769822


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   Oh, then I'll research and see if I can capture multi line comments as one 
token.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857758636


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {

Review Comment:
   > s/space/indent/ ?
   
   This is because `org.apache.jute.compiler.JType#genCDecl` also contains 
spaces, so I also used spaces. 😂



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857741746


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   > Or how about  ? 
(https://stackoverflow.com/questions/34550579/multiline-comments-in-javacc)
   
   > I think it is easy to reconstruct original format if multi-line comment is 
captured into one token.
   
   I am not sure the `LOOKAHEAD` could help here. I tasted and did not succeed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857758636


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {

Review Comment:
   > s/space/indent/ ?
   
   This is because ` org.apache.jute.compiler JType # genCDecl ` also contains 
spaces, so I also used spaces. 😂



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857740621


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   > all leading spaces in mult-line comment are dropped in generation
   
   ```
   /**
** Something.
**/
   ```
   
   For example, above comment will be formatted as following.

   ```
   /**
   ** Something.
   **/
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857741746


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   > Or how about  ? 
(https://stackoverflow.com/questions/34550579/multiline-comments-in-javacc)
   
   > I think it is easy to reconstruct original format if multi-line comment is 
captured into one token.
   
   I am not sure the `LOOKAHEAD` could help here. I tried but did not succeed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-25 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1857737011


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {
+if (jField == null || jField.getTypeToken() == null || 
jField.getNextToken() == null) {
+return "";
+}
+
+// get the comment before the line
+List comments = 
getComments(jField.getTypeToken().specialToken);
+
+// get the end-of-line comments of fields
+// If the current field and the next field are on the same line,
+// the leading field comment of the next field should be discarded
+Token tmpToken = jField.getNextToken();
+while (tmpToken.specialToken != null) {
+
+if (jField.getTypeToken().beginLine == 
tmpToken.specialToken.beginLine) {
+Token endLineComments = tmpToken.specialToken;
+tmpToken.specialToken = null;
+comments.addAll(getComments(endLineComments));
+break;
+}
+
+tmpToken = tmpToken.specialToken;
+}
+
+return formatComments(space, comments);
+}
+
+public String getRecordComments() {
+if (getRecordToken() == null || getRecordToken().specialToken == null) 
{
+return "";
+}
+
+// get the comments before the class
+return formatComments("", getComments(getRecordToken().specialToken));
+}
+
+private static String formatComments(String space, List 
commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return "";
+}
+
+String comments = String.join("", commentList);
+
+StringBuilder formatBuilder = new StringBuilder();
+for (String s : comments.split("\r?\n")) {
+formatBuilder.append("\n")
+.append(space)
+.append(s.replaceAll("^[ \t]+", ""));
+}
+
+return formatBuilder.append("\n").toString();
+}
+
+private static List getComments(Token token) {

Review Comment:
   Currently, all leading spaces in mult-line comment are dropped in 
generation. Can we reconstruct these for multi-line comment ?
   
   Or how about `` ? 
(https://stackoverflow.com/questions/34550579/multiline-comments-in-javacc)
   
   I find no hole despite there is one comment say that  "not correct with 
respect to the language definition".
   
   I think it is easy to reconstruct original format if multi-line comment is 
captured into one token.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/generated/rcc.jj:
##
@@ -111,17 +119,7 @@ SKIP :
 
 SPECIAL_TOKEN :
 {
-  "//" : WithinOneLineComment
-}
-
- SPECIAL_TOKEN :
-{
-  <("\n" | "\r" | "\r\n" )> : DEFAULT
-}
-
- MORE :
-{
-  <~[]>
+  

Review Comment:
   Should be "ONE_LINE_COMMENT" ? It is as a "Token.kind" constant in generated 
in "RccConstants".



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +793,88 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+public String getJavaFieldComments(JField jField) {
+return getFieldComments(jField, "  ");
+}
+
+public String getCFieldComments(JField jField) {
+return getFieldComments(jField, "");
+}
+
+private String getFieldComments(JField jField, String space) {

Review Comment:
   s/space/indent/ ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-21 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2491334165

   Hi @kezhuw,  Thank you for your review last time. The code feels much more 
concise now.
   Could you take another look at it for me?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-18 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1847769128


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JCommentGenerator.java:
##
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jute.compiler;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Optional;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import org.apache.jute.compiler.generated.Token;
+
+public class JCommentGenerator {
+
+/**
+ * {fieldCommentsBeginLineNumber : {fieldCommentsBeginColumn : 
fieldCommentsToken}}.
+ */
+private final TreeMap> 
fieldCommentsTokenMap = new TreeMap<>();
+
+/**
+ * {recordCommentsBeginLineNumber : {recordCommentsBeginColumn : 
recordCommentsToken}}.
+ */
+private final TreeMap> 
recordCommentsTokenMap = new TreeMap<>();
+
+private final TreeSet jFieldTreeSet = new TreeSet<>();
+
+private final TreeSet jRecordTreeSet = new TreeSet<>();
+
+public void addJRecord(JRecord jRecord) {
+jRecordTreeSet.add(jRecord);
+}
+
+public void addJField(JField jField) {
+jFieldTreeSet.add(jField);

Review Comment:
   OK, I will modify it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-18 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1847720246


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JCommentGenerator.java:
##
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jute.compiler;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Optional;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import org.apache.jute.compiler.generated.Token;
+
+public class JCommentGenerator {
+
+/**
+ * {fieldCommentsBeginLineNumber : {fieldCommentsBeginColumn : 
fieldCommentsToken}}.
+ */
+private final TreeMap> 
fieldCommentsTokenMap = new TreeMap<>();
+
+/**
+ * {recordCommentsBeginLineNumber : {recordCommentsBeginColumn : 
recordCommentsToken}}.
+ */
+private final TreeMap> 
recordCommentsTokenMap = new TreeMap<>();
+
+private final TreeSet jFieldTreeSet = new TreeSet<>();
+
+private final TreeSet jRecordTreeSet = new TreeSet<>();
+
+public void addJRecord(JRecord jRecord) {
+jRecordTreeSet.add(jRecord);
+}
+
+public void addJField(JField jField) {
+jFieldTreeSet.add(jField);
+}
+
+public void putFieldSpecialToken(Token fieldToken) {
+putSpecialToken(fieldToken, fieldCommentsTokenMap);
+}
+
+public void putRecordSpecialToken(Token recordToken) {
+putSpecialToken(recordToken, recordCommentsTokenMap);
+}
+
+private static void putSpecialToken(Token classToken, TreeMap> commentsTokenMap) {
+if (classToken == null || classToken.specialToken == null || 
commentsTokenMap == null) {
+return;
+}
+
+Token tmp = classToken;
+while ((tmp = tmp.specialToken) != null && tmp.image != null) {
+commentsTokenMap.computeIfAbsent(tmp.beginLine, key -> new 
TreeMap<>())
+.putIfAbsent(tmp.beginColumn, tmp);
+}
+}
+
+public List getRecordCommentsList(JRecord jRecord) {
+return getCommentsList(jRecord, jRecordTreeSet, 
recordCommentsTokenMap);
+}
+
+public List getFieldCommentsList(JField jField) {
+return getCommentsList(jField, jFieldTreeSet, fieldCommentsTokenMap);
+}
+
+private static List getCommentsList(RangeInfo rangeInfo,

Review Comment:
   I expect this should be much clear. Given that we support only comments in 
three places:
   1. Comments before `class` for record types.
   2. Comments before fields.
   3. Line ending comments for fields.
   
   For the first, we could extract comments from `Token::specialToken`. For the 
second, similar to the first, but we have to drop comments(could be in either 
`//` or `/* */` form) belong to previous field. For the third, additional 
lookup is required, also drop comments belong to next field.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JCommentGenerator.java:
##
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jute.compiler;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.NavigableMap;
+import java.util.Optional;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import org.apache.jute.comp

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-13 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2475410906

   Hi @kezhuw, These days, I have researched how JavaParser and Protobuf handle 
comments again.
   
   The 
[Comment#content](https://github.com/javaparser/javaparser/blob/master/javaparser-core/src/main/java/com/github/javaparser/ast/comments/Comment.java)
 method of Javaparser can retrieve the content inside the comment, which will 
remove the first layer`/**   */` or `/*   */` or `//` .  
   I think I might also be able to obtain the internal content of comments in 
this way for formatting purposes.
   
   The creatCommentFromToken method is the process of handling content
   
https://github.com/javaparser/javaparser/blob/master/javaparser-core/src/main/javacc-support/com/github/javaparser/GeneratedJavaParserTokenManagerBase.java
   
   
   Then I also tested the process of protobuf handling comments.
   
   protobuf will format `/**  */` or `/*  */` or `//` as `/**  */`. 
   https://github.com/user-attachments/assets/dfd81b38-a7e8-46a1-a75a-9c6d50339808";>
   
   Will convert multi line comments nested within single line comments into 
`Html Entity`. (Because multi line comments cannot be nested within each other)
   
   https://github.com/user-attachments/assets/eae37bd6-dbc4-4e1a-8083-c49be44e7b4e";>
   
   https://github.com/user-attachments/assets/a7b62be8-128b-4eac-8f9d-a4c9c61b697c";>
   
   So I think I just need to handle the following single line comments and a 
few situations
   ```text
   /*   /*/** /**
 * *
 * *
   */*/*/  */
   ```
   For internal comments, I only need to convert`/*` and `*/` to `HTML Entity` 
(because multi line comments cannot be nested)
   
   So can I make further modifications this way or are there any issues I 
haven't thought of?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-13 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2475347667

   Hi @kezhuw, These days, I have researched how JavaParser and Protobuf handle 
comments again。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



kezhuw commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2462189756

   > for the end-of-line comments, should I continue to keep them at the end of 
the line or is it better to move them above the code?
   
   I prefer to move them above the code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1832790508


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +790,53 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+private String getJavaFieldComments(JField jField) {
+StringJoiner javaFieldCommentsJoiner =
+new StringJoiner("\n   * ", "  /**\n   * ", "\n   
*/\n").setEmptyValue("");
+return getFieldComments(jField, javaFieldCommentsJoiner);
+}
+
+private String getCFieldComments(JField jField) {
+StringJoiner cFieldCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getFieldComments(jField, cFieldCommentsJoiner);
+}
+
+private String getFieldComments(JField jField, StringJoiner 
fieldCommentsJoiner) {
+List fieldSpecialToken = 
jCommentGenerator.getFieldCommentsList(jField);
+return getComments(fieldCommentsJoiner, fieldSpecialToken);
+}
+
+private String getRecordComments() {
+List classSpecialToken = 
jCommentGenerator.getRecordCommentsList(this);
+
+StringJoiner recordCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getComments(recordCommentsJoiner, classSpecialToken);
+}
+
+private static String getComments(StringJoiner commentsJoiner, 
List commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return commentsJoiner.toString();
+}
+
+for (String commentMulti : commentList) {
+if (commentMulti == null || commentMulti.isEmpty()) {
+continue;
+}
+
+String[] commentArray = commentMulti.split("\n");
+for (String comment : commentArray) {
+
+comment = comment.replaceAll("//|/\\*\\*|/\\*|\\*\\*/|\\*/", 
"")

Review Comment:
   > I do think people will want to maintain this kind of code.
   I'll amend that and I won't touch the comments so they're not needed here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2462473973

   Hi @kezhuw  ,  I made some changes, could you take a look at it for me?
   
   I won't touch the annotation content now.
   
   > But I still formatted it a little bit here, because for example, field 
needs a few spaces, class does not need spaces, and the number of spaces in 
java and c is also different.
   ```java
 private static String getComments(String space, List commentList) {
 if (commentList == null || commentList.isEmpty()) {
 return "";
 }
   
 String comments = String.join("", commentList);
   
 StringBuilder formatBuilder = new StringBuilder();
 for (String s : comments.split("\r?\n")) {
 formatBuilder.append("\n")
 .append(space)
 .append(s.replaceAll("^[ \t]+", ""));
 }
   
 return formatBuilder.append("\n").toString();
 }
   
   
   Now the comment is like this
   https://github.com/user-attachments/assets/d018d3cf-d30f-49b3-a4fd-90008f0e7f06";>
   https://github.com/user-attachments/assets/38ba86f4-cc0b-4156-8bc7-6540abe3b20e";>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-246210

   Hi @kezhuw , I made some changes, could you take a look at it for me?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1832739515


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +790,53 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+private String getJavaFieldComments(JField jField) {
+StringJoiner javaFieldCommentsJoiner =
+new StringJoiner("\n   * ", "  /**\n   * ", "\n   
*/\n").setEmptyValue("");
+return getFieldComments(jField, javaFieldCommentsJoiner);
+}
+
+private String getCFieldComments(JField jField) {
+StringJoiner cFieldCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getFieldComments(jField, cFieldCommentsJoiner);
+}
+
+private String getFieldComments(JField jField, StringJoiner 
fieldCommentsJoiner) {
+List fieldSpecialToken = 
jCommentGenerator.getFieldCommentsList(jField);
+return getComments(fieldCommentsJoiner, fieldSpecialToken);
+}
+
+private String getRecordComments() {
+List classSpecialToken = 
jCommentGenerator.getRecordCommentsList(this);
+
+StringJoiner recordCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getComments(recordCommentsJoiner, classSpecialToken);
+}
+
+private static String getComments(StringJoiner commentsJoiner, 
List commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return commentsJoiner.toString();
+}
+
+for (String commentMulti : commentList) {
+if (commentMulti == null || commentMulti.isEmpty()) {
+continue;
+}
+
+String[] commentArray = commentMulti.split("\n");
+for (String comment : commentArray) {
+
+comment = comment.replaceAll("//|/\\*\\*|/\\*|\\*\\*/|\\*/", 
"")

Review Comment:
   s/do/don't/



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2462204400

   > > for the end-of-line comments, should I continue to keep them at the end 
of the line or is it better to move them above the code?
   > 
   > I prefer to move them above the code.
   
   Thanks, I thought so too, I'll modify the code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-07 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2461577162

   > @luozongle01 Thank you for your work!
   > 
   > It might be late, I am sorry for this, but I think we probably should 
align on how comments from jute should be formatted in generated sources ? I 
see two approaches.
   > 
   > 1. Keep it in origin format.
   > 2. Transfer comments from one format to another.
   > 
   > I saw this pr is going on the second approach. I am good to unify the 
generated comment format. But I think it could be challenge, as we should 
escape between two formats which could make generated comments 
strange/inaccurate.
   > 
   > ```
   > // /**
   > //  * Some dead comments.
   > //  */
   > //
   > // Things changed. We are going ..
   > ```
   > 
   > Currently, the generated comments blindly strip some special strings. I 
don't think it is a good thing.
   > 
   > I suggest we keep it simple and don't touch comment content, this is also 
a safe path. If we want javadoc style comments, I think it is ok to convert 
them manually.
   
   
   
   > @luozongle01 Thank you for your work!
   > 
   > It might be late, I am sorry for this, but I think we probably should 
align on how comments from jute should be formatted in generated sources ? I 
see two approaches.
   > 
   > 1. Keep it in origin format.
   > 2. Transfer comments from one format to another.
   > 
   > I saw this pr is going on the second approach. I am good to unify the 
generated comment format. But I think it could be challenge, as we should 
escape between two formats which could make generated comments 
strange/inaccurate.
   > 
   > ```
   > // /**
   > //  * Some dead comments.
   > //  */
   > //
   > // Things changed. We are going ..
   > ```
   > 
   > Currently, the generated comments blindly strip some special strings. I 
don't think it is a good thing.
   > 
   > I suggest we keep it simple and don't touch comment content, this is also 
a safe path. If we want javadoc style comments, I think it is ok to convert 
them manually.
   
   Thank you very much for your review. I really didn't consider that 😂😂.   I 
will keep the original format output.
   
   However, for the end-of-line comments, should I continue to keep them at the 
end of the line or is it better to move them above the code?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-11-06 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1832126265


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +790,53 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+private String getJavaFieldComments(JField jField) {
+StringJoiner javaFieldCommentsJoiner =
+new StringJoiner("\n   * ", "  /**\n   * ", "\n   
*/\n").setEmptyValue("");
+return getFieldComments(jField, javaFieldCommentsJoiner);
+}
+
+private String getCFieldComments(JField jField) {
+StringJoiner cFieldCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getFieldComments(jField, cFieldCommentsJoiner);
+}
+
+private String getFieldComments(JField jField, StringJoiner 
fieldCommentsJoiner) {
+List fieldSpecialToken = 
jCommentGenerator.getFieldCommentsList(jField);
+return getComments(fieldCommentsJoiner, fieldSpecialToken);
+}
+
+private String getRecordComments() {
+List classSpecialToken = 
jCommentGenerator.getRecordCommentsList(this);
+
+StringJoiner recordCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getComments(recordCommentsJoiner, classSpecialToken);
+}
+
+private static String getComments(StringJoiner commentsJoiner, 
List commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return commentsJoiner.toString();
+}
+
+for (String commentMulti : commentList) {
+if (commentMulti == null || commentMulti.isEmpty()) {
+continue;
+}
+
+String[] commentArray = commentMulti.split("\n");
+for (String comment : commentArray) {
+
+comment = comment.replaceAll("//|/\\*\\*|/\\*|\\*\\*/|\\*/", 
"")
+.replaceAll("^[* ]+", "").trim();
+
+if (!comment.isEmpty()) {

Review Comment:
   The generated comments has no empty line which is not good.



##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +790,53 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+private String getJavaFieldComments(JField jField) {
+StringJoiner javaFieldCommentsJoiner =
+new StringJoiner("\n   * ", "  /**\n   * ", "\n   
*/\n").setEmptyValue("");
+return getFieldComments(jField, javaFieldCommentsJoiner);
+}
+
+private String getCFieldComments(JField jField) {
+StringJoiner cFieldCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getFieldComments(jField, cFieldCommentsJoiner);
+}
+
+private String getFieldComments(JField jField, StringJoiner 
fieldCommentsJoiner) {
+List fieldSpecialToken = 
jCommentGenerator.getFieldCommentsList(jField);
+return getComments(fieldCommentsJoiner, fieldSpecialToken);
+}
+
+private String getRecordComments() {
+List classSpecialToken = 
jCommentGenerator.getRecordCommentsList(this);
+
+StringJoiner recordCommentsJoiner =
+new StringJoiner("\n * ", "/**\n * ", "\n 
*/\n").setEmptyValue("");
+return getComments(recordCommentsJoiner, classSpecialToken);
+}
+
+private static String getComments(StringJoiner commentsJoiner, 
List commentList) {
+if (commentList == null || commentList.isEmpty()) {
+return commentsJoiner.toString();
+}
+
+for (String commentMulti : commentList) {
+if (commentMulti == null || commentMulti.isEmpty()) {
+continue;
+}
+
+String[] commentArray = commentMulti.split("\n");
+for (String comment : commentArray) {
+
+comment = comment.replaceAll("//|/\\*\\*|/\\*|\\*\\*/|\\*/", 
"")

Review Comment:
   I do think people will want to maintain this kind of code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-10-29 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1821739303


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +780,30 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+protected String getRecordComments() {
+if (mRecordToken == null || mRecordToken.specialToken == null) {
+return null;
+}
+
+StringJoiner joiner = new StringJoiner("\n * ", "/**\n * ", "\n */\n");
+Token tmpTkn = mRecordToken.specialToken;
+while (tmpTkn.specialToken != null) {
+tmpTkn = tmpTkn.specialToken;
+}
+
+while (tmpTkn != null) {
+String image = tmpTkn.image.trim();
+tmpTkn = tmpTkn.next;
+if (image.startsWith("//")) {

Review Comment:
   > The capture does not shaped well for directly usage. I tasted the 
following capture, and it captured a complete line comment. This way we can 
output whatever it captured without post work.
   > 
   > ```
   > SPECIAL_TOKEN :
   > {
   >   
   > }
   > ```
   > 
   > See also https://javacc.github.io/javacc/tutorials/token-manager.html
   
   It seems like there's no need to change this now, because I get all the 
comments，then I use and distinguished them by line numbe. 😂😂



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-10-29 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2445635303

   Hi, @kezhuw, I made some modifications, thank you for your review last time. 
I feel much better now than last time. 
   I hope you can help me look again.
   
   Currently, multi-line comments and single-line comments are supported.
   
   I put `JField` or `JRecord` into a `TreeSet`, so that I can get the `line` 
or `column` information of the previous or next element. According to the 
`beginLine`, `beginColumn`, `endLine`, `endColumn` information, I determine the 
range of my comment area, and then format the output in line number order.
   
   Here is my testing process.
   1. Verify that the modification only affects the comments and does not 
affect the normal code.
   
   2. When verifying that each JField is on its own line, multi-line comments 
and single-line comments are generated normally.
   https://github.com/user-attachments/assets/28f797ca-f474-46d0-b178-f3a7abb5faf4";>
   https://github.com/user-attachments/assets/4e9fce69-3c38-4698-ac90-24882eea6196";>
   
   3. If the attributes are on the same line, the comment should belong to the 
previous Field.
   https://github.com/user-attachments/assets/77120d53-9f8b-4323-9cad-485d58f97aef";>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-10-27 Thread via GitHub



luozongle01 commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1818365785


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +780,30 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+protected String getRecordComments() {
+if (mRecordToken == null || mRecordToken.specialToken == null) {
+return null;
+}
+
+StringJoiner joiner = new StringJoiner("\n * ", "/**\n * ", "\n */\n");
+Token tmpTkn = mRecordToken.specialToken;
+while (tmpTkn.specialToken != null) {
+tmpTkn = tmpTkn.specialToken;
+}
+
+while (tmpTkn != null) {
+String image = tmpTkn.image.trim();
+tmpTkn = tmpTkn.next;
+if (image.startsWith("//")) {

Review Comment:
   Thank you very much for your comment. I will make some further 
modifications.😆



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-10-27 Thread via GitHub



kezhuw commented on code in PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#discussion_r1818339448


##
zookeeper-jute/src/main/java/org/apache/jute/compiler/JRecord.java:
##
@@ -767,4 +780,30 @@ public static String getCsharpFQName(String name) {
 }
 return fQName.toString();
 }
+
+protected String getRecordComments() {
+if (mRecordToken == null || mRecordToken.specialToken == null) {
+return null;
+}
+
+StringJoiner joiner = new StringJoiner("\n * ", "/**\n * ", "\n */\n");
+Token tmpTkn = mRecordToken.specialToken;
+while (tmpTkn.specialToken != null) {
+tmpTkn = tmpTkn.specialToken;
+}
+
+while (tmpTkn != null) {
+String image = tmpTkn.image.trim();
+tmpTkn = tmpTkn.next;
+if (image.startsWith("//")) {

Review Comment:
   The capture does not shaped well for directly usage. I tasted the following 
capture, and it captured a complete line comment. This way we can output 
whatever it captured without post work.
   
   ```
   SPECIAL_TOKEN :
   {
 
   }
   ```
   
   See also https://javacc.github.io/javacc/tutorials/token-manager.html



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] ZOOKEEPER-4880: Generate comments from zookeeper.jute into code. [zookeeper]

2024-10-23 Thread via GitHub



luozongle01 commented on PR #2206:
URL: https://github.com/apache/zookeeper/pull/2206#issuecomment-2432773020

   @kezhuw  Could you help me take a look? Thanks.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@zookeeper.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

45 matches

Mail list logo