Hi Tzurs, Craig.
I fixed the bug in the preprocessor core and in my own preprocessor plugin.
changes have been committed to Antenna's CVS.
Below are two patches:
* The first is for the Preprocessor (which is directly relevant to
EclipseME Code, use it to fix the code or just build from CVS.
* The second is a fix to Antenna's preprocessor plugin, use it as an
example to fix your own plugin (you may have copied the bug when you
integrated).
### Eclipse Workspace Patch 1.0
#P antenna.preprocessor
Index: src/antenna/preprocessor/v2/Preprocessor.java
===================================================================
RCS file:
/cvsroot/antenna/antenna.preprocessor/src/antenna/preprocessor/v2/Preprocessor.java,v
retrieving revision 1.3
diff -u -r1.3 Preprocessor.java
--- src/antenna/preprocessor/v2/Preprocessor.java 3 Aug 2007 07:02:42
-0000 1.3
+++ src/antenna/preprocessor/v2/Preprocessor.java 24 Oct 2007
05:38:43 -0000
@@ -44,7 +44,7 @@
private int m_currentState;
- private boolean m_verbose = true;
+ private boolean m_verbose = false;
private Defines m_defines;
@@ -127,13 +127,13 @@
Vector lines = new Vector();
loadStrings(lines, in, encoding);
boolean changed = preprocess(lines, encoding);
- saveStrings(lines, out);
+ saveStrings(lines, out, encoding);
return changed;
}
- public static void saveStrings(Vector lines, OutputStream out)
throws IOException
+ public static void saveStrings(Vector lines, OutputStream out,
String encoding) throws IOException
{
- BufferedWriter writer = new BufferedWriter(new
OutputStreamWriter(out));
+ BufferedWriter writer = new BufferedWriter(new
OutputStreamWriter(out, encoding));
try
{
int size = lines.size();
@@ -151,7 +151,7 @@
public static void loadStrings(Vector lines, InputStream in, String
encoding) throws IOException
{
- BufferedReader reader = new BufferedReader(new
InputStreamReader(in));
+ BufferedReader reader = new BufferedReader(new
InputStreamReader(in, encoding));
String s;
while ((s = reader.readLine()) != null)
{
### Eclipse Workspace Patch 1.0
#P antenna.eclipse.preprocessor
Index: src/antenna/eclipse/preprocessor/Activator.java
===================================================================
RCS file:
/cvsroot/antenna/antenna.eclipse.preprocessor/src/antenna/eclipse/preprocessor/Activator.java,v
retrieving revision 1.6
diff -u -r1.6 Activator.java
--- src/antenna/eclipse/preprocessor/Activator.java 1 Oct 2007
15:18:37 -0000 1.6
+++ src/antenna/eclipse/preprocessor/Activator.java 24 Oct 2007
05:40:45 -0000
@@ -89,7 +89,7 @@
try
{
String str = document.get();
- ByteArrayInputStream bin = new
ByteArrayInputStream(str.getBytes());
+ ByteArrayInputStream bin = new
ByteArrayInputStream(str.getBytes(fileBuffer.getEncoding()));
Vector lines = new Vector();
Preprocessor.loadStrings(lines, bin,
fileBuffer.getEncoding());
@@ -99,8 +99,8 @@
if (changed)
{
ByteArrayOutputStream bout = new
ByteArrayOutputStream();
- Preprocessor.saveStrings(lines, bout);
- String r = new String(bout.toByteArray());
+ Preprocessor.saveStrings(lines, bout,
fileBuffer.getEncoding());
+ String r = new String(bout.toByteArray(),
fileBuffer.getEncoding());
DocumentRewriteSession session =
document.startRewriteSession(DocumentRewriteSessionType.UNRESTRICTED);
document.set(r);
document.stopRewriteSession(session);
tzur Sayag wrote:
> Dear Craig,
>
> the file properties are set correctly to UTF-8 and the original file
> shows nicely in the editor as should,
> the preprocessed version however is still corrupted unless I disable
> the preprocessor,
> any hints? perhaps Omri can shed a light about "
> preprocessor.preprocess(is, bos, charset);" which seems to ignore the
> charset,
> any possibility to get a log of these operations?
> where is the code you quote taken from? the eclipseME plugin sources?
> perhaps I can debug it and help fix it,
>
> bests,
> --tzurs
>
> On 10/23/07, *Craig Setera* <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
> tzur,
>
> As of 1.7.6, the preprocessor is Omry's preprocessor and there is no
> other choice in the matter. Looking at the code, it goes
> something like
> this...
>
> File localFile = srcFile.getLocation ().toFile();
> String charset = srcFile.getContentDescription().getCharset();
> ....
> try {
> preprocessor.setSymbols(symbols.toString());
> preprocessor.preprocess(is, bos, charset);
>
> So, you can see that the character set is being pulled from the file's
> content description. That can be controlled via the properties of the
> file. Please check the file properties and see if the character set
> encoding is set correctly.
>
> Craig
>
> tzur Sayag wrote:
> > Hi Guys,
> >
> > I'm a very very happy user of eclipseME and generally have 0 issues
> > with it.
> > lately (I can't pin point the exact time) I have the following
> problem,
> > I have a java file (aa.java) which is saved as UTF-8 and
> contains many
> > strings in various languages.
> > when I run the midlet (both emulator and device), the strings
> are all
> > corrupted. by corrupted I mean that some letters (I can't seem
> to pin
> > point the issue) get corrupted and are displayed as 2 chars (square
> > and ?) as if the encoding was completely ignored.
> > I've tried everything possible with the file (it happens on various
> > files on this project) but nothing fixes the problem.
> > today I noticed that the problem is not just runtime, the java files
> > are copied (eclipseME I assume) to a directory called
> > .processed/
> > looking at the java files in there I see the exact problem, the
> files
> > are all corrupted (string literal wise)
> > I'm pretty sure something got corrupted in my eclipseME
> configuration
> > because everything ran fine until just a while ago,
> > i can easily attach the original file/s and the .processed ones if
> > this helps in any way,
> > if I disable preprocessing which I need, the problem is gone. (by
> > preprocessing I mean right click the project and click j2me),
> >
> > info: using eclipse 3.2.2 eclipseME 1.7.6
> > I'm not sure how to tell which preprocessor I'm using as I've seen
> > some notes about Omri's new preprocessor, but I'm not sure if
> this is
> > the one that
> > I'm using, I haven't done anything special to use or no use it
> other
> > than selecting the "enable preprocessing" thing on the j2me
> context menu.
> >
> > I would appreciate any hint because this is a major problem for me,
> > Best regards,
> > --tzurs
> >
> ------------------------------------------------------------------------
>
> >
> >
> -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems? Stop.
> > Now Search log events and configuration files using AJAX and a
> browser.
> > Download your FREE copy of Splunk now >> http://get.splunk.com/
> >
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Eclipseme-users mailing list
> > [email protected]
> <mailto:[email protected]>
> > https://lists.sourceforge.net/lists/listinfo/eclipseme-users
> >
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a
> browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Eclipseme-users mailing list
> [email protected]
> <mailto:[email protected]>
> https://lists.sourceforge.net/lists/listinfo/eclipseme-users
> <https://lists.sourceforge.net/lists/listinfo/eclipseme-users>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Eclipseme-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/eclipseme-users
>
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Eclipseme-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/eclipseme-users
