[jira] [Commented] (VELOCITY-880) DataSourceResourceLoader corrupts UTF-8 encoded characters in template
[
https://issues.apache.org/jira/browse/VELOCITY-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063787#comment-16063787
]
Michael Osipov commented on VELOCITY-880:
-
>From {{public synchronized Reader getResourceReader(final String name, String
>encoding)}}.
> DataSourceResourceLoader corrupts UTF-8 encoded characters in template
> --
>
> Key: VELOCITY-880
> URL: https://issues.apache.org/jira/browse/VELOCITY-880
> Project: Velocity
> Issue Type: Bug
>Affects Versions: 2.1.x
> Environment: Oracle12c and HSQLDB 2.3.4, JDK 1.8
>Reporter: James R Doyle
>Assignee: Claude Brisson
> Fix For: 2.0
>
> Attachments: project.tar.gz, velocity-880.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A long-withstanding bug in the DataSourceResourceLoader corrupts UTF-8
> templates retrieved from the database. The Unit Test suite for this resource
> loader has deficiencies that hide the bug.
> The cause of the problem is this:
> {code}
> InputStream rawStream = rs.getAsciiStream(templateColumn);
> The resolution of the problem is simply:
> Reader r = rs.getCharacterStream(templateColumn);
> InputStream rawStream = null;
>try {
> rawStream = IOUtils.toInputStream(IOUtils.toString(r),
> encoding);
> } catch (IOException ioe) {}
> {code}
> Once done, the test failure vanishes:
> org.junit.ComparisonFailure: Unicode test failed.
> Expected :The Euro Currency Symbol € is a two-byte UTF-8 encoded
> character.
> Actual :The Euro Currency Symbol ? is a two-byte UTF-8 encoded
> character.
> The bug was verified and the fix was tested against Oracle12c and HSQLDB
> 2.3.4 using a CLOB column to store the template data.
> The Unit Tests for this resource loader need attention.
> Please see VELOCITY-599 ; long standing problem, which has been erroneously
> marked as resolved but has been in the codebase for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org
For additional commands, e-mail: dev-h...@velocity.apache.org
[jira] [Commented] (VELOCITY-880) DataSourceResourceLoader corrupts UTF-8 encoded characters in template
[
https://issues.apache.org/jira/browse/VELOCITY-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063780#comment-16063780
]
Claude Brisson commented on VELOCITY-880:
-
The method takes the name of the column, so the user can potentially use it to
build the reader with the encoding he wants. Otherwise than that; where would
that encoding come from?
Also, storing the template in a BLOB column rather than in a CLOB column is
obviously a design mistake that I don't think we should care about.
> DataSourceResourceLoader corrupts UTF-8 encoded characters in template
> --
>
> Key: VELOCITY-880
> URL: https://issues.apache.org/jira/browse/VELOCITY-880
> Project: Velocity
> Issue Type: Bug
>Affects Versions: 2.1.x
> Environment: Oracle12c and HSQLDB 2.3.4, JDK 1.8
>Reporter: James R Doyle
>Assignee: Claude Brisson
> Fix For: 2.0
>
> Attachments: project.tar.gz, velocity-880.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A long-withstanding bug in the DataSourceResourceLoader corrupts UTF-8
> templates retrieved from the database. The Unit Test suite for this resource
> loader has deficiencies that hide the bug.
> The cause of the problem is this:
> {code}
> InputStream rawStream = rs.getAsciiStream(templateColumn);
> The resolution of the problem is simply:
> Reader r = rs.getCharacterStream(templateColumn);
> InputStream rawStream = null;
>try {
> rawStream = IOUtils.toInputStream(IOUtils.toString(r),
> encoding);
> } catch (IOException ioe) {}
> {code}
> Once done, the test failure vanishes:
> org.junit.ComparisonFailure: Unicode test failed.
> Expected :The Euro Currency Symbol € is a two-byte UTF-8 encoded
> character.
> Actual :The Euro Currency Symbol ? is a two-byte UTF-8 encoded
> character.
> The bug was verified and the fix was tested against Oracle12c and HSQLDB
> 2.3.4 using a CLOB column to store the template data.
> The Unit Tests for this resource loader need attention.
> Please see VELOCITY-599 ; long standing problem, which has been erroneously
> marked as resolved but has been in the codebase for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org
For additional commands, e-mail: dev-h...@velocity.apache.org
[jira] [Commented] (VELOCITY-880) DataSourceResourceLoader corrupts UTF-8 encoded characters in template
[
https://issues.apache.org/jira/browse/VELOCITY-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063610#comment-16063610
]
Michael Osipov commented on VELOCITY-880:
-
Just checked the changes. Look good to me, but I am uncertain for the
{{DataSourceResourceLoader#getReader()}} method. The protected one does not
have the encoding passed. Given that a template could also be stored in a BLOB,
a custom loader wouldn't be able to return a proper reader. WDYT?
> DataSourceResourceLoader corrupts UTF-8 encoded characters in template
> --
>
> Key: VELOCITY-880
> URL: https://issues.apache.org/jira/browse/VELOCITY-880
> Project: Velocity
> Issue Type: Bug
>Affects Versions: 2.1.x
> Environment: Oracle12c and HSQLDB 2.3.4, JDK 1.8
>Reporter: James R Doyle
>Assignee: Claude Brisson
> Fix For: 2.0
>
> Attachments: project.tar.gz, velocity-880.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A long-withstanding bug in the DataSourceResourceLoader corrupts UTF-8
> templates retrieved from the database. The Unit Test suite for this resource
> loader has deficiencies that hide the bug.
> The cause of the problem is this:
> {code}
> InputStream rawStream = rs.getAsciiStream(templateColumn);
> The resolution of the problem is simply:
> Reader r = rs.getCharacterStream(templateColumn);
> InputStream rawStream = null;
>try {
> rawStream = IOUtils.toInputStream(IOUtils.toString(r),
> encoding);
> } catch (IOException ioe) {}
> {code}
> Once done, the test failure vanishes:
> org.junit.ComparisonFailure: Unicode test failed.
> Expected :The Euro Currency Symbol € is a two-byte UTF-8 encoded
> character.
> Actual :The Euro Currency Symbol ? is a two-byte UTF-8 encoded
> character.
> The bug was verified and the fix was tested against Oracle12c and HSQLDB
> 2.3.4 using a CLOB column to store the template data.
> The Unit Tests for this resource loader need attention.
> Please see VELOCITY-599 ; long standing problem, which has been erroneously
> marked as resolved but has been in the codebase for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org
For additional commands, e-mail: dev-h...@velocity.apache.org
[jira] [Resolved] (VELOCITY-880) DataSourceResourceLoader corrupts UTF-8 encoded characters in template
[
https://issues.apache.org/jira/browse/VELOCITY-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claude Brisson resolved VELOCITY-880.
-
Resolution: Fixed
Assignee: Claude Brisson
Fix Version/s: 2.0
Using getCharacterStream() is the proper solution.
Tested against HSQLDB, Derby, Oracle, SQL Server and PostgreSQL.
> DataSourceResourceLoader corrupts UTF-8 encoded characters in template
> --
>
> Key: VELOCITY-880
> URL: https://issues.apache.org/jira/browse/VELOCITY-880
> Project: Velocity
> Issue Type: Bug
>Affects Versions: 2.1.x
> Environment: Oracle12c and HSQLDB 2.3.4, JDK 1.8
>Reporter: James R Doyle
>Assignee: Claude Brisson
> Fix For: 2.0
>
> Attachments: project.tar.gz, velocity-880.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> A long-withstanding bug in the DataSourceResourceLoader corrupts UTF-8
> templates retrieved from the database. The Unit Test suite for this resource
> loader has deficiencies that hide the bug.
> The cause of the problem is this:
> {code}
> InputStream rawStream = rs.getAsciiStream(templateColumn);
> The resolution of the problem is simply:
> Reader r = rs.getCharacterStream(templateColumn);
> InputStream rawStream = null;
>try {
> rawStream = IOUtils.toInputStream(IOUtils.toString(r),
> encoding);
> } catch (IOException ioe) {}
> {code}
> Once done, the test failure vanishes:
> org.junit.ComparisonFailure: Unicode test failed.
> Expected :The Euro Currency Symbol € is a two-byte UTF-8 encoded
> character.
> Actual :The Euro Currency Symbol ? is a two-byte UTF-8 encoded
> character.
> The bug was verified and the fix was tested against Oracle12c and HSQLDB
> 2.3.4 using a CLOB column to store the template data.
> The Unit Tests for this resource loader need attention.
> Please see VELOCITY-599 ; long standing problem, which has been erroneously
> marked as resolved but has been in the codebase for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org
For additional commands, e-mail: dev-h...@velocity.apache.org
Re: [RESULT] [VOTE] Engine 2.0 RC8 Release quality
Maybe bump commons-lang3 to 3.6? Just released in June. Cheers Greg On 18 June 2017 at 22:13, Claude Brisson wrote: > I should be able to find the time, yes. > > Claude > > > On 18/06/2017 10:46, Michael Osipov wrote: > >> Am 2017-06-17 um 11:46 schrieb Claude Brisson: >> >>> Result: >>> >>> Two binding votes and two non-binding votes for GA. >>> >>> While at least two PMCs did express their will to vote, a user raised an >>> issue about the DataSourceResourceLoader [1] before they do so. >>> >>> So let's hit the road again for another RC. >>> >>> Claude >>> >>> [1] https://issues.apache.org/jira/browse/VELOCITY-880 >>> >> >> Salut Claude, >> >> unfortunately I wasn't able to perform any tests and I won't have access >> to those database instances before the end of the month. Were you able to >> test anything? >> >> Michael >> >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org >> For additional commands, e-mail: dev-h...@velocity.apache.org >> >> >> > > - > To unsubscribe, e-mail: dev-unsubscr...@velocity.apache.org > For additional commands, e-mail: dev-h...@velocity.apache.org > >
