We've migrated a national public sector e-procurement system to Torque
from plain JDBC... it's a big production use, and for indexing a
catalogue of many tens of thousands of records with Lucene, LargeSelect
is an absolute necessity!
Bit concerned about your comments on turbine community shrinkage, what's
going on here do you think?
I'll have a look at the scarab thingy. I was just concerned that I was
finding things, fixing things, and other people weren't getting any
benefit (which is kinda the point of this open source thing). I wasn't
sure if that was because my patches and add-ons were not acceptable or
if they just weren't being noticed because they were in the wrong place.
Unified diff below. I've no idea how you could produce a junit testcase
for this one either. Btw, I have problems producing diffs for some of my
changes because they are cumulative on top of other more major
enhancements (such as the betwixt XML support) I added so I can't diff
against the HEAD of that file and get anything meaningful.. is there a
good way around this?
Index: LargeSelect.java
===================================================================
RCS file: /home/cvspublic/jakarta-turbine-
torque/src/java/org/apache/torque/util/LargeSelect.java,v
retrieving revision 1.7
diff -u -r1.7 LargeSelect.java
--- LargeSelect.java 27 Sep 2002 16:09:20 -0000 1.7
+++ LargeSelect.java 15 Oct 2002 16:40:37 -0000
@@ -644,7 +644,11 @@
+ "start-blockBegin (" + fromIndex + ") through "
+ "fromIndex + Math.min(size, results.size() -
fromIndex) ("
+ toIndex + ")");
- List returnResults = results.subList(fromIndex, toIndex);
+ List returnResults;
+
+ synchronized (results) {
+ returnResults = new ArrayList(results.subList(fromIndex,
toIndex));
+ }
if (null != returnBuilderClass)
{
@@ -716,9 +720,11 @@
List tempResults
= BasePeer.getSelectResults(qds, size, false);
- for (int i = 0, n = tempResults.size(); i < n; i++)
- {
- results.add(tempResults.get(i));
+ synchronized (results) {
+ for (int i = 0, n = tempResults.size(); i < n; i++)
+ {
+ results.add(tempResults.get(i));
+ }
}
currentlyFilledTo += tempResults.size();
@@ -728,7 +734,9 @@
// on the last page but we must now get rid of it.
if (results.size() == memoryLimit + 1)
{
- results.remove(currentlyFilledTo--);
+ synchronized (results) {
+ results.remove(currentlyFilledTo--);
+ }
perhapsLastPage = false;
}
Regards
Gareth
On Tuesday, October 15, 2002, at 02:13 pm, Scott Eade wrote:
> Posting to -dev is good for discussion (but the level of discussion
> ain't
> what it used to be). You can also submit issues (including patches) to
> Scarab (http://nagoya.apache.org/scarab/issues/) which a committer will
> hopefully notice and deal with.
>
> I have to say that the probability of seeing any direct follow up at
> present
> to anything raised on any turbine related list or Scarab is extremely
> low.
> It appears that most of the committers are either occupied elsewhere or
> have
> permanently left the turbine community.
>
> That said, I am happy to look into any issues that arise with
> LargeSelect
> because I have a production system that uses it extensively. I will
> have a
> look to see if I can figure out where your changes fit in. It would be
> helpful if you could repost your patch in unidiff format (cvs diff
> -u). The
> multithreaded nature of LargeSelect makes it a right PITA to test. At
> one
> stage, before marking a few fields as volatile, some of the test cases
> would
> seemingly randomly fail. In suspect that a test case targeted at the
> problem you describe would suffer from a similar problem.
>
> BTW: Posting code to the -user list (including patches) is fine. You
> will
> find that as you become a more serious user/contributor you will
> gravitate
> to the -dev list quite naturally.
>
> And please don't cross-post to both lists (Hello to Partyk ;-).
>
> Cheers,
>
> Scott
> --
> Scott Eade
> Backstage Technologies Pty. Ltd.
> http://www.backstagetech.com.au
>
>
>> From: Gareth Boden <[EMAIL PROTECTED]>
>> Reply-To: "Turbine Torque Users List" <turbine-torque-
>> [EMAIL PROTECTED]>
>> Date: Tue, 15 Oct 2002 12:42:35 +0100
>> To: "Turbine Torque Users List" <turbine-torque-
>> [EMAIL PROTECTED]>
>> Subject: Re: Bug in LargeSelect
>>
>> Apologies for sending to the wrong list, I've now subscribed to the dev
>> one.
>>
>> btw, this is not the first code-related thing I've posted in here, and
>> I've not had much response.. is anyone doing anything with the stuff I
>> send? Is there a better way to submit patches and changes? In future I
>> was planning to post to -dev list, but if there's a better way than
>> that....?
>>
>> G.
>>
>> On Tuesday, October 15, 2002, at 09:34 am, Gareth Boden wrote:
>>
>>> Hi folks,
>>>
>>> Found a problem in LargeSelect - the list of results is passed using
>>> ArrayList.sublist to the populateObjects() method, but in the meantime
>>> results may still be being added to the list by a separate Thread,
>>> which results in a ConcurrentModificationException.
>>>
>>> Need to construct a new ArrayList containing results to pass to
>>> populateObjects(), and synchronize on returnResults when constructing
>>> the new list or adding/removing from the main one.
>>>
>>> I might be wrong, but I think this patch is worthwhile:
>>>
>>> /usr/bin/cvs diff -w LargeSelect.java
>>> Index: LargeSelect.java
>>> ===================================================================
>>> RCS file: /home/cvspublic/jakarta-turbine-
>>> torque/src/java/org/apache/torque/util/LargeSelect.java,v
>>> retrieving revision 1.7
>>> diff -w -r1.7 LargeSelect.java
>>> 647c647,651
>>> < List returnResults = results.subList(fromIndex, toIndex);
>>> ---
>>>> List returnResults;
>>>>
>>>> synchronized (results) {
>>>> returnResults = new ArrayList(results.subList(fromIndex,
>>> toIndex));
>>>> }
>>> 718a723
>>>> synchronized (results) {
>>> 722a728
>>>> }
>>> 730a737
>>>> synchronized (results) {
>>> 731a739
>>>> }
>>>
>>> Gareth
>>>
>> ---
>> Development Team Leader, eGS, 5-7 Singer Street, London EC2A 4BQ. +44
>> (0)20 7336 1440
>>
>> CONFIDENTIALITY: This is email is confidential and intended solely for
>> the use of the individual to whom it is addressed.� Any views or
>> opinions presented are solely those of the author and do not
>> necessarily
>> represent those of eGovernment Solutions (UK) Ltd.� If you are not the
>> intended recipient, be advised that you have received this email in
>> error and that any use, dissemination, forwarding, or copying of this
>> email is strictly prohibited.� If you have received this email in error
>> please contact the sender.
>>
>>
>> --
>> To unsubscribe, e-mail:
>> <mailto:[EMAIL PROTECTED]>
>> For additional commands, e-mail:
>> <mailto:[EMAIL PROTECTED]>
>>
>
>
> --
> To unsubscribe, e-mail: <mailto:turbine-torque-user-
> [EMAIL PROTECTED]>
> For additional commands, e-mail: <mailto:turbine-torque-user-
> [EMAIL PROTECTED]>
>
>
---
Development Team Leader, eGS, 5-7 Singer Street, London EC2A 4BQ. +44
(0)20 7336 1440
CONFIDENTIALITY: This is email is confidential and intended solely for
the use of the individual to whom it is addressed.� Any views or
opinions presented are solely those of the author and do not necessarily
represent those of eGovernment Solutions (UK) Ltd.� If you are not the
intended recipient, be advised that you have received this email in
error and that any use, dissemination, forwarding, or copying of this
email is strictly prohibited.� If you have received this email in error
please contact the sender.
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
