[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446040#comment-17446040 ] Will Dazey commented on OPENJPA-2767: - [~struberg] Sorry for the delay in responding! Yes, this should be resolved. > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 3.2.1 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v4_p2.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315102#comment-17315102 ] Mark Struberg commented on OPENJPA-2767: Hi [~dazeydev] I think this is finally resolved, isn't? > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 3.1.3 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v4_p2.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005285#comment-17005285 ] Jonathan Pomes commented on OPENJPA-2767: - I confirm that our environment is now stable in QA (Websphere / 10+ ear / load balancing) and no issue occurs anymore. No performance issues have been noticed either. > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v4_p2.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964967#comment-16964967 ] Will Dazey commented on OPENJPA-2767: - I am adding back the extra synchronization to `MetaDataRepository.processRegisteredClasses()` for `_registered` so that `MetaDataRepository.register()` can safely add to `_registered`. This synchronization block was in the original code, but was removed in the previous fix as I thought method synchronization was enough. This was an oversight in my previous fix and I shouldn't have removed that sync block. Exception: ``` Caused by: java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextNode(HashMap.java:1456) at java.util.HashMap$KeyIterator.next(HashMap.java:1480) at java.util.AbstractCollection.finishToArray(AbstractCollection.java:243) at java.util.AbstractCollection.toArray(AbstractCollection.java:210) at org.apache.openjpa.meta.MetaDataRepository.processRegisteredClasses(MetaDataRepository.java:1638) at org.apache.openjpa.meta.ClassMetaData.getPCSubclasses(ClassMetaData.java:377) ``` Due to the difficulties in testing these race conditions, I did not include a test with the patch. However, for what it is worth, the patch was tested in an application server getting these intermittent ConcurrentModificationExceptions and they noticed them go away. I am currently having our performance team make sure performance is not negatively impacted by this change. > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v4_p2.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964891#comment-16964891 ] Jody Grassel commented on OPENJPA-2767: --- LGTM > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v4_p2.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863173#comment-16863173 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit 7c54dd855ba283cb6bb0ff87355a00fd60edfbd4 in openjpa's branch refs/heads/master from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=7c54dd8 ] OPENJPA-2767: fix for checkstyles (#48) Signed-off-by: Will Dazey > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844094#comment-16844094 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit 02ca4645733c8250d07d5003bfff17b7e5677fc9 in openjpa's branch refs/heads/2.2.x from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=02ca464 ] Merge pull request #47 from dazey3/2767_2.2.x OPENJPA-2767: fix for checkstyles > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844093#comment-16844093 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit 4d673f3715b2f3ae28d9bb4d70fad431eece8cb6 in openjpa's branch refs/heads/2.2.x from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=4d673f3 ] OPENJPA-2767: fix for checkstyles Signed-off-by: Will Dazey > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3, 3.1.1 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841630#comment-16841630 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit 389a82db97b96bd2bf6b519afdc64e833f5c0478 in openjpa's branch refs/heads/master from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=389a82d ] OPENJPA-2767: Incomplete ValueMapDiscriminatorStrategy cache and MetaDataRepository race condition Signed-off-by: Will Dazey > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 50m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841631#comment-16841631 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit cd81d3602436083614afc77bacda4099d3f269c5 in openjpa's branch refs/heads/master from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=cd81d36 ] Merge pull request #45 from dazey3/2767_master OPENJPA-2767: Incomplete ValueMapDiscriminatorStrategy cache and MetaDataRepository race condition > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 50m > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841633#comment-16841633 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit a41cd39da8a55fb9dd1a8ff7b84823958a0bcd55 in openjpa's branch refs/heads/2.2.x from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=a41cd39 ] Merge pull request #43 from dazey3/2767_2.2.x OPENJPA-2767: Incomplete ValueMapDiscriminatorStrategy cache and MetaDataRepository race condition > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 1h > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841632#comment-16841632 ] ASF subversion and git services commented on OPENJPA-2767: -- Commit 29b82b23efa1db5024706851063c1988fb130398 in openjpa's branch refs/heads/2.2.x from Will Dazey [ https://gitbox.apache.org/repos/asf?p=openjpa.git;h=29b82b2 ] OPENJPA-2767: Incomplete ValueMapDiscriminatorStrategy cache and MetaDataRepository race condition Signed-off-by: Will Dazey > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch > > Time Spent: 1h > Remaining Estimate: 0h > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839147#comment-16839147 ] Jonathan Pomes commented on OPENJPA-2767: - I've been checking this solution in Websphere Z-OS for about 2 weeks (10 ears + load balancing environment) and It looks like it's stable and works fine. No performances issues have been noticed. > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch > > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[
https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838841#comment-16838841
]
Will Dazey commented on OPENJPA-2767:
-
A further issue I noticed was that for large ORMs, a race condition occurs in
MetaDataRepository.processRegisteredClasses. This race condition causes other
threads to initialize other caches too early.
To explain this race condition, take a look at
MetaDataRepository.processRegisteredClasses():
{code:java}
Class[] processRegisteredClasses(ClassLoader envLoader) {
if (_registered.isEmpty())
return EMPTY_CLASSES;
// copy into new collection to avoid concurrent mod errors on reentrant
// registrations
Class[] reg;
synchronized (_registered) {
reg = _registered.toArray(new Class[_registered.size()]);
_registered.clear();
}
//Code then proceeds to process the 'reg' copy
...
}
{code}
The race condition revolves around the fact that `_registered` is cleared, but
there is still more processing to do before `MetaDataRepository._subs` actually
becomes populated. This creates a very small window in time for a second thread
to call `MetaDataRepository.processRegisteredClasses()`, see that
`_registered.isEmpty()` is true, and therefore assume it's safe to try to get
subclasses from `MetaDataRepository._subs`. We can observe this sort of call
sequence in ClassMetaData.getPCSubclasses (which is used to initialize
ValueMapDiscriminator's cache):
{code:java}
public Class[] getPCSubclasses() {
if (_owner != null)
return MetaDataRepository.EMPTY_CLASSES;
_repos.processRegisteredClasses(_loader);
if (_subs == null) {
Collection> subs = _repos.getPCSubclasses(_type);
_subs = (Class[]) subs.toArray(new Class[subs.size()]);
}
return _subs;
}
{code}
*notice that `ClassMetaData._subs` is populated from
`MetaDataRepository.getPCSubclasses()` (which uses the
'MetaDataRepository._subs' cache I mentioned above).
Now, imagine the scenario where one thread (T1) comes into this method and
kicks off `MetaDataRepository.processRegisteredClasses()`, which will clear
`MetaDataRepository._registered`. In the mean time, another thread (T2) also
comes through this method, but since `MetaDataRepository._registered` is empty,
it returns immediately. For T2, `ClassMetaData._subs` will be null and T1 is
still in the middle of processing registered classes. This is the window where
issues can start appearing and dirty reads are made.
T2 will possibly initialize `ClassMetaData._subs` as an empty List since T1 is
still busy populating `MetaDataRepository._subs`. ValueMapDiscriminator will
then get an empty result in its cache, even though `MetaDataRepository._subs`
will soon populate with the correct value.
Now, `MetaDataRepository.processRegisteredClasses()` will clear
`ClassMetaData._subs` so that it can be reinitialized later. But how many times
does ValueMapDiscriminator need to call getSubClasses until the value is
finally populated? The most straightforward answer to this is to synchronize on
`MetaDataRepository.processRegisteredClasses()` so that other threads wont make
dirty reads.
> Incomplete ValueMapDiscriminatorStrategy cache
> ---
>
> Key: OPENJPA-2767
> URL: https://issues.apache.org/jira/browse/OPENJPA-2767
> Project: OpenJPA
> Issue Type: Bug
>Reporter: Will Dazey
>Assignee: Will Dazey
>Priority: Minor
> Fix For: 2.2.3
>
> Attachments: OJ2767_2.2.x_v4.patch
>
>
> There exists the possibility that the discriminator class cache in
> ValueMapDiscriminatorStrategy was originally created before the server fully
> loaded all classes. If this happens, the cache is incomplete and needs to be
> rebuilt. A simple enough strategy would be to just try rebuilding the cache,
> before giving up and throwing a CNFE, and check again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OPENJPA-2767) Incomplete ValueMapDiscriminatorStrategy cache
[ https://issues.apache.org/jira/browse/OPENJPA-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770853#comment-16770853 ] Jonathan Pomes commented on OPENJPA-2767: - I'v been trying this fix and it's not effective. The fix here tries to reload the ValueMapDiscriminator cache from the open jpa registry if it fail once. However we cannot assume that the OpenJpa registry is ready yet even after a second cache load. One way to grantee this to work might be to wait for the OpenJpa registry to by fully loaded with all the jpa Classes available in the ClassLoader (not very effective thougth). > Incomplete ValueMapDiscriminatorStrategy cache > --- > > Key: OPENJPA-2767 > URL: https://issues.apache.org/jira/browse/OPENJPA-2767 > Project: OpenJPA > Issue Type: Bug >Reporter: Will Dazey >Assignee: Will Dazey >Priority: Minor > Fix For: 2.2.3 > > Attachments: OJ2767_2.2.x_v4.patch, OJ2767_2.2.x_v5.patch > > > There exists the possibility that the discriminator class cache in > ValueMapDiscriminatorStrategy was originally created before the server fully > loaded all classes. If this happens, the cache is incomplete and needs to be > rebuilt. A simple enough strategy would be to just try rebuilding the cache, > before giving up and throwing a CNFE, and check again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
