RE: Getting all the ClassMetaDatas
Compass provides two main features with JPA: Mirroring and Indexing. Mirroring mirrors changes made through the JPA API into the search engine (through lifecycle listeners), and Indexing allows to automatically index all your database using both the JPA and Searchable classes. The indexing process requires to fetch or intersect with the current classes that are persistent. The indexing process fetches all the indexable entities and then iterate (in parallel) them in order to index them into the search engine. So, I am guessing that if classes are introduced to JPA at runtime, the user would need to pre-register them with OpenJPA (when using the OpenJPA plugin in order to locate persistent entities) in one of the ways that OpenJPA provides. The user could, if only Annotations are used, to use the default entities locator that comes with Compass, which basically check for the @Entity annotation. The only main drawback with this one is that it does not support xml or other mechanism to introduce new mappings for classes. -Shay Patrick Linskey wrote: Is there any reason why you need to eagerly get information about classes to process? In general, as you've noticed, OpenJPA does allow dynamic registration of persistent types. One possibility would be to declare that in order to use Compass searching with OpenJPA, one must provide a static list of classes (or tell OpenJPA to compute a static list of classes), using one of the options that Marc pointed out earlier. Alternately, you could potentially just register the right type of listener with OpenJPA and do whatever initialization is necessary lazily as new classes are encountered via the callbacks. -Patrick -- Patrick Linskey BEA Systems, Inc. ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. -Original Message- From: Shay Banon [mailto:[EMAIL PROTECTED] Sent: Monday, January 01, 2007 1:11 PM To: open-jpa-dev@incubator.apache.org Subject: Getting all the ClassMetaDatas Hi, First, I hope that this is the correct forum for posting questions, so sorry if it isn't. I have an external list of classes that I would like to match against the persistent classes that are defined/identified by OpenJPA. I would really like to get the ClassMetaData for each one, since it has a lot of information that I could use. This intersection happens after the EntityManagerFactory has been created. I have tried using:ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); But it seems like the meta data repository and ClassMetaData information are lazily loaded (i.e. when some operation is performed on a Class, the relevant meta data is fetched if not found in cache). So, what I get is an empty array (even though I can see the OpenJPA identified the classes). I wonder how I would be able to get all the class meta data? Something that I was thinking about is since I have the list of classes that I would like to check if they are persistent, I could call: getMetaData(Class cls, ClassLoader envLoader, boolean mustExist), with Thread context class loader and false in mustExists. I am guessing that it will load the ClassMetaData if not found. My main problem here is that OpenJPA might be configured with a different class loader (though it defaults to the thread context one). Any suggestions? p.s. I am the author of Compass, so once I have this nailed down, we will have Search capabilities to OpenJPA ;) -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426 .html#a8116958 Sent from the open-jpa-dev mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426.html#a8121024 Sent from the open-jpa-dev mailing list archive at Nabble.com.
Re: Getting all the ClassMetaDatas
I tried to open the entity manager before I get the ClassMetaData, but I still get an empty array. Here is what I do: OpenJPAEntityManagerFactory emf = OpenJPAPersistence.cast(entityManagerFactory); EntityManager entityManager = emf.createEntityManager(); entityManager.close(); ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); I do enumerate the classes in my persistence context, and I can see in the logging that OpenJPA parses the classes. Marc Prud wrote: Shay- Have you already obtained an EM from the EMF before you make this call? If you try to get the metadatas after calling emf.getEntityManager(), do you still see an empty list? Also, note that unless you enumerate the classes in your persistence.xml file (in the class elements), the only way the system will be able to know about your classes before they are lazily evaluated is if you enable one of the scanning features (e.g., but packaging all your classes in a jar and specifying the jar-file element in the persistence.xml, which will be automatically scanned for persistent classes). You might want to enable verbose logging and watch the make sure the class metadatas are registered before you try to get the list from the repository. On Jan 1, 2007, at 4:11 PM, Shay Banon wrote: Hi, First, I hope that this is the correct forum for posting questions, so sorry if it isn't. I have an external list of classes that I would like to match against the persistent classes that are defined/identified by OpenJPA. I would really like to get the ClassMetaData for each one, since it has a lot of information that I could use. This intersection happens after the EntityManagerFactory has been created. I have tried using:ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); But it seems like the meta data repository and ClassMetaData information are lazily loaded (i.e. when some operation is performed on a Class, the relevant meta data is fetched if not found in cache). So, what I get is an empty array (even though I can see the OpenJPA identified the classes). I wonder how I would be able to get all the class meta data? Something that I was thinking about is since I have the list of classes that I would like to check if they are persistent, I could call: getMetaData(Class cls, ClassLoader envLoader, boolean mustExist), with Thread context class loader and false in mustExists. I am guessing that it will load the ClassMetaData if not found. My main problem here is that OpenJPA might be configured with a different class loader (though it defaults to the thread context one). Any suggestions? p.s. I am the author of Compass, so once I have this nailed down, we will have Search capabilities to OpenJPA ;) -- View this message in context: http://www.nabble.com/Getting-all-the- ClassMetaDatas-tf2905426.html#a8116958 Sent from the open-jpa-dev mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426.html#a8121096 Sent from the open-jpa-dev mailing list archive at Nabble.com.
RE: Getting all the ClassMetaDatas
-Original Message- From: Patrick Linskey Sent: Tuesday, January 02, 2007 1:44 AM To: open-jpa-dev@incubator.apache.org Subject: RE: Getting all the ClassMetaDatas You may also be interested in the StateManager.getDirty() method, which returns a BitSet corresponding to the entries in StateManager.getMetaData().getFields(). The BitSet identifies which fields in a given object are modified. On top of that, you could also take advantage of StateManager.getFlushed(), which returns another BitSet indicating which fields have already been flushed. Combining the two, you can compute which fields are dirty and unflushed; in a pre-flush callback, these are the fields that have been mutated since the last time flush() was invoked (directly or indirectly). Correction: both of those methods are in OpenJPAStateManager, not StateManager. Sorry for any confusion. -Patrick ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
RE: Getting all the ClassMetaDatas
Regarding getting dirty fields, Compass does not make use of it. When an object changes, it must be completely reindexed by Compass (or deleted). In my listeners, I simply get the source, and perform the appropriate operation on Compass. It would be nice if Compass supported dirty fields, but it is not simple at all to support it (mainly because of Lucene). It is really fast though, especially because of how Compass supports transactions (explained in the next paragraph). Compass extends Lucene core classes to add 2pc transaction support. By default, the transactional data is stored in memory, and flushed to the index during commit. Compass also supports storing the transactional data on the file system, which basically allows for much longer running transactions. This is both a configuration setting and runtime setting. Note as well, during the Indexing operation (different than the mirroring one) uses different transactional isolation called batch insert, which has no problems to perform long running *fresh* indexing process. On top of the Lucene extension, Compass integrates nicely with different transaction managers. Namely, JTA (both JTA synchronization and XA) and Spring PlatformTransactionManager. The only soft point is when using OpenJPA in Resource Local transaction mode without any transaction manager. Compass could (as you suggested) integrate its transaction management with OpenJPA in such cases. I will look into it after I get the first integration stuff working. Last, regarding savepoints, Compass does not support savepoints currently, though with the current transaction architecture it could be easily added. The main point (as you mentioned) is integrating it with other savepoints enabled transaction strategies. Cheers, Shay Patrick Linskey wrote: You may also be interested in the StateManager.getDirty() method, which returns a BitSet corresponding to the entries in StateManager.getMetaData().getFields(). The BitSet identifies which fields in a given object are modified. On top of that, you could also take advantage of StateManager.getFlushed(), which returns another BitSet indicating which fields have already been flushed. Combining the two, you can compute which fields are dirty and unflushed; in a pre-flush callback, these are the fields that have been mutated since the last time flush() was invoked (directly or indirectly). Speaking of incremental flushing, is Compass transactional? IOW, is it possible to periodically (at flush() time) update Compass with mutations, and then only make the changes visible outside the current transactional scope at commit time? If so, it'd be interesting to also explore how we could hook up OpenJPA savepoints (when available). If not, then we should make sure we figure out what the memory implications are of using Compass + OpenJPA incremental flushes + large transactions. OpenJPA has features designed for optimizing memory handling in large transactions; Compass/OpenJPA work could probably dovetail nicely into some or all of these existing integration points. -Patrick -- Patrick Linskey BEA Systems, Inc. ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. -Original Message- From: Shay Banon [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 02, 2007 12:22 AM To: open-jpa-dev@incubator.apache.org Subject: RE: Getting all the ClassMetaDatas Compass provides two main features with JPA: Mirroring and Indexing. Mirroring mirrors changes made through the JPA API into the search engine (through lifecycle listeners), and Indexing allows to automatically index all your database using both the JPA and Searchable classes. The indexing process requires to fetch or intersect with the current classes that are persistent. The indexing process fetches all the indexable entities and then iterate (in parallel) them in order to index them into the search engine. So, I am guessing that if classes are introduced to JPA at runtime, the user would need to pre-register them with OpenJPA (when using the OpenJPA plugin in order to locate persistent entities) in one of the ways that OpenJPA provides. The user could, if only Annotations are used, to use the default entities locator that comes with Compass, which basically check for the @Entity annotation. The only main drawback with this one is that it does not support xml or other mechanism to introduce new mappings for classes. -Shay Patrick
RE: Getting all the ClassMetaDatas
I still need to provide a classloader for this method. Another thing, I get only the classes, where I need to ClassMetaData in order to check if it extends another mapped class (in such cases I exclude it from the indexing process since the select of the base class will return the derived classes as well). I was wondering if maybe I work in the same way the MappingTool works (since it needs to get all the ClassMappings as well). I had a quick look at the code, an it does some stuff that I am not sure that I should do. What do you say? Patrick Linskey wrote: What happens if you use MetaDataRepository.getPersistentTypeNames() instead? -Patrick -- Patrick Linskey BEA Systems, Inc. ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. -Original Message- From: Shay Banon [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 02, 2007 12:34 AM To: open-jpa-dev@incubator.apache.org Subject: Re: Getting all the ClassMetaDatas I tried to open the entity manager before I get the ClassMetaData, but I still get an empty array. Here is what I do: OpenJPAEntityManagerFactory emf = OpenJPAPersistence.cast(entityManagerFactory); EntityManager entityManager = emf.createEntityManager(); entityManager.close(); ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); I do enumerate the classes in my persistence context, and I can see in the logging that OpenJPA parses the classes. Marc Prud wrote: Shay- Have you already obtained an EM from the EMF before you make this call? If you try to get the metadatas after calling emf.getEntityManager(), do you still see an empty list? Also, note that unless you enumerate the classes in your persistence.xml file (in the class elements), the only way the system will be able to know about your classes before they are lazily evaluated is if you enable one of the scanning features (e.g., but packaging all your classes in a jar and specifying the jar-file element in the persistence.xml, which will be automatically scanned for persistent classes). You might want to enable verbose logging and watch the make sure the class metadatas are registered before you try to get the list from the repository. On Jan 1, 2007, at 4:11 PM, Shay Banon wrote: Hi, First, I hope that this is the correct forum for posting questions, so sorry if it isn't. I have an external list of classes that I would like to match against the persistent classes that are defined/identified by OpenJPA. I would really like to get the ClassMetaData for each one, since it has a lot of information that I could use. This intersection happens after the EntityManagerFactory has been created. I have tried using:ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); But it seems like the meta data repository and ClassMetaData information are lazily loaded (i.e. when some operation is performed on a Class, the relevant meta data is fetched if not found in cache). So, what I get is an empty array (even though I can see the OpenJPA identified the classes). I wonder how I would be able to get all the class meta data? Something that I was thinking about is since I have the list of classes that I would like to check if they are persistent, I could call: getMetaData(Class cls, ClassLoader envLoader, boolean mustExist), with Thread context class loader and false in mustExists. I am guessing that it will load the ClassMetaData if not found. My main problem here is that OpenJPA might be configured with a different class loader (though it defaults to the thread context one). Any suggestions? p.s. I am the author of Compass, so once I have this nailed down, we will have Search capabilities to OpenJPA ;) -- View this message in context: http://www.nabble.com/Getting-all-the- ClassMetaDatas-tf2905426.html#a8116958 Sent from the open-jpa-dev mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426 .html#a8121096 Sent from the open-jpa-dev mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426.html#a8132249 Sent from the open-jpa
RE: Getting all the ClassMetaDatas
-Original Message- From: Shay Banon [mailto:[EMAIL PROTECTED] Sent: Monday, January 01, 2007 1:11 PM To: open-jpa-dev@incubator.apache.org Subject: Getting all the ClassMetaDatas ... p.s. I am the author of Compass, so once I have this nailed down, we will have Search capabilities to OpenJPA ;) Cool! FYI, you'll probably be interested in org.apache.openjpa.event.TransactionListener and org.apache.openjpa.event.LifecycleListener (and associated superinterfaces). -Patrick ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.
RE: Getting all the ClassMetaDatas
Is there any reason why you need to eagerly get information about classes to process? In general, as you've noticed, OpenJPA does allow dynamic registration of persistent types. One possibility would be to declare that in order to use Compass searching with OpenJPA, one must provide a static list of classes (or tell OpenJPA to compute a static list of classes), using one of the options that Marc pointed out earlier. Alternately, you could potentially just register the right type of listener with OpenJPA and do whatever initialization is necessary lazily as new classes are encountered via the callbacks. -Patrick -- Patrick Linskey BEA Systems, Inc. ___ Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. -Original Message- From: Shay Banon [mailto:[EMAIL PROTECTED] Sent: Monday, January 01, 2007 1:11 PM To: open-jpa-dev@incubator.apache.org Subject: Getting all the ClassMetaDatas Hi, First, I hope that this is the correct forum for posting questions, so sorry if it isn't. I have an external list of classes that I would like to match against the persistent classes that are defined/identified by OpenJPA. I would really like to get the ClassMetaData for each one, since it has a lot of information that I could use. This intersection happens after the EntityManagerFactory has been created. I have tried using:ClassMetaData[] classMetaDatas = emf.getConfiguration().getMetaDataRepositoryInstance().getMetaDatas(); But it seems like the meta data repository and ClassMetaData information are lazily loaded (i.e. when some operation is performed on a Class, the relevant meta data is fetched if not found in cache). So, what I get is an empty array (even though I can see the OpenJPA identified the classes). I wonder how I would be able to get all the class meta data? Something that I was thinking about is since I have the list of classes that I would like to check if they are persistent, I could call: getMetaData(Class cls, ClassLoader envLoader, boolean mustExist), with Thread context class loader and false in mustExists. I am guessing that it will load the ClassMetaData if not found. My main problem here is that OpenJPA might be configured with a different class loader (though it defaults to the thread context one). Any suggestions? p.s. I am the author of Compass, so once I have this nailed down, we will have Search capabilities to OpenJPA ;) -- View this message in context: http://www.nabble.com/Getting-all-the-ClassMetaDatas-tf2905426 .html#a8116958 Sent from the open-jpa-dev mailing list archive at Nabble.com.
RE: Getting all the ClassMetaDatas
BTW, I should probably eludicate a bit. When working with
LifecycleEvents, you can get the instance associated with the event, and
from that you can get the corresponding StateManager and ClassMetaData:
Broker broker = (Broker) ((PersistenceCapable) event.getSource())
.pcGetGenericContext();
StateManager sm = broker.getStateManager(event.getSource());
ClassMetaData meta = sm.getMetaData();
With TransactionEvents, you can do:
Broker broker = (Broker) event.getSource();
for (Object o : event.getTransactionalObjects()) {
StateManager sm = broker.getStateManager(o);
ClassMetaData meta = sm.getMetaData();
if (!sm.isDirty())
continue;
// update indexes
}
Also, IMO, searching is something that shouldn't depend on the JPA spec
-- ideally, you should be able to implement all this without any
dependencies on the org.apache.openjpa.persistence package or
sub-packages.
-Patrick
--
Patrick Linskey
BEA Systems, Inc.
___
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
-Original Message-
From: Patrick Linskey
Sent: Monday, January 01, 2007 5:51 PM
To: open-jpa-dev@incubator.apache.org
Subject: RE: Getting all the ClassMetaDatas
-Original Message-
From: Shay Banon [mailto:[EMAIL PROTECTED]
Sent: Monday, January 01, 2007 1:11 PM
To: open-jpa-dev@incubator.apache.org
Subject: Getting all the ClassMetaDatas
...
p.s.
I am the author of Compass, so once I have this nailed down,
we will have
Search capabilities to OpenJPA ;)
Cool! FYI, you'll probably be interested in
org.apache.openjpa.event.TransactionListener and
org.apache.openjpa.event.LifecycleListener (and associated
superinterfaces).
-Patrick
__
_
Notice: This email message, together with any attachments,
may contain
information of BEA Systems, Inc., its subsidiaries and
affiliated
entities, that may be confidential, proprietary,
copyrighted and/or
legally privileged, and is intended solely for the use of the
individual
or entity named in this message. If you are not the intended
recipient,
and have received this message in error, please immediately
return this
by email and then delete it.
