Re: [hibernate-dev] [infinispan-dev] [jboss-as7-dev] AS7-4007 missing Infinispan dependency for clustered JPA second level cache
On 6 Mar 2012, at 14:45, Scott Marlow wrote: - Medium term: Have a way to pass in marshalling configurations per cache manager and per-cache (or an abstraction of it), which allows the right class resolver to be passed in. (***) https://issues.jboss.org/browse/ISPN-1367 ISPN-1367 is targeted to Infinispan 5.2, any idea of the target release date for that? I'm curious as to which AS release it might align with. Definitely post EAP 6. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Hibernate OGM URL
Wahey! On 12 Oct 2011, at 06:19, Emmanuel Bernard wrote: http://ogm.hibernate.org is now live. It's a redirect to the more complicated http://www.hibernate.org/subprojects/ogm That should make it easier to mention in your slides. ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Hibernate OGM documentation
Emmanuel, Some initial thoughts: * Section 1: replace NoSQL Front-End with NoSQL Back-end? :-) * Why is this version 3.0? That's pretty arbitrary. ;) Hibernate core is in v. 4.0, Infinispan is 5.0... maybe this should be a 1.0? Or even 0.1? * Page 5: Why do you need a @TableGenerator? Is this to demonstrate compatibility with Hibernate Core annotations? A table generator shouldn't be required, right? * I believe the JTA exclusions are fixed in JBoss TS 4.15.0.Final. Will make the JTA bits look a lot less scary. * Maybe highlight the lines in persistence.xml that are specific to OGM? I don't think DocBook will allow you to highlight a line in a code snippet or make it bold or something, but perhaps an XML comment before the relevant bits? Something to the effect of !-- This is the only bit that is different from your usual persistence.xml -- * Good diagrams! Looking at the storage format diagrammatically, it seems as though what we store is a combination of actual data and indexes (in the conceptual sense) as well. I.e., you have the ability to look up relationships directly, without doing a table scan. Nice. So the drawback (duplicates data) isn't really a drawback: an RDBMS would do this too, except it would call it an index. :) * What are the implications of this 'soft schema' approach? Cheers Manik On 22 Apr 2011, at 17:43, Emmanuel Bernard wrote: Hi all I've been working on Hibernate OGM documentation. I am still very unhappy with what I have but that's a start. You can read it at http://docs.jboss.org/hibernate/ogm/3.0/reference/en-US/html_single/ You can contribute to it by forking on github https://github.com/hibernate/hibernate-ogm To build the documentation, - go to hibernate-ogm-doucumentation/manual - run mvn install -DbuildDocs=true You will get the result in target/docbook/publish/en-US I'm interested in all kind of feedback: - general feel - what part is confusing - what you think is missing (besides the TODOs) And of course if you can contribute some part, that would be awesome. Emmanuel ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Hibernate OGM documentation
Sorry for not providing any feedback on this as yet - I have an imminent transatlantic flight and I intend to catch up on all this reading then. :) On 22 Apr 2011, at 17:43, Emmanuel Bernard wrote: Hi all I've been working on Hibernate OGM documentation. I am still very unhappy with what I have but that's a start. You can read it at http://docs.jboss.org/hibernate/ogm/3.0/reference/en-US/html_single/ You can contribute to it by forking on github https://github.com/hibernate/hibernate-ogm To build the documentation, - go to hibernate-ogm-doucumentation/manual - run mvn install -DbuildDocs=true You will get the result in target/docbook/publish/en-US I'm interested in all kind of feedback: - general feel - what part is confusing - what you think is missing (besides the TODOs) And of course if you can contribute some part, that would be awesome. Emmanuel ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Querying Infinispan -- Not fetching results
Hi Vidya
Please post on the Infinispan user forum rather than the dev mail list.
Cheers
Manik
On 15 Feb 2010, at 13:19, Vidya Sethuraman wrote:
Hi,
I am able to load data in cache and display. When I try to query, I am
getting an empty list.
Following is the code snippet. Please let me know what I am doing wrong.
-TestRecord.java
@ProvidedId(bridge = @FieldBridge(impl = StringBridge.class))
@Indexed(index = testRecord)
public class TestRecord implements Serializable {
//@Field(store = Store.YES)
private Date today = new java.util.Date();
//@Field(store = Store.YES)
private Timestamp time = new Timestamp(today.getTime());
@Field(store = Store.YES)
private String type;
@Field(store = Store.YES)
private String eventType;
@Field(store = Store.YES)
private int loggingLevel;
@Field(store = Store.YES)
private String message;
//@Field(store = Store.YES)
private String logTime = time.toString();
...// getters and setters
}
-TestQuery.java--
cache = cacheMgr.getCache();
QueryHelper qh = new QueryHelper(cache, new Properties(), TestRecord.class);
QueryFactory qf = new QueryFactory(cache, qh);
CacheQuery cq = qf.getBasicQuery(message, searchText);
auditLogRecordList = cq.list();
int hits = cq.getResultSize();
log.info(hits= + hits);
The hits is appearing empty.
Regards,
Vidya
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
--
Manik Surtani
ma...@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Hibernate Search 3.3.0
Hi guys Do you have an ETA on Hibernate Search 3.3.0, even early betas, etc? I'm particularly keen on HSEARCH-397 (to be able to finalise ISPN-194). Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Trunk failing because of infinispan tests
Apologies for that, that was a bad snapshot that did not completely upload. I'll push another one up today. On 1 Dec 2009, at 15:11, Brian Stansberry wrote: I'm forwarding this to the infinispan-dev list so the Infinispan guys can fix it. Pinging them on IM as well. On 12/01/2009 05:55 AM, Juraci Paixao Krohling wrote: Guys, As I know there are some Infinispan guys on this list, I'm sending it here. It seems that Infinispan maven repo used to contain some -tests.jar artifacts, but they weren't published on Nov 30, causing Hibernate trunk build to fail (look at cache-infinispan/pom.xml). Question is: should we remove the dependency from cache-infinispan/pom.xml or will Infinispan guys resume publishing the -tests.jar into their repository? Path to dependency: 1) org.hibernate:hibernate-infinispan:jar:3.5.0-SNAPSHOT 2) org.infinispan:infinispan-core:test-jar:tests:4.0.0-SNAPSHOT It tried to download this: http://snapshots.jboss.org/maven2/org/infinispan/infinispan-core/4.0.0-SNAPSHOT/infinispan-core-4.0.0-20091130.165452-15-tests.jar - Juca. ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Brian Stansberry Lead, AS Clustering JBoss by Red Hat ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Exposing transient/mortality information externally
On 30 Nov 2009, at 21:49, Brian Stansberry wrote: Be cautious about exposing cache contents (even just keys) via JMX. If you do, make it something that people can turn off, without turning off the basic JMX interface. Reason is the same as in recent discussions about providing access to the cache itself via JMX. People who deploy a JMX-enabled cache are not necessarily expecting that any code that can access the MBean server can also inspect cache contents. For example, a simple session id string is a likely key, and session ids are essentially a security token that shouldn't be exposed willy-nilly. In general, for the Hibernate 2LC use case, IMHO it's better if the Hibernate statistics API is improved, rather than having people access the cache directly. +100 How the data is stored in the cache should be an internal implementation detail that can be changed when needed. That's part of why your addition of eviction configuration to Hibernate Configuration properties was so nice. All that said, if information like [Person#1:[created:123456,lifespan:12,maxIdle:6,lasUsed:13456], Person#2:[created:234567,lifespan:12,maxIdle:6,lasUsed:24567], ...] is readily available internally, having a mechanism for authorized users to get at it sounds pretty cool. :-) Cool, but still though, that should be a Hibernate Statistics API. On 11/26/2009 03:16 AM, Galder Zamarreno wrote: Hi, Re: http://www.jboss.org/index.html?module=bbop=viewtopicp=4267469#4267469 I think Brian has a good point here. We don't expose any internal information wrt each cache entry's expiry/eviction values. Brian has a good point that this could guide him in finding out which entries have been last been used, how long they've been in memory and how it could tweak expiration/eviction accordingly. If we don't do anything, I think we run the risk of people being forced to get hold of the container, looping through it and getting information that they need from inspecting internal classes. So, I'd suggest that we add a JMX method to CacheDelegate called something like: MapString, MapString, long getTransientAndMortalityInformation(). (I'm open to suggestions to other names. This is the 1st thing that came to my mind) This would return a Map where the key is the toString form of the cache key and the value part is a map where individual transient/mortal properties are returned. I.e. We could event add calculated properties such as 'age' which is current - created. This would vary with each call obviously. As indicated in the forum entry, at least based on this use case, I don't see an immediate need to query this type of information given a key, although could be potentially useful for other use cases. The reason this would not help much right now is because it is Hibernate that creates the keys of 2nd level cache (i.e. CacheKey) and these are nor meant to be recreated externally, so it'd be hard for external apps to get something out of the Infinispan cache directly without going through the Hibernate integration layer. My suggested JMX method could allow for individual transient/mortality information to be queried by tools, or other client jmx code. Maybe some tools could use that to create a table or something like that which could be ordered based on a column? i.e. age. Also, tools or client jmx code could convert those longs into whatever they want: seconds, minutes...etc The reason I think JMX is a good candidate here is this is inherently monitoring information and it allows us to expose internal information via primitives without having to expose internal classes. Thoughts? Cheers, -- Brian Stansberry Lead, AS Clustering JBoss by Red Hat ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Exposing transient/mortality information externally
On 26 Nov 2009, at 09:16, Galder Zamarreno wrote: Hi, Re: http://www.jboss.org/index.html?module=bbop=viewtopicp=4267469#4267469 I think Brian has a good point here. We don't expose any internal information wrt each cache entry's expiry/eviction values. Brian has a good point that this could guide him in finding out which entries have been last been used, how long they've been in memory and how it could tweak expiration/eviction accordingly. If we don't do anything, I think we run the risk of people being forced to get hold of the container, looping through it and getting information that they need from inspecting internal classes. So, I'd suggest that we add a JMX method to CacheDelegate called something like: MapString, MapString, long getTransientAndMortalityInformation(). (I'm open to suggestions to other names. This is the 1st thing that came to my mind) This would return a Map where the key is the toString form of the cache key and the value part is a map where individual transient/mortal properties are returned. I.e. [Person#1:[created:123456,lifespan:12,maxIdle:6,lasUsed:13456], Person#2:[created:234567,lifespan:12,maxIdle:6,lasUsed:24567], ...] We could event add calculated properties such as 'age' which is current - created. This would vary with each call obviously. Surely that would be hugely expensive? To iterate through the entire collection? And what when you have JOPR polling this value every 5 seconds or something? ;) As indicated in the forum entry, at least based on this use case, I don't see an immediate need to query this type of information given a key, although could be potentially useful for other use cases. The reason this would not help much right now is because it is Hibernate that creates the keys of 2nd level cache (i.e. CacheKey) and these are nor meant to be recreated externally, so it'd be hard for external apps to get something out of the Infinispan cache directly without going through the Hibernate integration layer. My suggested JMX method could allow for individual transient/mortality information to be queried by tools, or other client jmx code. Maybe some tools could use that to create a table or something like that which could be ordered based on a column? i.e. age. Also, tools or client jmx code could convert those longs into whatever they want: seconds, minutes...etc The reason I think JMX is a good candidate here is this is inherently monitoring information and it allows us to expose internal information via primitives without having to expose internal classes. Thoughts? Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] atomic operations for Lucene's LockManager on Infinispan
On 22 Oct 2009, at 16:01, Mircea Markus wrote: On Oct 21, 2009, at 9:50 PM, Sanne Grinovero wrote: Hello, I've spoken with Manik in IRC about this, so wanted to share this, especially because he mentioned to ask someone to help me. I've applied the patch and reproduced the failure. Looking into it right now. To confirm, you only have this failure when using the cache in a clustered mode? I.e., using it in LOCAL mode works fine? Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Resetting Lucene lock at Directory initialization
On 16 Oct 2009, at 15:50, Sanne Grinovero wrote: Hello, Lucene does - in default LockManager implementation - a sort of lock cleanup at index creation: if it detects a lock on the index at startup, this is cleared. Łukasz translated the exact same semantic on the Infinispan Directory; current implementation inserts a lock marker at a conventional key, like Infinispan was a filesystem. So what is done in this case is to just delete the value from this key, if any, at startup (for precision: at lockFactory.clearLock()). But in this situation I would need to steal the lock from another node, if it exists. IMHO this Lucene approach is not considering concurrent initializations of the FSDirectory. So my doubts: 1) Is there some API in Infinispan capable to invalidate an existing Lock on a key, in case another node is still holding it (and will I have the other node failing?) 2) Does it make sense at all? looks like a bad practice to steal stuff. I am considering to write this lock using SKIP_CACHE_STORE, in which case I could assume that if there exists one, there's a good reason to not delete the lock as other nodes are running and using the index. In case all nodes are going down, the lock doesn't exists as it wasn't stored. So my proposal is to do a no-op on lockFactory.clearLock(), and use SKIP_CACHE_STORE when the lock is created. When an IndexWriter re-creates an index (basically making an existing one empty) it first uses clearLock(), then it tries to acquire one, so it looks like it should be safe. WDYT? this concept of SKIP_CACHE_STORE is totally new for me, maybe I just misunderstood the usage. You could use SKIP_CACHE_STORE, but that only means the lock marker won't be loaded off disk or some other cache store. It could still be loaded from a neighbouring node. -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Resetting Lucene lock at Directory initialization
I would think you need a separate tx for the lifecycle of the lock.
On 19 Oct 2009, at 12:22, Sanne Grinovero wrote:
Sorry I'll try to explain myself better, I think there's some
confusion about what my problem is.
Javadoc for LockFactory.clearLock - which the interface of what we
have to implement - is about an explicit force-cleanup:
/**
* Attempt to clear (forcefully unlock and remove) the
* specified lock. Only call this at a time when you are
* certain this lock is no longer in use.
* @param name name of the lock to be cleared.
*/
public void clearLock(String name) throws IOException {
So yes I would agree in avoiding all automagics here and just remove
the lock, but when IndexWriter opens the Directory in create mode
it does:
[...]
if (create) {
// Clear the write lock in case it's leftover:
directory.clearLock(WRITE_LOCK_NAME);
}
Lock writeLock = directory.makeLock(WRITE_LOCK_NAME);
if (!writeLock.obtain(writeLockTimeout)) // obtain write lock
throw new LockObtainFailedException(Index locked for write: +
writeLock);
this.writeLock = writeLock; // save it
[...]
basically stealing the lock ownership from existing running
processes, if any, and then by using the stolen lock apply changes to
the index.
Apparently this was working fine in Lucene's Filesystem-based
Directory, but this would fail on Infinispan as we are using
transactions: concurrent access is guaranteed to happen on the same
keys as the lock which is being ignored was meant to prevent this. And
I'm happy for this to be illegal as the result would really be
unpredictable :-)
My understanding of the IndexWriter's code is that it uses this
clearLock to make sure it's able to start even after a previous crash,
so I'd like to implement the same functionality but need to detect if
the left-over lock is really a left over and not a working lock from
another node / IndexWriter instance. If the index is live it's fine
for this IndexWriter to re-create it (making it empty) but it still
should coordinate nicely with the other nodes.
IMHO the IndexWriter wanting to do a cleanup should block until it
properly gets the lock; as we get an eager lock on
writeLock.obtain(writeLockTimeout) it means my implementation of
clearLock() could be no-op, provided we can guarantee to make a
difference from a crash-leftover lock and an in-use lock.
Manik you're idea is very interesting, but this lock is not shared:
just one owner. I could store the single lock owner as you suggest, or
is there some simpler way for this one-owner case? I understood that I
can't use Infinispan's eager lock as this ownership is spanning
multiple transactions; am I right on this? It would be quite useful if
I could keep owning the lock even after the transaction commit, or
have a separate TX running for the lock lifecycle, like a LockManager
transaction, also because I expect Infinispan to clear this locks
automatically in case the lock owner crashed/disconnects.
thanks for all comments,
Sanne
2009/10/19 Manik Surtani ma...@jboss.org:
On 19 Oct 2009, at 08:16, Emmanuel Bernard wrote:
On the Lucene side, it seems to me that manually asking for a lock
clear is cleaner / safer than this automagic approach.
Yeah, I agree with Emmanuel - a more explicit form would work
better IMO.
Perhaps what you could do is something like this:
1) Create an entry, name sharedlock, value address of current
lock
owner.
2) Any time a node needs a lock, adds its addess to the
sharedlock entry
only if it doesn't exist (putIfAbsent)
3) If the entry exists , check if the address is still in the
cluster
(check using CacheManager.getMembers()). If the address doesn't
exist
(stale lock) remove and overwrite (using replace() to prevent
concurrent
overwrites)
WDYT?
- Manik
--
Manik Surtani
ma...@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
--
Manik Surtani
ma...@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Continuous Query Caching
On 29 Sep 2009, at 09:57, Mircea Markus wrote: Hi, Again, this is a feature from Coherence[1]. Basic idea is to execute a query against the cache, and hold the result object. This result object will always have up to date query result; this means that whenever something is modified in the cache the result itself is updated. Advantage: if one performs the same query very often(e.g. several times every millisecond) the response will be fast and the system will not be overloaded. Is it really faster? Surely all you save is the construction of the various query objects, but the query itself would have to be re-run every time. Or does it attach a listener to the cache and check whether any new additions/removals should be used to update the result set? I don't see how that could be much faster though. Adding Hibernate-dev in cc so that the HIbernate Search guys can comment too. E.g. Filter filter = new AndFilter(new EqualsFilter(getTrader, traderid), new EqualsFilter(getStatus, Status.OPEN)); ContinuousQueryCache cacheOpenTrades = new ContinuousQueryCache (cache, filter); Iterator iter = cacheOpenTrades.entrySet().iterator(); //*this entrySet call will be instant!* FOr a full list of scenario in which this can be used take a look at [1]. Shall we consider adding something similar? Cheers, Mircea [1] http://download.oracle.com/docs/cd/E14526_01/coh.350/e14509/continuousquery.htm#BABBEIAH ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Continuous Query Caching
On 29 Sep 2009, at 10:19, Mircea Markus wrote: On Sep 29, 2009, at 12:08 PM, Manik Surtani wrote: On 29 Sep 2009, at 09:57, Mircea Markus wrote: Hi, Again, this is a feature from Coherence[1]. Basic idea is to execute a query against the cache, and hold the result object. This result object will always have up to date query result; this means that whenever something is modified in the cache the result itself is updated. Advantage: if one performs the same query very often(e.g. several times every millisecond) the response will be fast and the system will not be overloaded. Is it really faster? Surely all you save is the construction of the various query objects, but the query itself would have to be re- run every time. Or does it attach a listener to the cache and check whether any new additions/removals should be used to update the result set? this is the way it works. It is a sort of a near-cache, just that instead of being invalidated it is updated whenever the cache is updated. The documentation also suggests that they are using listeners. I don't see how that could be much faster though. I think it might be if the you are running *the same query* tons of times. Basically you don't do a map-reduce on all the nodes, but rather on every insertion (especially if the number of insertion is relative small compared to the number of same-query-bring-run) you updated (if necessary) the cached query result. Hmm. It would be pretty use-case-specific. It's hard to see how this _generally_ performs better, since you need to make sure you are aware of all changes happening all over the cluster to keep this result set up to date (REPL-style scalability bottleneck!) Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 27 Sep 2009, at 11:26, Sanne Grinovero wrote: Next version of Lucene will provide helpers and tools to make it easy to create your own QueryParser, so that everyone can make his one based on business-specific needs (Everybody can already, but is not as easy). So I would avoid reinventing that, and focus on a good API. To make this API very cool IMHO it should integrate with Hibernate Search to exploit all knowledge about object mapping, declarative Analyzer and Filter definitions, and so on... We concluded in another thread that to make best use of this information we need to give the type of what you're searching for as a parameter, so that from the specific mapping the analyzers, fields and fieldbridges can be matched. Should be fine, it means we can provide typesafe results? Are you suggesting that this is made to be specific to Hibernate Search, rather than more generic, for Lucene? As far as Infinispan is concerned I couldn't care either way since we just depend on HS. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 28 Sep 2009, at 10:08, Emmanuel Bernard wrote: On 28 sept. 09, at 10:48, Manik Surtani wrote: On 27 Sep 2009, at 11:26, Sanne Grinovero wrote: Next version of Lucene will provide helpers and tools to make it easy to create your own QueryParser, so that everyone can make his one based on business-specific needs (Everybody can already, but is not as easy). So I would avoid reinventing that, and focus on a good API. To make this API very cool IMHO it should integrate with Hibernate Search to exploit all knowledge about object mapping, declarative Analyzer and Filter definitions, and so on... We concluded in another thread that to make best use of this information we need to give the type of what you're searching for as a parameter, so that from the specific mapping the analyzers, fields and fieldbridges can be matched. Should be fine, it means we can provide typesafe results? Are you suggesting that this is made to be specific to Hibernate Search, rather than more generic, for Lucene? As far as Infinispan is concerned I couldn't care either way since we just depend on HS. Yes, if you check the thread history we've discussed the benefits of tying the DSL to Hibernate Search's metadata. We could basically save a lot of pain points like analyzers and type conversion for users (and we can't if we stay pure Lucene based). Yup, makes sense. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
Lexers are parsers are not difficult (a simple ANTLR grammar takes very little effort, for example), but that is orthoginal to this discussion. I believe the very term DSL is misleading (since DSL implies a separate grammar), but this is more for an intuitive builder API for query creation. And yes, while this does imply knowledge of the API, it is certainly far less verbose and more expressive than the existing query construction mechanism. Perhaps as a next step, a grammar can be built on top of this. - Manik On 25 Sep 2009, at 18:41, Navin Surtani wrote: Incase this email didn't go to the full dev-list. I got it as a separate thread so forwarding on. Begin forwarded message: From: johng@gmail.com Date: 25 September 2009 16:00:53 BST To: Navin Surtani nsurt...@redhat.com Subject: Re: Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new All, I think Hardy's original push back came from the first pass' use of the decorator pattern to try to come up with a DSL. That really isn't much better than knowing the API. The alternate is to come up with a more natural language implementation but that leads to parsers, lexers, etc... I'm not saying it's not worth while but it may be a lot of work. John Griffin On Sep 25, 2009 8:12am, Navin Surtani nsurt...@redhat.com wrote: Just wanted to get this topic re-started again. Essentially what I think this project/DSL/module/thingy-bob is thought to become: - A simple package where a user can build Lucene queries without having to know too much about Lucene itself. If I'm headed down the wrong thought path then just thwack me. On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote: On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote: I've been thinking about a DSL to build Lucene queries in the last day. What do you think of this proposal? What do you really gain compared to native Lucene queries? What's gained I believe is the fact that people can build complex lucene queries easier. Currently, it's a bit clunky imo so if we provide a cleaner way to build them it can prove beneficial to any lucene user (myself included for querying on Infinispan). Any other thoughts? If your API achieves exactly the same as what's possible with Lucene it is just a 'useless' wrapper. A wrapper around native Lucene queries would make sense if it could somehow use some of the Hibernate Search specific meta data. As an extreme example one could generate some meta classes a la JPA2. This way one could ensure that you can get help with which field names are available. --Hardy ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev Navin Surtani Intern Infinispan Intern JBoss Cache Searchable ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev Navin Surtani Intern Infinispan Intern JBoss Cache Searchable ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Hibernate 3.5.0.Beta-1 released
Try typing hibernate default cache provider into Google. ;-) Note that the (official?) docs [1] state: Note that versions prior to 3.2 use EhCache as the default cache provider. So perhaps most of the hits refer to old versions of Hibernate. And if the reason for removing a default altogether is to pass a TCK and multiple inits, we can have a chat about this to see how we can make this work (Infinispan has a concept of a default cache, which just works and is always available, if that helps with multiple inits). All the same, it would be good to have some sort of preference expressed in the docs even if you don't want to ship a default, but that's up to you guys. Cheers Manik [1] http://docs.jboss.org/hibernate/stable/core/reference/en-US/html/performance.html#performance-cache On 9 Sep 2009, at 18:04, Emmanuel Bernard wrote: Let me say it for the third time (complementing Steve's first two). There is no default cache provider, a user has to chose one. One of the reasons for this move was that the JPA TCK was having issues with cache systems and multiple initializations. On 9 sept. 09, at 18:58, Steve Ebersole wrote: Again, there is no default cache provider. Users must decide which to use. On Wed, 2009-09-09 at 11:42 -0500, Sanne Grinovero wrote: I'd like to see Infinispan as default cache provider, AFAIK ehcache is the default now. IMHO the default is to some extend a recommendation to the users, at least that's how I perceived it when I was new to hibernate: if they have chosen it, it must be good. It would need however to be able to work at least in single-vm mode without configuration, to keep it as simple as possible for beginners. 2009/9/9 Galder Zamarreno galder.zamarr...@redhat.com: Steve, taking in account that at least in the community version you ship a few different cache providers (correct me if I'm wrong), what about putting something in the Hibernate docs where you recommend using Infinispan? i.e. Infinispan is the recommended 2LC provider On 08/31/2009 06:02 PM, Steve Ebersole wrote: We do not make any cache integration the default. There are too many variables that have to come in to play... On Mon, 2009-08-31 at 17:55 +0200, Galder Zamarreno wrote: Bugger, this was released hours before I committed the Infinispan cache provider :( Steve, let's catch up before next 3.5 release in order to figure out what's needed to make Infinispan the default cache provider. Do you make such decision based on any performance test or the configuration itself? My aim is to make Infinispan the best 2nd level cache provider out there. On 08/21/2009 06:43 PM, Steve Ebersole wrote: Hibernate 3.5.0.Beta-1 has been released. Please see http://in.relation.to/12153.lace for details. -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Steve Ebersole st...@hibernate.org Hibernate.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Benchmarking 2nd level cache providers?
Do we have a JIRA for this? Are we tracking this somewhere? On 2 Sep 2009, at 14:25, Steve Ebersole wrote: The best bet for something like this is a separate project which pulls in the proper hibernate dependencies. Then we can talk with Juca about getting them set up on a scheduled run. However, if you want to have the configuration varied for different runs, I suggest you wait until after the discussions with the Maven developers to address the numerous specific issues we have encountered with Maven. This is exactly one of the issues on the docket. We have issues with this varied config piece in the core/testsuite module. On Wed, 2009-09-02 at 06:20 -0500, Sanne Grinovero wrote: Totally agree with this, it would be useful also to benchmark other parts of Hibernate (not just the cache) and as general regression test. Also with Hibernate Search I have some private tests which are very time consuming (an hour or so) which I can't commit on trunk as unit tests; I'd like to have them somewhere, maybe regularly executed by Continuous Integration (too much work for existing Hudson?). Sanne 2009/9/1 Galder Zamarreno galder.zamarr...@redhat.com: On 08/26/2009 05:48 PM, Mircea Markus wrote: Can't we write a plugin for the CacheBenchmark fwk? We could potentially do so but we'd be missing loads of things and we'd have to spend a fair bit of time replicating the way Hibernate uses the cache which is not straightforward by any means. Take in account as well that there're several caches and cache usages involved here: entity, collection, query and timestamps and each has a different usage pattern. Bottom line: I don't think trying to emulate this is worth the effort. What is needed here is a Hibernate/JPA load/perf test environment where the most common Hibernate usage patterns are exercised and then you swap 2nd level cache providers. This is really what counts to the users: how quickly I can persist N entities, how quick I can read N times an entity that I've just persisted, how fast I can execute same query N times...etc, these are the use cases where the cache provider must show off. I haven't heard anything from the HB team wrt to this, so I assume they don't have anything like this. On Aug 19, 2009, at 1:14 PM, Galder Zamarreno wrote: While talking to Manik online, the topic of 2nd level cache benchmarking came up and was wondering if there's a way to benchmark different 2nd level cache providers in Hibernate? Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Steve Ebersole st...@hibernate.org Hibernate.org ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Feedback on Infinispan patch
://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev InfinispanDirectoryProvider_22_09_2009 .patch___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
Very elegant. I'm generally a big fan of 'builder' patterns like
this, but this really isn't a DSL, is it? :) When you first
mentioned a DSL I had visions of defining a new grammar and an ANTLR
parser, etc. But that is overkill.
This approach certainly works, and will almost certainly perform
better too. One question: for the sake of brevity, why
SealedQueryBuilder instead of QueryBuilder ? :)
Also, I still think that if this is a generic helper factory that
helps you build Lucene queries - and has no knowledge of how and where
the query is used (why should it?) - then this should be something
people can use outside of HS or Infinispan. E.g., directly with Lucene.
On 26 Aug 2009, at 12:39, Emmanuel Bernard wrote:
I've been thinking about a DSL to build Lucene queries in the last
day.
What do you think of this proposal?
A few remarks:
- it asks the analyzer so that we correctly apply the analyzer on
terms
- it has a few query factory methods
- it contains a few orthogonal operations
- I am not quite satisfied with how boolean is handled, any idea?
Examples
SealedQueryBuilder qb =
searchFactory.withEntityAnalyzer(Address.class);
Query luceneQuery =
qb.must(Occurs.MUST)
.add(
qb.boolean(Occurs.Should)
.add( qb.term(city,
Atlanta).boostedTo(4).createQuery() )
.add( qb.term(address1,
Peachtree).fuzzy().createQuery() )
)
.add(
qb.from(movingDate,
200604).to(201201).exclusive().createQuery()
)
.createQuery();
Analyzer choice
queryBuilder.withAnalyzer(Analyzer)
queryBuilder.withEntityAnalyzer(Class?)
queryBuilder.basedOnEntityAnalyzer(Class?)
.overridesForField(String field, Analyzer)
.overridesForField(String field, Analyzer)
.build() //sucky name
returns a SealedQueryBuilder //sucky name
SealedQueryBuilder contains the factory methods
Factory methods
Hosted onSealedQueryBuilder
.term(String field, String text) //define a new query
.term(String field, String text) //define a new query
.ignoreAnalyzer() //ignore the analyzer, optional
.fuzzy() //API prevent wildcard calls, optional
.threshold() //optional
.prefixLengh() //optional
.term(String field, String value)
.wildcard() //API prevent fuzzy calls, optional
//range query
.from(String field, String text)
.exclusive() //optional
.to(String text)
.exclusive() //optional
.constantScore() //optional, due to constantScoreRangeQuery but
in practice inherited from the common operations
//match all docs
.all()
//phrase query
.phrase(String field)
.ignoreAnalyzer() //ignore the analyzer, optional
.addWord(String text) //at least one
.addWord(String text)
.sentence(String text) //do we need that?
.slop() //optional
//search multiple fields for same value
.searchInMultipleFields()
.onField(String field)
.boostedTo(float) //optional
.ignoreAnalyzer() //optional
.onField(String field)
.forWords(String) //do we need that?
.forWord(String)
Boolean operations
SealedQueryBuilder contains the boolean methods
.boolean(Occurs occurs)
.add( qb.from().to() )
.add( ... )
Works on all queries
.boostedTo()
.constantScore()
.filter(Filter) //filter the current query
.scoreMultipliedByField(field) //FieldScoreQuery +
FunctionQuery?? //Not backed
.createQuery()
Todo
Span*Queries
MultiPhraseQuery - needs to fillup all accepted terms
FieldScoreQuery
ValueSourceQuery
FuzzyLikeThis
MoreLikeThis
On 25 août 09, at 16:43, Manik Surtani wrote:
On 25 Aug 2009, at 13:34, Emmanuel Bernard wrote:
On 25 août 09, at 14:27, Manik Surtani wrote:
A DSL would work, but I'd rather not define our own language here.
Which is why I asked for a standard. Perhaps something based on
SQL/
JPA-QL? Or are you thinking DSL specific to Lucene - which could
be used by any/all of {Lucene, Hibernate Search, Infinispan}? In
which case the DSL should ideally be a Lucene project.
Yes I was thinking about a DSL used for Hibernate Search and maybe
all
of Lucene if the HS integration benefits offer no value towards
simplicity (but I think i can offer value).
Ok, this should be interesting. Lets chat about this some more -
have
you drafted any thoughts around this DSL somewhere?
___
infinispan-dev mailing list
infinispan-...@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
ma...@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 27 Aug 2009, at 13:06, Emmanuel Bernard wrote: On 27 août 09, at 13:07, Manik Surtani wrote: Very elegant. I'm generally a big fan of 'builder' patterns like this, but this really isn't a DSL, is it? :) When you first mentioned a DSL I had visions of defining a new grammar and an ANTLR parser, etc. But that is overkill. This is called an internal DSL, ie you use Java as the language, not some external representation. This approach certainly works, and will almost certainly perform better too. One question: for the sake of brevity, why SealedQueryBuilder instead of QueryBuilder ? :) The name is not right yet There are two things: - the query builder that lets you define the analyzer - the query builder that has an analyzer assigned and lets you build query What name is best for each of them. I thought this stuff you mentioned made sense: queryBuilder.withAnalyzer(Analyzer) queryBuilder.withEntityAnalyzer(Class?) queryBuilder.basedOnEntityAnalyzer(Class?) .overridesForField(String field, Analyzer) .overridesForField(String field, Analyzer) .build() //sucky name Perhaps rename the static factory methods to something like: QueryBuilder.getQueryBuilder(Analyzer) QueryBuilder.getQueryBuilder(Class?) and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). Why do you need the build() method at the end? Also, I still think that if this is a generic helper factory that helps you build Lucene queries - and has no knowledge of how and where the query is used (why should it?) - then this should be something people can use outside of HS or Infinispan. E.g., directly with Lucene. As of today this code is technically pure Lucene but to be honest the idea of passing an analyzer multiplexer (like the one we receive from searchFactory.getAnalyzerClass?)) is not wildly spread in Lucene and potentially cumbersome wo the declarative approach of HSearch. The second problem is that some potential improvements will require inner knowledge of HSearch: - object parameters (and not string params) do require to know the FieldBridge of the property. This is a pure HSearch notion. - property literal like JPA is introducing could be added to replace the String-based field approach in some situations. Though I don't think that it would be a perfect fit. - spell checker (the old idea we had) That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. Not it doesn't have to be either ASL or even hosted at Apache. I guess what I am suggesting is perhaps even a separate project - LuceneQueryBuilder or something - which plain-old-Lucene users could use as well. Doesn't matter where it's hosted or what the license is - as long as its ASL or LGPL :) -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 27 Aug 2009, at 16:10, Emmanuel Bernard wrote: queryBuilder.withAnalyzer(Analyzer) queryBuilder.withEntityAnalyzer(Class?) queryBuilder.basedOnEntityAnalyzer(Class?) .overridesForField(String field, Analyzer) .overridesForField(String field, Analyzer) .build() //sucky name Perhaps rename the static factory methods to something like: QueryBuilder.getQueryBuilder(Analyzer) QueryBuilder.getQueryBuilder(Class?) and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). Why do you need the build() method at the end? if you do that, all of the sudden, a QB can change it's analyzer on the fly making it immutable. Also the overridesForField methods would pollute the API when it's time to create a query. One of the advantages of a fluent API in a strongly typed environment is that we can hide methods that are meaningless in a given context. That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. Not it doesn't have to be either ASL or even hosted at Apache. I guess what I am suggesting is perhaps even a separate project - LuceneQueryBuilder or something - which plain-old-Lucene users could use as well. Doesn't matter where it's hosted or what the license is - as long as its ASL or LGPL :) Let's start it under the Hibernate Search umbrella due to potential synergies and spin it out if needed. Ok. Just make sure we use a different maven module or something so that there are no dependencies on the rest of HS or Hibernate. Otherwise spinning out will be a PITA. Lucene should be the only dependencies of this code. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Feedback on Infinispan patch
are stored indexes for entity Movie. Indexes for all other entities are stored in default HSInfinispanCache. Emmanuel Begin forwarded message: From: Łukasz Moreń lukasz.mo...@gmail.com Date: 21 août 2009 02:11:03 HAEC To: Emmanuel Bernard emman...@hibernate.org Subject: GSoC patch with Infinispan Directory Provider I'm sending patch and piece of documentation - not much but necessary information are included. There are some todos but I didn't manage to finish it yet. I changed maven jgroups dependency to 2.8.beta2, before version was clashed with used by infinispan. In pom file there was dependency on hibernate common annotations 3.2.shapshot. It should't be 3.5? Cheers, Lukasz GSoC2009_summary.pdf___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] COMMITTED tx status handling changed from JBC to ISPN
Wahey! We're on our way. We should blog about this. On 19 Aug 2009, at 08:32, Galder Zamarreno wrote: A FYI: Integrated with HB trunk and test now passes without any probs. Many thanks Steve! On 08/18/2009 10:57 PM, Manik Surtani wrote: Great! On 18 Aug 2009, at 20:40, Galder Zamarreno wrote: Right, so basically it's time for me to integrate this into HB trunk. I'll start with that work asap. On 08/18/2009 08:36 PM, Steve Ebersole wrote: To clarify further... Within txn here there are calls to: 1) cacheAccess.lockRegion() 2) cacheAccess.removeAll() In after-completion phase there is a call to: 3) cacheAccess.unlockRegion(lock-from-#1) A transactional cache would not care about #1 nor #3... On Tue, 2009-08-18 at 13:25 -0500, Steve Ebersole wrote: On Tue, 2009-08-18 at 16:50 +0200, Galder Zamarreno wrote: This change of behaivour is making Infinispan cache provider tests that do bulk modifications to fail. The reason it fails is because Hibernate has a javax.transaction.Synchronization implementation called CacheSynchronization that in it's afterCompletion(), it leads to call BulkOperationCleanupAction.evictEntityRegions() which clears the cache for the affected entities. Now, since the tx status is COMMITTED, the test fails. This is no longer accurate. There was a bug in how BulkOperationCleanupAction worked because it was still using the older Hibrnate cache SPIs. See http://opensource.atlassian.com/projects/hibernate/browse/HHH-4034 -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] COMMITTED tx status handling changed from JBC to ISPN
On 18 Aug 2009, at 15:50, Galder Zamarreno wrote: Hi all, More stuff related to the infinispan cache provider. The way we deal with transactions that are not ACTIVE or PREPARING has changed from JBoss Cache to Infinispan. In JBoss Cache, TransactionTable.getCurrentTransaction() logged a message if the transaction's status was committed whereas Infinispan simply throws an IllegalStateException if the status is neither ACTIVE nor PREPARING. This change of behaivour is making Infinispan cache provider tests that do bulk modifications to fail. The reason it fails is because Hibernate has a javax.transaction.Synchronization implementation called CacheSynchronization that in it's afterCompletion(), it leads to call BulkOperationCleanupAction.evictEntityRegions() which clears the cache for the affected entities. Now, since the tx status is COMMITTED, the test fails. Surely though, at that stage the tx has already committed (and hence the name of the callback - afterCompletion()). If anything, operations here should happen outside the scope of the tx. Can't the cleanup call happen in beforeCompletion(), after any Hibernate resources have performed their beforeCompletion() steps? Would there be any problems in maintaining the previous logic? While I don't have any problems reverting to the older logic, I just feel that doing so hides bugs elsewhere. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] COMMITTED tx status handling changed from JBC to ISPN
On 18 Aug 2009, at 16:21, Galder Zamarreno wrote: Would there be any problems in maintaining the previous logic? While I don't have any problems reverting to the older logic, I just feel that doing so hides bugs elsewhere. I agree but seeing that the logic was present in JBC, I assumed that this might have been discussed in the past. The JBC logic here was not put in place for Hibernate. -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] COMMITTED tx status handling changed from JBC to ISPN
Great! On 18 Aug 2009, at 20:40, Galder Zamarreno wrote: Right, so basically it's time for me to integrate this into HB trunk. I'll start with that work asap. On 08/18/2009 08:36 PM, Steve Ebersole wrote: To clarify further... Within txn here there are calls to: 1) cacheAccess.lockRegion() 2) cacheAccess.removeAll() In after-completion phase there is a call to: 3) cacheAccess.unlockRegion(lock-from-#1) A transactional cache would not care about #1 nor #3... On Tue, 2009-08-18 at 13:25 -0500, Steve Ebersole wrote: On Tue, 2009-08-18 at 16:50 +0200, Galder Zamarreno wrote: This change of behaivour is making Infinispan cache provider tests that do bulk modifications to fail. The reason it fails is because Hibernate has a javax.transaction.Synchronization implementation called CacheSynchronization that in it's afterCompletion(), it leads to call BulkOperationCleanupAction.evictEntityRegions() which clears the cache for the affected entities. Now, since the tx status is COMMITTED, the test fails. This is no longer accurate. There was a bug in how BulkOperationCleanupAction worked because it was still using the older Hibrnate cache SPIs. See http://opensource.atlassian.com/projects/hibernate/browse/HHH-4034 -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Infinispan cache loaders in Hibernate Search
On 15 Aug 2009, at 00:36, Łukasz Moreń wrote: Infinispan offers several persistance stores to persist content of cache, i.e. based on file system, database, S3 amazon service. Is there some recommendation which one should be used as a default for Hibernate Search directory provider based on Infinispan cache? Depends, really. If you expect more indexes than available memory a file system approach is best. You have the FileCacheStore, or BdbjeCacheStore as well. I would not really recommend the JdbcCacheStore for this. The S3CacheStore is a good option if you are deployed on an Amazon cloud (e.g., on EC2) otherwise latency will become a problem. What about cache loaders configuration strategy: one cache store per node, one common store for all nodes, only single node allowed to write to persistance store. Using a common store only makes sense if you have a common storage location, e.g. with S3CacheStore or JdbcCacheStore. With file system approaches, this only makes sense if it is mounted on a shared volume such as NFS or a SAN, but I wouldn't recommend using NFS with any of the cache stores. So anyway, as far as defaults are concerned, I think the best is to use the FileCacheStore (no external dependencies that way), and one cache store per node (so there is no constraint on a shared volume). But it is a good idea to note down the thoughts above in a README or a wiki page somewhere. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Infinispan tx, config and multithreading
On 13 Aug 2009, at 19:11, Emmanuel Bernard wrote: Infini guys, do you support transactional operation spanning several concurrent threads? The *same* tx on different threads? That's disallowed by the JTA spec. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Infinispan tx, config and multithreading
2. if it is necessary, index writer delegates new threads to do merge work. Those merge threads do not see changes made so far from begin of transaction, and are looking for segments which are not yet in index. Changes will be visible when AD.3 is completed. For tests i tried to commit transaction when merge starts and then everything worked well. But then i need to start it again. 3. index writer releases lock - transaction is commited, all changes made in this transaction are visible for other threads. Maybe using some other transaction manager could help? What about Infinispan cache configuration? Some configuration mechanism should be exposed to the user, or we can hardcoded one in InfinispanDirectoryProvider is enough? 2009/8/12 Emmanuel Bernard emman...@hibernate.org why? Emmanuel Bernard Pending you there? Emmanuel Bernard Pending Ok please describe in details what is going on. From what you are describing the tx cannot see all segments which looks like an infinispan bug to me. Pending As a back up you can try wo transaction and see if that works Emmanuel Bernard Pending technically the lucene index should cope with that Emmanuel Bernard 11:16 but I like this approach less Let's try and chat by email IF I'm not online, I need to run on some errands today. ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Jason T. Greene JBoss, a division of Red Hat -- Jason T. Greene JBoss, a division of Red Hat ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Infinispan tx, config and multithreading
to explain it more clear. I'm using DummyTransactionManager available for Infinispan. It associates transaction with the calling thread. Steps to update index: 1. index writer acquires lock - begin of transaction 2. if it is necessary, index writer delegates new threads to do merge work. Those merge threads do not see changes made so far from begin of transaction, and are looking for segments which are not yet in index. Changes will be visible when AD.3 is completed. For tests i tried to commit transaction when merge starts and then everything worked well. But then i need to start it again. 3. index writer releases lock - transaction is commited, all changes made in this transaction are visible for other threads. Maybe using some other transaction manager could help? What about Infinispan cache configuration? Some configuration mechanism should be exposed to the user, or we can hardcoded one in InfinispanDirectoryProvider is enough? 2009/8/12 Emmanuel Bernard emman...@hibernate.org why? Emmanuel Bernard Pending you there? Emmanuel Bernard Pending Ok please describe in details what is going on. From what you are describing the tx cannot see all segments which looks like an infinispan bug to me. Pending As a back up you can try wo transaction and see if that works Emmanuel Bernard Pending technically the lucene index should cope with that Emmanuel Bernard 11:16 but I like this approach less Let's try and chat by email IF I'm not online, I need to run on some errands today. ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Jason T. Greene JBoss, a division of Red Hat -- Jason T. Greene JBoss, a division of Red Hat ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 11 Aug 2009, at 16:34, Galder Zamarreno wrote: On 08/10/2009 01:09 PM, Galder Zamarreno wrote: On 08/10/2009 12:42 PM, Manik Surtani wrote: /snip Well, I think we need both. getConfiguration(String name) to retrieve a configuration template and modify it. defineCache(String newName, String templateName, Configuration overrides) to create a new configuration based on an existing one. Maybe this name should change to something more intuitive? Perhaps instead of defineCache, we have: Configuration defineConfiguration(String configurationName, Configuration overrides) // registers and names a NEW configuration, based on the default cfg and the overrides passed in Configuration defineConfiguration(String configurationName, String templateName, Configuration overrides) // registers and names a NEW configuration, based on an existing, predefined configuration and the overrides passed in WDYT? That would be a great way to combine the two API requirements. I assume that the returned Configuration instances returned would have had the overrides applied to them already. Let me work on this APIs in parallel to the work I'm doing for ISPN-6 so that I can fully exercise/test and see if there's anything we'd need to change. https://jira.jboss.org/jira/browse/ISPN-152 A quick update on this. I have implemented ISPN-152, integrated it with the cache provider and the tests I have are now passing. Just a heads up on a change of behaviour. defineConfiguration(*) methods do no longer throw DuplicateCacheNameException since this limits the API a lot disabling any type of configuration redefininitions/overriding. Example: defineConfiguration(entity, entity, overrides); This means, take the configuration for named cache entity, apply overrides and use that as new entity cache name configuration. If we're throwing DuplicateCacheNameException(s), we wouldn't be able to do this and this seems like a valid use case to me. By the way, I've already noted this in the javadoc but doing the following: Configuration c = defineConfiguration(entity, new Configuration()); is a way to return entity named cache's configuration! (equivalent to doing something like: manager.getConfiguration(entity) if such method existed!) IOW, this defineConfiguration method is saying: define a configuration, based on entity and apply no overrides (cos no setters have been called in the Configuration instance!) and return new configuration instance I've detailed all this in CacheManager's javadoc that you can find attached. If you get the chance, have a read to it and let me know if anything is not clear enough. Most of it sounds good, but the javadoc on defineConfiguration() is a bit misleading. * p/ * If cache name hasn't been defined before, this method creates a clone of the configuration of the cache whose * name matches the given template cache name, then applies the configuration overrides passed and returns this * configuration instance. * p/ * If cache name has been previously defined, this method creates a clone of this cache's existing configuration, * applies the overrides and return this configuration instance. * p/ The first bit makes sense. The second bit though, I would expect the behaviour to be to clone the template cfg, apply changes, and overwrite the existing named config rather than use the existing named config as the template (rather than templateCacheName). Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 11 Aug 2009, at 16:59, Galder Zamarreno wrote: On 08/11/2009 05:42 PM, Manik Surtani wrote: On 11 Aug 2009, at 16:34, Galder Zamarreno wrote: On 08/10/2009 01:09 PM, Galder Zamarreno wrote: On 08/10/2009 12:42 PM, Manik Surtani wrote: /snip Well, I think we need both. getConfiguration(String name) to retrieve a configuration template and modify it. defineCache(String newName, String templateName, Configuration overrides) to create a new configuration based on an existing one. Maybe this name should change to something more intuitive? Perhaps instead of defineCache, we have: Configuration defineConfiguration(String configurationName, Configuration overrides) // registers and names a NEW configuration, based on the default cfg and the overrides passed in Configuration defineConfiguration(String configurationName, String templateName, Configuration overrides) // registers and names a NEW configuration, based on an existing, predefined configuration and the overrides passed in WDYT? That would be a great way to combine the two API requirements. I assume that the returned Configuration instances returned would have had the overrides applied to them already. Let me work on this APIs in parallel to the work I'm doing for ISPN-6 so that I can fully exercise/test and see if there's anything we'd need to change. https://jira.jboss.org/jira/browse/ISPN-152 A quick update on this. I have implemented ISPN-152, integrated it with the cache provider and the tests I have are now passing. Just a heads up on a change of behaviour. defineConfiguration(*) methods do no longer throw DuplicateCacheNameException since this limits the API a lot disabling any type of configuration redefininitions/overriding. Example: defineConfiguration(entity, entity, overrides); This means, take the configuration for named cache entity, apply overrides and use that as new entity cache name configuration. If we're throwing DuplicateCacheNameException(s), we wouldn't be able to do this and this seems like a valid use case to me. By the way, I've already noted this in the javadoc but doing the following: Configuration c = defineConfiguration(entity, new Configuration()); is a way to return entity named cache's configuration! (equivalent to doing something like: manager.getConfiguration(entity) if such method existed!) IOW, this defineConfiguration method is saying: define a configuration, based on entity and apply no overrides (cos no setters have been called in the Configuration instance!) and return new configuration instance I've detailed all this in CacheManager's javadoc that you can find attached. If you get the chance, have a read to it and let me know if anything is not clear enough. Most of it sounds good, but the javadoc on defineConfiguration() is a bit misleading. * p/ * If cache name hasn't been defined before, this method creates a clone of the configuration of the cache whose * name matches the given template cache name, then applies the configuration overrides passed and returns this * configuration instance. * p/ * If cache name has been previously defined, this method creates a clone of this cache's existing configuration, * applies the overrides and return this configuration instance. * p/ The first bit makes sense. The second bit though, I would expect the behaviour to be to clone the template cfg, apply changes, and overwrite the existing named config rather than use the existing named config as the template (rather than templateCacheName). Hmmm, right, so defineConfiguration(String, String, Configuration) should work the same way regardless of whether the configuration for the named cache exists or not, correct? In both cases, a clone a clone of the template cache's configuration would be made, apply a clone of the overrides, and overwrite existing named cache's configuration. Yup. In fact, a side thing, this has reminded my that then applies the configuration overrides passed should say then applies a clone of the configuration overrides passed in, this makes it clear that the configuration passed in cannot be modified after calling defineConfiguration() and expect its changes to be reflected in the CacheManager. +1. -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 8 Aug 2009, at 08:14, Galder Zamarreno wrote: On 08/07/2009 02:06 PM, Manik Surtani wrote: On 7 Aug 2009, at 12:51, Galder Zamarreno wrote: Manik, While I working on this, I've noted the following. If we define default cache configurations for entity/query/...etc in an infinispan configuration file, currently, they cannot be overriden afterwards programatically. IOW, imagine there's a cache named entity that applies to all entities in a sample infinispan-configs.xml and then someone tweaks it in the hibernate config like this: hibernate.cache.infinispan.entity.eviction.wake_up_interval=3000 The way I'd deal with this is that I'd create a Configuration with this new interval and call defineCache(entity, cfg). However, when the configuration file was processed, entity cache was added to the configuration overrides and hence when I call defineCache with entity, it throws DuplicateCacheNameException. So, I think DFC.initialize() shouldn't add such configurations to configuration overridesm map. Thoughts? Hmm. Perhaps what is needed is a mechanism to retrieve these configurations and alter them. E.g., CacheManager.getConfiguration(); // retrieves the default cfg CacheManager.getConfiguration(String name); Yeah, without starting the cache! Right now, the only way to get a configuration is after doing manager.getCache() which effectively starts the cache and hence, the Configuration's ComponentRegistry is in RUNNING mode, so you can't just clone it and modify it to be used somewhere else. Even the clone is in running state and didn't find an easy way to change that. There's a second possibility here too: My above suggestion won't just work cos what defineCache(entity, cfg) does is take the *default* cache configuration, apply the overrides and create one with name entity. But I want to do is: take the entity named cache configuration, apply the overrides and create a brand new one with the same name entity. So, a method like this: defineCache(String cacheName, String baseNamedCache, Configuration overrides). That would work as well. Javadocs need to be clear though as to what is cloned and ready for use, etc. So, then apart from doing defineCache(entity, entity, overrides), I could also do defineCache(com.acme.Person, entity, overrides). The latter would define a cache for com.acme.Person entity type, based on entity named cache and applying overrides defined via hibernate.cfg.xml These Configuration objects - built based on what's in the XML file - can then be tweaked. These should be adequately javadoc'd to make it clear that caches already started using such a configuration will not be affected when a configuration changes. Indeed, getConfiguration() would return clones who's running state has been cleared. Maybe Configuration.clone() should already be doing that? This should be true as well. The state should never be cloned. I guess this also renders CacheManager.defineCache() API useless/obsolete, since getConfiguration() should always return a configuration, even if one didn't exist /wasn't defined in XML. Yeah, it's either enhanced defineCache() or getConfiguration(). I think CacheManager.getConfiguration() is likely to be a more powerful and simpler API than defineCache(*). Well, I think we need both. getConfiguration(String name) to retrieve a configuration template and modify it. defineCache(String newName, String templateName, Configuration overrides) to create a new configuration based on an existing one. Maybe this name should change to something more intuitive? Perhaps instead of defineCache, we have: Configuration defineConfiguration(String configurationName, Configuration overrides) // registers and names a NEW configuration, based on the default cfg and the overrides passed in Configuration defineConfiguration(String configurationName, String templateName, Configuration overrides) // registers and names a NEW configuration, based on an existing, predefined configuration and the overrides passed in WDYT? Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 7 Aug 2009, at 12:51, Galder Zamarreno wrote: Manik, While I working on this, I've noted the following. If we define default cache configurations for entity/query/...etc in an infinispan configuration file, currently, they cannot be overriden afterwards programatically. IOW, imagine there's a cache named entity that applies to all entities in a sample infinispan-configs.xml and then someone tweaks it in the hibernate config like this: hibernate.cache.infinispan.entity.eviction.wake_up_interval=3000 The way I'd deal with this is that I'd create a Configuration with this new interval and call defineCache(entity, cfg). However, when the configuration file was processed, entity cache was added to the configuration overrides and hence when I call defineCache with entity, it throws DuplicateCacheNameException. So, I think DFC.initialize() shouldn't add such configurations to configuration overridesm map. Thoughts? Hmm. Perhaps what is needed is a mechanism to retrieve these configurations and alter them. E.g., CacheManager.getConfiguration(); // retrieves the default cfg CacheManager.getConfiguration(String name); These Configuration objects - built based on what's in the XML file - can then be tweaked. These should be adequately javadoc'd to make it clear that caches already started using such a configuration will not be affected when a configuration changes. I guess this also renders CacheManager.defineCache() API useless/ obsolete, since getConfiguration() should always return a configuration, even if one didn't exist /wasn't defined in XML. Cheers Manik -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 6 Aug 2009, at 08:21, Galder Zamarreno wrote: On 08/05/2009 06:52 PM, Brian Stansberry wrote: Galder Zamarreno wrote: On 08/05/2009 04:04 PM, Brian Stansberry wrote: Galder Zamarreno wrote: /snip Sounds like this has diverged quite a bit from the JBC integration then. In your initial message you were discussing names: hibernate.cache.region.ispn4.cfg.entity hibernate.cache.region.ispn4.cfg.collection hibernate.cache.region.ispn4.cfg.query hibernate.cache.region.ispn4.cfg.timestamps What were those to be used for? With JBC they identify the name of a cache configuration, which is used to obtain an appropriately configured org.jboss.cache.Cache from the JBC CacheManager. My assumption on this thread was the same basic approach would be used with Infinispan. The region name that Hibernate passes is not meant to be the name of the cache configuration. It could be a unique identifier for the cache that's created using that configuration, but it's not the name of the configuration. Those names are not yet in use. They're just initial suggestions I had in mind to map JBC2/3 cache integration to ISPN. Shortly after I realised that actually, for each entity/collection, a cache was being created. If you follow that approach, you use the above properties to establish defaults for each of the 4 data types. You then use the techniques you discuss below to override those defaults if people need specialized configs for certain entities. And I suppose that using those 4 properties follows the same kind of default pattern as previous cache integration layer which is a good thing. Cool. Being able to configure different caches per entity type will be kinda nice. Semi-tangent: in general I really dislike if people have to configure JBC/Infinispan to get standard behaviors (e.g. eviction). Much better if people can use the standard configuration mechanism of whatever service is using JBC/Infinispan, and only touch JBC/Infinispan configs for exotic stuff. So, for example, web session passivation is configured via jboss-web.xml, not via a JBC eviction region. If the properties needed to configure 2nd Level Cache eviction could be reduced to 2 or 3, being able to express them via the SessionFactory config would be nice, e.g. hibernate.cache.infinispan.Users.max_age=5000 Perhaps just support LRU that way; if people want exotic stuff beyond LRU they have to go to the Infinispan config. I like the idea a lot. This would limit the number of files to modify for the most commonly touched things to 1 and that's a great thing from a usability perspective. I think this is very doable as well since ISPN has good support programmatic configuration (see http://www.jboss.org/community/wiki/infinispan-eviction) Infinispan has currently 5 settings in total for eviction/expiration and so they're little enough that we support them all, i.e. hibernate.cache.infinispan.entity.strategy=LRU hibernate.cache.infinispan.entity.wake_up_interval=2000 hibernate.cache.infinispan.entity.max_entries=5000 hibernate.cache.infinispan.entity.lifespan=6 // equivalent of maxAge hibernate.cache.infinispan.entity.max_idle=3 // equivalent of timeToLive Nice! -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] Spliting Lucene segments in Infinispan Directory
For some reason this got swallowed. Thanks for resending, Emmanuel. My comments inline. On 6 Aug 2009, at 15:19, Emmanuel Bernard wrote: Begin forwarded message: From: Emmanuel Bernard emman...@hibernate.org Date: August 4, 2009 08:53:07 EDT To: Manik Surtani ma...@jboss.org Cc: hibernate-dev@lists.jboss.org, infinispan-...@lists.jboss.org Subject: Re: [infinispan-dev] [hibernate-dev] Spliting Lucene segments in Infinispan Directory Manik, do you have some insight? We can't really understand why this is split. Emmanuel On Jul 30, 2009, at 19:51, Łukasz Moreń wrote: Hi, The JBoss Cache directory for Lucene splits each Lucene segment into pieces - chunks. Similar solutions exists in Lucene RamDirectory implementation. Smaller chunks mean smaller replication units, and finer grained locking (although the latter may not really be useful since you want to lock the entire 'file' each time). Are there some pros to use such splitting approach in Infinispan directory case? Some buffer size is recommended? For Infinispan though, I suggest not really bothering with this, unless you feel that individual files would really be very large and possibly too large to fit in memory - in which case chunking would make sense again. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 4 Aug 2009, at 09:37, Galder Zamarreno wrote: Hi, Re: https://jira.jboss.org/jira/browse/ISPN-6 Source code for this is currently located in an Infinispan branch in the Hibernate SVN repo: https://svn.jboss.org/repos/hibernate/core/branches/INFINISPAN/ http://anonsvn.jboss.org/repos/hibernate/core/branches/INFINISPAN/ I've picked this JIRA from Chris Bredesen. I'm waiting to get an answer to an email I sent him yesterday but in the mean time, here's a list of TODOs: 1. Current Infinispan region factory needs to point to a config with named caches. Suggested property name: hibernate.cache.region.ispn4.configs hibernate.cache.infinispan.cfg ? - why do we need 'region' in there, non-intuitive to end-users? - I'd rather use infinispan rather than ispn. 2. Need a equivalent version of this region factory where cache manager is retrieved from JNDI. Suggsted property name: hibernate.cache.region.ispn4.manager s/region.ispn4/infinispan 3. Configuration properties for named cache names. Suggested property names: hibernate.cache.region.ispn4.cfg.entity hibernate.cache.region.ispn4.cfg.collection hibernate.cache.region.ispn4.cfg.query hibernate.cache.region.ispn4.cfg.timestamps s/region.ispn4/infinispan 4. Resolve TransactionalAccess, ReadOnlyAccess and BaseRegion TODOs. 5. Enhance query results region so that query changes are not propagated if invalidation is used and add query.localonly equivalent property. Suggested name: hibernate.cache.region.ispn4.query.localonly s/region.ispn4/infinispan 6. Add more unit tests! 7. Document in wiki. Good stuff, thanks for taking this on! - Manik Some notes I've made while investigating this: - Whereas with JBC2/3, we had the possibility of a shared cache for all types (entities, collections, query,...etc) and a multiplexed version where each type had a specific cache, in Infinispan, it only makes the latter. Amongst other reasons because we don't have eviction regions any more and so we can't exclude the timestamp modification region as we did in JBC2/3. Overall, having a single option is a good thing from a configuration and usability perspective! Remember how complex eviction region definition could get for entities... Finally, a question to the list, specially for Brian/Steve who worked on the JBC2/3 integration layer: - Do we need a similar timestamp region local cache implementation for an ISPN based cache provider? Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [ISPN-6] (Infinispan cache provider for Hibernate) Remaining TODOs, notes and questions
On 4 Aug 2009, at 14:02, Galder Zamarreno wrote: s/region.ispn4/infinispan Ok. One thing here though. Chris's original solution works in such way that for each entity/collection, a new cache is retrieved from the cache manager using the region name, so for this example 3 caches would be created: Cache1 for [org.hibernate.test.cache.infinispan.functional.VersionedItem] Cache2 for [org.hibernate.test.cache.infinispan.functional.Item] Cache2 for [org.hibernate.test.cache.infinispan.functional.Item.items] Can we confirm this is the intented way? In https://jira.jboss.org/jira/browse/ISPN-6 the following is mentioned: Use a separate named cache per entity. This cache would hold entity instances as well as collections pertaining to that entity. So, if that is followed and we bear in mind the above example, there should only be 2 cache instances created rather than the current 3. What is clear is that there's no need for hibernate.cache.infinispan.cfg.entity or hibernate.cache.region.ispn4.cfg.collection. Simply stick the default cache configuration for entity/collections in the default section of configuration. I don't we need hibernate.cache.infinispan.cfg.query and hibernate.cache.infinispan.cfg.timestamps either since we can simply name the caches with the corresponding region names (org.hibernate.cache.UpdateTimestampsCache]and org.hibernate.cache.StandardQueryCache) and that's it. I suppose that would depend on the need for different eviction characteristics for different entity types. So from that perspective (the ability to use) a different cache per entity is useful. E.g., NoEvictionCache for [CountryList] NoEvictionCache for [SomeOtherDropDown] AggressivelyEvictedLRUCache for [Users] AggressivelyEvictedLRUCache for [Orders] LargeCapacityFIFOCache for [ProductsCatalog] etc. may well prove useful. Brian/Steve - care to chime in? Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: [infinispan-dev] Hibernate Search alternative Directory distribution
On 13 Jul 2009, at 17:10, Łukasz Moreń wrote: 1. share the same grid cache between the master and the slaves Infinispan has a flat structure. The key has to contain: - the index name - the chunk name Both with essentially be the unique identifier. I suppose in this idea all indexes are stored in a one single grid. What about one Infinispan grid per directory, similarly to RAMDirectory or FSDirectory? IMHO it could make some simplifications i.e. in metadata or key names. Are there any Infinispan drawbacks to have a high number of caches in the network? Sharing JGroups channels can help in that? They already share JGroups channels and other heavy components wherever possible. Its just that configuration becomes more of a pain, etc. When you say one cache per index, how do you define an index? Does 1 index mean all indexed data for a single java type? In which case couldn't these scale up dynamically and potentially on-demand? No wait - these are fixed in Hibernate Search on startup, correct? Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: [infinispan-dev] Hibernate Search alternative Directory distribution
On 9 Jul 2009, at 14:57, Emmanuel Bernard wrote: Here is the concall notes on how to cluster and copy Hibernate indexes using non file system approaches. Forget JBoss Cache, forget plain JGroups and focus on Infinispan Start with Infinispan in replication mode (the most stable code) and then try distribution. It should be interesting to test the dist algo and see how well L1 cache behaves in a search environment. For the architecture, we will try the following approach in decreasing interest )If the first one works like a charm we stick with it): 1. share the same grid cache between the master and the slaves 2. have a local cache on the master where indexing is done and manually copy over the chuncks of changed data to the grid This requires to store some metadata (namely the list of chunks for a given index and the lastupdate for each chunk) to implement the same algorithm as the one implemented in FSMaster/ SlaveDirectoryProvider (incremental copy). 3. have a local cache on the master where indexing is done and manually copy over the chuncks of changed data to the grid. Each slave copy from the grid to a local version of the index and use the local version for search. When writing the InfinispanDirectory (inspired by the RAMDirectory and the JBossCacheDirectory), one need to consider than Infinispan has a flat structure. The key has to contain: - the index name - the chunk name Both with essentially be the unique identifier. Each chunk should have its size limited (Lucene does that already AFAIK) Question on the metadata. one need ot keep the last update and the list of chuncks. Because Infinispan is not queryable, we need to store that as metadata: - should it be on each chunk (ie last time on each chunk, the size of a chunk) - on a dedicated metadata chunk ie one metadata chunk per chunk + a chink containing the list - on a single metadata chunk (I fear conflicts and inconsistencies) On changes or read explore the use of Infinispan transaction to ensure RR semantic. Is it necessary? A file system does not guarantee that anyway. In the case of replication, make sure a FD back end can be activated in case the grid goes to the unreachable clouds of total inactivity. FD backend? I presume you mean a cache store. Have a look at the different cache stores we ship with, I reckon a FileCacheStore would do the trick for you. http://infinispan.sourceforge.net/4.0/apidocs/org/infinispan/loaders/CacheStore.html http://infinispan.sourceforge.net/4.0/apidocs/org/infinispan/loaders/file/FileCacheStore.html Question to Manik: do you have a cluster to play with once we reach this stage? The cluster team does have a set of lab servers used to test, benchmark, etc. You will need to book time on this cluster though since it is shared between JBC/Infinispan, JGroups and JBoss AS clustering devs. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: [jbosscache-dev] JBoss Cache Lucene Directory
Sanne, Agreed. Could all involved please make sure we post to both hibernate- dev as well as infinispan-dev (rather than jbosscache-dev) when discussing anything to do with such integration work. As there are parallel efforts which can be brought together. Cheers Manik On 25 May 2009, at 10:53, Sanne Grinovero wrote: Hello, I'm forwarding this email to Emmanuel and Hibernate Search dev, as I believe we should join the discussion. Could we keep both dev-lists (jbosscache-...@lists.jboss.org, hibernate-dev@lists.jboss.org ) on CC ? Sanne 2009/4/29 Manik Surtani ma...@jboss.org: On 27 Apr 2009, at 05:18, Andrew Duckworth wrote: Hello, I have been working on a Lucene Directory provider based on JBoss Cache, my starting point was an implementation Manik had already written which pretty much worked with a few minor tweaks. Our use case was to cluster a Lucene index being used with Hibernate Search in our application, with the requirements that searching needed to be fast, there was no shared file system and it was important that the index was consistent across the cluster in a relatively short time frame. Maniks code used a token node in the cache to implement the distributed lock. During my testing I set up multiple cache copies with multiple threads reading/writing to each cache copy. I was finding a lot of transactions to acquire or release this lock were timing out, not understanding JBC well I modified the distributed lock to use JGroups DistrubutedLockManager. This worked quite well, however the time taken to acquire/release the lock (~100 ms for both) dwarfed the time to process the index update, lowering throughput. Even using Hibernate Search with an async worker thread, there was still a lot of contention for the single lock which seemed to limit the scalability of the solution. I thinkl part of the problem was that our use of HB Search generates a lot of small units of work (remove index entry, add index entry) and each of these UOW acquire a new IndexWriter and new write lock on the underlying Lucene Directory implementation. Out of curiosity, I created an alternative implementation based on the Hibernate Search JMS clustering strategy. Inside JBoss Cache I created a queue node and each slave node in the cluster creates a separate queue underneath where indexing work is written: /queue/slave1/[work0, work1, work2 ] /slave2 /slave3 etc In each cluster member a background thread runs continuously when it wakes up, it decides if it is the master node or not (currently checks if it is the view coordinator, but I'm considering changing it to use a longer lived distributed lock). If it is the master it merges the tasks from each slave queue, and updates the JBCDirectory in one go, it can safely do this with only local VM locking. This approach means that in all the slave nodes they can write to their queue without needing a global lock that any other slave or the master would be using. On the master, it can perform multiple updates in the context of a single Lucene index writer. With a cache loader configured, work that is written into the slave queue is persistent, so it can survive the master node crashing with automatic fail over to a new master meaning that eventually all updates should be applied to the index. Each work element in the queue is time stamped to allow them to be processed in order (requires! time synchronisation across the cluster) by the master. For our workload the master/slave pattern seems to improve the throughput of the system. Currently I'm refining the code and I have a few JBoss Cache questions which I hope you can help me with: 1) I have noticed that under high load I get LockTimeoutExceptions writing to /queue/slave0 when the lock owner is a transaction working on /queue/slave1 , i.e. the same lock seems to be used for 2 unrelated nodes in the cache. I'm assuming this is a result of the lock striping algorithm, if you could give me some insight into how this works that would be very helpful. Bumping up the cache concurrency level from 500 to 2000 seemed to reduce this problem, however I'm not sure if it just reduces the probability of a random event of if there is some level that will be sufficient to eliminate the issue. It could well be the lock striping at work. As of JBoss Cache 3.1.0 you can disable lock striping and have one lock per node. While this is expensive in that if you have a lot of nodes, you end up with a lot of locks, if you have a finite number of nodes this may help you a lot. 2) Is there a reason to use separate nodes for each slave queue ? Will it help with locking, or can each slave safely insert to the same parent node in separate transactions without interfering or blocking each other ? If I can reduce it to a single queue I thin that would be a more elegant solution. I am
Re: [jbosscache-dev] Re: [hibernate-dev] cache-jbosscache3 module for Hibernate Core
lol! thanks, will fix this in trunk now and publish a snapshot of 3.0.0. 2008/10/18 Brian Stansberry [EMAIL PROTECTED] Elias Ross was kind enough to notify me privately that the difference in method signature isn't the generics (which of course don't matter at runtime), it's the change in return type from CacheFactory to DefaultCacheFactory. DOH! Maxim: if the only remaining answer is impossible, don't decide the impossible is the answer. Look again and see the blindingly obvious possible answer. :-) Brian Stansberry wrote: Sorry, Manik, tests were failing w/ NoSuchMethodError and I saw your recent change in hibernate trunk to remove use of DefaultCacheFactory.getInstance() and assumed the method was gone. My bad :-( Perhaps problem is loss of generics info in the class file? 2.1.0.GA: public static K, V CacheFactoryK, V getInstance() 3.0.0.CR1: public static DefaultCacheFactory getInstance() Failure I see is: java.lang.NoSuchMethodError: org.jboss.cache.DefaultCacheFactory.getInstance()Lorg/jboss/cache/CacheFactory; at org.hibernate.cache.jbc2.builder.SharedCacheInstanceManager.createSharedCache(SharedCacheInstanceManager.java:193) I don't remember exactly how I tested this yesterday, but messing with it today, I can reproduce by: 1) revert pom in my checkout of the 3.3.1 tag so it uses JBC 2.1.0.GA 2) revert any compiled classes back to the original 3.3.1 mvn -Dmaven.test.skip.exec=true clean install 3) change the pom so it uses JBC 3.3.0.CR1 and JGroups 2.6.5 4) run the testsuite mvn -Ptest test Tests run: 225, Failures: 0, Errors: 21, Skipped: 0 5) force a new compile and then retest: mvn -Ptest clean test ... Tests run: 225, Failures: 0, Errors: 0, Skipped: 0 Problem is people using a 3.3.1 binary don't get to recompile. ;) Manik Surtani wrote: Weird, getInstance() was removed in early ALPHAs, and was replaced again pretty quickly - in 3.0.0.BETA1, even. http://fisheye.jboss.org/browse/JBossCache/core/tags/3.0.0.BETA1/src/main/java/org/jboss/cache/DefaultCacheFactory.java?r=6676 2008/10/18 Jason T. Greene [EMAIL PROTECTED] mailto: [EMAIL PROTECTED] Brian Stansberry wrote: Steve Ebersole wrote: so JBC 3 needs this change anyway? Yes, if it wants to go in, say, JBoss AS 5.2. Which I'm quite sure the JBC team wants, since they made a bunch of other more significant changes to ensure compatibility. This one's real trivial. at which point it would be a total drop-in replacement? Yes. I just did a diff between head of trunk (which passes 100% w/ JBC 3.0.0.CR1 plugged in) and the hibernate-3.3.1.GA http://hibernate-3.3.1.GA tag and the only differences are two places where the missing DefaultCacheFactory.getInstance() call was replaced. I fixed this compatibility problem awhile ago, but it could have been after CR1 was tagged. -- Jason T. Greene JBoss, a division of Red Hat ___ jbosscache-dev mailing list [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat [EMAIL PROTECTED] ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [jbosscache-dev] Re: [hibernate-dev] cache-jbosscache3 module for Hibernate Core
By the way, there's some feedback on using 3.0.0 as a 2nd level cache on Hibernate here (see comments at the end): http://jbosscache.blogspot.com/2008/09/first-cr-for-300.html I notice MVCC uses significantly less memory for my use case compared to both 2.2 and Coherence. 2008/10/19 Manik Surtani [EMAIL PROTECTED] lol! thanks, will fix this in trunk now and publish a snapshot of 3.0.0. 2008/10/18 Brian Stansberry [EMAIL PROTECTED] Elias Ross was kind enough to notify me privately that the difference in method signature isn't the generics (which of course don't matter at runtime), it's the change in return type from CacheFactory to DefaultCacheFactory. DOH! Maxim: if the only remaining answer is impossible, don't decide the impossible is the answer. Look again and see the blindingly obvious possible answer. :-) Brian Stansberry wrote: Sorry, Manik, tests were failing w/ NoSuchMethodError and I saw your recent change in hibernate trunk to remove use of DefaultCacheFactory.getInstance() and assumed the method was gone. My bad :-( Perhaps problem is loss of generics info in the class file? 2.1.0.GA: public static K, V CacheFactoryK, V getInstance() 3.0.0.CR1: public static DefaultCacheFactory getInstance() Failure I see is: java.lang.NoSuchMethodError: org.jboss.cache.DefaultCacheFactory.getInstance()Lorg/jboss/cache/CacheFactory; at org.hibernate.cache.jbc2.builder.SharedCacheInstanceManager.createSharedCache(SharedCacheInstanceManager.java:193) I don't remember exactly how I tested this yesterday, but messing with it today, I can reproduce by: 1) revert pom in my checkout of the 3.3.1 tag so it uses JBC 2.1.0.GA 2) revert any compiled classes back to the original 3.3.1 mvn -Dmaven.test.skip.exec=true clean install 3) change the pom so it uses JBC 3.3.0.CR1 and JGroups 2.6.5 4) run the testsuite mvn -Ptest test Tests run: 225, Failures: 0, Errors: 21, Skipped: 0 5) force a new compile and then retest: mvn -Ptest clean test ... Tests run: 225, Failures: 0, Errors: 0, Skipped: 0 Problem is people using a 3.3.1 binary don't get to recompile. ;) Manik Surtani wrote: Weird, getInstance() was removed in early ALPHAs, and was replaced again pretty quickly - in 3.0.0.BETA1, even. http://fisheye.jboss.org/browse/JBossCache/core/tags/3.0.0.BETA1/src/main/java/org/jboss/cache/DefaultCacheFactory.java?r=6676 2008/10/18 Jason T. Greene [EMAIL PROTECTED] mailto: [EMAIL PROTECTED] Brian Stansberry wrote: Steve Ebersole wrote: so JBC 3 needs this change anyway? Yes, if it wants to go in, say, JBoss AS 5.2. Which I'm quite sure the JBC team wants, since they made a bunch of other more significant changes to ensure compatibility. This one's real trivial. at which point it would be a total drop-in replacement? Yes. I just did a diff between head of trunk (which passes 100% w/ JBC 3.0.0.CR1 plugged in) and the hibernate-3.3.1.GA http://hibernate-3.3.1.GA tag and the only differences are two places where the missing DefaultCacheFactory.getInstance() call was replaced. I fixed this compatibility problem awhile ago, but it could have been after CR1 was tagged. -- Jason T. Greene JBoss, a division of Red Hat ___ jbosscache-dev mailing list [EMAIL PROTECTED] mailto: [EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat [EMAIL PROTECTED] ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [jbosscache-dev] Re: [hibernate-dev] cache-jbosscache3 module for Hibernate Core
Weird, getInstance() was removed in early ALPHAs, and was replaced again pretty quickly - in 3.0.0.BETA1, even. http://fisheye.jboss.org/browse/JBossCache/core/tags/3.0.0.BETA1/src/main/java/org/jboss/cache/DefaultCacheFactory.java?r=6676 2008/10/18 Jason T. Greene [EMAIL PROTECTED] Brian Stansberry wrote: Steve Ebersole wrote: so JBC 3 needs this change anyway? Yes, if it wants to go in, say, JBoss AS 5.2. Which I'm quite sure the JBC team wants, since they made a bunch of other more significant changes to ensure compatibility. This one's real trivial. at which point it would be a total drop-in replacement? Yes. I just did a diff between head of trunk (which passes 100% w/ JBC 3.0.0.CR1 plugged in) and the hibernate-3.3.1.GA tag and the only differences are two places where the missing DefaultCacheFactory.getInstance() call was replaced. I fixed this compatibility problem awhile ago, but it could have been after CR1 was tagged. -- Jason T. Greene JBoss, a division of Red Hat ___ jbosscache-dev mailing list [EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: [jbosscache-dev] REPEATABLE_READ in JBC as Hibernate 2nd Level Cache
On 18 Jul 2008, at 16:35, Brian Stansberry wrote: Galder Zamarreno wrote: Paul Ferraro wrote: refresh() always goes to the db, and only potentially triggers a 2LC update - not a read. [OT] A concern I have is whether the refresh() removes the item from the 2LC before doing the subsequent Hibernate cache put(). If not our putForExternalRead() usage will prevent the new value being placed in the cache. Either way this should be tested. In the cache-jbosscache2 module in hibernate-core, probably. Perhaps we should revisit whether the scenario below warrants the cost of repeatable_read, given that entity eviction from session cache is typically used only to minimize memory usage when iterating through large results, i.e. when an entity is not likely to be re-read within the current transaction. Yes. R_C will be the default config; this stuff is really something to document, and to add an alternate R_R config as a convenience for those who actually need it. See http://opensource.atlassian.com/projects/hibernate/browse/HHH-3390 This whole discussion has me thinking about the use of the OPTIMISTIC configs as the default. The main benefit of OPTIMISTIC is allowing an R_R-like semantic without blocking writes. But it comes as a cost. (Again, we're talking defaults here, not whether O/ L is 'supported'.) H, this sounds like a bad practice to me or at least not very performant. Evict/clear something within a transaction and then re- get it. Doing an evict() makes perfect sense in some scenarios (to control memory). Doing a subsequent re-read can easily happen if the logic is complex. (Agreed, if it happens a lot that's a sign of a design flaw in the app.) But to then expect R_R from the second read: that for sure makes it an edge case. Brian, did you use 2nd level cache in your previous use case? No. On Thu, 2008-07-17 at 16:59 -0500, Brian Stansberry wrote: Hmm, good point. Potentially also Session.refresh(...), although I'm not sure if the implementation of that method skips the 2LC and goes right to the db. Paul Ferraro wrote: After thinking this through, the only scenario I can think of where the 2LC would be subject to a repeated read is after a session cache eviction (i.e. via Session.clear() or Session.evict(...)). Without REPEATABLE_READ isolation on the 2LC, any subsequent request withing the same transaction for an evicted entity could return an updated value, if the cache was updated by a concurrent request. Paul On Thu, 2008-07-17 at 12:59 -0500, Brian Stansberry wrote: Can anyone see a reason to use REPEATABLE_READ as the JBoss Cache isolation level in the 2nd level cache use case? I'm not seeing one, and it certainly hurts performance by forcing cache writes to block waiting for an earlier tx that did a read to commit. There are 4 types of data cached: 1) Entities If an entity is read from the 2LC, for the life of the tx it will be cached in the Session, so AIUI there should be no second read during the tx. So no benefit to RR. 2) Collections Same as entities. 3) Queries If an application executes a query twice in the same tx, I wouldn't think they'd expect the same result. In any case, if an update to the query cache is blocking waiting for a tx that previously read the query result to release, the existence of the update that means the underlying entities and their timestamps have changed. So a repeated read of the cached query will just result in it being discarded as out of date anyway. 4) Timestamps Here you don't want an RR semantic. You always want to get the most up-to-date data. Anyone see any holes in my thinking? -- Brian Stansberry Lead, JBoss AS Clustering JBoss, a division of Red Hat ___ jbosscache-dev mailing list [EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev ___ jbosscache-dev mailing list [EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat [EMAIL PROTECTED] ___ jbosscache-dev mailing list [EMAIL PROTECTED] https://lists.jboss.org/mailman/listinfo/jbosscache-dev -- Manik Surtani Lead, JBoss Cache [EMAIL PROTECTED] ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Thoughts on custom data versions in JBoss Cache 3.0.0
Hi guys,
would appreciate your thoughts and comments on the ${SUBJECT}:
http://www.jboss.com/index.html?module=bbop=viewtopicp=4163564
Cheers
--
Manik Surtani
Lead, JBoss Cache
[EMAIL PROTECTED]
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] JBoss Cache Searchable interfaces
On 26 Jun 2008, at 13:48, Emmanuel Bernard wrote: SNIP / * iterator(); don't think it's that useful in practice, scroll is probably more practical as it lazily loads Lucene data (contrary to iterator) Then why don't we just lazily load Lucene data into the iterator and drop scroll()? I see supporting both as superfluous. And iterator() is more familiar to devs. That's the point of Scroll, but you can rename it if you want. What's important is that you need a .close() method to release the lucene resources. also scroll() could return null if the object is not found in the cache (inconsistency), iterate() will just ignore the entry. +1, good point. We'd need a close() method on the QueryResultIterator. I see just returning a null if the object is not in the cache as something people would expect. I think that any benefit of a scrollable result set window preventing loading unnecessary objects from a DB are lost when your backing store is a cache and the objects are in memory anyway. And besides, any further optimisations can be in the iterator implementation, such as just maintaining a list of CachedEntityIds (a composite of Fqn and key) and fetching the objects from the cache lazily, as required. iterate() lazily load object from the DB but eagerly loads Lucene stuffs scroll() lazily load everything but need a .close() If you think eager loading of Lucene data is something people would find useful, we could have 2 separate implementations of the QueryResultIterator - a QRILazyImpl and a QRIEagerImpl. And the interface to obtain these could be on the CacheQuery interface: CacheQuery.iterator(boolean lazy); You can use your own Scrolalble interface, that's fair. Essentially Search should look natural to cache users, not Hibernate users :) Yup. :-) -- Manik Surtani Lead, JBoss Cache [EMAIL PROTECTED] ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Re: JBoss Cache and Hibernate Integration
back through Hibernate; Hibernate can then manage ordering the callbacks? I don't recall more progress on that topic. Don't forget this one - manik ? :) I think he's teaching this week?? -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat [EMAIL PROTECTED] -- Manik Surtani Lead, JBoss Cache JBoss, a division of Red Hat Email: [EMAIL PROTECTED] Telephone: +44 7786 702 706 MSN: [EMAIL PROTECTED] Yahoo/AIM/Skype: maniksurtani ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: JBoss Cache and Hibernate Integration
): 00 800 226 1625 International toll free (Poland): 00 800 111 48 26 International toll free (Portugal): 800 819 366 International toll free (Russia): 810 800 2535 1012 International toll free (Singapore): 800 101 1767 International toll free (Slovenia): 0 800 80802 International toll free (South Africa): 0 800 999 571 International toll free (South Korea): 003 0813 1634 International toll free (Spain): 900 987 056 International toll free (Sweden): 02 079 5157 International toll free (Switzerland): 0 800 562 271 International toll free (Thailand): 001 800 156 205 1625 International toll free (Trinidad-Tobago): 1 800 205 1625 International toll free (UK): 0 800 051 3876 International toll free (Uruguay): 0004 019 0088 International toll free (Venezuela): 0 800 100 5202 Manik Surtani wrote: +1 On 22 Feb 2007, at 18:19, Steve Ebersole wrote: Brian Stansberry wrote: Sent this out a while back, but the conf call didn't come off due to scheduling issues. Time to try again. How does next Monday, Feb 26 at 10:00 AM EST sound? Once we have an agreed time I'll send out conference call details. Those cc'ed on the mailing lists are welcome to join in. Also including Owen explicitly, since I am not sure if he is part of either list. That date/time is fine for me. -- Manik Surtani Lead, JBoss Cache JBoss, a division of Red Hat Email: [EMAIL PROTECTED] Telephone: +44 7786 702 706 MSN: [EMAIL PROTECTED] Yahoo/AIM/Skype: maniksurtani -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat [EMAIL PROTECTED] -- Manik Surtani Lead, JBoss Cache JBoss, a division of Red Hat Email: [EMAIL PROTECTED] Telephone: +44 7786 702 706 MSN: [EMAIL PROTECTED] Yahoo/AIM/Skype: maniksurtani ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Re: JBoss Cache and Hibernate Integration
+1 On 22 Feb 2007, at 18:19, Steve Ebersole wrote: Brian Stansberry wrote: Sent this out a while back, but the conf call didn't come off due to scheduling issues. Time to try again. How does next Monday, Feb 26 at 10:00 AM EST sound? Once we have an agreed time I'll send out conference call details. Those cc'ed on the mailing lists are welcome to join in. Also including Owen explicitly, since I am not sure if he is part of either list. That date/time is fine for me. -- Manik Surtani Lead, JBoss Cache JBoss, a division of Red Hat Email: [EMAIL PROTECTED] Telephone: +44 7786 702 706 MSN: [EMAIL PROTECTED] Yahoo/AIM/Skype: maniksurtani ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
