Re: [hibernate-dev] Chat migration - D minus 115 until the death of HipChat
On 12/06/2018 10:40 AM, Guillaume Smet wrote: > So, I'm using Zulip right now on a daily basis. > > I maintain my first impression that it's really not user friendly. > > The fact that you are required to create topics for discussions Why is creating a new topic (=typing three extra words) an issue? This is the killer feature why many people love Zulip. (I would not spend more than 5 seconds checking for any existing topic on similar matter) > (or find a > suitable topic in a list of a gazillion topics previously created, > obviously without a search engine where you need it - you have a global one > at the top where you can find topics) is a pain. You also need to use > ctrl+enter to send a message, the default enter is a new line in your > message. There's a checkbox just next to the Send button, check it once and the setting persists. > The UI is not very good and I don't see any improvement since the > last time I tested it so I'm wondering if they are investing in it. Maybe they don't have feedback? Besides zooming in (View - Zoom In) I am perfectly satisfied with the UI. Actually one thing that I am not 100% comfortable with is the active/inactive status, which marks people inactive despite they have the client just on the background. Radim > > We could decide to use it as a dev team as I suppose we would get used to > it, but I seriously don't think it's a good alternative for our users to > occasionally come chat with us. > > As for Gitter, I agree with the notification issue, the web client is all > buggy. Haven't tested the desktop client yet. > > I must admit that I prefer using Gitter. Probably until I get bitten by the > 1-1 history issue :). > > From what I can see, GitLab doesn't invest much in Gitter either so I > wonder if it's gonna be viable in the long term. > > I suppose we'll see. > -- Radim Vansa JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Stride
;>>>> >>>>>> If they were optional, that would do but they are not and you always >>> need >>>>> 2 >>>>>> clicks to share (e.g. go to the right stream, then either choose a >>> topic >>>>> or >>>>>> create new topic), whereas you're at most one click away on HipChat. >>> For >>>>>> our usage I find it a bit suboptimal. >>>>>> >>>>>> I use the HipChat mobile client from time to time, not sure how bad >>>>> Zulip's >>>>>> is. Can you at least follow the streams and post messages? >>>>>> >>>>>> Anyway, it's more a -0 than a -1 for Zulip. >>>>>> >>>>>> Never used Slack. >>>>>> >>>>>> -- >>>>>> Guillaume >>>>> ___ >>>>> hibernate-dev mailing list >>>>> hibernate-dev@lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>> >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Stride
On 05/18/2018 03:56 PM, Scott Marlow wrote: > Does Zulip have a Fedora (native) client that can be installed by > Fedora dnf tool? No, they don't have RPM, I use the provided AppImage. Yes, nuisance (updates are not automatic). Issue [1] from 2015 looks stale. htop tells me that it's consuming 114 MB of RES memory and 12% CPU (!) when idle, though - I was hoping for better. [1] https://github.com/zulip/zulip/issues/41 > I've been using Zulip in a browser [1] (as I do with hipchat) and it > seems at least as good as hipchat. > > Has anyone looked at the Zulip multiple organization (team) support [2]? Client supporting multiple organizations was among the key requirements; noone likes the situation around isolated WF & Hibernate HipChats. R. > > Scott > > [1] https://infinispan.zulipchat.com/ > [2] > https://zulip.readthedocs.io/en/1.7.1/prod-multiple-organizations.html > > On 05/18/2018 02:57 AM, Radim Vansa wrote: >> Just out of curiosity, when choosing a replacement for HipChat, have you >> considered Zulip? >> >> Infinispan uses that for about a month now and besides being too >> colourful (similar to HipChat) there's been positive feedback, >> especially due to the threading feature. >> >> Radim [trying to lure everyone to the same client] >> >> On 05/18/2018 12:16 AM, Sanne Grinovero wrote: >>> On 17 May 2018 at 20:32, Guillaume Smet <guillaume.s...@gmail.com> >>> wrote: >>>> By the way, you say it’s clear we don’t want to stay on HipChat. >>>> >>>> When did we decide that? I don’t remember a discussion about it. >>> I didn't say it was decided, but I think we're working on that since >>> Steve asked about it today. To which I agree *because* it seems clear >>> to me that we don't want to stay on it - a notion I inferred from >>> multiple previous discussions. >>> >>> Steve pointed out multiple flaws e.g. the native client packaging >>> broken on Fedora, to which Atlassian pretty much replied by letting us >>> know they won't invest in HipChat as the future is Stride. We can >>> choose when to switch but staying doesn't look sensible to me as it >>> certainly won't improve; it's also likely that they'll want everyone >>> migrated eventually so to shut the existing service down. >>> >>> I for one gave up as well installing the native client and have been >>> using the web client since setting up my new workstation, as I was >>> expecting Stride to arrive soon. >>> >>> The other day some people tried to join and gave up because of login >>> complexity - that's IMO a very bad sign: not welcoming community >>> people means it's failing its primary requirements. >>> >>> And let's not forget all authentication nonsense; especially days that >>> I'm working more on the WildFly side of things and need to login to >>> multiple instances I really look forward to a better system (hopefully >>> it is!?). >>> >>> Question, since you want a decision: are you only suggesting to delay >>> or suggesting that you should rather stay on HipChat? >>> >>> Personally, I'm fine delaying a bit even though I can live happily >>> without Jenkins notifications, but let's hear the others as well. >>> >>> Thanks, >>> Sanne >>> >>> >>>> For sure, we probably won’t have a choice because there’s a good >>>> chance Atlassian will close the service but what are the problems >>>> that make a migration so urgent? >>>> >>>>> Le 17 mai 2018 à 20:16, Sanne Grinovero <sa...@hibernate.org> a >>>>> écrit : >>>>> >>>>> lol, I was writing about the same to the team list. >>>>> >>>>> +1 to have people register, it's better for them anyway. I checked >>>>> it's easier to self-register. >>>>> >>>>> +1 to migrate quickly. It's clear we don't want to stay on >>>>> HipChat, if >>>>> this doesn't work out we'll see. >>>>> >>>>> Refer to my parallel email for Fedora instructions. >>>>> >>>>> Thanks, >>>>> Sanne >>>>> >>>>> >>>>>> On 17 May 2018 at 19:03, Steve Ebersole <st...@hibernate.org> wrote: >>>>>> I got an email from Atlassian this morning about the migration >>>>>> from HipChat >>>>>> to Stride. Basically they have not gotten Stride >>>&g
Re: [hibernate-dev] Cache region names are not prefixed in 5.3
Ignore me, panicked too quickly... the dot is added there in 5.1 as well. On 05/18/2018 09:37 AM, Radim Vansa wrote: > One thing I've stumbled upon: it seems that RegionFactory should call > its method qualify(String regionName) to produce the prefixed region > name. Following Hibernate's implementation I use RegionNameQualifier > for this, which uses prefix + '.' + regionName. > > In previous versions the dot was missing from the qualifier - is this > something that could break backwards compatibility? E.g. in WF to let > 2LC use specific cache configuration you need to set > `hibernate.cache.infinispan.deployment#entity.name.cfg` - I wonder if > the qualified region name becomes deployment#.entity.name now... > > Radim > > On 05/18/2018 09:02 AM, Radim Vansa wrote: >> On 05/18/2018 02:54 AM, Gail Badner wrote: >>> >>> >>> On Thu, May 17, 2018 at 5:24 PM, Gail Badner <gbad...@redhat.com >>> <mailto:gbad...@redhat.com>> wrote: >>> >>> I see that HHH-11356 removed prefixes from region names used by >>> Hibernate. >>> >>> I also noticed that entity regions are unprefixed and the package >>> name is removed. >>> >>> >>> Actually, the package names are being included in the region names. >>> The test I was looking at did not have a package. >>> >>> >>> I've created 2 issues: >>> HHH-12599 to add Javadoc to make it clear that region names are >>> unprefixed; >>> HHH-12598 to add the package onto entity region names. >>> >>> >>> I've rejected HHH-12598. >>> >>> >>> Should I create an ISPN issue for hibernate-infinispan (or >>> whatever it's referred to now) to add the prefixes? >>> >> >> I take it as confirmed that RegionFactory is supposed to do this. >> Created https://issues.jboss.org/browse/ISPN-9174 >> >> R. >> >>> >>> On Thu, May 17, 2018 at 4:55 AM, Sanne Grinovero >>> <sa...@hibernate.org <mailto:sa...@hibernate.org>> wrote: >>> >>> On 17 May 2018 at 12:09, Radim Vansa <rva...@redhat.com >>> <mailto:rva...@redhat.com>> wrote: >>> > I basically agree with Sanne here that having the prefix >>> isolated opens >>> > space for performance improvements, though if certain call >>> is prefixed, >>> > RegionFactory can always drop that prefix. The important >>> thing is to mention >>> > if the region name is prefixed or not in javadocs. I would >>> also prefer if >>> > *all* region names that RegionFactory is supposed to access >>> followed the >>> > same strategy (prefixed/not-prefixed). >>> > >>> > 5.1 included method >>> > >>> > QueryResultsRegion buildQueryResultsRegion(String >>> regionName, Properties >>> > properties) >>> > >>> > where StandardQueryCache did the prefix for us, the change >>> in 5.3 to >>> > >>> > QueryResultsRegion buildQueryResultsRegion(String >>> regionName, >>> > SessionFactoryImplementor sessionFactory) >>> > >>> > does not indicate that the prefix won't be there and the >>> javadoc is missing >>> > completely. >>> > >>> > Also, DomainDataRegionConfig.getRegionName() has no >>> javadoc and >>> > RegionFactory does not add the prefix. >>> > >>> > @Steve could you please amend the javadoc and confirm if RF >>> should prefix >>> > region names? >>> > >>> > @Sanne while cache manager per deployment is an obvious way >>> to distinguish >>> > regions@deployments, CM holds some heavy resources (e.g. >>> threads) - so I >>> > while we are supposed to scale number of caches up to >>> thousands (and we've >>> > fixed some problems that arise when you have for example ~3k >>> caches), ATM >>> > you're not supposed to scale CMs. And I don't think that >>> we'll focus in this >>> > direction since
Re: [hibernate-dev] Cache region names are not prefixed in 5.3
One thing I've stumbled upon: it seems that RegionFactory should call its method qualify(String regionName) to produce the prefixed region name. Following Hibernate's implementation I use RegionNameQualifier for this, which uses prefix + '.' + regionName. In previous versions the dot was missing from the qualifier - is this something that could break backwards compatibility? E.g. in WF to let 2LC use specific cache configuration you need to set `hibernate.cache.infinispan.deployment#entity.name.cfg` - I wonder if the qualified region name becomes deployment#.entity.name now... Radim On 05/18/2018 09:02 AM, Radim Vansa wrote: > On 05/18/2018 02:54 AM, Gail Badner wrote: >> >> >> On Thu, May 17, 2018 at 5:24 PM, Gail Badner <gbad...@redhat.com >> <mailto:gbad...@redhat.com>> wrote: >> >> I see that HHH-11356 removed prefixes from region names used by >> Hibernate. >> >> I also noticed that entity regions are unprefixed and the package >> name is removed. >> >> >> Actually, the package names are being included in the region names. >> The test I was looking at did not have a package. >> >> >> I've created 2 issues: >> HHH-12599 to add Javadoc to make it clear that region names are >> unprefixed; >> HHH-12598 to add the package onto entity region names. >> >> >> I've rejected HHH-12598. >> >> >> Should I create an ISPN issue for hibernate-infinispan (or >> whatever it's referred to now) to add the prefixes? >> > > I take it as confirmed that RegionFactory is supposed to do this. > Created https://issues.jboss.org/browse/ISPN-9174 > > R. > >> >> On Thu, May 17, 2018 at 4:55 AM, Sanne Grinovero >> <sa...@hibernate.org <mailto:sa...@hibernate.org>> wrote: >> >> On 17 May 2018 at 12:09, Radim Vansa <rva...@redhat.com >> <mailto:rva...@redhat.com>> wrote: >> > I basically agree with Sanne here that having the prefix >> isolated opens >> > space for performance improvements, though if certain call >> is prefixed, >> > RegionFactory can always drop that prefix. The important >> thing is to mention >> > if the region name is prefixed or not in javadocs. I would >> also prefer if >> > *all* region names that RegionFactory is supposed to access >> followed the >> > same strategy (prefixed/not-prefixed). >> > >> > 5.1 included method >> > >> > QueryResultsRegion buildQueryResultsRegion(String >> regionName, Properties >> > properties) >> > >> > where StandardQueryCache did the prefix for us, the change >> in 5.3 to >> > >> > QueryResultsRegion buildQueryResultsRegion(String >> regionName, >> > SessionFactoryImplementor sessionFactory) >> > >> > does not indicate that the prefix won't be there and the >> javadoc is missing >> > completely. >> > >> > Also, DomainDataRegionConfig.getRegionName() has no javadoc >> and >> > RegionFactory does not add the prefix. >> > >> > @Steve could you please amend the javadoc and confirm if RF >> should prefix >> > region names? >> > >> > @Sanne while cache manager per deployment is an obvious way >> to distinguish >> > regions@deployments, CM holds some heavy resources (e.g. >> threads) - so I >> > while we are supposed to scale number of caches up to >> thousands (and we've >> > fixed some problems that arise when you have for example ~3k >> caches), ATM >> > you're not supposed to scale CMs. And I don't think that >> we'll focus in this >> > direction since the bright future lies in microservices :) >> >> Right, good points. It's a possible optimization I'd like to see >> eventually but it's not trivial to get there yet. >> >> Yet assuming a microservices scenario, this does become >> trivial to >> benefit from: the container knows there's a single deployment, a >> single CM. So no need to isolate them at all, just need to >> possibly >> isolate multiple PUs defined in the same servic
Re: [hibernate-dev] Stride
Just out of curiosity, when choosing a replacement for HipChat, have you considered Zulip? Infinispan uses that for about a month now and besides being too colourful (similar to HipChat) there's been positive feedback, especially due to the threading feature. Radim [trying to lure everyone to the same client] On 05/18/2018 12:16 AM, Sanne Grinovero wrote: > On 17 May 2018 at 20:32, Guillaume Smet <guillaume.s...@gmail.com> wrote: >> By the way, you say it’s clear we don’t want to stay on HipChat. >> >> When did we decide that? I don’t remember a discussion about it. > I didn't say it was decided, but I think we're working on that since > Steve asked about it today. To which I agree *because* it seems clear > to me that we don't want to stay on it - a notion I inferred from > multiple previous discussions. > > Steve pointed out multiple flaws e.g. the native client packaging > broken on Fedora, to which Atlassian pretty much replied by letting us > know they won't invest in HipChat as the future is Stride. We can > choose when to switch but staying doesn't look sensible to me as it > certainly won't improve; it's also likely that they'll want everyone > migrated eventually so to shut the existing service down. > > I for one gave up as well installing the native client and have been > using the web client since setting up my new workstation, as I was > expecting Stride to arrive soon. > > The other day some people tried to join and gave up because of login > complexity - that's IMO a very bad sign: not welcoming community > people means it's failing its primary requirements. > > And let's not forget all authentication nonsense; especially days that > I'm working more on the WildFly side of things and need to login to > multiple instances I really look forward to a better system (hopefully > it is!?). > > Question, since you want a decision: are you only suggesting to delay > or suggesting that you should rather stay on HipChat? > > Personally, I'm fine delaying a bit even though I can live happily > without Jenkins notifications, but let's hear the others as well. > > Thanks, > Sanne > > >> For sure, we probably won’t have a choice because there’s a good chance >> Atlassian will close the service but what are the problems that make a >> migration so urgent? >> >>> Le 17 mai 2018 à 20:16, Sanne Grinovero <sa...@hibernate.org> a écrit : >>> >>> lol, I was writing about the same to the team list. >>> >>> +1 to have people register, it's better for them anyway. I checked >>> it's easier to self-register. >>> >>> +1 to migrate quickly. It's clear we don't want to stay on HipChat, if >>> this doesn't work out we'll see. >>> >>> Refer to my parallel email for Fedora instructions. >>> >>> Thanks, >>> Sanne >>> >>> >>>> On 17 May 2018 at 19:03, Steve Ebersole <st...@hibernate.org> wrote: >>>> I got an email from Atlassian this morning about the migration from HipChat >>>> to Stride. Basically they have not gotten Stride feature-complete in terms >>>> of HipChat which is the trigger for the mass migration. However, they are >>>> reaching out to all waiting teams to see if any want to migrate anyway. >>>> The list of missing features they sent me are: >>>> >>>> >>>>1. Guest access >>>>2. Some admin controls and compliance settings >>>>3. Integrations with Atlassian server products (the Jira Server app is >>>>currently in beta and coming soon) and some other popular integrations. >>>> See >>>>all available Stride integrations >>>> >>>> <http://click.mailer.atlassian.com/?qs=6712850cb7a4f2d4a207f46e5f190a454855738b8db6340974f28f3c2e1e6aa7c674a6314ad5df11014e5e2ee39bd3d4326606d2826b241a> >>>>. >>>>4. User management via API >>>>5. Dark mode >>>> >>>> I am not really sure exactly what is missing WRT (2). (3) is nice-to-have, >>>> but not blocker IMO assuming it gets added at some point. >>>> >>>> I think (1) is the only one that is concerning. Though TBH for myself >>>> personally, I do not think registering is a big deal. >>>> >>>> Unless I hear otherwise, I plan on asking them to proceed with our >>>> migration to Stride. >>>> ___ >>>> hibernate-dev mailing list >>>> hibernate-dev@lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Cache region names are not prefixed in 5.3
On 05/18/2018 02:54 AM, Gail Badner wrote: > > > On Thu, May 17, 2018 at 5:24 PM, Gail Badner <gbad...@redhat.com > <mailto:gbad...@redhat.com>> wrote: > > I see that HHH-11356 removed prefixes from region names used by > Hibernate. > > I also noticed that entity regions are unprefixed and the package > name is removed. > > > Actually, the package names are being included in the region names. > The test I was looking at did not have a package. > > > I've created 2 issues: > HHH-12599 to add Javadoc to make it clear that region names are > unprefixed; > HHH-12598 to add the package onto entity region names. > > > I've rejected HHH-12598. > > > Should I create an ISPN issue for hibernate-infinispan (or > whatever it's referred to now) to add the prefixes? > I take it as confirmed that RegionFactory is supposed to do this. Created https://issues.jboss.org/browse/ISPN-9174 R. > > On Thu, May 17, 2018 at 4:55 AM, Sanne Grinovero > <sa...@hibernate.org <mailto:sa...@hibernate.org>> wrote: > > On 17 May 2018 at 12:09, Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > I basically agree with Sanne here that having the prefix > isolated opens > > space for performance improvements, though if certain call > is prefixed, > > RegionFactory can always drop that prefix. The important > thing is to mention > > if the region name is prefixed or not in javadocs. I would > also prefer if > > *all* region names that RegionFactory is supposed to access > followed the > > same strategy (prefixed/not-prefixed). > > > > 5.1 included method > > > > QueryResultsRegion buildQueryResultsRegion(String > regionName, Properties > > properties) > > > > where StandardQueryCache did the prefix for us, the change > in 5.3 to > > > > QueryResultsRegion buildQueryResultsRegion(String > regionName, > > SessionFactoryImplementor sessionFactory) > > > > does not indicate that the prefix won't be there and the > javadoc is missing > > completely. > > > > Also, DomainDataRegionConfig.getRegionName() has no javadoc and > > RegionFactory does not add the prefix. > > > > @Steve could you please amend the javadoc and confirm if RF > should prefix > > region names? > > > > @Sanne while cache manager per deployment is an obvious way > to distinguish > > regions@deployments, CM holds some heavy resources (e.g. > threads) - so I > > while we are supposed to scale number of caches up to > thousands (and we've > > fixed some problems that arise when you have for example ~3k > caches), ATM > > you're not supposed to scale CMs. And I don't think that > we'll focus in this > > direction since the bright future lies in microservices :) > > Right, good points. It's a possible optimization I'd like to see > eventually but it's not trivial to get there yet. > > Yet assuming a microservices scenario, this does become trivial to > benefit from: the container knows there's a single deployment, a > single CM. So no need to isolate them at all, just need to > possibly > isolate multiple PUs defined in the same service. > > So it will be easy to run hundreds of isolated microservices > on the > same physical CPU and kill each other via process contention :P > > > > > Radim > > > > > > On 05/17/2018 12:23 PM, Sanne Grinovero wrote: > >> > >> On 17 May 2018 at 11:11, Sanne Grinovero > <sa...@hibernate.org <mailto:sa...@hibernate.org>> wrote: > >>> > >>> I think this is the RegionFactory's responsibility, as > Hibernate ORM > >>> alone can't know if this is necessary. > >>> > >>> Prefixing is one of many options to isolate caches; a > Cache technology > >>> might wish to use a different approach by implementing a > custom > >>> `org.hibernate.cache.spi.CacheKeysFactory`. > >> > >> Hum, I
Re: [hibernate-dev] Why hibernate-jcache is not enough (for Infinispan)
luggable. > > P.S. JTA is a spec. JTA components are accessible from a "well known > location" (aka, easily accessible by anyone). So I really just do not > get your argument that `hibernate-jcache` + `infinispan` somehow loses > the ability to leverage JTA. And btw this was absolutely not the > intent with `CacheTransactionSynchronization`, which was instead > intended to allow unified processing of "cache events" at the "end of > a transaction" (as known to Hibernate) regardless of whether JTA or > straight JDBC transactions are used. I have complained about JCache + JTA being undefined, nothing about leveraging JTA in our impl. > > P.S.S. I totally agree with Sanne. The cache should be as correct as > possible, however it is *always* possible to simply evict a piece of > data from the cache to avoid conflicts. The database is *always* the > "truth of the system". This in in fact exactly the principle that the > collection cache works - any changes to that collection simply > invalidate (evict) the data from the cache. When you simply evict a piece of data from the cache you can't be sure that that piece won't end up in there right away because another concurrent request holding stale data hits the cache again. R. > > > On Thu, Mar 29, 2018 at 3:03 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > Hi Steve, > > on HipChat you've asked why hibernate-jcache + Infinispan's > implementation of JCache is not enough. It ultimately boils down to > > 1. performance > 2. correctness > > where (2) can be fine with some providers but then (1) suffers. > Infinispan offers transactional mode, but it's not serializable (gosh, > sometimes it's even read_uncommitted) and has its quirks. The > performance can't be as good as with non-tx mode, too. That's why the > native transactional caches support will be dropped in 5.3 and we'll > just emit a warning to update their configuration (and continue with > non-tx config). > > As a demonstration of this we can use the putFromLoad. If you > implement > this as a (ideal serializable) transactional cache putIfAbsent, the > provider must either > a) lock the key (pessimistic approach) - but we don't want to block > other nodes removing data from the cache (on write) or putFromLoading > (on read!) > b) resolve conflicts when the transaction is committing: you > figure out > that there are two concurrent updates and rollback one of the > transactions - that's not acceptable to us either, as any failure in > cache should not affect DB transaction. And there's a risk of blocking > between the 2 phases of commit, too. > > Theoretically you could just wipe any modified data on conflict - I > don't know if anyone does that, 'drop everything and proceed with > commit' is not something I'd expect from a general-purpose (NoSQL) > DB. I > recall Alex's JCache implementation (for 5.2) storing some 'lock' > objects in the cache, and you probably don't want to wipe those. > > Interaction with evictAll/removeAll could be also problematic: not > sure > about the other providers but Infinispan's clear() operation is > non-transactional even on tx cache (since Infinispan 7 or so) because > it's impractical to resolve all conflicts. I don't know details how > others provide that operation but there may be a hidden problem. > > Last but not least, you assume that the provider is transactional > and it > provides JCache interface. JCache does not define interaction with > JTA, > because it was hard to get agreement on non-tx behaviour (why did it > take 13 years to complete the JSR?) and it would be even harder > for JTA. > So what you expect is just your extrapolation or wishful thinking, and > it's up to integrators to verify that the unwritten contract is > fulfilled within the scope of hibernate-jcache module use. Not > that SPI > implementors would be in a better position, but at least we are aware > that (for us) it's not enough to implement those 3 classes and > job's done. > > Of course the correctness aspect may be ignored with 'it's just a > cache' > implying 'users expect stale/uncommitted data' as Sanne (who is much > closer to the customers than me) keeps repeating. However this is not > what 2LC promises as I understand it: the same results as DB would do. > > I am really grateful that in 5.3 you've provided the > CacheTransactionSynchronization that will help us boost (1) even > f
[hibernate-dev] Why hibernate-jcache is not enough (for Infinispan)
Hi Steve, on HipChat you've asked why hibernate-jcache + Infinispan's implementation of JCache is not enough. It ultimately boils down to 1. performance 2. correctness where (2) can be fine with some providers but then (1) suffers. Infinispan offers transactional mode, but it's not serializable (gosh, sometimes it's even read_uncommitted) and has its quirks. The performance can't be as good as with non-tx mode, too. That's why the native transactional caches support will be dropped in 5.3 and we'll just emit a warning to update their configuration (and continue with non-tx config). As a demonstration of this we can use the putFromLoad. If you implement this as a (ideal serializable) transactional cache putIfAbsent, the provider must either a) lock the key (pessimistic approach) - but we don't want to block other nodes removing data from the cache (on write) or putFromLoading (on read!) b) resolve conflicts when the transaction is committing: you figure out that there are two concurrent updates and rollback one of the transactions - that's not acceptable to us either, as any failure in cache should not affect DB transaction. And there's a risk of blocking between the 2 phases of commit, too. Theoretically you could just wipe any modified data on conflict - I don't know if anyone does that, 'drop everything and proceed with commit' is not something I'd expect from a general-purpose (NoSQL) DB. I recall Alex's JCache implementation (for 5.2) storing some 'lock' objects in the cache, and you probably don't want to wipe those. Interaction with evictAll/removeAll could be also problematic: not sure about the other providers but Infinispan's clear() operation is non-transactional even on tx cache (since Infinispan 7 or so) because it's impractical to resolve all conflicts. I don't know details how others provide that operation but there may be a hidden problem. Last but not least, you assume that the provider is transactional and it provides JCache interface. JCache does not define interaction with JTA, because it was hard to get agreement on non-tx behaviour (why did it take 13 years to complete the JSR?) and it would be even harder for JTA. So what you expect is just your extrapolation or wishful thinking, and it's up to integrators to verify that the unwritten contract is fulfilled within the scope of hibernate-jcache module use. Not that SPI implementors would be in a better position, but at least we are aware that (for us) it's not enough to implement those 3 classes and job's done. Of course the correctness aspect may be ignored with 'it's just a cache' implying 'users expect stale/uncommitted data' as Sanne (who is much closer to the customers than me) keeps repeating. However this is not what 2LC promises as I understand it: the same results as DB would do. I am really grateful that in 5.3 you've provided the CacheTransactionSynchronization that will help us boost (1) even further by allowing us to execute all operations in parallel. And it's good that you've made the SPI more expressive with the intent; there'll be a bunch of TODOs in the 5.3 implementation to cover use cases that were not handled in previous versions but now are obvious. Cheers Radim -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Async JDBC proposal
Emmanuel, is this a correct link? That one is ORM 5.1 vs. 5.3 binary compatibility analysis... I have seen [1] posted earlier today on; is this what you have in mind? I am very happy that JDBC will have an async variant; we have some plans for asynchronous persistence in Infinispan so this might would be definitely useful to our JDBC stores. I only wish there'd be an async thread-independent JTA next, too. Radim [1] https://blogs.oracle.com/java/jdbc-next%3a-a-new-asynchronous-api-for-connecting-to-a-database On 03/20/2018 12:01 PM, Emmanuel Bernard wrote: > Nothing really new for people at the edge of news but a nice > presentation showing how async JDBC will likely work > https://docs.google.com/document/d/1jH0znbYwgvGKHC-110zcjRaXLllBsvRKw-pdHrMzRzw > > Emmanuel > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] 5.3 - hibernate-ehcache
On 03/15/2018 02:56 PM, Steve Ebersole wrote: > Inline > > On Thu, Mar 15, 2018 at 8:44 AM Sanne Grinovero <sa...@hibernate.org> wrote: > >> On 15 March 2018 at 13:04, Steve Ebersole <st...@hibernate.org> wrote: >>> Given the changes to the second-level cache SPI, I wonder if we want to >>> take that as an opportunity to consider dropping the `hibernate-ehcache` >>> module? >>> >>> There are a few reasons I am considering this... >>> >>> 1. We have the JCache integration in place and users can use Ehcache >>> via that support already. >>> 2. This is really the same discussion we had wrt Spring Cache[1]. IMO >>> the answer should be consistent. We moved hibernate-infinispan for >> many of >>> these same reasons... >>> 3. A large part of the argument for keeping hibernate-ehcache was that >>> it is already in place and therefore did not require much effort to >>> support/maintain. However, given the new SPI there is actually a >> pretty >>> big effort to adapt hibernate-ehcache to those new SPIs. So far I am >> the >>> one doing that - which in and of itself is fine since I am the one who >>> changed the SPIs ;) But it makes me think it is far less of a >> maintenance >>> effort just to drop it and support just hibernate-ehcache. >> I guess you meant "support just JCache" ? >> > Oops, yes :) > > > +1 since the new SPI is different. I do wonder if the JCache API will >> be a straitjacket for futher performance improvements, but I guess we >> could re-insert this module if there's more concrete elements? At >> least any work due to SPI changes could be deferred. >> > For sure part of the new SPI design was to allow the cache providers to > make some performance decisions up front. Any caching libraries > integrating with Hibernate via hibernate-jcache would definitely lose that > opportunity (hibernate-jcache would have to make generalized decisions). > However, such providers are welcome to implement and provide their own > RegionFactory that makes these kind of decisions. That flexibility is not > going away - all that goes away is the specific adaptations of our caching > SPI to individual caching libraries. > > > We'll still be testing with Ehcache - over JCache - I hope? > Yeah, we'd probably use Ehcache as the provider for hibernate-jcache > testing. Ideally we'd set up a "matrix" style testing to be able to plug > in different providers - possibly even testing against both ehcache and > spring-cache. infinispan-hibernate-cache would not fit this model as it > does not integrate via the JCache abstraction, though we definitely want to > make sure that integration is tested as well. Before Infinispan provides native implementation to 5.3/6.0 SPIs, JCache is a good way to bridge the gap. So if you're setting up a matrix testing, adding Infinispan JCache provider would give us some confidence that we can recommend that bridge in the meantime. R. > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Hiberante OGM Infinispan Remote dialect - Execute the command queue as a single Infispan Remote Server Task
On 02/06/2018 02:55 PM, Sanne Grinovero wrote: > Thanks for starting this, great ideas. > > We should start exploring such options but I am thinking of some more > limitations that we'll have to overcome. To keep the discussion from > getting hopelessly complex, let's try to clarify the purpose: > > The goal is to solve two problems which are strongly related: ># performance / efficiency ># better ACIDity of "transactions" > > # Efficiency > Radim makes a great point, it would be nice to be able to address the > optimal node for any "task" we might want to perform. > JIRA: > - https://issues.jboss.org/browse/ISPN-8770 > > However while we wait for the Infinispan team to provide such > refinements, I think we should go ahead with this plan already, not > least for the other reasons, but also to be able to eventually provide > quick feedback on such improvements. > > Clearly this means we'll incur (possibly?) a performance hit when > running a simple task - such as writing a single key - but it > shouldn't matter much for more complex operations: especially with a > small sized cluster chances are high to hit a node owning at least one > of these. > > For the moment our client could detect if it's preferrable to run a > task or a "simple op"; essentially we could implement some of > ISPN-8770 in our Dialect but rather than making this too complex on > our side I'd contribute to the Infinispan improvement first. > > # "one transaction, one task" ? > > Just to clarify, we can't promise that the ORM won't need to flush > multiple times before the transaction is finally committed. Make sure > you read about auto-flush strategies, either in our docs or there are > many good articles about the topic. > In a nutshell, if there are dirty entities of some type A, and then a > query is run on that same type A, the ORM engine needs to flush the > pending changes on all As first to make sure the queries are accurate. > We could debate if this is appropriate and maybe we shouldn't do it on > all NoSQL stores, but it's certainly debatable as it's a change in > semantics compared to what people are used to. > > It's not entirely bad news though; while a single transaction might > actually be performed by multiple, independent although ordered "Task > RPC"s this will give us some opportunity to at least make sure that > operations which really need to be atomic will happen atomically; for > example when we delete an entity we can make sure that all references > are cleared up correctly. However it's not a real transaction still > and we need to be clear about that, no big deal as "real transactions" > will come soon in Infinispan Remote too. About 'real transactions': beware that current design for the Hot Rod transactions does not involve executing scripts withing the scope of the ongoing transaction. Since the server-side is already in master (for a while now), the protocol that lacks support for script execution in transaction is set. It was a 'start small' decision, so if this appears to be critical for your use cases please shout aloud on infinispan-dev list to influence that. > > # Usability > > Just a note on this. I hope we can try to work on this while expecting > that the Infinispan Server won't need to have specific code deployed > which depends on the specific application, such as the object model. > If it helps we can think of an "OGM extensions" to be included with > the Infinispan Server, but the code we include would then need to be > able to work with any schema. Ideally, even with multiple versions of > OGM, so I'd hope that we can design the models of each task we'll need > as Protobuf Entities. I hope you won't end up with OGM generating javascript to be run on the server :) Radim > > Fabio, do you want to start working on such a POC? > > Thanks, > Sanne > > > > > > On 5 February 2018 at 15:05, Fabio Massimo Ercoli <ferc...@redhat.com> wrote: >> Hi Radim, >> >> thank you for the alternative, it is very interesting. >> I've to study deeply the feature :P >> >> Fabio >> >> >> On Mon, Feb 5, 2018 at 9:53 AM, Radim Vansa <rva...@redhat.com> wrote: >> >>> Hi Fabio, >>> >>> thinking about the cons, Hot Rod can intelligently pick the server where >>> should a given key reside, to reduce the number of hops (therefore, >>> latency). RemoteCache does not expose any way to route according to some >>> key; feel free to file a feature request in Infinispan JIRA. >>> >>> Note that in order to reduce the number of round-trips, Infinispan 9.2 >>> will su
Re: [hibernate-dev] Hiberante OGM Infinispan Remote dialect - Execute the command queue as a single Infispan Remote Server Task
Hi Fabio, thinking about the cons, Hot Rod can intelligently pick the server where should a given key reside, to reduce the number of hops (therefore, latency). RemoteCache does not expose any way to route according to some key; feel free to file a feature request in Infinispan JIRA. Note that in order to reduce the number of round-trips, Infinispan 9.2 will support true-async operations: Previously the putAsync et all just moved the execution to different thread, since 9.2.0.CR2 it sends the request to the wire right await and the response is received through a CompletableFuture. At this moment multiple requests will need a distinct TCP connection for each concurrent operation, but this limitation is likely to be removed in 9.3. The goal is to let you write the batch down one-by-one using async API and just wait for all the operations to complete. I know this won't help for all the types of operation (a lack of client-side API sometimes forces OGM to use get() + CAS in a loop). I am not trying to talk you out of the remote execution API, just spreading news about the emerging alternatives. Radim On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote: > I'm Fabio, nice to meet you. > > Speaking of the current implementation of the Infinispan remote dialect of > Hibernate OGM, working on issue OGM-1206 and talking with Davide I noticed > that the unit of work commands are executed (flushed to datastore) at the > end of the transaction itself. > In particular I noticed that the commands are stored in a transaction > scoped object of type org.hibernate.ogm.dialect.batch.spi.OperationsQueue. > > Instead of perfom one remote invocation for each command in the method > org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch > maybe we could invoke a proper Infispan Remote Server Task to execute the > command queue on server side as a bulk operation. > > Moving the execution of the server-side command list (Infinispan) we would > have the advantage of reducing remote interactions. Moreover and above all > the execution of the command queue would be a transactional work unit, on > which could be apply a Repeteable Read isolation level, for instance. > > The solution would not solve the need for an XA client instead, but it > could be adopted by customers interested in local transactions. > > What do you think about it? > Can I open a Jira issue? > > Fabio > -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SynchronizationRegistry: expose read (lookup) operations?
t;>> such
>>> > an abstraction : SynchronizationRegistry
>>> >
>>> > On Wed, Oct 25, 2017 at 10:22 AM Steve Ebersole
>>> <st...@hibernate.org <mailto:st...@hibernate.org>
>>> > <mailto:st...@hibernate.org <mailto:st...@hibernate.org>>>
>>> wrote:
>>> >
>>> > Yes that would work for me, but thinking about the
>>> > implementation it
>>> > implies you'd need to hold on to both a Set and a
>>> Map, and
>>> > then we'd
>>> > be exposed to silly usage like people adding the same
>>> > synchronization
>>> > twice in two different ways?
>>> >
>>> >
>>> > Does it? Nothing in the SPI requires us to store things
>>> in any
>>> > specific way. E.g. we can keep just a Map - when we are
>>> passed
>>> > a KeyableSynchronization we'd use that key, when we are
>>> passed a
>>> > non-KeyableSynchronization Synchronization we'd generate
>>> one
>>> > ourselves.
>>> >
>>> > And we cant stop people from every conceivable "silly
>>> usage".
>>> > At some point we are professional developers and should
>>> be able
>>> > to do the non-silly things ;)
>>> >
>>> > And as far as your "register the thing twice" worry...
>>> > rhetorically, what stops them from calling:
>>> >
>>> > reg.register( "abc", MySync.INSTANCE )
>>> > reg.register( "123", MySync.INSTANCE )
>>> >
>>> > Nothing.
>>> >
>>> >
>>> > I'd rather expose a single consistent way: having to
>>> make up
>>> > an id
>>> > doesn't seem too inconvenient considering it's an SPI.
>>> >
>>> >
>>> > Well, again, I don't see how KeyableSynchronization is a
>>> > "inconsistent" approach. In fact out of the 2, it is my
>>> preferance.
>>> >
>>>
>>> --
>>> Registered in England and Wales under Company Registration No.
>>> 03798903
>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
>>>
>> --
>> Registered in England and Wales under Company Registration No. 03798903
>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
--
Radim Vansa <rva...@redhat.com>
JBoss Performance Team
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SynchronizationRegistry: expose read (lookup) operations?
I had the same issue in 2LC and we ended up passing CacheTransactionContext to all SPI calls (for ORM6). In case of Infinispan impl this will be a stateful object, probably implementing the Synchronization iface as well (to reduce object count), and 2LC will store all the beforeCompletion/afterCompletion work in there. Currently we can workaround by registering multiple synchronizations but as the RPCs can be done asynchronously doing it in parallel will reduce the latency. The only drawback is that JTA's suspend and resume cannot be honored properly but I've been told that the rest of ORM is not working perfectly either when you try to run two concurrent transactions using single Session. Radim On 10/25/2017 03:32 PM, Sanne Grinovero wrote: > Hi Steve, > > do you think it would be sensible for me to explore introducing some > kind of synchronization lookup method on > org.hibernate.resource.transaction.spi.SynchronizationRegistry ? > > Today it only exposes a `registerSynchronization` method, which we use > extensively, but then we also have quite some complexity in the Search > code caused by the fact that we can't look the synchronizations up in > a later phase. > Essentially our Synchronization is stateful and we need to update it later. > > I'd love to propose a change for ORM6 so allow registering such things > under some kind of id (a string?) so that one can look them back. > > current SPI: > > public void registerSynchronization(Synchronization synchronization) > > temptative proposal (didn't try it yet..): > > public void registerSynchronization(String id, Synchronization > synchronization); > > public void Synchronization getSynchronization(String id); > > > does it sound reasonable in principle? > > This would imply other users should make up an id unique for their use > case. Alternatively I could live with a Class used as an id, or we > could have the new methods in addition to the existing method for > people not interested in looking things up. > > thanks, > Sanne > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Testing DB lock timeouts
Hi, since hibernate-infinispan testsuite has been set on by default, recently I've set myself to improve the execution time which is several minutes due to various sleeps and timeouts. Many of the tests test concurrency issues, and that often involves issuing two writes to single table/row in DB. In H2, this results in waiting 10 seconds (as configured default lock timeout), and since the tests are executed sequentially, the testsuite takes much longer than it should. Obvious workaround is reducing this timeout to, say, 100 ms, but this could lead to a) false positives and b) those 100 ms add up and with over thousand of tests (for various configurations), this could be minutes anyway. Q: is there any infrastructure in testsuite to hook into the DB, assert that it's waiting in lock and let the thread time out if everything is as expected? Radim -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Broken nightly builds
Hi, seems something went wrong with HHH-11083 integration and I've broken the nightly builds. Fix is in [1], please integrate asap. Radim [1] https://github.com/hibernate/hibernate-orm/pull/1647 -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Change in DB lock acquisition in ORM?
Aha, that explains why the test take much longer. I was just about to look this up. I'll see if I can shorten this for tests where we expect such situations. Nevertheless, the test seems to be failing because it used to throw org.hibernate. PessimisticLockException and now it is throwing javax.persistence.PessimisticLockException (the hibernate exception is provided as cause). Radim On 11/21/2016 12:15 PM, Sanne Grinovero wrote: > Hi Radim, > I was wondering the same; I am not sure yet if it's related, but > noticed that the JDBC connection URL changed from > > jdbc:h2:mem:db1;DB_CLOSE_DELAY=-1 > > to > > jdbc:h2:mem:db1;DB_CLOSE_DELAY=-1;LOCK_TIMEOUT=1 > > I'm running the testsuite now reverting that change locally, but I > can't say yet as the tests take ages to run... > > If anyone knows of other locking related changes please let us know :) > > Thanks, > Sanne > > > On 21 November 2016 at 10:04, Radim Vansa <rva...@redhat.com> wrote: >> Hi, >> >> I am investigating the failures in hibernate-infinispan testsuite and >> I've found that [1] is failing as this uses two threads that both do >> >> 1. load entity >> 2. delete entity >> 3. flush session >> 4. commit tx >> >> on the same entity. There is a synchronization blocking the commit until >> both threads flush, and since the first thread holds a H2 DB lock on the >> entity, the other thread is blocked doing the flush on this lock. >> >> It makes sense to me, but I wonder why did the test work in the past. >> Was there a change in some locking defaults (optimistic/pessimistic) or >> was there anything delegating the lock acquisition to the commit instead >> of flush? The test works on 5.0.10. >> >> Radim >> >> [1] >> https://github.com/hibernate/hibernate-orm/blob/master/hibernate-infinispan/src/test/java/org/hibernate/test/cache/infinispan/functional/TombstoneTest.java#L37 >> >> >> -- >> Radim Vansa <rva...@redhat.com> >> JBoss Performance Team >> >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Change in DB lock acquisition in ORM?
Hi, I am investigating the failures in hibernate-infinispan testsuite and I've found that [1] is failing as this uses two threads that both do 1. load entity 2. delete entity 3. flush session 4. commit tx on the same entity. There is a synchronization blocking the commit until both threads flush, and since the first thread holds a H2 DB lock on the entity, the other thread is blocked doing the flush on this lock. It makes sense to me, but I wonder why did the test work in the past. Was there a change in some locking defaults (optimistic/pessimistic) or was there anything delegating the lock acquisition to the commit instead of flush? The test works on 5.0.10. Radim [1] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-infinispan/src/test/java/org/hibernate/test/cache/infinispan/functional/TombstoneTest.java#L37 -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [hibernate-announce] Releasing Hibernate OGM 5.1.0.Beta1: now with Hot Rod support!
I understand that you don't want to scare users off, but I would mention that at least in the section about deciding between embedded and Hot Rod: "When connecting to an /Infinispan Server/ over the /Hot Rod client/, the architecture is similar to having Hibernate connect to traditional database: the data is stored on the /Infinispan Server/ nodes, and Hibernate OGM uses a client with a pool of TCP connections to talk to the server." Traditional databases are transactional, so I would put a notice about no-transactions here, with a link to "9.3.4. Storage Principles of the Infinispan Remote dataprovider" Most people probably understand what *is* the loss of referential integrity, but 9.3.4 is not really specific *when* does this happen. As you speak about "interrupting Hibernate OGM", I would assume that this is limited to a case when some operation fails (due to network breakage), but it's not clear that this can happen even with successful concurrent operations. Therefore, an example that can lead to broken integrity could be useful, along with information about concurrent operations that are safe (basically saying that if each session uses distinct set of entities, you are safe). Is there any option to detect (and fix) the problems? (by a batch job...?) Radim On 11/08/2016 01:53 PM, Sanne Grinovero wrote: > Thanks Radim! > > the blog attempts to be short, I mention the Referential Integrity > problem in the reference documentation: > > " > Referential integrity > While we can use relations based on foreign keys, Infinispan has no > notion of referential integrity. Hibernate is able to maintain the > integrity as it won’t "forget" stale references, > but since the > storage doesn’t support transactions either it is possible to > interrupt Hibernate OGM during such maintenance and introduce breaks > of integrity. > " > - > https://docs.jboss.org/hibernate/ogm/5.1/reference/en-US/html_single/#storage_principles_of_the_infinispan_remote_dataprovider > > It's not explicitly listed in the known limitations, as I consider it > part of the "there's no transactions" point here: > - > https://docs.jboss.org/hibernate/ogm/5.1/reference/en-US/html_single/#known_limitations_future_improvements > > You think that's enough? It's a though one, as I want to be clear > about the limitations but w/o scaring people off by repeating > limitations too many times. > There are also various big highlighted baloons mentioning: > > " > Caution > The Hibernate OGM support for Infinispan Remote is considered experimental. " > > Happy to clarify the docs as needed. > > Thanks! > Sanne > > > On 8 November 2016 at 17:50, Radim Vansa <rva...@redhat.com> wrote: >> Wouldn't it be worth mentioning the lack of referential integrity among >> the limitations? >> >> Anyway, thumbs up! >> >> Radim >> >> On 11/08/2016 12:16 PM, Sanne Grinovero wrote: >>> Hello everyone, >>> >>> we can finally announce that Hibernate OGM 5.1.0.Beta1 is released, >>> and now includes support for Infinispan Server, alias Hot Rod, also >>> known as Infinispan Remote .. >>> >>> - http://in.relation.to/2016/11/08/hibernate-ogm-with-hotrod-support >>> >>> We also released Hibernate OGM 5.0.3.Final >>> >>> Kind Regards, >>> Sanne >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> -- >> Radim Vansa <rva...@redhat.com> >> JBoss Performance Team >> >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [hibernate-announce] Releasing Hibernate OGM 5.1.0.Beta1: now with Hot Rod support!
Wouldn't it be worth mentioning the lack of referential integrity among the limitations? Anyway, thumbs up! Radim On 11/08/2016 12:16 PM, Sanne Grinovero wrote: > Hello everyone, > > we can finally announce that Hibernate OGM 5.1.0.Beta1 is released, > and now includes support for Infinispan Server, alias Hot Rod, also > known as Infinispan Remote .. > > - http://in.relation.to/2016/11/08/hibernate-ogm-with-hotrod-support > > We also released Hibernate OGM 5.0.3.Final > > Kind Regards, > Sanne > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Is org.hibernate.cache.spi.OptimisticCacheSource#getVersionComparator obsolete?
On 06/18/2016 06:14 PM, Steve Ebersole wrote: > TIMESTAMP is a deprecated synonym for the TSQL rownumber datatype, at least > on SQL Server. Not sure specifically about Sybase. So I'll use "rownumber" > here... > > As we discussed in HipChat yesterday OptimisticCacheSource was added a long > time ago to facilitate leveraging JBossCache's MVCC support for versioned > data. The idea was to allow JBossCache to reuse a versioned entity's > version as its MVCC version via the comparator. > > JBossCache is of course no more, now morphed into Infinispan. To my > knowledge, Infinispan does not have the same MVCC support if any. Either > way, hibernate-infinispan does not support MVCC in the Infinispan cache by > leveraging the entity version. Radim, Galder - is this something we want > to consider? Versions can remove the need to lock some entries. At this point the comparison is used only in the nonstrict read-write access strategy, and it does not use Infinispan MVCC (I can't recall exactly why, probably it was more convenient to have the version in entry value than in metadata, as there's really no OOTB support). One reason this is not leveraged more is that it imposes limitation on the types of entries; though it can be transparent to the user, the effort is still reduced by the fact that it's only a portion of use cases. > > Depending on the version (and I forget the exact version this changed) the > role played by OptimisticCacheSource is actually now handled by > org.hibernate.cache.spi.CacheDataDescription. > So that is one of the reasons you do not find any uses of > OptimisticCacheSource#getVersionComparator. Do a usage search for > CacheDataDescription#getVersionComparator instead. > > Same question, different contract. Never seen OptimisticCacheSource, CDD was there at hand. Infinispan does not need to keep that in API. > > Some of the cache providers do leverage this version comparator (via > CacheDataDescription#getVersionComparator) for read-write locking. So > really the question is whether we want to allow the combination of: > > > 1. these cache providers with those access strategies using the version > comparator > 2. versions based on TSQL rowversion datatype. > > Aside from a general dislike of versions based on TSQL rowversion datatype > (for many reasons), there is no fundamental reason to not allow it as a > comparator (provided we find the right comparison algorithm). If you have > found the secret incantation for writing a proper Java Comparator based on > the TSQL rowversion datatype, then I think we should just implement that. If DB has that info, it's fair that Cache can have that, too. I would keep the support, though it does not have to have 100% coverage on the possible datatypes (if there are 2% that are too complex to implement). Radim > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [ORM] EhCache 3 and JCache
bernate.org> >>>>> wrote: >>>>> >>>>>> What we had decided before during a discussion with myself, Alex Snaps, >>>>>> Radim and Sanne was to develop a JCache-based L2 case module and that >>>>>> Ehcache 3 would be supported through that mechanism. We'd continue >>>>>> support >>>>>> for the current Ehcahce 2 based hibernate-ehcache module for a short >>>>>> period >>>>>> of time. >>>>>> >>>>>> On Fri, May 20, 2016 at 7:48 AM Guillaume Smet < >>>>>> guillaume.s...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On Fri, May 20, 2016 at 9:54 AM, Emmanuel Bernard < >>>>>> emman...@hibernate.org> >>>>>>> wrote: >>>>>>> >>>>>>>> 3. change the Ehcache module and move it from v2 to v3 >>>>>>>> >>>>>>> Please don't do that. >>>>>>> >>>>>>> I'm pretty sure users will need to test Ehcache 3 before going live >>>>>> and it >>>>>>> shouldn't be tied to an Hibernate upgrade. Cache is a very sensible >>>>>> subject >>>>>>> and I'm pretty sure moving to Ehcache 3 will come with its >>>>>> challenges. We >>>>>>> should at least have 1 or 2 versions allowing both Ehcache 2 and 3. >>>>>>> >>>>>>> Moreover, last time I checked, there was no Jgroups replication yet >>>>>> in >>>>>>> Ehcache 3 (or it's not documented). >>>>>>> >>>>>>> -- >>>>>>> Guillaume >>>>>>> ___ >>>>>>> hibernate-dev mailing list >>>>>>> hibernate-dev@lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>>>> >>>>>> ___ >>>>>> hibernate-dev mailing list >>>>>> hibernate-dev@lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>>> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [ORM] EhCache 3 and JCache
Will there ever be a possibility to use Ehcache in clustered environment, with opensource-only? Radim On 05/20/2016 10:56 AM, Louis Jacomet wrote: > Hi all, > > Thanks Emmanuel for the email and no worry on the delay, the fact that I > did not act sooner means it was pretty busy on our side too. But the timing > is right, so let's move forward! > > Chris Dennis (added in CC) was the last one looking into the code and > bringing it up to date with Hibernate master. You can see the current state > of a JCache integration module in his fork - > https://github.com/chrisdennis/hibernate-orm/commits/jsr107 > > It might be helpful to get feedback on this already. > > With regards to the options for having a full blown Ehcache 3 support, I > believe dropping Ehcache 2 in a minor release makes little sense given how > much different (and incompatible) it is. > So if we want to benefit from the transactional support in Ehcache 3 (as > one example of a feature not in JCache), I believe options 1 and 2 are the > open ones. > Of course, depending on the approach picked, we could discuss deprecating > Ehcache 2.x support for Hibernate 6. > > Now I think Sane had a conversation back in JavaOne 2015 with Alex Snaps > but not sure if that was one of the covered topics. > > For sure, in order of priority for us: > 1. Add JCache support to Hibernate so that Ehcache 3 but also others > providers can act as 2nd level caches. > 2. Extended Ehcache 3 support - exact solution to be defined. > > I will definitely start looking into the resources provided by Vlad, thanks. > > Regards, > Louis > > > On Fri, May 20, 2016 at 10:01 AM Vlad Mihalcea <mihalcea.v...@gmail.com> > wrote: > >> Hi Louis, >> >> The Caching section from the 5.0 docs is a good place to start to getting >> to know the basics of Hibernate 2nd level cache. >> >> If you want to get into lots of details, you can check my Hibernate Master >> Class: >> >> https://vladmihalcea.com/tutorials/hibernate/ >> >> There you can find a lot of resources on how each cache concurrency >> strategy works, about collection and query cache. >> If you have any questions, you can ask me anytime. >> >> Vlad >> >> On Fri, May 20, 2016 at 10:54 AM, Emmanuel Bernard <emman...@hibernate.org >>> wrote: >>> Hello guys, >>> >>> (I meant to send that almost a month ago, apologies Louis). >>> >>> I’ve met Louis Jacomet from Ehcache at Devoxx France and we discussed how >>> to restart the progress around JCache and EhCache 3 integration. >>> Louis is willing to step up to make these a reality but will need a bit >>> of help to ramp up knowledge wise. Let me list the key subjects. >>> >>> == JCache >>> >>> I think Sanne has the info somewhere in his brain, what is the status of >>> the current code? Any detail on what was missing? It would be nice to drive >>> this one home. >>> >>> == Ehcache 3 >>> >>> Ehcache 3 has been out recently and it would be nice to get an >>> integration. The API is very different. >>> There are a few options: >>> >>> 1. make the JCache support extensible to build upon it and use Ehcache >>> specific features when necessary or useful. Ehcache 3 is a JCache >>> implementation. >>> 2. create an new module dedicated to Ehcache 3 and go for it from scratch >>> basically >>> 3. change the Ehcache module and move it from v2 to v3 >>> >>> 1. might be the conceptual nicest but it’s unclear how doable that is >>> 2. vs 3. is about migration path for users and whether we consider it a >>> “breaking change” in a 5.x series >>> >>> == Doc and pointers >>> >>> As I said, Louis know Ehcache well but needs some input to step into the >>> magic world of second level caching in Hibernate ORM. Things like where is >>> the code, how to test it, is there a doc or an example of a well written >>> one etc. >>> >>> Can someone give him a hand? >>> >>> Emmanuel >>> >> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] ReadWrite 2LC "read uncommitted" data anomaly
Hi Vlad, I think that this change is safe (it uses only the supported flow of 2LC SPI calls), though it introduces a bit more overhead for the refresh. FYI, besides unit tests, there's CorrectnessTestCase [1] - I've used this one to find out most of flaws in Infinispan 2LC impl. Could be worth a try running this on EHCache, too. I've noticed some of the added tests explicitly use H2. Hibernate uses 1.3 for the testsuite - I was running tests with both 1.3 and 1.4 and the behaviour can differ a lot, since 1.4 can lock separate rows and 1.3 locks just whole tables (pardon me if I recall that incorrectly) - this can lead to different timing, and eventually to deadlock/test failure. Just to let you know that I had problems with these. Radim [1] https://github.com/hibernate/hibernate-orm/blob/master/hibernate-infinispan/src/test/java/org/hibernate/test/cache/infinispan/stress/CorrectnessTestCase.java On 04/06/2016 12:17 PM, Vlad Mihalcea wrote: > Hi Radim, > > I pushed this fix on master and 5.1, and I managed to add an EHCache > where the same behavior was replicated as well: > > https://github.com/hibernate/hibernate-orm/commit/cbdab9d87f05b4255c7930a32fe995f87f0f3e0b > > For Infinispan, I think it's better if you can investigate if this is > an issue or if the change does not break anything either (all > Infinispan tests ran fine, so hopefully this should not be the case). > > Thanks, > Vlad > > On Wed, Apr 6, 2016 at 12:58 PM, Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > On 04/05/2016 04:13 PM, Vlad Mihalcea wrote: > > Hi, > > > > I'd definitely fix it for the refresh operation, which does an > implicit > > cache eviction too. > > In this case, the proposed PR solution is fine since it simply > locks the > > entry right after it is evicted from the cache and releases the > lock after > > the transaction is ended. > > This way, we won't push an uncommitted entity into 2PL during > the two-phase > > loading phase that is triggered by the refresh operation. > > > > For sessionFactory.getCache.evictEntity/evictCollection, if > there is a > > current Session going on, we could propagate a > > CacheEvictEvent/CollectionCacheEvictEvent which can apply the > lock on that > > particular entity/collection, and we release it right after the > current > > transaction is ended, similar to what refresh should do as well. > > > > For every other use case, like evictAll/evictRegion, we should just > > document the behavior. > > > > I saw that Radim has added such a warning for Infinispan in the > new User > > Guide: > > > > read-write mode is supported on non-transactional > distributed/replicated > >> caches, however, eviction should not be used in this > configuration. Use of > >> eviction can lead to consistency issues. > > This is a different matter; in Infinispan 2LC impl you store locks and > entries either in two different caches (the entries cache is > invalidation, locks is local), or in single cache > (replicated/distributed). As we don't want to lose locks randomly, and > eviction picks entries unpredictably, its use is discouraged. > > I think that this issue does not apply to Infinispan with the > invalidation configuration, since evict/evictAll does not remove any > locks, and the lock blocks further updates (including putFromLoads) to > the entry in cache until the transaction commits. In case of > replicated/distributed cache, it seems that the evict is ignored after > update, but evictAll is not (that would qualify as a bug) - so after > evictAll you could observe the uncommitted read. Nevertheless I would > have to test this. > > Radim > > > > > Unfortunately, the EhCache documentation > > > > <http://www.ehcache.org/documentation/2.8/integrations/hibernate.html#read-write> > > makes a wrong assumption: > > > > read-write - Caches data that is sometimes updated while > maintaining the > >> semantics of “read committed” isolation level. > > > > Vlad > > > > On Tue, Apr 5, 2016 at 4:30 PM, Sanne Grinovero > <sa...@hibernate.org <mailto:sa...@hibernate.org>> wrote: > > > >> On 5 April 2016 at 14:11, Vlad Mihalcea > <mihalcea.v...@gmail.com <mailto:mihalcea.v...@gmail.com>> wrote: > >>> Hi, > >>> > >>> While reviewing th
Re: [hibernate-dev] ReadWrite 2LC "read uncommitted" data anomaly
On 04/06/2016 11:58 AM, Radim Vansa wrote: > On 04/05/2016 04:13 PM, Vlad Mihalcea wrote: >> Hi, >> >> I'd definitely fix it for the refresh operation, which does an implicit >> cache eviction too. >> In this case, the proposed PR solution is fine since it simply locks the >> entry right after it is evicted from the cache and releases the lock after >> the transaction is ended. >> This way, we won't push an uncommitted entity into 2PL during the two-phase >> loading phase that is triggered by the refresh operation. >> >> For sessionFactory.getCache.evictEntity/evictCollection, if there is a >> current Session going on, we could propagate a >> CacheEvictEvent/CollectionCacheEvictEvent which can apply the lock on that >> particular entity/collection, and we release it right after the current >> transaction is ended, similar to what refresh should do as well. >> >> For every other use case, like evictAll/evictRegion, we should just >> document the behavior. >> >> I saw that Radim has added such a warning for Infinispan in the new User >> Guide: >> >> read-write mode is supported on non-transactional distributed/replicated >>> caches, however, eviction should not be used in this configuration. Use of >>> eviction can lead to consistency issues. > This is a different matter; in Infinispan 2LC impl you store locks and > entries either in two different caches (the entries cache is > invalidation, locks is local), or in single cache > (replicated/distributed). As we don't want to lose locks randomly, and > eviction picks entries unpredictably, its use is discouraged. ... it's use is discouraged in the repl/dist mode as you can't allow eviction on entries but prohibit on locks with both of these in single cache. > > I think that this issue does not apply to Infinispan with the > invalidation configuration, since evict/evictAll does not remove any > locks, and the lock blocks further updates (including putFromLoads) to > the entry in cache until the transaction commits. In case of > replicated/distributed cache, it seems that the evict is ignored after > update, but evictAll is not (that would qualify as a bug) - so after > evictAll you could observe the uncommitted read. Nevertheless I would > have to test this. > > Radim > >> Unfortunately, the EhCache documentation >> <http://www.ehcache.org/documentation/2.8/integrations/hibernate.html#read-write> >>makes a wrong assumption: >> >> read-write - Caches data that is sometimes updated while maintaining the >>> semantics of “read committed” isolation level. >> Vlad >> >> On Tue, Apr 5, 2016 at 4:30 PM, Sanne Grinovero <sa...@hibernate.org> wrote: >> >>> On 5 April 2016 at 14:11, Vlad Mihalcea <mihalcea.v...@gmail.com> wrote: >>>> Hi, >>>> >>>> While reviewing the PR for this issue: >>>> >>>> https://hibernate.atlassian.net/browse/HHH-10649 >>>> >>>> I realized that the ReadWrite cache concurrency strategy has a flaw that >>>> permits "read uncommitted" anomalies. >>>> The RW cache concurrency strategy guards any modifications with Lock >>>> entries, as explained in this post that I wrote some time ago: >>>> >>>> >>> http://vladmihalcea.com/2015/05/25/how-does-hibernate-read_write-cacheconcurrencystrategy-work/ >>>> Every time we update/delete an entry, a Lock is put in the cache under >>> the >>>> entity key, and, this way, "read uncommitted" anomalies should be >>> prevented. >>>> The problem comes when entries are evicted either explicitly: >>>> >>>> session.getSessionFactory().getCache().evictEntity( CacheableItem.class, >>>> item1.getId() ); >>>> >>>> or implicitly: >>>> >>>> session.refresh( item1 ); >>> Good catch! >>> >>> I think this is caused as we generally don't expect the evict >>> operation to be controlled explicitly. >>> In my personal experience, I would use the evictAll method to nuke the >>> cache state after some significant operation, like restoring a >>> backup.. and no other Session would have been opened in the meantime. >>> I never used an explicit single-shot evict so I can't say what the use >>> case would be. >>> >>> But of course you're right that it might be used differently, or at >>> least such a limitation should be clear. >>> >>>> During eviction, the 2PL will remove the Lock entry, and if the use
Re: [hibernate-dev] ReadWrite 2LC "read uncommitted" data anomaly
ntries. >> I'm not sure I understood this part. Shouldn't it rather be allowed to >> delete everything, except any existing locks? >> Then rather than turn the remaining locks into locks, it would be >> enough to leave them. >> >>> For the evict method, this is not really a problem, but evictAll would >>> imply taking all entries and replacing them with Locks, and that might not >>> perform very well in a distributed-cache scenario. >>> >>> Ideally, lock entries would be stored separately than actual cached value >>> entries, and this problem would be fixed in a much cleaner fashion. >> I'd leave this as a detail to the Cache implementation, some might be >> able to perform some operation more efficiently. >> Probably a good idea to clarify this expectation on the javadocs of >> the SPI methods. >> >> Thanks, >> Sanne >> >> >>> Let me know what you think about this. >>> >>> Vlad >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Java 1.8-specific code in hibernate-infinispan
Btw., when does ORM plan to drop pre-8 support altogether? 6.0? Radim On 03/01/2016 10:18 PM, Steve Ebersole wrote: > Correct. hibernate-infinispan can use Java 8. As Sanne says, Infinispan > itself requires Java 8 so limiting hibernate-infinispan to > 8 really makes > no sense, > > On Tue, Mar 1, 2016 at 3:03 PM Sanne Grinovero <sa...@hibernate.org> wrote: > >> In general, probably yes. The Infinispan module is a bit special >> though, as Infinispan itself requires Java8 since Infinispan 8 so >> noone will be able to use those modules on previous Java versions.. so >> there's no point in avoiding pre-Java 8 code in there. >> >> On 1 March 2016 at 20:55, Gail Badner <gbad...@redhat.com> wrote: >>> I see a pull request on master for hibernate-infinispan that uses lambda >>> expressions. [1] >>> >>> There is a separate pull request for 5.0 that does not use them, so >> there's >>> no need to backport in this case. [2] >>> >>> In general, should we avoid using lambda expressions in master? >>> >>> Thanks, >>> Gail >>> >>> [1] https://github.com/hibernate/hibernate-orm/pull/1269/files >>> [2] https://github.com/hibernate/hibernate-orm/pull/1268/files >>> ___ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] 2LC docs
On 01/22/2016 05:26 PM, Steve Ebersole wrote: > On Fri, Jan 22, 2016 at 9:30 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > On 01/22/2016 03:11 PM, Steve Ebersole wrote: > > On Fri, Jan 22, 2016 at 7:21 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com> > > <mailto:rva...@redhat.com <mailto:rva...@redhat.com>>> wrote: > > > > Why should the strategy 'never be used if serializable > transaction > > isolation level is required'? What guarantees it gives, and what > > in ORM > > core depends on this? When I've asked the last time, Steve said > > that all modes but the > > > > nonstrict one require that the 2LC is absolutely transparent > > (consistency-wise), so you always get the same answer as if > you were > > directly talking to DB. > > > > > > I would guess this is talking about "serializable isolation" at the > > application layer. Yes extended across both the application and > > database. In our original implementations we had no L2 cache > > providers that would support serializable isolation. Does > > hibernate-infinispan? If we ask for a certain entry from the > cache in > > T1, T2 adds that entry and commits, and then we ask for it again > in T1 > > do we still see it as "not existing"? I'd highly doubt it, but > if it > > does then lets make note of that. > > No, without a transactional cache, it does not. Thanks for the > example. > But will the request get to 2LC, or will it be served already from > Session cache? > > > It won't work even with a transactional cache I believe. It won't work > with Infinispan e.g. I do not think. Hibernate does not keep reference > to "non-existing" entities. That's the only way the Session could > "serve" the fact that the first T1 lookup found nothing. Again, this > gets right back to that idea of consistency. Without L2 caching, in > this scenario with serializable isolation the database would return me > "no row" in both T1 SELECTs. Infinispan keeps 'transactional context' for the current transaction and stores all reads there, even if this is a null read. However, as I've checked the distribution code, it still does the remote lookup (which escapes the transaction) and the value could get there even with so-called repeatable reads. I'll check infinispan-dev why. > > > Does the ' you should ensure that the transaction is completed when > > `Session.close()` or `Session.disconnect()` is called' still > hold, or > > does the transactional rework in 5.0 somehow obsolete this info? > > > > > > I cannot say why this is discussed in a chapter on caching. > > Session#disconnect is largely deprecated (its main use case is > handled > > much more transparently now). IMO it's always a good idea to make > > sure a transaction against a resource is completed prior to closing > > that transaction. That's no different for a Hibernate Session > then it > > is for a JDBC Connection, etc. > > Did you meant 'commit the transaction before closing the session'? If > the Session.close() is called with tx open, will the transaction be > committed? But any way, this should be really the same as without 2LC. > > > I meant to say " make sure a transaction against a resource is > completed prior to closing that resource". Saying "complete the > transaction" != "commit the transaction". Completion might be either > commit or rollback. But the idea is that it is in a definitive state. > > Historically what a stranded transaction at the time of Session#close > meant depended on the JDBC driver. Most drivers rollback back on a > stranded transaction; Oracle has always been the notable exception as > they would commit a stranded transaction. But regardless in terms of > Session locks etc in the cache that would strand the locks as well iirc. > > In developing 5.0 and the new transaction handling I know we talked > about making this more deterministic, specifically always handling > this as if a rollback had been called. But to be honest, that's not > what I am seeing in the code. Andrea, do you remember? If not, we > should definitely add some tests for this to see what happens atm and > make sure its really what we want to have happen moving forward. > > > > Basically this passage is a poorly w
Re: [hibernate-dev] 2LC docs
On 01/25/2016 11:48 AM, Radim Vansa wrote: > On 01/22/2016 05:26 PM, Steve Ebersole wrote: >> On Fri, Jan 22, 2016 at 9:30 AM Radim Vansa <rva...@redhat.com >> <mailto:rva...@redhat.com>> wrote: >> >> On 01/22/2016 03:11 PM, Steve Ebersole wrote: >> > On Fri, Jan 22, 2016 at 7:21 AM Radim Vansa <rva...@redhat.com >> <mailto:rva...@redhat.com> >> > <mailto:rva...@redhat.com <mailto:rva...@redhat.com>>> wrote: >> > >> > Why should the strategy 'never be used if serializable >> transaction >> > isolation level is required'? What guarantees it gives, and >> what >> > in ORM >> > core depends on this? When I've asked the last time, Steve >> said >> > that all modes but the >> > >> > nonstrict one require that the 2LC is absolutely transparent >> > (consistency-wise), so you always get the same answer as if >> you were >> > directly talking to DB. >> > >> > >> > I would guess this is talking about "serializable isolation" at >> the >> > application layer. Yes extended across both the application and >> > database. In our original implementations we had no L2 cache >> > providers that would support serializable isolation. Does >> > hibernate-infinispan? If we ask for a certain entry from the >> cache in >> > T1, T2 adds that entry and commits, and then we ask for it again >> in T1 >> > do we still see it as "not existing"? I'd highly doubt it, but >> if it >> > does then lets make note of that. >> >> No, without a transactional cache, it does not. Thanks for the >> example. >> But will the request get to 2LC, or will it be served already from >> Session cache? >> >> >> It won't work even with a transactional cache I believe. It won't >> work with Infinispan e.g. I do not think. Hibernate does not keep >> reference to "non-existing" entities. That's the only way the >> Session could "serve" the fact that the first T1 lookup found >> nothing. Again, this gets right back to that idea of consistency. >> Without L2 caching, in this scenario with serializable isolation the >> database would return me "no row" in both T1 SELECTs. > > Infinispan keeps 'transactional context' for the current transaction > and stores all reads there, even if this is a null read. However, as > I've checked the distribution code, it still does the remote lookup > (which escapes the transaction) and the value could get there even > with so-called repeatable reads. I'll check infinispan-dev why. So it seems that Infinispan handles repeatable reads of null correctly. > >> >> > Does the ' you should ensure that the transaction is completed >> when >> > `Session.close()` or `Session.disconnect()` is called' still >> hold, or >> > does the transactional rework in 5.0 somehow obsolete this >> info? >> > >> > >> > I cannot say why this is discussed in a chapter on caching. >> > Session#disconnect is largely deprecated (its main use case is >> handled >> > much more transparently now). IMO it's always a good idea to make >> > sure a transaction against a resource is completed prior to >> closing >> > that transaction. That's no different for a Hibernate Session >> then it >> > is for a JDBC Connection, etc. >> >> Did you meant 'commit the transaction before closing the >> session'? If >> the Session.close() is called with tx open, will the transaction be >> committed? But any way, this should be really the same as without >> 2LC. >> >> >> I meant to say " make sure a transaction against a resource is >> completed prior to closing that resource". Saying "complete the >> transaction" != "commit the transaction". Completion might be either >> commit or rollback. But the idea is that it is in a definitive state. >> >> Historically what a stranded transaction at the time of Session#close >> meant depended on the JDBC driver. Most drivers rollback back on a >> stranded transaction; Oracle has always been the notable exception as >> they would commit a stranded transaction. But regardless in terms of >> Session locks etc in the cache that would
Re: [hibernate-dev] 2LC docs
I'd like to ask for some details on these (probably Steve): read-write:: If the application needs to update data, a read-write cache might be appropriate. This cache strategy should never be used if serializable transaction isolation level is required. If the cache is used in a JTA environment, you must specify the `hibernate.transaction.jta.platform` property. In other environments, you should ensure that the transaction is completed when `Session.close()` or `Session.disconnect()` is called. If you want to use this strategy in a cluster, you should ensure that the underlying cache implementation supports locking. nonstrict-read-write:: If the application only occasionally needs to update data (e.g. if it is extremely unlikely that two transactions would try to update the same item simultaneously) and strict transaction isolation is not required, a nonstrict-read-write cache might be appropriate. If the cache is used in a JTA environment, you must specify the `hibernate.transaction.jta.platform` property. In other environments, you should ensure that the transaction is completed when `Session.close()` or `Session.disconnect()` is called. Why should the strategy 'never be used if serializable transaction isolation level is required'? What guarantees it gives, and what in ORM core depends on this? When I've asked the last time, Steve said that all modes but the nonstrict one require that the 2LC is absolutely transparent (consistency-wise), so you always get the same answer as if you were directly talking to DB. I also don't really see what's the meaning of 'underlying cache implementation supports locking' - IMO it's implementation detail how the application achieves required level of consistency. The 'locking' term itself is a bit vague. Does the ' you should ensure that the transaction is completed when `Session.close()` or `Session.disconnect()` is called' still hold, or does the transactional rework in 5.0 somehow obsolete this info? Thanks Radim On 01/21/2016 04:34 PM, Radim Vansa wrote: > Seems much more comprehensive, though I can't find e.g. > 'Cache-provider/concurrency-strategy compatibility' table (already > out-of-date). Nevertheless, thanks! > > I'll try to find some timeslot for updating it early next week. > > Radim > > On 01/21/2016 04:17 PM, Vlad Mihalcea wrote: >> Hi Radim, >> >> I've just committed the updated version of the Caching chapter. >> To build it, you need to: >> >> 1. go to the documentation folder >> 2. run gradle rUG >> 3. The new user guide is located under: >> documentation/target/asciidoc/userguide/html/Hibernate_User_Guide.html >> >> Let me know what you think. >> >> Vlad >> >> On Wed, Jan 20, 2016 at 11:57 AM, Vlad Mihalcea >> <mihalcea.v...@gmail.com <mailto:mihalcea.v...@gmail.com>> wrote: >> >> Hi Radim, >> >> I'm now filling in the missing sections on the Caching chapter in >> the 5.1 User Guide. >> >> Vlad >> >> On Wed, Jan 20, 2016 at 11:32 AM, Radim Vansa <rva...@redhat.com >> <mailto:rva...@redhat.com>> wrote: >> >> I would say that the best source is 4.x docs, since the cited >> source is >> what I describe by 'close to nothing'. >> >> I understand that for 5.1 the transformation might be >> unfinished, but I >> was surprised by the same for 5.0 - missing information that's >> written, >> maybe just not formatted properly for asciiidoc. In such case, >> link to >> appropriate chapter in older docs would be useful. >> >> Radim >> >> On 01/19/2016 07:04 PM, Steve Ebersole wrote: >> > The docs are in a state of flux for 5.1. For now your best >> bet is the >> > source: >> >> documentation/src/main/asciidoc/userguide/chapters/caching/Caching.adoc >> > >> > On Fri, Jan 15, 2016 at 3:48 AM Radim Vansa >> <rva...@redhat.com <mailto:rva...@redhat.com> >> > <mailto:rva...@redhat.com <mailto:rva...@redhat.com>>> wrote: >> > >> > Hi, >> > >> > I was about to fill some gaps in 2LC docs regarding >> improvements in >> > 5.0/5.1, but when I took a look into 5.0 docs [1] >> there's close to >> > nothing; is it by purpose? In 4.3 docs [2], there's much >> more on this >> > subject. Am I looking in a wrong place? >>
Re: [hibernate-dev] 2LC docs
Seems much more comprehensive, though I can't find e.g. 'Cache-provider/concurrency-strategy compatibility' table (already out-of-date). Nevertheless, thanks! I'll try to find some timeslot for updating it early next week. Radim On 01/21/2016 04:17 PM, Vlad Mihalcea wrote: > Hi Radim, > > I've just committed the updated version of the Caching chapter. > To build it, you need to: > > 1. go to the documentation folder > 2. run gradle rUG > 3. The new user guide is located under: > documentation/target/asciidoc/userguide/html/Hibernate_User_Guide.html > > Let me know what you think. > > Vlad > > On Wed, Jan 20, 2016 at 11:57 AM, Vlad Mihalcea > <mihalcea.v...@gmail.com <mailto:mihalcea.v...@gmail.com>> wrote: > > Hi Radim, > > I'm now filling in the missing sections on the Caching chapter in > the 5.1 User Guide. > > Vlad > > On Wed, Jan 20, 2016 at 11:32 AM, Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > I would say that the best source is 4.x docs, since the cited > source is > what I describe by 'close to nothing'. > > I understand that for 5.1 the transformation might be > unfinished, but I > was surprised by the same for 5.0 - missing information that's > written, > maybe just not formatted properly for asciiidoc. In such case, > link to > appropriate chapter in older docs would be useful. > > Radim > > On 01/19/2016 07:04 PM, Steve Ebersole wrote: > > The docs are in a state of flux for 5.1. For now your best > bet is the > > source: > > documentation/src/main/asciidoc/userguide/chapters/caching/Caching.adoc > > > > On Fri, Jan 15, 2016 at 3:48 AM Radim Vansa > <rva...@redhat.com <mailto:rva...@redhat.com> > > <mailto:rva...@redhat.com <mailto:rva...@redhat.com>>> wrote: > > > > Hi, > > > > I was about to fill some gaps in 2LC docs regarding > improvements in > > 5.0/5.1, but when I took a look into 5.0 docs [1] > there's close to > > nothing; is it by purpose? In 4.3 docs [2], there's much > more on this > > subject. Am I looking in a wrong place? > > > > Radim > > > > [1] > > > > http://docs.jboss.org/hibernate/orm/5.0/userGuide/en-US/html_single/#caching > > [2] > > > > http://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html_single/#performance-cache > > > > -- > > Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com> <mailto:rva...@redhat.com > <mailto:rva...@redhat.com>>> > > JBoss Performance Team > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > <mailto:hibernate-dev@lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org>> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > > -- > Radim Vansa <rva...@redhat.com <mailto:rva...@redhat.com>> > JBoss Performance Team > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] 2LC docs
I would say that the best source is 4.x docs, since the cited source is what I describe by 'close to nothing'. I understand that for 5.1 the transformation might be unfinished, but I was surprised by the same for 5.0 - missing information that's written, maybe just not formatted properly for asciiidoc. In such case, link to appropriate chapter in older docs would be useful. Radim On 01/19/2016 07:04 PM, Steve Ebersole wrote: > The docs are in a state of flux for 5.1. For now your best bet is the > source: > documentation/src/main/asciidoc/userguide/chapters/caching/Caching.adoc > > On Fri, Jan 15, 2016 at 3:48 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > Hi, > > I was about to fill some gaps in 2LC docs regarding improvements in > 5.0/5.1, but when I took a look into 5.0 docs [1] there's close to > nothing; is it by purpose? In 4.3 docs [2], there's much more on this > subject. Am I looking in a wrong place? > > Radim > > [1] > > http://docs.jboss.org/hibernate/orm/5.0/userGuide/en-US/html_single/#caching > [2] > > http://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html_single/#performance-cache > > -- > Radim Vansa <rva...@redhat.com <mailto:rva...@redhat.com>> > JBoss Performance Team > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/hibernate-dev > -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] 2LC docs
Hi, I was about to fill some gaps in 2LC docs regarding improvements in 5.0/5.1, but when I took a look into 5.0 docs [1] there's close to nothing; is it by purpose? In 4.3 docs [2], there's much more on this subject. Am I looking in a wrong place? Radim [1] http://docs.jboss.org/hibernate/orm/5.0/userGuide/en-US/html_single/#caching [2] http://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html_single/#performance-cache -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] should immutable entities in the second level cache be invalidated when they are removed from the database?
On 01/05/2016 06:51 PM, Scott Marlow wrote: > On 01/05/2016 12:36 PM, Steve Ebersole wrote: >> Sorry Scott, I am not sure I understand exactly what you are asking. I >> will try to answer what I think you are asking as best I can.. >> >> >> On Tue, Jan 5, 2016 at 9:35 AM Scott Marlow <smar...@redhat.com >> <mailto:smar...@redhat.com>> wrote: >> >> I missed adding the WildFly clustering configuration for the Hibernate >> immutable-entity cache. I opened WFLY-5923 [1] for adding the cache but >> am unsure of whether org.hibernate.annotations.Immutable means that the >> application handles clearing the 2lc of stale immutable entities or if >> that should happen automagically when immutable entities are removed >> from the database. >> >> >> Perhaps there is a confusion that an immutable entity can in fact be >> created and deleted? Immutable simply means the state of an existing >> entity cannot be changed. In SQL terms, if you will, we can have an >> INSERT or a DELETE but never an UPDATE. > Thanks for the answer, this is exactly what I needed to know. > >> If an application has asked that Hibernate cache data, the expectation >> is that Hibernate would handle the caching of that data not the >> application *so long as it knows about changes to the cached data*. Now >> if by "when immutable entities are removed from the database" you are >> asking what should happen when cached data is changed in the database >> directly, that answer is always the same and the question of >> (im)mutablity is completely irrelevant there: if the underlying data is >> changed directly in the database it is up to the application to make >> sure that the cache is consistent with those outside changes. >> >> In a cluster, I expect that the performance of caching >> immutable-entities, would be faster if we assume they will not be >> invalidated from the 2lc (e.g. let applications clear them from the >> cache). >> >> >> I think really this comes down to the question of what "invalidated" >> means to the underlying cache and how a deletion of data correlates to >> that. But my guess is that the invalidation still needs to handled (and >> propagated) to properly handle the removal case. > Yes, I agree (based on your explanation). > >> Since we are using Infinispan for the 2lc in WildFly, I would like to >> use an Infinispan simple-cache for immutable-entities, which does no >> invalidation. However, just because I would like to do this, doesn't >> make it right. :-) Which is why I'm asking, what the design intention >> is, so I can configure the WildFly clustering configuration (for >> immutable entities), to align with the intent. >> >> >> Right, which is what I am trying to describe I guess in my previous >> paragraph. Based on my (granted, very limited) understanding of >> Infinispan I assume we still need the invalidation. One possible option >> is to allow the application to configure whether they expect removals >> from an immutable dataset. > This would be a nice performance enhancement for clustered environments, > as the (no removals allowed) immutable-entities cache would not need to > be cluster aware. ATM in Infinispan 2LC there's no optimization for Immutable entities, because the necessary handling of removals does not differ much from updates. In the past, the immutable-entities configuration was added as Galder suggested that in 4.x implementation these can use non-transactional cache, but in the end the implementation had flaws causing inconsistencies. For Immutable and never removed entities, you can AFAIK safely use local cache. However I am not aware of any means that would mark entity as never removed, or relax the consistency for removals. Radim > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Infinispan versions for Hibernate OGM
On 12/16/2015 10:41 PM, Gunnar Morling wrote: > Hi, > > Is there an actual need for 8.1 at this point (so does it provide > features we really need in OGM?) or is this more a general/theoretical > proposal? > > I like the idea in general, but we must carefully think through all > the implications, such as what module slot to depend on in the OGM > dialect and so on. I suppose the alias stuff may come in handy here. I think that the most important part is being independent, than choosing the 'right' version. Infinispan 8.0 brought a few regressions performance-wise, and developers are actively working on fixing those (not sure how much of the effort landed in 8.1). 8.2 could bring quite interesting scalability (in terms of concurrently processed requests) improvements. > > Question on Remote: can the 8.0 libs in WF talk to a 8.1 remote? Old HotRod client can always talk to new HotRod server. New HotRod client may require configuration setting (limiting certain features) in order to talk to old server. In library=embedded mode, compatibility of different version in the same cluster is *not* guaranteed even for micro releases. Radim > > All in all, if there is a strong benefit for going to the latest ISPN > right now, let's do it, otherwise I'd prefer if we sticked for now to > what's provided and focused on getting the remote dialect fly. Let's > life out the module obsession once we actually gain something from it > ;) > > --Gunnar > > > > 2015-12-16 14:04 GMT+01:00 Sanne Grinovero <sa...@hibernate.org>: >> Hello all, >> >> we generally have been trying to target the versions of Infinispan >> provided by the WildFly version we're targeting for compatibility with >> a specific OGM release cycle. >> >> I would like to change that, and for example now switch to the latest >> Infinispan 8.1.0.Final (rather than the one in WildFly 10 candidate >> release, which would be 8.0.2.Final). >> >> Several reasons: >> >> # shall not use the WildFly Infinispan distribution >>in WildFly the specific Infinispan version being integrated is an >> implementation detail. >> People wanting to use Infinispan directly, or for other means than the >> ones addressed by the WildFly clustering features which are based on >> Infinispan (but don't expose it), should be encouraged to download the >> Infinispan modules from the Infinispan project. We should apply (and >> encourage) this practice too. >> >> # pick our own version >>it's obviously nicer for us to reserve ourselves the practical >> benefits of being able to pick our own version according to needs, >> rather than stick with wathever WildFly requires. >> >> # we might have a need for latest Infinispan >>probably no need to explain ;) >> I don't with us to wait for an app-server update cycle when we need an >> upstream patch. >> >> # don't aim at a single WildFly version >>while we're currently running integration tests with latest WildFly, >> I think it's desirable to use that as a testing bed for the modules we >> provide but not as a coupling factor for our dependency matrix. >> In other words, let's prepare OGM to be able to produce modules for >> different versions of the application servers (and not least other >> application servers although that's unrelated). >> Not least, the fact that the app server is sticking with some version >> shouldn't have an impact to all of our users who have no interest in >> this particular appserver at all. >> >> Am I missing any important counter argument? >> >> Thanks, >> Sanne >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Fwd: Link to test case templates
You can mock the JDBC driver and test the emitted SQL from there; I am using mockrunner-jdbc [1] to do that. Since Hibernate needs to chat with DB a bit before doing the logic of that test, it's convenient to switch between real DB and the mocks at runtime. I've created a little tool [2] to do that - actually this disables recording of executed statements (since I am using this for performance testing rather than behaviour verification), but it's trivial to revert that ([3] - just use default ctor). Radim [1] https://github.com/mockrunner/mockrunner [2] https://github.com/rvansa/perfmock [3] https://github.com/rvansa/perfmock/blob/master/src/main/java/org/perfmock/PerfMockDriver.java#L179 On 11/01/2015 11:01 PM, Sanne Grinovero wrote: > Adding back the list. > > To answer Martijn: personally I don't know the answer, but it sounds > like an excellent idea to implement that, if it's not possible > already. > Keep in mind that Hibernate might need to generate multiple SQL > statements per operation. > > Sanne > > -- Forwarded message -- > From: Martijn Dashorst <martijn.dasho...@gmail.com> > Date: 31 October 2015 at 05:43 > Subject: Re: [hibernate-dev] Link to test case templates > To: Sanne Grinovero <sa...@hibernate.org> > > > Good to see making test cases easier. > > Is it possible to test the actual SQL that is emitted by Hibernate > from a test case? I've looked at the github repo but didn't find a > matching method. > > Martijn > > On Friday, 30 October 2015, Sanne Grinovero <sa...@hibernate.org> wrote: >> We should add a page on hibernate.org describing the idea, and from >> there point to github. >> >> On 30 October 2015 at 07:49, Steve Ebersole <st...@hibernate.org> wrote: >>> But that was not the purpose of the content at the old link. Yes the >>> templates are nice but that's not the whole picture of what makes a good >>> test case >>> >>> On Fri, Oct 30, 2015, 9:41 AM Gunnar Morling <gun...@hibernate.org> wrote: >>> >>>> 2015-10-30 15:16 GMT+01:00 Steve Ebersole <st...@hibernate.org>: >>>>> It looks like that may just be an invalid URL. >>>> Yes, the link should point to >>>> https://github.com/hibernate/hibernate-test-case-templates instead. >>>> There are the test case templates and also a description of their >>>> usage. >>>> >>>> >>>>> It looks like the content >>>>> that was at that URL was not migrated over in the website migration. >>>>> >>>>> This ties in with an uneasiness that has been growing on me tbh... We >>>> have >>>>> too many places users have to look for potential information. The >>>> website, >>>>> the JBoss wiki, the GitHub wiki, README.mds, CONTRIBUTING.mds. It's >>>> hard to >>>>> keep straight :) >>>>> >>>>> Ideally a lot of this would live under hibernate.org website umbrella. >>>> But >>>>> to be frank, I find developing content for hibernate.org and >>>> in.relation.to >>>>> to be cumbersome. We can get into "why" in a separate subject. >>>>> >>>>> >>>>> >>>>> On Fri, Oct 30, 2015 at 8:53 AM Steve Ebersole <st...@hibernate.org> >>>> wrote: >>>>>> But for some reason it directs me back to JIra. Even just clicking that >>>>>> link in the email does. I wonder if someone set up a bad redirect on >>>> the >>>>>> hibernate.org website for that? >>>>>> >>>>>> On Fri, Oct 30, 2015 at 8:52 AM Steve Ebersole <st...@hibernate.org> >>>>>> wrote: >>>>>>> The link target is >>>> http://www.hibernate.org/issuetracker.html#testcases. >>>>>>> That's not the "JIRA main page". >>>>>>> >>>>>>> >>>>>>> On Fri, Oct 30, 2015 at 8:44 AM Gunnar Morling <gun...@hibernate.org> >>>>>>> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> When creating a new HHH issue, there is a link "...should generally be >>>>>>>> accompanied by a test case" but it directs to the JIRA main page. >>>>>>>> >>>>>>>> Can we let it point to the test case template repo instead: >>>>>>>> >>>>>>>> https://github.com/hibernat
Re: [hibernate-dev] hibernate-infinispan tests - part... oh I've lost track
Since we can't reproduce the test on local machine (I was once runnning the testsuite whole night again and again and did not get any crash), the only option I can think of is running hibernate-infinispan with verbose logging in CI. To properly diagnose what has happened, we should set up org.jgroups TRACE org.infinispan TRACE org.infinispan.marshall DEBUG org.infinispan.commons.marshall DEBUG (the other two just reduce the verbosity in the parts we don't need). If the build fails in CI and I can get my hands on the log, I can investigate. Steve, should I prepare a PR switching the log level when running testsuite, or could you do that only in the CI? Radim On 09/25/2015 04:28 PM, Steve Ebersole wrote: > We are again running into problems with hibernate-infinispan tests. We are > seeing false test failures on CI. I cannot reproduce these failures > locally, and even out on CI the test(s) that fil change each time. > > http://ci.hibernate.org/job/hibernate-orm-master-h2/1121/ > http://ci.hibernate.org/job/hibernate-orm-master-h2/1122/ > http://ci.hibernate.org/job/hibernate-orm-master-h2/1123/ > > Are 3 job runs showing what I mean. There is no code change between any of > these runs. > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Consistency guarantees of second level cache
On 09/09/2015 06:16 PM, Steve Ebersole wrote: > Some comments inline and then a general discussion at the end... > > On Wed, Sep 9, 2015 at 10:32 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > Thanks for correcting the terms, I'll try to use 'isolation'. > > TX2 reading B = 1 is not READ_UNCOMMITTED - value B = 1 was committed > long ago (it's the initial value). It's reading A = 2 what can be > considered read uncommitted (not isolated enough), but as the > cache has > nothing to do with that entry, we can't really prevent it - it's > already > in the DB. So it's really rather a stale read of B. If my terms I > wrong, > I apologize. > > > I said that "TX2 reading "B->1" before TX1 commits is a question of > isolation and preventing READ_UNCOMMITTED access to the data". In > other words TX2 reading "B->1" in your "timeline" is in fact an > example of the cache preventing READ_UNCOMMITTED access to the data. > So we are saying the same thing there. But after that is where we > deviate. > > The issue with "isolation" is that it is always relative to "the truth > of the system". This is a historical problem between Hibernate and > every manifestation of caching that has come from JBoss ;) In this > usage (second level caching) the cache IS NOT the truth of the system; > the database is. > > So interestingly the data here *is* stale when looked at from the > perspective of the database (again the truth of the system). And that > is fundamentally a problem. I 100% agree that database is the source of the truth, and that the data is stale. My question is whether it is the problem (something we need to avoid by default), or whether something similar can be exhibited in session caching. Okay, I see that it is the problem. By the way the current implementation does not suffer of that, I am rather exploring further optimizations. Therefore, relaxing this should go to the nonstrict read-write mode. > > > "as close together as possible" is not enough - either you allow > certain > situation to happen (although you might try to minimize how often), or > you guarantee that it does not happen. So, do I understand it > correctly > that 2LC should check ' hibernate.connection.isolation' and behave > accordingly? > > > Again, the problem is that you are registering your own syncs to do > things. I understand that getting these 2 "events" as close together > as possible is just minimizing the risk. But thats is an important > minimization. Yes you still need to decide what to do when something > happens to occur between them. But minimizing those cases (by > shrinking the gap) is important. > > And in regards to 'hibernate.connection.isolation'.. uh, no, > absolutely not. I never said that. Not even sure how you go there.. I've diverted a bit here - I have just looked up how is the isolation level set. If you set it for READ_UNCOMMITTED, there's no need to have the cache in sync with DB providing non-isolated results. But I'd rather not abuse this configuration option. > > In 2LC code I am sometimes registering synchronizations but always > through > > SessionImplementor.getTransactionCoordinator() > .getLocalSynchronizations().registerSynchronization(...) > > - I hope this is the right way and not asking for trouble. I usually > just need to do something when I know whether the DB has written the > data or not - Hibernate calls the *AccessStrategy methods well > enough in > the beforeCompletion part (or I should rather say during flush()) but > sometimes I need to delegate some work to the afterCompletion part. > > > Well let's look at the example you gave in detail. And for reference, > this is outlined in the EntityRegionAccessStrategy javadocs. > > So for an UPDATE to an entity we'd get the following call sequence: > > 1) Session1 transaction begins > 2) Session1 flush is triggered; we deem that both A and B have > changed and need to be written to the database. > 2.a) SQL UPDATE issued for A > 2.b) EntityRegionAccessStrategy#update called for A > 2.c) SQL UPDATE issued for B > 2.d) EntityRegionAccessStrategy#update called for B > 3) Session1 transaction commits[1] > 3.a) "before completion callbacks" (for this discussion, there are none) > 3.b) JDBC transaction committed > 3.c) "after completion callbacks" > 3.c.1) EntityRegionAccessStrategy#afterUpdate called for A > 3.c.2) EntityRegionAccessStrategy#afterUpdate called for B > > And again, that is the flow outli
[hibernate-dev] Consistency guarantees of second level cache
Hi, I've been fixing a lot of consistency issues in Infinispan 2LC lately and also trying to improve performance. When reasoning about consistency guarantees I've usually assumed that we don't want to provide stale entries from the cache after the DB commits - that means, we have to invalidate them before the DB commit. This is a useful property if there are some application constraints on the data (e.g. that two entities have equal attributes). On the other hand, if we want the cache synchronized with DB only after the commit fully finishes, we could omit some pre-DB-commit RPCs and improve the performance a bit. To illustrate the difference, imagine that we wouldn't require such atomicity of transactions: when we update the two entities in TX1 and one of them is cached and the other is not, in TX2 we could see updated value of the non-cached value but we could still hit cache for the other entity, seeing stale value, since TX1 has committed the DB but did not finish the commit yet on ORM side: A = 1, B = 1 TX1: begin TX1: (from flush) write A -> 2 TX1: (from flush) write B -> 2 TX1: DB (XA resource) commit TX2: read A -> 2 (handled from DB) TX2: read B -> 1 (cached entry) TX1: cache commit (registered as synchronization) -> cache gets updated to B = 2 TX1 is completed, control flow returns to caller Naturally, after TX1 returns from transaction commit, no stale values should be provided. Since I don't have any deep experience with DBs (I assume that they behave really in the ACID way). I'd like to ask what are the guarantees that we want from 2LC, and if there's anything in the session caching that would loosen this ACIDity. I know we have the nonstrict-read-write mode (that could implement the less strict way), but I imagine this as something that breaks the contract a bit more, allowing even larger performance gains (going the best-effort way without any guarantees). Thanks for your insight! Radim -- Radim Vansa <rva...@redhat.com> JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Consistency guarantees of second level cache
Thanks for correcting the terms, I'll try to use 'isolation'. TX2 reading B = 1 is not READ_UNCOMMITTED - value B = 1 was committed long ago (it's the initial value). It's reading A = 2 what can be considered read uncommitted (not isolated enough), but as the cache has nothing to do with that entry, we can't really prevent it - it's already in the DB. So it's really rather a stale read of B. If my terms I wrong, I apologize. "as close together as possible" is not enough - either you allow certain situation to happen (although you might try to minimize how often), or you guarantee that it does not happen. So, do I understand it correctly that 2LC should check ' hibernate.connection.isolation' and behave accordingly? In 2LC code I am sometimes registering synchronizations but always through SessionImplementor.getTransactionCoordinator() .getLocalSynchronizations().registerSynchronization(...) - I hope this is the right way and not asking for trouble. I usually just need to do something when I know whether the DB has written the data or not - Hibernate calls the *AccessStrategy methods well enough in the beforeCompletion part (or I should rather say during flush()) but sometimes I need to delegate some work to the afterCompletion part. Radim On 09/09/2015 04:51 PM, Steve Ebersole wrote: > To be precise when you talk the stale data you are really asking about > isolation. TX2 reading "B->1" before TX1 commits is a question of > isolation and preventing READ_UNCOMMITTED access to the data. The > problem is the split in the notion of "commit". Those should be "as > close together as possible". For what it is worth, Hibernate commits > its work via Synchronization as well. My preference (and this is > based on years of fighting problems specifically between Hibernate and > TreeCache/JBossCache/Infinispan in regards to Synchronization > ordering) is that hibernate-infinispan piggyback on Hibernate's > transaction handling. Actually, I thought this is why we made some of > the transaction changes we did in Hibernate.. so that you could have a > consistent view of the transaction across jdbc/jta in > hibernate-infinispan. In my experience, having > hibernate-infinispan/Infinispan register its own Synchronization to > control this stuff is just asking for a lot of trouble. > > Anyway, this also gets into the meaning of the concurrent access > strategies. Which access strategy are you talking about in > particular? I assume you mean the `transactional` strategy, just > making sure. > > > > > On Wed, Sep 9, 2015 at 6:58 AM Radim Vansa <rva...@redhat.com > <mailto:rva...@redhat.com>> wrote: > > Hi, > > I've been fixing a lot of consistency issues in Infinispan 2LC lately > and also trying to improve performance. When reasoning about > consistency > guarantees I've usually assumed that we don't want to provide stale > entries from the cache after the DB commits - that means, we have to > invalidate them before the DB commit. This is a useful property if > there > are some application constraints on the data (e.g. that two entities > have equal attributes). On the other hand, if we want the cache > synchronized with DB only after the commit fully finishes, we > could omit > some pre-DB-commit RPCs and improve the performance a bit. > > To illustrate the difference, imagine that we wouldn't require such > atomicity of transactions: when we update the two entities in TX1 and > one of them is cached and the other is not, in TX2 we could see > updated > value of the non-cached value but we could still hit cache for the > other > entity, seeing stale value, since TX1 has committed the DB but did not > finish the commit yet on ORM side: > > A = 1, B = 1 > TX1: begin > TX1: (from flush) write A -> 2 > TX1: (from flush) write B -> 2 > TX1: DB (XA resource) commit > TX2: read A -> 2 (handled from DB) > TX2: read B -> 1 (cached entry) > TX1: cache commit (registered as synchronization) -> cache gets > updated > to B = 2 > TX1 is completed, control flow returns to caller > > Naturally, after TX1 returns from transaction commit, no stale values > should be provided. > > Since I don't have any deep experience with DBs (I assume that they > behave really in the ACID way). I'd like to ask what are the > guarantees > that we want from 2LC, and if there's anything in the session caching > that would loosen this ACIDity. I know we have the > nonstrict-read-write > mode (that could implement the less strict way), but I imagine this as > som
Re: [hibernate-dev] Hibernate 2lc settings in WildFly
I think that it was my mistake to not make this 'immutable-entities' get its configuration from 'entities', when it's not declared explicitly. I believe that adding that cache configuration to WF configs is the easiest way to fix things; though, if users use their own configs, they won't be notified that something is missing in their standalone.xml - the same problem as with overridden infinispan configuration, though this shouldn't be that common and users should pay more attention if they do that. AFAIK, there is no place but the default infinispan config where it is documented which caches need to be defined and what kind of config do they need. There is also no validation in the code. The defaults are just in the default config, not in the code, and therefore user can override all or nothing. Users can even think that all entities are stored in single cache (while each entity type has its own) . I don't think that serializing Infinispan configuration into properties is the best way. I plan to add some configuration sanity checks as part of HHH-10030, but I have not thought about any systematic approach, so I'd welcome any discussion. Radim On 08/18/2015 07:44 PM, Scott Marlow wrote: On 08/18/2015 01:10 PM, Sanne Grinovero wrote: I think such options should not be an aspect of the Cache configuration but only a property of the Hibernate configuration. The time to do that would of been about four years ago (now we have to be configuration compatible on the WildFly side at least). Still, not all applications include Hibernate configuration settings but they do expect to use a second level cache. If we were to remove the Hibernate section from [1], where would that configuration live and what code would deal with that? Why should that be known to the Infinispan configuration? In WildFly, that happens to be where we start/stop Infinispan caches, which makes sense since that is where we also deal with clustering. On 18 August 2015 at 14:29, Scott Marlow smar...@redhat.com wrote: I didn't know about 'immutable-entity', should that be listed in [1] for the WildFly standalone*.xml, so users can configure 'immutable-entity'? [2] describes the possible Infinispan subsystem settings for [1]. Scott [1] https://github.com/wildfly/wildfly/blob/master/clustering/infinispan/extension/src/main/resources/subsystem-templates/infinispan.xml#L39 [2] https://github.com/wildfly/wildfly/blob/master/clustering/infinispan/extension/src/main/resources/schema/jboss-as-infinispan_4_0.xsd ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa rva...@redhat.com JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] InfinispanTestingSetup
Right now I want to do that purely in Hibernate integration part, as I don't see why such user approach should not work - replacing remove(key) calls with put(key, tombstone singleton, expiration args). It's possible that I hit the wall somewhere and have to go down to Infinispan. My general plan is to do stuff in Hibernate now, see what's needed and then we could discuss possible infinispan core enhancements (that would get rid of the custom interceptors I've written) on clustering meeting, when I'll see all the trouble in the big picture. In my opinion, infinispan core is in need of user-managed versioning API (used internally for conditional commands, too) rather than tombstones alone. Functional API which is about to appear in 8.0 soon should give us the opportunity to emulate proper versioning (though, with tombstones for removal, too). Radim On 08/17/2015 01:37 PM, Sanne Grinovero wrote: Great, I also thought tombstones would be essential to improve the 2lc. Are you going to bake that feature within Infinispan or as a customization within the Hibernate code? On 17 August 2015 at 08:26, Radim Vansa rva...@redhat.com wrote: OK, thanks for the info. I am truly trying to rewrite the refactor the testsuite as part of [1] so that we can run the tests against all concurrency strategy modes and cache configurations (I am working on tombstone-based 2LC implementation). Also, I hope I can speed up the testsuite (I see that some tests hang for 15 seconds due to some problem in JGroups). Radim [1] https://hibernate.atlassian.net/browse/HHH-10030 On 08/14/2015 12:14 PM, Sanne Grinovero wrote: Context is this IRC question: rvansa [08:14:33] sannegrinovero: Hi, may I ask why have you sometimes added InfinispanTestingSetup as @ClassRule and sometimes as @Rule in HHH-10001? [11:06] jbossbot [08:14:34] jira [HHH-10001] Make the testsuite compatible with Infinispan 8 [Closed (Fixed) Task, Major, Sanne Grinovero] https://hibernate.atlassian.net/browse/HHH-10001 Hi Radim, I generally wanted to set it as test rule only as that gives a stricter cleanup verification, but in some cases the excessive isolation rules would have it fail: apparently some are relying on previous tests. That's something which should be cleaned up in the tests I guess but I only wanted to workaround the API change in the Infinispan testsuite without risking a semantic change of the tests.. I also didn't have a good understanding of the purpose of those tests. If you're interesting in cleaning that up, you should be able to try changing the @ClassRule instances to @Rule instances and you'll see the test fail. Sanne -- Radim Vansa rva...@redhat.com JBoss Performance Team -- Radim Vansa rva...@redhat.com JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] started release
I think that the failure is caused by HHH-7898 fix. The put is executed only after the current transaction is finished (as synchronization), until then the query is cached only for the current session and therefore not reflected in statistics. Radim On 07/29/2015 09:02 PM, Scott Marlow wrote: I missed a WildFly second level query cache failure yesterday in my testing http://pastebin.com/kg3YhxxL Test steps: - creates an entity - queries for the entity - verify that the query cache hit count is zero. - query again for the entity - verify that the query cache hit count is one (this fails as the query cache hit count is still zero) The failing test code is at https://github.com/wildfly/wildfly/blob/master/testsuite/integration/basic/src/test/java/org/jboss/as/test/integration/jpa/secondlevelcache/SFSB2LC.java#L255 And is called from https://github.com/wildfly/wildfly/blob/master/testsuite/integration/basic/src/test/java/org/jboss/as/test/integration/jpa/secondlevelcache/JPA2LCTestCase.java#L229 This hurts my WF10 upgrade to ORM 5.0.0.CR3 https://github.com/wildfly/wildfly/pull/7842 On 07/29/2015 11:37 AM, Steve Ebersole wrote: The release is done minus announcing and SF upload. Also I decided to not upload the docs yet since they are still incomplete and i still have work to do there prior to Final. On Wed, Jul 29, 2015 at 10:06 AM Steve Ebersole st...@hibernate.org wrote: per $subject ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev -- Radim Vansa rva...@redhat.com JBoss Performance Team ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
