Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Tristan Tarrant 
We also need: backup priority for internal caches as well as conflict 
resolution for backups to avoid broken data replicating in the wrong 
direction.

Tristan

On 4/12/18 10:13 PM, Tristan Tarrant wrote:
> I think we can certainly make it additive, especially now that we have 
> configuration templates in place: the user supplies a base template, and 
> the internal cache logic override with what is needed so that broken 
> configs are less probable (but still possible). Alternatively, instead 
> of overriding, we just check that it matches the requirements.
> 
> Tristan
> 
> On 4/12/18 10:10 PM, Adrian Nistor wrote:
>> Backing up caches with protobuf payload to a remote site will not work 
>> if they are indexed, unless the remote site already has the schema for 
>> the types in question, or else indexing will fail. If the cache is not 
>> indexed it matters less.
>>
>> So the replication of protobuf metadata cache has to be arranged 
>> somehow before any other data is replicated. Manual replication is 
>> indeed PITA.
>>
>> I remember in the very early version of remote query the protobuf 
>> metadata cache configuration was created programatically on startup 
>> unless a manually defined configuration with that name was found, 
>> already provided by the user. In that case the user's config was used. 
>> This approach had the benefit of allowing the user to gain control if 
>> needed. But can also lead to gloom and doom. Was that too bad to do it 
>> again :)))?
>>
>> Adrian
>>
>> On 04/12/2018 10:27 PM, Tristan Tarrant wrote:
>>> It is definitely an internal cache. Because of this, automatically
>>> backing it up to a remote site might not be such a good idea.
>>>
>>> Backups are enabled per-cache, and therefore just blindly replicating
>>> the schema cache to the other site is not a good idea.
>>>
>>> I think that we need a cache-manager-level backup setting that does the
>>> right thing.
>>>
>>> Tristan
>>>
>>> On 4/12/18 7:01 PM, Pedro Ruivo wrote:
 Wouldn't be better to assume the protobuf cache doesn't fit the 
 internal
 cache use case? :)

 On 12-04-2018 17:21, Galder Zamarreno wrote:
> Ok, we do need to find a better way to deal with this.
>
> JIRA: https://issues.jboss.org/browse/ISPN-9074
>
> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo  > wrote:
>
>
>
>   On 12-04-2018 15:49, Galder Zamarreno wrote:
>    > Hi,
>    >
>    > We have an issue with protobuf metadata cache.
>    >
>    > If you run in a multi-site scenario, protobuf metadata
>   information does
>    > not travel across sites by default.
>    >
>    > Being an internal cache, is it possible to somehow
>   override/reconfigure
>    > it so that cross-site configuration can be added in 
> standalone.xml?
>
>   No :( since it is an internal cache, its configuration can't 
> be changed.
>
>    >
>    > We're currently running a periodic job that checks if the
>   metadata is
>    > present and if not present add it. So, we have a 
> workaround for
>   it, but
>    > it'd be not very user friendly for end users.
>    >
>    > Thoughts?
>
>   Unfortunately none... it is the first time an internal cache 
> needs
>   to do
>   some x-site.
>
>    >
>    > Cheers,
>    > Galder
>    >
>    >
>    > ___
>    > infinispan-dev mailing list
>    > infinispan-dev@lists.jboss.org
>   
>    > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>    >
>   ___
>   infinispan-dev mailing list
>   infinispan-dev@lists.jboss.org 
> 
>   https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
 ___
 infinispan-dev mailing list
 infinispan-dev@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/infinispan-dev

>>
> 

-- 
Tristan Tarrant
Infinispan Lead and Data Grid Architect
JBoss, a division of Red Hat
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Tristan Tarrant 
I think we can certainly make it additive, especially now that we have 
configuration templates in place: the user supplies a base template, and 
the internal cache logic override with what is needed so that broken 
configs are less probable (but still possible). Alternatively, instead 
of overriding, we just check that it matches the requirements.

Tristan

On 4/12/18 10:10 PM, Adrian Nistor wrote:
> Backing up caches with protobuf payload to a remote site will not work 
> if they are indexed, unless the remote site already has the schema for 
> the types in question, or else indexing will fail. If the cache is not 
> indexed it matters less.
> 
> So the replication of protobuf metadata cache has to be arranged somehow 
> before any other data is replicated. Manual replication is indeed PITA.
> 
> I remember in the very early version of remote query the protobuf 
> metadata cache configuration was created programatically on startup 
> unless a manually defined configuration with that name was found, 
> already provided by the user. In that case the user's config was used. 
> This approach had the benefit of allowing the user to gain control if 
> needed. But can also lead to gloom and doom. Was that too bad to do it 
> again :)))?
> 
> Adrian
> 
> On 04/12/2018 10:27 PM, Tristan Tarrant wrote:
>> It is definitely an internal cache. Because of this, automatically
>> backing it up to a remote site might not be such a good idea.
>>
>> Backups are enabled per-cache, and therefore just blindly replicating
>> the schema cache to the other site is not a good idea.
>>
>> I think that we need a cache-manager-level backup setting that does the
>> right thing.
>>
>> Tristan
>>
>> On 4/12/18 7:01 PM, Pedro Ruivo wrote:
>>> Wouldn't be better to assume the protobuf cache doesn't fit the internal
>>> cache use case? :)
>>>
>>> On 12-04-2018 17:21, Galder Zamarreno wrote:
 Ok, we do need to find a better way to deal with this.

 JIRA: https://issues.jboss.org/browse/ISPN-9074

 On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo > wrote:



   On 12-04-2018 15:49, Galder Zamarreno wrote:
    > Hi,
    >
    > We have an issue with protobuf metadata cache.
    >
    > If you run in a multi-site scenario, protobuf metadata
   information does
    > not travel across sites by default.
    >
    > Being an internal cache, is it possible to somehow
   override/reconfigure
    > it so that cross-site configuration can be added in 
 standalone.xml?

   No :( since it is an internal cache, its configuration can't 
 be changed.

    >
    > We're currently running a periodic job that checks if the
   metadata is
    > present and if not present add it. So, we have a workaround 
 for
   it, but
    > it'd be not very user friendly for end users.
    >
    > Thoughts?

   Unfortunately none... it is the first time an internal cache 
 needs
   to do
   some x-site.

    >
    > Cheers,
    > Galder
    >
    >
    > ___
    > infinispan-dev mailing list
    > infinispan-dev@lists.jboss.org
   
    > https://lists.jboss.org/mailman/listinfo/infinispan-dev
    >
   ___
   infinispan-dev mailing list
   infinispan-dev@lists.jboss.org 
 
   https://lists.jboss.org/mailman/listinfo/infinispan-dev



 ___
 infinispan-dev mailing list
 infinispan-dev@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/infinispan-dev

>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
> 

-- 
Tristan Tarrant
Infinispan Lead and Data Grid Architect
JBoss, a division of Red Hat
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Adrian Nistor 
Backing up caches with protobuf payload to a remote site will not work 
if they are indexed, unless the remote site already has the schema for 
the types in question, or else indexing will fail. If the cache is not 
indexed it matters less.

So the replication of protobuf metadata cache has to be arranged somehow 
before any other data is replicated. Manual replication is indeed PITA.

I remember in the very early version of remote query the protobuf 
metadata cache configuration was created programatically on startup 
unless a manually defined configuration with that name was found, 
already provided by the user. In that case the user's config was used. 
This approach had the benefit of allowing the user to gain control if 
needed. But can also lead to gloom and doom. Was that too bad to do it 
again :)))?

Adrian

On 04/12/2018 10:27 PM, Tristan Tarrant wrote:
> It is definitely an internal cache. Because of this, automatically
> backing it up to a remote site might not be such a good idea.
>
> Backups are enabled per-cache, and therefore just blindly replicating
> the schema cache to the other site is not a good idea.
>
> I think that we need a cache-manager-level backup setting that does the
> right thing.
>
> Tristan
>
> On 4/12/18 7:01 PM, Pedro Ruivo wrote:
>> Wouldn't be better to assume the protobuf cache doesn't fit the internal
>> cache use case? :)
>>
>> On 12-04-2018 17:21, Galder Zamarreno wrote:
>>> Ok, we do need to find a better way to deal with this.
>>>
>>> JIRA: https://issues.jboss.org/browse/ISPN-9074
>>>
>>> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo >> > wrote:
>>>
>>>
>>>
>>>   On 12-04-2018 15:49, Galder Zamarreno wrote:
>>>> Hi,
>>>>
>>>> We have an issue with protobuf metadata cache.
>>>>
>>>> If you run in a multi-site scenario, protobuf metadata
>>>   information does
>>>> not travel across sites by default.
>>>>
>>>> Being an internal cache, is it possible to somehow
>>>   override/reconfigure
>>>> it so that cross-site configuration can be added in standalone.xml?
>>>
>>>   No :( since it is an internal cache, its configuration can't be 
>>> changed.
>>>
>>>>
>>>> We're currently running a periodic job that checks if the
>>>   metadata is
>>>> present and if not present add it. So, we have a workaround for
>>>   it, but
>>>> it'd be not very user friendly for end users.
>>>>
>>>> Thoughts?
>>>
>>>   Unfortunately none... it is the first time an internal cache needs
>>>   to do
>>>   some x-site.
>>>
>>>>
>>>> Cheers,
>>>> Galder
>>>>
>>>>
>>>> ___
>>>> infinispan-dev mailing list
>>>> infinispan-dev@lists.jboss.org
>>>   
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>   ___
>>>   infinispan-dev mailing list
>>>   infinispan-dev@lists.jboss.org 
>>>   https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>>
>>>
>>> ___
>>> infinispan-dev mailing list
>>> infinispan-dev@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>

___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Tristan Tarrant 
It is definitely an internal cache. Because of this, automatically 
backing it up to a remote site might not be such a good idea.

Backups are enabled per-cache, and therefore just blindly replicating 
the schema cache to the other site is not a good idea.

I think that we need a cache-manager-level backup setting that does the 
right thing.

Tristan

On 4/12/18 7:01 PM, Pedro Ruivo wrote:
> Wouldn't be better to assume the protobuf cache doesn't fit the internal
> cache use case? :)
> 
> On 12-04-2018 17:21, Galder Zamarreno wrote:
>> Ok, we do need to find a better way to deal with this.
>>
>> JIRA: https://issues.jboss.org/browse/ISPN-9074
>>
>> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo > > wrote:
>>
>>
>>
>>  On 12-04-2018 15:49, Galder Zamarreno wrote:
>>   > Hi,
>>   >
>>   > We have an issue with protobuf metadata cache.
>>   >
>>   > If you run in a multi-site scenario, protobuf metadata
>>  information does
>>   > not travel across sites by default.
>>   >
>>   > Being an internal cache, is it possible to somehow
>>  override/reconfigure
>>   > it so that cross-site configuration can be added in standalone.xml?
>>
>>  No :( since it is an internal cache, its configuration can't be changed.
>>
>>   >
>>   > We're currently running a periodic job that checks if the
>>  metadata is
>>   > present and if not present add it. So, we have a workaround for
>>  it, but
>>   > it'd be not very user friendly for end users.
>>   >
>>   > Thoughts?
>>
>>  Unfortunately none... it is the first time an internal cache needs
>>  to do
>>  some x-site.
>>
>>   >
>>   > Cheers,
>>   > Galder
>>   >
>>   >
>>   > ___
>>   > infinispan-dev mailing list
>>   > infinispan-dev@lists.jboss.org
>>  
>>   > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>   >
>>  ___
>>  infinispan-dev mailing list
>>  infinispan-dev@lists.jboss.org 
>>  https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> ___
>> infinispan-dev mailing list
>> infinispan-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

-- 
Tristan Tarrant
Infinispan Lead and Data Grid Architect
JBoss, a division of Red Hat
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Pedro Ruivo 
Wouldn't be better to assume the protobuf cache doesn't fit the internal 
cache use case? :)

On 12-04-2018 17:21, Galder Zamarreno wrote:
> Ok, we do need to find a better way to deal with this.
> 
> JIRA: https://issues.jboss.org/browse/ISPN-9074
> 
> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo  > wrote:
> 
> 
> 
> On 12-04-2018 15:49, Galder Zamarreno wrote:
>  > Hi,
>  >
>  > We have an issue with protobuf metadata cache.
>  >
>  > If you run in a multi-site scenario, protobuf metadata
> information does
>  > not travel across sites by default.
>  >
>  > Being an internal cache, is it possible to somehow
> override/reconfigure
>  > it so that cross-site configuration can be added in standalone.xml?
> 
> No :( since it is an internal cache, its configuration can't be changed.
> 
>  >
>  > We're currently running a periodic job that checks if the
> metadata is
>  > present and if not present add it. So, we have a workaround for
> it, but
>  > it'd be not very user friendly for end users.
>  >
>  > Thoughts?
> 
> Unfortunately none... it is the first time an internal cache needs
> to do
> some x-site.
> 
>  >
>  > Cheers,
>  > Galder
>  >
>  >
>  > ___
>  > infinispan-dev mailing list
>  > infinispan-dev@lists.jboss.org
> 
>  > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>  >
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org 
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> 
> 
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Galder Zamarreno 
Ok, we do need to find a better way to deal with this.

JIRA: https://issues.jboss.org/browse/ISPN-9074

On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo  wrote:

>
>
> On 12-04-2018 15:49, Galder Zamarreno wrote:
> > Hi,
> >
> > We have an issue with protobuf metadata cache.
> >
> > If you run in a multi-site scenario, protobuf metadata information does
> > not travel across sites by default.
> >
> > Being an internal cache, is it possible to somehow override/reconfigure
> > it so that cross-site configuration can be added in standalone.xml?
>
> No :( since it is an internal cache, its configuration can't be changed.
>
> >
> > We're currently running a periodic job that checks if the metadata is
> > present and if not present add it. So, we have a workaround for it, but
> > it'd be not very user friendly for end users.
> >
> > Thoughts?
>
> Unfortunately none... it is the first time an internal cache needs to do
> some x-site.
>
> >
> > Cheers,
> > Galder
> >
> >
> > ___
> > infinispan-dev mailing list
> > infinispan-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

Re: [infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Pedro Ruivo 


On 12-04-2018 15:49, Galder Zamarreno wrote:
> Hi,
> 
> We have an issue with protobuf metadata cache.
> 
> If you run in a multi-site scenario, protobuf metadata information does 
> not travel across sites by default.
> 
> Being an internal cache, is it possible to somehow override/reconfigure 
> it so that cross-site configuration can be added in standalone.xml?

No :( since it is an internal cache, its configuration can't be changed.

> 
> We're currently running a periodic job that checks if the metadata is 
> present and if not present add it. So, we have a workaround for it, but 
> it'd be not very user friendly for end users.
> 
> Thoughts?

Unfortunately none... it is the first time an internal cache needs to do 
some x-site.

> 
> Cheers,
> Galder
> 
> 
> ___
> infinispan-dev mailing list
> infinispan-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev

[infinispan-dev] Protobuf metadata cache and x-site

2018-04-12 Thread Galder Zamarreno 
Hi,

We have an issue with protobuf metadata cache.

If you run in a multi-site scenario, protobuf metadata information does not
travel across sites by default.

Being an internal cache, is it possible to somehow override/reconfigure it
so that cross-site configuration can be added in standalone.xml?

We're currently running a periodic job that checks if the metadata is
present and if not present add it. So, we have a workaround for it, but
it'd be not very user friendly for end users.

Thoughts?

Cheers,
Galder
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev