Re: Ignite 2.16 - Multiple query fields are associated with the same alias

[Pavel Tupitsyn]

Same thing. Fields get flattened for SQL, so you've got one column from
Employee and another from EmpInfo causing a conflict.

On Wed, Apr 24, 2024 at 2:06 PM Charlin S  wrote:

> Hi,
> Thank you for the email.
> What about EmpInfo? Main class and child class having EmpName field (error
> 1)
>
> Thanks and Regards,
> Charlin
>
>
> On Wed, 24 Apr, 2024, 4:25 pm Pavel Tupitsyn, 
> wrote:
>
>> Employee class has two fields with the same name:
>>
>>   [QuerySqlField]
>>   public string EMPCode { get; set; }
>>
>>   [QuerySqlField]
>>   public string EmpCode { get; set; }
>>
>>
>> Ignite 2.10 ignored this and used only one of those for SQL, which may
>> lead to subtle bugs. So we added this validation.
>>
>> You'll have to rename one of the fields, or override SQL field name in
>> the attribute:
>> [QuerySqlField(Name = "EMPCode2")]
>>
>> On Wed, Apr 24, 2024 at 10:01 AM Charlin S 
>> wrote:
>>
>>> Hi All,
>>> I am upgrading Ignite from 2.10 to 2.16 and getting Multiple query
>>> fields are associated with the same alias error for *EMPCode* and *EmpName,
>>> *which is working with Ignite 2.10. unable to create Employee cache.
>>>
>>> Error 1:
>>> System.Exception: IgniteException for chache ::  Employee
>>>  ---> Apache.Ignite.Core.Cache.CacheException: class
>>> org.apache.ignite.IgniteCheckedException: Multiple query fields are
>>> associated with the same alias [alias=EMPNAME]
>>>
>>> Error 2:
>>>   System.Exception: IgniteException for chache ::  Employee
>>>  ---> Apache.Ignite.Core.Cache.CacheException: class
>>> org.apache.ignite.IgniteCheckedException: Multiple query fields are
>>> associated with the same alias [alias=EMPCODE]
>>>
>>> Error 2 can be corrected by removing TmcCode property but why same error
>>> for EmpName?
>>>
>>> my class file
>>> Public class Employee: IBinarizable
>>> {
>>> [QuerySqlField]
>>> public string EMPCode { get; set; }
>>> [QuerySqlField]
>>> public string EmpCode { get; set; }
>>> [QuerySqlField]
>>> public string EmpName { get; set; }
>>> [QuerySqlField]
>>> public EmpInfo EmpInformation { get; set; }
>>>
>>> public void WriteBinary(IBinaryWriter writer)
>>> {
>>> if (writer != null)
>>> {
>>> writer.WriteString("empcode", EMPCode);
>>> writer.WriteString("empname",  EmpName );
>>> writer.WriteObject(" empInformation",  EmpInformation
>>> );
>>> }
>>> }
>>>  public void ReadBinary(IBinaryReader reader)
>>> {
>>> if (reader != null)
>>> {
>>>  EMPCode = reader.ReadString("empcode");
>>>  EmpName = reader.ReadString("empname");
>>>   EmpInformation = reader.ReadString("
>>> empInformation");
>>> }
>>>  }
>>>}
>>>
>>>   public class EmpInfo : IBinarizable
>>>   {
>>> [QuerySqlField]
>>> public string EmpName { get; set; }
>>> [QuerySqlField]
>>> public string EmpAddress{ get; set; }
>>>
>>>public void WriteBinary(IBinaryWriter writer)
>>> {
>>> if (writer != null)
>>> {
>>> writer.WriteString(" empname  ",  EmpName   );
>>> writer.WriteString(" empaddress  ",   EmpAddress  );
>>> }
>>> }
>>>  public void ReadBinary(IBinaryReader reader)
>>> {
>>> if (reader != null)
>>> {
>>>   EmpName   = reader.ReadString(" empname  ");
>>>   EmpAddress  = reader.ReadString(" empaddress  ");
>>> }
>>>  }
>>>   }
>>>
>>> Thanks & Regards,
>>> Charlin
>>>
>>>
>>>

Re: Ignite 2.16 - Multiple query fields are associated with the same alias

[Pavel Tupitsyn]

Employee class has two fields with the same name:

  [QuerySqlField]
  public string EMPCode { get; set; }

  [QuerySqlField]
  public string EmpCode { get; set; }


Ignite 2.10 ignored this and used only one of those for SQL, which may lead
to subtle bugs. So we added this validation.

You'll have to rename one of the fields, or override SQL field name in the
attribute:
[QuerySqlField(Name = "EMPCode2")]

On Wed, Apr 24, 2024 at 10:01 AM Charlin S  wrote:

> Hi All,
> I am upgrading Ignite from 2.10 to 2.16 and getting Multiple query fields
> are associated with the same alias error for *EMPCode* and *EmpName, *which
> is working with Ignite 2.10. unable to create Employee cache.
>
> Error 1:
> System.Exception: IgniteException for chache ::  Employee
>  ---> Apache.Ignite.Core.Cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: Multiple query fields are
> associated with the same alias [alias=EMPNAME]
>
> Error 2:
>   System.Exception: IgniteException for chache ::  Employee
>  ---> Apache.Ignite.Core.Cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: Multiple query fields are
> associated with the same alias [alias=EMPCODE]
>
> Error 2 can be corrected by removing TmcCode property but why same error
> for EmpName?
>
> my class file
> Public class Employee: IBinarizable
> {
> [QuerySqlField]
> public string EMPCode { get; set; }
> [QuerySqlField]
> public string EmpCode { get; set; }
> [QuerySqlField]
> public string EmpName { get; set; }
> [QuerySqlField]
> public EmpInfo EmpInformation { get; set; }
>
> public void WriteBinary(IBinaryWriter writer)
> {
> if (writer != null)
> {
> writer.WriteString("empcode", EMPCode);
> writer.WriteString("empname",  EmpName );
> writer.WriteObject(" empInformation",  EmpInformation   );
> }
> }
>  public void ReadBinary(IBinaryReader reader)
> {
> if (reader != null)
> {
>  EMPCode = reader.ReadString("empcode");
>  EmpName = reader.ReadString("empname");
>   EmpInformation = reader.ReadString("
> empInformation");
> }
>  }
>}
>
>   public class EmpInfo : IBinarizable
>   {
> [QuerySqlField]
> public string EmpName { get; set; }
> [QuerySqlField]
> public string EmpAddress{ get; set; }
>
>public void WriteBinary(IBinaryWriter writer)
> {
> if (writer != null)
> {
> writer.WriteString(" empname  ",  EmpName   );
> writer.WriteString(" empaddress  ",   EmpAddress  );
> }
> }
>  public void ReadBinary(IBinaryReader reader)
> {
> if (reader != null)
> {
>   EmpName   = reader.ReadString(" empname  ");
>   EmpAddress  = reader.ReadString(" empaddress  ");
> }
>  }
>   }
>
> Thanks & Regards,
> Charlin
>
>
>

Re: CVE-2024-22243 - Spring Framework Vulnerability

[Pavel Tupitsyn]

Ignite is not affected.

The description says "Applications that use UriComponentsBuilder...", and
we do not use UriComponentsBuilder at all, not a single match in the source
code [1]

[1]
https://github.com/search?q=repo%3Aapache%2Fignite%20UriComponentsBuilder=code



On Thu, Feb 29, 2024 at 6:13 AM David Horton 
wrote:

> Hi Team,
>
> Could you please advise if Ignite 2.14/2.15 is vulnerable to
> https://spring.io/security/cve-2024-22243 and if so whether a patch is
> planned?
>
> Thanks.
>

Re: DataModelling in Ignite .NET with Inheritance.

[Pavel Tupitsyn]

Hi Satyajit,

Here you go:

using Apache.Ignite.Core;
using Apache.Ignite.Core.Cache;

using var ignite = Ignition.Start();
var cache = ignite.GetOrCreateCache("myCache");

cache[1] = new Base(1, "base-1");
cache[2] = new Derived(2, "derived-2", 42);

foreach (ICacheEntry entry in cache)
{
Console.WriteLine(entry);
}

public record Base(int Id, string Name);
public record Derived(int Id, string Name, long Foo) : Base(Id, Name);


Result:

CacheEntry [Key=1, Value=Base { Id = 1, Name = base-1 }]
CacheEntry [Key=2, Value=Derived { Id = 2, Name = derived-2, Foo = 42 }]


On Wed, Apr 17, 2024 at 3:45 PM  wrote:

> Hi  Pavel,
>
>
>
> Do  we  have  any  working example  with  Parent and Derived  class  data
> model  being saved in  Ignite ?.
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Question regarding text search in ignite

[Pavel Tupitsyn]

Hi Satyajit, we (Ignite community members) are subscribed to Stackoverflow
too. I don't have anything to add to the existing answer there.

On Mon, Apr 15, 2024 at 8:09 AM  wrote:

> Hi  Pavel/Team,
>
>
>
> Is the  below  explanation  holds  good  from  your  side. Need some
> confirmation on  highlighted  line in  yellow.  We  are  not  using
> Native  persistence.  Our cluster is  in  memory  with  external  cache
> store enabled.
>
>
>
>
> https://stackoverflow.com/questions/70232020/ignite-search-query-not-returning-results-after-cluster-restart-with-native-pers
>
>
>
>
>
>
>
> UPD: in case of configured backups for a partitioned cache it should work
> as a regular index. For example if you add an additional node to the
> baseline topology you would see a rebalance
>  
> happening.
> Rebalance uses regular cache operations to insert entries. Lucene index
> would be built
> 
>  on
> the new node. On the contrary if you remove a node from a cluster you
> would still have a full copy of data including text indexes.
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: ignitevisorcmd tool - alternate option

[Pavel Tupitsyn]

control.sh is the replacement:
https://ignite.apache.org/docs/latest/tools/control-script

On Wed, Apr 10, 2024 at 3:56 PM Rajiv Shah  wrote:

> Hi Charlin,
>
> I am working as a Principal Architect at GridGain.
> Let me know if you would like to have a call to discuss your project and
> needs, so I can figure out how to help you.
>
> Regards,
> Rajiv Shah
>
> On Wed, Apr 10, 2024 at 8:01 AM Charlin S  wrote:
>
>> Hi All,
>> Currently I am using Ignite 2.10 and planning to upgrade Ignite 2.16 and
>> found that
>> ignitevisorcmd  tool had been removed from Ignite 2.15 onwards. What is
>> the alternative way for checking grids statistics and cache details without
>> ignitevisorcmd tool.
>>
>> Thanks & Regards,
>> Charlin
>>
>>

Re: Use of distributed computing in C++

[Pavel Tupitsyn]

I thought the question was about thin client, which can only run Java tasks.

The attached code will work from a C++ thick client or server node, the
requirement is to invoke RegisterComputeFunc [1] on every server node
beforehand.

[1]
https://ignite.apache.org/releases/latest/cppdoc/classignite_1_1IgniteBinding.html#a25339becdaa6ea01c6ef2d4ec8da

On Mon, Apr 8, 2024 at 4:30 PM Louis C  wrote:

> Thanks for your answer, it answers a part of my question.
>
> But what about this code that is pure C++ ? (can be seen on the previous
> link I sent) :
>
>
> How does one make sure that the server knows this task ?
>
> Best regards,
> Louis
> ------
> *De :* Pavel Tupitsyn 
> *Envoyé :* lundi 8 avril 2024 11:34
> *À :* user@ignite.apache.org 
> *Objet :* Re: Use of distributed computing in C++
>
> С++ client can execute Compute tasks that are written in Java and deployed
> to the server nodes:
>
>
> https://ignite.apache.org/releases/latest/cppdoc/classignite_1_1thin_1_1compute_1_1ComputeClient.html
>
> On Fri, Apr 5, 2024 at 3:41 PM Louis C  wrote:
>
> Hello everyone,
>
> I was interested in the distributed computed in C++ in Ignite, as
> described here :
>
> https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing
>
> I noted that the classes that will be executed must be known by the
> server, as said here :
>
>
> *In order to run tasks on the remote nodes, make sure the class
> definitions of the tasks are available on the nodes. You can do this in two
> ways:*
>
>-
>
>*Add the classes to the classpath of the nodes;*
>-
>
>*Enable peer class loading
>
> <https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading>.*
>
> However these 2 points are mostly related to Java-based tasks, and I could
> not find anything regarding C++.
> So my question will be : how can a C++ client share an execution task on
> the server nodes ? Is it transparent ?
> Must we add a dll/.so on the server nodes ? In this case how to load this
> dll ?
>
> Thanks in advance,
> Louis C
>
>

Re: Use of distributed computing in C++

[Pavel Tupitsyn]

С++ client can execute Compute tasks that are written in Java and deployed
to the server nodes:

https://ignite.apache.org/releases/latest/cppdoc/classignite_1_1thin_1_1compute_1_1ComputeClient.html

On Fri, Apr 5, 2024 at 3:41 PM Louis C  wrote:

> Hello everyone,
>
> I was interested in the distributed computed in C++ in Ignite, as
> described here :
>
> https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing
>
> I noted that the classes that will be executed must be known by the
> server, as said here :
>
>
> *In order to run tasks on the remote nodes, make sure the class
> definitions of the tasks are available on the nodes. You can do this in two
> ways:*
>
>-
>
>*Add the classes to the classpath of the nodes;*
>-
>
>*Enable peer class loading
>
> .*
>
> However these 2 points are mostly related to Java-based tasks, and I could
> not find anything regarding C++.
> So my question will be : how can a C++ client share an execution task on
> the server nodes ? Is it transparent ?
> Must we add a dll/.so on the server nodes ? In this case how to load this
> dll ?
>
> Thanks in advance,
> Louis C
>

Re: Spring Native support

[Pavel Tupitsyn]

Have a look at [1], there is a POC linked [2]

[1] https://www.mail-archive.com/dev@ignite.apache.org/msg53182.html
[2] https://github.com/scottmf/graalvm-ignite

On Fri, Mar 1, 2024 at 8:01 AM Kramer  wrote:

> I believe this is not supported with Ignite 2.x due to Native Buffer
> access. It will only be part of Ignite 3 which is still Beta.
>
> I haven't tested it myself though, so my comment should be confirmed by
> others.
> On 2/29/24, 14:14 Dinakar Devineni  wrote:
>
>> Hi,
>>
>>
>>
>> Has  anyone tried   spring  native  image  with   ignite?  If  so  is it
>> efficient  and what  are  the  pros and cons.
>>
>> Spring Native documentation
>> 
>>
>>
>>
>>
>>
>> Thanks
>>
>> Dina
>>
>

Re: Ignite transactions

[Pavel Tupitsyn]

1. Not sure I understand
2. Messaging is not transactional
3. No
4. No, transactions are tied to a specific thread

On Wed, Feb 14, 2024 at 11:01 AM Нестрогаев Андрей Викторович <
a.nestrog...@flexsoft.com> wrote:

> Hi All,
>
>
>
> Maybe someone has already researched these questions:
>
> 1. How can you organize nested/autonomous transactions in ignite? For
> example, for the purpose of writing a protocol to another cache, so that
> the protocol is saved regardless of the result of the main transaction.
>
> 2. If you use Messaging in ignite within a transaction, does it take it
> into account, or is the message sent without taking into account the
> transaction?
>
> 3. Does a transaction started on the current node extend to the code sent
> to another node (IgniteRunnable, IgniteClosure)?
>
> 4. Does a transaction span another thread started from current?
>
>
>
> Thanks for the help in advance.
>
>
>
> Andrey
>
>
> *Настоящее  сообщение или любые приложения к нему могут содержать
> конфиденциальную информацию и другую информацию, защищаемую от раскрытия и
> принадлежащую АО «ФлексСофт». Ее разглашение или иное использование без
> согласования с АО «ФлексСофт» является нарушением законодательства
> Российской Федерации. Любое действие, направленное на копирование,
> передачу, распространение каким-либо образом и с помощью каких-либо средств
> как самого письма, так и информации, содержащейся в нем (в том числе в виде
> приложений), запрещено. Отправитель настоящего сообщения не несет
> ответственность за точность и полноту передачи информации, содержащейся в
> настоящем сообщении, а также за своевременность ее получения. Если Вы
> получили настоящее сообщение по ошибке, пожалуйста, сообщите об этом
> отправителю, а затем удалите его и любые копии с Вашего компьютера.
> Настоящее электронное сообщение и содержащаяся в нем информация, или любые
> приложения к нему,  не является официальной позицией АО «ФлексСофт» и не
> влечет финансовые или иные обязательства АО «ФлексСофт»; не могут
> использоваться и не являются какого-либо рода офертой,  акцептом оферты,
> или предложением делать оферты, или совершать акцепт оферты, не является
> рекламой или профессиональным советом,  прогнозом любых событий,  если иное
> прямо не предусмотрено в настоящем сообщении или любых приложениях к нему.
> АО «ФлексСофт» не несет ответственность за любые прямые или косвенные
> убытки от использования получателем или иным лицом сведений настоящего
> сообщения и/или приложений к нему. Информация, передаваемая по сети
> Интернет, без использования технических средств защиты, является не
> защищенной от противоправных действий третьих лиц и может содержать
> вредоносные программные средства. АО «ФлексСофт» не несет ответственности
> за данные действия.*
>

Re: Ignite Text Query in .NET

[Pavel Tupitsyn]

Hi Satyajit, looks like the linked stackoverflow page explains it all, I
don't have anything to add.

Pavel

On Mon, Feb 12, 2024 at 4:28 AM  wrote:

> Hi  Pavel,
>
>
>
> We  have  observed  that  when  one  node went down  and  we restarted
> the node , Ignite Text  query  didn’t return  full set of data.  However
> when we checked  the cache count it  was as expected. Is  this
> observation  correct  that Lucene index  is  not  copied to  RAM  of
> restarted  node? Is  it  a  bug?.  We are using  external  cache store with
> cache.
>
>
>
>
> https://stackoverflow.com/questions/71797569/no-getting-any-results-from-a-textquery-after-restarting-the-node
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Ignite Write Behind + .NET + External Cache Store

[Pavel Tupitsyn]

1. Write-behind mode invokes cache store in background, so there is no
simple way to confirm if a particular write (cache put) has been persisted,
no matter how many backups.
2. AFAIK, cache store is only invoked on the primary node for the given key

On Tue, Jan 9, 2024 at 5:04 PM  wrote:

> Hi  Pavel,
>
>
>
> Under  Write Behind = true  setting  does  Ignite  will  try  to  save
> data  from  backup  node( partition) if  lets say  one  node crashes while
> writing   into  external  cache store? Document  says  if  node  crashes
> then  some data  can be  lost  from  saving  into  external  cache store
> but  what  if  we set Backup =1 and  WriteSynchronizationMode as  Full
> Sync?  Will  the data  get  written to  cache store  from backup node?
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
> Restricted - External
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Data consistency in Ignite and BackUp Filter in Ignite .net node

[Pavel Tupitsyn]

Could you test the simplest use case:
- In-memory cache without any persistence
- 1 backup
- WriteSynchronizationMode = FullSync
- Check count with ICache.GetSize

Then we can add other parts (persistence) one by one to see where the
problem might be.

On Thu, Dec 7, 2023 at 5:17 AM  wrote:

> Hi  Pavel,
>
>
>
> The  cluster  consist  of  some  caches  using  Native  persistence  and
> some  caches  connected  to external  cache  store. BackUp = 1 is  set
> for  every  cache.
>
> I  checked  the  count  of  cache  connected  to  external  cache  store
> once  it  was  prepopulated and  then stopping  one  node. Ignite  is
> running  as  .NET  Embedded  Service.
>
>
>
> Am using  Jolokia  to  connect  to  Ignite JMX  metrics  as  Rest  Api
> and  checking  counts? Any  thoughts or  alternate way to  check  count?
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Mandal, Satyajit: IT (PUN)
> *Sent:* Wednesday, December 6, 2023 11:36 AM
> *To:* Pavel Tupitsyn 
> *Cc:* user@ignite.apache.org
> *Subject:* RE: Data consistency in Ignite and BackUp Filter in Ignite
> .net node
>
>
>
> Hi  Pavel,
>
>
>
> I  am  using  JMX Metrics  to  count  the  cache  entries.
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Wednesday, December 6, 2023 11:34 AM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Data consistency in Ignite and BackUp Filter in Ignite
> .net node
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> With 1 backup and 1 node loss there should not be any data loss. How do
> you count the entries?
>
>
>
> On Wed, Dec 6, 2023 at 8:00 AM  wrote:
>
> Hi Pavel,
>
>
>
> We  did  testing  with  BackUp = 1  and  with  WriteSynchronizationMode
> as  PrimarySync  as  well  as  FullSync  we  found  that  when  we  are
> stopping  one  node and  comparing  the  count we observed  that  count
> of  cache  size  is not  matching. There  is minimal  difference. So  is
> it  that  if  we  lose  node  then  there  is  possibility  of  data  loss
> and  can’t  prevent  it  for  in  memory cluster? Cluster has  6 nodes.
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Tuesday, June 27, 2023 8:13 PM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Data consistency in Ignite and BackUp Filter in Ignite
> .net node
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> > ensure  backup partitions  of  a  particular  node  is  created  in
> another  node  and  not  in  same  node
>
> No need to do anything, this is the default behavior: backup partitions
> always reside on a different node.
>
> To put it another way: every partition has a primary node and one or more
> backup nodes. All those nodes are different (given enough nodes in the
> cluster).
>
>
>
> - AffinityBackupFilter is for complex use cases when the cluster spreads
> multiple racks or availability zones [1] [2]
>
> - If you have multiple nodes per machine, consider
> RendezvousAffinityFunction.ExcludeNeighbors property [3]
>
>
>
> [1]
> https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.Rendezvous.ClusterNodeAttributeAffinityBackupFilter.html
> <https://clicktime.symantec.com/15sM1Gy5PdsLM3siGkM23?h=cO_QJLdrtzTxbFFZGD0aLyUUsqezz5X0GCiXSjqzYYg==https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.Rendezvous.ClusterNodeAttributeAffinityBackupFilter.html>
>
> [2]
> https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution#backup-filter
> <https://clicktime.symantec.com/15sM67AMrFYvkzhdpJkAf?h=-Jbv1-ZUpqMUY2xn5IJddtZ2wk_5th5Uy1TKW0AfIqU==https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution%23backup-filter>
>
> [3]
> https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.AffinityFunctionBase.html#Apache_Ignite_Core_Cache_Affinity_AffinityFunctionBase_ExcludeNeighbors
> <https://clicktime.symantec.com/15sLvSmnw2Bjw73njBwsR?h=AdE7UMHeKx0TYXHG0iydrlc2PuNzZ7b4z1rZN_ZVUjw==https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.AffinityFunctionB

Re: Data consistency in Ignite and BackUp Filter in Ignite .net node

[Pavel Tupitsyn]

With 1 backup and 1 node loss there should not be any data loss. How do you
count the entries?

On Wed, Dec 6, 2023 at 8:00 AM  wrote:

> Hi Pavel,
>
>
>
> We  did  testing  with  BackUp = 1  and  with  WriteSynchronizationMode
> as  PrimarySync  as  well  as  FullSync  we  found  that  when  we  are
> stopping  one  node and  comparing  the  count we observed  that  count
> of  cache  size  is not  matching. There  is minimal  difference. So  is
> it  that  if  we  lose  node  then  there  is  possibility  of  data  loss
> and  can’t  prevent  it  for  in  memory cluster? Cluster has  6 nodes.
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Tuesday, June 27, 2023 8:13 PM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Data consistency in Ignite and BackUp Filter in Ignite
> .net node
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> > ensure  backup partitions  of  a  particular  node  is  created  in
> another  node  and  not  in  same  node
>
> No need to do anything, this is the default behavior: backup partitions
> always reside on a different node.
>
> To put it another way: every partition has a primary node and one or more
> backup nodes. All those nodes are different (given enough nodes in the
> cluster).
>
>
>
> - AffinityBackupFilter is for complex use cases when the cluster spreads
> multiple racks or availability zones [1] [2]
>
> - If you have multiple nodes per machine, consider
> RendezvousAffinityFunction.ExcludeNeighbors property [3]
>
>
>
> [1]
> https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.Rendezvous.ClusterNodeAttributeAffinityBackupFilter.html
> <https://clicktime.symantec.com/15sM1Gy5PdsLM3siGkM23?h=cO_QJLdrtzTxbFFZGD0aLyUUsqezz5X0GCiXSjqzYYg==https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.Rendezvous.ClusterNodeAttributeAffinityBackupFilter.html>
>
> [2]
> https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution#backup-filter
> <https://clicktime.symantec.com/15sM67AMrFYvkzhdpJkAf?h=-Jbv1-ZUpqMUY2xn5IJddtZ2wk_5th5Uy1TKW0AfIqU==https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution%23backup-filter>
>
> [3]
> https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.AffinityFunctionBase.html#Apache_Ignite_Core_Cache_Affinity_AffinityFunctionBase_ExcludeNeighbors
> <https://clicktime.symantec.com/15sLvSmnw2Bjw73njBwsR?h=AdE7UMHeKx0TYXHG0iydrlc2PuNzZ7b4z1rZN_ZVUjw==https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.AffinityFunctionBase.html%23Apache_Ignite_Core_Cache_Affinity_AffinityFunctionBase_ExcludeNeighbors>
>
>
>
> On Tue, Jun 27, 2023 at 4:02 PM  wrote:
>
> Hi  Pavel,
>
>
>
> We are  running  6  node cluster  with  Active Active  setup. 3  nodes  in
> one  Datacenter  and  3  in  another datacenter with  external  cache
> store  enabled  for  some caches  and  native  persistence  enabled  for
> others.  Now  if  we  want  to  setup  backup  = 1  for  in memory  caches
> how  that  should  be  done  to  ensure  backup partitions  of  a
> particular  node  is  created  in another  node  and  not  in  same  node.
> Also  is  there  any specific settings  to  ensure data  consistency  for
> in  memory  caches.
>
>
>
> Can you  also  share  how  this settings can  be  achieved  in  Ignite(.NET) 
> ( AffinityBackupFilter in RendezvousAffinityFunction)
>
>
>
> Regards
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> app

Re: On the issue of continuous query function monitoring in thinclients

[Pavel Tupitsyn]

> I am using gridgain-community-8.8.9 version.

This mailing list is about Apache Ignite. Please contact GridGain support.

On Fri, Oct 13, 2023 at 5:47 PM Mengyu Jing  wrote:

> Hi! Mikhail, Pavel
>
>
>
> Indeed, the two pieces of information provided by Pavel may not be very
> similar to my situation.
>
>
>
> I am using gridgain-community-8.8.9 version.
>
>
>
> The server is configured in file ignite-server-config1.xml, while the thin
> client and continuous query are configured in file ThinClientExample.java.
>
>
>
> The scenario you described is very accurate. As a supplement, in the
> second step, 40 thin clients need to be used to register 40 listeners. I
> have tried using up to 10 listeners before, but this situation did not
> occur. It seems to be related to the number of listeners.
>
>
>
> 从 Windows 版邮件 <https://go.microsoft.com/fwlink/?LinkId=550986>发送
>
>
>
> *发件人: *Mikhail Petrov 
> *发送时间: *2023年10月13日 18:42
> *收件人: *user@ignite.apache.org
> *主题: *Re: On the issue of continuous query function monitoring in
> thinclients
>
>
>
> Pavel, in fact, I don't think that the mentioned ticket has anything to do
> with the described problem.
>
> Mengyu Jing, What version of the Apache Ignite are you using? Could you
> please attach the configuration of the Ignite server node, thin client
> configuration and continuous query you are running?
>
> Do I understand your scenario correctly:
>
> 1. Start 12 server nodes
> 2. Register a Continuous Query via thin clients
> 3. Start loading the cache using thin clients
> 4. Kill one of the server nodes
> 5. Restart the disconnected node so that it rejoins the cluster
> 6. CacheContinuousQueryEventBuffe#backupQ is no longer being cleared on
> the restarted node
>
> On 13.10.2023 07:30, Pavel Tupitsyn wrote:
>
> Hello,
>
>
>
> We are working on this issue [1] [2]
>
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-20424
>
> [2] https://github.com/apache/ignite/pull/10980
>
>
>
> On Thu, Oct 12, 2023 at 6:27 PM Mengyu Jing  wrote:
>
> Hello, Igniters!
>
>
>
> I started an Ignite cluster with 12 nodes, but the cluster did not have
> persistence enabled.
>
>
>
> I used 40 thin clients to listen for cache changes through continuous
> query function, and there were also some other thin clients constantly
> inserting data into the cache. When I hung up a node in the cluster using
> the kill command, the node quickly stopped and was pulled back up by K8S,
> but the node continued to restart itself.
>
>
>
> By analyzing the logs, it can be found that the cluster was kicked out due
> to the continuous growth of heap memory. Through analyzing the heap, I
> found that the reason is that the variable named backupQ in the
> CacheContinuousQueryEventBuffer class is occupying memory. It seems that
> the data in this queue is no longer deleted, but is growing infinitely.
>
>
>
> Will restarting cause this node to not receive
> CacheContinuousQueryBatchAck messages?
>
> Has anyone encountered this situation before?
>
>
>
> 从 Windows 版邮件 <https://go.microsoft.com/fwlink/?LinkId=550986>发送
>
>
>
>
>

Re: On the issue of continuous query function monitoring in thin clients

[Pavel Tupitsyn]

Hello,

We are working on this issue [1] [2]

[1] https://issues.apache.org/jira/browse/IGNITE-20424
[2] https://github.com/apache/ignite/pull/10980

On Thu, Oct 12, 2023 at 6:27 PM Mengyu Jing  wrote:

> Hello, Igniters!
>
>
>
> I started an Ignite cluster with 12 nodes, but the cluster did not have
> persistence enabled.
>
>
>
> I used 40 thin clients to listen for cache changes through continuous
> query function, and there were also some other thin clients constantly
> inserting data into the cache. When I hung up a node in the cluster using
> the kill command, the node quickly stopped and was pulled back up by K8S,
> but the node continued to restart itself.
>
>
>
> By analyzing the logs, it can be found that the cluster was kicked out due
> to the continuous growth of heap memory. Through analyzing the heap, I
> found that the reason is that the variable named backupQ in the
> CacheContinuousQueryEventBuffer class is occupying memory. It seems that
> the data in this queue is no longer deleted, but is growing infinitely.
>
>
>
> Will restarting cause this node to not receive
> CacheContinuousQueryBatchAck messages?
>
> Has anyone encountered this situation before?
>
>
>
> 从 Windows 版邮件 发送
>
>
>

Re: Arithmetic overflow exception ( Ignite logs)

[Pavel Tupitsyn]

Workaround: disable Java console redirect [1]
IgniteConfiguration.RedirectJavaConsoleOutput = false;

[1]
https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.IgniteConfiguration.html#Apache_Ignite_Core_IgniteConfiguration_DefaultRedirectJavaConsoleOutput

On Fri, Oct 6, 2023 at 10:21 AM Pavel Tupitsyn  wrote:

> Hi Satyajit,
>
> It is a bug, I've created a ticket [1]
> Looks like a critical issue, so we'll fix it soon. Thanks for the report!
>
> [1] https://issues.apache.org/jira/browse/IGNITE-20586
>
> On Fri, Oct 6, 2023 at 9:24 AM  wrote:
>
>> Hi  Pavel/Team,
>>
>>
>>
>> One  Ignite  node  went  down with below error .  Can  you  please  let
>> us  know  why  this  error  came. The  whole  log  is  attached.
>>
>>
>>
>> ignite-instance-cfa2d711-f2b7-4572-84f3-b880e25aa9e4%][] JVM will be
>> halted immediately due to the failure: [failureCtx=FailureContext
>> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException:
>> Arithmetic operation resulted in an overflow.
>>
>>
>>
>>
>>
>> Regards
>>
>> Satyajit
>>
>> Restricted - External
>>
>> Barclays Execution Services Limited registered in England. Registered No.
>> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>>
>> Barclays Execution Services Limited provides support and administrative
>> services across Barclays group. Barclays Execution Services Limited is an
>> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
>> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
>> Bank plc are authorised by the Prudential Regulation Authority and
>> regulated by the Financial Conduct Authority and the Prudential Regulation
>> Authority. Clydesdale Financial Services Limited is authorised and
>> regulated by the Financial Conduct Authority.
>>
>> This email and any attachments are confidential and intended solely for
>> the addressee and may also be privileged or exempt from disclosure under
>> applicable law. If you are not the addressee, or have received this email
>> in error, please notify the sender and immediately delete it and any
>> attachments from your system. Do not copy, use, disclose or otherwise act
>> on any part of this email or its attachments.
>>
>> Internet communications are not guaranteed to be secure or virus-free.
>> The Barclays group does not accept responsibility for any loss arising from
>> unauthorised access to, or interference with, any internet communications
>> by any third party, or from the transmission of any viruses. Replies to
>> this email may be monitored by the Barclays group for operational or
>> business reasons.
>>
>> Any opinion or other information in this email or its attachments that
>> does not relate to the business of the Barclays group is personal to the
>> sender and is not given or endorsed by the Barclays group.
>>
>> Unless specifically indicated, this e-mail is not an offer to buy or sell
>> or a solicitation to buy or sell any securities, investment products or
>> other financial product or service, an official confirmation of any
>> transaction, or an official statement of Barclays.
>>
>

Re: Arithmetic overflow exception ( Ignite logs)

[Pavel Tupitsyn]

Hi Satyajit,

It is a bug, I've created a ticket [1]
Looks like a critical issue, so we'll fix it soon. Thanks for the report!

[1] https://issues.apache.org/jira/browse/IGNITE-20586

On Fri, Oct 6, 2023 at 9:24 AM  wrote:

> Hi  Pavel/Team,
>
>
>
> One  Ignite  node  went  down with below error .  Can  you  please  let
> us  know  why  this  error  came. The  whole  log  is  attached.
>
>
>
> ignite-instance-cfa2d711-f2b7-4572-84f3-b880e25aa9e4%][] JVM will be
> halted immediately due to the failure: [failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException:
> Arithmetic operation resulted in an overflow.
>
>
>
>
>
> Regards
>
> Satyajit
>
> Restricted - External
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Using REST api seems to "lock" record?

[Pavel Tupitsyn]

Use 'valueType' URL parameter [1] to specify the type, e.g.:
ignite?cmd=put=MYKEY=123=myCache=integer

[1] https://ignite.apache.org/docs/latest/restapi#data-types

On Fri, Sep 29, 2023 at 8:58 PM John Smith  wrote:

> Ok so I figured out the issue. I'm getting a class cast Exception.
>
> The cache in the application is declared as  when I use
> the REST API to put a value. I guess it changes the value to String and my
> application catches the error. Well it's actually swallowing the error but
> I can fix that.
>
> My question now would be, if using the HTTP REST Api, can we specify the
> type of the value when doing a put?
>
>
> On Fri, Sep 29, 2023 at 1:29 PM John Smith  wrote:
>
>> Hi, running 2.12
>>
>> When I use the put command like so:
>> http://xx/ignite?cmd=put=carrier-ids-for-phones=15149838779=10009=60001
>>
>> Then I call the Java get async function, it seems to block and doesn't
>> return.
>>
>> If I use this command:
>> http://xx/ignite?cmd=put=carrier-ids-for-phones=15149838779=10009=60001
>>
>> The java async Api blocks, but once the record is expired, the java async
>> Api returns.
>>
>

Re: External Cache Store + Ignite .NET

[Pavel Tupitsyn]

CacheConfiguration.WriteBehindEnabled = true

This enables background asynchronous cache store updates [1] [2]

[1]
https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Configuration.CacheConfiguration.html#Apache_Ignite_Core_Cache_Configuration_CacheConfiguration_WriteBehindEnabled
[2]
https://ignite.apache.org/docs/latest/persistence/external-storage#write-behind-caching

On Tue, Sep 26, 2023 at 12:34 PM  wrote:

> Hi  Pavel,
>
>
>
> What  is  the setting  for  asynchrous  write  to  external  cache  store
> using  Ignite  .NET  Api?  Do  you  have  any  sample  code?
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Read Through and WriteThrough and Native Persistence + Ignite .NET

[Pavel Tupitsyn]

Hi Satyajit,

Have you tried it? I'm not sure how a combination of CacheStore + native
persistence works.

On Thu, Sep 7, 2023 at 2:42 PM  wrote:

> Hi Pavel
>
>
>
> Can  we do  a  setting  where  we  enable  Native  Persistence as  well
> as  ( ReadThrough  = true  and  WriteThrough  = true)   and   save data
> in  disk  as well as  in External  cachestore.  And  after cache restart
> we don’t  do  initial  loading  from  external cache store and  take the
> advantage  of  native  persistence  of  Ignite(lazy loading).
>
>
>
> Regards
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Graceful shutdown of DotNet Ignite nodes running in Kubernetes pods

[Pavel Tupitsyn]

Ignite handles SIGTERM and stops the node.
Use IIgnite.Stopped [1] to shut down the app gracefully.

Example: run this program and use "kill" on it - "Stopped." is printed and
the process exits with code 0.

var ignite = Ignition.Start();
ignite.Stopped += (_, _) =>
{
Console.WriteLine("Stopped.");
Environment.Exit(0);
};
Thread.Sleep(-1);


[1]
https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.IIgnite.html#Apache_Ignite_Core_IIgnite_Stopped

On Thu, Sep 7, 2023 at 12:52 AM Raymond Wilson 
wrote:

> If you have an Apache Ignite deployment on Kubernetes with Linux
> containers using the DotNet C# Ignite client, how do you trigger
> graceful shutdown of the node?
>
> Kubernetes emits a SIGTERM signal to the pod when it wants to remove it.
> That signal is relayed to the process running in the pod identified in the
> Docker configuration
>
> In our YAML file we start the Ignite node in the pod like this:
>
> command: ["dotnet"]
> args: ["SomeNode.dll"]
>
> When it comes time to stop that Ignite node Kubernetes emits the SIGTERM
> to the pod. It appears the 'dotnet' context catches the SIGTERM and it is
> not relayed into the SomeNode.dll logic.
>
> We have several means of catching the SIGTERM configured in our
> application startup logic:
>
> AppDomain.CurrentDomain.ProcessExit += (s, e) =>  SigTermHandler ;
> AssemblyLoadContext.Default.Unloading += SigTermHandler;
> Console.CancelKeyPress += (s, e) =>  SigTermHandler  ;
>
> However the SigTermHandler is never called in our application logic, which
> means the node is then hard killed with a SIGKILL after the termination
> grace period configured for the pod.
>
> If you have a similar tool chain and deployment context, how are you
> ensuring the Ignite node implementation gets the SIGTERM and shuts down
> gracefully?
>
> Thanks,
> Raymond.
>
> --
> 
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> 
>

Re: Query regarding JVM and GC settings with Ignite .NET

[Pavel Tupitsyn]

Hi Satyajit,

Yes, the explanation in the Stackoverflow answer is correct.
Data regions are allocated in offheap (unmanaged heap) memory, so Xms/Xmx
settings are not relevant.

For example, there are production setups out there with Xmx16G and data
regions approaching terabyte size.

On Fri, Aug 25, 2023 at 6:38 AM  wrote:

>
>
> Hi  Pavel,
>
>
>
> Can  you please confirm if  below explanation  is  correct in
> stackoverflow regarding  offheap and  onheap  settings.
>
>
>
>
> https://stackoverflow.com/questions/52977420/how-to-set-off-heap-or-on-heap-memory-in-apache-ignite
>
>
>
> *In your case, you're committing more memory to Ignire than you have. You
> commit 12GB off-heap + 15GB heap = 27GB, which is obviously more than your
> 16 GB RAM.*
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> *From:* Mandal, Satyajit: IT (PUN)
> *Sent:* Thursday, August 24, 2023 9:19 PM
> *To:* Pavel Tupitsyn 
> *Cc:* user@ignite.apache.org
> *Subject:* Query regarding JVM and GC settings with Ignite .NET
>
>
>
> Hi  Pavel,
>
>
>
> Have  query  regarding  JVM  and  GC settings.  If  my  data region  has
> requirement  of  20 GB ( MAX  size)  then  we will  need  to  set  -Xmx as
> more  than  20G?  We faced  Java.lang.outofMemory exception  so  wanted
> to  confirm  abt  the JVM  tuning?
>
>
>
>
> https://ignite.apache.org/docs/latest/perf-and-troubleshooting/memory-tuning#java-heap-and-gc-tuning
>
>
>
>
>
>
>
>
>
> -server
>
> -Xms10g
>
> *-Xmx24g*
>
> -XX:+AlwaysPreTouch
>
> -XX:+UseG1GC
>
> -XX:+ScavengeBeforeFullGC
>
> -XX:+DisableExplicitGC
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Query regarding Ignite .NET Authentication and Authorization

[Pavel Tupitsyn]

Hi Satyajit,

Ignite does not provide role-based access control, only authentication.
Some vendors have plugins/forks where this is available, though.

Pavel

On Fri, Aug 18, 2023 at 5:48 AM  wrote:

> Hi  Pavel,
>
>
>
> We have a  requirement  where we are loading  static data  into Ignite
> cache  as SQL  tables  using Ignite .NET library.  As  per document  we
> can  enable  authentication  if  persistence  is  enabled  for  one data
> region.
>
> How can we extend  the security  in  Ignite .NET  to  authorize  only
> few  members  to  view and edit the data  in cache  and  for  others  to
> only have readonly permission.  Does  Ignite .NET provides  any custom
> plugins  through  which  we can  integrate  with  our  organizations  JWT
> token  Api to  achieve  the same.?  Currently as  per  docs  we can
> create  users  with passwords  from sqlline which  does  not  provide
> enterprise  level  security.
>
>
>
> https://ignite.apache.org/docs/latest/security/authentication
>
>
>
>
>
> Any  example  or  blog  will be  helpful.
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Query regarding Generic Entity Framework Cache Store.

[Pavel Tupitsyn]

Hi Satyajit,

Generic cache store usage is commented out in the sample repo:
https://github.com/ptupitsyn/ignite-net-examples/blob/master/EFCacheStore/IgniteEFCacheStore/PostCacheStore.cs#L124

Have you tried replacing "return new PostCacheStore();" with the commented
code?

On Mon, Aug 14, 2023 at 5:15 AM  wrote:

> Hi  Pavel,
>
>
>
> Do  you  have  the  example  code  which  calls  the  generic  entity
> framework cache  store.  In the example  mentioned  here  I don’t see  the
> generic  entity framework cache  store  getting  called. Can you  share
> the  code  through  which  you  tested  the  generic entity framework
>  cache store?
>
>
>
>
>
> https://ptupitsyn.github.io/Entity-Framework-Cache-Store/
>
>
>
>
> https://github.com/ptupitsyn/ignite-net-examples/blob/master/EFCacheStore/IgniteEFCacheStore/EntityFrameworkCacheStore.cs
>
>
>
> Need an  example  which  calls  EntityFrameworkCacheStore.
>
>
>
> Regards
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Unable to make rest api calls to ignite on local host

[Pavel Tupitsyn]

Answered on StackOverflow [1]:

"it's required to move the ignite-rest-http module from the
{IGNITE_HOME}/libs/optional/ directory to the {IGNITE_HOME}/libs to enable
REST API."

[1]
https://stackoverflow.com/questions/76790017/unable-to-make-rest-api-calls-to-ignite-on-local-host

On Fri, Jul 28, 2023 at 8:56 PM Amitkumar Maheshwari <
amitkumar.maheshw...@automationanywhere.com> wrote:

> I am referring this document https://ignite.apache.org/docs/latest/setup
>
> I have downloaded the Ignite 2.15.0 binaries
>
> and followed every steps of setting up and rest-api document
>
> https://ignite.apache.org/docs/latest/setup
> https://ignite.apache.org/docs/latest/quick-start/restapi#ignite-quick-start-guide-for-rest-api
>
> I have java version 18
>
> I can see when I ran two nodes with default configurations (as suggested
> in docs), i can see the proper command line entries.
>
> However, when i try to make a rest call,
>
> by curl http://localhost:8080/ignite?cmd=version
>
> it fails with the error Error: connect ECONNREFUSED 127.0.0.1:8080
>
> Even though the document clearly saying that there isn't any additional
> configuration needed for it, is there anything I am missing?
>
>
>
>
>
> Thanks and Regards,
>
> *Amitkumar Maheshwari ** | * Sr, Software Engineer
>
> www.automationanywhere.com/in
>
> Automation Anywhere
>
> Ground Floor (North West Part) Alembic Business Park Premises Alembic Road
> Gorwa.
>
>
>
>
>

Re: Cache write synchronization mode

[Pavel Tupitsyn]

> If the primary node 'comes back' after the primary node failure would you
expect the new value to propagate to all nodes?

I don't think so, but I'm not 100% sure - could you ask about this specific
case in a separate thread?

On Tue, Jul 25, 2023 at 8:50 AM Raymond Wilson 
wrote:

> >>  However, if a primary node fails before at least 1 backup
> node receives an update, then the update will be lost, and all nodes will
> have the old value.
>
> Does this imply that it is a good idea to have the FullSync write
> synchronization mode? If the primary node 'comes back' after the primary
> node failure would you expect the new value to propagate to all nodes?
>
>
> On Tue, Jul 25, 2023 at 5:22 PM Pavel Tupitsyn 
> wrote:
>
>> > if a hard failure occurs to one of the backup servers in the replicated
>> cache will the server that failed have an inconsistent (old) copy of that
>> element in the replicated cache when it restarts
>>
>> If only a backup server fails and restarts, it will get new data from the
>> primary node, no issue here.
>> However, if a primary node fails before at least 1 backup node receives
>> an update, then the update will be lost, and all nodes will have the old
>> value.
>>
>> Related: CacheConfiguration.ReadFromBackup property is true by default,
>> meaning that with PrimarySync it is possible to get old value from a backup
>> node after an update, before backups receive new data.
>>
>> On Mon, Jul 24, 2023 at 11:51 PM Raymond Wilson <
>> raymond_wil...@trimble.com> wrote:
>>
>>> Hi Pavel,
>>>
>>> I understand the differences between the sync modes in terms of when the
>>> write returns. What I want to understand is if there are consistency risks
>>> with the PrimarySync versus FullSync modes.
>>>
>>> For example, if I have 4 nodes participating in the replicated cache
>>> (and am using the default PrimarySync mode), then the write will return
>>> once the primary node in the replicated cache has completed the write. At
>>> that point if a hard failure occurs to one of the backup servers in the
>>> replicated cache will the server that failed have an inconsistent (old)
>>> copy of that element in the replicated cache when it restarts?
>>>
>>> Raymond.
>>>
>>>
>
> --
> <http://www.trimble.com/>
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>

Re: Cache write synchronization mode

[Pavel Tupitsyn]

> if a hard failure occurs to one of the backup servers in the replicated
cache will the server that failed have an inconsistent (old) copy of that
element in the replicated cache when it restarts

If only a backup server fails and restarts, it will get new data from the
primary node, no issue here.
However, if a primary node fails before at least 1 backup node receives an
update, then the update will be lost, and all nodes will have the old value.

Related: CacheConfiguration.ReadFromBackup property is true by default,
meaning that with PrimarySync it is possible to get old value from a backup
node after an update, before backups receive new data.

On Mon, Jul 24, 2023 at 11:51 PM Raymond Wilson 
wrote:

> Hi Pavel,
>
> I understand the differences between the sync modes in terms of when the
> write returns. What I want to understand is if there are consistency risks
> with the PrimarySync versus FullSync modes.
>
> For example, if I have 4 nodes participating in the replicated cache (and
> am using the default PrimarySync mode), then the write will return once the
> primary node in the replicated cache has completed the write. At that point
> if a hard failure occurs to one of the backup servers in the replicated
> cache will the server that failed have an inconsistent (old) copy of that
> element in the replicated cache when it restarts?
>
> Raymond.
>
>

Re: Ignite data region off-heap allocation

[Pavel Tupitsyn]

> If this flag is true will Ignite proactively allocate and use all pages
in a data region, rather than incrementally?

LazyMemoryAllocation means whether memory for DataRegion will be allocated
only when the first cache is created in that region (when true), or
immediately (when false)

https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/DataRegionConfiguration.html#isLazyMemoryAllocation--


On Wed, Jul 19, 2023 at 12:50 PM Raymond Wilson 
wrote:

> Just FYI, we have held off any memory pressure changes in the meantime
> while we continue to investigate the memory issues we have.
>
> On Tue, 18 Jul 2023 at 9:07 AM, Raymond Wilson 
> wrote:
>
>> Hi Pavel,
>>
>> This area is confusing. There is no indication that the memory pressure
>> applies to any individual object or allocation, so there is clearly no
>> association between memory pressure and any particular resource.
>>
>> I get your argument that .Net can 'see' allocated memory. What is unclear
>> is whether it cares about actually allocated and used pages, or committed
>> pages.
>>
>> I see there is a LazyMemoryAllocation (default: true) for data regions.
>> Some data regions set this to false, eg:
>>
>> ^--   sysMemPlc region [type=internal, persistence=true,
>> lazyAlloc=false,
>> ^--   metastoreMemPlc region [type=internal, persistence=true,
>> lazyAlloc=false,
>> ^--   TxLog region [type=internal, persistence=true, lazyAlloc=false,
>>
>> The documentation is not clear on the effect of this flag other than to
>> say it is for 'Lazy memory allocation'. If this flag is true will Ignite
>> proactively allocate and use all pages in a data region, rather than
>> incrementally?
>>
>> Thanks,
>> Raymond.
>>
>>
>> On Tue, Jul 11, 2023 at 10:55 PM Pavel Tupitsyn 
>> wrote:
>>
>>> > I can’t see another way of letting . Net know that it can’t have
>>> access to all the ‘free’ memory in the process
>>>
>>> You don't need to tell .NET how much memory is currently available. It
>>> is the job of the OS. .NET can "see" the size of the unmanaged heap.
>>>
>>> To quote another explanation [1]:
>>>
>>> > The point of AddMemoryPressure is to tell the garbage collector that
>>> there's a large amount of memory allocated with that object.
>>> > If it's unmanaged, the garbage collector doesn't know about it; only
>>> the managed portion.
>>> > Since the managed portion is relatively small, the GC may let it pass
>>> for garbage collection several times, essentially wasting memory that might
>>> need to be freed.
>>>
>>> I really don't think AddMemoryPressure is the right thing to do in your
>>> case.
>>> If you run into OOM issues, then look into Ignite memory region settings
>>> [2] and/or adjust application memory usage on the .NET side, so that the
>>> sum of those is not bigger than available RAM.
>>>
>>> [1]
>>> https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource
>>> [2]
>>> https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-default-data-region
>>>
>>> On Tue, Jul 11, 2023 at 11:48 AM Raymond Wilson <
>>> raymond_wil...@trimble.com> wrote:
>>>
>>>> How do Ignite .Net server nodes manage this memory issue in other
>>>> projects?
>>>>
>>>> On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson <
>>>> raymond_wil...@trimble.com> wrote:
>>>>
>>>>> Oops, commutes => committed
>>>>>
>>>>> On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson <
>>>>> raymond_wil...@trimble.com> wrote:
>>>>>
>>>>>> I can’t see another way of letting . Net know that it can’t have
>>>>>> access to all the ‘free’ memory in the process when a large slab of that 
>>>>>> is
>>>>>> spoken for in terms of memory commutes to Ignite data regions.
>>>>>>
>>>>>> In the current setup, as time goes on and Ignite progressively fills
>>>>>> the allocated cache ram then system behaviour changes and can result in 
>>>>>> out
>>>>>> of memory issues. I think I would prefer consistent system behaviour wrt 
>>>>>> to
>>>>>> allocated resources from the start.
>>>>>>
>>>>>> Raymond.
&

Re: Cache write synchronization mode

[Pavel Tupitsyn]

Please check this reply on StackOverflow - does it help?

https://stackoverflow.com/questions/45497095/explicit-setting-of-write-synchronization-mode-full-sync-needed-for-replicated-c


On Wed, Jul 19, 2023 at 1:38 AM Raymond Wilson 
wrote:

> I have a query regarding the CacheWriteSynchronizationMode in
> CacheConfiguration.
>
> This enum is defined like this in the .Net client:
>
>   public enum CacheWriteSynchronizationMode
>   {
> /// 
> /// Mode indicating that Ignite should wait for write or commit
> replies from all nodes.
> /// This behavior guarantees that whenever any of the atomic or
> transactional writes
> /// complete, all other participating nodes which cache the written
> data have been updated.
> /// 
> FullSync,
> /// 
> /// Flag indicating that Ignite will not wait for write or commit
> responses from participating nodes,
> /// which means that remote nodes may get their state updated a bit
> after any of the cache write methods
> /// complete, or after {@link Transaction#commit()} method completes.
> /// 
> FullAsync,
> /// 
> /// This flag only makes sense for {@link CacheMode#PARTITIONED} mode.
> When enabled, Ignite will wait
> /// for write or commit to complete on primary node, but will not wait
> for backups to be updated.
> /// 
> PrimarySync,
>   }
>
> We have some replicated caches (where cfg.CacheMode =
> CacheMode.Replicated), but we don't specify the WriteSynchronizationMode.
>
> I note in the comment for PrimarySync (the default) that this "only makes
> sense" for Partitioned caches. Given we don't set this mode for our
> replicated caches then they will be using the PrimarySync write
> synchronization mode.
>
> The core Ignite help does not distinguish these synchronization modes and
> strongly implies that all three synchronization modes have equivalent
> consistency guarantees, but the help comment implies that replicated caches
> should use either FullSync or FullAsync to ensure all replicated contexts
> receive the written value.
>
> As a background, I am investigating an issue in our system that could be
> explained by replicated caches not having consistent values and am writing
> some triage tooling to prove if that is the case or not by comparing the
> stored values in each of the replicates cache nodes, However, I'm also
> doing some due diligence on our configuration and ran into this item.
>
> Thanks,
> Raymond.
>
>
> --
> 
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> 
>

Re: Ignite data region off-heap allocation

[Pavel Tupitsyn]

> I can’t see another way of letting . Net know that it can’t have access
to all the ‘free’ memory in the process

You don't need to tell .NET how much memory is currently available. It is
the job of the OS. .NET can "see" the size of the unmanaged heap.

To quote another explanation [1]:

> The point of AddMemoryPressure is to tell the garbage collector that
there's a large amount of memory allocated with that object.
> If it's unmanaged, the garbage collector doesn't know about it; only the
managed portion.
> Since the managed portion is relatively small, the GC may let it pass for
garbage collection several times, essentially wasting memory that might
need to be freed.

I really don't think AddMemoryPressure is the right thing to do in your
case.
If you run into OOM issues, then look into Ignite memory region settings
[2] and/or adjust application memory usage on the .NET side, so that the
sum of those is not bigger than available RAM.

[1]
https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource
[2]
https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-default-data-region

On Tue, Jul 11, 2023 at 11:48 AM Raymond Wilson 
wrote:

> How do Ignite .Net server nodes manage this memory issue in other
> projects?
>
> On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson 
> wrote:
>
>> Oops, commutes => committed
>>
>> On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson <
>> raymond_wil...@trimble.com> wrote:
>>
>>> I can’t see another way of letting . Net know that it can’t have access
>>> to all the ‘free’ memory in the process when a large slab of that is spoken
>>> for in terms of memory commutes to Ignite data regions.
>>>
>>> In the current setup, as time goes on and Ignite progressively fills the
>>> allocated cache ram then system behaviour changes and can result in out of
>>> memory issues. I think I would prefer consistent system behaviour wrt to
>>> allocated resources from the start.
>>>
>>> Raymond.
>>>
>>> On Tue, 11 Jul 2023 at 3:57 PM, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> Are you sure this is necessary?
>>>>
>>>> GC.AddMemoryPressure documentation [1] states that this will "improve
>>>> performance only for types that exclusively depend on finalizers".
>>>>
>>>> [1]
>>>> https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0
>>>>
>>>> On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson <
>>>> raymond_wil...@trimble.com> wrote:
>>>>
>>>>> I'm making changes to add memory pressure to the GC to take into
>>>>> account memory committed to the Ignite data regions as this will be
>>>>> unmanaged memory allocations from the perspective of the GC.
>>>>>
>>>>> I don't call seeing anything related to this for .Net clients in the
>>>>> documentation. Are you aware of any?
>>>>>
>>>>> Raymond.
>>>>>
>>>>> On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson <
>>>>> raymond_wil...@trimble.com> wrote:
>>>>>
>>>>>> Thanks Pavel, this makes sense.
>>>>>>
>>>>>> Querying the .Net Process instance shows this as the difference
>>>>>> between PagesMemorySize (includes committed) versus WorkingSet (includes
>>>>>> uses/written to) size.
>>>>>> Raymond.
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> <http://www.trimble.com/>
>>>>> Raymond Wilson
>>>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>>>> 11 Birmingham Drive |
>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail=g>Christchurch,
>>>>> New Zealand
>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail=g>
>>>>> raymond_wil...@trimble.com
>>>>>
>>>>>
>>>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>>>>>
>>>> --
>>> <http://www.trimble.com/>
>>> Raymond Wilson
>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>> 11 Birmingham Drive | Christchurch, New Zealand
>>> raymond_wil...@trimble.com
>>>
>>>
>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>>>
>> --
>> <http://www.trimble.com/>
>> Raymond Wilson
>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>> 11 Birmingham Drive | Christchurch, New Zealand
>> raymond_wil...@trimble.com
>>
>>
>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>>
>
>
> --
> <http://www.trimble.com/>
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>

Re: Ignite data region off-heap allocation

[Pavel Tupitsyn]

Are you sure this is necessary?

GC.AddMemoryPressure documentation [1] states that this will "improve
performance only for types that exclusively depend on finalizers".

[1]
https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0

On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson 
wrote:

> I'm making changes to add memory pressure to the GC to take into account
> memory committed to the Ignite data regions as this will be unmanaged
> memory allocations from the perspective of the GC.
>
> I don't call seeing anything related to this for .Net clients in the
> documentation. Are you aware of any?
>
> Raymond.
>
> On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson 
> wrote:
>
>> Thanks Pavel, this makes sense.
>>
>> Querying the .Net Process instance shows this as the difference between
>> PagesMemorySize (includes committed) versus WorkingSet (includes
>> uses/written to) size.
>> Raymond.
>>
>>
>
> --
> 
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> 
>

Re: Ignite data region off-heap allocation

[Pavel Tupitsyn]

It is similar with .NET, try the following code:

Console.WriteLine("Press any key to allocate 100MB");
Console.ReadKey();
var bytes = new byte[100_000_000];

// At this point Linux reports ~6MB of memory used by the process.
Console.WriteLine("Array allocated, press any key to fill with 1s");
Console.ReadKey();
bytes.AsSpan().Fill(1);

// And now we see ~106MB of memory used by the process.
Console.WriteLine("Array filled, press any key to exit");
Console.ReadKey();

On Mon, Jul 10, 2023 at 9:17 AM Raymond Wilson 
wrote:

> Hi Pavel,
>
> I want to say this should be included in the ‘used’ memory for a process,
> but perhaps that is not correct.
>
> Raymond.
>
> On Mon, 10 Jul 2023 at 5:07 PM, Pavel Tupitsyn 
> wrote:
>
>> Hi Raymond,
>>
>> "allocated=94407MB" reported by Ignite is "committed" memory - requested
>> from the OS, but not entirely used/touched.
>>
>>
>> See
>> -
>> https://github.com/apache/ignite/blob/df685afb08e3c2297adb8fc6df435a7310e95e50/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L2369
>> -
>> https://serverfault.com/questions/1008584/committed-allocated-memory-in-linux-is-less-than-used-memory-how-is-that-possib
>> -
>> https://unix.stackexchange.com/questions/137773/is-inactive-memory-related-to-the-commited-but-unused
>>
>> On Sat, Jul 8, 2023 at 11:44 AM Raymond Wilson <
>> raymond_wil...@trimble.com> wrote:
>>
>>> Hi,
>>>
>>> We have an Ignite node reporting off-heap data region allocation like
>>> this in the logs:
>>>
>>> ^-- Off-heap memory [used=37077MB, free=60.81%, allocated=94407MB]
>>>
>>> The same process (.Net 7 running in a Kubernetes pod with 124Gb
>>> allocated out of 128Gb available on the node), reports this level of
>>> managed memory usage:
>>>
>>> Heartbeat: Total managed memory use: 43836.083Mb
>>>
>>> Clearly ~94Gb + ~44Gb (138Gb) is a lot more than both 128Gb and 124Gb
>>>
>>> The node in question has the initial and maximum allocation for the data
>>> region as 94208Mb (plus the system data region etc), so I expect the Ignite
>>> node to have allocated that much (which is indicated by the 94407Mb
>>> allocated figure noted in the log line.
>>>
>>> However, the .Net CLR is reporting nearly 48Gb of managed RAM usage in
>>> .Net, so something does not add up. Either .Net is lying about how much it
>>> is using, or Ignite is lying about how much RAM it actually allocated.
>>>
>>> I feel I am missing something here!
>>>
>>> Thanks,
>>> Raymond.
>>>
>>> --
>>> <http://www.trimble.com/>
>>> Raymond Wilson
>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>> 11 Birmingham Drive |
>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail=g>Christchurch,
>>> New Zealand
>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail=g>
>>> raymond_wil...@trimble.com
>>>
>>>
>>> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>>>
>> --
> <http://www.trimble.com/>
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> <https://worksos.trimble.com/?utm_source=Trimble_medium=emailsign_campaign=Launch>
>

Re: Ignite data region off-heap allocation

[Pavel Tupitsyn]

Hi Raymond,

"allocated=94407MB" reported by Ignite is "committed" memory - requested
from the OS, but not entirely used/touched.


See
-
https://github.com/apache/ignite/blob/df685afb08e3c2297adb8fc6df435a7310e95e50/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L2369
-
https://serverfault.com/questions/1008584/committed-allocated-memory-in-linux-is-less-than-used-memory-how-is-that-possib
-
https://unix.stackexchange.com/questions/137773/is-inactive-memory-related-to-the-commited-but-unused

On Sat, Jul 8, 2023 at 11:44 AM Raymond Wilson 
wrote:

> Hi,
>
> We have an Ignite node reporting off-heap data region allocation like this
> in the logs:
>
> ^-- Off-heap memory [used=37077MB, free=60.81%, allocated=94407MB]
>
> The same process (.Net 7 running in a Kubernetes pod with 124Gb allocated
> out of 128Gb available on the node), reports this level of managed memory
> usage:
>
> Heartbeat: Total managed memory use: 43836.083Mb
>
> Clearly ~94Gb + ~44Gb (138Gb) is a lot more than both 128Gb and 124Gb
>
> The node in question has the initial and maximum allocation for the data
> region as 94208Mb (plus the system data region etc), so I expect the Ignite
> node to have allocated that much (which is indicated by the 94407Mb
> allocated figure noted in the log line.
>
> However, the .Net CLR is reporting nearly 48Gb of managed RAM usage in
> .Net, so something does not add up. Either .Net is lying about how much it
> is using, or Ignite is lying about how much RAM it actually allocated.
>
> I feel I am missing something here!
>
> Thanks,
> Raymond.
>
> --
> 
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> 
>

Re: Data consistency in Ignite and BackUp Filter in Ignite .net node

[Pavel Tupitsyn]

Hi Satyajit,

> ensure  backup partitions  of  a  particular  node  is  created  in
another  node  and  not  in  same  node
No need to do anything, this is the default behavior: backup partitions
always reside on a different node.
To put it another way: every partition has a primary node and one or more
backup nodes. All those nodes are different (given enough nodes in the
cluster).

- AffinityBackupFilter is for complex use cases when the cluster spreads
multiple racks or availability zones [1] [2]
- If you have multiple nodes per machine, consider
RendezvousAffinityFunction.ExcludeNeighbors property [3]

[1]
https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.Rendezvous.ClusterNodeAttributeAffinityBackupFilter.html
[2]
https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution#backup-filter
[3]
https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cache.Affinity.AffinityFunctionBase.html#Apache_Ignite_Core_Cache_Affinity_AffinityFunctionBase_ExcludeNeighbors

On Tue, Jun 27, 2023 at 4:02 PM  wrote:

> Hi  Pavel,
>
>
>
> We are  running  6  node cluster  with  Active Active  setup. 3  nodes  in
> one  Datacenter  and  3  in  another datacenter with  external  cache
> store  enabled  for  some caches  and  native  persistence  enabled  for
> others.  Now  if  we  want  to  setup  backup  = 1  for  in memory  caches
> how  that  should  be  done  to  ensure  backup partitions  of  a
> particular  node  is  created  in another  node  and  not  in  same  node.
> Also  is  there  any specific settings  to  ensure data  consistency  for
> in  memory  caches.
>
>
>
> Can you  also  share  how  this settings can  be  achieved  in  Ignite(.NET) 
> ( AffinityBackupFilter in RendezvousAffinityFunction)
>
>
>
> Regards
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Monitoring Ignite .NET Nodes.

[Pavel Tupitsyn]

There is no difference. control.sh works the same way with embedded nodes.

Have you tried it? Any issues?

On Thu, Jun 8, 2023 at 12:32 PM  wrote:

> Hi  Pavel,
>
>
>
> I  have  hosted  Ignite as usual  windows service  and  not  using  Ignite
> distribution .  Ignite  is  embedded  to  my  .net core windows service
>  and  java  libraries are installed  as  part  of  Apache.Ignite Nuget
> package.
>
>
>
> So  now  how will  visor  and control script  will  connect  to  Ignite
> cluster  which  is embedded  to  my .NET service?
>
>
>
> Thanks
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Thursday, June 8, 2023 2:40 PM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Monitoring Ignite .NET Nodes.
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> .NET nodes are built on top of Java nodes, so everything applies the same
> way. Visor should work, as well as system views and JMX.
>
>
>
> The steps to use control.sh with .NET node:
>
> * Download the full distribution package
> https://dlcdn.apache.org/ignite/2.15.0/apache-ignite-2.15.0-bin.zip
> <https://clicktime.symantec.com/15t5ZtGoEHWjE1JjR6iaP?h=eh8Leht8OkV2FwrGjyXxZZaKXxfnAjK5pB7fMKpDhcY==https://dlcdn.apache.org/ignite/2.15.0/apache-ignite-2.15.0-bin.zip>
>
> * Unzip
>
> * Start .NET node from platforms/dotnet/bin/net6.0: dotnet
> ./Apache.Ignite.Executable.dll
>
> * Run control script from the package root: ./bin/control.sh --diagnostic
>
>
>
>
>
> On Thu, Jun 8, 2023 at 11:17 AM  wrote:
>
> Hi  Pavel,
>
>
>
> Question 1.  If  we are running  Ignite  as  .NET nodes  can  we activate
> the cluster  using  control  scripts  and  monitor  the same using  Visor
> tools  which  comes  with  Ignite distribution? Can  you  provide  the
> steps  if  yes.
>
>
>
> Question 2.  Can  we  use System  views  using  .NET library  and
> publish  the  metrics  to  Prometheus?
> https://ignite.apache.org/docs/latest/monitoring-metrics/system-views
> <https://clicktime.symantec.com/15t5eiU5guCKdx8exf7j1?h=GlEKwiZsDSSaCq340grfhciwzj3SBk7wzO8BPFt-tZI==https://ignite.apache.org/docs/latest/monitoring-metrics/system-views>
> :  Document  has  mentioned  examples  of  Java  client only? What  are
> the other  alternatives  for  having  effective monitoring  for  .NET
> nodes. JMX?
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>
> Barclays Execution Services Limited registered in E

Re: Monitoring Ignite .NET Nodes.

[Pavel Tupitsyn]

Hi Satyajit,

.NET nodes are built on top of Java nodes, so everything applies the same
way. Visor should work, as well as system views and JMX.

The steps to use control.sh with .NET node:
* Download the full distribution package
https://dlcdn.apache.org/ignite/2.15.0/apache-ignite-2.15.0-bin.zip
* Unzip
* Start .NET node from platforms/dotnet/bin/net6.0: dotnet
./Apache.Ignite.Executable.dll
* Run control script from the package root: ./bin/control.sh --diagnostic


On Thu, Jun 8, 2023 at 11:17 AM  wrote:

> Hi  Pavel,
>
>
>
> Question 1.  If  we are running  Ignite  as  .NET nodes  can  we activate
> the cluster  using  control  scripts  and  monitor  the same using  Visor
> tools  which  comes  with  Ignite distribution? Can  you  provide  the
> steps  if  yes.
>
>
>
> Question 2.  Can  we  use System  views  using  .NET library  and
> publish  the  metrics  to  Prometheus?
> https://ignite.apache.org/docs/latest/monitoring-metrics/system-views
> :  Document  has  mentioned  examples  of  Java  client only? What  are
> the other  alternatives  for  having  effective monitoring  for  .NET
> nodes. JMX?
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: 2 caches joining error

[Pavel Tupitsyn]

Hello, can you please share the code that does not work for you?

On Thu, Jun 1, 2023 at 7:52 AM Jiang Jacky  wrote:

> Hello
> Can someone share me with some good examples that how to make a join
> between 2 caches?
> I am using ignite 2.7 and mocking up the example from the video
> https://youtu.be/4CQalha3MyY
> I can query the table separately, however when I get the cache from
> “persons” and tried to join with “companies” it complains table “Company”
> is not existing.
>
> Please share me the complete example including the configuration.
>
> Thank you
>
> Get Outlook for iOS 
>

Re: Apache Ignite 3.0 Questions

[Pavel Tupitsyn]

1) There is no set date for now
2) Maksim is right, 2.x will live on for a long time
3) Yes, it is in the works

On Fri, May 26, 2023 at 3:01 PM Maksim Timonin 
wrote:

> Hi David,
>
>  2) What is the end of life matrix or end of support of 2.0?
>
>
> There is a stable community of great engineers that improve and support
> Ignite 2. Support of Ignite 2 will not be terminated in the foreseeable
> future.
>
> On Fri, May 26, 2023 at 1:04 PM David Bucek  wrote:
>
>> Hello,
>>
>> we are considering to use Apache Ignite for our software and we have
>> couple of questions:
>>
>> 1) When will Apache Ignite 3.0 be the stable version?
>> 2) What is the end of life matrix or end of support of 2.0?
>> 3) Will the 3.0 version still be embeddable to the Java application?
>> (very useful for us).
>>
>> Thank you for answers
>>
>>
>> David Bucek | Software Architect | MANTA
>>
>> *This email is intended solely for the addressee(s) and all its contents,
>> including all attachments and files transmitted with it, represent
>> confidential information. Unauthorized distribution, modification or
>> disclosure of its contents and unauthorized reliance on its contents are
>> prohibited. If you have received this email in error, please notify the
>> sender immediately by return email. Please then delete the email from your
>> system and do not (i) copy or distribute it, (ii) rely on its contents, or
>> (iii) disclose its contents to any person.*
>
>

Re: Ignite thin client continuous query listener cannot listen to all events

[Pavel Tupitsyn]

Thank you for the bug report, I will have a look.

On Thu, May 25, 2023 at 5:10 AM LonesomeRain  wrote:

> Hi, Jeremy McMillan
>
>
>
> I have created a Jira about this bug.
>
>
>
> https://issues.apache.org/jira/browse/IGNITE-19561
>
>
>
> By executing these two test codes in the description, it is easy to
> reproduce the problem.
>
>
>
> What should I do next? Continuously following this Jira?
>
>
>
>
>
> *发件人: *Jeremy McMillan 
> *发送时间: *2023年5月24日 23:24
> *收件人: *user@ignite.apache.org
> *主题: *Re: Ignite thin client continuous query listener cannot listen to
> all events
>
>
>
> Thanks for bringing this up!
>
>
>
>
> https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees
>
>
>
> This sounds like you may have found a bug, but the details you've provided
> are not sufficient to help others recreate and observe it for themselves,
> and this effort needs to be recorded in a ticket. Would you be able to sign
> up for a Jira account 
> and detail steps to reproduce this behavior?
>
> You may also want to research this:
>   https://issues.apache.org/jira/browse/IGNITE-8035
>
>
>
> On Mon, May 22, 2023 at 6:52 AM lonesomerain  wrote:
>
> *Hi,*
>
> *I have a question while using ignite 2.15.0*
>
>
>
> *Problem scenario:*
>
> Start the Ignite server of one node, start one thin client and create a
> continuous query listener, and then use 50 threads to add 500 data to the
> cache concurrently.
>
> *Problem phenomenon:*
>
> Through the information printed on the listener, it was found that the
> number of events listened to each time varies, possibly 496, 499 or 500...
>
> *Test Code:*
>
> public class StartServer {
>
> public static void main(String[] args) {
>
> Ignite ignite = Ignition.start();
>
> }
>
> }
>
>
>
> public class StartThinClient {
>
> public static void main(String[] args) throws InterruptedException {
>
> String addr = "127.0.0.1:10800";
>
>
>
> int threadNmu = 50;
>
>
>
> ClientConfiguration clientConfiguration = new
> ClientConfiguration();
>
> clientConfiguration.setAddresses(addr);
>
>
>
> IgniteClient client1 = Ignition.startClient(clientConfiguration);
>
>
>
> ClientCache cache1 =
> client1.getOrCreateCache("test");
>
>
>
> ContinuousQuery query = new ContinuousQuery<>();
>
> query.setLocalListener(new CacheEntryUpdatedListener Object>() {
>
> @Override
>
> public void onUpdated(Iterable>
> cacheEntryEvents) throws CacheEntryListenerException {
>
> Iterator> iterator =
> cacheEntryEvents.iterator();
>
> while (iterator.hasNext()) {
>
> CacheEntryEvent next = iterator.next();
>
> System.out.println("" + next.getKey());
>
> }
>
> }
>
> });
>
>
>
> cache1.query(query);
>
>
>
> IgniteClient client2 = Ignition.startClient(clientConfiguration);
>
> ClientCache cache2 = client2.cache("test");
>
>
>
> Thread[] threads = new Thread[threadNmu];
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i] = new Thread(new OperationInsert(cache2, i, 500,
> threadNmu));
>
> }
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i].start();
>
> }
>
> for (Thread thread : threads) {
>
> thread.join();
>
> }
>
>
>
> Thread.sleep(6);
>
>
>
> }
>
>
>
> static class OperationInsert implements Runnable {
>
>
>
> private ClientCache cache;
>
> private int k;
>
> private Integer test_rows;
>
> private Integer thread_cnt;
>
>
>
> public OperationInsert(ClientCache cache, int k,
> Integer test_rows, Integer thread_cnt) {
>
> this.cache = cache;
>
> this.k = k;
>
> this.test_rows = test_rows;
>
> this.thread_cnt = thread_cnt;
>
> }
>
>
>
> @Override
>
> public void run() {
>
> for (int i = 100 + (test_rows/thread_cnt) * k; i < 100
> + (test_rows/thread_cnt) * (k + 1); i++) {
>
> cache.put("" + i, "aaa");
>
> }
>
> }
>
> }
>
>
>
> }
>
>
>
> *Version:*
>
> The testing program uses Ignite version 2.15.0
>
> I attempted to insert data using one thread and did not observe any event
> loss. In addition, I also attempted an Ignite cluster with two or three
> nodes, which can still listen to all 500 events even when inserting data
> using multiple threads. May I ask if this issue only occurs at a single
> node? Are there any good solutions?
>
>
>

Re: Query :.NET Ignite Node deployment and Monitoring

[Pavel Tupitsyn]

I've put together a short example:
https://gist.github.com/ptupitsyn/633bf86dd4beaf48d604fab97abb2dd1

It uses Ignite.NET from NuGet, and loads the REST module from an external
directory.

On Wed, May 3, 2023 at 8:14 AM  wrote:

> Hi Pavel,
>
>
>
> Basically  want  to  collect  cluster  metrics  and  show  it  in some
> dashboard.  So  wanted  to  take  help  of  Ignite  rest  points.
>
> https://ignite.apache.org/docs/latest/restapi
> <https://clicktime.symantec.com/15t5Zt4yLtMwuRxhLNAQz?h=9IiYXYTQoz6LcsqZt3SYnge_GM-KLLF49vTu-M5IZfk==https://ignite.apache.org/docs/latest/restapi>
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* satyajit.mandal.barclays.com via user 
> *Sent:* Wednesday, May 3, 2023 10:31 AM
> *To:* ptupit...@apache.org
> *Cc:* user@ignite.apache.org
> *Subject:* RE: Query :.NET Ignite Node deployment and Monitoring
>
>
>
> CAUTION: This email originated from outside our organisation -
> user@ignite.apache.org Do not click on links, open attachments, or
> respond unless you recognize the sender and can validate the content is
> safe.
>
> Hi  Pavel,
>
>
>
> Am  referring  to  this  https://ignite.apache.org/docs/latest/restapi
> <https://clicktime.symantec.com/15t5Zt4yLtMwuRxhLNAQz?h=9IiYXYTQoz6LcsqZt3SYnge_GM-KLLF49vTu-M5IZfk==https://ignite.apache.org/docs/latest/restapi>
>
> To enable HTTP connectivity, make sure that the ignite-rest-http module
> is enabled. If you use the binary distribution, copy the ignite-rest-http 
> module
> from IGNITE_HOME/libs/optional/ to the IGNITE_HOME/libs folder
>
>
>
>
>
> How  this  feature  can be  done  via  .NET code ?
>
>
>
> Thanks
>
> Satyajit
>
>
>
>
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Wednesday, May 3, 2023 10:21 AM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Query :.NET Ignite Node deployment and Monitoring
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> NuGet deployment refers to the fact that you install Ignite from NuGet.
>
>
>
> Single-file deployment is about packaging your entire app (including your
> code, .NET code, Ignite dlls and jars) as a single file:
>
>
> https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview?tabs=cli
> <https://clicktime.symantec.com/15t5eiGFfx1sjHPBnfBVH?h=rXQIwARAWC6FIrgZmaBiIdFnGHbaEJCdYNMnvtFRwHs==https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview?tabs%3Dcli>
>
>
>
>
>
> > Is  this  issue  fixed  with  .NET 6  AND  Ignite 2.14?
>
> Yes.
>
>
>
>
>
> > How  can  we  enable  Rest  points  so  that  we  can  collect  metrics
>  for  Ignite cluster  like  heap  memory  usage  etc.. when  we are
>  hosting  Ignite as  .net node from  code.?
>
> Can you please clarify - which "Rest points" would you like to enable?
>
>
>
> On Tue, May 2, 2023 at 3:47 PM  wrote:
>
> Hi  Pavel,
>
>
>
> We  are planning  to  deploy WebApi  as  windows  service  which  will
> start Ignite  on  startup. Going  through the  Ignite deployment
> documentation  specific  to  .NET  and  it  mentions  these two  options.
>
> *NuGet Deployment:*
>
> *Single File Deployment:*
>
> *What’s  the  difference  between  these two deployments? *
>
> ·For  Single File  Deployment it  mentions  that  See Troubleshooting:
> *DllNotFoundException*
> <https://clicktime.symantec.com/15t5Zt4yDLLHKLZGF6nLf?h=JTE1fXE9H6ZIUWyT-bU4e81riTS8sBpW5vY2ZmyLHso==https://ignite.apache.org/docs/latest/net-specific/net-troubleshooting.html%23libcoreclr-not-found>
>  for
> a workaround that is required on .NET 5 with some Ignite versions. Is
> this  issue  fixed  with  .NET 6  AND  Ignite 2.14?
>
> *2nd  Query*  :  How  can  we  enable  Rest  points  so  that  we  can
> collect  metrics  for  Ignite cluster  like  heap  memory  usage  etc..
> when  we are  hosting  Ignite as  .net node from  code.?
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Bar

Re: Query :.NET Ignite Node deployment and Monitoring

[Pavel Tupitsyn]

Hi Satyajit,

NuGet deployment refers to the fact that you install Ignite from NuGet.

Single-file deployment is about packaging your entire app (including your
code, .NET code, Ignite dlls and jars) as a single file:
https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview?tabs=cli


> Is  this  issue  fixed  with  .NET 6  AND  Ignite 2.14?
Yes.


> How  can  we  enable  Rest  points  so  that  we  can  collect  metrics
 for  Ignite cluster  like  heap  memory  usage  etc.. when  we are
 hosting  Ignite as  .net node from  code.?
Can you please clarify - which "Rest points" would you like to enable?

On Tue, May 2, 2023 at 3:47 PM  wrote:

> Hi  Pavel,
>
>
>
> We  are planning  to  deploy WebApi  as  windows  service  which  will
> start Ignite  on  startup. Going  through the  Ignite deployment
> documentation  specific  to  .NET  and  it  mentions  these two  options.
>
> *NuGet Deployment:*
>
> *Single File Deployment:*
>
> *What’s  the  difference  between  these two deployments? *
>
> ·For  Single File  Deployment it  mentions  that  See Troubleshooting:
> *DllNotFoundException*
> 
>  for
> a workaround that is required on .NET 5 with some Ignite versions. Is
> this  issue  fixed  with  .NET 6  AND  Ignite 2.14?
>
> *2nd  Query*  :  How  can  we  enable  Rest  points  so  that  we  can
> collect  metrics  for  Ignite cluster  like  heap  memory  usage  etc..
> when  we are  hosting  Ignite as  .net node from  code.?
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Query on External CacheStore implemented using .NET

[Pavel Tupitsyn]

External cache store will work in any case - server, thick client, thin
client.

On Wed, Apr 26, 2023 at 4:05 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

>
>
> Hi  Pavel,
>
>
>
> We  have  implemented  custom  external  cache store.  We  are  running
> three nodes  as  server nodes  with  external  cache  store  implemented.
> Now  if  we  start  a  server  with  clientmode  = true  ( thick  client)
> and  do  some cache  operations  will  external  cache store  work in  this
> case ? Or  all  the cache  operations  has  to  be  done  via  server
> nodes?
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: SQL objects in Ignite ( .NET) not showing up with JDBC drivers

[Pavel Tupitsyn]

The table goes into the "cacheName" schema - you have to load and expand
other schemas in IntelliJ to see it:

[image: image.png]

On Tue, Apr 18, 2023 at 7:19 AM  wrote:

> Hi  Pavel,
>
>
>
> Here  is  my  query  entity  class
>
>
>
> public class Person
>
> {
>
> // Indexed field. Will be visible to the SQL engine.
>
> [QuerySqlField(IsIndexed = true)] public long Id;
>
>
>
> //Queryable field. Will be visible to the SQL engine
>
> [QuerySqlField] public string Name;
>
>
>
> //Will NOT be visible to the SQL engine.
>
> public int Age;
>
>
>
> [QuerySqlField(IsIndexed = true, IsDescending = true)]
>
> public float Salary;
>
> }
>
>
>
>
>
> And  here  is  the  code to  create table using  .NET Api.
>
>
>
> var client = Ignition.Start(clientCfg);
>
>
>
> Console.WriteLine("Total nodes: " + client.GetCluster().GetNodes().Count);
>
>
>
> var cacheCfg = new CacheConfiguration
>
> {
>
> Name = "cacheName",
>
> QueryEntities = new[]
>
>  {
>
> new QueryEntity(typeof(int), typeof(Person))
>
> }
>
> };
>
>
>
> var cache = client.CreateCache(cacheCfg);
>
>
>
> Person val = new Person();
>
> val.Name = "Satya";
>
> val.Id = 5;
>
> val.Age = 39;
>
> val.Salary = 25000;
>
> cache.Put(5, val);
>
>
>
>
>
> Now  we  want  to  see  this  table  to  be  shown  up  automatically  in
> Intellij  using  JDBC  driver(ignite-core-2.14.0.jar).
>
>
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Tuesday, April 18, 2023 9:40 AM
> *To:* user@ignite.apache.org; Mandal, Satyajit: IT (PUN) <
> satyajit.man...@barclays.com>
> *Subject:* Re: SQL objects in Ignite ( .NET) not showing up with JDBC
> drivers
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> How do you create tables in .NET? Are you sure QueryEntities are
> configured correctly? Can you share the code please?
>
> Normally tables should show up in JDBC.
>
>
>
> On Mon, Apr 17, 2023 at 3:27 PM satyajit.mandal.barclays.com
> <https://clicktime.symantec.com/15siFABQmz4yreNgp3kzV?h=LWuZz2wR-l0xerqVBEyXEUuUAVfNfwuN0R01xD0-M0A==http://satyajit.mandal.barclays.com>
> via user  wrote:
>
> Hi  Pavel,
>
>
>
> When  we  create  SQL objects ( Tables)  using  .NET  Ignite  Api they
> don’t  show up in  IntellijIdea  with  Ignite JDBC  drivers.  We want  to
> see  the  tables  so  that  we can  query  the  cache  using  normal
> Select  statements  using  JDBC  driver.
>
>
>
> Is  there  any  alternative  for same to  view  the  tables  in  cache
> when created  using  .NET Api?
>
>
>
> Thanks
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not rela

Re: SQL objects in Ignite ( .NET) not showing up with JDBC drivers

[Pavel Tupitsyn]

Hi Satyajit,

How do you create tables in .NET? Are you sure QueryEntities are configured
correctly? Can you share the code please?
Normally tables should show up in JDBC.

On Mon, Apr 17, 2023 at 3:27 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Pavel,
>
>
>
> When  we  create  SQL objects ( Tables)  using  .NET  Ignite  Api they
> don’t  show up in  IntellijIdea  with  Ignite JDBC  drivers.  We want  to
> see  the  tables  so  that  we can  query  the  cache  using  normal
> Select  statements  using  JDBC  driver.
>
>
>
> Is  there  any  alternative  for same to  view  the  tables  in  cache
> when created  using  .NET Api?
>
>
>
> Thanks
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Ignite configuration using .NET Thick client

[Pavel Tupitsyn]

I don't see any issue with this setup. It is a perfectly valid use case.

Make sure to check this doc regarding IIS:
https://ignite.apache.org/docs/latest/net-specific/net-deployment-options#iis-application-pool-lifecycle-appdomains-and-ignite-net

On Thu, Apr 13, 2023 at 3:15 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Pavel,
>
>
>
> Thanks for  below  information.  One  clarification  needed  from  your
> side . Am  want  to  run  .net WebApi  in three different machines and  it
> will  start  ignite as  server  node and do  cache  operations  ( CRUD
> operations )  using  controller exposed. These webapi’s  will  be  hosted
> in  IIS.  So a  cluster  with  three server  nodes  will  form  and  it
> will  serve  cache  operations.  Do  you  see  any  issue  with  this
> setup?Load  balancing  will be  done  using  .NET frameworks like Dapr etc..
>
>
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Friday, April 7, 2023 10:17 AM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Ignite configuration using .NET Thick client
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> You said you wanted 4 nodes in 1 process. Sorry, I'm not sure I understand
> the question.
>
>
>
> TcpDiscoveryMulticastIpFinder uses multicast, you don't need to specify
> endpoints. Nodes will find each other in the same subnet, no matter if on
> one machine or different ones.
>
> However, it is not recommended for production use.
>
>
>
> For proper setup, use TcpDiscoveryStaticIpFinder (same as
> TcpDiscoveryVmIpFinder in Java)
>
>
>
> DiscoverySpi = new TcpDiscoverySpi
> {
> IpFinder = new TcpDiscoveryStaticIpFinder
> {
> Endpoints = new[]
> {
> "machine1:47500..47505",
> "machine2:47500..47505",
> "machine3:47500..47505",
> }
> }
> }
>
>
>
> https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery
> <https://clicktime.symantec.com/15siFA7ph2RLqD7k2XKr7?h=ZX1pgZKpFCJizmKFIfdRWXfCeqtql4quzCYpgiQfUgQ==https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery>
>
>
>
> On Thu, Apr 6, 2023 at 7:39 PM  wrote:
>
> Hi  Pavel,
>
>
>
> Below  code  snippet  doesn’t  mentions  the  IP  address  of  4
> different  nodes( different  machines) so  how  will  this  make the cache
> run  in  different  node  and  form  a  cluster  of  4  nodes.
>
>
>
> Do  we  need  to  start  the  thick  client  in  all  nodes  one  by  one
> with  the same server cfg  and  add this in  server cfg. .
>
>
>
>   DiscoverySpi = new TcpDiscoverySpi
>
> {
>
> IpFinder = new TcpDiscoveryMulticastIpFinder
>
> {
>
> Endpoints = new[]
>
> {
>
> "xxx.0.0.xxx:47500..47502",
>
>"yyy.0.0.:47500..47502",
>
>"zzz.0.0.:47500..47502"
>
>
>
> }
>
> }
>
> },
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Thursday, April 6, 2023 9:51 PM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Ignite configuration using .NET Thick client
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> using Apache.Ignite.Core;
> using Apache.Ignite.Core.Deployment;
> using Apache.Ignite.Core.Log;
>
> var serverCfg = new IgniteConfiguration
> {
> Localhost = "127.0.0.1",
> JvmOptions = new[]
> {
> "-DIGNITE_QUIET=true",
> "-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true"
> },
> Logger = new ConsoleLogger
> {
> MinLevel = LogLevel.
> *Error *},
> PeerAssemblyLoadingMode = PeerAssemblyLoadingMode.*CurrentAppDomain*,
> AutoGenerateIgniteInstanceName = true
> };
>
> var clientCfg = new IgniteConfiguration(serverCfg)
> {
> ClientMode = true
> *// Thick client. *};
>
>
> *// Start 3 server nodes. *var servers = Enumerab

Re: Ignite configuration using .NET Thick client

[Pavel Tupitsyn]

You said you wanted 4 nodes in 1 process. Sorry, I'm not sure I understand
the question.

TcpDiscoveryMulticastIpFinder uses multicast, you don't need to specify
endpoints. Nodes will find each other in the same subnet, no matter if on
one machine or different ones.
However, it is not recommended for production use.

For proper setup, use TcpDiscoveryStaticIpFinder (same as
TcpDiscoveryVmIpFinder in Java)

DiscoverySpi = new TcpDiscoverySpi
{
IpFinder = new TcpDiscoveryStaticIpFinder
{
Endpoints = new[]
{
"machine1:47500..47505",
"machine2:47500..47505",
"machine3:47500..47505",
}
}
}

https://ignite.apache.org/docs/latest/clustering/tcp-ip-discovery

On Thu, Apr 6, 2023 at 7:39 PM  wrote:

> Hi  Pavel,
>
>
>
> Below  code  snippet  doesn’t  mentions  the  IP  address  of  4
> different  nodes( different  machines) so  how  will  this  make the cache
> run  in  different  node  and  form  a  cluster  of  4  nodes.
>
>
>
> Do  we  need  to  start  the  thick  client  in  all  nodes  one  by  one
> with  the same server cfg  and  add this in  server cfg. .
>
>
>
>   DiscoverySpi = new TcpDiscoverySpi
>
> {
>
> IpFinder = new TcpDiscoveryMulticastIpFinder
>
> {
>
> Endpoints = new[]
>
> {
>
> "xxx.0.0.xxx:47500..47502",
>
>"yyy.0.0.:47500..47502",
>
>"zzz.0.0.:47500..47502"
>
>
>
> }
>
> }
>
> },
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Thursday, April 6, 2023 9:51 PM
> *To:* Mandal, Satyajit: IT (PUN) 
> *Cc:* user@ignite.apache.org
> *Subject:* Re: Ignite configuration using .NET Thick client
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> using Apache.Ignite.Core;
> using Apache.Ignite.Core.Deployment;
> using Apache.Ignite.Core.Log;
>
> var serverCfg = new IgniteConfiguration
> {
> Localhost = "127.0.0.1",
> JvmOptions = new[]
> {
> "-DIGNITE_QUIET=true",
> "-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true"
> },
> Logger = new ConsoleLogger
> {
> MinLevel = LogLevel.
> *Error *},
> PeerAssemblyLoadingMode = PeerAssemblyLoadingMode.*CurrentAppDomain*,
> AutoGenerateIgniteInstanceName = true
> };
>
> var clientCfg = new IgniteConfiguration(serverCfg)
> {
> ClientMode = true
> *// Thick client. *};
>
>
> *// Start 3 server nodes. *var servers = Enumerable.*Range*(1, 3).Select(_
> => Ignition.*Start*(serverCfg)).ToList();
>
>
> *// Start a thick client node. *var client = Ignition.*Start*(clientCfg);
>
> Console.*WriteLine*("Total nodes: " + client.GetCluster().GetNodes().Count
> );
>
>
>
> On Thu, Apr 6, 2023 at 5:39 PM  wrote:
>
> Hi  Pavel,
>
>
>
> I  need  to  start  cache  in  4  different  nodes  using  .NET Thick
> client in one  process.  Can you  share  me  the sample  Ignite
> Configuration  and  steps  to  do  that.
>
>
>
> Previously  I  was  using  thin  client  and  that  time  I used  to
> start  Ignite  nodes  using  Spring.xml  one  by  one  in  each  node.
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Thursday, April 6, 2023 7:57 PM
> *To:* user@ignite.apache.org; Mandal, Satyajit: IT (PUN) <
> satyajit.man...@barclays.com>
> *Subject:* Re: Ignite configuration using .NET Thick client
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> 1. You don't need to use Spring XML unless you have to set a property that
> is not exposed in .NET API
>
> 2. Do you want to start 4 nodes in one process? Or multiple processes on
> the same machine? What's your desired setup?
>
>
>
> Pavel
>
>
>
> On Thu, Apr 6, 2023 at 4:40 PM satyajit.mandal.barclays.com
> <https://clicktime.symantec.com/15sLvSJyuA6fEjRJ369u2?h=LSypiaGaE8p6-HR8_7rXGT-aHM5RsH9EHUzLHQuAzSc==http://satyajit.mandal.barclays.com>
> via user  wrote:
>
> Hi  Pavel,
>
>
>
> Need  to  understand  one  thi

Re: Ignite configuration using .NET Thick client

[Pavel Tupitsyn]

using Apache.Ignite.Core;
using Apache.Ignite.Core.Deployment;
using Apache.Ignite.Core.Log;

var serverCfg = new IgniteConfiguration
{
Localhost = "127.0.0.1",
JvmOptions = new[]
{
"-DIGNITE_QUIET=true",
"-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true"
},
Logger = new ConsoleLogger
{
MinLevel = LogLevel.Error
},
PeerAssemblyLoadingMode = PeerAssemblyLoadingMode.CurrentAppDomain,
AutoGenerateIgniteInstanceName = true
};

var clientCfg = new IgniteConfiguration(serverCfg)
{
ClientMode = true // Thick client.
};

// Start 3 server nodes.
var servers = Enumerable.Range(1, 3).Select(_ => Ignition.Start(serverCfg)).
ToList();

// Start a thick client node.
var client = Ignition.Start(clientCfg);

Console.WriteLine("Total nodes: " + client.GetCluster().GetNodes().Count);

On Thu, Apr 6, 2023 at 5:39 PM  wrote:

> Hi  Pavel,
>
>
>
> I  need  to  start  cache  in  4  different  nodes  using  .NET Thick
> client in one  process.  Can you  share  me  the sample  Ignite
> Configuration  and  steps  to  do  that.
>
>
>
> Previously  I  was  using  thin  client  and  that  time  I used  to
> start  Ignite  nodes  using  Spring.xml  one  by  one  in  each  node.
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Thursday, April 6, 2023 7:57 PM
> *To:* user@ignite.apache.org; Mandal, Satyajit: IT (PUN) <
> satyajit.man...@barclays.com>
> *Subject:* Re: Ignite configuration using .NET Thick client
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi Satyajit,
>
>
>
> 1. You don't need to use Spring XML unless you have to set a property that
> is not exposed in .NET API
>
> 2. Do you want to start 4 nodes in one process? Or multiple processes on
> the same machine? What's your desired setup?
>
>
>
> Pavel
>
>
>
> On Thu, Apr 6, 2023 at 4:40 PM satyajit.mandal.barclays.com
> <https://clicktime.symantec.com/15sLvSJyuA6fEjRJ369u2?h=LSypiaGaE8p6-HR8_7rXGT-aHM5RsH9EHUzLHQuAzSc==http://satyajit.mandal.barclays.com>
> via user  wrote:
>
> Hi  Pavel,
>
>
>
> Need  to  understand  one  thing  from  you.  Am  using .NET  thick
> client. What’s  the  best way  to  put  the Ignite configuration . Should
> this  be  put  on  code  or  in spring  xml. If  we  put  on  code and
> lets  say  I  need  4  nodes  as  part  of  cluster  then  do  I  need  to
> start  this  service in  all  4  nodes  and  put  those 4 ip  address  in
> code ?  Don’t  want  to  use  spring.xml.
>
>
>
> return new IgniteConfiguration
>
> {
>
> Localhost = "127.0.0.1",
>
> DiscoverySpi = new TcpDiscoverySpi
>
> {
>
> IpFinder = new TcpDiscoveryMulticastIpFinder
>
> {
>
> Endpoints = new[]
>
> {
>
> "127.0.0.1:47500..47502"
>
> }
>
> }
>
> },
>
> JvmOptions = new[]
>
> {
>
> "-DIGNITE_QUIET=true",
>
> "-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true"
>
> },
>
> Logger = new ConsoleLogger
>
> {
>
> MinLevel = LogLevel.Error
>
> },
>
> PeerAssemblyLoadingMode =
> PeerAssemblyLoadingMode.CurrentAppDomain
>
> };
>
> }
>
>
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have

Re: Ignite configuration using .NET Thick client

[Pavel Tupitsyn]

Hi Satyajit,

1. You don't need to use Spring XML unless you have to set a property that
is not exposed in .NET API
2. Do you want to start 4 nodes in one process? Or multiple processes on
the same machine? What's your desired setup?

Pavel

On Thu, Apr 6, 2023 at 4:40 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Pavel,
>
>
>
> Need  to  understand  one  thing  from  you.  Am  using .NET  thick
> client. What’s  the  best way  to  put  the Ignite configuration . Should
> this  be  put  on  code  or  in spring  xml. If  we  put  on  code and
> lets  say  I  need  4  nodes  as  part  of  cluster  then  do  I  need  to
> start  this  service in  all  4  nodes  and  put  those 4 ip  address  in
> code ?  Don’t  want  to  use  spring.xml.
>
>
>
> return new IgniteConfiguration
>
> {
>
> Localhost = "127.0.0.1",
>
> DiscoverySpi = new TcpDiscoverySpi
>
> {
>
> IpFinder = new TcpDiscoveryMulticastIpFinder
>
> {
>
> Endpoints = new[]
>
> {
>
> "127.0.0.1:47500..47502"
>
> }
>
> }
>
> },
>
> JvmOptions = new[]
>
> {
>
> "-DIGNITE_QUIET=true",
>
> "-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true"
>
> },
>
> Logger = new ConsoleLogger
>
> {
>
> MinLevel = LogLevel.Error
>
> },
>
> PeerAssemblyLoadingMode =
> PeerAssemblyLoadingMode.CurrentAppDomain
>
> };
>
> }
>
>
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: java thin client idle like 1 hour , ignite server close client port

[Pavel Tupitsyn]

Yes, heartbeats can help with connection stability.

Also check ClientConnectorConfiguration#idleTimeout setting on the server
[1]

[1]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#getIdleTimeout--

On Tue, Apr 4, 2023 at 11:37 AM f cad  wrote:

> in this case, java thin client need set setHeartbeatEnabled(true)?
>

Re: Query regarding Thick Client using .NET

[Pavel Tupitsyn]

> Am  able  to  run  the same.

All good, no issues? Great!

On Fri, Mar 31, 2023 at 6:42 PM  wrote:

> Hi  Pavel,
>
>
>
> Am  able  to  run  the same.
>
>
>
> Regards
>
> Satyajit
>
>
>
> *From:* satyajit.mandal.barclays.com via user 
> *Sent:* Friday, March 31, 2023 7:59 PM
> *To:* user@ignite.apache.org; ptupit...@apache.org
> *Subject:* RE: Query regarding Thick Client using .NET
>
>
>
> CAUTION: This email originated from outside our organisation -
> user@ignite.apache.org Do not click on links, open attachments, or
> respond unless you recognize the sender and can validate the content is
> safe.
>
> Hi  Pavel,
>
>
>
> Am trying  to  add  controllers  and  do  the  basic cache  operations
> and  expose  the controller  end point  as  part  of  Api  for  other
> Api’s  to consume.
>
>
>
> Do  you  have  any  working  example  using  webapi  controllers?
>
>
>
> Regards
>
> Satyajit
>
>
>
>
>
> *From:* Pavel Tupitsyn 
> *Sent:* Friday, March 31, 2023 7:47 PM
> *To:* user@ignite.apache.org; Mandal, Satyajit: IT (PUN) <
> satyajit.man...@barclays.com>
> *Subject:* Re: Query regarding Thick Client using .NET
>
>
>
> CAUTION: This email originated from outside our organisation -
> ptupit...@apache.org Do not click on links, open attachments, or respond
> unless you recognize the sender and can validate the content is safe.
>
> Hi,
>
>
>
> I don't think WebApi makes any difference. You can use Ignite in the same
> way.
>
> DI lifetime is probably the only concern. IIgnite instance should be
> registered as a singleton, or wrapped within a singleton service.
>
>
>
> Let me know if you have any specific questions.
>
>
>
> Pavel
>
>
>
>
>
> On Fri, Mar 31, 2023 at 4:08 PM satyajit.mandal.barclays.com
> <https://clicktime.symantec.com/15t5ZstJwssT8uaerASgn?h=jbMy_fjnwUsQcJvepxoPHwxcVxnQRwG2-JbaVeX6d_8==http://satyajit.mandal.barclays.com>
> via user  wrote:
>
> Hi  Pavel/Team,
>
>
>
> Can  we  run  webapi  service written  in  .NET  which  will  connect  to
> Ignite cluster  as  thick  client  and  fetch  data?  All  examples  which
> we have seen is  using  console application? Can you  share  a  .NET
> webapi  service  as a  thick  client?
>
>
>
>
>
> Thanks
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of 

Re: Query regarding Thick Client using .NET

[Pavel Tupitsyn]

Hi,

I don't think WebApi makes any difference. You can use Ignite in the same
way.
DI lifetime is probably the only concern. IIgnite instance should be
registered as a singleton, or wrapped within a singleton service.

Let me know if you have any specific questions.

Pavel


On Fri, Mar 31, 2023 at 4:08 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Pavel/Team,
>
>
>
> Can  we  run  webapi  service written  in  .NET  which  will  connect  to
> Ignite cluster  as  thick  client  and  fetch  data?  All  examples  which
> we have seen is  using  console application? Can you  share  a  .NET
> webapi  service  as a  thick  client?
>
>
>
>
>
> Thanks
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Maximum concurrency of ComputeTask

[Pavel Tupitsyn]

Two things can limit the number of active compute tasks:

IgniteConfiguration#publicThreadPoolSize (defaults to max(8,
AVAILABLE_PROC_CNT))
ThinClientConfiguration#maxActiveComputeTasksPerConnection (defaults to 0
=> compute from thin clients is disabled)

Please check those settings on your servers.


https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setPublicThreadPoolSize-int-
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ThinClientConfiguration.html#setMaxActiveComputeTasksPerConnection-int-

On Fri, Mar 10, 2023 at 5:17 AM y  wrote:

> Hi Everyone：
>
>   When *multiple users* were calling the same *c**omputeTask *through the 
> *thin
> client*, I noticed that there is a limit to the maximum number of
> computing tasks running simultaneously on the server.  In my server, 30 or
> 31 ComputeTasks can be run simultaneously at most. No matter how increasing
> the number of users, it still can only run 30 computeTask on Server.  How
> can i improve the maximum number of running task？
>
> Thanks,
> Hu ty
>
>
>

Re: Performance benchmarks for Thin Client and Thick Client

[Pavel Tupitsyn]

Thick and thin clients have similar performance on basic use cases.
For example, here is a DataStreamer benchmark, .NET thin vs thick:
https://github.com/apache/ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.BenchmarkDotNet/ThinClient/ThinClientDataStreamerBenchmark.cs

But you should always measure your own end-to-end scenario, don't rely on
synthetic benchmarks.

On Tue, Feb 14, 2023 at 2:43 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Team,
>
>
>
> Is  there  any  performance  benchmark  documented  for  .NET thin  client
> vs  thick  client. Can  someone  share  details  about  performance
> comparison  between  thin  and  thick  client.
>
>
>
> Regards
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: ScanQuery and PeerAssemblyLoading - Issue 1

[Pavel Tupitsyn]

Jay,

Peer Assembly Loading feature only exists for Compute API.

- When you perform a ScanQuery the 1st time, the assembly does not exist on
the remote node and you get an error.
- After you run a Compute task, your entire assembly (including
SalaryFilter class) gets loaded on the remote node, and now this class can
be used for ScanQuery too.

Basically, Compute can be used as a workaround to pre-load any assemblies
on remote nodes.

Pavel

On Mon, Jan 23, 2023 at 8:29 PM  wrote:

> Dear Community,
>
>
>
> I’ve done quite a lot of testing and somehow remotely loading a cache
> entry filter from client to server only works after a compute-call was
> made.
>
> This appears somewhat strange – maybe someone can explain (/or reproduce
> the behaviour? Full code at the end of this mail).
>
>
>
> The setup: Ignite 2.14 with PeerAssemblyLoadingMode.CurrentAppDomain; .NET
> 5.0 with incremental assembly versioning; (Thick) .NET client running
> queries against a .NET-embedded server.
>
>
>
> Summary of the client code:
>
> - Create a cache and put some users
>
> - Run ScanQuery (1st time) -> Exception “No matching type found for object
> [typeId=327128514, typeName=SalaryFilter]. This usually indicates that
> assembly with specified type is not loaded on a node.”
>
> - Run (any) compute task
>
> - Run ScanQuery (2nd time) -> Works
>
> Same behaviour can be observed when restarting / recompiling in-between
> any of the above stages, repeating stages multiple times, and so on.
>
>
>
> Thank you for any input on this.
>
>
>
> Jay
>
>
>
>
>
>
>
> SERVER CODE:
>
>
>
> public static class Program
>
> {
>
> public static async Task Main(string[] args)
>
> {
>
> IgniteConfiguration IgniteConfig = new IgniteConfiguration();
>
> IgniteConfig.PeerAssemblyLoadingMode =
> PeerAssemblyLoadingMode.CurrentAppDomain;
>
>
>
> var ignite = Ignition.Start(IgniteConfig);
>
> ignite.GetCluster().SetActive(true);
>
>
>
> Console.ReadLine();
>
> }
>
> }
>
>
>
>
>
> CLIENT CODE:
>
>
>
> public static class Program
>
> {
>
> public static async Task Main(string[] args)
>
> {
>
> IgniteConfiguration IgniteConfig = new IgniteConfiguration();
>
> IgniteConfig.PeerAssemblyLoadingMode =
> PeerAssemblyLoadingMode.CurrentAppDomain;
>
> IgniteConfig.ClientMode = true;
>
>
>
> var ignite = Ignition.Start(IgniteConfig);
>
>
>
> var cache = ignite.GetOrCreateCache(
> "Name_User_Cache").WithKeepBinary();
>
>
>
> IBinaryObjectBuilder builder = ignite.GetBinary().GetBuilder(
> "User");
>
> IBinaryObject user = builder.SetField("Salary",
> 1001).Build();
>
> cache.Put("First User", user);
>
> user = builder.SetField("Salary", 1000).Build();
>
> cache.Put("Second User", user);
>
> user = builder.SetField("Salary", 1002).Build();
>
> cache.Put("Third User", user);
>
>
>
> Console.WriteLine("First search:");
>
> RunScanQuery(cache);
>
>
>
> Console.WriteLine("Broadcasting compute job");
>
> ignite.GetCompute().Broadcast(new HelloAction());
>
>
>
> Console.WriteLine("Second search:");
>
> RunScanQuery(cache);
>
>
>
> Console.ReadLine();
>
> }
>
>
>
> static void RunScanQuery(ICache cache)
>
> {
>
> try
>
> {
>
> using (var cursor = cache.Query(new ScanQuery IBinaryObject>(new SalaryFilter(
>
> {
>
> foreach (var entry in cursor)
>
>{
>
> Console.WriteLine("Key = " + entry.Key + ",
> Value.Salary = " + entry.Value.GetField("Salary").ToString());
>
> }
>
> }
>
> }
>
> catch (Exception ex)
>
> {
>
> Console.WriteLine("Exception: " + ex.Message);
>
> }
>
> }
>
> }
>
>
>
> class HelloAction : IComputeAction
>
> {
>
> public void Invoke()
>
> {
>
> Console.WriteLine("Hello, World!");
>
> }
>
> }
>
>
>
> class SalaryFilter : ICacheEntryFilter
>
> {
>
> public bool Invoke(ICacheEntry entry)
>
> {
>
> return entry.Value.GetField("Salary") > 1000;
>
> }
>
> }
>
>
>
>
>
> appreciate the help
>
>
>
>
>

Re: Question on BinaryObject

[Pavel Tupitsyn]

Hello, yes, your understanding is correct.

On Tue, Jan 17, 2023 at 6:46 PM Peter  wrote:

> Hello,
>
> Do I understand correctly, that each BinaryObject that is returned by
> IgniteCache.get() and  IgniteCache.getAll() method calls on a local node
> contains an internal on-heap byte array, and object unmarshalling occurs
> from that array, and not from off-heap memory?
>

Re: C# Ignite thin client - how to connect specific grid

[Pavel Tupitsyn]

Set different client connector ports for different grids like this:







...

On Thu, Jan 12, 2023 at 2:01 PM Stephen Darlington <
stephen.darling...@gridgain.com> wrote:

> Those ports are for thick-clients and servers. Thin clients connect to
> 10800. The second server to start will be on 10801, etc.
>
> You can configure the thin-client port with ClientConnectorConfiguration:
> getting-started-with-thin-clients
> <https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients#configuring-thin-client-connector>
>
> On 12 Jan 2023, at 11:56, Charlin S  wrote:
>
> Hi Pavel,
> Thanks for your reply.
> I am having two grids in one server and grid configuration as below. Which
> port should I use for connecting grid1 and grid2?
>
> 
>
> As per my understanding, I should be able to connect thin clients like
> this. Please correct me if I am wrong.
>
> //for Grid 1
>
> var clientCfgGrid1 = new IgniteClientConfiguration
> {
> Endpoints = new[] { "10.201.30.116: 
> <http://127.0.0.1:10901/>48500","10.201.30.117: 
> <http://127.0.0.1:10901/>48500" }
>
> };
>
> // for Grid 2
>
> var clientCfgGrid2 = new IgniteClientConfiguration
> {
> Endpoints = new[] { "10.201.30.116: 
> <http://127.0.0.1:10901/>55800","10.201.30.117: 
> <http://127.0.0.1:10901/>55800" }
>
> };
>
>
> Regards,
> Charlin
>
>
>
>
>
>
>
>
>
> On Thu, 12 Jan 2023 at 15:37, Pavel Tupitsyn  wrote:
>
>>
>> C# thin client can connect to any port, specify it in the endpoint string
>> like this:
>>
>> var clientCfg = new IgniteClientConfiguration("127.0.0.1:10901");
>> var client = Ignition.StartClient(clientCfg);
>>
>>
>> or
>>
>> var clientCfg2 = new IgniteClientConfiguration
>> {
>> Endpoints = new[] { "127.0.0.1:10901" }
>> };
>>
>>
>> On Thu, Jan 12, 2023 at 11:41 AM Charlin S 
>> wrote:
>>
>>> Hi All,
>>> My requirement is to connect a specific grid(server has multiple grids)
>>> from c# thin client, since its support only 10800 ports.
>>>
>>> Regards,
>>> Charlin
>>>
>>>
>>>
>>>
>

Re: C# Ignite thin client - how to connect specific grid

[Pavel Tupitsyn]

C# thin client can connect to any port, specify it in the endpoint string
like this:

var clientCfg = new IgniteClientConfiguration("127.0.0.1:10901");
var client = Ignition.StartClient(clientCfg);


or

var clientCfg2 = new IgniteClientConfiguration
{
Endpoints = new[] { "127.0.0.1:10901" }
};


On Thu, Jan 12, 2023 at 11:41 AM Charlin S  wrote:

> Hi All,
> My requirement is to connect a specific grid(server has multiple grids)
> from c# thin client, since its support only 10800 ports.
>
> Regards,
> Charlin
>
>
>
>

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

You can find Cassandra jars inside the full distribution [1] at
apache-ignite-2.10.0-bin.zip/apache-ignite-2.10.0-bin/libs/optional/ignite-cassandra-store

[1] https://ignite.apache.org/download.cgi

On Mon, Jan 9, 2023 at 2:01 PM Charlin S  wrote:

> Hi Pavel,
> Where can I find the Ignite cassandra jar file? Could you please share.
> Note: I am using Ignite 2.10 now.
>
> Regards,
> Charlin
>
>
> On Mon, 9 Jan 2023 at 16:34, Charlin S  wrote:
>
>> Hi Pavel,
>> Thank you for your response.
>> I have placed the ignite-cassandra-store
>> & ignite-cassandra-serializers(which is available in optional) folder
>> under /home/soft/apache-ignite-2.10.0-bin/libs.
>> and  classpath has been set as by command export
>> CLASSPATH="/home/soft/apache-ignite-2.10.0-bin/libs".
>>
>> Regards,
>> Charlin
>>
>>
>> On Mon, 9 Jan 2023 at 15:53, Pavel Tupitsyn  wrote:
>>
>>> Check inner exceptions, probably Cassandra jars are missing
>>>
>>> On Mon, Jan 9, 2023 at 10:01 AM Charlin S 
>>> wrote:
>>>
>>>> Hi Pavel,
>>>> Thank you for your reply.
>>>> I have tried the following options.
>>>> 1. Moved connection-settings.xml inside the Ignite spring xml
>>>> configuration file.
>>>> 2. added classpath by this command export
>>>> CLASSPATH="/home/soft/apache-ignite-2.10.0-bin/libs".
>>>> but still same error class org.apache.ignite.IgniteException: Failed to
>>>> instantiate Spring XML application context (make sure all classes used in
>>>> Spring configuration are present at CLASSPATH)
>>>> [springUrl=file:/home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml]
>>>>
>>>> Regards,
>>>> Charlin
>>>>
>>>>
>>>>
>>>> On Mon, 9 Jan 2023 at 11:58, Pavel Tupitsyn 
>>>> wrote:
>>>>
>>>>> Ok, it seems this requires quite a bit of configuration, we can't just
>>>>> use those linked configs directly.
>>>>> 'import resource="classpath: ' looks into Java classpath, so this
>>>>> would depend on your setup.
>>>>>
>>>>> You can replace  with the contents of connection-settings.xml.
>>>>> But there are other things to fix. You'll need Cassandra jars in
>>>>> IgniteConfiguration.JvmClasspath, etc.
>>>>>
>>>>> Unfortunately, I don't have the capacity to develop a fully working
>>>>> sample right now. Please try to follow the docs, source code, and error
>>>>> messages.
>>>>>
>>>>> On Fri, Jan 6, 2023 at 6:08 AM Charlin S 
>>>>> wrote:
>>>>>
>>>>>> Hi Pavel,
>>>>>> I have specified absolute path(/org/apache/ignite/tests/
>>>>>> cassandra/connection-settings.xml) in the config file but in
>>>>>> exception it's showing only org/apache/ignite/tests/
>>>>>> cassandra/connection-settings.xml.
>>>>>>
>>>>>> Verified file content using cat command and file path.
>>>>>>
>>>>>> Regards,
>>>>>> Charlin
>>>>>>
>>>>>>
>>>>>> On Thu, 5 Jan 2023 at 20:02, Pavel Tupitsyn 
>>>>>> wrote:
>>>>>>
>>>>>>> > org/apache/ignite/tests/cassandra/connection-settings.xml cannot
>>>>>>> be opened because it does not exist
>>>>>>>
>>>>>>> Try specifying an absolute path to this xml file. Looks like the
>>>>>>> relative path is not correct.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 5, 2023 at 1:12 PM Charlin S 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi All,
>&

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

Check inner exceptions, probably Cassandra jars are missing

On Mon, Jan 9, 2023 at 10:01 AM Charlin S  wrote:

> Hi Pavel,
> Thank you for your reply.
> I have tried the following options.
> 1. Moved connection-settings.xml inside the Ignite spring xml
> configuration file.
> 2. added classpath by this command export
> CLASSPATH="/home/soft/apache-ignite-2.10.0-bin/libs".
> but still same error class org.apache.ignite.IgniteException: Failed to
> instantiate Spring XML application context (make sure all classes used in
> Spring configuration are present at CLASSPATH)
> [springUrl=file:/home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml]
>
> Regards,
> Charlin
>
>
>
> On Mon, 9 Jan 2023 at 11:58, Pavel Tupitsyn  wrote:
>
>> Ok, it seems this requires quite a bit of configuration, we can't just
>> use those linked configs directly.
>> 'import resource="classpath: ' looks into Java classpath, so this would
>> depend on your setup.
>>
>> You can replace  with the contents of connection-settings.xml.
>> But there are other things to fix. You'll need Cassandra jars in
>> IgniteConfiguration.JvmClasspath, etc.
>>
>> Unfortunately, I don't have the capacity to develop a fully working
>> sample right now. Please try to follow the docs, source code, and error
>> messages.
>>
>> On Fri, Jan 6, 2023 at 6:08 AM Charlin S  wrote:
>>
>>> Hi Pavel,
>>> I have specified absolute path(/org/apache/ignite/tests/
>>> cassandra/connection-settings.xml) in the config file but in exception
>>> it's showing only org/apache/ignite/tests/cassandra/connection-settings.
>>> xml.
>>>
>>> Verified file content using cat command and file path.
>>>
>>> Regards,
>>> Charlin
>>>
>>>
>>> On Thu, 5 Jan 2023 at 20:02, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> > org/apache/ignite/tests/cassandra/connection-settings.xml cannot be
>>>> opened because it does not exist
>>>>
>>>> Try specifying an absolute path to this xml file. Looks like the
>>>> relative path is not correct.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jan 5, 2023 at 1:12 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> Ignite server node not starting with the following configuration( all
>>>>> configuration files attached)
>>>>> 1. created org/apache/ignite/tests/cassandra/ and
>>>>> placed connection-settings.xml
>>>>> 2. created /org/apache/ignite/tests/cassandra/persistence and
>>>>> placed persistence-settings-1.xml
>>>>> 3. placed Ignite server bean xml (ignite-config-cassandra) under
>>>>> /home/soft/apache-ignite-2.10.0-bin/config
>>>>> 4. starting Ignite by command nohup
>>>>> /home/soft/apache-ignite-2.10.0-bin/bin/ignite.sh
>>>>> /home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml &
>>>>>
>>>>> below error while starting server node
>>>>> Caused by:
>>>>> org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
>>>>> Configuration problem: Failed to import bean definitions from URL location
>>>>> [classpath:/org/apache/ignite/tests/cassandra/connection-settings.xml]
>>>>> Offending resource: URL
>>>>> [file:/home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml];
>>>>> nested exception is
>>>>> org.springframework.beans.factory.BeanDefinitionStoreException: 
>>>>> IOException
>>>>> parsing XML document from class path resource
>>>>> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
>>>>> exception is java.io.FileNotFoundException: class path resource
>>>>> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
>>>>> opened because it does not exist
>>>>>
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>>
>>>>> On Tue, 3 Jan 2023 at 21:52, Pavel Tupitsyn 
>>>>> wrote:
>>>>

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

Ok, it seems this requires quite a bit of configuration, we can't just use
those linked configs directly.
'import resource="classpath: ' looks into Java classpath, so this would
depend on your setup.

You can replace  with the contents of connection-settings.xml.
But there are other things to fix. You'll need Cassandra jars in
IgniteConfiguration.JvmClasspath, etc.

Unfortunately, I don't have the capacity to develop a fully working sample
right now. Please try to follow the docs, source code, and error messages.

On Fri, Jan 6, 2023 at 6:08 AM Charlin S  wrote:

> Hi Pavel,
> I have specified absolute path(/org/apache/ignite/tests/
> cassandra/connection-settings.xml) in the config file but in exception
> it's showing only org/apache/ignite/tests/cassandra/connection-settings.
> xml.
>
> Verified file content using cat command and file path.
>
> Regards,
> Charlin
>
>
> On Thu, 5 Jan 2023 at 20:02, Pavel Tupitsyn  wrote:
>
>> > org/apache/ignite/tests/cassandra/connection-settings.xml cannot be
>> opened because it does not exist
>>
>> Try specifying an absolute path to this xml file. Looks like the relative
>> path is not correct.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jan 5, 2023 at 1:12 PM Charlin S  wrote:
>>
>>> Hi All,
>>> Ignite server node not starting with the following configuration( all
>>> configuration files attached)
>>> 1. created org/apache/ignite/tests/cassandra/ and
>>> placed connection-settings.xml
>>> 2. created /org/apache/ignite/tests/cassandra/persistence and
>>> placed persistence-settings-1.xml
>>> 3. placed Ignite server bean xml (ignite-config-cassandra) under
>>> /home/soft/apache-ignite-2.10.0-bin/config
>>> 4. starting Ignite by command nohup
>>> /home/soft/apache-ignite-2.10.0-bin/bin/ignite.sh
>>> /home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml &
>>>
>>> below error while starting server node
>>> Caused by:
>>> org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
>>> Configuration problem: Failed to import bean definitions from URL location
>>> [classpath:/org/apache/ignite/tests/cassandra/connection-settings.xml]
>>> Offending resource: URL
>>> [file:/home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml];
>>> nested exception is
>>> org.springframework.beans.factory.BeanDefinitionStoreException: IOException
>>> parsing XML document from class path resource
>>> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
>>> exception is java.io.FileNotFoundException: class path resource
>>> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
>>> opened because it does not exist
>>>
>>> Regards,
>>> Charlin
>>>
>>>
>>> On Tue, 3 Jan 2023 at 21:52, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> Leave it as is. You can grab the settings file at
>>>>
>>>> https://github.com/apache/ignite/blob/231ead01d186c75ebb48f1d19e5a95fc9c459202/modules/cassandra/store/src/test/resources/org/apache/ignite/tests/persistence/primitive/persistence-settings-1.xml
>>>>
>>>> There is nothing C# specific at all in this example, just start the
>>>> node with the config file and you should be set.
>>>>
>>>> On Tue, Jan 3, 2023 at 6:08 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi Pavel,
>>>>> Thanks for updating, how does the support below xml config  works in
>>>>> C#, or do I need to skip this part for cassandra persistence
>>>>> storage implementation in c#.
>>>>>
>>>>> >>>> class="org.apache.ignite.cache.store.cassandra.utils.persistence.KeyValuePersistenceSettings">
>>>>> >>>> value="classpath:org/apache/ignite/tests/persistence/blob/persistence-settings-1.xml"
>>>>>  />
>>>>>
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>> On Tue, 3 Jan 2023 at 21:07, Pavel Tupitsyn 
>>>>> wrote:
>>>>>
>>>>>> Hi Charlin, happy new year!
>>>>>>
>>>>>> Those examples are mostly XML configs, you can use them as is in C#,
>>>>>> something like:
>>>>>> var cfg = new IgniteConfiguration { SpringConfigUrl =
>>>>>> "ignite-cassandra.xml" };
>>>>>> Ignition.Start(cfg);
>>>>>>
>>>>>> Let me know if you need more details.
>>>>>>
>>>>>> Pavel
>>>>>>
>>>>>> On Tue, Jan 3, 2023 at 1:50 PM Charlin S 
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>> Happy new year to all !
>>>>>>>
>>>>>>>
>>>>>>> https://ignite.apache.org/docs/latest/extensions-and-integrations/cassandra/usage-examples
>>>>>>>
>>>>>>> As per documentation, we see examples for support of data types in
>>>>>>> Java. we have not seen support for dotnet data types.
>>>>>>> Could you please advise on this for C# implementation?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Charlin
>>>>>>>
>>>>>>>

Re: Text Search from Ignite Cache

[Pavel Tupitsyn]

Ignite has full text search capabilities (based on Lucene engine) with
TextQuery [1] that should perform better for partial matches.
Example: [2]

[1]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/TextQuery.html
[2]
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheQueryExample.java#L147

On Fri, Jan 6, 2023 at 9:22 AM Biraj Deb <
biraj...@decisionmanagementsolutions.com> wrote:

> Hi all,
> What is the best way to search partial text and ignore cases from a table
> to get best performance?
> Our backend code in *Java. *Our environment has  RAM: 61 gib,  cpu: 16
> vcpus, core : 8.
>  I have tried to index on that field and I'm still getting results in 4
> sec.
> Query Is :
> *SELECT OBJ.PROP_VALUE_STR1 as name FROM DFM.BRANCH B INNER JOIN
> DFM.OBJECT OBJ ON 1=1 WHERE (OBJ.BRANCH_ID = B.ID ) AND
> ((CAST(OBJ.PROP_VALUE_STR1 AS VARCHAR_IGNORECASE) LIKE 'qualify%') AND
> ((OBJ.ID  <> '6a7a51ce-3fc9-4b35-a845-2be95c8180a1') AND
> ((OBJ.REFERENCE_ID <> '6a7a51ce-3fc9-4b35-a845-2be95c8180a1') AND
> ((B.PROJECT_ID = 'cad78e78-a38b-4340-817b-d689b988ec17') AND
> (OBJ.OBJECT_TYPE_ID =1)*
> In above Query join two table in that  *OBJECT*   table has index on
> BRANCH_ID and  PROP_VALUE_STR1 and  *BRANCH* table has index on
> PROJECT_ID. And Object table has more than a million entry
>
>
>
>
>

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

> org/apache/ignite/tests/cassandra/connection-settings.xml cannot be
opened because it does not exist

Try specifying an absolute path to this xml file. Looks like the relative
path is not correct.

























On Thu, Jan 5, 2023 at 1:12 PM Charlin S  wrote:

> Hi All,
> Ignite server node not starting with the following configuration( all
> configuration files attached)
> 1. created org/apache/ignite/tests/cassandra/ and
> placed connection-settings.xml
> 2. created /org/apache/ignite/tests/cassandra/persistence and
> placed persistence-settings-1.xml
> 3. placed Ignite server bean xml (ignite-config-cassandra) under
> /home/soft/apache-ignite-2.10.0-bin/config
> 4. starting Ignite by command nohup
> /home/soft/apache-ignite-2.10.0-bin/bin/ignite.sh
> /home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml &
>
> below error while starting server node
> Caused by:
> org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
> Configuration problem: Failed to import bean definitions from URL location
> [classpath:/org/apache/ignite/tests/cassandra/connection-settings.xml]
> Offending resource: URL
> [file:/home/soft/apache-ignite-2.10.0-bin/config/ignite-config-cassandra.xml];
> nested exception is
> org.springframework.beans.factory.BeanDefinitionStoreException: IOException
> parsing XML document from class path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml]; nested
> exception is java.io.FileNotFoundException: class path resource
> [org/apache/ignite/tests/cassandra/connection-settings.xml] cannot be
> opened because it does not exist
>
> Regards,
> Charlin
>
>
> On Tue, 3 Jan 2023 at 21:52, Pavel Tupitsyn  wrote:
>
>> Leave it as is. You can grab the settings file at
>>
>> https://github.com/apache/ignite/blob/231ead01d186c75ebb48f1d19e5a95fc9c459202/modules/cassandra/store/src/test/resources/org/apache/ignite/tests/persistence/primitive/persistence-settings-1.xml
>>
>> There is nothing C# specific at all in this example, just start the node
>> with the config file and you should be set.
>>
>> On Tue, Jan 3, 2023 at 6:08 PM Charlin S  wrote:
>>
>>> Hi Pavel,
>>> Thanks for updating, how does the support below xml config  works in C#,
>>> or do I need to skip this part for cassandra persistence
>>> storage implementation in c#.
>>>
>>> >> class="org.apache.ignite.cache.store.cassandra.utils.persistence.KeyValuePersistenceSettings">
>>> >> value="classpath:org/apache/ignite/tests/persistence/blob/persistence-settings-1.xml"
>>>  />
>>>
>>> Regards,
>>> Charlin
>>>
>>> On Tue, 3 Jan 2023 at 21:07, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> Hi Charlin, happy new year!
>>>>
>>>> Those examples are mostly XML configs, you can use them as is in C#,
>>>> something like:
>>>> var cfg = new IgniteConfiguration { SpringConfigUrl =
>>>> "ignite-cassandra.xml" };
>>>> Ignition.Start(cfg);
>>>>
>>>> Let me know if you need more details.
>>>>
>>>> Pavel
>>>>
>>>> On Tue, Jan 3, 2023 at 1:50 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> Happy new year to all !
>>>>>
>>>>>
>>>>> https://ignite.apache.org/docs/latest/extensions-and-integrations/cassandra/usage-examples
>>>>>
>>>>> As per documentation, we see examples for support of data types in
>>>>> Java. we have not seen support for dotnet data types.
>>>>> Could you please advise on this for C# implementation?
>>>>>
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>>

Re: Ignite Cassandra Integration

[Pavel Tupitsyn]

Hi,

Cassandra integration works through CassandraCacheStore [1].
To combine it with another storage (e.g. File), you can implement a wrapper
around CassandraCacheStore that performs additional reads/writes to another
storage.

https://ignite.apache.org/docs/latest/extensions-and-integrations/cassandra/overview

On Wed, Jan 4, 2023 at 1:08 PM satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  Pavel,
>
>
>
> Is  it  possible to   persist  data  both  in  File as  well  as  in
> Cassandra  at  the same  time without  using  connectors?
>
>
>
> Regards
>
> Satyajit
>
>
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

Leave it as is. You can grab the settings file at
https://github.com/apache/ignite/blob/231ead01d186c75ebb48f1d19e5a95fc9c459202/modules/cassandra/store/src/test/resources/org/apache/ignite/tests/persistence/primitive/persistence-settings-1.xml

There is nothing C# specific at all in this example, just start the node
with the config file and you should be set.

On Tue, Jan 3, 2023 at 6:08 PM Charlin S  wrote:

> Hi Pavel,
> Thanks for updating, how does the support below xml config  works in C#,
> or do I need to skip this part for cassandra persistence
> storage implementation in c#.
>
>  class="org.apache.ignite.cache.store.cassandra.utils.persistence.KeyValuePersistenceSettings">
>  value="classpath:org/apache/ignite/tests/persistence/blob/persistence-settings-1.xml"
>  />
>
> Regards,
> Charlin
>
> On Tue, 3 Jan 2023 at 21:07, Pavel Tupitsyn  wrote:
>
>> Hi Charlin, happy new year!
>>
>> Those examples are mostly XML configs, you can use them as is in C#,
>> something like:
>> var cfg = new IgniteConfiguration { SpringConfigUrl =
>> "ignite-cassandra.xml" };
>> Ignition.Start(cfg);
>>
>> Let me know if you need more details.
>>
>> Pavel
>>
>> On Tue, Jan 3, 2023 at 1:50 PM Charlin S  wrote:
>>
>>> Hi All,
>>> Happy new year to all !
>>>
>>>
>>> https://ignite.apache.org/docs/latest/extensions-and-integrations/cassandra/usage-examples
>>>
>>> As per documentation, we see examples for support of data types in Java.
>>> we have not seen support for dotnet data types.
>>> Could you please advise on this for C# implementation?
>>>
>>> Regards,
>>> Charlin
>>>
>>>

Re: Ignite Cassandra Integration Configuration for C#

[Pavel Tupitsyn]

Hi Charlin, happy new year!

Those examples are mostly XML configs, you can use them as is in C#,
something like:
var cfg = new IgniteConfiguration { SpringConfigUrl =
"ignite-cassandra.xml" };
Ignition.Start(cfg);

Let me know if you need more details.

Pavel

On Tue, Jan 3, 2023 at 1:50 PM Charlin S  wrote:

> Hi All,
> Happy new year to all !
>
>
> https://ignite.apache.org/docs/latest/extensions-and-integrations/cassandra/usage-examples
>
> As per documentation, we see examples for support of data types in Java.
> we have not seen support for dotnet data types.
> Could you please advise on this for C# implementation?
>
> Regards,
> Charlin
>
>

Re: Behavior of java thin client connection

[Pavel Tupitsyn]

Ignite thin client retry mechanism is tuned to work with multi-node
clusters where some nodes are expected to fail or restart time to time,
but other nodes remain online. Retry logic tries all known server
addresses, but if all nodes are down, it gives up.
That is why it does not work for you - we don't expect single node clusters.

A workaround is to pass the same address to setAddresses multiple times to
force more aggressive reconnect:
cfg.setAddresses("127.0.0.1:10800", "127.0.0.1:10800", "127.0.0.1:10800")


On Tue, Oct 25, 2022 at 4:23 PM Bernáth Máté 
wrote:

>
>
> Hello,
>
> We use this code snippet for the connection:
> *„ClientConfiguration cfg = new ClientConfiguration();*
>
> *cfg = cfg.setReconnectThrottlingPeriod(1000);*
>
> *cfg = cfg.setReconnectThrottlingRetries(2);*
>
> *cfg.setAddresses("127.0.0.1:10800 <http://127.0.0.1:10800>");*
>
> *cfg.setSslMode(SslMode.DISABLED);*
>
>
>
>
>
> *IgniteClient client = Ignition.startClient(cfg);” *We also tried that
> version of this code snippet where we don’t set the
> setReconnectThrottlingPeriod and setReconnectThrottlingRetries parameters.
>
> Best regards,
>
> *Máté Bernáth*
>
> Developer
>
>
>
> Intalion Group
>
> E-mail: mate.bern...@intalion.hu <%20mate.bern...@intalion.hu>
>
> Mobile: +36 70 428 5671
>
> Phone: +36 1 999 6197
>
> H-1012 Budapest, Vérmező út 4.
>
> <https://www.intalion.hu/>
>
>
>
> *Feladó:* Pavel Tupitsyn 
> *Küldve:* 2022. október 25., kedd 15:14
> *Címzett:* user@ignite.apache.org
> *Tárgy:* Re: Behavior of java thin client connection
>
>
>
> Hello,
>
>
>
> Can you please share your ClientConfiguration?
>
>
>
> On Tue, Oct 25, 2022 at 2:41 PM Bernáth Máté 
> wrote:
>
> Hello,
>
>
>
> The following is our experience using the Apache Ignite.
>
>
>
> We use the Apache Ignite for java applications. These applications are
> part of ESB applications.
>
> For the client connection (between our java application and apache ignite
> server) we use the java thin client connection.
>
>
>
> First we start the apache ignite server. When we start our java
> application the connection is successfully created to the Apache Ignite
> server. The connection is only created once in the java application. After
> the Apache Ignite server is restarted our application call the apache
> ignite to get some data, after that our java application throws this
> exception: org.apache.ignite.client.ClientConnectionException: Channel is
> closed. After few unsuccessful  data requests (1-10 requests) to the apache
> ignite server our java application get the requested data successfully.
>
> We use these java classes to build connection between our java application
> and apache ignite server:
>
>- org.apache.ignite.configuration.ClientConfiguration
>- org.apache.ignite.client.IgniteClient
>
>
>
> We also tried to configure the ClientConfiguration java class with
> setReconnectThrottlingPeriod and setReconnectThrottlingRetries methods, but
> it didn’t help to get data faster if the apache ignite server is restarted.
>
>
>
> Versions we use:
>
>- Apache Ignite server: 2.13.0
>- Used jar to client connection: ignite-core-2.13.0.jar (
>https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.13.0
>)
>
> Is it the default and expected behavior when we use java thin client
> connection?
>
>
>
> Best regards,
>
> *Máté Bernáth*
>
> Developer
>
>
>
> Intalion Group
>
> E-mail: mate.bern...@intalion.hu <%20mate.bern...@intalion.hu>
>
> Mobile: +36 70 428 5671
>
> Phone: +36 1 999 6197
>
> H-1012 Budapest, Vérmező út 4.
>
> [image: A képet a feladó eltávolította.] <https://www.intalion.hu/>
>
>
>
> Jelen üzenet és annak mellékletei bizalmas információkat, üzleti titkokat,
> személyes adatokat tartalmazhatnak, azokat kizárólag a címzett ismerheti
> meg, illetve használhatja fel. Felhívjuk a figyelmét, hogy az üzenet és
> mellékletei tartalmának illetéktelen megismerése, tárolása, másolása,
> továbbítása, közzététele, vagy azzal végzett bármely más művelet szigorúan
> tilos. Amennyiben a jelen üzenetet Ön tévedésből kapta, vagy nem Ön az
> üzenet címzettje, kérjük, hogy azt - mellékletekkel együtt - gépéről
> visszavonhatatlanul törölni, és bennünket a fenti e-mail címen
> haladéktalanul értesíteni szíveskedjék.
>
> Jelen üzenet és annak mellékletei bizalmas információkat, üzleti titkokat,
> személyes adatokat tartalmazhatnak, azokat kizárólag a címzett ismerheti
> meg, illetve használhatja fel. Felhívjuk a figyelmét, hogy az üzenet és
> mellékletei tartalmának illetéktelen megismerése, tárolása, másolása,
> továbbítása, közzététele, vagy azzal végzett bármely más művelet szigorúan
> tilos. Amennyiben a jelen üzenetet Ön tévedésből kapta, vagy nem Ön az
> üzenet címzettje, kérjük, hogy azt - mellékletekkel együtt - gépéről
> visszavonhatatlanul törölni, és bennünket a fenti e-mail címen
> haladéktalanul értesíteni szíveskedjék.
>

Re: Behavior of java thin client connection

[Pavel Tupitsyn]

Hello,

Can you please share your ClientConfiguration?

On Tue, Oct 25, 2022 at 2:41 PM Bernáth Máté 
wrote:

> Hello,
>
>
>
> The following is our experience using the Apache Ignite.
>
>
>
> We use the Apache Ignite for java applications. These applications are
> part of ESB applications.
>
> For the client connection (between our java application and apache ignite
> server) we use the java thin client connection.
>
>
>
> First we start the apache ignite server. When we start our java
> application the connection is successfully created to the Apache Ignite
> server. The connection is only created once in the java application. After
> the Apache Ignite server is restarted our application call the apache
> ignite to get some data, after that our java application throws this
> exception: org.apache.ignite.client.ClientConnectionException: Channel is
> closed. After few unsuccessful  data requests (1-10 requests) to the apache
> ignite server our java application get the requested data successfully.
>
> We use these java classes to build connection between our java application
> and apache ignite server:
>
>- org.apache.ignite.configuration.ClientConfiguration
>- org.apache.ignite.client.IgniteClient
>
>
>
> We also tried to configure the ClientConfiguration java class with
> setReconnectThrottlingPeriod and setReconnectThrottlingRetries methods, but
> it didn’t help to get data faster if the apache ignite server is restarted.
>
>
>
> Versions we use:
>
>- Apache Ignite server: 2.13.0
>- Used jar to client connection: ignite-core-2.13.0.jar (
>https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.13.0
>)
>
> Is it the default and expected behavior when we use java thin client
> connection?
>
>
>
> Best regards,
>
> *Máté Bernáth*
>
> Developer
>
>
>
> Intalion Group
>
> E-mail: mate.bern...@intalion.hu <%20mate.bern...@intalion.hu>
>
> Mobile: +36 70 428 5671
>
> Phone: +36 1 999 6197
>
> H-1012 Budapest, Vérmező út 4.
>
> 
>
>
> Jelen üzenet és annak mellékletei bizalmas információkat, üzleti titkokat,
> személyes adatokat tartalmazhatnak, azokat kizárólag a címzett ismerheti
> meg, illetve használhatja fel. Felhívjuk a figyelmét, hogy az üzenet és
> mellékletei tartalmának illetéktelen megismerése, tárolása, másolása,
> továbbítása, közzététele, vagy azzal végzett bármely más művelet szigorúan
> tilos. Amennyiben a jelen üzenetet Ön tévedésből kapta, vagy nem Ön az
> üzenet címzettje, kérjük, hogy azt - mellékletekkel együtt - gépéről
> visszavonhatatlanul törölni, és bennünket a fenti e-mail címen
> haladéktalanul értesíteni szíveskedjék.
>

Re: Call an operation on all deployed remote services

[Pavel Tupitsyn]

Hi, there is no such method.

I suggest using IgniteCompute.broadcast - it is the way to invoke some code
on multiple nodes simultaneously.

If you absolutely have to do this with services:
1. Call IgniteServices.serviceDescriptors
2. Find the right service in the resulting collection
3. Iterate over ServiceDescriptor.topologySnapshot
4. Obtain a service proxy for every node from 3:
ignite.services(ignite.cluster().forNodeId(uuid)).serviceProxy(...)
5. Invoke the method on every proxy

I don't recommend this though - not a right thing to do, and perf would not
be good.

Pavel



On Thu, Oct 20, 2022 at 5:12 PM Joan Pujol  wrote:

> Hi,
>
> Is there a way that I can call a remote service method in all of the
> nodes where the service is deployed?
>
> A lot of thanks in advance.
>
> --
> Joan Jesús Pujol Espinar
> http://www.joanpujol.cat
>

Re: [ANNOUNCE] Apache Ignite 2.14.0 Released

[Pavel Tupitsyn]

A small post about .NET changes:
https://ptupitsyn.github.io/Whats-New-In-Ignite-Net-2.14/

Cheers!

On Tue, Oct 11, 2022 at 10:56 AM Taras Ledkov  wrote:

> The Apache Ignite Community is pleased to announce the release of
> Apache Ignite 2.14.0.
>
> Apache Ignite® is an in-memory computing platform for transactional,
> analytical, and streaming workloads delivering in-memory speeds at
> a petabyte scale.
> https://ignite.apache.org
>
> For the full list of changes, you can refer to the RELEASE_NOTES list
> which is trying to catalogue the most significant improvements for
> this version of the platform.
> https://ignite.apache.org/releases/2.14.0/release_notes.html
>
> Download the latest Ignite version from here:
> https://ignite.apache.org/download.cgi
>
> Please let us know if you encounter any problems:
> https://ignite.apache.org/community/resources.html#ask
>
> --
> With best regards,
> Taras Ledkov
>

Re: Ignite Async API call continutation behaviour

[Pavel Tupitsyn]

Hi Raymond,

Yes, you can remove those ContinueWith() calls. Default behavior is now
safe.

On Mon, Oct 3, 2022 at 2:25 AM Raymond Wilson 
wrote:

> Jay,
>
> Thanks for the confirmation and the heads up regarding transactions.
>
> Raymond.
>
> On Mon, Oct 3, 2022 at 12:18 AM  wrote:
>
>> Hi Raymond,
>>
>>
>>
>> That’s what we did and it works perfectly.
>>
>>
>>
>> Please note however that async/await currently does not work inside
>> transactions, should that concern you.
>>
>>
>>
>> Pavel wrote in Jan 2022:
>>
>> “
>>
>> > Is using async/await supported for optimistic serializable transactions
>> (and we’re doing something wrong)
>>
>> > or are transactional operations currently supposed to be blocking in
>> .NET?
>>
>>
>>
>> Short answer - yes, please use synchronous cache APIs when working with
>> transactions.
>>
>>
>>
>> A transaction is owned by a specific thread. Transactional API calls
>> should be done from this thread, and `Commit` should be called from this
>> thread.
>>
>> When doing `await`, we most likely end up on another thread and lose
>> transaction context.
>>
>>
>>
>> There were plans to make transactions work across multiple threads, but
>> the ticket looks abandoned:
>>
>> https://issues.apache.org/jira/browse/IGNITE-4887
>>
>> “
>>
>>
>>
>> Issue 4887 is currently still open as I see it.
>>
>>
>>
>> Best Regard
>>
>> Jay
>>
>>
>>
>>
>>
>>
>>
>> *From:* Raymond Wilson 
>> *Sent:* Sunday, 2 October 2022 11:24
>> *To:* user 
>> *Subject:* Ignite Async API call continutation behaviour
>>
>>
>>
>> Way back in Ignite 2.8 we ran into an issue with continuations from
>> Ignite Async method calls that meant our code could continue to run on an
>> Ignite striped thread pool thread, with sometimes bad results.
>>
>>
>>
>> At the time the pattern suggested was to use a .ContinueWith()
>> continuation, like this:
>>
>>
>>
>>  await _cache.PutAsync(key, value).ContinueWith(t => { },
>> CancellationToken.None, TaskContinuationOptions.None,
>> TaskScheduler.Default);
>>
>>
>>
>> to ensure that the continuation after the PutAsync occurred on a .Net
>> thread pool thread. This worked with no further bad results.
>>
>>
>>
>> A little while ago we upgraded to 2.13, and I note in this link (
>> https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations)
>> that the default behaviour has been changed.
>>
>>
>>
>> My reading of this is that the example above can now be changed to:
>>
>>
>>
>>   await _cache.PutAsync(key, value);
>>
>>
>>
>> with the continuation occurring on a .Net threadpool thread by default.
>> Is that correct?
>>
>>
>>
>> This will allow us to remove the (quite a few) .ContinueWith() statements
>> which do add a layer of overhead on async calls.
>>
>>
>>
>> Thanks,
>> Raymond.
>>
>>
>>
>> --
>>
>> 
>>
>> *Raymond Wilson*Trimble Distinguished Engineer, Civil Construction
>> Software (CCS)
>> 11 Birmingham Drive | Christchurch, New Zealand
>> raymond_wil...@trimble.com
>>
>>
>> 
>>
>>
>>
>
>
> --
> 
> Raymond Wilson
> Trimble Distinguished Engineer, Civil Construction Software (CCS)
> 11 Birmingham Drive | Christchurch, New Zealand
> raymond_wil...@trimble.com
>
>
> 
>

Re: Thin client reconnection handling inside kubernetes

[Pavel Tupitsyn]

Hello,

> does not re-attempt to re-resolve addresses
Yes, that's a known issue [1], it is fixed in upcoming Ignite 2.14
The release is expected within the next month.

For now I can only recommend recycling IgniteClient instances when scaling
k8s up/down.

[1] https://issues.apache.org/jira/browse/IGNITE-17217

On Tue, Sep 27, 2022 at 11:46 PM Łukasz Dywicki  wrote:

> Hello everyone,
> I am coming to user mailing list with fairly basic issue, which probably
> can be easily answered, but it got me stuck for quite a bit already.
>
> I got a fairly basic ignite application with map/reduce task. A caller
> of a task uses thin client and async api to dispatch request. All works
> as expected until ignite cluster is being restarted. What I found is
> that thin client is not able to renew its connection if member IP
> addresses are shifted.
> Strange enough client marks channel as closed, but does not re-attempt
> to re-resolve addresses, it just tries to connect same address over and
> over. Last day I tested several configurations, however none of them
> helped (throttling, heartbeat, timeout).
> Additionally I found out that when cluster member pods are rolled out
> (ie. through scale down), re-connection attempt hangs for fairly long
> time. Turns out that connection timeout is set to fairly large amount:
>
> https://github.com/apache/ignite/blob/ignite-2.13/modules/core/src/main/java/org/apache/ignite/internal/client/thin/io/gridnioserver/GridNioClientConnectionMultiplexer.java#L125
>
> My client configuration (still using ignite 2.13) is as below:
> ClientConfiguration cfg = new ClientConfiguration();
> cfg.setReconnectThrottlingPeriod(0);
> cfg.setReconnectThrottlingRetries(0);
> cfg.setHeartbeatEnabled(true);
> cfg.setTimeout(500);
> cfg.setPartitionAwarenessEnabled(false);
>
> k8cfg = new KubernetesConnectionConfiguration();
> k8cfg.setServiceName(kubernetesServiceName);
> cfg.setAddressesFinder(new ThinClientKubernetesAddressFinder(k8cfg));
>
> Client call is just:
> igniteClient.compute()
>.executeAsync2("sampleTask", taskArgument)
>.toCompletableFuture()
>.whenComplete((r, e) -> ...)
>
> I wanted to confirm an expected behavior here, maybe I miss something
> obvious or miss understand the documentation.
>
> Kind regards,
> Łukasz
>

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Sorry, the code you sent is not working.

- There is a stack overflow due to recursion
at UnitOfWork.testModelRepository
- Cache does not exist exception at IgniteCacheInstance (changed
to GetOrCreateCache)
- All exceptions are swallowed (empty catch blocks)

I made it work with some changes, now I get exit code 0, not sure what to
do with that.

On Wed, Sep 21, 2022 at 1:09 PM Charlin S  wrote:

> Hi,
> Please let me know if there are any findings in my configuration or
> implementation.
>
> Regards,
> Charlin
>
>
> On Tue, 20 Sept 2022 at 11:51, Charlin S  wrote:
>
>> Hi,
>> I already attached server config in the same mail thread( along with
>> sample code).
>>
>> Regards,
>> Charlin
>>
>>
>> On Tue, 20 Sept 2022 at 11:50, Pavel Tupitsyn 
>> wrote:
>>
>>> Now the client starts, but cannot connect to a server. How do I start a
>>> server, do you use a config file or something else?
>>>
>>> On Tue, Sep 20, 2022 at 9:15 AM Charlin S 
>>> wrote:
>>>
>>>> Hi,
>>>> Sorry for inconvenience, I have removed local machine path dependency.
>>>>
>>>> Regards,
>>>> Charlin
>>>>
>>>>
>>>> On Tue, 20 Sept 2022 at 11:04, Pavel Tupitsyn 
>>>> wrote:
>>>>
>>>>> Still can't run, sorry. It has hardcoded directories from your machine
>>>>> (IgniteSettings.IGNITE_HOME, IgniteSettings.WorkDirectory - commented them
>>>>> out), and also GlobalVariables.IgniteSettings.ConfigurationPath:
>>>>>
>>>>> > Spring XML configuration path is invalid:
>>>>> D:\NGSourceCode\IgniteConfig\Labs/common_dynamiccache_client_config.xml
>>>>>
>>>>> Please make sure the zip file has everything required to run the
>>>>> application on any machine.
>>>>>
>>>>> On Mon, Sep 19, 2022 at 5:07 PM Charlin S 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I have attached here.
>>>>>> Regards,
>>>>>> Charlin
>>>>>>
>>>>>>
>>>>>> On Mon, 19 Sept 2022 at 19:00, Pavel Tupitsyn 
>>>>>> wrote:
>>>>>>
>>>>>>> I get an exception when trying to run the code:
>>>>>>>
>>>>>>> Unhandled exception. System.NullReferenceException: Object reference
>>>>>>> not set to an instance of an object.
>>>>>>>at ConsoleApp4.IgniteInstance..ctor() in
>>>>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>>>>>>> 47
>>>>>>>at ConsoleApp4.IgniteInstance.get_Instance() in
>>>>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>>>>>>> 33
>>>>>>>at ConsoleApp4.UnitOfWork..ctor() in
>>>>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/UnitOfWork.cs:line
>>>>>>> 15
>>>>>>>at ConsoleApp4.Program.Main(String[] args) in
>>>>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/Program.cs:line
>>>>>>>  12
>>>>>>>
>>>>>>> On Mon, Sep 19, 2022 at 3:04 PM Charlin S 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> Cache EexpiryPolicy is 60 minutes. but cache key failing in between
>>>>>>>> and return data after some time so its not due to expiry policy.
>>>>>>>> I have prepared a separate solution and attached it here. I am
>>>>>>>> sharing a cache data reading time screenshot for your reference .
>>>>>>>> [image: image.png]
>>>>>>>> Regards,
>>>>>>>> Charlin
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, 19 Sept 2022 at 11:00, Pavel Tupitsyn 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> The code you provided above uses ExpiryPolicy. It might be the
>>>>>>>>> reason for TryGet to return false - cache entry expires.
>>>>>>>>> To provide a definitive answer we need 

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Now the client starts, but cannot connect to a server. How do I start a
server, do you use a config file or something else?

On Tue, Sep 20, 2022 at 9:15 AM Charlin S  wrote:

> Hi,
> Sorry for inconvenience, I have removed local machine path dependency.
>
> Regards,
> Charlin
>
>
> On Tue, 20 Sept 2022 at 11:04, Pavel Tupitsyn 
> wrote:
>
>> Still can't run, sorry. It has hardcoded directories from your machine
>> (IgniteSettings.IGNITE_HOME, IgniteSettings.WorkDirectory - commented them
>> out), and also GlobalVariables.IgniteSettings.ConfigurationPath:
>>
>> > Spring XML configuration path is invalid:
>> D:\NGSourceCode\IgniteConfig\Labs/common_dynamiccache_client_config.xml
>>
>> Please make sure the zip file has everything required to run the
>> application on any machine.
>>
>> On Mon, Sep 19, 2022 at 5:07 PM Charlin S  wrote:
>>
>>> Hi,
>>> I have attached here.
>>> Regards,
>>> Charlin
>>>
>>>
>>> On Mon, 19 Sept 2022 at 19:00, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> I get an exception when trying to run the code:
>>>>
>>>> Unhandled exception. System.NullReferenceException: Object reference
>>>> not set to an instance of an object.
>>>>at ConsoleApp4.IgniteInstance..ctor() in
>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>>>> 47
>>>>at ConsoleApp4.IgniteInstance.get_Instance() in
>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>>>> 33
>>>>at ConsoleApp4.UnitOfWork..ctor() in
>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/UnitOfWork.cs:line
>>>> 15
>>>>at ConsoleApp4.Program.Main(String[] args) in
>>>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/Program.cs:line 
>>>> 12
>>>>
>>>> On Mon, Sep 19, 2022 at 3:04 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> Cache EexpiryPolicy is 60 minutes. but cache key failing in between
>>>>> and return data after some time so its not due to expiry policy.
>>>>> I have prepared a separate solution and attached it here. I am sharing
>>>>> a cache data reading time screenshot for your reference .
>>>>> [image: image.png]
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 19 Sept 2022 at 11:00, Pavel Tupitsyn 
>>>>> wrote:
>>>>>
>>>>>> The code you provided above uses ExpiryPolicy. It might be the reason
>>>>>> for TryGet to return false - cache entry expires.
>>>>>> To provide a definitive answer we need a project that runs and
>>>>>> demonstrates the problem.
>>>>>>
>>>>>> On Fri, Sep 16, 2022 at 5:12 PM Charlin S 
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> sorry for inconvenience. It's 90% actual code only. I have changed
>>>>>>> cache name and grid name as per policy. I have to prepare a separate
>>>>>>> solution and provide you if required.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Charlin
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, 16 Sept 2022 at 19:11, Pavel Tupitsyn 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It does not have to be your actual code, just something that
>>>>>>>> demonstrates the problem
>>>>>>>>
>>>>>>>> On Fri, Sep 16, 2022 at 4:40 PM Pavel Tupitsyn <
>>>>>>>> ptupit...@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Can you please share a complete project that I can run and see
>>>>>>>>> the issue?
>>>>>>>>>
>>>>>>>>> On Fri, Sep 16, 2022 at 4:30 PM Charlin S 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> Sorry I am not able to share a complete solution. bean xml
>>>>>>>>>> attached  and code snippet in below
>>>>>>>>>> IgniteConfi

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Still can't run, sorry. It has hardcoded directories from your machine
(IgniteSettings.IGNITE_HOME, IgniteSettings.WorkDirectory - commented them
out), and also GlobalVariables.IgniteSettings.ConfigurationPath:

> Spring XML configuration path is invalid:
D:\NGSourceCode\IgniteConfig\Labs/common_dynamiccache_client_config.xml

Please make sure the zip file has everything required to run the
application on any machine.

On Mon, Sep 19, 2022 at 5:07 PM Charlin S  wrote:

> Hi,
> I have attached here.
> Regards,
> Charlin
>
>
> On Mon, 19 Sept 2022 at 19:00, Pavel Tupitsyn 
> wrote:
>
>> I get an exception when trying to run the code:
>>
>> Unhandled exception. System.NullReferenceException: Object reference not
>> set to an instance of an object.
>>at ConsoleApp4.IgniteInstance..ctor() in
>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>> 47
>>at ConsoleApp4.IgniteInstance.get_Instance() in
>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
>> 33
>>at ConsoleApp4.UnitOfWork..ctor() in
>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/UnitOfWork.cs:line
>> 15
>>at ConsoleApp4.Program.Main(String[] args) in
>> /home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/Program.cs:line 12
>>
>> On Mon, Sep 19, 2022 at 3:04 PM Charlin S  wrote:
>>
>>> Hi,
>>> Cache EexpiryPolicy is 60 minutes. but cache key failing in between and
>>> return data after some time so its not due to expiry policy.
>>> I have prepared a separate solution and attached it here. I am sharing a
>>> cache data reading time screenshot for your reference .
>>> [image: image.png]
>>> Regards,
>>> Charlin
>>>
>>>
>>>
>>>
>>>
>>> On Mon, 19 Sept 2022 at 11:00, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> The code you provided above uses ExpiryPolicy. It might be the reason
>>>> for TryGet to return false - cache entry expires.
>>>> To provide a definitive answer we need a project that runs and
>>>> demonstrates the problem.
>>>>
>>>> On Fri, Sep 16, 2022 at 5:12 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> sorry for inconvenience. It's 90% actual code only. I have changed
>>>>> cache name and grid name as per policy. I have to prepare a separate
>>>>> solution and provide you if required.
>>>>>
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>>
>>>>>
>>>>> On Fri, 16 Sept 2022 at 19:11, Pavel Tupitsyn 
>>>>> wrote:
>>>>>
>>>>>> It does not have to be your actual code, just something that
>>>>>> demonstrates the problem
>>>>>>
>>>>>> On Fri, Sep 16, 2022 at 4:40 PM Pavel Tupitsyn 
>>>>>> wrote:
>>>>>>
>>>>>>> Can you please share a complete project that I can run and see
>>>>>>> the issue?
>>>>>>>
>>>>>>> On Fri, Sep 16, 2022 at 4:30 PM Charlin S 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> Sorry I am not able to share a complete solution. bean xml
>>>>>>>> attached  and code snippet in below
>>>>>>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>>>>>>> igniteGridIg.AutoGenerateIgniteInstanceName = true;
>>>>>>>> if
>>>>>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.IGNITE_HOME))
>>>>>>>> {
>>>>>>>> igniteGridIg.IgniteHome =
>>>>>>>> GlobalVariables.IgniteSettings.IGNITE_HOME;
>>>>>>>> }
>>>>>>>> var gridDetails =
>>>>>>>> GlobalVariables.IgniteSettings.GridDetails.Where(gd =>
>>>>>>>> gd.GridType.Equals("DynamicGrid",
>>>>>>>> StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
>>>>>>>> //setting ignite working directory if not ignite will
>>>>>>>> run with default work directory
>>>>>>>> if
>>>>>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.WorkDire

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

I get an exception when trying to run the code:

Unhandled exception. System.NullReferenceException: Object reference not
set to an instance of an object.
   at ConsoleApp4.IgniteInstance..ctor() in
/home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
47
   at ConsoleApp4.IgniteInstance.get_Instance() in
/home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/IgniteInstance.cs:line
33
   at ConsoleApp4.UnitOfWork..ctor() in
/home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/UnitOfWork.cs:line
15
   at ConsoleApp4.Program.Main(String[] args) in
/home/pavel/Downloads/ConsoleApp4/ConsoleApp4/ConsoleApp4/Program.cs:line 12

On Mon, Sep 19, 2022 at 3:04 PM Charlin S  wrote:

> Hi,
> Cache EexpiryPolicy is 60 minutes. but cache key failing in between and
> return data after some time so its not due to expiry policy.
> I have prepared a separate solution and attached it here. I am sharing a
> cache data reading time screenshot for your reference .
> [image: image.png]
> Regards,
> Charlin
>
>
>
>
>
> On Mon, 19 Sept 2022 at 11:00, Pavel Tupitsyn 
> wrote:
>
>> The code you provided above uses ExpiryPolicy. It might be the reason for
>> TryGet to return false - cache entry expires.
>> To provide a definitive answer we need a project that runs and
>> demonstrates the problem.
>>
>> On Fri, Sep 16, 2022 at 5:12 PM Charlin S  wrote:
>>
>>> Hi,
>>> sorry for inconvenience. It's 90% actual code only. I have changed cache
>>> name and grid name as per policy. I have to prepare a separate solution and
>>> provide you if required.
>>>
>>> Regards,
>>> Charlin
>>>
>>>
>>>
>>> On Fri, 16 Sept 2022 at 19:11, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> It does not have to be your actual code, just something that
>>>> demonstrates the problem
>>>>
>>>> On Fri, Sep 16, 2022 at 4:40 PM Pavel Tupitsyn 
>>>> wrote:
>>>>
>>>>> Can you please share a complete project that I can run and see
>>>>> the issue?
>>>>>
>>>>> On Fri, Sep 16, 2022 at 4:30 PM Charlin S 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Sorry I am not able to share a complete solution. bean xml attached
>>>>>> and code snippet in below
>>>>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>>>>> igniteGridIg.AutoGenerateIgniteInstanceName = true;
>>>>>> if
>>>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.IGNITE_HOME))
>>>>>> {
>>>>>> igniteGridIg.IgniteHome =
>>>>>> GlobalVariables.IgniteSettings.IGNITE_HOME;
>>>>>> }
>>>>>> var gridDetails =
>>>>>> GlobalVariables.IgniteSettings.GridDetails.Where(gd =>
>>>>>> gd.GridType.Equals("DynamicGrid",
>>>>>> StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
>>>>>> //setting ignite working directory if not ignite will run
>>>>>> with default work directory
>>>>>> if
>>>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.WorkDirectory))
>>>>>> {
>>>>>> igniteGridIg.WorkDirectory =
>>>>>> Path.Combine(GlobalVariables.IgniteSettings.WorkDirectory,
>>>>>> gridDetails.CacheType);
>>>>>> }
>>>>>> if (null != gridDetails)
>>>>>> {
>>>>>> GridName = gridDetails.GridName;
>>>>>> igniteGridIg.SpringConfigUrl =
>>>>>> Path.Combine(GlobalVariables.IgniteSettings.ConfigurationPath,
>>>>>> gridDetails.SpringXMLFileName);
>>>>>> igniteGridIg.ConsistentId =
>>>>>> Guid.NewGuid().ToString().ToUpper();
>>>>>> ICollection lstJvmOptions = null;
>>>>>>
>>>>>> //"IgniteJVMOption":
>>>>>> "-XX:+DisableExplicitGC,-XX:+UseG1GC,-Xms1g,-Xmx2g"
>>>>>> if (string.IsNullOrEmpty(gridDetails.IgniteJVMOption))
>>>>>> {
>>>>>> lstJvmOptions =
>>>>>> GlobalVariables.IgniteSettings.IgniteJVMOptionDefault.Sp

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

The code you provided above uses ExpiryPolicy. It might be the reason for
TryGet to return false - cache entry expires.
To provide a definitive answer we need a project that runs and demonstrates
the problem.

On Fri, Sep 16, 2022 at 5:12 PM Charlin S  wrote:

> Hi,
> sorry for inconvenience. It's 90% actual code only. I have changed cache
> name and grid name as per policy. I have to prepare a separate solution and
> provide you if required.
>
> Regards,
> Charlin
>
>
>
> On Fri, 16 Sept 2022 at 19:11, Pavel Tupitsyn 
> wrote:
>
>> It does not have to be your actual code, just something that demonstrates
>> the problem
>>
>> On Fri, Sep 16, 2022 at 4:40 PM Pavel Tupitsyn 
>> wrote:
>>
>>> Can you please share a complete project that I can run and see the issue?
>>>
>>> On Fri, Sep 16, 2022 at 4:30 PM Charlin S 
>>> wrote:
>>>
>>>> Hi,
>>>> Sorry I am not able to share a complete solution. bean xml attached
>>>> and code snippet in below
>>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>>> igniteGridIg.AutoGenerateIgniteInstanceName = true;
>>>> if
>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.IGNITE_HOME))
>>>> {
>>>> igniteGridIg.IgniteHome =
>>>> GlobalVariables.IgniteSettings.IGNITE_HOME;
>>>> }
>>>> var gridDetails =
>>>> GlobalVariables.IgniteSettings.GridDetails.Where(gd =>
>>>> gd.GridType.Equals("DynamicGrid",
>>>> StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
>>>> //setting ignite working directory if not ignite will run
>>>> with default work directory
>>>> if
>>>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.WorkDirectory))
>>>> {
>>>> igniteGridIg.WorkDirectory =
>>>> Path.Combine(GlobalVariables.IgniteSettings.WorkDirectory,
>>>> gridDetails.CacheType);
>>>> }
>>>> if (null != gridDetails)
>>>> {
>>>> GridName = gridDetails.GridName;
>>>> igniteGridIg.SpringConfigUrl =
>>>> Path.Combine(GlobalVariables.IgniteSettings.ConfigurationPath,
>>>> gridDetails.SpringXMLFileName);
>>>> igniteGridIg.ConsistentId =
>>>> Guid.NewGuid().ToString().ToUpper();
>>>> ICollection lstJvmOptions = null;
>>>>
>>>> //"IgniteJVMOption": "-XX:+DisableExplicitGC,-XX:+UseG1GC,-Xms1g,-Xmx2g"
>>>> if (string.IsNullOrEmpty(gridDetails.IgniteJVMOption))
>>>> {
>>>> lstJvmOptions =
>>>> GlobalVariables.IgniteSettings.IgniteJVMOptionDefault.Split(',').Where(x =>
>>>> !string.IsNullOrWhiteSpace(x)).ToList();
>>>> }
>>>> else
>>>> lstJvmOptions =
>>>> gridDetails.IgniteJVMOption.Split(',').Where(x =>
>>>> !string.IsNullOrWhiteSpace(x)).ToList();
>>>>
>>>> if (lstJvmOptions.Any())
>>>> igniteGridIg.JvmOptions = lstJvmOptions;
>>>> int RetryCount = 0;
>>>> while (InstanceObject == null)
>>>> {
>>>> try
>>>> {
>>>> RetryCount++;
>>>> //Try to start Ignite
>>>> if
>>>> (!String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteInstanceRetryLimit"))
>>>> && RetryCount >
>>>> (ApplicationSettingKeys.Get("IgniteInstanceRetryLimit")))
>>>> {
>>>> break;
>>>> }
>>>> //Try to start Ignite
>>>> if
>>>> (String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid")))
>>>> InstanceObject =
>>>> Ignition.Start(igniteGridIg);
>>>> else
>>>> InstanceObject =
>>>> Ignition.TryGetIgnite(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid"))
&g

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

It does not have to be your actual code, just something that demonstrates
the problem

On Fri, Sep 16, 2022 at 4:40 PM Pavel Tupitsyn  wrote:

> Can you please share a complete project that I can run and see the issue?
>
> On Fri, Sep 16, 2022 at 4:30 PM Charlin S  wrote:
>
>> Hi,
>> Sorry I am not able to share a complete solution. bean xml attached  and
>> code snippet in below
>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>> igniteGridIg.AutoGenerateIgniteInstanceName = true;
>> if
>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.IGNITE_HOME))
>> {
>> igniteGridIg.IgniteHome =
>> GlobalVariables.IgniteSettings.IGNITE_HOME;
>> }
>> var gridDetails =
>> GlobalVariables.IgniteSettings.GridDetails.Where(gd =>
>> gd.GridType.Equals("DynamicGrid",
>> StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
>> //setting ignite working directory if not ignite will run
>> with default work directory
>> if
>> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.WorkDirectory))
>> {
>> igniteGridIg.WorkDirectory =
>> Path.Combine(GlobalVariables.IgniteSettings.WorkDirectory,
>> gridDetails.CacheType);
>> }
>> if (null != gridDetails)
>> {
>> GridName = gridDetails.GridName;
>> igniteGridIg.SpringConfigUrl =
>> Path.Combine(GlobalVariables.IgniteSettings.ConfigurationPath,
>> gridDetails.SpringXMLFileName);
>> igniteGridIg.ConsistentId =
>> Guid.NewGuid().ToString().ToUpper();
>> ICollection lstJvmOptions = null;
>>
>> //"IgniteJVMOption": "-XX:+DisableExplicitGC,-XX:+UseG1GC,-Xms1g,-Xmx2g"
>> if (string.IsNullOrEmpty(gridDetails.IgniteJVMOption))
>> {
>> lstJvmOptions =
>> GlobalVariables.IgniteSettings.IgniteJVMOptionDefault.Split(',').Where(x =>
>> !string.IsNullOrWhiteSpace(x)).ToList();
>> }
>> else
>> lstJvmOptions =
>> gridDetails.IgniteJVMOption.Split(',').Where(x =>
>> !string.IsNullOrWhiteSpace(x)).ToList();
>>
>> if (lstJvmOptions.Any())
>> igniteGridIg.JvmOptions = lstJvmOptions;
>> int RetryCount = 0;
>> while (InstanceObject == null)
>> {
>> try
>> {
>> RetryCount++;
>> //Try to start Ignite
>> if
>> (!String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteInstanceRetryLimit"))
>> && RetryCount >
>> (ApplicationSettingKeys.Get("IgniteInstanceRetryLimit")))
>> {
>> break;
>> }
>> //Try to start Ignite
>> if
>> (String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid")))
>> InstanceObject = Ignition.Start(igniteGridIg);
>> else
>> InstanceObject =
>> Ignition.TryGetIgnite(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid"))
>> ?? Ignition.Start(igniteGridIg);
>> }
>> catch (Exception ex) //If failing to start Ignite,
>> wait a bit for the previous AppDomain to stop the Ignite running instance...
>> {
>> logContextInfo.ErrorLogger(ex);
>>
>> }
>> }
>>
>>  public ICache testModelICache { get; set; }
>>
>> using (new Log4NetTimeLogHelper(logContextInfo, "testModelModel",
>> "testModelModel"))
>> {
>>
>> testModelICache = testModelICache ??
>> InstanceObject.GetCache("testModelModel")
>> .WithExpiryPolicy(new ExpiryPolicy(
>> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime),
>> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime),
>> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime)
>> ));
>> }
>>
>>
>> public CacheServiceResponse GetCacheById(string Key)
>> {
>> CacheServiceResponse response = new CacheServiceResponse();
>> var model = (T)Act

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Can you please share a complete project that I can run and see the issue?

On Fri, Sep 16, 2022 at 4:30 PM Charlin S  wrote:

> Hi,
> Sorry I am not able to share a complete solution. bean xml attached  and
> code snippet in below
> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
> igniteGridIg.AutoGenerateIgniteInstanceName = true;
> if
> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.IGNITE_HOME))
> {
> igniteGridIg.IgniteHome =
> GlobalVariables.IgniteSettings.IGNITE_HOME;
> }
> var gridDetails =
> GlobalVariables.IgniteSettings.GridDetails.Where(gd =>
> gd.GridType.Equals("DynamicGrid",
> StringComparison.OrdinalIgnoreCase)).FirstOrDefault();
> //setting ignite working directory if not ignite will run with
> default work directory
> if
> (!string.IsNullOrEmpty(GlobalVariables.IgniteSettings.WorkDirectory))
> {
> igniteGridIg.WorkDirectory =
> Path.Combine(GlobalVariables.IgniteSettings.WorkDirectory,
> gridDetails.CacheType);
> }
> if (null != gridDetails)
> {
> GridName = gridDetails.GridName;
> igniteGridIg.SpringConfigUrl =
> Path.Combine(GlobalVariables.IgniteSettings.ConfigurationPath,
> gridDetails.SpringXMLFileName);
> igniteGridIg.ConsistentId =
> Guid.NewGuid().ToString().ToUpper();
> ICollection lstJvmOptions = null;
>
> //"IgniteJVMOption": "-XX:+DisableExplicitGC,-XX:+UseG1GC,-Xms1g,-Xmx2g"
> if (string.IsNullOrEmpty(gridDetails.IgniteJVMOption))
> {
> lstJvmOptions =
> GlobalVariables.IgniteSettings.IgniteJVMOptionDefault.Split(',').Where(x =>
> !string.IsNullOrWhiteSpace(x)).ToList();
> }
> else
> lstJvmOptions =
> gridDetails.IgniteJVMOption.Split(',').Where(x =>
> !string.IsNullOrWhiteSpace(x)).ToList();
>
> if (lstJvmOptions.Any())
> igniteGridIg.JvmOptions = lstJvmOptions;
> int RetryCount = 0;
> while (InstanceObject == null)
> {
> try
> {
> RetryCount++;
> //Try to start Ignite
> if
> (!String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteInstanceRetryLimit"))
> && RetryCount >
> (ApplicationSettingKeys.Get("IgniteInstanceRetryLimit")))
> {
> break;
> }
> //Try to start Ignite
> if
> (String.IsNullOrEmpty(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid")))
> InstanceObject = Ignition.Start(igniteGridIg);
> else
> InstanceObject =
> Ignition.TryGetIgnite(ApplicationSettingKeys.Get("IgniteDynamicCacheGrid"))
> ?? Ignition.Start(igniteGridIg);
> }
> catch (Exception ex) //If failing to start Ignite,
> wait a bit for the previous AppDomain to stop the Ignite running instance...
> {
> logContextInfo.ErrorLogger(ex);
>
> }
> }
>
>  public ICache testModelICache { get; set; }
>
> using (new Log4NetTimeLogHelper(logContextInfo, "testModelModel",
> "testModelModel"))
> {
>
> testModelICache = testModelICache ??
> InstanceObject.GetCache("testModelModel")
> .WithExpiryPolicy(new ExpiryPolicy(
> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime),
> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime),
> TimeSpan.FromMinutes(GlobalVariables.testModelExpiryTime)
> ));
> }
>
>
> public CacheServiceResponse GetCacheById(string Key)
> {
> CacheServiceResponse response = new CacheServiceResponse();
> var model = (T)Activator.CreateInstance(typeof(T));
> try
> {
>
> IgniteCache.TryGet(Key, out var value);
> var k = IgniteCache.TryGetAsync(Key);
> if (null != value)
> {
> model = (T)value;
> response.SingleObject = model;
> response.Status = true;
> response.StatusCode = System.Net.HttpStatusCode.OK;
> }
>   

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Can you share a reproducer?

On Fri, Sep 16, 2022 at 3:13 PM Charlin S  wrote:

> Hi,
> Data loss is not possible, because some other time I am able to read data
> with the same key.
>
> FYI: data update not in our workflow(Please ignore my previous mail
> statement it is possible to update the cache by another process/workflow).
> Always data read only once cache inserted by key.
>
> Regards,
> Charlin
>
>
> On Fri, 16 Sept 2022 at 17:21, Pavel Tupitsyn 
> wrote:
>
>> Ok, so the issue is that the key is not present in the cache, and this is
>> not expected.
>> Data loss is possible if one of Ignite nodes goes down, and backups [1]
>> are not configured.
>> Please check server logs for errors.
>>
>> [1]
>> https://ignite.apache.org/docs/latest/configuring-caches/configuring-backups
>>
>> On Fri, Sep 16, 2022 at 2:38 PM Charlin S  wrote:
>>
>>> Hi,
>>> There is no exception and executing the else part. it is possible to
>>> update the cache by another process/workflow.
>>> //response is custom class object
>>> var model = (T)Activator.CreateInstance(typeof(T));
>>> IgniteCache.TryGet(Key, out var value);
>>> if (null != value)
>>> {
>>> model = (T)value;
>>> response.SingleObject = model;
>>> response.Status = true;
>>> response.StatusCode =
>>> System.Net.HttpStatusCode.OK;
>>> }
>>> else
>>> {
>>> response.Status = false;
>>>     response.Message = "Given key " + Key + " not
>>> present in the cache.";
>>> response.StatusCode =
>>> System.Net.HttpStatusCode.NoContent;
>>> }
>>>
>>> Regards,
>>> Charlin
>>>
>>> On Fri, 16 Sept 2022 at 16:53, Pavel Tupitsyn 
>>> wrote:
>>>
>>>> Hi Charlin,
>>>>
>>>> Please provide full exception details.
>>>>
>>>> On Fri, Sep 16, 2022 at 2:04 PM Charlin S 
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> Cache.TryGet failed to read intermittently and was unable to read some
>>>>> time later.
>>>>>
>>>>> having 3 server nodes and 11 client nodes. cache mode is partition and
>>>>> atomicitymode not been set.
>>>>>
>>>>> Regards,
>>>>> Charlin
>>>>>
>>>>>

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Ok, so the issue is that the key is not present in the cache, and this is
not expected.
Data loss is possible if one of Ignite nodes goes down, and backups [1] are
not configured.
Please check server logs for errors.

[1]
https://ignite.apache.org/docs/latest/configuring-caches/configuring-backups

On Fri, Sep 16, 2022 at 2:38 PM Charlin S  wrote:

> Hi,
> There is no exception and executing the else part. it is possible to
> update the cache by another process/workflow.
> //response is custom class object
> var model = (T)Activator.CreateInstance(typeof(T));
> IgniteCache.TryGet(Key, out var value);
> if (null != value)
> {
> model = (T)value;
> response.SingleObject = model;
> response.Status = true;
> response.StatusCode = System.Net.HttpStatusCode.OK;
> }
> else
> {
> response.Status = false;
> response.Message = "Given key " + Key + " not
> present in the cache.";
> response.StatusCode =
> System.Net.HttpStatusCode.NoContent;
>     }
>
> Regards,
> Charlin
>
> On Fri, 16 Sept 2022 at 16:53, Pavel Tupitsyn 
> wrote:
>
>> Hi Charlin,
>>
>> Please provide full exception details.
>>
>> On Fri, Sep 16, 2022 at 2:04 PM Charlin S  wrote:
>>
>>> Hi All,
>>> Cache.TryGet failed to read intermittently and was unable to read some
>>> time later.
>>>
>>> having 3 server nodes and 11 client nodes. cache mode is partition and
>>> atomicitymode not been set.
>>>
>>> Regards,
>>> Charlin
>>>
>>>

Re: Ignite cache failed to read by key intermittently

[Pavel Tupitsyn]

Hi Charlin,

Please provide full exception details.

On Fri, Sep 16, 2022 at 2:04 PM Charlin S  wrote:

> Hi All,
> Cache.TryGet failed to read intermittently and was unable to read some
> time later.
>
> having 3 server nodes and 11 client nodes. cache mode is partition and
> atomicitymode not been set.
>
> Regards,
> Charlin
>
>

Re: Why does Thin client connect to all servers in connection string???

[Pavel Tupitsyn]

Hi, this is called "partition awareness" [1] - thin client establishes
connections to all known nodes to be able to send requests directly to the
primary node for the given entry.
Multiple connections are also useful for load balancing and improved
reliability.

You can disable this behavior with
ClientConfiguration#setPartitionAwareness(false) [2]

[1]
https://ignite.apache.org/docs/latest/thin-clients/java-thin-client#partition-awareness
[2]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConfiguration.html#setPartitionAwarenessEnabled-boolean-

On Tue, Sep 6, 2022 at 2:09 AM Gregory Sylvain 
wrote:

>
>
> Hi,
>
>
>
> I am running an Ignite 2.12.0 Service Grid with 5 Server Nodes and 32 Thin
> Client nodes in a private cloud of RHEL 7.9 VMs.  (no docker images).
>
>
>
> Doing a sqlline CLI query for connected clients, I get the expected 32
> connected clients.  However, if I execute netstat (or ss) on all
> ServerNodes, I see an ESTABLISHed connection from each client to each
> ServerNode on port 10800?  These connections seem to be maintained (e.g.
> they do not timeout after 2 hours).
>
>
>
> I am using Spring XML to configure both client and server nodes.  The
> server nodes are also utilizing systemd for reliability.
>
>
>
> Any idea what is going on?
>
>
>
> Thanks in advance.
>
> Greg
>
>
>

Re: Error on Ignite.close(): class java.io.ObjectInputStream$Caches$1 cannot be cast to class java.util.Map

[Pavel Tupitsyn]

Roman, good catch - I've cherry-picked IGNITE-17481
 to 2.14

On Mon, Aug 29, 2022 at 9:47 AM Roman Puchkovskiy <
roman.puchkovs...@gmail.com> wrote:

> Looks like this has been fixed in
> https://issues.apache.org/jira/browse/IGNITE-17481
> Not sure if this is going to land in 2.14. Probably it should, the fix
> is tiny, but the issue can be very annoying on recent Java versions.
>
> пн, 15 авг. 2022 г. в 11:45, Maksim Timonin :
> >
> > Hi Rafael,
> >
> > Getting `java.lang.ClassCastException` looks like a bug. I'm not sure
> but it's required to have different cleaning for jdk8 and jdk11. Thanks for
> finding that!
> >
> > > Do you have a guideline "howto contribute" if this is wanted?
> >
> > Please check this guide:
> https://github.com/apache/ignite/blob/master/CONTRIBUTING.md
> >
> > Feel free to ask any questions.
> >
> >
> >
> > On Tue, Aug 9, 2022 at 4:20 PM Rafael Troilo 
> wrote:
> >>
> >> Hi,
> >>
> >> @Paolo de Dios, thank you for creating an issue for this
> >> - https://issues.apache.org/jira/browse/IGNITE-17481
> >>
> >> I attached a patch for this issue.
> >>
> >> Do you have a guideline "howto contribute" if this is wanted?
> >>
> >> Best,
> >> Rafael
> >>
> >>
> >>
> >> On 8/5/22 16:18, Rafael Troilo wrote:
> >> > Hi,
> >> >
> >> > in case it wasn't reported before.
> >> >
> >> > On Ignite.close we got an Error:
> >> >
> >> >
> >> > ```
> >> > SEVERE: Failed to stop component (ignoring): GridManagerAdapter
> [enabled=true, name=o.a.i.i.managers.deployment.GridDeploymentManager]
> >> > java.lang.ClassCastException:
> (java.io.ObjectInputStream$Caches$1 and java.util.Map are in module
> java.base of loader 'bootstrap')
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentStoreAdapter.clearSerializationCache(GridDeploymentStoreAdapter.java:151)
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentStoreAdapter.clearSerializationCaches(GridDeploymentStoreAdapter.java:120)
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.undeploy(GridDeploymentLocalStore.java:565)
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore.stop(GridDeploymentLocalStore.java:101)
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.storesStop(GridDeploymentManager.java:630)
> >> >  at
> org.apache.ignite.internal.managers.deployment.GridDeploymentManager.stop(GridDeploymentManager.java:137)
> >> >  at
> org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:1928)
> >> >  at
> org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:1806)
> >> >  at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2382)
> >> >  at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2205)
> >> >  at
> org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:350)
> >> >  at org.apache.ignite.Ignition.stop(Ignition.java:230)
> >> >  at
> org.apache.ignite.internal.IgniteKernal.close(IgniteKernal.java:2776)
> >> > ```
> >> >
> >> > ver. 2.13.0#20220420-sha1:551f6ece
> >> > OS: Linux 4.15.0-189-generic amd64
> >> > VM information: OpenJDK Runtime Environment
> 11.0.16+8-post-Ubuntu-0ubuntu118.04 Ubuntu OpenJDK 64-Bit Server VM
> 11.0.16+8-post-Ubuntu-0ubuntu118.04
> >> >
> >> > The reason for this Exception is an access to an field of
> ObjectOutputStream$Caches.subclassAudits which used to be of type
> java.util.Map but it changed to type java.lang.ClassValue!
> >> >
> >> > ```
> >> >
> org.apache.ignite.internal.managers.deployment.GridDeploymentStoreAdapter::clearSerializationCaches
> >> >
> >> >
> >> > clearSerializationCache(Class.forName("java.io.ObjectInputStream$Caches"),
> "subclassAudits");
> >> >
> >> > clearSerializationCache(Class.forName("java.io.ObjectOutputStream$Caches"),
> "subclassAudits");
> >> >
> >> > clearSerializationCache(Class.forName("java.io.ObjectStreamClass$Caches"),
> "localDescs");
> >> >
> >> > clearSerializationCache(Class.forName("java.io.ObjectStreamClass$Caches"),
> "reflectors");
> >> > ```
> >> >
> >> > Is it safe to ignore this Exception? Any workarounds?
> >> >
> >> > Thank you,
> >> > Best,
> >> > Rafael
> >> >
> >> >
> >> >
> >>
> >> --
> >> Rafael Troilo
> >> HeiGIT gGmbH
> >> Heidelberg Institute for Geoinformation Technology at Heidelberg
> University
> >>
> >> https://heigit.org | rafael.tro...@heigit.org | phone +49-6221-533 484
> >>
> >> Postal address: Schloss-Wolfsbrunnenweg 33 | 69118 Heidelberg | Germany
> >> Offices: Berliner Str. 45 | 69120 Heidelberg | Germany
> >>
> >> Amtsgericht Mannheim | HRB 733765
> >> Managing Directors: Prof. Dr. Alexander Zipf | Dr. Gesa Schönberger
>

Re: distributed-computing error System.Runtime.Serialization.ISerializable

[Pavel Tupitsyn]

> Sorry to bother you，does scan in distribute compute faster than use a
scan in client？

Depends on the use case. Always measure your own scenario.

On Wed, Aug 24, 2022 at 2:34 PM wkhapy...@gmail.com 
wrote:

> Sorry to bother you，does scan in distribute compute faster than use a scan
> in client？
>
> ---Original---
> *From:* "Pavel Tupitsyn"
> *Date:* Wed, Aug 24, 2022 19:14 PM
> *To:* "user";
> *Subject:* Re: distributed-computing error
> System.Runtime.Serialization.ISerializable
>
> This line is the problem:
> public ICache IgniteCache { get; set; }
>
> You can't serialize an Ignite cache. Instead, use [InstanceResource] -
> uncomment the line you already have, then get the cache by name:
>
> [InstanceResource] private readonly IIgnite _ignite;
> public string CacheName { get; set; }
>
> ...
>
> var cache = _ignite.GetCache(CacheName);
> cache.Query(...)
>
> On Wed, Aug 24, 2022 at 1:31 PM Charlin S  wrote:
>
>> Hi,
>> The exception details are as follows.
>> System.AggregateException
>>   HResult=0x80131500
>>   Message=One or more errors occurred. (Serializing delegates is not
>> supported on this platform.)
>>   Source=System.Private.CoreLib
>>   StackTrace:
>>at System.Threading.Tasks.Task`1.GetResultCore(Boolean
>> waitCompletionNotification)
>>at System.Threading.Tasks.Task`1.get_Result()
>>at Apache.Ignite.Core.Impl.Common.Future`1.Get()
>>at
>> Apache.Ignite.Core.Impl.Compute.Compute.Apply[TArg,TJobRes](IComputeFunc`2
>> clo, TArg arg)
>>at ConsoleApp2.CacheUtils.ComputeTest_OnIgnite(String searchCriteria)
>> in D:\NGSourceCode\POC\ConsoleApp2\ConsoleApp2\CacheUtils.cs:line 76
>>at ConsoleApp2.Program.Main(String[] args) in
>> D:\NGSourceCode\POC\ConsoleApp2\ConsoleApp2\Program.cs:line 19
>>
>>   This exception was originally thrown at this call stack:
>>
>> System.MulticastDelegate.GetObjectData(System.Runtime.Serialization.SerializationInfo,
>> System.Runtime.Serialization.StreamingContext)
>>
>> Apache.Ignite.Core.Impl.Binary.SerializableSerializer.WriteBinary(T,
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
>>
>> Apache.Ignite.Core.Impl.Binary.BinaryReflectiveSerializerInternal.Apache.Ignite.Core.Impl.Binary.IBinarySerializerInternal.WriteBinary(T,
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
>>
>> Apache.Ignite.Core.Impl.Binary.BinaryReflectiveSerializerInternal.Apache.Ignite.Core.Impl.Binary.IBinarySerializerInternal.WriteBinary(T,
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
>> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
>> ...
>> [Call Stack Truncated]
>>
>> Inner Exception 1:
>> SerializationException: Serializing delegates is not supported on this
>> platform.
>>
>> Note: I am having two 2 server nodes and running the POC application on a
>> client node.
>>
>> Regards,
>> Charlin
>>
>>
>>
>> On Wed, 24 Aug 2022 at 15:02, Pavel Tupitsyn 
>> wrote:
>>
>>> Please share the exception with full stack trace (you've only shared
>>> some warnings, which may or may not be a problem).
>>>
>>> On Wed, Aug 24, 2022 at 10:26 AM Charlin S 
>>> wrote:
>>>
>>>>
>>>> I have a requirement to fetch the data using distributed-computing
>>>> function and  error at
>>>> var res =
>>>> CacheInstance.Instance.InstanceObject.GetCompute().Call(calls);
>>>>
>>>> //  InstanceObject singleton object for Ignite
>>>>
>>>> [11:16:34,865][WARNING][main][Marshaller] Type
>>>> 'System.Collections.Generic.Dictionary`2[System.Int64,Apache.Ignite.Core.Impl.Binary.BinaryFullTypeDescriptor]'
>>>> implements 'System.Runtime.Serialization.ISerializable'. It will be written
>>>> in Ignite binary format, however, the following limitations apply: DateTime
>>>> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
>>>> work in DML.
>>>> [11:16:34,889][WARNING][main][Marshaller] Type
>>>> 'System.Collections.Generic.Dictionary`2[System.String,System.Byte]'
>>>> implements 'System.Runtime.Serialization.ISerializable'. It will be writt

Re: distributed-computing error System.Runtime.Serialization.ISerializable

[Pavel Tupitsyn]

This line is the problem:
public ICache IgniteCache { get; set; }

You can't serialize an Ignite cache. Instead, use [InstanceResource] -
uncomment the line you already have, then get the cache by name:

[InstanceResource] private readonly IIgnite _ignite;
public string CacheName { get; set; }

...

var cache = _ignite.GetCache(CacheName);
cache.Query(...)

On Wed, Aug 24, 2022 at 1:31 PM Charlin S  wrote:

> Hi,
> The exception details are as follows.
> System.AggregateException
>   HResult=0x80131500
>   Message=One or more errors occurred. (Serializing delegates is not
> supported on this platform.)
>   Source=System.Private.CoreLib
>   StackTrace:
>at System.Threading.Tasks.Task`1.GetResultCore(Boolean
> waitCompletionNotification)
>at System.Threading.Tasks.Task`1.get_Result()
>at Apache.Ignite.Core.Impl.Common.Future`1.Get()
>at
> Apache.Ignite.Core.Impl.Compute.Compute.Apply[TArg,TJobRes](IComputeFunc`2
> clo, TArg arg)
>at ConsoleApp2.CacheUtils.ComputeTest_OnIgnite(String searchCriteria)
> in D:\NGSourceCode\POC\ConsoleApp2\ConsoleApp2\CacheUtils.cs:line 76
>at ConsoleApp2.Program.Main(String[] args) in
> D:\NGSourceCode\POC\ConsoleApp2\ConsoleApp2\Program.cs:line 19
>
>   This exception was originally thrown at this call stack:
>
> System.MulticastDelegate.GetObjectData(System.Runtime.Serialization.SerializationInfo,
> System.Runtime.Serialization.StreamingContext)
>
> Apache.Ignite.Core.Impl.Binary.SerializableSerializer.WriteBinary(T,
> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
>
> Apache.Ignite.Core.Impl.Binary.BinaryReflectiveSerializerInternal.Apache.Ignite.Core.Impl.Binary.IBinarySerializerInternal.WriteBinary(T,
> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
>
> Apache.Ignite.Core.Impl.Binary.BinaryReflectiveSerializerInternal.Apache.Ignite.Core.Impl.Binary.IBinarySerializerInternal.WriteBinary(T,
> Apache.Ignite.Core.Impl.Binary.BinaryWriter)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.Write(T)
> Apache.Ignite.Core.Impl.Binary.BinaryWriter.WriteObject(string, T)
> ...
> [Call Stack Truncated]
>
> Inner Exception 1:
> SerializationException: Serializing delegates is not supported on this
> platform.
>
> Note: I am having two 2 server nodes and running the POC application on a
> client node.
>
> Regards,
> Charlin
>
>
>
> On Wed, 24 Aug 2022 at 15:02, Pavel Tupitsyn  wrote:
>
>> Please share the exception with full stack trace (you've only shared some
>> warnings, which may or may not be a problem).
>>
>> On Wed, Aug 24, 2022 at 10:26 AM Charlin S 
>> wrote:
>>
>>>
>>> I have a requirement to fetch the data using distributed-computing
>>> function and  error at
>>> var res =
>>> CacheInstance.Instance.InstanceObject.GetCompute().Call(calls);
>>>
>>> //  InstanceObject singleton object for Ignite
>>>
>>> [11:16:34,865][WARNING][main][Marshaller] Type
>>> 'System.Collections.Generic.Dictionary`2[System.Int64,Apache.Ignite.Core.Impl.Binary.BinaryFullTypeDescriptor]'
>>> implements 'System.Runtime.Serialization.ISerializable'. It will be written
>>> in Ignite binary format, however, the following limitations apply: DateTime
>>> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
>>> work in DML.
>>> [11:16:34,889][WARNING][main][Marshaller] Type
>>> 'System.Collections.Generic.Dictionary`2[System.String,System.Byte]'
>>> implements 'System.Runtime.Serialization.ISerializable'. It will be written
>>> in Ignite binary format, however, the following limitations apply: DateTime
>>> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
>>> work in DML.
>>> [11:16:34,895][WARNING][main][Marshaller] Type
>>> 'System.Func`2[Apache.Ignite.Core.Impl.Binary.BinaryReader,Apache.Ignite.Core.Impl.Compute.ComputeJobHolder]'
>>> implements 'System.Runtime.Serialization.ISerializable'. It will be written
>>> in Ignite binary format, however, the following limitations apply: DateTime
>>> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
>>> work in DML.
>>>
>>> Kindly help me on this and I have shared the implementation below.
>>>
>>>  public class ComputeTestModel : IBinarizable
>>> {
>>>
>>> [QuerySqlField]
>>> public st

Re: distributed-computing error System.Runtime.Serialization.ISerializable

[Pavel Tupitsyn]

Please share the exception with full stack trace (you've only shared some
warnings, which may or may not be a problem).

On Wed, Aug 24, 2022 at 10:26 AM Charlin S  wrote:

>
> I have a requirement to fetch the data using distributed-computing
> function and  error at
> var res = CacheInstance.Instance.InstanceObject.GetCompute().Call(calls);
>
> //  InstanceObject singleton object for Ignite
>
> [11:16:34,865][WARNING][main][Marshaller] Type
> 'System.Collections.Generic.Dictionary`2[System.Int64,Apache.Ignite.Core.Impl.Binary.BinaryFullTypeDescriptor]'
> implements 'System.Runtime.Serialization.ISerializable'. It will be written
> in Ignite binary format, however, the following limitations apply: DateTime
> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
> work in DML.
> [11:16:34,889][WARNING][main][Marshaller] Type
> 'System.Collections.Generic.Dictionary`2[System.String,System.Byte]'
> implements 'System.Runtime.Serialization.ISerializable'. It will be written
> in Ignite binary format, however, the following limitations apply: DateTime
> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
> work in DML.
> [11:16:34,895][WARNING][main][Marshaller] Type
> 'System.Func`2[Apache.Ignite.Core.Impl.Binary.BinaryReader,Apache.Ignite.Core.Impl.Compute.ComputeJobHolder]'
> implements 'System.Runtime.Serialization.ISerializable'. It will be written
> in Ignite binary format, however, the following limitations apply: DateTime
> fields would not work in SQL; sbyte, ushort, uint, ulong fields would not
> work in DML.
>
> Kindly help me on this and I have shared the implementation below.
>
>  public class ComputeTestModel : IBinarizable
> {
>
> [QuerySqlField]
> public string TestField1 { get; set; }
> [QuerySqlField]
> public string TestField2 { get; set; }
> [QuerySqlField]
> public string TestField3 { get; set; }
>
>
> public void ReadBinary(IBinaryReader reader){}//not added
> implementation here
> public void WriteBinary(IBinaryWriter writer){}//not added
> implementation here
> }
>
>
> class TestModelComputeFunc : IComputeFunc>
> {
> // [InstanceResource] private readonly IIgnite _ignite;
> public ICache IgniteCache { get; set; }
>
> public string _searchCriteria { get; set; }
> public TestModelComputeFunc(ICache igniteCache, string
> searchCriteria)
> {
> IgniteCache = igniteCache;
> _searchCriteria = searchCriteria;
> }
> public List Invoke()
> {
> List objs = new List();
> string query = "select * from ComputeTestModel where
>  TestField1 = ?";
> SqlFieldsQuery fieldsQuery =  new SqlFieldsQuery(query,
> _searchCriteria);
> IFieldsQueryCursor queryCursor =
> IgniteCache.Query(fieldsQuery);
> if (queryCursor != null)
> {
> foreach (var cacheEntry in queryCursor)
> {
> objs.Add(new ComputeTestModel
> {
> TestField1 = cacheEntry[0] as string,
> TestField2 = cacheEntry[1] as string,
> TestField3 = cacheEntry[2] as string
>
> });
> }
> return objs;
>
> }
> return null;
> }
> }
> }
>
> public static class CacheUtilsTest
> {
> public static List ComputeTest_OnIgnite(string
> searchCriteria)
> {
> //CacheInstance.Instance.ComputeTestICache defined in another class public
> ICache ComputeTestModelICache { get; set; }
>
> ComputeTestComputeFunc calls = new
> ComputeTestComputeFunc(CacheInstance.Instance.ComputeTestICache,
> searchCriteria);
>
> // Execute the collection of calls on the cluster.
> var res =
> CacheInstance.Instance.InstanceObject.GetCompute().Call(calls);
>  //exception thrown here
>
>
> /*
> //another aproach
> ComputeTestComputeFunc cacheObject = new
> ComputeTestComputeFunc(CacheInstance.Instance.ComputeTestICache,
> searchCriteria);
> // var calls = CacheInstance.Instance.ComputeTestICache.Select(s =>
> cacheObject).ToList();
>
>  // Execute the collection of calls on the cluster.
> var res =
> CacheInstance.Instance.InstanceObject.GetCompute().Call(calls);
>  //exception thrown here
> */
> return null;
> }
>
> }
>
>
> class Program
> {
>
> static void Main(string[] args)
> {
> // Console.WriteLine("Hello World!");
> var saticCache = CacheInstance.Instance.InstanceObject;
>
> var res = CacheUtilsTest.ComputeTest_OnIgnite("A");
> int ans= res.Count();
> Console.ReadKey();
>
> }
> }
>
> Regards,
> Charlin
>
>
>

Re: Which cacheKeyConfiguration

[Pavel Tupitsyn]

The intent is the same - to define an affinity key for a given type.
IgniteConfiguration does it globally, CacheConfiguration just for a
specific cache.

On Mon, Aug 1, 2022 at 9:12 PM Mike Wiesenberg 
wrote:

>
> Hi
>  I'm wondering why there are CacheKeyConfiguration setters on both the
> IgniteConfiguration and the CacheConfiguration classes. Looking briefly at
> the code, they seem to be used differently but I can't decipher the intent.
>
> Thanks
>  Mike
>

Re: Cursor in ThinClient failure

[Pavel Tupitsyn]

Hi, sorry for the late reply, your message got in my spam folder for some
reason.

> If the thin client fails during the execution of the above code, will it
leave an unclosed cursor on the server side?

No, if the query fails, all resources are cleared.

On Thu, Jul 7, 2022 at 1:26 PM 38797715 <38797...@qq.com> wrote:

> Does the thin client have a mechanism like Session?
>

Re: cannot unsubscribe

[Pavel Tupitsyn]

Please send any text to user-unsubscr...@ignite.apache.org

‪On Thu, Jul 14, 2022 at 10:24 AM ‫אריאל מסלאטון‬‎ 
wrote:‬

> I have been unsubscribing from the mailing list but I keep getting emails.
> I sent unsubscribe messages to every mailing list and it didn't help.
>
> Can you please assist?
> Regards
>
> --
> *054-2116997*
> *arielmasla...@gmail.com *
>

Re: TiKV vs Ignite 3

[Pavel Tupitsyn]

What about Ignite 2? It is great for KV scenarios, and the Java client is
first-class.

On Sat, Jul 9, 2022 at 6:51 PM Pavel Tupitsyn  wrote:

> Hi,
>
> Ignite 3 is in alpha stage, it is way too early to do any benchmarks.
> We are aware of existing performance issues, those will be fixed. Right
> now the focus is on correctness and features.
>
> > Are there any plans for Ignite 3 to have more low level functions like
> the scan api to iterate over rocksdb keys quickly
> Yes, I think so.
>
> On Sat, Jul 9, 2022 at 4:28 PM Matej  wrote:
>
>> Hi All.
>>
>> We like TiKV, it's fast, low level with low latency. But the problem is
>> their java client is useless.
>>
>> Ignite 3 seems to go more in the storage direction. But he API seems
>> high-level. We testes for speed and it seems much more slower then Ignite 2
>> and Tikv .
>>
>> Are there any plans for Ignite 3 to have more low level functions like
>> the scan api to iterate over rocksdb keys quickly. And have everything more
>> binary to not loose to much on schema stuff.
>>
>> Overall I really like the ignite 3 direction.
>>
>> BR
>>
>> Matej.
>>
>>

Re: TiKV vs Ignite 3

[Pavel Tupitsyn]

Hi,

Ignite 3 is in alpha stage, it is way too early to do any benchmarks.
We are aware of existing performance issues, those will be fixed. Right now
the focus is on correctness and features.

> Are there any plans for Ignite 3 to have more low level functions like
the scan api to iterate over rocksdb keys quickly
Yes, I think so.

On Sat, Jul 9, 2022 at 4:28 PM Matej  wrote:

> Hi All.
>
> We like TiKV, it's fast, low level with low latency. But the problem is
> their java client is useless.
>
> Ignite 3 seems to go more in the storage direction. But he API seems
> high-level. We testes for speed and it seems much more slower then Ignite 2
> and Tikv .
>
> Are there any plans for Ignite 3 to have more low level functions like the
> scan api to iterate over rocksdb keys quickly. And have everything more
> binary to not loose to much on schema stuff.
>
> Overall I really like the ignite 3 direction.
>
> BR
>
> Matej.
>
>

Re: Thin client cache operations takes longer and longer for subsequent retries to throw exception when server down

[Pavel Tupitsyn]

This behavior is controlled by the following fields in ClientConfiguration
[1]:
retryLimit, reconnectThrottlingRetries, reconnectThrottlingPeriod

To fail fast, set reconnectThrottlingPeriod to 0 and retryLimit to 1.

[1]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConfiguration.html

On Fri, Jul 8, 2022 at 4:46 PM Sumit Deshinge 
wrote:

> Correction in subject
>
> On Fri, Jul 8, 2022 at 7:12 PM Sumit Deshinge 
> wrote:
>
>> Hi,
>>
>> I have ignite thin clients connecting to ignite server.
>> When server node is down, ClientCache operations like get, exists takes
>> incremental times to throw clientconnectiontimeout exception. e.g. 40s,
>> 60s, 80s, ... upto ~600s and then again it starts with around 40s time for
>> response.
>>
>> Which setting/property do I need to change in order to reduce this time?
>> Basically I want these client cache operations to fail fast during
>> subsequent retries.
>>
>> --
>> Regards,
>> Sumit Deshinge
>>
>>
>
> --
> Regards,
> Sumit Deshinge
>
>

Re: Live fallback/backup scenario

[Pavel Tupitsyn]

> a) The persistent directory on the crashed server is still available
> (but most likely is not up-to-date with latest data from the other
> caches). Do I add the server to the base topology and it will
> automatically get the data that it missed during its downtime?

Restarted node is still a member of the baseline topology. You don't need
to do anything, the latest data will be synced automatically, and the
cluster continues to operate seamlessly.


> b) The server was wiped and the persistent directory is empty when it's
> restarted. Do I add the server to the base topology and it will
> automatically participate in the shared data caches to become a
> potential full backup in case another server crashes?

As long as IgniteConfiguration.consistentId is the same as before restart,
again, no manual steps are required. Baseline remains the same and wiped
data will be synced up from other nodes.

If consistentId changes, it is considered a new node, not a member of
baseline. You can add it to the baseline manually.


> How long does such data sync for the restarted server take? Is there an
> event for this?

The time depends on the data size and hardware, from seconds to minutes or
hours.
The event is CacheRebalancingEvent.

https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/events/CacheRebalancingEvent.html

On Sat, Jul 2, 2022 at 3:40 PM  wrote:

> Hi Igniters,
>
> can you please comment or correct the scenario described below? I assume
> zero-downtime and fault-tolerance against data loss on node crashes is a
> key feature of Ignite but the procedure in failure scenarios is not
> clear to me.
>
> Thanks!
>
> On 19.06.22 11:32, jay.et...@gmx.de wrote:
> > Hi,
> >
> > yes, your 4-point initial setup is how we've been operating our cluster
> so far. For maintenance we go in reverse order (leave out point 2).
> >
> > I'm very curious myself about the second part of your mail on how to
> correctly restore the cluster after a node crash.
> >
> > Can anyone comment on this? Any help appreciated here.
> >
> > Jay
> >
> >
> >
> > -Original Message-
> > From: don.tequ...@gmx.de 
> > Sent: Saturday, 11 June 2022 14:33
> > To: user@ignite.apache.org
> > Subject: Live fallback/backup scenario
> >
> > Hi,
> >
> > I'm experimenting with a Ignite cluster with multiple server nodes and
> multiple client nodes. My understanding is that with Ignite I can avoid
> data loss of all persistent caches and can avoid downtime for all clients.
> >
> > If the above assumption is correct, how do I manage the servers and
> baseline topology for this scenario?
> >
> > Caches are configured with persistence enabled and:
> >   
> >   
> >   
> >value="FULL_SYNC"/>
> >
> > Is this procedure correct for initial setup?
> >
> > 1. Start all server nodes and keep the cluster inactive.
> > 2. Once all server nodes are connected, set the baseline topology to all
> server nodes.
> > 3. Activate the cluster.
> > 4. Connect clients and start application operation with compute and
> persistent caches.
> >
> > Let's say one server node crashes, I can see operation continues without
> interruption and no data loss. However, what's the scenario after the
> crashed server node was restarted and connected again to the cluster? I can
> see it does not automatically get a member of the baseline topology.
> >
> > What's the correct procedure in the two below scenarios:
> >
> > a) The persistent directory on the crashed server is still available
> (but most likely is not up-to-date with latest data from the other caches).
> Do I add the server to the base topology and it will automatically get the
> data that it missed during its downtime?
> >
> > b) The server was wiped and the persistent directory is empty when it's
> restarted. Do I add the server to the base topology and it will
> automatically participate in the shared data caches to become a potential
> full backup in case another server crashes?
> >
> > How long does such data sync for the restarted server take? Is there an
> event for this?
> >
> > Thanks for some background on this scenario.
> >
>

Re: ThinClient connection refused / disconnect

[Pavel Tupitsyn]

Thanks for confirmation!

On Sat, Jul 2, 2022 at 3:32 PM  wrote:

> Fixed it. It was indeed a wrongly configured IP address!
>
>
> On 15.06.22 22:29, don.tequ...@gmx.de wrote:
>
> Both server node and thin client are on the same computer, each running in
> a separate Docker container with network „host“. CPU load does not seem to
> be an issue and other server nodes on the same computer and from other
> computers connect to the cluster just fine without such connection issues.
>
> I‘ll check the server logs more thoroughly.
>
>
>
>
> On 15.06.22 at 21:14, Pavel Tupitsyn wrote:
>
> From: "Pavel Tupitsyn"  
> Date: 15. June 2022
> To: "user"  
> Cc:
> Subject: Re: ThinClient connection refused / disconnect
> Idle timeout is disabled by default.
>
> Regarding connection issues, either the server is overloaded and can't
> accept connections, or there are network issues.
> 1. Check if the network between client and server is reliable enough
> (measure speed, check packet loss)
> 2. Check server logs and resource utilization
>
> On Wed, Jun 15, 2022 at 5:47 PM  wrote:
>
>> Hi,
>>
>> I'm experiencing a connection issue when using Java ThinClient at the
>> customer site. It is using Ignite 2.13.
>>
>>  From time to time it happens that the ThinClient cannot connect at
>> startup and gets a "connection refused". After several tries it
>> connects, but frequently is "loses" connection again and all further
>> attempts to use the ignite client instance lead to connection refused
>> again. Then after some time it connects again.
>>
>> at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> at
>> java.base/sun.nio.ch
>> .SocketChannelImpl.finishConnect(SocketChannelImpl.java:774)
>> at java.base/sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120)
>> at
>>
>> org.apache.ignite.internal.client.thin.io.gridnioserver.GridNioClientConnectionMultiplexer.open(GridNioClientConnectionMultiplexer.java:125)
>> ... 25 common frames omitted
>> Suppressed: org.apache.ignite.client.ClientConnectionException:
>> Connection refused
>> ... 26 common frames omitted
>> Caused by: java.net.ConnectException: Connection refused
>> at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> at
>> java.base/sun.nio.ch
>> .SocketChannelImpl.finishConnect(SocketChannelImpl.java:774)
>> at java.base/sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120)
>> at
>>
>> org.apache.ignite.internal.client.thin.io.gridnioserver.GridNioClientConnectionMultiplexer.open(GridNioClientConnectionMultiplexer.java:125)
>>
>> I can see there is a idle time out but I'm not setting this. So I assume
>> the default value 0 means that no timeout should happen?
>>
>>
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#DFLT_IDLE_TIMEOUT
>>
>> Thanks!
>>
>>
>>

Re: Performance and large numbers of servers

[Pavel Tupitsyn]

Thank you for tracking this down! An additional map by name is a good idea
there.

> CONCURRENCY NOTE: these two maps need to update concurrently
All updates are triggered by discovery events, which are raised under
"synchronized (discoEvtMux)" in GridDiscoveryManager,
so it is safe to update two maps together.

>  is desc.name() unique?
Yes



On Wed, Jun 29, 2022 at 2:06 AM Arthur Naseef  wrote:

> The following is taking most of the time:
>
> @Nullable private ServiceInfo lookupInRegisteredServices(String name) {
> for (ServiceInfo desc : registeredServices.values()) {
> if (desc.name().equals(name))
> return desc;
> }
>
>
> return null;
> }
>
> After changing that to use a Map lookup:
>
>- 50,000 service startup in *8s* (down from around 70s)
>- 100,000 service startup in *14s* (right around 2x of the 50K timing)
>
>
> Here's the change I tested (note it's shortened) - it's not 100%, but fine
> for my test cast, I believe:
>
> private final ConcurrentMap registeredServicesByName
> = new ConcurrentHashMap<>();
>
>
> @Nullable private ServiceInfo lookupInRegisteredServices(String name) {
> return registeredServicesByName.get(name);
> }
>
> private void registerService(ServiceInfo desc) {
> desc.context(ctx);
>
>
> // (CONCURRENCY NOTE: these two maps need to update concurrently)
> registeredServices.put(desc.serviceId(), desc);
> registeredServicesByName.put(desc.name(), desc);
> }
>
>
> That's in IgniteServiceProcessor.java.
>
> Any thoughts?  I'll gladly clean this up and make  PR - would appreciate
> feedback to help address possible questions with this change (e.g. is
> desc.name() unique?).
>
> Art
>
>
> On Tue, Jun 28, 2022 at 12:27 PM Arthur Naseef 
> wrote:
>
>> Yes.  The "services" in our case will be schedules that periodically
>> perform fast operations.
>>
>> For example a service could be, "ping this device every  seconds".
>>
>> Art
>>
>> On Tue, Jun 28, 2022 at 12:20 PM Pavel Tupitsyn 
>> wrote:
>>
>>> > we do not plan to make cross-cluster calls into the services
>>>
>>> If you are making local calls, I think there is no point in using Ignite
>>> services.
>>> Can you describe the use case - what are you trying to achieve?
>>>
>>> On Tue, Jun 28, 2022 at 8:55 PM Arthur Naseef 
>>> wrote:
>>>
>>>> Hello - I'm getting started with Ignite and looking seriously at using
>>>> it for a specific use-case.
>>>>
>>>> Working on a Proof-Of-Concept (POC), I am finding a question related to
>>>> performance, and wondering if the solution, using Ignite Services, is a
>>>> good fit for the use-case.
>>>>
>>>> In my testing, I am getting the following timings:
>>>>
>>>>- Startup of 20,000 ignite services takes 30 seconds
>>>>- Startup of 50,000 ignite services takes 250 seconds
>>>>- The 2.5x increase from 20,000 to 50,000 yielded > 8x cost in
>>>>startup time (appears to be exponential growth)
>>>>
>>>> Watching the JVM during this time, I see the following:
>>>>
>>>>- Heap usage is not significant (do not see signs of GC)
>>>>- CPU usage is only slightly increased - on the order of 20% total
>>>>(system has 12 cores/24 threads)
>>>>- Network utilization is reasonable
>>>>- Futex system call (measured with "strace -r") appears to be
>>>>taking the most time by far.
>>>>
>>>> The use-case involves the following:
>>>>
>>>>- Startup of up-to hundreds-of-thousands of services at cluster
>>>>spin-up
>>>>- Frequent, small adjustments to the services running over time
>>>>- Need to rebalance when a new node joins the cluster, or an old
>>>>one leaves the cluster
>>>>- Once the services are deployed, we do not plan to make
>>>>cross-cluster calls into the services (i.e. we do *not* plan to use
>>>>ignite's services().serviceProxy() on these)
>>>>- Jobs don't look like a fit because these (1) are "long-running"
>>>>(actually periodically scheduled tasks) and (2) they need to 
>>>> redistribute
>>>>even after they start running
>>>>
>>>> This is starting to get long.  I have more details to share.  Here is
>>>> the repo with the code being used to test, and a link to a wiki page with
>>>> some of the details:
>>>>
>>>> https://github.com/opennms-forge/distributed-scheduling-poc/
>>>>
>>>>
>>>> https://github.com/opennms-forge/distributed-scheduling-poc/wiki/Ignite-Startup-Performance
>>>>
>>>>
>>>> Questions I have in mind:
>>>>
>>>>- Are services a good fit here?  We expect to reach upwards of
>>>>500,000 services in a cluster with multiple nodes.
>>>>- Any thoughts on tracking down the bottleneck and alleviating it?
>>>>(I have started taking timing measurements in the Ignite code)
>>>>
>>>> Stopping here - please ask questions and I'll gladly fill in details.
>>>> Any tips are welcome, including ideas for tracking down just where the
>>>> bottleneck exists.
>>>>
>>>> Art
>>>>
>>>>

Re: Performance and large numbers of servers

[Pavel Tupitsyn]

> we do not plan to make cross-cluster calls into the services

If you are making local calls, I think there is no point in using Ignite
services.
Can you describe the use case - what are you trying to achieve?

On Tue, Jun 28, 2022 at 8:55 PM Arthur Naseef  wrote:

> Hello - I'm getting started with Ignite and looking seriously at using it
> for a specific use-case.
>
> Working on a Proof-Of-Concept (POC), I am finding a question related to
> performance, and wondering if the solution, using Ignite Services, is a
> good fit for the use-case.
>
> In my testing, I am getting the following timings:
>
>- Startup of 20,000 ignite services takes 30 seconds
>- Startup of 50,000 ignite services takes 250 seconds
>- The 2.5x increase from 20,000 to 50,000 yielded > 8x cost in startup
>time (appears to be exponential growth)
>
> Watching the JVM during this time, I see the following:
>
>- Heap usage is not significant (do not see signs of GC)
>- CPU usage is only slightly increased - on the order of 20% total
>(system has 12 cores/24 threads)
>- Network utilization is reasonable
>- Futex system call (measured with "strace -r") appears to be taking
>the most time by far.
>
> The use-case involves the following:
>
>- Startup of up-to hundreds-of-thousands of services at cluster spin-up
>- Frequent, small adjustments to the services running over time
>- Need to rebalance when a new node joins the cluster, or an old one
>leaves the cluster
>- Once the services are deployed, we do not plan to make cross-cluster
>calls into the services (i.e. we do *not* plan to use ignite's
>services().serviceProxy() on these)
>- Jobs don't look like a fit because these (1) are "long-running"
>(actually periodically scheduled tasks) and (2) they need to redistribute
>even after they start running
>
> This is starting to get long.  I have more details to share.  Here is the
> repo with the code being used to test, and a link to a wiki page with some
> of the details:
>
> https://github.com/opennms-forge/distributed-scheduling-poc/
>
>
> https://github.com/opennms-forge/distributed-scheduling-poc/wiki/Ignite-Startup-Performance
>
>
> Questions I have in mind:
>
>- Are services a good fit here?  We expect to reach upwards of 500,000
>services in a cluster with multiple nodes.
>- Any thoughts on tracking down the bottleneck and alleviating it?  (I
>have started taking timing measurements in the Ignite code)
>
> Stopping here - please ask questions and I'll gladly fill in details.  Any
> tips are welcome, including ideas for tracking down just where the
> bottleneck exists.
>
> Art
>
>

Re: Re: ignite client can not reconnect to ignite Kubernetes cluster,after pod restart

[Pavel Tupitsyn]

The fix has been merged.
Thanks for the contribution, wkhapy123!

On Fri, Jun 24, 2022 at 8:57 AM Pavel Tupitsyn  wrote:

> Please request contributor permissions on d...@ignite.apache.org
>
> On Fri, Jun 24, 2022 at 6:53 AM wkhapy...@gmail.com 
> wrote:
>
>> Hi,
>>
>> I want to contribute to Apache ignite .
>>
>> Would you please give me the contributor permission?
>> below is my account
>> --
>> wkhapy...@gmail.com
>>
>>
>> *From:* wkhapy...@gmail.com
>> *Date:* 2022-06-23 22:34
>> *To:* Pavel Tupitsyn 
>> *Subject:* Re: ignite client can not reconnect to ignite Kubernetes
>> cluster,after pod restart
>> thank you!i will follow  wiki.
>> ---Original---
>> *From:* "Pavel Tupitsyn"
>> *Date:* Thu, Jun 23, 2022 22:25 PM
>> *To:* "wkhapy123";"user";
>> *Subject:* Re: ignite client can not reconnect to ignite Kubernetes
>> cluster,after pod restart
>>
>> > I want to fix this bug . I think it is good opportunity to study ignite.
>>
>> Great! Please go ahead.
>> Make sure to check our wiki, register as a contributor, and assign the
>> ticket to yourself.
>>
>>
>> https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=177047163#content/view/177047163
>>
>>
>> On Thu, Jun 23, 2022, 17:02 wkhapy...@gmail.com 
>> wrote:
>>
>>>
>>> I want to fix this bug . I think it is good opportunity to study ignite.
>>> ---Original---
>>> *From:* "wkhapy...@gmail.com"
>>> *Date:* Thu, Jun 23, 2022 21:41 PM
>>> *To:* "Pavel Tupitsyn";
>>> *Subject:* Re: ignite client can not reconnect to ignite Kubernetes
>>> cluster,after pod restart
>>>
>>>  I am interested in ignite
>>>
>>> ---Original---
>>> *From:* "wkhapy...@gmail.com"
>>> *Date:* Thu, Jun 23, 2022 21:37 PM
>>> *To:* "Pavel Tupitsyn";
>>> *Subject:* Re: ignite client can not reconnect to ignite Kubernetes
>>> cluster,after pod restart
>>>
>>> can I repair it
>>>
>>> ---Original---
>>> *From:* "Pavel Tupitsyn"
>>> *Date:* Thu, Jun 23, 2022 20:17 PM
>>> *To:* "user";
>>> *Subject:* Re: Re: ignite client can not reconnect to ignite Kubernetes
>>> cluster,after pod restart
>>>
>>> It is a bug - addresses are not reloaded from AddressFinder on
>>> connection loss, so we still try old pod address and fail:
>>> https://issues.apache.org/jira/browse/IGNITE-17217
>>>
>>> Thanks for reporting this.
>>>
>>> On Thu, Jun 23, 2022 at 3:00 PM wkhapy...@gmail.com 
>>> wrote:
>>>
>>>> as you can see,address is 104
>>>> but addressFinder.getAddress new  ip is 87,and retrylimit is 5 (i set)
>>>> --
>>>> wkhapy...@gmail.com
>>>>
>>>>
>>>> *From:* wkhapy...@gmail.com
>>>> *Date:* 2022-06-23 16:02
>>>> *To:* Maksim Timonin 
>>>> *Subject:* Re: Re: ignite client can not reconnect to ignite
>>>> Kubernetes cluster,after pod restart
>>>> sorry i did not add,i will add and retry.
>>>>
>>>> --
>>>> wkhapy...@gmail.com
>>>>
>>>>
>>>> *From:* Maksim Timonin 
>>>> *Date:* 2022-06-23 15:55
>>>> *To:* user ; wkhapy123 
>>>> *Subject:* Re: Re: ignite client can not reconnect to ignite
>>>> Kubernetes cluster,after pod restart
>>>> Did you set any value to `ClientConfiguration#setRetryLimit`? If you
>>>> check it with a single pod then any value greater than 1 should help (2 or
>>>> 3).
>>>>
>>>> Could you please confirm that you have the failure even with this
>>>> setting?
>>>>
>>>> On Thu, Jun 23, 2022 at 10:49 AM wkhapy...@gmail.com <
>>>> wkhapy...@gmail.com> wrote:
>>>>
>>>>> Hi:
>>>>> i find it still get Connection timed out exeception.
>>>>> and i add cfg.setPartitionAwarenessEnabled(true).
>>>>> and its errorMsg
>>>>> org.apache.ignite.client.ClientConnectionException: Connection timed
>>>>> out
>>>>> at
>>>>> org.apache.ignite.internal.client.thin.io.gridnioserver.GridNioClientConnectionMultiplexer.open(GridNioCli

Re: Sql Query to read data for a partition of a table

[Pavel Tupitsyn]

> How can i do that using JDBC?

Why would you want to do that, when we are already inside the Ignite
Compute task?
Ignite SQL API call will be a lot faster.

On Sun, Jun 26, 2022 at 8:19 AM Sachin janani 
wrote:

> Thanks for a quick response. How can i do that using JDBC?
>
>
> Thanks and Regards,
> Sachin Janani
>
> On Sat, Jun 25, 2022 at 1:46 PM Pavel Tupitsyn 
> wrote:
>
>> SqlQuery#partitions does just that:
>>
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/SqlQuery.html#setPartitions-int...-
>>
>> On Sat, Jun 25, 2022 at 10:36 AM Sachin janani <
>> sachin.janani...@gmail.com> wrote:
>>
>>> Hi Team,
>>> Is it possible in ignite to query data for a specific partition in SQL
>>> query. I have a requirement where i have to do colocated task by partition
>>> using SQL query so just want to know is there any way in SQL query to
>>> achieve this.
>>>
>>>
>>> Thanks and Regards,
>>> *Sachin Janani*
>>>
>>>
>>
>
> --
> *Sachin Janani*
>
>

Re: Sql Query to read data for a partition of a table

[Pavel Tupitsyn]

SqlQuery#partitions does just that:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/SqlQuery.html#setPartitions-int...-

On Sat, Jun 25, 2022 at 10:36 AM Sachin janani 
wrote:

> Hi Team,
> Is it possible in ignite to query data for a specific partition in SQL
> query. I have a requirement where i have to do colocated task by partition
> using SQL query so just want to know is there any way in SQL query to
> achieve this.
>
>
> Thanks and Regards,
> *Sachin Janani*
>
>