SQL: setting key field name for types registered via CacheConfiguration.setIndexedTypes

[Denis Magda]

Guys,

Let’s suppose there is a cache that stores entries like that:

* key - int
* value - custom object which fields are annotated by @QuerySqlField

Before 2.0 if I wanted to refer to the key in an SQL query I had to use “_key” 
keyword. In 2.0 this is no longer the requirement. Any name can be used in the 
condition it’s set via QueryEntity.setKeyFieldName(keyName).

This concept works perfectly fine if both key's and value's types are 
configured using QueryEntities directly. But what can I do if the value is 
annotated with @QuerySqlField as in the example above and passed to SQL engine 
via CacheConfiguration.setIndexedTypes(…) method? How can I set 
QueryEntity.setKeyFieldName()?

Looks like the only workable way is this one:
- register key and value types via CacheConfiguration.setIndexedTypes(…)
- find the QueryEntity corresponding to the just registered types in a 
collection of CacheConfiguration.getQueryEntities()
- call QueryEntity.setKeyFieldName(keyName) for the entity of interest.

Basically, this is a workaround but not a user friendly approach. Is there any 
other way to achieve this?

In the meanwhile, what if we add QueryEntity(Class keyType, Class valueType) 
that will do totally the same things that CacheConfiguration.setIndexedTypes(…) 
with the only difference that right after that the user will be able to call 
QueryEntity.setKeyFieldName(keyName)?

—
Denis

Re: 10X decrease in performance with Ignite 2.0.0

[Yakov Zhdanov]

Cross-posting to devlist.

--Yakov

Re: Add ability to enable and disable rebalancing per-node

[npordash]

Hi Alexandr,

Yes, I'm using a StreamReceiver since I'm mostly using the cache as a
router. The implementation is more-or-less like the following:



The StreamPipeline is managed by the data node which contains the stages
that make up the pipeline and what service is responsible for each stage
(the service exposes a handle method which is used for side-effects or
modifying entries as they go through the pipeline). When a service is
started or cancelled it will poke the pipeline so it can add or remove that
stage from the pipeline. The pipeline is also aware if all required services
have started or not.

I didn't mention it initially here, but the classes for these services are
_not_ available to Ignite when the node starts. Instead, the node is
actually starting a generic service which fetches required jar files from
IGFS and then starts the real service using a URLClassLoader (once that is
done the pipeline poke I mentioned above occurs). I'm doing this to get
around existing service grid limitations with regards to dynamic code
deployment.

Because of the above, I wouldn't assume service startup would be "fast" when
compared to map partitions being rebalanced to the new node. 

Can you please elaborate on the following?

"Service should start quickly after node has joined the topology, and will
process all the data has been collected by local partitions the moments
before."

It sounds like you're implying that data is being buffered somewhere, but
not quite following.

Thanks!
-Nick 



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Add-ability-to-enable-and-disable-rebalancing-per-node-tp17494p17600.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Add ability to enable and disable rebalancing per-node

[Yakov Zhdanov]

As far as I can see from ticket title reporter wants to disable rebalancing
on per-node basis. So, some nodes would not load partitions from other
nodes while others should load normally. Disabling partition map exchange
will stop rebalancing all over the grid (however, I find this pretty useful
feature in some cases and it probably deserves a separate ticket).

I would ask Alexey Goncharuk to elaborate a little bit and add some
background for the ticket he filed.

--Yakov

[GitHub] ignite pull request #1930: logging of memory configuration improvements on n...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/1930

logging of memory configuration improvements on node start 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5202

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1930.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1930


commit 5565da03a4636cbd82af656b2cf6ce6e54e926a3
Author: Sergey Chugunov 
Date:   2017-05-11T14:27:02Z

IGNITE-5202 logging memory configuration improved




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5206) Test mesos user and mesos role parameters on real mesos cluster.

[Vadim Opolski (JIRA)]

Vadim Opolski created IGNITE-5206:
-

 Summary: Test mesos user and mesos role parameters on real mesos 
cluster.
 Key: IGNITE-5206
 URL: https://issues.apache.org/jira/browse/IGNITE-5206
 Project: Ignite
  Issue Type: Task
Reporter: Vadim Opolski


In current implementation Ignite Mesos Framework connects to MESOS cluster via 
system env properties (user and role). 
Need to test this locally, on real mesos cluster.

We need to understand which exception user will get if user name and/or role 
incorrect.

Documentation:
https://apacheignite.readme.io/docs/mesos-deployment
http://mesos.apache.org/gettingstarted/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1929: Removed condition, simplified logic

[yzhdanov]

GitHub user yzhdanov opened a pull request:

https://github.com/apache/ignite/pull/1929

Removed condition, simplified logic



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yzhdanov/ignite atomic-sequence-optimization

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1929.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1929


commit bf696acb7932f40590f905cb0b232434a1f5b352
Author: Yakov Zhdanov 
Date:   2017-05-11T14:23:56Z

Removed condition, simplified logic




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1928: Ignite 5193

[iveselovskiy]

GitHub user iveselovskiy opened a pull request:

https://github.com/apache/ignite/pull/1928

Ignite 5193



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5193

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1928.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1928


commit bfb00b6e61f9709718c30971997aeb0ac79e86b4
Author: Alexandr Kuramshin 
Date:   2016-11-18T20:12:28Z

IgniteTcpCommunicationBigClusterTest added

commit 02dd92e605b9b53f5a16c7ec5f8e7b5698b15ba4
Author: Alexandr Kuramshin 
Date:   2016-11-18T21:55:37Z

IgniteTcpCommunicationBigClusterTest update

commit 6acf193a3d356d1bad4c02a53ac76833ed1008d0
Author: Alexandr Kuramshin 
Date:   2016-11-19T09:55:45Z

Have got TcpCommunicationSpi error

commit 4fd39653d24f62f19f70b4dffba8497185cc46fb
Author: Alexandr Kuramshin 
Date:   2016-11-19T16:39:10Z

Some discovery have been done

commit c2c181922c7c24ea457577e32d2af897c8bec87f
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:11:28Z

Prove that problem is not in the onFirstMessage hang

commit f8076edba097f6077229b2090ee3ff1a3369878c
Author: Alexandr Kuramshin 
Date:   2016-11-19T20:26:37Z

Revert: Prove that problem is not in the onFirstMessage hang

commit 6e1f2dfc2acb3dbb8f24aa51ed67b2ee447b4585
Author: Alexandr Kuramshin 
Date:   2016-11-21T08:55:09Z

Revert: pushing unnecessary changes to the master

commit ed794ca815f6bb1471af15779279d287576b39cc
Author: Alexandr Kuramshin 
Date:   2016-11-21T09:08:00Z

Revert: pushing unnecessary changes to the master

commit 3a57d63668bed239e21eca588134272783472e97
Author: iveselovskiy 
Date:   2016-12-09T14:13:37Z

Merge branch 'master' of https://github.com/apache/ignite

commit 166ad21d560c41c42a6de81cf5910537baa3ac92
Author: iveselovskiy 
Date:   2016-12-12T18:46:44Z

Merge branch 'master' of https://github.com/apache/ignite

commit 1eaba3e91bfe54c2bf6f761ac3f238a3b181a8e0
Author: iveselovskiy 
Date:   2017-03-27T18:41:07Z

Merge branch 'master' of https://github.com/apache/ignite

commit cef7775c9dcfc32b11a4cc4678b3a69a1e1b17da
Author: iveselovskiy 
Date:   2017-03-30T13:05:23Z

Merge branch 'master' of https://github.com/apache/ignite

commit 84e26d6fa2f6aec742fafb4d8145fbc9f697da70
Author: iveselovskiy 
Date:   2017-03-31T09:35:07Z

Merge branch 'master' of https://github.com/apache/ignite

commit be360213d74c9f005e4e3a1beb661323262bb7fb
Author: iveselovskiy 
Date:   2017-03-31T12:18:43Z

Merge branch 'master' of https://github.com/apache/ignite

commit ec3de454894f4bba4497b6c0ca0564a730525244
Author: iveselovskiy 
Date:   2017-04-17T15:22:00Z

Merge branch 'master' of https://github.com/apache/ignite

commit 88de0693b76c80fdc6533a799e0475fd53a4b0fc
Author: iveselovskiy 
Date:   2017-05-02T13:20:47Z

Merge branch 'master' of https://github.com/apache/ignite

commit 04d8f4ef69ea92b96642ecc3b797703858574c69
Author: iveselovskiy 
Date:   2017-05-11T13:23:05Z

Merge branch 'master' of https://github.com/apache/ignite

commit a50794a79881c399da294c2235080a419b974093
Author: iveselovskiy 
Date:   2017-05-11T13:36:30Z

IGNITE-5193




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5204) The Unicode character in the value of a field which are included in an un-unique index will cause "stack overhead" exception

[Chris Wang (JIRA)]

Chris Wang created IGNITE-5204:
--

 Summary: The Unicode character in the value of a field which are 
included in an un-unique index will cause "stack overhead" exception
 Key: IGNITE-5204
 URL: https://issues.apache.org/jira/browse/IGNITE-5204
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.0
 Environment: windows server 2012, JDK 1.8, X64
Reporter: Chris Wang
Priority: Critical


When put  "草DX009090" as the value of BillId, which is a field of entity Bill. 
If I define a index includes the BillId, and execute the query like "select * 
from Bill where BillId=’草DX009090‘ in the H2 debug console, there throws an 
exception by the H2 with a code 5000. 
another scenario is, I have two entities, "Bill" and "Detail", both have field 
"BillId". If either of them have value like "草DX009090" and execute the query 
like "select bill.* from bill left join detail on bill.billid=detail.billid", 
the whole ignite cache node will halt ( suppose there should be an stack 
overhead exception, dead loop).
==
I think the issue should relate to hash computing on the unicode character.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5203) JDBC driver should support BLOB fields

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-5203:
---

 Summary: JDBC driver should support BLOB fields
 Key: IGNITE-5203
 URL: https://issues.apache.org/jira/browse/IGNITE-5203
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Andrey Gura


JDBD driver doesn't support BLOB fields. Need to implement BLOBs, at least 
simple byte array based implementation.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5202) Default mem policy name is not changed after setting defaultMemoryPolicyName property

[Ksenia Rybakova (JIRA)]

Ksenia Rybakova created IGNITE-5202:
---

 Summary: Default mem policy name is not changed after setting 
defaultMemoryPolicyName property
 Key: IGNITE-5202
 URL: https://issues.apache.org/jira/browse/IGNITE-5202
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Ksenia Rybakova
 Fix For: 2.1


Default mem policy name is not changed after setting defaultMemoryPolicyName 
property.

Config:
{noformat}

http://www.springframework.org/schema/beans;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="
http://www.springframework.org/schema/beans 
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd;>



  

...
  


  


  




{noformat}

{noformat}

http://www.springframework.org/schema/beans;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="
http://www.springframework.org/schema/beans 
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd;>

...



   ...
   
   
   
   
   

   
   ...
   
   
   
 
 
   

   




   ...

{noformat}

According to this config default memory policy should be "default2", but it 
remains "default" (while SystemCacheInitialSize was changed successfully to 
50Mb as it's set in config). 

{noformat}
[12:48:12,226][INFO ][main][IgniteKernal] System cache's MemoryPolicy size is 
configured to 50 MB. Use MemoryConfiguration.systemCacheMemorySize property to 
change the setting.
[12:48:12,226][INFO ][main][IgniteKernal] Configured caches [in 'default' 
memoryPolicy: ['ignite-sys-cache', 'ignite-atomics-sys-cache', 'atomic-part', 
'atomic-rplc', 'atomic-part-fat-values', 'atomic-part-onheap-evict-fifo', 
'atomic-rplc-onheap-evict-lru', 'tx-part', 'tx-rplc', 'tx-part-fat-values', 
'compute', 'tx-part-onheap-evict-lru', 'tx-rplc-onheap-evict-fifo', 
'atomic-index', 'query', 'orgCache']]
{noformat}

Also it would be nice to have default policy name instead of "null"  in logs 
when printing caches info:
{noformat}
<12:48:12> Cache configured with the following parameters: 
CacheConfiguration [name=atomic-part, memPlcName=null, 
storeConcurrentLoadAllThreshold=5, ...
{noformat}
{noformat}
[12:48:13,239][INFO ][main][GridCacheProcessor] Started cache 
[name=atomic-part, memoryPolicyName=null, mode=PARTITIONED]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Ignite 2.1 scope

[Pavel Tupitsyn]

As for .NET, I would propose to concentrate on peer deployment (IGNITE-2492)
and related stuff, like IGNITE-1894 .NET: Delegate support in the API via
extension methods.

SQL Dependency does not look important to me, we can reschedule it for
later versions.

On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan 
wrote:

> Vyacheslav, I think it is worth the research, but you should always keep
> data querying and indexing in mind. For example, I don't see how by-page
> compression will solve it.
>
> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur 
> wrote:
>
> > Dmitriy,
> >
> > I'm researching a best way for this future.
> >
> > At the moment I found only one way (querying and indexing compatible),
> this
> > is per-objects-field compression.
> >
> > But there is a good proffit only for long strings or fields with large
> > objects.
> >
> > Maybe it makes sense just to introduce compression for string fileds.
> >
> > I'm researching the new page-memory architecture as applied to by-page
> > compression.
> >
> > 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan :
> >
> > > On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <
> > daradu...@gmail.com>
> > > wrote:
> > >
> > > > Denis,
> > > >
> > > > The described roadmap looks great!
> > > >
> > > > Additional, I vote for introducing an ability (OOTB) to store objects
> > in
> > > a
> > > > cache in a compressed form.
> > > > This will allow to store more data at the cost of incriasing of CPU
> > > > utilization.
> > > >
> > >
> > > One of the problems with compression is indexing and querying. How do
> we
> > > index the data if it is compressed?
> > >
> > >
> > > >
> > > >
> > > > 2017-05-11 4:23 GMT+03:00 Denis Magda :
> > > >
> > > > > Igniters,
> > > > >
> > > > > Let me start a discussion around the scope for 2.1 release.
> > > > >
> > > > > In my vision the main direction of our ongoing efforts should be
> > > > > implementing in life a use case of Ignite as a transactional
> > > distributed
> > > > > SQL database and HTAP platform. The current use cases (database
> > cache,
> > > > data
> > > > > grid, micro services platform, etc.) will be supported as usual, no
> > > > changes
> > > > > on that frontier.
> > > > >
> > > > > Keeping this in mind, the roadmap needs to include essential SQL
> > > related
> > > > > features as well as disk based capabilities, MVCC support, advanced
> > DDL
> > > > > implementation and so on so forth. This is for Ignite as a SQL
> > > database.
> > > > >
> > > > > Next, Machine Learning will be a great addition to Ignite as an
> HTAP
> > > > > platform offering. This is why we should keep investing our time
> and
> > > > > resources in that recently released component.
> > > > >
> > > > > Having this said, I see the scope for 2.1 release this way:
> > > > >
> > > > > 1. Distributed Persistent Store - if the donation is accepted by
> ASF.
> > > The
> > > > > decision is to be done in separate discussion. W/o the store
> > > > >Ignite can only be used as In-Memory SQL database.
> > > > >
> > > > > 2. SQL Grid:
> > > > > - CREATE & DROP table commands: https://issues.apache.org/
> > > > > jira/browse/IGNITE-4651
> > > > > - Renewed JDBC driver: https://issues.apache.org/
> > > > > jira/browse/IGNITE-4922
> > > > > - Collocation based routing of SQL queries:
> > > > https://issues.apache.org/
> > > > > jira/browse/IGNITE-4510,
> > > > >https://issues.apache.org/jira/browse/IGNITE-4509
> > > > > -
> > > > >
> > > > > 3. .NET:
> > > > > - Peer-class loading: https://issues.apache.org/
> > > > > jira/browse/IGNITE-2492
> > > > > - SQLDependency: https://issues.apache.org/
> > jira/browse/IGNITE-2657
> > > > >
> > > > > 4. C++:
> > > > > - Compute Grid: https://issues.apache.org/
> > jira/browse/IGNITE-3574
> > > > >
> > > > > 5. ML Grid:
> > > > > - Linear regression algorithms: https://issues.apache.org/
> > > > > jira/browse/IGNITE-5012
> > > > > - K-means clustering: https://issues.apache.org/
> > > > > jira/browse/IGNITE-5113
> > > > >
> > > > >
> > > > > Please join the thread and share your thoughts, ideas and concerns.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Vyacheslav
> > > >
> > >
> >
> >
> >
> > --
> > Best Regards, Vyacheslav
> >
>

[jira] [Created] (IGNITE-5201) .NET: SegmentationResolver

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5201:
--

 Summary: .NET: SegmentationResolver
 Key: IGNITE-5201
 URL: https://issues.apache.org/jira/browse/IGNITE-5201
 Project: Ignite
  Issue Type: New Feature
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.2


Support {{SegmentationResolver}} natively in .NET



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Ignite 2.1 scope

[Dmitriy Setrakyan]

Vyacheslav, I think it is worth the research, but you should always keep
data querying and indexing in mind. For example, I don't see how by-page
compression will solve it.

On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur 
wrote:

> Dmitriy,
>
> I'm researching a best way for this future.
>
> At the moment I found only one way (querying and indexing compatible), this
> is per-objects-field compression.
>
> But there is a good proffit only for long strings or fields with large
> objects.
>
> Maybe it makes sense just to introduce compression for string fileds.
>
> I'm researching the new page-memory architecture as applied to by-page
> compression.
>
> 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan :
>
> > On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur <
> daradu...@gmail.com>
> > wrote:
> >
> > > Denis,
> > >
> > > The described roadmap looks great!
> > >
> > > Additional, I vote for introducing an ability (OOTB) to store objects
> in
> > a
> > > cache in a compressed form.
> > > This will allow to store more data at the cost of incriasing of CPU
> > > utilization.
> > >
> >
> > One of the problems with compression is indexing and querying. How do we
> > index the data if it is compressed?
> >
> >
> > >
> > >
> > > 2017-05-11 4:23 GMT+03:00 Denis Magda :
> > >
> > > > Igniters,
> > > >
> > > > Let me start a discussion around the scope for 2.1 release.
> > > >
> > > > In my vision the main direction of our ongoing efforts should be
> > > > implementing in life a use case of Ignite as a transactional
> > distributed
> > > > SQL database and HTAP platform. The current use cases (database
> cache,
> > > data
> > > > grid, micro services platform, etc.) will be supported as usual, no
> > > changes
> > > > on that frontier.
> > > >
> > > > Keeping this in mind, the roadmap needs to include essential SQL
> > related
> > > > features as well as disk based capabilities, MVCC support, advanced
> DDL
> > > > implementation and so on so forth. This is for Ignite as a SQL
> > database.
> > > >
> > > > Next, Machine Learning will be a great addition to Ignite as an HTAP
> > > > platform offering. This is why we should keep investing our time and
> > > > resources in that recently released component.
> > > >
> > > > Having this said, I see the scope for 2.1 release this way:
> > > >
> > > > 1. Distributed Persistent Store - if the donation is accepted by ASF.
> > The
> > > > decision is to be done in separate discussion. W/o the store
> > > >Ignite can only be used as In-Memory SQL database.
> > > >
> > > > 2. SQL Grid:
> > > > - CREATE & DROP table commands: https://issues.apache.org/
> > > > jira/browse/IGNITE-4651
> > > > - Renewed JDBC driver: https://issues.apache.org/
> > > > jira/browse/IGNITE-4922
> > > > - Collocation based routing of SQL queries:
> > > https://issues.apache.org/
> > > > jira/browse/IGNITE-4510,
> > > >https://issues.apache.org/jira/browse/IGNITE-4509
> > > > -
> > > >
> > > > 3. .NET:
> > > > - Peer-class loading: https://issues.apache.org/
> > > > jira/browse/IGNITE-2492
> > > > - SQLDependency: https://issues.apache.org/
> jira/browse/IGNITE-2657
> > > >
> > > > 4. C++:
> > > > - Compute Grid: https://issues.apache.org/
> jira/browse/IGNITE-3574
> > > >
> > > > 5. ML Grid:
> > > > - Linear regression algorithms: https://issues.apache.org/
> > > > jira/browse/IGNITE-5012
> > > > - K-means clustering: https://issues.apache.org/
> > > > jira/browse/IGNITE-5113
> > > >
> > > >
> > > > Please join the thread and share your thoughts, ideas and concerns.
> > > >
> > > > —
> > > > Denis
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav
> > >
> >
>
>
>
> --
> Best Regards, Vyacheslav
>

Re: [jira] [Commented] (IGNITE-4052) Add ability to set up users for MESOS

[Вадим Опольский]

Hello guys!

I want to update https://apacheignite.readme.io/docs/mesos-deployment with
new text according with
https://cwiki.apache.org/confluence/display/IGNITE/Documentation
But have a problem with authentication, registered login
vaopols...@gmail.com.
Help me, please.

New text:

3.Copy the following application definition (in JSON format) and save to
marathon.json file. Update any parameters which would like to change.



*A role name must be a valid directory name, so it cannot:  • Be an empty
string  • Be . or ..  • Start with -  • Contain any slash, backspace, or
whitespace character*
 If doesn't set restriction on cluster then the framework will try to
occupy all resources in Mesos cluster. See Configuration section below.

JSON
{
  "id": "ignition",
  "instances": 1,
  "cpus": 2,
  "mem": 2048,
  "ports": [0],
  "uris": [
"http://host/ignite-mesos--jar-with-dependencies.jar"
  ],
  "env": {
"IGNITE_NODE_COUNT": "4",
"MESOS_MASTER_URL": "zk://localhost:2181/mesos",
"IGNITE_RUN_CPU_PER_NODE": "2",
"IGNITE_MEMORY_PER_NODE": "2048",
"IGNITE_VERSION": "1.0.5",


*"MESOS_USER" : "userA","MESOS_ROLE" :  "role1"*  },
  "cmd": "java -jar ignite-mesos--jar-with-dependencies.jar"
}

Vadim


2017-04-27 19:29 GMT+03:00 Nikolay Tikhonov (JIRA) :

>
> [ https://issues.apache.org/jira/browse/IGNITE-4052?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel=15986959#comment-15986959 ]
>
> Nikolay Tikhonov commented on IGNITE-4052:
> --
>
> [~javaller],
> Thank you for your contribution. I've fixed some minors issue and pushed
> your changes into {{ignite-4052}} branch. Please look at the changes.
>
> >I dont work with Mesos and think that anyone who has experience should
> make it. OK?
> I think that it good time to try it. ;) It looks strange when developer
> don't run own code. [2] Also would be great to update docs [1]. Use for it
> {{suggest edits}} button.
>
> 1. https://apacheignite.readme.io/docs/mesos-deployment
> 2. http://mesos.apache.org/gettingstarted/
>
> > Add ability to set up users for MESOS
> > -
> >
> > Key: IGNITE-4052
> > URL: https://issues.apache.org/jira/browse/IGNITE-4052
> > Project: Ignite
> >  Issue Type: Improvement
> >  Components: general
> >Affects Versions: 1.7
> >Reporter: Nikolay Tikhonov
> >Assignee: Vadim Opolski
> >Priority: Trivial
> >
> > In current implementation Ignite Mesos Framework connects to MESOS
> cluster via current user. Need to add ability to configure this parameters
> via system env properties. Also need to add properties for mesos role.
> > See org/apache/ignite/mesos/IgniteFramework.java:537
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)
>

Re: Apache Ignite 2.1 scope

[Vyacheslav Daradur]

Dmitriy,

I'm researching a best way for this future.

At the moment I found only one way (querying and indexing compatible), this
is per-objects-field compression.

But there is a good proffit only for long strings or fields with large
objects.

Maybe it makes sense just to introduce compression for string fileds.

I'm researching the new page-memory architecture as applied to by-page
compression.

2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan :

> On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur 
> wrote:
>
> > Denis,
> >
> > The described roadmap looks great!
> >
> > Additional, I vote for introducing an ability (OOTB) to store objects in
> a
> > cache in a compressed form.
> > This will allow to store more data at the cost of incriasing of CPU
> > utilization.
> >
>
> One of the problems with compression is indexing and querying. How do we
> index the data if it is compressed?
>
>
> >
> >
> > 2017-05-11 4:23 GMT+03:00 Denis Magda :
> >
> > > Igniters,
> > >
> > > Let me start a discussion around the scope for 2.1 release.
> > >
> > > In my vision the main direction of our ongoing efforts should be
> > > implementing in life a use case of Ignite as a transactional
> distributed
> > > SQL database and HTAP platform. The current use cases (database cache,
> > data
> > > grid, micro services platform, etc.) will be supported as usual, no
> > changes
> > > on that frontier.
> > >
> > > Keeping this in mind, the roadmap needs to include essential SQL
> related
> > > features as well as disk based capabilities, MVCC support, advanced DDL
> > > implementation and so on so forth. This is for Ignite as a SQL
> database.
> > >
> > > Next, Machine Learning will be a great addition to Ignite as an HTAP
> > > platform offering. This is why we should keep investing our time and
> > > resources in that recently released component.
> > >
> > > Having this said, I see the scope for 2.1 release this way:
> > >
> > > 1. Distributed Persistent Store - if the donation is accepted by ASF.
> The
> > > decision is to be done in separate discussion. W/o the store
> > >Ignite can only be used as In-Memory SQL database.
> > >
> > > 2. SQL Grid:
> > > - CREATE & DROP table commands: https://issues.apache.org/
> > > jira/browse/IGNITE-4651
> > > - Renewed JDBC driver: https://issues.apache.org/
> > > jira/browse/IGNITE-4922
> > > - Collocation based routing of SQL queries:
> > https://issues.apache.org/
> > > jira/browse/IGNITE-4510,
> > >https://issues.apache.org/jira/browse/IGNITE-4509
> > > -
> > >
> > > 3. .NET:
> > > - Peer-class loading: https://issues.apache.org/
> > > jira/browse/IGNITE-2492
> > > - SQLDependency: https://issues.apache.org/jira/browse/IGNITE-2657
> > >
> > > 4. C++:
> > > - Compute Grid: https://issues.apache.org/jira/browse/IGNITE-3574
> > >
> > > 5. ML Grid:
> > > - Linear regression algorithms: https://issues.apache.org/
> > > jira/browse/IGNITE-5012
> > > - K-means clustering: https://issues.apache.org/
> > > jira/browse/IGNITE-5113
> > >
> > >
> > > Please join the thread and share your thoughts, ideas and concerns.
> > >
> > > —
> > > Denis
> > >
> > >
> > >
> >
> >
> >
> >
> > --
> > Best Regards, Vyacheslav
> >
>



-- 
Best Regards, Vyacheslav

Re: Apache Ignite 2.1 scope

[Dmitriy Setrakyan]

On Thu, May 11, 2017 at 12:44 AM, Vyacheslav Daradur 
wrote:

> Denis,
>
> The described roadmap looks great!
>
> Additional, I vote for introducing an ability (OOTB) to store objects in a
> cache in a compressed form.
> This will allow to store more data at the cost of incriasing of CPU
> utilization.
>

One of the problems with compression is indexing and querying. How do we
index the data if it is compressed?


>
>
> 2017-05-11 4:23 GMT+03:00 Denis Magda :
>
> > Igniters,
> >
> > Let me start a discussion around the scope for 2.1 release.
> >
> > In my vision the main direction of our ongoing efforts should be
> > implementing in life a use case of Ignite as a transactional distributed
> > SQL database and HTAP platform. The current use cases (database cache,
> data
> > grid, micro services platform, etc.) will be supported as usual, no
> changes
> > on that frontier.
> >
> > Keeping this in mind, the roadmap needs to include essential SQL related
> > features as well as disk based capabilities, MVCC support, advanced DDL
> > implementation and so on so forth. This is for Ignite as a SQL database.
> >
> > Next, Machine Learning will be a great addition to Ignite as an HTAP
> > platform offering. This is why we should keep investing our time and
> > resources in that recently released component.
> >
> > Having this said, I see the scope for 2.1 release this way:
> >
> > 1. Distributed Persistent Store - if the donation is accepted by ASF. The
> > decision is to be done in separate discussion. W/o the store
> >Ignite can only be used as In-Memory SQL database.
> >
> > 2. SQL Grid:
> > - CREATE & DROP table commands: https://issues.apache.org/
> > jira/browse/IGNITE-4651
> > - Renewed JDBC driver: https://issues.apache.org/
> > jira/browse/IGNITE-4922
> > - Collocation based routing of SQL queries:
> https://issues.apache.org/
> > jira/browse/IGNITE-4510,
> >https://issues.apache.org/jira/browse/IGNITE-4509
> > -
> >
> > 3. .NET:
> > - Peer-class loading: https://issues.apache.org/
> > jira/browse/IGNITE-2492
> > - SQLDependency: https://issues.apache.org/jira/browse/IGNITE-2657
> >
> > 4. C++:
> > - Compute Grid: https://issues.apache.org/jira/browse/IGNITE-3574
> >
> > 5. ML Grid:
> > - Linear regression algorithms: https://issues.apache.org/
> > jira/browse/IGNITE-5012
> > - K-means clustering: https://issues.apache.org/
> > jira/browse/IGNITE-5113
> >
> >
> > Please join the thread and share your thoughts, ideas and concerns.
> >
> > —
> > Denis
> >
> >
> >
>
>
>
>
> --
> Best Regards, Vyacheslav
>

[GitHub] ignite pull request #1926: IGNITE-4597: CPP: Add mechanism to reset argument...

[vinx13]

GitHub user vinx13 opened a pull request:

https://github.com/apache/ignite/pull/1926

IGNITE-4597: CPP: Add mechanism to reset arguments for sql queries.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vinx13/ignite ignite-4597

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1926.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1926


commit 262a06abcb0d7b2588072985cf5f94f70e6c57da
Author: Wuwei Lin 
Date:   2017-05-10T14:12:03Z

Add methods to reset arguments in sql query




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

IGNITE-1094 Ignite.createCache(CacheConfiguration) hangs if some exception occurs during cache initialization

[ALEKSEY KUZNETSOV]

Igntrs! Plz, review my changes
https://issues.apache.org/jira/browse/IGNITE-1094

*https://github.com/apache/ignite/pull/1904
*
-- 

*Best Regards,*

*Kuznetsov Aleksey*
-- 

*Best Regards,*

*Kuznetsov Aleksey*

[jira] [Created] (IGNITE-5200) Web Console: Watch in ignite_modules works incorrect after migration on webpack2

[Andrey Novikov (JIRA)]

Andrey Novikov created IGNITE-5200:
--

 Summary: Web Console: Watch in ignite_modules works incorrect 
after migration on webpack2
 Key: IGNITE-5200
 URL: https://issues.apache.org/jira/browse/IGNITE-5200
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Affects Versions: 2.0
Reporter: Andrey Novikov
 Fix For: 2.1






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Ignite 2.1 scope

[Vyacheslav Daradur]

Denis,

The described roadmap looks great!

Additional, I vote for introducing an ability (OOTB) to store objects in a
cache in a compressed form.
This will allow to store more data at the cost of incriasing of CPU
utilization.


2017-05-11 4:23 GMT+03:00 Denis Magda :

> Igniters,
>
> Let me start a discussion around the scope for 2.1 release.
>
> In my vision the main direction of our ongoing efforts should be
> implementing in life a use case of Ignite as a transactional distributed
> SQL database and HTAP platform. The current use cases (database cache, data
> grid, micro services platform, etc.) will be supported as usual, no changes
> on that frontier.
>
> Keeping this in mind, the roadmap needs to include essential SQL related
> features as well as disk based capabilities, MVCC support, advanced DDL
> implementation and so on so forth. This is for Ignite as a SQL database.
>
> Next, Machine Learning will be a great addition to Ignite as an HTAP
> platform offering. This is why we should keep investing our time and
> resources in that recently released component.
>
> Having this said, I see the scope for 2.1 release this way:
>
> 1. Distributed Persistent Store - if the donation is accepted by ASF. The
> decision is to be done in separate discussion. W/o the store
>Ignite can only be used as In-Memory SQL database.
>
> 2. SQL Grid:
> - CREATE & DROP table commands: https://issues.apache.org/
> jira/browse/IGNITE-4651
> - Renewed JDBC driver: https://issues.apache.org/
> jira/browse/IGNITE-4922
> - Collocation based routing of SQL queries: https://issues.apache.org/
> jira/browse/IGNITE-4510,
>https://issues.apache.org/jira/browse/IGNITE-4509
> -
>
> 3. .NET:
> - Peer-class loading: https://issues.apache.org/
> jira/browse/IGNITE-2492
> - SQLDependency: https://issues.apache.org/jira/browse/IGNITE-2657
>
> 4. C++:
> - Compute Grid: https://issues.apache.org/jira/browse/IGNITE-3574
>
> 5. ML Grid:
> - Linear regression algorithms: https://issues.apache.org/
> jira/browse/IGNITE-5012
> - K-means clustering: https://issues.apache.org/
> jira/browse/IGNITE-5113
>
>
> Please join the thread and share your thoughts, ideas and concerns.
>
> —
> Denis
>
>
>




-- 
Best Regards, Vyacheslav

Re: Add ability to enable and disable rebalancing per-node

[Alexandr Kuramshin]

to Nick,

could you please describe in more detail the use of DataStreamer (do you
use StreamReceiver)?

It seems that you've unnecessary care about synchronous service startup and
cache rebalance. Service should start quickly after node has joined the
topology, and will process all the data has been collected by local
partitions the moments before. You may use rebalance delay to minimize the
amount of collected data before the service has been be started.

But if your service depends on external resources (another service),
managing rebalance won't help you because external resource may get
unavailable even after your service gets started and rebalance has occur.
You can't unrebalance partitions in such the case. In addition, if some
event-cache should be supplied with other caches (storing additional data
for service processing), there is always the gap between rebalancing
partition of first and the last cache containing collocated data. I think
you should not worry about additional network calls while rebalancing in
progress.

to Sasha,

I think we need configuration property enablePartitionExchange (in addition
to MBean flag) to have an ability to disable partition exchange at node
startup.

2017-05-06 2:51 GMT+07:00 npordash :

> I can outline a use-case I have which may help define requirements for this
> task. For context, I was originally going to try and address the below
> use-case by disabling automatic rebalancing on a per-cache basis and use a
> cluster-wide task to orchestrate manual rebalancing; however, this issue
> sounds like it may provide a better approach.
>
> I have caches setup for the sole purpose of routing data to nodes via a
> Data
> Streamer. The logic in the streamer is simply to access a plugin on the
> data
> node which exposes a processing pipeline and runs the received cache
> entries
> through it. The data in this case is monitoring related and there is one
> cache (or logical stream) per data type (f.e. logs, events, metrics).
>
> The pipeline is composed of N services which are deployed as node
> singletons
> and have a service filter which targets a particular cache. These services
> can be deployed and un-deployed as processing requirements change or bugs
> are fixed without requiring clients to know or care about it.
>
> The catch here is that when nodes are added I don't want map partitions to
> rebalance to a new node until I know all of the necessary services are
> running, otherwise we may have a small window where data is processed
> through a pipeline that isn't completely initialized yet which would result
> in a data quality issue. Alternatively, I could have the pipeline raise an
> error which would cause the streamer to retry, but I'd like this to be
> handled more gracefully, if possible.
>
> In addition, it will probably be the case were these caches eventually have
> node filters so that we can isolate resources for these streams across
> different computes. This means that, for example, if we add a node only for
> metrics then deferring rebalancing should ideally only impact caches that
> would get assigned to that node.
>
> Going even further... so far we've talked about one cache which is used
> just
> for streaming, but at least one of the services would create its own set of
> caches as an in-memory storage layer which maintains an inverted index and
> time series data for elements coming through the stream. The storage caches
> in this case would only exist on nodes where the stream cache is and most
> of
> the write activity to these caches would be local since they would use the
> same affinity as the stream cache (if most writes were remote this wouldn't
> scale well). So... these caches would need to rebalance at the same time in
> order to minimize the possibility of additional network calls.
>
> The main concern I have is how to avoid the race condition of another node
> joining the topology _after_ it has been determined rebalancing should
> happen, but _before_ rebalancing is triggered. If this is controlled on a
> per-node (+cache) basis - as the ticket describes - it's probably a
> non-issue, but it's definitely an issue if it's only on a per-cache basis.
>
> -Nick
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Add-ability-to-enable-and-
> disable-rebalancing-per-node-tp17494p17529.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>



-- 
Thanks,
Alexandr Kuramshin