[jira] [Created] (IGNITE-4552) Optimize GridDhtLocalPartition.rmvQueue

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-4552:
-

 Summary: Optimize GridDhtLocalPartition.rmvQueue
 Key: IGNITE-4552
 URL: https://issues.apache.org/jira/browse/IGNITE-4552
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.6
Reporter: Alexei Scherbakov
 Fix For: 2,0


Current implementation stores deferred entry removals in rmvQueue for 
consistency guaranties.

This can lead to significant heap over-usage(I observed several Gbs) in case of 
many caches with removals, because currently queue is cleared lazily after 
reaching max capacity(200_000 by default).

This can be mitigated by using lower IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, 
but can lead to consistency issues in case of frequent cache updates.

Possible optimizations:

* Use single fixed size queue per all caches to overcome limitations of 
IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE workaround.

* Do queue cleaning in background

* Move queue to an off-heap.







--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

scripting languages base cases using swig

[Evgeniy Stanilovskiy]

Hi all.
Not so long ago i had to know that ignite had reduced functionality  
support in scripting languages.
So, idea was to take an existing C++ client and using SWIG  
(http://www.swig.org) as automatic wrapper, generate clients for absence  
scripting languages.

What do you think about this case ?

Thanks !

Re: DDL implementation details

[Sergey Kozlov]

For the approach schema==cache it is not clear how it works for some cases
(especialy if SQL and regular cache operations are mixed):

1. I want to store two tables in same cache (same schema) and both tables
have identical structure like id Integer, name String.
How they will be processed for regular cache operations
put/putall/get/getall etc where we get cache instance by its name (on SQL
level we use the table name) ?

2. What's about ALTER TABLE statement? Will the changes for one table lock
all tables in the cache?

3. In the future if we will introduce table-based features like SQL roles
(as next step of SQL implementation) "many tables one cache" rule looks
like more difficult for its implementation.

p.s. may be we need to think about new entity called "table" (based on type
descriptor) inside cache to better support of SQL DDL


On Mon, Jan 16, 2017 at 11:06 PM, Sergi Vladykin 
wrote:

> Dima,
>
> I agree that cache==table is definitely a wrong choice, but as far as I see
> Vova suggests having cache==tablespace instead of cache==schema. I tend to
> agree with this decoupling of physical and logical grouping, but the
> concern is that it will require much more work to do.
>
> Sergi
>
> 2017-01-16 21:35 GMT+03:00 Dmitriy Setrakyan :
>
> > Vova,
> >
> > Currently I see only 2 ways we can proceed here:
> >
> >1. cache == table
> >2. cache == schema
> >
> > I agree that "cache==table" may be more flexible, but I don't think it
> will
> > work in Ignite.
> > We may end up with 1,000s of caches, which will carry significant
> overhead
> > on memory and cluster overall. I think that we have no choice but to take
> > "cache==schema" approach.
> >
> > D.
> >
> > On Mon, Jan 16, 2017 at 1:00 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Sergi, Dima,
> > >
> > > In the scope of Ignite 1.x it is perfectly fine to have "schema =
> cache".
> > > Nobody suffers from it because nobody use Ignite as database. But in
> > > future, thanks to page memory, we are going to target real database use
> > > cases. Users will have multiple tables in Ignite. Plus views, triggers,
> > > constraints, etc.. All these features are very useful and easy to
> > implement
> > > provided that we already have table and index implementations. And in
> > > databases all related objects are *logically *grouped in a "schema".
> This
> > > is convenient for users: less boilerplate in SQL, better manageability
> > > (remember that database users will definitely need some console and/or
> UI
> > > tools to manage Ignite as a database).
> > >
> > > What you offer is to group database objects *physically *rather than
> > > logically. It will lead to:
> > > - Boilerplate in queries
> > > - Inconvenient database management. All the things database users are
> > used
> > > to - import/export tools, UIs, "USING" keyword, etc, will look weird in
> > > Ignite as there will be no way to group arbitrary objects logically.
> > >
> > > With this approach almost every user will have to use two schemes
> instead
> > > of one - one for operational data (PARTITIONED) and one for reference
> > data
> > > (REPLICATED). No conventional database works this way.
> > >
> > > Vladimir.
> > >
> > > On Fri, Jan 13, 2017 at 9:18 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > I will join Sergi here. It seems like "schema = cache" will take care
> > of
> > > > all different configuration properties required for different groups
> of
> > > > caches. In addition, it cleanly maps into current Ignite
> architecture.
> > We
> > > > will need to have a very strong reason to move away from it.
> > > >
> > > > D.
> > > >
> > > > On Fri, Jan 13, 2017 at 2:39 AM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Correct, it worked, because Ignite has never had real database use
> > case
> > > > in
> > > > > mind. Unfortunately, if our global plans go as expected, it will
> not
> > > work
> > > > > for Ignite 2.x+.
> > > > >
> > > > > On Fri, Jan 13, 2017 at 11:53 AM, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Lets move on with SQL schema == Ignite cache. It worked always
> like
> > > > > this, I
> > > > > > see no reasons to change this.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-01-13 11:20 GMT+03:00 Vladimir Ozerov  >:
> > > > > >
> > > > > > > "Tablespace" (Oracle, PostgreSQL) is what maps better than
> > "schema"
> > > > to
> > > > > > our
> > > > > > > cache. But not ideally still.
> > > > > > >
> > > > > > > On Fri, Jan 13, 2017 at 11:10 AM, Vladimir Ozerov <
> > > > > voze...@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Alex,
> > > > > > > >
> > > > > > > > Currently Ignite is not used as database. It is used as
> search
> > > > > engine -
> > > > > > > > several types, several tables, several joins. This is why
> > having
> > > > > > "SCHEMA
> > > > > > > ==
> > > > > > > > cache" was never a 

Re: DDL implementation details

[Sergi Vladykin]

Dima,

I agree that cache==table is definitely a wrong choice, but as far as I see
Vova suggests having cache==tablespace instead of cache==schema. I tend to
agree with this decoupling of physical and logical grouping, but the
concern is that it will require much more work to do.

Sergi

2017-01-16 21:35 GMT+03:00 Dmitriy Setrakyan :

> Vova,
>
> Currently I see only 2 ways we can proceed here:
>
>1. cache == table
>2. cache == schema
>
> I agree that "cache==table" may be more flexible, but I don't think it will
> work in Ignite.
> We may end up with 1,000s of caches, which will carry significant overhead
> on memory and cluster overall. I think that we have no choice but to take
> "cache==schema" approach.
>
> D.
>
> On Mon, Jan 16, 2017 at 1:00 AM, Vladimir Ozerov 
> wrote:
>
> > Sergi, Dima,
> >
> > In the scope of Ignite 1.x it is perfectly fine to have "schema = cache".
> > Nobody suffers from it because nobody use Ignite as database. But in
> > future, thanks to page memory, we are going to target real database use
> > cases. Users will have multiple tables in Ignite. Plus views, triggers,
> > constraints, etc.. All these features are very useful and easy to
> implement
> > provided that we already have table and index implementations. And in
> > databases all related objects are *logically *grouped in a "schema". This
> > is convenient for users: less boilerplate in SQL, better manageability
> > (remember that database users will definitely need some console and/or UI
> > tools to manage Ignite as a database).
> >
> > What you offer is to group database objects *physically *rather than
> > logically. It will lead to:
> > - Boilerplate in queries
> > - Inconvenient database management. All the things database users are
> used
> > to - import/export tools, UIs, "USING" keyword, etc, will look weird in
> > Ignite as there will be no way to group arbitrary objects logically.
> >
> > With this approach almost every user will have to use two schemes instead
> > of one - one for operational data (PARTITIONED) and one for reference
> data
> > (REPLICATED). No conventional database works this way.
> >
> > Vladimir.
> >
> > On Fri, Jan 13, 2017 at 9:18 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > Vova,
> > >
> > > I will join Sergi here. It seems like "schema = cache" will take care
> of
> > > all different configuration properties required for different groups of
> > > caches. In addition, it cleanly maps into current Ignite architecture.
> We
> > > will need to have a very strong reason to move away from it.
> > >
> > > D.
> > >
> > > On Fri, Jan 13, 2017 at 2:39 AM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Correct, it worked, because Ignite has never had real database use
> case
> > > in
> > > > mind. Unfortunately, if our global plans go as expected, it will not
> > work
> > > > for Ignite 2.x+.
> > > >
> > > > On Fri, Jan 13, 2017 at 11:53 AM, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Lets move on with SQL schema == Ignite cache. It worked always like
> > > > this, I
> > > > > see no reasons to change this.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-01-13 11:20 GMT+03:00 Vladimir Ozerov :
> > > > >
> > > > > > "Tablespace" (Oracle, PostgreSQL) is what maps better than
> "schema"
> > > to
> > > > > our
> > > > > > cache. But not ideally still.
> > > > > >
> > > > > > On Fri, Jan 13, 2017 at 11:10 AM, Vladimir Ozerov <
> > > > voze...@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Alex,
> > > > > > >
> > > > > > > Currently Ignite is not used as database. It is used as search
> > > > engine -
> > > > > > > several types, several tables, several joins. This is why
> having
> > > > > "SCHEMA
> > > > > > ==
> > > > > > > cache" was never a problem. Users have never build complex SQL
> > > > > > applications
> > > > > > > on top of Ignite. But we are going towards database. And my
> > > question
> > > > > > stands
> > > > > > > still - suppose it is Y2019, how is user going to migrate his
> > > > database
> > > > > > > containing 20-30-50-100 tables in a single schema in Oracle to
> > > > Ignite?
> > > > > > >
> > > > > > > Single cache for all tables? Doens't work - not flexible. Users
> > > will
> > > > > > > definitely require different cache modes, different co-location
> > > > rules,
> > > > > > > different number of backups, etc..
> > > > > > > Schema per table? Doesn't work either - unmanageable and not
> > > > convenient
> > > > > > > for users even for relatively small databases.
> > > > > > >
> > > > > > > From user perspective schema is logical grouping of database
> > > objects,
> > > > > > > nothing more.
> > > > > > >
> > > > > > > For Ignite schema could be a logical group of resources (nodes,
> > > > memory
> > > > > > > pools, caches, etc.). And multiple tables over multiple caches
> > > should
> > > > > > > reside in it. To the contrast, table definition governs how
> data
> > is
> > > 

Re: DDL implementation details

[Dmitriy Setrakyan]

Vova,

Currently I see only 2 ways we can proceed here:

   1. cache == table
   2. cache == schema

I agree that "cache==table" may be more flexible, but I don't think it will
work in Ignite.
We may end up with 1,000s of caches, which will carry significant overhead
on memory and cluster overall. I think that we have no choice but to take
"cache==schema" approach.

D.

On Mon, Jan 16, 2017 at 1:00 AM, Vladimir Ozerov 
wrote:

> Sergi, Dima,
>
> In the scope of Ignite 1.x it is perfectly fine to have "schema = cache".
> Nobody suffers from it because nobody use Ignite as database. But in
> future, thanks to page memory, we are going to target real database use
> cases. Users will have multiple tables in Ignite. Plus views, triggers,
> constraints, etc.. All these features are very useful and easy to implement
> provided that we already have table and index implementations. And in
> databases all related objects are *logically *grouped in a "schema". This
> is convenient for users: less boilerplate in SQL, better manageability
> (remember that database users will definitely need some console and/or UI
> tools to manage Ignite as a database).
>
> What you offer is to group database objects *physically *rather than
> logically. It will lead to:
> - Boilerplate in queries
> - Inconvenient database management. All the things database users are used
> to - import/export tools, UIs, "USING" keyword, etc, will look weird in
> Ignite as there will be no way to group arbitrary objects logically.
>
> With this approach almost every user will have to use two schemes instead
> of one - one for operational data (PARTITIONED) and one for reference data
> (REPLICATED). No conventional database works this way.
>
> Vladimir.
>
> On Fri, Jan 13, 2017 at 9:18 PM, Dmitriy Setrakyan 
> wrote:
>
> > Vova,
> >
> > I will join Sergi here. It seems like "schema = cache" will take care of
> > all different configuration properties required for different groups of
> > caches. In addition, it cleanly maps into current Ignite architecture. We
> > will need to have a very strong reason to move away from it.
> >
> > D.
> >
> > On Fri, Jan 13, 2017 at 2:39 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Correct, it worked, because Ignite has never had real database use case
> > in
> > > mind. Unfortunately, if our global plans go as expected, it will not
> work
> > > for Ignite 2.x+.
> > >
> > > On Fri, Jan 13, 2017 at 11:53 AM, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Lets move on with SQL schema == Ignite cache. It worked always like
> > > this, I
> > > > see no reasons to change this.
> > > >
> > > > Sergi
> > > >
> > > > 2017-01-13 11:20 GMT+03:00 Vladimir Ozerov :
> > > >
> > > > > "Tablespace" (Oracle, PostgreSQL) is what maps better than "schema"
> > to
> > > > our
> > > > > cache. But not ideally still.
> > > > >
> > > > > On Fri, Jan 13, 2017 at 11:10 AM, Vladimir Ozerov <
> > > voze...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Alex,
> > > > > >
> > > > > > Currently Ignite is not used as database. It is used as search
> > > engine -
> > > > > > several types, several tables, several joins. This is why having
> > > > "SCHEMA
> > > > > ==
> > > > > > cache" was never a problem. Users have never build complex SQL
> > > > > applications
> > > > > > on top of Ignite. But we are going towards database. And my
> > question
> > > > > stands
> > > > > > still - suppose it is Y2019, how is user going to migrate his
> > > database
> > > > > > containing 20-30-50-100 tables in a single schema in Oracle to
> > > Ignite?
> > > > > >
> > > > > > Single cache for all tables? Doens't work - not flexible. Users
> > will
> > > > > > definitely require different cache modes, different co-location
> > > rules,
> > > > > > different number of backups, etc..
> > > > > > Schema per table? Doesn't work either - unmanageable and not
> > > convenient
> > > > > > for users even for relatively small databases.
> > > > > >
> > > > > > From user perspective schema is logical grouping of database
> > objects,
> > > > > > nothing more.
> > > > > >
> > > > > > For Ignite schema could be a logical group of resources (nodes,
> > > memory
> > > > > > pools, caches, etc.). And multiple tables over multiple caches
> > should
> > > > > > reside in it. To the contrast, table definition governs how data
> is
> > > > > stored.
> > > > > > This is similar to, for example, MySQL approach, where you define
> > how
> > > > you
> > > > > > store data on per-table level, and on schema level you define
> only
> > > > minor
> > > > > > things like collation.
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 13, 2017 at 10:33 AM, Alexander Paschenko <
> > > > > > alexander.a.pasche...@gmail.com> wrote:
> > > > > >
> > > > > >> Vova,
> > > > > >>
> > > > > >> 2017-01-13 4:56 GMT+08:00 Vladimir Ozerov  >:
> > > > > >> > I am not quite sure I understand the idea of "SCHEMA ==
> cache".
> > > > > Consider
> 

[jira] [Created] (IGNITE-4551) Reconsider cache key/value peer class loading

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-4551:


 Summary: Reconsider cache key/value peer class loading
 Key: IGNITE-4551
 URL: https://issues.apache.org/jira/browse/IGNITE-4551
 Project: Ignite
  Issue Type: Sub-task
Reporter: Alexey Goncharuk
 Fix For: 2.0


In new cache implementation after entry is written in offheap information about 
key/value classloaders in lost (before classloader ids were stored in 
swap/offheap see GridCacheMapEntry.swap in 'master').
Need decide how it should work with new architecture (maybe single type per 
cache can simplify implementation).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ignite ASF blog

[Konstantin Boudnik]

The blog
https://blogs.apache.org/ignite/

has been configured. I have admin rights so please let me know if you want to
start contributing and I will grant you the credentials for it.

Cheers,
  Cos

On Fri, Jan 13, 2017 at 08:39AM, Konstantin Boudnik wrote:
> FYI
> https://issues.apache.org/jira/browse/INFRA-13324
> 
> once the JIRA is finished, we'd have an office Apache blog for Ignite, listed
> and syndicated across all ASF resources. Hence raising the visibility of the
> project.
> 
> Regards,
>   Cos
> 
> On Sat, Nov 26, 2016 at 01:21PM, Konstantin Boudnik wrote:
> > Gents, 
> > 
> > I would encourage you to use
> > 
> > https://blogs.apache.org/
> > 
> > as this place gets contant attention of the ASF media team and all the blogs
> > here got quite a bit of the spot light. In fact, I would encourage this 
> > group
> > to start using it. I'd be happy to help to set this up.
> > 
> > Cos
> > 
> > On Wed, Nov 23, 2016 at 10:34AM, Denis Magda wrote:
> > > Pavel,
> > > 
> > > Do you mind blogging about ASP.NET session state [1] and Entity Framework 
> > > 2nd level cache [2] support after 1.8 release goes public?
> > > 
> > > Also I would remind everyone in the community that it’s always welcomed 
> > > to share your experience or Ignite related news over blog posts. We track 
> > > all the posts on this page.
> > > https://ignite.apache.org/blogs.html 
> > > 
> > > 
> > > 
> > > [1] https://apacheignite-net.readme.io/docs/aspnet-session-state-caching 
> > > 
> > > [2] 
> > > https://apacheignite-net.readme.io/docs/entity-framework-second-level-cache
> > >  
> > > 
> > > 
> > > —
> > > Denis

[GitHub] ignite pull request #1433: IGNITE-4548 Implemented mapping of enum fields fo...

[vsisko]

GitHub user vsisko opened a pull request:

https://github.com/apache/ignite/pull/1433

IGNITE-4548 Implemented mapping of enum fields for POJO STORE



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4548-2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1433.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1433


commit ab956f2d970507d8c7fe414c10ceda061bf99ba8
Author: vsisko 
Date:   2017-01-16T11:11:30Z

IGNITE-4548 Mapping of enum types.

commit e4e3ce4c388abaf9595964a33544fc255347abb1
Author: vsisko 
Date:   2017-01-16T12:10:22Z

IGNITE-4548 Mapping of enum types test.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1432: IGNITE-4046: Implemented BinaryArrayIdentityResol...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/1432

IGNITE-4046: Implemented BinaryArrayIdentityResolver.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4046

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1432.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1432


commit fe371527a91322ff062b432e6982947dcf8ef0c9
Author: Igor Sapego 
Date:   2016-12-07T13:56:04Z

IGNITE-4046: Implemented BinaryArrayIdentityResolver.

commit ea8f064c94baf9b3e29f706ae85fad06bbf4bc90
Author: Igor Sapego 
Date:   2016-12-07T16:46:36Z

IGNITE-4046: Added tests.

commit cc11bd2d3abafd47714e123b7ced294477bbb7d3
Author: Igor Sapego 
Date:   2016-12-09T13:37:36Z

IGNITE-4046: Test fixed.

commit 0b1312fb208b867fb4cbd76e8734213a39b5c97d
Author: Igor Sapego 
Date:   2016-12-09T14:05:44Z

IGNITE-4046: Makefile fixes.

commit 4e7d0cbfcef642dff0146222e2e82ebe70fbb5cc
Author: Igor Sapego 
Date:   2016-12-26T15:34:36Z

Merge remote-tracking branch 'upstream/master' into ignite-4046

commit 4da71b259540141ab8f2840b397a62548da6a095
Author: Igor Sapego 
Date:   2016-12-26T18:31:57Z

IGNITE-4046: Merge-related changes.

commit 1ecf5738de7c6e52470fb0279181240bcf30a830
Author: Igor Sapego 
Date:   2017-01-16T14:33:30Z

Merge remote-tracking branch 'upstream/master' into ignite-4046




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1337: IGNITE-4046: Implemented BinaryArrayIdentityResol...

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/1337


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1309: IGNITE-4045 .NET: Support DML API

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1309


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1417: IGNITE-3400: add available size check before data...

[tledkov-gridgain]

Github user tledkov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/1417


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Product ID for Apache Ignite

[Mark Thomas]

On 03/01/2017 17:52, Denis Magda wrote:
> Hi Mark,
> 
> I reached out both MITRE and cvedetails.com
>  folks as you suggested earlier. Below you can
> see the answer from MITRE. CVE guys have not replied yet.
> 
> One of the things suggested by MITRE is the following
> 
>> One last item to note is that Apache is a CVE CNA. You can find more
>> information about the CNA program
>> at http://cve.mitre.org/cve/cna.html. We realize that there are many
>> Apache products, but you may want to investigate this and reach out to
>> the appropriate folks within Apache to not only share the CVE ID pool,
>> but also potentially communicate when vulnerabilities are found in
>> Apache Ignite.
> 
> Do you guys keep in eye on all Apache vulnerabilities or subscribe to
> the updates? If so, could you update Apache Ignite community every time
> an Ignite vulnerability has discovered?

That isn't how vulnerability handling works.

See http://www.apache.org/security/committers.html

Any vulnerability reports for Apache Ignite received by the security
team will be passed privately to the project for resolution.

Mark


> 
> Regards,
> Denis
> 
>> On Dec 29, 2016, at 10:03 AM, Coffin, Chris > > wrote:
>>
>> Denis,
>>  
>> The cvedetails.com  web site is not affiliated
>> with MITRE and you would need to contact them directly if you wanted
>> to see a change in the URL you had provided. The contact information
>> for cvedetails.com  can be found
>> at http://www.cvedetails.com/about-contact.php.
>>  
>> The MITRE CVE team does not currently provide any notifications for
>> CVEs, but has considered this in the recent past. One thought was to
>> create a registry of product vendors that is used for contact purposes
>> when a CVE ID is published and affects the vendor. If this is
>> something that would be of interest to you, please let us know.
>>  
>> One last item to note is that Apache is a CVE CNA. You can find more
>> information about the CNA program
>> at http://cve.mitre.org/cve/cna.html. We realize that there are many
>> Apache products, but you may want to investigate this and reach out to
>> the appropriate folks within Apache to not only share the CVE ID pool,
>> but also potentially communicate when vulnerabilities are found in
>> Apache Ignite.
>>  
>> Regards,
>>  
>> Chris Coffin
>> The CVE Team
>>  
>> *From:* Denis Magda [mailto:dma...@apache.org] 
>> *Sent:* Wednesday, December 28, 2016 3:18 PM
>> *To:* Common Vulnerabilities & Exposures > >
>> *Cc:* priv...@ignite.apache.org 
>> *Subject:* Fwd: Product ID for Apache Ignite 
>>  
>> Dear Sir/Madam,
>>  
>> I’m writing you on behalf of Apache Ignite [1] community to check if
>> there is a way to obtain a product ID for our project. The whole
>> purpose of that is to be proactive by handling vulnerabilities as soon
>> as they appear in the CVE database. 
>>  
>> For instance, we can use services like that [2] to subscribe for
>> vulnerabilities related updates. To do that, both vendor ID and
>> product ID have to be known. In our case the vendor is 45 (Apache
>> Foundation) while there is no product ID for Apache Ignite yet. 
>>  
>> Could you assist and register product ID for Apache Ignite?
>>  
>> [1] https://ignite.apache.org 
>> [2] http://www.cvedetails.com/product-list/vendor_id-45/Apache.html
>>  
>> Regards,
>> Denis Magda
>> Apache Ignite PMC Chair
>>
>>
>> Begin forwarded message:
>>  
>> *From: *Mark Thomas mailto:ma...@apache.org>>
>> *Subject: Re: Product ID for Apache Ignite at CVE*
>> *Date: *December 12, 2016 at 9:01:58 AM PST
>> *To: *priv...@ignite.apache.org 
>> *Cc: *secur...@apache.org 
>> *Reply-To: *priv...@ignite.apache.org
>> 
>>  
>> On 08/12/2016 01:59, Denis Magda wrote:
>>
>> Hello,
>>
>> I’m writing on behalf of Apache Ignite [1] community. We would
>> like to
>> register Apache Ignite in CVE database so that it appears in
>> the list of
>> Apache products [2] already registered there and has its own
>> unique
>> product ID.
>>
>> Who can assist us with this or provide a guidance?
>>
>>
>> Sorry, not a clue.
>>
>> I suspect updates are made as new products issue vulnerability
>> announcements. cvedetails.com  isn't part
>> of Mitre so I suggest you
>> contact cvedetails.com  directly with your
>> query.
>>
>> Mark
>>
>>
>>
>>
>>
>> [1] https://ignite.apache.org 
>> [2] http://www.cvedetails.com/product-list/vendor_id-45/Apache.html
>>
>> Regards,
>> Denis
> 

[GitHub] ignite pull request #1397: Ignite 4515

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1397


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4550) Move service deployment to certain test

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-4550:


 Summary: Move service deployment to certain test
 Key: IGNITE-4550
 URL: https://issues.apache.org/jira/browse/IGNITE-4550
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 1.8
Reporter: Andrew Mashenkov
 Fix For: 1.9


GridCacheAbstractFullApiSelfTest has service deployment in beforeTest() method.
It look like there is no need to do this for all tests and service deployment 
should be moved to testTransformResourceInjection().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: DDL implementation details

[Alexey Goncharuk]

>
> Right, however, this section of the docs
>
> https://apacheignite.readme.io/docs/jcache#section-dynamic-cache
>
> does not clarify Ignite's behavior regarding dynamic caches when a new
> node joins the cluster. Will it just pick up dynamic cache
> configurations from the peers without being explicitly configured? If
> yes, I suggest that we at least mention it in the docs. (Sorry if I'm
> missing something but I skimmed through the documentation and have not
> found a clear answer to this.) Could someone clarify please?
>

Alex,

Yes, newly joined nodes will fetch all started caches configuration and
create local data structured accordingly. If we use schema=cache approach,
nothing beyond cache creation should be done to create the schema. I've
updated the doc you mentioned.

--AG

Re: Contributing to Ignite.NET

[Pavel Tupitsyn]

Sergey, sure, go ahead and take it!
Please assign to yourself and click "Start Progress".

Let me know if you have any questions.

Thanks,
Pavel

On Mon, Jan 16, 2017 at 2:49 PM, Guru Stron 
wrote:

> Hi igniters,
>
> I want to improve LINQ support in IGNITE.NET.
>
> Is it ok if i take this task: IGNITE-4425 .NET: Support "Array.Contains" in
> LINQ 
>
> On 9 January 2017 at 15:40, Pavel Tupitsyn  wrote:
>
> > Sergey, I've merged your fix for IGNITE-4412.
> >
> > Looking forward to more contributions from you!
> >
> > Pavel
> >
> > On Thu, Jan 5, 2017 at 8:15 PM, Pavel Tupitsyn 
> > wrote:
> >
> > > Hi Sergey,
> > >
> > > I've added you to project contributors in Jira.
> > >
> > > I can suggest the following .NET tickets to get familiar with the
> > project:
> > >
> > > IGNITE-4412 .NET: NLogLoggerTest is not thread safe
> > > 
> > > IGNITE-3170 .NET: Add user-friendly ToString overrides for public types
> > > 
> > > IGNITE-1508 .NET: Create common ToString() method
> > > 
> > >
> > > Let me know if you have any questions.
> > >
> > > Thanks,
> > >
> > > Pavel.
> > >
> > >
> > > On Thu, Jan 5, 2017 at 8:10 PM, Guru Stron 
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> My name is Sergey, i would like to contribute to Ignite.NET.
> > >> Can you please suggest a ticket for me to start. My Jira username is
> > >> GuruStron.
> > >>
> > >
> > >
> >
>

Re: Contributing to Ignite.NET

[Guru Stron]

Hi igniters,

I want to improve LINQ support in IGNITE.NET.

Is it ok if i take this task: IGNITE-4425 .NET: Support "Array.Contains" in
LINQ 

On 9 January 2017 at 15:40, Pavel Tupitsyn  wrote:

> Sergey, I've merged your fix for IGNITE-4412.
>
> Looking forward to more contributions from you!
>
> Pavel
>
> On Thu, Jan 5, 2017 at 8:15 PM, Pavel Tupitsyn 
> wrote:
>
> > Hi Sergey,
> >
> > I've added you to project contributors in Jira.
> >
> > I can suggest the following .NET tickets to get familiar with the
> project:
> >
> > IGNITE-4412 .NET: NLogLoggerTest is not thread safe
> > 
> > IGNITE-3170 .NET: Add user-friendly ToString overrides for public types
> > 
> > IGNITE-1508 .NET: Create common ToString() method
> > 
> >
> > Let me know if you have any questions.
> >
> > Thanks,
> >
> > Pavel.
> >
> >
> > On Thu, Jan 5, 2017 at 8:10 PM, Guru Stron 
> > wrote:
> >
> >> Hi,
> >>
> >> My name is Sergey, i would like to contribute to Ignite.NET.
> >> Can you please suggest a ticket for me to start. My Jira username is
> >> GuruStron.
> >>
> >
> >
>

Re: DDL implementation details

[Alexander Paschenko]

Sergi,

Regarding CREATE SCHEMA/TABLE parsing:

2017-01-12 18:51 GMT+03:00 Sergi Vladykin :
> Hi,
>
> 1. For now I'm against inventing any custom SQL syntax and implementing
> parsing.
> Currently H2 supports the following syntax:
>
> CREATE TABLE test(...) WITH "myCustomParamString"
>
> This is enough for us to pass the needed parameters.

H2's grammar requires to supply table engine class name as well, you
can't just go with WITH:

http://www.h2database.com/html/grammar.html#create_table

And code of H2's org.h2.command.Parser works exactly like this - you
can't just set default engine in connection string, in order to use
WITH, you have to specify both engine and WITH.

So we have either to force users to specify engine in their queries
(ugly and hardly acceptable), or have to somehow analyze CREATE TABLE
and sneak ENGINE param into the query ourselves.

But this again smells custom parsing, if you ask me.

>
> 2. Agree with AG, we have to separate cache creation from table creation.
> Cache == SQL schema for us. We just have to add the same WITH syntax in H2
> for schema creation like this:
>
> CREATE SCHEMA "MyCacheName" WITH "cacheConfig=myCache.xml"
>

Currently H2 does not allow using WITH with CREATE SCHEMA, proof:
http://www.h2database.com/html/grammar.html#create_schema

Do you suggest that we should somehow introduce this syntax to H2 engine first?

In the light of this, CREATE CACHE might seem legit. Thus, if we
depart from CREATE SCHEMA == create cache and introduce custom syntax
for CREATE CACHE, then we'll end up closer to what Vlad suggest –
schema is a logical object, caches exist on their own right.

Also, to add to Vlad's thoughts on Ignite becoming a database, I would
like to point out that for now we have talked only about schemas,
tables, and caches, but there are also *databases* as organizational
entities. Shouldn't we start thinking about those as well?

- Alex

Re: DDL implementation details

[Alexander Paschenko]

Dima,

2017-01-13 21:20 GMT+03:00 Dmitriy Setrakyan :
> How about dynamic caches? Isn't the metadata already maintained across all
> nodes every time a new cache is created? What am I missing here?

Right, however, this section of the docs

https://apacheignite.readme.io/docs/jcache#section-dynamic-cache

does not clarify Ignite's behavior regarding dynamic caches when a new
node joins the cluster. Will it just pick up dynamic cache
configurations from the peers without being explicitly configured? If
yes, I suggest that we at least mention it in the docs. (Sorry if I'm
missing something but I skimmed through the documentation and have not
found a clear answer to this.) Could someone clarify please?

- Alex

[jira] [Created] (IGNITE-4549) .NET: Add LINQPad samples to logging NuGet packages

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4549:
--

 Summary: .NET: Add LINQPad samples to logging NuGet packages
 Key: IGNITE-4549
 URL: https://issues.apache.org/jira/browse/IGNITE-4549
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 1.8
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Priority: Trivial
 Fix For: 2.0


Add LINQPad samples to Log4Net and NLog NuGet packages.

Samples should include simple in-code file-based log configuration and writing 
to the log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1431: Ignite 1.7.5 p1

[avinogradovgg]

GitHub user avinogradovgg opened a pull request:

https://github.com/apache/ignite/pull/1431

Ignite 1.7.5 p1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.7.5-p1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1431.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1431


commit ccfaaf8d060ef984678d2376b16b5a17e7c17e9d
Author: vozerov-gridgain 
Date:   2016-09-28T11:38:17Z

Merge remote-tracking branch 'upstream/ignite-1.6.9' into ignite-1.6.9

commit c7fa918c10d771efa91cde1017662c26d0a61085
Author: Pavel Tupitsyn 
Date:   2016-09-28T11:47:17Z

Merge remote-tracking branch 'remotes/community/ignite-1.6.9' into 
ignite-1.7.3

commit b9105df77cc70b532f94899c754fba47e3e05e9a
Author: vozerov-gridgain 
Date:   2016-09-28T11:53:20Z

IGNITE-3989: Moved failing JTA tests to ignore module.

commit d595345765db2151ff432beecd478ce056393593
Author: vozerov-gridgain 
Date:   2016-09-28T12:08:38Z

IGNITE-3990: Moved failing Spring tests to "ignore" module.

commit e3f13455d4273e615727d0410783e3719db98f76
Author: sboikov 
Date:   2016-09-28T09:56:17Z

ignite-2833 Need call 'touch' for cache entry if it was obtained using 
'entryEx'.

(cherry picked from commit 17c2fc0)

commit b2faa339acb2eea24e6dd5e0c21fc3d3d0592ff6
Author: sboikov 
Date:   2016-09-28T10:47:51Z

ignite-3621 Fixed 'testEvictExpired'.

(cherry picked from commit 4ff19c2)

commit 74d2fc2416b8e6bc0598152552021f984a013061
Author: sboikov 
Date:   2016-09-28T11:31:24Z

ignite-3621 Fixed 'testEvictExpired'.

(cherry picked from commit bfe4458)

commit d2563dacceea61b19bb6e083e29ebacc28fdd323
Author: vozerov-gridgain 
Date:   2016-09-28T12:51:55Z

IGNITE-3993: Added failing client tests to "ignored" test suite.

commit 33d34941390c00e8d6a2488e8f2e11e6abba8a01
Author: sboikov 
Date:   2016-09-28T12:54:52Z

Merge remote-tracking branch 'community/ignite-1.6.9' into ignite-1.6.9

commit 78144c4c9d6200ceef8b666a186039685f053381
Author: vozerov-gridgain 
Date:   2016-09-28T13:52:13Z

Fixed incorrect test count calculation leading to afterTestsStopped() not 
being called.

commit e3dfdecc3607b5f3183bfcb1ce36c57543a8965f
Author: Alexander Paschenko 
Date:   2016-09-28T13:46:46Z

Validate hash code presence in BinaryObject. Fixes #928

commit 53229e290f7d6aab9b504693bd2b93155ecd2bad
Author: Pavel Tupitsyn 
Date:   2016-09-28T14:07:47Z

IGNITE-3995 .NET: Introduced default non-null ASP.NET Session-State Store 
Provider cache name. This closes #1128.

commit d1e3a78ae569fa5d5692816db44f2c677e1b8283
Author: Valentin Kulichenko 
Date:   2016-09-29T00:53:42Z

IGNITE-3191 - Fixed ordering of fields in binary objects

commit b280c3efa1eb84c6bc8abbe31ba669b0c24323d8
Author: vozerov-gridgain 
Date:   2016-09-29T07:23:54Z

Fixed missing Apache header.

commit 3e8a1c6b045c231dbb3c972463000f824386aee9
Author: vozerov-gridgain 
Date:   2016-09-29T08:08:23Z

Added node stop to IgniteCacheExpireAndUpdateConsistencyTest.

commit 062b9b69aa851d5253dfb7f44066b7d749f1ca12
Author: Alexey Goncharuk 
Date:   2016-09-29T08:26:50Z

IGNITE-3633 - Enforce key validation for tests.

commit a4d7aa343df23a5e00df11c5080cc1762b187161
Author: sboikov 
Date:   2016-09-29T09:04:44Z

ignite-3621 Fixed 'testEvictExpired'.

commit ad613afd7165ed2d8bf0a62805fc2506dfe6c59a
Author: sboikov 
Date:   2016-09-29T09:04:44Z

ignite-3621 Fixed 'testEvictExpired'.

(cherry picked from commit a4d7aa3)

commit 22dc2c9e05e5183f838865023c9cb7a8291ac67f
Author: Alexander Paschenko 
Date:   2016-09-29T12:43:20Z

GridNearCacheAdapter, GridDhtCacheAdapter - test fix

commit f447559ecba55149452e4e48e2bf44ef1fa1b4d7
Author: Alexey Goncharuk 
Date:   2016-09-29T14:10:14Z

ignite-1.6.9 - Fixing tests

commit a4c63ae113931615e940ada2411ebce32dba5db8
Author: iveselovskiy 
Date:   2016-09-30T07:45:13Z

IGNITE-3998: IGFS: Enabled 
IgfsAbstractSelfTest.testCreateConsistencyMultithreaded. This closes #1129.

commit a12298c0baf19407f3110ba3c4f0aab88b67946d
Author: Igor Sapego 
Date:   2016-09-30T08:00:51Z

IGNITE-3868: ODBC: Fixed connection string parsing. This closes #1051.

commit c32082fe8b1e02758179c1b7bb61a75be53534fe
Author: Alexey Kuznetsov 
Date:   2016-09-30T08:20:11Z

IGNITE-4007 Fixed update of QueryMetrics.minimumTime() metric.  Tests 
added. Added Visor reset metrics task.

commit f745371af76baa676385c22257b4577d2e697d16
Author: vozerov-gridgain 
Date:   2016-10-03T07:07:26Z

Revert "Fixed missing Apache header."

This reverts commit b280c3efa1eb84c6bc8abbe31ba669b0c24323d8.

commit de50287d493acc73186926d34431733bc76c549b
Author: vozerov-gridgain 
Date:   2016-10-03T07:07:33Z

Revert "IGNITE-3191 - Fixed ordering

Re: DDL implementation details

[Vladimir Ozerov]

Sergi, Dima,

In the scope of Ignite 1.x it is perfectly fine to have "schema = cache".
Nobody suffers from it because nobody use Ignite as database. But in
future, thanks to page memory, we are going to target real database use
cases. Users will have multiple tables in Ignite. Plus views, triggers,
constraints, etc.. All these features are very useful and easy to implement
provided that we already have table and index implementations. And in
databases all related objects are *logically *grouped in a "schema". This
is convenient for users: less boilerplate in SQL, better manageability
(remember that database users will definitely need some console and/or UI
tools to manage Ignite as a database).

What you offer is to group database objects *physically *rather than
logically. It will lead to:
- Boilerplate in queries
- Inconvenient database management. All the things database users are used
to - import/export tools, UIs, "USING" keyword, etc, will look weird in
Ignite as there will be no way to group arbitrary objects logically.

With this approach almost every user will have to use two schemes instead
of one - one for operational data (PARTITIONED) and one for reference data
(REPLICATED). No conventional database works this way.

Vladimir.

On Fri, Jan 13, 2017 at 9:18 PM, Dmitriy Setrakyan 
wrote:

> Vova,
>
> I will join Sergi here. It seems like "schema = cache" will take care of
> all different configuration properties required for different groups of
> caches. In addition, it cleanly maps into current Ignite architecture. We
> will need to have a very strong reason to move away from it.
>
> D.
>
> On Fri, Jan 13, 2017 at 2:39 AM, Vladimir Ozerov 
> wrote:
>
> > Correct, it worked, because Ignite has never had real database use case
> in
> > mind. Unfortunately, if our global plans go as expected, it will not work
> > for Ignite 2.x+.
> >
> > On Fri, Jan 13, 2017 at 11:53 AM, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >
> > wrote:
> >
> > > Lets move on with SQL schema == Ignite cache. It worked always like
> > this, I
> > > see no reasons to change this.
> > >
> > > Sergi
> > >
> > > 2017-01-13 11:20 GMT+03:00 Vladimir Ozerov :
> > >
> > > > "Tablespace" (Oracle, PostgreSQL) is what maps better than "schema"
> to
> > > our
> > > > cache. But not ideally still.
> > > >
> > > > On Fri, Jan 13, 2017 at 11:10 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Alex,
> > > > >
> > > > > Currently Ignite is not used as database. It is used as search
> > engine -
> > > > > several types, several tables, several joins. This is why having
> > > "SCHEMA
> > > > ==
> > > > > cache" was never a problem. Users have never build complex SQL
> > > > applications
> > > > > on top of Ignite. But we are going towards database. And my
> question
> > > > stands
> > > > > still - suppose it is Y2019, how is user going to migrate his
> > database
> > > > > containing 20-30-50-100 tables in a single schema in Oracle to
> > Ignite?
> > > > >
> > > > > Single cache for all tables? Doens't work - not flexible. Users
> will
> > > > > definitely require different cache modes, different co-location
> > rules,
> > > > > different number of backups, etc..
> > > > > Schema per table? Doesn't work either - unmanageable and not
> > convenient
> > > > > for users even for relatively small databases.
> > > > >
> > > > > From user perspective schema is logical grouping of database
> objects,
> > > > > nothing more.
> > > > >
> > > > > For Ignite schema could be a logical group of resources (nodes,
> > memory
> > > > > pools, caches, etc.). And multiple tables over multiple caches
> should
> > > > > reside in it. To the contrast, table definition governs how data is
> > > > stored.
> > > > > This is similar to, for example, MySQL approach, where you define
> how
> > > you
> > > > > store data on per-table level, and on schema level you define only
> > > minor
> > > > > things like collation.
> > > > >
> > > > > Vladimir.
> > > > >
> > > > >
> > > > > On Fri, Jan 13, 2017 at 10:33 AM, Alexander Paschenko <
> > > > > alexander.a.pasche...@gmail.com> wrote:
> > > > >
> > > > >> Vova,
> > > > >>
> > > > >> 2017-01-13 4:56 GMT+08:00 Vladimir Ozerov :
> > > > >> > I am not quite sure I understand the idea of "SCHEMA == cache".
> > > > Consider
> > > > >> > some small database with, say, ~30 tables. And user wants to
> > migrate
> > > > to
> > > > >> > Ignite. How is he supposed to do so? 30 schemas leading to
> rewrite
> > > of
> > > > >> all
> > > > >> > his SQL scripts? Or 30 key-value pairs in a single cache leading
> > to
> > > > >> lack of
> > > > >> > flexibility and performance problems?
> > > > >>
> > > > >> But currently schema *is* semantically equal to cache while table
> is
> > > > >> equal to type descriptor (i.e. type of stored entities), nothing
> new
> > > > >> here.
> > > > >>
> > > > >> Say, in single cache we may have entities of types Person and
> > > > >> Organization, those map to two table