Recap of Ignite ML presentation at ApacheCon

[Denis Magda]

Hey Ignite dev and user communities,

Those of you who are passionate about ML and Ignite ML in particular, we
continue spreading the word about this so capable module.

Had an opportunity to join ApacheCon this year and here is a
professionally written recap on my talk:
https://www.infoq.com/news/2019/09/continuous-deep-learning-ignite

Slides for those of you who are curious or keep promoting Ignite ML in
other parts of the world (ping me if you need PPT for your presentations):
https://www.slideshare.net/DenisMagda/continuous-machine-and-deep-learning-with-apache-ignite

-
Denis

Re: Ignite query performance with lots of joins

[Denis Magda]

Ignite SQL engine cannot be considered as a competitor (in a single-node
scenario) to old-good RDBMS like Postgres or MySQL. Those DBs have been
being developed for decades and are optimized for single-machine
deployments. While our efforts (Ignite community) were put into distributed
optimizations when an application has to scale out and utilize RAM in the
distributed fashion. That's why we have that recommendation to avoid any
misunderstanding. Probably, it needs to be rewritten a bit for more clarity.

Btw, if you are around Silicon Valley, step by my session at PostgresConf
this week:
https://postgresconf.org/conferences/SV2019/program/proposals/postgresql-with-in-memory-computing-faster-transactions-and-analytics-d6bb1225-4721-46ce-beda-e44f5e7c333e

As for your specific case, I think that a primary bottleneck is a number of
JOINs. It makes sense to rewrite this query first and then scale-out for
bigger benefits.

Ivan Pavluknin, Stan Lukyanov, Pavel Vinokurov, could you folks please
check this thread out and this repo suggesting any optimizations?
https://github.com/spoutnik-be/h2-ignite-perf.git


-
Denis


On Mon, Sep 16, 2019 at 6:08 AM spoutnik_be  wrote:

> About the quote on Stackoverflow:
> "Ignite/GridGain is optimized for multi-nodes deployments with RAM as a
> primary storage. Don’t try to compare a single-node GridGain cluster to a
> relational database that was optimized for such single-node configurations.
> You should deploy a multi-node GridGain cluster with the whole copy of data
> in RAM."
>
> Not sure how to interpret the above statement. The support for SQL is an
> attractive feature of Ignite/Gridgain, but if it doesn't perform on a
> single
> node with little data I don't see how it will perform on a multi-node
> cluster.
>
> What would be then your recommendation? Should we implement a SQL converter
> to translate queries into something else Ignite could run faster?
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: [IGNITE-9836] Invalid check of ea java versions

[Denis Magda]

Stephen,

Thanks for sending the patch! Seems that Igniters are already actively
reviewing it in JIRA.

-
Denis


On Mon, Sep 16, 2019 at 7:03 AM Stephen Darlington <
stephen.darling...@gridgain.com> wrote:

> Hi,
>
> Would someone mind taking a quick look at this ticket? Basically, a clean
> download of Ignite won’t start if the version of Java you’re using has a
> number like “java version "1.8.0_202-ea””. (This is the default if you get
> your JDK using Homebrew on a Mac.)
>
> > https://issues.apache.org/jira/browse/IGNITE-9836 <
> https://issues.apache.org/jira/browse/IGNITE-9836>
>
> This has been bugging me for ages and now that I look at it I find that
> there’s already a tiny, working patch available.
>
> Regards,
> Stephen

[RESULT][VOTE] Release Apache Ignite 2.7.6-rc2

[Alexey Goncharuk]

The vote for the new release candidate is closed now.

Vote result:
Vote passed with 7 votes +1 (5 binding +1 votes), no 0 and no -1 votes.

+1 votes:
Alexey Zinoviev
Ilya Kasnacheev (binding)
Maxim Muzafarow
Denis Magda (binding)
Pavel Tupitsyn (binding)
Alexey Kuznetsov (binding)
Andrey Gura (binding)

Vote thread:
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-Apache-Ignite-2-7-6-rc2-td43468.html

RE: Unsubscribe

[Alexandr Shapkin]

Hello,  

Please,  send an email to user-unsubscr...@ignite.apache.org and 
dev-unsubscr...@ignite.apache.org

More details https://ignite.apache.org/community/resources.html#mail-lists

From: Deepa Kolwalkar
Sent: Monday, September 16, 2019 12:37 PM
To: u...@ignite.apache.org; dev@ignite.apache.org
Subject: Unsubscribe

=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Nikolay Izhikov]

> From my point of view is still task of current issue

What is exactly the task of current issue?
We have fully ready `Walkers` for current system views.
They are correct and easy to use and maintain.

The current PR is already relatively big.
I don't think we should overcomplicate it with the autocompile code.

> But views and metrics are not the same entities.

OK, this is the main difference in our view.

My arguments are following:

1. We exports information about node state to Ignite administrator.
2. Metrics and system view column is the different representation of the same 
data.
   In one case it's more convinient to watch over 'sytem view', in other make a 
pretty chart from metric values.
3. With single manager we obtain more clear solution.

So, I think, one manager is enough for both entities.

Maybe there is anyone else, who want to participate in this discussion?

В Пн, 16/09/2019 в 18:01 +0300, Andrey Gura пишет:
> > I think, we should improve Ignite step by step.
> 
> From my point of view is still task of current issue - system views,
> that is exactly one step.
> 
> > > What is part of exported data?
> > Some SPI implementation(JMX, SQL view) exports both metrics and views.
> > Some exports only metrics.
> > It's fine to me.
> > What do you think?
> 
> As I already wrote early metrics and views are different things and
> they should not be exported by the same SPIs. This approach allows
> strongly specify SPI responsibility and excludes problems in reasoning
> about it. This SPI exports metrics and views while another only
> metrics. Why I should care about it? All I want it get SPI that export
> only data that I actually need.
> 
> > I don't propose that.
> > What I propose is to have one manager for the same entities.
> 
> But views and metrics are not the same entities.
> 
> > Please, don't overact my words.
> 
> Just some hyberbole, no more.
> 
> On Mon, Sep 16, 2019 at 4:46 PM Nikolay Izhikov  wrote:
> > 
> > > Why? It is still part of the same task. Master branch should not see
> > > intermediate changes.
> > 
> > I don't propose any "intermediate" feature.
> > We would have full feature of "system views" after merge.
> > 
> > I think, we should improve Ignite step by step.
> > Why we should postpone merge of "system views" because you prefer 
> > auto-compile approache comparing auto-generate one?
> > The product will stay the same.
> > Nothing will change from the public API point of view or ongoing 
> > development complexity.
> > 
> > It's a "nice to have" feature.
> > And we will have it, shortly.
> > I'll take care of it.
> > 
> > > What is part of exported data?
> > 
> > Some SPI implementation(JMX, SQL view) exports both metrics and views.
> > Some exports only metrics.
> > 
> > It's fine to me.
> > 
> > What do you think?
> > 
> > > First, views and metrics are entities from different worlds/domains.
> > 
> > It's a one domain entities for me.
> > They reflects current state of the node.
> > 
> > > We can live with one manager for absolutely all entities in the system
> > > but we don't do it, right? :)
> > 
> > I don't propose that.
> > What I propose is to have one manager for the same entities.
> > 
> > Please, don't overact my words.
> > 
> > В Пн, 16/09/2019 в 16:24 +0300, Andrey Gura пишет:
> > > > > Views have wider meaning than metrics.
> > > > 
> > > > Yes! I agree, that's why I wrote 'extension' :)
> > > 
> > > No, no, no. Wider meaning isn't equal to extension :)
> > > 
> > > > > IMO using the same code at
> > > > > runtime for view generation is better approach.
> > > > 
> > > > OK for me.
> > > > Let's do it in another ticket?
> > > > I will create one.
> > > 
> > > Why? It is still part of the same task. Master branch should not see
> > > intermediate changes.
> > > 
> > > > Seems, it's OK if some SPI implementation supports only part of 
> > > > exported data.
> > > 
> > > What is part of exported data? I understand why we have to export
> > > metrics but defineitely have no idea why views should be exported
> > > through out any special SPI.
> > > 
> > > > Are we use "lists" or "view" term? :)
> > > 
> > > Views for our task. I mean lists in general sense.
> > > 
> > > > > We can have single manager for metrics and views.
> > > > > Why do we need one more manager in the system?
> > > > > We can live without it.
> > > 
> > > First, views and metrics are entities from different worlds/domains.
> > > Second, we will have less conflicts on GridMetricManager because we
> > > are still working on metrics and views concurrently.
> > > We can live with one manager for absolutely all entities in the system
> > > but we don't do it, right? :)
> > > 
> > > On Mon, Sep 16, 2019 at 2:52 PM Nikolay Izhikov  
> > > wrote:
> > > > 
> > > > > Views have wider meaning than metrics.
> > > > 
> > > > Yes! I agree, that's why I wrote 'extension' :)
> > > > 
> > > > > IMO using the same code at
> > > > > runtime for view generation is better approach.
> > > > 
> > > > OK for 

[jira] [Created] (IGNITE-12174) NPE during expiration of spatial data when durable memory is active

[Jira]

Jan Schäfer created IGNITE-12174:


 Summary: NPE during expiration of spatial data when durable memory 
is active
 Key: IGNITE-12174
 URL: https://issues.apache.org/jira/browse/IGNITE-12174
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.7.5
 Environment: Ignite 2.7.5
 * ignite-core
 * ignite-indexing
 * ignite-geospatial

Durable memory activated

Geospatial indexing in use
Reporter: Jan Schäfer
 Attachments: IgniteBug.java

I get a NullPointerException when I restart my Ignite instance with durable 
memory activated and geospatial data in use:

{code:java}
SEVERE: Critical system error detected. Will be handled accordingly to 
configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, 
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext 
[type=SYSTEM_WORKER_TERMINATION, err=java.lang.NullPointerException]]SEVERE: 
Critical system error detected. Will be handled accordingly to configured 
handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, 
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext 
[type=SYSTEM_WORKER_TERMINATION, 
err=java.lang.NullPointerException]]java.lang.NullPointerException at 
org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.remove(GridH2SpatialIndex.java:249)
 at 
org.apache.ignite.internal.processors.query.h2.opt.GridH2SpatialIndex.removex(GridH2SpatialIndex.java:263)
 at 
org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.remove(GridH2Table.java:525)
 at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.remove(IgniteH2Indexing.java:801)
 at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.remove(GridQueryProcessor.java:2557)
 at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.remove(GridCacheQueryManager.java:435)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishRemove(IgniteCacheOffheapManagerImpl.java:2736)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.remove(IgniteCacheOffheapManagerImpl.java:2713)
 at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.remove(GridCacheOffheapManager.java:2131)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.remove(IgniteCacheOffheapManagerImpl.java:663)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.removeValue(GridCacheMapEntry.java:4441)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onExpired(GridCacheMapEntry.java:4134)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4056)
 at 
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:61)
 at 
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:52)
 at 
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:38)
 at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2413)
 at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2343)
 at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:980)
 at 
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:207)
 at 
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) 
at java.lang.Thread.run(Thread.java:748)
{code}

Minimal reproducing code sample is attached:

# start IgniteBug one time
# wait some seconds
# start IgniteBug a second time
# see NullPointerException from above



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Andrey Gura]

> I think, we should improve Ignite step by step.

>From my point of view is still task of current issue - system views,
that is exactly one step.

>> What is part of exported data?

> Some SPI implementation(JMX, SQL view) exports both metrics and views.
> Some exports only metrics.

> It's fine to me.

> What do you think?

As I already wrote early metrics and views are different things and
they should not be exported by the same SPIs. This approach allows
strongly specify SPI responsibility and excludes problems in reasoning
about it. This SPI exports metrics and views while another only
metrics. Why I should care about it? All I want it get SPI that export
only data that I actually need.

> I don't propose that.
> What I propose is to have one manager for the same entities.

But views and metrics are not the same entities.

> Please, don't overact my words.

Just some hyberbole, no more.

On Mon, Sep 16, 2019 at 4:46 PM Nikolay Izhikov  wrote:
>
> > Why? It is still part of the same task. Master branch should not see
> > intermediate changes.
>
> I don't propose any "intermediate" feature.
> We would have full feature of "system views" after merge.
>
> I think, we should improve Ignite step by step.
> Why we should postpone merge of "system views" because you prefer 
> auto-compile approache comparing auto-generate one?
> The product will stay the same.
> Nothing will change from the public API point of view or ongoing development 
> complexity.
>
> It's a "nice to have" feature.
> And we will have it, shortly.
> I'll take care of it.
>
> > What is part of exported data?
>
> Some SPI implementation(JMX, SQL view) exports both metrics and views.
> Some exports only metrics.
>
> It's fine to me.
>
> What do you think?
>
> > First, views and metrics are entities from different worlds/domains.
>
> It's a one domain entities for me.
> They reflects current state of the node.
>
> > We can live with one manager for absolutely all entities in the system
> > but we don't do it, right? :)
>
> I don't propose that.
> What I propose is to have one manager for the same entities.
>
> Please, don't overact my words.
>
> В Пн, 16/09/2019 в 16:24 +0300, Andrey Gura пишет:
> > > > Views have wider meaning than metrics.
> > > Yes! I agree, that's why I wrote 'extension' :)
> >
> > No, no, no. Wider meaning isn't equal to extension :)
> >
> > > > IMO using the same code at
> > > > runtime for view generation is better approach.
> > > OK for me.
> > > Let's do it in another ticket?
> > > I will create one.
> >
> > Why? It is still part of the same task. Master branch should not see
> > intermediate changes.
> >
> > > Seems, it's OK if some SPI implementation supports only part of exported 
> > > data.
> >
> > What is part of exported data? I understand why we have to export
> > metrics but defineitely have no idea why views should be exported
> > through out any special SPI.
> >
> > > Are we use "lists" or "view" term? :)
> >
> > Views for our task. I mean lists in general sense.
> >
> > > > We can have single manager for metrics and views.
> > > > Why do we need one more manager in the system?
> > > > We can live without it.
> >
> > First, views and metrics are entities from different worlds/domains.
> > Second, we will have less conflicts on GridMetricManager because we
> > are still working on metrics and views concurrently.
> > We can live with one manager for absolutely all entities in the system
> > but we don't do it, right? :)
> >
> > On Mon, Sep 16, 2019 at 2:52 PM Nikolay Izhikov  wrote:
> > >
> > > > Views have wider meaning than metrics.
> > >
> > > Yes! I agree, that's why I wrote 'extension' :)
> > >
> > > > IMO using the same code at
> > > > runtime for view generation is better approach.
> > >
> > > OK for me.
> > > Let's do it in another ticket?
> > > I will create one.
> > >
> > > > What is the reaal life uses cases for exporting views?
> > > > Is there any database which exports some lists to somewhere?
> > > > Especially on push based model, not on demand.
> > >
> > > I don't know such dbms.
> > > Seems, it's OK if some SPI implementation supports only part of exported 
> > > data.
> > >
> > > Are we use "lists" or "view" term? :)
> > >
> > > My point is:
> > >
> > > We can have single manager for metrics and views.
> > > Why do we need one more manager in the system?
> > > We can live without it.
> > >
> > > В Пн, 16/09/2019 в 13:53 +0300, Andrey Gura пишет:
> > > > Hi,
> > > >
> > > > > > I think akso that GridMetricManager is bad candidate for lists 
> > > > > > (system views) management.
> > > > >
> > > > > For me, it seems that views and metrics is extension of one another.
> > > > > If the user want to know some instant values(cache put count, cahe 
> > > > > get latency) he use metrics
> > > > > and one want to know list of running SQL queries one take a look into 
> > > > > views.
> > > >
> > > > Views are about system state and they answer to question "what
> > > > entities exist in the system 

[IGNITE-9836] Invalid check of ea java versions

[Stephen Darlington]

Hi,

Would someone mind taking a quick look at this ticket? Basically, a clean 
download of Ignite won’t start if the version of Java you’re using has a number 
like “java version "1.8.0_202-ea””. (This is the default if you get your JDK 
using Homebrew on a Mac.)

> https://issues.apache.org/jira/browse/IGNITE-9836 
> 

This has been bugging me for ages and now that I look at it I find that there’s 
already a tiny, working patch available.

Regards,
Stephen

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Nikolay Izhikov]

> Why? It is still part of the same task. Master branch should not see
> intermediate changes.

I don't propose any "intermediate" feature.
We would have full feature of "system views" after merge.

I think, we should improve Ignite step by step.
Why we should postpone merge of "system views" because you prefer auto-compile 
approache comparing auto-generate one?
The product will stay the same.
Nothing will change from the public API point of view or ongoing development 
complexity.

It's a "nice to have" feature.
And we will have it, shortly.
I'll take care of it.

> What is part of exported data?

Some SPI implementation(JMX, SQL view) exports both metrics and views.
Some exports only metrics.

It's fine to me.

What do you think?

> First, views and metrics are entities from different worlds/domains.

It's a one domain entities for me.
They reflects current state of the node.

> We can live with one manager for absolutely all entities in the system
> but we don't do it, right? :)

I don't propose that.
What I propose is to have one manager for the same entities.

Please, don't overact my words.

В Пн, 16/09/2019 в 16:24 +0300, Andrey Gura пишет:
> > > Views have wider meaning than metrics.
> > Yes! I agree, that's why I wrote 'extension' :)
> 
> No, no, no. Wider meaning isn't equal to extension :)
> 
> > > IMO using the same code at
> > > runtime for view generation is better approach.
> > OK for me.
> > Let's do it in another ticket?
> > I will create one.
> 
> Why? It is still part of the same task. Master branch should not see
> intermediate changes.
> 
> > Seems, it's OK if some SPI implementation supports only part of exported 
> > data.
> 
> What is part of exported data? I understand why we have to export
> metrics but defineitely have no idea why views should be exported
> through out any special SPI.
> 
> > Are we use "lists" or "view" term? :)
> 
> Views for our task. I mean lists in general sense.
> 
> > > We can have single manager for metrics and views.
> > > Why do we need one more manager in the system?
> > > We can live without it.
> 
> First, views and metrics are entities from different worlds/domains.
> Second, we will have less conflicts on GridMetricManager because we
> are still working on metrics and views concurrently.
> We can live with one manager for absolutely all entities in the system
> but we don't do it, right? :)
> 
> On Mon, Sep 16, 2019 at 2:52 PM Nikolay Izhikov  wrote:
> > 
> > > Views have wider meaning than metrics.
> > 
> > Yes! I agree, that's why I wrote 'extension' :)
> > 
> > > IMO using the same code at
> > > runtime for view generation is better approach.
> > 
> > OK for me.
> > Let's do it in another ticket?
> > I will create one.
> > 
> > > What is the reaal life uses cases for exporting views?
> > > Is there any database which exports some lists to somewhere?
> > > Especially on push based model, not on demand.
> > 
> > I don't know such dbms.
> > Seems, it's OK if some SPI implementation supports only part of exported 
> > data.
> > 
> > Are we use "lists" or "view" term? :)
> > 
> > My point is:
> > 
> > We can have single manager for metrics and views.
> > Why do we need one more manager in the system?
> > We can live without it.
> > 
> > В Пн, 16/09/2019 в 13:53 +0300, Andrey Gura пишет:
> > > Hi,
> > > 
> > > > > I think akso that GridMetricManager is bad candidate for lists 
> > > > > (system views) management.
> > > > 
> > > > For me, it seems that views and metrics is extension of one another.
> > > > If the user want to know some instant values(cache put count, cahe get 
> > > > latency) he use metrics
> > > > and one want to know list of running SQL queries one take a look into 
> > > > views.
> > > 
> > > Views are about system state and they answer to question "what
> > > entities exist in the system (caches)?" or "what processes are
> > > executing by system (tx, queries)?"
> > > Metrics are about system behavior in some retrospective. They answers
> > > on questions how system behaves?
> > > 
> > > Views have wider meaning than metrics.
> > > 
> > > > > Code generation for walkers is also redundant.
> > > > 
> > > > If you don't want, you can not use it.
> > > > I find it pretty usefull during development.
> > > 
> > > I talk not about wishes of somebody ) Moreover, if it will depend on
> > > wishes it potentially can lead to misusing. IMO using the same code at
> > > runtime for view generation is better approach.
> > > 
> > > > > I really don't understand why we should export system views content
> > > > > (especially periodically). Real life use case is take view content on
> > > > > demand. So we should have public API for it, SQL API and JMX. There is
> > > > > no need any exporters.
> > > > 
> > > > What if we want to export lists to log or via http, etc?
> > > 
> > > If we will have public API for views then we can use REST for access
> > > to this API. Also you can use public API directly. What is the reaal
> > > life uses cases 

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Andrey Gura]

>> Views have wider meaning than metrics.

> Yes! I agree, that's why I wrote 'extension' :)

No, no, no. Wider meaning isn't equal to extension :)

>> IMO using the same code at
>> runtime for view generation is better approach.

> OK for me.
> Let's do it in another ticket?
> I will create one.

Why? It is still part of the same task. Master branch should not see
intermediate changes.

> Seems, it's OK if some SPI implementation supports only part of exported data.

What is part of exported data? I understand why we have to export
metrics but defineitely have no idea why views should be exported
through out any special SPI.

> Are we use "lists" or "view" term? :)

Views for our task. I mean lists in general sense.

>> We can have single manager for metrics and views.
>> Why do we need one more manager in the system?
>> We can live without it.

First, views and metrics are entities from different worlds/domains.
Second, we will have less conflicts on GridMetricManager because we
are still working on metrics and views concurrently.
We can live with one manager for absolutely all entities in the system
but we don't do it, right? :)

On Mon, Sep 16, 2019 at 2:52 PM Nikolay Izhikov  wrote:
>
> > Views have wider meaning than metrics.
>
> Yes! I agree, that's why I wrote 'extension' :)
>
> > IMO using the same code at
> > runtime for view generation is better approach.
>
> OK for me.
> Let's do it in another ticket?
> I will create one.
>
> > What is the reaal life uses cases for exporting views?
> > Is there any database which exports some lists to somewhere?
> > Especially on push based model, not on demand.
>
> I don't know such dbms.
> Seems, it's OK if some SPI implementation supports only part of exported data.
>
> Are we use "lists" or "view" term? :)
>
> My point is:
>
> We can have single manager for metrics and views.
> Why do we need one more manager in the system?
> We can live without it.
>
> В Пн, 16/09/2019 в 13:53 +0300, Andrey Gura пишет:
> > Hi,
> >
> > > > I think akso that GridMetricManager is bad candidate for lists (system 
> > > > views) management.
> > > For me, it seems that views and metrics is extension of one another.
> > > If the user want to know some instant values(cache put count, cahe get 
> > > latency) he use metrics
> > > and one want to know list of running SQL queries one take a look into 
> > > views.
> >
> > Views are about system state and they answer to question "what
> > entities exist in the system (caches)?" or "what processes are
> > executing by system (tx, queries)?"
> > Metrics are about system behavior in some retrospective. They answers
> > on questions how system behaves?
> >
> > Views have wider meaning than metrics.
> >
> > > > Code generation for walkers is also redundant.
> > > If you don't want, you can not use it.
> > > I find it pretty usefull during development.
> >
> > I talk not about wishes of somebody ) Moreover, if it will depend on
> > wishes it potentially can lead to misusing. IMO using the same code at
> > runtime for view generation is better approach.
> >
> > > > I really don't understand why we should export system views content
> > > > (especially periodically). Real life use case is take view content on
> > > > demand. So we should have public API for it, SQL API and JMX. There is
> > > > no need any exporters.
> > > What if we want to export lists to log or via http, etc?
> >
> > If we will have public API for views then we can use REST for access
> > to this API. Also you can use public API directly. What is the reaal
> > life uses cases for exporting views? Is there any database which
> > exports some lists to somewhere? Especially on push based model, not
> > on demand.
> >
> > On Fri, Sep 13, 2019 at 4:36 PM Nikolay Izhikov  wrote:
> > >
> > > Hello, Andrey.
> > >
> > > > I really don't like name MonitoringList. First of all because it isn't
> > > > about monitoring at all while can be useful for monitoring purposes.
> > > > We already have SQL system views and I think that system view is good
> > > > candidate for naming of new entity.
> > >
> > > SystemView is OK for me.
> > > I will rename enity in the PR.
> > >
> > > > I think akso that GridMetricManager is bad candidate for lists (system 
> > > > views) management.
> > >
> > > For me, it seems that views and metrics is extension of one another.
> > > If the user want to know some instant values(cache put count, cahe get 
> > > latency) he use metrics
> > > and one want to know list of running SQL queries one take a look into 
> > > views.
> > >
> > > > There is no any interaction with lists on hot path of code flow
> > > > and there is no any performance impact.
> > >
> > > OK, let's remove it.
> > >
> > > > Code generation for walkers is also redundant.
> > >
> > > If you don't want, you can not use it.
> > > I find it pretty usefull during development.
> > >
> > > > I really don't understand why we should export system views content
> > > > (especially periodically). Real 

[jira] [Created] (IGNITE-12173) Fix http links

[Peter Ivanov (Jira)]

Peter Ivanov created IGNITE-12173:
-

 Summary: Fix http links
 Key: IGNITE-12173
 URL: https://issues.apache.org/jira/browse/IGNITE-12173
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.7.5
Reporter: Peter Ivanov
Assignee: Peter Ivanov
 Fix For: 2.8


h3. [Apache Ignite|https://github.com/apache]
# 
{{[http://h2database.com/m2-repo|https://github.com/apache/ignite/blob/6e9ec03659e5f35c0f18605079fbced1e42ad130/parent/pom.xml#L233]}}
# 
{{[http://hadoop.apache.org/docs/current/api/|https://github.com/apache/ignite/blob/6e9ec03659e5f35c0f18605079fbced1e42ad130/parent/pom.xml#L339]}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (IGNITE-12172) [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite

[Andrey Gura (Jira)]

Andrey Gura created IGNITE-12172:


 Summary: [IEP-35] OpenCensusMetricExporterSpiTest isn't added to 
any test suite
 Key: IGNITE-12172
 URL: https://issues.apache.org/jira/browse/IGNITE-12172
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
 Fix For: 2.8


{{OpenCensusMetricExporterSpiTest}} isn't added to any test suite. As result 
tests do not run on TC.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Nikolay Izhikov]

> Views have wider meaning than metrics.

Yes! I agree, that's why I wrote 'extension' :)

> IMO using the same code at
> runtime for view generation is better approach.

OK for me.
Let's do it in another ticket? 
I will create one.

> What is the reaal life uses cases for exporting views?
> Is there any database which exports some lists to somewhere?
> Especially on push based model, not on demand.

I don't know such dbms.
Seems, it's OK if some SPI implementation supports only part of exported data.

Are we use "lists" or "view" term? :)

My point is: 

We can have single manager for metrics and views.
Why do we need one more manager in the system?
We can live without it.

В Пн, 16/09/2019 в 13:53 +0300, Andrey Gura пишет:
> Hi,
> 
> > > I think akso that GridMetricManager is bad candidate for lists (system 
> > > views) management.
> > For me, it seems that views and metrics is extension of one another.
> > If the user want to know some instant values(cache put count, cahe get 
> > latency) he use metrics
> > and one want to know list of running SQL queries one take a look into views.
> 
> Views are about system state and they answer to question "what
> entities exist in the system (caches)?" or "what processes are
> executing by system (tx, queries)?"
> Metrics are about system behavior in some retrospective. They answers
> on questions how system behaves?
> 
> Views have wider meaning than metrics.
> 
> > > Code generation for walkers is also redundant.
> > If you don't want, you can not use it.
> > I find it pretty usefull during development.
> 
> I talk not about wishes of somebody ) Moreover, if it will depend on
> wishes it potentially can lead to misusing. IMO using the same code at
> runtime for view generation is better approach.
> 
> > > I really don't understand why we should export system views content
> > > (especially periodically). Real life use case is take view content on
> > > demand. So we should have public API for it, SQL API and JMX. There is
> > > no need any exporters.
> > What if we want to export lists to log or via http, etc?
> 
> If we will have public API for views then we can use REST for access
> to this API. Also you can use public API directly. What is the reaal
> life uses cases for exporting views? Is there any database which
> exports some lists to somewhere? Especially on push based model, not
> on demand.
> 
> On Fri, Sep 13, 2019 at 4:36 PM Nikolay Izhikov  wrote:
> > 
> > Hello, Andrey.
> > 
> > > I really don't like name MonitoringList. First of all because it isn't
> > > about monitoring at all while can be useful for monitoring purposes.
> > > We already have SQL system views and I think that system view is good
> > > candidate for naming of new entity.
> > 
> > SystemView is OK for me.
> > I will rename enity in the PR.
> > 
> > > I think akso that GridMetricManager is bad candidate for lists (system 
> > > views) management.
> > 
> > For me, it seems that views and metrics is extension of one another.
> > If the user want to know some instant values(cache put count, cahe get 
> > latency) he use metrics
> > and one want to know list of running SQL queries one take a look into views.
> > 
> > > There is no any interaction with lists on hot path of code flow
> > > and there is no any performance impact.
> > 
> > OK, let's remove it.
> > 
> > > Code generation for walkers is also redundant.
> > 
> > If you don't want, you can not use it.
> > I find it pretty usefull during development.
> > 
> > > I really don't understand why we should export system views content
> > > (especially periodically). Real life use case is take view content on
> > > demand. So we should have public API for it, SQL API and JMX. There is
> > > no need any exporters.
> > 
> > What if we want to export lists to log or via http, etc?
> > 
> > > Also it would be great to involve more people to this discussion.
> > 
> > Any feedback are welcome!
> > 
> > 
> > В Пт, 13/09/2019 в 15:13 +0300, Andrey Gura пишет:
> > > Nikolay,
> > > 
> > > thanks a lot for clarification! I added some comments to Upsource review 
> > > [1].
> > > 
> > > Here I want to discuss some high-level issues.
> > > 
> > > 1. Naming
> > > 
> > > "There are only two hard things in Computer Science: cache
> > > invalidation and naming things."
> > > -- Phil Karlton
> > > 
> > > I really don't like name MonitoringList. First of all because it isn't
> > > about monitoring at all while can be useful for monitoring purposes.
> > > 
> > > We already have SQL system views and I think that system view is good
> > > candidate for naming of new entity. As result we will have consistent
> > > naming which better describes domain.
> > > 
> > > I think akso that GridMetricManager is bad candidate for lists (system
> > > views) management. Because it isn't about metrics. May be new
> > > SystemViewManager will better fit to this purposes.
> > > 
> > > 2. Management
> > > 
> > > Lists (aka system views) have life cycle now. I believe 

Re: [IEP-35] Monitoring & Profiling. Phase 2

[Andrey Gura]

Hi,

>> I think akso that GridMetricManager is bad candidate for lists (system 
>> views) management.

>For me, it seems that views and metrics is extension of one another.
>If the user want to know some instant values(cache put count, cahe get 
>latency) he use metrics
>and one want to know list of running SQL queries one take a look into views.

Views are about system state and they answer to question "what
entities exist in the system (caches)?" or "what processes are
executing by system (tx, queries)?"
Metrics are about system behavior in some retrospective. They answers
on questions how system behaves?

Views have wider meaning than metrics.

>> Code generation for walkers is also redundant.

>If you don't want, you can not use it.
>I find it pretty usefull during development.

I talk not about wishes of somebody ) Moreover, if it will depend on
wishes it potentially can lead to misusing. IMO using the same code at
runtime for view generation is better approach.

>> I really don't understand why we should export system views content
>> (especially periodically). Real life use case is take view content on
>> demand. So we should have public API for it, SQL API and JMX. There is
>> no need any exporters.

> What if we want to export lists to log or via http, etc?

If we will have public API for views then we can use REST for access
to this API. Also you can use public API directly. What is the reaal
life uses cases for exporting views? Is there any database which
exports some lists to somewhere? Especially on push based model, not
on demand.

On Fri, Sep 13, 2019 at 4:36 PM Nikolay Izhikov  wrote:
>
> Hello, Andrey.
>
> > I really don't like name MonitoringList. First of all because it isn't
> > about monitoring at all while can be useful for monitoring purposes.
> > We already have SQL system views and I think that system view is good
> > candidate for naming of new entity.
>
> SystemView is OK for me.
> I will rename enity in the PR.
>
> > I think akso that GridMetricManager is bad candidate for lists (system 
> > views) management.
>
> For me, it seems that views and metrics is extension of one another.
> If the user want to know some instant values(cache put count, cahe get 
> latency) he use metrics
> and one want to know list of running SQL queries one take a look into views.
>
> > There is no any interaction with lists on hot path of code flow
> > and there is no any performance impact.
>
> OK, let's remove it.
>
> > Code generation for walkers is also redundant.
>
> If you don't want, you can not use it.
> I find it pretty usefull during development.
>
> > I really don't understand why we should export system views content
> > (especially periodically). Real life use case is take view content on
> > demand. So we should have public API for it, SQL API and JMX. There is
> > no need any exporters.
>
> What if we want to export lists to log or via http, etc?
>
> > Also it would be great to involve more people to this discussion.
>
> Any feedback are welcome!
>
>
> В Пт, 13/09/2019 в 15:13 +0300, Andrey Gura пишет:
> > Nikolay,
> >
> > thanks a lot for clarification! I added some comments to Upsource review 
> > [1].
> >
> > Here I want to discuss some high-level issues.
> >
> > 1. Naming
> >
> > "There are only two hard things in Computer Science: cache
> > invalidation and naming things."
> > -- Phil Karlton
> >
> > I really don't like name MonitoringList. First of all because it isn't
> > about monitoring at all while can be useful for monitoring purposes.
> >
> > We already have SQL system views and I think that system view is good
> > candidate for naming of new entity. As result we will have consistent
> > naming which better describes domain.
> >
> > I think akso that GridMetricManager is bad candidate for lists (system
> > views) management. Because it isn't about metrics. May be new
> > SystemViewManager will better fit to this purposes.
> >
> > 2. Management
> >
> > Lists (aka system views) have life cycle now. I believe that it is
> > redundant functionality. There is no any reason for enabling/disabling
> > lists. There is no any interaction with lists on hot path of code flow
> > and there is no any performance impact.
> >
> > So lists management can be reduced to lists creation and registration
> > operations (which executes only on node start).
> >
> > 3. Code generation
> >
> > Code generation for walkers is also redundant. Amount of system views
> > in the system is strongly limited (units not dozens) so it is easier
> > to change walker by hand literally than navigate to code generator and
> > run it. Moreover, first you should add Order annotation in the proper
> > place and it make generator practically useless.
> >
> > If you still see benefit that can bring Order annotation you can use
> > reflection. Motivation is simple, system views are on not hot path and
> > I expected that API for system views will not called frequently.
> >
> > 4. Export
> >
> > I really don't 

[jira] [Created] (IGNITE-12171) Remove the MetricRegistry.remove(String name) method.

[Amelchev Nikita (Jira)]

Amelchev Nikita created IGNITE-12171:


 Summary: Remove the MetricRegistry.remove(String name) method.
 Key: IGNITE-12171
 URL: https://issues.apache.org/jira/browse/IGNITE-12171
 Project: Ignite
  Issue Type: Task
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita
 Fix For: 2.8


I suggest removing the unused {{MetricRegistry.remove(String name)}} method due 
to we can remove a subset of metrics using the 
{{GridMetricManager.remove(String regName)}} method with notifying mechanism. 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [VOTE] Release Apache Ignite 2.7.6-rc2

[Andrey Gura]

+1 (binding)

On Mon, Sep 16, 2019 at 12:50 PM Alexey Kuznetsov  wrote:
>
> +1 (binding)
>
> On Mon, Sep 16, 2019 at 3:58 PM Pavel Tupitsyn  wrote:
>
> > +1 (binding)
> >
> > On Fri, Sep 13, 2019 at 8:09 PM Denis Magda  wrote:
> >
> > > +1 (binding)
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Sep 13, 2019 at 1:30 AM Alexey Goncharuk <
> > > alexey.goncha...@gmail.com>
> > > wrote:
> > >
> > > > Dear Community,
> > > >
> > > > We have uploaded release candidate to
> > > > https://dist.apache.org/repos/dist/dev/ignite/2.7.6-rc2/
> > > >
> > > > The following staging can be used for a dependent project for testing:
> > > >
> > https://repository.apache.org/content/repositories/orgapacheignite-1471/
> > > >
> > > > This is a maintenance release for Ignite 2.7.5 addressing several
> > > usability
> > > > and stability issues.
> > > >
> > > > Tag name is 2.7.6-rc2:
> > > >
> > > >
> > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.7.6-rc2
> > > >
> > > > 2.7.6 changes:
> > > > Ignite:
> > > >   * Changed default persistence directory location when IGNITE_HOME is
> > > not
> > > > set, native persistence files will not be stored in the temp directory
> > > > anymore
> > > >   * Fixed a bug that caused a SELECT query with an equality predicate
> > on
> > > a
> > > > part of the primary compound key to return a single row even if the
> > query
> > > > matched multiple rows
> > > >   * Fixed an issue that could cause data corruption during
> > checkpointing
> > > >   * Fixed an issue where a row size was calculated incorrectly for
> > shared
> > > > cache groups, which caused a tree corruption
> > > >   * Reduced java heap footprint by optimizing
> > > GridDhtPartitionsFullMessage
> > > > maps in exchange history
> > > >   * Fixed an issue where an outdated node with a destroyed cache caused
> > > the
> > > > cluster to hang
> > > >   * Fixed a bug that made it impossible to change the inline_size
> > > property
> > > > of an existing index after it was dropped and recreated with a
> > different
> > > > value
> > > >   * Fixed an issue causing silent control.sh script fail when JAVA_HOME
> > > is
> > > > not set
> > > >   * Fixed an issue causing sporadic node failure when persistence is
> > > > enabled and WAL mmap is disabled
> > > >   * Fixed an issue causing potential PDS corruption when a node is
> > killed
> > > > during checkpoint mark phase
> > > >
> > > >   Ignite .NET:
> > > >   * Native persistence now works with a custom affinity function
> > > >   * Fixed missing CacheEntryEventType.Removed event
> > > >
> > > > Complete list of closed issues:
> > > >
> > > >
> > >
> > https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20in%20(%272.7.6%27)%20AND%20status%20IN%20(Resolved%2C%20Closed))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > >
> > > > DEVNOTES
> > > >
> > > >
> > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> > > >
> > > > RELEASE NOTES
> > > >
> > > >
> > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=810ecfd3baeeb378d868a0cf230cbe5c0fe298ea;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> > > >
> > > > Release Verification Task run
> > > >
> > > >
> > >
> > https://ci.ignite.apache.org/viewLog.html?buildId=4594148=ApacheIgniteReleaseJava8_PrepareVote4CheckRcLicensesChecksum
> > > >
> > > > The vote is formal, see voting guidelines
> > > > https://www.apache.org/foundation/voting.html
> > > >
> > > > +1 - to accept Apache Ignite 2.7.6-rc2
> > > > 0 - don't care either way
> > > > -1 - DO NOT accept Apache Ignite Ignite 2.7.6-rc2 (explain why)
> > > >
> > > > See notes on how to verify release here
> > > > https://www.apache.org/info/verification.html
> > > > and
> > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-ReleaseVerification
> > > >
> > > > This vote will be open for at least 72 hours till September 16, 09:00
> > > UTC.
> > > >
> > > >
> > >
> > https://www.timeanddate.com/countdown/to?year=2019=9=16=09=0=0=utc-1
> > > >
> > > > -- Alexey Goncharuk
> > > >
> > >
> >
>
>
> --
> Alexey Kuznetsov

[jira] [Created] (IGNITE-12170) DOC: Advanced security should be better documented

[Andrey Aleksandrov (Jira)]

Andrey Aleksandrov created IGNITE-12170:
---

 Summary: DOC: Advanced security should be better documented
 Key: IGNITE-12170
 URL: https://issues.apache.org/jira/browse/IGNITE-12170
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.7.5
Reporter: Andrey Aleksandrov
 Fix For: 2.8


[https://apacheignite.readme.io/docs/advanced-security#section-enable-authentication]

After applying the example, in the server log you can see:

[12:44:11] Security status [authentication=off, tls/ssl=off]

After that, I was able to connect the client without any username/password.

So looks like we should add into documentation the information about how it 
works. 

I checked that it worked with thin clients and web console.  But for user it 
can be not clear that authentication between nodes doesn' work out of the boxes.

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [VOTE] Release Apache Ignite 2.7.6-rc2

[Alexey Kuznetsov]

+1 (binding)

On Mon, Sep 16, 2019 at 3:58 PM Pavel Tupitsyn  wrote:

> +1 (binding)
>
> On Fri, Sep 13, 2019 at 8:09 PM Denis Magda  wrote:
>
> > +1 (binding)
> >
> > -
> > Denis
> >
> >
> > On Fri, Sep 13, 2019 at 1:30 AM Alexey Goncharuk <
> > alexey.goncha...@gmail.com>
> > wrote:
> >
> > > Dear Community,
> > >
> > > We have uploaded release candidate to
> > > https://dist.apache.org/repos/dist/dev/ignite/2.7.6-rc2/
> > >
> > > The following staging can be used for a dependent project for testing:
> > >
> https://repository.apache.org/content/repositories/orgapacheignite-1471/
> > >
> > > This is a maintenance release for Ignite 2.7.5 addressing several
> > usability
> > > and stability issues.
> > >
> > > Tag name is 2.7.6-rc2:
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.7.6-rc2
> > >
> > > 2.7.6 changes:
> > > Ignite:
> > >   * Changed default persistence directory location when IGNITE_HOME is
> > not
> > > set, native persistence files will not be stored in the temp directory
> > > anymore
> > >   * Fixed a bug that caused a SELECT query with an equality predicate
> on
> > a
> > > part of the primary compound key to return a single row even if the
> query
> > > matched multiple rows
> > >   * Fixed an issue that could cause data corruption during
> checkpointing
> > >   * Fixed an issue where a row size was calculated incorrectly for
> shared
> > > cache groups, which caused a tree corruption
> > >   * Reduced java heap footprint by optimizing
> > GridDhtPartitionsFullMessage
> > > maps in exchange history
> > >   * Fixed an issue where an outdated node with a destroyed cache caused
> > the
> > > cluster to hang
> > >   * Fixed a bug that made it impossible to change the inline_size
> > property
> > > of an existing index after it was dropped and recreated with a
> different
> > > value
> > >   * Fixed an issue causing silent control.sh script fail when JAVA_HOME
> > is
> > > not set
> > >   * Fixed an issue causing sporadic node failure when persistence is
> > > enabled and WAL mmap is disabled
> > >   * Fixed an issue causing potential PDS corruption when a node is
> killed
> > > during checkpoint mark phase
> > >
> > >   Ignite .NET:
> > >   * Native persistence now works with a custom affinity function
> > >   * Fixed missing CacheEntryEventType.Removed event
> > >
> > > Complete list of closed issues:
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20in%20(%272.7.6%27)%20AND%20status%20IN%20(Resolved%2C%20Closed))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20
> > >
> > > DEVNOTES
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> > >
> > > RELEASE NOTES
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=810ecfd3baeeb378d868a0cf230cbe5c0fe298ea;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> > >
> > > Release Verification Task run
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4594148=ApacheIgniteReleaseJava8_PrepareVote4CheckRcLicensesChecksum
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept Apache Ignite 2.7.6-rc2
> > > 0 - don't care either way
> > > -1 - DO NOT accept Apache Ignite Ignite 2.7.6-rc2 (explain why)
> > >
> > > See notes on how to verify release here
> > > https://www.apache.org/info/verification.html
> > > and
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-ReleaseVerification
> > >
> > > This vote will be open for at least 72 hours till September 16, 09:00
> > UTC.
> > >
> > >
> >
> https://www.timeanddate.com/countdown/to?year=2019=9=16=09=0=0=utc-1
> > >
> > > -- Alexey Goncharuk
> > >
> >
>


-- 
Alexey Kuznetsov

[jira] [Created] (IGNITE-12169) Dockerfile: Thin Client and REST API ports are not exposed

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12169:
---

 Summary: Dockerfile: Thin Client and REST API ports are not exposed
 Key: IGNITE-12169
 URL: https://issues.apache.org/jira/browse/IGNITE-12169
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn


Expose Thin Client (10800) and REST API (8080) ports in Dockerfiles (both 
apache-ignite and apache-ignite-net) 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: After IGNITE-12148 the Examples suite has unstable tests

[Alexey Zinoviev]

Happy to hear!

пн, 16 сент. 2019 г. в 10:44, Denis Garus :

> Alexey, I run tests two times on your branch, all are green.
> Thank you, great job!
>
> пт, 13 сент. 2019 г. в 23:04, Alexey Zinoviev :
>
> > Fixed it, please have a look and test it
> >
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
> >
> >
> > пт, 13 сент. 2019 г. в 21:16, Alexey Zinoviev :
> >
> > > Ok, got it, I've found a solution
> > >
> > > пт, 13 сент. 2019 г. в 21:09, Denis Garus :
> > >
> > >> Alexey, about agents was my assumption, and it looks like wrong.
> > >> I didn't dive so deep.
> > >>
> > >> пт, 13 сент. 2019 г. в 20:54, Alexey Zinoviev  >:
> > >>
> > >> > Could you help me recognize the difference between agents to exactly
> > >> > reproduce the issue and be sure that I fix it?
> > >> >
> > >> >
> > >> > пт, 13 сент. 2019 г. в 20:51, Alexey Zinoviev <
> zaleslaw@gmail.com
> > >:
> > >> >
> > >> > > Thank you, I'll try to fix it, ticket is here
> > >> > > https://issues.apache.org/jira/browse/IGNITE-12168
> > >> > >
> > >> > > пт, 13 сент. 2019 г. в 20:41, Denis Garus :
> > >> > >
> > >> > >> Alexey, sure.
> > >> > >> My first build today is [1], and the last build is [2].
> > >> > >> 28 tests became flaky.
> > >> > >>
> > >> > >> 1.
> > >> > >>
> > >> > >>
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4594295=IgniteTests24Java8_Examples
> > >> > >> 2.
> > >> > >>
> > >> > >>
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4595102=IgniteTests24Java8_Examples
> > >> > >>
> > >> > >> пт, 13 сент. 2019 г. в 20:25, Alexey Zinoviev <
> > >> zaleslaw@gmail.com>:
> > >> > >>
> > >> > >> > Dear @ Denis Garus
> > >> > >> >
> > >> > >> > Could you please what kind of tests became unstable?
> > >> > >> > Because I have no troubles with examples (and run them of
> course
> > >> > before
> > >> > >> > merging) and many builds have no troubles too
> > >> > >> >
> > >> > >> > Look at
> > >> > >> >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
> > >> > >> > -
> > >> > >> > it's broken on Scala component a few weeks ago
> > >> > >> >
> > >> > >> > Could you please send me exact link on the builds that were ran
> > by
> > >> > >> yourself
> > >> > >> > today
> > >> > >> >
> > >> > >> > But yes, the IGNITE-12148 could influence on many examples in
> ML
> > >> > module
> > >> > >> > because a few of resources were moved.
> > >> > >> > I will try to fix today or tomorrow (but I need more
> information
> > to
> > >> > >> > reproduce the situation)
> > >> > >> >
> > >> > >> >
> > >> > >> > пт, 13 сент. 2019 г. в 19:52, Denis Magda :
> > >> > >> >
> > >> > >> > > Alex Zinoviev, could you please double-check and confirm if
> > >> > >> IGNITE-12148
> > >> > >> > > affects the test suite?
> > >> > >> > >
> > >> > >> > > -
> > >> > >> > > Denis
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > On Fri, Sep 13, 2019 at 1:55 AM Denis Garus <
> > garus@gmail.com
> > >> >
> > >> > >> wrote:
> > >> > >> > >
> > >> > >> > > > Hello, Igniters!
> > >> > >> > > >
> > >> > >> > > > I ran two times the Examples suite [1] on the master branch
> > >> today
> > >> > >> and
> > >> > >> > get
> > >> > >> > > > different results. It looks like some tests become unstable
> > >> after
> > >> > >> > merging
> > >> > >> > > > of task IGNITE-12148 [2]. I think tests result depend on an
> > >> Agent
> > >> > >> that
> > >> > >> > > > executes suite.
> > >> > >> > > >
> > >> > >> > > > 1.
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
> > >> > >> > > >
> > >> > >> > > > 2.
> > >> > >> > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> https://github.com/apache/ignite/commit/cd5e39b783bc499837b569c2cf974a8fe308bcf2
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [VOTE] Release Apache Ignite 2.7.6-rc2

[Pavel Tupitsyn]

+1 (binding)

On Fri, Sep 13, 2019 at 8:09 PM Denis Magda  wrote:

> +1 (binding)
>
> -
> Denis
>
>
> On Fri, Sep 13, 2019 at 1:30 AM Alexey Goncharuk <
> alexey.goncha...@gmail.com>
> wrote:
>
> > Dear Community,
> >
> > We have uploaded release candidate to
> > https://dist.apache.org/repos/dist/dev/ignite/2.7.6-rc2/
> >
> > The following staging can be used for a dependent project for testing:
> > https://repository.apache.org/content/repositories/orgapacheignite-1471/
> >
> > This is a maintenance release for Ignite 2.7.5 addressing several
> usability
> > and stability issues.
> >
> > Tag name is 2.7.6-rc2:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.7.6-rc2
> >
> > 2.7.6 changes:
> > Ignite:
> >   * Changed default persistence directory location when IGNITE_HOME is
> not
> > set, native persistence files will not be stored in the temp directory
> > anymore
> >   * Fixed a bug that caused a SELECT query with an equality predicate on
> a
> > part of the primary compound key to return a single row even if the query
> > matched multiple rows
> >   * Fixed an issue that could cause data corruption during checkpointing
> >   * Fixed an issue where a row size was calculated incorrectly for shared
> > cache groups, which caused a tree corruption
> >   * Reduced java heap footprint by optimizing
> GridDhtPartitionsFullMessage
> > maps in exchange history
> >   * Fixed an issue where an outdated node with a destroyed cache caused
> the
> > cluster to hang
> >   * Fixed a bug that made it impossible to change the inline_size
> property
> > of an existing index after it was dropped and recreated with a different
> > value
> >   * Fixed an issue causing silent control.sh script fail when JAVA_HOME
> is
> > not set
> >   * Fixed an issue causing sporadic node failure when persistence is
> > enabled and WAL mmap is disabled
> >   * Fixed an issue causing potential PDS corruption when a node is killed
> > during checkpoint mark phase
> >
> >   Ignite .NET:
> >   * Native persistence now works with a custom affinity function
> >   * Fixed missing CacheEntryEventType.Removed event
> >
> > Complete list of closed issues:
> >
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20in%20(%272.7.6%27)%20AND%20status%20IN%20(Resolved%2C%20Closed))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20
> >
> > DEVNOTES
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> >
> > RELEASE NOTES
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=810ecfd3baeeb378d868a0cf230cbe5c0fe298ea;hb=21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
> >
> > Release Verification Task run
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4594148=ApacheIgniteReleaseJava8_PrepareVote4CheckRcLicensesChecksum
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept Apache Ignite 2.7.6-rc2
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite Ignite 2.7.6-rc2 (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-ReleaseVerification
> >
> > This vote will be open for at least 72 hours till September 16, 09:00
> UTC.
> >
> >
> https://www.timeanddate.com/countdown/to?year=2019=9=16=09=0=0=utc-1
> >
> > -- Alexey Goncharuk
> >
>

Re: Cache expiry policy not deleting records from disk(native persistence)

[Artem Budnikov]

Hi Denis,

That's on the expiry policies page: 
https://apacheignite.readme.io/docs/expiry-policies


On 13.09.2019 19:46, Denis Magda wrote:
Artem, thanks, could you please share a reference to the updated page? 
Can't find anything here:

https://apacheignite.readme.io/docs/evictions

Shiva, I've restarted the discussion on the dev list, to get to the 
bottom of this gap and how it can be addressed:

http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html

-
Denis


On Fri, Sep 13, 2019 at 7:05 AM Artem Budnikov 
mailto:a.budnikov.ign...@gmail.com>> wrote:


Hi Denis,

I updated the page about eviction policies. Not freeing up space
on disk was not implemented for reasons explained in the dev-list
thread. I'll update the page once more if/when a solution is
implemented.

Artem

On 13.09.2019 00:34, Denis Magda wrote:

Shiva,

Hopefully, someone from the dev community will pick this ticket
up soon and solve the task. In the meantime, Artem, would you
mind documenting this limitation referring to ticket 10862?

-
Denis


On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar
mailto:shivakumar@gmail.com>> wrote:

I have filed a bug
https://issues.apache.org/jira/browse/IGNITE-12152 but this
is same as https://issues.apache.org/jira/browse/IGNITE-10862
Any idea on the timeline of these tickets?
In the documentation
https://apacheignite.readme.io/v2.7/docs/expiry-policies
it says when native persistence is enabled "*expired entries
are removed from both memory and disk tiers*" but in the disk
it just mark the pages as unwanted pages and same disk space
used by these unwanted pages will be used to store new pages
but it will not remove unwanted pages from disk and so it
will not release disk space used by these unwanted pages.

here is the developer's discussion link

http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html


On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar
mailto:shivakumar@gmail.com>>
wrote:

Hi
I have deployed ignite on kubernetes and configured two
seperate persistent volume for WAL and persistence.
The issue Iam facing is same as
https://issues.apache.org/jira/browse/IGNITE-10862

Thanks
Shiva

On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov,
mailto:aealexsand...@gmail.com>> wrote:

Hello,

I guess that generated WAL will take this disk space.
Please read about WAL here:

https://apacheignite.readme.io/docs/write-ahead-log

Please provide the size of every folder under
/opt/ignite/persistence.

BR,
Andrei

9/6/2019 9:45 PM, Shiva Kumar пишет:

Hi all,
I have set cache expiry policy like this


 
 
          
              
                
                
                
                
                
                  

                      


                      

                  
                

              
          
 


And batch inserting records to one of the table
which is created with above cache template.
Around 10 minutes, I ingested ~1.5GB of data and
after 10 minutes records started reducing(expiring)
when I monitored from sqlline.

0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;


COUNT(ID)


248896

1 row selected (0.86 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;


COUNT(ID)


222174

1 row selected (0.313 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;

Re: After IGNITE-12148 the Examples suite has unstable tests

[Denis Garus]

Alexey, I run tests two times on your branch, all are green.
Thank you, great job!

пт, 13 сент. 2019 г. в 23:04, Alexey Zinoviev :

> Fixed it, please have a look and test it
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
>
>
> пт, 13 сент. 2019 г. в 21:16, Alexey Zinoviev :
>
> > Ok, got it, I've found a solution
> >
> > пт, 13 сент. 2019 г. в 21:09, Denis Garus :
> >
> >> Alexey, about agents was my assumption, and it looks like wrong.
> >> I didn't dive so deep.
> >>
> >> пт, 13 сент. 2019 г. в 20:54, Alexey Zinoviev :
> >>
> >> > Could you help me recognize the difference between agents to exactly
> >> > reproduce the issue and be sure that I fix it?
> >> >
> >> >
> >> > пт, 13 сент. 2019 г. в 20:51, Alexey Zinoviev  >:
> >> >
> >> > > Thank you, I'll try to fix it, ticket is here
> >> > > https://issues.apache.org/jira/browse/IGNITE-12168
> >> > >
> >> > > пт, 13 сент. 2019 г. в 20:41, Denis Garus :
> >> > >
> >> > >> Alexey, sure.
> >> > >> My first build today is [1], and the last build is [2].
> >> > >> 28 tests became flaky.
> >> > >>
> >> > >> 1.
> >> > >>
> >> > >>
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4594295=IgniteTests24Java8_Examples
> >> > >> 2.
> >> > >>
> >> > >>
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4595102=IgniteTests24Java8_Examples
> >> > >>
> >> > >> пт, 13 сент. 2019 г. в 20:25, Alexey Zinoviev <
> >> zaleslaw@gmail.com>:
> >> > >>
> >> > >> > Dear @ Denis Garus
> >> > >> >
> >> > >> > Could you please what kind of tests became unstable?
> >> > >> > Because I have no troubles with examples (and run them of course
> >> > before
> >> > >> > merging) and many builds have no troubles too
> >> > >> >
> >> > >> > Look at
> >> > >> >
> >> > >> >
> >> > >>
> >> >
> >>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
> >> > >> > -
> >> > >> > it's broken on Scala component a few weeks ago
> >> > >> >
> >> > >> > Could you please send me exact link on the builds that were ran
> by
> >> > >> yourself
> >> > >> > today
> >> > >> >
> >> > >> > But yes, the IGNITE-12148 could influence on many examples in ML
> >> > module
> >> > >> > because a few of resources were moved.
> >> > >> > I will try to fix today or tomorrow (but I need more information
> to
> >> > >> > reproduce the situation)
> >> > >> >
> >> > >> >
> >> > >> > пт, 13 сент. 2019 г. в 19:52, Denis Magda :
> >> > >> >
> >> > >> > > Alex Zinoviev, could you please double-check and confirm if
> >> > >> IGNITE-12148
> >> > >> > > affects the test suite?
> >> > >> > >
> >> > >> > > -
> >> > >> > > Denis
> >> > >> > >
> >> > >> > >
> >> > >> > > On Fri, Sep 13, 2019 at 1:55 AM Denis Garus <
> garus@gmail.com
> >> >
> >> > >> wrote:
> >> > >> > >
> >> > >> > > > Hello, Igniters!
> >> > >> > > >
> >> > >> > > > I ran two times the Examples suite [1] on the master branch
> >> today
> >> > >> and
> >> > >> > get
> >> > >> > > > different results. It looks like some tests become unstable
> >> after
> >> > >> > merging
> >> > >> > > > of task IGNITE-12148 [2]. I think tests result depend on an
> >> Agent
> >> > >> that
> >> > >> > > > executes suite.
> >> > >> > > >
> >> > >> > > > 1.
> >> > >> > > >
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Examples?branch=%3Cdefault%3E
> >> > >> > > >
> >> > >> > > > 2.
> >> > >> > > >
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://github.com/apache/ignite/commit/cd5e39b783bc499837b569c2cf974a8fe308bcf2
> >> > >> > > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> > >
> >> >
> >>
> >
>