Re: Call for presentations for ApacheCon North America 2020 now open
Hi Denis, Congrats!!! It looks like both of our talks are on same day, Tuesday, September 29th https://apachecon.com/acah2020/tracks/ignite.html I still have not received confirmation on schedule yet and maybe it can change. Regards, Saikat On Tue, Aug 4, 2020 at 11:08 PM Denis Magda wrote: > Congrats, Saikat! I received a similar message that my talk (In-memory > computing essentials for software engineers) was accepted as well. So, at > least two talks by the Ignite folks. > > Once you share the time your presentation is scheduled for, I'll go ahead > and update on the events' page on the website. > https://ignite.apache.org/events.html > > - > Denis > > > On Tue, Aug 4, 2020 at 6:40 PM Saikat Maitra > wrote: > > > Hi, > > > > I learned that my proposal for talk about Data Streaming using Apache > > Ignite > > and Apache Flink has been accepted. > > > > I have not yet received the schedule yet. I will share as soon as I have > > it. > > > > Regards, > > Saikat > > > > On Wed, Jan 22, 2020 at 8:09 PM Saikat Maitra > > wrote: > > > > > Hi Denis, > > > > > > Thank you for your email. I have submitted a talk about Data Streaming > > > using Apache Ignite and Apache Flink. > > > > > > I would also like to co-present on Ignite internals and usecases deep > > dive. > > > > > > Regards, > > > Saikat > > > > > > On Wed, Jan 22, 2020 at 6:22 PM Denis Magda wrote: > > > > > >> Igniters, > > >> > > >> I was submitting an Ignite talk to the conference and found out that > > this > > >> time ASF created a separate category for Ignite-specific proposals. > You > > can > > >> see an abstract submitted by me: > > >> > > > https://drive.google.com/file/d/1woaEOWaIFxN8UIJ7nvFbYoc53mYsUsSN/view?usp=sharing > > >> > > >> Who else is ready to submit? It can be a deep-dive about Ignite > > internals > > >> or Ignite use case overview. We can co-present. Also, GridGain is > ready > > to > > >> sponsor your trip if the talk is accepted. > > >> > > >> > > >> > > >> - > > >> Denis > > >> > > >> > > >> On Tue, Jan 21, 2020 at 5:22 PM Denis Magda > wrote: > > >> > > >>> Ignite folks, just bringing to your attention and encourage anybody > > >>> interested to submit an Ignite-related session. Both Alex Zinoviev > and > > I > > >>> presented at ApacheCons the previous year -- the events are worth > > attending > > >>> and speaking at. > > >>> > > >>> - > > >>> Denis > > >>> > > >>> > > >>> -- Forwarded message - > > >>> From: Rich Bowen > > >>> Date: Tue, Jan 21, 2020 at 7:06 AM > > >>> Subject: Call for presentations for ApacheCon North America 2020 now > > open > > >>> To: plann...@apachecon.com > > >>> > > >>> > > >>> Dear Apache enthusiast, > > >>> > > >>> (You’re receiving this message because you are subscribed to one or > > more > > >>> project mailing lists at the Apache Software Foundation.) > > >>> > > >>> The call for presentations for ApacheCon North America 2020 is now > open > > >>> at https://apachecon.com/acna2020/cfp > > >>> > > >>> ApacheCon will be held at the Sheraton, New Orleans, September 28th > > >>> through October 2nd, 2020. > > >>> > > >>> As in past years, ApacheCon will feature tracks focusing on the > various > > >>> technologies within the Apache ecosystem, and so the call for > > >>> presentations will ask you to select one of those tracks, or > “General” > > >>> if the content falls outside of one of our already-organized tracks. > > >>> These tracks are: > > >>> > > >>> Karaf > > >>> Internet of Things > > >>> Fineract > > >>> Community > > >>> Content Delivery > > >>> Solr/Lucene (Search) > > >>> Gobblin/Big Data Integration > > >>> Ignite > > >>> Observability > > >>> Cloudstack > > >>> Geospatial > > >>> Graph > > >>> Camel/Integration > > >>> Flagon > > >>> Tomcat > > >>> Cassandra > > >>> Groovy > > >>> Web/httpd > > >>> General/Other > > >>> > > >>> The CFP will close Friday, May 1, 2020 8:00 AM (America/New_York > time). > > >>> > > >>> Submit early, submit often, at https://apachecon.com/acna2020/cfp > > >>> > > >>> Rich, for the ApacheCon Planners > > >>> > > >> > > >
Apache Ignite 3.0
Igniters, I would like to kick off a discussion regarding Ignite 3.0. Ignite 2.0 exists for more than 3 years now and we've already collected a significant list [1] of changes that we would like to have, but cannot implement without breaking compatibility. I think it's time to start planning for the next major release and discussing what should be included. I've already gathered some information and feedback, and have some thoughts on how to approach this. In the next few days, I will put everything into a Wiki page and will share it once this is done. Stay tuned! I'm willing to drive the 3.0 activities going forward as well. In the meantime, if there are any immediate thoughts or ideas, please feel free to join the thread and share them. [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0+Wishlist Regards, Val
[MTCGA]: new failures in builds [5515108] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master-nightly DatasetAffinityFunctionWrapperTest.unnecessary Mockito stubbings https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-749995274864701629=%3Cdefault%3E=testDetails Changes may lead to failure were done by - alexey kuznetsov https://ci.ignite.apache.org/viewModification.html?modId=905349 - alexey kuznetsov https://ci.ignite.apache.org/viewModification.html?modId=905344 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 22:44:19 05-08-2020
[jira] [Created] (IGNITE-13329) Use checkstyle from command line check on validate phase
Maxim Muzafarov created IGNITE-13329: Summary: Use checkstyle from command line check on validate phase Key: IGNITE-13329 URL: https://issues.apache.org/jira/browse/IGNITE-13329 Project: Ignite Issue Type: Task Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov Fix For: 2.10 Currently, the checkstyle plugin configured to run on the `compile` maven phase due to the custom OverrideAnnotationOnTheSameLineCheck check must be compiled. However, it will be faster to run checkstyle on `validate` maven phase check all the configured rules locally prior to pushing by running the `mvn checkstyle:check` command. [https://maven.apache.org/plugins/maven-checkstyle-plugin/usage.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Cache warmup
Kirill, Thank you for driving this discussion and implementation. A few points from my side: * Agree that it will be best to keep the strategy interface private because it will be very dependent on the persistent storage implementation. We would need to expose page IDs and types to public API, which is very restrictive. The configuration part obviously needs to be public, and ability to pull the strategy implementation from plugin is a good idea. * I was also thinking of adding the warmup configuration straight to the IgniteConfiguration, but I like Stan's idea of adding it to DataRegionConfiguration. No strong preference here. * I do not think we need to deprecate preloadPartition() method. One of the use-cases for this method was to process partitions sequentially while a node is running. This method is able to fetch the partition from disk much (from times to orders of magnitude) faster than sequential scan. * Being able to cancel the warmup during startup is a great feature. We should be able to support it from control.sh because the warmup runs before discovery which starts the last, so the control.sh handler should be already running.
[jira] [Created] (IGNITE-13328) Control.sh bash script swallow return code of CommandHandler and always return 0
Ivan Daschinskiy created IGNITE-13328:
-
Summary: Control.sh bash script swallow return code of
CommandHandler and always return 0
Key: IGNITE-13328
URL: https://issues.apache.org/jira/browse/IGNITE-13328
Project: Ignite
Issue Type: Bug
Affects Versions: 2.8.1, 2.8
Reporter: Ivan Daschinskiy
Fix For: 2.9
After merging [IGNITE-12367|https://issues.apache.org/jira/browse/IGNITE-12367],
control.sh always return 0, despite the fact that CommandHandler returns
correct code.
For example:
Ignite 2.8.1
{code}
Failed to execute baseline command='collect'
Latest topology update failed.
Connection to cluster failed. Latest topology update failed.
Command [BASELINE] finished with code: 2
Control utility has completed execution at: 2020-08-05T15:01:34.123
Execution time: 26627 ms
>>> echo $?
0
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Distinguishing node-local metrics from cluster-wide
Hi Igniters! My team and I are building a monitoring system on top of the new metrics framework described in the following IEP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392 So far it's going well, but we'd like to improve the way metrics are exported from Ignite. There are different kinds of metrics that you can access through this framework. Some of them are local for a node, like used heap, or CPU load. It makes sense to send them independently from every node to the centralized storage. Let's assume that we attach nodeID to metric names, so that we can distinguish between metrics coming from different nodes. It makes sense to work with local metrics using some kind of patterns on metric names. For example, if I want to draw a chart for CPU load on every node, I can use a pattern similar to the following one: sys.CpuLoad.* There are also the kind of metrics that have the same value, no matter which node the metric is taken from. For example, cache size, progress of rebalance or topology version are global things that don't depend on the node. If I take any of the metrics matching the pattern pme.Duration.*, I will get what I need. I wonder, what is the recommended approach to global metrics? I know that there are tools like Prometheus and Graphite that allow similar manipulations with metric names. Is it supposed that global and local metrics are differentiated on the side of monitoring tools using functions like any(pme.Duration.*) ? It seems that Graphite is lacking one, for example. Maybe it makes sense to introduce a property for metrics that will let the exporters distinguish between them and not parameterize the names with node ID? What do you think? Denis
[jira] [Created] (IGNITE-13327) Add a metric for processed keys when rebuilding indexes.
Kirill Tkalenko created IGNITE-13327: Summary: Add a metric for processed keys when rebuilding indexes. Key: IGNITE-13327 URL: https://issues.apache.org/jira/browse/IGNITE-13327 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.10 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [jira] [Created] (IGNITE-13133) Add integration with QuerydslPredicateExecutor for spring-data integrations
Hi, alamar and akuznetsov!
Could you please review my PR https://github.com/apache/ignite/pull/7912
Regards,
Sergey
пн, 8 июн. 2020 г. в 19:21, Moldachev Sergey (Jira) :
> Moldachev Sergey created IGNITE-13133:
> -
>
> Summary: Add integration with QuerydslPredicateExecutor for
> spring-data integrations
> Key: IGNITE-13133
> URL: https://issues.apache.org/jira/browse/IGNITE-13133
> Project: Ignite
> Issue Type: New Feature
> Components: springdata
> Reporter: Moldachev Sergey
> Assignee: Moldachev Sergey
>
>
> We have a pretty ignite-spring-data integration but we don't have a
> support of `QuerydslPredicateExecutor` which provide ability to filter
> cache entities by dynamic criterias.
>
> Example of usage:
>
> {code:java}
> /**
> * Simple entity.
> */
> @QueryEntity
> public class Person {
> /** First name. */
> @QuerySqlField(index = true)
> private String firstName;
>
> /** Second name. */
> @QuerySqlField(index = true)
> private String secondName;
>
> /** Age. **/
> @QuerySqlField(index = true)
> private int age;
> }
>
> /**
> * Implement QuerydslPredicateExecutor interface.
> */
> public interface PersonRepository extends IgniteRepository Integer>, QuerydslPredicateExecutor {
>
> }
>
> /**
> * Now we can filter our entites by firstName prdicate.
> */
> List persons = (List) repo.findAll(
> QPerson.person.firstName.eq(firstName_1)
> .or(QPerson.person.firstName.eq(firstName_2))
> );
> {code}
>
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)
>
Re: [DISCUSSION] Cache warmup
Hi, Stas!
After talking with Anton and Alexy about "IP40", I changed description of
implementation in form of a response to Slava, here [1]. In short, I made three
separate interfaces, first public for strategy configuration, second internal
for strategy implementation, and third for possible delivery of strategies from
different plugins.
I will try to think about this and implement it. Warm-up phase will be up to
"discovery" and while I'm not sure that it will be possible to connect via
control.sh, perhaps it will be possible via jmx, but I think it will be better
via control.sh
> Will there be a way to interrupt warmup phase and continue startup (e.g. via
> JMX, REST and/or control.sh)? Can we have it please?
I was thinking about how and where to make warm-up configuration and I think it
would be better to do it in IgniteConfiguration since each strategy can work
for caches, groups, regions, etc.
> I think that ideally warmup should be configured per-cache - I believe this
> is what a user would expect to do.
> However, cache configs are immutable. We need a way for existing users to
> enjoy the cache warmup feature, as well as for early adopters to switch to
> more > > > sophisticated strategies as they will be released (or as their
> dataset grows).
> Because of that I propose to add the cache warmup configuration to the
> DataRegionConfiguration. Data regions can be changed between restarts,
> independently > on each node allowing for a rolling change.
Possible.
> Will preloadPartition() method be deprecated together with this change? I
> assume yes?
I think it can be done as a new strategy, but this is at discretion of
developers.
> How hard would it be to implement a "load all indexes, metapages and
> freelists" strategy in addition to the "load everything"?
> I think it would be an MVP for environments with a datasets larger than RAM.
> A "load everything" strategy will not work in this environments pretty much
> at all,
> and "load indexes" will be a significant improvement to no warmup at all.
[1] -
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Cache-warmup-td48582.html#a48649
04.08.2020, 23:22, "Stanislav Lukyanov" :
> Kirill,
>
> Thanks for driving this. This is awaited by many users.
>
> A few comments and questions.
>
> I would keep CacheWarmup interface purely internal and never view it as an
> interface which a user would be implementing.
> There are multiple reasons for that:
> - The logic of the cache warmup is very low-level; how a user is supposed to
> know which pages they want?
> - A sophisticated strategy will require accessing private APIs for sure; say,
> I need a strategy which loads the last known memory state before the restart;
> how can I even implement that without breaking into various internals?
> - In fact there aren't many implementations which make sense ("load
> everything", "load indexes", "load last memory state", "load N GB at
> random"); every use case I've seen would be solved by a "load everything"
> strategy (if disk is < RAM) or "load last memory state" strategy
> - Warmup will be a critical phase, and a custom user implementation is all
> too likely to cause issues. We should avoid executing user code in critical
> stages if we can help it
> To summarize, if we give warmup strategies in users' hands they will be hard
> to write, will require breaking into internals or a lot of additional public
> interfaces for these internals, will likely cause issues with the cluster,
> and everyone will be implementing the same few general strategies.
> Basically, I expect only fellow Ignite developers to be implementing their
> own strategies.
> Because of that I propose to keep the interfaces private, and only give a
> single public parameter. The parameter can take an enum of the supported
> strategies. New useful strategies should be added to Ignite codebase.
>
> Will there be a way to interrupt warmup phase and continue startup (e.g. via
> JMX, REST and/or control.sh)? Can we have it please?
>
> I think that ideally warmup should be configured per-cache - I believe this
> is what a user would expect to do.
> However, cache configs are immutable. We need a way for existing users to
> enjoy the cache warmup feature, as well as for early adopters to switch to
> more sophisticated strategies as they will be released (or as their dataset
> grows).
> Because of that I propose to add the cache warmup configuration to the
> DataRegionConfiguration. Data regions can be changed between restarts,
> independently on each node allowing for a rolling change.
>
> Will preloadPartition() method be deprecated together with this change? I
> assume yes?
>
> How hard would it be to implement a "load all indexes, metapages and
> freelists" strategy in addition to the "load everything"?
> I think it would be an MVP for environments with a datasets larger than RAM.
> A "load everything" strategy will not work in this
