[jira] [Created] (IGNITE-12954) Remove checkstyle suppression for XGBoostModelLexer,XGBoostModelParser
Maxim Muzafarov created IGNITE-12954:
Summary: Remove checkstyle suppression for
XGBoostModelLexer,XGBoostModelParser
Key: IGNITE-12954
URL: https://issues.apache.org/jira/browse/IGNITE-12954
Project: Ignite
Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Currently, there are some redundant suppressions for the checkstyle rules which
can be removed:
{code}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12953) Add support for SingleSpaceSeparator to checkstyle rules
Amelchev Nikita created IGNITE-12953: Summary: Add support for SingleSpaceSeparator to checkstyle rules Key: IGNITE-12953 URL: https://issues.apache.org/jira/browse/IGNITE-12953 Project: Ignite Issue Type: Task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Add a new rule to checkstyle according to Apache Ignite Whitespaces and empty lines conventions. https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Whitespacesandemptylines -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12952) Enable travis ci for ignite-extensions
Saikat Maitra created IGNITE-12952: -- Summary: Enable travis ci for ignite-extensions Key: IGNITE-12952 URL: https://issues.apache.org/jira/browse/IGNITE-12952 Project: Ignite Issue Type: Sub-task Components: streaming Reporter: Saikat Maitra Enable travis ci for ignite-extensions. We have recently enabled travis ci in ignite repo and checking for build results for PR and merge to master process. This subtask is to enable travis ci in ignite-extensions repo. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12951) Update README for migrated extensions
Saikat Maitra created IGNITE-12951: -- Summary: Update README for migrated extensions Key: IGNITE-12951 URL: https://issues.apache.org/jira/browse/IGNITE-12951 Project: Ignite Issue Type: Sub-task Components: streaming Reporter: Saikat Maitra Update README for migrated extensions. The newly migrated extensions modules have suffix -ext in the artifact name. Review and update extensions README docs to correct artifact name. An example for correct artifact name would be ignite-flink-ext -- This message was sent by Atlassian Jira (v8.3.4#803005)
IGNITE-12357 Migrate Twitter module to ignite-extensions
Hi, I have raised PRs for migration of twitter module in ignite-extensions repository. Jira https://issues.apache.org/jira/browse/IGNITE-12357 PRs https://github.com/apache/ignite-extensions/pull/12 https://github.com/apache/ignite/pull/7733 Please review and share your feedback. Regards, Saikat
Re: [DISCUSSION] Pull-request checks on GitHub
Hi Maxim, Thank you for enabling travis ci in ignite repo. It is very helpful to see PR build results integrated in PR request. I will enable it in ignite-extensions repo as well. Regards, Saikat On Mon, Apr 20, 2020 at 12:14 PM Pavel Tupitsyn wrote: > Maxim, pull request template is a great idea. > We can put a checklist there along with the links to the guidelines, > something like this: > > [ ] Coding Guidelines are followed > [ ] TeamCity build passes > [ ] JIRA ticked is in Patch Available state, review has been requested in > comments > [ ] Something else? > > On Mon, Apr 20, 2020 at 8:09 PM Maxim Muzafarov wrote: > > > Pavel, > > > > Sorry for the incomplete message. > > > > 2. I mentioned it too. Hopefully, builds > 4 hrs would not be too often. > > The reason - there are limited job-workers shared between all the > > Apache projects. I found some details here [1] [2]. > > > > > > [1] > > > https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E > > [2] https://issues.apache.org/jira/browse/INFRA-18533 > > > > On Mon, 20 Apr 2020 at 20:03, Maxim Muzafarov wrote: > > > > > > Pavel, > > > > > > 1. Agree here. What if we create a default template pull request > > > description with all the links required by our development process? > > > [1] It's will be friendly for contributors to have everything they > > > need in the PR. > > > > > > 2. > > > > > > [1] > > > https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository > > > > > > On Mon, 20 Apr 2020 at 19:46, Pavel Tupitsyn > > wrote: > > > > > > > > Maxim, > > > > > > > > Good news, thank you. > > > > > > > > However, I see two issues with this: > > > > > > > > 1. False sense of a ready-to-merge PR > > > > Now that we have a reassuring green checkmark on the PR, contributors > > might > > > > think that build passes and all is well. > > > > But this is not true - we only check that the code compiles. TeamCity > > run > > > > is still required. > > > > My proposal is to change the text somehow to make this clear, maybe > > add a > > > > link to the contribution guidelines automatically. > > > > > > > > 2. Builds seem to spend a lot of time in the queue. > > > > I've created this PR 4 hours ago, still no results: [1] > > > > Any ideas? I use Travis on some other GitHub projects and it usually > > runs > > > > in a minute or two. > > > > > > > > [1] https://github.com/apache/ignite/pull/7698 > > > > > > > > On Mon, Apr 20, 2020 at 3:16 PM Maxim Muzafarov > > wrote: > > > > > > > > > Igniters, > > > > > > > > > > > > > > > The Travis-ci build configured for running on the Apache Ignite PRs > > > > > and the master branch [1] [2]. > > > > > > > > > > Build run under: > > > > > openjdk8 > > > > > openjdk11 > > > > > > > > > > Example of PR: > > > > > https://github.com/apache/ignite/pull/7695 > > > > > > > > > > > > > > > [1] https://travis-ci.org/github/apache/ignite > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-12916 > > > > > > > > > > On Tue, 14 Apr 2020 at 21:00, Maxim Muzafarov > > wrote: > > > > > > > > > > > > Petr, > > > > > > > > > > > > I think it's doable. It has custom `install-jdk` script, so even > > the > > > > > > latest JDKs can be used. > > > > > > > > > > > > [1] https://github.com/sormuras/bach#install-jdksh > > > > > > > > > > > > On Tue, 14 Apr 2020 at 18:31, Petr Ivanov > > wrote: > > > > > > > > > > > > > > We do not need JDK10 — it is out of support already. > > > > > > > Instead, how about adding JDK14? > > > > > > > > > > > > > > > On 14 Apr 2020, at 17:30, Maxim Muzafarov > > > wrote: > > > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > I forgot to mention one more important thing of this tool. We > > can > > > > > > > > configure build and checks simultaneously for several JDK > > versions. > > > > > > > > > > > > > > > > jdk: > > > > > > > > - oraclejdk8 > > > > > > > > - openjdk10 > > > > > > > > - openjdk11 > > > > > > > > > > > > > > > > On Tue, 14 Apr 2020 at 17:17, Maxim Muzafarov < > > mmu...@apache.org> > > > > > wrote: > > > > > > > >> > > > > > > > >> Folks, > > > > > > > >> > > > > > > > >> +1 Travis-ci > > > > > > > >> > > > > > > > >> I see no disadvantages of having multiple CI tools due to: > > > > > > > >> - it's free for open-source and can be disabled at any time > > without > > > > > > > >> any consequences; > > > > > > > >> - it will free TeamCity from running builds on each PR and > TC > > can > > > > > > > >> focus on tests execution; > > > > > > > >> - we can perform more sophisticated checks with this tool > > like a PR > > > > > > > >> title format (e.g. IGNITE-X: Sample) > > > > > > > >> > > > > > > > >> It seems the TC.Bot can also be integrated with GitHub > checks > > via > > > > > REST API [1]. > > > > > > > >> > > > > > > > >> > > > > > > > >> I've checked locally the Ignite build procedure with > > travis-ci and > > > > > > >
Re: Tool for performance statistics reports
Hello Nikolay, > Who deprecated visor and when? Maybe I miss something? On the one hand, there was technically no community consensus that this tool should be obsolete. On the other hand, my opinion based on the following topic: http://apache-ignite-developers.2346864.n4.nabble.com/Re-Visor-plugin-tp44879p44939.html Moreover, it seems to me, currently, the control utility is widely used and actively developed, instead of the visor. > It's true that, for now, Ignite doesn't have "tool strategy" I think it's a big issue from the user's point of view. I absolutely agree with that. > We should solve it in the nearest time. Feel free to start this activity I have no plan at the moment. However, at the first stage, we could understand the difference between visor and control utility. All useful features from visor should be moved/implemented in control utility and after that visor tool and should be marked as deprecated/obsoleted. > You need to throw in control.sh also, which does some kind of statistics too, such as idle_verify. > Please, clarify your idea: >We should use some info from control.sh to the report? >The report should be generated by some control.sh subcommand? If I am not mistaken, the oracle database has AWR tool (mentioned on the IEP page) which is a command-line utility that generates HTML reports. I like this idea and I think this is a good approach that can be realized in the control utility. If we have a case that cannot be implemented in this way, we have to clearly states the difference between these tools so as not to confuse our users. What do you think? Thanks, Slava. сб, 25 апр. 2020 г. в 12:00, Nikolay Izhikov : > Hello, Slava, Ilya, Denis. > > Thanks for joining this discussion! > > > - visor (which is deprecated) > > Who deprecated visor and when? > Maybe I miss something? > > > - web-console (to be honest, I don't quite understand the status of this > tool) > > +1. > > > I am not against the new tool, I just want to understand the motivation > to not improve the existing sub-projects. > > It's true that, for now, Ignite doesn't have "tool strategy" > I think it's a big issue from the user's point of view. > We should solve it in the nearest time. > Feel free to start this activity. > > > - new ignite-profiling (which is a monitoring tool as well, judging by > the provided link [1] ) > > The general idea is the following: > > 1. We should have some profiling mechanism that will generate a node-local > event log > 2. We should have a tool that can export events to some third-party > system. This system can be an Elastic Search(Kibana) or Ignite performance > report or Kafka log, whatever. > 3. Ignite performance report, in the first release, should be a "static" > tool. > This means we take static logs(that is not rewritten in the analysis > time) and feed them in the script(or tool or control.sh, whatever) > The script produces static report that can be used for overall > performance analysis. > > The primary users of this report is a developer of Ignite based > applications and performance engineers. > > Ilya, > > > You need to throw in control.sh also, which does some kind of statistics > too, such as idle_verify. > > Please, clarify your idea: > We should use some info from control.sh to the report? > The report should be generated by some control.sh subcommand? > > > Denis, > > > Speaking of the probes/statistics collection approach, is it supposed to > reuse tracing capabilities that are to be added as part of IEP-35? > > For now, we don't have any results of tracing development available in > Apache Ignite. > Hopefully, we got some in a couple of weeks. > After it, we can start a discussion of how to merge two improvements. > > > > > 24 апр. 2020 г., в 20:32, Denis Magda написал(а): > > > >> > >> Tracing is more deeply takes statistics. If it will be possible, I'm for > >> reuse. > > > > > > Looks like we need to sync up on these activities/initiatives to ensure > we > > don't do a duplicate job. If you think a separate discussion is necessary > > let's kick it off. > > > > - > > Denis > > > > > > On Fri, Apr 24, 2020 at 9:18 AM Nikita Amelchev > > wrote: > > > >> Denis, Ilya, > >> > >> I will try to integrate profiling functionality into control.sh utility. > >> > >>> Speaking of the probes/statistics collection approach, is it supposed > to > >>> reuse tracing capabilities that are to be added as part of IEP-35? > >> Tracing is more deeply takes statistics. If it will be possible, I'm for > >> reuse. > >> > >> пт, 24 апр. 2020 г. в 18:59, Ilya Kasnacheev >: > >>> > >>> Hello! > >>> > >>> I suggest that it's one of the places where it could be put instead of > >>> adding a new tool. > >>> > >>> Regards, > >>> -- > >>> Ilya Kasnacheev > >>> > >>> > >>> пт, 24 апр. 2020 г. в 18:56, Nikita Amelchev : > >>> > Ilya, > > You suggest using control.sh to build the report? > > пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev < >
Re: Hello.
Welcome Ivan, I added your account to the contributors list in JIRA. Now you can assign tickets to yourself. Best regards, Ivan Pavlukhin вс, 26 апр. 2020 г. в 13:18, Ivan Mironovich : > > My name is Ivan Mironovich and I want to contribute.Need access to > Ignite Jira. My username is imironovich. > Thanks in advance.Best regards, > Ivan Mironovich
Hello.
My name is Ivan Mironovich and I want to contribute.Need access to Ignite Jira. My username is imironovich. Thanks in advance.Best regards, Ivan Mironovich
[jira] [Created] (IGNITE-12950) Partitions validator must check sizes even if update counters are different
Ivan created IGNITE-12950:
-
Summary: Partitions validator must check sizes even if update
counters are different
Key: IGNITE-12950
URL: https://issues.apache.org/jira/browse/IGNITE-12950
Project: Ignite
Issue Type: Improvement
Components: cache
Reporter: Ivan
Fix For: 2.9
We have method in GridDhtPartitionsStateValidator:
{code:java}
// public void validatePartitionCountersAndSizes(
GridDhtPartitionsExchangeFuture fut,
GridDhtPartitionTopology top,
Map messages
) throws IgniteCheckedException {
final Set ignoringNodes = new HashSet<>();
// Ignore just joined nodes.
for (DiscoveryEvent evt : fut.events().events()) {
if (evt.type() == EVT_NODE_JOINED)
ignoringNodes.add(evt.eventNode().id());
}
AffinityTopologyVersion topVer =
fut.context().events().topologyVersion();
// Validate update counters.
Map> result =
validatePartitionsUpdateCounters(top, messages, ignoringNodes);
if (!result.isEmpty())
throw new IgniteCheckedException("Partitions update counters are
inconsistent for " + fold(topVer, result));
// For sizes validation ignore also nodes which are not able to send
cache sizes.
for (UUID id : messages.keySet()) {
ClusterNode node = cctx.discovery().node(id);
if (node != null &&
node.version().compareTo(SIZES_VALIDATION_AVAILABLE_SINCE) < 0)
ignoringNodes.add(id);
}
if (!cctx.cache().cacheGroup(top.groupId()).mvccEnabled()) { // TODO:
Remove "if" clause in IGNITE-9451.
// Validate cache sizes.
result = validatePartitionsSizes(top, messages, ignoringNodes);
if (!result.isEmpty())
throw new IgniteCheckedException("Partitions cache sizes are
inconsistent for " + fold(topVer, result));
}
}
{code}
{{}}
We should check paritions sizes even if update counters are different. It could
be helpful for debug problems on production.
We must print information about all copies, if partition is in inconsistent
state. Now we could get message on cache group with 3 backups:
{code:java}
// Partition states validation has failed for group:
CACHEGROUP_PARTICLE_union-module_com.sbt.processing.data.partition.dpl.PartitionKey.
Partitions update counters are inconsistent for Part 3415:
[10.104.6.10:47500=2577263 10.104.6.12:47500=2577263 10.104.6.23:47500=2577262
10.104.6.9:47500=2577263 ] Part 4960: [10.104.6.11:47500=2560994
10.104.6.23:47500=2560993 ]
{code}
(part 4960 contains information about 2 copies only)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
