[jira] [Created] (IGNITE-8267) Web console: cluster client connector configuration fields are too narrow

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-8267:


 Summary: Web console: cluster client connector configuration 
fields are too narrow
 Key: IGNITE-8267
 URL: https://issues.apache.org/jira/browse/IGNITE-8267
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov
 Attachments: image-2018-04-16-11-31-15-136.png

*How to reproduce:*
1. Go to advanced cluster configuration.
2. Open "client connector configuration" section.

*What happens:*
"Socket send buffer size" and "Socket receive buffer size" field labels wrap to 
second line.

 !image-2018-04-16-11-31-15-136.png! 

*What should happen:*
"Socket send buffer size" and "Socket receive buffer size" field labels should 
be on a single line.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IGNITE-6879

[Роман Меерсон]

Hi all!

So guys let’s make a decision. We leave code in current state as I suggest
or change module naming according Dmitry’s suggestion.
ср, 11 апр. 2018 г. в 1:54, Denis Magda :

> Roman,
>
> Your suggestion sounds reasonable to me. Backing it up.
>
> --
> Denis
>
> On Tue, Apr 10, 2018 at 2:50 PM, Роман Меерсон 
> wrote:
>
> > Hi all!
> >
> > IMHO if we do so we'll produce big pain for everybody while migrating on
> > new version, because ones should change method and others should change
> > their poms. This change would be backward incompatible so it probably
> > should follow with major version upgrade, but I'm not sure about it.
> >
> > Otherwise if we leave current state( spring-data for old and
> > spring-data-2.0 for new) we could support old users who probably use
> spring
> > data 1.0 (because spring data 2.0 release was not so long ago) and
> provide
> > new functionality for users who want to use new spring data.
> > In this case old users wouldn't change any in their code except ignite
> > version, and new users would include ignite in their Pom anyway and could
> > choose which module of spring data to bring.
> >
> > After some time (probably on 3.0 release) we could change naming as Denis
> > suggested.
> >
> > Anyway I leave this decision up to you, just tell me what is the way to
> > finish this PR.
> >
> > Regards, Roman.
> >
> > ср, 11 апр. 2018 г. в 1:35, Denis Magda :
> >
> > > In our Hibernate integration we define following two modules to
> > > distinguish incompatible versions:
> > >
> > >- ignite-hiberbate_4.2
> > >- ignite-hibernate_5.1
> > >
> > > In Spark we have:
> > >
> > >- ignite-spark
> > >- ignite-spark_2.10
> > >
> > > After thinking this over, I would do the following with Spring Data:
> > >
> > >- ignite-spring-data for the latest Sprind Data 2.0
> > >- ignite-spring-data_1.0
> > >
> > > What do you think?
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Apr 10, 2018 at 3:50 AM, Dmitry Pavlov 
> > > wrote:
> > >
> > >> Thank you, Roman.
> > >>
> > >> Igniters,
> > >>
> > >> IMO we should consider one more alternative - renaming of old module
> and
> > >> package names. Users, which prefer to stay on previous version will be
> > >> requiered to update their pom's. In the same time users which are
> ready
> > to
> > >> migrate to spring data 2.0 will need to update methods naming.
> > >>
> > >> Denis M, what would you say?
> > >>
> > >> Sincerely,
> > >> Dmitriy Pavlov
> > >>
> > >> вт, 10 апр. 2018 г. в 11:27, Роман Меерсон :
> > >>
> > >>> Hi Dmitry!
> > >>>
> > >>> I`ve just commited new fix. I renamed package of new module to
> > >>> springdata20, it helps us to separate old implementation from new and
> > also
> > >>> should fix all compilation errors.
> > >>>
> > >>> пн, 9 апр. 2018 г. в 23:54, Роман Меерсон :
> > >>>
> >  Ok, I'll check it, but I haven't face this problem.
> >  If I'll find same issue, what is the proper way? Renaming to
> something
> >  like Ignite2QueryGenerator or module removing?
> >  пн, 9 апр. 2018 г. в 23:40, Dmitry Pavlov :
> > 
> > > There are 2 classes IgniteQueryGenerator with same package name.
> > > Ignite in Idea can't compile.
> > >
> > >
> > > пн, 9 апр. 2018 г., 21:38 Роман Меерсон :
> > >
> > >> Hi Dmitry!
> > >
> > >
> > >> Could you specify where you find conflict? Because I don’t have
> any.
> > >> пн, 9 апр. 2018 г. в 21:09, Dmitry Pavlov  >:
> > >>
> > >>> Hi Denis,
> > >>>
> > >>> could we support just one version instead of leaving compatible
> > >>> module?
> > >>>
> > >>> Sincerely,
> > >>> Dmitriy Pavlov
> > >>>
> > >>> пн, 9 апр. 2018 г. в 20:08, Dmitry Pavlov  >:
> > >>>
> > 
> > 
> >  пн, 9 апр. 2018 г. в 20:07, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > 
> > > Hi Roman,
> > >
> > > I've applied PR locally and I have class name conflict at least
> > > for
> > > org.apache.ignite.springdata.repository.query.
> > IgniteQueryGenerator
> > >
> > > How could we solve it? Is it better to rename class for new
> > plugin
> > > version?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пт, 6 апр. 2018 г. в 17:38, Dmitry Pavlov <
> dpavlov@gmail.com
> > >:
> > >
> > >> Excellend picture. I remember about this change.
> > >>
> > >> If Denis M. would be able to look througt the changes faster
> > than
> > >> me, I can merge without detailed review.
> > >>
> > >> пт, 6 апр. 2018 г. в 16:15, Роман Меерсон <
> homich1...@gmail.com
> > >:
> > >>
> > >>> OK
> > 

Efficiently determining if cache keys belong to the local server node

[Raymond Wilson]

I have a type of query that asks for potentially large numbers of
information elements to be computed. Each element has an affinity key that
maps it to a server node through an IAffinityFunction.



The way the question is asked means that a single query broadcast to the
compute projection (owning the cache containing the source data for the
request) contains the identities of all the pieces of information needed to
be processed.



Each server node then scans the elements requested and identifies which
ones are its responsibility according to the affinity key.



Calculating the partition ID from the affinity key is simple (I have an
affinity function set up and supplied to the cache configuration, or I
could use IAffinity.GetPartition()), so the question became: How do I know
the server node executing the query is responsible for that partition, and
so should process this element? IE: I need to derive the vector of primary
or backup  partitions that this node is responsible for.



I can query the partition map and return it, like this:



ICacheAffinity affinity = Cache.Ignite.GetAffinity(Cache.Name);

public Dictionary primaryPartitions =
affinity.GetPrimaryPartitions(Cache.Ignite.GetCluster().GetLocalNode()).ToDictionary(k
=> k, v => true);



This lets me do a dictionary lookup, but its less efficient that having a
complete partition map with simple array lookup semantics, like this:



ICacheAffinity affinity = Cache.Ignite.GetAffinity(Cache.Name);

bool[] partitionMap = new bool[affinity.Partitions];



foreach (int partition in
affinity.GetBackupPartitions(Cache.Ignite.GetCluster().GetLocalNode()))

partitionMap[partition] = true;



This is a nice lookup for the query to determine which elements are its
responsibility from the overall request.



I’m not sure of the performance profile of this approach if I end up doing
it a lot, so I’m considering caching this lookup and invalidate it if any
event occurs that could modify the key -> partition map.



Questions:



   1. How big is the penalty when determining the full partition map like
   this?
   2. If I decide to invalidate the cached map, what are all the events I’d
   need to listen to?
  1. Rebalancing events?:I found CacheRebalancingEvent, but I’m not
  sure if this gives visibility to the points in time when a rebalanced
  partition becomes active on the new node and so the partition map changes
  2. Topology change events? (eg: adding a new backup node without
  rebalancing (if that is a thing) I looked for an event like that but have
  not found it so far, though I do know the affinity function can
respond to
  this via AssignPartitions()
   3. How do I provide my own affinity key mapper to for keys to partition
   IDs, but allow Ignite to map the partitions to nodes. The IAffinityFunction
   implementation requires both steps to be implemented. I’d prefer not to
   have the partition -> server mapping responsibility as this requires
   persistent configuration on the nodes to ensure stable mapping.



Thanks,

Raymond.

Re: Apache Ignite RPM packaging (Stage II)

[Ilya Kasnacheev]

Hello!

With existing binary archive, user can move directories from
apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to activate
them.

But with RPM, user should not contemplate moving directories from
/usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.


User lacks permissions for that operation and it would defeat the purpose
of having a RPM (imagine trying to upgrade Ignite RPM version with a setup
like that). "Turning distribution into Slackware" should not be an offering.

How it could work: Imagine we create /var/lib/ignite, use it as
IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
way, we can add and remove symlinks to modules in optional/, and thus
enable and disable them as user sees fit.

This also answers the problem of "what's IGNITE_HOME for visor launched
from /usr/bin"

But obviously this will require dedication and effort.


-- 
Ilya Kasnacheev

2018-04-14 8:03 GMT+03:00 Peter Ivanov :

> Current packages design (after installation) does not differ from binary
> archive - everything works (except necessity to run service instead
> ignite.sh) just the same way, including libs/optional.
>
> Also, there can be issues with system JDK version by default, but every
> problem will be in journalctl and/or  /var/log, and package has strong
> dependency on any implementation of JDK 1.8.
>
>
> I am lacking enough feedback about Apache Ignite from packages from real
> users, don’t know real use cases so still "moving in the dark".
>
>
> On Fri, 13 Apr 2018 at 22:18, Denis Magda  wrote:
>
> > Ilya,
> >
> > Thanks for your inputs. The reason why we decided to split Ignite into
> > several packages mimics the reason why Java community introduced modular
> > subsystem for JDK. That's all about size. Ignite distribution is too big,
> > and we're trying to separate it into several components so that people
> can
> > install only the features they need.
> >
> > The point of a package is to ship something into root file system that
> can
> > > be used from root file system. If cpp files require compilation we
> should
> > > not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> > > there's no mechanism to add optional libs to Ignite classpath, we
> should
> > > not ship optional libs. Moreover, some of 'optional' modules such as
> yarn
> > > don't make sense here because they're not supposed to be used with
> > > standalone Ignite.
> >
> >
> > Agree that we need to ship the code that is ready to be run. As for the
> > classpath thing, if an optional package is installed into the root (core)
> > package directory, then its jars have to be added to "ignite/libs"
> folder.
> > After that, the one needs to restart a cluster node, nd it will add the
> > just installed optional libs to the classpath. *Petr*, does it work this
> > way or can be implemented this way to address Ilya's concerns?
> >
> > --
> > Denis
> >
> > On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > 2018-04-13 7:42 GMT+03:00 Peter Ivanov :
> > >
> > > > On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > Moreover I did not find a way to start service if default installed
> > JVM
> > > > is
> > > > > Java 7 :( I understand it's EOL, still this is something that hit
> me.
> > > >
> > > >
> > > > Apache Ignite >=2.4 does not support Java 7 - it is said in
> > > documentation,
> > > > DEVNOTES and even in startup scripts.
> > > >
> > > >  I have Java 8 too, but I could not get service from package to start
> > > Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> > > matter). Is it possible to specify it while running packaged Ignite?
> > >
> > >
> > > >
> > > > >
> > > > > apache-ignite-libs is a totally unexpected package name.
> > apache-ignite
> > > > core
> > > > > doesn't depend on it. It doesn't enable anything out of the box.
> The
> > > > > package is huge.
> > > >
> > > > ‘apache-ignite-libs’ is an aggregation package (for now) for all
> > optional
> > > > libs we are delivering. Possibly later they will be split more
> granular
> > > or
> > > > even package per lib (like php, perl, python, etc. do for their
> libs).
> > > > This package dependency on ‘apache-ignite-core’ may seem confusing
> > > though,
> > > > I will try to explain it in IEP at least for current iteration.
> > > >
> > >
> > > Okay, but how do you add optional libs to be included into Ignite
> > classpath
> > > while being launched by service? Is it even possible? If it isn't, I
> > think
> > > it doesn't make sense to ship apache-ignite-libs at all.
> > >
> > >
> > > >
> > > > Further naming may become clear when we’ll start initiative on
> > including
> > > > packages to popular Linux distributions and theirs community will
> join
> > > > naming discussions.
> > > >
> > > Renaming packages once