[jira] [Created] (IGNITE-13935) Update Ignite docker image description

2020-12-29 Thread Denis A. Magda (Jira)
Denis A. Magda created IGNITE-13935:
---

 Summary: Update Ignite docker image description
 Key: IGNITE-13935
 URL: https://issues.apache.org/jira/browse/IGNITE-13935
 Project: Ignite
  Issue Type: Improvement
Reporter: Denis A. Magda


Apache Ignite is redefined as "a distributed database for in-memory speed and 
high-performance computing".

Update the [docker image 
description|https://hub.docker.com/r/apacheignite/ignite]:
 # The title needs to say "Apache Ignite - Distributed Database"
 # The description of the "What is Apache Ignite?" section needs to be 
identical to a similar section of GitHub's README.md (definition + quick links 
to docs): https://github.com/apache/ignite



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: IEP-54: Schema-first approach for 3.0

2020-12-29 Thread Valentin Kulichenko
Hi Mike,

Thanks for providing your feedback. Please see my comments below.

I would also encourage you to go through the IEP-54 [1] - it has a lot of
detail on the topic.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach

-Val

On Mon, Dec 28, 2020 at 11:22 PM Michael Cherkasov <
michael.cherka...@gmail.com> wrote:

> Hi all,
>
> I reviewed the mail thread and proposal page and I still don't fully
> understand what is going to be changed, I would really appreciate it if you
> will answer a few questions:
>
> 1. Are you going to leave only one schema per cache? if so, will be there
> an option to have a table with arbitrary objects(pure KV case)?
>

My opinion is that KV case should be natively supported. I think this still
needs to be thought over, my current view on this is that we should have
separate APIs for KV and more generic storages. KV storage can be
implemented as a "table" with two BLOB fields where we will store
serialized key-value pairs. That would imply deserialization on read, but I
believe this is OK for KV use cases. I'm happy to hear other ideas though :)


> 2. What options will Apache Ignite 3.0 have to define schema? SchemaBuilder
> and SQL only? Is there an option to put the schema definition to the
> configuration?(I really don't like this, I would prefer to have
> separate scripts to create schemas)
>

There will be no such thing as a static configuration in the first place.
Tables and schemas are created in runtime. Even if there is a file provided
on node startup, this file is only applied in the scope of the 'start'
operation. All configurations will be stored in a meta storage available to
all nodes, as opposed to individual files.


> 3. Is there a way to change field type? if yes, can it be done in runtime?
>

Absolutely! IEP-54 has a whole section about schema evolution.


> 4. Looks like BinaryMarshaller is going to be re-worked too, is there any
> IEP for this?
>

BinaryMarshaller as a tool for arbitrary object serialization will be gone,
but we will reuse a lot of its concept to implement an internal tuple
serialization mechanism. IEP-54 has the description of the proposed data
format.


> 5. I don't like automatic schema evaluation when a new field is added
> automatically on record put, so is there a way to prohibit this behavior?
>  I think all schema changes should be done only explicitly except initial
> schema creation.
>

The way I see it is that we should have two modes: schema-first and
schema-last. Schema-first means exactly what you've described - schemas are
defined and updated explicitly by the user. In the schema-last mode,
the user does not deal with schemas, as they are inferred from the data
inserted into tables. We should definitely not mix these modes - it has to
be one or another. And it probably makes sense to discuss which mode should
be the default one.


>
> Thanks,
> Mike.
>
> пн, 21 дек. 2020 г. в 06:40, Andrey Mashenkov  >:
>
> > Hi, Igniters.
> >
> > We all know that the current QueryEntity API is not convenient and needs
> to
> > be reworked.
> > So, I'm glad to share PR [1] with schema configuration public API for
> > Ignite 3.0.
> >
> > New schema configuration uses Builder pattern, which looks more
> comfortable
> > to use.
> >
> > In the PR you will find a 'schema' package with the API itself, and a
> draft
> > implementation in 'internal' sub-package,
> > and a test that demonstrates how the API could be used.
> >
> > Please note:
> >
> > * Entrypoint is 'SchemaBuilders' class with static factory methods.
> > * The implementation is decoupled and can be easily extracted to separate
> > module if we decide to do so.
> > * Some columns types (e.g. Date/Time) are missed, they will be added
> lately
> > in separate tickes.
> > * Index configuration extends marker interface that makes possible to
> > implement indexes of new types in plugins.
> > Hopfully, we could add a persistent geo-indices support in future.
> > * Supposedly, current table schema can be changed via builder-like
> > structure as it is done if JOOQ project. See 'TableModificationBuilder'
> for
> > details.
> > I'm not sure 'SchemaTable' should have 'toBuilder()' converter for that
> > purpose as it is a Schema Manager responsibility to create mutator
> objects
> > from the current schema,
> > but implementing the Schema manager is out of scope and will be designed
> > within the next task.
> > * Interfaces implementations are out of scope. I did not intend to merge
> > them right now, but for test/demostration purposes.
> >
> > It is NOT the final version and some may be changed before the first
> > release of course.
> > For now, we have to agree if we can proceed with this approach or some
> > issues should be resolved at first.
> >
> > Any thoughts or objections?
> > Are interfaces good enough to be merged within the current ticket?
> >
> >
> > https://issues.apache.org/jira/browse/IGNITE-13748
> >
> > On Thu, Nov 26, 2020 at 2:33 

Re: [ANNOUNCE] Apache Ignite 2.9.1 Released

2020-12-29 Thread Denis Magda
Congrats, community!

Yaroslav, thanks for driving the release. Look forward to collaborating
with all of you in 2021. Let's rock and happy New Year!

-
Denis


On Tue, Dec 29, 2020 at 4:17 AM Yaroslav Molochkov 
wrote:

> The Apache Ignite Community is pleased to announce the release of
> Apache Ignite 2.9.1.
>
> Apache Ignite [1] is a memory-centric distributed database, caching
> and processing platform for transactional, analytical, and streaming
> workloads delivering in-memory speeds at petabyte scale.
>
> This release fixes several critical issues, please, see the release notes:
> https://ignite.apache.org/releases/2.9.1/release_notes.html
>
> Download the latest Ignite version here:
> https://ignite.apache.org/download.cgi
>
> Please let us know [2] if you encounter any problems.
>
> Regards,
> Yaroslav Molochkov
>
> [1] https://ignite.apache.org
> [2] https://ignite.apache.org/community/resources.html#ask
>


[jira] [Created] (IGNITE-13934) Some of the source files are missing license headers

2020-12-29 Thread Valentin Kulichenko (Jira)
Valentin Kulichenko created IGNITE-13934:


 Summary: Some of the source files are missing license headers
 Key: IGNITE-13934
 URL: https://issues.apache.org/jira/browse/IGNITE-13934
 Project: Ignite
  Issue Type: Improvement
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
 Fix For: 3.0.0-alpha1


Need to add the license check to the build process, and fix all the missing 
headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13933) Add an ability to customize documentation source repo

2020-12-29 Thread Valentin Kulichenko (Jira)
Valentin Kulichenko created IGNITE-13933:


 Summary: Add an ability to customize documentation source repo
 Key: IGNITE-13933
 URL: https://issues.apache.org/jira/browse/IGNITE-13933
 Project: Ignite
  Issue Type: Improvement
  Components: website
Reporter: Valentin Kulichenko
 Fix For: 3.0.0-alpha1


Documentation deployment script current has the original Ignite repo hardcoded: 
https://github.com/apache/ignite-website/blob/master/_docs/build.sh#L42

Ignite 3.x is located in a different repo: https://github.com/apache/ignite-3

Suggestion: add a parameter that would allow specifying the version (2.x or 
3.x). The script should then pick the corresponding repo URL and fetch the 
sources from there.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

2020-12-29 Thread Denis Magda
Thanks, Maksim,

Considering that most of the contributors will be on holidays through Jan
10th, it's highly unlikely that all documentation tickets will be ready by
Jan 18th (the vote date).

Anyway, Nikita confirmed that he'll be actively working on the tickets
starting Jan 11th. As for me, unfortunately, I have no time for
contributions in January.

Special thanks for preparing a list of resolved but undocumented features.
Most of those just need to be donated from the GridGain documentation page.
I'll take care of those.

-
Denis


On Tue, Dec 29, 2020 at 12:43 AM Maxim Muzafarov  wrote:

> Folks,
>
> 1.
> I've prepared the list of unhandled important documentation issues for
> the 2.10 release, please check the wiki page link [1]. Note, that some
> of the important (from my point of view) documentation issues even not
> created.
>
> 2.
> Here is the list of resolved issues which seems to be documented, but
> an appropriate documentation task doesn't exist. Should we create
> appropriate documentation issues for them? Check the list [3].
>
> 3.
> Some of the documentation pages must be completed prior to 2.10
> release or removed, right? Check the [2] page for example.
>
>
> Denis, Nikita
>
> > So we should probably allocate more time for this work.
> Let's start the work, track the progress and looks at how it goes.
> I'll try to ask some folks for the help with documentation and also
> plan to help with the issues by myself. We can shift the dates if it's
> necessary, but currently, I see no reasons.
>
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
> [2] https://ignite.apache.org/docs/latest/cpp-specific/
> [3] The list of issues without documentation tasks :
>
> https://issues.apache.org/jira/browse/IGNITE-13488
> Add command to control.(sh|bin) to get an arbitrary Metric
>
> https://issues.apache.org/jira/browse/IGNITE-13467
> Add events for snapshot operation state
>
> https://issues.apache.org/jira/browse/IGNITE-13535
> Support cluster snapshot security permissions
>
> https://issues.apache.org/jira/browse/IGNITE-13426
> Add command to control.(sh|bin) to get an arbitrary SystemView
>
> https://issues.apache.org/jira/browse/IGNITE-13327
> Add a metric for processed keys when rebuilding indexes.
>
> https://issues.apache.org/jira/browse/IGNITE-13310
> Add tracing of SQL queries.
>
> https://issues.apache.org/jira/browse/IGNITE-12380
> Add event for node validation failure.
>
> https://issues.apache.org/jira/browse/IGNITE-7623
> Thin client Java API - async API
>
> https://issues.apache.org/jira/browse/IGNITE-13380
> Output IgniteSystemProperties via ignite.sh
>
> https://issues.apache.org/jira/browse/IGNITE-11731
> CPP: Implement Cluster API
>
> https://issues.apache.org/jira/browse/IGNITE-13308
> C++: Thin client transactions
>
> https://issues.apache.org/jira/browse/IGNITE-12754
> .NET: Thin Client: Service invocation
>
> On Fri, 25 Dec 2020 at 22:43, Никита Сафонов 
> wrote:
> >
> > Hi Denis, Maxim,
> >
> > I'd like to clarify that I'm pretty sure to complete the tasks below
> before
> > the voting date:
> >
> > https://issues.apache.org/jira/browse/IGNITE-13659
> > https://issues.apache.org/jira/browse/IGNITE-13796
> >
> > I'll start to work on them right after the long Russian NY holidays.
> > But I'm not quite sure that we'll meet the deadline with other tickets
> > that should be on the list.
> > So we should probably allocate more time for this work.
> >
> > I'd be glad to get your opinion on this.
> >
> > Regards,
> > Nikita
> >
> > сб, 19 дек. 2020 г. в 11:39, Maxim Muzafarov :
> >
> > > Denis, Nikita
> > >
> > > Yes, the list of important documentation issues will be ready prior to
> > > the scope freeze date. I'll prepare it.
> > >
> > > I mean if we want to start working right now on some documentation
> > > task we can choose any important from those list of opened tasks.
> > >
> > > On Fri, 18 Dec 2020 at 19:55, Denis Magda  wrote:
> > > >
> > > > Maxim,
> > > >
> > > > Presently, there are 70 documentation tickets. Probably, these are
> all
> > > the
> > > > tickets that we are moving from a release to a release. Could you
> help to
> > > > prepare a list of 2.10-specific tickets (features, enhancements,
> required
> > > > updates due to a change in the behavior)? As before, we can track
> these
> > > > important 2.10-specific tickets under the "Documentation tasks for
> > > > important tasks" features:
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
> > > >
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Fri, Dec 18, 2020 at 8:14 AM Maxim Muzafarov 
> > > wrote:
> > > >
> > > > > Hello Nikita,
> > > > >
> > > > > Thanks for your support.
> > > > >
> > > > > You can find the 2.10 related documentation issues on the release
> wiki
> > > > > page [1]. Currently, some of the important issues 

[COMMUNITY] Recognizing those who contribute to user support and project awareness

2020-12-29 Thread Kseniya Romanova
Hi, Igniters!


ASF welcomes all types of contributions, including responses to user
questions and promotion of projects. And, the GriGain DelRel team knows
that people spend a lot of their private time on such activities. Our code
and documentation contributors are recognized by being promoted to the role
of " committer." But, unfortunately, we don't have a formal way to
recognize our user-support and project-promotion contributors.


In 2020, more than 40 people contributed their time and experience to
promote project visibility and encourage adoption by developers:



   -

   59 talks to 2400 developers and architects at meetup and conference
   events
   -

   60+ blog posts and tutorials
   -

   2000+ questions answered on the User list and Stackoverflow


Let’s praise some of our most active Ignite supporters in 2020:



   -

   In user support, Ilya Kasnacheev is our absolute user support champion.
   He alone answered 756 questions on Stackoverflow and the User list.
   -

   Val Kulichenko posted 9 articles and presented at 6 meetups.
   -

   Denis Magda created 10 demos and 4 tutorials and presented at 14 meetups
   and 7 conferences.
   -

   Alexey Zinoviev wrote 4 articles about Ignite ML and presented at 1
   meetup and 1 conference.
   -

   Pavel Tupitsyn created 4 articles and helped Ignite users on the mailing
   list and Stackoverflow.
   -

   Saikat Maitra presented at 2 conferences.
   -

   Ivan Pavlukhin helped many new developers to start contributing to
   Ignite.


As a sign of our gratitude for their efforts, these contributors will
receive special New Year gifts.


In 2021, we want to track this type of contribution and ensure that the
contributors are recognized within the community. The Alfa version of our
tracking system is already functioning inside GridGain, so the next step is
to upgrade the current system and start a discussion with PMC members.
Those who help to attract and retain users are definitely worthy of praise
and promotion, as the value of their contribution matches the value that is
provided through code and documentation.


Keep being awesome and have a Happy New Year!


Kseniya

DevRel at GridGain


[jira] [Created] (IGNITE-13932) Extend Ignite thread pools documentation

2020-12-29 Thread Nikita Safonov (Jira)
Nikita Safonov created IGNITE-13932:
---

 Summary: Extend Ignite thread pools documentation
 Key: IGNITE-13932
 URL: https://issues.apache.org/jira/browse/IGNITE-13932
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.9
Reporter: Nikita Safonov
Assignee: Nikita Safonov


There are references in the code to three thread pools that don't appear to be 
referenced in the Ignite thread pool documentation at 
[https://apacheignite.readme.io/docs/thread-pools]. 
 
These are:
 
asyncCallbackThreadPoolSize
managementThreadPoolSize
utilityCacheThreadPoolSize

 
By default these pools will be set to Max(8, Environment.ProcessorCount) []we 
use the IA C# client]
 
Note: These configurations are referenced in the Ignite Javadoc pages, but the 
descriptions are just single sentences mirroring the names of the thread pools 
themselves.
 
The thread pool documentation should be extended to include these additional 
pools, particularly with respect to their purpose and roles, and the 
circumstances when you would change them from the default value.
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [ANNOUNCE] Apache Ignite 2.9.1 Released

2020-12-29 Thread Nikolay Izhikov
Great news!

Congrats to all the community with one more release!
Yaroslav and Maxim thanks for driving it!

> 29 дек. 2020 г., в 15:17, Yaroslav Molochkov  
> написал(а):
> 
> The Apache Ignite Community is pleased to announce the release of
> Apache Ignite 2.9.1.
> 
> Apache Ignite [1] is a memory-centric distributed database, caching
> and processing platform for transactional, analytical, and streaming
> workloads delivering in-memory speeds at petabyte scale.
> 
> This release fixes several critical issues, please, see the release notes:
> https://ignite.apache.org/releases/2.9.1/release_notes.html
> 
> Download the latest Ignite version here:
> https://ignite.apache.org/download.cgi
> 
> Please let us know [2] if you encounter any problems.
> 
> Regards,
> Yaroslav Molochkov
> 
> [1] https://ignite.apache.org
> [2] https://ignite.apache.org/community/resources.html#ask



[ANNOUNCE] Apache Ignite 2.9.1 Released

2020-12-29 Thread Yaroslav Molochkov
The Apache Ignite Community is pleased to announce the release of
Apache Ignite 2.9.1.

Apache Ignite [1] is a memory-centric distributed database, caching
and processing platform for transactional, analytical, and streaming
workloads delivering in-memory speeds at petabyte scale.

This release fixes several critical issues, please, see the release notes:
https://ignite.apache.org/releases/2.9.1/release_notes.html

Download the latest Ignite version here:
https://ignite.apache.org/download.cgi

Please let us know [2] if you encounter any problems.

Regards,
Yaroslav Molochkov

[1] https://ignite.apache.org
[2] https://ignite.apache.org/community/resources.html#ask


[jira] [Created] (IGNITE-13931) .NET: Service can't assign correct type to passed array parameters (.Net -> .Net call)

2020-12-29 Thread Nikolay Izhikov (Jira)
Nikolay Izhikov created IGNITE-13931:


 Summary: .NET: Service can't assign correct type to passed array 
parameters (.Net -> .Net call)
 Key: IGNITE-13931
 URL: https://issues.apache.org/jira/browse/IGNITE-13931
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.9
Reporter: Yaroslav Molochkov
Assignee: Nikolay Izhikov
 Fix For: 2.10


This issue relates to IGNITE-12823.

Ignite client calls deployed service and fails with the following exception:
{code:java}
Apache.Ignite.Core.Common.JavaException: class 
org.apache.ignite.IgniteException: Failed to invoke proxy: there is no method 
'processSomething' in type 'SomeService' with (Int32, Object[]) arguments
at 
org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.inLongOutLong(Native
 Method)
at 
org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.serviceInvokeMethod(PlatformCallbackGateway.java:942)
at 
org.apache.ignite.internal.processors.platform.services.PlatformAbstractService.invokeMethod(PlatformAbstractService.java:214)
at 
org.apache.ignite.internal.processors.platform.services.PlatformAbstractService.invokeMethod(PlatformAbstractService.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.ignite.internal.processors.service.GridServiceProxy$ServiceProxyCallable.callService(GridServiceProxy.java:491)
at 
org.apache.ignite.internal.processors.service.GridServiceProxy$ServiceProxyCallable.call(GridServiceProxy.java:469)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1847)
at 
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:598)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7077)
at 
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:592)
at 
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:521)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1368)
at 
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:2130)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:241)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at 
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
Service interface itself looks like this:
{code:java}
public interface SomeService : IService
{
int processSomething(int foo, Parameter[] parameters);

int processSomething(int foo, int bar);
}

public class Parameter
{

private int id;

private ParamValue[] _values;


public Parameter(int id, ParamValue[] _values)
{
 
   this.id = id;

   this._values = _values;

}

}



public class ParamValue
{

private int id;

private double val;


public ParamValue(int id, double val)
{
 
   this.id = id;
   
   this.val = val;
  
}

}

{code}
And the call to the service:
{code:java}
var svc = igniteServices.GetServiceProxy(name, false);
var result = svc.processSomething(id, parameters);
{code}
Please note, that if there's no overloaded methods, at least in our experiments 
we found out that it does reproduce with equal number of parameters in 
overloaded methods (e.g. here we have overloaded methods that take 2 parameters 
each), then the code works like a charm.

Fix in IGNITE-12823 addresses particular code execution path where the 
execution flow goes through PlatformServices class. Yet in this case our code 
goes through PlatformAbstractService. I think that the fix of casting arrays 
should be positioned a little bit lower in the call stack (or 

Re: [RESULT] [VOTE] Release Apache Ignite 2.9.1 (RC1)

2020-12-29 Thread Yaroslav Molochkov
Sure, I will do it right away, thanks a lot!

On Tue, Dec 29, 2020 at 2:02 PM Maxim Muzafarov  wrote:

> Folks,
>
> I've finished (with some help) all the release steps.
> I think we are ready for the announcement.
>
>
> Yaroslav,
>
> Can you check that everything is ok?
>
> On Mon, 28 Dec 2020 at 13:47, Ilya Kasnacheev 
> wrote:
> >
> > Hello!
> >
> > Please make sure that it gets pushed before the long holidays. I don't
> > think we can withhold a successfully voted release for so long.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > чт, 24 дек. 2020 г. в 19:44, Yaroslav Molochkov :
> >
> > > Me and mainly Maxim (since I don’t have sufficient permissions) are
> > > working on it, there were a couple of mishaps with docs (due to docs
> were
> > > in a separate branch for 2.9) and access rights (bintray in particular)
> > > that delayed us but rest assured the release will be available ASAP.
> > >
> > > Sorry for the inconvenience, guys.
> > >
> > > > On 24 Dec 2020, at 17:11, Ilya Kasnacheev  >
> > > wrote:
> > > >
> > > > Hello!
> > > >
> > > > Where's the release then? Almost a week has passed but it's not
> available
> > > > for download on ignite.apache.org. Neither in Maven.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > пт, 18 дек. 2020 г. в 22:56, Yaroslav Molochkov <
> molochko...@gmail.com>:
> > > >
> > > >> Hello, Igniters!
> > > >>
> > > >> Apache Ignite 2.9.1 release (RC1) has been accepted.
> > > >>
> > > >> 6 "+1" votes received.
> > > >>
> > > >> Here are the votes received (in order received):
> > > >>
> > > >>   - Pavel Tupitsyn (binding)
> > > >>   - Ilya Kasnacheev (binding)
> > > >>   - Nikolay Izhikov (binding)
> > > >>   - Denis Magda (binding)
> > > >>   - Alex Plehanov (binding)
> > > >>   - Maxim Muzafarov (binding)
> > > >>
> > > >> Here is the link to the voting thread -
> > > >>
> > > >>
> > >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-Apache-Ignite-2-9-1-RC1-td50653.html
> > > >>
> > > >> Thank you!
> > > >>
> > >
>


[jira] [Created] (IGNITE-13930) Update pom depencencies to 2.11.0-SNAPSHOT version

2020-12-29 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-13930:


 Summary: Update pom depencencies to 2.11.0-SNAPSHOT version
 Key: IGNITE-13930
 URL: https://issues.apache.org/jira/browse/IGNITE-13930
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov


The pom dependencies must be updated due to the ignite-2.10 branch has been 
created.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: IEP-54: Schema-first approach for 3.0

2020-12-29 Thread Andrey Mashenkov
Michael, thanks for feedback.

1. However, there is no decision approved by the community, but I heard a
lot about it would be nice to have one table\schema per cache.
Actually, we already have CacheGroup, a very useful feature that allows us
to share memory region/persistence files between caches and resolves many
performance issues.
AFAIK, CacheGroup idea will be present in Ignite 3.0, but may be slightly
reworked regarding a new ideology.

Cache with arbitrary objects was always a headache and this is totally
unusable for SQL.
You ask a good question about pure KV-case. I think we could allow users to
create a schemaless table for arbitrary objects, but it will NOT be
possible to use it in SQL.
There is a big question how to serialize arbitrary objects correctly and
effectively. I suggest storing arbitrary objects as byte[] or kind of
blobs.

Looking forward, users will want to access arbitrary fields of such objects
and will need a BinaryObject interface, but on this step we have to deal
with some schema...
So, I think we can go with one of next ways
* treating KV objects as blob/byte[]
* strict schema approach, schema is created and propagated to grid on first
object put and can never be changed until table destroyed.
* use some kind of BinaryObject with a schema in footer, that may have huge
overhead.

2. One can use SQL or SchemaBuilder or generate schema from Class using
annotations.
I thought it should be possible to pass the result of the schema
builder/generator to initial configuration or to createTable(schema)
method.
What kind of script do you mean?

3. Generally, no. With a strict schema mode it is not possible to do it.
With live-schema mode we could make some trivial conversions e.g. int->long
transparently, but change int->date looks impossible for automatic
conversion in run-time.
Offline conversion utils for persistence files look possible for any
changes.

I think the next sequence of user actions can be applied to change field
type:
* Add new_field with new type. This will up a schema version.
* Change mapping on all nodes nodes to write to new_field, but read from
old_field then convert if needed.
This allows the user to read/convert values saved before the rising schema
version.
* Start scan query to copy/convert old_field data to new one. No node will
write with the old schema version at this point.
* Old_field can be safely dropped. This will up the schema version.
* Rename new_field to old_field via change mapping on nodes. Yes, once
again.
Obviously, users may need to update Java classes for this. In fact, they
will do it twice.

Ok, a rename field command should be added to the schema modification API.

4. We don't need BinaryMarshaller with a new approach. What purpose do you
need a BinaryMarshaller for?
5. Yes, strict schema mode is what you are looking for.


On Tue, Dec 29, 2020 at 10:22 AM Michael Cherkasov <
michael.cherka...@gmail.com> wrote:

> Hi all,
>
> I reviewed the mail thread and proposal page and I still don't fully
> understand what is going to be changed, I would really appreciate it if you
> will answer a few questions:
>
> 1. Are you going to leave only one schema per cache? if so, will be there
> an option to have a table with arbitrary objects(pure KV case)?
> 2. What options will Apache Ignite 3.0 have to define schema? SchemaBuilder
> and SQL only? Is there an option to put the schema definition to the
> configuration?(I really don't like this, I would prefer to have
> separate scripts to create schemas)
> 3. Is there a way to change field type? if yes, can it be done in runtime?
> 4. Looks like BinaryMarshaller is going to be re-worked too, is there any
> IEP for this?
> 5. I don't like automatic schema evaluation when a new field is added
> automatically on record put, so is there a way to prohibit this behavior?
>  I think all schema changes should be done only explicitly except initial
> schema creation.
>
> Thanks,
> Mike.
>
> пн, 21 дек. 2020 г. в 06:40, Andrey Mashenkov  >:
>
> > Hi, Igniters.
> >
> > We all know that the current QueryEntity API is not convenient and needs
> to
> > be reworked.
> > So, I'm glad to share PR [1] with schema configuration public API for
> > Ignite 3.0.
> >
> > New schema configuration uses Builder pattern, which looks more
> comfortable
> > to use.
> >
> > In the PR you will find a 'schema' package with the API itself, and a
> draft
> > implementation in 'internal' sub-package,
> > and a test that demonstrates how the API could be used.
> >
> > Please note:
> >
> > * Entrypoint is 'SchemaBuilders' class with static factory methods.
> > * The implementation is decoupled and can be easily extracted to separate
> > module if we decide to do so.
> > * Some columns types (e.g. Date/Time) are missed, they will be added
> lately
> > in separate tickes.
> > * Index configuration extends marker interface that makes possible to
> > implement indexes of new types in plugins.
> > Hopfully, we could add a persistent geo-indices 

Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

2020-12-29 Thread Maxim Muzafarov
Folks,

The branch `ignite-2.10` for the release has been created.

The list of issues related to 2.10 release [1]. If I excluded your
issue you are working on from the release feel free to set the
`fixVersion` back.


[1]  https://s.apache.org/qa6kk

On Tue, 29 Dec 2020 at 11:43, Maxim Muzafarov  wrote:
>
> Folks,
>
> 1.
> I've prepared the list of unhandled important documentation issues for
> the 2.10 release, please check the wiki page link [1]. Note, that some
> of the important (from my point of view) documentation issues even not
> created.
>
> 2.
> Here is the list of resolved issues which seems to be documented, but
> an appropriate documentation task doesn't exist. Should we create
> appropriate documentation issues for them? Check the list [3].
>
> 3.
> Some of the documentation pages must be completed prior to 2.10
> release or removed, right? Check the [2] page for example.
>
>
> Denis, Nikita
>
> > So we should probably allocate more time for this work.
> Let's start the work, track the progress and looks at how it goes.
> I'll try to ask some folks for the help with documentation and also
> plan to help with the issues by myself. We can shift the dates if it's
> necessary, but currently, I see no reasons.
>
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
> [2] https://ignite.apache.org/docs/latest/cpp-specific/
> [3] The list of issues without documentation tasks :
>
> https://issues.apache.org/jira/browse/IGNITE-13488
> Add command to control.(sh|bin) to get an arbitrary Metric
>
> https://issues.apache.org/jira/browse/IGNITE-13467
> Add events for snapshot operation state
>
> https://issues.apache.org/jira/browse/IGNITE-13535
> Support cluster snapshot security permissions
>
> https://issues.apache.org/jira/browse/IGNITE-13426
> Add command to control.(sh|bin) to get an arbitrary SystemView
>
> https://issues.apache.org/jira/browse/IGNITE-13327
> Add a metric for processed keys when rebuilding indexes.
>
> https://issues.apache.org/jira/browse/IGNITE-13310
> Add tracing of SQL queries.
>
> https://issues.apache.org/jira/browse/IGNITE-12380
> Add event for node validation failure.
>
> https://issues.apache.org/jira/browse/IGNITE-7623
> Thin client Java API - async API
>
> https://issues.apache.org/jira/browse/IGNITE-13380
> Output IgniteSystemProperties via ignite.sh
>
> https://issues.apache.org/jira/browse/IGNITE-11731
> CPP: Implement Cluster API
>
> https://issues.apache.org/jira/browse/IGNITE-13308
> C++: Thin client transactions
>
> https://issues.apache.org/jira/browse/IGNITE-12754
> .NET: Thin Client: Service invocation
>
> On Fri, 25 Dec 2020 at 22:43, Никита Сафонов  
> wrote:
> >
> > Hi Denis, Maxim,
> >
> > I'd like to clarify that I'm pretty sure to complete the tasks below before
> > the voting date:
> >
> > https://issues.apache.org/jira/browse/IGNITE-13659
> > https://issues.apache.org/jira/browse/IGNITE-13796
> >
> > I'll start to work on them right after the long Russian NY holidays.
> > But I'm not quite sure that we'll meet the deadline with other tickets
> > that should be on the list.
> > So we should probably allocate more time for this work.
> >
> > I'd be glad to get your opinion on this.
> >
> > Regards,
> > Nikita
> >
> > сб, 19 дек. 2020 г. в 11:39, Maxim Muzafarov :
> >
> > > Denis, Nikita
> > >
> > > Yes, the list of important documentation issues will be ready prior to
> > > the scope freeze date. I'll prepare it.
> > >
> > > I mean if we want to start working right now on some documentation
> > > task we can choose any important from those list of opened tasks.
> > >
> > > On Fri, 18 Dec 2020 at 19:55, Denis Magda  wrote:
> > > >
> > > > Maxim,
> > > >
> > > > Presently, there are 70 documentation tickets. Probably, these are all
> > > the
> > > > tickets that we are moving from a release to a release. Could you help 
> > > > to
> > > > prepare a list of 2.10-specific tickets (features, enhancements, 
> > > > required
> > > > updates due to a change in the behavior)? As before, we can track these
> > > > important 2.10-specific tickets under the "Documentation tasks for
> > > > important tasks" features:
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
> > > >
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Fri, Dec 18, 2020 at 8:14 AM Maxim Muzafarov 
> > > wrote:
> > > >
> > > > > Hello Nikita,
> > > > >
> > > > > Thanks for your support.
> > > > >
> > > > > You can find the 2.10 related documentation issues on the release wiki
> > > > > page [1]. Currently, some of the important issues don't have the right
> > > > > priorities, so it's better to sort them first.
> > > > >
> > > > > As for the time estimates, we've previously agreed that documentation
> > > > > should be ready prior to the release voting date. You can find the
> > > > > actual release dates here [2].
> 

[jira] [Created] (IGNITE-13929) Don't print sensitive information in logs by default

2020-12-29 Thread Sergey Uttsel (Jira)
Sergey Uttsel created IGNITE-13929:
--

 Summary: Don't print sensitive information in logs by default
 Key: IGNITE-13929
 URL: https://issues.apache.org/jira/browse/IGNITE-13929
 Project: Ignite
  Issue Type: Improvement
Reporter: Sergey Uttsel
Assignee: Sergey Uttsel


Right now, by default, node prints entries in logs of PME and long running 
operations. It’s not secure, because it disclose sensitive data. However 
printing of entries might help with certain issues such as deadlock. So we can 
print hash of entries in log.

 

 

*Summary of the changes:*
1. IGNITE_TO_STRING_INCLUDE_SENSITIVE is deprecated
2. IGNITE_SENSITIVE_DATA_LOGGING is a new system property with three possible 
values:
"plain" - print as is
"hash" - print hash (primitives are printed as is)
"none" - don’t print anything
3. "hash" is default value
4. If a node starts with explicit IGNITE_TO_STRING_INCLUDE_SENSITIVE the value 
converts to IGNITE_SENSITIVE_DATA_LOGGING:
true -> plain
false -> none



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [RESULT] [VOTE] Release Apache Ignite 2.9.1 (RC1)

2020-12-29 Thread Maxim Muzafarov
Folks,

I've finished (with some help) all the release steps.
I think we are ready for the announcement.


Yaroslav,

Can you check that everything is ok?

On Mon, 28 Dec 2020 at 13:47, Ilya Kasnacheev  wrote:
>
> Hello!
>
> Please make sure that it gets pushed before the long holidays. I don't
> think we can withhold a successfully voted release for so long.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 24 дек. 2020 г. в 19:44, Yaroslav Molochkov :
>
> > Me and mainly Maxim (since I don’t have sufficient permissions) are
> > working on it, there were a couple of mishaps with docs (due to docs were
> > in a separate branch for 2.9) and access rights (bintray in particular)
> > that delayed us but rest assured the release will be available ASAP.
> >
> > Sorry for the inconvenience, guys.
> >
> > > On 24 Dec 2020, at 17:11, Ilya Kasnacheev 
> > wrote:
> > >
> > > Hello!
> > >
> > > Where's the release then? Almost a week has passed but it's not available
> > > for download on ignite.apache.org. Neither in Maven.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 18 дек. 2020 г. в 22:56, Yaroslav Molochkov :
> > >
> > >> Hello, Igniters!
> > >>
> > >> Apache Ignite 2.9.1 release (RC1) has been accepted.
> > >>
> > >> 6 "+1" votes received.
> > >>
> > >> Here are the votes received (in order received):
> > >>
> > >>   - Pavel Tupitsyn (binding)
> > >>   - Ilya Kasnacheev (binding)
> > >>   - Nikolay Izhikov (binding)
> > >>   - Denis Magda (binding)
> > >>   - Alex Plehanov (binding)
> > >>   - Maxim Muzafarov (binding)
> > >>
> > >> Here is the link to the voting thread -
> > >>
> > >>
> > http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-Apache-Ignite-2-9-1-RC1-td50653.html
> > >>
> > >> Thank you!
> > >>
> >


Re: IEP-54: Schema-first approach for 3.0

2020-12-29 Thread Andrey Mashenkov
Ilya,

Thanks for feedback.

1. It is a Nested Builder pattern [1]. With a nested builder user will have
a single entry point to build a schema.
When you have a number of builders and one builder need a result of some
another builder, users can be confused and waste a time while looking for
proper implementation.
So, I thought it would be polite to either use a nested builder or have a
factory\helper class for all related builders.
Ok, I've got it.
Mixing nested with chained builders may be non intuitive and your
suggestion (to go the same way with PK as with secondary indices) looks
good.

2. I don't like to pass a builder to another builder because it will look
like you will implicitly use a passed builder.
Actually you can't do anything other than call the "build()" method (of
passed builder) as the method (alretColumn) signature expects some interface
and you can't rely on any expected internal implementation will be ever
passed.
I'd think a Command object should be passed here:
TableModificationBuilder.alretColumn(AlterColumnCommand acc)

Your approach requires an additional interface for Command(s) and some
entry point class to users could find the builder easily.
Thus, I'd proceed with a nested builder approach in that particular case.
JOOQ goes the same way and it looks ok.

3. I think an additional explanation needed here.
We thought that in Ignite 3.0 it should be possible to create schema via
SQL and via pure Java code.
Current QueryEntity has a bad and confusing API, so we decided to rework it
using builders.
Also, we thought about initial schema (as a part of initial configuration)
that can be applied only once on the first grid start and then it can only
be modified.

SchemaTable builder is the way users can create initial
schema/configuration programmatically. It creates some standalone objects
perpesening a schema

Modification builder is the way users can modify the schema like an "ALTER"
SQL command. It creates and applies some sequence of internal commands,
which can be useful (in "live-schema" mode) to create automatic converters
from one schema version to another.
Thus Modification builder produces objects that are wired and interact with
the node internal components.
Modification builders can be requested from Schema manager components that
are out of scope now.

So, you may note these builders are very different, but I think we could
have a similar style in both parts of API and even reuse some
interfaces\objects, As you can see I reuse IndexBuilders.


[1] https://dzone.com/articles/nested-builder



On Tue, Dec 29, 2020 at 11:54 AM Ilya Kazakov 
wrote:

> Hello, Andrei!
>
> I have read this thread and PR. The builder idea and API looks good. But I
> have some questions.
>
> 1.
> In SchemaTableBuilder:
> When I use a builder in chain style, I expect in every step the same result
> type. And as I see this idea is implemented anywhere except pk(), where you
> return PrimryKeyBuilder. As for me, it will be more intuitive if we will
> use something like this:
>
> SchemaTableBuilder.pk(PrimryKeyBuilder pkb)
>
> 2.
> In TableModificationBuilder:
> I have the same question. Some methods returns TableModificationBuilder,
> but alretColumn returns AlterColumnBuilder. As for me, it will be more
> intuitive if we will use something like
>
> TableModificationBuilder.alretColumn(AlterColumnBuilder acb)
>
> But in general, these two points are only a matter of taste.
>
> 3.
> In your examples, I can't understand how we can alter some table, which we
> have already created previously (how we can get its SchemaTable object).
> But maybe this question is out of scope?
>
> --
> Thanks
> Ilya Kazakov
>
> вт, 29 дек. 2020 г. в 15:22, Michael Cherkasov <
> michael.cherka...@gmail.com
> >:
>
> > Hi all,
> >
> > I reviewed the mail thread and proposal page and I still don't fully
> > understand what is going to be changed, I would really appreciate it if
> you
> > will answer a few questions:
> >
> > 1. Are you going to leave only one schema per cache? if so, will be there
> > an option to have a table with arbitrary objects(pure KV case)?
> > 2. What options will Apache Ignite 3.0 have to define schema?
> SchemaBuilder
> > and SQL only? Is there an option to put the schema definition to the
> > configuration?(I really don't like this, I would prefer to have
> > separate scripts to create schemas)
> > 3. Is there a way to change field type? if yes, can it be done in
> runtime?
> > 4. Looks like BinaryMarshaller is going to be re-worked too, is there any
> > IEP for this?
> > 5. I don't like automatic schema evaluation when a new field is added
> > automatically on record put, so is there a way to prohibit this behavior?
> >  I think all schema changes should be done only explicitly except initial
> > schema creation.
> >
> > Thanks,
> > Mike.
> >
> > пн, 21 дек. 2020 г. в 06:40, Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> >
> > > Hi, Igniters.
> > >
> > > We all know that the 

[jira] [Created] (IGNITE-13928) Index defragmentation: only one index defragmented

2020-12-29 Thread Semyon Danilov (Jira)
Semyon Danilov created IGNITE-13928:
---

 Summary: Index defragmentation: only one index defragmented
 Key: IGNITE-13928
 URL: https://issues.apache.org/jira/browse/IGNITE-13928
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Reporter: Semyon Danilov
Assignee: Semyon Danilov
 Fix For: 2.10


In IndexingDefragmentation.java:
{code:java}
// code placeholder
H2TreeIndex oldH2Idx = (H2TreeIndex)indexes.get(2);
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: IEP-54: Schema-first approach for 3.0

2020-12-29 Thread Ilya Kasnacheev
Hello!

Is there a way to define schema via ignitectl utility and its
hierarchical properties? How would it look like?

Regards,
-- 
Ilya Kasnacheev


пн, 21 дек. 2020 г. в 17:40, Andrey Mashenkov :

> Hi, Igniters.
>
> We all know that the current QueryEntity API is not convenient and needs to
> be reworked.
> So, I'm glad to share PR [1] with schema configuration public API for
> Ignite 3.0.
>
> New schema configuration uses Builder pattern, which looks more comfortable
> to use.
>
> In the PR you will find a 'schema' package with the API itself, and a draft
> implementation in 'internal' sub-package,
> and a test that demonstrates how the API could be used.
>
> Please note:
>
> * Entrypoint is 'SchemaBuilders' class with static factory methods.
> * The implementation is decoupled and can be easily extracted to separate
> module if we decide to do so.
> * Some columns types (e.g. Date/Time) are missed, they will be added lately
> in separate tickes.
> * Index configuration extends marker interface that makes possible to
> implement indexes of new types in plugins.
> Hopfully, we could add a persistent geo-indices support in future.
> * Supposedly, current table schema can be changed via builder-like
> structure as it is done if JOOQ project. See 'TableModificationBuilder' for
> details.
> I'm not sure 'SchemaTable' should have 'toBuilder()' converter for that
> purpose as it is a Schema Manager responsibility to create mutator objects
> from the current schema,
> but implementing the Schema manager is out of scope and will be designed
> within the next task.
> * Interfaces implementations are out of scope. I did not intend to merge
> them right now, but for test/demostration purposes.
>
> It is NOT the final version and some may be changed before the first
> release of course.
> For now, we have to agree if we can proceed with this approach or some
> issues should be resolved at first.
>
> Any thoughts or objections?
> Are interfaces good enough to be merged within the current ticket?
>
>
> https://issues.apache.org/jira/browse/IGNITE-13748
>
> On Thu, Nov 26, 2020 at 2:33 PM Юрий  wrote:
>
> > A little bit my thoughts about unsigned types:
> >
> > 1. Seems we may support unsign types
> > 2. It requires adding new types to the internal representation, protocol,
> > e.t.c.
> > 3. internal representation should be the same as we keep sign types. So
> it
> > will not requires more memory
> > 4. User should be aware of specifics such types for platforms which not
> > support unsigned types. For example, a user could derive -6 value in Java
> > for 250 unsigned byte value (from bits perspective will be right). I
> think
> > We shouldn't use more wide type for such cases, especially it will be bad
> > for unsigned long when we require returns BigInteger type.
> > 5. Possible it requires some suffix/preffix for new types like a '250u' -
> > it means that 250 is an unsigned value type.
> > 6. It requires a little bit more expensive comparison logic for indexes
> > 7. It requires new comparison logic for expressions. I think it not
> > possible for the current H2 engine and probably possible for the new
> > Calcite engine. Need clarification from anybody who involved in this part
> >
> > WDYT?
> >
> > вт, 24 нояб. 2020 г. в 18:36, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > > Actually, we can support comparisons in 3.0: once we the actual type
> > > information, we can make proper runtime adjustments and conversions to
> > > treat those values as unsigned - it will be just a bit more expensive.
> > >
> > > вт, 24 нояб. 2020 г. в 18:32, Pavel Tupitsyn :
> > >
> > > > > SQL range queries it will break
> > > > > WHERE x > y may return wrong results
> > > >
> > > > Yes, range queries, inequality comparisons and so on are broken
> > > > for unsigned data types, I think I mentioned this somewhere above.
> > > >
> > > > Again, in my opinion, we can document that SQL is not supported on
> > those
> > > > types,
> > > > end of story.
> > > >
> > > > On Tue, Nov 24, 2020 at 6:25 PM Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com>
> > > > wrote:
> > > >
> > > > > Folks, I think this is a reasonable request. I thought about this
> > when
> > > I
> > > > > was drafting the IEP, but hesitated to add these types right away.
> > > > >
> > > > > > That is how it works in Ignite since the beginning with .NET and
> > C++
> > > :)
> > > > > I have some doubts that it actually works as expected, it needs
> some
> > > > > checking (will be glad if my concerns are false):
> > > > >
> > > > >- It's true that equality check works properly, but for SQL
> range
> > > > >queries it will break unless some special care is taken on Java
> > > side:
> > > > > for
> > > > >u8 255 > 10, but in Java (byte)255 will be converted to -1,
> which
> > > will
> > > > >break the comparison. Since we don't have unsigned types now, I
> > > doubt
> > > > it
> > > > >works.
> > > > >- There is an obvious 

Re: IEP-54: Schema-first approach for 3.0

2020-12-29 Thread Ilya Kazakov
Hello, Andrei!

I have read this thread and PR. The builder idea and API looks good. But I
have some questions.

1.
In SchemaTableBuilder:
When I use a builder in chain style, I expect in every step the same result
type. And as I see this idea is implemented anywhere except pk(), where you
return PrimryKeyBuilder. As for me, it will be more intuitive if we will
use something like this:

SchemaTableBuilder.pk(PrimryKeyBuilder pkb)

2.
In TableModificationBuilder:
I have the same question. Some methods returns TableModificationBuilder,
but alretColumn returns AlterColumnBuilder. As for me, it will be more
intuitive if we will use something like

TableModificationBuilder.alretColumn(AlterColumnBuilder acb)

But in general, these two points are only a matter of taste.

3.
In your examples, I can't understand how we can alter some table, which we
have already created previously (how we can get its SchemaTable object).
But maybe this question is out of scope?

--
Thanks
Ilya Kazakov

вт, 29 дек. 2020 г. в 15:22, Michael Cherkasov :

> Hi all,
>
> I reviewed the mail thread and proposal page and I still don't fully
> understand what is going to be changed, I would really appreciate it if you
> will answer a few questions:
>
> 1. Are you going to leave only one schema per cache? if so, will be there
> an option to have a table with arbitrary objects(pure KV case)?
> 2. What options will Apache Ignite 3.0 have to define schema? SchemaBuilder
> and SQL only? Is there an option to put the schema definition to the
> configuration?(I really don't like this, I would prefer to have
> separate scripts to create schemas)
> 3. Is there a way to change field type? if yes, can it be done in runtime?
> 4. Looks like BinaryMarshaller is going to be re-worked too, is there any
> IEP for this?
> 5. I don't like automatic schema evaluation when a new field is added
> automatically on record put, so is there a way to prohibit this behavior?
>  I think all schema changes should be done only explicitly except initial
> schema creation.
>
> Thanks,
> Mike.
>
> пн, 21 дек. 2020 г. в 06:40, Andrey Mashenkov  >:
>
> > Hi, Igniters.
> >
> > We all know that the current QueryEntity API is not convenient and needs
> to
> > be reworked.
> > So, I'm glad to share PR [1] with schema configuration public API for
> > Ignite 3.0.
> >
> > New schema configuration uses Builder pattern, which looks more
> comfortable
> > to use.
> >
> > In the PR you will find a 'schema' package with the API itself, and a
> draft
> > implementation in 'internal' sub-package,
> > and a test that demonstrates how the API could be used.
> >
> > Please note:
> >
> > * Entrypoint is 'SchemaBuilders' class with static factory methods.
> > * The implementation is decoupled and can be easily extracted to separate
> > module if we decide to do so.
> > * Some columns types (e.g. Date/Time) are missed, they will be added
> lately
> > in separate tickes.
> > * Index configuration extends marker interface that makes possible to
> > implement indexes of new types in plugins.
> > Hopfully, we could add a persistent geo-indices support in future.
> > * Supposedly, current table schema can be changed via builder-like
> > structure as it is done if JOOQ project. See 'TableModificationBuilder'
> for
> > details.
> > I'm not sure 'SchemaTable' should have 'toBuilder()' converter for that
> > purpose as it is a Schema Manager responsibility to create mutator
> objects
> > from the current schema,
> > but implementing the Schema manager is out of scope and will be designed
> > within the next task.
> > * Interfaces implementations are out of scope. I did not intend to merge
> > them right now, but for test/demostration purposes.
> >
> > It is NOT the final version and some may be changed before the first
> > release of course.
> > For now, we have to agree if we can proceed with this approach or some
> > issues should be resolved at first.
> >
> > Any thoughts or objections?
> > Are interfaces good enough to be merged within the current ticket?
> >
> >
> > https://issues.apache.org/jira/browse/IGNITE-13748
> >
> > On Thu, Nov 26, 2020 at 2:33 PM Юрий 
> wrote:
> >
> > > A little bit my thoughts about unsigned types:
> > >
> > > 1. Seems we may support unsign types
> > > 2. It requires adding new types to the internal representation,
> protocol,
> > > e.t.c.
> > > 3. internal representation should be the same as we keep sign types. So
> > it
> > > will not requires more memory
> > > 4. User should be aware of specifics such types for platforms which not
> > > support unsigned types. For example, a user could derive -6 value in
> Java
> > > for 250 unsigned byte value (from bits perspective will be right). I
> > think
> > > We shouldn't use more wide type for such cases, especially it will be
> bad
> > > for unsigned long when we require returns BigInteger type.
> > > 5. Possible it requires some suffix/preffix for new types like a
> '250u' -
> > > it means that 250 is an 

Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)

2020-12-29 Thread Maxim Muzafarov
Folks,

1.
I've prepared the list of unhandled important documentation issues for
the 2.10 release, please check the wiki page link [1]. Note, that some
of the important (from my point of view) documentation issues even not
created.

2.
Here is the list of resolved issues which seems to be documented, but
an appropriate documentation task doesn't exist. Should we create
appropriate documentation issues for them? Check the list [3].

3.
Some of the documentation pages must be completed prior to 2.10
release or removed, right? Check the [2] page for example.


Denis, Nikita

> So we should probably allocate more time for this work.
Let's start the work, track the progress and looks at how it goes.
I'll try to ask some folks for the help with documentation and also
plan to help with the issues by myself. We can shift the dates if it's
necessary, but currently, I see no reasons.


[1] 
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
[2] https://ignite.apache.org/docs/latest/cpp-specific/
[3] The list of issues without documentation tasks :

https://issues.apache.org/jira/browse/IGNITE-13488
Add command to control.(sh|bin) to get an arbitrary Metric

https://issues.apache.org/jira/browse/IGNITE-13467
Add events for snapshot operation state

https://issues.apache.org/jira/browse/IGNITE-13535
Support cluster snapshot security permissions

https://issues.apache.org/jira/browse/IGNITE-13426
Add command to control.(sh|bin) to get an arbitrary SystemView

https://issues.apache.org/jira/browse/IGNITE-13327
Add a metric for processed keys when rebuilding indexes.

https://issues.apache.org/jira/browse/IGNITE-13310
Add tracing of SQL queries.

https://issues.apache.org/jira/browse/IGNITE-12380
Add event for node validation failure.

https://issues.apache.org/jira/browse/IGNITE-7623
Thin client Java API - async API

https://issues.apache.org/jira/browse/IGNITE-13380
Output IgniteSystemProperties via ignite.sh

https://issues.apache.org/jira/browse/IGNITE-11731
CPP: Implement Cluster API

https://issues.apache.org/jira/browse/IGNITE-13308
C++: Thin client transactions

https://issues.apache.org/jira/browse/IGNITE-12754
.NET: Thin Client: Service invocation

On Fri, 25 Dec 2020 at 22:43, Никита Сафонов  wrote:
>
> Hi Denis, Maxim,
>
> I'd like to clarify that I'm pretty sure to complete the tasks below before
> the voting date:
>
> https://issues.apache.org/jira/browse/IGNITE-13659
> https://issues.apache.org/jira/browse/IGNITE-13796
>
> I'll start to work on them right after the long Russian NY holidays.
> But I'm not quite sure that we'll meet the deadline with other tickets
> that should be on the list.
> So we should probably allocate more time for this work.
>
> I'd be glad to get your opinion on this.
>
> Regards,
> Nikita
>
> сб, 19 дек. 2020 г. в 11:39, Maxim Muzafarov :
>
> > Denis, Nikita
> >
> > Yes, the list of important documentation issues will be ready prior to
> > the scope freeze date. I'll prepare it.
> >
> > I mean if we want to start working right now on some documentation
> > task we can choose any important from those list of opened tasks.
> >
> > On Fri, 18 Dec 2020 at 19:55, Denis Magda  wrote:
> > >
> > > Maxim,
> > >
> > > Presently, there are 70 documentation tickets. Probably, these are all
> > the
> > > tickets that we are moving from a release to a release. Could you help to
> > > prepare a list of 2.10-specific tickets (features, enhancements, required
> > > updates due to a change in the behavior)? As before, we can track these
> > > important 2.10-specific tickets under the "Documentation tasks for
> > > important tasks" features:
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures
> > >
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Dec 18, 2020 at 8:14 AM Maxim Muzafarov 
> > wrote:
> > >
> > > > Hello Nikita,
> > > >
> > > > Thanks for your support.
> > > >
> > > > You can find the 2.10 related documentation issues on the release wiki
> > > > page [1]. Currently, some of the important issues don't have the right
> > > > priorities, so it's better to sort them first.
> > > >
> > > > As for the time estimates, we've previously agreed that documentation
> > > > should be ready prior to the release voting date. You can find the
> > > > actual release dates here [2].
> > > >
> > > > [1]
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Allunresolveddocumentationtasks
> > > > [2]
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Releasephases
> > > >
> > > > On Fri, 18 Dec 2020 at 18:48, Никита Сафонов <
> > vlasovpavel2...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi guys,
> > > > >
> > > > > I'd be more than happy to support the 2.10 release and contribute to
> > the
> > > > > documentation.
> > > > > So could you please give