[jira] [Created] (IGNITE-13998) [Spark] Upgrade ignite-spark to Spark 3.0.1/Scala 2.12 required for Java 11

[Rajkumar Arumugham (Jira)]

Rajkumar Arumugham created IGNITE-13998:
---

 Summary: [Spark] Upgrade ignite-spark to Spark 3.0.1/Scala 2.12 
required for Java 11
 Key: IGNITE-13998
 URL: https://issues.apache.org/jira/browse/IGNITE-13998
 Project: Ignite
  Issue Type: Sub-task
  Components: spark
Affects Versions: 2.8
Reporter: Rajkumar Arumugham
Assignee: Alexey Zinoviev
 Fix For: 2.8


This solution provides the initial support of spark 2.4 with copied codebase 
from spark and with initial support of ExternalCatalog refactoring



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: SHA-512 for Maven deployment

[Andrey Mashenkov]

Val, I didn't found the way to make a local deploy. So I just make
'install'.

Yes you are right, only source jar is signed.
Seems, we need to configure checksum plugin for signing binary jars as it
is done in Maven-parent or any other project.

чт, 14 янв. 2021 г., 23:14 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Andrey,
>
> Did you try on the 2.x or 3.x?
>
> I've just tried to do the same in ignite-3, but it didn't work for me. I've
> updated the parent pom version to 23 and ran "mvn clean deploy
> -Papache-release". The source package is now signed with SHA512, which is
> good, but there was no effect on the JAR artifacts. As a matter of fact, I
> don't see any checksum files for them. My guess is that by default they are
> generated by the deploy plugin, during the upload to Maven. Here is the
> resulting staging (still MD5 and SHA1):
> https://repository.apache.org/content/repositories/orgapacheignite-1505/
>
> Does it behave in the same way for you?
>
> -Val
>
> On Thu, Jan 14, 2021 at 3:30 AM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> wrote:
>
> > I've made "mvn clean install" with enabled "apache-release" profile and
> see
> > *.sha-512 checksum files in target directories.
> > So, upgrading to the latest apache parent looks sufficient.
> >
> >
> > On Thu, Jan 14, 2021 at 12:30 PM Petr Ivanov 
> wrote:
> >
> > > Is seems that parent is already updated in
> > > https://issues.apache.org/jira/browse/IGNITE-13987 <
> > > https://issues.apache.org/jira/browse/IGNITE-13987>
> > >
> > >
> > >
> > > > On 14 Jan 2021, at 01:57, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > > Andrey,
> > > >
> > > > This sounds even better. Can you create a ticket for this change?
> > > >
> > > > -Val
> > > >
> > > > On Wed, Jan 13, 2021 at 2:34 PM Andrey Mashenkov <
> > > andrey.mashen...@gmail.com>
> > > > wrote:
> > > >
> > > >> Val,
> > > >>
> > > >> I've just found Maven projects use SHA-512.
> > > >> I passed through commits and found they just switched to newer
> parent
> > > >> org.apache:apache pom.
> > > >> I've compared our current parent pom with the latest available one
> > > >> (org.apache:apache:16 vs org.apache:apache:23)
> > > >> and then found checksum-maven-plugin was added [1] somewhen in
> > between.
> > > >>
> > > >> So, seems we have to switched to newer apache pom and maybe add
> > > >> checksum-maven-plugin
> > > >> to our main pom.
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://github.com/apache/maven-apache-parent/commit/a46aa52b4b56d9b7aa62e1b8cbea5ff0af434a
> > > >>
> > > >> On Wed, Jan 13, 2021 at 10:41 PM Valentin Kulichenko <
> > > >> valentin.kuliche...@gmail.com> wrote:
> > > >>
> > > >>> Hi Andrey,
> > > >>>
> > > >>> This indeed sounds like the cleanest way. I don't know how much
> > effort
> > > >> that
> > > >>> would be though.
> > > >>>
> > > >>> -Val
> > > >>>
> > > >>> On Wed, Jan 13, 2021 at 11:01 AM Andrey Mashenkov <
> > > >>> andrey.mashen...@gmail.com> wrote:
> > > >>>
> > >  Maybe, we could donate to maven plugin possibility to switch to
> > > >> SHA-512.
> > >  Hopefully, a new plugin version will be released before we have
> any
> > > >>> release
> > >  candidate.
> > > 
> > >  Is it looks like a big deal?
> > > 
> > >  ср, 13 янв. 2021 г., 21:32 Valentin Kulichenko <
> > >  valentin.kuliche...@gmail.com>:
> > > 
> > > > Hi Ivan,
> > > >
> > > > No, I haven't found a way yet. SHA1 still works, but I believe we
> > > >>> should
> > > > consider using better options in future releases.
> > > >
> > > > Do you have any ideas on how to implement this?
> > > >
> > > > -Val
> > > >
> > > > On Wed, Jan 13, 2021 at 8:21 AM Ivan Pavlukhin <
> > vololo...@gmail.com>
> > > > wrote:
> > > >
> > > >> Folks,
> > > >>
> > > >> Were you able to resolve this?
> > > >>
> > > >> 2020-12-28 22:15 GMT+03:00, Valentin Kulichenko <
> > > >> valentin.kuliche...@gmail.com>:
> > > >>> Hi Ivan,
> > > >>>
> > > >>> Thanks for your response. I've looked into the PGP plugin, and
> > > >>> unfortunately it looks like it only can create signatures, but
> > > >> not
> > > >>> checksums.
> > > >>>
> > > >>> -Val
> > > >>>
> > > >>> On Sun, Dec 27, 2020 at 11:54 PM Ivan Bessonov <
> > >  bessonov...@gmail.com>
> > > >>> wrote:
> > > >>>
> > >  Hi,
> > > 
> > >  I've never done this before, but it seems like we need
> > > > maven-gpg-plugin
> > >  for
> > >  it [1].
> > > 
> > >  Algorithm configuration would look like this:
> > >  
> > > --digest-algo=SHA512
> > >  
> > > 
> > >  Maybe this will help.
> > > 
> > >  [1]
> > > 
> > > 
> > > >>
> > > >
> > > 
> > > >>>
> > > >>
> > >
> >
> 

Re: Apache Ignite Extensions TeamCity project

[Saikat Maitra]

Hi Petr,

I am able to access the project.

Thank you

Regards,
Saikat

On Wed, Jan 13, 2021 at 2:17 AM Petr Ivanov  wrote:

> Added you to Ignite Test Managers.
> Please check access level.
>
>
> > On 13 Jan 2021, at 03:49, Saikat Maitra  wrote:
> >
> > Hi Petr,
> >
> > Thank you for your response. I am not able to access the project due to
> > error
> >
> > "You do not have enough permissions to access project with id
> > IgniteExtensions_Tests"
> >
> > Can you please provide me access? My username is samaitra
> >
> > Regards,
> > Saikat
> >
> > On Mon, Jan 11, 2021 at 2:55 AM Petr Ivanov  wrote:
> >
> >> Hi.
> >>
> >>
> >> I've refactored the hierarchy of the project and moved it here:
> >> https://ci.ignite.apache.org/project/IgniteExtensions_Tests?mode=builds
> <
> >> https://ci.ignite.apache.org/project/IgniteExtensions_Tests?mode=builds
> >
> >> Check your permissions, please.
> >>
> >>> On 11 Jan 2021, at 01:01, Saikat Maitra 
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> We had a project for IgniteExtensions under Tests main project in
> >> TeamCity.
> >>>
> >>>
> >>
> https://ci.ignite.apache.org/project.html?projectId=Tests_IgniteExtensions
> >>>
> >>> I am no longer able to access the IgniteExtensions project.
> >>>
> >>> Was it refactored and moved to another location?
> >>>
> >>> Regards,
> >>> Saikat
> >>
> >>
>
>

Re: SHA-512 for Maven deployment

[Valentin Kulichenko]

Andrey,

Did you try on the 2.x or 3.x?

I've just tried to do the same in ignite-3, but it didn't work for me. I've
updated the parent pom version to 23 and ran "mvn clean deploy
-Papache-release". The source package is now signed with SHA512, which is
good, but there was no effect on the JAR artifacts. As a matter of fact, I
don't see any checksum files for them. My guess is that by default they are
generated by the deploy plugin, during the upload to Maven. Here is the
resulting staging (still MD5 and SHA1):
https://repository.apache.org/content/repositories/orgapacheignite-1505/

Does it behave in the same way for you?

-Val

On Thu, Jan 14, 2021 at 3:30 AM Andrey Mashenkov 
wrote:

> I've made "mvn clean install" with enabled "apache-release" profile and see
> *.sha-512 checksum files in target directories.
> So, upgrading to the latest apache parent looks sufficient.
>
>
> On Thu, Jan 14, 2021 at 12:30 PM Petr Ivanov  wrote:
>
> > Is seems that parent is already updated in
> > https://issues.apache.org/jira/browse/IGNITE-13987 <
> > https://issues.apache.org/jira/browse/IGNITE-13987>
> >
> >
> >
> > > On 14 Jan 2021, at 01:57, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > >
> > > Andrey,
> > >
> > > This sounds even better. Can you create a ticket for this change?
> > >
> > > -Val
> > >
> > > On Wed, Jan 13, 2021 at 2:34 PM Andrey Mashenkov <
> > andrey.mashen...@gmail.com>
> > > wrote:
> > >
> > >> Val,
> > >>
> > >> I've just found Maven projects use SHA-512.
> > >> I passed through commits and found they just switched to newer parent
> > >> org.apache:apache pom.
> > >> I've compared our current parent pom with the latest available one
> > >> (org.apache:apache:16 vs org.apache:apache:23)
> > >> and then found checksum-maven-plugin was added [1] somewhen in
> between.
> > >>
> > >> So, seems we have to switched to newer apache pom and maybe add
> > >> checksum-maven-plugin
> > >> to our main pom.
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://github.com/apache/maven-apache-parent/commit/a46aa52b4b56d9b7aa62e1b8cbea5ff0af434a
> > >>
> > >> On Wed, Jan 13, 2021 at 10:41 PM Valentin Kulichenko <
> > >> valentin.kuliche...@gmail.com> wrote:
> > >>
> > >>> Hi Andrey,
> > >>>
> > >>> This indeed sounds like the cleanest way. I don't know how much
> effort
> > >> that
> > >>> would be though.
> > >>>
> > >>> -Val
> > >>>
> > >>> On Wed, Jan 13, 2021 at 11:01 AM Andrey Mashenkov <
> > >>> andrey.mashen...@gmail.com> wrote:
> > >>>
> >  Maybe, we could donate to maven plugin possibility to switch to
> > >> SHA-512.
> >  Hopefully, a new plugin version will be released before we have any
> > >>> release
> >  candidate.
> > 
> >  Is it looks like a big deal?
> > 
> >  ср, 13 янв. 2021 г., 21:32 Valentin Kulichenko <
> >  valentin.kuliche...@gmail.com>:
> > 
> > > Hi Ivan,
> > >
> > > No, I haven't found a way yet. SHA1 still works, but I believe we
> > >>> should
> > > consider using better options in future releases.
> > >
> > > Do you have any ideas on how to implement this?
> > >
> > > -Val
> > >
> > > On Wed, Jan 13, 2021 at 8:21 AM Ivan Pavlukhin <
> vololo...@gmail.com>
> > > wrote:
> > >
> > >> Folks,
> > >>
> > >> Were you able to resolve this?
> > >>
> > >> 2020-12-28 22:15 GMT+03:00, Valentin Kulichenko <
> > >> valentin.kuliche...@gmail.com>:
> > >>> Hi Ivan,
> > >>>
> > >>> Thanks for your response. I've looked into the PGP plugin, and
> > >>> unfortunately it looks like it only can create signatures, but
> > >> not
> > >>> checksums.
> > >>>
> > >>> -Val
> > >>>
> > >>> On Sun, Dec 27, 2020 at 11:54 PM Ivan Bessonov <
> >  bessonov...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hi,
> > 
> >  I've never done this before, but it seems like we need
> > > maven-gpg-plugin
> >  for
> >  it [1].
> > 
> >  Algorithm configuration would look like this:
> >  
> > --digest-algo=SHA512
> >  
> > 
> >  Maybe this will help.
> > 
> >  [1]
> > 
> > 
> > >>
> > >
> > 
> > >>>
> > >>
> >
> http://maven.apache.org/plugins-archives/maven-gpg-plugin-LATEST/sign-mojo.html
> > 
> >  пн, 28 дек. 2020 г. в 01:25, Valentin Kulichenko <
> >  valentin.kuliche...@gmail.com>:
> > 
> > > Igniters,
> > >
> > > I've been preparing the 3.0.0-alpha1 release and got confused
> >  about
> > >> the
> > > requirements for checksums in Maven deployments. The Apache
> > >> instruction
> >  [1]
> > > states that MD5 is deprecated and SHA1 should be avoided in
> > >>> favor
> >  of
> > > SHA-256 or SHA-512. However, it looks like we are still using
> > >>> the
> >  MD5/SHA1
> > > combination (at least 

[jira] [Created] (IGNITE-13997) CPP Thin: Transactions can cause deadlock when client shared in multiple threads

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-13997:


 Summary: CPP Thin: Transactions can cause deadlock when client 
shared in multiple threads
 Key: IGNITE-13997
 URL: https://issues.apache.org/jira/browse/IGNITE-13997
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 2.9.1
Reporter: Igor Sapego


In our tests we have detected a deadlock when following piece of code is
executed for more than one thread on our application:

{code:cpp}
ClientTransactions transactions = client.ClientTransactions();
ClientTransaction tx = transactions.TxStart(PESSIMISTIC, READ_COMMITTED);

// This call should atomically get the current value for "key" and put
"value" instead, locking the "key" cache entry at the same time
auto oldValue = cache.GetAndPut(key, value);

// Only the thread able of locking "key" should reach this code. Others have
to wait for tx.Commit() to complete
cache.Put (key, newValue);

// After this call, other thread waiting in GetAndPut for "key" to be
released should be able of continuing
tx.Commit ();
{code:cpp}

The thread reaching "cache.Put (key, newValue);" call, gets blocked in
there, concretely in the lockGuard object created at the beginning of
DataChannel::InternalSyncMessage function (data_channel.cpp:108).  After
debugging, we realized that this lockGuard is owned by a different thread,
which is currently waiting on socket while executing GetAndPut function.
According to this, my guess is that data routing for C++ Thin Clients is not
multithread friendly.  

Reported in 
http://apache-ignite-users.70518.x6.nabble.com/Multithread-transactions-in-a-C-Thin-Client-td35145.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13996) JMX beans in a format not understood by Prometheus

[Stephen Darlington (Jira)]

Stephen Darlington created IGNITE-13996:
---

 Summary: JMX beans in a format not understood by Prometheus
 Key: IGNITE-13996
 URL: https://issues.apache.org/jira/browse/IGNITE-13996
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.9.1, 2.8.1, 2.9
Reporter: Stephen Darlington


When I connect Ignite to Prometheus' JMX exporter, I get the following 
exception:

{{ }}
{{ Jan 13, 2021 5:31:49 PM 
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxCollector collect}}
{{ SEVERE: JMX scrape failed: java.lang.IllegalArgumentException: Not an 
Attribute: 
javax.management.openmbean.TabularDataSupport(tabularType=javax.management.openmbean.TabularType(name=org.apache.ignite.spi.systemview.view.ScanQueryView,rowType=javax.management.openmbean.CompositeType(name=org.apache.ignite.spi.systemview.view.ScanQueryView,items=((itemName=cacheGroupId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=cacheGroupName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=cacheId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=cacheName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=canceled,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=duration,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=filter,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=keepBinary,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=local,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=originNodeId,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=pageSize,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=partition,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=queryId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=startTime,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=subjectId,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=systemViewRowId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=taskName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=topology,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=transformer,itemType=javax.management.openmbean.SimpleType(name=java.lang.String,indexNames=(systemViewRowId)),contents={})}}
{{ at javax.management.AttributeList.adding(AttributeList.java:328)}}
{{ at javax.management.AttributeList.adding(AttributeList.java:335)}}
{{ at javax.management.AttributeList.asList(AttributeList.java:165)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.scrapeBean(JmxScraper.java:156)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.doScrape(JmxScraper.java:117)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.jmx.JmxCollector.collect(JmxCollector.java:460)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)}}
{{ at 
io.prometheus.jmx.shaded.io.prometheus.client.exporter.HTTPServer$HTTPMetricHandler.handle(HTTPServer.java:59)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)}}
{{ at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)}}
{{ at 
sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)}}
{{ at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)}}
{{ at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{ at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{ at java.lang.Thread.run(Thread.java:748)}}
{{  }}

And only the basic JVM metrics appear in Prometheus. A workaround would be to 
use OpenCensus, but a lot of people seem to prefer JMX.

It's not clear to me Ignite's JMX output is incorrect ([Prometheus' developers 
don't seem keen to resolve on their 
side|https://github.com/prometheus/jmx_exporter/issues/483]) but ideally, 
Ignite would work correctly with a common monitoring tool.



--
This message was sent by Atlassian Jira

[jira] [Created] (IGNITE-13994) Rebalance huge cache for in-memory cluster

[Ivan Daschinskiy (Jira)]

Ivan Daschinskiy created IGNITE-13994:
-

 Summary: Rebalance huge cache for in-memory cluster
 Key: IGNITE-13994
 URL: https://issues.apache.org/jira/browse/IGNITE-13994
 Project: Ignite
  Issue Type: Test
Reporter: Ivan Daschinskiy


There are some evidence, that rebalancing huge cache without rebalance 
throttling can cause OOM on supplier. We need to cover this scenario.

Scenario:
1. Start two nodes and 1 replicated cache with data region much more than heap.
2. Stop one of the node.
3. Load data to cache almost equal to size of data region.
4. Start node.

Goal is to run experiments with parameters
1. Heap size
2. Cache size
3. Rebalance batch size.
4. Rebalance throttle



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13993) Ignite-extensions: Change version of Ignite dependency to 2.11.0

[Mikhail Petrov (Jira)]

Mikhail Petrov created IGNITE-13993:
---

 Summary: Ignite-extensions: Change version of Ignite dependency to 
2.11.0
 Key: IGNITE-13993
 URL: https://issues.apache.org/jira/browse/IGNITE-13993
 Project: Ignite
  Issue Type: Bug
Reporter: Mikhail Petrov
Assignee: Mikhail Petrov


It's needed to change version of Ignite dependency to 2.11.0-SNAPSHOT. Now it 
causes TC ignite-extensions build failure.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13992) Migrate spring-transactions integration to ignite-extensions

[Mikhail Petrov (Jira)]

Mikhail Petrov created IGNITE-13992:
---

 Summary: Migrate spring-transactions integration to 
ignite-extensions
 Key: IGNITE-13992
 URL: https://issues.apache.org/jira/browse/IGNITE-13992
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov
Assignee: Mikhail Petrov


It's needed to migrate spring-transactions integration to ignite-extensions 
repository.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Migration of Spring Cache and Spring Transactions integrations to ignite-extensions.

[Mikhail Petrov]

Igniters,

I propose to migrate the Spring Transactions [1] and Spring Cache [2] 
integrations, that are currently stored in the ignite-spring module [3], 
to the ignite-extensions repository and split them as two separate 
modules - ignite-spring-transactions-ext and ignite-spring-cache-ext.



The motivation is the same as with migration of Spring Data integration 
[4] - to remove Spring integration releases cycle dependency on Ignite 
releases.
It can also help us reuse classes that are common across all Spring 
integrations.


WDYT? Any objections?

Regards,
Mikhail.

[1] - 
https://github.com/apache/ignite/tree/master/modules/spring/src/main/java/org/apache/ignite/transactions/spring
[2] - 
https://github.com/apache/ignite/tree/master/modules/spring/src/main/java/org/apache/ignite/cache/spring

[3] - https://github.com/apache/ignite/tree/master/modules/spring
[4] - 
http://apache-ignite-developers.2346864.n4.nabble.com/Migration-of-spring-data-modules-to-ignite-extensions-td49566.html

Re: SHA-512 for Maven deployment

[Andrey Mashenkov]

I've made "mvn clean install" with enabled "apache-release" profile and see
*.sha-512 checksum files in target directories.
So, upgrading to the latest apache parent looks sufficient.


On Thu, Jan 14, 2021 at 12:30 PM Petr Ivanov  wrote:

> Is seems that parent is already updated in
> https://issues.apache.org/jira/browse/IGNITE-13987 <
> https://issues.apache.org/jira/browse/IGNITE-13987>
>
>
>
> > On 14 Jan 2021, at 01:57, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> >
> > Andrey,
> >
> > This sounds even better. Can you create a ticket for this change?
> >
> > -Val
> >
> > On Wed, Jan 13, 2021 at 2:34 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> > wrote:
> >
> >> Val,
> >>
> >> I've just found Maven projects use SHA-512.
> >> I passed through commits and found they just switched to newer parent
> >> org.apache:apache pom.
> >> I've compared our current parent pom with the latest available one
> >> (org.apache:apache:16 vs org.apache:apache:23)
> >> and then found checksum-maven-plugin was added [1] somewhen in between.
> >>
> >> So, seems we have to switched to newer apache pom and maybe add
> >> checksum-maven-plugin
> >> to our main pom.
> >>
> >> [1]
> >>
> >>
> https://github.com/apache/maven-apache-parent/commit/a46aa52b4b56d9b7aa62e1b8cbea5ff0af434a
> >>
> >> On Wed, Jan 13, 2021 at 10:41 PM Valentin Kulichenko <
> >> valentin.kuliche...@gmail.com> wrote:
> >>
> >>> Hi Andrey,
> >>>
> >>> This indeed sounds like the cleanest way. I don't know how much effort
> >> that
> >>> would be though.
> >>>
> >>> -Val
> >>>
> >>> On Wed, Jan 13, 2021 at 11:01 AM Andrey Mashenkov <
> >>> andrey.mashen...@gmail.com> wrote:
> >>>
>  Maybe, we could donate to maven plugin possibility to switch to
> >> SHA-512.
>  Hopefully, a new plugin version will be released before we have any
> >>> release
>  candidate.
> 
>  Is it looks like a big deal?
> 
>  ср, 13 янв. 2021 г., 21:32 Valentin Kulichenko <
>  valentin.kuliche...@gmail.com>:
> 
> > Hi Ivan,
> >
> > No, I haven't found a way yet. SHA1 still works, but I believe we
> >>> should
> > consider using better options in future releases.
> >
> > Do you have any ideas on how to implement this?
> >
> > -Val
> >
> > On Wed, Jan 13, 2021 at 8:21 AM Ivan Pavlukhin 
> > wrote:
> >
> >> Folks,
> >>
> >> Were you able to resolve this?
> >>
> >> 2020-12-28 22:15 GMT+03:00, Valentin Kulichenko <
> >> valentin.kuliche...@gmail.com>:
> >>> Hi Ivan,
> >>>
> >>> Thanks for your response. I've looked into the PGP plugin, and
> >>> unfortunately it looks like it only can create signatures, but
> >> not
> >>> checksums.
> >>>
> >>> -Val
> >>>
> >>> On Sun, Dec 27, 2020 at 11:54 PM Ivan Bessonov <
>  bessonov...@gmail.com>
> >>> wrote:
> >>>
>  Hi,
> 
>  I've never done this before, but it seems like we need
> > maven-gpg-plugin
>  for
>  it [1].
> 
>  Algorithm configuration would look like this:
>  
> --digest-algo=SHA512
>  
> 
>  Maybe this will help.
> 
>  [1]
> 
> 
> >>
> >
> 
> >>>
> >>
> http://maven.apache.org/plugins-archives/maven-gpg-plugin-LATEST/sign-mojo.html
> 
>  пн, 28 дек. 2020 г. в 01:25, Valentin Kulichenko <
>  valentin.kuliche...@gmail.com>:
> 
> > Igniters,
> >
> > I've been preparing the 3.0.0-alpha1 release and got confused
>  about
> >> the
> > requirements for checksums in Maven deployments. The Apache
> >> instruction
>  [1]
> > states that MD5 is deprecated and SHA1 should be avoided in
> >>> favor
>  of
> > SHA-256 or SHA-512. However, it looks like we are still using
> >>> the
>  MD5/SHA1
> > combination (at least that's what the staging for 2.9.1 [2]
> > contains).
> >
> > On top of that, I can't find an easy way to switch to another
> > checksum
> > -
> > Maven deploy plugin [3] creates MD5 and SHA1 files
> >> automatically
>  and
> > doesn't seem to have any options to tweak this behavior.
> >
> > That said, I have two questions:
> >
> >   1. Are we required to use SHA512 or MD5/SHA1 is OK for now?
> >   2. Is there a painless way to include SHA512 in addition to
> > MD5/SHA1?
> >
> > Can anyone shed some light on this?
> >
> > [1] https://infra.apache.org/release-signing.html#basic-facts
> > [2]
> >
> >
> 
> >>
> >
> 
> >>>
> >>
> https://repository.apache.org/content/repositories/orgapacheignite-1490/org/apache/ignite/ignite-core/2.9.1/
> > [3]
> 
>  

[jira] [Created] (IGNITE-13991) Add checkstyle test suite for pom files on TC.

[Andrey Mashenkov (Jira)]

Andrey Mashenkov created IGNITE-13991:
-

 Summary: Add checkstyle test suite for pom files on TC.
 Key: IGNITE-13991
 URL: https://issues.apache.org/jira/browse/IGNITE-13991
 Project: Ignite
  Issue Type: Improvement
  Components: build
Reporter: Andrey Mashenkov
Assignee: Peter Ivanov


Create separate test suite for TC that should check that there are violations 
of dependencyManagement/pluginManagement approach.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: SHA-512 for Maven deployment

[Petr Ivanov]

Is seems that parent is already updated in 
https://issues.apache.org/jira/browse/IGNITE-13987 




> On 14 Jan 2021, at 01:57, Valentin Kulichenko  
> wrote:
> 
> Andrey,
> 
> This sounds even better. Can you create a ticket for this change?
> 
> -Val
> 
> On Wed, Jan 13, 2021 at 2:34 PM Andrey Mashenkov 
> wrote:
> 
>> Val,
>> 
>> I've just found Maven projects use SHA-512.
>> I passed through commits and found they just switched to newer parent
>> org.apache:apache pom.
>> I've compared our current parent pom with the latest available one
>> (org.apache:apache:16 vs org.apache:apache:23)
>> and then found checksum-maven-plugin was added [1] somewhen in between.
>> 
>> So, seems we have to switched to newer apache pom and maybe add
>> checksum-maven-plugin
>> to our main pom.
>> 
>> [1]
>> 
>> https://github.com/apache/maven-apache-parent/commit/a46aa52b4b56d9b7aa62e1b8cbea5ff0af434a
>> 
>> On Wed, Jan 13, 2021 at 10:41 PM Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>> 
>>> Hi Andrey,
>>> 
>>> This indeed sounds like the cleanest way. I don't know how much effort
>> that
>>> would be though.
>>> 
>>> -Val
>>> 
>>> On Wed, Jan 13, 2021 at 11:01 AM Andrey Mashenkov <
>>> andrey.mashen...@gmail.com> wrote:
>>> 
 Maybe, we could donate to maven plugin possibility to switch to
>> SHA-512.
 Hopefully, a new plugin version will be released before we have any
>>> release
 candidate.
 
 Is it looks like a big deal?
 
 ср, 13 янв. 2021 г., 21:32 Valentin Kulichenko <
 valentin.kuliche...@gmail.com>:
 
> Hi Ivan,
> 
> No, I haven't found a way yet. SHA1 still works, but I believe we
>>> should
> consider using better options in future releases.
> 
> Do you have any ideas on how to implement this?
> 
> -Val
> 
> On Wed, Jan 13, 2021 at 8:21 AM Ivan Pavlukhin 
> wrote:
> 
>> Folks,
>> 
>> Were you able to resolve this?
>> 
>> 2020-12-28 22:15 GMT+03:00, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com>:
>>> Hi Ivan,
>>> 
>>> Thanks for your response. I've looked into the PGP plugin, and
>>> unfortunately it looks like it only can create signatures, but
>> not
>>> checksums.
>>> 
>>> -Val
>>> 
>>> On Sun, Dec 27, 2020 at 11:54 PM Ivan Bessonov <
 bessonov...@gmail.com>
>>> wrote:
>>> 
 Hi,
 
 I've never done this before, but it seems like we need
> maven-gpg-plugin
 for
 it [1].
 
 Algorithm configuration would look like this:
 
--digest-algo=SHA512
 
 
 Maybe this will help.
 
 [1]
 
 
>> 
> 
 
>>> 
>> http://maven.apache.org/plugins-archives/maven-gpg-plugin-LATEST/sign-mojo.html
 
 пн, 28 дек. 2020 г. в 01:25, Valentin Kulichenko <
 valentin.kuliche...@gmail.com>:
 
> Igniters,
> 
> I've been preparing the 3.0.0-alpha1 release and got confused
 about
>> the
> requirements for checksums in Maven deployments. The Apache
>> instruction
 [1]
> states that MD5 is deprecated and SHA1 should be avoided in
>>> favor
 of
> SHA-256 or SHA-512. However, it looks like we are still using
>>> the
 MD5/SHA1
> combination (at least that's what the staging for 2.9.1 [2]
> contains).
> 
> On top of that, I can't find an easy way to switch to another
> checksum
> -
> Maven deploy plugin [3] creates MD5 and SHA1 files
>> automatically
 and
> doesn't seem to have any options to tweak this behavior.
> 
> That said, I have two questions:
> 
>   1. Are we required to use SHA512 or MD5/SHA1 is OK for now?
>   2. Is there a painless way to include SHA512 in addition to
> MD5/SHA1?
> 
> Can anyone shed some light on this?
> 
> [1] https://infra.apache.org/release-signing.html#basic-facts
> [2]
> 
> 
 
>> 
> 
 
>>> 
>> https://repository.apache.org/content/repositories/orgapacheignite-1490/org/apache/ignite/ignite-core/2.9.1/
> [3]
 
 https://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html
> 
> -Val
> 
 
 
 --
 Sincerely yours,
 Ivan Bessonov
 
>>> 
>> 
>> 
>> --
>> 
>> Best regards,
>> Ivan Pavlukhin
>> 
> 
 
>>> 
>> 
>> 
>> --
>> Best regards,
>> Andrey V. Mashenkov
>> 

[jira] [Created] (IGNITE-13990) Improve maven plugin management.

[Andrey Mashenkov (Jira)]

Andrey Mashenkov created IGNITE-13990:
-

 Summary: Improve maven plugin management.
 Key: IGNITE-13990
 URL: https://issues.apache.org/jira/browse/IGNITE-13990
 Project: Ignite
  Issue Type: Improvement
  Components: build
Reporter: Andrey Mashenkov


Plugins like RAT should not be configured in parent at all, because it will be 
used only in root module, while current configuration will run it for every 
module. Thus, configuration should be moved to root module.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] .Net BinaryTypes transparency

[Nikolay Izhikov]

Hello, Pavel.

Agree. Will implement it. 

> 13 янв. 2021 г., в 20:28, Pavel Tupitsyn  написал(а):
> 
>> I propose to implicitly register classes only in the case they are sent
> to the Java side.
> 
> Ok, I guess we can do this when all of the following is true
> - We are inside ExecuteJavaTask or Java service call
> - FQN mapper is used (default or explicit)
> - There is no BinaryIdMapper on .NET or Java side
> 
> 
> On Wed, Jan 13, 2021 at 6:30 PM Nikolay Izhikov  wrote:
> 
>> Pavel.
>> 
>>> Yes, but they are registered separately for .NET and Java, please see
>> how MarshallerContextImpl stores typeId->typeName mappings.
>> 
>> This will not change. We still will have separate type registration.
>> 
>>> With the proposal we put all class names in the same pile.
>> 
>> Implementation of type registration will not be changed.
>> What I propose is to simplify user life in the exact places we can do it
>> for free.
>> 
>>> Now imagine that the user has a cluster with C# and Java clients.
>>> C# client uses class "foo.bar.Aa" for Compute,
>>> Java client uses class "foo.bar.BB" for Services - independent use
>> cases,
>>> all is well.
>> 
>> I propose to implicitly register classes only in the case they are sent to
>> the Java side.
>> So, for all classes that is used only in .Net nothing will change.
>> 
>> 
>>> 13 янв. 2021 г., в 15:14, Pavel Tupitsyn 
>> написал(а):
>>> 
 Classes MUST be registered to be used in compute or cache.
>>> 
>>> Yes, but they are registered separately for .NET and Java,
>>> please see how MarshallerContextImpl stores typeId->typeName mappings.
>>> 
>>> With the proposal we put all class names in the same pile.
>>> 
>>> Now imagine that the user has a cluster with C# and Java clients.
>>> C# client uses class "foo.bar.Aa" for Compute,
>>> Java client uses class "foo.bar.BB" for Services - independent use
>> cases,
>>> all is well.
>>> 
>>> As soon as we put all class names in the same map, the app breaks,
>>> because "foo.bar.Aa" and "foo.bar.BB" have the same hash code,
>>> and DuplicateTypeIdException is thrown by Ignite.
>>> 
>>> On Wed, Jan 13, 2021 at 1:54 PM Nikolay Izhikov 
>> wrote:
>>> 
 Hello, Pavel.
 
> Imagine that some Ignite user has lots of .NET and Java classes being
 stored in cache, used in compute,
 
 Classes MUST be registered to be used in compute or cache.
 So, in your example, user registered classes by hand, already.
 
> Let's add a flag like BinaryConfiguration.AssumeSameJavaTypeNames,
 
 We already have this «flag».
 It a default INameMapper implementation that assumes we use the same
>> class
 names both in Java and .Net.
 
> This is not up for discussion, we do not break compatibility like this.
 
 Sorry, I don’t see compatibility issues in the proposal.
 Can you, please, provide an example?
 
> 13 янв. 2021 г., в 13:46, Pavel Tupitsyn 
 написал(а):
> 
> Nikolay,
> 
> I agree with your proposal, with one addition:
> this behavior must be disabled by default for compatibility reasons.
> 
> Imagine that some Ignite user has lots of .NET and Java classes being
> stored in cache,
> used in compute, etc. Now with the proposal implemented, all .NET
>> classes
> are registered
> as Java classes too, which has a potential for hash code collisions,
> resulting in DuplicateTypeIdException.
> 
> So the user can't upgrade to 2.11, their app does not work anymore,
>> which
> is not acceptable.
> This is not up for discussion, we do not break compatibility like this.
> 
> Let's add a flag like BinaryConfiguration.AssumeSameJavaTypeNames,
> which enables the behavior globally and both ways:
> all type names become shared across all platforms in
 MarshallerContextImpl.
> 
> 
> On Wed, Jan 13, 2021 at 11:21 AM Nikolay Izhikov 
> wrote:
> 
>> Hello, Pavel
>> 
>> My proposal is the following:
>> 
>> *When Ignite configured with FQN NameMapper.*
>> And  user invokes
>> 
>> 1. Service method from .Net to Java.
>> 2. Compute methods `ExecuteJavaTask`, `ExecuteJavaTaskAsync`
>> 3. Cache operations.
>> 
>> Ignite should implicitly register types both for .Net and Java
>> platform.
>> 
>> =
>> My intentions is to simplify usage of the Ignite .Net platform by
>> eliminating necessity of BinaryConfiguration in .Net.
>> 
>> Approach when user should modify Ignite configuration on both sides
>> Java
>> and .Net every time he use new class as a service parameter looks
>> ugly,
 for
>> me.
>> 
>> You can see an example of my idea in services test [1]
>> In current release, user forced to enlist each type in configuration.
>> Now, you just deploy Java service and use it.
>> 
>> Please, Note:
>> 
>> 1. FQN NameMapper is a default value.
>> 2. FQN NameMapper requires from