[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323946#comment-17323946 ] Ilya Kasnacheev commented on IGNITE-14346: -- Thank you for this module [~atri] It have been merged to master. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement > Components: integrations >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Fix For: 2.11 > > Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot > 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png, > Screenshot 2021-03-23 at 8.45.05 PM.png, azure-container.png > > Time Spent: 3h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323929#comment-17323929 ] Ilya Kasnacheev commented on IGNITE-14346: -- LGTM. I will re-run basic tests and then hope to merge. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Labels: ReviewNeeded > Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot > 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png, > Screenshot 2021-03-23 at 8.45.05 PM.png, azure-container.png > > Time Spent: 3h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[
https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320434#comment-17320434
]
Ignite TC Bot commented on IGNITE-14346:
{panel:title=Branch: [pull/8897/head] Base: [master] : Possible Blockers
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Core Linux){color} [[tests 1 TC_SERVICE_MESSAGE
|https://ci.ignite.apache.org/viewLog.html?buildId=5965398]]
* dll: CacheQueriesTest.TestScanQueryPartitions(True) - Test has low fail rate
in base branch 0,0% and is not flaky
{color:#d04437}[Missing Tests]{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=5965422]]
{panel}
{panel:title=Branch: [pull/8897/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5965424buildTypeId=IgniteTests24Java8_RunAll]
> Implement Azure Blob Storage Based IP Finder
>
>
> Key: IGNITE-14346
> URL: https://issues.apache.org/jira/browse/IGNITE-14346
> Project: Ignite
> Issue Type: Improvement
>Reporter: Atri Sharma
>Assignee: Atri Sharma
>Priority: Major
> Labels: ReviewNeeded
> Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot
> 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png,
> Screenshot 2021-03-23 at 8.45.05 PM.png, azure-container.png
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[
https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317910#comment-17317910
]
Ilya Kasnacheev commented on IGNITE-14346:
--
I have tried the fresh build of the PR, now the dependencies issue seems
resolved, but there are at least two new issues:
If IPv6 is active, I get the following error:
{code}
class org.apache.ignite.IgniteException: URLDecoder: Illegal hex characters in
escape (%) pattern - Error at index 0 in: "lo"
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1098)
at org.apache.ignite.Ignition.start(Ignition.java:356)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
Caused by: class org.apache.ignite.IgniteCheckedException: URLDecoder: Illegal
hex characters in escape (%) pattern - Error at index 0 in: "lo"
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1454)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2112)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1758)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1061)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:947)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:846)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:685)
at org.apache.ignite.Ignition.start(Ignition.java:353)
... 1 more
Caused by: java.lang.IllegalArgumentException: URLDecoder: Illegal hex
characters in escape (%) pattern - Error at index 0 in: "lo"
at java.base/java.net.URLDecoder.decode(URLDecoder.java:232)
at java.base/java.net.URLDecoder.decode(URLDecoder.java:142)
at com.azure.storage.common.Utility.decode(Utility.java:91)
at com.azure.storage.common.Utility.urlDecode(Utility.java:82)
at
com.azure.storage.blob.BlobContainerAsyncClient.getBlobAsyncClient(BlobContainerAsyncClient.java:144)
at
com.azure.storage.blob.BlobContainerAsyncClient.getBlobAsyncClient(BlobContainerAsyncClient.java:125)
at
com.azure.storage.blob.BlobContainerClient.getBlobClient(BlobContainerClient.java:76)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.azure.TcpDiscoveryAzureBlobStoreIpFinder.registerAddresses(TcpDiscoveryAzureBlobStoreIpFinder.java:140)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:66)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:401)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:457)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2166)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:278)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:981)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1965)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1323)
... 10 more
Failed to start grid: URLDecoder: Illegal hex characters in escape (%) pattern
- Error at index 0 in: "lo"
{code}
I believe this is because IPv6 binding addresses have %interface prepended to
them (::1%localhost, ::1%eth0), it looks like they need some extra escaping in
your use case.
When IPv6 is on, azure discovery will start all right but cluster will not
assemble - neither of my two nodes will discover itself and start. Maybe I'm
doing something wrong? My Azure container has some entries (see attach) but
cluster does not come online in a few minutes. When using static discovery, the
cluster is up and running instantly.
Please advise.
> Implement Azure Blob Storage Based IP Finder
>
>
> Key: IGNITE-14346
> URL: https://issues.apache.org/jira/browse/IGNITE-14346
> Project: Ignite
> Issue Type: Improvement
>Reporter: Atri Sharma
>Assignee: Atri Sharma
>Priority: Major
> Labels: ReviewNeeded
> Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot
> 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png,
> Screenshot 2021-03-23 at 8.45.05 PM.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312071#comment-17312071 ] Atri Sharma commented on IGNITE-14346: -- [~ilyak] Tried with a build after including the Azure directory and the Ignite node started up fine (after the latest iteration). Please see and let me know. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Labels: ReviewNeeded > Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot > 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png, > Screenshot 2021-03-23 at 8.45.05 PM.png > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[
https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312070#comment-17312070
]
Atri Sharma commented on IGNITE-14346:
--
[~ilyak] Updated. I also started an Ignite node after building a release with
the Azure module included.
I did {{mvn clean install -DskipTests}} and then {{mvn initialize -Prelease and
then started the node.}}
> Implement Azure Blob Storage Based IP Finder
>
>
> Key: IGNITE-14346
> URL: https://issues.apache.org/jira/browse/IGNITE-14346
> Project: Ignite
> Issue Type: Improvement
>Reporter: Atri Sharma
>Assignee: Atri Sharma
>Priority: Major
> Labels: ReviewNeeded
> Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot
> 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png,
> Screenshot 2021-03-23 at 8.45.05 PM.png
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310694#comment-17310694 ] Ilya Kasnacheev commented on IGNITE-14346: -- It is also possible that IP finder is waiting silently after this happens. Either way, I don't see how to go forward from here. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Labels: ReviewNeeded > Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot > 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png, > Screenshot 2021-03-23 at 8.45.05 PM.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[
https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310691#comment-17310691
]
Ilya Kasnacheev commented on IGNITE-14346:
--
Azure also seems to interfere with slf4j settings in some way that I can no
longer see anything printed to logs after azure discovery starts
The last that I see is
{code}
[14:19:06,989][INFO][main][GridTcpRestProtocol] Command protocol successfully
started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
[14:19:07,250][INFO][main][GridJettyRestProtocol] Command protocol successfully
started [name=Jetty REST, host=/0.0.0.0, port=8080]
[14:19:07,314][INFO][main][IgniteKernal] Non-loopback local IPs: 10.1.1.4,
fe80:0:0:0:222:48ff:fe1f:b80f%eth0
[14:19:07,314][INFO][main][IgniteKernal] Enabled local MACs: 0022481FB80F
[14:19:07,322][INFO][main][ClusterProcessor] Cluster ID and tag has been read
from metastorage: null
[14:19:07,327][INFO][main][IgniteClusterImpl] Shutdown policy was updated
[oldVal=null, newVal=null]
log4j:WARN No appenders could be found for logger
(io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
{code}
> Implement Azure Blob Storage Based IP Finder
>
>
> Key: IGNITE-14346
> URL: https://issues.apache.org/jira/browse/IGNITE-14346
> Project: Ignite
> Issue Type: Improvement
>Reporter: Atri Sharma
>Assignee: Atri Sharma
>Priority: Major
> Labels: ReviewNeeded
> Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot
> 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png,
> Screenshot 2021-03-23 at 8.45.05 PM.png
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[
https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310678#comment-17310678
]
Ilya Kasnacheev commented on IGNITE-14346:
--
[~atri] I have added some more comments.
I can also see this problem:
{code}
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name
'org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryAzureBlobStoreIpFinder#70be0a2b'
defined in URL
[file:/home/gg_rw_user/apache-ignite-2.11.0-SNAPSHOT-bin/config/default-config.xml]:
Instantiation of bean failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryAzureBlobStoreIpFinder]:
No default constructor found; nested exception is
java.lang.NoClassDefFoundError: com/azure/core/exception/HttpResponseException
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1158)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1102)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at
org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
... 28 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to
instantiate
[org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryAzureBlobStoreIpFinder]:
No default constructor found; nested exception is
java.lang.NoClassDefFoundError: com/azure/core/exception/HttpResponseException
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:85)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1150)
... 32 more
Caused by: java.lang.NoClassDefFoundError:
com/azure/core/exception/HttpResponseException
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at
java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at
java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at
java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
at java.base/java.lang.Class.getConstructor0(Class.java:3342)
at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2553)
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:80)
... 33 more
Caused by: java.lang.ClassNotFoundException:
com.azure.core.exception.HttpResponseException
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 47 more
{code}
When Apache Ignite releases are built, dependencies are not resolved in
transient manner. All dependencies need to be listed explicitly in module's
pom.xml. Please collect and add them.
> Implement Azure Blob Storage Based IP Finder
>
>
> Key: IGNITE-14346
> URL: https://issues.apache.org/jira/browse/IGNITE-14346
> Project: Ignite
> Issue Type: Improvement
>Reporter: Atri Sharma
>Assignee: Atri Sharma
>Priority: Major
> Labels: ReviewNeeded
> Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot
> 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png,
> Screenshot 2021-03-23 at 8.45.05 PM.png
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
--
This message was sent by
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307164#comment-17307164 ] Atri Sharma commented on IGNITE-14346: -- [~ilyak] I have uploaded screenshots of test suite with Azure credentials and actual storage. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Attachments: Screenshot 2021-03-23 at 8.42.31 PM.png, Screenshot > 2021-03-23 at 8.43.31 PM.png, Screenshot 2021-03-23 at 8.44.11 PM.png, > Screenshot 2021-03-23 at 8.45.05 PM.png > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14346) Implement Azure Blob Storage Based IP Finder
[ https://issues.apache.org/jira/browse/IGNITE-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307094#comment-17307094 ] Ilya Kasnacheev commented on IGNITE-14346: -- I did not have chance to try it on Azure, but I have left some comments regarding CC. > Implement Azure Blob Storage Based IP Finder > > > Key: IGNITE-14346 > URL: https://issues.apache.org/jira/browse/IGNITE-14346 > Project: Ignite > Issue Type: Improvement >Reporter: Atri Sharma >Assignee: Atri Sharma >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
