[
https://issues.apache.org/jira/browse/IGNITE-13520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-13520:
--------------------------------------
Description:
Currently, when a client node joins a cluster with a static encrypted cache
configuration, it generates an encryption key for that cache and sends it to
the cluster (just like the server node does).
This is unexpected behavior, it happens due to IGNITE-13567 (see
GridEncryptionManager#collectJoiningNodeData). The client node should not
generate encryption keys and should be able to start without configuring
EncryptionSPI.
After doing some research on possible solutions, we decided to reject node
joining in such a situation, because there is no clean and simple way to
distribute the same encryption key between server nodes that are already in the
cluster (we have to either add discovery overhead, block the exchange, or add
an additional exchange to be able to distribute keys between server nodes that
are already in the cluster).
was:
Configuration: 1 server node, 1 client node, 1 statically configured cache
group with enabled encryption
Expected: client node can join the cluster without specifying a custom
EncrptionSPI implementation.
Actual: client node cannot join the cluster due to the following exception:
{noformat}
class org.apache.ignite.IgniteCheckedException: Failed to start manager:
GridManagerAdapter [enabled=true,
name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager] at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1938)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1289)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2096)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1748)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:641)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1229)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1150)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startClientGrid(GridAbstractTest.java:1088)
at
org.apache.ignite.internal.encryption.EncryptedCacheNodeJoinTest.testClientNodeJoinWithPreconfiguredCache(EncryptedCacheNodeJoinTest.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2373)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI:
TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000,
marsh=JdkMarshaller
[clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@48073af2],
reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5,
forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null,
skipAddrsRandomization=false]
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:281)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:974)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1933)
... 19 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: You have to
configure custom EncryptionSpi implementation.
at
org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi.create(NoopEncryptionSpi.java:45)
at
org.apache.ignite.internal.managers.encryption.GridEncryptionManager.newEncryptionKeys(GridEncryptionManager.java:894)
at
org.apache.ignite.internal.managers.encryption.GridEncryptionManager.collectJoiningNodeData(GridEncryptionManager.java:442)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.collect(GridDiscoveryManager.java:892)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.collectExchangeData(TcpDiscoverySpi.java:2089)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl.sendJoinRequest(ClientImpl.java:767)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl.joinTopology(ClientImpl.java:629)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl.access$1000(ClientImpl.java:150)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.tryJoin(ClientImpl.java:2108)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1751)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.spi.discovery.tcp.ClientImpl$1.body(ClientImpl.java:317)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
{noformat}
*Update*:
After investigating, I found that there are 2 problems here.
The first problematic case is when we start all nodes with the same statically
configured cache.
In this case, we just shouldn't try to generate keys on the client node.
The second problem is when we try to join a client node with a new statically
configured cache (which is not present on the server nodes).
This case is more difficult because in the usual case, for a new cache, the
key is generated on the node where the cache was defined
(collectJoiningNodeData), and the other nodes add this key to themselves
(onJoiningNodeDataReceived), the joining node saves the key when it receives a
response from the coordinator (onGridDataReceived), thus the generated key is
the same on all nodes.
We can't generate encryption key on client node because it should be able to
start without custom _EncryptionSPI_ implementation.
If we try to generate a key, for example, on the coordinator, then we will not
be able to transfer it to the server nodes that are already in the cluster.
Also, we cannot transfer the key (for the new cache) from the coordinator to
the server nodes using regular partition map exchange, because when the client
node joins, the server nodes don't receive the full message.
*Solution*: prevent key generation on client nodes and prevent cache start with
missed encryption key.
In this case client node successfully joins the cluster, the server nodes
ignore new cache (with missed encryption key).
On exchange init (after joining), *the client node will automatically start
this cache dynamically* (see
_GridDhtPartitionsExchangeFuture.ensureClientCachesStarted,_ implemented in
IGNITE-5789).
> Client node with a static encrypted cache configuration cannot join a cluster
> without EncryptionSPI configured.
> ---------------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-13520
> URL: https://issues.apache.org/jira/browse/IGNITE-13520
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.9
> Reporter: Pavel Pereslegin
> Assignee: Pavel Pereslegin
> Priority: Major
> Labels: encryption
> Fix For: 2.10
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently, when a client node joins a cluster with a static encrypted cache
> configuration, it generates an encryption key for that cache and sends it to
> the cluster (just like the server node does).
> This is unexpected behavior, it happens due to IGNITE-13567 (see
> GridEncryptionManager#collectJoiningNodeData). The client node should not
> generate encryption keys and should be able to start without configuring
> EncryptionSPI.
> After doing some research on possible solutions, we decided to reject node
> joining in such a situation, because there is no clean and simple way to
> distribute the same encryption key between server nodes that are already in
> the cluster (we have to either add discovery overhead, block the exchange, or
> add an additional exchange to be able to distribute keys between server nodes
> that are already in the cluster).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
