[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859813#comment-15859813
]
Taras Ledkov edited comment on IGNITE-2356 at 2/10/17 9:11 AM:
---
Fixed. Ignite nodes management is changed.
I've added the method {{T2 getOrStart(IgniteConfiguration
cfg)}} at the IgnitionEx to know ignite instance started by call this method or
an existing instance is returned.
[Tests|http://195.239.208.174/project.html?projectId=IgniteTests_IgniteTests=pull%2F1251%2Fhead]
was (Author: tledkov-gridgain):
Fixed. Ignite nodes management is changed.
I've added the method {{T2 getOrStart(IgniteConfiguration
cfg)}} to the IgnitionEx to know ignite instance started or an existing
instance is returned.
[Tests|http://195.239.208.174/project.html?projectId=IgniteTests_IgniteTests=pull%2F1251%2Fhead]
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842510#comment-15842510
]
Taras Ledkov edited comment on IGNITE-2356 at 1/27/17 1:02 PM:
---
1. We have to use remote node to check both {{InProc, Client}} modes for
{{HadoopIgfsWrapper}}. e.g. {{InProc}} mode is default and client node is
started never if the ignite node available at the same process.
2. Sure. The concurrency with {{synchronized}} become simpler then with
lock-free (*Fixed*).
[Tests|http://195.239.208.174/project.html?projectId=IgniteTests_IgniteTests=pull%2F1251%2Fhead]
are re-run
was (Author: tledkov-gridgain):
1. We have to use remote node to check both {{InProc, Client}} modes for
{{HadoopIgfsWrapper}}. e.g. {{InProc}} mode is default and client node is
started never if the ignite node available at the same process.
2. Sure. The concurrency with {{synchronized}} become simpler then with
lock-free (*Fixed*).
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 2.0
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761193#comment-15761193
]
Taras Ledkov edited comment on IGNITE-2356 at 12/19/16 1:44 PM:
*1, 2* - done;
*3* - Should we support the case when delegate shared ignite instance between
pure in-proc connection and Ignite client connection? Looks like it doesn't
make sense because the case is happen when the grid name of {{endpoint}} equals
to client name. It is a very strange configuration. Correct?
Also, should we support creation of several clients with different
configuration?
was (Author: tledkov-gridgain):
*1, 2* - done;
*3* - Should we support the case when delegate shared ignite instance between
pure in-proc connection and Ignite client connection? Looks like it doesn't
make sense because the case is happen when the grid name of {{endpoint}} equals
to client name. It is a very strange configuration. Correct?
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 2.0
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761193#comment-15761193
]
Taras Ledkov edited comment on IGNITE-2356 at 12/19/16 1:42 PM:
*1, 2* - done;
*3* - Should we support the case when delegate shared ignite instance between
pure in-proc connection and Ignite client connection? Looks like it doesn't
make sense because the case is happen when the grid name of {{endpoint}} equals
to client name. It is a very strange configuration. Correct?
was (Author: tledkov-gridgain):
*1, 2* - done;
3 - Should we support the case when delegate shared ignite instance between
pure in-proc connection and Ignite client connection? Looks like it doesn't
make sense because the case is happen when the grid name of {{endpoint}} equals
to client name. It is a very strange configuration. Correct?
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 2.0
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588898#comment-15588898
]
Taras Ledkov edited comment on IGNITE-2356 at 10/20/16 3:13 PM:
The tests that check file's owner on Ignite-client based mode are failed due to
current user isn't passed to client tasks.
Tests:
{code}
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner
{code}
was (Author: tledkov-gridgain):
The tests that check file's owner on Ignite-client based mode are failed due to
IGNITE-3667.
Tests:
{code}
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner
{code}
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591619#comment-15591619
]
Taras Ledkov edited comment on IGNITE-2356 at 10/20/16 11:59 AM:
-
No, the tests {{IgniteHadoopFileSystemClientBased*}} extends
{{IgniteHadoopFileSystemAbstractSelfTest}} that doesn't use
{{LocalSecondaryFileSystem}}.
I don't see any reasons to use {{LocalSecondaryFileSystem}}. These tests check
different types of connection of an IGFS client.
was (Author: tledkov-gridgain):
No, the tests IgniteHadoopFileSystemClientBased* extends
IgniteHadoopFileSystemAbstractSelfTest that doesn't use
LocalSecondaryFileSystem.
I don't see any reasons to use LocalSecondaryFileSystem. These tests check
different types of connection of an IGFS client.
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586008#comment-15586008
]
Taras Ledkov edited comment on IGNITE-2356 at 10/19/16 2:49 PM:
Looks like the tests {{IgniteHadoopFileSystemShmemEmbedded*}} are not useful
because when the flag {{skipEmbed}} is set to {{true}} these tests are
equivalent to the {{IgniteHadoopFileSystemLoopbackEmbedded*}}. It happens
because in-process Ignite is always available for the tests. So, when it is not
skipped this type of delegate is used with the highest priority.
I'll add 4 tests for Ignite client -based IGFS delegate (for primary, proxy &
both dual modes).
There is a problem with client reconnection to the topology after all servers
are down. Root cause is the inconsistent configuration of the client node &
server nodes.
was (Author: tledkov-gridgain):
Looks like the tests {{IgniteHadoopFileSystemShmemEmbedded*}} are not useful
because when the flag {{skipEmbed}} is set to {{true}} these tests are
equivalent to the {{IgniteHadoopFileSystemLoopbackEmbedded*}}. It happens
because in-process Ignite is available always for the tests. So, when it is not
skipped this type of delegate is used with the highest priority.
I'll add 4 tests for Ignite client -based IGFS delegate (for primary, proxy &
both dual modes).
There is a problem with client reconnection to the topology after all servers
are down. Root cause is the inconsistent configuration of the client node &
server nodes.
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[
https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588898#comment-15588898
]
Taras Ledkov edited comment on IGNITE-2356 at 10/19/16 2:44 PM:
The tests that check file's owner on Ignite-client based mode are failed due to
IGNITE-3667.
Tests:
{code}
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner
{code}
was (Author: tledkov-gridgain):
The tests that check file's owner on Ignite-client based mode are failed due to
IGNITE-3667.
Tests:
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs
IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner
> IGFS client should be able to failover in case of server crash.
> ---
>
> Key: IGNITE-2356
> URL: https://issues.apache.org/jira/browse/IGNITE-2356
> Project: Ignite
> Issue Type: Sub-task
> Components: IGFS
>Affects Versions: ignite-1.4
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either
> TCP or shmem.
> Only single endpoint can be specified. As such, should the server went down,
> IgntieHadoopFileSystem (either new or existing) is no longer operational.
> We need to let user specify several endpoints and failover/balance between
> them.
> Look at Hadoop HA first to get an ideas on how to configure multiple
> addresses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[ https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588898#comment-15588898 ] Taras Ledkov edited comment on IGNITE-2356 at 10/19/16 2:19 PM: The tests that check file's owner on Ignite-client based mode are failed due to IGNITE-3667. Tests: IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner was (Author: tledkov-gridgain): The tests that check file owner on Ignite-client based mode are failed due to IGNITE-3667. Tests: IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner > IGFS client should be able to failover in case of server crash. > --- > > Key: IGNITE-2356 > URL: https://issues.apache.org/jira/browse/IGNITE-2356 > Project: Ignite > Issue Type: Sub-task > Components: IGFS >Affects Versions: ignite-1.4 >Reporter: Vladimir Ozerov >Assignee: Taras Ledkov >Priority: Critical > Fix For: 1.8 > > > IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either > TCP or shmem. > Only single endpoint can be specified. As such, should the server went down, > IgntieHadoopFileSystem (either new or existing) is no longer operational. > We need to let user specify several endpoints and failover/balance between > them. > Look at Hadoop HA first to get an ideas on how to configure multiple > addresses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[ https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588898#comment-15588898 ] Taras Ledkov edited comment on IGNITE-2356 at 10/19/16 2:19 PM: The tests that check file owner on Ignite-client based mode are failed due to IGNITE-3667. Tests: IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualAsyncSelfTest.testSetOwner IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testOpen IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testMkdirs IgniteHadoopFileSystemClientBasedDualSyncSelfTest.testSetOwner was (Author: tledkov-gridgain): The tests that check file owner on Ignite-client based mode are failed due to IGNITE-3667. > IGFS client should be able to failover in case of server crash. > --- > > Key: IGNITE-2356 > URL: https://issues.apache.org/jira/browse/IGNITE-2356 > Project: Ignite > Issue Type: Sub-task > Components: IGFS >Affects Versions: ignite-1.4 >Reporter: Vladimir Ozerov >Assignee: Taras Ledkov >Priority: Critical > Fix For: 1.8 > > > IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either > TCP or shmem. > Only single endpoint can be specified. As such, should the server went down, > IgntieHadoopFileSystem (either new or existing) is no longer operational. > We need to let user specify several endpoints and failover/balance between > them. > Look at Hadoop HA first to get an ideas on how to configure multiple > addresses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-2356) IGFS client should be able to failover in case of server crash.
[ https://issues.apache.org/jira/browse/IGNITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568838#comment-15568838 ] Taras Ledkov edited comment on IGNITE-2356 at 10/12/16 2:15 PM: Fixed. [Tests|http://195.239.208.174/viewQueued.html?itemId=342134=queuedBuildOverviewTab] was (Author: tledkov-gridgain): Fixed > IGFS client should be able to failover in case of server crash. > --- > > Key: IGNITE-2356 > URL: https://issues.apache.org/jira/browse/IGNITE-2356 > Project: Ignite > Issue Type: Sub-task > Components: IGFS >Affects Versions: ignite-1.4 >Reporter: Vladimir Ozerov >Assignee: Taras Ledkov >Priority: Critical > Fix For: 2.0 > > > IGFS client (IgniteHadoopFileSystem) communicates IGFS over endpoint - either > TCP or shmem. > Only single endpoint can be specified. As such, should the server went down, > IgntieHadoopFileSystem (either new or existing) is no longer operational. > We need to let user specify several endpoints and failover/balance between > them. > Look at Hadoop HA first to get an ideas on how to configure multiple > addresses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
