[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Hadoop Flags: Reviewed
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
>Assignee: Virajith Jalaparti
> Fix For: 3.1.0
>
> Attachments: HDFS-12093-HDFS-9806.001.patch,
> HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch,
> HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Status: Open (was: Patch Available)
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch,
> HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Attachment: HDFS-12093-HDFS-9806.002.patch
Posting the patch rebased on latest version of feature branch.
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch,
> HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Status: Patch Available (was: Open)
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch,
> HDFS-12093-HDFS-9806.002.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Status: Patch Available (was: Open)
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Attachment: HDFS-12093-HDFS-9806.001.patch
Attaching a patch which instantiates {{remoteFS}} in {{ProvidedVolumeImpl}},
and passes it to {{ProvidedReplica}} when they are created.
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
> Attachments: HDFS-12093-HDFS-9806.001.patch
>
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12093) [READ] Share remoteFS between ProvidedReplica instances.
[
https://issues.apache.org/jira/browse/HDFS-12093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Virajith Jalaparti updated HDFS-12093:
--
Description:
When a Datanode comes online using Provided storage, it fills the
{{ReplicaMap}} with the known replicas. With Provided Storage, this includes
{{ProvidedReplica}} instances. Each of these objects, in their constructor,
will construct an FileSystem using the Service Provider. This can result in
contacting the remote file system and checking that the credentials are correct
and that the data is there. For large systems this is a prohibitively expensive
operation to perform per replica.
Instead, the {{ProvidedVolumeImpl}} should own the reference to the
{{remoteFS}} and should share it with the {{ProvidedReplica}} objects on their
creation.
was:
Then a Datanode comes online using Provided storage, it fills the
{{ReplicaMap}} with the known replicas. With Provided Storage, this includes
{{ProvidedReplica}} instances. Each of these objects, in their constructor,
will construct an FileSystem using the Service Provider. This can result in
contacting the remote file system and checking that the credentials are correct
and that the data is there. For large systems this is a prohibitively expensive
operation to perform per replica.
Instead, the {{ProvidedVolumeImpl}} should own the reference to the
{{remoteFS}} and should share it with the {{ProvidedReplica}} objects on their
creation.
> [READ] Share remoteFS between ProvidedReplica instances.
>
>
> Key: HDFS-12093
> URL: https://issues.apache.org/jira/browse/HDFS-12093
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Ewan Higgs
>
> When a Datanode comes online using Provided storage, it fills the
> {{ReplicaMap}} with the known replicas. With Provided Storage, this includes
> {{ProvidedReplica}} instances. Each of these objects, in their constructor,
> will construct an FileSystem using the Service Provider. This can result in
> contacting the remote file system and checking that the credentials are
> correct and that the data is there. For large systems this is a prohibitively
> expensive operation to perform per replica.
> Instead, the {{ProvidedVolumeImpl}} should own the reference to the
> {{remoteFS}} and should share it with the {{ProvidedReplica}} objects on
> their creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
