[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4679: - Attachment: HIVE-4679.2.patch HIVE-4679.2.patch specifies valuedefault/value for the queue.name property rather than empty string. WebHCat e2e tests still don't run clean due to HIVE-4703 but the failures are the same with this patch and w/o. WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-4679.2.patch, HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4679: - Status: Patch Available (was: Open) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-4679.2.patch, HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-4679: - Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed. WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-4679.2.patch, HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-4679: - Status: Open (was: Patch Available) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4679: - Fix Version/s: 0.12.0 WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4679: - Attachment: HIVE-4679.patch WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4679) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers
[ https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4679: - Status: Patch Available (was: Open) WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers - Key: HIVE-4679 URL: https://issues.apache.org/jira/browse/HIVE-4679 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.10.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-4679.patch o In the current Templeton design, each time a Job is submitted thru the REST API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot. Given that the number of map slots is finite in the cluster (16 node cluster will have 32 map slots), in some circumstances, a user can deadlock the cluster if Templeton job submission pipeline takes over all map slots (Templeton map tasks will wait for the actual underlying jobs to complete, what will never happen, given that Hadoop has no free map slots to schedule new tasks). o HCat queries use a different mechanism and do not contribute to the deadlock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
