[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13113865#comment-13113865 ] Hudson commented on HIVE-2181: -- Integrated in Hive-trunk-h0.21 #974 (See [https://builds.apache.org/job/Hive-trunk-h0.21/974/]) HIVE-2181. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. (Chinna Rao Lalam via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1174892 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ServerUtils.java * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml * /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java * /hive/trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.4.patch, HIVE-2181.5.patch, HIVE-2181.6.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13112796#comment-13112796 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/ --- (Updated 2011-09-22 18:26:19.812602) Review request for hive. Changes --- In testcases server startup is removed and added unit tests for cleanUpScratchDir(). Summary --- Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir This addresses bug HIVE-2181. https://issues.apache.org/jira/browse/HIVE-2181 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/common/ServerUtils.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1174277 trunk/conf/hive-default.xml 1174277 trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1174277 trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1174277 Diff: https://reviews.apache.org/r/1481/diff Testing --- Added test case for this scenario. Thanks, chinna Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.4.patch, HIVE-2181.5.patch, HIVE-2181.6.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13112881#comment-13112881 ] John Sichi commented on HIVE-2181: -- +1. Will commit when tests pass. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.4.patch, HIVE-2181.5.patch, HIVE-2181.6.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13109494#comment-13109494 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/ --- (Updated 2011-09-21 13:30:22.283181) Review request for hive. Changes --- spelling mistake corrected Summary --- Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir This addresses bug HIVE-2181. https://issues.apache.org/jira/browse/HIVE-2181 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1173539 trunk/conf/hive-default.xml 1173539 trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1173539 trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1173539 Diff: https://reviews.apache.org/r/1481/diff Testing --- Added test case for this scenario. Thanks, chinna Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.4.patch, HIVE-2181.5.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108956#comment-13108956 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/#review1977 --- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java https://reviews.apache.org/r/1481/#comment4467 SCRATCH needs a T in it (everywhere). - John On 2011-09-16 17:30:40, chinna wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1481/ bq. --- bq. bq. (Updated 2011-09-16 17:30:40) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. bq. bq. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir bq. bq. bq. This addresses bug HIVE-2181. bq. https://issues.apache.org/jira/browse/HIVE-2181 bq. bq. bq. Diffs bq. - bq. bq.trunk/conf/hive-default.xml 1171488 bq.trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1171488 bq. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1171488 bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1171488 bq. bq. Diff: https://reviews.apache.org/r/1481/diff bq. bq. bq. Testing bq. --- bq. bq. Added test case for this scenario. bq. bq. bq. Thanks, bq. bq. chinna bq. bq. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.4.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13106615#comment-13106615 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/ --- (Updated 2011-09-16 17:30:40.980271) Review request for hive. Changes --- Rebased the patch on latest trunk Summary --- Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir This addresses bug HIVE-2181. https://issues.apache.org/jira/browse/HIVE-2181 Diffs (updated) - trunk/conf/hive-default.xml 1171488 trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1171488 trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1171488 trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1171488 Diff: https://reviews.apache.org/r/1481/diff Testing --- Added test case for this scenario. Thanks, chinna Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.3.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105007#comment-13105007 ] Hudson commented on HIVE-2181: -- Integrated in Hive-trunk-h0.21 #954 (See [https://builds.apache.org/job/Hive-trunk-h0.21/954/]) HIVE-2181. Avoid null pointer exception when executing UDF (Chinna Rao Lalam via jvs) jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1170721 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java * /hive/trunk/ql/src/test/queries/clientnegative/udfnull.q * /hive/trunk/ql/src/test/results/clientnegative/udfnull.q.out * /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input20.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input8.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/udf6.q.xml Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105014#comment-13105014 ] John Sichi commented on HIVE-2181: -- Oops, looks like I typed in the wrong JIRA issue number in the commit message :( Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1308#comment-1308 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/ --- (Updated 2011-08-22 18:23:15.749654) Review request for hive. Changes --- Fixed review comments and updated the patch. Summary --- Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir This addresses bug HIVE-2181. https://issues.apache.org/jira/browse/HIVE-2181 Diffs (updated) - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1158629 trunk/conf/hive-default.xml 1158629 trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1158629 trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1158629 Diff: https://reviews.apache.org/r/1481/diff Testing --- Added test case for this scenario. Thanks, chinna Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086293#comment-13086293 ] jirapos...@reviews.apache.org commented on HIVE-2181: - bq. On 2011-08-13 14:08:14, M IS wrote: bq. trunk/data/conf/hive-site.xml, line 181 bq. https://reviews.apache.org/r/1481/diff/1/?file=32387#file32387line181 bq. bq. This overrides the value set in hive-default.xml. And also deviates from the existing behavior. So, the property needn't be mentioned in hive-site.xml This is added for the testing purpose. Now it is removed. bq. On 2011-08-13 14:08:14, M IS wrote: bq. trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java, line 713 bq. https://reviews.apache.org/r/1481/diff/1/?file=32388#file32388line713 bq. bq. Why create a reference, when the same can be logged directly and easily readable as well. Creating a reference for the log message is removed bq. On 2011-08-13 14:08:14, M IS wrote: bq. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java, line 394 bq. https://reviews.apache.org/r/1481/diff/1/?file=32389#file32389line394 bq. bq. There needs to be a test case to test the other use-case also, where the user would have chosen not to set the property to true. Added a testcase for other user-case (Test should not clear scratchDir) - chinna --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/#review1444 --- On 2011-08-12 19:10:40, chinna wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1481/ bq. --- bq. bq. (Updated 2011-08-12 19:10:40) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. bq. bq. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir bq. bq. bq. This addresses bug HIVE-2181. bq. https://issues.apache.org/jira/browse/HIVE-2181 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1156493 bq.trunk/conf/hive-default.xml 1156493 bq.trunk/data/conf/hive-site.xml 1156493 bq.trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1156493 bq. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1156493 bq. bq. Diff: https://reviews.apache.org/r/1481/diff bq. bq. bq. Testing bq. --- bq. bq. Added test case for this scenario. bq. bq. bq. Thanks, bq. bq. chinna bq. bq. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.2.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084620#comment-13084620 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/#review1444 --- trunk/data/conf/hive-site.xml https://reviews.apache.org/r/1481/#comment3342 This overrides the value set in hive-default.xml. And also deviates from the existing behavior. So, the property needn't be mentioned in hive-site.xml trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java https://reviews.apache.org/r/1481/#comment3343 Why create a reference, when the same can be logged directly and easily readable as well. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java https://reviews.apache.org/r/1481/#comment3344 There needs to be a test case to test the other use-case also, where the user would have chosen not to set the property to true. - M On 2011-08-12 19:10:40, chinna wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/1481/ bq. --- bq. bq. (Updated 2011-08-12 19:10:40) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. bq. bq. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir bq. bq. bq. This addresses bug HIVE-2181. bq. https://issues.apache.org/jira/browse/HIVE-2181 bq. bq. bq. Diffs bq. - bq. bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1156493 bq.trunk/conf/hive-default.xml 1156493 bq.trunk/data/conf/hive-site.xml 1156493 bq.trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1156493 bq. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1156493 bq. bq. Diff: https://reviews.apache.org/r/1481/diff bq. bq. bq. Testing bq. --- bq. bq. Added test case for this scenario. bq. bq. bq. Thanks, bq. bq. chinna bq. bq. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Attachments: HIVE-2181.1.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13084309#comment-13084309 ] jirapos...@reviews.apache.org commented on HIVE-2181: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1481/ --- Review request for hive. Summary --- Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. Implemented a cleanup method in HiveServer and it will trigger based on the property value hive.start.cleanup.scrachdir This addresses bug HIVE-2181. https://issues.apache.org/jira/browse/HIVE-2181 Diffs - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1156493 trunk/conf/hive-default.xml 1156493 trunk/data/conf/hive-site.xml 1156493 trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1156493 trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1156493 Diff: https://reviews.apache.org/r/1481/diff Testing --- Added test case for this scenario. Thanks, chinna Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Labels: patch Fix For: 0.8.0 Attachments: HIVE-2181.1.patch, HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[
https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13083306#comment-13083306
]
MIS commented on HIVE-2181:
---
-1 for the issue.
What if I'm running multiple hive servers on different port in the same machine
{With my metastore db on a mysql server}, then if one of the server instances
restarts, it would end up deleting the scratch dir, which would affect other
running instances as well. Even if we specify different scratch dir for each of
the instances, I doubt about the value add from this property.
Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
Key: HIVE-2181
URL: https://issues.apache.org/jira/browse/HIVE-2181
Project: Hive
Issue Type: Bug
Components: Server Infrastructure
Affects Versions: 0.8.0
Environment: Suse linux, Hadoop 20.1, Hive 0.8
Reporter: sanoj mathew
Assignee: Chinna Rao Lalam
Priority: Minor
Labels: patch
Fix For: 0.8.0
Attachments: HIVE-2181.patch
Original Estimate: 48h
Remaining Estimate: 48h
Now queries leaves the map outputs under scratch.dir after execution. If the
hive server is stopped we need not keep the stopped server's map oputputs. So
whle starting the server we can clear the scratch.dir. This can help in
improved disk usage.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2181) Clean up the scratch.dir (tmp/hive-root) while restarting Hive server.
[ https://issues.apache.org/jira/browse/HIVE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13083401#comment-13083401 ] Chinna Rao Lalam commented on HIVE-2181: Hi MIS, Thanks for the point as u said if we have multiple instances with the same scratch dir on same machine it wont help. But in this case if we give different value for the scratch dir it may help(I will double check this point). I will introduce one propety for this like hive.start.cleanup.scrachdir . This cleanup can trigger based on this property value. By default it will be turned off. If cleanup need to do while starting the server turn on. Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. Key: HIVE-2181 URL: https://issues.apache.org/jira/browse/HIVE-2181 Project: Hive Issue Type: Bug Components: Server Infrastructure Affects Versions: 0.8.0 Environment: Suse linux, Hadoop 20.1, Hive 0.8 Reporter: sanoj mathew Assignee: Chinna Rao Lalam Priority: Minor Labels: patch Fix For: 0.8.0 Attachments: HIVE-2181.patch Original Estimate: 48h Remaining Estimate: 48h Now queries leaves the map outputs under scratch.dir after execution. If the hive server is stopped we need not keep the stopped server's map oputputs. So whle starting the server we can clear the scratch.dir. This can help in improved disk usage. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
