BELUGA BEHR created HIVE-20832: ---------------------------------- Summary: Locate Operation Logs Under Session Directory Key: HIVE-20832 URL: https://issues.apache.org/jira/browse/HIVE-20832 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 3.1.0, 4.0.0 Reporter: BELUGA BEHR
If I understand the Hive entity relationship model correctly, each Hive Session will have 0 or more Operations associated with it. Here is the current session setup sequence: {code} 2018-10-24 21:06:03,771 INFO org.apache.hadoop.hive.ql.session.SessionState: [HiveServer2-Handler-Pool: Thread-510932]: Created local directory: /tmp/hive/7650c8ff-2ba5-4bbb-964f-fcecc45e5193 2018-10-24 21:06:03,779 INFO org.apache.hadoop.hive.ql.session.SessionState: [HiveServer2-Handler-Pool: Thread-510932]: Created HDFS directory: /tmp/hive/hive/7650c8ff-2ba5-4bbb-964f-fcecc45e5193/_tmp_space.db 2018-10-24 21:06:03,782 INFO org.apache.hive.service.cli.session.HiveSessionImpl: [HiveServer2-Handler-Pool: Thread-510932]: Operation log session directory is created: /var/log/hive/operation_logs/7650c8ff-2ba5-4bbb-964f-fcecc45e5193 {code} The Hive Session gets its own directory on the local FS and operation logs get their own space as well. Can we please merge so that all of the Operation directories are stored within their associated Hive session directory? Something like... {code} 2018-10-24 21:06:03,771 INFO org.apache.hadoop.hive.ql.session.SessionState: [HiveServer2-Handler-Pool: Thread-510932]: Created local directory: /tmp/hive/7650c8ff-2ba5-4bbb-964f-fcecc45e5193 2018-10-24 21:06:03,779 INFO org.apache.hadoop.hive.ql.session.SessionState: [HiveServer2-Handler-Pool: Thread-510932]: Created HDFS directory: /tmp/hive/hive/7650c8ff-2ba5-4bbb-964f-fcecc45e5193/_tmp_space.db 2018-10-24 21:06:03,782 INFO org.apache.hive.service.cli.session.HiveSessionImpl: [HiveServer2-Handler-Pool: Thread-510932]: Operation log session directory is created: /tmp/hive/7650c8ff-2ba5-4bbb-964f-fcecc45e5193/operation_logs/7650c8ff-2ba5-4bbb-964f-fcecc45e5193 {code} Allows removal of configuration {{hive.server2.logging.operation.log.location}}. One less thing an operator needs to worry/know about. One set of logs are in {{/tmp}} and the other is {{/var}}. A bit confusing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)