[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894076#comment-13894076 ] Mithun Radhakrishnan commented on HIVE-3012: Sorry, all. Looks like what's checked in is a more generic fix (that differs from the patch here). hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.10.0 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.3.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880629#comment-13880629 ] Mithun Radhakrishnan commented on HIVE-3012: Was this patch checked in? I don't see the code in this patch in branch-0.10/, branch-0.12/ or in trunk/. Perhaps I'm missing something. hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.10.0 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.3.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547850#comment-13547850 ] Hudson commented on HIVE-3012: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3012 hive custom scripts do not work well if the data contains new lines (njain via kevinwilfong) (Revision 1336986) Result = ABORTED kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1336986 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/scripts/newline.py * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TextRecordReader.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TextRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java * /hive/trunk/ql/src/test/queries/clientpositive/newline.q * /hive/trunk/ql/src/test/results/clientpositive/newline.q.out hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.10.0 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-3012.D3099.3.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13273125#comment-13273125 ] Hudson commented on HIVE-3012: -- Integrated in Hive-trunk-h0.21 #1424 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1424/]) HIVE-3012 hive custom scripts do not work well if the data contains new lines (njain via kevinwilfong) (Revision 1336986) Result = FAILURE kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1336986 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/data/scripts/newline.py * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TextRecordReader.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TextRecordWriter.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java * /hive/trunk/ql/src/test/queries/clientpositive/newline.q * /hive/trunk/ql/src/test/results/clientpositive/newline.q.out hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch, HIVE-3012.D3099.2.patch, HIVE-3012.D3099.3.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271677#comment-13271677 ] Namit Jain commented on HIVE-3012: -- https://reviews.facebook.net/D3099 hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271703#comment-13271703 ] Phabricator commented on HIVE-3012: --- kevinwilfong has commented on the revision HIVE-3012 [jira] hive custom scripts do not work well if the data contains new lines. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java:100-119 Rather than use methods here, wouldn't it be easier to just use a final static variable? E.g. static final byte[] newLineEscapeBytes = \\n.getBytes(); common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:393 Could you add this to conf/hive-default.xml.template with a description of what it does? REVISION DETAIL https://reviews.facebook.net/D3099 hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271759#comment-13271759 ] Ashutosh Chauhan commented on HIVE-3012: This has been fixed upstream: https://issues.apache.org/jira/browse/HADOOP-7096 If thats there, do we still need this? hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271827#comment-13271827 ] Namit Jain commented on HIVE-3012: -- We still need it - a lot of hive clients are using a older hadoop version hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch, HIVE-3012.D3099.2.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271829#comment-13271829 ] Namit Jain commented on HIVE-3012: -- comments addressed hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch, HIVE-3012.D3099.2.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3012) hive custom scripts do not work well if the data contains new lines
[ https://issues.apache.org/jira/browse/HIVE-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271846#comment-13271846 ] Phabricator commented on HIVE-3012: --- kevinwilfong has accepted the revision HIVE-3012 [jira] hive custom scripts do not work well if the data contains new lines. +1 Running tests. REVISION DETAIL https://reviews.facebook.net/D3099 BRANCH svn To: JIRA, kevinwilfong, njain Cc: kevinwilfong hive custom scripts do not work well if the data contains new lines --- Key: HIVE-3012 URL: https://issues.apache.org/jira/browse/HIVE-3012 Project: Hive Issue Type: Improvement Reporter: Namit Jain Assignee: Namit Jain Attachments: HIVE-3012.D3099.1.patch, HIVE-3012.D3099.2.patch, HIVE-3012.D3099.3.patch If the data contain newline, it will be passed as is to the script. The script has no way of splitting the data based on the new line. An option should be added to hive to escape/unescape the new lines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
