[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135847#comment-14135847 ] Gopal V commented on HIVE-8090: --- Test failures look unrelated - +1. Assigned to myself till [~rpalamut] gets contributor access. Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.14.0 Reporter: Ted Yu Assignee: Gopal V Priority: Minor Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch, HIVE-8090.4.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134475#comment-14134475 ] Rahul Palamuttam commented on HIVE-8090: Thanks Gopal. I will send an email to dev-subscr...@hive.apache.org Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Ted Yu Priority: Minor Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch, HIVE-8090.4.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134819#comment-14134819 ] Hive QA commented on HIVE-8090: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12668839/HIVE-8090.4.patch {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6276 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.ql.parse.TestParse.testParse_union {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/811/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/811/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-811/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12668839 Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Ted Yu Priority: Minor Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch, HIVE-8090.4.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133124#comment-14133124 ] Rahul Palamuttam commented on HIVE-8090: Please ignore that patch Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133139#comment-14133139 ] Hive QA commented on HIVE-8090: --- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12668648/HIVE-8090.1.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/789/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/789/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-789/ Messages: {noformat} This message was trimmed, see log for full details Decision can match input such as LPAREN KW_CASE KW_MAP using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN LPAREN KW_ARRAY using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN LPAREN KW_MAP using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN KW_NULL LESSTHANOREQUALTO using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN KW_CASE KW_ARRAY using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN KW_NULL LESSTHAN using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as LPAREN KW_NOT KW_DATE using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:115:5: Decision can match input such as KW_CLUSTER KW_BY LPAREN using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:127:5: Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:138:5: Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:149:5: Decision can match input such as KW_SORT KW_BY LPAREN using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:166:7: Decision can match input such as STAR using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as KW_NULL using multiple alternatives: 1, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as KW_TRUE using multiple alternatives: 3, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as KW_FALSE using multiple alternatives: 3, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as KW_DATE StringLiteral using multiple alternatives: 2, 3 As a result, alternative(s) 3 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER KW_BY using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE KW_BY using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER KW_BY using multiple alternatives: 2, 9 As a result, alternative(s) 9 were
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133360#comment-14133360 ] Rahul Palamuttam commented on HIVE-8090: My apologies on the previous patch - a bit embarrassing. This one should be fine. Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133363#comment-14133363 ] Rahul Palamuttam commented on HIVE-8090: Waiting for code review Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133362#comment-14133362 ] Gopal V commented on HIVE-8090: --- [~rpalamut]: You patch turns off this feature for ZlibCodec, which is the only codec which implements the modify() operation today. In this context, I think I missed a null check within the ZlibCodec::modify() check. Can you add that there instead of disabling the feature? Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133399#comment-14133399 ] Rahul Palamuttam commented on HIVE-8090: I followed the comment guidelines in CompressionCodec.java. /** * Produce a modified compression codec if the underlying algorithm allows * modification. * * This does not modify the current object, but returns a new object if * modifications are possible. Returns the same object if no modifications * are possible. * @param modifiers compression modifiers * @return codec for use after optional modification */ CompressionCodec modify(EnumSetModifier modifiers); Thus if modifiers are null then no modifications are possible. As per the comment the function returns the same object. Waiting for code review. Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133410#comment-14133410 ] Hive QA commented on HIVE-8090: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12668672/HIVE-8090.2.patch {color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 6275 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_ptf org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_ptf org.apache.hadoop.hive.ql.io.orc.TestFileDump.testDictionaryThreshold org.apache.hadoop.hive.ql.io.orc.TestFileDump.testDump org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/792/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/792/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-792/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 18 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12668672 Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133426#comment-14133426 ] Gopal V commented on HIVE-8090: --- Nice, that's what I meant - null implies no modifications. I will need to re-format your changes for this patch, please submit patches with no extra whitespace and following the hive code guidelines. {code} if(modifiers == null) return this; {code} should be {code} if (modifiers == null) { return this; } {code} I'll pick this up next week and commit after a re-test. Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Ted Yu Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8090) Potential null pointer reference in WriterImpl#StreamFactory#createStream()
[ https://issues.apache.org/jira/browse/HIVE-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133465#comment-14133465 ] Hive QA commented on HIVE-8090: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12668678/HIVE-8090.3.patch {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6275 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/794/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/794/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-794/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12668678 Potential null pointer reference in WriterImpl#StreamFactory#createStream() --- Key: HIVE-8090 URL: https://issues.apache.org/jira/browse/HIVE-8090 Project: Hive Issue Type: Bug Affects Versions: 0.14.0 Reporter: Ted Yu Priority: Minor Attachments: HIVE-8090.1.patch, HIVE-8090.2.patch, HIVE-8090.3.patch {code} switch (kind) { ... default: modifiers = null; break; } BufferedStream result = streams.get(name); if (result == null) { result = new BufferedStream(name.toString(), bufferSize, codec == null ? codec : codec.modify(modifiers)); {code} In case modifiers is null and codec is ZlibCodec, there would be NPE in ZlibCodec#modify(EnumSetModifier modifiers) : {code} for (Modifier m : modifiers) { {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)