[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435707#comment-15435707 ] Krishna Anisetty commented on HIVE-10485: - We are using Hive 1.1.0. We dont have any plans on upgrading to 2.0.0. But is there is any standalone way to just install this function. May be as UDF? > Create md5 UDF > -- > > Key: HIVE-10485 > URL: https://issues.apache.org/jira/browse/HIVE-10485 > Project: Hive > Issue Type: Task > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, > HIVE-10485.3.patch > > > MD5(str) > Calculates an MD5 128-bit checksum for the string. The value is returned as a > string of 32 hex digits, or NULL if the argument was NULL. The return value > can, for example, be used as a hash key. > Example: > {code} > SELECT MD5('udf_md5'); > 'ce62ef0d2d27dc37b6d488b92f4b24fd' > {code} > online md5 generator: http://www.md5.cz/ > MySQL has md5 function: > https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 > PostgreSQL also has md5 function: > http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183384#comment-15183384 ] Duryea Lee commented on HIVE-10485: --- I am getting the same error that Russell Pierce received October 2015. What should I be doing that I am not. > Create md5 UDF > -- > > Key: HIVE-10485 > URL: https://issues.apache.org/jira/browse/HIVE-10485 > Project: Hive > Issue Type: Task > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, > HIVE-10485.3.patch > > > MD5(str) > Calculates an MD5 128-bit checksum for the string. The value is returned as a > string of 32 hex digits, or NULL if the argument was NULL. The return value > can, for example, be used as a hash key. > Example: > {code} > SELECT MD5('udf_md5'); > 'ce62ef0d2d27dc37b6d488b92f4b24fd' > {code} > online md5 generator: http://www.md5.cz/ > MySQL has md5 function: > https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 > PostgreSQL also has md5 function: > http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970469#comment-14970469 ] Russell Pierce commented on HIVE-10485: --- Is there anything that must be done to support the md5 function? In: Hive 1.0.0-amzn-1 Subversion git://ip-10-155-25-185/workspace/workspace/bigtop.all-rpm-4.1.0/build/hive/rpm/BUILD/apache-hive-1.0.0-amzn-1-src -r edd5a97db145470a8723dde24f38c83724e0959c Compiled by ec2-user on Fri Sep 25 15:34:36 UTC 2015 >From source with checksum af54f19f33a1535ad110eda19817fa7e I was getting: hive> SELECT MD5('udf_md5'); FAILED: SemanticException [Error 10011]: Line 1:7 Invalid function 'MD5' > Create md5 UDF > -- > > Key: HIVE-10485 > URL: https://issues.apache.org/jira/browse/HIVE-10485 > Project: Hive > Issue Type: Task > Components: UDF >Reporter: Alexander Pivovarov >Assignee: Alexander Pivovarov > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, > HIVE-10485.3.patch > > > MD5(str) > Calculates an MD5 128-bit checksum for the string. The value is returned as a > string of 32 hex digits, or NULL if the argument was NULL. The return value > can, for example, be used as a hash key. > Example: > {code} > SELECT MD5('udf_md5'); > 'ce62ef0d2d27dc37b6d488b92f4b24fd' > {code} > online md5 generator: http://www.md5.cz/ > MySQL has md5 function: > https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 > PostgreSQL also has md5 function: > http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698098#comment-14698098 ] Ruslan Dautkhanov commented on HIVE-10485: -- that's great. would it be possible to have another md5 function that returns base64 representation and not hex? or returns binary so we one can use base64() function? Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Fix For: 1.3.0, 2.0.0 Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, HIVE-10485.3.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698117#comment-14698117 ] Alexander Pivovarov commented on HIVE-10485: Ruslan, try unhex(md5(string/binary)) Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Fix For: 1.3.0, 2.0.0 Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, HIVE-10485.3.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531001#comment-14531001 ] Alexander Pivovarov commented on HIVE-10485: added md5 UDF description to Misc. Functions https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-Misc.Functions Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Fix For: 1.3.0 Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, HIVE-10485.3.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518945#comment-14518945 ] Hive QA commented on HIVE-10485: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12728862/HIVE-10485.3.patch {color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 8829 tests executed *Failed tests:* {noformat} TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a TEST-*.xml file TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did not produce a TEST-*.xml file {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3639/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3639/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3639/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 13 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12728862 - PreCommit-HIVE-TRUNK-Build Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, HIVE-10485.3.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518838#comment-14518838 ] Jason Dere commented on HIVE-10485: --- +1 if tests look good Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch, HIVE-10485.3.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10485) Create md5 UDF
[ https://issues.apache.org/jira/browse/HIVE-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517517#comment-14517517 ] Hive QA commented on HIVE-10485: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12728709/HIVE-10485.2.patch {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 8821 tests executed *Failed tests:* {noformat} TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a TEST-*.xml file TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3628/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3628/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3628/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 14 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12728709 - PreCommit-HIVE-TRUNK-Build Create md5 UDF -- Key: HIVE-10485 URL: https://issues.apache.org/jira/browse/HIVE-10485 Project: Hive Issue Type: Task Components: UDF Reporter: Alexander Pivovarov Assignee: Alexander Pivovarov Attachments: HIVE-10485.1.patch, HIVE-10485.2.patch MD5(str) Calculates an MD5 128-bit checksum for the string. The value is returned as a string of 32 hex digits, or NULL if the argument was NULL. The return value can, for example, be used as a hash key. Example: {code} SELECT MD5('udf_md5'); 'ce62ef0d2d27dc37b6d488b92f4b24fd' {code} online md5 generator: http://www.md5.cz/ MySQL has md5 function: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html#function_md5 PostgreSQL also has md5 function: http://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)