[jira] [Created] (HIVE-23771) load数据到hive,limit 显示用户名中文正确,where 用户名乱码,并且不能使用用户名比对
wang created HIVE-23771: --- Summary: load数据到hive,limit 显示用户名中文正确,where 用户名乱码,并且不能使用用户名比对 Key: HIVE-23771 URL: https://issues.apache.org/jira/browse/HIVE-23771 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 2.1.1 Reporter: wang Fix For: 2.1.1 Attachments: image-2020-06-29-15-04-23-999.png, image-2020-06-29-15-08-25-923.png, image-2020-06-29-15-10-10-310.png 建表语句:create table smg_t_usr_inf_23( Usr_ID string, RlgnSvcPltfrmUsr_TpCd string, Rlgn_InsID string, Usr_Nm string , ) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ("field.delim"="|@|") stored as textfile 导入数据:LOAD DATA LOCAL INPATH '/home/ap/USR_INF 20200622_0001.dat' INTO TABLE usr_inf select * from usr_inf limit 10;显示数据: !image-2020-06-29-15-04-23-999.png! select * from usr_inf where usr_nm = '胡学玲' ;无显示数据: !image-2020-06-29-15-08-25-923.png! 其他select * from usr_inf where usr_id='***';显示数据 !image-2020-06-29-15-10-10-310.png! . 求大神解答,为什么导入的数据是中文但是where就有问题,直接insert into table aa select * from usr_inf;新表 的usr_nm 字段也是同上 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23772) Reallocate calcite-core to prevent NoSuchFiledError
Ayush Saxena created HIVE-23772: --- Summary: Reallocate calcite-core to prevent NoSuchFiledError Key: HIVE-23772 URL: https://issues.apache.org/jira/browse/HIVE-23772 Project: Hive Issue Type: Bug Affects Versions: 4.0.0 Reporter: Ayush Saxena Assignee: Ayush Saxena Exception trace due to conflict with {{calcite-core}} {noformat} Caused by: java.lang.NoSuchFieldError: operands at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitCall(ASTConverter.java:785) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter$RexVisitor.visitCall(ASTConverter.java:509) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) ~[calcite-core-1.21.0.jar:1.21.0] at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convert(ASTConverter.java:239) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convertSource(ASTConverter.java:437) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convert(ASTConverter.java:124) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter.convert(ASTConverter.java:112) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1620) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:555) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12456) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:433) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:290) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:184) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:602) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:548) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:542) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:125) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199) ~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT] {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23774) Reduce log level at aggrColStatsForPartitions in MetaStoreDirectSql.java
Barnabas Maidics created HIVE-23774: --- Summary: Reduce log level at aggrColStatsForPartitions in MetaStoreDirectSql.java Key: HIVE-23774 URL: https://issues.apache.org/jira/browse/HIVE-23774 Project: Hive Issue Type: Improvement Reporter: Barnabas Maidics Assignee: Barnabas Maidics [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1589] This log is not needed at INFO log level. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23773) Support multi-key probe MapJoins
Panagiotis Garefalakis created HIVE-23773: - Summary: Support multi-key probe MapJoins Key: HIVE-23773 URL: https://issues.apache.org/jira/browse/HIVE-23773 Project: Hive Issue Type: Sub-task Reporter: Panagiotis Garefalakis Assignee: Panagiotis Garefalakis -- This message was sent by Atlassian Jira (v8.3.4#803005)
Announcing ApacheCon @Home 2020
Hi, Apache enthusiast! (You’re receiving this because you’re subscribed to one or more dev or user mailing lists for an Apache Software Foundation project.) The ApacheCon Planners and the Apache Software Foundation are pleased to announce that ApacheCon @Home will be held online, September 29th through October 1st, 2020. We’ll be featuring content from dozens of our projects, as well as content about community, how Apache works, business models around Apache software, the legal aspects of open source, and many other topics. Full details about the event, and registration, is available at https://apachecon.com/acah2020 Due to the confusion around how and where this event was going to be held, and in order to open up to presenters from around the world who may previously have been unable or unwilling to travel, we’ve reopened the Call For Presentations until July 13th. Submit your talks today at https://acna2020.jamhosted.net/ We hope to see you at the event! Rich Bowen, VP Conferences, The Apache Software Foundation
Re: Open old PRs
Hey all! Around friday I've made a small change: and enabled daily repo scanning of apache/hive; it picked up ~80 pending prs for testruns - during the weekeng it was able to finish with all of them :) The periodic repo scanning adds an extra safety net which could help mitigate issues in case the PR event was lost for some reason. cheers, Zoltan On 5/30/20 4:15 PM, Zoltan Haindrich wrote: Hey All, The new test executor will pick up any PR which doesn't yet have a test result - now that the patch is on the master; every PR which is mergeable with the master branch is a good candidate - so the right move would be to clean up our PR backlog. I would like to ask everyone to look at https://github.com/apache/hive/pulls and close some PRs which are already submitted or just leftovers from - primarily I would ask you to look at PRs opened by yourself... cheers, Zoltan
[jira] [Created] (HIVE-23775) investigate windowing spec when an order by is present
Zoltan Haindrich created HIVE-23775: --- Summary: investigate windowing spec when an order by is present Key: HIVE-23775 URL: https://issues.apache.org/jira/browse/HIVE-23775 Project: Hive Issue Type: Bug Reporter: Zoltan Haindrich Assignee: Zoltan Haindrich this is some weird stuff came up during review https://github.com/apache/hive/pull/1126#discussion_r442266978 Order by spec -> range, unbounded preceding, current row This also aligns with most RDBMSs implementation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23776) Retire quickstats autocollection
Zoltan Haindrich created HIVE-23776: --- Summary: Retire quickstats autocollection Key: HIVE-23776 URL: https://issues.apache.org/jira/browse/HIVE-23776 Project: Hive Issue Type: Improvement Reporter: Zoltan Haindrich Assignee: Zoltan Haindrich this is about: * num files * datasize (sum of filesizes) * num erasure coded files right now these are scanned during every BasicStatsTask execution - which means some filesystem reads/etc - for small inserts these are visible in case the fs is a bit slower (s3 and friends) I don't think they are really in use...we rely more on columnstats which are more accurate ; and because of the datasize in this case is for "offline" (ondisk) - while we should be insted calculate with "online" sizes... proposal: * remove collection and storage of this data * collect it on the fly during "desc formatted" statements to provide them for informational purposes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23777) hive.semantic.analyzer.hook missing documentation
liuyan created HIVE-23777: - Summary: hive.semantic.analyzer.hook missing documentation Key: HIVE-23777 URL: https://issues.apache.org/jira/browse/HIVE-23777 Project: Hive Issue Type: Improvement Components: Documentation Affects Versions: 3.1.2, 3.1.1, 3.1.0, 3.0.0 Reporter: liuyan hive.semantic.analyzer.hook is missing documentation from the [https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties] Page -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23778) Unable to start Hive - java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProvider
sathyanarayanan created HIVE-23778: -- Summary: Unable to start Hive - java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProvider Key: HIVE-23778 URL: https://issues.apache.org/jira/browse/HIVE-23778 Project: Hive Issue Type: Test Components: CLI Affects Versions: 2.3.7 Environment: Hadoop 2.3.0-cdh5.0.0 Hive 2.3.7 VM - Oracle Linux 7 Bash profile - export JAVA_HOME=/usr/local/java/jdk1.8.0_251/ export PATH=$PATH:$JAVA_HOME/bin export HADOOP_HOME=/usr/local/hadoop/hadoop-2.3.0-cdh5.0.0 export PATH=$PATH:$HADOOP_HOME/bin export HADOOP_HOME=/usr/local/hadoop/hadoop-2.3.0-cdh5.0.0 export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin export HADOOP_INSTALL=$HADOOP_HOME export HIVE_HOME=/usr/local/hive/apache-hive-2.3.7-bin export PATH=$PATH:$HIVE_HOME/bin export CLASSPATH=$CLASSPATH:/usr/local/hive/apache-hive-2.3.7-bin/lib/ export HADOOP_CLASSPATH=/usr/local/hive/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/hive/apache-hive-2.3.7-bin/lib/*.jar export HIVE_CLASSPATH=/usr/local/hive/apache-hive-2.3.7-bin/lib/:/usr/local/hadoop/hadoop-2.3.0-cdh5.0.0/share/hadoop/mapreduce1/lib/*.jar export DERBY_HOME=/usr/local/derby/db-derby-10.4.2.0-bin export PATH=$PATH:$DERBY_HOME/bin export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar [root@hadhiv bin]# Reporter: sathyanarayanan My Hadoop version is Hadoop 2.3.0-cdh5.0.0 and my Hive 2.3.7, while executive Hive, i'm getting the following error message. I could sense that there is some JAR file missing in the classpath. But could not get the relevant classpath. Logging initialized using configuration in jar:file:/usr/local/hive/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true Exception in thread "main" java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProvider at org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:91) at org.apache.hadoop.hive.ql.exec.Utilities.createDirsWithPermission(Utilities.java:3323) at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:709) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:654) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:553) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProvider at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hive.shims.ShimLoader.createShim(ShimLoader.java:129) at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:124) at org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:88) ... 12 more Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.crypto.key.KeyProvider at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 17 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23779) BasicStatsTask Info is not getting printed in beeline console
Naresh P R created HIVE-23779: - Summary: BasicStatsTask Info is not getting printed in beeline console Key: HIVE-23779 URL: https://issues.apache.org/jira/browse/HIVE-23779 Project: Hive Issue Type: Bug Reporter: Naresh P R Assignee: Naresh P R After HIVE-16061, partition basic stats are not getting printed in beeline console. {code:java} INFO : Partition {dt=2020-06-29} stats: [numFiles=21, numRows=22, totalSize=14607, rawDataSize=0]{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-23780) Fail dropTable if acid cleanup fails
Mustafa Iman created HIVE-23780: --- Summary: Fail dropTable if acid cleanup fails Key: HIVE-23780 URL: https://issues.apache.org/jira/browse/HIVE-23780 Project: Hive Issue Type: Bug Components: Metastore, Standalone Metastore, Transactions Reporter: Mustafa Iman Assignee: Mustafa Iman Acid cleanup happens after dropTable is committed. If cleanup fails for some reason, there are leftover entries in acid tables. This later causes dropped table's name to be unusable by new tables. [~pvary] [~ngangam] -- This message was sent by Atlassian Jira (v8.3.4#803005)