thesuperzapper commented on issue #638: [WIP] [HUDI-12] Upgrade to Spark 2.4, Avro 1.8.2, Parquet 1.10.0... (WIP) URL: https://github.com/apache/incubator-hudi/pull/638#issuecomment-486269374 @bvaradar I have brought my branch up to speed with master (but it will be a bit of work to squash those commits nicely once we are done, as XML and git don't like each-other) When I build the images locally and run tests, they all pass. (Integration, and otherwise) However when running the demo everything works until this: (Which is the same as the initial PR) ```bash 0: jdbc:hive2://hiveserver:10000> select symbol, max(ts) from stock_ticks_mor_rt group by symbol HAVING symbol = 'GOOG'; WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Error: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257) at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) (state=08S01,code=2) ``` This might be related to: * Replacing `com.twitter:parquet-avro` with `org.apache.parquet:parquet-avro` * Replacing `com.twitter:parquet-hadoop` with `org.apache.parquet:parquet-hadoop` (Which I mostly did due to `com.twitter:parquet-avro` using avro 1.7.6 rather than 1.8.2)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
