[jira] [Commented] (GRIFFIN-274) Error: java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
[
https://issues.apache.org/jira/browse/GRIFFIN-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908133#comment-16908133
]
Eugene commented on GRIFFIN-274:
[~Renan Leme]
I think you can try change spark lib dependecies from 2.2.1 to 2.11, it seems a
mismatch issue
spark-streaming_2.11
spark-core_2.11
spark-sql_2.11
Eugene
> Error: java.lang.NoClassDefFoundError:
> scala/collection/GenTraversableOnce$class
>
>
> Key: GRIFFIN-274
> URL: https://issues.apache.org/jira/browse/GRIFFIN-274
> Project: Griffin
> Issue Type: Bug
> Components: completeness-batch
>Affects Versions: 0.5.0, 0.6.0
>Reporter: Renan Paes Leme
>Priority: Major
>
> I'm trying to run one completeness job to test Griffin features but when I
> try to run it, reading from one Avro file I got this error:
>
> {code:java}
> 19/08/07 15:38:47 ERROR executor.Executor: Exception in task 2.2 in stage 2.0
> (TID 13)
> java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
> at
> com.databricks.spark.avro.DefaultSource$$anonfun$buildReader$1$$anon$1.(DefaultSource.scala:205)
> at
> com.databricks.spark.avro.DefaultSource$$anonfun$buildReader$1.apply(DefaultSource.scala:205)
> at
> com.databricks.spark.avro.DefaultSource$$anonfun$buildReader$1.apply(DefaultSource.scala:160)
> at
> org.apache.spark.sql.execution.datasources.FileFormat$$anon$1.apply(FileFormat.scala:136)
> at
> org.apache.spark.sql.execution.datasources.FileFormat$$anon$1.apply(FileFormat.scala:120)
> at
> org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.org$apache$spark$sql$execution$datasources$FileScanRDD$$anon$$readCurrentFile(FileScanRDD.scala:124)
> at
> org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:174)
> at
> org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:105)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
> Source)
> at
> org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
> at
> org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
> at
> org.apache.spark.sql.execution.columnar.InMemoryRelation$$anonfun$1$$anon$1.hasNext(InMemoryRelation.scala:133)
> at
> org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:215)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1038)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1029)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:969)
> at
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1029)
> at
> org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:760)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1038)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1029)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:969)
> at
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1029)
> at
> org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:760)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:32
[jira] [Created] (GRIFFIN-279) Upgrade Spring boot to 2.1.x as Spring Boot 1.x EOL Aug 1st 2019
Johnnie created GRIFFIN-279: --- Summary: Upgrade Spring boot to 2.1.x as Spring Boot 1.x EOL Aug 1st 2019 Key: GRIFFIN-279 URL: https://issues.apache.org/jira/browse/GRIFFIN-279 Project: Griffin Issue Type: Improvement Reporter: Johnnie As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate to 2.1.x. Below is the announcement [https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019] Migrate Guide is [https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (GRIFFIN-279) Upgrade Spring boot to 2.1.7.RELEASE
[ https://issues.apache.org/jira/browse/GRIFFIN-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnnie updated GRIFFIN-279: Summary: Upgrade Spring boot to 2.1.7.RELEASE (was: Upgrade Spring boot to 2.1.x as Spring Boot 1.x EOL Aug 1st 2019) > Upgrade Spring boot to 2.1.7.RELEASE > > > Key: GRIFFIN-279 > URL: https://issues.apache.org/jira/browse/GRIFFIN-279 > Project: Griffin > Issue Type: Improvement >Reporter: Johnnie >Priority: Major > > As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate > to 2.1.x. > Below is the announcement > [https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019] > Migrate Guide is > [https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide] > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (GRIFFIN-280) Issues with travis-ci build
Johnnie created GRIFFIN-280:
---
Summary: Issues with travis-ci build
Key: GRIFFIN-280
URL: https://issues.apache.org/jira/browse/GRIFFIN-280
Project: Griffin
Issue Type: Bug
Reporter: Johnnie
Actually, some of the test cases are failed without external dependencies such
as hive.
Below is the travis-cli and we can find some exceptions
[https://travis-ci.org/apache/griffin]
2019-08-15 08:40:25.987 ERROR 5041 --- [ main] o.a.g.c.m.h.HiveMetaStoreProxy
[70] : Failed to connect hive metastore. {}
568
569org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to
meta store using any of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused (Connection refused)
570 at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
571 at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:478)
572 at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:286)
573 at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:211)
574 at
org.apache.griffin.core.metastore.hive.HiveMetaStoreProxy.initHiveMetastoreClient(HiveMetaStoreProxy.java:68)
575 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
576 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
577 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
578 at java.lang.reflect.Method.invoke(Method.java:498)
579 at
I think we might need to change travis config to execute docker-compose -f to
start up the dockers before we run the CI test.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [griffin] joohnnie opened a new pull request #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE
joohnnie opened a new pull request #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE URL: https://github.com/apache/griffin/pull/523 As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate to 2.1.x. Below is the announcement https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019 Migrate Guide is https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GRIFFIN-279) Upgrade Spring boot to 2.1.7.RELEASE
[ https://issues.apache.org/jira/browse/GRIFFIN-279?focusedWorklogId=295841&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-295841 ] ASF GitHub Bot logged work on GRIFFIN-279: -- Author: ASF GitHub Bot Created on: 15/Aug/19 22:48 Start Date: 15/Aug/19 22:48 Worklog Time Spent: 10m Work Description: joohnnie commented on pull request #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE URL: https://github.com/apache/griffin/pull/523 As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate to 2.1.x. Below is the announcement https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019 Migrate Guide is https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 295841) Time Spent: 10m Remaining Estimate: 0h > Upgrade Spring boot to 2.1.7.RELEASE > > > Key: GRIFFIN-279 > URL: https://issues.apache.org/jira/browse/GRIFFIN-279 > Project: Griffin > Issue Type: Improvement >Reporter: Johnnie >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate > to 2.1.x. > Below is the announcement > [https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019] > Migrate Guide is > [https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide] > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (GRIFFIN-280) Issues with travis-ci build config
[
https://issues.apache.org/jira/browse/GRIFFIN-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johnnie updated GRIFFIN-280:
Summary: Issues with travis-ci build config (was: Issues with travis-ci
build )
> Issues with travis-ci build config
> --
>
> Key: GRIFFIN-280
> URL: https://issues.apache.org/jira/browse/GRIFFIN-280
> Project: Griffin
> Issue Type: Bug
>Reporter: Johnnie
>Priority: Major
>
> Actually, some of the test cases are failed without external dependencies
> such as hive.
> Below is the travis-cli and we can find some exceptions
> [https://travis-ci.org/apache/griffin]
>
> 2019-08-15 08:40:25.987 ERROR 5041 --- [ main] o.a.g.c.m.h.HiveMetaStoreProxy
> [70] : Failed to connect hive metastore. {}
> 568
> 569org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to
> meta store using any of the URIs provided. Most recent failure:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> 570 at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
> 571 at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:478)
> 572 at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:286)
> 573 at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:211)
> 574 at
> org.apache.griffin.core.metastore.hive.HiveMetaStoreProxy.initHiveMetastoreClient(HiveMetaStoreProxy.java:68)
> 575 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 576 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 577 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 578 at java.lang.reflect.Method.invoke(Method.java:498)
> 579 at
>
> I think we might need to change travis config to execute docker-compose -f to
> start up the dockers before we run the CI test.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [griffin] joohnnie commented on issue #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE
joohnnie commented on issue #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE URL: https://github.com/apache/griffin/pull/523#issuecomment-521874232 the failure is caused by JIRA https://issues.apache.org/jira/browse/GRIFFIN-280 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (GRIFFIN-279) Upgrade Spring boot to 2.1.7.RELEASE
[ https://issues.apache.org/jira/browse/GRIFFIN-279?focusedWorklogId=296047&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296047 ] ASF GitHub Bot logged work on GRIFFIN-279: -- Author: ASF GitHub Bot Created on: 16/Aug/19 03:49 Start Date: 16/Aug/19 03:49 Worklog Time Spent: 10m Work Description: joohnnie commented on issue #523: GRIFFIN-279 Upgrade Spring boot to 2.1.7.RELEASE URL: https://github.com/apache/griffin/pull/523#issuecomment-521874232 the failure is caused by JIRA https://issues.apache.org/jira/browse/GRIFFIN-280 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 296047) Time Spent: 20m (was: 10m) > Upgrade Spring boot to 2.1.7.RELEASE > > > Key: GRIFFIN-279 > URL: https://issues.apache.org/jira/browse/GRIFFIN-279 > Project: Griffin > Issue Type: Improvement >Reporter: Johnnie >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As spring boot 1.x is end of life Aug 1st 2019, it would be great to migrate > to 2.1.x. > Below is the announcement > [https://spring.io/blog/2018/07/30/spring-boot-1-x-eol-aug-1st-2019] > Migrate Guide is > [https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide] > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Apache Griffin compatible with Hive 2.3.3?
Hello,
Does anybody know if Apache Griffin is compatible with Hive 2.3.3?
We have an error in our environment when Griffin is getting databases from
Hive, our Hive version is Hive 2.3.3-mapr-1808 and Griffin is using hive
metastore 2.2.0
2019-08-16 05:25:46.910 ERROR 73551 --- [nio-8080-exec-1] h.log
: Converting exception to MetaException
2019-08-16 05:25:46.911 INFO 73551 --- [nio-8080-exec-1] h.metastore
: Closed a connection to metastore, current connections: 0
2019-08-16 05:25:46.911 INFO 73551 --- [nio-8080-exec-1] h.metastore
: Trying to connect to metastore with URI
thrift://inhas68626.eu.boehringer.com:9083
2019-08-16 05:25:46.914 INFO 73551 --- [nio-8080-exec-1] h.metastore
: Opened a connection to metastore, current connections: 1
2019-08-16 05:25:46.914 INFO 73551 --- [nio-8080-exec-1] h.metastore
: Connected to metastore.
2019-08-16 05:25:46.914 ERROR 73551 --- [nio-8080-exec-1]
o.a.g.c.m.h.HiveMetaStoreService : Can not get databases : {}
org.apache.hadoop.hive.metastore.api.MetaException: Got exception:
org.apache.thrift.transport.TTransportException null
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1342)
~[hive-metastore-2.2.0.jar!/:2.2.0]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1156)
~[hive-metastore-2.2.0.jar!/:2.2.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_191]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2295)
~[hive-metastore-2.2.0.jar!/:2.2.0]
at com.sun.proxy.$Proxy142.getAllDatabases(Unknown Source) ~[?:?]
at
org.apache.griffin.core.metastore.hive.HiveMetaStoreServiceImpl.getAllDatabases(HiveMetaStoreServiceImpl.java:69)
[classes!/:0.4.0]
at
org.apache.griffin.core.metastore.hive.HiveMetaStoreServiceImpl.getAllTable(HiveMetaStoreServiceImpl.java:125)
[classes!/:0.4.0]
at
org.apache.griffin.core.metastore.hive.HiveMetaStoreServiceImpl$$FastClassBySpringCGLIB$$d0fbb087.invoke()
[classes!/:0.4.0]
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
[spring-core-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
[spring-aop-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
[spring-aop-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:52)
[spring-context-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.cache.interceptor.CacheAspectSupport.invokeOperation(CacheAspectSupport.java:345)
[spring-context-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:408)
[spring-context-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:327)
[spring-context-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61)
[spring-context-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
[spring-aop-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
[spring-aop-4.3.6.RELEASE.jar!/:4.3.6.RELEASE]
at
org.apache.griffin.core.metastore.hive.HiveMetaStoreServiceImpl$$EnhancerBySpringCGLIB$$a8c6e34f.getAllTable()
[classes!/:0.4.0]
at
org.apache.griffin.core.metastore.hive.HiveMetaStoreController.getAllTables(HiveMetaStoreController.java:57)
[classes!/:0.4.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_191]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
[spring-web-4.3.6.RELEAS
