[jira] [Updated] (BEAM-8407) [SQL] Some Hive tests throw NullPointerException, but get marked as passing (Direct Runner)

2020-04-23 Thread Kenneth Knowles (Jira) 


 [ 
https://issues.apache.org/jira/browse/BEAM-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kenneth Knowles updated BEAM-8407:
--
Status: Open  (was: Triage Needed)

> [SQL] Some Hive tests throw NullPointerException, but get marked as passing 
> (Direct Runner)
> ---
>
> Key: BEAM-8407
> URL: https://issues.apache.org/jira/browse/BEAM-8407
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.15.0
>Reporter: Kirill Kozlov
>Priority: Major
>  Labels: test
>
> One of the tests with this issue:
> {code:java}
> ./gradlew -p sdks/java/extensions/sql/ test --tests 
> org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb
>  --info
> {code}
> Stack trace:
> {code:java}
> [direct-runner-worker] ERROR hive.log - Got exception: 
> java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - 
> Got exception: java.lang.NullPointerException 
> nulljava.lang.NullPointerException 
> at 
> org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77)
>  ~[hive-exec-2.1.0.jar:2.1.0] 
> at 
> org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54)
>  ~[hive-exec-2.1.0.jar:2.1.0] 
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126)
>  [hive-metastore-2.1.0.jar:2.1.0] 
> at 
> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369)
>  [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] 
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_222] 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_222] 
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] 
> at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150)
>  [hive-metastore-2.1.0.jar:2.1.0] 
> at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?] 
> at 
> org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) 
> [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at 
> org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
>  [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at 
> org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42)
>  [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
> at 
> org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323)
>  [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
> at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_222] 
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] 
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [?:1.8.0_222] 
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  [?:1.8.0_222] 
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
> ... more NullPointerExceptions{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (BEAM-8407) [SQL] Some Hive tests throw NullPointerException, but get marked as passing (Direct Runner)

2020-04-23 Thread Kenneth Knowles (Jira) 


 [ 
https://issues.apache.org/jira/browse/BEAM-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kenneth Knowles updated BEAM-8407:
--
Priority: Critical  (was: Major)

> [SQL] Some Hive tests throw NullPointerException, but get marked as passing 
> (Direct Runner)
> ---
>
> Key: BEAM-8407
> URL: https://issues.apache.org/jira/browse/BEAM-8407
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Affects Versions: 2.15.0
>Reporter: Kirill Kozlov
>Priority: Critical
>  Labels: test
>
> One of the tests with this issue:
> {code:java}
> ./gradlew -p sdks/java/extensions/sql/ test --tests 
> org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb
>  --info
> {code}
> Stack trace:
> {code:java}
> [direct-runner-worker] ERROR hive.log - Got exception: 
> java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - 
> Got exception: java.lang.NullPointerException 
> nulljava.lang.NullPointerException 
> at 
> org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77)
>  ~[hive-exec-2.1.0.jar:2.1.0] 
> at 
> org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54)
>  ~[hive-exec-2.1.0.jar:2.1.0] 
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126)
>  [hive-metastore-2.1.0.jar:2.1.0] 
> at 
> org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369)
>  [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] 
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_222] 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_222] 
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] 
> at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150)
>  [hive-metastore-2.1.0.jar:2.1.0] 
> at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?] 
> at 
> org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) 
> [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at 
> org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
>  [hive-hcatalog-core-2.1.0.jar:2.1.0] 
> at 
> org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42)
>  [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
> at 
> org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323)
>  [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
> at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at 
> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124)
>  [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_222] 
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] 
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [?:1.8.0_222] 
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  [?:1.8.0_222] 
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
> ... more NullPointerExceptions{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (BEAM-8407) [SQL] Some Hive tests throw NullPointerException, but get marked as passing (Direct Runner)

2019-10-15 Thread Kirill Kozlov (Jira) 


 [ 
https://issues.apache.org/jira/browse/BEAM-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Kozlov updated BEAM-8407:

Description: 
One of the tests with this issue:
{code:java}
./gradlew -p sdks/java/extensions/sql/ test --tests 
org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb
 --info
{code}
Stack trace:
{code:java}
[direct-runner-worker] ERROR hive.log - Got exception: 
java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got 
exception: java.lang.NullPointerException nulljava.lang.NullPointerException 
at 
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77)
 ~[hive-exec-2.1.0.jar:2.1.0] 
at 
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54)
 ~[hive-exec-2.1.0.jar:2.1.0] 
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126)
 [hive-metastore-2.1.0.jar:2.1.0] 
at 
org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369)
 [hive-hcatalog-core-2.1.0.jar:2.1.0] 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] 
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_222] 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_222] 
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] 
at 
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150)
 [hive-metastore-2.1.0.jar:2.1.0] 
at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?] 
at 
org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) 
[hive-hcatalog-core-2.1.0.jar:2.1.0] 
at 
org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
 [hive-hcatalog-core-2.1.0.jar:2.1.0] 
at 
org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42)
 [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
at 
org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323)
 [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
at 
org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172)
 [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at 
org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148)
 [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at 
org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160)
 [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at 
org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124)
 [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[?:1.8.0_222] 
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] 
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_222] 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_222] 
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

... more NullPointerExceptions{code}

  was:
One of the tests with this issue:
{code:java}
./gradlew -p sdks/java/extensions/sql/ test --tests 
org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb
 --info
{code}
Stack trace:
{code:java}
[direct-runner-worker] ERROR hive.log - Got exception: 
java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got 
exception: java.lang.NullPointerException nulljava.lang.NullPointerException at 
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77)
 ~[hive-exec-2.1.0.jar:2.1.0] at 
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54)
 ~[hive-exec-2.1.0.jar:2.1.0] at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126)
 [hive-metastore-2.1.0.jar:2.1.0] at 
org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369)
 [hive-hcatalog-core-2.1.0.jar:2.1.0] at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_222] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcc