Re: Review Request 72882: SENTRY-2558: Issue in creating full snapshot when the storage descriptor for a table is null.
> On Sept. 17, 2020, 3:44 p.m., Na Li wrote: > > Ship It! Thanks for the review. Will merge the change after the unit tests are green. - kalyan kumar --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72882/#review221882 --- On Sept. 17, 2020, 3:40 p.m., kalyan kumar kalvagadda wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72882/ > --- > > (Updated Sept. 17, 2020, 3:40 p.m.) > > > Review request for sentry and Na Li. > > > Bugs: SENTRY-2558 > https://issues.apache.org/jira/browse/SENTRY-2558 > > > Repository: sentry > > > Description > --- > > When the storage descriptor for the table is null, sentry throws NullPointer > Exception and fails to create a snapshot. > > 020-09-17 15:08:43,647 ERROR > org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute > task java.lang.NullPointerException at > org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 > INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db = test_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInit ializer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) > Instead, it should ignore the object and move-on. > > > Diffs > - > > > sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java > 668a4ca05 > > > Diff: https://reviews.apache.org/r/72882/diff/1/ > > > Testing > --- > > I instrumented the failure by manually updating HMS database made sure that > logs look similar to one in the description. I verified ths issue is not > observed with the fix made. > > > Thanks, > > kalyan kumar kalvagadda > >
Re: Review Request 72882: SENTRY-2558: Issue in creating full snapshot when the storage descriptor for a table is null.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72882/#review221882 --- Ship it! Ship It! - Na Li On Sept. 17, 2020, 3:40 p.m., kalyan kumar kalvagadda wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72882/ > --- > > (Updated Sept. 17, 2020, 3:40 p.m.) > > > Review request for sentry and Na Li. > > > Bugs: SENTRY-2558 > https://issues.apache.org/jira/browse/SENTRY-2558 > > > Repository: sentry > > > Description > --- > > When the storage descriptor for the table is null, sentry throws NullPointer > Exception and fails to create a snapshot. > > 020-09-17 15:08:43,647 ERROR > org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute > task java.lang.NullPointerException at > org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) > at > org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 > INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db = test_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInit ializer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) > Instead, it should ignore the object and move-on. > > > Diffs > - > > > sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java > 668a4ca05 > > > Diff: https://reviews.apache.org/r/72882/diff/1/ > > > Testing > --- > > I instrumented the failure by manually updating HMS database made sure that > logs look similar to one in the description. I verified ths issue is not > observed with the fix made. > > > Thanks, > > kalyan kumar kalvagadda > >
Review Request 72882: SENTRY-2558: Issue in creating full snapshot when the storage descriptor for a table is null.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72882/ --- Review request for sentry and Na Li. Bugs: SENTRY-2558 https://issues.apache.org/jira/browse/SENTRY-2558 Repository: sentry Description --- When the storage descriptor for the table is null, sentry throws NullPointer Exception and fails to create a snapshot. 020-09-17 15:08:43,647 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.lang.NullPointerException at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db = t est_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitia lizer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Instead, it should ignore the object and move-on. Diffs - sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java 668a4ca05 Diff: https://reviews.apache.org/r/72882/diff/1/ Testing --- I instrumented the failure by manually updating HMS database made sure that logs look similar to one in the description. I verified ths issue is not observed with the fix made. Thanks, kalyan kumar kalvagadda
