Taewoo, The reason you found the exception in only one is that that one was the metadata node as can be seen in the stack trace.
Cheers, Abdullah. > On Feb 19, 2017, at 9:52 PM, Taewoo Kim <[email protected]> wrote: > > @Till: Thanks for the answer. I usually saw this error when I change one > binary to a newer binary and there was a meta-data change in-between. But, > the strange part is that I was using the same binary for 10 days now. I > just stopped the instance and restarted instance and saw that error on one > NC node. The symptom is that the system doesn't accept any query nor > generates any errors. That's why I found this exception on one NC. CC and > all the other NCs (8 nodes) were fine. > > Best, > Taewoo > > On Sun, Feb 19, 2017 at 8:47 PM, Till Westmann <[email protected]> wrote: > >> NoSuchMethodError certainly sounds like something is not right with >> the classes on the classpath. Since we’re not playing too much with >> the classpath at runtime, it seems that a change of binaries or an >> error reading the jars are not unlikely. But maybe the same message was >> there before as well. >> Does this error affect the system behavior? >> >> Cheers, >> Till >> >> On 19 Feb 2017, at 15:05, Taewoo Kim wrote: >> >>> I see. I checked the disk related errors and found nothing special. >>> >>> Best, >>> Taewoo >>> >>> On Sun, Feb 19, 2017 at 3:01 PM, Mike Carey <[email protected]> wrote: >>> >>>> Got it. (So not attributable to the all-too-frequent Asterix cluster >> I/O >>>> misbehavior...) >>>> >>>> >>>> >>>> On 2/19/17 2:55 PM, Taewoo Kim wrote: >>>> >>>>> It happened on the Sensorium cluster. >>>>> >>>>> Best, >>>>> Taewoo >>>>> >>>>> On Sun, Feb 19, 2017 at 2:54 PM, Mike Carey <[email protected]> wrote: >>>>> >>>>> Which cluster hardware? >>>>>> >>>>>> >>>>>> >>>>>> On 2/19/17 2:42 PM, Taewoo Kim wrote: >>>>>> >>>>>> Nope. I was keeping the same binary for 10 days now. >>>>>>> >>>>>>> Best, >>>>>>> Taewoo >>>>>>> >>>>>>> On Sun, Feb 19, 2017 at 2:20 PM, abdullah alamoudi < >> [email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Taewoo, >>>>>>> >>>>>>>> Did you change the binary in between? >>>>>>>> >>>>>>>> >>>>>>>> On Feb 19, 2017, at 2:18 PM, Taewoo Kim <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> I have a CC and nine NCs. After I stopped the cluster and restarted >>>>>>>>> the >>>>>>>>> instance, I saw the following exception on one NC. Has anybody seen >>>>>>>>> this >>>>>>>>> error? >>>>>>>>> >>>>>>>>> Feb 19, 2017 2:05:21 PM org.apache.hyracks.control.nc.NCDriver >> main >>>>>>>>> SEVERE: Setting uncaught exception handler >>>>>>>>> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@ >> 51565ec2 >>>>>>>>> Feb 19, 2017 2:05:21 PM org.apache.hyracks.control.nc. >>>>>>>>> >>>>>>>>> NodeControllerService >>>>>>>> >>>>>>>> start >>>>>>>>> INFO: Starting NodeControllerService >>>>>>>>> Feb 19, 2017 2:05:21 PM >>>>>>>>> org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start >>>>>>>>> INFO: Starting Asterix node controller: ss1120_nc4 >>>>>>>>> Feb 19, 2017 2:05:24 PM >>>>>>>>> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager >>>>>>>>> >>>>>>>>> uncaughtException >>>>>>>> >>>>>>>> SEVERE: Uncaught Exception from thread main message: >>>>>>>>> org.apache.hyracks.storage.am.common.api.IPageManager. >>>>>>>>> >>>>>>>>> createMetadataFrame()Lorg/apache/hyracks/storage/am/common/api/ >>>>>>>> ITreeIndexMetaDataFrame; >>>>>>>> >>>>>>>> java.lang.NoSuchMethodError: >>>>>>>>> org.apache.hyracks.storage.am.common.api.IPageManager. >>>>>>>>> >>>>>>>>> createMetadataFrame()Lorg/apache/hyracks/storage/am/common/api/ >>>>>>>> ITreeIndexMetaDataFrame; >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.btree.impls.BTreeOpContext.< >>>>>>>>> >>>>>>>>> init>(BTreeOpContext.java:104) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.btree.impls.BTree. >>>>>>>>> >>>>>>>>> createOpContext(BTree.java:835) >>>>>>>> >>>>>>>> at org.apache.hyracks.storage.am.btree.impls.BTree.access$100( >>>>>>>>> >>>>>>>>> BTree.java:74) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.btree.impls.BTree$ >>>>>>>>> >>>>>>>>> BTreeAccessor.<init>(BTree.java:915) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.btree.impls.BTree. >>>>>>>>> >>>>>>>>> createAccessor(BTree.java:896) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeOpCont >>>>>>>>> ext.<init>( >>>>>>>>> >>>>>>>>> LSMBTreeOpContext.java:93) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree. >>>>>>>>> >>>>>>>>> createOpContext(LSMBTree.java:770) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree. >>>>>>>>> >>>>>>>>> createAccessor(LSMBTree.java:777) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree. >>>>>>>>> >>>>>>>>> createAccessor(LSMBTree.java:86) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.asterix.metadata.MetadataNode. >> initializeDatasetIdFactory( >>>>>>>>> >>>>>>>>> MetadataNode.java:1193) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.asterix.metadata.MetadataManager.initializeDatase >>>>>>>>> tIdFactory( >>>>>>>>> >>>>>>>>> MetadataManager.java:634) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.asterix.metadata.bootstrap.MetadataBootstrap.star >>>>>>>>> tUniverse( >>>>>>>>> >>>>>>>>> MetadataBootstrap.java:183) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.asterix.app.nc.NCAppRuntimeContext.initializeMetadata( >>>>>>>>> >>>>>>>>> NCAppRuntimeContext.java:449) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint. >>>>>>>>> >>>>>>>>> notifyStartupComplete(NCApplicationEntryPoint.java:227) >>>>>>>> >>>>>>>> at >>>>>>>>> org.apache.hyracks.control.nc.NodeControllerService.start( >>>>>>>>> >>>>>>>>> NodeControllerService.java:310) >>>>>>>> >>>>>>>> at org.apache.hyracks.control.nc.NCDriver.main(NCDriver.java:47) >>>>>>>>> Feb 19, 2017 2:05:24 PM >>>>>>>>> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager stopAll >>>>>>>>> SEVERE: Lifecycle management not >>>>>>>>> configuredorg.apache.hyracks.api.lifecycle. >> LifeCycleComponentManager@ >>>>>>>>> >>>>>>>>> 51565ec2 >>>>>>>> >>>>>>>> Best, >>>>>>>>> Taewoo >>>>>>>>> >>>>>>>>> >>>>>>>> >>>> >>
