Hi Pieter, I was able to reproduce this problem. It looks like it is an issue with doing a get inside of a transaction along with a replicated region using persistence and overflow. The value is still on disk, and for whatever reason if you do the get inside of a transaction it is returning you this bogus NOT_AVAILABLE token instead of reading the value off disk.
I'll create a JIRA and attach my test. In the meantime, you could do the get outside of a transaction, or you could change your region to not use overflow. If you try changing the region to not use overflow, I think you'll also have to set the system property gemfire.disk.recoverValuesSync to true to make sure that in all cases you never have to read from disk. Thanks, -Dan On Mon, Apr 30, 2018 at 3:47 AM, Pieter van Zyl <[email protected]> wrote: > Good day. > > I am constantly seeing this error below when we stop and start Geode > server after a data import. > > When the client connects the second time after the restart we get > NotSerializableException: > org.apache.geode.internal.cache.Token$NotAvailable > > Any ideas why we are getting this error or why it would state > "NotAvailable"? > > *Versions:* > > compile 'org.springframework.data:spring-data-geode:2.1.0.M2' > compile group: 'org.apache.geode', name: 'geode-core', version: '1.5.0' > > Trying to access this region on startup: > > > > > > *<gfe:replicated-region id="ClassID-ClassName-LookUp" > disk-store-ref="tauDiskStore" persistent="true"> > <gfe:eviction type="HEAP_PERCENTAGE" > action="OVERFLOW_TO_DISK"/></gfe:replicated-region>* > > *Server config:* > > > > > > > > > > > > *<util:properties id="gemfire-props"><prop > key="log-level">info</prop><prop key="locators">pvz-dell[10334]</prop><prop > key="start-locator">pvz-dell[10334]</prop><prop > key="mcast-port">0</prop><prop key="http-service-port">0</prop><prop > key="jmx-manager">true</prop><prop key="jmx-manager-port">1099</prop><prop > key="jmx-manager-start">true</prop></util:properties>* > > > > > > > > > > > *<gfe:cache properties-ref="gemfire-props" > pdx-serializer-ref="pdxSerializer" > pdx-persistent="true"pdx-disk-store="pdx-disk-store" /><gfe:cache-server > port="40404" max-connections="300" socket-buffer-size="65536" > max-threads="200"/><gfe:transaction-manager id="txManager"/><bean > id="pdxSerializer" > class="org.rdb.geode.mapping.RDBGeodeSerializer"> <constructor-arg > value="org.rdb.*,net.lautus.*"/></bean>* > > The server seems to be up and running > *Cache server connection listener bound to address > pvz-dell-/0:0:0:0:0:0:0:0:40404 with backlog 1,000.* > > *[info 2018/04/30 12:32:30.483 SAST <main> tid=0x1] > ClientHealthMonitorThread maximum allowed time between pings: 60,000* > > *[warn 2018/04/30 12:32:30.485 SAST <main> tid=0x1] Handshaker max Pool > size: 4* > > *[info 2018/04/30 12:32:30.486 SAST <Cache Server Selector > /0:0:0:0:0:0:0:0:40404 local port: 40404> tid=0x4f] SELECTOR enabled* > > *[info 2018/04/30 12:32:30.491 SAST <main> tid=0x1] CacheServer > Configuration: port=40404 max-connections=300 max-threads=200 > notify-by-subscription=true socket-buffer-size=65536 > maximum-time-between-pings=60000 maximum-message-count=230000 > message-time-to-live=180 eviction-policy=none capacity=1 overflow > directory=. groups=[] loadProbe=ConnectionCountProbe loadPollInterval=5000 > tcpNoDelay=true* > > *server running on port 40404* > *Press <Enter> to terminate the server* > > > Exception in thread "main" > org.apache.geode.cache.client.ServerOperationException: > remote server on pvz-dell(23128:loner):38042:2edf1c16: > org.apache.geode.SerializationException: > failed serializing object > at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException( > OpExecutorImpl.java:669) > at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException( > OpExecutorImpl.java:742) > at org.apache.geode.cache.client.internal.OpExecutorImpl.handleException( > OpExecutorImpl.java:611) > at org.apache.geode.cache.client.internal.OpExecutorImpl.executeOnServer( > OpExecutorImpl.java:373) > at org.apache.geode.cache.client.internal.OpExecutorImpl. > executeWithServerAffinity(OpExecutorImpl.java:220) > at org.apache.geode.cache.client.internal.OpExecutorImpl. > execute(OpExecutorImpl.java:129) > at org.apache.geode.cache.client.internal.OpExecutorImpl. > execute(OpExecutorImpl.java:116) > at org.apache.geode.cache.client.internal.PoolImpl.execute( > PoolImpl.java:774) > at org.apache.geode.cache.client.internal.GetOp.execute(GetOp.java:91) > at org.apache.geode.cache.client.internal.ServerRegionProxy. > get(ServerRegionProxy.java:113) > at org.apache.geode.internal.cache.tx.ClientTXRegionStub. > findObject(ClientTXRegionStub.java:72) > at org.apache.geode.internal.cache.TXStateStub.findObject( > TXStateStub.java:453) > at org.apache.geode.internal.cache.TXStateProxyImpl. > findObject(TXStateProxyImpl.java:496) > at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1366) > at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1300) > at org.apache.geode.internal.cache.LocalRegion.get(LocalRegion.java:1285) > at org.apache.geode.internal.cache.AbstractRegion.get( > AbstractRegion.java:320) > ...... > Caused by: org.apache.geode.SerializationException: failed serializing > object > at org.apache.geode.internal.cache.tier.sockets.Message. > serializeAndAddPart(Message.java:399) > at org.apache.geode.internal.cache.tier.sockets.Message. > addPartInAnyForm(Message.java:360) > at org.apache.geode.internal.cache.tier.sockets.command. > Get70.writeResponse(Get70.java:424) > at org.apache.geode.internal.cache.tier.sockets.command. > Get70.cmdExecute(Get70.java:211) > at org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute( > BaseCommand.java:157) > at org.apache.geode.internal.cache.tier.sockets. > ServerConnection.doNormalMsg(ServerConnection.java:797) > at org.apache.geode.internal.cache.tier.sockets.LegacyServerConnection. > doOneMessage(LegacyServerConnection.java:85) > at org.apache.geode.internal.cache.tier.sockets.ServerConnection.run( > ServerConnection.java:1148) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > at org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$4$1.run( > AcceptorImpl.java:641) > at java.lang.Thread.run(Thread.java:748) > *Caused by: java.io <http://java.io>.NotSerializableException: > org.apache.geode.internal.cache.Token$NotAvailable* > at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) > at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) > at org.apache.geode.internal.InternalDataSerializer. > writeSerializableObject(InternalDataSerializer.java:2341) > at org.apache.geode.internal.InternalDataSerializer.basicWriteObject( > InternalDataSerializer.java:2216) > at org.apache.geode.DataSerializer.writeObject(DataSerializer.java:2936) > at org.apache.geode.internal.util.BlobHelper.serializeTo( > BlobHelper.java:66) > at org.apache.geode.internal.cache.tier.sockets.Message. > serializeAndAddPart(Message.java:397) > > > Kindly > Pieter >
