This worked! :)

Thanks a lot for your help Yusaku, Sid and Jeff!

Just to understand why this happened I have few questions, so we can address the issue if it is commonly occurring on machine failure thus reducing manual intervention. /(I will try to find answers for the below questions myself in the source, would be great if I can get help here). /

 * Why did Postgresql corrupted on machine crash and did not recover on
   restart ?
 * If we had configured MySQL instead of Postgresql, will the problem
   re-occur on similar situation ? (I know this may take a while to
   test and infer conclusion)
 * What is stored in key_value_store and which component uses it ?
 * Can we have this as a quick fix in UI when the Web UI pops up the
   error ? Thus allowing admin to perform step by step guidance to
   recover from the failure(manual or self healing process, need
   discussion on this).
 * What did the POST call to http://localhost:8080/api/v1/persist API do ?

--
Suraj Nayak

On Friday 11 July 2014 05:17 AM, Yusaku Sako wrote:
Please try the following (worked for me on my test cluster).

1. Delete all rows from key_value_store
1. ambari-server restart
2. Wait until the server comes up
3. curl -i -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d
'{"user-pref-admin-dashboard":"","CLUSTER_CURRENT_STATUS":"{\"clusterState\":\"CLUSTER_STARTED_5\"}","admin-settings-show-bg-admin":"true","showJobsForNonAdmin":"true"}'
http://localhost:8080/api/v1/persist
4. Log back in to ambari-server

Hope this helps.
Yusaku

On Thu, Jul 10, 2014 at 11:31 AM, Suraj Nayak M <[email protected]> wrote:
Thanks Sid,

Found the username, password and database name used by ambari :)

Yusaku,
     Is it safe to truncate the table key_value_store table ? The old cluster
state will be restored after following steps mentioned below (confirmation
needed!):

Truncate table key_value_store from postgresql.
Call the API as mentioned below in the email chain.

--
Suraj Nayak


On Thursday 10 July 2014 10:29 PM, Siddharth Wagle wrote:

Hi Suraj,

You should find all db info in /etc/ambari-server/conf/ambari.properties.
(grep for jdbc).

I believe you can just truncate the table key_value_store and then try
making the API call.
Since UI utilizes this table Yusaku would be the right person to +1 this
advice.

-Sid


On Thu, Jul 10, 2014 at 1:34 AM, Suraj Nayak M <[email protected]> wrote:
Yes Sid, Even I suspected the Postgres corruption.

I tried the API provided by Yasaku. I got 500 Internal error message
mentioned below :

HTTP/1.1 500  Internal Exception: org.postgresql.util.PSQLException:
ERROR: missing chunk number 0 for toast value 50294 in pg_toast_16548 Error
Code: 0 Call: SELECT "key", "value" FROM key_value_store WHERE ("key" = ?)
bind => [1 parameter bound] Query: ReadObjectQuery(name="readObject"
referenceClass=KeyValueEntity sql="SELECT "key", "value" FROM
key_value_store WHERE ("key" = ?)")
Set-Cookie: AMBARISESSIONID=39gi1jtp0amp1orp572oipb8k;Path=/
Content-Type: text/plain;charset=ISO-8859-1
Content-Length: 467
Server: Jetty(7.6.7.v20120910)

{
   "status": 500,
   "message": "\nInternal Exception: org.postgresql.util.PSQLException:
ERROR: missing chunk number 0 for toast value 50294 in pg_toast_16548\nError
Code: 0\nCall: SELECT \"key\", \"value\" FROM key_value_store WHERE (\"key\"
\u003d ?)\n\tbind \u003d\u003e [1 parameter bound]\nQuery:
ReadObjectQuery(name\u003d\"readObject\" referenceClass\u003dKeyValueEntity
sql\u003d\"SELECT \"key\", \"value\" FROM key_value_store WHERE (\"key\"
\u003d ?)\")"

I can notice a specific key, which is highlighter in red. Does deleting
this solve the problem ?

I am not able to take a dump of existing databases as I don't know the
username and password of postgre.

What is the default possource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Ambari Server 'start' completed successfully.
[root@vm1 ~]#  curl -i -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d
curl: option -d: requires parameter
curl: try 'curl --help' or 'curl --manual' for more information
[root@vm1 ~]# 
'{"user-pref-admin-dashboard":"","CLUSTER_CURRENT_STATUS":"{\"clusterState\":\"CLUSTER_STARTED_5\"}","admin-settings-show-bg-admin":"true","showJobsForNonAdmin":"true"}'
-bash: 
{"user-pref-admin-dashboard":"","CLUSTER_CURRENT_STATUS":"{\"clusterState\":\"CLUSTER_STARTED_5\"}","admin-settings-show-bg-admin":"true","showJobsForNonAdmin":"true"}:
 command not found
[root@vm1 ~]# 
'{"user-pref-admin-dashboard":"","CLUSTER_CURRENT_STATUS":"{\"clusterState\":\"CLUSTER_STARTED_5\"}","admin-settings-show-bg-admin":"true","showJotgres
 username, password and database name that
Ambari uses ?
I suspect the user is posgres, which is the default user name. If I hard
reset the postgres which is mentioned in the this link, will ambari face
problem ? Which is the configuration file Ambari uses to store postgres
username and password?

--
Suraj Nayak


On Thursday 10 July 2014 09:31 AM, Yusaku Sako wrote:

Suraj,

Here's the API call that Sid mentioned for restoring key_value_store data:

curl -i -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{
"CLUSTER_CURRENT_STATUS": "{\"clusterState\":\"CLUSTER_STARTED_5\"}"
}' http://localhost:8080/api/v1/persist

Hopefully this helps.

Yusaku

On Wed, Jul 9, 2014 at 8:49 PM, Siddharth Wagle <[email protected]>
wrote:

Hi Suraj,

Found a similar thread on postgres forum,

http://postgresql.1045698.n5.nabble.com/select-table-indicate-missing-chunk-number-0-for-toast-value-96635-in-pg-toast-2619-td5682176.html

The cleanup commands at the end of the topic seem worth a try.

Note: If key_value_store has corrupt data it can be deleted without
affection state of the system, there is a API call to do it, Yusaku can
probably help you get the API call for it.

1. Backup the database as it is.
2. I would try the API call and see if everything recovers.
3. Run the analyze command and re index on pg_statistic.

-Sid


On Wed, Jul 9, 2014 at 8:32 PM, Yusaku Sako <[email protected]>
wrote:

"Caused by: org.postgresql.util.PSQLException: ERROR: missing chunk
number 0 for toast value 50294 in pg_toast_16548"

I think this suggests that your Postgres DB is corrupt.

Yusaku


On Wed, Jul 9, 2014 at 10:58 AM, Suraj Nayak M <[email protected]> wrote:

Jeff,

I tried restarting both ambari-server and postgresql. Still the error
exist.
I have attached a part of the log file(log after ambari-server restart)
so
it might help to pin point the issue.

I could see following line before the Exception :

11:03:04,784 ERROR [qtp2062231465-43] HeartBeatHandler:127 -
CurrentResponseId unknown for vm1.somedomain.com - send register command
11:03:09,016 ERROR [qtp2062231465-43] HeartBeatHandler:127 -
CurrentResponseId unknown for vm3.somedomain.com - send register command
11:03:20,792 ERROR [qtp2062231465-44] HeartBeatHandler:127 -
CurrentResponseId unknown for vm2.somedomain.com - send register command

Below are the output of the commands(Restart and status) :

-------------------------------

Status :

[root@vm1 ~]# ambari-server status
Using python  /usr/bin/python2.6
Ambari-server status
Ambari Server running
Found Ambari Server PID: '4895 at:
/var/run/ambari-server/ambari-server.pid
[root@vm1 ~]# ps -ef | grep postgres
postgres  1947     1  0 04:16 ?        00:00:01 /usr/bin/postmaster -p
5432
-D /var/lib/pgsql/data
postgres  1949  1947  0 04:16 ?        00:00:00 postgres: logger process
postgres  1951  1947  0 04:16 ?        00:00:03 postgres: writer process
postgres  1952  1947  0 04:16 ?        00:00:02 postgres: wal writer
process
postgres  1953  1947  0 04:16 ?        00:00:00 postgres: autovacuum
launcher process
postgres  1954  1947  0 04:16 ?        00:00:01 postgres: stats
collector
process
postgres  4915  1947  1 04:43 ?        00:05:05 postgres: ambari ambari
127.0.0.1(57245) idle
postgres  5018  1947  0 04:44 ?        00:00:00 postgres: ambari ambari
127.0.0.1(57267) idle
postgres  5173  1947  0 04:45 ?        00:00:00 postgres: ambari ambari
127.0.0.1(57282) idle
postgres  5174  1947  0 04:45 ?        00:00:00 postgres: ambari ambari
127.0.0.1(57283) idle
postgres  5175  1947  0 04:45 ?        00:00:00 postgres: ambari ambari
127.0.0.1(57284) idle
root     30657 30493  0 10:59 pts/0    00:00:00 grep postgres
[root@vm1 ~]# service postgresql status
postmaster (pid  1947) is running...

Restart :
[root@vm1 ~]# service postgresql restart
Stopping postgresql service:                               [  OK  ]
Starting postgresql service:                               [  OK  ]
[root@vm1 ~]# ambari-server restart
Using python  /usr/bin/python2.6
Restarting ambari-server
Using python  /usr/bin/python2.6
Stopping ambari-server
Ambari Server stopped
Using python  /usr/bin/python2.6
Starting ambari-server
Ambari Server running with 'root' privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Ambari Server 'start' completed successfully.

Status :
[root@vm1 ~]# service postgresql status
postmaster (pid  31003) is running...

[root@vm1 ~]# ps -ef | grep postgres
postgres 31003     1  0 11:02 ?        00:00:00 /usr/bin/postmaster -p
5432
-D /var/lib/pgsql/data
postgres 31005 31003  0 11:02 ?        00:00:00 postgres: logger process
postgres 31007 31003  0 11:02 ?        00:00:00 postgres: writer process
postgres 31008 31003  0 11:02 ?        00:00:00 postgres: wal writer
process
postgres 31009 31003  0 11:02 ?        00:00:00 postgres: autovacuum
launcher process
postgres 31010 31003  0 11:02 ?        00:00:00 postgres: stats
collector
process
postgres 31079 31003  1 11:02 ?        00:00:03 postgres: ambari ambari
127.0.0.1(59377) idle
postgres 31350 31003  0 11:05 ?        00:00:00 postgres: ambari ambari
127.0.0.1(59417) idle
postgres 31351 31003  0 11:05 ?        00:00:00 postgres: ambari ambari
127.0.0.1(59418) idle
postgres 31352 31003  0 11:05 ?        00:00:00 postgres: ambari ambari
127.0.0.1(59419) idle
root     31501 30493  0 11:06 pts/0    00:00:00 grep postgres

[root@vm1 ~]# ambari-server status
Using python  /usr/bin/python2.6
Ambari-server status
Ambari Server running
Found Ambari Server PID: '31061 at:
/var/run/ambari-server/ambari-server.pid

----------------------------


--
Suraj Nayak


On Wednesday 09 July 2014 07:15 PM, Jeff Sposetti wrote:

Can you confirm ambari server is running? And stop/start as necessary?

ambari-server status
Also, confirm postgres is running after you stop/start ambari-server?

ps -ef | grep postgres

service postgresql status



On Wed, Jul 9, 2014 at 8:49 AM, Suraj Nayak <[email protected]> wrote:

Hi,

I have a 3 node cluster with HA enabled via Ambari. Accidentally the
power
supply to all the nodes was disrupted. After the power was restored,
the 3
servers were turned on. Now if I try to login to Ambari Web UI, an
error
pops up  Error in retrieving web client state from ambari server.(After
entering username and password)

It doesn't show the cluster dashboard which was showing before. Instead
it
shows "Cluster Install Wizard" to setup a new cluster. How can I
recover
from the error? I do not want to restart the installation process

Below is the error I found in /var/log/ambari-server/ambari-server.log
log
file.

I found an email archive at


http://mail-archives.apache.org/mod_mbox/ambari-user/201402.mbox/%3ccf1bc3ed.2e43f%[email protected]%3E.
But I need suggestions on recovering but not starting the installation
from
scratch.

Error :

Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services -
2.4.0.v20120608-r11652):
org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: missing
chunk number 0 for toast value 50294 in pg_toast_16548
Error Code: 0
Call: SELECT "key", "value" FROM key_value_store WHERE ("key" = ?)
     bind => [1 parameter bound]
Query: ReadObjectQuery(name="readObject" referenceClass=KeyValueEntity
sql="SELECT "key", "value" FROM key_value_store WHERE ("key" = ?)")
     at


org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
     at


org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:646)
     at


org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:537)
     at


org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1800)
     at


org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:566)
     at


org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:207)
     at


org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:193)
     at


org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:668)
     at


org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2744)
     at


org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2697)
     at


org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:452)
     at


org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1149)
     at


org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:852)
     at


org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1108)
     at


org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:420)
     at


org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1196)
     at


org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2875)
     at


org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1602)
     at


org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1584)
     at


org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1535)
     at


org.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:838)
     at


org.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:778)
     at


org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:671)
     at


org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:543)
     at


org.apache.ambari.server.orm.dao.KeyValueDAO.findByKey(KeyValueDAO.java:42)
     at


org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:53)
     at


org.apache.ambari.server.api.services.PersistKeyValueImpl.getValue(PersistKeyValueImpl.java:50)
     at


org.apache.ambari.server.api.services.PersistKeyValueService.getKey(PersistKeyValueService.java:83)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at


sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at


sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at


com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
     at


com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
     at


com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
     at


com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
     at


com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
     at


com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
     at


com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
     at


com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
     at


com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
     at


com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
     at


com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
     at


com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
     at


com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
     at


com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
     at


com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:708)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
     at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:652)
     at


org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1329)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
     at


org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
     at


org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.apache.ambari.server.security.authorization.internal.InternalTokenAuthenticationFilter.doFilter(InternalTokenAuthenticationFilter.java:53)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
     at


org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
     at


org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
     at


org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
     at


org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
     at


org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
     at


org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300)
     at


org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:48)
     at


org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300)
     at

org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:445)
     at


org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
     at


org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:559)
     at


org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
     at


org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1038)
     at

org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:374)
     at


org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189)
     at


org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:972)
     at


org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
     at

org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
     at


org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
     at org.eclipse.jetty.server.Server.handle(Server.java:363)
     at


org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
     at


org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
     at


org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
     at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
     at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
     at


org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
     at


org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627)
     at


org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51)
     at


org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
     at


org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
     at java.lang.Thread.run(Thread.java:745)
Caused by: org.postgresql.util.PSQLException: ERROR: missing chunk
number
0 for toast value 50294 in pg_toast_16548
     at


org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
     at


org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
     at


org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
     at


org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:559)
     at


org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
     at


org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:302)
     at


org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:938)
     at


org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:609)
     ... 98 more


--
Thanks
Suraj Nayak

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity
to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the
reader of
this message is not the intended recipient, you are hereby notified that
any
printing, copying, dissemination, distribution, disclosure or forwarding
of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete
it
from your system. Thank You.


--

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity
to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity
to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of
this message is not the intended recipient, you are hereby notified that
any
printing, copying, dissemination, distribution, disclosure or forwarding
of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete
it
from your system. Thank You.



CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.



Reply via email to