Re: [Gluster-users] java application crushes while reading a zip file

2019-01-28 Thread Dmitry Isakbayev
for you to automate the tests, it would be great > to have it in our regression, so we can always be sure your setup would > never break in future releases. > > Regards, > Amar > > On Mon, Jan 7, 2019 at 11:42 PM Dmitry Isakbayev > wrote: > >> This system is going into p

Re: [Gluster-users] [External] Re: A broken file that can not be deleted

2019-01-10 Thread Dmitry Isakbayev
having the software rebuild them. > *Wondering whether its a case of split brain.*Very possible. All 3 servers were rebooted. It brought down linux cluster running on the same 3 servers as well. > On Thu, Jan 10, 2019 at 10:00 AM Raghavendra Gowdappa > wrote: > >> &g

[Gluster-users] A broken file that can not be deleted

2019-01-09 Thread Dmitry Isakbayev
I am seeing a broken file that exists on 2 out of 3 nodes. The application trying to use the file throws file permissions error. ls, rm, mv, touch all throw "Input/output error" $ ls -la ls: cannot access .download_suspensions.memo: Input/output error drwxrwxr-x. 2 ossadmin ossadmin 4096 Jan

Re: [Gluster-users] java application crushes while reading a zip file

2019-01-07 Thread Dmitry Isakbayev
This system is going into production. I will try to replicate this problem on the next installation. On Wed, Jan 2, 2019 at 9:25 PM Raghavendra Gowdappa wrote: > > > On Wed, Jan 2, 2019 at 9:59 PM Dmitry Isakbayev wrote: > >> Still no JVM crushes. Is it possible that

Re: [Gluster-users] java application crushes while reading a zip file

2019-01-02 Thread Dmitry Isakbayev
Still no JVM crushes. Is it possible that running glusterfs with performance options turned off for a couple of days cleared out the "stale metadata issue"? On Mon, Dec 31, 2018 at 1:38 PM Dmitry Isakbayev wrote: > The software ran with all of the options turned off over the week

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-31 Thread Dmitry Isakbayev
try. Can you provide the following debug info I asked earlier: > > * strace -ff -v ... of java application > * dump of the I/O traffic seen by the mountpoint (use --dump-fuse while > mounting). > > regards, > Raghavendra > > On Sat, Dec 29, 2018 at 2:08 AM Dmitry Isakbayev

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-28 Thread Dmitry Isakbayev
: on* *performance.read-ahead: on* performance.client-io-threads: off nfs.disable: on transport.address-family: inet On Fri, Dec 28, 2018 at 10:24 AM Dmitry Isakbayev wrote: > Turning a single option on at a time still worked fine. I will keep > trying. > > We had used 4.1.5 on KVM/CentOS7.5 at

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-28 Thread Dmitry Isakbayev
) CPE OS Name: cpe:/o:centos:centos:7 Kernel: Linux 3.10.0-862.3.2.el7.x86_64 Architecture: x86-64 On Fri, Dec 28, 2018 at 8:56 AM Raghavendra Gowdappa wrote: > > > On Fri, Dec 28, 2018 at 7:23 PM Dmitry Isakbayev > wrote: > >> Ok. I will

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-28 Thread Dmitry Isakbayev
Ok. I will try different options. This system is scheduled to go into production soon. What version would you recommend to roll back to? On Thu, Dec 27, 2018 at 10:55 PM Raghavendra Gowdappa wrote: > > > On Fri, Dec 28, 2018 at 3:13 AM Dmitry Isakbayev > wrote: > >> Rag

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-27 Thread Dmitry Isakbayev
[No such device or address] [2018-12-28 01:01:20.218775] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 12-epoll: Failed to dispatch handler Regards, Dmitry On Thu, Dec 27, 2018 at 4:43 PM Dmitry Isakbayev wrote: > Raghavendra, > > Thank for the suggestion. > &g

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-27 Thread Dmitry Isakbayev
rfs are you using? It might be either > * a stale metadata issue. > * inconsistent ctime issue. > > Can you try turning off all performance xlators? If the issue is 1, that > should help. > > On Fri, Dec 28, 2018 at 1:51 AM Dmitry Isakbayev > wrote: > >> Attempte

Re: [Gluster-users] java application crushes while reading a zip file

2018-12-27 Thread Dmitry Isakbayev
Attempted to set 'performance.read-ahead off` according to https://jira.apache.org/jira/browse/AMQ-7041 That did not help. On Mon, Dec 24, 2018 at 2:11 PM Dmitry Isakbayev wrote: > The core file generated by JVM suggests that it happens because the file > is changing while it is bein

[Gluster-users] java application crushes while reading a zip file

2018-12-24 Thread Dmitry Isakbayev
The core file generated by JVM suggests that it happens because the file is changing while it is being read - https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8186557. The application reads in the zipfile and goes through the zip entries, then reloads the file and goes the zip entries again.