The snapinstaller script opens a new searcher by calling commit. From the
attached debug output it looks like that actually worked:
+ /opt/solr/bin/commit
+ [[ 0 != 0 ]]
+ logExit ended 0
Try running the /opt/solr/bin/commit directly with the -V option.
Bill
On 9/5/07, Matthew Runo <[EMAIL PROTECTED]> wrote:
>
> If it helps anyone, this index is around a gig in size.
>
> +--------------------------------------------------------+
> | Matthew Runo
> | Zappos Development
> | [EMAIL PROTECTED]
> | 702-943-7833
> +--------------------------------------------------------+
>
>
> On Sep 5, 2007, at 3:14 PM, Matthew Runo wrote:
>
> > It seems that the scripts cannot open new searchers at the end of
> > the process, for some reason. Here's a message from cron, but I'm
> > not sure what to make of it... It looks like the files properly
> > copied over, but failed the install. I removed the temp* directory,
> > but still SOLR could not launch a new searcher. I don't see any
> > activity in catalina.out though...
> >
> >
> > started by tomcat5
> > command: /opt/solr/bin/snappuller -M search1 -P 18080 -D /opt/solr/
> > data -S /opt/solr/logs -d /opt/solr/data -v
> > pulling snapshot temp-snapshot.20070905150504
> > receiving file list ... done
> > deleting segments_1ine
> > deleting _164h_1.del
> > deleting _164h.tis
> > deleting _164h.tii
> > deleting _164h.prx
> > deleting _164h.nrm
> > deleting _164h.frq
> > deleting _164h.fnm
> > deleting _164h.fdx
> > deleting _164h.fdt
> > deleting _164g_1.del
> > deleting _164g.tis
> > deleting _164g.tii
> > deleting _164g.prx
> > deleting _164g.nrm
> > deleting _164g.frq
> > deleting _164g.fnm
> > deleting _164g.fdx
> > deleting _164g.fdt
> > deleting _164f_1.del
> > deleting _164f.tis
> > deleting _164f.tii
> > deleting _164f.prx
> > deleting _164f.nrm
> > deleting _164f.frq
> > deleting _164f.fnm
> > deleting _164f.fdx
> > deleting _164f.fdt
> > deleting _164e_1.del
> > deleting _164e.tis
> > deleting _164e.tii
> > deleting _164e.prx
> > deleting _164e.nrm
> > deleting _164e.frq
> > deleting _164e.fnm
> > deleting _164e.fdx
> > deleting _164e.fdt
> > deleting _164d_1.del
> > deleting _164d.tis
> > deleting _164d.tii
> > deleting _164d.prx
> > deleting _164d.nrm
> > deleting _164d.frq
> > deleting _164d.fnm
> > deleting _164d.fdx
> > deleting _164d.fdt
> > deleting _164c_1.del
> > deleting _164c.tis
> > deleting _164c.tii
> > deleting _164c.prx
> > deleting _164c.nrm
> > deleting _164c.frq
> > deleting _164c.fnm
> > deleting _164c.fdx
> > deleting _164c.fdt
> > deleting _164b_1.del
> > deleting _164b.tis
> > deleting _164b.tii
> > deleting _164b.prx
> > deleting _164b.nrm
> > deleting _164b.frq
> > deleting _164b.fnm
> > deleting _164b.fdx
> > deleting _164b.fdt
> > deleting _164a_1.del
> > deleting _164a.tis
> > deleting _164a.tii
> > deleting _164a.prx
> > deleting _164a.nrm
> > deleting _164a.frq
> > deleting _164a.fnm
> > deleting _164a.fdx
> > deleting _164a.fdt
> > deleting _163z_3.del
> > deleting _163z.tis
> > deleting _163z.tii
> > deleting _163z.prx
> > deleting _163z.nrm
> > deleting _163z.frq
> > deleting _163z.fnm
> > deleting _163z.fdx
> > deleting _163z.fdt
> > deleting _163o_3.del
> > deleting _163o.tis
> > deleting _163o.tii
> > deleting _163o.prx
> > deleting _163o.nrm
> > deleting _163o.frq
> > deleting _163o.fnm
> > deleting _163o.fdx
> > deleting _163o.fdt
> > deleting _163d_4.del
> > deleting _163d.tis
> > deleting _163d.tii
> > deleting _163d.prx
> > deleting _163d.nrm
> > deleting _163d.frq
> > deleting _163d.fnm
> > deleting _163d.fdx
> > deleting _163d.fdt
> > deleting _1632_6.del
> > deleting _1632.tis
> > deleting _1632.tii
> > deleting _1632.prx
> > deleting _1632.nrm
> > deleting _1632.frq
> > deleting _1632.fnm
> > deleting _1632.fdx
> > deleting _1632.fdt
> > deleting _162r_7.del
> > deleting _162r.tis
> > deleting _162r.tii
> > deleting _162r.prx
> > deleting _162r.nrm
> > deleting _162r.frq
> > deleting _162r.fnm
> > deleting _162r.fdx
> > deleting _162r.fdt
> > deleting _162g_d.del
> > deleting _162g.tis
> > deleting _162g.tii
> > deleting _162g.prx
> > deleting _162g.nrm
> > deleting _162g.frq
> > deleting _162g.fnm
> > deleting _162g.fdx
> > deleting _162g.fdt
> > deleting _1625_m.del
> > deleting _1625.tis
> > deleting _1625.tii
> > deleting _1625.prx
> > deleting _1625.nrm
> > deleting _1625.frq
> > deleting _1625.fnm
> > deleting _1625.fdx
> > deleting _1625.fdt
> > deleting _161u_w.del
> > deleting _161u.tis
> > deleting _161u.tii
> > deleting _161u.prx
> > deleting _161u.nrm
> > deleting _161u.frq
> > deleting _161u.fnm
> > deleting _161u.fdx
> > deleting _161u.fdt
> > deleting _161j_16.del
> > ./
> > _161j_17.del
> > _164m.fdt
> > _164m.fdx
> > _164m.fnm
> > _164m.frq
> > _164m.nrm
> > _164m.prx
> > _164m.tii
> > _164m.tis
> > _164m_1.del
> > _164x.fdt
> > _164x.fdx
> > _164x.fnm
> > _164x.frq
> > _164x.nrm
> > _164x.prx
> > _164x.tii
> > _164x.tis
> > _164x_1.del
> > segments.gen
> > segments_1inv
> >
> > sent 516 bytes received 105864302 bytes 30247090.86 bytes/sec
> > total size is 966107226 speedup is 9.13
> > + [[ -z search1 ]]
> > + [[ -z /opt/solr/logs ]]
> > + fixUser -M search1 -S /opt/solr/logs -d /opt/solr/data -V
> > + [[ -z tomcat5 ]]
> > ++ whoami
> > + [[ tomcat5 != tomcat5 ]]
> > ++ who -m
> > ++ cut '-d ' -f1
> > ++ sed '-es/^.*!//'
> > + oldwhoami=
> > + [[ '' == '' ]]
> > +++ pgrep -g0 snapinstaller
> > ++ tail -1
> > ++ cut -f1 '-d '
> > ++ ps h -Hfp 3621 3629 3630 3631
> > + oldwhoami=tomcat5
> > + [[ -z /opt/solr/data ]]
> > ++ echo /opt/solr/data
> > ++ cut -c1
> > + [[ / != \/ ]]
> > ++ echo /opt/solr/logs
> > ++ cut -c1
> > + [[ / != \/ ]]
> > ++ date +%s
> > + start=1189030205
> > + logMessage started by tomcat5
> > ++ timeStamp
> > ++ date '+%Y/%m/%d %H:%M:%S'
> > + echo 2007/09/05 15:10:05 started by tomcat5
> > + [[ -n '' ]]
> > + logMessage command: /opt/solr/bin/snapinstaller -M search1 -S /
> > opt/solr/logs -d /opt/solr/data -V
> > ++ timeStamp
> > ++ date '+%Y/%m/%d %H:%M:%S'
> > + echo 2007/09/05 15:10:05 command: /opt/solr/bin/snapinstaller -M
> > search1 -S /opt/solr/logs -d /opt/solr/data -V
> > + [[ -n '' ]]
> > ++ ls /opt/solr/data
> > ++ grep 'snapshot\.'
> > ++ grep -v wip
> > ++ sort -r
> > ++ head -1
> > + name=temp-snapshot.20070905150504
> > + trap 'echo "caught INT/TERM, exiting now but partial installation
> > may have already occured";/bin/rm -rf ${data_dir"/index.tmp$
> > $;logExit aborted 13' INT TERM
> > + [[ temp-snapshot.20070905150504 == '' ]]
> > + name=/opt/solr/data/temp-snapshot.20070905150504
> > ++ cat /opt/solr/logs/snapshot.current
> > + [[ /opt/solr/data/temp-snapshot.20070905150504 == /opt/solr/data/
> > snapshot.20070905150005 ]]
> > + ssh -o StrictHostKeyChecking=no search1 mkdir -p /opt/solr/logs
> > + logMessage installing snapshot /opt/solr/data/temp-snapshot.
> > 20070905150504
> > ++ timeStamp
> > ++ date '+%Y/%m/%d %H:%M:%S'
> > + echo 2007/09/05 15:10:05 installing snapshot /opt/solr/data/temp-
> > snapshot.20070905150504
> > + [[ -n '' ]]
> > + cp -lr /opt/solr/data/temp-snapshot.20070905150504/ /opt/solr/
> > data/index.tmp3621
> > + /bin/rm -rf /opt/solr/data/index
> > + mv -f /opt/solr/data/index.tmp3621 /opt/solr/data/index
> > + echo /opt/solr/data/temp-snapshot.20070905150504
> > ++ uname -n
> > + scp -q -o StrictHostKeyChecking=no /opt/solr/logs/
> > snapshot.current search1:/opt/solr/logs/snapshot.current.search2
> > + logMessage notifing Solr to open a new Searcher
> > ++ timeStamp
> > ++ date '+%Y/%m/%d %H:%M:%S'
> > + echo 2007/09/05 15:10:06 notifing Solr to open a new Searcher
> > + [[ -n '' ]]
> > + /opt/solr/bin/commit
> > + [[ 0 != 0 ]]
> > + logExit ended 0
> > ++ date +%s
> > + end=1189030210
> > ++ expr 1189030210 - 1189030205
> > + diff=5
> > ++ timeStamp
> > ++ date '+%Y/%m/%d %H:%M:%S'
> > + echo '2007/09/05 15:10:10 ended (elapsed time: 5 sec)'
> > + exit 0
> >
> >
> >
> > +--------------------------------------------------------+
> > | Matthew Runo
> > | Zappos Development
> > | [EMAIL PROTECTED]
> > | 702-943-7833
> > +--------------------------------------------------------+
> >
> >
> > On Aug 31, 2007, at 6:11 PM, Bill Au wrote:
> >
> >>> latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already
> >>> installed
> >>
> >> It looks like you have a directory named temp-snapshot.20070816120113
> >> in your data directory. You should remove it. One of the other
> >> script might have left that behind somehow.
> >>
> >> I will update the snapinstaller script to ignore non-snapshot when
> >> looking for the latest snapshot to install.
> >>
> >> Bill
> >>
> >> On 8/31/07, Matthew Runo <[EMAIL PROTECTED]> wrote:
> >>> Hello!
> >>>
> >>> On a somewhat related note, our replication seems very much broken.
> >>> I've added -v to all my cron jobs, and I think I've seen the error
> >>> (below).
> >>>
> >>> As you can see, it's rsyncing an updated index, but then doesn't
> >>> seem
> >>> to know to install it. I'm not sure why though.. no errors are
> >>> reported anywhere via the -v. Any help would be most appreciated,
> >>> I'm
> >>> sure I'm just missing something. You can see the cronjob
> >>> command in
> >>> the subject of the forwarded message.
> >>>
> >>> +--------------------------------------------------------+
> >>> | Matthew Runo
> >>> | Zappos Development
> >>> | [EMAIL PROTECTED]
> >>> | 702-943-7833
> >>> +--------------------------------------------------------+
> >>>
> >>>
> >>> Begin forwarded message:
> >>>
> >>>> From: [EMAIL PROTECTED] (Cron Daemon)
> >>>> Date: August 31, 2007 1:02:36 PM PDT
> >>>> To: [EMAIL PROTECTED]
> >>>> Subject: Cron <[EMAIL PROTECTED]> /opt/solr/bin/snappuller -M search1
> >>>> -P 18080 -D /opt/solr/data -S /opt/solr/logs -d /opt/solr/data -v;/
> >>>> opt/solr/bin/snapinstaller -M search1 -S /opt/solr/logs -d /opt/
> >>>> solr/data -v
> >>>>
> >>>> started by tomcat5
> >>>> command: /opt/solr/bin/snappuller -M search1 -P 18080 -D /opt/solr/
> >>>> data -S /opt/solr/logs -d /opt/solr/data -v
> >>>> pulling snapshot snapshot.20070831130005
> >>>> receiving file list ... done
> >>>> deleting segments_19zm
> >>>> deleting _10bg.tis
> >>>> deleting _10bg.tii
> >>>> deleting _10bg.prx
> >>>> deleting _10bg.nrm
> >>>> deleting _10bg.frq
> >>>> deleting _10bg.fnm
> >>>> deleting _10bg.fdx
> >>>> deleting _10bg.fdt
> >>>> ./
> >>>> _14ff.fdt
> >>>> _14ff.fdx
> >>>> _14ff.fnm
> >>>> _14ff.frq
> >>>> _14ff.nrm
> >>>> _14ff.prx
> >>>> _14ff.tii
> >>>> _14ff.tis
> >>>> _14ff_9.del
> >>>> _14fq.fdt
> >>>> _14fq.fdx
> >>>> _14fq.fnm
> >>>> _14fq.frq
> >>>> _14fq.nrm
> >>>> _14fq.prx
> >>>> _14fq.tii
> >>>> _14fq.tis
> >>>> _14fq_3.del
> >>>> _14fr.fdt
> >>>> _14fr.fdx
> >>>> _14fr.fnm
> >>>> _14fr.frq
> >>>> _14fr.nrm
> >>>> _14fr.prx
> >>>> _14fr.tii
> >>>> _14fr.tis
> >>>> _14fr_2.del
> >>>> _14fs.fdt
> >>>> _14fs.fdx
> >>>> _14fs.fnm
> >>>> _14fs.frq
> >>>> _14fs.nrm
> >>>> _14fs.prx
> >>>> _14fs.tii
> >>>> _14fs.tis
> >>>> _14fs_1.del
> >>>> segments.gen
> >>>> segments_1fza
> >>>> write.lock
> >>>>
> >>>> sent 871 bytes received 843185604 bytes 24440187.68 bytes/sec
> >>>> total size is 843080453 speedup is 1.00
> >>>> started by tomcat5
> >>>> command: /opt/solr/bin/snapinstaller -M search1 -S /opt/solr/logs -
> >>>> d /opt/solr/data -v
> >>>> latest snapshot /opt/solr/data/temp-snapshot.20070816120113 already
> >>>> installed
> >>>>
> >>>
> >>>
> >>
> >
>
>
