Eric, thanks for the help. This is on 1.3.2, looks like we will be able to go to 1.3.4 very shortly. >From the below JIRA, it looks like it is fixed in 1.3.6. I will send this up as I'm not an approving authority on installed/running versions and will also recommend the trace password in the site.xml.
Once we upgrade to the minor version, I will report back on whether the WAL GC runs in that version. -Kris On Wed, Apr 11, 2012 at 1:00 PM, Eric Newton <[email protected]> wrote: > Also you will need to restart. > > What version of accumulo are you running? This problem should have been > fixed with ACCUMULO-243. > > -Eric > > On Wed, Apr 11, 2012 at 12:28 PM, Eric Newton <[email protected]>wrote: > >> in conf/accumulo-site.xml, add this: >> >> <property> >> <name>trace.password</name> >> <value>your-root-password-here</value> >> </property> >> >> -Eric >> >> On Wed, Apr 11, 2012 at 12:14 PM, Kristopher Kane >> <[email protected]>wrote: >> >>> I know I haven't answered some of the previous questions to me, but, the >>> GC talk brought me to the source of the problem. our WAL GC job is not >>> successfully running. The GC logs state a bad username/password, so, will >>> investigate that. >>> >>> Thanks for the help! >>> >>> -Kris >>> >>> On Wed, Apr 11, 2012 at 11:18 AM, Keith Turner <[email protected]> wrote: >>> >>>> When data is written to the accumulo its written to memory and the >>>> write ahead logs. The data in memory is sorted and the data in the >>>> write ahead logs is written asis (unsorted). When the data in memory >>>> is flushed to HDFS, the write ahead logs that also contain that data >>>> are no longer needed. >>>> >>>> On Wed, Apr 11, 2012 at 11:12 AM, Eric Newton <[email protected]> >>>> wrote: >>>> > Logs are only put into HDFS during a recovery. >>>> > >>>> > Flush removes references to WALs, and the accumulo gc will ask the >>>> loggers >>>> > to delete them when there are no references to them. >>>> > >>>> > -Eric >>>> > >>>> > >>>> > On Wed, Apr 11, 2012 at 11:02 AM, Kristopher Kane < >>>> [email protected]> >>>> > wrote: >>>> >> >>>> >> >>>> >> >>>> >> On Wed, Apr 11, 2012 at 10:48 AM, Keith Turner <[email protected]> >>>> wrote: >>>> >>> >>>> >>> How big is the partition? Are the same number of logger servers >>>> >>> running as tablet servers? >>>> >>> >>>> >>> >>>> >>> You can scan the metadata table to look for tablets that have alot >>>> of >>>> >>> write-ahead logs. I think the command below will show you how many >>>> >>> write-ahead logs each tablet has. Look for any tablets that have >>>> too >>>> >>> many. I think it should sort the tablets with the most tablets to >>>> the >>>> >>> top, but not positive. >>>> >>> >>>> >>> ./bin/accumulo shell -u root -p secret -e 'scan -t !METADATA -c >>>> >>> log' | cut -f 1 -d ' ' | uniq -c | sort -r -n >>>> >>> >>>> >>> I think the following command will show you how many active log each >>>> >>> logger has. This should be even. >>>> >>> >>>> >>> ./bin/accumulo shell -u root -p secret -e 'scan -t !METADATA -c >>>> >>> log' | cut -f 2 -d ' ' | cut -d ':' -f 2 | sort | uniq -c >>>> >>> >>>> >>> You can use the "flush -p" command in the shell to force data in >>>> >>> memory to disk and stop referencing write-ahead logs. Maybe execute >>>> >>> the commands above before and after flushing. >>>> >>> >>>> >>> Keith >>>> >>> >>>> >>> >>>> >> >>>> >> >>>> >> >>>> >> Thanks for the replies. I read about the flush command in the docs >>>> but >>>> >> didn't make a connection between "memory" to the write ahead logs. >>>> Is >>>> >> that correct? Flush writes write ahead log data to hdfs? >>>> >> >>>> >> Thanks! >>>> >> >>>> >> -Kris >>>> > >>>> > >>>> >>> >>> >> >
