The auto tests are a bit flakey, especially when they rely on local conditions, like the speed at which my workstation will run something. If you have found a bug in one of the scripts, please make a Jira ticket.
Right now several of the tests assume they have superuser rights to scribble all over the distributed file system, see ACCUMULO-355. -Eric On Sun, Apr 8, 2012 at 10:01 AM, Keys Botzum <[email protected]> wrote: > Billie, > > I apologize for taking so long to reply to your message. As soon as I read > it I thought to myself "this is a very good argument." I did some more > research myself and talked with folks at MapR and we agreed that the right > thing to do here is to change MapR to be tolerant of a seek to position > just after the last byte. > > I've delayed posting this because I wanted to report the results of the > change and retest. This was delayed for reasons that have nothing to do > with Accumulo or really MapR, I just was too busy. > > In any case, we've tried the change in MapR (being tracked as change 6539) > and this does indeed address the issue below. The test case in question now > passes. Thank you VERY much for your help. It is much appreciated. > > I am continuing testing and have encountered additional issues but I > haven't yet had time to analyze them. I will post something when I have a > more coherent question. I would just like to confirm that all of the auto > tests do run successfully on stock Apache Hadoop? I ask because at least > one of the tests appears to have a bug in the script that runs it. I just > want to make sure I'm testing the right stuff. > > Incidentally, I have upgrade to the official Accumulo 1.4.0 release. > > > Keys > > > On Fri, Mar 30, 2012 at 11:15 PM, Billie J Rinaldi < > [email protected]> wrote: > >> It is my understanding that the file length is a valid file position. If >> you are at position 0 and read one byte, you get the first byte of the >> file. If you are at position length-1 and read one byte, you get the last >> byte of the file. At that point you are at position equal to the file >> length, and attempting to read an additional byte will give an EOF >> indicator (for an InputStream read(), this would mean returning -1). >> Seeking to the file length should just cause a read to give an EOF. >> >> Billie >> >> >> ----- Original Message ----- >> > From: "Keys Botzum" <[email protected]> >> > To: [email protected] >> > Sent: Friday, March 30, 2012 10:12:00 PM >> > Subject: Re: Accumulo on MapR Continued >> > Billie, >> > >> > >> > Thanks for the quick reply. I did some checking and the Hadoop docs >> > indicate that seeking past the end of the file is illegal. >> > >> > >> > >> > >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FSDataInputStream.html#seek%28long%29 >> > seek >> > public void seek (long desired) >> > throws IOException Description copied from interface: Seekable Seek to >> > the given offset from the start of the file. The next read() will be >> > from that location. Can't seek past the end of the file. >> > >> > Specified by: seek in interface Seekable Throws: IOException >> > >> > >> > The code below that is seeking to the file length is actually seeking >> > one byte past the end of the file since the first byte in a file is >> > byte zero. To me this seems like an Accumulo bug. Can this be fixed? >> > >> > >> > Thanks, >> > Keys >> > ________________________________ >> > Keys Botzum >> > Senior Principal Technologist >> > WW Systems Engineering >> > [email protected] >> > 443-718-0098 >> > MapR Technologies >> > http://www.mapr.com >> > >> > >> > >> > >> > >> > On Mar 30, 2012, at 5:04 PM, Billie J Rinaldi wrote: >> > >> > >> > >> > I ran the following JUnit test and verified that the value printed out >> > was the file length. >> > >> > @Test >> > public void testSeekToEOF() throws IOException { >> > FileSystem fs = FileSystem.get(new Configuration()); >> > FSDataInputStream fdi = fs.open(new Path("README")); >> > System.out.println("seeking to end: " + fdi.available()); >> > fdi.seek(fdi.available()); >> > fdi.close(); >> > } >> > >> > >> > ----- Original Message ----- >> > >> > >> > From: "Keys Botzum" < [email protected] > >> > >> > >> > To: [email protected] >> > >> > >> > Sent: Friday, March 30, 2012 4:56:09 PM >> > >> > >> > Subject: Re: Accumulo on MapR Continued >> > >> > >> > Can you share the program? I can try running it and answer your >> > >> > >> > question. I can also of course check with MapR engineering to see if >> > >> > >> > that's the issue. >> > >> > >> > >> > >> > >> > >> > >> > >> > Much appreciated, >> > >> > >> > Keys >> > >> > >> > >> > >> > >> > ________________________________ >> > >> > >> > Keys Botzum >> > >> > >> > Senior Principal Technologist >> > >> > >> > WW Systems Engineering >> > >> > >> > [email protected] >> > >> > >> > 443-718-0098 >> > >> > >> > MapR Technologies >> > >> > >> > http://www.mapr.com >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Mar 30, 2012, at 4:48 PM, Billie J Rinaldi wrote: >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Friday, March 30, 2012 2:07:49 PM, "Keys Botzum" < >> > >> > >> > [email protected] > wrote: >> > >> > >> > >> > >> > >> > >> > >> > >> > 30 11:00:15,778 [tabletserver.Tablet] ERROR: Failed to find lastkey >> > >> > >> > >> > >> > >> > >> > >> > >> > Seeking beyond EOF, filelen 733, wantpos 733 >> > >> > >> > >> > >> > >> > It looks like Accumulo is trying to seek to the last position in a >> > >> > >> > file. I just wrote a little program that opens a file in HDFS and >> > >> > >> > seeks to its file length, and it didn't throw an error. Does MapR not >> > >> > >> > allow this? If that is the desired MapR behavior, we could try to >> > >> > >> > determine whether Accumulo could avoid performing that seek. >> > >> > >> > >> > >> > >> > Billie >> > > > > -- > ________________________________ > Keys Botzum > Senior Principal Technologist > WW Systems Engineering > [email protected] > 443-718-0098 > MapR Technologies > http://www.mapr.com <http://mapr.com> > >
