Re: [External] Re: odd issue with accumulo 1.10.0 starting up

2022-03-17 Thread Zachary Radtka 
I have had this exact error before:

ERROR: read a frame size of 1195725856, which is bigger than the maximum
allowable buffer size for ALL connections.

My cluster was on a client's AWS account which would regularly have
security scans on the weekends. Logging in on Monday the master would
always be down. We didn't know what the security scans were, but we did
solve our issue by placing our servers in a security group that only
allowed the accumulo servers to talk with each other. We also restricted
inbound traffic to security groups for our other systems that were
accessing Accumulo directly.

-Zach



On Thu, Mar 17, 2022 at 11:45 AM Mike Miller  wrote:

> Are you still running Replication? I would turn it off if you can.
>
> On Thu, Mar 17, 2022 at 7:44 AM dev1  wrote:
>
>> When an Accumulo process abnormally terminates, there may be a file
>> create with the exception of the problem – the files may be names *.out (or
>> *.err) can’t recall which. Normally the files have 0 size, but on
>> termination will have some text.
>>
>>
>>
>> Are you seeing those files and do they point to the issue?
>>
>>
>>
>> Do you have the jvm configured to terminate on out of memory – and print
>> that error condition? Maybe the manager is running out of memory.
>>
>>
>>
>> Ed Coleman
>>
>>
>>
>> *From:* Ligade, Shailesh [USA] 
>> *Sent:* Wednesday, March 16, 2022 3:31 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* RE: [External] Re: odd issue with accumulo 1.10.0 starting up
>>
>>
>>
>> Thanks,
>>
>>
>>
>> I think we are having the same or similar issue with virus scan/security
>> scan. However that should not bring down the master, can it??
>>
>>
>>
>> I am still digging thru the logs.
>>
>>
>>
>> -S
>>
>>
>>
>> *From:* Adam J. Shook 
>> *Sent:* Wednesday, March 16, 2022 2:46 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* Re: [External] Re: odd issue with accumulo 1.10.0 starting up
>>
>>
>>
>> This is certainly anecdotal, but we've seen this "ERROR: Read a frame
>> size of (large number)" before on our Accumulo cluster that would show up
>> at a regular and predictable frequency. The root cause was due to a routine
>> scan done by the security team looking for vulnerabilities across the
>> entire enterprise (nothing Accumulo-specific). I don't have any additional
>> information about the specifics of the scan. From all that we can tell, it
>> has no impact on our Accumulo cluster outside of these error messages.
>>
>>
>>
>> --Adam
>>
>>
>>
>> On Wed, Mar 16, 2022 at 8:35 AM Christopher  wrote:
>>
>> Since that error message is coming from the libthrift library, and not
>> Accumulo code, we would need a lot more context to even begin helping you
>> troubleshoot it. For example, the complete stack trace that shows the
>> Accumulo code that called into the Thrift library, would be extremely
>> helpful.
>>
>> It's a bit concerning that you're trying to send a single buffer over
>> thrift that's over a gigabyte in size, according to that number. You've
>> said before that you use live ingest. Are you trying to send a 1GB mutation
>> to a tablet server? Or are you using replication and the stack trace looks
>> like it's sending 1GB of replication data?
>>
>>
>>
>> On Wed, Mar 16, 2022 at 7:14 AM Ligade, Shailesh [USA] <
>> ligade_shail...@bah.com> wrote:
>>
>> Well, I re-initialized accumulo but I still see
>>
>>
>>
>> ERROR: Read a frame size of 1195725856, which is bigger than the maximum
>> allowable buffer size for ALL connections.
>>
>>
>>
>> Is there a setting that I can increase to get past it?
>>
>>
>>
>> -S
>>
>>
>>
>>
>> --
>>
>> *From:* Ligade, Shailesh [USA] 
>> *Sent:* Tuesday, March 15, 2022 12:47 PM
>> *To:* user@accumulo.apache.org 
>> *Subject:* Re: [External] Re: odd issue with accumulo 1.10.0 starting up
>>
>>
>>
>> Not daily but  over weekend.
>> --
>>
>> *From:* Mike Miller 
>> *Sent:* Tuesday, March 15, 2022 10:39 AM
>> *To:* user@accumulo.apache.org 
>> *Subject:* Re: [External] Re: odd issue with accumulo 1.10.0 starting up
>>
>>
>>
>> Why are you bringing the cluster down every night? That is not ideal.
>>
>>
>>
>> On Tue, Mar 15, 2022 at 9:24 AM Ligade, Shailesh [USA] <
>> ligade_shail...@bah.com> wrote:
>>
>> Thanks Mike,
>>
>>
>>
>> We bring the servers down nightly. these are on aws. This worked
>> yesterday (Monday) but this (Tuesday) i went on to check on it and it was
>> down, I guess i didn't check yesterday. I assume it was up as no one
>> complained., but it was up and kicking last week for sure.
>>
>>
>>
>> So not exactly sure when or what caused it, all services are up (tserver,
>> master) so services are not crashing themselves.
>>
>>
>>
>> I guess worst case, i can re-initialize and recreate tables form hdfs..:-(
>>
>>
>>
>> -S
>> --
>>
>> *From:* Mike Miller 
>> *Sent:* Tuesday, March 15, 2022 9:16 AM
>> *To:* user@accumulo.apache.org 
>> *Subject:* Re: [External] Re: odd issue with accumulo

Re: Null source for iterator

2014-02-27 Thread Zachary Radtka 
Hey Michael, I ran into something similar when I didn't implement the 
deepcopy() method on my custom iterator. Inside the deepCopy method you will 
need to make sure you copy any options your iterator implements.

On Thu, Feb 27, 2014 at 1:36 PM, Giordano, Michael
michael.giord...@vistronix.com wrote:

 Given this code :
 BatchScanner scanner = connector.createBatchScanner(tableName, 
 authorizations, threadCount);
 IteratorSetting iterSetting = new IteratorSetting(10, filterName, 
 CustomFilter.class);
 iterSetting.addOption(option1, value1);
 iterSetting.addOption(option2, value2);
 scanner.addScanIterator(iterSetting);
 IteratorEntryKey,Value iter = scanner.iterator();
 iter.hasNext();
 I am receiving this exception when hasNext() is called :
 2014-02-27 15:47:08,845 [thrift.ProcessFunction] ERROR: Internal error 
 processing startMultiScan
 java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
 java.lang.IllegalStateException: getting null source
 The full stacktrace can be found at http://pastebin.com/nkpKnwAh
 If anyone can shed light on what I am doing wrong, I would deeply appreciate 
 it.
 Thanks,
 Mike G.
 This communication, along with its attachments, is considered confidential 
 and proprietary to Vistronix.  It is intended only for the use of the 
 person(s) named above.  Note that unauthorized disclosure or distribution of 
 information not generally known to the public is strictly prohibited.  If you 
 are not the intended recipient, please notify the sender immediately.

Re: Inserting a negative value in the shell.

2013-10-22 Thread Zachary Radtka 
David,

I just tried the same thing and it looks like it is a bug.

If you did want to insert -1 from the shell you could:
insert row cf cq \-1\

That does put 's around the value, but it is a way to get around the
problem.


On Tue, Oct 22, 2013 at 10:03 AM, David Medinets
david.medin...@gmail.comwrote:

 I just tried to do this:

   insert row cf cq -1

 The negative sign is being interpreted as an unrecognized option. There
 does not appear a workaround but perhaps I'm missing something?