[jira] [Commented] (ACCUMULO-4542) Tablet left in bad state after bulk import timeout
[ https://issues.apache.org/jira/browse/ACCUMULO-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796461#comment-15796461 ] Christopher Tubbs commented on ACCUMULO-4542: - This seems really hard to reproduce. [~kturner] tells me he believes there is a final check before it moves, and it might do a copy instead of a move, if it has failed for some tablets but not others (in the case of the file overlapping several tablets). If he's right, then it's possible there was a failure reading the metadata table to confirm, and the system treated this failure to validate as a false-positive failure to assign. I'm not sure there's a sane way to handle that case... which is better than the result you saw. > Tablet left in bad state after bulk import timeout > -- > > Key: ACCUMULO-4542 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4542 > Project: Accumulo > Issue Type: Bug >Affects Versions: 1.7.2 >Reporter: John Vines > > On a cluster we saw a large amount of network issues at one point. Cause > still has not been pinpointed, but it did result in us seeing a lot of rpc > exceptions and the like. > While these network issues happened, a bulk import was kicked off for a > single file. This single file was assigned to two tablets (which both > happened to be on the same server). Unfortunately, in the 3 attempts bulk > import made to assign this file to this tablet, there were 3 rpc exceptions > due to a socket timeout. After the three failures the bulk import went ahead > and moved this file to the failures directory and carried on. > Unfortunately, this file was actually assigned to the tablet succesfully on > the first attempt. The following 2 attempts logged about how the server had > already been assigned this file. It was shortly afterward a query came in > (and then later major compactions) which then complained about how the file > could not be found because the bulk import moved it to the failures directory. > I think in this event we need some sort of final validation the record didn't > end up in the metadata table before we move it to the failures. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4541) shell `grep -o` writes incomplete files
[
https://issues.apache.org/jira/browse/ACCUMULO-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs updated ACCUMULO-4541:
Labels: newbie (was: )
> shell `grep -o` writes incomplete files
> ---
>
> Key: ACCUMULO-4541
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4541
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.6.6, 1.7.2
>Reporter: Michael Berman
> Labels: newbie
> Fix For: 1.7.3, 1.8.1, 2.0.0
>
>
> {{grep -o}} appears to be writing truncated files. missing a flush, maybe? i
> do not observe the same behavior with {{scan -o}}
> {code}
> $ accumulo shell ... -e 'grep a -t tablename -np -o /tmp/grep_-o.out'
> 2016-12-21 23:38:29,350 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:29,358 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,488 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,763 [trace.DistributedTrace] INFO : SpanReceiver
> org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
> $ accumulo shell ... -e 'grep a -t tablename -np' > /tmp/grep_redirect.out
> $ wc /tmp/grep_*.out
> 44030 194670 4669440 /tmp/grep_-o.out
> 44165 195171 4761183 /tmp/grep_redirect.out
> {code}
> There are a couple extra rows in the redirect output explained by the log
> messages, but not 100. The final line in grep_-o.out is truncated in the
> middle of the row column.
> If I include some terms in the grep to limit the output, I often get
> zero-length files via -o despite seeing results when run interactively.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4541) shell `grep -o` writes incomplete files
[
https://issues.apache.org/jira/browse/ACCUMULO-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796436#comment-15796436
]
Christopher Tubbs commented on ACCUMULO-4541:
-
{{ScanCommand::execute}} does something like the following:
{code:java}
if (printFile != null) {
printFile.close();
}
{code}
{{GrepCommand::execute}} does not call close. I'm guessing close flushes.
> shell `grep -o` writes incomplete files
> ---
>
> Key: ACCUMULO-4541
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4541
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.6.6, 1.7.2
>Reporter: Michael Berman
> Fix For: 1.7.3, 1.8.1, 2.0.0
>
>
> {{grep -o}} appears to be writing truncated files. missing a flush, maybe? i
> do not observe the same behavior with {{scan -o}}
> {code}
> $ accumulo shell ... -e 'grep a -t tablename -np -o /tmp/grep_-o.out'
> 2016-12-21 23:38:29,350 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:29,358 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,488 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,763 [trace.DistributedTrace] INFO : SpanReceiver
> org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
> $ accumulo shell ... -e 'grep a -t tablename -np' > /tmp/grep_redirect.out
> $ wc /tmp/grep_*.out
> 44030 194670 4669440 /tmp/grep_-o.out
> 44165 195171 4761183 /tmp/grep_redirect.out
> {code}
> There are a couple extra rows in the redirect output explained by the log
> messages, but not 100. The final line in grep_-o.out is truncated in the
> middle of the row column.
> If I include some terms in the grep to limit the output, I often get
> zero-length files via -o despite seeing results when run interactively.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4541) shell `grep -o` writes incomplete files
[
https://issues.apache.org/jira/browse/ACCUMULO-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs updated ACCUMULO-4541:
Fix Version/s: 2.0.0
1.8.1
1.7.3
> shell `grep -o` writes incomplete files
> ---
>
> Key: ACCUMULO-4541
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4541
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.6.6, 1.7.2
>Reporter: Michael Berman
> Fix For: 1.7.3, 1.8.1, 2.0.0
>
>
> {{grep -o}} appears to be writing truncated files. missing a flush, maybe? i
> do not observe the same behavior with {{scan -o}}
> {code}
> $ accumulo shell ... -e 'grep a -t tablename -np -o /tmp/grep_-o.out'
> 2016-12-21 23:38:29,350 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:29,358 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,488 [client.ClientConfiguration] WARN : Found no
> client.conf in default paths. Using default client configuration values.
> 2016-12-21 23:38:31,763 [trace.DistributedTrace] INFO : SpanReceiver
> org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
> $ accumulo shell ... -e 'grep a -t tablename -np' > /tmp/grep_redirect.out
> $ wc /tmp/grep_*.out
> 44030 194670 4669440 /tmp/grep_-o.out
> 44165 195171 4761183 /tmp/grep_redirect.out
> {code}
> There are a couple extra rows in the redirect output explained by the log
> messages, but not 100. The final line in grep_-o.out is truncated in the
> middle of the row column.
> If I include some terms in the grep to limit the output, I often get
> zero-length files via -o despite seeing results when run interactively.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4547) Broken Links in Release Notes
[ https://issues.apache.org/jira/browse/ACCUMULO-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4547: Labels: newbie (was: ) > Broken Links in Release Notes > - > > Key: ACCUMULO-4547 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4547 > Project: Accumulo > Issue Type: Bug > Components: website >Reporter: Michael Miller >Assignee: Christopher Tubbs >Priority: Minor > Labels: newbie > > I think the recent changes to the website broke some of the links in the > release notes. Here are snippets from some of the links that I noticed for > https://accumulo.apache.org/release/accumulo-1.8.0/ > - "brand new public API for reading" > Not Found > The requested URL > /release/1.8/apidocs/org/apache/accumulo/core/client/rfile/RFile.html was not > found on this server. > - "See the sample Readme example and javadocs on the new APIs" > Not Found > The requested URL /release/1.8/examples/sample was not found on this server. > - "Please see the Accumulo User Manual chapter on Iterator Testing for more > information" > Not Found > The requested URL /release/1.8/accumulo_user_manual.html was not found on > this server. > The JIRA links seem OK. It looks like its all the relative links -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4546) IllegalTableTransitionException should include a default message that logs the requested state transition
[ https://issues.apache.org/jira/browse/ACCUMULO-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4546: Fix Version/s: 2.0.0 1.8.1 1.7.3 > IllegalTableTransitionException should include a default message that logs > the requested state transition > - > > Key: ACCUMULO-4546 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4546 > Project: Accumulo > Issue Type: Bug > Components: server-base >Affects Versions: 1.6.6 >Reporter: John Vines > Fix For: 1.7.3, 1.8.1, 2.0.0 > > > While trying to track down the root of an Illegal state transition for a > table, I hit a dead end when the original transition to bring a table online > failed. The IllegalTableTransitionException takes in the old and new states > in the contstructor for the exception, but these states are not used to > construct any sort of message, so this information isn't available in the > logs. We should have a default message for this constructor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4545) Update User Manual to use Slate
[ https://issues.apache.org/jira/browse/ACCUMULO-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796312#comment-15796312 ] Christopher Tubbs commented on ACCUMULO-4545: - Slate is really great for documenting commands and APIs, in particular. It's a bit confusing for long amounts of text. However, the biggest problem I can see with using it, is that it's not easy to generate using our existing build tooling, and the output is not the single-file-manuscript that we currently have today, which is really convenient for re-publication (like a PDF). > Update User Manual to use Slate > --- > > Key: ACCUMULO-4545 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4545 > Project: Accumulo > Issue Type: Improvement > Components: docs >Reporter: Luis Tavarez >Priority: Minor > Labels: documentation > > I was looking at the Timely documentation > (https://nationalsecurityagency.github.io/timely/docs/) and saw that they use > Slate for it, which looks really nice (specially for big documents such as > the Accumulo manual). I was wondering if it would be possible to update the > Accumulo User Manual to use Slate instead of asciidocs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4543) backslashes in shell commands are consumed when using up key
[
https://issues.apache.org/jira/browse/ACCUMULO-4543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796301#comment-15796301
]
Christopher Tubbs commented on ACCUMULO-4543:
-
Oh, interesting. Let me know if you get a chance to look at this. I'm very
interested in making future improvements to the shell and related tooling. In
particular, I think it's frustrating how we handle multi-byte characters. (What
if I want to use as a delimiter?)
> backslashes in shell commands are consumed when using up key
>
>
> Key: ACCUMULO-4543
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4543
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.7.1
>Reporter: Michael Berman
>
> If you pull a command out of your shell history with the up key, one slash
> gets consumed from each sequence of slashes. As in:
> {code}
> $ grep \\x00something\\x00
> > results...
> $ [up]
> $ grep \x00something\x00
> > no results because now you're not escaped
> $ [up]
> $ grep x00somethingx00
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4061) Add way to determine Accumulo version from a running tserver
[ https://issues.apache.org/jira/browse/ACCUMULO-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796296#comment-15796296 ] Christopher Tubbs commented on ACCUMULO-4061: - Excellent. I added some comments to your commit. https://github.com/lstav/accumulo/commit/ecd4234aae90a5b1971f06c36a454274d17f15d8#diff-d0b489f3f342ab286859731869eeb622 > Add way to determine Accumulo version from a running tserver > > > Key: ACCUMULO-4061 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4061 > Project: Accumulo > Issue Type: New Feature > Components: tserver >Reporter: Ed Coleman >Assignee: Luis Tavarez >Priority: Minor > Fix For: 1.7.3, 1.8.1, 2.0.0 > > > When upgrading by performing a rolling restart, realized that there was no > way to determine the version of a running tserver other than using ps to get > start time or maybe a versioned package on the classpath. > A typical deployment structure seems to be lay down the Accumulo directories > with a version and then use a symbolic link to point to the version that is > used at run-time. As an example: > /usr/local/accumulo > /usr/local/accumulo/accumulo-1.6.2 > /usr/local/accumulo/accumulo-1.6.3 > /usr/local/accumulo/accumulo-latest --> /usr/local/accumulo/accumulo-1.6.3 > To upgrade without a complete shutdown, we lay down the new version, update > the symbolic link and then perform a rolling restart of the tservers, with > scripts, env,... using the symbolic link to specify which version are used. > Realized that if the rolling shutdown failed for any particular tserver, it > would keep running the previous version. The only way to determine if we > were running the desired version was to use ps and check the running time of > the tserver process. > The could also be a situation were a "dead" server would be offline during > the upgrade and not receive the new version. If the server was resurrected > and services started before updated versions are installed it could be > difficult to determine exactly what version of the tserver was running. > It would be nice if there was some way to ask a running tserver what version > it is. That way, post-upgrade we could confirm that all running tservers > reply with the expected version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4061) Add way to determine Accumulo version from a running tserver
[ https://issues.apache.org/jira/browse/ACCUMULO-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-4061: Fix Version/s: 2.0.0 > Add way to determine Accumulo version from a running tserver > > > Key: ACCUMULO-4061 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4061 > Project: Accumulo > Issue Type: New Feature > Components: tserver >Reporter: Ed Coleman >Assignee: Luis Tavarez >Priority: Minor > Fix For: 1.7.3, 1.8.1, 2.0.0 > > > When upgrading by performing a rolling restart, realized that there was no > way to determine the version of a running tserver other than using ps to get > start time or maybe a versioned package on the classpath. > A typical deployment structure seems to be lay down the Accumulo directories > with a version and then use a symbolic link to point to the version that is > used at run-time. As an example: > /usr/local/accumulo > /usr/local/accumulo/accumulo-1.6.2 > /usr/local/accumulo/accumulo-1.6.3 > /usr/local/accumulo/accumulo-latest --> /usr/local/accumulo/accumulo-1.6.3 > To upgrade without a complete shutdown, we lay down the new version, update > the symbolic link and then perform a rolling restart of the tservers, with > scripts, env,... using the symbolic link to specify which version are used. > Realized that if the rolling shutdown failed for any particular tserver, it > would keep running the previous version. The only way to determine if we > were running the desired version was to use ps and check the running time of > the tserver process. > The could also be a situation were a "dead" server would be offline during > the upgrade and not receive the new version. If the server was resurrected > and services started before updated versions are installed it could be > difficult to determine exactly what version of the tserver was running. > It would be nice if there was some way to ask a running tserver what version > it is. That way, post-upgrade we could confirm that all running tservers > reply with the expected version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ACCUMULO-4549) Remove duplicate init functions in TabletBalancer
Adam J Shook created ACCUMULO-4549:
--
Summary: Remove duplicate init functions in TabletBalancer
Key: ACCUMULO-4549
URL: https://issues.apache.org/jira/browse/ACCUMULO-4549
Project: Accumulo
Issue Type: Improvement
Reporter: Adam J Shook
Priority: Minor
There are two {{init}} functions with similar signatures in the
{{TabletLoadBalancer}} class that could result in an end-user overriding the
wrong function and their object may never be initialized.
{{Master}} (1) calls the {{TabletBalancer#init(ServerConfigurationFactory)}}
which is not what the regex load balancer is overriding (2), and the
{{TabletBalancer}} doesn't call the other init function (3).
Come up with a plan for compatibility and remove the duplicate functions.
(1)
https://github.com/apache/accumulo/blob/1.7/server/master/src/main/java/org/apache/accumulo/master/Master.java#L589
(2)
https://github.com/apache/accumulo/blob/1.7/server/base/src/main/java/org/apache/accumulo/server/master/balancer/HostRegexTableLoadBalancer.java#L257
(3)
https://github.com/apache/accumulo/blob/1.7/server/base/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java#L57
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Accumulo-Pull-Requests - Build # 588 - Aborted
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #588) Status: Aborted Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/588/ to view the results.
[jira] [Updated] (ACCUMULO-4057) Duplicated code in IteratorUtil
[
https://issues.apache.org/jira/browse/ACCUMULO-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs updated ACCUMULO-4057:
Fix Version/s: 2.0.0
> Duplicated code in IteratorUtil
> ---
>
> Key: ACCUMULO-4057
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4057
> Project: Accumulo
> Issue Type: Improvement
>Reporter: Josh Elser
>Assignee: Luis Tavarez
>Priority: Trivial
> Fix For: 2.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Duplicated code in
> https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java#L236
> {code}
> public static ,V extends Writable>
> SortedKeyValueIterator loadIterators(IteratorScope scope,
> SortedKeyValueIterator source, KeyExtent extent,
> AccumuloConfiguration conf, List ssiList,
> Map> ssio,
> IteratorEnvironment env, boolean useAccumuloClassLoader) throws
> IOException {
> List iters = new ArrayList(ssiList);
> Map> allOptions = new
> HashMap>();
> parseIteratorConfiguration(scope, iters, ssio, allOptions, conf);
> return loadIterators(source, iters, allOptions, env,
> useAccumuloClassLoader, conf.get(Property.TABLE_CLASSPATH));
> }
> public static ,V extends Writable>
> SortedKeyValueIterator loadIterators(IteratorScope scope,
> SortedKeyValueIterator source, KeyExtent extent,
> AccumuloConfiguration conf, List ssiList,
> Map> ssio,
> IteratorEnvironment env, boolean useAccumuloClassLoader, String
> classLoaderContext) throws IOException {
> List iters = new ArrayList(ssiList);
> Map> allOptions = new
> HashMap>();
> parseIteratorConfiguration(scope, iters, ssio, allOptions, conf);
> return loadIterators(source, iters, allOptions, env,
> useAccumuloClassLoader, classLoaderContext);
> }
> {code}
> I thought I had commented on https://github.com/apache/accumulo/pull/51 about
> this, but maybe I forgot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ACCUMULO-4548) Multiple grammar errors in Accumulo Website
[ https://issues.apache.org/jira/browse/ACCUMULO-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Tavarez updated ACCUMULO-4548: --- Summary: Multiple grammar errors in Accumulo Website (was: Multiple grammar errors) > Multiple grammar errors in Accumulo Website > --- > > Key: ACCUMULO-4548 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4548 > Project: Accumulo > Issue Type: Improvement > Components: website >Reporter: Luis Tavarez >Assignee: Luis Tavarez >Priority: Trivial > > While looking at the website, I noticed that there are a couple of grammar > errors on different pages (and some sentences are confusing to read). It > would be nice to fix them to make the website easier to read (specially for > people whose first language is not English). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4548) Multiple grammar errors
[ https://issues.apache.org/jira/browse/ACCUMULO-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795790#comment-15795790 ] Luis Tavarez commented on ACCUMULO-4548: Started work on this. I'm going through the website pages to see if I find any grammar errors or sentences that are hard/confusing to read. > Multiple grammar errors > --- > > Key: ACCUMULO-4548 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4548 > Project: Accumulo > Issue Type: Improvement > Components: website >Reporter: Luis Tavarez >Assignee: Luis Tavarez >Priority: Trivial > > While looking at the website, I noticed that there are a couple of grammar > errors on different pages (and some sentences are confusing to read). It > would be nice to fix them to make the website easier to read (specially for > people whose first language is not English). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ACCUMULO-4548) Multiple grammar errors
Luis Tavarez created ACCUMULO-4548: -- Summary: Multiple grammar errors Key: ACCUMULO-4548 URL: https://issues.apache.org/jira/browse/ACCUMULO-4548 Project: Accumulo Issue Type: Improvement Components: website Reporter: Luis Tavarez Assignee: Luis Tavarez Priority: Trivial While looking at the website, I noticed that there are a couple of grammar errors on different pages (and some sentences are confusing to read). It would be nice to fix them to make the website easier to read (specially for people whose first language is not English). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
