[jira] [Commented] (ACCUMULO-1703) Thrift generated code isn't properly checked for optional fields
[ https://issues.apache.org/jira/browse/ACCUMULO-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834988#comment-17834988 ] Christopher Tubbs commented on ACCUMULO-1703: - This can be fixed by the mandatory use of Thrift private members, as per https://github.com/apache/accumulo/pull/3405 > Thrift generated code isn't properly checked for optional fields > > > Key: ACCUMULO-1703 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1703 > Project: Accumulo > Issue Type: Bug >Reporter: Christopher Tubbs >Priority: Minor > > In our thrift definition files, we declare some fields as optional. However, > we don't actually check to see if they are sent. This may be okay for > Objects, which we can check for null, but it won't work for checking optional > primitives. > Example: > ProxyServer.createBatchScanner() accepts a BatchScanOptions object, which has > the number of threads as an optional parameter. Instead of calling > opts.threads, we should check if opts.isSetThreads() is true, and then > retrieve it with opts.getThreads(). > This is an essential check for all optional arguments (non-optional args are > validated by the generated validate() method). > The importance of the optional field is high, if we want to support RPC > compatibility across versions. As far as I know, this is one of the goals of > the client proxy. So, we must use the thrift features (such as the optional > flag) correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1703) Thrift generated code isn't properly checked for optional fields
[ https://issues.apache.org/jira/browse/ACCUMULO-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1703. - Resolution: Duplicate > Thrift generated code isn't properly checked for optional fields > > > Key: ACCUMULO-1703 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1703 > Project: Accumulo > Issue Type: Bug >Reporter: Christopher Tubbs >Priority: Minor > > In our thrift definition files, we declare some fields as optional. However, > we don't actually check to see if they are sent. This may be okay for > Objects, which we can check for null, but it won't work for checking optional > primitives. > Example: > ProxyServer.createBatchScanner() accepts a BatchScanOptions object, which has > the number of threads as an optional parameter. Instead of calling > opts.threads, we should check if opts.isSetThreads() is true, and then > retrieve it with opts.getThreads(). > This is an essential check for all optional arguments (non-optional args are > validated by the generated validate() method). > The importance of the optional field is high, if we want to support RPC > compatibility across versions. As far as I know, this is one of the goals of > the client proxy. So, we must use the thrift features (such as the optional > flag) correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3121) Add ability to watch configuration changes to API
[ https://issues.apache.org/jira/browse/ACCUMULO-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3121. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Add ability to watch configuration changes to API > - > > Key: ACCUMULO-3121 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3121 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: Christopher Tubbs >Priority: Minor > > In ACCUMULO-2841, an improvement was proposed by [~vines] to have the ability > to watch certain configuration attributes. > We currently don't support this in our API, but in theory, it's possible, > because we store these things in ZooKeeper, which does support watchers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-671) Create utility to re-initialize just ZooKeeper from existing Accumulo install
[ https://issues.apache.org/jira/browse/ACCUMULO-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-671. Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Create utility to re-initialize just ZooKeeper from existing Accumulo install > - > > Key: ACCUMULO-671 > URL: https://issues.apache.org/jira/browse/ACCUMULO-671 > Project: Accumulo > Issue Type: New Feature > Components: master >Reporter: Krishmin Rai >Priority: Minor > > In a case where ZooKeeper data is lost but the Accumulo footprint in HDFS > still exists, it would be nice to be able to properly re-initialize just > ZooKeeper (to the extent possible). > See [mailing > list|http://mail-archives.apache.org/mod_mbox/accumulo-user/201207.mbox/%3CCAGUtCHrfNCApFb5iAVJX9R%3DuzwP5zT7FHw7zBrytMQwH1voJcw%40mail.gmail.com%3E] > for discussion -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-463) use term cardinalities to optimize the AndIterator
[ https://issues.apache.org/jira/browse/ACCUMULO-463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-463. Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > use term cardinalities to optimize the AndIterator > -- > > Key: ACCUMULO-463 > URL: https://issues.apache.org/jira/browse/ACCUMULO-463 > Project: Accumulo > Issue Type: Improvement > Components: wikisearch >Reporter: Eric C. Newton >Priority: Minor > > The wikisearch example stores term cardinalities as metadata, but does not > use the term frequency to determine the best initial seek term for > conjunctions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1344) Shell gives erroneous error messages during long compaction
[
https://issues.apache.org/jira/browse/ACCUMULO-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-1344.
-
Resolution: Abandoned
Closing old inactive issue; if this is still a problem, please create a new
issue or pull request at https://github.com/apache/accumulo
> Shell gives erroneous error messages during long compaction
> ---
>
> Key: ACCUMULO-1344
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1344
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.4.2
>Reporter: Mike Drob
>Priority: Minor
>
> When waiting for a large compaction to finish that was kicked off in the
> shell via {{compact -w}}, it may warn that {{Thread "shell" stuck on IO}}
> after ten minutes.
> This warning may be misleading, as the compaction could be running just fine,
> and simply running over a large table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3088) StatsIterator readCounter could be more accurate
[ https://issues.apache.org/jira/browse/ACCUMULO-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3088. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > StatsIterator readCounter could be more accurate > > > Key: ACCUMULO-3088 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3088 > Project: Accumulo > Issue Type: Bug >Reporter: Adam Fuchs >Priority: Trivial > Labels: newbie > > When testing seek performance I noticed that the read count displayed on > Accumulo's monitor page registered zero entries read per second and a > positive number of entries per second returned. This should not be possible. > There is a fencepost error in the StatsIterator where seeks don't count as > entries read. > Successful completion of this ticket should handle the cases where a seek > returns no entries, as well as where next is never called on an iterator. > The cost of incrementing an AtomicLong is something like 10ns, which is at > least an order of magnitude more than the desired overhead. The current way > of batching updates to the read counter should be preserved for performance. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2953) Maxima in memory stress test are one less than configuration options
[
https://issues.apache.org/jira/browse/ACCUMULO-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-2953.
-
Resolution: Abandoned
Closing old inactive issue; if this is still a problem, please create a new
issue or pull request at https://github.com/apache/accumulo
> Maxima in memory stress test are one less than configuration options
>
>
> Key: ACCUMULO-2953
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2953
> Project: Accumulo
> Issue Type: Test
> Components: test
>Reporter: Bill Havanki
>Priority: Trivial
> Labels: boundary, test
>
> The {{RandomWithinRange}} class in the memory stress test takes care of
> generating random integers across a range. The range is inclusive at the
> beginning and exclusive at the end. That means that the maxima for various
> test configuration options, such as key component and value lengths, are
> actually one less than the option values. For example, with
> --min-value-size=1 and --max-value-size=10, value lengths will range only
> from 1 to 9.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2055) configure random walkers to send logs to the monitor
[ https://issues.apache.org/jira/browse/ACCUMULO-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2055. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo (Also, this should be easily doable with the latest monitor appender configured by default in our reference log4j config for servers) > configure random walkers to send logs to the monitor > > > Key: ACCUMULO-2055 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2055 > Project: Accumulo > Issue Type: Improvement > Components: test >Reporter: Eric C. Newton >Priority: Minor > Labels: newbie > > There's a bit of a hack to configure WARN/ERROR messages to a shared NFS file > system so that you can monitor all the walkers from one place. However, if > you don't have a shared NFS, it's painful to monitor the logs distributed > throughout the cluster. We already have a single place to watch the logs: > the monitor. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4559) Create status API for Accumulo services
[ https://issues.apache.org/jira/browse/ACCUMULO-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4559. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Create status API for Accumulo services > --- > > Key: ACCUMULO-4559 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4559 > Project: Accumulo > Issue Type: Improvement > Components: gc, master, tserver >Reporter: Luis Tavarez >Priority: Minor > > Create a status API to the accumulo services, similar to the tservers status. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4493) Shell should be able to use keytab login
[ https://issues.apache.org/jira/browse/ACCUMULO-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4493. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Shell should be able to use keytab login > > > Key: ACCUMULO-4493 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4493 > Project: Accumulo > Issue Type: New Feature > Components: shell >Reporter: Sean Busbey >Priority: Minor > > Users should be able to launch the shell in a kerberos deployment using a > keytab. > current workaround: use the system shell to kinit with the keytab, then > launch the shell, then kdestroy > Workaround doesn't allow re-login from keytab for long running shell. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4595) Simple validator for inserting data
[ https://issues.apache.org/jira/browse/ACCUMULO-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4595. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Simple validator for inserting data > --- > > Key: ACCUMULO-4595 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4595 > Project: Accumulo > Issue Type: New Feature >Reporter: Michael Wall >Priority: Minor > > It would be useful to be able to write custom validation for specific parts > of a Key or Value that could be applied to both mutations on live ingest and > writing RFiles for bulk loads. Initially I thought this could be done with a > table constraint, but didn't consider the importdirectory use case where > RFiles are bulk imported. This validation does not need to track state > though. Thinking this is really only for simple validation. Couple of > contrived examples. > 1) Validate the CF starts with the string "abcd". > 2) Validate the CV only contains strings from a predefined list of strings. > 3) Validate the row only contains letters or numbers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2596) Document failure mode around persistent version upgrades
[ https://issues.apache.org/jira/browse/ACCUMULO-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2596. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Document failure mode around persistent version upgrades > > > Key: ACCUMULO-2596 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2596 > Project: Accumulo > Issue Type: Task > Components: docs, master >Affects Versions: 1.4.5, 1.5.1, 1.6.0 >Reporter: Sean Busbey >Priority: Minor > Labels: upgrade > > updateAccumuloVersion does a non-atomic update of the persisted version, and > getPersistentVersion relies on the ordering of the underlying fs/Volume to > determine which version to return. > We should document what happens when there's a partial failure, improve > reporting in it to the operator, and document how to fix it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2607) Add "list tables" flag to LogReader
[ https://issues.apache.org/jira/browse/ACCUMULO-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2607. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Add "list tables" flag to LogReader > --- > > Key: ACCUMULO-2607 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2607 > Project: Accumulo > Issue Type: Improvement > Components: logger >Affects Versions: 1.4.5, 1.5.1, 1.6.0 >Reporter: Sean Busbey >Priority: Minor > Labels: newbie > > Add a flag to LogReader that has it just print out the list of tables that > have entries in a passed WAL. > I was doing an upgrade test for ACCUMULO-2519, and making sure I had WALs > needed to recover the !METADATA table would have been much easier with this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2772) Bulk Import of incorrect RFile version should give error message that explains incompatibility
[
https://issues.apache.org/jira/browse/ACCUMULO-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-2772.
-
Resolution: Abandoned
Closing old inactive issue; if this is still a problem, please create a new
issue or pull request at https://github.com/apache/accumulo
> Bulk Import of incorrect RFile version should give error message that
> explains incompatibility
> --
>
> Key: ACCUMULO-2772
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2772
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.5.0, 1.6.0
>Reporter: Sean Busbey
>Priority: Minor
> Labels: error-feedback, logging
>
> In ACCUMULO-1079 this comment showed incorrect error handling
> {quote}
> David Medinets added a comment - 19 minutes ago
> ~[busbey] the error did not mention rfile version mismatch. There was no
> error at the command line. The rfile was moved to the failures directory and
> the 'unable to find tablets that overlap file' message was in the logs.
> {quote}
> We should make sure that we correctly report RFile mismatches during bulk
> load attempts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4560) Create status API for Master
[ https://issues.apache.org/jira/browse/ACCUMULO-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4560. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Create status API for Master > > > Key: ACCUMULO-4560 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4560 > Project: Accumulo > Issue Type: Sub-task > Components: master >Reporter: Luis Tavarez >Priority: Minor > > Create status API for Master, to include Accumulo version of master. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1512) Provide example using KeyRangePartitioner
[ https://issues.apache.org/jira/browse/ACCUMULO-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1512. - Resolution: Abandoned Closing old inactive issue; if this is still a problem, please create a new issue or pull request at https://github.com/apache/accumulo > Provide example using KeyRangePartitioner > - > > Key: ACCUMULO-1512 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1512 > Project: Accumulo > Issue Type: Improvement >Affects Versions: 1.4.1, 1.5.0, 1.6.0 >Reporter: David Medinets >Priority: Minor > Labels: Documentation > > The KeyRangePartitioner class lets mappers output Accumulo Key objects > instead of Text. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1543) tablets without WALs can be loaded in parallel
[ https://issues.apache.org/jira/browse/ACCUMULO-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1543. - Resolution: Duplicate Closing as duplicate of https://github.com/apache/accumulo/pull/4429 > tablets without WALs can be loaded in parallel > -- > > Key: ACCUMULO-1543 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1543 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: Eric C. Newton >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Clean tablets can be loaded in parallel. There are concerns that loading > tablets that require replay of mutations from WALs would require too much > memory (ACCUMULO-1085). However, tablets that do not refer to WALs can be > loaded quickly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1949) Tablet server does not track tablet memory usage during recovery
[ https://issues.apache.org/jira/browse/ACCUMULO-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1949. - Resolution: Not A Problem As far as I can tell, this doesn't seem to have been a problem since this issue was created. Closing this stale issue. If this is still a desired feature, please open a new issue or PR at https://github.com/apache/accumulo > Tablet server does not track tablet memory usage during recovery > > > Key: ACCUMULO-1949 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1949 > Project: Accumulo > Issue Type: Bug >Reporter: Keith Turner >Priority: Major > > It seems like the tablet server does not track memory usage of a tablet thats > recovering. The following is an example situation I am thinking of. > # Tablet server is hosting tablets that are using a alot of memory > # Tablet server is asked to load a tablet the needs recovery > # Tablet server recovers tablet and load a lot of data into memory, this > exceeds amount of memory tablet server should use. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2431) Allow localization of errors and shell
[ https://issues.apache.org/jira/browse/ACCUMULO-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2431. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Allow localization of errors and shell > -- > > Key: ACCUMULO-2431 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2431 > Project: Accumulo > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Minor > Labels: i18n, localization > > It would be nice to allow localization of commands and error messages for > non-english speakers. > Initial implementation for this ticket just needs to: > * add an articulation point in the code for localization > * move existing text into an EN-US implementation > Future Implementer, please try to use an existing i18n framework rather than > rolling our own. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-483) Create purge locality utility
[ https://issues.apache.org/jira/browse/ACCUMULO-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-483. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Create purge locality utility > - > > Key: ACCUMULO-483 > URL: https://issues.apache.org/jira/browse/ACCUMULO-483 > Project: Accumulo > Issue Type: New Feature > Components: client >Reporter: John Vines >Priority: Minor > Labels: newbie > > For some high capacity ingest, the desired path is to do some pre-splits, > bulk import, and then let it naturally split down the rest of the way. If all > of the pre-split tablets made split evenly, then the system will have > continuous ranges bundled together on tservers. This poor distribution can > impact performance depending on the operations performed. This could be > handled in the load balancer, but it could be tricky. You don't want to > randomly reassign tablets with any sort of frequency. Rather, you want to do > a one-time operation in doing so. Given the initial assignment code is a bit > random (needs to be validated), this could easily be done by offlining a > table, purging all location records for it from the !METADATA table, and > bringing it back online. The balancer will assign the table randomly, at > which point the user could force a major compaction to restablish locality > (as well some permanence in tablet assignment). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-594) Show per table plots
[ https://issues.apache.org/jira/browse/ACCUMULO-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-594. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Show per table plots > > > Key: ACCUMULO-594 > URL: https://issues.apache.org/jira/browse/ACCUMULO-594 > Project: Accumulo > Issue Type: New Feature > Components: monitor >Reporter: Keith Turner >Priority: Trivial > Labels: gsoc2013, mentor > > The monitor overview page plots accumulo performance data over time for all > tables. I would like to be able to see this data over time per table (and > maybe per tablet server). Could add this to the monitor page. Eric > suggested maybe using OpenTSDB to collect and view this data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2797) Localization for non-english
[ https://issues.apache.org/jira/browse/ACCUMULO-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2797. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Localization for non-english > > > Key: ACCUMULO-2797 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2797 > Project: Accumulo > Issue Type: Improvement > Components: client, docs >Reporter: Sean Busbey >Priority: Minor > Labels: i18n > > This ticket will probably need subtasks. I'd like to improve our support for > non-English. > * Docs would be a big step. English. Before I start pushing to get > translations done, it would be nice if our build process already had the > means to handle multiple languages. > * Client API messages should have pluggable message langauges > * Server log messages as well > Those last two will probably require finding some kind of library to build > on. Or we could do something simple with Properties and the Java Services API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3418) create a microbenchmark for tablet splitting
[ https://issues.apache.org/jira/browse/ACCUMULO-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3418. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > create a microbenchmark for tablet splitting > > > Key: ACCUMULO-3418 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3418 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Affects Versions: 1.7.0 >Reporter: Sean Busbey >Priority: Minor > > It would be nice if we had a targeted benchmark that could easily be run to > see the impact of optimization choices on the time it takes to complete > tablet splits. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3943) volumn definition agreement with default settings
[ https://issues.apache.org/jira/browse/ACCUMULO-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3943. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > volumn definition agreement with default settings > - > > Key: ACCUMULO-3943 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3943 > Project: Accumulo > Issue Type: Bug > Components: gc, master, tserver >Reporter: Eric C. Newton >Priority: Minor > > I was helping a new user trying to use Accumulo. They managed to set up HDFS, > running on hdfs://localhost:8020. But they didn't set it up with specific > settings, and just used the default port. Accumulo worked initially, but > would not allow a bulk import. > During the bulk import process, the servers need to move the files into the > accumulo volumes, but keeping the volume the same. This makes the move > efficient, since nothing is copied between namespaces. In this case it > refused the import because it could not find the correct volume. > Accumulo needs to be more nuanced when comparing hdfs://localhost:8020, and > hdfs://localhost. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4552) Create abstract class to encapsulate ZK lock logic
[ https://issues.apache.org/jira/browse/ACCUMULO-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4552. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo (Note: I think this exists now, as ServiceLock` > Create abstract class to encapsulate ZK lock logic > -- > > Key: ACCUMULO-4552 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4552 > Project: Accumulo > Issue Type: Sub-task > Components: monitor >Reporter: Luis Tavarez >Priority: Minor > > [~elserj] > [commented|https://github.com/apache/accumulo/pull/195#issuecomment-270422597] > on Github that we could create a common abstract class which encapsulates > the logic of acquiring a ZK lock. This would reduce the duplication of > obtaining a lock before the services start. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-400) continuous random walk execution
[ https://issues.apache.org/jira/browse/ACCUMULO-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-400. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > continuous random walk execution > > > Key: ACCUMULO-400 > URL: https://issues.apache.org/jira/browse/ACCUMULO-400 > Project: Accumulo > Issue Type: Improvement >Reporter: Adam Fuchs >Priority: Major > > Random walk is finding bugs like a boss, but we can anticipate future usage > in which the current setup will be limiting. In particular, with a larger > development team knocking off bugs and writing new tests we might get to the > point where the most obvious bug is the only one that we find in a given run > of all of the random walkers. Consider hundreds of random walkers walking > over all of the tests. Many of these tests will find bugs > non-deterministically. If we add one test that finds one bug with high > probability, all of the walkers will find that bug and halt. None of the > other bugs will be found until the one bug is fixed or the test is removed. > Here are some things we could do to improve this situation and migrate to > more of a continual random walk setup: > 1. Stop executing a test after some number of walkers have found a bug when > running it. > 2. Store the random walk graph in a database and have the walkers re-query it > with some regularity. This will let us add new tests to running walkers. > 3. Have the walkers snapshot the relevant parts of the overall system when > they find a bug. We currently rely on the walkers halting to preserve the > state of the system so that we can manually extract all of the relevant > details that may have led to the bug. Dynamically snapshotting the system > makes it possible to continue to run tests without rolling over logs and > forensic information. Exactly what information needs to be kept is TBD. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-425) tservers should fail into read only mode
[ https://issues.apache.org/jira/browse/ACCUMULO-425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-425. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo (Note: this could be done with a user-defined Constraint) > tservers should fail into read only mode > > > Key: ACCUMULO-425 > URL: https://issues.apache.org/jira/browse/ACCUMULO-425 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: John Vines >Priority: Major > > Aaron came up with the thought to have tservers fail into a read only mode. > Perhaps once hdfs reaches X percent full, all writes are blocked until space > improves. There's potential here, but it may involve some work to be optimal. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-549) Create infrastructure that supports debugging.
[ https://issues.apache.org/jira/browse/ACCUMULO-549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-549. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Create infrastructure that supports debugging. > -- > > Key: ACCUMULO-549 > URL: https://issues.apache.org/jira/browse/ACCUMULO-549 > Project: Accumulo > Issue Type: New Feature >Reporter: Keith Turner >Priority: Major > > Before a release an extensive amount of testing is done that usually involves > running test like continuous ingest and random walk on a cluster for extended > periods of time. When bugs are found by these test it can take a lot of time > to track the issue down sometimes. Inorder to make tracking these issues > down easier the write ahead logs are archived. These walogs archives make it > possible to answer questions about a tablets history because everything ever > written to the metadata table is there. It would be nice to always have this > capability on an accumulo system, and have it be easy to use. Spelunking > around in the write ahead logs is not an easy task. > It would be nice if accumulo could answer questions like the following. > > * Where has a tablet been assigned > * What compactions has a tablet done > * What split or merge created a tablet > These questions can currently be answered with walogs and log4j logs, but its > painful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1737) MultiTableBatchScanner
[ https://issues.apache.org/jira/browse/ACCUMULO-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1737. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > MultiTableBatchScanner > -- > > Key: ACCUMULO-1737 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1737 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: John Vines >Priority: Major > > We currently have the BatchScanner, which does not guarantee in order results > with the benefit of higher parallellization. Unfortunately, if you want to > query multiple tables, the only option is to chain BatchScanner's together, > but you lose something of the optimizations under the hood for both using > system resources well as well as getting data back in order. We have another > ticket for multi-table accumulo input format for mapreduce, but I would like > to see a multi table version of a batch scanner for regular client use. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-902) Have a common resource pool for minor and major compactions
[ https://issues.apache.org/jira/browse/ACCUMULO-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-902. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo With compaction strategies and external compactions and compaction queues, this feature may not make sense anymore. > Have a common resource pool for minor and major compactions > --- > > Key: ACCUMULO-902 > URL: https://issues.apache.org/jira/browse/ACCUMULO-902 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: John Vines >Priority: Major > > Currently we have a defined threadpool for minor and major compactions, > independent of one another. However, there are situations where a system may > be minor compaction heavy with no major, or vice versa. I would like to see a > common threadpool which is accessible to both operations for work to be done, > with guarantees for certain resources to be available to the other type of > work. That is, it should be a defined pool size with a (configurable) minimum > of resources maintained for the other to maintain a certain QoS. Of course, > major is heavier than minor, so some weighting of operations needs to be done > to keep workloads reasonable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2147) Create test to verify mappers are running locally
[ https://issues.apache.org/jira/browse/ACCUMULO-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2147. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Create test to verify mappers are running locally > - > > Key: ACCUMULO-2147 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2147 > Project: Accumulo > Issue Type: Improvement >Reporter: Keith Turner >Priority: Major > > Map task that read from an Accumulo tablet should run on the node where that > tablet is hosted. We need a test that verifies that this is working. One > possible way of doing this is to add something to the continuous ingest > verify step. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-255) Improve Iterator Debugging
[ https://issues.apache.org/jira/browse/ACCUMULO-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-255. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Improve Iterator Debugging > -- > > Key: ACCUMULO-255 > URL: https://issues.apache.org/jira/browse/ACCUMULO-255 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: John Vines >Priority: Major > > There is currently little to no information provided to end users when there > is a failure in custom iterators. We do not want to provide the full stack > trace for security purposes, but that doesn't mean we can't provide some > error information like error name (particularly ClassNotFoundException) or > any exception which stems directly from the written iterator. Or we could > have it return the partial stack trace from just within the iterator itself. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-272) Prefer local tablet server for client operations
[ https://issues.apache.org/jira/browse/ACCUMULO-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-272. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Prefer local tablet server for client operations > > > Key: ACCUMULO-272 > URL: https://issues.apache.org/jira/browse/ACCUMULO-272 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: Keith Turner >Priority: Major > > Accumulo client code will pick a random tablet server for certain operations > (like security operations such checking permissions). This code should > choose a local tablet server if there is one, otherwise chose a random tablet > server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3374) master should take action if tablets are not loaded in a reasonable time frame
[ https://issues.apache.org/jira/browse/ACCUMULO-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3374. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > master should take action if tablets are not loaded in a reasonable time frame > -- > > Key: ACCUMULO-3374 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3374 > Project: Accumulo > Issue Type: Improvement > Components: master >Reporter: Eric C. Newton >Priority: Major > > The master should warn if a tablet assignment is not picked up by the tablet > server in a reasonable timeframe. The monitor does display offline tablets > as a "red" condition, but if an assignment isn't processed in a reasonable > time frame, a bigger warning, and identification of the tablet and server > would be helpful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4107) Need metrics on conditional mutations
[ https://issues.apache.org/jira/browse/ACCUMULO-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4107. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Need metrics on conditional mutations > - > > Key: ACCUMULO-4107 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4107 > Project: Accumulo > Issue Type: Improvement >Reporter: Keith Turner >Priority: Major > > Tracking down uneven load in Fluo applications would be much easier if > Accumulo reported some per tablet server conditional mutation metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2614) take advantage of block placement to reduce MTTR
[ https://issues.apache.org/jira/browse/ACCUMULO-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2614. - Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > take advantage of block placement to reduce MTTR > > > Key: ACCUMULO-2614 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2614 > Project: Accumulo > Issue Type: Improvement > Components: logger, tserver >Reporter: Sean Busbey >Priority: Major > Labels: recovery > > We should look at the block placement for WALs that need to be recovered and > then attempt to have the recovery task run on a tserver that is a minimum > distance from the set. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-577) Allow option for ignoring repeatedly failed iterators during compaction
[ https://issues.apache.org/jira/browse/ACCUMULO-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-577. Resolution: Abandoned Closing this old issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo > Allow option for ignoring repeatedly failed iterators during compaction > --- > > Key: ACCUMULO-577 > URL: https://issues.apache.org/jira/browse/ACCUMULO-577 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Affects Versions: 1.4.0, 1.4.1 >Reporter: John Vines >Priority: Major > > Currently when an iterator is causing errors, either stemming from the > iterator configuration or from instantiating, it will endlessy try over and > over again. I think we should provide a table configuration to allow eventual > fail over for these circumstances. I think in these circumstances all user > configured iterators (including versioning) should be disabled to ensure the > data is written (or rewritten) and once the configuration is fixed, then the > next major compaction will sort it all out. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4492) operations tool to migrate existing users when enabling Kerberos
[
https://issues.apache.org/jira/browse/ACCUMULO-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-4492.
-
Resolution: Abandoned
Closing this old issue. If this is still a problem, please open a new issue or
PR at https://github.com/apache/accumulo
> operations tool to migrate existing users when enabling Kerberos
>
>
> Key: ACCUMULO-4492
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4492
> Project: Accumulo
> Issue Type: New Feature
>Reporter: Sean Busbey
>Priority: Major
>
> When converting an existing cluster to use Kerberos, existing user
> permissions aren't much use unless the user names happen to be formatted like
> Kerberos principals.
> An offline tool that folks migrating can use to map existing user names to
> principals would be super useful.
> Essentially something like:
> {code}
> $ accumulo kerberos-migration --include-users=* --exclude-users=root
> --no-instance --realm=EXAMPLE.COM
> Migrating users matching '*' and not matching 'root'.
> User principals will not have an instance component.
> User principals will be in the realm 'EXAMPLE.COM'
> Found user 'auser', converted to 'au...@example.com'
> Found user 'another_user', converted to 'another_u...@example.com'
> Found user 'hpnewton', converted to 'hpnew...@example.com'
> Found user 'root', skipped due to exclusion rule
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4491) operations tool to enable the administrative user in kerberos installations w/o security reset
[
https://issues.apache.org/jira/browse/ACCUMULO-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-4491.
-
Resolution: Abandoned
Closing this old issue. If this is still a problem, please open a new issue or
PR at https://github.com/apache/accumulo
> operations tool to enable the administrative user in kerberos installations
> w/o security reset
> --
>
> Key: ACCUMULO-4491
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4491
> Project: Accumulo
> Issue Type: Improvement
>Affects Versions: 1.7.1, 1.7.2, 1.8.0
>Reporter: Sean Busbey
>Priority: Critical
>
> Right now converting an existing cluster to use Kerberos requires using the
> {{accumulo init --reset-security}} tool in order to replace the "root" user
> with an appropriate administrative user principal. This has the side effect
> of dumping existing user permission information.
> That means downstream folks have to use the config dumping tools or the like
> in order to save existing permissions if they want to refer to them later
> while setting up kerberos users.
> It'd be preferable for us to have a cli tool to set the administrative user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1068) Improve Javadoc for Instance Operations
[ https://issues.apache.org/jira/browse/ACCUMULO-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1068. - Resolution: Duplicate Duplicated by https://github.com/apache/accumulo/pull/3188 > Improve Javadoc for Instance Operations > --- > > Key: ACCUMULO-1068 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1068 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: Jim Klucar >Priority: Minor > Labels: newbie > Time Spent: 10m > Remaining Estimate: 0h > > The difference between InstanceOperations.getSiteConfiguration and > getSystemConfiguration could be highlighted more clearly in the Javadoc. Also > how they relate to getProperty and setProperty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3277) Document vulnerability reporting
[ https://issues.apache.org/jira/browse/ACCUMULO-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3277. - Resolution: Duplicate Closing as duplicate of https://github.com/apache/accumulo-website/pull/372 > Document vulnerability reporting > > > Key: ACCUMULO-3277 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3277 > Project: Accumulo > Issue Type: Task > Components: docs >Reporter: Sean Busbey >Priority: Critical > Labels: website > Time Spent: 0.5h > Remaining Estimate: 0h > > We currently have neither a security disclosure page nor instructions on how > people should report vulnerabilities to us. > secur...@apache.org will already use our private@accumulo list for forwarding > issues to us. Presuming we don't want to create security@accumulo, we should > advertise that private@ is the preferred way for others to contact us. > in addition to a dedicate page, a reference on the mailing lists page would > be a good idea (since it's the closest thing we have to a 'contact us' page). > ref: [ASF guide|http://www.apache.org/security/committers.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1245) firewall a bad server
[ https://issues.apache.org/jira/browse/ACCUMULO-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1245. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > firewall a bad server > - > > Key: ACCUMULO-1245 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1245 > Project: Accumulo > Issue Type: Bug > Components: master >Reporter: Eric C. Newton >Priority: Major > > Tablet servers that lose their locks don't always kill themselves. In one > case, the lock is lost, detected, but the class that is supposed to execute > fails because it cannot be loaded from the file system. Other threads > continue to run, unaware that they are no longer part of a valid server. > Consider firewalling, somehow, this rogue server from the rest of the > cluster. Perhaps it can be an external utility that will power down the > node, or actually set up firewall rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1207) document the stats available via JSON/xml/getMasterStats
[ https://issues.apache.org/jira/browse/ACCUMULO-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1207. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > document the stats available via JSON/xml/getMasterStats > > > Key: ACCUMULO-1207 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1207 > Project: Accumulo > Issue Type: Bug > Components: docs >Reporter: Eric C. Newton >Priority: Major > > Document each of the values, in particular "scans" and "scanssessions" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-361) Need utility to help recover from common problems
[ https://issues.apache.org/jira/browse/ACCUMULO-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-361. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Need utility to help recover from common problems > - > > Key: ACCUMULO-361 > URL: https://issues.apache.org/jira/browse/ACCUMULO-361 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: Keith Turner >Priority: Major > > Need a utility like fsck to automatically recover from issues that occur > sometimes. This could be a shell command. > * detect and fix missing files > * detect and fix files with missing or bad blocks > This allows users to make progress even though data was lost. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1450) TServers should provide better diagnostic information when running low on memory
[ https://issues.apache.org/jira/browse/ACCUMULO-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1450. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > TServers should provide better diagnostic information when running low on > memory > > > Key: ACCUMULO-1450 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1450 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Affects Versions: 1.4.3, 1.5.0 >Reporter: Mike Drob >Priority: Major > > When a tserver runs out of memory, there is not enough information available > regarding the state that led to the problem. It would be good to get more > from it when it is running low on memory but before it falls over so that the > problem can be diagnosed and addressed in the future. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1666) Provide a way to declare iterator dependencies on other iterators
[ https://issues.apache.org/jira/browse/ACCUMULO-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1666. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Provide a way to declare iterator dependencies on other iterators > - > > Key: ACCUMULO-1666 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1666 > Project: Accumulo > Issue Type: Improvement >Reporter: Brian Loss >Priority: Major > > Sometimes iterators really only make sense when they are used in conjunction > with other iterators, but it doesn't make sense to implement the > functionality in a single iterator. It would be nice if Accumulo had a way > to declare that an iterator depends on another iterator or iterators. > Perhaps this could be done by annotating the iterator class, and then when > the tablet server adds an iterator to the stack, it could ensure that the > dependent iterators are already on the stack and throw an error if not. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1816) track metrics during scan session
[
https://issues.apache.org/jira/browse/ACCUMULO-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-1816.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> track metrics during scan session
> -
>
> Key: ACCUMULO-1816
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1816
> Project: Accumulo
> Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Mike Drob
>Priority: Major
>
> It would be useful to track and display certain metrics on scan sessions.
> Initially, I would be interested in seeing bytes read, bytes returned,
> entries read, entries returned, and seeks performed by a scan session.
> This information should be available through the {{listscans}} command. Some
> of it is currently on the monitor, but aggregated and not useful for
> debugging.
> Providing that information may be a security concern because it suggests the
> existence of rows that a user cannot see, but I think if we hide the command
> behind some permission then operators can limit it to authorized folks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-415) support iterator profiles
[ https://issues.apache.org/jira/browse/ACCUMULO-415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-415. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > support iterator profiles > - > > Key: ACCUMULO-415 > URL: https://issues.apache.org/jira/browse/ACCUMULO-415 > Project: Accumulo > Issue Type: Improvement >Reporter: John Vines >Priority: Major > > Currentlyif a user wants to have multiple iterator configurations for a > table, they must be individually set on the scanner. This makes it difficult, > so we should provide the ability for a table to have profiles. By default we > should also support a naked profile for use by client side deletes so > transformed keys do not create problems. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-359) Need to create automatic performance tests
[ https://issues.apache.org/jira/browse/ACCUMULO-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-359. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Need to create automatic performance tests > -- > > Key: ACCUMULO-359 > URL: https://issues.apache.org/jira/browse/ACCUMULO-359 > Project: Accumulo > Issue Type: New Feature > Components: test >Reporter: John Vines >Priority: Major > > We have many tests for validating functionality of the system, but we lack > any standard tests for any form of benchmarking. We should create appropriate > performance tests in order to test performance differences between versions, > impacts from code changes, and for just generally charting our growth as a > project. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2843) Apache Crunch integration
[ https://issues.apache.org/jira/browse/ACCUMULO-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2843. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Apache Crunch integration > - > > Key: ACCUMULO-2843 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2843 > Project: Accumulo > Issue Type: Wish > Components: client >Affects Versions: 1.7.0 >Reporter: Corey J. Nolet >Priority: Major > Labels: crunch, mapreduce > > Similar to Pig, this will probably be as easy as filling in their custom > input format wrapper to return PCollections from Accumulo Key/Value objects. > It would be really useful if this was made generic like the HBase integration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2859) Refactor GarbageCollectionAlgorithm et al. to add unit tests
[ https://issues.apache.org/jira/browse/ACCUMULO-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2859. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Refactor GarbageCollectionAlgorithm et al. to add unit tests > > > Key: ACCUMULO-2859 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2859 > Project: Accumulo > Issue Type: Test > Components: gc, test >Affects Versions: 1.6.0 >Reporter: Bill Havanki >Assignee: Sean Kane >Priority: Major > Labels: gc, testing > > Recent investigation and testing of the garbage collector by [~busbey] and > [~mdrob] (in the context of upgrades) showed that the garbage collector could > use some (more) unit tests to verify its behavior. This ticket is to perform > refactorings, and add minor improvements, needed to add more unit tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-712) InputFormat that supports more than one tablet per input split if on the same tablet server
[ https://issues.apache.org/jira/browse/ACCUMULO-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-712. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > InputFormat that supports more than one tablet per input split if on the same > tablet server > --- > > Key: ACCUMULO-712 > URL: https://issues.apache.org/jira/browse/ACCUMULO-712 > Project: Accumulo > Issue Type: New Feature > Components: client >Reporter: Jim Donofrio >Priority: Major > > Similar to CombineFileInputFormat for hdfs files, there should be an > InputFormat that supports more than one tablet per input split if on the same > tablet server. This should also work with ACCUMULO-391 to allow adding > tablets from different compatible tables into a single input split. This > allows mappers that finish very quickly to have more work to do. This could > be a new InputFormat that extends AccumuloInputFormat or an option added to > AccumuloInputFormat. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1675) Accumulo maven plugin should be skip-able with a property
[ https://issues.apache.org/jira/browse/ACCUMULO-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1675. - Resolution: Works for Me Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo-maven-plugin > Accumulo maven plugin should be skip-able with a property > - > > Key: ACCUMULO-1675 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1675 > Project: Accumulo > Issue Type: Improvement > Components: maven-plugin >Affects Versions: 1.6.0 >Reporter: Keith Turner >Priority: Major > > I am using the accumulo-maven-plugin to test an Accumulo project I am working > on. When I run "mvn install -DskipTests" it starts and stops mini accumulo. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1758) Implement temporary tables
[ https://issues.apache.org/jira/browse/ACCUMULO-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1758. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Implement temporary tables > -- > > Key: ACCUMULO-1758 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1758 > Project: Accumulo > Issue Type: Improvement >Reporter: Michael Allen >Priority: Major > > It'd be useful to be able to set an Accumulo table to expire in a certain > period of time or a certain period from the last activity upon it. This > feature will facilitate things like query specific searches that require > building up a set of data but then not needing it past the lifetime of a > session. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2933) Modify ContinuousIngest to allow multiple columns per row
[ https://issues.apache.org/jira/browse/ACCUMULO-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2933. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo-testing > Modify ContinuousIngest to allow multiple columns per row > - > > Key: ACCUMULO-2933 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2933 > Project: Accumulo > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Priority: Major > > Not sure what this would necessarily look like, but It might be good to allow > continuous ingest to write multiple columns to the same row. The number of > columns would, ideally, be configurable. > This was inspired by the discussion around ACCUMULO-2827. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2868) Make master configurable in when it kills tablet servers
[
https://issues.apache.org/jira/browse/ACCUMULO-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-2868.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> Make master configurable in when it kills tablet servers
>
>
> Key: ACCUMULO-2868
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2868
> Project: Accumulo
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.6.0
>Reporter: Bill Havanki
>Priority: Major
> Labels: admin, configuration, master
>
> On a cluster with a flaky network, the master may be unable to contact a
> tserver for some moderate amount of time and then direct it to terminate,
> even though the tserver is still up. (See {{gatherTableInformation()}} and
> {{StatusThread}}. It does not appear possible to configure the master to be
> more forgiving in these checks. Relevant constants:
> * {{DEFAULT_WAIT_FOR_WATCHER}} - interval between server checks
> * {{MAX_BAD_STATUS_COUNT}} - the maximum number of failed attempts allowed
> before killing the tserver
> Making one or both of those configurable, or some other pertinent parameter
> configurable, would allow cluster admins to cope with mild network maladies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3044) ConditionalMutation should support conditional updates
[
https://issues.apache.org/jira/browse/ACCUMULO-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-3044.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> ConditionalMutation should support conditional updates
> --
>
> Key: ACCUMULO-3044
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3044
> Project: Accumulo
> Issue Type: Improvement
> Components: client, tserver
>Affects Versions: 1.6.0
>Reporter: Ryan Leary
>Priority: Major
>
> Currently, the ConditionalMutation object operates on a single row, and
> applies updates if and only if a set of Conditions are met.
> A new type of ConditionalMutation which operates on a single row and applies
> an update if and only if a set of Conditions for that particular update would
> be more flexible.
> This behavior is possible currently by creating a new ConditionalMutation for
> each update. In the case where there are a large number of updates for a
> single row, however, there is a steep penalty paid due to the row-level
> locking. Another acceptable solution to this ticket would be optimizing the
> ConditionalWriter to apply multiple conditional mutations on the same row
> within the same atomic lock. The order of execution would need to be
> guaranteed.
> The API would look something like:
> {code}
> ConditionalMutation cm = new ConditionalMutation(rowKey);
> cm.put(Iterable, CF, CQ, CV, Val);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3083) capability balancing
[
https://issues.apache.org/jira/browse/ACCUMULO-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-3083.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> capability balancing
>
>
> Key: ACCUMULO-3083
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3083
> Project: Accumulo
> Issue Type: New Feature
> Components: master
>Reporter: Eric C. Newton
>Priority: Major
>
> I was thinking about balancing, in much the way that new container-based
> (yarn, mesos, etc) resource managers work. The existing {{Balancers}} try to
> keep the number of tablets even over presumably homogeneous nodes.
> But as clusters age and are improved and expanded in piecemeal, the hardware
> capabilities will drift apart. Some nodes may be capable of handling more
> load than others. We may even want some nodes to have a trivial number of
> tablets so the data they serve is always cached.
> Tablet servers could advertise an attribute (or several). Really, it would
> just be a name, like {{fatnode}} or {{metaonly}}. These would then be used
> by the {{Balancer}} to favor or ignore tservers for balancing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3345) Redundant logging on failed multi-scans
[ https://issues.apache.org/jira/browse/ACCUMULO-3345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3345. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Redundant logging on failed multi-scans > --- > > Key: ACCUMULO-3345 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3345 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: John Vines >Priority: Major > > When a multi scan fails, for whatever reason, it gets double logged. This > makes it harder to track down. We should settle on the logging coming from > one location. > The locations in question are- > TabletServer.LookupTask.run, there's a catch Throwable that log.warns > "exception while doing multi-scan " with the exception. > Then higher in the stack under > TabletServer.ThriftClientHandler.continueMultiScan, there's another catch > Throwable that log.warns "Failed to get multiscan result" > Then higher in the stack under thrift in > ProcessFunction.process, there is another catch Throwable. This log.error > "internal error processing *" and logs the exception. > We should > A. settle on multi-scan or multiscan or something else > B. decide which one we prefer > C. remove/mute the other ones in our default logging configuration -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3095) Document Backup and Recovery
[ https://issues.apache.org/jira/browse/ACCUMULO-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3095. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo-website > Document Backup and Recovery > > > Key: ACCUMULO-3095 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3095 > Project: Accumulo > Issue Type: Bug > Components: docs >Reporter: Dave Marion >Priority: Major > > Document best practices using existing Hadoop and Accumulo capabilities for > backup and recovery, to include: > normal instance recovery process > how to create a backup of Accumulo (what / where / how) > how to recover Accumulo from a backup -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3412) Consistent application of runtime configuration changes
[ https://issues.apache.org/jira/browse/ACCUMULO-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3412. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Consistent application of runtime configuration changes > --- > > Key: ACCUMULO-3412 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3412 > Project: Accumulo > Issue Type: Improvement > Components: zookeeper >Affects Versions: 1.7.0 >Reporter: Sean Busbey >Priority: Major > > Right now changes to runtime configurable properties are lazily applied > across the cluster. Define stricter rules about when rules are in affect. > * Minimal goal: allow for client notification when a configuration change has > been applied across the cluster. > * Stretch: do something fancy so that we have consistent application across > the cluster (e.g. either all tablet servers are applying a constraint to a > table or none are) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3530) alterTable/NamespaceProperty should use Fate locks
[ https://issues.apache.org/jira/browse/ACCUMULO-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3530. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > alterTable/NamespaceProperty should use Fate locks > -- > > Key: ACCUMULO-3530 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3530 > Project: Accumulo > Issue Type: Bug >Reporter: John Vines >Priority: Major > > Fate operations, such as clone table, have logic in place to ensure > consistency as the operation occurs. However, operaitons like > alterTableProperty can still interfere because there is no locking done. We > should add identical locking to these methods in MasterClientServiceHandler > to help ensure consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3156) Document full release process
[ https://issues.apache.org/jira/browse/ACCUMULO-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3156. - Resolution: Works for Me A lot of this information, if not all of it, is now on the website. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Document full release process > - > > Key: ACCUMULO-3156 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3156 > Project: Accumulo > Issue Type: Task >Reporter: Corey J. Nolet >Assignee: Corey J. Nolet >Priority: Major > > This should apply both to people who are familiar with the release process > but want a quick checklist or people who are new to the release process and > have never done it before. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3648) Ability to Scan Table Keys in Reverse Order
[ https://issues.apache.org/jira/browse/ACCUMULO-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3648. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Ability to Scan Table Keys in Reverse Order > --- > > Key: ACCUMULO-3648 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3648 > Project: Accumulo > Issue Type: New Feature > Components: master, tserver >Reporter: Mike Fagan >Priority: Major > Labels: features, performance > > There are cases where you would like to scan a table in reverse key order. > For example: > Consider a table using an inverted timestamp key. > This allows for very quick access to the latest records. > But It would be helpful to quickly access earliest records without having to: > A) scan through all the records to build a collection to reverse sort > B) store the table again using the reverse key -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4055) Add test in ConditionalWriterIT for setting context name on Scanner
[ https://issues.apache.org/jira/browse/ACCUMULO-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4055. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Add test in ConditionalWriterIT for setting context name on Scanner > --- > > Key: ACCUMULO-4055 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4055 > Project: Accumulo > Issue Type: Task > Components: test >Affects Versions: 1.8.0 >Reporter: Dave Marion >Priority: Major > > Create a clone of ConditionalWriterIT.testIterators() that loads the > iterators from a context not defined on a table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3980) Create a new WAL when the number of replicas is less than we expect
[ https://issues.apache.org/jira/browse/ACCUMULO-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3980. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Create a new WAL when the number of replicas is less than we expect > --- > > Key: ACCUMULO-3980 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3980 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: Eric C. Newton >Assignee: Eric C. Newton >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > As suggested by [~elserj] in ACCUMULO-3976, we could roll the WALog when the > number of replicas drops below the desired replica count. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4054) Add optional context name to TableOperations.testClassLoad
[ https://issues.apache.org/jira/browse/ACCUMULO-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4054. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Add optional context name to TableOperations.testClassLoad > -- > > Key: ACCUMULO-4054 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4054 > Project: Accumulo > Issue Type: Task >Affects Versions: 1.8.0 >Reporter: Dave Marion >Priority: Major > > Scanner.setContext() allows a user to set a context name that will override > the context set on a table. We could fail fast at setup time if we could > check the existence of the iterator in that context on the server side. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1619) Provide a way to override parent configuration with a delete
[ https://issues.apache.org/jira/browse/ACCUMULO-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1619. - Resolution: Won't Fix Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Provide a way to override parent configuration with a delete > > > Key: ACCUMULO-1619 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1619 > Project: Accumulo > Issue Type: Bug >Reporter: Christopher Tubbs >Priority: Major > > Per-table configuration options are inherited from a hierarchy of > configuration scopes. In some cases, it would be useful to denote a key as > "do not inherit from the parent" (treat it as though it doesn't exist, > possibly falling back on the default if the property is defined in the > default configuration). > In such cases, a property key could be prefixed with a "!" character (as in > "NOT") to "unset" a key that is set with a value in a parent scope. > This ticket removes the previously undocumented restriction that empty string > is the same as a property not being set at all. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4561) Crash when using ping on a non-existing server
[
https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-4561.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> Crash when using ping on a non-existing server
> --
>
> Key: ACCUMULO-4561
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4561
> Project: Accumulo
> Issue Type: Bug
> Components: shell
>Reporter: Luis Tavarez
>Priority: Major
>
> While working on ACCUMULO-4558, I tried running
> {code}ping -ts localhost:9995{code} (localhost:9995 does not have a a tserver
> on my setup.)
> And it caused the shell to exit (crashed) and show the following message.
> {code}#
> # java.lang.OutOfMemoryError: Java heap space
> # -XX:OnOutOfMemoryError="kill -9 %p"
> # Executing /bin/sh -c "kill -9 25561"...
> /home/lmtavar/git/uno/bin/uno: line 44: 25561 Killed
> "$ACCUMULO_HOME"/bin/accumulo shell -u "$ACCUMULO_USER" -p
> "$ACCUMULO_PASSWORD" "${@:2}"
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1991) listiter command in shell should tell you which configuration source the iterator is coming from
[ https://issues.apache.org/jira/browse/ACCUMULO-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1991. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > listiter command in shell should tell you which configuration source the > iterator is coming from > > > Key: ACCUMULO-1991 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1991 > Project: Accumulo > Issue Type: Improvement >Reporter: Chris McCubbin >Priority: Major > > namespace, table, systemwide, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2164) minicluster should accept properties via command line
[ https://issues.apache.org/jira/browse/ACCUMULO-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2164. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > minicluster should accept properties via command line > - > > Key: ACCUMULO-2164 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2164 > Project: Accumulo > Issue Type: Bug > Components: mini >Reporter: John Vines >Priority: Major > > bin/accumulo minicluster is an easy way to start up a mini cluster. However, > the only way to give it properties is to put them in a file with -p flag. I > should be able to just pass them as flags directly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2336) support user-defined table metadata
[ https://issues.apache.org/jira/browse/ACCUMULO-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2336. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > support user-defined table metadata > --- > > Key: ACCUMULO-2336 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2336 > Project: Accumulo > Issue Type: Improvement >Reporter: Adam Fuchs >Priority: Major > > An Accumulo table can have all sorts of external information that > applications use to interact with the table. One example is schema > definition. Another might be information about what data or data sources have > been loaded into the table. It would be nice to encapsulate all of this > information together with the table's data for easier management, > particularly when cloning tables or discovering tables that are shared across > applications (multi-tenancy). > How can we support generic table metadata properties so that users can > innovate independently? These could be accessible via the table properties, > or this could be a separate set of public API methods if appropriate. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1534) Tablet Server using large number of decompressors during a scan
[ https://issues.apache.org/jira/browse/ACCUMULO-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1534. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Tablet Server using large number of decompressors during a scan > --- > > Key: ACCUMULO-1534 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1534 > Project: Accumulo > Issue Type: Bug >Affects Versions: 1.4.3 >Reporter: Mike Drob >Assignee: Keith Turner >Priority: Major > > I believe this issue is similar to ACCUMULO-665. We've run into a situation > where a complex iterator tree creates a large number of decompressors from > the underlying CodecPool for serving scans. Each decompressor holds on to a > large buffer and the total volume ends up killing the tserver. > We have verified that turning off compression makes this problem go away. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2033) monitor XML doesn't say anything about GC
[ https://issues.apache.org/jira/browse/ACCUMULO-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2033. - Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > monitor XML doesn't say anything about GC > - > > Key: ACCUMULO-2033 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2033 > Project: Accumulo > Issue Type: Bug > Components: monitor >Reporter: Michael Berman >Priority: Major > > in order to test a fix for ACCUMULO-2032, i need to programmatically verify > that the monitor can talk to the GC, but GC stats aren't exposed in either > the xml or json monitor outputs. I guess I can just slurp through the html, > but that doesn't seem ideal. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3710) Scanning with many singleton ranges crashes tserver
[
https://issues.apache.org/jira/browse/ACCUMULO-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-3710.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> Scanning with many singleton ranges crashes tserver
> ---
>
> Key: ACCUMULO-3710
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3710
> Project: Accumulo
> Issue Type: Bug
> Components: client, tserver
>Affects Versions: 1.6.1
>Reporter: Shana Hutchison
>Priority: Major
>
> Setup: single-node standalone 1.6.1 Accumulo instance.
> Use case: scan ~1M individual rows, scattered across a ~15GB table.
> The following steps crash the TabletServer:
> 1. Gather a List of Range objects, each one a singleton range spanning an
> entire row.
> 2. Create a BatchScanner with one read thread.
> 3. Set the ranges via BatchScanner.setRanges()
> 4. Start iterating through the scanner.
> One solution is to batch the reads into groups of ~10k ranges idea.
> Comment from Josh Elser:
> {quote}
> Taking a quick glance at the code, it looks like this would be a good place
> to do some optimization in the BatchScanner's impl
> (TabletServerBatchReaderImpl). The BatchScanner will bin the ranges to the
> tablets and the servers hosting those tablets. Normally, this would be spread
> out, but, in your single server case, all 1M rows would all go to a single
> TabletServer in one RPC call.
> I'm guessing a good optimization here would be to check the size of a batch
> of Ranges for a single tabletserver, and when above a certain threshold,
> split the batch in half and try to reprocess each half (the recursion would
> naturally keep splitting until we get down to some high-watermark).
> Point being, if your client VM constructed the Ranges without issue, the
> BatchScanner impl should be smart enough to not knock over a TabletServer.
> {quote}
> Verified to cause an OOME via tserver_localhost.out:
> {quote}
> #
> # java.lang.OutOfMemoryError: Java heap space
> # -XX:OnOutOfMemoryError="kill -9 %p"
> # Executing /bin/sh -c "kill -9 12833"...
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3813) single row look-ups are slow
[
https://issues.apache.org/jira/browse/ACCUMULO-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-3813.
-
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please create a new issue
or PR at https://github.com/apache/accumulo
> single row look-ups are slow
>
>
> Key: ACCUMULO-3813
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3813
> Project: Accumulo
> Issue Type: Improvement
> Components: client, tserver
>Reporter: Eric C. Newton
>Priority: Major
>
> Emilio Lahr-Vivaz writes on the user mailing list:
> {quote}
> I've found that scanning lots of non-sequential single-row ranges is pretty
> slow in accumulo. Your best approach is probably to create an index table on
> whatever you are originally trying to query (assuming those 1 ids came
> from some other query).
> {quote}
> Specifically, the use case is fetching many single items, all of which are
> present (so, bloom filters aren't going to help).
> Since this is one of the use cases Accumulo was designed to handle, look into
> actual performance and figure out if there are any obvious bottlenecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3805) No means to monitor/interact with FATE programmatically
[ https://issues.apache.org/jira/browse/ACCUMULO-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3805. - Resolution: Not A Problem The admin command can do some of these things. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > No means to monitor/interact with FATE programmatically > --- > > Key: ACCUMULO-3805 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3805 > Project: Accumulo > Issue Type: Bug > Components: fate >Reporter: John Vines >Priority: Major > > With FATE, we have the ability to have strong tracking of operations. > However, we hide a lot of this from the client. We have admin utilities > plumbed into the shell (FateCommand), but they're not actual APIs. This means > there is no API available for failing/deleting/listing fate operations, but > it also means there no way to be aware of the txid for a fate operation. This > is critical for instances where a potentially distributed client wants to > perform a FATEd operation. > In an ideal world, client would be able to pre-seed the transaction, as we do > now, and then issue the command with that id. This would allow the client to > be aware of the transaction before it's started so it could be shared. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4049) create a generic counting iterator
[ https://issues.apache.org/jira/browse/ACCUMULO-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4049. - Resolution: Won't Fix This can be done with summaries, a new feature in 2.x. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > create a generic counting iterator > -- > > Key: ACCUMULO-4049 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4049 > Project: Accumulo > Issue Type: Improvement > Components: core >Reporter: Adam Fuchs >Priority: Major > > As a user I want to be able to count the number of key/values, rows, > row+column family, etc. exist in a range. This could be done via a simple > iterator like the CountingIterator added at scan time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4090) BatchWriter close not cleaning up all resources
[ https://issues.apache.org/jira/browse/ACCUMULO-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4090. - Resolution: Cannot Reproduce Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > BatchWriter close not cleaning up all resources > --- > > Key: ACCUMULO-4090 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4090 > Project: Accumulo > Issue Type: Bug > Components: client >Affects Versions: 1.7.0 >Reporter: Eric C. Newton >Assignee: Eric C. Newton >Priority: Major > > I'm debugging an issue with a long-running ingestor, similar to the > TraceServer. > After realizing that BatchWriter close needs to be called when a > MutationsRejectedException occurs (see ACCUMULO-4088), a close was added, and > the client became more stable. > However, after a day, or so, the client became sluggish. When inspecting a > heap dump, many TabletServerBatchWriter objects were still referenced. This > server should only have two BatchWriter instances at any one time, and this > server had >100. > Still debugging. > The error that initiates the issue is a SessionID not found, presumably > because the session timed out. This is the cause of the > MutationsRejectedException seen by the client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4092) metadata table corruption on recovery
[ https://issues.apache.org/jira/browse/ACCUMULO-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4092. - Resolution: Cannot Reproduce This was reported against 1.6. We are now on 2.1, and the WALs have gone through several major updates since 1.6. This is likely no longer relevant, but if this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > metadata table corruption on recovery > - > > Key: ACCUMULO-4092 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4092 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.6.4 > Environment: large production system, 1.6.2 with local patches, > hadoop 2.2 >Reporter: Eric C. Newton >Priority: Major > > I suspect that we are getting metadata table corruption on WAL recovery. > There have been several hints that this has occurred over the past 2 years, > but I have not had strong evidence for it until today. > A large production cluster was recently upgraded to 1.6.4. Upon shutdown, it > had several consistency check failures. > When a tablet is unloaded, it double-checks the entries for the tablet held > in memory against the metadata for the tablet. When the production system was > restarted for the upgrade, this check failed for several tablets. In > particular, there were file references for the tablet, that did not exist in > memory. > This particular system has a very large table which is organized by date. > Almost all of the tablets that failed the check occurred on the same date. If > the metadata tablet for those tablets was recovered on that date, and there > is some bug recovering the WAL entries, they would have affected multiple > tablets on the same day. > After searching around the logs, we did find that the metadata tablet for the > corrupt tablets did experience a recovery on the date in question. > Unfortunately, the WAL files were GC'd many weeks ago. > We need more information to track down the bug. Some possible ways to get > this information include: > 1) add periodic consistency checks: It's simple, and would detect problems > earlier. In a test environment, we might be able to keep all the archived > WALs. > 2) upon metadata tablet recovery, the master could issue a request for > consistency checks for the affected tablets. If checks fail, the recovery > logs could be archived. > 3) add metadata splits to the long-running tests which would add many more > metadata tablet recoveries > I suspect the bug is subtle, and may not cause data loss, since we don't see > data loss in continuous ingest tests. But that doesn't mean that deleted > data isn't being returned to a table, since the CI test does not delete data. > The uptime for this system is measured in months and includes several hundred > nodes. The metadata tablet is spread over most of the cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2338) add RangeInterpreter to the shell
[ https://issues.apache.org/jira/browse/ACCUMULO-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2338. - Resolution: Won't Fix > add RangeInterpreter to the shell > - > > Key: ACCUMULO-2338 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2338 > Project: Accumulo > Issue Type: New Feature > Components: shell >Reporter: Eric C. Newton >Priority: Major > > The shell doesn't allow for very sophisticated scans. You can't specify > ranges on columns, which is often used in document-partitioned indexing > schemes. Similar to the ScanInterpreter feature, in which the scan args are > translated, it would be nice if we supported a RangeInterpreter which would > translate the range of the scan. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ACCUMULO-2338) add RangeInterpreter to the shell
[ https://issues.apache.org/jira/browse/ACCUMULO-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627913#comment-17627913 ] Christopher Tubbs commented on ACCUMULO-2338: - JShell is now a viable option for more sophisticated situations. If this is still a problem or a desired feature, please create a new issue or PR at https://github.com/apache/accumulo > add RangeInterpreter to the shell > - > > Key: ACCUMULO-2338 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2338 > Project: Accumulo > Issue Type: New Feature > Components: shell >Reporter: Eric C. Newton >Priority: Major > > The shell doesn't allow for very sophisticated scans. You can't specify > ranges on columns, which is often used in document-partitioned indexing > schemes. Similar to the ScanInterpreter feature, in which the scan args are > translated, it would be nice if we supported a RangeInterpreter which would > translate the range of the scan. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4544) pluggable parser for insert shell command
[ https://issues.apache.org/jira/browse/ACCUMULO-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4544. - Resolution: Won't Fix I believe formatters are now being phased out of the shell, and at the very least, JShell is now a viable option for doing custom things with data manipulation. If this is still a problem or a desired feature, please create a new issue or PR at https://github.com/apache/accumulo > pluggable parser for insert shell command > - > > Key: ACCUMULO-4544 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4544 > Project: Accumulo > Issue Type: New Feature > Components: shell >Affects Versions: 1.7.1 >Reporter: Michael Berman >Priority: Major > > Just as you can attach custom formatters for scan output in the shell, it > would be great to be able to provide the inverse transformation for use when > inserting. right now, if you are using a formatter on a table, you typically > cannot copy lines to reinsert with modifications, without also manually > reversing whatever your formatter is doing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-416) reevaluate limiting the number of open files given HDFS improvements
[ https://issues.apache.org/jira/browse/ACCUMULO-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-416. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > reevaluate limiting the number of open files given HDFS improvements > > > Key: ACCUMULO-416 > URL: https://issues.apache.org/jira/browse/ACCUMULO-416 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: Adam Fuchs >Assignee: Keith Turner >Priority: Major > > Tablet servers limit the number of files that can be opened for scans and for > major compactions. The two main reasons for this limit was to reduce our > impact on HDFS, primarily regarding connections to data nodes, and to limit > our memory usage related to preloading file indexes. A third reason might be > that disk thrashing could become a problem if we try to read from too many > places at once. > Two improvements may have made (or may soon make) this limit obsolete: HDFS > now pools connections, and RFile now uses a multi-level index. With these > improvements, is it reasonable to lift some of our open file restrictions? > The tradeoff on query side might be availability vs. overall resource usage. > On the compaction side, the tradeoff is probably write replication vs. > thrashing on reads. I think we can make an argument that queries should be > available at almost any cost, but the compaction tradeoff is not as clear. We > should test the efficiency of compacting a large number of files to get a > better feeling for how the two extremes effect read and write performance > across the system. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-543) why are we still reusing RFile.Readers?
[ https://issues.apache.org/jira/browse/ACCUMULO-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-543. Resolution: Abandoned Closing this stale issue. If this is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > why are we still reusing RFile.Readers? > --- > > Key: ACCUMULO-543 > URL: https://issues.apache.org/jira/browse/ACCUMULO-543 > Project: Accumulo > Issue Type: Improvement > Components: tserver >Reporter: Adam Fuchs >Assignee: Adam Fuchs >Priority: Major > > The TabletServer includes a bunch of code to manage reuse of RFile.Reader > objects across scans. There are a few reasons for doing this that certainly > made sense in the past, including: > * Reducing index reads. If we have already found a location in the RFile > index, and we can easily determine that this is the location that we need for > the next read, then we can skip doing a binary search in the RFile index. > Index caching and hierarchical indexes may improve this aspect to where we > don't need this optimization. > * Reducing re-reading of the same data block. Re-reading a block used to mean > that HDFS would create a new socket rather than reusing the current socket. > Seeking to the same spot HDFS now reuses connections, so this is not as big a > deal. > * Reducing namenode operations. Re-opening an RFile requires knowledge of the > RFile's size. Even if we have the block locations cached, the file size still > includes an HDFS namenode operation. This is something we should probably be > caching within the TabletServer, since our RFiles are immutable. > There are also a few problems with reusing RFiles, including: > * Increased TabletServer complexity. We need to manage the resource pool > among many scans. Not reusing readers along with ACCUMULO-416 may make it > possible to eliminate coordination of RFile.Readers at the TabletServer level > altogether. > * Risk of information leakage. RFile.Readers hold state, and we don't want > that state passing between scan sessions. Sharing RFile.Readers makes it more > likely that a bug in this class exists. > * Risk of interference. RFile.Readers are not thread safe now, but even if > they were we wouldn't want separate scan sessions (or maybe even multiple > threads in the same scan session) affecting each other through the > RFile.Reader. Sharing RFile.Readers also makes it more likely that a bug in > this class exists. > The relevant question is what would be the effect on creating a new > RFile.Reader anytime we want to read an RFile instead of grabbing one from > the pool? ACCUMULO-416 is certainly a prerequisite for this ticket, and we > know we need to cache the RFile length. Are there any other prerequisites? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3848) Tracer service exits when port fuzzer scans
[ https://issues.apache.org/jira/browse/ACCUMULO-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-3848. - Resolution: Not A Problem The Tracer service no longer exists. > Tracer service exits when port fuzzer scans > --- > > Key: ACCUMULO-3848 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3848 > Project: Accumulo > Issue Type: Bug > Components: trace >Affects Versions: 1.6.0 >Reporter: Sean Busbey >Priority: Major > > I had a security scanner knock over a Tracer role when it fuzzed the port. > still trying to get logs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-3196) Add support scripts and instructions for manual zookeeper cleanup
[
https://issues.apache.org/jira/browse/ACCUMULO-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-3196.
-
Resolution: Duplicate
This was done in https://github.com/apache/accumulo-website/pull/338
> Add support scripts and instructions for manual zookeeper cleanup
> -
>
> Key: ACCUMULO-3196
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3196
> Project: Accumulo
> Issue Type: New Feature
> Components: docs, scripts
>Affects Versions: 1.5.2, 1.6.1, 1.7.0
>Reporter: Sean Busbey
>Assignee: Michael Allen
>Priority: Major
>
> [From user@
> thread|http://mail-archives.apache.org/mod_mbox/accumulo-user/201410.mbox/%3CCAGZGg1idg7U_YVJi_dH-KASK7VVY%3D4r9aReqxGWukVO1s0dmLw%40mail.gmail.com%3E]:
> {quote}
> ZooKeeper has security features built into it by way of access control
> lists (ACLs) on nodes. Once set, these ACLs can be very hard to get rid
> of, especially if errant code has set up nodes that you no longer have any
> password for. This how-to guide shows you how to set up a root user inside
> of ZooKeeper that can wipe out any ACLed node.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4562) Consider Adding Java 8 Stream support to scanners
[
https://issues.apache.org/jira/browse/ACCUMULO-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs resolved ACCUMULO-4562.
-
Resolution: Duplicate
Done in https://github.com/apache/accumulo/pull/2636
> Consider Adding Java 8 Stream support to scanners
> -
>
> Key: ACCUMULO-4562
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4562
> Project: Accumulo
> Issue Type: Improvement
>Reporter: Keith Turner
>Priority: Major
>
> For a test I wanted to find the min and max timestamp of an Accumulo table.
> I used Java 8 streams to do that as follows. The code
> {{StreamSupport.stream(scanner.spliterator(), false)}} is a standard way in
> Java 8 to create a stream from an Iterable.
> {code:java}
> try(Scanner scanner = c.createScanner(table, Authorizations.EMPTY)){
> Stream> stream =
> StreamSupport.stream(scanner.spliterator(), false);
> LongSummaryStatistics stats = stream
> .mapToLong(e -> e.getKey().getTimestamp())
> .summaryStatistics();
> System.out.println(stats);
> }
> {code}
> In Java 8, collections have the {{stream()}} and {{parallelStream()}}
> methods. If ScannerBase had those methods in Accumulo, then the following
> could be written w/o using {{StreamSupport}}
> {code:java}
> try(Scanner scanner = c.createScanner(table, Authorizations.EMPTY)){
> LongSummaryStatistics stats = scanner.stream()
> .mapToLong(e -> e.getKey().getTimestamp())
> .summaryStatistics();
> System.out.println(stats);
> }
> {code}
> For the BatchScanner I think we could implement a parallel stream. One way
> to do this would be a to create an internal batch scanner queue for each Java
> 8 split iterator. Currently the BatchScanner has one queue that all
> background threads put batches of key values on. With multiple queues, each
> background thread could break its batches into equal sizes and put a subset
> on each queue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1609) Block conditional writes if too much memory used
[ https://issues.apache.org/jira/browse/ACCUMULO-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1609. - Resolution: Abandoned Closing this stale issue. If it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Block conditional writes if too much memory used > > > Key: ACCUMULO-1609 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1609 > Project: Accumulo > Issue Type: Improvement > Components: client, tserver >Reporter: Keith Turner >Priority: Major > > Add support for blocking writes to the conditional writer if too much memory > is used. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2189) Document the Repo interface
[ https://issues.apache.org/jira/browse/ACCUMULO-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2189. - Resolution: Abandoned Closing this stale issue. If it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Document the Repo interface > --- > > Key: ACCUMULO-2189 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2189 > Project: Accumulo > Issue Type: Sub-task > Components: docs, fate >Reporter: Vikram Srivastava >Priority: Major > > The Repo interface is lacking java doc. For example: > Repo.isReady return a long and there is no documentation about what its > return value should be and how it is used. > Currently it's used by Fate.run to decide whether to run "Repo.call" or not. > If that's the only purpose it is used for, then isReady should return a > boolean. If we do want it to return a long, then we should at least use > constants by the various derived classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-1535) monitor deepCopy to understand Compressor usage as a tserver resource
[ https://issues.apache.org/jira/browse/ACCUMULO-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1535. - Resolution: Abandoned Closing this stale issue. If it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > monitor deepCopy to understand Compressor usage as a tserver resource > - > > Key: ACCUMULO-1535 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1535 > Project: Accumulo > Issue Type: Sub-task > Components: tserver >Reporter: Eric C. Newton >Priority: Major > > With IntersectingIterator trees, the number of deepCopied handles to files > can multiply rapidly. The tablet server should monitor this resource because > the Decompressors can take up a lot of space permanently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4637) Master should rip out locks of servers with bad system credentials
[ https://issues.apache.org/jira/browse/ACCUMULO-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4637. - Resolution: Abandoned Closing this stale issue. If it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Master should rip out locks of servers with bad system credentials > -- > > Key: ACCUMULO-4637 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4637 > Project: Accumulo > Issue Type: Bug >Affects Versions: 1.7.2 >Reporter: John Vines >Priority: Major > > Saw at a customer where one of their servers had not been properly synced. > Specifically, one of the instance values was mismatched, which means the > system password was mismatched. Server logs show a lot of bad auth failures > for the system user (as expected). > Meanwhile, the master is spinning, trying to get all the tablets online, > constantly attempting to assign them to this one server (because it has 0 > assigned, so it's a prime contender). Instead of this behavior, the master > should rip out the servers zk lock because it's not a member of the cluster > anymore (not until it's accumulo-site is fixed). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ACCUMULO-4810) Make session manager reservations more strict
[ https://issues.apache.org/jira/browse/ACCUMULO-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627893#comment-17627893 ] Christopher Tubbs commented on ACCUMULO-4810: - It's not clear what "more strict" means here. If it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Make session manager reservations more strict > - > > Key: ACCUMULO-4810 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4810 > Project: Accumulo > Issue Type: Improvement >Reporter: Keith Turner >Priority: Major > > While working on ACCUMULO-4782 I noticed that the session manager was not > strict for the following cases. > * Removing a reserved session > * Unreserving a removed session > For ACCUMULO-4782 I wanted to preserve existing behavior and switch to a > concurrent map. I think it would be nice to make reservations more strict > and work through any bugs this causes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-4810) Make session manager reservations more strict
[ https://issues.apache.org/jira/browse/ACCUMULO-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4810. - Resolution: Abandoned > Make session manager reservations more strict > - > > Key: ACCUMULO-4810 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4810 > Project: Accumulo > Issue Type: Improvement >Reporter: Keith Turner >Priority: Major > > While working on ACCUMULO-4782 I noticed that the session manager was not > strict for the following cases. > * Removing a reserved session > * Unreserving a removed session > For ACCUMULO-4782 I wanted to preserve existing behavior and switch to a > concurrent map. I think it would be nice to make reservations more strict > and work through any bugs this causes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2200) Implement blocking password change
[ https://issues.apache.org/jira/browse/ACCUMULO-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2200. - Assignee: (was: Vicky Kak) Resolution: Not A Problem In general, this hasn't really been a problem. I'll close this issue for now, but if it is still a problem, please create a new issue or PR at https://github.com/apache/accumulo > Implement blocking password change > -- > > Key: ACCUMULO-2200 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2200 > Project: Accumulo > Issue Type: Improvement > Components: client >Reporter: Mike Drob >Priority: Major > > We should have a blocking password change method that users can use. Our > tests would benefit from this by no longer needing to use blind sleep > statements, and users could be assured that the change has propogated before > starting up other clients. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ACCUMULO-2069) ThriftTransportPool too aggressive in closing connections
[ https://issues.apache.org/jira/browse/ACCUMULO-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-2069. - Resolution: Duplicate Duplicated / superseded by https://github.com/apache/accumulo/pull/2612 > ThriftTransportPool too aggressive in closing connections > - > > Key: ACCUMULO-2069 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2069 > Project: Accumulo > Issue Type: Bug > Components: client >Affects Versions: 1.4.4 >Reporter: Dave Marion >Priority: Major > > org.apache.accumulo.core.client.impl.ThriftTransportPool closes unused thrift > connections after 3 seconds of inactivty. This can impact performance by > causing tablet server connections to be constantly torn down and set up. > Think of the case where you have a MTBW writer with a small number of threads > and a moderate amount of tservers. Make the timeout configurable. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (ACCUMULO-1298) Document Fate zookeeper layout
[ https://issues.apache.org/jira/browse/ACCUMULO-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1298. - Resolution: Duplicate > Document Fate zookeeper layout > -- > > Key: ACCUMULO-1298 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1298 > Project: Accumulo > Issue Type: Task > Components: fate >Reporter: John Vines >Priority: Minor > > Taken from TODO: > Fate has a specific layout in zookeeper. It should be documented. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (ACCUMULO-1298) Document Fate zookeeper layout
[ https://issues.apache.org/jira/browse/ACCUMULO-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs reopened ACCUMULO-1298: - > Document Fate zookeeper layout > -- > > Key: ACCUMULO-1298 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1298 > Project: Accumulo > Issue Type: Task > Components: fate >Reporter: John Vines >Priority: Minor > > Taken from TODO: > Fate has a specific layout in zookeeper. It should be documented. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (ACCUMULO-1297) Fate's ZooStore should use zoocache or curator
[ https://issues.apache.org/jira/browse/ACCUMULO-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-1297. - Resolution: Duplicate > Fate's ZooStore should use zoocache or curator > -- > > Key: ACCUMULO-1297 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1297 > Project: Accumulo > Issue Type: Sub-task > Components: fate >Reporter: John Vines >Assignee: John Vines >Priority: Major > > Taken from TODOs > ZooStore does all of it's ZooKeeper work directly. It should either hedge on > ZooCache or Curator for it's operations instead. This also means better > handling for error conditions. -- This message was sent by Atlassian Jira (v8.20.1#820001)
