[jira] [Assigned] (ACCUMULO-4774) Conditional Writer creates a non daemon thread that is keeping processes alive

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs reassigned ACCUMULO-4774:
---

Assignee: Keith Turner

> Conditional Writer creates a non daemon thread that is keeping processes alive
> --
>
> Key: ACCUMULO-4774
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4774
> Project: Accumulo
>  Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Keith Turner
>Assignee: Keith Turner
>  Labels: pull-request-available
> Fix For: 1.7.4, 1.8.2, 2.0.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The Conditional writer has a static thread pool that creates a non-daemon 
> thread that keeping processes alive. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] asfgit closed pull request #348: ACCUMULO-4774 Made conditional writer thread into daemon

2017-12-21 Thread GitBox 
asfgit closed pull request #348: ACCUMULO-4774 Made conditional writer thread 
into daemon
URL: https://github.com/apache/accumulo/pull/348
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/core/src/main/java/org/apache/accumulo/core/client/impl/ConditionalWriterImpl.java
 
b/core/src/main/java/org/apache/accumulo/core/client/impl/ConditionalWriterImpl.java
index 4812378ea7..2199ad241e 100644
--- 
a/core/src/main/java/org/apache/accumulo/core/client/impl/ConditionalWriterImpl.java
+++ 
b/core/src/main/java/org/apache/accumulo/core/client/impl/ConditionalWriterImpl.java
@@ -36,6 +36,7 @@
 import java.util.concurrent.Delayed;
 import java.util.concurrent.LinkedBlockingQueue;
 import java.util.concurrent.ScheduledThreadPoolExecutor;
+import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.ThreadPoolExecutor;
 import java.util.concurrent.TimeUnit;
 
@@ -96,7 +97,18 @@
 
 class ConditionalWriterImpl implements ConditionalWriter {
 
-  private static ThreadPoolExecutor cleanupThreadPool = new 
ThreadPoolExecutor(1, 1, 10, TimeUnit.SECONDS, new 
LinkedBlockingQueue());
+  private static class CleanupThreadFactory implements ThreadFactory {
+
+@Override
+public Thread newThread(Runnable r) {
+  Thread t = new Thread(r, "Conditional Writer Cleanup Thread ");
+  t.setDaemon(true);
+  return t;
+}
+  }
+
+  private static ThreadPoolExecutor cleanupThreadPool = new 
ThreadPoolExecutor(1, 1, 3, TimeUnit.SECONDS, new 
LinkedBlockingQueue(),
+  new CleanupThreadFactory());
 
   static {
 cleanupThreadPool.allowCoreThreadTimeOut(true);


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ctubbsii commented on issue #348: ACCUMULO-4774 Made conditional writer thread into daemon

2017-12-21 Thread GitBox 
ctubbsii commented on issue #348: ACCUMULO-4774 Made conditional writer thread 
into daemon
URL: https://github.com/apache/accumulo/pull/348#issuecomment-353525118
 
 
   I merged into 1.7, and then 1.8 and master branches, then converted the new 
ThreadFactory to a lambda (less code) for the master branch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (ACCUMULO-4770) Accumulo monitor overview page is not listing all Zookeeper nodes

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs updated ACCUMULO-4770:

Priority: Blocker  (was: Major)

> Accumulo monitor overview page is not listing all Zookeeper nodes
> -
>
> Key: ACCUMULO-4770
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4770
> Project: Accumulo
>  Issue Type: Bug
>  Components: monitor
>Affects Versions: 2.0.0
> Environment: Ran Accumulo 2.0.0-SNAPSHOT on a cluster with 3 
> Zookeeper nodes.  Only one is being listed in the Accumulo monitor overview 
> page.
>Reporter: Mike Walch
>Priority: Blocker
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ACCUMULO-4770) Accumulo monitor overview page is not listing all Zookeeper nodes

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs updated ACCUMULO-4770:

Component/s: monitor

> Accumulo monitor overview page is not listing all Zookeeper nodes
> -
>
> Key: ACCUMULO-4770
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4770
> Project: Accumulo
>  Issue Type: Bug
>  Components: monitor
>Affects Versions: 2.0.0
> Environment: Ran Accumulo 2.0.0-SNAPSHOT on a cluster with 3 
> Zookeeper nodes.  Only one is being listed in the Accumulo monitor overview 
> page.
>Reporter: Mike Walch
>Priority: Blocker
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ACCUMULO-4770) Accumulo monitor overview page is not listing all Zookeeper nodes

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs updated ACCUMULO-4770:

Fix Version/s: 2.0.0

> Accumulo monitor overview page is not listing all Zookeeper nodes
> -
>
> Key: ACCUMULO-4770
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4770
> Project: Accumulo
>  Issue Type: Bug
>Affects Versions: 2.0.0
> Environment: Ran Accumulo 2.0.0-SNAPSHOT on a cluster with 3 
> Zookeeper nodes.  Only one is being listed in the Accumulo monitor overview 
> page.
>Reporter: Mike Walch
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ACCUMULO-4769) Sanity check for valid CryptoModule and KeyEncryptionStrategy in config

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs updated ACCUMULO-4769:

Fix Version/s: 2.0.0

> Sanity check for valid CryptoModule and KeyEncryptionStrategy in config
> ---
>
> Key: ACCUMULO-4769
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4769
> Project: Accumulo
>  Issue Type: Improvement
>Reporter: Nick Felts
>Assignee: Nick Felts
>  Labels: pull-request-available
> Fix For: 2.0.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The current code will log a warning and default to an unencrypted mode if an 
> unknown class is provided for the cryptomodule or for the 
> keyencryptionstrategy. These should be checked when they are read from the 
> config to confirm they exist and implement the appropriate interface.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ctubbsii commented on a change in pull request #345: ACCUMULO-4769 Sanity check for valid CryptoModule and KeyEncryptionStrategy in config

2017-12-21 Thread GitBox 
ctubbsii commented on a change in pull request #345: ACCUMULO-4769 Sanity check 
for valid CryptoModule and KeyEncryptionStrategy in config
URL: https://github.com/apache/accumulo/pull/345#discussion_r158139376
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/conf/ConfigSanityCheck.java
 ##
 @@ -88,6 +90,33 @@ else if (!prop.getType().isValidFormat(value))
   if (key.equals(Property.CRYPTO_CIPHER_KEY_ALGORITHM_NAME.getKey())) {
 keyAlgorithm = Objects.requireNonNull(value);
   }
+  if 
(key.equals(Property.CRYPTO_SECRET_KEY_ENCRYPTION_STRATEGY_CLASS.getKey()) && 
!(value == null || value.equals("NullSecretKeyEncryptionStrategy"))) {
+@SuppressWarnings("rawtypes")
 
 Review comment:
   Should not suppress warnings which can be addressed with proper generics 
use. Can use `` here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ctubbsii commented on a change in pull request #345: ACCUMULO-4769 Sanity check for valid CryptoModule and KeyEncryptionStrategy in config

2017-12-21 Thread GitBox 
ctubbsii commented on a change in pull request #345: ACCUMULO-4769 Sanity check 
for valid CryptoModule and KeyEncryptionStrategy in config
URL: https://github.com/apache/accumulo/pull/345#discussion_r158141223
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/security/crypto/CryptoModuleFactory.java
 ##
 @@ -172,23 +164,17 @@ private static SecretKeyEncryptionStrategy 
instantiateSecreteKeyEncryptionStrate
 }
 
 if (!implementsSecretKeyStrategy) {
-  log.warn("Configured Accumulo secret key encryption strategy \"%s\" does 
not implement the SecretKeyEncryptionStrategy interface. No encryption will be 
used.");
-  return new NullSecretKeyEncryptionStrategy();
+  throw new RuntimeException("Configured Accumulo secret key encryption 
strategy \"%s\" does not implement the SecretKeyEncryptionStrategy interface.");
 } else {
   try {
 strategy = (SecretKeyEncryptionStrategy) 
keyEncryptionStrategyClazz.newInstance();
 
 log.debug("Successfully instantiated secret key encryption strategy 
{}", className);
 
   } catch (InstantiationException e) {
-log.warn("Got instantiation exception {} when instantiating secret key 
encryption strategy \"{}\".  No encryption will be used.", e.getCause()
-.getClass().getName(), className);
-log.warn("InstantiationException {}", e.getCause());
-return new NullSecretKeyEncryptionStrategy();
+throw new RuntimeException("Got instantiation exception {} when 
instantiating secret key encryption strategy " + className);
 
 Review comment:
   Since a lot of these catch blocks have essentially the same error, it'd 
probably be better to use the multi-catch syntax, just to clean things up a 
bit. The original exception should be passed as a parameter, so we don't lose 
portions of the stack trace.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (ACCUMULO-4768) Monitor Look and Feel

2017-12-21 Thread Christopher Tubbs (JIRA) 

[ 
https://issues.apache.org/jira/browse/ACCUMULO-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300832#comment-16300832
 ] 

Christopher Tubbs commented on ACCUMULO-4768:
-

Are there any subtasks? Or is this just an umbrella ticket because our 
convention is to include a JIRA number in our commits?
If it's just the latter, I think that sort of highlights a failure in our 
conventions. Perhaps we should relax them a bit... omitting a JIRA issue number 
from a commit log... especially for a change that will probably be reviewed in 
a PR is probably fine, IMO.

> Monitor Look and Feel
> -
>
> Key: ACCUMULO-4768
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4768
> Project: Accumulo
>  Issue Type: Improvement
>  Components: monitor
>Affects Versions: 2.0.0
>Reporter: Michael Miller
>Assignee: Michael Miller
>Priority: Minor
> Fix For: 2.0.0
>
>
> Catch all ticket for minor look and feel improvements for the new Monitor.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ACCUMULO-3902) Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs updated ACCUMULO-3902:

Fix Version/s: (was: 1.7.4)
   (was: 1.8.2)

Given that the current pull request is focusing on the master branch, and the 
fact that we want to limit changes to previous branches unless they are 
targeted at fixing a known bug, I'm dropping the 1.7 and 1.8 versions from this 
issue.

> Ensure [Batch]Scanners are closed in ITs
> 
>
> Key: ACCUMULO-3902
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3902
> Project: Accumulo
>  Issue Type: Bug
>  Components: test
>Reporter: Josh Elser
>Assignee: Jared R
>Priority: Trivial
>  Labels: newbie, pull-request-available
> Fix For: 2.0.0
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Do an audit of the integration tests to verify that we actually close 
> Scanners and BatchScanners. This is a best practice that we should be 
> encouraging (by doing it in our tests). It can also lead to bugs in other 
> test cases (ACCUMULO-3888).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (ACCUMULO-4765) Improve sortTable js function

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs resolved ACCUMULO-4765.
-
   Resolution: Won't Fix
Fix Version/s: (was: 2.0.0)

I'm closing this as "Won't Fix". The suggestion to use DataTables is so much 
better, that I don't think this is even worth pursuing.

> Improve sortTable js function
> -
>
> Key: ACCUMULO-4765
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4765
> Project: Accumulo
>  Issue Type: Improvement
>  Components: monitor
>Affects Versions: 2.0.0
>Reporter: Michael Miller
>Priority: Minor
>  Labels: javascript, newbie
>
> The sortTable() javascript function is used quite a lot (and has many 
> implementations) in the Monitor but takes the column number as a parameter.  
> While this is a simple solution that works well, it could break if the order 
> of the columns change.  It would be nice to have a sortTable function that is 
> smarter and could automatically detect which column to sort.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (ACCUMULO-4732) No APIs to configure iterators and locality groups for new table

2017-12-21 Thread Christopher Tubbs (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christopher Tubbs resolved ACCUMULO-4732.
-
Resolution: Fixed

> No APIs to configure iterators and locality groups for new table
> 
>
> Key: ACCUMULO-4732
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4732
> Project: Accumulo
>  Issue Type: Improvement
>Reporter: Keith Turner
>Assignee: Mark Owens
>  Labels: newbie, pull-request-available
> Fix For: 2.0.0
>
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>
> In Accumulo 1.7 the ability to set table properties at table creation time 
> was added.  For existing tables there are APIs in table operations that allow 
> setting locality groups and iterators for existing tables.  When setting 
> table properties at table creation time there is not good API for iterators 
> and locality groups. There should be some way in the API to do this.  There 
> may be other things besides iterators and locality groups that should also be 
> supported at table creation time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ctubbsii closed pull request #337: ACCUMULO-4732 No APIs to configure iterators or locality groups for new tables

2017-12-21 Thread GitBox 
ctubbsii closed pull request #337: ACCUMULO-4732 No APIs to configure iterators 
or locality groups for new tables
URL: https://github.com/apache/accumulo/pull/337
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/core/src/main/java/org/apache/accumulo/core/client/admin/NewTableConfiguration.java
 
b/core/src/main/java/org/apache/accumulo/core/client/admin/NewTableConfiguration.java
index 9d5d31a159..b0dbf4ec27 100644
--- 
a/core/src/main/java/org/apache/accumulo/core/client/admin/NewTableConfiguration.java
+++ 
b/core/src/main/java/org/apache/accumulo/core/client/admin/NewTableConfiguration.java
@@ -21,16 +21,28 @@
 
 import java.util.Arrays;
 import java.util.Collections;
+import java.util.EnumSet;
 import java.util.HashMap;
 import java.util.Map;
-
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.IteratorSetting;
+import org.apache.accumulo.core.client.impl.TableOperationsHelper;
 import org.apache.accumulo.core.client.sample.SamplerConfiguration;
 import org.apache.accumulo.core.client.summary.Summarizer;
 import org.apache.accumulo.core.client.summary.SummarizerConfiguration;
+import org.apache.accumulo.core.conf.Property;
 import org.apache.accumulo.core.iterators.IteratorUtil;
+import org.apache.accumulo.core.iterators.IteratorUtil.IteratorScope;
 import org.apache.accumulo.core.iterators.user.VersioningIterator;
 import org.apache.accumulo.core.sample.impl.SamplerConfigurationImpl;
 import org.apache.accumulo.core.summary.SummarizerConfigurationUtil;
+import org.apache.accumulo.core.util.LocalityGroupUtil;
+import org.apache.hadoop.io.Text;
 
 /**
  * This object stores table creation parameters. Currently includes: {@link 
TimeType}, whether to include default iterators, and user-specified initial
@@ -48,6 +60,8 @@
   private Map properties = Collections.emptyMap();
   private Map samplerProps = Collections.emptyMap();
   private Map summarizerProps = Collections.emptyMap();
+  private Map localityProps = Collections.emptyMap();
+  private Map iteratorProps = new HashMap<>();
 
   private void checkDisjoint(Map props, Map 
derivedProps, String kind) {
 checkArgument(Collections.disjoint(props.keySet(), derivedProps.keySet()), 
"Properties and derived %s properties are not disjoint", kind);
@@ -62,7 +76,6 @@ private void checkDisjoint(Map props, 
Map derivedP
*/
   public NewTableConfiguration setTimeType(TimeType tt) {
 checkArgument(tt != null, "TimeType is null");
-
 this.timeType = tt;
 return this;
   }
@@ -88,7 +101,7 @@ public NewTableConfiguration withoutDefaultIterators() {
   }
 
   /**
-   * Sets additional properties to be applied to tables created with this 
configuration. Additional calls to this method replaces properties set by 
previous
+   * Sets additional properties to be applied to tables created with this 
configuration. Additional calls to this method replace properties set by 
previous
* calls.
*
* @param props
@@ -99,6 +112,9 @@ public NewTableConfiguration 
setProperties(Map props) {
 checkArgument(props != null, "properties is null");
 checkDisjoint(props, samplerProps, "sampler");
 checkDisjoint(props, summarizerProps, "summarizer");
+checkDisjoint(props, localityProps, "locality group");
+checkDisjoint(props, iteratorProps, "iterator");
+checkTableProperties(props);
 this.properties = new HashMap<>(props);
 return this;
   }
@@ -111,13 +127,14 @@ public NewTableConfiguration 
setProperties(Map props) {
   public Map getProperties() {
 Map propertyMap = new HashMap<>();
 
-if (limitVersion) {
+if (limitVersion)
   
propertyMap.putAll(IteratorUtil.generateInitialTableProperties(limitVersion));
-}
 
 propertyMap.putAll(summarizerProps);
 propertyMap.putAll(samplerProps);
 propertyMap.putAll(properties);
+propertyMap.putAll(iteratorProps);
+propertyMap.putAll(localityProps);
 return Collections.unmodifiableMap(propertyMap);
   }
 
@@ -146,4 +163,91 @@ public NewTableConfiguration 
enableSummarization(SummarizerConfiguration... conf
 summarizerProps = tmp;
 return this;
   }
+
+  /**
+   * Configures a table's locality groups prior to initial table creation.
+   *
+   * Allows locality groups to be set prior to table creation. Additional 
calls to this method prior to table creation will overwrite previous locality 
group
+   * mappings.
+   *
+   * @param groups
+

Accumulo-Pull-Requests - Build # 934 - Unstable

2017-12-21 Thread Apache Jenkins Server 
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #934)

Status: Unstable

Check console output at 
https://builds.apache.org/job/Accumulo-Pull-Requests/934/ to view the results.

[GitHub] asfgit commented on issue #348: ACCUMULO-4774 Made conditional writer thread into daemon

2017-12-21 Thread GitBox 
asfgit commented on issue #348: ACCUMULO-4774 Made conditional writer thread 
into daemon
URL: https://github.com/apache/accumulo/pull/348#issuecomment-353487600
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (ACCUMULO-4774) Conditional Writer creates a non daemon thread that is keeping processes alive

2017-12-21 Thread ASF GitHub Bot (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ACCUMULO-4774:
-
Labels: pull-request-available  (was: )

> Conditional Writer creates a non daemon thread that is keeping processes alive
> --
>
> Key: ACCUMULO-4774
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4774
> Project: Accumulo
>  Issue Type: Bug
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Keith Turner
>  Labels: pull-request-available
> Fix For: 1.7.4, 1.8.2, 2.0.0
>
>
> The Conditional writer has a static thread pool that creates a non-daemon 
> thread that keeping processes alive. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] keith-turner opened a new pull request #348: ACCUMULO-4774 Made conditional writer thread into daemon

2017-12-21 Thread GitBox 
keith-turner opened a new pull request #348: ACCUMULO-4774 Made conditional 
writer thread into daemon
URL: https://github.com/apache/accumulo/pull/348
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (ACCUMULO-4774) Conditional Writer creates a non daemon thread that is keeping processes alive

2017-12-21 Thread Keith Turner (JIRA) 
Keith Turner created ACCUMULO-4774:
--

 Summary: Conditional Writer creates a non daemon thread that is 
keeping processes alive
 Key: ACCUMULO-4774
 URL: https://issues.apache.org/jira/browse/ACCUMULO-4774
 Project: Accumulo
  Issue Type: Bug
Affects Versions: 1.8.1, 1.7.3
Reporter: Keith Turner
 Fix For: 1.7.4, 1.8.2, 2.0.0


The Conditional writer has a static thread pool that creates a non-daemon 
thread that keeping processes alive. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ctubbsii commented on a change in pull request #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
ctubbsii commented on a change in pull request #341: ACCUMULO-3902 Ensure 
[Batch]Scanners are closed in ITs
URL: https://github.com/apache/accumulo/pull/341#discussion_r158381566
 
 

 ##
 File path: test/src/main/java/org/apache/accumulo/test/MetaGetsReadersIT.java
 ##
 @@ -112,6 +113,7 @@ public void test() throws Exception {
 t2.interrupt();
 t1.join();
 t2.join();
+m.close();
 
 Review comment:
   Oh, interesting. I must have misread the code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ctubbsii commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
ctubbsii commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are 
closed in ITs
URL: https://github.com/apache/accumulo/pull/341#issuecomment-353460637
 
 
   @jkrdev In the case of the variable being reassigned, I think it makes sense 
to use a new variable (can keep the same name, but redeclared in each block). 
It results in cleaner code, and it's not like variable names are a limited 
commodity.
   
   For the stale ITs you mention... if you see anything significant, you can 
just create a new issue in JIRA at: 
https://issues.apache.org/jira/secure/CreateIssue!default.jspa for the 
"ACCUMULO" project (or just click on the "Create" button to create a new issue 
after logging in to that JIRA). If you have questions about whether you made 
the issue correctly, you can always ask on the dev@ mailing list, but chances 
are, somebody will triage the new issue whether you ask or not.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Accumulo-Pull-Requests - Build # 933 - Fixed

2017-12-21 Thread Apache Jenkins Server 
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #933)

Status: Fixed

Check console output at 
https://builds.apache.org/job/Accumulo-Pull-Requests/933/ to view the results.

[GitHub] bfach10 commented on a change in pull request #347: ACCUMULO-4746 Fluent API for Mutation

2017-12-21 Thread GitBox 
bfach10 commented on a change in pull request #347: ACCUMULO-4746 Fluent API 
for Mutation
URL: https://github.com/apache/accumulo/pull/347#discussion_r158338652
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -772,6 +772,236 @@ public void putDelete(byte[] columnFamily, byte[] 
columnQualifier, ColumnVisibil
 put(columnFamily, columnQualifier, columnVisibility.getExpression(), true, 
timestamp, true, EMPTY_BYTES);
   }
 
+  /**
+   * Fluent API for Mutation
+   *
+   * 
+   * Each step of the Mutation is represented by an interface. Inheritance 
order ensures that chained methods follow prescribed order of:
+   *
+   * 
+   * row, family, qualifier, visibility, timestamp
+   *
+   * 
+   * set and delete methods end the chain and finalize the mutation
+   */
+  public interface RowStep extends FamilyStep {
 
 Review comment:
   I wasn't sure about this one so I didn't include any implementation for it. 
Thanks for the clarification! I've removed the row methods and interface in my 
most recent commit.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Accumulo-Pull-Requests - Build # 932 - Failure

2017-12-21 Thread Apache Jenkins Server 
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #932)

Status: Failure

Check console output at 
https://builds.apache.org/job/Accumulo-Pull-Requests/932/ to view the results.

Accumulo-Pull-Requests - Build # 931 - Fixed

2017-12-21 Thread Apache Jenkins Server 
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #931)

Status: Fixed

Check console output at 
https://builds.apache.org/job/Accumulo-Pull-Requests/931/ to view the results.

[jira] [Created] (ACCUMULO-4773) Add sanity check to table properties prior to creation of fate operation

2017-12-21 Thread Mark Owens (JIRA) 
Mark Owens created ACCUMULO-4773:


 Summary: Add sanity check to table properties prior to creation of 
fate operation
 Key: ACCUMULO-4773
 URL: https://issues.apache.org/jira/browse/ACCUMULO-4773
 Project: Accumulo
  Issue Type: Improvement
  Components: fate
Reporter: Mark Owens
Priority: Minor
 Fix For: 2.0.0


On the server side, table properties could be sanity checked before the fate 
operation is created.

Could be added prior to 

https://github.com/apache/accumulo/blob/rel/1.8.1/server/master/src/main/java/org/apache/accumulo/master/FateServiceHandler.java#L151

This would prevent situations where user assumes they have set table properties 
when in fact the table properties are mis-configured and ignored by the fate 
operation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ACCUMULO-4772) Update Accumulo shell to utilize new NewTableConfiguration methods

2017-12-21 Thread Mark Owens (JIRA) 
Mark Owens created ACCUMULO-4772:


 Summary: Update Accumulo shell to utilize new 
NewTableConfiguration methods
 Key: ACCUMULO-4772
 URL: https://issues.apache.org/jira/browse/ACCUMULO-4772
 Project: Accumulo
  Issue Type: Improvement
  Components: shell
Reporter: Mark Owens
Assignee: Mark Owens
Priority: Minor
 Fix For: 2.0.0


ACCUMULO-4732 adds the capability for NewTableConfiguration to preconfigure 
iterators and locality groups prior to table creation. Update the Accumulo 
shell to allow the same capability. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] jkrdev commented on a change in pull request #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
jkrdev commented on a change in pull request #341: ACCUMULO-3902 Ensure 
[Batch]Scanners are closed in ITs
URL: https://github.com/apache/accumulo/pull/341#discussion_r158302664
 
 

 ##
 File path: test/src/main/java/org/apache/accumulo/test/MetaGetsReadersIT.java
 ##
 @@ -112,6 +113,7 @@ public void test() throws Exception {
 t2.interrupt();
 t1.join();
 t2.join();
+m.close();
 
 Review comment:
   Actually the scanner is called m here let me change that ha. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jkrdev commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
jkrdev commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed 
in ITs
URL: https://github.com/apache/accumulo/pull/341#issuecomment-353366635
 
 
   I have noticed that throughout many of the ITs there is outdated code and in 
some instances whole folders (proxy) of ITs that have code that is deprecated. 
I don't know how I got about making a ticket but I think one should be opened 
about it. Theres code that can be replaced and some ITs that need to be fully 
refactored. I don't know if there is already a ticket about this but would be a 
nice fix. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jkrdev commented on a change in pull request #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
jkrdev commented on a change in pull request #341: ACCUMULO-3902 Ensure 
[Batch]Scanners are closed in ITs
URL: https://github.com/apache/accumulo/pull/341#discussion_r158286817
 
 

 ##
 File path: 
test/src/main/java/org/apache/accumulo/test/BatchWriterInTabletServerIT.java
 ##
 @@ -104,24 +104,34 @@ private void test(String t1, String t2, Connector c, 
IteratorSetting itset, int
 c.tableOperations().attachIterator(t2, summer);
 
 Map.Entry actual;
-// Scan t1 with an iterator that writes to table t2
-Scanner scanner = c.createScanner(t1, Authorizations.EMPTY);
-scanner.addScanIterator(itset);
-actual = Iterators.getOnlyElement(scanner.iterator());
-Assert.assertTrue(actual.getKey().equals(k, 
PartialKey.ROW_COLFAM_COLQUAL));
-Assert.assertEquals(BatchWriterIterator.SUCCESS_VALUE, actual.getValue());
-scanner.close();
+Scanner scanner = null;
+try {
+  // Scan t1 with an iterator that writes to table t2
+  scanner = c.createScanner(t1, Authorizations.EMPTY);
+  scanner.addScanIterator(itset);
+  actual = Iterators.getOnlyElement(scanner.iterator());
+  Assert.assertTrue(actual.getKey().equals(k, 
PartialKey.ROW_COLFAM_COLQUAL));
+  Assert.assertEquals(BatchWriterIterator.SUCCESS_VALUE, 
actual.getValue());
+} finally {
+  if (scanner != null) {
+scanner.close();
+  }
+}
 
-// ensure entries correctly wrote to table t2
-scanner = c.createScanner(t2, Authorizations.EMPTY);
-actual = Iterators.getOnlyElement(scanner.iterator());
-log.debug("t2 entry is " + actual.getKey().toStringNoTime() + " -> " + 
actual.getValue());
-Assert.assertTrue(actual.getKey().equals(k, 
PartialKey.ROW_COLFAM_COLQUAL));
-Assert.assertEquals(numEntriesToWritePerEntry, 
Integer.parseInt(actual.getValue().toString()));
-scanner.close();
+try {
+  // ensure entries correctly wrote to table t2
+  scanner = c.createScanner(t2, Authorizations.EMPTY);
+  actual = Iterators.getOnlyElement(scanner.iterator());
+  log.debug("t2 entry is " + actual.getKey().toStringNoTime() + " -> " + 
actual.getValue());
+  Assert.assertTrue(actual.getKey().equals(k, 
PartialKey.ROW_COLFAM_COLQUAL));
+  Assert.assertEquals(numEntriesToWritePerEntry, 
Integer.parseInt(actual.getValue().toString()));
+} finally {
+  if (scanner != null) {
+scanner.close();
+  }
+}
 
 Review comment:
   I missed this comment, so my general comment, is sort of answered by this. I 
will do this or at least try. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent API for Mutation

2017-12-21 Thread GitBox 
keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent 
API for Mutation
URL: https://github.com/apache/accumulo/pull/347#discussion_r158283704
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -772,6 +772,236 @@ public void putDelete(byte[] columnFamily, byte[] 
columnQualifier, ColumnVisibil
 put(columnFamily, columnQualifier, columnVisibility.getExpression(), true, 
timestamp, true, EMPTY_BYTES);
   }
 
+  /**
+   * Fluent API for Mutation
+   *
+   * 
+   * Each step of the Mutation is represented by an interface. Inheritance 
order ensures that chained methods follow prescribed order of:
+   *
+   * 
+   * row, family, qualifier, visibility, timestamp
+   *
+   * 
+   * set and delete methods end the chain and finalize the mutation
+   */
+  public interface RowStep extends FamilyStep {
+FamilyStep row(byte[] r);
+
+FamilyStep row(CharSequence r);
+
+FamilyStep row(ByteBuffer r);
+
+FamilyStep row(Text t);
+  }
+
+  public interface FamilyStep extends QualifierStep {
+QualifierStep family(byte[] f);
+
+QualifierStep family(CharSequence f);
+
+QualifierStep family(ByteBuffer f);
+
+QualifierStep family(Text t);
+  }
+
+  public interface QualifierStep extends VisibilityStep {
+VisibilityStep qualifier(byte[] q);
+
+VisibilityStep qualifier(CharSequence q);
+
+VisibilityStep qualifier(ByteBuffer q);
+
+VisibilityStep qualifier(Text t);
+  }
+
+  public interface VisibilityStep extends TimestampStep {
+TimestampStep visibility(byte[] cv);
+
+TimestampStep visibility(ColumnVisibility cv);
+  }
+
+  public interface TimestampStep extends MutationStep {
+MutationStep timestamp(long t);
+  }
+
+  public interface MutationStep {
+void set(byte[] v, boolean delete);
+
+void set(ByteBuffer v);
+
+void set(Value v);
+
+void delete();
+  }
+
+  public RowStep at() {
 
 Review comment:
   This method will need a `@since 2.0` javadoc tag.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent API for Mutation

2017-12-21 Thread GitBox 
keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent 
API for Mutation
URL: https://github.com/apache/accumulo/pull/347#discussion_r158282911
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -772,6 +772,236 @@ public void putDelete(byte[] columnFamily, byte[] 
columnQualifier, ColumnVisibil
 put(columnFamily, columnQualifier, columnVisibility.getExpression(), true, 
timestamp, true, EMPTY_BYTES);
   }
 
+  /**
+   * Fluent API for Mutation
+   *
+   * 
+   * Each step of the Mutation is represented by an interface. Inheritance 
order ensures that chained methods follow prescribed order of:
+   *
+   * 
+   * row, family, qualifier, visibility, timestamp
+   *
+   * 
+   * set and delete methods end the chain and finalize the mutation
+   */
+  public interface RowStep extends FamilyStep {
 
 Review comment:
   There should not be a row step.  All changes to a mutation are for a single 
row.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent API for Mutation

2017-12-21 Thread GitBox 
keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent 
API for Mutation
URL: https://github.com/apache/accumulo/pull/347#discussion_r158282561
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -772,6 +772,236 @@ public void putDelete(byte[] columnFamily, byte[] 
columnQualifier, ColumnVisibil
 put(columnFamily, columnQualifier, columnVisibility.getExpression(), true, 
timestamp, true, EMPTY_BYTES);
   }
 
+  /**
+   * Fluent API for Mutation
+   *
+   * 
+   * Each step of the Mutation is represented by an interface. Inheritance 
order ensures that chained methods follow prescribed order of:
+   *
+   * 
+   * row, family, qualifier, visibility, timestamp
+   *
+   * 
+   * set and delete methods end the chain and finalize the mutation
+   */
+  public interface RowStep extends FamilyStep {
+FamilyStep row(byte[] r);
+
+FamilyStep row(CharSequence r);
+
+FamilyStep row(ByteBuffer r);
+
+FamilyStep row(Text t);
 
 Review comment:
   Personally I think consistency is important.  As long as Text methods 
elsewhere are not deprecated I think it should be supported here.  For better 
or worse Text is widely used in the Accumulo API.  Some Accumulo methods only 
accept Text, which means in some cases users may already have Text objects that 
could also be used here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent API for Mutation

2017-12-21 Thread GitBox 
keith-turner commented on a change in pull request #347: ACCUMULO-4746 Fluent 
API for Mutation
URL: https://github.com/apache/accumulo/pull/347#discussion_r158283504
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -772,6 +772,236 @@ public void putDelete(byte[] columnFamily, byte[] 
columnQualifier, ColumnVisibil
 put(columnFamily, columnQualifier, columnVisibility.getExpression(), true, 
timestamp, true, EMPTY_BYTES);
   }
 
+  /**
+   * Fluent API for Mutation
+   *
+   * 
+   * Each step of the Mutation is represented by an interface. Inheritance 
order ensures that chained methods follow prescribed order of:
+   *
+   * 
+   * row, family, qualifier, visibility, timestamp
+   *
+   * 
+   * set and delete methods end the chain and finalize the mutation
+   */
+  public interface RowStep extends FamilyStep {
+FamilyStep row(byte[] r);
+
+FamilyStep row(CharSequence r);
+
+FamilyStep row(ByteBuffer r);
+
+FamilyStep row(Text t);
+  }
+
+  public interface FamilyStep extends QualifierStep {
 
 Review comment:
   All of these interfaces will need a javadoc `@since 2.0` tag.  Do not need 
it on the method as they will inherit it from the class level javadoc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jkrdev commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed in ITs

2017-12-21 Thread GitBox 
jkrdev commented on issue #341: ACCUMULO-3902 Ensure [Batch]Scanners are closed 
in ITs
URL: https://github.com/apache/accumulo/pull/341#issuecomment-353340658
 
 
   So I will go through again and check them but most of the time when I use a 
`scanner = null;` outside of a try block followed by a `finally `block it is 
because the scanner gets re-assigned later on. When in a try-with-resources 
block the scanner becomes final therefore it can't be re-assigned later on. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jmark99 commented on a change in pull request #337: ACCUMULO-4732 No APIs to configure iterators or locality groups for new tables

2017-12-21 Thread GitBox 
jmark99 commented on a change in pull request #337: ACCUMULO-4732 No APIs to 
configure iterators or locality groups for new tables
URL: https://github.com/apache/accumulo/pull/337#discussion_r158261473
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/client/admin/NewTableConfiguration.java
 ##
 @@ -217,17 +215,19 @@ public NewTableConfiguration 
attachIterator(IteratorSetting setting) throws Accu
*  object specifying the properties of the iterator
* @param scopes
*  enumerated set of iterator scopes
-   * @throws AccumuloException
-   *   if a general error occurs
*
* @since 2.0.0
*
* @see TableOperations#attachIterator(String, IteratorSetting, EnumSet)
*/
-  public NewTableConfiguration attachIterator(IteratorSetting setting, 
EnumSet scopes) throws AccumuloException {
+  public NewTableConfiguration attachIterator(IteratorSetting setting, 
EnumSet scopes) {
 Objects.requireNonNull(setting, "setting cannot be null!");
 Objects.requireNonNull(scopes, "scopes cannot be null!");
-TableOperationsHelper.checkIteratorConflicts(iteratorProps, setting, 
scopes);
+try {
+  TableOperationsHelper.checkIteratorConflicts(iteratorProps, setting, 
scopes);
+} catch (AccumuloException e) {
+  throw new IllegalArgumentException(e.getMessage());
 
 Review comment:
   Thanks for the suggestion @ctubbsii. Updated and committed the change.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services