date:20181017

[GitHub] milleruntime commented on a change in pull request #205: Separated setup into install & run command

2018-10-17 Thread GitBox

milleruntime commented on a change in pull request #205: Separated setup into 
install & run command
URL: https://github.com/apache/fluo-uno/pull/205#discussion_r226086096
 
 

 ##
 File path: plugins/accumulo-encryption.sh
 ##
 @@ -0,0 +1,30 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+source "$UNO_HOME"/bin/impl/util.sh
+
+if [[ $ACCUMULO_VERSION =~ ^1\..*$ ]]; then
+  echo "Encryption cannot be enabled for Accumulo 1.x"
+  exit 1
+fi
+
+accumulo_conf=$ACCUMULO_HOME/conf/accumulo.properties
+encrypt_key=$ACCUMULO_HOME/conf/data-encryption.key
+openssl rand -out $encrypt_key 32
+echo "instance.crypto.opts.key.provider=uri" >> "$accumulo_conf"
+echo "instance.crypto.opts.key.location=file://$encrypt_key" >> 
"$accumulo_conf"
 
 Review comment:
   key.provider has been removed and key.location is now key.uri


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner closed pull request #1053: Accumulo 2.0 and Hadoop 3.0 updates

2018-10-17 Thread GitBox

keith-turner closed pull request #1053: Accumulo 2.0 and Hadoop 3.0 updates
URL: https://github.com/apache/fluo/pull/1053
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.travis.yml b/.travis.yml
index 5a97b504..b6850d15 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -27,6 +27,5 @@ before_script:
   - unset _JAVA_OPTIONS
 env:
   - BUILD_ARGS="clean verify javadoc:jar"
-  - BUILD_ARGS="clean verify javadoc:jar -Daccumulo.version=1.9.2"
 script:
   - mvn $BUILD_ARGS
diff --git a/modules/accumulo/pom.xml b/modules/accumulo/pom.xml
index c423d859..e43759c2 100644
--- a/modules/accumulo/pom.xml
+++ b/modules/accumulo/pom.xml
@@ -42,7 +42,7 @@
 
 
   org.apache.hadoop
-  hadoop-client
+  hadoop-client-api
 
 
   org.apache.zookeeper
diff --git 
a/modules/accumulo/src/main/java/org/apache/fluo/accumulo/iterators/TimestampSkippingIterator.java
 
b/modules/accumulo/src/main/java/org/apache/fluo/accumulo/iterators/TimestampSkippingIterator.java
index 75ba96f4..3279ec2f 100644
--- 
a/modules/accumulo/src/main/java/org/apache/fluo/accumulo/iterators/TimestampSkippingIterator.java
+++ 
b/modules/accumulo/src/main/java/org/apache/fluo/accumulo/iterators/TimestampSkippingIterator.java
@@ -4,9 +4,9 @@
  * copyright ownership. The ASF licenses this file to you under the Apache 
License, Version 2.0 (the
  * "License"); you may not use this file except in compliance with the 
License. You may obtain a
  * copy of the License at
- * 
+ *
  * http://www.apache.org/licenses/LICENSE-2.0
- * 
+ *
  * Unless required by applicable law or agreed to in writing, software 
distributed under the License
  * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 
KIND, either express
  * or implied. See the License for the specific language governing permissions 
and limitations under
@@ -16,11 +16,9 @@
 package org.apache.fluo.accumulo.iterators;
 
 import java.io.IOException;
-import java.lang.reflect.Field;
 import java.util.Collection;
 import java.util.Map;
 
-import com.google.common.annotations.VisibleForTesting;
 import org.apache.accumulo.core.data.ByteSequence;
 import org.apache.accumulo.core.data.Key;
 import org.apache.accumulo.core.data.PartialKey;
@@ -28,14 +26,7 @@
 import org.apache.accumulo.core.data.Value;
 import org.apache.accumulo.core.iterators.IteratorEnvironment;
 import org.apache.accumulo.core.iterators.SortedKeyValueIterator;
-import org.apache.accumulo.core.iterators.SortedMapIterator;
-import org.apache.accumulo.core.iterators.WrappingIterator;
-import org.apache.accumulo.core.iterators.system.DeletingIterator;
-import org.apache.accumulo.core.iterators.system.SourceSwitchingIterator;
-import org.apache.accumulo.core.iterators.system.SynchronizedIterator;
 import org.apache.fluo.accumulo.util.ColumnConstants;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 
 /**
  * The purpose of this iterator is to make seeking within a columns timestamp 
range efficient.
@@ -49,12 +40,6 @@
   private Range range;
   private Collection fams;
   private boolean inclusive;
-  private boolean hasSeeked = false;
-
-  private boolean removedDeletingIterator = false;
-  private int removalFailures = 0;
-
-  private static final Logger log = 
LoggerFactory.getLogger(TimestampSkippingIterator.class);
 
   public TimestampSkippingIterator(SortedKeyValueIterator source) {
 this.source = source;
@@ -82,7 +67,7 @@ public void skipToTimestamp(Key curCol, long timestamp) 
throws IOException {
 while (source.hasTop()
 && curCol.equals(source.getTopKey(), 
PartialKey.ROW_COLFAM_COLQUAL_COLVIS)
 && timestamp < source.getTopKey().getTimestamp()) {
-  if (count == 10 && shouldSeek()) {
+  if (count == 10) {
 // seek to prefix
 Key seekKey = new Key(curCol);
 seekKey.setTimestamp(timestamp);
@@ -124,93 +109,8 @@ public void skipColumn(Key curCol) throws IOException {
 }
   }
 
-  @SuppressWarnings("unchecked")
-  private static SortedKeyValueIterator getParent(
-  SortedKeyValueIterator iter) {
-try {
-  if (iter instanceof WrappingIterator) {
-Field field = WrappingIterator.class.getDeclaredField("source");
-field.setAccessible(true);
-return (SortedKeyValueIterator) field.get(iter);
-  } else if (iter instanceof SourceSwitchingIterator) {
-// TODO could sync on SSI
-Field field = SourceSwitchingIterator.class.getDeclaredField("iter");
-field.setAccessible(true);
-return (SortedKeyValueIterator) field.get(iter);
-  } else if (iter instanceof SynchronizedIterator) {
-Field field = SynchronizedIterator.class.getDeclaredField("source");
-field.setAccessible(true);
-return

Build failed in Jenkins: Fluo #185

2018-10-17 Thread Apache Jenkins Server

See 

Changes:

[kturner] Updates for Accumulo 2 and Hadoop 3

[kturner] Use Accumulo 2.0.0-alpha-1

[kturner] Use new Accumulo create table functionality

[kturner] Fix accumulo-maven-plugin

[kturner] Disable delete and remove iterator reflection code

[kturner] update travis.yml

[ctubbsii] Add pluginRepository for accumulo-maven-plugin SNAPSHOT

--
[...truncated 770.31 KB...]
[ERROR] at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81)
[ERROR] at org.eclipse.sisu.wire.BeanProviders.firstOf(BeanProviders.java:179)
[ERROR] at org.eclipse.sisu.wire.BeanProviders$7.get(BeanProviders.java:160)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
[ERROR] at 
com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:133)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63)
[ERROR] at 
com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45)
[ERROR] at 
com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[ERROR] at 
com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[ERROR] at 
com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:104)
[ERROR] at 
com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32)
[ERROR] at 
com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:89)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:133)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
[ERROR] at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:87)
[ERROR] at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
[ERROR] at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
[ERROR] at 
com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
[ERROR] at 
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
[ERROR] at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
[ERROR] at org.eclipse.sisu.inject.Guice4$1.get(Guice4.java:162)
[ERROR] at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81)
[ERROR] at org.eclipse.sisu.wire.BeanProviders.firstOf(BeanProviders.java:179)
[ERROR] at org.eclipse.sisu.wire.BeanProviders$7.get(BeanProviders.java:160)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
[ERROR] at 
com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:65)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:133)
[ERROR] at 
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
[ERROR] at 
com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:63)
[ERROR] at 
com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45)
[ERROR] at 
com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[ERROR] at 
com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[ERROR] at 
com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:104)
[ERROR] at 
com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32)
[ERROR] at 
com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:89)
[ERROR] at

[GitHub] mikewalch opened a new pull request #205: Separated setup into install & run command

2018-10-17 Thread GitBox

mikewalch opened a new pull request #205: Separated setup into install & run 
command
URL: https://github.com/apache/fluo-uno/pull/205
 
 
   * Created plugin system
   * Updated source code headers for Apache


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] mikewalch commented on a change in pull request #205: Separated setup into install & run command

2018-10-17 Thread GitBox

mikewalch commented on a change in pull request #205: Separated setup into 
install & run command
URL: https://github.com/apache/fluo-uno/pull/205#discussion_r226087021
 
 

 ##
 File path: plugins/accumulo-encryption.sh
 ##
 @@ -0,0 +1,30 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+source "$UNO_HOME"/bin/impl/util.sh
+
+if [[ $ACCUMULO_VERSION =~ ^1\..*$ ]]; then
+  echo "Encryption cannot be enabled for Accumulo 1.x"
+  exit 1
+fi
+
+accumulo_conf=$ACCUMULO_HOME/conf/accumulo.properties
+encrypt_key=$ACCUMULO_HOME/conf/data-encryption.key
+openssl rand -out $encrypt_key 32
+echo "instance.crypto.opts.key.provider=uri" >> "$accumulo_conf"
+echo "instance.crypto.opts.key.location=file://$encrypt_key" >> 
"$accumulo_conf"
 
 Review comment:
   was this done for 2.0.0-alpha-1?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] milleruntime commented on a change in pull request #205: Separated setup into install & run command

2018-10-17 Thread GitBox

milleruntime commented on a change in pull request #205: Separated setup into 
install & run command
URL: https://github.com/apache/fluo-uno/pull/205#discussion_r226090568
 
 

 ##
 File path: plugins/accumulo-encryption.sh
 ##
 @@ -0,0 +1,30 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+source "$UNO_HOME"/bin/impl/util.sh
+
+if [[ $ACCUMULO_VERSION =~ ^1\..*$ ]]; then
+  echo "Encryption cannot be enabled for Accumulo 1.x"
+  exit 1
+fi
+
+accumulo_conf=$ACCUMULO_HOME/conf/accumulo.properties
+encrypt_key=$ACCUMULO_HOME/conf/data-encryption.key
+openssl rand -out $encrypt_key 32
+echo "instance.crypto.opts.key.provider=uri" >> "$accumulo_conf"
+echo "instance.crypto.opts.key.location=file://$encrypt_key" >> 
"$accumulo_conf"
 
 Review comment:
   Yup I think it was the last thing that got in... 
https://github.com/apache/accumulo/commit/9247fbeadeb9c63e040947e0d7649de95e2252a2


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner opened a new issue #1054: Use summarizers to make compaction decisions

2018-10-17 Thread GitBox

keith-turner opened a new issue #1054: Use summarizers to make compaction 
decisions
URL: https://github.com/apache/fluo/issues/1054
 
 
   In Accumulo 2.0 summarizes were introduced.   Fluo could use these to count 
column types and notifications.  These counts could be use to make compaction 
decisions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner opened a new issue #1055: Look into a dedicated scan executor for notification scans

2018-10-17 Thread GitBox

keith-turner opened a new issue #1055: Look into a dedicated scan executor for 
notification scans
URL: https://github.com/apache/fluo/issues/1055
 
 
   Scanning for notifications is a continuous activity that Fluo does.   It may 
make sense to do one of the following.
   
 * Have a dedicated thread pool for notifcation scans with a small number 
of threads like Math.max(1,0.10 * numCores)
 * Have a prioritizer that gives notification scans a lower priority. 
   
   Fluo could offer the prioritizer and/or dispatcher AND utilities or docs to 
configure them.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on issue #1055: Look into a dedicated scan executor for notification scans

2018-10-17 Thread GitBox

keith-turner commented on issue #1055: Look into a dedicated scan executor for 
notification scans
URL: https://github.com/apache/fluo/issues/1055#issuecomment-430618511
 
 
   For prioritization, Fluo could use the built in prioritizer with scan 
execution hints.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] milleruntime commented on a change in pull request #205: Separated setup into install & run command

[GitHub] keith-turner closed pull request #1053: Accumulo 2.0 and Hadoop 3.0 updates

Build failed in Jenkins: Fluo #185

[GitHub] mikewalch opened a new pull request #205: Separated setup into install & run command

[GitHub] mikewalch commented on a change in pull request #205: Separated setup into install & run command

[GitHub] milleruntime commented on a change in pull request #205: Separated setup into install & run command

[GitHub] keith-turner opened a new issue #1054: Use summarizers to make compaction decisions

[GitHub] keith-turner opened a new issue #1055: Look into a dedicated scan executor for notification scans

[GitHub] keith-turner commented on issue #1055: Look into a dedicated scan executor for notification scans

9 matches

Site Navigation

Mail list logo

Footer information