[GitHub] activemq-artemis pull request #2494: ARTEMIS-2224 Reduce contention on LiveP...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2494#discussion_r246601217 --- Diff: artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/ConcurrentAppendOnlyChunkedList.java --- @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.activemq.artemis.utils.collections; + +import java.util.Objects; +import java.util.concurrent.atomic.AtomicLongFieldUpdater; +import java.util.concurrent.atomic.AtomicReferenceArray; +import java.util.function.IntFunction; + +/** + * This collection is a concurrent append-only list that grows in chunks. + * It's safe to be used by many threads concurrently and has a max capacity of {@link Integer#MAX_VALUE}. + */ +public final class ConcurrentAppendOnlyChunkedList { --- End diff -- This is a lot better! ---
[GitHub] activemq-artemis pull request #2494: ARTEMIS-2224 Reduce contention on LiveP...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2494#discussion_r246601169 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/LivePageCacheImpl.java --- @@ -48,54 +82,228 @@ public long getPageId() { } @Override - public synchronized int getNumberOfMessages() { - return messages.size(); + public int getNumberOfMessages() { + while (true) { + final long size = producerIndex; + if (size == RESIZING) { +Thread.yield(); +continue; + } + return (int) Math.min(size, Integer.MAX_VALUE); + } } @Override - public synchronized void setMessages(PagedMessage[] messages) { + public void setMessages(PagedMessage[] messages) { // This method shouldn't be called on liveCache, but we will provide the implementation for it anyway for (PagedMessage msg : messages) { addLiveMessage(msg); } } @Override - public synchronized PagedMessage getMessage(int messageNumber) { - if (messageNumber < messages.size()) { - return messages.get(messageNumber); - } else { + public PagedMessage getMessage(int messageNumber) { + if (messageNumber < 0) { return null; } + //it allow to perform less cache invalidations vs producerIndex if there are bursts of appends + long size = lastSeenProducerIndex; + if (messageNumber >= size) { + while ((size = producerIndex) == RESIZING) { +Thread.yield(); + } + //it is a message over the current size? + if (messageNumber >= size) { +return null; + } + //publish it for others consumers + LAST_PRODUCER_INDEX_UPDATER.lazySet(this, size); + } + final AtomicChunk buffer; + final int offset; + if (messageNumber >= chunkSize) { + offset = messageNumber & chunkMask; + //slow path is moved in a separate method + buffer = jump(messageNumber, size); + } else { + offset = messageNumber; + buffer = consumerBuffer; + } + //NOTE: producerIndex is being updated before setting a new value ie on consumer side need to spin until a not null value is set + PagedMessage msg; + while ((msg = buffer.get(offset)) == null) { + Thread.yield(); + } + return msg; + } + + /** +* Implements a lock-free version of the optimization used on {@link java.util.LinkedList#get(int)} to speed up queries +* ie backward search of a node if needed. +*/ + private AtomicChunk jump(final int messageNumber, final long size) { + //fast division by a power of 2 + final int jumps = messageNumber >> chunkSizeLog2; + //size is never allowed to be > Integer.MAX_VALUE + final int lastChunkIndex = (int) size >> chunkSizeLog2; + int requiredJumps = jumps; + AtomicChunk jumpBuffer = null; + boolean jumpForward = true; + int distanceFromLastChunkIndex = lastChunkIndex - jumps; + //it's worth to go backward from lastChunkIndex? + //trying first to check against the value we already have: if it won't worth, won't make sense to load the producerBuffer + if (distanceFromLastChunkIndex < jumps) { + final AtomicChunk producer = producerBuffer; + //producer is a potential moving, always increasing, target ie better to re-check the distance + distanceFromLastChunkIndex = producer.index - jumps; + if (distanceFromLastChunkIndex < jumps) { +//we're saving some jumps ie is fine to go backward from here +jumpBuffer = producer; +requiredJumps = distanceFromLastChunkIndex; +jumpForward = false; + } + } + //start from the consumer buffer only is needed + if (jumpBuffer == null) { + jumpBuffer = consumerBuffer; + } + for (int i = 0; i < requiredJumps; i++) { + //next chunk is always set if below a read producerIndex value + //previous chunk is final and can be safely read + jumpBuffer = jumpForward ? jumpBuffer.next : jumpBuffer.prev; + } + return jumpBuffer; } @Override - public synchronized boolean isLive() { + public boolean isLive() { return isLive; } @Override - public synchronized void addLiveMessage(PagedMessage message) {
[GitHub] activemq-artemis pull request #2494: ARTEMIS-2224 Reduce contention on LiveP...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2494#discussion_r246558406 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/LivePageCacheImpl.java --- @@ -28,15 +28,49 @@ /** * This is the same as PageCache, however this is for the page that's being currently written. */ --- End diff -- Isn't this mixing the collection implementation itself into the LivePageCache? isn't there a way to implement this logic into its own structure? Like PageCache using a generic ChunkArray (a name I just came up here)? I'm a bit concerned on maintaining the business side of this issue (that is the PageCache) with the speedy implementation of a collection. ---
[GitHub] activemq-artemis pull request #2492: ARTEMIS-2222 why the position remains u...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2492#discussion_r246538095 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java --- @@ -1315,9 +1315,7 @@ private PagedReference moveNext() { } } - if (!ignored) { - position = message.getPosition(); - } + position = message.getPosition(); --- End diff -- I think this made sense at some point, but after a few fixes later it's harmless. I don't think this needs a JIRA as there's no issue I think. I will merge and keep the JIRA, but I will rename the commit from being a question into something more affirmative. ---
[GitHub] activemq-artemis issue #2467: ARTEMIS-2205 Performance improvements on AMQP ...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2467 @michaelandrepearce / @franz1981 all done. I squashed my commits. It was hard to keep them separated due to these changes being inter-related. ---
[GitHub] activemq-artemis pull request #:
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/commit/0f9bf15788096ad9dc795954174c3b496861932c#commitcomment-31886537 In artemis-ra/src/main/java/org/apache/activemq/artemis/ra/inflow/ActiveMQMessageHandler.java: In artemis-ra/src/main/java/org/apache/activemq/artemis/ra/inflow/ActiveMQMessageHandler.java on line 347: @andytaylor I had a lot of headache here. Weeks testing this while sitting close to a production/test environment. I would double check this carefully. This kind of thing is hard to be tested. ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r246058995 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/ProtonServerSenderContext.java --- @@ -730,22 +793,29 @@ public int deliverMessage(MessageReference messageReference, int deliveryCount, if (preSettle) { // Presettled means the client implicitly accepts any delivery we send it. - sessionSPI.ack(null, brokerConsumer, messageReference.getMessage()); + try { + sessionSPI.ack(null, brokerConsumer, messageReference.getMessage()); + } catch (Exception e) { + log.debug(e.getMessage(), e); + } delivery.settle(); } else { sender.advance(); } connection.flush(); } finally { -connection.unlock(); +synchronized (creditsLock) { + pending.decrementAndGet(); +} +if (releaseRequired) { + ((NettyReadable) sendBuffer).getByteBuf().release(); +} } + } catch (Exception e) { + log.warn(e.getMessage(), e); - return size; - } finally { - if (releaseRequired) { -((NettyReadable) sendBuffer).getByteBuf().release(); - } + // important todo: Error treatment --- End diff -- I'm working on it. I'm out today on a meeting... will be done tomorrow (Wed) ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r245777806 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/AMQPConnectionContext.java --- @@ -359,59 +371,77 @@ public boolean flowControl(ReadyListener readyListener) { @Override public void onRemoteOpen(Connection connection) throws Exception { - lock(); + handler.requireHandler(); try { - try { -initInternal(); - } catch (Exception e) { -log.error("Error init connection", e); - } - if (!validateConnection(connection)) { -connection.close(); - } else { -connection.setContext(AMQPConnectionContext.this); -connection.setContainer(containerId); -connection.setProperties(connectionProperties); - connection.setOfferedCapabilities(getConnectionCapabilitiesOffered()); -connection.open(); - } - } finally { - unlock(); + initInternal(); + } catch (Exception e) { + log.error("Error init connection", e); + } + if (!validateConnection(connection)) { + connection.close(); + } else { + connection.setContext(AMQPConnectionContext.this); + connection.setContainer(containerId); + connection.setProperties(connectionProperties); + connection.setOfferedCapabilities(getConnectionCapabilitiesOffered()); + connection.open(); } initialise(); - /* - * This can be null which is in effect an empty map, also we really don't need to check this for in bound connections - * but its here in case we add support for outbound connections. - * */ + /* + * This can be null which is in effect an empty map, also we really don't need to check this for in bound connections + * but its here in case we add support for outbound connections. + * */ if (connection.getRemoteProperties() == null || !connection.getRemoteProperties().containsKey(CONNECTION_OPEN_FAILED)) { long nextKeepAliveTime = handler.tick(true); if (nextKeepAliveTime != 0 && scheduledPool != null) { -scheduledPool.schedule(new Runnable() { - @Override - public void run() { - Long rescheduleAt = handler.tick(false); - if (rescheduleAt == null) { - // this mean tick could not acquire a lock, we will just retry in 10 milliseconds. - scheduledPool.schedule(this, 10, TimeUnit.MILLISECONDS); - } else if (rescheduleAt != 0) { - scheduledPool.schedule(this, rescheduleAt - TimeUnit.NANOSECONDS.toMillis(System.nanoTime()), TimeUnit.MILLISECONDS); - } - } -}, (nextKeepAliveTime - TimeUnit.NANOSECONDS.toMillis(System.nanoTime())), TimeUnit.MILLISECONDS); +scheduledPool.schedule(new ScheduleRunnable(), (nextKeepAliveTime - TimeUnit.NANOSECONDS.toMillis(System.nanoTime())), TimeUnit.MILLISECONDS); } } } + class TickerRunnable implements Runnable { + + final ScheduleRunnable scheduleRunnable; + + TickerRunnable(ScheduleRunnable scheduleRunnable) { + this.scheduleRunnable = scheduleRunnable; + } + + @Override + public void run() { + try { +Long rescheduleAt = handler.tick(false); +if (rescheduleAt == null) { + // this mean tick could not acquire a lock, we will just retry in 10 milliseconds. + scheduledPool.schedule(scheduleRunnable, 10, TimeUnit.MILLISECONDS); +} else if (rescheduleAt != 0) { + scheduledPool.schedule(scheduleRunnable, rescheduleAt - TimeUnit.NANOSECONDS.toMillis(System.nanoTime()), TimeUnit.MILLISECONDS); +} + } catch (Exception e) { +log.warn(e.getMessage(), e); + } --- End diff -- I'm removing the catch here. I don't think we need to use specific loggers on generic handlers like this though. it was a generic handler.. but it's being removed. ---
[GitHub] activemq-artemis issue #2469: NO-JIRA avoid unnecessary Bindings instance cr...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2469 I think this warrants a JIRA. ---
[GitHub] activemq-artemis issue #2484: ARTEMIS-2216 Use a specific executor for pageS...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2484 during initial development years ago I needed to be careful when to leave paging. this needed some sync points to make sure depage would set it synchronously. I'm not saying anything against the change (I'm still in vacation mode, I'm back on monday).. just saying what was the original intent when it was written. ---
[GitHub] activemq-artemis issue #2484: ARTEMIS-2216 Use a specific executor for pageS...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2484 The receiver could be slower because of more IO. Thatâs not a bad thing. Iâm on vacation this week. I wonât be able to get to this. Just saying why the receiver could be slower. ---
[GitHub] activemq-artemis pull request #2474: [ARTEMIS-1536]: Incorrect Journal files...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2474#discussion_r243338608 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java --- @@ -163,13 +163,17 @@ protected void init(Configuration config, IOCriticalErrorListener criticalErrorL int fileSize = config.getJournalFileSize(); // we need to correct the file size if its not a multiple of the alignement - int modulus = fileSize % journalFF.getAlignment(); - if (modulus != 0) { - int difference = modulus; - int low = config.getJournalFileSize() - difference; - int high = low + journalFF.getAlignment(); - fileSize = difference < journalFF.getAlignment() / 2 ? low : high; - ActiveMQServerLogger.LOGGER.invalidJournalFileSize(config.getJournalFileSize(), fileSize, journalFF.getAlignment()); + if (fileSize <= journalFF.getAlignment()) { + fileSize = journalFF.getAlignment(); --- End diff -- I will just mere it.. but if you can add an unit test... ---
[GitHub] activemq-artemis pull request #2474: [ARTEMIS-1536]: Incorrect Journal files...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2474#discussion_r243338301 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java --- @@ -163,13 +163,17 @@ protected void init(Configuration config, IOCriticalErrorListener criticalErrorL int fileSize = config.getJournalFileSize(); // we need to correct the file size if its not a multiple of the alignement - int modulus = fileSize % journalFF.getAlignment(); - if (modulus != 0) { - int difference = modulus; - int low = config.getJournalFileSize() - difference; - int high = low + journalFF.getAlignment(); - fileSize = difference < journalFF.getAlignment() / 2 ? low : high; - ActiveMQServerLogger.LOGGER.invalidJournalFileSize(config.getJournalFileSize(), fileSize, journalFF.getAlignment()); + if (fileSize <= journalFF.getAlignment()) { + fileSize = journalFF.getAlignment(); --- End diff -- Lets not complicate things... but a unit test would be nice :) ---
[GitHub] activemq-artemis pull request #2474: [ARTEMIS-1536]: Incorrect Journal files...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2474#discussion_r243283678 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java --- @@ -163,13 +163,17 @@ protected void init(Configuration config, IOCriticalErrorListener criticalErrorL int fileSize = config.getJournalFileSize(); // we need to correct the file size if its not a multiple of the alignement - int modulus = fileSize % journalFF.getAlignment(); - if (modulus != 0) { - int difference = modulus; - int low = config.getJournalFileSize() - difference; - int high = low + journalFF.getAlignment(); - fileSize = difference < journalFF.getAlignment() / 2 ? low : high; - ActiveMQServerLogger.LOGGER.invalidJournalFileSize(config.getJournalFileSize(), fileSize, journalFF.getAlignment()); + if (fileSize <= journalFF.getAlignment()) { + fileSize = journalFF.getAlignment(); --- End diff -- I would actually throw an exception here. I wouldn't expect a block size that low. you can only have the header on the journal and nothing else if you specify fileSize == alignment. ---
[GitHub] activemq-artemis issue #2467: ARTEMIS-2205 Performance improvements on AMQP ...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2467 @franz1981 I have pulled your netty change here for correctness. (Now with everything async I had an issue and I almost came up with a similar fix..yours was better and ready) :) ---
[GitHub] activemq-artemis pull request #2473: ARTEMIS-1058 Jars in web tmp dir locked...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2473#discussion_r243083234 --- Diff: artemis-web/src/main/java/org/apache/activemq/artemis/component/WebServerComponent.java --- @@ -237,28 +236,39 @@ public void start() throws Exception { public void internalStop() throws Exception { server.stop(); if (webContexts != null) { + File tmpdir = null; + StringBuilder strBuilder = new StringBuilder(); + boolean found = false; for (WebAppContext context : webContexts) { tmpdir = context.getTempDirectory(); -if (tmpdir != null && !context.isPersistTempDirectory()) { +if (tmpdir != null && tmpdir.exists() && !context.isPersistTempDirectory()) { //tmpdir will be removed by deleteOnExit() - //somehow when broker is stopped and restarted quickly - //this tmpdir won't get deleted sometimes - boolean fileDeleted = TimeUtils.waitOnBoolean(false, 5000, tmpdir::exists); - - if (!fileDeleted) { - //because the execution order of shutdown hooks are - //not determined, so it's possible that the deleteOnExit - //is executed after this hook, in that case we force a delete. - FileUtil.deleteDirectory(tmpdir); - logger.debug("Force to delete temporary file on shutdown: " + tmpdir.getAbsolutePath()); - if (tmpdir.exists()) { - ActiveMQWebLogger.LOGGER.tmpFileNotDeleted(tmpdir); - } + //However because the URLClassLoader never release/close its opened + //jars the jar file won't be able to get deleted on Windows platform + //until after the process fully terminated. To fix this here arranges + //a separate process to try clean up the temp dir + FileUtil.deleteDirectory(tmpdir); + if (tmpdir.exists()) { + ActiveMQWebLogger.LOGGER.tmpFileNotDeleted(tmpdir); + strBuilder.append(tmpdir); + strBuilder.append(","); + found = true; } } } + + if (found) { +String artemisHome = System.getProperty("artemis.home"); --- End diff -- can't you get the property from other means? Fileconfiguration for instance? ---
[GitHub] activemq-artemis issue #2470: Fixes for alerts from lgtm.com
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2470 all lgtm, pun intended If we could run the full testsuite just to be sure. ---
[GitHub] activemq-artemis pull request #2470: Fixes for alerts from lgtm.com
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2470#discussion_r243073035 --- Diff: artemis-cli/src/main/java/org/apache/activemq/artemis/cli/commands/tools/PrintData.java --- @@ -216,63 +216,63 @@ private static void printPages(DescribeJournal describeJournal, if (pgStore != null) { folder = pgStore.getFolder(); --- End diff -- There's a semantic change here, are you sure? ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242969136 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/handler/ExecutorNettyAdapter.java --- @@ -0,0 +1,221 @@ +/* + * Copyright 2005-2014 Red Hat, Inc. + * Red Hat licenses this file to you under the Apache License, version + * 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + *http://www.apache.org/licenses/LICENSE-2.0 + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +package org.apache.activemq.artemis.protocol.amqp.proton.handler; + +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; + +import io.netty.channel.Channel; +import io.netty.channel.ChannelFuture; +import io.netty.channel.ChannelPromise; +import io.netty.channel.EventLoop; +import io.netty.channel.EventLoopGroup; +import io.netty.util.concurrent.EventExecutor; +import io.netty.util.concurrent.Future; +import io.netty.util.concurrent.ProgressivePromise; +import io.netty.util.concurrent.Promise; +import io.netty.util.concurrent.ScheduledFuture; +import org.apache.activemq.artemis.utils.actors.ArtemisExecutor; + +/** Test cases may supply a simple executor instead of the real Netty Executor + * On that case this is a simple adapter for what's needed from these tests. + * Not intended to be used in production. + * + * TODO: This could be refactored out of the main codebase but at a high cost. + *We may do it some day if we find an easy way that won't clutter the code too much. + * */ +public class ExecutorNettyAdapter implements EventLoop { --- End diff -- @franz1981 that's correct. Placing it on the testsuite only would require some work that I do not want to go through. ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242968808 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/ProtonServerSenderContext.java --- @@ -730,22 +793,29 @@ public int deliverMessage(MessageReference messageReference, int deliveryCount, if (preSettle) { // Presettled means the client implicitly accepts any delivery we send it. - sessionSPI.ack(null, brokerConsumer, messageReference.getMessage()); + try { + sessionSPI.ack(null, brokerConsumer, messageReference.getMessage()); + } catch (Exception e) { + log.debug(e.getMessage(), e); + } delivery.settle(); } else { sender.advance(); } connection.flush(); } finally { -connection.unlock(); +synchronized (creditsLock) { + pending.decrementAndGet(); +} +if (releaseRequired) { + ((NettyReadable) sendBuffer).getByteBuf().release(); +} } + } catch (Exception e) { + log.warn(e.getMessage(), e); - return size; - } finally { - if (releaseRequired) { -((NettyReadable) sendBuffer).getByteBuf().release(); - } + // important todo: Error treatment --- End diff -- ouch... let me see ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242745363 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/ServerSessionImpl.java --- @@ -159,7 +160,7 @@ private final SimpleString managementAddress; - protected final RoutingContext routingContext = new RoutingContextImpl(null); + protected final RoutingContext _routingContext; --- End diff -- @franz1981 I reverted the change that's not needed any longer. all amended. ---
[GitHub] activemq-artemis pull request #2471: ARTEMIS-2207 Page Showing Log.warns for...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2471 ARTEMIS-2207 Page Showing Log.warns for regular acked messages You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis paging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2471.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2471 commit 9984b638af54ac6c2270604e046d739ed6b6f328 Author: Clebert Suconic Date: 2018-12-18T23:17:03Z ARTEMIS-2207 Page Showing Log.warns for regular acked messages ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242727332 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/ServerSessionImpl.java --- @@ -159,7 +160,7 @@ private final SimpleString managementAddress; - protected final RoutingContext routingContext = new RoutingContextImpl(null); + protected final RoutingContext _routingContext; --- End diff -- @franz1981 I don't follow you ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242727059 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/proton/handler/ExecutorNettyAdapter.java --- @@ -0,0 +1,221 @@ +/* + * Copyright 2005-2014 Red Hat, Inc. + * Red Hat licenses this file to you under the Apache License, version + * 2.0 (the "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + *http://www.apache.org/licenses/LICENSE-2.0 + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. See the License for the specific language governing + * permissions and limitations under the License. + */ + +package org.apache.activemq.artemis.protocol.amqp.proton.handler; + +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; + +import io.netty.channel.Channel; +import io.netty.channel.ChannelFuture; +import io.netty.channel.ChannelPromise; +import io.netty.channel.EventLoop; +import io.netty.channel.EventLoopGroup; +import io.netty.util.concurrent.EventExecutor; +import io.netty.util.concurrent.Future; +import io.netty.util.concurrent.ProgressivePromise; +import io.netty.util.concurrent.Promise; +import io.netty.util.concurrent.ScheduledFuture; +import org.apache.activemq.artemis.utils.actors.ArtemisExecutor; + +/** Test cases may supply a simple executor instead of the real Netty Executor + * On that case this is a simple adapter for what's needed from these tests. + * Not intended to be used in production. + * + * TODO: This could be refactored out of the main codebase but at a high cost. + *We may do it some day if we find an easy way that won't clutter the code too much. + * */ +public class ExecutorNettyAdapter implements EventLoop { --- End diff -- @franz1981 this is basically for unit testing, I did not want to invest a lot of code on it. ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2467#discussion_r242725348 --- Diff: artemis-protocols/artemis-amqp-protocol/src/main/java/org/apache/activemq/artemis/protocol/amqp/broker/ActiveMQProtonRemotingConnection.java --- @@ -119,10 +119,15 @@ public boolean isDestroyed() { @Override public void disconnect(boolean criticalError) { - ErrorCondition errorCondition = new ErrorCondition(); - errorCondition.setCondition(AmqpSupport.CONNECTION_FORCED); - amqpConnection.close(errorCondition); - getTransportConnection().close(); + amqpConnection.runLater(() -> { + ErrorCondition errorCondition = new ErrorCondition(); + errorCondition.setCondition(AmqpSupport.CONNECTION_FORCED); + amqpConnection.close(errorCondition); + amqpConnection.flush(); + }); + amqpConnection.runLater(() -> { --- End diff -- @franz1981 During my work, the first one was runNow, the second Later. (I would accept better names BTW) later I changed the first to later and forgot the other one :) fixing it on my next ammend ---
[GitHub] activemq-artemis issue #2466: ARTEMIS-2206 The MQTT consumer reconnection ca...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2466 If you don't know git, do some googling about squash and interactive rebase. git push -f will update the Pull Request. ---
[GitHub] activemq-artemis issue #2466: ARTEMIS-2206 The MQTT consumer reconnection ca...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2466 you do this on your branch: git rebase -i HEAD~4 for each line, squash all 3 later commits into the first one, Once you squash you will be sent to a vim (or another editor you configured) where you can change the tittle. make the title reflect the JIRA, git push origin -f (your branch name) That will be enough to update this PR. ---
[GitHub] activemq-artemis issue #2466: ARTEMIS-2206 The MQTT consumer reconnection ca...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2466 Please... squash your commits, and use the JIRA on the commit. It's kind of hard for me to review your changes here. what I saw on the first commit makes this a nogo.. but I didn't know if you fixed it on a later commit or not? ---
[GitHub] activemq-artemis pull request #2466: ARTEMIS-2206 The MQTT consumer reconnec...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2466#discussion_r242541448 --- Diff: artemis-protocols/artemis-mqtt-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/mqtt/MQTTSubscriptionManager.java --- @@ -194,12 +191,13 @@ private synchronized void removeSubscription(String address) throws Exception { SimpleString internalQueueName = getQueueNameForTopic(internalAddress); session.getSessionState().removeSubscription(address); - - ServerConsumer consumer = consumers.get(address); + Set queueConsumers = session.getServer().queueConsumersQuery(internalQueueName); --- End diff -- I think this is wrong. You should only check for consumers on this Session. Your logic is closing every consumer to the same address. You keep multiple connections and you have a failure. ---
[GitHub] activemq-artemis pull request #2466: ARTEMIS-2206 The MQTT consumer reconnec...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2466#discussion_r242540859 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/ActiveMQServerImpl.java --- @@ -113,32 +113,7 @@ import org.apache.activemq.artemis.core.security.SecurityAuth; import org.apache.activemq.artemis.core.security.SecurityStore; import org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl; -import org.apache.activemq.artemis.core.server.ActivateCallback; --- End diff -- This will fail checkstyle. no * imports ---
[GitHub] activemq-artemis issue #2462: ARTEMIS-2197 Page deleted before transaction f...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2462 @gaohoward can you close this. It's merged after a rebase (I fixed a test failure you had here). Merge doesn't work well on 2.6.x. ---
[GitHub] activemq-artemis pull request #2467: ARTEMIS-2205 Performance improvements o...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2467 ARTEMIS-2205 Performance improvements on AMQP and other parts You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis amqp-PR Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2467.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2467 commit 3199e32d2c98e7cdfd9ef157b9ea0cb19ca85e75 Author: Clebert Suconic Date: 2018-12-17T14:11:54Z ARTEMIS-2205 Refactor AMQP Processing into Single Threaded (per connection) https://issues.apache.org/jira/browse/ARTEMIS-2205 commit a72c73fad3b8b4ca03737c6733d018dbd385bfbd Author: Clebert Suconic Date: 2018-12-17T14:12:07Z ARTEMIS-2205 Broker Improvement by caching Routing for most common cases During AMQP Perf tests this became more relevant on profiling. It is a general improvement done as part of this AMQP performance task. commit 6d93d0aff908c23e68a135041d663c3c39701c72 Author: Clebert Suconic Date: 2018-12-17T14:12:14Z ARTEMIS-2205 Avoid new Runnable for every message sent As we now use the netty executor, creating a new Runnable for every message received became a relevant CPU, memory and putting extra GC pressure. By doing this change alone I was able to improve performance by 5% more or less. https://issues.apache.org/jira/browse/ARTEMIS-2205 commit c11605e00737f570ee4eddc62d9f2bacebcdfb2b Author: Francesco Nigro Date: 2018-12-17T14:12:19Z ARTEMIS-2205 Optimizing some Lambda usages https://issues.apache.org/jira/browse/ARTEMIS-2205 ---
[GitHub] activemq-artemis issue #2462: ARTEMIS-2197 Page deleted before transaction f...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2462 I think you should keep separated commits. or if you squash, make it clear that this is a two commits squashed please? ---
[GitHub] activemq-artemis issue #2462: ARTEMIS-2197 Page deleted before transaction f...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2462 There are tests broken on master. ---
[GitHub] activemq-artemis pull request #2458: ARTEMIS-2199 PagingStore leak when dele...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2458#discussion_r240419559 --- Diff: artemis-jms-server/src/main/java/org/apache/activemq/artemis/jms/server/impl/JMSServerManagerImpl.java --- @@ -1719,11 +1720,11 @@ public void callback(SimpleString queueName) throws Exception { @Override public void callback(SimpleString address, SimpleString queueName) throws Exception { Queue queue = server.locateQueue(address); - Collection bindings = server.getPostOffice().getBindingsForAddress(address).getBindings(); --- End diff -- Is there a master counterpart for this fix? ---
[GitHub] activemq-artemis issue #2287: ARTEMIS-2069 Backup doesn't activate after sha...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2287 My point is. -1 is forever. If you set it forever it will be failing at 2 seconds. What happens if you just configure the system at 2 seconds instead of -1? ---
[GitHub] activemq-artemis pull request #2287: ARTEMIS-2069 Backup doesn't activate af...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2287#discussion_r240232734 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/FileLockNodeManager.java --- @@ -49,6 +49,8 @@ private static final byte NOT_STARTED = 'N'; + private static final long LOCK_ACCESS_FAILURE_WAIT_TIME = 2000; --- End diff -- You are here effectively ignoring the acquire timeout where the user can configure it. I'm not sure this is correct. your test should play with a configured timeout and see if you get the expected behaviour. Adding this you are forcing your own timeout bypassing the configured one. ---
[GitHub] activemq-artemis issue #2457: ARTEMIS-2193 Artemis fails on OutOfMemoryError...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2457 Is this specific to drop ? ---
[GitHub] activemq-artemis issue #2449: ARTEMIS-2191 Makes PostOfficeImpl::route commo...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2449 @franz1981 the failures I mentioned are on my branch. I am asking you to defer your PR until after I merge mine on AMQP performance. This route change I'm making is a bit more functional than yours. It seems yours is a refactoring while mine had functionality involved.. hence I'm asking you to defer it Adding your change in top of mine added failues (the one you sent). ---
[GitHub] activemq-artemis issue #2449: ARTEMIS-2191 Makes PostOfficeImpl::route commo...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2449 @franz1981 your changes (on my branch) are causing a few failures. I really like your optimizations here. but can you close this. .and resend once I'm done? ---
[GitHub] activemq-artemis issue #2449: ARTEMIS-2191 Makes PostOfficeImpl::route commo...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2449 @franz1981 close this please? it will cause me a lot of headache if someone merged it before my branch? I'm pulling your change with my PR. ---
[GitHub] activemq-artemis issue #2449: ARTEMIS-2191 Makes PostOfficeImpl::route commo...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2449 @franz1981 can you hold this? or better.. can you send these optimizations into my new-amqp branch? I have optimized routing to cache it.. and it would take me longer to refactor your changes in than yourself doing it. ---
[GitHub] activemq-artemis issue #2444: ARTEMIS-2186 Large message incomplete when ser...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2444 I would just always sync on releaseResources. I don't see any case where we wanted to close without a previous sync given the semantic I missed. ---
[GitHub] activemq-artemis issue #2444: ARTEMIS-2186 Large message incomplete when ser...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2444 nice catch then.. i will merge this and back port into 2.6.x ---
[GitHub] activemq-artemis issue #2444: ARTEMIS-2186 Large message incomplete when ser...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2444 The release resources thing. ---
[GitHub] activemq-artemis issue #2444: ARTEMIS-2186 Large message incomplete when ser...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2444 I am a bit confused as a file.close would issue a sync. ---
[GitHub] activemq-artemis pull request #2444: ARTEMIS-2186 Large message incomplete w...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2444#discussion_r236681572 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/protocol/core/ServerSessionPacketHandler.java --- @@ -1018,6 +1018,7 @@ private void sendContinuations(final int packetSize, currentLargeMessage.addBytes(body); if (!continues) { +currentLargeMessage.sync(); currentLargeMessage.releaseResources(); --- End diff -- wouldn't releaseResources make a sync when the file is closed? ---
[GitHub] activemq-artemis pull request #2427: ARTEMIS-2170 Optimized CoreMessage's ch...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2427#discussion_r234767548 --- Diff: artemis-core-client/src/main/java/org/apache/activemq/artemis/core/message/impl/CoreMessage.java --- @@ -52,6 +52,11 @@ * consumers */ public class CoreMessage extends RefCountMessage implements ICoreMessage { + // We use properties to establish routing context on clustering. + // However if the client resends the message after receiving, it needs to be removed + private static final Predicate INTERNAL_PROPERTY_NAMES_CLEANUP_FILTER = --- End diff -- @franz1981 not in top of my mind. I would first check if properties = null, or empty... ? or add a parsed boolean? but i would need to debug to remember the best way. ---
[GitHub] activemq-artemis pull request #2432: [ARTEMIS-2176] RA connection properties...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2432#discussion_r233860659 --- Diff: artemis-service-extensions/src/main/java/org/apache/activemq/artemis/service/extensions/xa/recovery/XARecoveryConfig.java --- @@ -45,16 +45,41 @@ private final Map properties; private final ClientProtocolManagerFactory clientProtocolManager; + // ServerLocator properties --- End diff -- @spyrkob ?? ^^^ ---
[GitHub] activemq-artemis pull request #2432: [ARTEMIS-2176] RA connection properties...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2432#discussion_r233540277 --- Diff: artemis-service-extensions/src/main/java/org/apache/activemq/artemis/service/extensions/xa/recovery/XARecoveryConfig.java --- @@ -45,16 +45,41 @@ private final Map properties; private final ClientProtocolManagerFactory clientProtocolManager; + // ServerLocator properties --- End diff -- why you had to bring those here? ---
[GitHub] activemq-artemis pull request #2416: [ARTEMIS-2163]: Classloading issue if a...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2416#discussion_r233539372 --- Diff: artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientSessionFactoryImpl.java --- @@ -982,7 +982,13 @@ protected ConnectorFactory instantiateConnectorFactory(final String connectorFac return AccessController.doPrivileged(new PrivilegedAction() { @Override public ConnectorFactory run() { -return (ConnectorFactory) ClassloadingUtil.newInstanceFromClassLoader(connectorFactoryClassName); +ClassLoader cl = Thread.currentThread().getContextClassLoader(); --- End diff -- @ehsavoie can you amend it then? ---
[GitHub] activemq-artemis pull request #2416: [ARTEMIS-2163]: Classloading issue if a...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2416#discussion_r233483727 --- Diff: artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientSessionFactoryImpl.java --- @@ -982,7 +982,13 @@ protected ConnectorFactory instantiateConnectorFactory(final String connectorFac return AccessController.doPrivileged(new PrivilegedAction() { @Override public ConnectorFactory run() { -return (ConnectorFactory) ClassloadingUtil.newInstanceFromClassLoader(connectorFactoryClassName); +ClassLoader cl = Thread.currentThread().getContextClassLoader(); --- End diff -- Couldn't you change ClassLoadingUtil as ```java public static Object newInstanceFromClassLoader(final String className) { return newInstanceFromClassLoader(ClassloadingUtil.class, className); } public static Object newInstanceFromClassLoader(Class classOwner, final String className) { ClassLoader loader = classOwner.getClassLoader(); try { Class clazz = loader.loadClass(className); return clazz.newInstance(); } catch (Throwable t) { if (t instanceof InstantiationException) { System.out.println(INSTANTIATION_EXCEPTION_MESSAGE); } loader = Thread.currentThread().getContextClassLoader(); if (loader == null) throw new RuntimeException("No local context classloader", t); try { return loader.loadClass(className).newInstance(); } catch (InstantiationException e) { throw new RuntimeException(INSTANTIATION_EXCEPTION_MESSAGE + " " + className, e); } catch (ClassNotFoundException e) { throw new IllegalStateException(e); } catch (IllegalAccessException e) { throw new RuntimeException(e); } } } ``` and pass in the class parameter on these cases? Or would this have issues with Security on the JDK? If there are no issues I would prefer the parameter added? ---
[GitHub] activemq-artemis issue #2425: ARTEMIS-2142 Refactor of Patchfix ServerJMSMes...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2425 I am a bit confused on how to merge this into 2.6.x. If you want a 2.6.x patch can you pull it yourself? ---
[GitHub] activemq-artemis issue #2423: ARTEMIS-2168 Fix "populate-validated-user" fea...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2423 Any modification has to be on extra properties. ---
[GitHub] activemq-artemis issue #2423: ARTEMIS-2168 Fix "populate-validated-user" fea...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2423 Amqp headers cannot be changed per spec. I am kind of confused on whatâs going on Iâm a bit lost on a week or vacation. Can I have a TL;dr ---
[GitHub] activemq-artemis pull request #:
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/commit/270b383e80296fb47dba6a719ef1616ddcaab1ef#commitcomment-31162500 I canât do any work this weekend. But looking through the iPhone it looks really good. If tests pass letâs merge it. Lgtm ---
[GitHub] activemq-artemis pull request #2413: NO-JIRA fixing tests
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2413 NO-JIRA fixing tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis fixTest Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2413.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2413 commit 50cf73f03eeb5457929735489b1ede8fac38dbc0 Author: Clebert Suconic Date: 2018-11-02T19:41:52Z NO-JIRA fixing tests ---
[GitHub] activemq-artemis issue #2412: DO NOT MERGE
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2412 What is this? :) ---
[GitHub] activemq-artemis issue #2401: ARTEMIS-1710 Allow management msgs to exceed g...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2401 @franz1981 / @michaelandrepearce : Management messages exist to manage the broker. they are passed right into QueueController and are not queued. Hence they should not be managed by blocking / paging or anything. They should just be consumed right away, and not parsed by PagingManager.. never. I actually thought this was the case.. This could block you out. Say you need to drop a queue because the broker memory is off limits. if your only option is to use management messages, you're locked out of our own broker. Hence... management messages shouldn't be blocked or paged. they should just be consumed right away. i consider this a bug. .and it should be fixed without adding any configuration. if (isManagement) maxSize = -1;... don't use any blocking / paging / fail! ---
[GitHub] activemq-artemis pull request #2409: ARTEMIS-2159 Fixing OpenWire Blocker Pr...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2409 ARTEMIS-2159 Fixing OpenWire Blocker Producer Previous change on Flow control in OpenWire broke Blocked cases This is a better fix. You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2159 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2409.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2409 commit a6359d2ada1f880bf9ed537bab812d9a06fb0bfa Author: Clebert Suconic Date: 2018-11-01T19:33:03Z ARTEMIS-2159 Fixing OpenWire Blocker Producer Previous change on Flow control in OpenWire broke Blocked cases This is a better fix. ---
[GitHub] activemq-artemis issue #2401: ARTEMIS-1710 Allow management msgs to pass the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2401 @lulf / @franz1981 ok, Why not make a simpler change.. and return -1 on PaginManager if the management address. That's a lot less code to get around the sizing. @franz1981 Also, please rename the commit and JIRA to *exceed* size. ---
[GitHub] activemq-artemis pull request #2401: ARTEMIS-1710 Allow management msgs to p...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2401#discussion_r229835269 --- Diff: artemis-server/src/main/java/org/apache/activemq/artemis/core/config/Configuration.java --- @@ -493,6 +493,17 @@ Configuration addDiscoveryGroupConfiguration(String key, */ Configuration setManagementAddress(SimpleString address); + /** +* Sets whether {@link #getManagementAddress()} ignores Global Max Size limit. +*/ + Configuration setManagementAddressIgnoreGlobalMaxSize(boolean value); + + /** +* Returns {@code true} if {@link #getManagementAddress()} ignores Global Max Size limit. +*/ + boolean isManagementAddressIgnoreGlobalMaxSize(); --- End diff -- Why not simply configure the management address as -1, which would be unlimited? ---
[GitHub] activemq-artemis issue #2401: ARTEMIS-1710 Allow management msgs to pass the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2401 @franz1981 Did you mean exceed global-max-size? when I read this originally I had the impression you were setting global-max-size using management messages. I'm still trying to get my head around basic use cases.. such as.. why not simply set the management address size as -1? I'm confused on basic stuff here. ---
[GitHub] activemq-artemis issue #2406: New profile to allow skipping or including the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2406 how to run a single test: ``` mvn -Ptests -DfailIfNoTests=false -Pextra-tests -DskipStyleCheck=true -DskipPerformanceTests=false -Dtest=$1 test ``` ^^^ the following is provided as ./scripts/one-test.sh from main. and TBH: most people just use the IDE to start a single test. or if you just want to run the integration-tests, you can call mvn -Ptests within integration-tests I really don't see a reason for this profile. ---
[GitHub] activemq-artemis issue #2406: New profile to allow skipping or including the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2406 There is a script. One-test.sh. Or you can run the test in an ide. ---
[GitHub] activemq-artemis issue #2406: New profile to allow skipping or including the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2406 This is not needed at all. can you close it? ---
[GitHub] activemq-artemis pull request #2407: openWire would allow one extra send
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2407 openWire would allow one extra send You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2159 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2407.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2407 commit d1fd86ece23bc499c54b295316bcd94494a764ad Author: Otavio Rodolfo Piske Date: 2018-10-31T11:10:52Z ARTEMIS-2159 Reproducer test that demonstrates that it is possible to force the broker to accept messages even after the queue is full and an exception is raised commit 2ab0f7b0eeb3e7e05e784646839580354dbf62cb Author: Clebert Suconic Date: 2018-10-31T13:13:05Z ARTEMIS-2159 OpenWire would allow one extra send ---
[GitHub] activemq-artemis issue #2400: ARTEMIS-2151 JMS Selectors broken in some case...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2400 The some cases you mention is just AMQGroupID? If that's the case can you rename the JIRA and commit here? something like AMQGroupID is ignored on filter? Or am I misunderstanding what you changed here? ---
[GitHub] activemq-artemis issue #2401: ARTEMIS-1710 Allow management msgs to pass the...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2401 @franz1981 did you run a full testsuite on this? ---
[GitHub] activemq-artemis pull request #2404: ARTEMIS-2157 Extra information on Criti...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2404 ARTEMIS-2157 Extra information on CriticalAnalyzer and its components You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2157 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2404.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2404 commit 692c30a701374f28dceeb491ffcf5e16077a04dd Author: Clebert Suconic Date: 2018-10-30T20:10:35Z ARTEMIS-2157 Extra information on CriticalAnalyzer and its components ---
[GitHub] activemq-artemis issue #2396: ARTEMIS-2149 Protecting message.sendBuffer fro...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2396 @michaelandrepearce I added a test to encode / decode in multi-thread. it should be enough. ---
[GitHub] activemq-artemis issue #2396: ARTEMIS-2149 Protecting message.sendBuffer fro...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2396 about previously from 2.0.0 I can only think that we have made improvements that probably allowed more load. I ran the same test against 2.0.0 and it fails as well. Prior to 2.0.0 we would always copy the buffer during send and duplicate it. I tried to create a single buffer upon receiving and call saveToBuffer using the read-only-buffer. During the process I missed this race. Even though I tested it quite a lot. Regarding validBuffer, The application is single threaded upon receiving a message. We only need to validate it later during encode, which I added the synchronize. So i think it's ok? ---
[GitHub] activemq-artemis issue #2396: ARTEMIS-2149 Protecting message.sendBuffer fro...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2396 @michaelandrepearce I will merge this now.. but if you could still review it after I merged it? we can send further commits. ---
[GitHub] activemq-artemis issue #2396: ARTEMIS-2149 Protecting message.sendBuffer fro...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2396 This is now ready to be merged.. (I had a word WIP on the name here before); ---
[GitHub] activemq-artemis issue #2396: ARTEMIS-2149 Protecting message.sendBuffer fro...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2396 @michaelandrepearce can you review this one please. ---
[GitHub] activemq-artemis pull request #2396: ARTEMIS-2149 Protecting message.sendBuf...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2396 ARTEMIS-2149 Protecting message.sendBuffer from races encoding it You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2149 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2396.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2396 commit 2823cdf7e222faf4dacdc9274febb1fba3ede45c Author: Clebert Suconic Date: 2018-10-25T21:27:47Z ARTEMIS-2149 Protecting message.sendBuffer from races encoding it ---
[GitHub] activemq-artemis issue #2369: ARTEMIS-2123 Paging not stopped if there are n...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2369 @wy96f are you waiting a release to apply your patch? ---
[GitHub] activemq-artemis pull request #2395: NO-JIRA Adding log.warn statements in c...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2395 NO-JIRA Adding log.warn statements in case properties decode fails You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis NO-JIRA Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2395.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2395 commit a408bfbf9b7d347ccaa2373e02a8ce466e4f0c66 Author: Clebert Suconic Date: 2018-10-25T15:23:40Z NO-JIRA Adding log.warn statements in case properties decode fails ---
[GitHub] activemq-artemis pull request #2394: ARTEMIS-2148 Fixing typo where getDoubl...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2394 ARTEMIS-2148 Fixing typo where getDoubleProperty marks body as changed You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2148 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2394.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2394 commit 04e278cd155de0e88ab1bea659da1a2b71e96c39 Author: Clebert Suconic Date: 2018-10-25T14:41:02Z ARTEMIS-2148 Fixing typo where getDoubleProperty marks body as changed ---
[GitHub] activemq-artemis pull request #2393: ARTEMIS-2146 Avoiding NPE on AMQP Flow ...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2393 ARTEMIS-2146 Avoiding NPE on AMQP Flow Control AMQP Flow control will disable consumer flow control (setting credits to null) This will avoid a race checking flow control. You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2146 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2393.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2393 commit 1c91539bad06f96b3ebee84b900b8dcdb1d66d94 Author: Clebert Suconic Date: 2018-10-23T16:43:36Z ARTEMIS-2146 Avoiding NPE on AMQP Flow Control AMQP Flow control will disable consumer flow control (setting credits to null) This will avoid a race checking flow control. ---
[GitHub] activemq-artemis issue #2391: [ENTMQBR-2035][JBEAP-15599] add page decoding ...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2391 You shouldnât reference an external JiRA on a commit. ---
[GitHub] activemq-artemis issue #2383: ARTEMIS-2096 Adding a test
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2383 @gaohoward this is actually ARTEMIS-2135, which I fixed in 2.6.x and added a test. Also verified that @tabish121 ' s refactoring fixed it.. but the actual issue for this test is ARTEMIS-2135. Which I had already fixed in 48e0fc8f42346d96bc809593a150e05a586787ee at 2.6.x. ---
[GitHub] activemq-artemis pull request #:
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/commit/292566e3906c510cfa055e5adaafec922af54ef7#commitcomment-30997805 In artemis-jms-client/src/main/java/org/apache/activemq/artemis/jms/client/ActiveMQMapMessage.java: In artemis-jms-client/src/main/java/org/apache/activemq/artemis/jms/client/ActiveMQMapMessage.java on line 39: This is just reverting the patch that would require properties on JMS Messages using Core. I'm planning to revisit this into 2.7.0 with more extensive testing. I'm a bit suck now with how Wildfly is working ATM. ---
[GitHub] activemq-artemis pull request #:
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/commit/292566e3906c510cfa055e5adaafec922af54ef7#commitcomment-30997542 In artemis-jms-client/src/main/java/org/apache/activemq/artemis/jms/client/ActiveMQMapMessage.java: In artemis-jms-client/src/main/java/org/apache/activemq/artemis/jms/client/ActiveMQMapMessage.java on line 39: It has.. however Wildfly is still having compatibility issues. So, I decided to revert for now on 2.6.x stream, and revisit this into master. ---
[GitHub] activemq-artemis pull request #2250: ARTEMIS-1996 MappedSequentialFileFactor...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2250#discussion_r227024184 --- Diff: tests/unit-tests/src/test/java/org/apache/activemq/artemis/tests/unit/core/journal/impl/fakes/FakeSequentialFileFactory.java --- @@ -417,6 +418,18 @@ public synchronized void writeDirect(final ByteBuffer bytes, final boolean sync, } + @Override + public synchronized void blockingWriteDirect(ByteBuffer bytes, --- End diff -- do you really need synchronized here? this is synchronizing the factory. ---
[GitHub] activemq-artemis issue #2383: ARTEMIS-2096 Adding a test
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2383 @gaohoward - I wouldn't mention external JIRAs here... - Please find a more inclusive message.. when I do git log, a message such as "Adding a Test" doesn't help much. ---
[GitHub] activemq-artemis issue #2380: ARTEMIS-2136 Adding synchronization on Copy Co...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2380 @jbertram can you take over? I will close my PR. ---
[GitHub] activemq-artemis pull request #2380: ARTEMIS-2136 Adding synchronization on ...
Github user clebertsuconic closed the pull request at: https://github.com/apache/activemq-artemis/pull/2380 ---
[GitHub] activemq-artemis issue #2380: ARTEMIS-2136 Adding synchronization on Copy Co...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2380 The test for this was a complex application. I still don't have a testcase for this.. I may come up with something next week. ---
[GitHub] activemq-artemis pull request #2380: ARTEMIS-2136 Adding synchronization on ...
GitHub user clebertsuconic opened a pull request: https://github.com/apache/activemq-artemis/pull/2380 ARTEMIS-2136 Adding synchronization on Copy Constructor I missed this synchronization block during the AMQP Refactoring This could have issues on Diverts Or Clustering bridges. You can merge this pull request into a Git repository by running: $ git pull https://github.com/clebertsuconic/activemq-artemis ARTEMIS-2136 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/2380.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2380 commit 45e152644069ffcf10034aca997fabeda005ff04 Author: Clebert Suconic Date: 2018-10-18T21:18:45Z ARTEMIS-2136 Adding synchronization on Copy Constructor I missed this synchronization block during the AMQP Refactoring This could have issues on Diverts Or Clustering bridges. ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 I will merge this now.. if there's any issues we can review later ---
[GitHub] activemq-artemis issue #2375: NO-JIRA small fix to avoid re-distributor bein...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2375 I think this needs a JIRA.. perhaps even a test? ---
[GitHub] activemq-artemis pull request #:
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/commit/48d8a54135732b2b34251f571aa3a5cadc44d3a9#commitcomment-30947882 In artemis-protocols/artemis-amqp-protocol/src/test/java/org/apache/activemq/artemis/protocol/amqp/broker/AMQPMessageTest.java: In artemis-protocols/artemis-amqp-protocol/src/test/java/org/apache/activemq/artemis/protocol/amqp/broker/AMQPMessageTest.java on line 178: Thatâs correct it does not apply to master. Tim Bish âs refactoring on amqp message fixed it. No need for a fix. ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 @michaelandrepearce I tested directly at your branch.. I just rebased on a new branch and ran it and it passed fine. There was a few fixes that you were behind I assume. I will merge it. ---
[GitHub] activemq-artemis issue #2250: ARTEMIS-1996 MappedSequentialFileFactory may c...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2250 Can you rebase this? lots of commits since you sent this. I will merge it along with the other PR on the journal. ---
[GitHub] activemq-artemis pull request #2377: ARTEMIS-2133 Artemis tab not showing on...
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/2377#discussion_r225992639 --- Diff: artemis-hawtio/artemis-plugin/src/main/webapp/plugin/js/artemisPlugin.js --- @@ -362,7 +362,7 @@ var ARTEMIS = (function(ARTEMIS) { workspace.subLevelTabs = subLevelTabs; - preLogoutTasks.addTask("clearArtemisCredentials", () => { --- End diff -- This will be enough to fix IE? ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 I had to take kids to a piano lesson tonight. I donât have a computer now. I will have to connect later tonight. I will do some debug before I post as well. (Will run it again ) ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 the testsuite is stalling... I will have to figure out where and put some context here. ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 I'm running a whole testsuite before we can merge this.. will have results in 3 hours. ---
[GitHub] activemq-artemis issue #2362: ARTEMIS-2117 Add custom LVQ Key and Non Destru...
Github user clebertsuconic commented on the issue: https://github.com/apache/activemq-artemis/pull/2362 My colleagues educated me on the non-descructive. Only question I have now is the deprecated method. ---