[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938273#comment-15938273 ] Hudson commented on BOOKKEEPER-998: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1695 (See [https://builds.apache.org/job/bookkeeper-master/1695/]) BOOKKEEPER-998: Increased the max entry size to 5MB (sijie: rev 28c264c8655f74e09ceee5ad499942cfb0dc65dc) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieNettyServer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 4.5.0 > > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937387#comment-15937387 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user asfgit closed the pull request at: https://github.com/apache/bookkeeper/pull/104 > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 4.5.0 > > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880609#comment-15880609 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user jvrao commented on the issue: https://github.com/apache/bookkeeper/pull/104 +1 LGTM. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868400#comment-15868400 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user eolivelli commented on the issue: https://github.com/apache/bookkeeper/pull/104 For me is +1 but I am not a committer. Usually @sijie merges PR or maybe @merlimat > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868210#comment-15868210 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user revans2 commented on the issue: https://github.com/apache/bookkeeper/pull/104 @eolivelli and @jvrao is there anything more I need to do to get this merged in? > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859642#comment-15859642 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user revans2 commented on the issue: https://github.com/apache/bookkeeper/pull/104 Thanks everyone I moved the configs to AbstractConfiguration and squashed the commits. Hopefully this is good to go now. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859060#comment-15859060 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user jvrao commented on the issue: https://github.com/apache/bookkeeper/pull/104 1MB is good enough for us at least for now. Did not test anything larger. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857472#comment-15857472 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user sijie commented on the issue: https://github.com/apache/bookkeeper/pull/104 I think 1MB limitation was coming before we used netty for the network transportation. Increasing it to 5MB sounds okay to me. I don't think it breaks any backward compatibility issue. I don't have any strong preference here on using a system property or configuration setting for it. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856815#comment-15856815 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user revans2 commented on the issue: https://github.com/apache/bookkeeper/pull/104 @eolivelli I added in configs for both the server and the client. Please take a look again. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856288#comment-15856288 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user revans2 commented on the issue: https://github.com/apache/bookkeeper/pull/104 @eolivelli Thanks I'll look at making them separate. If someone from salesforce wants to take a look that would be great. If they are storing large BLOBs on BK there is a high likelihood that they have similar code too. > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856249#comment-15856249 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- Github user eolivelli commented on the issue: https://github.com/apache/bookkeeper/pull/104 @revans2 IMHO it would be better to make it configurable as system property or in ServerConfiguration it would be even better, so it would be more "testable". I'm not aware of some system property naming convention, maybe you can suggest one and then the discuss is open. IMHO the max entry size can be related to the max Netty frame size but maybe we can use two different limits. For instance on the wire we need headers @sijie what to you think ? I think that at Salesforce they make large use of big entries (as they store BLOBs on BK), maybe JV as some number to share For me, I'm starting a new project which will need to store "huge" (more than 10 MB) entries and so I will need to set this limit to a greater value that 5MB > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854904#comment-15854904 ] ASF GitHub Bot commented on BOOKKEEPER-998: --- GitHub user revans2 opened a pull request: https://github.com/apache/bookkeeper/pull/104 BOOKKEEPER-998: Increased the max entry size to 5MB Full disclosure, most of these changes were actually done by @merlimat For this I am mostly just putting them back into open source. You can merge this pull request into a Git repository by running: $ git pull https://github.com/revans2/bookkeeper BOOKKEEPER-998 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bookkeeper/pull/104.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #104 commit 614d9cd5bfeeedf35f360adb757d1668429bb8ea Author: Robert (Bobby) Evans Date: 2017-02-06T22:33:49Z BOOKKEEPER-998: Increased the max entry size to 5MB > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854893#comment-15854893 ] Robert Joseph Evans commented on BOOKKEEPER-998: [~mmerli], Thanks > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB
[ https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854870#comment-15854870 ] Robert Joseph Evans commented on BOOKKEEPER-998: I would love to start working on this. Is there someone who can assign this to me? > Increased the max entry size to 5MB > --- > > Key: BOOKKEEPER-998 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Robert Joseph Evans > > In practice we have found the Bookkeeper can handle 5MB entries without > issue. We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the > {{BookieNettyServer.maxMessageSize}} and the > {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and > ideally 5MB. -- This message was sent by Atlassian JIRA (v6.3.15#6346)