[jira] [Comment Edited] (AMQ-6441) Incorrect File System Size Reported with Amazon Elastic File System (EFS)
[
https://issues.apache.org/jira/browse/AMQ-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529181#comment-15529181
]
Christopher L. Shannon edited comment on AMQ-6441 at 9/28/16 10:17 AM:
---
Right that's basically what I had in mind with mocking the File API, but you'll
want to wire that in for a test with the BrokerService to test that a real
Broker instance still starts up when the value is negative (if the new flag is
set to disable the disk usage check, etc). If you take a look at the unit test
project you'll see a ton of examples of tests and how to configure a
BrokerService to start a real broker in JUnit.
I'm not sure how familiar you are with the code base so if you can't figure out
how to set up a test or don't have time then I should be able to do it when I
get a chance.
was (Author: christopher.l.shannon):
Right that's basically what I had in mind with mocking the File API, but you'll
want to wire that in for a test with the BrokerService to test that the a real
Broker instance still starts up when the value is negative (if the new flag is
set to disable the disk usage check, etc). If you take a look at the unit test
project you'll see a ton of examples of tests and how to configure a
BrokerService to start a real broker in JUnit.
I'm not sure how familiar you are with the code base so if you can't figure out
how to set up a test or don't have time then I should be able to do it when I
get a chance.
> Incorrect File System Size Reported with Amazon Elastic File System (EFS)
> -
>
> Key: AMQ-6441
> URL: https://issues.apache.org/jira/browse/AMQ-6441
> Project: ActiveMQ
> Issue Type: Bug
>Affects Versions: 5.11.1
>Reporter: Ismail Bhana
>
> I've set up Active MQ in production with a shared file system master/slave
> configuration (KahaDB). I've set everything up and mounted the EFS on both
> EC2 instances.
> When I check the disk free stats I get 8 exabytes for the shared file system:
> {code}
> $ df -h
> eu-west-1a.***.efs.eu-west-1.amazonaws.com:/ 8.0E 0 8.0E 0% /mnt/efs
> {code}
> Unfortunately, ActiveMQ cannot interpret this number (8 exabytes). This may
> be due to integer truncation.
> Here is a snippet of the log:
> {code}
> Store limit is 102400 mb (current store usage is 0 mb). The data directory:
> /mnt/efs/kahadb only has -8796093022208 mb of usable space - resetting to
> maximum available disk space: -8796093022207 mb
> Store limit is -8796093022207 mb, whilst the max journal file size for the
> store is: 32 mb, the store will not accept any data when used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (AMQ-6441) Incorrect File System Size Reported with Amazon Elastic File System (EFS)
[
https://issues.apache.org/jira/browse/AMQ-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527178#comment-15527178
]
Christopher L. Shannon edited comment on AMQ-6441 at 9/27/16 7:27 PM:
--
This should be tested in a unit test regardless so my recommendation is to not
worry about that and see if you could use something like mockito to mock/spy on
the File api interface and then you can configure it to return whatever value
you want (ie 8 EB) to test. Off the top of my head maybe something like
mocking out what the getPersistenceAdapter() method returns in BrokerService
could work so that when the usage checks get the directory to check the size it
will return your Directory/File mock etc.
was (Author: christopher.l.shannon):
This should be tested in a unit test regardless so my recommendation is to not
worry about that and see if you could use something like mockito to mock/spy on
the File api interface and then you can configure it to return whatever value
you want (ie 8 EB) to test. Off the top of my head maybe something like
mocking out out the getPersistenceAdapter() method in BrokerService so that
when it returns a directory for the usage checks it will return your
Directory/File mock etc.
> Incorrect File System Size Reported with Amazon Elastic File System (EFS)
> -
>
> Key: AMQ-6441
> URL: https://issues.apache.org/jira/browse/AMQ-6441
> Project: ActiveMQ
> Issue Type: Bug
>Affects Versions: 5.11.1
>Reporter: Ismail Bhana
>
> I've set up Active MQ in production with a shared file system master/slave
> configuration (KahaDB). I've set everything up and mounted the EFS on both
> EC2 instances.
> When I check the disk free stats I get 8 exabytes for the shared file system:
> {code}
> $ df -h
> eu-west-1a.***.efs.eu-west-1.amazonaws.com:/ 8.0E 0 8.0E 0% /mnt/efs
> {code}
> Unfortunately, ActiveMQ cannot interpret this number (8 exabytes). This may
> be due to integer truncation.
> Here is a snippet of the log:
> {code}
> Store limit is 102400 mb (current store usage is 0 mb). The data directory:
> /mnt/efs/kahadb only has -8796093022208 mb of usable space - resetting to
> maximum available disk space: -8796093022207 mb
> Store limit is -8796093022207 mb, whilst the max journal file size for the
> store is: 32 mb, the store will not accept any data when used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (AMQ-6441) Incorrect File System Size Reported with Amazon Elastic File System (EFS)
[
https://issues.apache.org/jira/browse/AMQ-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517036#comment-15517036
]
William Crowell edited comment on AMQ-6441 at 9/23/16 5:25 PM:
---
I have a theory that a long is not large enough to represent 8,796,093,022,208
megabytes, and we are running into an overflow issue.
Given that a long primitive type in Java 7 has a range of -2 to the 63rd power
and a maximum value of 2 to the 63rd power -1.
This means that a long can have values -9,223,372,036,854,775,808 to
9,223,372,036,854,775,807.
If I convert 8796093022208 megabytes into bytes then that value is
9,223,372,036,854,780,000 which is 4,193 outside the maximum value of a long.
If you happen to be running Java 8, then a unsigned 64-bit long can have a
maximum value of 18,446,744,073,709,500,000 which would be large enough.
This means that the call to Java's java.io.File.getTotalSpace() would overflow
since it returns a long:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html#getTotalSpace()
In Java 8, this probably works fine. Could I propose changing the variable on
line 2050 of org.apache.activemq.broker.BrokerService to a double instead of a
long?
...
double totalSpace = dir.getTotalSpace();
...
>From the link:
>https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
"The double data type is a double-precision 64-bit IEEE 754 floating point."
was (Author: william.crowell):
I have a theory that a long is not large enough to represent 8,796,093,022,208
megabytes, and we are running into an overflow issue.
Given that a long primitive type in Java 7 has a range of -2 to the 63rd power
and a maximum value of 2 to the 63rd power -1.
This means that a long can have values -9,223,372,036,854,775,808 to
9,223,372,036,854,775,807.
If I convert 8796093022208 megabytes into bytes then that value is
9,223,372,036,854,780,000 which is 4,193 outside the maximum value of a long.
If you happen to be running Java 8, then a unsigned 64-bit long can have a
maximum value of 18,446,744,073,709,500,000.
This means that the call to Java's java.io.File.getTotalSpace() would overflow
since it returns a long:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html#getTotalSpace()
In Java 8, this probably works fine. Could I propose changing the variable on
line 2050 of org.apache.activemq.broker.BrokerService to a double instead of a
long?
...
double totalSpace = dir.getTotalSpace();
...
>From the link:
>https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
"The double data type is a double-precision 64-bit IEEE 754 floating point."
> Incorrect File System Size Reported with Amazon Elastic File System (EFS)
> -
>
> Key: AMQ-6441
> URL: https://issues.apache.org/jira/browse/AMQ-6441
> Project: ActiveMQ
> Issue Type: Bug
>Affects Versions: 5.11.1
>Reporter: Ismail Bhana
>
> I've set up Active MQ in production with a shared file system master/slave
> configuration (KahaDB). I've set everything up and mounted the EFS on both
> EC2 instances.
> When I check the disk free stats I get 8 exabytes for the shared file system:
> {code}
> $ df -h
> eu-west-1a.***.efs.eu-west-1.amazonaws.com:/ 8.0E 0 8.0E 0% /mnt/efs
> {code}
> Unfortunately, ActiveMQ cannot interpret this number (8 exabytes). This may
> be due to integer truncation.
> Here is a snippet of the log:
> {code}
> Store limit is 102400 mb (current store usage is 0 mb). The data directory:
> /mnt/efs/kahadb only has -8796093022208 mb of usable space - resetting to
> maximum available disk space: -8796093022207 mb
> Store limit is -8796093022207 mb, whilst the max journal file size for the
> store is: 32 mb, the store will not accept any data when used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
