Hi Devs,
Base question:
How to run Cassandra 4.0 (trunk) version on a Windows based environment
without getting always the errors like this?
org.apache.cassandra.io.FSReadError: java.io.IOException: Invalid folder
descriptor trying to create log replica
D:\workspace\java-projects\java-automated-test-suite\target\embeddedCassandra\data\test_keyspace\test_table-f6498e1075ca11e9a59ed3a612c57aaf
Background / Motivation:
In our project we want to use Map/List data types heavily - also
combined with IN operator. Due to
https://issues.apache.org/jira/browse/CASSANDRA-12654 this does not work
in 3.x but solved in 4.0. Also extended features around the above
complex data types in 4.0 excites me up. So I decided to do a test
deployment on our test servers (Linux based) - features worked like
charm there.
As a next step I extended our jBehave testing framework to also be able
to fire up an embedded Cassandra 4.0 for running tests. Since I am
developing on Windows I started to get the above mentioned FSReadError
exceptions however folder path is valid.
Same happens if I try to fire up Cassandra 4.0 instances on Windows systems.
More advanced question:
Based on stack traces I took a look into the source. In
org.apache.cassandra.db.lifecycle.LogReplica and e.g. in open(File file)
method (along with others) it is using the
NativeLibrary.tryOpenDirectory(file.getParentFile().getPath()) call - if
it fails I get the above FSReadError for sure.
And here we arrive to the main point... On Windows the
org.apache.cassandra.utils.NativeLibraryWindows class is used in the
background. But in this class basically almost all method implementation
is just "throw new UnsatisfiedLinkError();" - bingo! This shows me that
I have no really hope... If I am on Windows I will fail like hell with
above errors always.
1. Are you planning to do something around it?
What I did as an ass-saver I have added a "mock" implementation for 3
problematic methods by implementing in Java in NativeLibraryWindows
class fake_callOpen(), fake_callFsync(), fake_callClose() kinda
simulating something. fake_callOpen() stores the fake fdId => path
mapping in an internal Map object. fake_callClose() flushes that out.
fake_callFsync() gives a WARN if the fdId is uinknown (so
fake_callOpen() was not invoked earlier). Issue has gone for me after
this, Cassandra 4.0 runs fine with all of our test cases - I am happy!
:-) So.... and but...
2. My quick and dirty implementation shows that I regularly get lots of
warnings from my fake_callFsync(int fd): "file descriptor {} is unknown
so invalid!" which means callOpen(int fd) was NOT invoked in prior
calling callFsync() for the given descriptor. Is that normal??? Should
not callOpen() always be invoked??? Could it be a bug somehow???
3. I am really not planning ever run Cassandra (or any other) in Windows
based PROD system. But developing on Windows still. Wouldn't it make
sense to at least provide some workaround here for fellow developers
like me? At least to be able to run the server in Windows (for
testing/development purposes only) ...
thanks!
--
Attila Wind
http://www.linkedin.com/in/attilaw
Mobile: +36 31 7811355
