Have you modified in Storm.yaml settings related to ACL? The defaults
one disabled ACL for blob
On 03/04/2017 18:27, I PVP wrote:
Thanks for everyone that is answering and helping to look into this issue.
I tried from /tmp. Same error.
I tried submitting the jar from the ec2-user $HOME directory. Same error
I created a new directory to which storm:storm is the owner and tried
submitting the jar from there. Same error.
The error log is not really clear on what is going on.
Where/what is setup-path that is mentioned on the log? is it the
/opt/storm where storm was “installed” ?
What is the exact "No such file or directory” that is missing or
inaccessible ?
First it says that :
—
2017-04-03 13:13:00.148 pool-15-thread-30 o.a.s.d.nimbus [INFO]
Finished uploading file from client:
/opt/storm/storm-local/nimbus/inbox/stormjar-f90ff743-e9ec-4e59-b945-878a47fd9057.j
ar
2017-04-03 13:13:00.173 pool-15-thread-32 o.a.s.d.nimbus [INFO]
Received topology submission for <COMPANYNAME>Topology with conf
{"topology.max.task.parallelism" nil, "topology
.submitter.principal" "", "topology.acker.executors" nil,
"topology.eventlogger.executors" 0, "topology.workers" 1,
"topology.debug" false, "storm.zookeeper.superACL" nil, "topology.us
ers" (), "topology.submitter.user" "storm", "topology.kryo.register"
nil, "topology.kryo.decorators" (), "storm.id"
"<COMPANYNAME>Topology-3-1491235980", "topology.name" "Emplo
yeePasswordResetTopology"}
2017-04-03 13:13:00.189 pool-15-thread-32 o.a.s.d.nimbus [INFO]
uploadedJar
/opt/storm/storm-local/nimbus/inbox/stormjar-f90ff743-e9ec-4e59-b945-878a47fd9057.jar
2017-04-03 13:13:00.189 pool-15-thread-32
o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
..
----
Before trowing the exception it says :
——
2017-04-03 13:13:00.207 pool-15-thread-32 o.a.s.cluster [INFO]
setup-path/blobstore/<COMPANYNAME>Topology-3-1491235980-stormjar.jar/ip-172-31-5-153.sa-east-1.compute.internal:6
627-1
2017-04-03 13:13:00.207 pool-15-thread-32-EventThread
o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2017-04-03 13:13:00.217 pool-15-thread-32 o.a.s.d.nimbus [WARN]
Topology submission exception. (topology name='<COMPANYNAME>Topology')
#error {
:cause No such file or directory
:via
[{:type java.lang.RuntimeException
:message java.io.IOException: No such file or directory
:at [org.apache.storm.blobstore.LocalFsBlobStore createBlob
LocalFsBlobStore.java 117]}
...
-----
best,
IPVP
On April 3, 2017 at 12:12:02 PM, Santosh Pingale
([email protected] <mailto:[email protected]>) wrote:
Can you try to submit the jar from a directory which is writable by
storm user. Try from /tmp maybe.
On Mon, Apr 3, 2017 at 3:56 PM, I PVP <[email protected]
<mailto:[email protected]>> wrote:
that is the error:
2017-04-03 10:31:44.535 pool-15-thread-56 o.a.s.d.nimbus [INFO]
Uploading file from client to
/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.820 pool-15-thread-17 o.a.s.d.nimbus [INFO]
Finished uploading file from client:
/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.842 pool-15-thread-20 o.a.s.d.nimbus [INFO]
Received topology submission for <COMPANYNAME>Topology with conf
{"topology.max.task.parallelism" nil,
"topology.submitter.principal" "", "topology.acker.executors"
nil, "topology.eventlogger.executors" 0, "topology.workers" 1,
"topology.debug" false, "storm.zookeeper.superACL" nil,
"topology.users" (), "topology.submitter.user" "storm",
"topology.kryo.register" nil, "topology.kryo.decorators" (),
"storm.id <http://storm.id>"
"<COMPANYNAME>Topology-2-1491226304", "topology.name
<http://topology.name>" "<COMPANYNAME>Topology"}
2017-04-03 10:31:44.846 pool-15-thread-20 o.a.s.d.nimbus [INFO]
uploadedJar
/opt/storm/storm-local/nimbus/inbox/stormjar-fceb1c76-b88e-415d-803d-ab73c0d2fafe.jar
2017-04-03 10:31:44.847 pool-15-thread-20
o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2017-04-03 10:31:44.848 pool-15-thread-20 o.a.s.s.o.a.z.ZooKeeper
[INFO] Initiating client connection,
connectString=172.31.5.153:2181
<http://172.31.5.153:2181>,172.31.3.29:2181
<http://172.31.3.29:2181>,172.31.0.218:2181/storm
<http://172.31.0.218:2181/storm> sessionTimeout=20000
watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@13bf52ea
2017-04-03 10:31:44.851
pool-15-thread-20-SendThread(172.31.0.218:2181
<http://172.31.0.218:2181>) o.a.s.s.o.a.z.ClientCnxn [INFO]
Opening socket connection to server
172.31.0.218/172.31.0.218:2181
<http://172.31.0.218/172.31.0.218:2181>. Will not attempt to
authenticate using SASL (unknown error)
2017-04-03 10:31:44.852
pool-15-thread-20-SendThread(172.31.0.218:2181
<http://172.31.0.218:2181>) o.a.s.s.o.a.z.ClientCnxn [INFO]
Socket connection established to 172.31.0.218/172.31.0.218:2181
<http://172.31.0.218/172.31.0.218:2181>, initiating session
2017-04-03 10:31:44.855
pool-15-thread-20-SendThread(172.31.0.218:2181
<http://172.31.0.218:2181>) o.a.s.s.o.a.z.ClientCnxn [INFO]
Session establishment complete on server
172.31.0.218/172.31.0.218:2181
<http://172.31.0.218/172.31.0.218:2181>, sessionid =
0x35b33b21dad0005, negotiated timeout = 20000
2017-04-03 10:31:44.855 pool-15-thread-20-EventThread
o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change:
CONNECTED
2017-04-03 10:31:44.860 Curator-Framework-0
o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO]
backgroundOperationsLoop exiting
2017-04-03 10:31:44.862 pool-15-thread-20 o.a.s.s.o.a.z.ZooKeeper
[INFO] Session: 0x35b33b21dad0005 closed
2017-04-03 10:31:44.862 pool-15-thread-20 o.a.s.cluster [INFO]
setup-path/blobstore/<COMPANYNAME>Topology-2-1491226304-stormjar.jar/ip-172-31-5-153.sa-east-1.compute.internal:6627-1
2017-04-03 10:31:44.862 pool-15-thread-20-EventThread
o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2017-04-03 10:31:44.877 pool-15-thread-20 o.a.s.d.nimbus [WARN]
Topology submission exception. (topology
name='<COMPANYNAME>Topology') #error {
:cause No such file or directory
:via
[{:type java.lang.RuntimeException
:message java.io.IOException: No such file or directory
:at [org.apache.storm.blobstore.LocalFsBlobStore createBlob
LocalFsBlobStore.java 117]}
{:type java.io.IOException
:message No such file or directory
:at [java.io.UnixFileSystem createFileExclusively
UnixFileSystem.java -2]}]
:trace
[[java.io.UnixFileSystem createFileExclusively
UnixFileSystem.java -2]
[java.io.File createNewFile File.java 1012]
[org.apache.storm.blobstore.LocalFsBlobStoreFile getOutputStream
LocalFsBlobStoreFile.java 109]
[org.apache.storm.blobstore.BlobStore$BlobStoreFileOutputStream
<init> BlobStore.java 316]
[org.apache.storm.blobstore.LocalFsBlobStore createBlob
LocalFsBlobStore.java 111]
[org.apache.storm.blobstore.BlobStore createBlob BlobStore.java 217]
[sun.reflect.NativeMethodAccessorImpl invoke0
NativeMethodAccessorImpl.java -2]
[sun.reflect.NativeMethodAccessorImpl invoke
NativeMethodAccessorImpl.java 62]
[sun.reflect.DelegatingMethodAccessorImpl invoke
DelegatingMethodAccessorImpl.java 43]
[java.lang.reflect.Method invoke Method.java 498]
[clojure.lang.Reflector invokeMatchingMethod Reflector.java 93]
[clojure.lang.Reflector invokeInstanceMethod Reflector.java 28]
[org.apache.storm.daemon.nimbus$setup_storm_code invoke
nimbus.clj 474]
[org.apache.storm.daemon.nimbus$fn__9109$exec_fn__1371__auto__$reify__9138
submitTopologyWithOpts nimbus.clj 1525]
[org.apache.storm.daemon.nimbus$fn__9109$exec_fn__1371__auto__$reify__9138
submitTopology nimbus.clj 1541]
[org.apache.storm.generated.Nimbus$Processor$submitTopology
getResult Nimbus.java 2912]
[org.apache.storm.generated.Nimbus$Processor$submitTopology
getResult Nimbus.java 2896]
[org.apache.storm.thrift.ProcessFunction process
ProcessFunction.java 39]
[org.apache.storm.thrift.TBaseProcessor process
TBaseProcessor.java 39]
[org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor
process SimpleTransportPlugin.java 162]
[org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer
invoke AbstractNonblockingServer.java 518]
[org.apache.storm.thrift.server.Invocation run Invocation.java 18]
[java.util.concurrent.ThreadPoolExecutor runWorker
ThreadPoolExecutor.java 1142]
[java.util.concurrent.ThreadPoolExecutor$Worker run
ThreadPoolExecutor.java 617]
[java.lang.Thread run Thread.java 745]]}
2017-04-03 10:31:44.877 pool-15-thread-20
o.a.s.t.s.AbstractNonblockingServer$FrameBuffer [ERROR]
Unexpected throwable while invoking!
java.lang.RuntimeException: java.io.IOException: No such file or
directory
at
org.apache.storm.blobstore.LocalFsBlobStore.createBlob(LocalFsBlobStore.java:117)
at
org.apache.storm.blobstore.BlobStore.createBlob(BlobStore.java:217)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
at
org.apache.storm.daemon.nimbus$setup_storm_code.invoke(nimbus.clj:474)
at
org.apache.storm.daemon.nimbus$fn__9109$exec_fn__1371__auto__$reify__9138.submitTopologyWithOpts(nimbus.clj:1525)
at
org.apache.storm.daemon.nimbus$fn__9109$exec_fn__1371__auto__$reify__9138.submitTopology(nimbus.clj:1541)
at
org.apache.storm.generated.Nimbus$Processor$submitTopology.getResult(Nimbus.java:2912)
at
org.apache.storm.generated.Nimbus$Processor$submitTopology.getResult(Nimbus.java:2896)
at
org.apache.storm.thrift.ProcessFunction.process(ProcessFunction.java:39)
at
org.apache.storm.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java:162)
at
org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)
at org.apache.storm.thrift.server.Invocation.run(Invocation.java:18)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1012)
at
org.apache.storm.blobstore.LocalFsBlobStoreFile.getOutputStream(LocalFsBlobStoreFile.java:109)
at
org.apache.storm.blobstore.BlobStore$BlobStoreFileOutputStream.<init>(BlobStore.java:316)
at
org.apache.storm.blobstore.LocalFsBlobStore.createBlob(LocalFsBlobStore.java:111)
... 20 more
—
Storm 1.0.3 on AWS Red Hat Enterprise Linux 7.3.
Storm path is /opt/storm
user: storm:storm
Thanks
IPVP
On March 31, 2017 at 3:10:05 AM, Harsh Choudhary
([email protected] <mailto:[email protected]>) wrote:
Can you show the exact error you getting?
/Cheers!/
Harsh Choudhary
On Thu, Mar 30, 2017 at 11:39 PM, I PVP <[email protected]
<mailto:[email protected]>> wrote:
that is how it is being done as of now:
sudo chown -R storm:storm /opt/storm
sudo chmod -R 700 /opt/storm
but still facing some issues while submitting topologies.
looks like Nimbus is not being able to create blob store
directories or files.
i just tried sudo chmod -R 777 /opt/storm. but the issue
still happening.
On March 30, 2017 at 3:02:07 PM, Harsh Choudhary
([email protected] <mailto:[email protected]>) wrote:
It depends on from which user, you are running storm. That
user must own the folders of storm. So, you need not use
chmod but chown.
/Cheers!/
Harsh Choudhary
On Thu, Mar 30, 2017 at 11:24 PM, I PVP <[email protected]
<mailto:[email protected]>> wrote:
What are the recommended files/folders permissions for
running Storm on Linux ?
Can I just set the entire folder tree to 700 or are
there specific files/folders that required a particular
permission ?
I am installing Storm 1.0.3 on Red Hat Enterprise Linux
7.3.
Storm path is /opt/storm
user: storm:storm.
Thanks
IPVP
--
My THALES email is [email protected].
+33 (0)5 62 88 84 40
Thales Services, Toulouse, France
