Re: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread Jonas Pfefferle
I'm talking subsystem NQN. Host NQN should not be relevant. Per default Host NQN of the clients is generated randomly (you can set it via the configuration file). On Wed, 26 Jun 2019 16:22:18 + David Crespi wrote: We’re talking host nqn right, not subsystem nqn? David 714-476-2692

Re: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread David Crespi
We’re talking host nqn right, not subsystem nqn? David 714-476-2692 > On Jun 26, 2019, at 09:17, Jonas Pfefferle wrote: > > There can only be 1 NQN in the whole system. For multiple storage classes > just use different ip/port combinations with the same NQN. e.g. start a NVMf > server on

Re: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread Jonas Pfefferle
There can only be 1 NQN in the whole system. For multiple storage classes just use different ip/port combinations with the same NQN. e.g. start a NVMf server on node 1 with storage class 1 with NQN X and start a server on node 2 with storage class 2 with NQN X. You can start another server on

RE: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread David Crespi
Sorry Jonas but I’m still confused. I started off doing this by having each storage class 1 connected to it’s own subsystem, with a single namespace. This is where I was seeing the correct IP address of that subsystem, but the wrong controller name. It was the controller name of the storage

RE: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread David Crespi
Yep. I wanted to use crail-store for temp results storage, storage class 2, AND the shuffle data, with storage class 1. Regards, David From: Jonas Pfefferle Sent: Wednesday, June 26, 2019 7:51:43 AM To: dev@crail.apache.org; David Crespi

RE: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread David Crespi
I’m a bit confused now. If it only ever handles 1 namespace, then multiple datanodes are useless. Or are you saying only 1 namespace for type-1? If you’re not saying only one for type-1, how do you mix types if limited to 1 namespace? Regards, David

Re: Question on crail type-1 with terasort shuffle data

2019-06-26 Thread Jonas Pfefferle
I forgot to add: Namespace has to be 1 for the moment. This is a limitation in how the namenode handles storage tier metadata (essentially it only stores ip/port). You can set it on the server side however it will not be respected by the client, i.e. the client will always use namespace id 1.