Re: Ports Drill Uses

[John Omernik]

Jacques -

I don't the behavior works how I interpreted your explanation

I set drill.exec.http.port to be 2, and drill.exec.rpc.user.server.port
to be 20001

I expected

web:2
user:20001
control:20002
data:20003

based on your summary.

What I got was

web:2
user:20001
control:31011
data:31012


Could there be two more settings, drill.exec.rpc.control.server.port and
drill.exec.rpc.data.server.port?




On Thu, Feb 11, 2016 at 2:25 PM, Bob Rumsby  wrote:

> Bridget is on vacation but will pick this up when she gets back next week.
> Kristine left the company in January.
>
> Bob
>
> On Thu, Feb 11, 2016 at 12:24 PM, John Omernik  wrote:
>
> > Awesome.  I would say that the page can stay on the Ports page, and just
> > removing the UDP thing, and then documenting those two settings and
> > examples of what that means (the start port) I can try to come up with
> > something, Is this something Kristine would want to jump on just based on
> > this thread or should I formalize something and let her handle it from
> > there?
> >
> > On Thu, Feb 11, 2016 at 1:42 PM, Jacques Nadeau 
> > wrote:
> >
> > > >
> > > >
> > > > Awesome.   Is there a doc page on this?
> > >
> > >
> > > There should be. Sounds like a great contribution :)
> > >
> > >
> > > > 1. "Starts" at, that indicates if I set
> drill.exec.rpc.user.server.port
> > > to
> > > > say 5, then instead of 31010, 31011, and 31012 (per the doc page)
> > > then
> > > > it would use 5, 50001, 50002?
> > > >
> > >
> > > yep
> > >
> > >
> > > > 2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
> > > > Infinispan. Needed for multi-node installation of Apache Drill.)?
> > > >
> > >
> > > We no longer use JGroups or Infinispan. No UDP needed.
> > >
> > >
> > > > 3.  So this can be on per node basis, in that when it registers with
> > the
> > > > cluster defined in drill-override.conf, it registers it's ports so
> the
> > > > other nodes are aware of the ports without any other sort of config?
> > > (I.e.
> > > > I can have bit one start on port 31010 by default, and then have bit
> > two
> > > > start at 5, etc)
> > > >
> > >
> > > Correct. Each node registers its information in zookeeper.
> > >
> > >
> > > > 4. Are there any odd issues that people have seen by using
> non-standard
> > > > ports or is it built to handle that fairly easily?
> > > >
> > >
> > > I don't think it has been used a lot beyond unit tests where we want to
> > run
> > > multiple Drillbits on the same node. The mechanism is fairly simple so
> I
> > > would expect major issues.
> > >
> >
>

Re: Ports Drill Uses

[John Omernik]

Ah ok,

so by setting drill.exec.rpc.bit.server.port to 20002, the "data" port did
that + 1


In summary
drill.exec -
http.port: 2
rpc.user.server.port: 20001
rpc:bit:server.port: 20002

resulted in

web:2
user: 20001
control: 20002
data: 20003





On Fri, Feb 12, 2016 at 11:37 AM, John Omernik  wrote:

> I found drill.exec.rpc.bit.server.port  in the over-example, I will set
> that to 20002  now to find the "Data" port
>
>
>
> On Fri, Feb 12, 2016 at 11:34 AM, John Omernik  wrote:
>
>> Jacques -
>>
>> I don't the behavior works how I interpreted your explanation
>>
>> I set drill.exec.http.port to be 2, and
>> drill.exec.rpc.user.server.port to be 20001
>>
>> I expected
>>
>> web:2
>> user:20001
>> control:20002
>> data:20003
>>
>> based on your summary.
>>
>> What I got was
>>
>> web:2
>> user:20001
>> control:31011
>> data:31012
>>
>>
>> Could there be two more settings, drill.exec.rpc.control.server.port and
>> drill.exec.rpc.data.server.port?
>>
>>
>>
>>
>> On Thu, Feb 11, 2016 at 2:25 PM, Bob Rumsby  wrote:
>>
>>> Bridget is on vacation but will pick this up when she gets back next
>>> week.
>>> Kristine left the company in January.
>>>
>>> Bob
>>>
>>> On Thu, Feb 11, 2016 at 12:24 PM, John Omernik  wrote:
>>>
>>> > Awesome.  I would say that the page can stay on the Ports page, and
>>> just
>>> > removing the UDP thing, and then documenting those two settings and
>>> > examples of what that means (the start port) I can try to come up with
>>> > something, Is this something Kristine would want to jump on just based
>>> on
>>> > this thread or should I formalize something and let her handle it from
>>> > there?
>>> >
>>> > On Thu, Feb 11, 2016 at 1:42 PM, Jacques Nadeau 
>>> > wrote:
>>> >
>>> > > >
>>> > > >
>>> > > > Awesome.   Is there a doc page on this?
>>> > >
>>> > >
>>> > > There should be. Sounds like a great contribution :)
>>> > >
>>> > >
>>> > > > 1. "Starts" at, that indicates if I set
>>> drill.exec.rpc.user.server.port
>>> > > to
>>> > > > say 5, then instead of 31010, 31011, and 31012 (per the doc
>>> page)
>>> > > then
>>> > > > it would use 5, 50001, 50002?
>>> > > >
>>> > >
>>> > > yep
>>> > >
>>> > >
>>> > > > 2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
>>> > > > Infinispan. Needed for multi-node installation of Apache Drill.)?
>>> > > >
>>> > >
>>> > > We no longer use JGroups or Infinispan. No UDP needed.
>>> > >
>>> > >
>>> > > > 3.  So this can be on per node basis, in that when it registers
>>> with
>>> > the
>>> > > > cluster defined in drill-override.conf, it registers it's ports so
>>> the
>>> > > > other nodes are aware of the ports without any other sort of
>>> config?
>>> > > (I.e.
>>> > > > I can have bit one start on port 31010 by default, and then have
>>> bit
>>> > two
>>> > > > start at 5, etc)
>>> > > >
>>> > >
>>> > > Correct. Each node registers its information in zookeeper.
>>> > >
>>> > >
>>> > > > 4. Are there any odd issues that people have seen by using
>>> non-standard
>>> > > > ports or is it built to handle that fairly easily?
>>> > > >
>>> > >
>>> > > I don't think it has been used a lot beyond unit tests where we want
>>> to
>>> > run
>>> > > multiple Drillbits on the same node. The mechanism is fairly simple
>>> so I
>>> > > would expect major issues.
>>> > >
>>> >
>>>
>>
>>
>

Re: Ports Drill Uses

[John Omernik]

I found drill.exec.rpc.bit.server.port  in the over-example, I will set
that to 20002  now to find the "Data" port



On Fri, Feb 12, 2016 at 11:34 AM, John Omernik  wrote:

> Jacques -
>
> I don't the behavior works how I interpreted your explanation
>
> I set drill.exec.http.port to be 2, and
> drill.exec.rpc.user.server.port to be 20001
>
> I expected
>
> web:2
> user:20001
> control:20002
> data:20003
>
> based on your summary.
>
> What I got was
>
> web:2
> user:20001
> control:31011
> data:31012
>
>
> Could there be two more settings, drill.exec.rpc.control.server.port and
> drill.exec.rpc.data.server.port?
>
>
>
>
> On Thu, Feb 11, 2016 at 2:25 PM, Bob Rumsby  wrote:
>
>> Bridget is on vacation but will pick this up when she gets back next week.
>> Kristine left the company in January.
>>
>> Bob
>>
>> On Thu, Feb 11, 2016 at 12:24 PM, John Omernik  wrote:
>>
>> > Awesome.  I would say that the page can stay on the Ports page, and just
>> > removing the UDP thing, and then documenting those two settings and
>> > examples of what that means (the start port) I can try to come up with
>> > something, Is this something Kristine would want to jump on just based
>> on
>> > this thread or should I formalize something and let her handle it from
>> > there?
>> >
>> > On Thu, Feb 11, 2016 at 1:42 PM, Jacques Nadeau 
>> > wrote:
>> >
>> > > >
>> > > >
>> > > > Awesome.   Is there a doc page on this?
>> > >
>> > >
>> > > There should be. Sounds like a great contribution :)
>> > >
>> > >
>> > > > 1. "Starts" at, that indicates if I set
>> drill.exec.rpc.user.server.port
>> > > to
>> > > > say 5, then instead of 31010, 31011, and 31012 (per the doc
>> page)
>> > > then
>> > > > it would use 5, 50001, 50002?
>> > > >
>> > >
>> > > yep
>> > >
>> > >
>> > > > 2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
>> > > > Infinispan. Needed for multi-node installation of Apache Drill.)?
>> > > >
>> > >
>> > > We no longer use JGroups or Infinispan. No UDP needed.
>> > >
>> > >
>> > > > 3.  So this can be on per node basis, in that when it registers with
>> > the
>> > > > cluster defined in drill-override.conf, it registers it's ports so
>> the
>> > > > other nodes are aware of the ports without any other sort of config?
>> > > (I.e.
>> > > > I can have bit one start on port 31010 by default, and then have bit
>> > two
>> > > > start at 5, etc)
>> > > >
>> > >
>> > > Correct. Each node registers its information in zookeeper.
>> > >
>> > >
>> > > > 4. Are there any odd issues that people have seen by using
>> non-standard
>> > > > ports or is it built to handle that fairly easily?
>> > > >
>> > >
>> > > I don't think it has been used a lot beyond unit tests where we want
>> to
>> > run
>> > > multiple Drillbits on the same node. The mechanism is fairly simple
>> so I
>> > > would expect major issues.
>> > >
>> >
>>
>
>

Re: Ports Drill Uses

[John Omernik]

Awesome.   Is there a doc page on this? Some questions I have below, if we
don't have a doc page on this, could be helpful.

1. "Starts" at, that indicates if I set drill.exec.rpc.user.server.port to
say 5, then instead of 31010, 31011, and 31012 (per the doc page) then
it would use 5, 50001, 50002?
2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
Infinispan. Needed for multi-node installation of Apache Drill.)?
3.  So this can be on per node basis, in that when it registers with the
cluster defined in drill-override.conf, it registers it's ports so the
other nodes are aware of the ports without any other sort of config? (I.e.
I can have bit one start on port 31010 by default, and then have bit two
start at 5, etc)
4. Are there any odd issues that people have seen by using non-standard
ports or is it built to handle that fairly easily?


Thanks for the answers!

John


On Thu, Feb 11, 2016 at 1:37 AM, Jacques Nadeau  wrote:

> You can change the ports that Drill starts at from 31010 to something else
> by setting:
>
> drill.exec.rpc.user.server.port
>
> You can change the HTTP port by setting:
>
> drill.exec.http.port
>
> Both of these are configured in drill-override.conf on a per node basis.
> You can also disable the web server on a particular node by setting
> drill.exec.http.enabled to false.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Feb 10, 2016 at 8:54 AM, John Omernik  wrote:
>
> > Using https://drill.apache.org/docs/ports-used-by-drill/ as  start, I
> was
> > wondering if it is possible in the drill-overide.conf to set these ports
> to
> > be a specific port?
> >
> > Do all ports on each drillbit have to be same from bit to bit in a
> specific
> > drill cluster?  I.e. if I set the ports, that will be the ports used on a
> > bits for that cluster? I suppose at least one port must be the same on
> each
> > bit per drill cluster, otherwise they wouldn't know how to find. Either
> > way, is it possible to set these to be different values?
> >
> > Thanks!
> >
> > John
> >
>

Re: Ports Drill Uses

[John Omernik]

Awesome.  I would say that the page can stay on the Ports page, and just
removing the UDP thing, and then documenting those two settings and
examples of what that means (the start port) I can try to come up with
something, Is this something Kristine would want to jump on just based on
this thread or should I formalize something and let her handle it from
there?

On Thu, Feb 11, 2016 at 1:42 PM, Jacques Nadeau  wrote:

> >
> >
> > Awesome.   Is there a doc page on this?
>
>
> There should be. Sounds like a great contribution :)
>
>
> > 1. "Starts" at, that indicates if I set drill.exec.rpc.user.server.port
> to
> > say 5, then instead of 31010, 31011, and 31012 (per the doc page)
> then
> > it would use 5, 50001, 50002?
> >
>
> yep
>
>
> > 2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
> > Infinispan. Needed for multi-node installation of Apache Drill.)?
> >
>
> We no longer use JGroups or Infinispan. No UDP needed.
>
>
> > 3.  So this can be on per node basis, in that when it registers with the
> > cluster defined in drill-override.conf, it registers it's ports so the
> > other nodes are aware of the ports without any other sort of config?
> (I.e.
> > I can have bit one start on port 31010 by default, and then have bit two
> > start at 5, etc)
> >
>
> Correct. Each node registers its information in zookeeper.
>
>
> > 4. Are there any odd issues that people have seen by using non-standard
> > ports or is it built to handle that fairly easily?
> >
>
> I don't think it has been used a lot beyond unit tests where we want to run
> multiple Drillbits on the same node. The mechanism is fairly simple so I
> would expect major issues.
>

Re: Ports Drill Uses

[Bob Rumsby]

Bridget is on vacation but will pick this up when she gets back next week.
Kristine left the company in January.

Bob

On Thu, Feb 11, 2016 at 12:24 PM, John Omernik  wrote:

> Awesome.  I would say that the page can stay on the Ports page, and just
> removing the UDP thing, and then documenting those two settings and
> examples of what that means (the start port) I can try to come up with
> something, Is this something Kristine would want to jump on just based on
> this thread or should I formalize something and let her handle it from
> there?
>
> On Thu, Feb 11, 2016 at 1:42 PM, Jacques Nadeau 
> wrote:
>
> > >
> > >
> > > Awesome.   Is there a doc page on this?
> >
> >
> > There should be. Sounds like a great contribution :)
> >
> >
> > > 1. "Starts" at, that indicates if I set drill.exec.rpc.user.server.port
> > to
> > > say 5, then instead of 31010, 31011, and 31012 (per the doc page)
> > then
> > > it would use 5, 50001, 50002?
> > >
> >
> > yep
> >
> >
> > > 2.  What about the UDP Port it uses (46655 UDP Used for JGroups and
> > > Infinispan. Needed for multi-node installation of Apache Drill.)?
> > >
> >
> > We no longer use JGroups or Infinispan. No UDP needed.
> >
> >
> > > 3.  So this can be on per node basis, in that when it registers with
> the
> > > cluster defined in drill-override.conf, it registers it's ports so the
> > > other nodes are aware of the ports without any other sort of config?
> > (I.e.
> > > I can have bit one start on port 31010 by default, and then have bit
> two
> > > start at 5, etc)
> > >
> >
> > Correct. Each node registers its information in zookeeper.
> >
> >
> > > 4. Are there any odd issues that people have seen by using non-standard
> > > ports or is it built to handle that fairly easily?
> > >
> >
> > I don't think it has been used a lot beyond unit tests where we want to
> run
> > multiple Drillbits on the same node. The mechanism is fairly simple so I
> > would expect major issues.
> >
>

Ports Drill Uses

[John Omernik]

Using https://drill.apache.org/docs/ports-used-by-drill/ as  start, I was
wondering if it is possible in the drill-overide.conf to set these ports to
be a specific port?

Do all ports on each drillbit have to be same from bit to bit in a specific
drill cluster?  I.e. if I set the ports, that will be the ports used on a
bits for that cluster? I suppose at least one port must be the same on each
bit per drill cluster, otherwise they wouldn't know how to find. Either
way, is it possible to set these to be different values?

Thanks!

John

Re: Ports Drill Uses

[Jacques Nadeau]

You can change the ports that Drill starts at from 31010 to something else
by setting:

drill.exec.rpc.user.server.port

You can change the HTTP port by setting:

drill.exec.http.port

Both of these are configured in drill-override.conf on a per node basis.
You can also disable the web server on a particular node by setting
drill.exec.http.enabled to false.

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, Feb 10, 2016 at 8:54 AM, John Omernik  wrote:

> Using https://drill.apache.org/docs/ports-used-by-drill/ as  start, I was
> wondering if it is possible in the drill-overide.conf to set these ports to
> be a specific port?
>
> Do all ports on each drillbit have to be same from bit to bit in a specific
> drill cluster?  I.e. if I set the ports, that will be the ports used on a
> bits for that cluster? I suppose at least one port must be the same on each
> bit per drill cluster, otherwise they wouldn't know how to find. Either
> way, is it possible to set these to be different values?
>
> Thanks!
>
> John
>