Re: Requesting reviews for ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol

[Atri Sharma]

I can help review this

On Apr 20, 2017 2:22 PM, "Abraham Fine"  wrote:

> Hello-
>
> I have been continuing work on ZOOKEEPER-236 and it would be great to
> get feedback from the community on the patch. The pull request can be
> found here: https://github.com/apache/zookeeper/pull/184
>
> ZOOKEEPER-236 provides the ability to use SSL/TLS to secure
> communication within the ZooKeeper quorum.
>
> Documentation will be handled in another pull request but the usage is
> very similar to our existing Client <-> Quorum functionality, here is an
> overview of the basic configuration.
>
> System properties are set on each member of the quorum, for example:
> -Dzookeeper.ssl.quorum.keyStore.location=keystore.jks
> -Dzookeeper.ssl.quorum.keyStore.password=password
> -Dzookeeper.ssl.quorum.trustStore.location=truststore.jks
>
> A flag is set in the cfg files:
> sslQuorum=true
>
> The best way to see all the functionality provided by this patch is to
> take a look at the integration tests:
> https://github.com/afine/zookeeper/blob/3c6c81b69b7105fa7c5235a0f27718
> a7eae195de/src/java/test/org/apache/zookeeper/test/QuorumSSLTest.java.
> The integration tests contain examples showing how hostname
> verification, rolling upgrades, cipher configuration, protocol
> configuration, and certificate revocation are handled.
>
> There is a current outstanding question regarding hostname verification,
> please provide input here:
> https://github.com/apache/zookeeper/pull/184#discussion_r111485824
>
> Looking forward to hearing everyone's thoughts.
>
> Thanks,
> Abraham Fine
>
>
>

[jira] [Commented] (ZOOKEEPER-2362) ZooKeeper multi / transaction allows partial read

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977383#comment-15977383
 ] 

Atri Sharma commented on ZOOKEEPER-2362:


I see this issue to be still there. If it is a release blocker, can I go ahead 
and take this? [~hanm]

> ZooKeeper multi / transaction allows partial read
> -
>
> Key: ZOOKEEPER-2362
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2362
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: Whitney Sorenson
>Priority: Critical
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In this thread 
> http://mail-archives.apache.org/mod_mbox/zookeeper-user/201602.mbox/%3CCAPbqGzicBkLLyVDm7RFM20z0y3X1v1P-C9-1%3D%3D1DDqRDTzdOmQ%40mail.gmail.com%3E
>  , I discussed an issue I've now seen in multiple environments:
> In a multi (using Curator), I write 2 new nodes. At some point, I issue 2 
> reads for these new nodes. In one read, I see one of the new nodes. In a 
> subsequent read, I fail to see the other new node:
> 1. Starting state : { /foo = , /bar =  }
> 2. In a multi, write: { /foo = A, /bar = B}
> 3. Read /foo as A
> 4. Read /bar as  
> #3 and #4 are issued 100% sequentially.
> It is not known at what point during #2, #3 starts.
> Note: the reads are getChildren() calls.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-1609) Improve ZooKeeper performance under mixed workload

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960675#comment-15960675
 ] 

Atri Sharma commented on ZOOKEEPER-1609:


Folks, I am planning to work on this. Please let me know if it is relevant and 
is not done yet. [~shralex]

> Improve ZooKeeper performance under mixed workload
> --
>
> Key: ZOOKEEPER-1609
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1609
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.4.3
>Reporter: Thawan Kooburat
>
> ZOOKEEPER-1505 allows 1 write or N reads to pass through the CommitProcessor 
> at any given time. I did performance experiment similar to 
> http://wiki.apache.org/hadoop/ZooKeeper/Performance and found that read 
> throughput drop dramatically when there are write requests. After a bit more 
> investigation, I found that
> the biggest bottleneck is at the request queue entering the CommitProcessor.
> When the CommitProcessor see any write request, it will need to block the 
> entire pipeline and wait until matching commit from the leader. This means 
> that all read requests (including ping request) won't be able to go through. 
> The time spent waiting for commit from the leader far exceed the time spent 
> waiting for 1 write to goes through the CommitProcessor. 
> The current plan is to create multiple request queues at the front of the 
> CommitProcessor. Requests are hashed using sessionId and send to one of the 
> queue. Whenever, the CommitProcessor saw a write request on one of the queue 
> it moves on to process read requests. It will have to unblock the write 
> requests in the same order that it sent to the leader, so it may need to 
> maintain a separate list to keep track of that. 
> The correctness is the same as having more learners in the ensemble. Sessions 
> which are hashed onto a different queue is similar to sessions connecting to 
> a different learners in the ensemble. 
> I am hoping that this will improve read throughput and reduce disconnect rate 
> on an ensemble with large number of clients  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2076) Improve Leader Change Mechanism

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950539#comment-15950539
 ] 

Atri Sharma commented on ZOOKEEPER-2076:


Hi Folks,

Is this still valid? [~shralex]

If nobody is working on this, I can take it up

> Improve Leader Change Mechanism
> ---
>
> Key: ZOOKEEPER-2076
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2076
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Alexander Shraer
>
> When a leader is removed during a reconfiguration, ZOOKEEPER-107 uses a 
> mechanism where the old leader nominates the new one. Although it reduces the 
> time for a new leader to be elected, it still takes too long. This JIRA is 
> for two things:
> 1. Improve the mechanism, e.g., avoid loading snapshots, etc. during the 
> handoff.
> 2. Make it a first-class citizen & export it as a client API. We get 
> questions about this once in a while - how do I cause a different leader to 
> be elected ? Currently the response is either kill or reconfigure the current 
> leader.
> Any one interested to work on this ?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-900) FLE implementation should be improved to use non-blocking sockets

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946569#comment-15946569
 ] 

Atri Sharma commented on ZOOKEEPER-900:
---

Ping?

> FLE implementation should be improved to use non-blocking sockets
> -
>
> Key: ZOOKEEPER-900
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-900
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Vishal Kher
>Assignee: Martin Kuchta
>Priority: Critical
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-900-part2.patch, ZOOKEEPER-900.patch, 
> ZOOKEEPER-900.patch1, ZOOKEEPER-900.patch2
>
>
> From earlier email exchanges:
> 1. Blocking connects and accepts:
> a) The first problem is in manager.toSend(). This invokes connectOne(), which 
> does a blocking connect. While testing, I changed the code so that 
> connectOne() starts a new thread called AsyncConnct(). AsyncConnect.run() 
> does a socketChannel.connect(). After starting AsyncConnect, connectOne 
> starts a timer. connectOne continues with normal operations if the connection 
> is established before the timer expires, otherwise, when the timer expires it 
> interrupts AsyncConnect() thread and returns. In this way, I can have an 
> upper bound on the amount of time we need to wait for connect to succeed. Of 
> course, this was a quick fix for my testing. Ideally, we should use Selector 
> to do non-blocking connects/accepts. I am planning to do that later once we 
> at least have a quick fix for the problem and consensus from others for the 
> real fix (this problem is big blocker for us). Note that it is OK to do 
> blocking IO in SenderWorker and RecvWorker threads since they block IO to the 
> respective peer.
> b) The blocking IO problem is not just restricted to connectOne(), but also 
> in receiveConnection(). The Listener thread calls receiveConnection() for 
> each incoming connection request. receiveConnection does blocking IO to get 
> peer's info (s.read(msgBuffer)). Worse, it invokes connectOne() back to the 
> peer that had sent the connection request. All of this is happening from the 
> Listener. In short, if a peer fails after initiating a connection, the 
> Listener thread won't be able to accept connections from other peers, because 
> it would be stuck in read() or connetOne(). Also the code has an inherent 
> cycle. initiateConnection() and receiveConnection() will have to be very 
> carefully synchronized otherwise, we could run into deadlocks. This code is 
> going to be difficult to maintain/modify.
> Also see: https://issues.apache.org/jira/browse/ZOOKEEPER-822



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: ZOOKEEPER-900

[Atri Sharma]

Hi Patrick,

I am fine with either. I tried pinging the owners but got no response.

Please let me know.

Regards,

Atri

On Tue, Mar 28, 2017 at 4:51 AM, Patrick Hunt <ph...@apache.org> wrote:
> Hi Atri. Which do you intend to work on? 900, 901, or both? Typically if
> someone is intending to work on something they will be listed as the
> "assigned" in JIRA. They would be the first person to check in with. If
> they no longer intend to work on something it's easy enough to reassign.
>
> Regards,
>
> Patrick
>
> On Sat, Mar 25, 2017 at 7:36 AM, Atri Sharma <atri.j...@gmail.com> wrote:
>
>> Hi folks,
>>
>> I was looking to work on ZOOKEEPER-901. Could anybody please let me
>> know if they are working on it?
>>
>> --
>> Regards,
>>
>> Atri
>> l'apprenant
>>



-- 
Regards,

Atri
l'apprenant

ZOOKEEPER-900

[Atri Sharma]

Hi folks,

I was looking to work on ZOOKEEPER-901. Could anybody please let me
know if they are working on it?

-- 
Regards,

Atri
l'apprenant

[jira] [Commented] (ZOOKEEPER-900) FLE implementation should be improved to use non-blocking sockets

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940894#comment-15940894
 ] 

Atri Sharma commented on ZOOKEEPER-900:
---

Could you please reassign it to me/ point me to another JIRA is this umbrella?

> FLE implementation should be improved to use non-blocking sockets
> -
>
> Key: ZOOKEEPER-900
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-900
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Vishal Kher
>Assignee: Martin Kuchta
>Priority: Critical
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-900-part2.patch, ZOOKEEPER-900.patch, 
> ZOOKEEPER-900.patch1, ZOOKEEPER-900.patch2
>
>
> From earlier email exchanges:
> 1. Blocking connects and accepts:
> a) The first problem is in manager.toSend(). This invokes connectOne(), which 
> does a blocking connect. While testing, I changed the code so that 
> connectOne() starts a new thread called AsyncConnct(). AsyncConnect.run() 
> does a socketChannel.connect(). After starting AsyncConnect, connectOne 
> starts a timer. connectOne continues with normal operations if the connection 
> is established before the timer expires, otherwise, when the timer expires it 
> interrupts AsyncConnect() thread and returns. In this way, I can have an 
> upper bound on the amount of time we need to wait for connect to succeed. Of 
> course, this was a quick fix for my testing. Ideally, we should use Selector 
> to do non-blocking connects/accepts. I am planning to do that later once we 
> at least have a quick fix for the problem and consensus from others for the 
> real fix (this problem is big blocker for us). Note that it is OK to do 
> blocking IO in SenderWorker and RecvWorker threads since they block IO to the 
> respective peer.
> b) The blocking IO problem is not just restricted to connectOne(), but also 
> in receiveConnection(). The Listener thread calls receiveConnection() for 
> each incoming connection request. receiveConnection does blocking IO to get 
> peer's info (s.read(msgBuffer)). Worse, it invokes connectOne() back to the 
> peer that had sent the connection request. All of this is happening from the 
> Listener. In short, if a peer fails after initiating a connection, the 
> Listener thread won't be able to accept connections from other peers, because 
> it would be stuck in read() or connetOne(). Also the code has an inherent 
> cycle. initiateConnection() and receiveConnection() will have to be very 
> carefully synchronized otherwise, we could run into deadlocks. This code is 
> going to be difficult to maintain/modify.
> Also see: https://issues.apache.org/jira/browse/ZOOKEEPER-822



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-900) FLE implementation should be improved to use non-blocking sockets

[Atri Sharma (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940766#comment-15940766
 ] 

Atri Sharma commented on ZOOKEEPER-900:
---

Hi Folks,

Is this still being worked on? I was thinking of taking this up.

Please let me know

> FLE implementation should be improved to use non-blocking sockets
> -
>
> Key: ZOOKEEPER-900
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-900
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Vishal Kher
>Assignee: Martin Kuchta
>Priority: Critical
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-900-part2.patch, ZOOKEEPER-900.patch, 
> ZOOKEEPER-900.patch1, ZOOKEEPER-900.patch2
>
>
> From earlier email exchanges:
> 1. Blocking connects and accepts:
> a) The first problem is in manager.toSend(). This invokes connectOne(), which 
> does a blocking connect. While testing, I changed the code so that 
> connectOne() starts a new thread called AsyncConnct(). AsyncConnect.run() 
> does a socketChannel.connect(). After starting AsyncConnect, connectOne 
> starts a timer. connectOne continues with normal operations if the connection 
> is established before the timer expires, otherwise, when the timer expires it 
> interrupts AsyncConnect() thread and returns. In this way, I can have an 
> upper bound on the amount of time we need to wait for connect to succeed. Of 
> course, this was a quick fix for my testing. Ideally, we should use Selector 
> to do non-blocking connects/accepts. I am planning to do that later once we 
> at least have a quick fix for the problem and consensus from others for the 
> real fix (this problem is big blocker for us). Note that it is OK to do 
> blocking IO in SenderWorker and RecvWorker threads since they block IO to the 
> respective peer.
> b) The blocking IO problem is not just restricted to connectOne(), but also 
> in receiveConnection(). The Listener thread calls receiveConnection() for 
> each incoming connection request. receiveConnection does blocking IO to get 
> peer's info (s.read(msgBuffer)). Worse, it invokes connectOne() back to the 
> peer that had sent the connection request. All of this is happening from the 
> Listener. In short, if a peer fails after initiating a connection, the 
> Listener thread won't be able to accept connections from other peers, because 
> it would be stuck in read() or connetOne(). Also the code has an inherent 
> cycle. initiateConnection() and receiveConnection() will have to be very 
> carefully synchronized otherwise, we could run into deadlocks. This code is 
> going to be difficult to maintain/modify.
> Also see: https://issues.apache.org/jira/browse/ZOOKEEPER-822



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Contributing to Zookeeper

[Atri Sharma]

Thanks, that is very helpful

On Mar 19, 2017 10:13 PM, "Jordan Zimmerman" <jor...@jordanzimmerman.com>
wrote:

> Atri,
>
> Docs regarding ZooKeeper are abundant on the Internet. Like all Apache
> projects, the best place to start is the project website:
> http://zookeeper.apache.org <http://zookeeper.apache.org/>. Also, like
> most Apache projects there is a wiki with tons of information:
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Index <
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Index>
>
> -Jordan
>
> > On Mar 19, 2017, at 11:04 AM, Atri Sharma <a...@apache.org> wrote:
> >
> > Thanks Camille.
> >
> > Could you please point me to some internals/code docs that I could
> > refer to for getting started?
> >
> > Regards,
> >
> > Atri
> >
> > On Sun, Mar 19, 2017 at 1:48 AM, Camille Fournier <cami...@apache.org>
> wrote:
> >> Hi Atri,
> >>
> >> We're always happy to have folks contribute. I would recommend starting
> by
> >> reading the various documentation on how to contribute, and hanging out
> on
> >> the mailing lists for a while to get a feel for the project. Once you
> see
> >> something that makes sense for your skill set, volunteer an answer or a
> >> patch. That's the way most of us have gotten involved here.
> >>
> >> Cheers,
> >> C
> >>
> >> On Sat, Mar 18, 2017 at 10:20 AM, Atri Sharma <atri.j...@gmail.com>
> wrote:
> >>
> >>> Hi folks,
> >>>
> >>> Please advise
> >>>
> >>> On Mar 17, 2017 10:25 PM, "Atri Sharma" <a...@apache.org> wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> I am a distributed systems engineer with experience across different
> >>>> spectrum of highly scalable systems and have worked with consistency
> >>>> and quorum protocols.
> >>>>
> >>>> I would be happy to help out on any ongoing/needed feature in
> >>>> Zookeeper. Please let me know.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Atri
> >>>>
> >>>
> >
> >
> >
> > --
> > Regards,
> >
> > Atri
> > Apache Concerted
>
>

Re: Contributing to Zookeeper

[Atri Sharma]

Thanks Camille.

Could you please point me to some internals/code docs that I could
refer to for getting started?

Regards,

Atri

On Sun, Mar 19, 2017 at 1:48 AM, Camille Fournier <cami...@apache.org> wrote:
> Hi Atri,
>
> We're always happy to have folks contribute. I would recommend starting by
> reading the various documentation on how to contribute, and hanging out on
> the mailing lists for a while to get a feel for the project. Once you see
> something that makes sense for your skill set, volunteer an answer or a
> patch. That's the way most of us have gotten involved here.
>
> Cheers,
> C
>
> On Sat, Mar 18, 2017 at 10:20 AM, Atri Sharma <atri.j...@gmail.com> wrote:
>
>> Hi folks,
>>
>> Please advise
>>
>> On Mar 17, 2017 10:25 PM, "Atri Sharma" <a...@apache.org> wrote:
>>
>> > Hi All,
>> >
>> > I am a distributed systems engineer with experience across different
>> > spectrum of highly scalable systems and have worked with consistency
>> > and quorum protocols.
>> >
>> > I would be happy to help out on any ongoing/needed feature in
>> > Zookeeper. Please let me know.
>> >
>> >
>> > Regards,
>> >
>> > Atri
>> >
>>



-- 
Regards,

Atri
Apache Concerted

Re: Contributing to Zookeeper

[Atri Sharma]

Hi folks,

Please advise

On Mar 17, 2017 10:25 PM, "Atri Sharma" <a...@apache.org> wrote:

> Hi All,
>
> I am a distributed systems engineer with experience across different
> spectrum of highly scalable systems and have worked with consistency
> and quorum protocols.
>
> I would be happy to help out on any ongoing/needed feature in
> Zookeeper. Please let me know.
>
>
> Regards,
>
> Atri
>

Contributing to Zookeeper

[Atri Sharma]

Hi All,

I am a distributed systems engineer with experience across different
spectrum of highly scalable systems and have worked with consistency
and quorum protocols.

I would be happy to help out on any ongoing/needed feature in
Zookeeper. Please let me know.


Regards,

Atri