Re: [lldb-dev] [RFC]The future of pexpect

2019-03-08 Thread Davide Italiano via lldb-dev
After discussing this privately with Pavel we decided to try updating
pexpect instead.

On Tue, Feb 26, 2019 at 4:42 AM Pavel Labath  wrote:
>
> On 25/02/2019 22:15, Davide Italiano wrote:
> > On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
> >>
> >> On 21/02/2019 19:48, Ted Woodward wrote:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: lldb-dev  On Behalf Of Pavel 
> >>>> Labath
> >>>> via lldb-dev
> >>>> Sent: Thursday, February 21, 2019 8:35 AM
> >>>> To: Davide Italiano 
> >>>> Cc: LLDB 
> >>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> >>>>
> >>>> On 21/02/2019 00:03, Davide Italiano wrote:
> >>>>> I found out that there are tests that effectively require
> >>>>> interactivity. Some of the lldb-mi ones are an example.
> >>>>> A common use-case is that of sending SIGTERM in a loop to make sure
> >>>>> `lldb-mi` doesn't crash and handle the signal correctly.
> >>>>>
> >>>>> This functionality is really hard to replicate in lit_as is_.
> >>>>> Any ideas on how we could handle this case?
> >>>>
> >>>> How hard is it to import a new version of pexpect which supports python3 
> >>>> and
> >>>> stuff?
> >>>>
> >>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
> >>>> most linux
> >>>> systems either already have it installed, or have an easy way to do so. 
> >>>> So we
> >>>> may not even be able to get away with just using the system one and 
> >>>> skipping
> >>>> tests when it's not present.
> >>>>
> >>>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
> >>>> :D.
> >>>> Interactivity is one thing, and I'm very much in favour of keeping that 
> >>>> ability,
> >>>> but pexpect is not a prerequisite for that. For me, the main advantage of
> >>>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
> >>>> need
> >>>> that stuff. It doesn't have any command line editing capabilities or 
> >>>> similar. It's
> >>>> expecting to communicate with an IDE over a pipe, and that's it.
> >>>>
> >>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
> >>>> work on top
> >>>> of the standard python "subprocess" library. While we're doing that, we 
> >>>> might
> >>>> actually fix some of the issues that have been bugging everyone in the 
> >>>> lldb-mi
> >>>> tests. At least for me, the most annoying thing was that when lldb-mi 
> >>>> fails to
> >>>> produce the expected output, the test does not fail immediately, but 
> >>>> instead
> >>>> the implementation of self.expect("^whatever") waits until the timeout
> >>>> expires, optimistically hoping that it will find some output that match 
> >>>> the
> >>>> pattern.
> >>>>
> >
> > Pavel, I think yours is a really nice idea.
> > I'm no python expert, but I found out making the conversion is
> > relatively simple.
> > I propose a proof-of-concept API and implementation here:
> >
> > https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> >
> > Comments appreciated! Once we agree on how this should look like, I do
> > recommend to have a new lldbMITest base class and incrementally start
> > moving the tests to it.
> > Once we're done, we can delete the old class.
> >
> > Does this sound reasonable?
> >
> > --
> > Davide
> >
>
> Sounds great. Let's ship it. :)
>
> pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-26 Thread Pavel Labath via lldb-dev

On 25/02/2019 22:15, Davide Italiano wrote:

On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:


On 21/02/2019 19:48, Ted Woodward wrote:




-Original Message-
From: lldb-dev  On Behalf Of Pavel Labath
via lldb-dev
Sent: Thursday, February 21, 2019 8:35 AM
To: Davide Italiano 
Cc: LLDB 
Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect

On 21/02/2019 00:03, Davide Italiano wrote:

I found out that there are tests that effectively require
interactivity. Some of the lldb-mi ones are an example.
A common use-case is that of sending SIGTERM in a loop to make sure
`lldb-mi` doesn't crash and handle the signal correctly.

This functionality is really hard to replicate in lit_as is_.
Any ideas on how we could handle this case?


How hard is it to import a new version of pexpect which supports python3 and
stuff?

I'm not sure how the situation is on darwin, but I'd expect (:P) that most linux
systems either already have it installed, or have an easy way to do so. So we
may not even be able to get away with just using the system one and skipping
tests when it's not present.

BTW, for lldb-mi I would actually argue that it should *not* use pexpect :D.
Interactivity is one thing, and I'm very much in favour of keeping that ability,
but pexpect is not a prerequisite for that. For me, the main advantage of
pexpect is that it emulates a real terminal. However, lldb-mi does not need
that stuff. It doesn't have any command line editing capabilities or similar. 
It's
expecting to communicate with an IDE over a pipe, and that's it.

Given that, it should be fairly easy to rewrite the lldb-mi tests to work on top
of the standard python "subprocess" library. While we're doing that, we might
actually fix some of the issues that have been bugging everyone in the lldb-mi
tests. At least for me, the most annoying thing was that when lldb-mi fails to
produce the expected output, the test does not fail immediately, but instead
the implementation of self.expect("^whatever") waits until the timeout
expires, optimistically hoping that it will find some output that match the
pattern.



Pavel, I think yours is a really nice idea.
I'm no python expert, but I found out making the conversion is
relatively simple.
I propose a proof-of-concept API and implementation here:

https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68

Comments appreciated! Once we agree on how this should look like, I do
recommend to have a new lldbMITest base class and incrementally start
moving the tests to it.
Once we're done, we can delete the old class.

Does this sound reasonable?

--
Davide



Sounds great. Let's ship it. :)

pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Adrian Prantl via lldb-dev


> On Feb 25, 2019, at 2:21 PM, Davide Italiano  wrote:
> 
> On Mon, Feb 25, 2019 at 1:57 PM Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Feb 25, 2019, at 1:40 PM, Davide Italiano  wrote:
>>> 
>>> On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Feb 25, 2019, at 1:15 PM, Davide Italiano  
>>>>> wrote:
>>>>> 
>>>>> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>>>>>> 
>>>>>> On 21/02/2019 19:48, Ted Woodward wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> -Original Message-----
>>>>>>>> From: lldb-dev  On Behalf Of Pavel 
>>>>>>>> Labath
>>>>>>>> via lldb-dev
>>>>>>>> Sent: Thursday, February 21, 2019 8:35 AM
>>>>>>>> To: Davide Italiano 
>>>>>>>> Cc: LLDB 
>>>>>>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
>>>>>>>> 
>>>>>>>> On 21/02/2019 00:03, Davide Italiano wrote:
>>>>>>>>> I found out that there are tests that effectively require
>>>>>>>>> interactivity. Some of the lldb-mi ones are an example.
>>>>>>>>> A common use-case is that of sending SIGTERM in a loop to make sure
>>>>>>>>> `lldb-mi` doesn't crash and handle the signal correctly.
>>>>>>>>> 
>>>>>>>>> This functionality is really hard to replicate in lit_as is_.
>>>>>>>>> Any ideas on how we could handle this case?
>>>>>>>> 
>>>>>>>> How hard is it to import a new version of pexpect which supports 
>>>>>>>> python3 and
>>>>>>>> stuff?
>>>>>>>> 
>>>>>>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
>>>>>>>> most linux
>>>>>>>> systems either already have it installed, or have an easy way to do 
>>>>>>>> so. So we
>>>>>>>> may not even be able to get away with just using the system one and 
>>>>>>>> skipping
>>>>>>>> tests when it's not present.
>>>>>>>> 
>>>>>>>> BTW, for lldb-mi I would actually argue that it should *not* use 
>>>>>>>> pexpect :D.
>>>>>>>> Interactivity is one thing, and I'm very much in favour of keeping 
>>>>>>>> that ability,
>>>>>>>> but pexpect is not a prerequisite for that. For me, the main advantage 
>>>>>>>> of
>>>>>>>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
>>>>>>>> need
>>>>>>>> that stuff. It doesn't have any command line editing capabilities or 
>>>>>>>> similar. It's
>>>>>>>> expecting to communicate with an IDE over a pipe, and that's it.
>>>>>>>> 
>>>>>>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
>>>>>>>> work on top
>>>>>>>> of the standard python "subprocess" library. While we're doing that, 
>>>>>>>> we might
>>>>>>>> actually fix some of the issues that have been bugging everyone in the 
>>>>>>>> lldb-mi
>>>>>>>> tests. At least for me, the most annoying thing was that when lldb-mi 
>>>>>>>> fails to
>>>>>>>> produce the expected output, the test does not fail immediately, but 
>>>>>>>> instead
>>>>>>>> the implementation of self.expect("^whatever") waits until the timeout
>>>>>>>> expires, optimistically hoping that it will find some output that 
>>>>>>>> match the
>>>>>>>> pattern.
>>>>>>>> 
>>>>> 
>>>>> Pavel, I think yours is a really nice idea.
>>>>> I'm no python expert, but I found out making the conversion is
>>>>> relatively simple.
>>>>> I propose a proof-of-concept API and implementation here:
>>>>> 
>>>>> https://gist.github.com/dcci/94a4936a227d9c7627b91a

Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Davide Italiano via lldb-dev
On Mon, Feb 25, 2019 at 1:57 PM Adrian Prantl  wrote:
>
>
>
> > On Feb 25, 2019, at 1:40 PM, Davide Italiano  wrote:
> >
> > On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
> >>
> >>
> >>
> >>> On Feb 25, 2019, at 1:15 PM, Davide Italiano  
> >>> wrote:
> >>>
> >>> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
> >>>>
> >>>> On 21/02/2019 19:48, Ted Woodward wrote:
> >>>>>
> >>>>>
> >>>>>> -Original Message-
> >>>>>> From: lldb-dev  On Behalf Of Pavel 
> >>>>>> Labath
> >>>>>> via lldb-dev
> >>>>>> Sent: Thursday, February 21, 2019 8:35 AM
> >>>>>> To: Davide Italiano 
> >>>>>> Cc: LLDB 
> >>>>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> >>>>>>
> >>>>>> On 21/02/2019 00:03, Davide Italiano wrote:
> >>>>>>> I found out that there are tests that effectively require
> >>>>>>> interactivity. Some of the lldb-mi ones are an example.
> >>>>>>> A common use-case is that of sending SIGTERM in a loop to make sure
> >>>>>>> `lldb-mi` doesn't crash and handle the signal correctly.
> >>>>>>>
> >>>>>>> This functionality is really hard to replicate in lit_as is_.
> >>>>>>> Any ideas on how we could handle this case?
> >>>>>>
> >>>>>> How hard is it to import a new version of pexpect which supports 
> >>>>>> python3 and
> >>>>>> stuff?
> >>>>>>
> >>>>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
> >>>>>> most linux
> >>>>>> systems either already have it installed, or have an easy way to do 
> >>>>>> so. So we
> >>>>>> may not even be able to get away with just using the system one and 
> >>>>>> skipping
> >>>>>> tests when it's not present.
> >>>>>>
> >>>>>> BTW, for lldb-mi I would actually argue that it should *not* use 
> >>>>>> pexpect :D.
> >>>>>> Interactivity is one thing, and I'm very much in favour of keeping 
> >>>>>> that ability,
> >>>>>> but pexpect is not a prerequisite for that. For me, the main advantage 
> >>>>>> of
> >>>>>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
> >>>>>> need
> >>>>>> that stuff. It doesn't have any command line editing capabilities or 
> >>>>>> similar. It's
> >>>>>> expecting to communicate with an IDE over a pipe, and that's it.
> >>>>>>
> >>>>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
> >>>>>> work on top
> >>>>>> of the standard python "subprocess" library. While we're doing that, 
> >>>>>> we might
> >>>>>> actually fix some of the issues that have been bugging everyone in the 
> >>>>>> lldb-mi
> >>>>>> tests. At least for me, the most annoying thing was that when lldb-mi 
> >>>>>> fails to
> >>>>>> produce the expected output, the test does not fail immediately, but 
> >>>>>> instead
> >>>>>> the implementation of self.expect("^whatever") waits until the timeout
> >>>>>> expires, optimistically hoping that it will find some output that 
> >>>>>> match the
> >>>>>> pattern.
> >>>>>>
> >>>
> >>> Pavel, I think yours is a really nice idea.
> >>> I'm no python expert, but I found out making the conversion is
> >>> relatively simple.
> >>> I propose a proof-of-concept API and implementation here:
> >>>
> >>> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> >>>
> >>> Comments appreciated! Once we agree on how this should look like, I do
> >>> recommend to have a new lldbMITest base class and incrementally start
> >>> moving the tests to it.
> >>> Once we're done, we can delete the old class.
> >>>
> &

Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Davide Italiano via lldb-dev
On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
>
>
>
> > On Feb 25, 2019, at 1:15 PM, Davide Italiano  wrote:
> >
> > On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
> >>
> >> On 21/02/2019 19:48, Ted Woodward wrote:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: lldb-dev  On Behalf Of Pavel 
> >>>> Labath
> >>>> via lldb-dev
> >>>> Sent: Thursday, February 21, 2019 8:35 AM
> >>>> To: Davide Italiano 
> >>>> Cc: LLDB 
> >>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> >>>>
> >>>> On 21/02/2019 00:03, Davide Italiano wrote:
> >>>>> I found out that there are tests that effectively require
> >>>>> interactivity. Some of the lldb-mi ones are an example.
> >>>>> A common use-case is that of sending SIGTERM in a loop to make sure
> >>>>> `lldb-mi` doesn't crash and handle the signal correctly.
> >>>>>
> >>>>> This functionality is really hard to replicate in lit_as is_.
> >>>>> Any ideas on how we could handle this case?
> >>>>
> >>>> How hard is it to import a new version of pexpect which supports python3 
> >>>> and
> >>>> stuff?
> >>>>
> >>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
> >>>> most linux
> >>>> systems either already have it installed, or have an easy way to do so. 
> >>>> So we
> >>>> may not even be able to get away with just using the system one and 
> >>>> skipping
> >>>> tests when it's not present.
> >>>>
> >>>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
> >>>> :D.
> >>>> Interactivity is one thing, and I'm very much in favour of keeping that 
> >>>> ability,
> >>>> but pexpect is not a prerequisite for that. For me, the main advantage of
> >>>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
> >>>> need
> >>>> that stuff. It doesn't have any command line editing capabilities or 
> >>>> similar. It's
> >>>> expecting to communicate with an IDE over a pipe, and that's it.
> >>>>
> >>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
> >>>> work on top
> >>>> of the standard python "subprocess" library. While we're doing that, we 
> >>>> might
> >>>> actually fix some of the issues that have been bugging everyone in the 
> >>>> lldb-mi
> >>>> tests. At least for me, the most annoying thing was that when lldb-mi 
> >>>> fails to
> >>>> produce the expected output, the test does not fail immediately, but 
> >>>> instead
> >>>> the implementation of self.expect("^whatever") waits until the timeout
> >>>> expires, optimistically hoping that it will find some output that match 
> >>>> the
> >>>> pattern.
> >>>>
> >
> > Pavel, I think yours is a really nice idea.
> > I'm no python expert, but I found out making the conversion is
> > relatively simple.
> > I propose a proof-of-concept API and implementation here:
> >
> > https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> >
> > Comments appreciated! Once we agree on how this should look like, I do
> > recommend to have a new lldbMITest base class and incrementally start
> > moving the tests to it.
> > Once we're done, we can delete the old class.
> >
> > Does this sound reasonable?
>
> What you are saying is that we would write the tests as Python tests in a way 
> similar to how lldbtest.expect() look in the dotest.py tests, banking on 
> synchronous mode taking care of all the synchronization? Are you thinking of 
> doing this for all the remaining tests or only the ones where a command input 
> depends on the result of a previous command?
>

I'm thinking to do this for all the remaining tests. Do you have any
concerns about this? (I'm aware your GSoC student introduced the `lit
lldb-mi` tests for a reason, I just don't know exactly what the reason
was).
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Adrian Prantl via lldb-dev


> On Feb 25, 2019, at 1:40 PM, Davide Italiano  wrote:
> 
> On Mon, Feb 25, 2019 at 1:35 PM Adrian Prantl  wrote:
>> 
>> 
>> 
>>> On Feb 25, 2019, at 1:15 PM, Davide Italiano  wrote:
>>> 
>>> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>>>> 
>>>> On 21/02/2019 19:48, Ted Woodward wrote:
>>>>> 
>>>>> 
>>>>>> -Original Message-
>>>>>> From: lldb-dev  On Behalf Of Pavel 
>>>>>> Labath
>>>>>> via lldb-dev
>>>>>> Sent: Thursday, February 21, 2019 8:35 AM
>>>>>> To: Davide Italiano 
>>>>>> Cc: LLDB 
>>>>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
>>>>>> 
>>>>>> On 21/02/2019 00:03, Davide Italiano wrote:
>>>>>>> I found out that there are tests that effectively require
>>>>>>> interactivity. Some of the lldb-mi ones are an example.
>>>>>>> A common use-case is that of sending SIGTERM in a loop to make sure
>>>>>>> `lldb-mi` doesn't crash and handle the signal correctly.
>>>>>>> 
>>>>>>> This functionality is really hard to replicate in lit_as is_.
>>>>>>> Any ideas on how we could handle this case?
>>>>>> 
>>>>>> How hard is it to import a new version of pexpect which supports python3 
>>>>>> and
>>>>>> stuff?
>>>>>> 
>>>>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that 
>>>>>> most linux
>>>>>> systems either already have it installed, or have an easy way to do so. 
>>>>>> So we
>>>>>> may not even be able to get away with just using the system one and 
>>>>>> skipping
>>>>>> tests when it's not present.
>>>>>> 
>>>>>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
>>>>>> :D.
>>>>>> Interactivity is one thing, and I'm very much in favour of keeping that 
>>>>>> ability,
>>>>>> but pexpect is not a prerequisite for that. For me, the main advantage of
>>>>>> pexpect is that it emulates a real terminal. However, lldb-mi does not 
>>>>>> need
>>>>>> that stuff. It doesn't have any command line editing capabilities or 
>>>>>> similar. It's
>>>>>> expecting to communicate with an IDE over a pipe, and that's it.
>>>>>> 
>>>>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to 
>>>>>> work on top
>>>>>> of the standard python "subprocess" library. While we're doing that, we 
>>>>>> might
>>>>>> actually fix some of the issues that have been bugging everyone in the 
>>>>>> lldb-mi
>>>>>> tests. At least for me, the most annoying thing was that when lldb-mi 
>>>>>> fails to
>>>>>> produce the expected output, the test does not fail immediately, but 
>>>>>> instead
>>>>>> the implementation of self.expect("^whatever") waits until the timeout
>>>>>> expires, optimistically hoping that it will find some output that match 
>>>>>> the
>>>>>> pattern.
>>>>>> 
>>> 
>>> Pavel, I think yours is a really nice idea.
>>> I'm no python expert, but I found out making the conversion is
>>> relatively simple.
>>> I propose a proof-of-concept API and implementation here:
>>> 
>>> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
>>> 
>>> Comments appreciated! Once we agree on how this should look like, I do
>>> recommend to have a new lldbMITest base class and incrementally start
>>> moving the tests to it.
>>> Once we're done, we can delete the old class.
>>> 
>>> Does this sound reasonable?
>> 
>> What you are saying is that we would write the tests as Python tests in a 
>> way similar to how lldbtest.expect() look in the dotest.py tests, banking on 
>> synchronous mode taking care of all the synchronization? Are you thinking of 
>> doing this for all the remaining tests or only the ones where a command 
>> input depends on the result of a previous command?
>> 
> 
> I'm thinking to do this for all the remaining tests. Do you have any
> concerns about this? (I'm aware your GSoC student introduced the `lit
> lldb-mi` tests for a reason, I just don't know exactly what the reason
> was).

I think the reason was that for tests that don't need synchronization and have 
a static command input, writing a lit/FileCheck test is straightforward and 
easy. At the time I thought we could just rewrite *all* lldb-mi tests as 
FileCheck tests. If we do need a python mechanism anyway, however, I don't 
actually think that having two ways of writing tests is better; I'd rather have 
all tests in one place. Thankfully there shouldn't be that many tests, so we 
should be able to just convert all of them over to whatever format we 
eventually settle on.
-- adrian


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Adrian Prantl via lldb-dev


> On Feb 25, 2019, at 1:15 PM, Davide Italiano  wrote:
> 
> On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>> 
>> On 21/02/2019 19:48, Ted Woodward wrote:
>>> 
>>> 
>>>> -Original Message-
>>>> From: lldb-dev  On Behalf Of Pavel Labath
>>>> via lldb-dev
>>>> Sent: Thursday, February 21, 2019 8:35 AM
>>>> To: Davide Italiano 
>>>> Cc: LLDB 
>>>> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
>>>> 
>>>> On 21/02/2019 00:03, Davide Italiano wrote:
>>>>> I found out that there are tests that effectively require
>>>>> interactivity. Some of the lldb-mi ones are an example.
>>>>> A common use-case is that of sending SIGTERM in a loop to make sure
>>>>> `lldb-mi` doesn't crash and handle the signal correctly.
>>>>> 
>>>>> This functionality is really hard to replicate in lit_as is_.
>>>>> Any ideas on how we could handle this case?
>>>> 
>>>> How hard is it to import a new version of pexpect which supports python3 
>>>> and
>>>> stuff?
>>>> 
>>>> I'm not sure how the situation is on darwin, but I'd expect (:P) that most 
>>>> linux
>>>> systems either already have it installed, or have an easy way to do so. So 
>>>> we
>>>> may not even be able to get away with just using the system one and 
>>>> skipping
>>>> tests when it's not present.
>>>> 
>>>> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
>>>> :D.
>>>> Interactivity is one thing, and I'm very much in favour of keeping that 
>>>> ability,
>>>> but pexpect is not a prerequisite for that. For me, the main advantage of
>>>> pexpect is that it emulates a real terminal. However, lldb-mi does not need
>>>> that stuff. It doesn't have any command line editing capabilities or 
>>>> similar. It's
>>>> expecting to communicate with an IDE over a pipe, and that's it.
>>>> 
>>>> Given that, it should be fairly easy to rewrite the lldb-mi tests to work 
>>>> on top
>>>> of the standard python "subprocess" library. While we're doing that, we 
>>>> might
>>>> actually fix some of the issues that have been bugging everyone in the 
>>>> lldb-mi
>>>> tests. At least for me, the most annoying thing was that when lldb-mi 
>>>> fails to
>>>> produce the expected output, the test does not fail immediately, but 
>>>> instead
>>>> the implementation of self.expect("^whatever") waits until the timeout
>>>> expires, optimistically hoping that it will find some output that match the
>>>> pattern.
>>>> 
> 
> Pavel, I think yours is a really nice idea.
> I'm no python expert, but I found out making the conversion is
> relatively simple.
> I propose a proof-of-concept API and implementation here:
> 
> https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68
> 
> Comments appreciated! Once we agree on how this should look like, I do
> recommend to have a new lldbMITest base class and incrementally start
> moving the tests to it.
> Once we're done, we can delete the old class.
> 
> Does this sound reasonable?

What you are saying is that we would write the tests as Python tests in a way 
similar to how lldbtest.expect() look in the dotest.py tests, banking on 
synchronous mode taking care of all the synchronization? Are you thinking of 
doing this for all the remaining tests or only the ones where a command input 
depends on the result of a previous command?

-- adrian

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-25 Thread Davide Italiano via lldb-dev
On Fri, Feb 22, 2019 at 6:32 AM Pavel Labath  wrote:
>
> On 21/02/2019 19:48, Ted Woodward wrote:
> >
> >
> >> -Original Message-
> >> From: lldb-dev  On Behalf Of Pavel Labath
> >> via lldb-dev
> >> Sent: Thursday, February 21, 2019 8:35 AM
> >> To: Davide Italiano 
> >> Cc: LLDB 
> >> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> >>
> >> On 21/02/2019 00:03, Davide Italiano wrote:
> >>> I found out that there are tests that effectively require
> >>> interactivity. Some of the lldb-mi ones are an example.
> >>> A common use-case is that of sending SIGTERM in a loop to make sure
> >>> `lldb-mi` doesn't crash and handle the signal correctly.
> >>>
> >>> This functionality is really hard to replicate in lit_as is_.
> >>> Any ideas on how we could handle this case?
> >>
> >> How hard is it to import a new version of pexpect which supports python3 
> >> and
> >> stuff?
> >>
> >> I'm not sure how the situation is on darwin, but I'd expect (:P) that most 
> >> linux
> >> systems either already have it installed, or have an easy way to do so. So 
> >> we
> >> may not even be able to get away with just using the system one and 
> >> skipping
> >> tests when it's not present.
> >>
> >> BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
> >> :D.
> >> Interactivity is one thing, and I'm very much in favour of keeping that 
> >> ability,
> >> but pexpect is not a prerequisite for that. For me, the main advantage of
> >> pexpect is that it emulates a real terminal. However, lldb-mi does not need
> >> that stuff. It doesn't have any command line editing capabilities or 
> >> similar. It's
> >> expecting to communicate with an IDE over a pipe, and that's it.
> >>
> >> Given that, it should be fairly easy to rewrite the lldb-mi tests to work 
> >> on top
> >> of the standard python "subprocess" library. While we're doing that, we 
> >> might
> >> actually fix some of the issues that have been bugging everyone in the 
> >> lldb-mi
> >> tests. At least for me, the most annoying thing was that when lldb-mi 
> >> fails to
> >> produce the expected output, the test does not fail immediately, but 
> >> instead
> >> the implementation of self.expect("^whatever") waits until the timeout
> >> expires, optimistically hoping that it will find some output that match the
> >> pattern.
> >>

Pavel, I think yours is a really nice idea.
I'm no python expert, but I found out making the conversion is
relatively simple.
I propose a proof-of-concept API and implementation here:

https://gist.github.com/dcci/94a4936a227d9c7627b91ae9575b7b68

Comments appreciated! Once we agree on how this should look like, I do
recommend to have a new lldbMITest base class and incrementally start
moving the tests to it.
Once we're done, we can delete the old class.

Does this sound reasonable?

--
Davide
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-22 Thread Pavel Labath via lldb-dev

On 21/02/2019 19:48, Ted Woodward wrote:




-Original Message-
From: lldb-dev  On Behalf Of Pavel Labath
via lldb-dev
Sent: Thursday, February 21, 2019 8:35 AM
To: Davide Italiano 
Cc: LLDB 
Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect

On 21/02/2019 00:03, Davide Italiano wrote:

I found out that there are tests that effectively require
interactivity. Some of the lldb-mi ones are an example.
A common use-case is that of sending SIGTERM in a loop to make sure
`lldb-mi` doesn't crash and handle the signal correctly.

This functionality is really hard to replicate in lit_as is_.
Any ideas on how we could handle this case?


How hard is it to import a new version of pexpect which supports python3 and
stuff?

I'm not sure how the situation is on darwin, but I'd expect (:P) that most linux
systems either already have it installed, or have an easy way to do so. So we
may not even be able to get away with just using the system one and skipping
tests when it's not present.

BTW, for lldb-mi I would actually argue that it should *not* use pexpect :D.
Interactivity is one thing, and I'm very much in favour of keeping that ability,
but pexpect is not a prerequisite for that. For me, the main advantage of
pexpect is that it emulates a real terminal. However, lldb-mi does not need
that stuff. It doesn't have any command line editing capabilities or similar. 
It's
expecting to communicate with an IDE over a pipe, and that's it.

Given that, it should be fairly easy to rewrite the lldb-mi tests to work on top
of the standard python "subprocess" library. While we're doing that, we might
actually fix some of the issues that have been bugging everyone in the lldb-mi
tests. At least for me, the most annoying thing was that when lldb-mi fails to
produce the expected output, the test does not fail immediately, but instead
the implementation of self.expect("^whatever") waits until the timeout
expires, optimistically hoping that it will find some output that match the
pattern.

If we change this to something like self.expect_reply("^whatever"), and make
the "expect_reply" function smart enough to know that lldb-mi's response
should come as a single line, and if the first line doesn't match, it should 
abort,
this problem would be fixed. While we're at it, we could also tune the failure
message so that it's more helpful than the current implementation. Plus, that
would solve the issue of not being able to run lldb-mi tests on windows.


This would be OK, I think, as long as "expect_reply" has the option to do a 
partial match,
or a regex match. Some of the lldb-mi tests only look for certain parts of the 
reply.


Yes, except from the difference in treating "messages" independently, 
the function could/should have the same matching capabilities as the 
current one.


I do see an opportunity to improve this to do some kind of 
structure-aware matching (expect_reply("library-loaded", 
target_name="foo.so", loaded_addr=0x47000)). IMO, that would make these 
tests superior even to the current lit tests, but I'm not an lldb-mi 
developer, so I'll probably stop short of doing that. :)





Also, until Python2 is declared dead and not supported at all by lldb, we 
should be able
to run this under 2 or 3.


Yes, of course.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Ted Woodward via lldb-dev


> -Original Message-
> From: lldb-dev  On Behalf Of Pavel Labath
> via lldb-dev
> Sent: Thursday, February 21, 2019 8:35 AM
> To: Davide Italiano 
> Cc: LLDB 
> Subject: [EXT] Re: [lldb-dev] [RFC]The future of pexpect
> 
> On 21/02/2019 00:03, Davide Italiano wrote:
> > I found out that there are tests that effectively require
> > interactivity. Some of the lldb-mi ones are an example.
> > A common use-case is that of sending SIGTERM in a loop to make sure
> > `lldb-mi` doesn't crash and handle the signal correctly.
> >
> > This functionality is really hard to replicate in lit_as is_.
> > Any ideas on how we could handle this case?
> 
> How hard is it to import a new version of pexpect which supports python3 and
> stuff?
> 
> I'm not sure how the situation is on darwin, but I'd expect (:P) that most 
> linux
> systems either already have it installed, or have an easy way to do so. So we
> may not even be able to get away with just using the system one and skipping
> tests when it's not present.
> 
> BTW, for lldb-mi I would actually argue that it should *not* use pexpect :D.
> Interactivity is one thing, and I'm very much in favour of keeping that 
> ability,
> but pexpect is not a prerequisite for that. For me, the main advantage of
> pexpect is that it emulates a real terminal. However, lldb-mi does not need
> that stuff. It doesn't have any command line editing capabilities or similar. 
> It's
> expecting to communicate with an IDE over a pipe, and that's it.
> 
> Given that, it should be fairly easy to rewrite the lldb-mi tests to work on 
> top
> of the standard python "subprocess" library. While we're doing that, we might
> actually fix some of the issues that have been bugging everyone in the lldb-mi
> tests. At least for me, the most annoying thing was that when lldb-mi fails to
> produce the expected output, the test does not fail immediately, but instead
> the implementation of self.expect("^whatever") waits until the timeout
> expires, optimistically hoping that it will find some output that match the
> pattern.
> 
> If we change this to something like self.expect_reply("^whatever"), and make
> the "expect_reply" function smart enough to know that lldb-mi's response
> should come as a single line, and if the first line doesn't match, it should 
> abort,
> this problem would be fixed. While we're at it, we could also tune the failure
> message so that it's more helpful than the current implementation. Plus, that
> would solve the issue of not being able to run lldb-mi tests on windows.

This would be OK, I think, as long as "expect_reply" has the option to do a 
partial match,
or a regex match. Some of the lldb-mi tests only look for certain parts of the 
reply.

Also, until Python2 is declared dead and not supported at all by lldb, we 
should be able
to run this under 2 or 3.

> Anyway, that's what I'd do. I was actually planning to look into that soon, 
> but
> then I roped myself into writing a yaml (de)serialization tool for minidumps, 
> so
> I have no idea when I will get back to that. I hope some of this is helpful
> nonetheless.
> 
> cheers,
> pavel
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Pavel Labath via lldb-dev

On 21/02/2019 15:35, Pavel Labath via lldb-dev wrote:
So we may not even be able to get away with just using the system one 
and skipping tests when it's not present.


s/may not/may/
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-21 Thread Pavel Labath via lldb-dev

On 21/02/2019 00:03, Davide Italiano wrote:

I found out that there are tests that effectively require
interactivity. Some of the lldb-mi ones are an example.
A common use-case is that of sending SIGTERM in a loop to make sure
`lldb-mi` doesn't crash and handle the signal correctly.

This functionality is really hard to replicate in lit_as is_.
Any ideas on how we could handle this case?


How hard is it to import a new version of pexpect which supports python3 
and stuff?


I'm not sure how the situation is on darwin, but I'd expect (:P) that 
most linux systems either already have it installed, or have an easy way 
to do so. So we may not even be able to get away with just using the 
system one and skipping tests when it's not present.


BTW, for lldb-mi I would actually argue that it should *not* use pexpect 
:D. Interactivity is one thing, and I'm very much in favour of keeping 
that ability, but pexpect is not a prerequisite for that. For me, the 
main advantage of pexpect is that it emulates a real terminal. However, 
lldb-mi does not need that stuff. It doesn't have any command line 
editing capabilities or similar. It's expecting to communicate with an 
IDE over a pipe, and that's it.


Given that, it should be fairly easy to rewrite the lldb-mi tests to 
work on top of the standard python "subprocess" library. While we're 
doing that, we might actually fix some of the issues that have been 
bugging everyone in the lldb-mi tests. At least for me, the most 
annoying thing was that when lldb-mi fails to produce the expected 
output, the test does not fail immediately, but instead the 
implementation of self.expect("^whatever") waits until the timeout 
expires, optimistically hoping that it will find some output that match 
the pattern.


If we change this to something like self.expect_reply("^whatever"), and 
make the "expect_reply" function smart enough to know that lldb-mi's 
response should come as a single line, and if the first line doesn't 
match, it should abort, this problem would be fixed. While we're at it, 
we could also tune the failure message so that it's more helpful than 
the current implementation. Plus, that would solve the issue of not 
being able to run lldb-mi tests on windows.


Anyway, that's what I'd do. I was actually planning to look into that 
soon, but then I roped myself into writing a yaml (de)serialization tool 
for minidumps, so I have no idea when I will get back to that. I hope 
some of this is helpful nonetheless.


cheers,
pavel
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-02-20 Thread Davide Italiano via lldb-dev
On Thu, Jan 31, 2019 at 11:40 AM Pavel Labath  wrote:
>
> On 31/01/2019 19:51, Zachary Turner wrote:
>  > FileCheck the ansi escape codes seems like one possibility.
>  >
>  > In general I think you don't actually need to test true interactivity,
>  > because the odds of there being a problem in the 2-3 lines of code that
>  > convert the keyboard press to something else in LLDB are very unlikely
>  > to be problematic, and the rest can be mocked.
>
>
> On 31/01/2019 20:02, Jim Ingham wrote:
> > All the traffic back and forth with the terminal happens in the 
> > IOHandlerEditLine.  We should be able to get our hands on the Debuggers 
> > IOHandler and feed characters directly to it, and read the results.  So we 
> > should be able to write this kind of test by driving the debugger to 
> > whatever state you need with SB API and then just run one command and get 
> > the output string directly from the IOHandler.  We should be able to then 
> > scan that output for color codes.  I don't think we need an external 
> > process inspection tool to do this sort of thing.
> >
>
>
> Libedit expect to work with a real terminal, so to test the code that
> interacts with libedit (and there's more than 3 lines of that), you'll
> need something that can create a pty, and read and write characters to
> it, regardless of whether you drive the test through FileCheck or SB API.
>
> "creating a pty, and reading and writing to it" is pretty much the
> definition of pexpect.
>
> I am not saying either of this approaches can't be made to work, but I
> am not sure who is going to do it. I fear that we are shooting ourselves
> in the foot banning pexpect and then pushing patches without tests
> because "it's hard".
>
> Just for fun, I tried to write a test to check the coloring of the
> prompt via pexpect. It was _literally_ three lines long:
>
> def test_colored_prompt_comes_out_right(self):
>  child = pexpect.spawn(lldbtest_config.lldbExec)
>  child.expect_exact("(lldb) \x1b[1G\x1b[2m(lldb) \x1b[22m\x1b[8G")
>
>
> BTW: I am not proposing we spend heroic efforts trying to port pexpect
> 2.4 to python3. But I would consider using a newer version of pexpect to
> write tests ***where it makes sense to do so***. At least until someone
> comes up with a better (and not vapourware) alternative...
>
> pl

I found out that there are tests that effectively require
interactivity. Some of the lldb-mi ones are an example.
A common use-case is that of sending SIGTERM in a loop to make sure
`lldb-mi` doesn't crash and handle the signal correctly.

This functionality is really hard to replicate in lit _as is_.
Any ideas on how we could handle this case?

Thanks,

--
Davide
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Pavel Labath via lldb-dev

On 31/01/2019 20:52, Zachary Turner wrote:
It's worth mentioning that pexpect is basically unusable on Windows, so 
there's still that.




Our interactive command line is basically unusable on windows, so there 
isn't anything to test anyway.


I expect (pun intended) that getting a working pexpect on windows will 
be much easier than getting a working interactive command line.

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Zachary Turner via lldb-dev
It's worth mentioning that pexpect is basically unusable on Windows, so
there's still that.

On Thu, Jan 31, 2019 at 11:40 AM Pavel Labath  wrote:

> On 31/01/2019 19:51, Zachary Turner wrote:
>  > FileCheck the ansi escape codes seems like one possibility.
>  >
>  > In general I think you don't actually need to test true interactivity,
>  > because the odds of there being a problem in the 2-3 lines of code that
>  > convert the keyboard press to something else in LLDB are very unlikely
>  > to be problematic, and the rest can be mocked.
>
>
> On 31/01/2019 20:02, Jim Ingham wrote:
> > All the traffic back and forth with the terminal happens in the
> IOHandlerEditLine.  We should be able to get our hands on the Debuggers
> IOHandler and feed characters directly to it, and read the results.  So we
> should be able to write this kind of test by driving the debugger to
> whatever state you need with SB API and then just run one command and get
> the output string directly from the IOHandler.  We should be able to then
> scan that output for color codes.  I don't think we need an external
> process inspection tool to do this sort of thing.
> >
>
>
> Libedit expect to work with a real terminal, so to test the code that
> interacts with libedit (and there's more than 3 lines of that), you'll
> need something that can create a pty, and read and write characters to
> it, regardless of whether you drive the test through FileCheck or SB API.
>
> "creating a pty, and reading and writing to it" is pretty much the
> definition of pexpect.
>
> I am not saying either of this approaches can't be made to work, but I
> am not sure who is going to do it. I fear that we are shooting ourselves
> in the foot banning pexpect and then pushing patches without tests
> because "it's hard".
>
> Just for fun, I tried to write a test to check the coloring of the
> prompt via pexpect. It was _literally_ three lines long:
>
> def test_colored_prompt_comes_out_right(self):
>  child = pexpect.spawn(lldbtest_config.lldbExec)
>  child.expect_exact("(lldb) \x1b[1G\x1b[2m(lldb) \x1b[22m\x1b[8G")
>
>
> BTW: I am not proposing we spend heroic efforts trying to port pexpect
> 2.4 to python3. But I would consider using a newer version of pexpect to
> write tests ***where it makes sense to do so***. At least until someone
> comes up with a better (and not vapourware) alternative...
>
> pl
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Pavel Labath via lldb-dev

On 31/01/2019 19:51, Zachary Turner wrote:
> FileCheck the ansi escape codes seems like one possibility.
>
> In general I think you don't actually need to test true interactivity,
> because the odds of there being a problem in the 2-3 lines of code that
> convert the keyboard press to something else in LLDB are very unlikely
> to be problematic, and the rest can be mocked.


On 31/01/2019 20:02, Jim Ingham wrote:

All the traffic back and forth with the terminal happens in the 
IOHandlerEditLine.  We should be able to get our hands on the Debuggers 
IOHandler and feed characters directly to it, and read the results.  So we 
should be able to write this kind of test by driving the debugger to whatever 
state you need with SB API and then just run one command and get the output 
string directly from the IOHandler.  We should be able to then scan that output 
for color codes.  I don't think we need an external process inspection tool to 
do this sort of thing.




Libedit expect to work with a real terminal, so to test the code that 
interacts with libedit (and there's more than 3 lines of that), you'll 
need something that can create a pty, and read and write characters to 
it, regardless of whether you drive the test through FileCheck or SB API.


"creating a pty, and reading and writing to it" is pretty much the 
definition of pexpect.


I am not saying either of this approaches can't be made to work, but I 
am not sure who is going to do it. I fear that we are shooting ourselves 
in the foot banning pexpect and then pushing patches without tests 
because "it's hard".


Just for fun, I tried to write a test to check the coloring of the 
prompt via pexpect. It was _literally_ three lines long:


def test_colored_prompt_comes_out_right(self):
child = pexpect.spawn(lldbtest_config.lldbExec)
child.expect_exact("(lldb) \x1b[1G\x1b[2m(lldb) \x1b[22m\x1b[8G")


BTW: I am not proposing we spend heroic efforts trying to port pexpect 
2.4 to python3. But I would consider using a newer version of pexpect to 
write tests ***where it makes sense to do so***. At least until someone 
comes up with a better (and not vapourware) alternative...


pl
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev


> On Jan 31, 2019, at 10:42 AM, Pavel Labath via lldb-dev 
>  wrote:
> 
> On 31/01/2019 19:26, Zachary Turner wrote:
>> Was the test failing specifically in the keyboard handler for up arrow, or 
>> was it failing in the command history searching code?  Because if it's the 
>> latter, then we could have a command which searches the command history.
> 
> The patch is r351313, if you want to look at it in detail. But, I don't think 
> this one example matters too much, since we will always have some code which 
> deals with the interactivity of the terminal. That will need to be tested 
> somehow.
> 
> Another example: we have a fairly complex piece of code that makes sure our 
> (lldb) prompt comes out in color. How do we write a test for that?

All the traffic back and forth with the terminal happens in the 
IOHandlerEditLine.  We should be able to get our hands on the Debuggers 
IOHandler and feed characters directly to it, and read the results.  So we 
should be able to write this kind of test by driving the debugger to whatever 
state you need with SB API and then just run one command and get the output 
string directly from the IOHandler.  We should be able to then scan that output 
for color codes.  I don't think we need an external process inspection tool to 
do this sort of thing.

Jim


> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Zachary Turner via lldb-dev
FileCheck the ansi escape codes seems like one possibility.

In general I think you don't actually need to test true interactivity,
because the odds of there being a problem in the 2-3 lines of code that
convert the keyboard press to something else in LLDB are very unlikely to
be problematic, and the rest can be mocked.



On Thu, Jan 31, 2019 at 10:42 AM Pavel Labath  wrote:

> On 31/01/2019 19:26, Zachary Turner wrote:
> > Was the test failing specifically in the keyboard handler for up arrow,
> > or was it failing in the command history searching code?  Because if
> > it's the latter, then we could have a command which searches the command
> > history.
> >
>
> The patch is r351313, if you want to look at it in detail. But, I don't
> think this one example matters too much, since we will always have some
> code which deals with the interactivity of the terminal. That will need
> to be tested somehow.
>
> Another example: we have a fairly complex piece of code that makes sure
> our (lldb) prompt comes out in color. How do we write a test for that?
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Pavel Labath via lldb-dev

On 31/01/2019 19:26, Zachary Turner wrote:
Was the test failing specifically in the keyboard handler for up arrow, 
or was it failing in the command history searching code?  Because if 
it's the latter, then we could have a command which searches the command 
history.




The patch is r351313, if you want to look at it in detail. But, I don't 
think this one example matters too much, since we will always have some 
code which deals with the interactivity of the terminal. That will need 
to be tested somehow.


Another example: we have a fairly complex piece of code that makes sure 
our (lldb) prompt comes out in color. How do we write a test for that?

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev
I don't think anybody uses these tests.  They are all time based benchmarks, 
and in the end there was just too much variability for them to be really 
useful.  We really need to do more work tracking performance, but I think a 
better approach is to focus on how much work we do (how many DIE's did you have 
to parse to do X, how many lookups did it take to compile an expression, or how 
many memory requests did a task take, things like that.  Those seem to me 
likely to be ore stable.

That said, I'm unclear why any of the benchmark tests would need pexpect to 
function.  The Driver used to have a lot more functionality in it that has 
since been moved into the SB API's.  Maybe there was good reason for doing 
performance testing through the driver but I can't think of any good reason for 
doing that nowadays. 

Jim

> On Jan 31, 2019, at 10:23 AM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
>> 
>> On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
>>> As you probably know (I didn’t), lldb embeds its own version of
>>> `pexpect-2.4`, which doesn’t support python3.
>>> This is the (relatively short) list of tests relying on pyexpect:
>>> 
>>> testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
>>> # 7 (EOF)
>>> testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
>>> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
>>> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
>>> testcases/lldbtest.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
>>> testcases/lldbpexpect.py:import pexpect
>>> testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
>>> testcases/darwin_log.py:import pexpect
>>> testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
>>> testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
>>> testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
>>>   import pexpect
>>> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>>>import pexpect
>>> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>>>import pexpect
>>> testcases/benchmarks/expression/TestExpressionCmd.py:import pexpect
>>> testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
>>> testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
>>> testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
>>> testcases/functionalities/command_regex/TestCommandRegex.py:
>>> import pexpect
>>> testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
>>>import pexpect
>>> testcases/functionalities/format/TestFormats.py:import pexpect
>>> 
>>> (I count 14, but there might be something else).
>>> 
>>> I audited all of them and from what I see they’re almost all testing the 
>>> driver.
>>> I had a chat with my coworkers and we agreed it's reasonable to
>>> replace them with lit tests (as they're just running commands).
>>> This would allow us to get rid of an external dependency, which
>>> happened to be cause of trouble in the past.
>>> 
>>> Are there any objections?
>>> 
>>> Thanks,
>>> 
>> 
>> I'm not a fan of pexpect, and if these tests can be converted to lit,
>> then I'm all for it. But I do have a question.
>> 
>> There is a class of tests that cannot be written in the current lit
>> framework, but they can with pexpect. A couple of weeks ago we had a
>> patch fixing a bug where pressing up arrow while searching through the
>> command history caused a crash. In the end a test for this was not
>> included because it was hard for a reason unrelated to pexpect, but
>> without pexpect (or something equivalent) writing a test for this would
>> be impossible.
>> 
> 
> I don't know about this, to be honest. Maybe lit should grow an
> interactive mode somehow to accomodate for this functionality?
> I'm not an expert in how it's implemented so that could be hard to achieve.
> FWIW, I haven't seen anything that really requires interactivity, but
> I have to admit I haven't looked really deeply.
> 
>> What's our story for testing interactive command-line functionalities?
>> The way I see it, if we don't use pexpect, we'll either have to use some
>> other tool which simulates a realistic terminal, or write our own. (We
>> already have one attempt for this in
>> unittests/Editline/EditlineTest.cpp, but this would need more work to be
>> fully functional.)
>> 
>> pl
>> 
>> 
>> PS: Does anyone actually use the benchmark tests? Can we just delete them?
> 
> I don't know. Maybe Jim knows. I personally 

Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Jim Ingham via lldb-dev
Even if it was the keyboard handler, lldb feeds characters to edit line through 
the IO Handler, so it should be possible to emulate the up arrow as well.  If 
there are reasons why that's not feasible, we should be able to make it work.  
This seems a tractable problem to me, and to me seems a better place to put 
effort than something like pexpect.

Jim


> On Jan 31, 2019, at 10:26 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Was the test failing specifically in the keyboard handler for up arrow, or 
> was it failing in the command history searching code?  Because if it's the 
> latter, then we could have a command which searches the command history.
> 
> On Thu, Jan 31, 2019 at 10:23 AM Davide Italiano via lldb-dev 
>  wrote:
> On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
> >
> > On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
> > > As you probably know (I didn’t), lldb embeds its own version of
> > > `pexpect-2.4`, which doesn’t support python3.
> > > This is the (relatively short) list of tests relying on pyexpect:
> > >
> > > testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
> > >  # 7 (EOF)
> > > testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/lldbtest.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/lldbpexpect.py:import pexpect
> > > testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
> > > testcases/darwin_log.py:import pexpect
> > > testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
> > > testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
> > > testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
> > >import pexpect
> > > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > > testcases/benchmarks/expression/TestExpressionCmd.py:import 
> > > pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import 
> > > pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import 
> > > pexpect
> > > testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
> > > testcases/functionalities/command_regex/TestCommandRegex.py:
> > > import pexpect
> > > testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
> > > import pexpect
> > > testcases/functionalities/format/TestFormats.py:import pexpect
> > >
> > > (I count 14, but there might be something else).
> > >
> > > I audited all of them and from what I see they’re almost all testing the 
> > > driver.
> > > I had a chat with my coworkers and we agreed it's reasonable to
> > > replace them with lit tests (as they're just running commands).
> > > This would allow us to get rid of an external dependency, which
> > > happened to be cause of trouble in the past.
> > >
> > > Are there any objections?
> > >
> > > Thanks,
> > >
> >
> > I'm not a fan of pexpect, and if these tests can be converted to lit,
> > then I'm all for it. But I do have a question.
> >
> > There is a class of tests that cannot be written in the current lit
> > framework, but they can with pexpect. A couple of weeks ago we had a
> > patch fixing a bug where pressing up arrow while searching through the
> > command history caused a crash. In the end a test for this was not
> > included because it was hard for a reason unrelated to pexpect, but
> > without pexpect (or something equivalent) writing a test for this would
> > be impossible.
> >
> 
> I don't know about this, to be honest. Maybe lit should grow an
> interactive mode somehow to accomodate for this functionality?
> I'm not an expert in how it's implemented so that could be hard to achieve.
> FWIW, I haven't seen anything that really requires interactivity, but
> I have to admit I haven't looked really deeply.
> 
> > What's our story for testing interactive command-line functionalities?
> > The way I see it, if we don't use pexpect, we'll either have to use some
> > other tool which simulates a realistic terminal, or write our own. (We
> > already have one attempt for this in
> > unittests/Editline/EditlineTest.cpp, but this would need more work to be
> > fully functional.)
> >
> > pl
> >
> >
> > PS: Does anyone actually use the benchmark tests? Can we just delete them?
> 
> I don't know. Maybe Jim knows. I personally don't use them.
> 

Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Zachary Turner via lldb-dev
Was the test failing specifically in the keyboard handler for up arrow, or
was it failing in the command history searching code?  Because if it's the
latter, then we could have a command which searches the command history.

On Thu, Jan 31, 2019 at 10:23 AM Davide Italiano via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
> >
> > On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
> > > As you probably know (I didn’t), lldb embeds its own version of
> > > `pexpect-2.4`, which doesn’t support python3.
> > > This is the (relatively short) list of tests relying on pyexpect:
> > >
> > > testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
> > >  # 7 (EOF)
> > > testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > > testcases/lldbtest.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > > testcases/lldbpexpect.py:import pexpect
> > > testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
> > > testcases/darwin_log.py:import pexpect
> > > testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
> > > testcases/benchmarks/stepping/TestSteppingSpeed.py:import
> pexpect
> > > testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
> > >import pexpect
> > >
> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > >
> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > > import pexpect
> > > testcases/benchmarks/expression/TestExpressionCmd.py:import
> pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import
> pexpect
> > > testcases/benchmarks/expression/TestRepeatedExprs.py:import
> pexpect
> > > testcases/benchmarks/startup/TestStartupDelays.py:import
> pexpect
> > > testcases/functionalities/command_regex/TestCommandRegex.py:
> > > import pexpect
> > >
> testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
> > > import pexpect
> > > testcases/functionalities/format/TestFormats.py:import pexpect
> > >
> > > (I count 14, but there might be something else).
> > >
> > > I audited all of them and from what I see they’re almost all testing
> the driver.
> > > I had a chat with my coworkers and we agreed it's reasonable to
> > > replace them with lit tests (as they're just running commands).
> > > This would allow us to get rid of an external dependency, which
> > > happened to be cause of trouble in the past.
> > >
> > > Are there any objections?
> > >
> > > Thanks,
> > >
> >
> > I'm not a fan of pexpect, and if these tests can be converted to lit,
> > then I'm all for it. But I do have a question.
> >
> > There is a class of tests that cannot be written in the current lit
> > framework, but they can with pexpect. A couple of weeks ago we had a
> > patch fixing a bug where pressing up arrow while searching through the
> > command history caused a crash. In the end a test for this was not
> > included because it was hard for a reason unrelated to pexpect, but
> > without pexpect (or something equivalent) writing a test for this would
> > be impossible.
> >
>
> I don't know about this, to be honest. Maybe lit should grow an
> interactive mode somehow to accomodate for this functionality?
> I'm not an expert in how it's implemented so that could be hard to achieve.
> FWIW, I haven't seen anything that really requires interactivity, but
> I have to admit I haven't looked really deeply.
>
> > What's our story for testing interactive command-line functionalities?
> > The way I see it, if we don't use pexpect, we'll either have to use some
> > other tool which simulates a realistic terminal, or write our own. (We
> > already have one attempt for this in
> > unittests/Editline/EditlineTest.cpp, but this would need more work to be
> > fully functional.)
> >
> > pl
> >
> >
> > PS: Does anyone actually use the benchmark tests? Can we just delete
> them?
>
> I don't know. Maybe Jim knows. I personally don't use them.
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Davide Italiano via lldb-dev
On Thu, Jan 31, 2019 at 10:09 AM Pavel Labath  wrote:
>
> On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:
> > As you probably know (I didn’t), lldb embeds its own version of
> > `pexpect-2.4`, which doesn’t support python3.
> > This is the (relatively short) list of tests relying on pyexpect:
> >
> > testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
> >  # 7 (EOF)
> > testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
> > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> > testcases/lldbtest.py:import pexpect
> > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> > testcases/lldbpexpect.py:import pexpect
> > testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
> > testcases/darwin_log.py:import pexpect
> > testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
> > testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
> > testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
> >import pexpect
> > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > import pexpect
> > testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
> > import pexpect
> > testcases/benchmarks/expression/TestExpressionCmd.py:import pexpect
> > testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
> > testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
> > testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
> > testcases/functionalities/command_regex/TestCommandRegex.py:
> > import pexpect
> > testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
> > import pexpect
> > testcases/functionalities/format/TestFormats.py:import pexpect
> >
> > (I count 14, but there might be something else).
> >
> > I audited all of them and from what I see they’re almost all testing the 
> > driver.
> > I had a chat with my coworkers and we agreed it's reasonable to
> > replace them with lit tests (as they're just running commands).
> > This would allow us to get rid of an external dependency, which
> > happened to be cause of trouble in the past.
> >
> > Are there any objections?
> >
> > Thanks,
> >
>
> I'm not a fan of pexpect, and if these tests can be converted to lit,
> then I'm all for it. But I do have a question.
>
> There is a class of tests that cannot be written in the current lit
> framework, but they can with pexpect. A couple of weeks ago we had a
> patch fixing a bug where pressing up arrow while searching through the
> command history caused a crash. In the end a test for this was not
> included because it was hard for a reason unrelated to pexpect, but
> without pexpect (or something equivalent) writing a test for this would
> be impossible.
>

I don't know about this, to be honest. Maybe lit should grow an
interactive mode somehow to accomodate for this functionality?
I'm not an expert in how it's implemented so that could be hard to achieve.
FWIW, I haven't seen anything that really requires interactivity, but
I have to admit I haven't looked really deeply.

> What's our story for testing interactive command-line functionalities?
> The way I see it, if we don't use pexpect, we'll either have to use some
> other tool which simulates a realistic terminal, or write our own. (We
> already have one attempt for this in
> unittests/Editline/EditlineTest.cpp, but this would need more work to be
> fully functional.)
>
> pl
>
>
> PS: Does anyone actually use the benchmark tests? Can we just delete them?

I don't know. Maybe Jim knows. I personally don't use them.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-31 Thread Pavel Labath via lldb-dev

On 31/01/2019 02:32, Davide Italiano via lldb-dev wrote:

As you probably know (I didn’t), lldb embeds its own version of
`pexpect-2.4`, which doesn’t support python3.
This is the (relatively short) list of tests relying on pyexpect:

testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
 # 7 (EOF)
testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
testcases/lldbtest.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/lldbpexpect.py:import pexpect
testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
testcases/darwin_log.py:import pexpect
testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
   import pexpect
testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
import pexpect
testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
import pexpect
testcases/benchmarks/expression/TestExpressionCmd.py:import pexpect
testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
testcases/functionalities/command_regex/TestCommandRegex.py:
import pexpect
testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
import pexpect
testcases/functionalities/format/TestFormats.py:import pexpect

(I count 14, but there might be something else).

I audited all of them and from what I see they’re almost all testing the driver.
I had a chat with my coworkers and we agreed it's reasonable to
replace them with lit tests (as they're just running commands).
This would allow us to get rid of an external dependency, which
happened to be cause of trouble in the past.

Are there any objections?

Thanks,



I'm not a fan of pexpect, and if these tests can be converted to lit, 
then I'm all for it. But I do have a question.


There is a class of tests that cannot be written in the current lit 
framework, but they can with pexpect. A couple of weeks ago we had a 
patch fixing a bug where pressing up arrow while searching through the 
command history caused a crash. In the end a test for this was not 
included because it was hard for a reason unrelated to pexpect, but 
without pexpect (or something equivalent) writing a test for this would 
be impossible.


What's our story for testing interactive command-line functionalities? 
The way I see it, if we don't use pexpect, we'll either have to use some 
other tool which simulates a realistic terminal, or write our own. (We 
already have one attempt for this in 
unittests/Editline/EditlineTest.cpp, but this would need more work to be 
fully functional.)


pl


PS: Does anyone actually use the benchmark tests? Can we just delete them?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-30 Thread Zachary Turner via lldb-dev
This would be great. All of these tests have always been disabled on
Windows so converting them to lit tests would increase test coverage there
as well
On Wed, Jan 30, 2019 at 6:00 PM Alex Langford via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> +1
>
> Thanks for bringing this up. I'd like to see this happen!
>
> - Alex
>
> On 1/30/19, 5:33 PM, "lldb-dev on behalf of Davide Italiano via lldb-dev"
> 
> wrote:
>
> As you probably know (I didn’t), lldb embeds its own version of
> `pexpect-2.4`, which doesn’t support python3.
> This is the (relatively short) list of tests relying on pyexpect:
>
> testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
> # 7 (EOF)
> testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
> testcases/lldbtest.py:import pexpect
> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> testcases/driver/batch_mode/TestBatchMode.py:import pexpect
> testcases/lldbpexpect.py:import pexpect
> testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
> testcases/darwin_log.py:import pexpect
> testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
> testcases/benchmarks/stepping/TestSteppingSpeed.py:import
> pexpect
> testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
>   import pexpect
>
> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>import pexpect
>
> testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
>import pexpect
> testcases/benchmarks/expression/TestExpressionCmd.py:import
> pexpect
> testcases/benchmarks/expression/TestRepeatedExprs.py:import
> pexpect
> testcases/benchmarks/expression/TestRepeatedExprs.py:import
> pexpect
> testcases/benchmarks/startup/TestStartupDelays.py:import
> pexpect
> testcases/functionalities/command_regex/TestCommandRegex.py:
> import pexpect
>
> testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
>import pexpect
> testcases/functionalities/format/TestFormats.py:import pexpect
>
> (I count 14, but there might be something else).
>
> I audited all of them and from what I see they’re almost all testing
> the driver.
> I had a chat with my coworkers and we agreed it's reasonable to
> replace them with lit tests (as they're just running commands).
> This would allow us to get rid of an external dependency, which
> happened to be cause of trouble in the past.
>
> Are there any objections?
>
> Thanks,
>
> --
> Davide
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_lldb-2Ddev=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=plcBe-Lvb3GcVnji0z26DNJmyn6uNsBq7AW-IQ7KAQQ=oo0_7ONGQhEkwtwF6DG8I6sVC2lUR-vlmka8pm4v1k0=zp1B92i8MPZxGtbFYUADj5J4GqHwpC1-g_x3fIN1hq0=
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [RFC]The future of pexpect

2019-01-30 Thread Alex Langford via lldb-dev
+1

Thanks for bringing this up. I'd like to see this happen!

- Alex

On 1/30/19, 5:33 PM, "lldb-dev on behalf of Davide Italiano via lldb-dev" 
 wrote:

As you probably know (I didn’t), lldb embeds its own version of
`pexpect-2.4`, which doesn’t support python3.
This is the (relatively short) list of tests relying on pyexpect:

testcases/tools/lldb-mi/syntax/TestMiSyntax.py:import pexpect
# 7 (EOF)
testcases/tools/lldb-mi/lldbmi_testcase.py:import pexpect
testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
testcases/tools/lldb-mi/signal/TestMiSignal.py:import pexpect
testcases/lldbtest.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/driver/batch_mode/TestBatchMode.py:import pexpect
testcases/lldbpexpect.py:import pexpect
testcases/terminal/TestSTTYBeforeAndAfter.py:import pexpect
testcases/darwin_log.py:import pexpect
testcases/macosx/nslog/TestDarwinNSLogOutput.py:import pexpect
testcases/benchmarks/stepping/TestSteppingSpeed.py:import pexpect
testcases/benchmarks/frame_variable/TestFrameVariableResponse.py:
  import pexpect
testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
   import pexpect
testcases/benchmarks/turnaround/TestCompileRunToBreakpointTurnaround.py:
   import pexpect
testcases/benchmarks/expression/TestExpressionCmd.py:import pexpect
testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
testcases/benchmarks/expression/TestRepeatedExprs.py:import pexpect
testcases/benchmarks/startup/TestStartupDelays.py:import pexpect
testcases/functionalities/command_regex/TestCommandRegex.py:
import pexpect

testcases/functionalities/single-quote-in-filename-to-lldb/TestSingleQuoteInFilename.py:
   import pexpect
testcases/functionalities/format/TestFormats.py:import pexpect

(I count 14, but there might be something else).

I audited all of them and from what I see they’re almost all testing the 
driver.
I had a chat with my coworkers and we agreed it's reasonable to
replace them with lit tests (as they're just running commands).
This would allow us to get rid of an external dependency, which
happened to be cause of trouble in the past.

Are there any objections?

Thanks,

--
Davide
___
lldb-dev mailing list
lldb-dev@lists.llvm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_lldb-2Ddev=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=plcBe-Lvb3GcVnji0z26DNJmyn6uNsBq7AW-IQ7KAQQ=oo0_7ONGQhEkwtwF6DG8I6sVC2lUR-vlmka8pm4v1k0=zp1B92i8MPZxGtbFYUADj5J4GqHwpC1-g_x3fIN1hq0=


___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev