Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[Shigio YAMAGUCHI]

Hi,
I have committed the patch (slightly modified version) to the repository.
Thank you.

Regards,
Shigio


2017-03-12 21:57 GMT+09:00 ishigane :

> Yes, this code is more safe than my patch on UNIX.
>
> > Can I change it as follows?
> >
> > -path = p.stdout.readline().rstrip()
> > +if sys.platform == 'win32' and sys.version_info >= (3,):
> > +path = io.TextIOWrapper(p.stdout,
> > encoding='latin1').readline().rstrip()
> > +else:
> > +path = p.stdout.readline().rstrip()
>
> I appreciate your kindness.
>
> Thanks,
> Seigo Ishigane
>



-- 
Shigio YAMAGUCHI 
PGP fingerprint:
26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB (Currently in use)
D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3 (Used until 2017/2)
___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[ishigane]

Yes, this code is more safe than my patch on UNIX.

> Can I change it as follows?
>
> -path = p.stdout.readline().rstrip()
> +if sys.platform == 'win32' and sys.version_info >= (3,):
> +path = io.TextIOWrapper(p.stdout,
> encoding='latin1').readline().rstrip()
> +else:
> +path = p.stdout.readline().rstrip()

I appreciate your kindness.

Thanks,
Seigo Ishigane

___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[Shigio YAMAGUCHI]

> I'm sorry, is indentation broken(mail text)?

No problem.

> -path = p.stdout.readline().rstrip()
> +if sys.version_info < (3,):
> +path = p.stdout.readline().rstrip()
> +else:
> +path = io.TextIOWrapper(p.stdout,
encoding='latin1').readline().rstrip()

Can I change it as follows?

-path = p.stdout.readline().rstrip()
+if sys.platform == 'win32' and sys.version_info >= (3,):
+path = io.TextIOWrapper(p.stdout,
encoding='latin1').readline().rstrip()
+else:
+path = p.stdout.readline().rstrip()

If 'encoding=...' is not needed for UNIX, I want to prevent the code
from running on UNIX.

Regards,
Shigio


2017-03-11 22:51 GMT+09:00 ishigane :

> I'm sorry, is indentation broken(mail text)?
>
> I bet, I failed to typing after paste to diff.
> Because I'm feeling nervous(it isn't good English?).
>
> No problem with attached diff.
>
> Seigo Ishigane
>



-- 
Shigio YAMAGUCHI 
PGP fingerprint:
26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB (Currently in use)
D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3 (Used until 2017/2)
___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[ishigane]

I'm sorry, is indentation broken(mail text)?

I bet, I failed to typing after paste to diff.
Because I'm feeling nervous(it isn't good English?).

No problem with attached diff.

Seigo Ishigane

___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[ishigane]

Shigio and Jason, Thank you for your reply.

I found that subprocess#Popen do a different behavior on Windows or not.
This issue occurs only Windows.

I wrote a test code, and tried on Windows10 and Debian 
stretch(currently, my unix-like machine is Debian only).


--
import subprocess
import sys

cmd = b'whoami'

p = subprocess.Popen(cmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)

if p.wait() == 0:
print(p.stdout.readline().rstrip())
--

result:
  Windows10 + python 3.6.0 -> TypeError: argument of type 'int' is not 
iterable

  Debian stretch + python 3.5.3 -> worker
  (worker = user name)

On Windows, subprocess#Popen rejects byte type argument. But on Linux, 
subprocess#Popen accepts byte type.

I rechecked "path"(in handle_requests function). Value is like that.

Windows: b'ctags'
Debian: b'/usr/bin/ctags'



I rewrite a patch for encoding='latin1' and returning str type value.
I checked the patch, Windows/Debian + python2/3 (4 patterns). Target is 
"test.c"(same as previos mail).

gtags outs function name into GTAGS at every pattern.

But I don't have macintosh(I cannot check MacOS X). So if this patch is 
rejected, that wouldn't be a problem.


--- pygments_parser.py.orig 2017-01-13 12:32:06 +0900
+++ pygments_parser.py  2017-03-11 20:48:42 +0900
@@ -238,7 +238,10 @@
 p = subprocess.Popen("gtags --config=ctagscom", shell=True,
 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 if p.wait() == 0:
-path = p.stdout.readline().rstrip()
+if sys.version_info < (3,):
+path = p.stdout.readline().rstrip()
+else:
+path = io.TextIOWrapper(p.stdout, 
encoding='latin1').readline().rstrip()

 return path

 def main():

Thank you for reading(Thank you for being patient with my bad English).

Seigo Ishigane
--- pygments_parser.py.orig 2017-01-07 23:22:40.0 +0900
+++ pygments_parser.py  2017-03-11 20:54:28.266765014 +0900
@@ -238,7 +238,10 @@
 p = subprocess.Popen("gtags --config=ctagscom", shell=True,
 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 if p.wait() == 0:
-path = p.stdout.readline().rstrip()
+if sys.version_info < (3,):
+path = p.stdout.readline().rstrip()
+else:
+path = io.TextIOWrapper(p.stdout, 
encoding='latin1').readline().rstrip()
 return path
 
 def main():
___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[Jason Hood]

On 9/03/2017 23:16, ishigane wrote:
> When run "gtags" with "--gtagslabel=pygments" option, GTAGS file don't have 
> any function names.
> CTagsParser (at "share/gtags/script/pygments_parser.py") failed to 
> initialize, gtags only use pygments.

I can confirm this occurs with Win7 and Python 3.4.

>   if p.wait() == 0:
>   path = p.stdout.readline().rstrip()
> +if sys.version_info >= (3, ):
> +path = path.decode()

I would suggest indenting the new lines, since there's no need to do it for the 
empty string.  Should probably also add "encoding='latin1'" to match what's 
used elsewhere, but it's doubtful the command would be anything other than 
ASCII, anyway.

-- 
Jason.

___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

Re: [python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[Shigio YAMAGUCHI]

Hi,
It was not reproduced in my environment.
GLOBAL-current
Python 3.6.0
MacOS 10.11.6

Since GLOBAL does not support Windows, I cannot say
it is a bug at this stage.
However, if it is a fix and it does not affect the UNIX
version, I will accept a patch.

Regards,
Shigio


2017-03-09 22:16 GMT+09:00 ishigane :

> Dear all,
>
> It might be a bug? I send a report.
> Target is "*.c"(C language file).
>
> When run "gtags" with "--gtagslabel=pygments" option, GTAGS file don't
> have any function names.
> CTagsParser (at "share/gtags/script/pygments_parser.py") failed to
> initialize, gtags only use pygments.
>
> This issue occurs only using python3. In the case of python 2.7.13, no
> problem.
>
> environments:
>Windows10 Pro x64
>global 6.5.6 (http://adoxa.altervista.org/global/)
>python 3.6.0 x64 + pygments 2.2.0
>exuberant ctags 5.8 (patched, http://hp.vector.co.jp/authors
> /VA025040/ctags/)
>
> Instructions:
> 1. I prepared "test.c". Content is like below.
> --
> #include 
>
> void test() {
> }
>
> int main(int argc, **char argv) {
> return 0;
> }
> ---
>
> 2. gtags --gtagslabel=pygments
> 3. gtags -d GTAGS
>
> expected result(same as python2):
>   __.COMPNAME __.COMPNAME
>   __.COMPRESS __.COMPRESS ddefine ttypedef
>   __.VERSION  __.VERSION 6
> main1 @n 6 int @n(int argc, **char argv) {
> test1 @n 3 void @n() {
>
> actual result(python3):
>   __.COMPNAME __.COMPNAME
>   __.COMPRESS __.COMPRESS ddefine ttypedef
>   __.VERSION  __.VERSION 6
>
> cause:
> "share/gtags/script/pygments_parser.py".
>
> At handle_requests(), CtagsParser failed to initialize.
> At CtagsParser.__init__, subprocess.Popen() failed.
>
> Because of "ctags_command"(argument of CtagsParser.__init__) is byte type.
>
> patch:
> --- pygments_parser.py.orig 2017-01-13 12:32:06 +0900
> +++ pygments_parser.py  2017-03-09 18:44:25 +0900
> @@ -239,6 +239,8 @@
>   stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>   if p.wait() == 0:
>   path = p.stdout.readline().rstrip()
> +if sys.version_info >= (3, ):
> +path = path.decode()
>   return path
>
>   def main():
>
> I don't know well about python(is checking/converting really ok?), it
> might be exists more better way.
>
> Regards,
> Seigo Ishigane
>
>
> ___
> Bug-global mailing list
> Bug-global@gnu.org
> https://lists.gnu.org/mailman/listinfo/bug-global
>
>


-- 
Shigio YAMAGUCHI 
PGP fingerprint:
26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB (Currently in use)
D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3 (Used until 2017/2)
___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global

[python3 only] when use "--gtagslabel=pygements", gtags don't output (C language) function names into GTAGS

[ishigane]

Dear all,

It might be a bug? I send a report.
Target is "*.c"(C language file).

When run "gtags" with "--gtagslabel=pygments" option, GTAGS file don't 
have any function names.
CTagsParser (at "share/gtags/script/pygments_parser.py") failed to 
initialize, gtags only use pygments.


This issue occurs only using python3. In the case of python 2.7.13, no 
problem.


environments:
   Windows10 Pro x64
   global 6.5.6 (http://adoxa.altervista.org/global/)
   python 3.6.0 x64 + pygments 2.2.0
   exuberant ctags 5.8 (patched, 
http://hp.vector.co.jp/authors/VA025040/ctags/)


Instructions:
1. I prepared "test.c". Content is like below.
--
#include 

void test() {
}

int main(int argc, **char argv) {
return 0;
}
---

2. gtags --gtagslabel=pygments
3. gtags -d GTAGS

expected result(same as python2):
  __.COMPNAME __.COMPNAME
  __.COMPRESS __.COMPRESS ddefine ttypedef
  __.VERSION  __.VERSION 6
main1 @n 6 int @n(int argc, **char argv) {
test1 @n 3 void @n() {

actual result(python3):
  __.COMPNAME __.COMPNAME
  __.COMPRESS __.COMPRESS ddefine ttypedef
  __.VERSION  __.VERSION 6

cause:
"share/gtags/script/pygments_parser.py".

At handle_requests(), CtagsParser failed to initialize.
At CtagsParser.__init__, subprocess.Popen() failed.

Because of "ctags_command"(argument of CtagsParser.__init__) is byte type.

patch:
--- pygments_parser.py.orig 2017-01-13 12:32:06 +0900
+++ pygments_parser.py  2017-03-09 18:44:25 +0900
@@ -239,6 +239,8 @@
  stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  if p.wait() == 0:
  path = p.stdout.readline().rstrip()
+if sys.version_info >= (3, ):
+path = path.decode()
  return path

  def main():

I don't know well about python(is checking/converting really ok?), it 
might be exists more better way.


Regards,
Seigo Ishigane

--- pygments_parser.py.orig 2017-01-13 12:32:06 +0900
+++ pygments_parser.py  2017-03-09 18:44:25 +0900
@@ -239,6 +239,8 @@
 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 if p.wait() == 0:
 path = p.stdout.readline().rstrip()
+if sys.version_info >= (3, ):
+path = path.decode()
 return path
 
 def main():
___
Bug-global mailing list
Bug-global@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-global