Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Dec 1, 2014, at 1:25 AM, Ethan Furman  wrote:
> 
> One argument that keeps coming up is transferability of knowledge:  knowing 
> git and/or GitHub, as many seem to, it
> therefore becomes easier to commit to the Python ecosystem.
> 
> What about the transferability of Python knowledge?  Because I know Python, I 
> can customize hg;  because I know Python I
> can customize Roundup.
> 
> I do not choose tools simply because they are written in Python -- I choose 
> them because, being written in Python, I can
> work on them if I need to:  I can enhance them, I can fix them, I can learn 
> from them.
> 
> --
> ~Ethan~
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Git uses the idea of small individual commands that do small things,
so you can write your own commands that work on text streams to
extend git and you can even write those in Python.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Ethan Furman]

On 11/30/2014 03:41 PM, Terry Reedy wrote:
> On 11/30/2014 2:27 PM, Ethan Furman wrote:
>> On 11/30/2014 11:15 AM, Guido van Rossum wrote:
>>> On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote:
 On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:
>
> All the use cases seem to be about adding some kind of getattr hook
> to modules. They all seem to involve modifying the CPython C code
> anyway. So why not tackle that problem head-on and modify 
> module_getattro()
> to look for a global named __getattr__ and if it exists, call that instead
> of raising AttributeError?

 Not sure if anyone thought of it. :) Seems like a reasonable solution to 
 me.
 Be curious to know what the benchmark suite said the impact was.
>>>
>>> Why would there be any impact? The __getattr__ hook would be similar to the
>>> one on classes -- it's only invoked at the point where otherwise 
>>> AttributeError
>>> would be raised.
>>
>> I think the bigger question is how do we support it back on 2.7?
> 
> I do not understand this question. We don't add new features to 2.7 and this 
> definitely is one.

My understanding of one of the use-cases was being able to issue warnings about 
deprecated attributes, which would be
most effective if a backport could be written for current versions.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ethan Furman]

One argument that keeps coming up is transferability of knowledge:  knowing git 
and/or GitHub, as many seem to, it
therefore becomes easier to commit to the Python ecosystem.

What about the transferability of Python knowledge?  Because I know Python, I 
can customize hg;  because I know Python I
can customize Roundup.

I do not choose tools simply because they are written in Python -- I choose 
them because, being written in Python, I can
work on them if I need to:  I can enhance them, I can fix them, I can learn 
from them.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Joining the PEP Editors team

[Chris Angelico]

In response to Guido's call for volunteers, I'm offering myself as a
PEP editor. Who is in charge of this kind of thing? Who manages public
key lists etc?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Stephen J. Turnbull]

Ben Finney writes:

 > Whether that is a *problem* is a matter of debate, but the fact that
 > Git's common workflow commonly discards information that some consider
 > valuable, is a simple fact.

It *was* a simple fact in git 0.99.  Since the advent of reflogs
(years ago), it is simply false.

Worse, common hg workflows (as developed at the same time as your
impression of "Git's common workflow") *also* discard information that
some (namely, me) consider valuable, because it *never gets recorded*.
Exploring the git reflog has taught me things about my workflow and
skills (and lack thereof ;-) that I'd never learn from an hg or bzr
branch.

In the end, the logs look very similar.  I can only conclude that the
rebasing that I do in git is implicit in the process of composing a
"coherent changeset" in hg or bzr.  I also typically have a bunch of
"loose commits" lying around in Mercurial queues or bzr stashes, which
amount to rebases when reapplied.

 > If Ethan chooses to make that a factor in his decisions about Git,
 > the facts are on his side.

Hardly.  All he needs to do is pretend git is hg, and avoid rebase.

The only thing that should matter is the annoyance of learning a new
tool.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

On Sun, Nov 30, 2014 at 5:42 PM, Nathaniel Smith  wrote:

> On Mon, Dec 1, 2014 at 1:27 AM, Guido van Rossum  wrote:
> > Nathaniel, did you look at Brett's LazyLoader? It overcomes the subclass
> > issue by using a module loader that makes all modules instances of a
> > (trivial) Module subclass. I'm sure this approach can be backported as
> far
> > as you need to go.
>
> The problem is that by the time your package's code starts running,
> it's too late to install such a loader. Brett's strategy works well
> for lazy-loading submodules (e.g., making it so 'import numpy' makes
> 'numpy.testing' available, but without the speed hit of importing it
> immediately), but it doesn't help if you want to actually hook
> attribute access on your top-level package (e.g., making 'numpy.foo'
> trigger a DeprecationWarning -- we have a lot of stupid exported
> constants that we can never get rid of because our rules say that we
> have to deprecate things before removing them).
>
> Or maybe you're suggesting that we define a trivial heap-allocated
> subclass of PyModule_Type and use that everywhere, as a
> quick-and-dirty way to enable __class__ assignment? (E.g., return it
> from PyModule_New?) I considered this before but hesitated b/c it
> could potentially break backwards compatibility -- e.g. if code A
> creates a PyModule_Type object directly without going through
> PyModule_New, and then code B checks whether the resulting object is a
> module by doing isinstance(x, type(sys)), this will break. (type(sys)
> is a pretty common way to get a handle to ModuleType -- in fact both
> types.py and importlib use it.) So in my mind I sorta lumped it in
> with my Option 2, "minor compatibility break". OTOH maybe anyone who
> creates a module object without going through PyModule_New deserves
> whatever they get.
>

Couldn't you install a package loader using some install-time hook?

Anyway, I still think that the issues with heap types can be overcome. Hm,
didn't you bring that up before here? Was the conclusion that it's
impossible?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 07:43 PM, Donald Stufft wrote:

The idea that unless Python as a project always picks something written in 
Python over something written in something else we’re somehow signaling to the 
world that if you want to write X kind of tool you should do it in some other 
language is laughable. It completely ignores anything at all about the tools 
except what language they are written in.


My point is not "Python should always" pick python over any other 
language. My point is: a tool written in python and with developer 
already involved with the community is something to takes in account. 
And is we simplify the technical debate (and shut it down as requested) 
by saying both are equivalent. This is part of the other consideration 
to take in account.


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 8:19 PM, Pierre-Yves David wrote:


Mercurial have robust Windows support for a long time. This support is
native (not using cygwin) and handle properly all kind of strange corner
case. We have large scale ecosystem (http://unity3d.com/) using
Mercurial on windows.

We also have full featured GUI client http://tortoisehg.bitbucket.org/.


Tortoisehg comes with Hg Workbench, which I use for everything I can. 
The only thing I do not use it for is file (versus repository/changeset) 
commands, like file annotation. I do that with right click in Windows 
Explorer.  In my current physical state, having to do everything at a 
Windows command prompt would be more onerous.


There exists a TortoiseGit, but it lacks the unifying Workbench GUI. 
There exists at least one 3rd party workbench that purports to work with 
both hg and git. I have not looked at it yet.



It is actively developed by people who stay in touch with the Mercurial
upstream so new feature tend to land in the GUI really fast.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 10:08 PM, Pierre-Yves David 
>  wrote:
> 
> 
> 
> On 11/29/2014 06:01 PM, Donald Stufft wrote:
>> The reason the PEP primarily focuses on the popularity of the the tool is
>> because as you mentioned, issues like poor documentation, bad support for a
>> particular platform, a particular workflow not being very good can be
>> solved by working with the tool authors to solve that particular problem. I 
>> wouldn’t
>> consider those issues in a vacuum to be a good reason to migrate away from 
>> that
>> tool.
> 
> As I understand it[1] my current employer (Facebook) picked Mercurial over 
> git because these very reason end up being important. And this analysis have 
> been validated by another big company[2].
> 
> Git implementation is very tied to the linux world and this slowed down its 
> gain of a windows support. This is not something that will change by 
> discussing with author: "btw can you rewrite your tool with different techno 
> and concept?".
> Mercurial is extensible in Python, very extensible. In my previous job one of 
> our client switched to Mercurial and was able to get an extension adding 
> commands to match it exact previous code-review workflow in a couple of 
> hundred line of python. (you could have the same for python).
> Mercurial developer are already connected to the Python community. They are 
> invited to language submit, regular pycon speaker and attendees etc.
> 
> All these things contradict "bah any project would not make a difference"
> 
>> However there’s very little that CPython can do to get more people using
>> Mercurial, and presumably the authors of Mercurial are already doing  what 
>> they
>> can to get people to use them.
> 
> Mercurial is an open source project. We have no communication department, no 
> communication budget actually. Over the year, more and more contributors are 
> actually paid to do so, but they usually focus on "making employer's users" 
> happy. Something that rarely involves getting more outside-world users. We 
> mostly rely on the network effect to gain more users, (yes, we are losing to 
> git on this, but still growing anyway). Part of this network effect is having 
> big project like CPython using Mercurial. It also imply that CPython dev are 
> willing to look at how the tools works and that the Project tries to take 
> advantage of the tools strength. This would turn the situation into mutual 
> benefits. You are happy with Mercurial and we are happy with Python.
> 
> However moving to git and github send a very different signal: If you want to 
> be a successful command line tool, use C and bash. If you want to be a 
> successful website use ruby on rails.

I want to adress this point specifically, because it’s not particularly related 
to the hg vs git discussion that Guido has asked people to stop doing.

The idea that unless Python as a project always picks something written in 
Python over something written in something else we’re somehow signaling to the 
world that if you want to write X kind of tool you should do it in some other 
language is laughable. It completely ignores anything at all about the tools 
except what language they are written in.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/29/2014 06:01 PM, Donald Stufft wrote:

The reason the PEP primarily focuses on the popularity of the the tool is
because as you mentioned, issues like poor documentation, bad support for a
particular platform, a particular workflow not being very good can be
solved by working with the tool authors to solve that particular problem. I 
wouldn’t
consider those issues in a vacuum to be a good reason to migrate away from that
tool.


As I understand it[1] my current employer (Facebook) picked Mercurial 
over git because these very reason end up being important. And this 
analysis have been validated by another big company[2].


Git implementation is very tied to the linux world and this slowed down 
its gain of a windows support. This is not something that will change by 
discussing with author: "btw can you rewrite your tool with different 
techno and concept?".
Mercurial is extensible in Python, very extensible. In my previous job 
one of our client switched to Mercurial and was able to get an extension 
adding commands to match it exact previous code-review workflow in a 
couple of hundred line of python. (you could have the same for python).
Mercurial developer are already connected to the Python community. They 
are invited to language submit, regular pycon speaker and attendees etc.


All these things contradict "bah any project would not make a difference"


However there’s very little that CPython can do to get more people using
Mercurial, and presumably the authors of Mercurial are already doing  what they
can to get people to use them.


Mercurial is an open source project. We have no communication 
department, no communication budget actually. Over the year, more and 
more contributors are actually paid to do so, but they usually focus on 
"making employer's users" happy. Something that rarely involves getting 
more outside-world users. We mostly rely on the network effect to gain 
more users, (yes, we are losing to git on this, but still growing 
anyway). Part of this network effect is having big project like CPython 
using Mercurial. It also imply that CPython dev are willing to look at 
how the tools works and that the Project tries to take advantage of the 
tools strength. This would turn the situation into mutual benefits. You 
are happy with Mercurial and we are happy with Python.


However moving to git and github send a very different signal: If you 
want to be a successful command line tool, use C and bash. If you want 
to be a successful website use ruby on rails.


--
Pierre-Yves David

[1] This is a personal statement is not to be linked to the opinion of 
my employer PR department.

[2] That I'm not naming in fear of there PR assasins.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 4:45 PM, Donald Stufft wrote:

I think you are stimulating more heated discussion than is necessary by 
trying to do too much, both in terms of physical changes and in terms of 
opinion persuasion.


I am reminded of the integer division change.  The initial discussion 
was initially over-heated partly because the actual change to Python was 
coupled, quite unnecessarily, with a metaphysical opinion as the the 
ontological status of natural numbers versus integers -- an opinion that 
many, including me, considered to be wrong.  Once the metaphysical 
question was dropped, discussion went more smoothly.



Here’s another idea for an experiment that might be more generally useful.


I think a true experiment with one repository is easily justified.  The 
PEP repository is an obvious choice because a) the main editor is in 
favor, b) many but not all core committers interact with it, and c) it 
is not tied to the tracker.  The easier and more controversial option 
would be to move it completely to GitHub.  I expect part of the result 
would be pull requests from committers who would otherwise commit 
directly to hg.  The harder and, I think, more useful (generalizable) 
option would be to set up a mirror (or keep the hg version as a mirror 
;-) and experiment with coordinating the two mirrors.


Such an experiment should not preclude other experiments.  If Brett 
wants to similarly experiment with devinabox on another site, let him. 
If the horrible-to-Nick prospect of possibly moving CPython to GitHub, 
if nothing else is done, provokes Nick to improve the workflow 
otherwise, great.


If the mirror experiment is successful, the devguide might be the next 
experiment.  It does not have any one maintainer, and *is* tied to the 
tracker.  But herein lies the problem with the devguide.  There are 22 
issues, down just 1 from about a year ago.  All but 2 are more than a 
year old.  Many (most?) have patches, but enough consensus for anyone to 
push is hard.  As with other doc issues, there is no 'test' for when a 
non-trivial patch is 'good enough' and hence, in my opinion, too much 
bikeshedding and pursuit of the perfect.



As we've said there are two sides to the coin here, non-comitters and
comitters, a lot of the benefit of moving to Github is focused at
non-comitters although there are benefits for comitters themselves.


For maintaining Idle, I do not see the benefit.  Downloading patches 
from the tracker to my dev directory is trivial. I then apply to the 
current 3.x maintenance version, possibly with some hand editing, revise 
(always, that I can remember), and test.  Once satisfied, I backport to 2.7.



What if we focused an experiment on the benefits to non-comitters?


Users benefit by more patches being applied.  How do non-commiters 
benefit, really, by making it easier for them to submit patches that sit 
for years?  Ignore that.  Guido says that working with PEPs on GitHub 
would benefit him as a committer.



It's possible to maintain a git mirror of a Mercurial repository, in fact
we already have that at github.com/python/cpython
. What if we permit people
to make PRs against that repository, and then take those PRs and paste them
into roundup? Sort of like the "Remote hg repo" field. Then we can
create some integration that would post a comment to the ticket whenever

> that PR is updated

(sort of like the notification that happens when a new patch is uploaded).
The cannonical repository would still be hg.python.org
 and in order to actually
commit the PR commiters would need to turn the PR into a patch
(trivially easy, just add .diff or .patch to the PR URL).


This would be the focus of an experiment with the devguide, even if we 
have to generate some somewhat artificial pull requests for testing.  I 
really hope you try to make the above work.


The 3rd stage would be to expand on the above for doc patches. This is 
one area where we would get small ready-to-commit patches -- that do not 
need to be reported to the tracker.  Would it be possible to automate 
the following? Turn a doc PR into a patch, apply the patch to all 3 
branches (perhaps guided by the PR message), and generate a report, 
along with, currently, a 2.7 and 3.4 PR.  (I am thinking about how to do 
some of this doc patches with hg on windows.)


[snip premature discussion of moving cpython 'wholehog' to github.]

Summary research plan: 3 experiments, each depending of the preceding.
1. Link 2 repositories, one with pull requests
2. Link the PRs with the tracker
3. Make PRs work better with our multibranch, 2-head monster.
Report after each experiment (ending with 'success' or 'give-up').

--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

On Sun, Nov 30, 2014 at 8:09 PM, Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

>
>
> On 11/30/2014 08:44 AM, Brett Cannon wrote:
>
>> For me personally, if I knew a simple patch integrated cleanly and
>> passed on at least one buildbot -- when it wasn't a platform-specific
>> fix -- then I could easily push a "Commit" button and be done with it
>> (although this assumes single branch committing; doing this across
>> branches makes all of this difficult unless we finally resolve our
>> Misc/NEWS conflict issues so that in some instances it can be
>> automated). Instead I have to wait until I have a clone I can push from,
>> download a patch, apply it, run the unit tests myself, do the commit,
>> and then repeat a subset of that to whatever branches make sense. It's a
>> lot of work for which some things could be automated.
>>
>
> The Misc/NEWS issue could be easily solved. Mercurial allow to specify a
> custom merge tool for specific file. And I already succesfully wrote
> dedicated merge tools for file with similar issue.
>
>
You might take a look at the hubflow/gitflow branching workflow diagrams?

https://datasift.github.io/gitflow/IntroducingGitFlow.html (GitFlow ->
Hubflow)

feature/name, develop, hotfix/name, releases/v0.0.1, master


I've already discussed that with various people Larry, Nick, etc. And what
> is needed now is someone actually doing the work.
>
> Once you have such tool, you can have automatic pull request
> merge/rebasing through a web ui. -However-, You can only do that if you
> actually own the said interface. Because propriétary plateform are not
> going to let your run arbitrary code on their machine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 09:09 AM, Donald Stufft wrote:

Even converting between two FLOSS tools is an amazing amount of work.  Look at
>what Eric Raymond did with reposurgeon to convert from Bazaar to git.

I fail to see how this is a reasonable argument to make at all since, as you
mentioned, converting between two FLOSS tools can be an amazing amount of work.
Realistically the amount of work is going to be predicated on whether or not
there is a tool that already handles the conversion for you. Assuming of course
that the data is available to you at all.


The statement that switch a whole infra from one tool to another is 
"cheap" and predictable sounds extremely naive to me.



As a particularly relevant example, switching from Mercurial to Git is as easy
as installing hg-git, creating a bookmark for master that tracks default, and
then pushing to a git repository.


Migrating the DVCS content is usually easy. The hard part is then to 
find all the script, tools, doc that rely on the previous tools to 
upgrade them to the new one, sometimes trying to find feature parity. 
This is rarely cheap and never "predictable"


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/29/2014 05:15 PM, Chris Angelico wrote:

On Sun, Nov 30, 2014 at 11:37 AM, Donald Stufft  wrote:

I also don’t know how to do this. When I’m doing multiple things for CPython
my “branching” strategy is essentially using hg diff to create a patch file
with my “branch” name (``hg diff > my-branch.patch``), then revert all of my
changes (``hg revert —all —no-backup``), then either work on a new “branch”
or switch to an old “branch” by applying the corresponding patch
(``patch -p1 < other-branch.patch``).


IMO, this is missing out on part of the benefit of a DVCS. When your
patches are always done purely on the basis of files, and have to be
managed separately, everything will be manual; and your edits won't
(normally) contain commit messages, authorship headers, date/time
stamps, and all the other things that a commit will normally have.
Using GitHub automatically makes all that available; when someone
forks the project and adds a commit, that commit will exist and have
its full identity, metadata, etc, and if/when it gets merged into
trunk, all that will be carried through automatically.


There is no reason to make this `hg diff` dance (but ignorance).

- You can make plain commit with your changes.
- You can export commit content using `hg export`
- You can change you patch content will all kind of tools (amend, 
rebase, etc)
- You can have multiple branches without any issue to handle concurrent 
workflow.


We (Mercurial developer) will be again sprinting at Pycon 2015. We can 
probably arrange some workflow discussion//training there.


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 08:30 AM, Donald Stufft wrote:



On Nov 30, 2014, at 2:08 AM, Larry Hastings mailto:la...@hastings.org>> wrote:


On 11/29/2014 04:37 PM, Donald Stufft wrote:

On Nov 29, 2014, at 7:15 PM, Alex Gaynor  wrote:

Despite being a regular hg
user for years, I have no idea how to create a local-only branch, or a branch
which is pushed to a remote (to use the git term).

I also don’t know how to do this.


Instead of collectively scratching your heads, could one of you guys
do the research and figure out whether or not hg supports this
workflow?  One of the following two things must be true:

 1. hg supports this workflow (or a reasonable fascimile), which may
lessen the need for this PEP.
 2. hg doesn't support this workflow, which may strengthen the need
for this PEP.

Saying "I've been using hg for years and I don't know whether it
supports this IMPORTANT THING" is not a particularly compelling argument.



Comments like this make me feel like I didn’t explain myself very well
in the
PEP.

While I do think that supporting this workflow via an extension is worse
than
supporting it in core,


I was about to point that bookmark are no longer an extension for more 
than 3 years, but someone was faster than me.


But I would like to reply a bit more on this extension FUD. There is 
three kind of Mercurial feature:


1) The one in Mercurial core
2) The one in official extension
3) the one in third party extension

(1) and (2) Have the -same- level of support and stability. All of them 
are part of the same repo and same test suite, offer the same backward 
compatibility promise and are installed as part of the same package.


Official extensions are usually not in core for various reasons:

1) It is exotic feature (eg: communication bugzilla extension)
2) Nobody did the work to move it into core (unfortunatly) (eg: progress 
bar extension) (similar situation: how long did it took to get pip 
shipped with python?)
3) We think it is a important feature but we are not happy with the 
current UX and would like somethign better before moving it into core 
(eg: histedit)


Not using official extensions because they are extensions is similar to 
not use the python standard library because "They are modules, not part 
of the core language"



this isn’t why this PEP exists. The current
workflow for
contributing is painful, for the repositories this is talking about if I’m a
non-comitter I have to email patches to a particular closed mailing list and
then sit around and wait.


Your workflow issue does not seems to be particularly tied to the tool 
(Mercurial) itself but more to (1) very partial usage of the tool (2) 
current project workflow. It would not be hard to (1) use the tools in a 
less painful way (2) rethink project workflow to reduce the pain. This 
seems orthogonal to changing the tool.



The Pull Request based workflow is *massively* better than
uploading/emailing
patches around. So the question then becomes, if we're going to move to a PR
based workflow how do we do it? PEP 474 says that we should install some
software that works with Mercurial and supports Pull Requests. Another
thread
suggested that we should just use to bitbucket which also supports
Mercurial
and use that.


(note: Yes Manual upload of patch is terrible and tracking email by 
patch is terrible too. But github is fairly bad at doing pull request. 
It emphasis the final result instead of the actual pull request content. 
Encouraging massive patches and making the history muddier.)



This PEP says that git and Github have the popular vote, which is extremely
compelling as a feature because:


Sure, Git//Github is more popular than Mercurial nowaday. But can I 
point to the irony of Python using the "more popular" argument? If most 
of people here would have seen this definitive argument, they would been 
currently writing Java code instead of discussing Python on this list.


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 08:44 AM, Brett Cannon wrote:

For me personally, if I knew a simple patch integrated cleanly and
passed on at least one buildbot -- when it wasn't a platform-specific
fix -- then I could easily push a "Commit" button and be done with it
(although this assumes single branch committing; doing this across
branches makes all of this difficult unless we finally resolve our
Misc/NEWS conflict issues so that in some instances it can be
automated). Instead I have to wait until I have a clone I can push from,
download a patch, apply it, run the unit tests myself, do the commit,
and then repeat a subset of that to whatever branches make sense. It's a
lot of work for which some things could be automated.


The Misc/NEWS issue could be easily solved. Mercurial allow to specify a 
custom merge tool for specific file. And I already succesfully wrote 
dedicated merge tools for file with similar issue.


I've already discussed that with various people Larry, Nick, etc. And 
what is needed now is someone actually doing the work.


Once you have such tool, you can have automatic pull request 
merge/rebasing through a web ui. -However-, You can only do that if you 
actually own the said interface. Because propriétary plateform are not 
going to let your run arbitrary code on their machine.


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Eric Snow]

On Sun, Nov 30, 2014 at 6:44 PM, Donald Stufft  wrote:
> Yea this is essentially what I meant. We already have “unofficial” mirrors
> for PEPs and CPython itself on Github that are updated a few times a day.
> It wouldn’t be very difficult I think to make them official mirrors and
> update them immediately after a push.
>
> Then just some integration with Roundup would enable people to send PRs
> on Github.

Exactly.  In my mind this eliminates almost all the controversy in
this discussion.  The question of switching hosting provider or DVCS
becomes something to discuss separately (and probably just drop since
at that point it doesn't improve much over the status quo).  Of
course, it does not address the more pressing concern of how to get
contributions landed more quickly/steadily. :(  However, that is
largely what Nick addresses in PEP 462. :)

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Eric Snow]

On Sun, Nov 30, 2014 at 6:40 PM, Donald Stufft  wrote:
> I’m not sure if it got lost in the discussion or if it was purposely left
> out. However I did come up with another idea, where we enable people to make
> PRs against these repositories with PR integration within roundup. Using the
> fact that it’s trivial to turn a PR into a patch core contributors (and the
> “single source of truth”) for the repositories can remain Mercurial with
> core contributors needing to download a .patch file from Github instead of a
> .patch from from Roundup. This could allow non-committers to use git if they
> want, including PRs but without moving things around.

Hah.  I just had a similar idea.

>
> The obvious cost is that since the committer side of things is still using
> the existing tooling there’s no “Merge button” or the other committer
> benefits of Github, it would strictly be enabling people who aren’t
> committing directly to the repository to use git and Github.

This is not an added cost.  It's just the status quo and something
that can be addressed separately.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 8:41 PM, Eric Snow  wrote:
> 
> On Sun, Nov 30, 2014 at 6:25 PM, Donald Stufft  wrote:
>> The technical benefits mostly come from Github generally being a higher
>> quality product than it’s competitors, both FOSS and not.
> 
> Here's a solution to allow contribution via PR while not requiring
> anything to switch VCS or hosting:
> 
> 1. Set up mirrors of a desired repo on any hosting providers we choose.
> 2. Set up a webhook for PRs that automatically creates/re-uses a
> tracker ticket with the diff from the PR.
> 
> The workflow does not change for the committer, but it gets easier to
> contribute.
> 
> I did something like this for juju (https://github.com/juju/juju) when
> we switched to github, weren't satisfied with their code review tool,
> and switched to something else.  We have a web hook that automatically
> creates a review request for new PRs and updates the review request
> when the PR gets updated.
> 
> -eric

Yea this is essentially what I meant. We already have “unofficial” mirrors
for PEPs and CPython itself on Github that are updated a few times a day.
It wouldn’t be very difficult I think to make them official mirrors and
update them immediately after a push.

Then just some integration with Roundup would enable people to send PRs
on Github.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Mon, Dec 1, 2014 at 1:27 AM, Guido van Rossum  wrote:
> Nathaniel, did you look at Brett's LazyLoader? It overcomes the subclass
> issue by using a module loader that makes all modules instances of a
> (trivial) Module subclass. I'm sure this approach can be backported as far
> as you need to go.

The problem is that by the time your package's code starts running,
it's too late to install such a loader. Brett's strategy works well
for lazy-loading submodules (e.g., making it so 'import numpy' makes
'numpy.testing' available, but without the speed hit of importing it
immediately), but it doesn't help if you want to actually hook
attribute access on your top-level package (e.g., making 'numpy.foo'
trigger a DeprecationWarning -- we have a lot of stupid exported
constants that we can never get rid of because our rules say that we
have to deprecate things before removing them).

Or maybe you're suggesting that we define a trivial heap-allocated
subclass of PyModule_Type and use that everywhere, as a
quick-and-dirty way to enable __class__ assignment? (E.g., return it
from PyModule_New?) I considered this before but hesitated b/c it
could potentially break backwards compatibility -- e.g. if code A
creates a PyModule_Type object directly without going through
PyModule_New, and then code B checks whether the resulting object is a
module by doing isinstance(x, type(sys)), this will break. (type(sys)
is a pretty common way to get a handle to ModuleType -- in fact both
types.py and importlib use it.) So in my mind I sorta lumped it in
with my Option 2, "minor compatibility break". OTOH maybe anyone who
creates a module object without going through PyModule_New deserves
whatever they get.

-n

> On Sun, Nov 30, 2014 at 5:02 PM, Nathaniel Smith  wrote:
>>
>> On Mon, Dec 1, 2014 at 12:59 AM, Nathaniel Smith  wrote:
>> > On Sun, Nov 30, 2014 at 10:14 PM, Mark Shannon  wrote:
>> >> Hi,
>> >>
>> >> This discussion has been going on for a while, but no one has
>> >> questioned the
>> >> basic premise. Does this needs any change to the language or
>> >> interpreter?
>> >>
>> >> I believe it does not. I'm modified your original metamodule.py to not
>> >> use
>> >> ctypes and support reloading:
>> >> https://gist.github.com/markshannon/1868e7e6115d70ce6e76
>> >
>> > Interesting approach!
>> >
>> > As written, your code will blow up on any python < 3.4, because when
>> > old_module gets deallocated it'll wipe the module dict clean. And I
>> > guess even on >=3.4, this might still happen if old_module somehow
>> > manages to get itself into a reference loop before getting
>> > deallocated. (Hopefully not, but what a nightmare to debug if it did.)
>> > However, both of these issues can be fixed by stashing a reference to
>> > old_module somewhere in new_module.
>> >
>> > The __class__ = ModuleType trick is super-clever but makes me
>> > irrationally uncomfortable. I know that this is documented as a valid
>> > method of fooling isinstance(), but I didn't know that until your
>> > yesterday, and the idea of objects where type(foo) is not
>> > foo.__class__ strikes me as somewhat blasphemous. Maybe this is all
>> > fine though.
>> >
>> > The pseudo-module objects generated this way will still won't pass
>> > PyModule_Check, so in theory this could produce behavioural
>> > differences. I can't name any specific places where this will break
>> > things, though. From a quick skim of the CPython source, a few
>> > observations: It means the PyModule_* API functions won't work (e.g.
>> > PyModule_GetDict); maybe these aren't used enough to matter. It looks
>> > like the __reduce__ methods on "method objects"
>> > (Objects/methodobject.c) have a special check for ->m_self being a
>> > module object, and won't pickle correctly if ->m_self ends up pointing
>> > to one of these pseudo-modules. I have no idea how one ends up with a
>> > method whose ->m_self points to a module object, though -- maybe it
>> > never actually happens. PyImport_Cleanup treats module objects
>> > differently from non-module objects during shutdown.
>>
>> Actually, there is one showstopper here -- in the first version where
>> reload() uses isinstance() is actually 3.4. Before that you need a
>> real module subtype for reload to work. But this is in principle
>> workaroundable by using subclassing + ctypes on old versions of python
>> and the __class__ = hack on new versions.
>>
>> > I guess it also has the mild limitation that it doesn't work with
>> > extension modules, but eh. Mostly I'd be nervous about the two points
>> > above.
>> >
>> > -n
>> >
>> > --
>> > Nathaniel J. Smith
>> > Postdoctoral researcher - Informatics - University of Edinburgh
>> > http://vorpus.org
>>
>>
>>
>> --
>> Nathaniel J. Smith
>> Postdoctoral researcher - Informatics - University of Edinburgh
>> http://vorpus.org
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubs

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Eric Snow]

On Sun, Nov 30, 2014 at 6:25 PM, Donald Stufft  wrote:
>The technical benefits mostly come from Github generally being a higher
> quality product than it’s competitors, both FOSS and not.

Here's a solution to allow contribution via PR while not requiring
anything to switch VCS or hosting:

1. Set up mirrors of a desired repo on any hosting providers we choose.
2. Set up a webhook for PRs that automatically creates/re-uses a
tracker ticket with the diff from the PR.

The workflow does not change for the committer, but it gets easier to
contribute.

I did something like this for juju (https://github.com/juju/juju) when
we switched to github, weren't satisfied with their code review tool,
and switched to something else.  We have a web hook that automatically
creates a review request for new PRs and updates the review request
when the PR gets updated.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 8:24 PM, Guido van Rossum  wrote:
> 
> Can we please stop the hg-vs-git discussion? We've established earlier that 
> the capabilities of the DVCS itself (hg or git) are not a differentiator, and 
> further he-said-she-said isn't going to change anybody's opinion.
> 
> What's left is preferences of core developers, possibly capabilities of the 
> popular websites (though BitBucket vs. GitHub seems to be a wash as well), 
> and preferences of contributors who aren't core developers (using popularity 
> as a proxy). It seems the preferences of the core developers are mixed, while 
> the preferences of non-core contributors are pretty clear, so we have a 
> problem weighing these two appropriately.
> 
> Also, let's not get distracted by the needs of the CPython repo, issue 
> tracker, and code review tool. Arguments about core developers vs. 
> contributors for CPython shouldn't affect the current discussion.
> 
> Next, two of the three repos mentioned in Donald's PEP 481 are owned by Brett 
> Cannon, according to the Contact column listed on hg.python.org 
> . I propose to let Brett choose whether to keep these 
> on hg.python.org , move to BitBucket, or move to 
> GitHub. @Brett, what say you? (Apart from "I'm tired of the whole thread." :-)
> 
> The third one is the peps repo, which has python-dev@python.org 
>  as Contact. It turns out that Nick is by far 
> the largest contributor (he committed 215 of the most recent 1000 changes) so 
> I'll let him choose.
> 
> Finally, I'd like to get a few more volunteers for the PEP editors list, 
> preferably non-core devs: the core devs are already spread too thinly, and I 
> really shouldn't be the one who picks new PEP numbers and checks that PEPs 
> are well-formed according to the rules of PEP 1. A PEP editor shouldn't have 
> to pass judgment on the contents of a PEP (though they may choose to correct 
> spelling and grammar). Knowledge of Mercurial is a plus. :-)
> 

I’m not sure if it got lost in the discussion or if it was purposely left out. 
However I did come up with another idea, where we enable people to make PRs 
against these repositories with PR integration within roundup. Using the fact 
that it’s trivial to turn a PR into a patch core contributors (and the “single 
source of truth”) for the repositories can remain Mercurial with core 
contributors needing to download a .patch file from Github instead of a .patch 
from from Roundup. This could allow non-committers to use git if they want, 
including PRs but without moving things around.

The obvious cost is that since the committer side of things is still using the 
existing tooling there’s no “Merge button” or the other committer benefits of 
Github, it would strictly be enabling people who aren’t committing directly to 
the repository to use git and Github.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Sun, Nov 30, 2014 at 8:54 PM, Guido van Rossum  wrote:
> On Sun, Nov 30, 2014 at 11:29 AM, Nathaniel Smith  wrote:
>>
>> On Sun, Nov 30, 2014 at 2:54 AM, Guido van Rossum 
>> wrote:
>> > All the use cases seem to be about adding some kind of getattr hook to
>> > modules. They all seem to involve modifying the CPython C code anyway.
>> > So
>> > why not tackle that problem head-on and modify module_getattro() to look
>> > for
>> > a global named __getattr__ and if it exists, call that instead of
>> > raising
>> > AttributeError?
>>
>> You need to allow overriding __dir__ as well for tab-completion, and
>> some people wanted to use the properties API instead of raw
>> __getattr__, etc. Maybe someone will want __getattribute__ semantics,
>> I dunno.
>
> Hm... I agree about __dir__ but the other things feel too speculative.
>
>> So since we're *so close* to being able to just use the
>> subclassing machinery, it seemed cleaner to try and get that working
>> instead of reimplementing bits of it piecewise.
>
> That would really be option 1, right? It's the one that looks cleanest from
> the user's POV (or at least from the POV of a developer who wants to build a
> framework using this feature -- for a simple one-off use case, __getattr__
> sounds pretty attractive). I think that if we really want option 1, the
> issue of PyModuleType not being a heap type can be dealt with.

Options 1-4 all have the effect of making it fairly simple to slot an
arbitrary user-defined module subclass into sys.modules. Option 1 is
the cleanest API though :-).

>>
>> That said, __getattr__ + __dir__ would be enough for my immediate use
>> cases.
>
>
>  Perhaps it would be a good exercise to try and write the "lazy submodule
> import"(*) use case three ways: (a) using only CPython 3.4; (b) using
> __class__ assignment; (c) using customizable __getattr__ and __dir__. I
> think we can learn a lot about the alternatives from this exercise. I
> presume there's already a version of (a) floating around, but if it's been
> used in practice at all, it's probably too gnarly to serve as a useful
> comparison (though its essence may be extracted to serve as such).

(b) and (c) are very straightforward and trivial. Probably I could do
a better job of faking dir()'s default behaviour on modules, but
basically:

# __class__ assignment__ #

import sys, types, importlib

class MyModule(types.ModuleType):
def __getattr__(self, name):
if name in _lazy_submodules:
# implicitly assigns submodule to self.__dict__[name]
return importlib.import_module(name, package=self.__package__)

def __dir__(self):
entries = set(self.__dict__)
entries.update(__lazy_submodules__)
return sorted(entries)

sys.modules[__name__].__class__ = MyModule
_lazy_submodules = {"foo", "bar"}

# customizable __getattr__ and __dir__ #

import importlib

def __getattr__(name):
if name in _lazy_submodules:
# implicitly assigns submodule to globals()[name]
return importlib.import_module(name, package=self.__package__)

def __dir__():
entries = set(globals())
entries.update(__lazy_submodules__)
return sorted(entries)

_lazy_submodules = {"foo", "bar"}

> FWIW I believe all proposals here have a big limitation: the module *itself*
> cannot benefit much from all these shenanigans, because references to
> globals from within the module's own code are just dictionary accesses, and
> we don't want to change that.

I think that's fine -- IMHO the main uses cases here are about
controlling the public API. And a module that really wants to can
always import itself if it wants to pull more shenanigans :-) (i.e.,
foo/__init__.py can do "import foo; foo.blahblah" instead of just
"blahblah".)

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Markus Unterwaditzer]

On 1 December 2014 01:17:02 CET, Ben Finney  wrote:
>Donald Stufft  writes:
>
>> I have never heard of git losing history.
>
>In my experience talking with Git users about this problem, that
>depends
>on a very narrow definition of “losing history”.
>
>Git encourages re-writing, and thereby losing prior versions of, the
>history of a branch. The commit information remains, but the history of
>how they link together is lost. That is a loss of information, which is
>not the case in the absence of such history re-writing.

"Losing data" is generally used in the sense that either the application or the 
filesystem accidentally deletes or overwrites data without the user's consent 
or knownledge. Rewriting and deleting (not "losing") history in git is 
explicitly done by the user, encouraged or not.

-- Markus
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

Nathaniel, did you look at Brett's LazyLoader? It overcomes the subclass
issue by using a module loader that makes all modules instances of a
(trivial) Module subclass. I'm sure this approach can be backported as far
as you need to go.

On Sun, Nov 30, 2014 at 5:02 PM, Nathaniel Smith  wrote:

> On Mon, Dec 1, 2014 at 12:59 AM, Nathaniel Smith  wrote:
> > On Sun, Nov 30, 2014 at 10:14 PM, Mark Shannon  wrote:
> >> Hi,
> >>
> >> This discussion has been going on for a while, but no one has
> questioned the
> >> basic premise. Does this needs any change to the language or
> interpreter?
> >>
> >> I believe it does not. I'm modified your original metamodule.py to not
> use
> >> ctypes and support reloading:
> >> https://gist.github.com/markshannon/1868e7e6115d70ce6e76
> >
> > Interesting approach!
> >
> > As written, your code will blow up on any python < 3.4, because when
> > old_module gets deallocated it'll wipe the module dict clean. And I
> > guess even on >=3.4, this might still happen if old_module somehow
> > manages to get itself into a reference loop before getting
> > deallocated. (Hopefully not, but what a nightmare to debug if it did.)
> > However, both of these issues can be fixed by stashing a reference to
> > old_module somewhere in new_module.
> >
> > The __class__ = ModuleType trick is super-clever but makes me
> > irrationally uncomfortable. I know that this is documented as a valid
> > method of fooling isinstance(), but I didn't know that until your
> > yesterday, and the idea of objects where type(foo) is not
> > foo.__class__ strikes me as somewhat blasphemous. Maybe this is all
> > fine though.
> >
> > The pseudo-module objects generated this way will still won't pass
> > PyModule_Check, so in theory this could produce behavioural
> > differences. I can't name any specific places where this will break
> > things, though. From a quick skim of the CPython source, a few
> > observations: It means the PyModule_* API functions won't work (e.g.
> > PyModule_GetDict); maybe these aren't used enough to matter. It looks
> > like the __reduce__ methods on "method objects"
> > (Objects/methodobject.c) have a special check for ->m_self being a
> > module object, and won't pickle correctly if ->m_self ends up pointing
> > to one of these pseudo-modules. I have no idea how one ends up with a
> > method whose ->m_self points to a module object, though -- maybe it
> > never actually happens. PyImport_Cleanup treats module objects
> > differently from non-module objects during shutdown.
>
> Actually, there is one showstopper here -- in the first version where
> reload() uses isinstance() is actually 3.4. Before that you need a
> real module subtype for reload to work. But this is in principle
> workaroundable by using subclassing + ctypes on old versions of python
> and the __class__ = hack on new versions.
>
> > I guess it also has the mild limitation that it doesn't work with
> > extension modules, but eh. Mostly I'd be nervous about the two points
> > above.
> >
> > -n
> >
> > --
> > Nathaniel J. Smith
> > Postdoctoral researcher - Informatics - University of Edinburgh
> > http://vorpus.org
>
>
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Guido van Rossum]

Can we please stop the hg-vs-git discussion? We've established earlier that
the capabilities of the DVCS itself (hg or git) are not a differentiator,
and further he-said-she-said isn't going to change anybody's opinion.

What's left is preferences of core developers, possibly capabilities of the
popular websites (though BitBucket vs. GitHub seems to be a wash as well),
and preferences of contributors who aren't core developers (using
popularity as a proxy). It seems the preferences of the core developers are
mixed, while the preferences of non-core contributors are pretty clear, so
we have a problem weighing these two appropriately.

Also, let's not get distracted by the needs of the CPython repo, issue
tracker, and code review tool. Arguments about core developers vs.
contributors for CPython shouldn't affect the current discussion.

Next, two of the three repos mentioned in Donald's PEP 481 are owned by
Brett Cannon, according to the Contact column listed on hg.python.org. I
propose to let Brett choose whether to keep these on hg.python.org, move to
BitBucket, or move to GitHub. @Brett, what say you? (Apart from "I'm tired
of the whole thread." :-)

The third one is the peps repo, which has python-dev@python.org as Contact.
It turns out that Nick is by far the largest contributor (he committed 215
of the most recent 1000 changes) so I'll let him choose.

Finally, I'd like to get a few more volunteers for the PEP editors list,
preferably non-core devs: the core devs are already spread too thinly, and
I really shouldn't be the one who picks new PEP numbers and checks that
PEPs are well-formed according to the rules of PEP 1. A PEP editor
shouldn't have to pass judgment on the contents of a PEP (though they may
choose to correct spelling and grammar). Knowledge of Mercurial is a plus.
:-)

On Sun, Nov 30, 2014 at 4:50 PM, Donald Stufft  wrote:

>
> > On Nov 30, 2014, at 7:43 PM, Ben Finney 
> wrote:
> >
> > Donald Stufft  writes:
> >
> >> It’s not lost, [… a long, presumably-accurate discourse of the many
> >> conditions that must be met before …] you can restore it.
> >
> > This isn't the place to discuss the details of Git's internals, I think.
> > I'm merely pointing out that:
> >
> >> The important thing to realize is that a “branch” isn’t anything
> >> special in git.
> >
> > Because of that, Ethan's impression – that Git's default behaviour
> > encourages losing history (by re-writing the history of commits to be
> > other than what they were) is true, and “Git never loses history” simply
> > isn't true.
> >
> > Whether that is a *problem* is a matter of debate, but the fact that
> > Git's common workflow commonly discards information that some consider
> > valuable, is a simple fact.
> >
> > If Ethan chooses to make that a factor in his decisions about Git, the
> > facts are on his side.
>
> Except it’s not true at all.
>
> That data is all still there if you want it to exist and it’s not a real
> differentiator between Mercurial and git because Mercurial has the ability
> to do the same thing. Never mind the fact that “lose” your history makes it
> sound accidental instead of on purpose. It’s like saying that ``rm
> foo.txt``
> will “lose” the data in foo.txt. So either it was a misunderstanding in
> which case I wanted to point out that those operations don’t magically lose
> information or it’s a purposely FUDish statement in which case I want to
> point out that the statement is inaccurate.
>
> The only thing that is true is that git users are more likely to use the
> ability to rewrite history than Mercurial users are, but you’ll typically
> find that people generally don’t do this on public branches, only on
> private
> branches. Which again doesn’t make much sense in this context since
> generally
> currently the way people are using Mercurial with CPython you’re using
> patches to transfer the changes from the contributor to the committer so
> you’re
> “losing” that history regardless.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 8:11 PM, Pierre-Yves David 
>  wrote:
> 
> 
> 
> On 11/30/2014 08:45 AM, Donald Stufft wrote:
>> I don’t make branches in Mercurial because
>> i’m afraid I’m going to push a permanent branch to hg.python.org
>>  and screw
>> something up.
> 
> There is no need to be afraid there, Mercurial is not going to let you push 
> new head/branch unless you explicitly use `hg push --force`.
> 
> I you are really paranoid about this, you can configure your Mercurial to 
> make all new commit as secret (no pushable) and explicitly make commit ready 
> to push as such. This can be achieved by adding
> 
> [phases]
> new-commit=secret
> 
> See `hg help phases` for details.

Yea Benjamin mentioned that the hg.python.org repositories have commit hooks to 
prevent that from happening too. To be clear the fact I don’t really know 
Mercurial very well isn’t what I think is a compelling argument for not using 
Mercurial. It’s mostly a tangent to this PEP which is primarily focused on the 
“network effects” of using a more popular tool. The technical benefits mostly 
come from Github generally being a higher quality product than it’s 
competitors, both FOSS and not.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 04:31 AM, Paul Moore wrote:

On 29 November 2014 at 23:27, Donald Stufft  wrote:

>In previous years there was concern about how well supported git was on Windows
>in comparison to Mercurial. However git has grown to support Windows as a first
>class citizen. In addition to that, for Windows users who are not well aquanted
>with the Windows command line there are GUI options as well.


Mercurial have robust Windows support for a long time. This support is 
native (not using cygwin) and handle properly all kind of strange corner 
case. We have large scale ecosystem (http://unity3d.com/) using 
Mercurial on windows.


We also have full featured GUI client http://tortoisehg.bitbucket.org/. 
It is actively developed by people who stay in touch with the Mercurial 
upstream so new feature tend to land in the GUI really fast.


--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 08:45 AM, Donald Stufft wrote:

I don’t make branches in Mercurial because
i’m afraid I’m going to push a permanent branch to hg.python.org
 and screw
something up.


There is no need to be afraid there, Mercurial is not going to let you 
push new head/branch unless you explicitly use `hg push --force`.


I you are really paranoid about this, you can configure your Mercurial 
to make all new commit as secret (no pushable) and explicitly make 
commit ready to push as such. This can be achieved by adding


[phases]
new-commit=secret

See `hg help phases` for details.

--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Pierre-Yves David]

On 11/30/2014 04:31 AM, Paul Moore wrote:

On 29 November 2014 at 23:27, Donald Stufft  wrote:

In previous years there was concern about how well supported git was on Windows
in comparison to Mercurial. However git has grown to support Windows as a first
class citizen. In addition to that, for Windows users who are not well aquanted
with the Windows command line there are GUI options as well.


I have little opinion on the PEP as a whole, but is the above
statement true? From the git website, version 2.2.0 is current, and
yet the downloadable Windows version is still 1.9.4. That's a fairly
significant version lag for a "first class citizen".

I like git, and it has a number of windows-specific extensions that
are really useful (more than Mercurial, AFAIK), but I wouldn't say
that the core product supported Windows on an equal footing to Linux.


I'm curious about These useful extension. Can you elaborate?

--
Pierre-Yves David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Mon, Dec 1, 2014 at 12:59 AM, Nathaniel Smith  wrote:
> On Sun, Nov 30, 2014 at 10:14 PM, Mark Shannon  wrote:
>> Hi,
>>
>> This discussion has been going on for a while, but no one has questioned the
>> basic premise. Does this needs any change to the language or interpreter?
>>
>> I believe it does not. I'm modified your original metamodule.py to not use
>> ctypes and support reloading:
>> https://gist.github.com/markshannon/1868e7e6115d70ce6e76
>
> Interesting approach!
>
> As written, your code will blow up on any python < 3.4, because when
> old_module gets deallocated it'll wipe the module dict clean. And I
> guess even on >=3.4, this might still happen if old_module somehow
> manages to get itself into a reference loop before getting
> deallocated. (Hopefully not, but what a nightmare to debug if it did.)
> However, both of these issues can be fixed by stashing a reference to
> old_module somewhere in new_module.
>
> The __class__ = ModuleType trick is super-clever but makes me
> irrationally uncomfortable. I know that this is documented as a valid
> method of fooling isinstance(), but I didn't know that until your
> yesterday, and the idea of objects where type(foo) is not
> foo.__class__ strikes me as somewhat blasphemous. Maybe this is all
> fine though.
>
> The pseudo-module objects generated this way will still won't pass
> PyModule_Check, so in theory this could produce behavioural
> differences. I can't name any specific places where this will break
> things, though. From a quick skim of the CPython source, a few
> observations: It means the PyModule_* API functions won't work (e.g.
> PyModule_GetDict); maybe these aren't used enough to matter. It looks
> like the __reduce__ methods on "method objects"
> (Objects/methodobject.c) have a special check for ->m_self being a
> module object, and won't pickle correctly if ->m_self ends up pointing
> to one of these pseudo-modules. I have no idea how one ends up with a
> method whose ->m_self points to a module object, though -- maybe it
> never actually happens. PyImport_Cleanup treats module objects
> differently from non-module objects during shutdown.

Actually, there is one showstopper here -- in the first version where
reload() uses isinstance() is actually 3.4. Before that you need a
real module subtype for reload to work. But this is in principle
workaroundable by using subclassing + ctypes on old versions of python
and the __class__ = hack on new versions.

> I guess it also has the mild limitation that it doesn't work with
> extension modules, but eh. Mostly I'd be nervous about the two points
> above.
>
> -n
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org



-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Sun, Nov 30, 2014 at 10:14 PM, Mark Shannon  wrote:
> Hi,
>
> This discussion has been going on for a while, but no one has questioned the
> basic premise. Does this needs any change to the language or interpreter?
>
> I believe it does not. I'm modified your original metamodule.py to not use
> ctypes and support reloading:
> https://gist.github.com/markshannon/1868e7e6115d70ce6e76

Interesting approach!

As written, your code will blow up on any python < 3.4, because when
old_module gets deallocated it'll wipe the module dict clean. And I
guess even on >=3.4, this might still happen if old_module somehow
manages to get itself into a reference loop before getting
deallocated. (Hopefully not, but what a nightmare to debug if it did.)
However, both of these issues can be fixed by stashing a reference to
old_module somewhere in new_module.

The __class__ = ModuleType trick is super-clever but makes me
irrationally uncomfortable. I know that this is documented as a valid
method of fooling isinstance(), but I didn't know that until your
yesterday, and the idea of objects where type(foo) is not
foo.__class__ strikes me as somewhat blasphemous. Maybe this is all
fine though.

The pseudo-module objects generated this way will still won't pass
PyModule_Check, so in theory this could produce behavioural
differences. I can't name any specific places where this will break
things, though. From a quick skim of the CPython source, a few
observations: It means the PyModule_* API functions won't work (e.g.
PyModule_GetDict); maybe these aren't used enough to matter. It looks
like the __reduce__ methods on "method objects"
(Objects/methodobject.c) have a special check for ->m_self being a
module object, and won't pickle correctly if ->m_self ends up pointing
to one of these pseudo-modules. I have no idea how one ends up with a
method whose ->m_self points to a module object, though -- maybe it
never actually happens. PyImport_Cleanup treats module objects
differently from non-module objects during shutdown.

I guess it also has the mild limitation that it doesn't work with
extension modules, but eh. Mostly I'd be nervous about the two points
above.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 7:43 PM, Ben Finney  wrote:
> 
> Donald Stufft  writes:
> 
>> It’s not lost, [… a long, presumably-accurate discourse of the many
>> conditions that must be met before …] you can restore it.
> 
> This isn't the place to discuss the details of Git's internals, I think.
> I'm merely pointing out that:
> 
>> The important thing to realize is that a “branch” isn’t anything
>> special in git.
> 
> Because of that, Ethan's impression – that Git's default behaviour
> encourages losing history (by re-writing the history of commits to be
> other than what they were) is true, and “Git never loses history” simply
> isn't true.
> 
> Whether that is a *problem* is a matter of debate, but the fact that
> Git's common workflow commonly discards information that some consider
> valuable, is a simple fact.
> 
> If Ethan chooses to make that a factor in his decisions about Git, the
> facts are on his side.

Except it’s not true at all.

That data is all still there if you want it to exist and it’s not a real
differentiator between Mercurial and git because Mercurial has the ability
to do the same thing. Never mind the fact that “lose” your history makes it
sound accidental instead of on purpose. It’s like saying that ``rm foo.txt``
will “lose” the data in foo.txt. So either it was a misunderstanding in
which case I wanted to point out that those operations don’t magically lose
information or it’s a purposely FUDish statement in which case I want to
point out that the statement is inaccurate.

The only thing that is true is that git users are more likely to use the
ability to rewrite history than Mercurial users are, but you’ll typically
find that people generally don’t do this on public branches, only on private
branches. Which again doesn’t make much sense in this context since generally
currently the way people are using Mercurial with CPython you’re using
patches to transfer the changes from the contributor to the committer so you’re
“losing” that history regardless.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ben Finney]

Donald Stufft  writes:

> It’s not lost, [… a long, presumably-accurate discourse of the many
> conditions that must be met before …] you can restore it.

This isn't the place to discuss the details of Git's internals, I think.
I'm merely pointing out that:

> The important thing to realize is that a “branch” isn’t anything
> special in git.

Because of that, Ethan's impression – that Git's default behaviour
encourages losing history (by re-writing the history of commits to be
other than what they were) is true, and “Git never loses history” simply
isn't true.

Whether that is a *problem* is a matter of debate, but the fact that
Git's common workflow commonly discards information that some consider
valuable, is a simple fact.

If Ethan chooses to make that a factor in his decisions about Git, the
facts are on his side.

-- 
 \   “One of the most important things you learn from the internet |
  `\   is that there is no ‘them’ out there. It's just an awful lot of |
_o__)‘us’.” —Douglas Adams |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 7:17 PM, Ben Finney  wrote:
> 
> Donald Stufft  writes:
> 
>> I have never heard of git losing history.
> 
> In my experience talking with Git users about this problem, that depends
> on a very narrow definition of “losing history”.
> 
> Git encourages re-writing, and thereby losing prior versions of, the
> history of a branch. The commit information remains, but the history of
> how they link together is lost. That is a loss of information, which is
> not the case in the absence of such history re-writing.
> 
> Git users differ in whether they consider that information loss
> important; but it is, objectively, losing history information. So
> Ethan's impression is correct on this point.
> 

It’s not lost, the only thing that’s “gone” is a pointer to the HEAD commit
of that branch. Each commit points to it’s parent commit so if you find the
HEAD and give it a name you’ll restore the branch. It just so happens inside
the reflog you’ll see a list of the old HEADs of branches so you can get the
old commit ID from the HEAD there. In addition depending on how you rewrote
the branch and if you did anything else there is likely a reference to the
old head at ORIG_HEAD. If you don’t have the reflog (this is per copy of the
repository, so a different computer or deleting the repo and recreating it
will lose it) and for similar reasons you don’t have the ORIG_HEAD, if you
have any reference to the previous HEAD (email, commit messages, whatever)
that’s enough to restore it assuming that the commits have not been garbage
collected yet (which happens in 90 days or 30 days depending on what kind of
unreferenced commit it is) you can restore it.

The important thing to realize is that a “branch” isn’t anything special in
git. All a branch does is act as a sort of symlink to a commit ID. Anything
more beyond “what is the HEAD commit in this branch” is stored as part
of the commits themselves and doesn’t rely on the branch to be named.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ben Finney]

Donald Stufft  writes:

> I have never heard of git losing history.

In my experience talking with Git users about this problem, that depends
on a very narrow definition of “losing history”.

Git encourages re-writing, and thereby losing prior versions of, the
history of a branch. The commit information remains, but the history of
how they link together is lost. That is a loss of information, which is
not the case in the absence of such history re-writing.

Git users differ in whether they consider that information loss
important; but it is, objectively, losing history information. So
Ethan's impression is correct on this point.

-- 
 \   “If you see an animal and you can't tell if it's a skunk or a |
  `\   cat, here's a good saying to help: ‘Black and white, stinks all |
_o__)  right. Tabby-colored, likes a fella.’” —Jack Handey |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Terry Reedy]

On 11/30/2014 2:27 PM, Ethan Furman wrote:

On 11/30/2014 11:15 AM, Guido van Rossum wrote:

On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote:

On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:


All the use cases seem to be about adding some kind of getattr hook
to modules. They all seem to involve modifying the CPython C code
anyway. So why not tackle that problem head-on and modify module_getattro()
to look for a global named __getattr__ and if it exists, call that instead
of raising AttributeError?


Not sure if anyone thought of it. :) Seems like a reasonable solution to me.
Be curious to know what the benchmark suite said the impact was.


Why would there be any impact? The __getattr__ hook would be similar to the
one on classes -- it's only invoked at the point where otherwise AttributeError
would be raised.


I think the bigger question is how do we support it back on 2.7?


I do not understand this question. We don't add new features to 2.7 and 
this definitely is one.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode decode exception

[Terry Reedy]

On 11/30/2014 3:07 AM, balaji marisetti wrote:

Hi,

When I try to iterate through the lines of a
file("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c"), I get a
UnicodeDecodeError (in python 3.4.0 on Ubuntu 14.04). But there is no
such error with python 2.7.6. What could be the problem?


Questions about using current version should be directed to python-list 
or other support forums.  Py-dev is for development of future versions 
and releases.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 1:05 PM, Guido van Rossum wrote:

I don't feel it's my job to accept or reject this PEP, but I do have an
opinion.

...

- I am basically the only remaining active PEP editor, so I see most PEP
contributions by non-core-committers. Almost all of these uses github.
Not bitbucket, not some other git host, but github. I spend a fair
amount of time applying patches. It would most definitely be easier if I
could get them to send me pull requests.


The scope of the PEP is still apparently somewhat fluid.  I said 
elsewhere that I think the principal maintainers of a specialized 
single-branch repository should have the principal say in where it 
lives.  So I think you should be the one to decide on a PEP limited to 
moving the PEP repository.


My understanding is that if the peps were moved to github, then I would 
be able to suggest changes via an online web form that would generate a 
pull request from edited text.  If so, I would say go ahead and move 
them and see how it goes.


To me, the multibranch CPython repository is a very different issue. I 
think it should stay where it is for now, especially with 2.7 support 
extended.  I think for this we should better focus on better use of 
developer time and getting more developers active.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode decode exception

[Bruno Cauet]

Did you try opening it as a binary file?
open(filename, 'rb'):



—
Tagada tsouin tsouin

On Sun, Nov 30, 2014 at 5:06 PM, balaji marisetti
 wrote:

> Hi,
> When I try to iterate through the lines of a
> file("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c"), I get a
> UnicodeDecodeError (in python 3.4.0 on Ubuntu 14.04). But there is no
> such error with python 2.7.6. What could be the problem?
> In [39]: with open("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
>  for line in f:
>  print (line)
> ---
> UnicodeDecodeErrorTraceback (most recent call last)
>  in ()
>   1 with open("../openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
> > 2 for line in f:
>   3 print (line)
>   4
> /usr/lib/python3.4/codecs.py in decode(self, input, final)
> 311 # decode input (taking the buffer into account)
> 312 data = self.buffer + input
> --> 313 (result, consumed) = self._buffer_decode(data,
> self.errors, final)
> 314 # keep undecoded input until the next call
> 315 self.buffer = data[consumed:]
> -- 
> :-)balaji
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/brunocauet%40gmail.com___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

On Sun, Nov 30, 2014 at 10:55 AM, Barry Warsaw  wrote:

> On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
>
> >- Migrating "data" from GitHub is easy. There are free-as-in-freedom
> >tools to do it and the only cost is the time it would take to monitor
> >the process
>
> *Extracting* data may be easy, but migrating it is a different story.  As
> the
> Mailman project has seen in trying to migrate from Confluence to Moin,
> there
> is a ton of very difficult work involved after extracting the data.
> Parsing
> the data, ensuring that you have all the bits you need, fitting it into the
> new system's schema, working out the edge cases, adapting to semantic
> differences and gaps, ensuring that all the old links are redirected, and
> so
> on, were all exceedingly difficult[*].
>

The GitHub API is currently at Version 3. These may be useful references
for the PEP:

https://developer.github.com/v3/
https://developer.github.com/libraries/
https://github.com/jaxbot/github-issues.vim (:Gissues)

https://developer.github.com/webhooks/

There are integrations for many platforms here:

https://zapier.com/developer/documentation/
https://zapier.com/zapbook/apps/#sort=popular&filter=developer-tools



>
> Even converting between two FLOSS tools is an amazing amount of work.
> Look at
> what Eric Raymond did with reposurgeon to convert from Bazaar to git.
>
> It's a good thing that your data isn't locked behind a proprietary door,
> for
> now.  That's only part of the story.  But also, because github is a closed
> system, there's no guarantee that today's data-freeing APIs will still
> exist,
> continue to be functional for practical purposes, remain complete, or stay
> at
> parity with new features.
>
> Cheers,
> -Barry
>
> [*] And our huge gratitude goes to Paul Boddie for his amazing amount of
> work
> on the project.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Mark Shannon]

Hi,

This discussion has been going on for a while, but no one has questioned 
the basic premise. Does this needs any change to the language or 
interpreter?


I believe it does not. I'm modified your original metamodule.py to not 
use ctypes and support reloading:

https://gist.github.com/markshannon/1868e7e6115d70ce6e76

Cheers,
Mark.

On 29/11/14 01:59, Nathaniel Smith wrote:

Hi all,

There was some discussion on python-ideas last month about how to make
it easier/more reliable for a module to override attribute access.
This is useful for things like autoloading submodules (accessing
'foo.bar' triggers the import of 'bar'), or for deprecating module
attributes that aren't functions. (Accessing 'foo.bar' emits a
DeprecationWarning, "the bar attribute will be removed soon".) Python
has had some basic support for this for a long time -- if a module
overwrites its entry in sys.modules[__name__], then the object that's
placed there will be returned by 'import'. This allows one to define
custom subclasses of module and use them instead of the default,
similar to how metaclasses allow one to use custom subclasses of
'type'.

In practice though it's very difficult to make this work safely and
correctly for a top-level package. The main problem is that when you
create a new object to stick into sys.modules, this necessarily means
creating a new namespace dict. And now you have a mess, because now
you have two dicts: new_module.__dict__ which is the namespace you
export, and old_module.__dict__, which is the globals() for the code
that's trying to define the module namespace. Keeping these in sync is
extremely error-prone -- consider what happens, e.g., when your
package __init__.py wants to import submodules which then recursively
import the top-level package -- so it's difficult to justify for the
kind of large packages that might be worried about deprecating entries
in their top-level namespace. So what we'd really like is a way to
somehow end up with an object that (a) has the same __dict__ as the
original module, but (b) is of our own custom module subclass. If we
can do this then metamodules will become safe and easy to write
correctly.

(There's a little demo of working metamodules here:
https://github.com/njsmith/metamodule/
but it uses ctypes hacks that depend on non-stable parts of the
CPython ABI, so it's not a long-term solution.)

I've now spent some time trying to hack this capability into CPython
and I've made a list of the possible options I can think of to fix
this. I'm writing to python-dev because none of them are obviously The
Right Way so I'd like to get some opinions/ruling/whatever on which
approach to follow up on.

Option 1: Make it possible to change the type of a module object
in-place, so that we can write something like

sys.modules[__name__].__class__ = MyModuleSubclass

Option 1 downside: The invariants required to make __class__
assignment safe are complicated, and only implemented for
heap-allocated type objects. PyModule_Type is not heap-allocated, so
making this work would require lots of delicate surgery to
typeobject.c. I'd rather not go down that rabbit-hole.



Option 2: Make PyModule_Type into a heap type allocated at interpreter
startup, so that the above just works.

Option 2 downside: PyModule_Type is exposed as a statically-allocated
global symbol, so doing this would involve breaking the stable ABI.



Option 3: Make it legal to assign to the __dict__ attribute of a
module object, so that we can write something like

new_module = MyModuleSubclass(...)
new_module.__dict__ = sys.modules[__name__].__dict__
sys.modules[__name__].__dict__ = {} # ***
sys.modules[__name__] = new_module

The line marked *** is necessary because the way modules are designed,
they expect to control the lifecycle of their __dict__. When the
module object is initialized, it fills in a bunch of stuff in the
__dict__. When the module object (not the dict object!) is
deallocated, it deletes everything from the __dict__. This latter
feature in particular means that having two module objects sharing the
same __dict__ is bad news.

Option 3 downside: The paragraph above. Also, there's stuff inside the
module struct besides just the __dict__, and more stuff has appeared
there over time.



Option 4: Add a new function sys.swap_module_internals, which takes
two module objects and swaps their __dict__ and other attributes. By
making the operation a swap instead of an assignment, we avoid the
lifecycle pitfalls from Option 3. By making it a builtin, we can make
sure it always handles all the module fields that matter, not just
__dict__. Usage:

new_module = MyModuleSubclass(...)
sys.swap_module_internals(new_module, sys.modules[__name__])
sys.modules[__name__] = new_module

Option 4 downside: Obviously a hack.



Option 3 or 4 both seem workable, it just depends on which way we
prefer to hold our nose. Option 4 is slightly more correct in that it
wor

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 2:33 PM, Donald Stufft wrote:


So a goal of mine here is to sort of use these as a bit of a test bed.
Moving CPython itself is a big and drastic change with a lot of
implications, but moving the “support” repositories is not nearly as
much, especially with a read only mirror on hg.python.org
 which would allow things like the PEP rendering
on www.python.org  to stay the same if we wanted
to. My hope was that we’d try this out, see how it works out, and if it
seems to be a good thing, then at a later time we can either look at
moving CPython itself or decide if it makes sense to do something
different. Maybe this should be spelled out in the PEP?

I’ve seen a few people say they were -1 because they didn’t want to
split between hg on the CPython side and git on the supporting repos
side.


Being only recently somewhat comfortable with hg, I do not really want 
to have to learn git at this time.



I’m not sure you can really get away from that because we’re
*already* in that situation, things like the docs building script is a
Git repo on Github,


As long as I can do test doc builds with the scripts in /Docs, this does 
not matter to me.  If the people who work with the site doc build script 
want it in git on Github, fine.



the python infrastructure itself is a git repo on Github,


Since I am not sure what you mean, its location does not matter to me.

> the new python.org  website is a git repo on

Github, the future PyPI is a git repo on GitHub.


Since I do not commit to either, ditto.  As far as I am concerned, the 
people involved with specialized repositories may as well have their 
preference.  This includes devinabox.


I have made a few commits to the PEP repository, but only to the one I 
co-authored, 434. Using the web form might make it easier to suggest 
changes to other peoples' PEPs.



IOW I’m not sure what the best way forward is. I think moving to these
tools for *all* repos is likely to be in the best interests of making
things better for both sides of that coin however I didn’t want to go
wholesale and try and make everything switch at all at once.


I agree with others that the current bottleneck is disposing of patches, 
especially those from non-core developers, not getting more patches to 
considier.  I am quite sure that if we significantly reduce the current 
backlog of 4600 issues and more expeditiously handles the contributions 
we do get, will will get more.  More than one person has said that they 
are disinclined to submit patches when they have multiple patches on the 
tracker already.


So I think the focus should be on making better use of developer time 
and having more of it.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 3:26 PM, Brett Cannon  wrote:
> 
> 
> 
> On Sun Nov 30 2014 at 2:33:35 PM Donald Stufft  > wrote:
> 
>> On Nov 30, 2014, at 2:19 PM, Brett Cannon > > wrote:
>> 
>> All very true, but if we can't improve both sides then we are simply going 
>> to end up with even more patches that we take a while to get around to. I 
>> want to end up with a solution that advances the situation for *both* 
>> committers and non-committers and I feel like that is being lost in the 
>> discussion somewhat. As the person who pushed for a migration to DVCS for 
>> non-committers I totally support improving the workflow for non-committers, 
>> but not at the cost of ignoring the latter half of the contribution workflow 
>> of committers which is a chronic problem.
>> 
>> As the PEP points out, the devguide, devinabox, and the PEPs have such a 
>> shallow development process that hosting them on Bitbucket wouldn't be a big 
>> thing. But if we don't view this as a long-term step towards moving cpython 
>> development somehow we are bifurcating our code contributors between git and 
>> hg which will be annoying. Now it could be argued that it doesn't matter for 
>> the peps and devguide since they are purely text and can be done easily 
>> through a web UI and a simple CI in Travis can be set up to make sure that 
>> the docs compile cleanly. But moving devinabox where there is going to be a 
>> code checkout in order to execute code for testing, etc. will be an issue.
>> 
>> So I guess my view is +0 for doc-only repos on GitHub as long as we make it 
>> clear we are doing it with the expectation that people will do everything 
>> through the web UI and never have to know git. But I can't advocate moving 
>> code over without moving ALL repos over to git -- hosting location doesn't 
>> matter to me -- to prevent having to know both DVCSs in order to do coding 
>> work related to Python; the cpython repo shouldn't become this vaunted repo 
>> that is special and so it's in hg long-term but everything else is on git.
> 
> 
> So a goal of mine here is to sort of use these as a bit of a test bed. Moving 
> CPython itself is a big and drastic change with a lot of implications, but 
> moving the “support” repositories is not nearly as much, especially with a 
> read only mirror on hg.python.org  which would allow 
> things like the PEP rendering on www.python.org  to 
> stay the same if we wanted to. My hope was that we’d try this out, see how it 
> works out, and if it seems to be a good thing, then at a later time we can 
> either look at moving CPython itself or decide if it makes sense to do 
> something different. Maybe this should be spelled out in the PEP? 
> 
> I’ve seen a few people say they were -1 because they didn’t want to split 
> between hg on the CPython side and git on the supporting repos side. I’m not 
> sure you can really get away from that because we’re *already* in that 
> situation, things like the docs building script is a Git repo on Github, the 
> python infrastructure itself is a git repo on Github, the new python.org 
>  website is a git repo on Github, the future PyPI is a 
> git repo on GitHub.
> 
> That doesn't bother as that is support infrastructure around CPython but in 
> no way directly tied to CPython releases. But devinabox, for instance, is 
> specifically for helping people contribute to CPython, so asking people to 
> use devinabox in git but then work in hg for some repos and git in others 
> that devinabox checks out is just asking for trouble (e.g. devinabox checks 
> out the peps, devguide, and cpython repos).

I’m not sure what you’re proposing here. If devinabox checks out peps, 
devguide, and cpython aren’t they going to have to use git and hg both unless 
we move all of those repositories together? Beyond just the tools the status 
quo of those is that if you do make a change you have different ways to submit 
a contribution depending on which repository it is.

>  
>  
> 
> IOW I’m not sure what the best way forward is. I think moving to these tools 
> for *all* repos is likely to be in the best interests of making things better 
> for both sides of that coin however I didn’t want to go wholesale and try and 
> make everything switch at all at once. If you think it makes sense to drop 
> devinabox and make the dividing line between Code and not code (although I’d 
> argue that line is already crossed with other code things already being on 
> github) that’s fine with me. Or I can expand the scope if people think that 
> makes more sense in the PEP too.
> 
> Depends what other people think, but for me it's "we are going to move to git 
> long-term and we are starting an experiment with docs on GitHub to see if 
> that impacts contributions and committer maintenance at least for docs, maybe 
> for code eventually”.

What do you mean by “docs”, is that

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

On Sun, Nov 30, 2014 at 1:12 PM, Antoine Pitrou  wrote:

> On Sun, 30 Nov 2014 11:15:50 -0800
> Guido van Rossum  wrote:
> > On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon  wrote:
> > >
> > >  On Sat, Nov 29, 2014, 21:55 Guido van Rossum 
> wrote:
> > >
> > > All the use cases seem to be about adding some kind of getattr hook to
> > > modules. They all seem to involve modifying the CPython C code anyway.
> So
> > > why not tackle that problem head-on and modify module_getattro() to
> look
> > > for a global named __getattr__ and if it exists, call that instead of
> > > raising AttributeError?
> > >
> > >  Not sure if anyone thought of it. :) Seems like a reasonable solution
> to
> > > me. Be curious to know what the benchmark suite said the impact was.
> > >
> > Why would there be any impact? The __getattr__ hook would be similar to
> the
> > one on classes -- it's only invoked at the point where otherwise
> > AttributeError would be raised.
>
> builtins are typically found by first looking up in the current globals
> (module) scope, failing, and then falling back on __builtins__.
>
> Depending on how much overhead is added to the "failing" step, there
> /might/ be a performance difference. Of course, that would only occur
> wherever a __getattr__ hook is defined.
>

The builtins lookup process never does a module attribute lookup -- it only
does dict lookups. So it would not be affected by a module __getattr__ hook
(unless we were to use dict proxies, which Nathaniel already rejected).

@Nathaniel: perhaps you could get what you want without any C code changes
using the approach of Brett's LazyLoader?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Antoine Pitrou]

On Sun, 30 Nov 2014 11:15:50 -0800
Guido van Rossum  wrote:
> On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon  wrote:
> >
> >  On Sat, Nov 29, 2014, 21:55 Guido van Rossum  wrote:
> >
> > All the use cases seem to be about adding some kind of getattr hook to
> > modules. They all seem to involve modifying the CPython C code anyway. So
> > why not tackle that problem head-on and modify module_getattro() to look
> > for a global named __getattr__ and if it exists, call that instead of
> > raising AttributeError?
> >
> >  Not sure if anyone thought of it. :) Seems like a reasonable solution to
> > me. Be curious to know what the benchmark suite said the impact was.
> >
> Why would there be any impact? The __getattr__ hook would be similar to the
> one on classes -- it's only invoked at the point where otherwise
> AttributeError would be raised.

builtins are typically found by first looking up in the current globals
(module) scope, failing, and then falling back on __builtins__.

Depending on how much overhead is added to the "failing" step, there
/might/ be a performance difference. Of course, that would only occur
wherever a __getattr__ hook is defined.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 4:05 PM, Antoine Pitrou  wrote:
> 
> On Sun, 30 Nov 2014 10:05:01 -0800
> Guido van Rossum  wrote:
>> 
>> I bring this up to emphasize that (unlike GNU software and the FSF) Python
>> has no additional hidden agenda of bringing freedom to all software.
> 
> As far as GNU and the FSF are concerned, I don't think the agenda is
> "hidden" at all ;-)
> 
>> Now some entirely practical points.
>> 
>> - I am basically the only remaining active PEP editor, so I see most PEP
>> contributions by non-core-committers. Almost all of these uses github. Not
>> bitbucket, not some other git host, but github. I spend a fair amount of
>> time applying patches. It would most definitely be easier if I could get
>> them to send me pull requests.
> 
> Are you sure that those contributors wouldn't want to use Bitbucket -
> or another hg-based hosting service?
> 
> A PEP contributor is someone who is likely to contribute CPython code as
> well - so they would have to know hg.

Speaking as someone who has written a number of PEPS, plans to write a number
more, and also has commit bit on hg.python.org and is maintaining code in 
CPython.
I would greatly prefer using Github for PEPs than I would any hg hosting service
I’ve seen to date. I can’t obviously speak for every single committer, however
Guido also stated that through his work in committing patches of incoming PEPs
he’s seen that a lot of the PEPs are being written on Github using git. I think
it’s fair to say that those people would prefer PRs on Github over using 
Bitbucket
as well since they were choosing Github over Bitbucket when there was no 
incentive
to do so.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Brett Cannon]

On Sun Nov 30 2014 at 3:55:39 PM Guido van Rossum  wrote:

> On Sun, Nov 30, 2014 at 11:29 AM, Nathaniel Smith  wrote:
>
>> On Sun, Nov 30, 2014 at 2:54 AM, Guido van Rossum 
>> wrote:
>> > All the use cases seem to be about adding some kind of getattr hook to
>> > modules. They all seem to involve modifying the CPython C code anyway.
>> So
>> > why not tackle that problem head-on and modify module_getattro() to
>> look for
>> > a global named __getattr__ and if it exists, call that instead of
>> raising
>> > AttributeError?
>>
>> You need to allow overriding __dir__ as well for tab-completion, and
>> some people wanted to use the properties API instead of raw
>> __getattr__, etc. Maybe someone will want __getattribute__ semantics,
>> I dunno.
>
>
> Hm... I agree about __dir__ but the other things feel too speculative.
>
>
>> So since we're *so close* to being able to just use the
>> subclassing machinery, it seemed cleaner to try and get that working
>> instead of reimplementing bits of it piecewise.
>>
>
> That would really be option 1, right? It's the one that looks cleanest
> from the user's POV (or at least from the POV of a developer who wants to
> build a framework using this feature -- for a simple one-off use case,
> __getattr__ sounds pretty attractive). I think that if we really want
> option 1, the issue of PyModuleType not being a heap type can be dealt with.
>
>
>> That said, __getattr__ + __dir__ would be enough for my immediate use
>> cases.
>
>
>  Perhaps it would be a good exercise to try and write the "lazy submodule
> import"(*) use case three ways: (a) using only CPython 3.4; (b) using
> __class__ assignment; (c) using customizable __getattr__ and __dir__. I
> think we can learn a lot about the alternatives from this exercise. I
> presume there's already a version of (a) floating around, but if it's been
> used in practice at all, it's probably too gnarly to serve as a useful
> comparison (though its essence may be extracted to serve as such).
>
> FWIW I believe all proposals here have a big limitation: the module
> *itself* cannot benefit much from all these shenanigans, because references
> to globals from within the module's own code are just dictionary accesses,
> and we don't want to change that.
>
> (*) I originally wrote "lazy import", but I realized that messing with the
> module class object probably isn't the best way to implement that -- it
> requires a proxy for the module that's managed by an import hook. But if
> you think it's possible, feel free to use this example, as "lazy import"
> seems a pretty useful thing to have in many situations. (At least that's
> how I would do it. And I would probably add some atrocious hack to patch up
> the importing module's globals once the module is actually loaded, to
> reduce the cost of using the proxy over the lifetime of the process.
>

Start at
https://hg.python.org/cpython/file/64bb01bce12c/Lib/importlib/util.py#l207
and read down the rest of the file. It really only requires changing
__class__ to drop the proxy and that's done immediately after the lazy
import. The approach also occurs *after* the finder so you don't get
ImportError for at least missing a file.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Antoine Pitrou]

On Sun, 30 Nov 2014 10:05:01 -0800
Guido van Rossum  wrote:
> 
> I bring this up to emphasize that (unlike GNU software and the FSF) Python
> has no additional hidden agenda of bringing freedom to all software.

As far as GNU and the FSF are concerned, I don't think the agenda is
"hidden" at all ;-)

> Now some entirely practical points.
> 
> - I am basically the only remaining active PEP editor, so I see most PEP
> contributions by non-core-committers. Almost all of these uses github. Not
> bitbucket, not some other git host, but github. I spend a fair amount of
> time applying patches. It would most definitely be easier if I could get
> them to send me pull requests.

Are you sure that those contributors wouldn't want to use Bitbucket -
or another hg-based hosting service?

A PEP contributor is someone who is likely to contribute CPython code as
well - so they would have to know hg.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Antoine Pitrou]

On Sun, 30 Nov 2014 19:19:50 +
Brett Cannon  wrote:
> 
> As the PEP points out, the devguide, devinabox, and the PEPs have such a
> shallow development process that hosting them on Bitbucket wouldn't be a
> big thing. But if we don't view this as a long-term step towards moving
> cpython development somehow we are bifurcating our code contributors
> between git and hg which will be annoying. Now it could be argued that it
> doesn't matter for the peps and devguide since they are purely text and can
> be done easily through a web UI and a simple CI in Travis can be set up to
> make sure that the docs compile cleanly.

Web-based text editors are a poor UI so, while it *can* be done over
the Web, it's not really a good thing to promote (unless we're talking
mini-edits such as fixing typos).

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

On Sun, Nov 30, 2014 at 11:29 AM, Nathaniel Smith  wrote:

> On Sun, Nov 30, 2014 at 2:54 AM, Guido van Rossum 
> wrote:
> > All the use cases seem to be about adding some kind of getattr hook to
> > modules. They all seem to involve modifying the CPython C code anyway. So
> > why not tackle that problem head-on and modify module_getattro() to look
> for
> > a global named __getattr__ and if it exists, call that instead of raising
> > AttributeError?
>
> You need to allow overriding __dir__ as well for tab-completion, and
> some people wanted to use the properties API instead of raw
> __getattr__, etc. Maybe someone will want __getattribute__ semantics,
> I dunno.


Hm... I agree about __dir__ but the other things feel too speculative.


> So since we're *so close* to being able to just use the
> subclassing machinery, it seemed cleaner to try and get that working
> instead of reimplementing bits of it piecewise.
>

That would really be option 1, right? It's the one that looks cleanest from
the user's POV (or at least from the POV of a developer who wants to build
a framework using this feature -- for a simple one-off use case,
__getattr__ sounds pretty attractive). I think that if we really want
option 1, the issue of PyModuleType not being a heap type can be dealt with.


> That said, __getattr__ + __dir__ would be enough for my immediate use
> cases.


 Perhaps it would be a good exercise to try and write the "lazy submodule
import"(*) use case three ways: (a) using only CPython 3.4; (b) using
__class__ assignment; (c) using customizable __getattr__ and __dir__. I
think we can learn a lot about the alternatives from this exercise. I
presume there's already a version of (a) floating around, but if it's been
used in practice at all, it's probably too gnarly to serve as a useful
comparison (though its essence may be extracted to serve as such).

FWIW I believe all proposals here have a big limitation: the module
*itself* cannot benefit much from all these shenanigans, because references
to globals from within the module's own code are just dictionary accesses,
and we don't want to change that.

(*) I originally wrote "lazy import", but I realized that messing with the
module class object probably isn't the best way to implement that -- it
requires a proxy for the module that's managed by an import hook. But if
you think it's possible, feel free to use this example, as "lazy import"
seems a pretty useful thing to have in many situations. (At least that's
how I would do it. And I would probably add some atrocious hack to patch up
the importing module's globals once the module is actually loaded, to
reduce the cost of using the proxy over the lifetime of the process.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ethan Furman]

On 11/30/2014 11:56 AM, Donald Stufft wrote:
>> On Nov 30, 2014, at 2:28 PM, Ethan Furman  wrote:
>>
>> My issues with GitHub range from selfish to philosophical:
>>
>>  - (selfish) I don't want to learn git
> 
> Note: That you don’t actually have to learn git, you can clone a git 
> repository
> with Mercurial using hg-git and continue to use Mercurial locally. The same of
> course can be said for the *other* way, but I’d argue that putting the burden 
> of
> using things like hg-git or git-remote-hg on the less popular tool is a better
> overall decision.

Fair enough.

>>  - (practical) from what I hear git can have issues with losing history -- 
>> in a
>>project that has many volunteer and part-time developers, using a tool 
>> that
>>can munge your data just doesn't seem very wise
> 
> I have never heard of git losing history. Git goes out of it’s way not to lose
> things. Even unreferences commits don’t go away for months unless you 
> purposely
> prune them. I’d personally call this FUD unless there’s some citation here.

Okay.


>>  - (practical) supporting git and hg means learning two different workflows
> 
> As I mentioned in my other email, we’re already supporting two different 
> tools,
> and it’s a hope of mine to use this as a sort of testbed to moving the other
> repositories as well.

That should be in the PEP then.


>>  - (philosophical) in a commercial world we vote with our dollars (don't 
>> like how
>>a company behaves?  don't buy their product); in an OSS world we vote by 
>> whose
>>services/software we use;  I don't want to support, or appear to support, 
>> a
>>company that is abusive and sexist towards its employees:  it is not what 
>> the
>>PSF supports, and it's definitely not what I support.
> 
> I’m assuming this is about Github.

Yes.

> I’ll say that Github has, at least publicly, made steps towards doing better 
> than
> they had previously there. I’m not a Github employee so I can’t speak towards 
> that.
> 
> It almost feels like there is some amount of co-opting this incident as a 
> shield
> to hide behind. Most people who make this statement are more than happy to 
> continue
> to use Linux for example, even though Linus is well documented being extremely
> abusive towards people and has more or less said that he’s never going to 
> change
> that.

Linux is not my choice.  ;)  Linus is also one person, not an entire company.

> I also think it’s hard to look at a company like bitbucket, for example, and
> say they are *better* than Github just because they didn’t have a public and
> inflammatory event.

In cases like this it's not "better" but "willing to work with".  BitBucket 
might be as hostile as GitHub, only someone
who has worked for both could really know.

> [...] than refusing to use a product of one group of people over another just 
> because
> the other group hasn’t had a public and inflammatory event.

We can only make decisions on information we have;  pretending we don't have 
it, or that some other company /could/ be
the same, is hiding our heads in the sand.

If git is the wave of the future, there are other git hosts besides GitHub.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Brett Cannon]

On Sun Nov 30 2014 at 2:33:35 PM Donald Stufft  wrote:

>
> On Nov 30, 2014, at 2:19 PM, Brett Cannon  wrote:
>
> All very true, but if we can't improve both sides then we are simply going
> to end up with even more patches that we take a while to get around to. I
> want to end up with a solution that advances the situation for *both*
> committers and non-committers and I feel like that is being lost in the
> discussion somewhat. As the person who pushed for a migration to DVCS for
> non-committers I totally support improving the workflow for non-committers,
> but not at the cost of ignoring the latter half of the contribution
> workflow of committers which is a chronic problem.
>
> As the PEP points out, the devguide, devinabox, and the PEPs have such a
> shallow development process that hosting them on Bitbucket wouldn't be a
> big thing. But if we don't view this as a long-term step towards moving
> cpython development somehow we are bifurcating our code contributors
> between git and hg which will be annoying. Now it could be argued that it
> doesn't matter for the peps and devguide since they are purely text and can
> be done easily through a web UI and a simple CI in Travis can be set up to
> make sure that the docs compile cleanly. But moving devinabox where there
> is going to be a code checkout in order to execute code for testing, etc.
> will be an issue.
>
> So I guess my view is +0 for doc-only repos on GitHub as long as we make
> it clear we are doing it with the expectation that people will do
> everything through the web UI and never have to know git. But I can't
> advocate moving code over without moving ALL repos over to git -- hosting
> location doesn't matter to me -- to prevent having to know both DVCSs in
> order to do coding work related to Python; the cpython repo shouldn't
> become this vaunted repo that is special and so it's in hg long-term but
> everything else is on git.
>
>
> So a goal of mine here is to sort of use these as a bit of a test bed.
> Moving CPython itself is a big and drastic change with a lot of
> implications, but moving the “support” repositories is not nearly as much,
> especially with a read only mirror on hg.python.org which would allow
> things like the PEP rendering on www.python.org to stay the same if we
> wanted to. My hope was that we’d try this out, see how it works out, and if
> it seems to be a good thing, then at a later time we can either look at
> moving CPython itself or decide if it makes sense to do something
> different. Maybe this should be spelled out in the PEP?
>
> I’ve seen a few people say they were -1 because they didn’t want to split
> between hg on the CPython side and git on the supporting repos side. I’m
> not sure you can really get away from that because we’re *already* in that
> situation, things like the docs building script is a Git repo on Github,
> the python infrastructure itself is a git repo on Github, the new
> python.org website is a git repo on Github, the future PyPI is a git repo
> on GitHub.
>

That doesn't bother as that is support infrastructure around CPython but in
no way directly tied to CPython releases. But devinabox, for instance, is
specifically for helping people contribute to CPython, so asking people to
use devinabox in git but then work in hg for some repos and git in others
that devinabox checks out is just asking for trouble (e.g. devinabox checks
out the peps, devguide, and cpython repos).


>
>
> IOW I’m not sure what the best way forward is. I think moving to these
> tools for *all* repos is likely to be in the best interests of making
> things better for both sides of that coin however I didn’t want to go
> wholesale and try and make everything switch at all at once. If you think
> it makes sense to drop devinabox and make the dividing line between Code
> and not code (although I’d argue that line is already crossed with other
> code things already being on github) that’s fine with me. Or I can expand
> the scope if people think that makes more sense in the PEP too.
>

Depends what other people think, but for me it's "we are going to move to
git long-term and we are starting an experiment with docs on GitHub to see
if that impacts contributions and committer maintenance at least for docs,
maybe for code eventually".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode decode exception

[Chris Angelico]

On Sun, Nov 30, 2014 at 7:07 PM, balaji marisetti
 wrote:
> Hi,

Hi. This list is for the development *of* Python, not development
*with* Python, so I'm sending this reply also to
python-l...@python.org where it can be better handled. You'll probably
want to subscribe here:

https://mail.python.org/mailman/listinfo/python-list

or alternatively, point a news reader at comp.lang.python. Let's
continue this conversation on python-list rather than python-dev.

> When I try to iterate through the lines of a
> file("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c"), I get a
> UnicodeDecodeError (in python 3.4.0 on Ubuntu 14.04). But there is no
> such error with python 2.7.6. What could be the problem?

The difference between the two Python versions is that 2.7 lets you be
a bit sloppy about Unicode vs bytes, but 3.4 requires that you keep
them properly separate.

> In [39]: with open("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
>  for line in f:
>  print (line)
>
> ---
> UnicodeDecodeErrorTraceback (most recent call last)
>  in ()
>   1 with open("../openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
> > 2 for line in f:
>   3 print (line)
>   4
>
> /usr/lib/python3.4/codecs.py in decode(self, input, final)
> 311 # decode input (taking the buffer into account)
> 312 data = self.buffer + input
> --> 313 (result, consumed) = self._buffer_decode(data,
> self.errors, final)
> 314 # keep undecoded input until the next call
> 315 self.buffer = data[consumed:]
>
>
> --
> :-)balaji

Most likely, the line of input that you just reached has a non-ASCII
character, and the default encoding is ASCII. (Though without the
actual exception message, I can't be sure of that.) The best fix would
be to know what the file's encoding is, and simply add that as a
parameter to your open() call - perhaps this:

with open("filename", encoding="utf-8") as f:

If you use the right encoding, and the file is correctly encoded, you
should have no errors. If you still have errors... welcome to data
problems, life can be hard. :|

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Chris Angelico]

On Mon, Dec 1, 2014 at 6:28 AM, Ethan Furman  wrote:
> My issues with GitHub range from selfish to philosophical:
>
>   - (selfish) I don't want to learn git

This ties in directly with the popularity argument. How many people
are there who know hg and don't know git? How many who know git and
don't know hg? So while this is a git issue for you, it's going to be
a Mercurial issue for a lot more people.

(And even people like me who kinda know both are going to spend a lot
more time with git than with hg, on average. My knowledge of git is
fairly advanced - I use git hooks, have scripts that manage things for
me, can repair a repository that's seen some mess in it, etc - but my
Mercurial knowledge is basic - I can clone and pull, not even sure I
can push as I have literally never done so, and I basically turn to a
dev guide to figure out how to make a patch in the appropriate way.)

>   - (practical) from what I hear git can have issues with losing history -- 
> in a
> project that has many volunteer and part-time developers, using a tool 
> that
> can munge your data just doesn't seem very wise

It is possible to rewrite history in git, but - assuming you're not
going to go to the ridiculous extent of SHA-1 cracking - it'll always
be done by creating a new stream of commits, and git is very careful
about confirming that you want to do this sort of thing. When you have
a central server, you can simply configure it to reject any
non-fast-forward pushes. (I'm not sure how to set
receive.denyNonFastForwards on GitHub, but it's likely to be possible.
In any case, you can always have an authoritative clone on python.org
somewhere which mandates this kind of thing.)

The git philosophy is: Your repository is yours. What you do with it
is up to you. If that means rewriting history, git will confirm with
you that you really want to do that, and then go right ahead and do
what you ask. But then if your repo is a clone of someone else's,
well, that someone else controls the other repo, so you might well not
be allowed to merge changes in.

>   - (practical) supporting git and hg means learning two different workflows

Already a problem for a lot of people. Unless CPython is the only
project you ever contribute to, chances are you're going to meet git
somewhere. If CPython used some mindblowingly brilliant but utterly
unheard-of DVCS, sure it might be possible for core devs to avoid
using any of the popular systems, but nobody else could.

Personally, and somewhat selfishly, I would love to see the PEPs repo
move to GitHub. For the two PEPs that I wrote, I had to juggle my own
personal PEP draft repo and the central Mercurial peps repo; making
sure changes got deployed from one to the other was basically a manual
job, without any tool support. If I could send PRs against a clone of
the peps repo, I would work that way instead of creating a dedicated
repo at all (at least until such time as I need other files; the PEP
463 repo grew a few other bits and bobs, but they could just as easily
have been in an ancillary repo without the PEP text in it).

There'll be easily as many people who selfishly want git as selfishly want hg :)

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 2:28 PM, Ethan Furman  wrote:
> 
> On 11/30/2014 10:05 AM, Guido van Rossum wrote:
> 
>> Python has a long history (all the way back to my choice of a MIT-style 
>> license for the first release) of mixing "free"
>> and "non-free" uses and tools -- for example on Windows we consciously chose 
>> to align ourselves with the platform
>> tooling rather than with the (minority) free tools available, Python has 
>> been ported to many commercial platforms, and
>> I've always encouraged use of Python in closed-source situations.
> 
> For this I am grateful, and agree with.
> 
>> Finally. And this may actually be the most important point. Python people 
>> should be doing stuff that makes Python better
>> (both taken in the most inclusive way possible). For stuff that's not unique 
>> to Python but can be used by many other
>> open-source projects, such as compilers, DVCS tools, or mailing lists, we 
>> should not be wasting our precious time on
>> building and maintaining our own tools or administering the servers on which 
>> they run. And historically we've not done a
>> great job on maintenance and administration.
> 
> My issues with GitHub range from selfish to philosophical:
> 
>  - (selfish) I don't want to learn git

Note: That you don’t actually have to learn git, you can clone a git repository 
with Mercurial using hg-git and continue to use Mercurial locally. The same of 
course can be said for the *other* way, but I’d argue that putting the burden 
of using things like hg-git or git-remote-hg on the less popular tool is a 
better overall decision.

> 
>  - (practical) from what I hear git can have issues with losing history -- in 
> a
>project that has many volunteer and part-time developers, using a tool that
>can munge your data just doesn't seem very wise

I have never heard of git losing history. Git goes out of it’s way not to lose 
things. Even unreferences commits don’t go away for months unless you purposely 
prune them. I’d personally call this FUD unless there’s some citation here.

> 
>  - (practical) supporting git and hg means learning two different workflows

As I mentioned in my other email, we’re already supporting two different tools, 
and it’s a hope of mine to use this as a sort of testbed to moving the other 
repositories as well.

> 
>  - (philosophical) in a commercial world we vote with our dollars (don't like 
> how
>a company behaves?  don't buy their product); in an OSS world we vote by 
> whose
>services/software we use;  I don't want to support, or appear to support, a
>company that is abusive and sexist towards its employees:  it is not what 
> the
>PSF supports, and it's definitely not what I support.

I’m assuming this is about Github. I’ll say that Github has, at least publicly,
made steps towards doing better than they had previously there. I’m not a Github
employee so I can’t speak towards that.

It almost feels like there is some amount of co-opting this incident as a shield
to hide behind. Most people who make this statement are more than happy to 
continue
to use Linux for example, even though Linus is well documented being extremely
abusive towards people and has more or less said that he’s never going to change
that. I also think it’s hard to look at a company like bitbucket, for example, 
and
say they are *better* than Github just because they didn’t have a public and
inflammatory event.

Attempting to reduce the cognitive burden for contributing and aligning 
ourselves
with the most popular tools allows us to take advantage of the network effects
of these tools popularity. This can be the difference between someone with 
limited
amount of time being able to contribute or not, which can make real inroads 
towards
making it easier for under privileged people to contribute much more than 
refusing
to use a product of one group of people over another just because the other 
group
hasn’t had a public and inflammatory event.


> 
> Not everyone is suited to demonstrate in the streets, but it shouldn't be 
> that hard to not use a company with
> acknowledged bad practices.
> 
> --
> ~Ethan~
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Sun, Nov 30, 2014 at 7:27 PM, Ethan Furman  wrote:
> On 11/30/2014 11:15 AM, Guido van Rossum wrote:
>> On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote:
>>> On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:

 All the use cases seem to be about adding some kind of getattr hook
 to modules. They all seem to involve modifying the CPython C code
 anyway. So why not tackle that problem head-on and modify module_getattro()
 to look for a global named __getattr__ and if it exists, call that instead
 of raising AttributeError?
>>>
>>> Not sure if anyone thought of it. :) Seems like a reasonable solution to me.
>>> Be curious to know what the benchmark suite said the impact was.
>>
>> Why would there be any impact? The __getattr__ hook would be similar to the
>> one on classes -- it's only invoked at the point where otherwise 
>> AttributeError
>> would be raised.
>
> I think the bigger question is how do we support it back on 2.7?

I think that's doable -- assuming I'm remembering correctly the
slightly weird class vs. instance lookup rules for special methods,
you can write a module subclass like

class GetAttrModule(types.ModuleType):
def __getattr__(self, name):
return self.__dict__["__getattr__"](name)

and then use ctypes hacks to get it into sys.modules[__name__].

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Brett Cannon]

On Sun Nov 30 2014 at 2:28:31 PM Ethan Furman  wrote:

> On 11/30/2014 11:15 AM, Guido van Rossum wrote:
> > On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote:
> >> On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:
> >>>
> >>> All the use cases seem to be about adding some kind of getattr hook
> >>> to modules. They all seem to involve modifying the CPython C code
> >>> anyway. So why not tackle that problem head-on and modify
> module_getattro()
> >>> to look for a global named __getattr__ and if it exists, call that
> instead
> >>> of raising AttributeError?
> >>
> >> Not sure if anyone thought of it. :) Seems like a reasonable solution
> to me.
> >> Be curious to know what the benchmark suite said the impact was.
> >
> > Why would there be any impact? The __getattr__ hook would be similar to
> the
> > one on classes -- it's only invoked at the point where otherwise
> AttributeError
> > would be raised.
>
> I think the bigger question is how do we support it back on 2.7?
>

You don't; you just can't shoehorn everything back to 2.7.

And just to make sure everyone participating in this discussion is up on
the latest import stuff, Python 3.4 does have Loader.create_module()

which lets you control what object is used for a module in the import
machinery (this is prior to loading, though, so you can't specify it in a
module but at the loader level only). This is how I was able to implement lazy
loading for 3.5

.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 2:19 PM, Brett Cannon  wrote:
> 
> All very true, but if we can't improve both sides then we are simply going to 
> end up with even more patches that we take a while to get around to. I want 
> to end up with a solution that advances the situation for *both* committers 
> and non-committers and I feel like that is being lost in the discussion 
> somewhat. As the person who pushed for a migration to DVCS for non-committers 
> I totally support improving the workflow for non-committers, but not at the 
> cost of ignoring the latter half of the contribution workflow of committers 
> which is a chronic problem.
> 
> As the PEP points out, the devguide, devinabox, and the PEPs have such a 
> shallow development process that hosting them on Bitbucket wouldn't be a big 
> thing. But if we don't view this as a long-term step towards moving cpython 
> development somehow we are bifurcating our code contributors between git and 
> hg which will be annoying. Now it could be argued that it doesn't matter for 
> the peps and devguide since they are purely text and can be done easily 
> through a web UI and a simple CI in Travis can be set up to make sure that 
> the docs compile cleanly. But moving devinabox where there is going to be a 
> code checkout in order to execute code for testing, etc. will be an issue.
> 
> So I guess my view is +0 for doc-only repos on GitHub as long as we make it 
> clear we are doing it with the expectation that people will do everything 
> through the web UI and never have to know git. But I can't advocate moving 
> code over without moving ALL repos over to git -- hosting location doesn't 
> matter to me -- to prevent having to know both DVCSs in order to do coding 
> work related to Python; the cpython repo shouldn't become this vaunted repo 
> that is special and so it's in hg long-term but everything else is on git.


So a goal of mine here is to sort of use these as a bit of a test bed. Moving 
CPython itself is a big and drastic change with a lot of implications, but 
moving the “support” repositories is not nearly as much, especially with a read 
only mirror on hg.python.org  which would allow things 
like the PEP rendering on www.python.org  to stay the 
same if we wanted to. My hope was that we’d try this out, see how it works out, 
and if it seems to be a good thing, then at a later time we can either look at 
moving CPython itself or decide if it makes sense to do something different. 
Maybe this should be spelled out in the PEP? 

I’ve seen a few people say they were -1 because they didn’t want to split 
between hg on the CPython side and git on the supporting repos side. I’m not 
sure you can really get away from that because we’re *already* in that 
situation, things like the docs building script is a Git repo on Github, the 
python infrastructure itself is a git repo on Github, the new python.org 
 website is a git repo on Github, the future PyPI is a git 
repo on GitHub. 

IOW I’m not sure what the best way forward is. I think moving to these tools 
for *all* repos is likely to be in the best interests of making things better 
for both sides of that coin however I didn’t want to go wholesale and try and 
make everything switch at all at once. If you think it makes sense to drop 
devinabox and make the dividing line between Code and not code (although I’d 
argue that line is already crossed with other code things already being on 
github) that’s fine with me. Or I can expand the scope if people think that 
makes more sense in the PEP too.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Nathaniel Smith]

On Sun, Nov 30, 2014 at 2:54 AM, Guido van Rossum  wrote:
> All the use cases seem to be about adding some kind of getattr hook to
> modules. They all seem to involve modifying the CPython C code anyway. So
> why not tackle that problem head-on and modify module_getattro() to look for
> a global named __getattr__ and if it exists, call that instead of raising
> AttributeError?

You need to allow overriding __dir__ as well for tab-completion, and
some people wanted to use the properties API instead of raw
__getattr__, etc. Maybe someone will want __getattribute__ semantics,
I dunno. So since we're *so close* to being able to just use the
subclassing machinery, it seemed cleaner to try and get that working
instead of reimplementing bits of it piecewise.

That said, __getattr__ + __dir__ would be enough for my immediate use cases.

-n

> On Sat, Nov 29, 2014 at 11:37 AM, Nathaniel Smith  wrote:
>>
>> On Sat, Nov 29, 2014 at 4:21 AM, Guido van Rossum 
>> wrote:
>> > Are these really all our options? All of them sound like hacks, none of
>> > them
>> > sound like anything the language (or even the CPython implementation)
>> > should
>> > sanction. Have I missed the discussion where the use cases and
>> > constraints
>> > were analyzed and all other approaches were rejected? (I might have some
>> > half-baked ideas, but I feel I should read up on the past discussion
>> > first,
>> > and they are probably more fit for python-ideas than for python-dev.
>> > Plus
>> > I'm just writing this email because I'm procrastinating on the type
>> > hinting
>> > PEP. :-)
>>
>> The previous discussions I was referring to are here:
>>   http://thread.gmane.org/gmane.comp.python.ideas/29487/focus=29555
>>   http://thread.gmane.org/gmane.comp.python.ideas/29788
>>
>> There might well be other options; these are just the best ones I
>> could think of :-). The constraints are pretty tight, though:
>> - The "new module" object (whatever it is) should have a __dict__ that
>> aliases the original module globals(). I can elaborate on this if my
>> original email wasn't enough, but hopefully it's obvious that making
>> two copies of the same namespace and then trying to keep them in sync
>> at the very least smells bad :-).
>> - The "new module" object has to be a subtype of ModuleType, b/c there
>> are lots of places that do isinstance(x, ModuleType) checks (notably
>> -- but not only -- reload()). Since a major goal here is to make it
>> possible to do cleaner deprecations, it would be really unfortunate if
>> switching an existing package to use the metamodule support itself
>> broke things :-).
>> - Lookups in the normal case should have no additional performance
>> overhead, because module lookups are extremely extremely common. (So
>> this rules out dict proxies and tricks like that -- we really need
>> 'new_module.__dict__ is globals()' to be true.)
>>
>> AFAICT there are three logically possible strategies for satisfying
>> that first constraint:
>> (a) convert the original module object into the type we want, in-place
>> (b) create a new module object that acts like the original module object
>> (c) somehow arrange for our special type to be used from the start
>>
>> My options 1 and 2 are means of accomplishing (a), and my options 3
>> and 4 are means of accomplishing (b) while working around the
>> behavioural quirks of module objects (as required by the second
>> constraint).
>>
>> The python-ideas thread did also consider several methods of
>> implementing strategy (c), but they're messy enough that I left them
>> out here. The problem is that somehow we have to execute code to
>> create the new subtype *before* we have an entry in sys.modules for
>> the package that contains the code for the subtype. So one option
>> would be to add a new rule, that if a file pkgname/__new__.py exists,
>> then this is executed first and is required to set up
>> sys.modules["pkgname"] before we exec pkgname/__init__.py. So
>> pkgname/__new__.py might look like:
>>
>> import sys
>> from pkgname._metamodule import MyModuleSubtype
>> sys.modules[__name__] = MyModuleSubtype(__name__, docstring)
>>
>> This runs into a lot of problems though. To start with, the 'from
>> pkgname._metamodule ...' line is an infinite loop, b/c this is the
>> code used to create sys.modules["pkgname"]. It's not clear where the
>> globals dict for executing __new__.py comes from (who defines
>> __name__? Currently that's done by ModuleType.__init__). It only works
>> for packages, not modules. The need to provide the docstring here,
>> before __init__.py is even read, is weird. It adds extra stat() calls
>> to every package lookup. And, the biggest showstopper IMHO: AFAICT
>> it's impossible to write a polyfill to support this code on old python
>> versions, so it's useless to any package which needs to keep
>> compatibility with 2.7 (or even 3.4). Sure, you can backport the whole
>> import system like importlib2, but telling everyone that they need to
>>

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Brett Cannon]

On Sun Nov 30 2014 at 2:16:18 PM Guido van Rossum  wrote:

> On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon  wrote:
>>
>>  On Sat, Nov 29, 2014, 21:55 Guido van Rossum  wrote:
>>
>> All the use cases seem to be about adding some kind of getattr hook to
>> modules. They all seem to involve modifying the CPython C code anyway. So
>> why not tackle that problem head-on and modify module_getattro() to look
>> for a global named __getattr__ and if it exists, call that instead of
>> raising AttributeError?
>>
>>  Not sure if anyone thought of it. :) Seems like a reasonable solution
>> to me. Be curious to know what the benchmark suite said the impact was.
>>
> Why would there be any impact? The __getattr__ hook would be similar to
> the one on classes -- it's only invoked at the point where otherwise
> AttributeError would be raised.
>

You're right. My brain was thinking __getattribute__ semantics for some
reason.

>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ethan Furman]

On 11/30/2014 10:05 AM, Guido van Rossum wrote:

> Python has a long history (all the way back to my choice of a MIT-style 
> license for the first release) of mixing "free"
> and "non-free" uses and tools -- for example on Windows we consciously chose 
> to align ourselves with the platform
> tooling rather than with the (minority) free tools available, Python has been 
> ported to many commercial platforms, and
> I've always encouraged use of Python in closed-source situations.

For this I am grateful, and agree with.

> Finally. And this may actually be the most important point. Python people 
> should be doing stuff that makes Python better
> (both taken in the most inclusive way possible). For stuff that's not unique 
> to Python but can be used by many other
> open-source projects, such as compilers, DVCS tools, or mailing lists, we 
> should not be wasting our precious time on
> building and maintaining our own tools or administering the servers on which 
> they run. And historically we've not done a
> great job on maintenance and administration.

My issues with GitHub range from selfish to philosophical:

  - (selfish) I don't want to learn git

  - (practical) from what I hear git can have issues with losing history -- in a
project that has many volunteer and part-time developers, using a tool that
can munge your data just doesn't seem very wise

  - (practical) supporting git and hg means learning two different workflows

  - (philosophical) in a commercial world we vote with our dollars (don't like 
how
a company behaves?  don't buy their product); in an OSS world we vote by 
whose
services/software we use;  I don't want to support, or appear to support, a
company that is abusive and sexist towards its employees:  it is not what 
the
PSF supports, and it's definitely not what I support.

Not everyone is suited to demonstrate in the streets, but it shouldn't be that 
hard to not use a company with
acknowledged bad practices.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Ethan Furman]

On 11/30/2014 11:15 AM, Guido van Rossum wrote:
> On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote: 
>> On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:
>>> 
>>> All the use cases seem to be about adding some kind of getattr hook
>>> to modules. They all seem to involve modifying the CPython C code
>>> anyway. So why not tackle that problem head-on and modify module_getattro()
>>> to look for a global named __getattr__ and if it exists, call that instead
>>> of raising AttributeError?
>> 
>> Not sure if anyone thought of it. :) Seems like a reasonable solution to me.
>> Be curious to know what the benchmark suite said the impact was.
> 
> Why would there be any impact? The __getattr__ hook would be similar to the
> one on classes -- it's only invoked at the point where otherwise 
> AttributeError
> would be raised.

I think the bigger question is how do we support it back on 2.7?

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Brett Cannon]

On Sun Nov 30 2014 at 12:00:20 PM Donald Stufft  wrote:

>
> On Nov 30, 2014, at 11:44 AM, Brett Cannon  wrote:
>
>
>
> On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco 
> wrote:
>
>> On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou 
>> wrote:
>> > On Sun, 30 Nov 2014 16:23:08 +1100
>> > Chris Angelico  wrote:
>> >>
>> >> Yes, GitHub is proprietary. But all of your actual code is stored in
>> >> git, which is free, and it's easy to push that to a new host somewhere
>> >> else, or create your own host. This proposal is for repositories that
>> >> don't need much in the way of issue trackers etc, so shifting away
>> >> from GitHub shouldn't demand anything beyond moving the repos
>> >> themselves.
>> >
>> > I hope we're not proposing to move the issue trackers to github,
>> > otherwise I'm -1 on this PEP.
>> >
>> > Regards
>> >
>> > Antoine.
>>
>> So I usually choose not to weigh in on discussions like these but
>> there seems to be a lot of misdirection in some of these arguments.
>>
>> To start, I'm generally neutral about this proposal or Nick's proposal
>> that spurred this one. I've found the most frustrating part of
>> contributing to anything involving CPython to be the lack of reviewer
>> time. I have had very small (2-line) patches take months (close to a
>> year in reality) to get through in spite of periodically pinging the
>> appropriate people. Moving to git/GitHub will not alleviate this at
>> all.
>>
>> To be clear, the main reasoning behind Nick's was being able to submit
>> changes without ever having to have a local copy of the repository in
>> question on your machine. Having a complete web workflow for editing
>> and contributing makes the barrier to entry far lower than switching
>> VCS or anything else. BitBucket (apparently, although I've never used
>> this) and GitHub both have this capability and *both* are
>> free-as-in-beer systems.
>>
>> No one as I understand it is proposing that we use the per-distro
>> proprietary interface to these websites.
>>
>> All data can be removed from GitHub using it's API and can generally
>> be converted to another platform. The same goes for BitBucket although
>> it's arguably easier to retrieve issue data from BitBucket than
>> GitHub. That said, *the issue tracker is not covered by these
>> proposals* so this is a moot point. Drop it already.
>>
>> If we're seriously considering moving to git as a DVCS, we should
>> consider the real free-as-in-freedom alternatives that come very close
>> to GitHub and can be improved by us (even though they're not written
>> in Python). One of those is GitLab. We can self-host a GitLab instance
>> easily or we can rely on gitlab.com. GitLab aims to provide a very
>> similar user experience to GitHub and it's slowly approaching feature
>> parity and experience parity. GitLab is also what a lot of people
>> chose to switch to after the incident Steven mentioned, which I don't
>> think is something we should dismiss or ignore.
>>
>> We should refocus the discussion with the following in mind:
>>
>> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
>> tools to do it and the only cost is the time it would take to monitor
>> the process
>>
>> - GitHub has a toxic company culture that we should be aware of before
>> moving to it. They have a couple blog posts about attempting to change
>> it but employees became eerily silent after the incident and have
>> remained so from what I've personally seen.
>>
>> - GitHub may be popular but there are popular FOSS solutions that
>> exist that we can also self-host at something like forge.python.org
>>
>> - bugs.python.org is not covered by any of these proposals
>>
>> - The main benefit this proposal (and the proposal to move to
>> BitBucket) are seeking to achieve is an online editing experience
>> allowing for *anyone with a browser and an account* to contribute.
>> This to me is the only reason I would be +1 for either of these
>> proposals (if we can reach consensus).
>>
>
> But that's not just it. As you pointed out, Ian, getting patch submissions
> committed faster would be a huge improvement over what we have today.
> GitHub/Bitbucket/whatever could help with this by giving core devs basic CI
> to know that I patch is sound to some extent as well as push button commits
> of patches.
>
> For me personally, if I knew a simple patch integrated cleanly and passed
> on at least one buildbot -- when it wasn't a platform-specific fix -- then
> I could easily push a "Commit" button and be done with it (although this
> assumes single branch committing; doing this across branches makes all of
> this difficult unless we finally resolve our Misc/NEWS conflict issues so
> that in some instances it can be automated). Instead I have to wait until I
> have a clone I can push from, download a patch, apply it, run the unit
> tests myself, do the commit, and then repeat a subset of that to whatever
> branches make sense. It's a lot of work for which some things could be
> au

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon  wrote:
>
>  On Sat, Nov 29, 2014, 21:55 Guido van Rossum  wrote:
>
> All the use cases seem to be about adding some kind of getattr hook to
> modules. They all seem to involve modifying the CPython C code anyway. So
> why not tackle that problem head-on and modify module_getattro() to look
> for a global named __getattr__ and if it exists, call that instead of
> raising AttributeError?
>
>  Not sure if anyone thought of it. :) Seems like a reasonable solution to
> me. Be curious to know what the benchmark suite said the impact was.
>
Why would there be any impact? The __getattr__ hook would be similar to the
one on classes -- it's only invoked at the point where otherwise
AttributeError would be raised.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 1:05 PM, Guido van Rossum  wrote:
> 
> I don't feel it's my job to accept or reject this PEP, but I do have an 
> opinion.

So here’s a question. If it’s not your job to accept or reject this PEP, whose 
is it? This is probably an issue we’re never going to get actual consensus on 
so unless there is an arbitrator of who gets to decide I feel it’s probably a 
waste of my time to try and convince absolutely *everyone*.

> 
> The scope of the PSF organization is far beyond just the Python language -- 
> it includes the Python developer community, the Python user community, 3rd 
> party Python packages and their communities (even if some have created their 
> own organizations). But I think that it is "scope creep" to try and be "pure" 
> in our tooling or workflows.
> 
> Python has a long history (all the way back to my choice of a MIT-style 
> license for the first release) of mixing "free" and "non-free" uses and tools 
> -- for example on Windows we consciously chose to align ourselves with the 
> platform tooling rather than with the (minority) free tools available, Python 
> has been ported to many commercial platforms, and I've always encouraged use 
> of Python in closed-source situations.
> 
> I bring this up to emphasize that (unlike GNU software and the FSF) Python 
> has no additional hidden agenda of bringing freedom to all software. At least 
> that's how I feel about it -- clearly some of the most vocal contributors to 
> this thread feel differently.

Yes, this is how I feel about Python too, that it’s the pragmatic 
language/community not the purist language/community. 

> 
> Now some entirely practical points.
> 
> - I am basically the only remaining active PEP editor, so I see most PEP 
> contributions by non-core-committers. Almost all of these uses github. Not 
> bitbucket, not some other git host, but github. I spend a fair amount of time 
> applying patches. It would most definitely be easier if I could get them to 
> send me pull requests.
> 
> - I am not worried about "lock in". The most important stuff is copied in the 
> local git repos of hundreds of core devs and thousands of others. Pull 
> requests are by nature short-lived -- and if you really want a history of the 
> back-and-forth that led to the eventual set of diffs that was integrated, you 
> could subscribe a mailing list to it to archive it. I'm sure there's a way to 
> back up the issue tracker too.
> 
> Finally. And this may actually be the most important point. Python people 
> should be doing stuff that makes Python better (both taken in the most 
> inclusive way possible). For stuff that's not unique to Python but can be 
> used by many other open-source projects, such as compilers, DVCS tools, or 
> mailing lists, we should not be wasting our precious time on building and 
> maintaining our own tools or administering the servers on which they run. And 
> historically we've not done a great job on maintenance and administration.
> 
> Of course it's fun to make tools in Python, and to see them used beyond the 
> Python world. But that's an entirely different argument from what I hear.
> 
> -- 
> --Guido van Rossum (python.org/~guido )

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ian Cordasco]

Can this discussion be split off into a separate discussion. It's
tangential to the PEP and clearly not actively progressing so it
doesn't seem productive. I don't care where it's taken, but I don't
think this belongs here. Speculation on the actions of the msysgit
project are not fair talk for this PEP.

On Sun, Nov 30, 2014 at 12:14 PM, Paul Moore  wrote:
> On 30 November 2014 at 16:08, Donald Stufft  wrote:
>>> On Nov 30, 2014, at 7:31 AM, Paul Moore  wrote:
>>>
>>> On 29 November 2014 at 23:27, Donald Stufft  wrote:
 In previous years there was concern about how well supported git was on 
 Windows
 in comparison to Mercurial. However git has grown to support Windows as a 
 first
 class citizen. In addition to that, for Windows users who are not well 
 aquanted
 with the Windows command line there are GUI options as well.
>>>
>>> I have little opinion on the PEP as a whole, but is the above
>>> statement true? From the git website, version 2.2.0 is current, and
>>> yet the downloadable Windows version is still 1.9.4. That's a fairly
>>> significant version lag for a "first class citizen".
>>>
>>> I like git, and it has a number of windows-specific extensions that
>>> are really useful (more than Mercurial, AFAIK), but I wouldn't say
>>> that the core product supported Windows on an equal footing to Linux.
>>>
>>> Paul
>>
>> I think so yes. I may be wrong, however while 1.9.4 may be the latest
>> downloadable version of git for Windows, there is no downloadable
>> version of the Linux clients at all, they just tell you to go use
>> your package manager which for instance is version 1.7 on Debian. On
>> OS X the latest version is 2.0.1.
>
> OTOH, presumably you can build your own copy of git from source on
> Linux/OSX. I haven't tried this on Windows but it looks pretty
> difficult (you start by downloading the msysgit development
> environment and go from there). Also, if it's easy to produce binaries
> for 2.2.0 on Windows, why haven't the msysgit project (still an
> external project, to an extent, AFAICT) done so?
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Paul Moore]

On 30 November 2014 at 16:08, Donald Stufft  wrote:
>> On Nov 30, 2014, at 7:31 AM, Paul Moore  wrote:
>>
>> On 29 November 2014 at 23:27, Donald Stufft  wrote:
>>> In previous years there was concern about how well supported git was on 
>>> Windows
>>> in comparison to Mercurial. However git has grown to support Windows as a 
>>> first
>>> class citizen. In addition to that, for Windows users who are not well 
>>> aquanted
>>> with the Windows command line there are GUI options as well.
>>
>> I have little opinion on the PEP as a whole, but is the above
>> statement true? From the git website, version 2.2.0 is current, and
>> yet the downloadable Windows version is still 1.9.4. That's a fairly
>> significant version lag for a "first class citizen".
>>
>> I like git, and it has a number of windows-specific extensions that
>> are really useful (more than Mercurial, AFAIK), but I wouldn't say
>> that the core product supported Windows on an equal footing to Linux.
>>
>> Paul
>
> I think so yes. I may be wrong, however while 1.9.4 may be the latest
> downloadable version of git for Windows, there is no downloadable
> version of the Linux clients at all, they just tell you to go use
> your package manager which for instance is version 1.7 on Debian. On
> OS X the latest version is 2.0.1.

OTOH, presumably you can build your own copy of git from source on
Linux/OSX. I haven't tried this on Windows but it looks pretty
difficult (you start by downloading the msysgit development
environment and go from there). Also, if it's easy to produce binaries
for 2.2.0 on Windows, why haven't the msysgit project (still an
external project, to an extent, AFAICT) done so?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Guido van Rossum]

I don't feel it's my job to accept or reject this PEP, but I do have an
opinion.

The scope of the PSF organization is far beyond just the Python language --
it includes the Python developer community, the Python user community, 3rd
party Python packages and their communities (even if some have created
their own organizations). But I think that it is "scope creep" to try and
be "pure" in our tooling or workflows.

Python has a long history (all the way back to my choice of a MIT-style
license for the first release) of mixing "free" and "non-free" uses and
tools -- for example on Windows we consciously chose to align ourselves
with the platform tooling rather than with the (minority) free tools
available, Python has been ported to many commercial platforms, and I've
always encouraged use of Python in closed-source situations.

I bring this up to emphasize that (unlike GNU software and the FSF) Python
has no additional hidden agenda of bringing freedom to all software. At
least that's how I feel about it -- clearly some of the most vocal
contributors to this thread feel differently.

Now some entirely practical points.

- I am basically the only remaining active PEP editor, so I see most PEP
contributions by non-core-committers. Almost all of these uses github. Not
bitbucket, not some other git host, but github. I spend a fair amount of
time applying patches. It would most definitely be easier if I could get
them to send me pull requests.

- I am not worried about "lock in". The most important stuff is copied in
the local git repos of hundreds of core devs and thousands of others. Pull
requests are by nature short-lived -- and if you really want a history of
the back-and-forth that led to the eventual set of diffs that was
integrated, you could subscribe a mailing list to it to archive it. I'm
sure there's a way to back up the issue tracker too.

Finally. And this may actually be the most important point. Python people
should be doing stuff that makes Python better (both taken in the most
inclusive way possible). For stuff that's not unique to Python but can be
used by many other open-source projects, such as compilers, DVCS tools, or
mailing lists, we should not be wasting our precious time on building and
maintaining our own tools or administering the servers on which they run.
And historically we've not done a great job on maintenance and
administration.

Of course it's fun to make tools in Python, and to see them used beyond the
Python world. But that's an entirely different argument from what I hear.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Glyph]

> On Nov 30, 2014, at 11:17, Chris Angelico  wrote:
> 
> On Sun, Nov 30, 2014 at 8:54 PM, Nick Coghlan  wrote:
>> On 30 November 2014 at 15:23, Chris Angelico  wrote:
>>> Python is already using quite a bit of non-free software in its
>>> ecosystem. The Windows builds of CPython are made with Microsoft's
>>> compiler, and the recent discussion about shifting to Cygwin or MinGW
>>> basically boiled down to "but it ought to be free software", and that
>>> was considered not a sufficiently strong argument. In each case, the
>>> decision has impact on other people (using MSVC for the official
>>> python.org installers means extension writers need to use MSVC too;
>>> and using GitHub means that contributors are strongly encouraged,
>>> possibly required, to use GitHub); so why is it acceptable to use a
>>> non-free compiler, but not acceptable to use a non-free host?
>> 
>> Relying on non-free software to support users of a non-free platform
>> is rather different from *requiring* the use of non-free software to
>> participate in core Python community design processes.
> 
> But what non-free software is required to use the community design
> processes? The GitHub client is entirely optional; I don't use it, I
> just use git itself. Using a free client to access a proprietary
> server isn't the same as using non-free software.

Also keep in mind that unless you are using a very esoteric hardware setup and 
dedicated leased lines that you purchased yourself, you are likely to be using 
copyrighted, patented, proprietary software at a number of levels:

the microcode implementation in your CPU
the firmware in your GPU
the firmware in your network card
the boot code (e.g.: BIOS or EFI implementation) of your motherboard
the firmware in your router
or the firmware in your cable or DSL modem, if you thought to get a free router 
with OpenWRT or something in it
the firmware in your ISP's router
the firmware in the backbone's routers
the firmware in the PSF's ISP's routers

Does this sound like ridiculous nitpicking?  Of course it does!  If you refused 
to use all that stuff you just wouldn't be able to access the internet at all, 
regardless of your personal choices.  Most layers of this stack are _less_ 
constrained to open standards and open data access than Github, and can require 
layers and layers of additional proprietary software or reverse engineering 
(ask anyone who has tried to play a video game on a Linux computer what the 
experience is like without gobs of proprietary blobs from nvidia or ATI).

And as the story of BitKeeper shows, if a proprietary platform decides to do 
something bad, if the cost of migration is within your means, you can just 
leave.  This is far more true of Github than Bitkeeper - Linux had to create a 
totally new VCS to migrate off of that, we just have to install Trac or Roundup 
or something again. (Which, as per the presently-proposed PEP, we don't even 
have to install them again, just change some configuration to put a few 
repositories back.)

The monoculture about Github concerns me.  I also have concerns about the 
long-term consequences of not having an all-free-software stack.  But focusing 
on avoiding services like Github at this point in history is just a gigantic 
waste of time; it's resolving dependencies in the wrong order.

The only reason to avoid Github is ideological purity, and even then it's not 
even "purity" because you still have to accept this other ideological 
contamination.  Except, what about other ideological concepts that are 
important to the Python core developers, like equitable representation for 
women and minorities, and compassionate and polite interactions within the 
community?  Clearly we can't require the use of Linux.  If we treat these 
ideals as an equivalent priority as free software (even if we ignore the 
"invisible" dependencies like the list I just made above), then there is 
literally no software left that we can use to develop Python. Linux kernel and 
GNU low-level user-land development are a perpetual cesspit, and their leaders 
serve as a continuous embarrassment to our community.

And speaking of equitable representation, one proven technique we have learned 
for dealing with that problem is making it for newcomers of all stripes to 
access the community.  Like it or not, Github's popularity means that it's a 
place where most newcomers to programming are learning to use version control 
and bug tracking.  This is a network effect that can have a real impact on 
people's lives.  Requiring newcomers to learn our weird technology religion 
before they can contribute creates a real barrier to entry, which in turn makes 
our community more insular and homogenous.

Some days you get the Git, and some days the Github gets you.  The sooner we, 
as a community and a culture, can accept this and move on, the more time will 
be available to actually build replacements for these things.

-glyph

__

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ethan Furman]

On 11/29/2014 10:14 PM, Demian Brecht wrote:
> 
> On Sat, Nov 29, 2014 at 3:27 PM, Donald Stufft  > wrote:
>> As promised in the "Move selected documentation repos to PSF BitBucket
>> account?" thread I've written up a PEP for moving selected repositories from
>> hg.python.org  to Github.
> 
> 
> FWIW, I'm a pretty solid -1 to this PEP.

Agreed.

-1 for the same reasons (plus most of Nick's).

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 11:30 AM, Donald Stufft  wrote:
> 
> Comments like this make me feel like I didn’t explain myself very well in the
> PEP.


It’s been pointed out to me that Mercurial bookmarks have been core since 1.8
and since I felt like the technical arguments were really secondary to the 
social
arguments I’ve gone ahead and removed that section from the PEP.

I’ve also added the read only Mercurial repository on hg.python.org 


See changes at https://hg.python.org/peps/rev/d31fe28e2766

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ian Cordasco]

On Nov 30, 2014 11:09 AM, "Donald Stufft"  wrote:
>
>
> > On Nov 30, 2014, at 11:55 AM, Barry Warsaw  wrote:
> >
> > On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
> >
> >> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
> >> tools to do it and the only cost is the time it would take to monitor
> >> the process
> >
> > *Extracting* data may be easy, but migrating it is a different story.
As the
> > Mailman project has seen in trying to migrate from Confluence to Moin,
there
> > is a ton of very difficult work involved after extracting the data.
Parsing
> > the data, ensuring that you have all the bits you need, fitting it into
the
> > new system's schema, working out the edge cases, adapting to semantic
> > differences and gaps, ensuring that all the old links are redirected,
and so
> > on, were all exceedingly difficult[*].
> >
> > Even converting between two FLOSS tools is an amazing amount of work.
Look at
> > what Eric Raymond did with reposurgeon to convert from Bazaar to git.
>
> I fail to see how this is a reasonable argument to make at all since, as
you
> mentioned, converting between two FLOSS tools can be an amazing amount of
work.
> Realistically the amount of work is going to be predicated on whether or
not
> there is a tool that already handles the conversion for you. Assuming of
course
> that the data is available to you at all.
>
> As a particularly relevant example, switching from Mercurial to Git is as
easy
> as installing hg-git, creating a bookmark for master that tracks default,
and
> then pushing to a git repository.
>
> >
> > It's a good thing that your data isn't locked behind a proprietary
door, for
> > now.  That's only part of the story.  But also, because github is a
closed
> > system, there's no guarantee that today's data-freeing APIs will still
exist,
> > continue to be functional for practical purposes, remain complete, or
stay at
> > parity with new features.
>
> This feels like Chicken Little-ing. If Github closed it’s APIs then you
could
> still get at that data by scraping the web interface. However why would
Github
> do that? That would piss off the vast majority of OSS projects who are
currently
> hosted there and is likely to cause a pretty big migration off of Github
for
> fear that Github is going to attempt to lock people onto Github. The
popularity
> of Github *is* Github’s killer feature and doing something that is going
to
> send the vast bulk of your users running for the hills sounds like
something that
> they would have to be particularly stupid to do.
>
> >
> > Cheers,
> > -Barry
> >
> > [*] And our huge gratitude goes to Paul Boddie for his amazing amount
of work
> > on the project.
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

I tend to agree with Donald that it's highly unlikely for GitHub to close
off their APIs but Barry is right that it isn't impossible. That can be
mitigated by regularly scheduling a back-up of that data using the tools we
have now so that the sky doesn't appear to be falling if the worst case
does occur.

I also tend to disagree with Barry that it will be extraordinarily
difficult because I have migrated issues and pull requests off of GitHub
and was able to automate the entirety of it reliably with python.
Admittedly, I'm very familiar with GitHub's API as the author of github3.py
so for me this is a trivial task. I would also be willing to help set up
migrations and back ups if we decide to use GitHub.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 12:09 PM, Donald Stufft  wrote:
> 
>> 
>> On Nov 30, 2014, at 11:55 AM, Barry Warsaw  wrote:
>> 
>> On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
>> 
>>> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
>>> tools to do it and the only cost is the time it would take to monitor
>>> the process
>> 
>> *Extracting* data may be easy, but migrating it is a different story.  As the
>> Mailman project has seen in trying to migrate from Confluence to Moin, there
>> is a ton of very difficult work involved after extracting the data.  Parsing
>> the data, ensuring that you have all the bits you need, fitting it into the
>> new system's schema, working out the edge cases, adapting to semantic
>> differences and gaps, ensuring that all the old links are redirected, and so
>> on, were all exceedingly difficult[*].
>> 
>> Even converting between two FLOSS tools is an amazing amount of work.  Look 
>> at
>> what Eric Raymond did with reposurgeon to convert from Bazaar to git.
> 
> I fail to see how this is a reasonable argument to make at all since, as you
> mentioned, converting between two FLOSS tools can be an amazing amount of 
> work.
> Realistically the amount of work is going to be predicated on whether or not
> there is a tool that already handles the conversion for you. Assuming of 
> course
> that the data is available to you at all.
> 
> As a particularly relevant example, switching from Mercurial to Git is as easy
> as installing hg-git, creating a bookmark for master that tracks default, and
> then pushing to a git repository.

When looking for a tool that did this (specifically Github -> Gitlab because the
two are most similar) I found
https://gitlab.com/sigmavirus24/issues-migration/blob/master/migrate.py
which happens to be written by Ian. I would guess that he is likely speaking 
from
experience about migrating off of Github to go to Gitlab.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 11:55 AM, Barry Warsaw  wrote:
> 
> On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
> 
>> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
>> tools to do it and the only cost is the time it would take to monitor
>> the process
> 
> *Extracting* data may be easy, but migrating it is a different story.  As the
> Mailman project has seen in trying to migrate from Confluence to Moin, there
> is a ton of very difficult work involved after extracting the data.  Parsing
> the data, ensuring that you have all the bits you need, fitting it into the
> new system's schema, working out the edge cases, adapting to semantic
> differences and gaps, ensuring that all the old links are redirected, and so
> on, were all exceedingly difficult[*].
> 
> Even converting between two FLOSS tools is an amazing amount of work.  Look at
> what Eric Raymond did with reposurgeon to convert from Bazaar to git.

I fail to see how this is a reasonable argument to make at all since, as you
mentioned, converting between two FLOSS tools can be an amazing amount of work.
Realistically the amount of work is going to be predicated on whether or not
there is a tool that already handles the conversion for you. Assuming of course
that the data is available to you at all.

As a particularly relevant example, switching from Mercurial to Git is as easy
as installing hg-git, creating a bookmark for master that tracks default, and
then pushing to a git repository.

> 
> It's a good thing that your data isn't locked behind a proprietary door, for
> now.  That's only part of the story.  But also, because github is a closed
> system, there's no guarantee that today's data-freeing APIs will still exist,
> continue to be functional for practical purposes, remain complete, or stay at
> parity with new features.

This feels like Chicken Little-ing. If Github closed it’s APIs then you could
still get at that data by scraping the web interface. However why would Github
do that? That would piss off the vast majority of OSS projects who are currently
hosted there and is likely to cause a pretty big migration off of Github for
fear that Github is going to attempt to lock people onto Github. The popularity
of Github *is* Github’s killer feature and doing something that is going to
send the vast bulk of your users running for the hills sounds like something 
that
they would have to be particularly stupid to do.

> 
> Cheers,
> -Barry
> 
> [*] And our huge gratitude goes to Paul Boddie for his amazing amount of work
> on the project.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 11:44 AM, Brett Cannon  wrote:
> 
> 
> 
> On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco  > wrote:
> On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou  > wrote:
> > On Sun, 30 Nov 2014 16:23:08 +1100
> > Chris Angelico mailto:ros...@gmail.com>> wrote:
> >>
> >> Yes, GitHub is proprietary. But all of your actual code is stored in
> >> git, which is free, and it's easy to push that to a new host somewhere
> >> else, or create your own host. This proposal is for repositories that
> >> don't need much in the way of issue trackers etc, so shifting away
> >> from GitHub shouldn't demand anything beyond moving the repos
> >> themselves.
> >
> > I hope we're not proposing to move the issue trackers to github,
> > otherwise I'm -1 on this PEP.
> >
> > Regards
> >
> > Antoine.
> 
> So I usually choose not to weigh in on discussions like these but
> there seems to be a lot of misdirection in some of these arguments.
> 
> To start, I'm generally neutral about this proposal or Nick's proposal
> that spurred this one. I've found the most frustrating part of
> contributing to anything involving CPython to be the lack of reviewer
> time. I have had very small (2-line) patches take months (close to a
> year in reality) to get through in spite of periodically pinging the
> appropriate people. Moving to git/GitHub will not alleviate this at
> all.
> 
> To be clear, the main reasoning behind Nick's was being able to submit
> changes without ever having to have a local copy of the repository in
> question on your machine. Having a complete web workflow for editing
> and contributing makes the barrier to entry far lower than switching
> VCS or anything else. BitBucket (apparently, although I've never used
> this) and GitHub both have this capability and *both* are
> free-as-in-beer systems.
> 
> No one as I understand it is proposing that we use the per-distro
> proprietary interface to these websites.
> 
> All data can be removed from GitHub using it's API and can generally
> be converted to another platform. The same goes for BitBucket although
> it's arguably easier to retrieve issue data from BitBucket than
> GitHub. That said, *the issue tracker is not covered by these
> proposals* so this is a moot point. Drop it already.
> 
> If we're seriously considering moving to git as a DVCS, we should
> consider the real free-as-in-freedom alternatives that come very close
> to GitHub and can be improved by us (even though they're not written
> in Python). One of those is GitLab. We can self-host a GitLab instance
> easily or we can rely on gitlab.com . GitLab aims to 
> provide a very
> similar user experience to GitHub and it's slowly approaching feature
> parity and experience parity. GitLab is also what a lot of people
> chose to switch to after the incident Steven mentioned, which I don't
> think is something we should dismiss or ignore.
> 
> We should refocus the discussion with the following in mind:
> 
> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
> tools to do it and the only cost is the time it would take to monitor
> the process
> 
> - GitHub has a toxic company culture that we should be aware of before
> moving to it. They have a couple blog posts about attempting to change
> it but employees became eerily silent after the incident and have
> remained so from what I've personally seen.
> 
> - GitHub may be popular but there are popular FOSS solutions that
> exist that we can also self-host at something like forge.python.org 
> 
> 
> - bugs.python.org  is not covered by any of these 
> proposals
> 
> - The main benefit this proposal (and the proposal to move to
> BitBucket) are seeking to achieve is an online editing experience
> allowing for *anyone with a browser and an account* to contribute.
> This to me is the only reason I would be +1 for either of these
> proposals (if we can reach consensus).
> 
> But that's not just it. As you pointed out, Ian, getting patch submissions 
> committed faster would be a huge improvement over what we have today. 
> GitHub/Bitbucket/whatever could help with this by giving core devs basic CI 
> to know that I patch is sound to some extent as well as push button commits 
> of patches.
> 
> For me personally, if I knew a simple patch integrated cleanly and passed on 
> at least one buildbot -- when it wasn't a platform-specific fix -- then I 
> could easily push a "Commit" button and be done with it (although this 
> assumes single branch committing; doing this across branches makes all of 
> this difficult unless we finally resolve our Misc/NEWS conflict issues so 
> that in some instances it can be automated). Instead I have to wait until I 
> have a clone I can push from, download a patch, apply it, run the unit tests 
> myself, do the commit, and then repeat a subset of that to whatever branches 
> make se

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Barry Warsaw]

On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:

>- Migrating "data" from GitHub is easy. There are free-as-in-freedom
>tools to do it and the only cost is the time it would take to monitor
>the process

*Extracting* data may be easy, but migrating it is a different story.  As the
Mailman project has seen in trying to migrate from Confluence to Moin, there
is a ton of very difficult work involved after extracting the data.  Parsing
the data, ensuring that you have all the bits you need, fitting it into the
new system's schema, working out the edge cases, adapting to semantic
differences and gaps, ensuring that all the old links are redirected, and so
on, were all exceedingly difficult[*].

Even converting between two FLOSS tools is an amazing amount of work.  Look at
what Eric Raymond did with reposurgeon to convert from Bazaar to git.

It's a good thing that your data isn't locked behind a proprietary door, for
now.  That's only part of the story.  But also, because github is a closed
system, there's no guarantee that today's data-freeing APIs will still exist,
continue to be functional for practical purposes, remain complete, or stay at
parity with new features.

Cheers,
-Barry

[*] And our huge gratitude goes to Paul Boddie for his amazing amount of work
on the project.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Benjamin Peterson]

On Sun, Nov 30, 2014, at 11:45, Donald Stufft wrote:
> 
> > On Nov 30, 2014, at 11:28 AM, Brett Cannon  wrote:
> > 
> > 
> > 
> > On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor  > > wrote:
> > Donald Stufft  stufft.io > writes:
> > 
> > >
> > > [words words words]
> > >
> > 
> > I strongly support this PEP. I'd like to share two pieces of information. 
> > Both
> > of these are personal anecdotes:
> > 
> > For the past several years, I've been a contributor on two major projects 
> > using
> > mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching,
> > basically all contributors regularly make branches in the main repo for 
> > their
> > work, and we're very free in giving people commit rights, so almost everyone
> > working on PyPy in any way has this level of access. This workflow works 
> > ok. I
> > don't love it as much as git, but it's fine, it's not an impediment to my 
> > work.
> > 
> > By contrast, CPython does not have this type of workflow, there are almost 
> > no
> > in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg
> > user for years, I have no idea how to create a local-only branch, or a 
> > branch
> > which is pushed to a remote (to use the git term). I also don't generally
> > commit my own work to CPython, even though I have push privledges,
> > because I
> > prefer to *always* get code review on my work. As a result, I use a git 
> > mirror
> > of CPython to do all my work, and generate patches from that.
> > 
> > The conclusion I draw from this is that hg's workflow is probably fine if
> > you're a committer on the project, or don't ever need to maintain multiple
> > patches concurrently (and thus can just leave everything uncommitted in the
> > repo). However, the hg workflow seems extremely defficient at non-committer
> > contributors.
> > 
> > One way to come close to that using hg is to have your own clone that you 
> > never push to hg.python.org/cpython  (e.g. 
> > cloning the Bitbucket clone of cpython or hosting on hg.python.org 
> >  a personal clone). You can then specify the repo 
> > and branch on the issue tracker to generate your patch: 
> > https://docs.python.org/devguide/triaging.html#mercurial-repository 
> >  . 
> > After that it's just like any other patch workflow for core devs. It's not 
> > quite as nice as maybe using named branches where you can just do a final 
> > hg merge/commit to get your changes committed, but if you're not going to 
> > commit your branches then you might as well get the automatic patch 
> > generation perk in the issue tracker rather than using git (unless there is 
> > some other git feature you use that you can't get in hg).
> 
> This doesn’t really work in a Pull Request workflow though I think is the
> point.
> Uploading patches is it’s own kind of terrible but yes if you’re just
> uploading
> patches you can probably do that. I don’t make branches in Mercurial
> because
> i’m afraid I’m going to push a permanent branch to hg.python.org
>  and screw
> something up.

Don't worry; we have a hook for that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Brett Cannon]

On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco 
wrote:

> On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou 
> wrote:
> > On Sun, 30 Nov 2014 16:23:08 +1100
> > Chris Angelico  wrote:
> >>
> >> Yes, GitHub is proprietary. But all of your actual code is stored in
> >> git, which is free, and it's easy to push that to a new host somewhere
> >> else, or create your own host. This proposal is for repositories that
> >> don't need much in the way of issue trackers etc, so shifting away
> >> from GitHub shouldn't demand anything beyond moving the repos
> >> themselves.
> >
> > I hope we're not proposing to move the issue trackers to github,
> > otherwise I'm -1 on this PEP.
> >
> > Regards
> >
> > Antoine.
>
> So I usually choose not to weigh in on discussions like these but
> there seems to be a lot of misdirection in some of these arguments.
>
> To start, I'm generally neutral about this proposal or Nick's proposal
> that spurred this one. I've found the most frustrating part of
> contributing to anything involving CPython to be the lack of reviewer
> time. I have had very small (2-line) patches take months (close to a
> year in reality) to get through in spite of periodically pinging the
> appropriate people. Moving to git/GitHub will not alleviate this at
> all.
>
> To be clear, the main reasoning behind Nick's was being able to submit
> changes without ever having to have a local copy of the repository in
> question on your machine. Having a complete web workflow for editing
> and contributing makes the barrier to entry far lower than switching
> VCS or anything else. BitBucket (apparently, although I've never used
> this) and GitHub both have this capability and *both* are
> free-as-in-beer systems.
>
> No one as I understand it is proposing that we use the per-distro
> proprietary interface to these websites.
>
> All data can be removed from GitHub using it's API and can generally
> be converted to another platform. The same goes for BitBucket although
> it's arguably easier to retrieve issue data from BitBucket than
> GitHub. That said, *the issue tracker is not covered by these
> proposals* so this is a moot point. Drop it already.
>
> If we're seriously considering moving to git as a DVCS, we should
> consider the real free-as-in-freedom alternatives that come very close
> to GitHub and can be improved by us (even though they're not written
> in Python). One of those is GitLab. We can self-host a GitLab instance
> easily or we can rely on gitlab.com. GitLab aims to provide a very
> similar user experience to GitHub and it's slowly approaching feature
> parity and experience parity. GitLab is also what a lot of people
> chose to switch to after the incident Steven mentioned, which I don't
> think is something we should dismiss or ignore.
>
> We should refocus the discussion with the following in mind:
>
> - Migrating "data" from GitHub is easy. There are free-as-in-freedom
> tools to do it and the only cost is the time it would take to monitor
> the process
>
> - GitHub has a toxic company culture that we should be aware of before
> moving to it. They have a couple blog posts about attempting to change
> it but employees became eerily silent after the incident and have
> remained so from what I've personally seen.
>
> - GitHub may be popular but there are popular FOSS solutions that
> exist that we can also self-host at something like forge.python.org
>
> - bugs.python.org is not covered by any of these proposals
>
> - The main benefit this proposal (and the proposal to move to
> BitBucket) are seeking to achieve is an online editing experience
> allowing for *anyone with a browser and an account* to contribute.
> This to me is the only reason I would be +1 for either of these
> proposals (if we can reach consensus).
>

But that's not just it. As you pointed out, Ian, getting patch submissions
committed faster would be a huge improvement over what we have today.
GitHub/Bitbucket/whatever could help with this by giving core devs basic CI
to know that I patch is sound to some extent as well as push button commits
of patches.

For me personally, if I knew a simple patch integrated cleanly and passed
on at least one buildbot -- when it wasn't a platform-specific fix -- then
I could easily push a "Commit" button and be done with it (although this
assumes single branch committing; doing this across branches makes all of
this difficult unless we finally resolve our Misc/NEWS conflict issues so
that in some instances it can be automated). Instead I have to wait until I
have a clone I can push from, download a patch, apply it, run the unit
tests myself, do the commit, and then repeat a subset of that to whatever
branches make sense. It's a lot of work for which some things could be
automated.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 11:28 AM, Brett Cannon  wrote:
> 
> 
> 
> On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor  > wrote:
> Donald Stufft  stufft.io > writes:
> 
> >
> > [words words words]
> >
> 
> I strongly support this PEP. I'd like to share two pieces of information. Both
> of these are personal anecdotes:
> 
> For the past several years, I've been a contributor on two major projects 
> using
> mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching,
> basically all contributors regularly make branches in the main repo for their
> work, and we're very free in giving people commit rights, so almost everyone
> working on PyPy in any way has this level of access. This workflow works ok. I
> don't love it as much as git, but it's fine, it's not an impediment to my 
> work.
> 
> By contrast, CPython does not have this type of workflow, there are almost no
> in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg
> user for years, I have no idea how to create a local-only branch, or a branch
> which is pushed to a remote (to use the git term). I also don't generally
> commit my own work to CPython, even though I have push privledges,
> because I
> prefer to *always* get code review on my work. As a result, I use a git mirror
> of CPython to do all my work, and generate patches from that.
> 
> The conclusion I draw from this is that hg's workflow is probably fine if
> you're a committer on the project, or don't ever need to maintain multiple
> patches concurrently (and thus can just leave everything uncommitted in the
> repo). However, the hg workflow seems extremely defficient at non-committer
> contributors.
> 
> One way to come close to that using hg is to have your own clone that you 
> never push to hg.python.org/cpython  (e.g. 
> cloning the Bitbucket clone of cpython or hosting on hg.python.org 
>  a personal clone). You can then specify the repo and 
> branch on the issue tracker to generate your patch: 
> https://docs.python.org/devguide/triaging.html#mercurial-repository 
>  . After 
> that it's just like any other patch workflow for core devs. It's not quite as 
> nice as maybe using named branches where you can just do a final hg 
> merge/commit to get your changes committed, but if you're not going to commit 
> your branches then you might as well get the automatic patch generation perk 
> in the issue tracker rather than using git (unless there is some other git 
> feature you use that you can't get in hg).

This doesn’t really work in a Pull Request workflow though I think is the point.
Uploading patches is it’s own kind of terrible but yes if you’re just uploading
patches you can probably do that. I don’t make branches in Mercurial because
i’m afraid I’m going to push a permanent branch to hg.python.org 
 and screw
something up.

>  
> 
> The seconds experience I have is that of Django's migration to git and github.
> For a long time we were on SVN, and we were very resistant to moving to
> DVCS in
> general, and github in particular. Multiple times I said that I didn't see how
> exporting a patch and uploading it to trac was more difficult than sending a
> pull request. That was very wrong on my part.
> 
> My primary observation is not about new contributors though, it's actually
> about the behavior of core developers. Before we were on github, it was fairly
> rare for core developers to ask for reviews for anything besides *gigantic*
> patches, we'd mostly just commit stuff to trunk. Since the switch to github,
> I've seen that core developers are *far* more likely to ask for reviews of
> their work before merging.
>  
> Why specifically? Did you have a web UI for reviewing patches previously? Do 
> you have CI set up for patches now and didn't before? What features did you 
> specifically gain from the switch to GitHub that you didn't have before? IOW 
> was it the "magic" of GitHub or some technical solution that you got as part 
> of the GitHub package and thus could theoretically be replicated on 
> python.org ?

In most of the cases there was some form of a web UI for reviewing patches. I
don’t believe there was a CI setup for patches previously and most projects do
end up with some sort of CI on Github since Travis makes it particularly easy.
Honestly though, I feel like the main reason for it is just that Github’s
solution is easy to use, it works with the VCS which is great because that’s
what I’m using already, and it’s highly polished.

OSS has a long history of having rather poor UX, and poor UX tends to mean that
people would rather not use some particular thing unless they *have* to. That
is the way I look at Rietveld to be honest. I get frustrated trying to use it
so I'd rather just not use it where I feel I can get away with i

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Alex Gaynor]

On Sun Nov 30 2014 at 10:28:50 AM Brett Cannon  wrote:

> Why specifically? Did you have a web UI for reviewing patches previously?
> Do you have CI set up for patches now and didn't before? What features did
> you specifically gain from the switch to GitHub that you didn't have
> before? IOW was it the "magic" of GitHub or some technical solution that
> you got as part of the GitHub package and thus could theoretically be
> replicated on python.org?
>
> -Brett
>

Previously someone looking for a review (read: any non-committer) would
export a diff from their VCS, upload it as a patch to trac, and then
reviewers could leave comments as trac comments. CPython's present process
is a clear improvement, insofar as Rietveld allows inlining commenting, but
it is otherwise basically the same.

By contrast, the Github process does not require a patch author to leave
their workflow, they simply "git push" to update a patch. We now also have
CI for PRs, but that's a recent addition.

It's not magic, it's a good UX :-)

Alex
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 6:18 AM, Ben Finney  wrote:
> 
> Donald Stufft  writes:
> 
>> I think there is a big difference here between using a closed source
>> VCS or compiler and using a closed source code host. Namely in that
>> the protocol is defined by git so switching from one host to another
>> is easy.
> 
> GitHub deliberately encourages proprietary features that create valuable
> data that cannot be exported — the proprietary GitHub-specific pull
> requests being a prime example.

Once a Pull Request is merged there is little benefit to the PR itself,
however if you want the diff you can create a .patch file by appending
.diff or .patch to the end of the PR URL.

> 
> So it is only true to say one can export the data to a different host if
> one entirely abandons all those features which create GitHub-proprietary
> data.

That’s not true at all. There is an API that allows you to access *all*
of the data that is stored there. Allowing you to replicate it to whatever
system you wish. 

> 
> If you want to re-write the PEP to be clear you are only talking about
> the repository hosting of GitHub, and *not* any of its extra features
> that make it so attractive to use their proprietary service, I'd be
> interested to see that.
> 
> On the other hand, if you want to promote those proprietary features as
> part of the PEP, then it is disingenuous to claim the data can be easily
> exported to a different host.
> 
> -- 
> \“Do not enter the lift backwards, and only when lit up.” |
>  `\—elevator, Leipzig |
> _o__)  |
> Ben Finney
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 2:08 AM, Larry Hastings  wrote:
> 
> 
> On 11/29/2014 04:37 PM, Donald Stufft wrote:
>> On Nov 29, 2014, at 7:15 PM, Alex Gaynor  
>>  wrote:
>>> Despite being a regular hg
>>> user for years, I have no idea how to create a local-only branch, or a 
>>> branch
>>> which is pushed to a remote (to use the git term).
>> I also don’t know how to do this.
> 
> Instead of collectively scratching your heads, could one of you guys do the 
> research and figure out whether or not hg supports this workflow?  One of the 
> following two things must be true:
> hg supports this workflow (or a reasonable fascimile), which may lessen the 
> need for this PEP.
> hg doesn't support this workflow, which may strengthen the need for this PEP.
> Saying "I've been using hg for years and I don't know whether it supports 
> this IMPORTANT THING" is not a particularly compelling argument.
> 

Comments like this make me feel like I didn’t explain myself very well in the
PEP.

While I do think that supporting this workflow via an extension is worse than
supporting it in core, this isn’t why this PEP exists. The current workflow for
contributing is painful, for the repositories this is talking about if I’m a
non-comitter I have to email patches to a particular closed mailing list and
then sit around and wait.

The Pull Request based workflow is *massively* better than uploading/emailing
patches around. So the question then becomes, if we're going to move to a PR
based workflow how do we do it? PEP 474 says that we should install some
software that works with Mercurial and supports Pull Requests. Another thread
suggested that we should just use to bitbucket whicih also supports Mercurial
and use that.

This PEP says that git and Github have the popular vote, which is extremely
compelling as a feature because:

* Popularity is the one thing you can't replicate featurewise. This feature or
  that you can find a way to work something like it out.
* With popularity comes the fact that people will already know the tooling
  involved and how to use it. This means that a new contributor whom already
  knows git will spend less time learning our specific toolset and more time
  being able to meaningfully contribute.
* With popularity also comes transferability, If I don't currently know a VCS
  tool and I learn git (and github) for the sake of these repositories then
  that knowledge will directly transfer to greater than 60% of the top 100
  projects on PyPI.
* With popularity also comes an increased amount of people writing about it,
  asking questions, etc. This means that if I have a problem while using this
  tool I am more likely to find someone who already had this problem and they
  are more likely to have found someone to help them solve it than I am with
  a less popular tool.

I'm not sure why people seem to try and focus on the fact that with this
extension or that you can make Mercurial replicate git better when that isn't
really the major point of the PEP at all. If Mercurial and git were neck in
neck in terms of mindshare and bitbucket and Github were as well then this
PEP probably wouldn't exist because on a techincal level I think the two tools
are probably close enough. However when most of the world is using one tool
and you're using another, that does represent a barrier to entry.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Brett Cannon]

On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor  wrote:

> Donald Stufft  stufft.io> writes:
>
> >
> > [words words words]
> >
>
> I strongly support this PEP. I'd like to share two pieces of information.
> Both
> of these are personal anecdotes:
>
> For the past several years, I've been a contributor on two major projects
> using
> mercurial, CPython and PyPy. PyPy has a strong culture of in-repo
> branching,
> basically all contributors regularly make branches in the main repo for
> their
> work, and we're very free in giving people commit rights, so almost
> everyone
> working on PyPy in any way has this level of access. This workflow works
> ok. I
> don't love it as much as git, but it's fine, it's not an impediment to my
> work.
>
> By contrast, CPython does not have this type of workflow, there are almost
> no
> in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular
> hg
> user for years, I have no idea how to create a local-only branch, or a
> branch
> which is pushed to a remote (to use the git term). I also don't generally
> commit my own work to CPython, even though I have push privledges,
> because I
> prefer to *always* get code review on my work. As a result, I use a git
> mirror
> of CPython to do all my work, and generate patches from that.
>
> The conclusion I draw from this is that hg's workflow is probably fine if
> you're a committer on the project, or don't ever need to maintain multiple
> patches concurrently (and thus can just leave everything uncommitted in the
> repo). However, the hg workflow seems extremely defficient at non-committer
> contributors.
>

One way to come close to that using hg is to have your own clone that you
never push to hg.python.org/cpython (e.g. cloning the Bitbucket clone of
cpython or hosting on hg.python.org a personal clone). You can then specify
the repo and branch on the issue tracker to generate your patch:
https://docs.python.org/devguide/triaging.html#mercurial-repository . After
that it's just like any other patch workflow for core devs. It's not quite
as nice as maybe using named branches where you can just do a final hg
merge/commit to get your changes committed, but if you're not going to
commit your branches then you might as well get the automatic patch
generation perk in the issue tracker rather than using git (unless there is
some other git feature you use that you can't get in hg).


>
> The seconds experience I have is that of Django's migration to git and
> github.
> For a long time we were on SVN, and we were very resistant to moving to
> DVCS in
> general, and github in particular. Multiple times I said that I didn't see
> how
> exporting a patch and uploading it to trac was more difficult than sending
> a
> pull request. That was very wrong on my part.
>
> My primary observation is not about new contributors though, it's actually
> about the behavior of core developers. Before we were on github, it was
> fairly
> rare for core developers to ask for reviews for anything besides *gigantic*
> patches, we'd mostly just commit stuff to trunk. Since the switch to
> github,
> I've seen that core developers are *far* more likely to ask for reviews of
> their work before merging.
>

Why specifically? Did you have a web UI for reviewing patches previously?
Do you have CI set up for patches now and didn't before? What features did
you specifically gain from the switch to GitHub that you didn't have
before? IOW was it the "magic" of GitHub or some technical solution that
you got as part of the GitHub package and thus could theoretically be
replicated on python.org?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Donald Stufft]

> On Nov 30, 2014, at 7:31 AM, Paul Moore  wrote:
> 
> On 29 November 2014 at 23:27, Donald Stufft  wrote:
>> In previous years there was concern about how well supported git was on 
>> Windows
>> in comparison to Mercurial. However git has grown to support Windows as a 
>> first
>> class citizen. In addition to that, for Windows users who are not well 
>> aquanted
>> with the Windows command line there are GUI options as well.
> 
> I have little opinion on the PEP as a whole, but is the above
> statement true? From the git website, version 2.2.0 is current, and
> yet the downloadable Windows version is still 1.9.4. That's a fairly
> significant version lag for a "first class citizen".
> 
> I like git, and it has a number of windows-specific extensions that
> are really useful (more than Mercurial, AFAIK), but I wouldn't say
> that the core product supported Windows on an equal footing to Linux.
> 
> Paul

I think so yes. I may be wrong, however while 1.9.4 may be the latest
downloadable version of git for Windows, there is no downloadable
version of the Linux clients at all, they just tell you to go use
your package manager which for instance is version 1.7 on Debian. On
OS X the latest version is 2.0.1.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Unicode decode exception

[balaji marisetti]

Hi,

When I try to iterate through the lines of a
file("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c"), I get a
UnicodeDecodeError (in python 3.4.0 on Ubuntu 14.04). But there is no
such error with python 2.7.6. What could be the problem?

In [39]: with open("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
 for line in f:
 print (line)

---
UnicodeDecodeErrorTraceback (most recent call last)
 in ()
  1 with open("../openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c") as f:
> 2 for line in f:
  3 print (line)
  4

/usr/lib/python3.4/codecs.py in decode(self, input, final)
311 # decode input (taking the buffer into account)
312 data = self.buffer + input
--> 313 (result, consumed) = self._buffer_decode(data,
self.errors, final)
314 # keep undecoded input until the next call
315 self.buffer = data[consumed:]


-- 
:-)balaji
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

- [ ] Paste-able links

On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner  wrote:

> - [ ] Stable URIs
> - [ ] Commit hashes
>
> On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner  wrote:
>
>> - [ ] Markdown
>> - [ ] ReStructuredText
>>
>> - [ ] Review (why are these out of band?)
>>
>> On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner 
>> wrote:
>>
>>> Specifically, which features are most ideal here?
>>>
>>> - [ ] Userbase
>>> - [ ] TTW editing only over SSL (see: Zope 2)
>>> - [ ] Pull Requests (see also: BitBucket, Torvalds rant)
>>> - [ ] Simple Issue Tagging
>>> - [ ] Pingbacks
>>> - [ ] CI Integration
>>>
>>>
>>> On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft 
>>> wrote:
>>>

 > On Nov 30, 2014, at 12:06 AM, Ben Finney 
 wrote:
 >
 > Nick Coghlan  writes:
 >
 >> 1. I strongly believe that the long term sustainability of the
 overall
 >> open source community requires the availability and use of open
 source
 >> infrastructure.
 >
 > I concur. This article >>> http://mako.cc/writing/hill-free_tools.html>
 > makes the arguments well, IMO.
 >
 >> 2. I also feel that this proposal is far too cavalier in not even
 >> discussing the possibility of helping out the Mercurial team […] we'd
 >> prefer to switch to something else entirely rather than organising a
 >> sprint with them at PyCon to help ensure that our existing Mercurial
 >> based infrastructure is approachable for git & GitHub users?
 >
 > Exactly. For such a core tool, instead of pushing proprietary
 platforms
 > at the expense of software freedom, the sensible strategy for a
 project
 > (Python) that hopes to be around in the long term is to use and
 improve
 > the free software platforms.

 I think there is a big difference here between using a closed source VCS
 or compiler and using a closed source code host. Namely in that the
 protocol is defined by git so switching from one host to another is
 easy.

 It’s akin to saying that if we chose to run the PyPI services on a
 Windows
 machine that it is somehow makes it less-free even though we could
 have chosen to run it on a “free” OS and we weren’t doing much, if
 anything,
 to tie us to that particular OS.

 If it makes people feel better we can continue to support the existing
 mechanisms of contribution, then people can choose between interacting
 with a “non free” host and “free” tooling. I suspect most people will
 choose
 the “non-free” tooling.
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com

>>>
>>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

https://kallithea-scm.org/repos/kallithea/files/tip/setup.py
https://github.com/codeinn/vcs/blob/master/setup.py



On Sun, Nov 30, 2014 at 1:16 AM, Wes Turner  wrote:

> Is there a kalithea celery task to mirror / SYNC with github, for pull
> requests and/or issues?
>
> https://pypi.python.org/pypi/Kallithea/
> https://kallithea-scm.org/
> https://kallithea-scm.org/repos/kallithea
> https://bitbucket.org/conservancy/kallithea
>
> http://pythonhosted.org//Kallithea
> http://kallithea.readthedocs.org/en/latest/
>
> http://pypi.python.org/pypi/vcs
> http://pythonhosted.org//vcs
> https://github.com/codeinn/vcs
>
>
> On Sun, Nov 30, 2014 at 12:50 AM, Wes Turner  wrote:
>
>> - [ ] Paste-able links
>>
>> On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner 
>> wrote:
>>
>>> - [ ] Stable URIs
>>> - [ ] Commit hashes
>>>
>>> On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner 
>>> wrote:
>>>
 - [ ] Markdown
 - [ ] ReStructuredText

 - [ ] Review (why are these out of band?)

 On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner 
 wrote:

> Specifically, which features are most ideal here?
>
> - [ ] Userbase
> - [ ] TTW editing only over SSL (see: Zope 2)
> - [ ] Pull Requests (see also: BitBucket, Torvalds rant)
> - [ ] Simple Issue Tagging
> - [ ] Pingbacks
> - [ ] CI Integration
>
>
> On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft 
> wrote:
>
>>
>> > On Nov 30, 2014, at 12:06 AM, Ben Finney <
>> ben+pyt...@benfinney.id.au> wrote:
>> >
>> > Nick Coghlan  writes:
>> >
>> >> 1. I strongly believe that the long term sustainability of the
>> overall
>> >> open source community requires the availability and use of open
>> source
>> >> infrastructure.
>> >
>> > I concur. This article > http://mako.cc/writing/hill-free_tools.html>
>> > makes the arguments well, IMO.
>> >
>> >> 2. I also feel that this proposal is far too cavalier in not even
>> >> discussing the possibility of helping out the Mercurial team […]
>> we'd
>> >> prefer to switch to something else entirely rather than organising
>> a
>> >> sprint with them at PyCon to help ensure that our existing
>> Mercurial
>> >> based infrastructure is approachable for git & GitHub users?
>> >
>> > Exactly. For such a core tool, instead of pushing proprietary
>> platforms
>> > at the expense of software freedom, the sensible strategy for a
>> project
>> > (Python) that hopes to be around in the long term is to use and
>> improve
>> > the free software platforms.
>>
>> I think there is a big difference here between using a closed source
>> VCS
>> or compiler and using a closed source code host. Namely in that the
>> protocol is defined by git so switching from one host to another is
>> easy.
>>
>> It’s akin to saying that if we chose to run the PyPI services on a
>> Windows
>> machine that it is somehow makes it less-free even though we could
>> have chosen to run it on a “free” OS and we weren’t doing much, if
>> anything,
>> to tie us to that particular OS.
>>
>> If it makes people feel better we can continue to support the existing
>> mechanisms of contribution, then people can choose between interacting
>> with a “non free” host and “free” tooling. I suspect most people will
>> choose
>> the “non-free” tooling.
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>>
>
>

>>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

Is there a kalithea celery task to mirror / SYNC with github, for pull
requests and/or issues?

https://pypi.python.org/pypi/Kallithea/
https://kallithea-scm.org/
https://kallithea-scm.org/repos/kallithea
https://bitbucket.org/conservancy/kallithea

http://pythonhosted.org//Kallithea
http://kallithea.readthedocs.org/en/latest/

http://pypi.python.org/pypi/vcs
http://pythonhosted.org//vcs
https://github.com/codeinn/vcs


On Sun, Nov 30, 2014 at 12:50 AM, Wes Turner  wrote:

> - [ ] Paste-able links
>
> On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner  wrote:
>
>> - [ ] Stable URIs
>> - [ ] Commit hashes
>>
>> On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner 
>> wrote:
>>
>>> - [ ] Markdown
>>> - [ ] ReStructuredText
>>>
>>> - [ ] Review (why are these out of band?)
>>>
>>> On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner 
>>> wrote:
>>>
 Specifically, which features are most ideal here?

 - [ ] Userbase
 - [ ] TTW editing only over SSL (see: Zope 2)
 - [ ] Pull Requests (see also: BitBucket, Torvalds rant)
 - [ ] Simple Issue Tagging
 - [ ] Pingbacks
 - [ ] CI Integration


 On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft 
 wrote:

>
> > On Nov 30, 2014, at 12:06 AM, Ben Finney 
> wrote:
> >
> > Nick Coghlan  writes:
> >
> >> 1. I strongly believe that the long term sustainability of the
> overall
> >> open source community requires the availability and use of open
> source
> >> infrastructure.
> >
> > I concur. This article  http://mako.cc/writing/hill-free_tools.html>
> > makes the arguments well, IMO.
> >
> >> 2. I also feel that this proposal is far too cavalier in not even
> >> discussing the possibility of helping out the Mercurial team […]
> we'd
> >> prefer to switch to something else entirely rather than organising a
> >> sprint with them at PyCon to help ensure that our existing Mercurial
> >> based infrastructure is approachable for git & GitHub users?
> >
> > Exactly. For such a core tool, instead of pushing proprietary
> platforms
> > at the expense of software freedom, the sensible strategy for a
> project
> > (Python) that hopes to be around in the long term is to use and
> improve
> > the free software platforms.
>
> I think there is a big difference here between using a closed source
> VCS
> or compiler and using a closed source code host. Namely in that the
> protocol is defined by git so switching from one host to another is
> easy.
>
> It’s akin to saying that if we chose to run the PyPI services on a
> Windows
> machine that it is somehow makes it less-free even though we could
> have chosen to run it on a “free” OS and we weren’t doing much, if
> anything,
> to tie us to that particular OS.
>
> If it makes people feel better we can continue to support the existing
> mechanisms of contribution, then people can choose between interacting
> with a “non free” host and “free” tooling. I suspect most people will
> choose
> the “non-free” tooling.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>


>>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

- [ ] Stable URIs
- [ ] Commit hashes

On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner  wrote:

> - [ ] Markdown
> - [ ] ReStructuredText
>
> - [ ] Review (why are these out of band?)
>
> On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner  wrote:
>
>> Specifically, which features are most ideal here?
>>
>> - [ ] Userbase
>> - [ ] TTW editing only over SSL (see: Zope 2)
>> - [ ] Pull Requests (see also: BitBucket, Torvalds rant)
>> - [ ] Simple Issue Tagging
>> - [ ] Pingbacks
>> - [ ] CI Integration
>>
>>
>> On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft  wrote:
>>
>>>
>>> > On Nov 30, 2014, at 12:06 AM, Ben Finney 
>>> wrote:
>>> >
>>> > Nick Coghlan  writes:
>>> >
>>> >> 1. I strongly believe that the long term sustainability of the overall
>>> >> open source community requires the availability and use of open source
>>> >> infrastructure.
>>> >
>>> > I concur. This article >> http://mako.cc/writing/hill-free_tools.html>
>>> > makes the arguments well, IMO.
>>> >
>>> >> 2. I also feel that this proposal is far too cavalier in not even
>>> >> discussing the possibility of helping out the Mercurial team […] we'd
>>> >> prefer to switch to something else entirely rather than organising a
>>> >> sprint with them at PyCon to help ensure that our existing Mercurial
>>> >> based infrastructure is approachable for git & GitHub users?
>>> >
>>> > Exactly. For such a core tool, instead of pushing proprietary platforms
>>> > at the expense of software freedom, the sensible strategy for a project
>>> > (Python) that hopes to be around in the long term is to use and improve
>>> > the free software platforms.
>>>
>>> I think there is a big difference here between using a closed source VCS
>>> or compiler and using a closed source code host. Namely in that the
>>> protocol is defined by git so switching from one host to another is easy.
>>>
>>> It’s akin to saying that if we chose to run the PyPI services on a
>>> Windows
>>> machine that it is somehow makes it less-free even though we could
>>> have chosen to run it on a “free” OS and we weren’t doing much, if
>>> anything,
>>> to tie us to that particular OS.
>>>
>>> If it makes people feel better we can continue to support the existing
>>> mechanisms of contribution, then people can choose between interacting
>>> with a “non free” host and “free” tooling. I suspect most people will
>>> choose
>>> the “non-free” tooling.
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>>>
>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Wes Turner]

- [ ] Markdown
- [ ] ReStructuredText

- [ ] Review (why are these out of band?)

On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner  wrote:

> Specifically, which features are most ideal here?
>
> - [ ] Userbase
> - [ ] TTW editing only over SSL (see: Zope 2)
> - [ ] Pull Requests (see also: BitBucket, Torvalds rant)
> - [ ] Simple Issue Tagging
> - [ ] Pingbacks
> - [ ] CI Integration
>
>
> On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft  wrote:
>
>>
>> > On Nov 30, 2014, at 12:06 AM, Ben Finney 
>> wrote:
>> >
>> > Nick Coghlan  writes:
>> >
>> >> 1. I strongly believe that the long term sustainability of the overall
>> >> open source community requires the availability and use of open source
>> >> infrastructure.
>> >
>> > I concur. This article http://mako.cc/writing/hill-free_tools.html
>> >
>> > makes the arguments well, IMO.
>> >
>> >> 2. I also feel that this proposal is far too cavalier in not even
>> >> discussing the possibility of helping out the Mercurial team […] we'd
>> >> prefer to switch to something else entirely rather than organising a
>> >> sprint with them at PyCon to help ensure that our existing Mercurial
>> >> based infrastructure is approachable for git & GitHub users?
>> >
>> > Exactly. For such a core tool, instead of pushing proprietary platforms
>> > at the expense of software freedom, the sensible strategy for a project
>> > (Python) that hopes to be around in the long term is to use and improve
>> > the free software platforms.
>>
>> I think there is a big difference here between using a closed source VCS
>> or compiler and using a closed source code host. Namely in that the
>> protocol is defined by git so switching from one host to another is easy.
>>
>> It’s akin to saying that if we chose to run the PyPI services on a Windows
>> machine that it is somehow makes it less-free even though we could
>> have chosen to run it on a “free” OS and we weren’t doing much, if
>> anything,
>> to tie us to that particular OS.
>>
>> If it makes people feel better we can continue to support the existing
>> mechanisms of contribution, then people can choose between interacting
>> with a “non free” host and “free” tooling. I suspect most people will
>> choose
>> the “non-free” tooling.
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Akira Li]

Larry Hastings  writes:

> On 11/29/2014 04:37 PM, Donald Stufft wrote:
>> On Nov 29, 2014, at 7:15 PM, Alex Gaynor  wrote:
>>> Despite being a regular hg
>>> user for years, I have no idea how to create a local-only branch, or a 
>>> branch
>>> which is pushed to a remote (to use the git term).
>> I also don’t know how to do this.
>
> Instead of collectively scratching your heads, could one of you guys
> do the research and figure out whether or not hg supports this
> workflow?  One of the following two things must be true:
>
> 1. hg supports this workflow (or a reasonable fascimile), which may
>lessen the need for this PEP.
> 2. hg doesn't support this workflow, which may strengthen the need for
>this PEP.
>

Assuming git's "all work is done in a local branch" workflow, you could
use bookmarks with hg 

http://lostechies.com/jimmybogard/2010/06/03/translating-my-git-workflow-with-local-branches-to-mercurial/
http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/#branching-with-bookmarks
http://mercurial.selenic.com/wiki/BookmarksExtension#Usage
http://stackoverflow.com/questions/1598759/git-and-mercurial-compare-and-contrast


--
Akira

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ian Cordasco]

On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou  wrote:
> On Sun, 30 Nov 2014 16:23:08 +1100
> Chris Angelico  wrote:
>>
>> Yes, GitHub is proprietary. But all of your actual code is stored in
>> git, which is free, and it's easy to push that to a new host somewhere
>> else, or create your own host. This proposal is for repositories that
>> don't need much in the way of issue trackers etc, so shifting away
>> from GitHub shouldn't demand anything beyond moving the repos
>> themselves.
>
> I hope we're not proposing to move the issue trackers to github,
> otherwise I'm -1 on this PEP.
>
> Regards
>
> Antoine.

So I usually choose not to weigh in on discussions like these but
there seems to be a lot of misdirection in some of these arguments.

To start, I'm generally neutral about this proposal or Nick's proposal
that spurred this one. I've found the most frustrating part of
contributing to anything involving CPython to be the lack of reviewer
time. I have had very small (2-line) patches take months (close to a
year in reality) to get through in spite of periodically pinging the
appropriate people. Moving to git/GitHub will not alleviate this at
all.

To be clear, the main reasoning behind Nick's was being able to submit
changes without ever having to have a local copy of the repository in
question on your machine. Having a complete web workflow for editing
and contributing makes the barrier to entry far lower than switching
VCS or anything else. BitBucket (apparently, although I've never used
this) and GitHub both have this capability and *both* are
free-as-in-beer systems.

No one as I understand it is proposing that we use the per-distro
proprietary interface to these websites.

All data can be removed from GitHub using it's API and can generally
be converted to another platform. The same goes for BitBucket although
it's arguably easier to retrieve issue data from BitBucket than
GitHub. That said, *the issue tracker is not covered by these
proposals* so this is a moot point. Drop it already.

If we're seriously considering moving to git as a DVCS, we should
consider the real free-as-in-freedom alternatives that come very close
to GitHub and can be improved by us (even though they're not written
in Python). One of those is GitLab. We can self-host a GitLab instance
easily or we can rely on gitlab.com. GitLab aims to provide a very
similar user experience to GitHub and it's slowly approaching feature
parity and experience parity. GitLab is also what a lot of people
chose to switch to after the incident Steven mentioned, which I don't
think is something we should dismiss or ignore.

We should refocus the discussion with the following in mind:

- Migrating "data" from GitHub is easy. There are free-as-in-freedom
tools to do it and the only cost is the time it would take to monitor
the process

- GitHub has a toxic company culture that we should be aware of before
moving to it. They have a couple blog posts about attempting to change
it but employees became eerily silent after the incident and have
remained so from what I've personally seen.

- GitHub may be popular but there are popular FOSS solutions that
exist that we can also self-host at something like forge.python.org

- bugs.python.org is not covered by any of these proposals

- The main benefit this proposal (and the proposal to move to
BitBucket) are seeking to achieve is an online editing experience
allowing for *anyone with a browser and an account* to contribute.
This to me is the only reason I would be +1 for either of these
proposals (if we can reach consensus).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Brett Cannon]

 On Sat, Nov 29, 2014, 21:55 Guido van Rossum  wrote:

All the use cases seem to be about adding some kind of getattr hook to
modules. They all seem to involve modifying the CPython C code anyway. So
why not tackle that problem head-on and modify module_getattro() to look
for a global named __getattr__ and if it exists, call that instead of
raising AttributeError?

 Not sure if anyone thought of it. :) Seems like a reasonable solution to
me. Be curious to know what the benchmark suite said the impact was.

-brett


On Sat, Nov 29, 2014 at 11:37 AM, Nathaniel Smith  wrote:

On Sat, Nov 29, 2014 at 4:21 AM, Guido van Rossum  wrote:
> Are these really all our options? All of them sound like hacks, none of
them
> sound like anything the language (or even the CPython implementation)
should
> sanction. Have I missed the discussion where the use cases and constraints
> were analyzed and all other approaches were rejected? (I might have some
> half-baked ideas, but I feel I should read up on the past discussion
first,
> and they are probably more fit for python-ideas than for python-dev. Plus
> I'm just writing this email because I'm procrastinating on the type
hinting
> PEP. :-)

The previous discussions I was referring to are here:
  http://thread.gmane.org/gmane.comp.python.ideas/29487/focus=29555
  http://thread.gmane.org/gmane.comp.python.ideas/29788

There might well be other options; these are just the best ones I
could think of :-). The constraints are pretty tight, though:
- The "new module" object (whatever it is) should have a __dict__ that
aliases the original module globals(). I can elaborate on this if my
original email wasn't enough, but hopefully it's obvious that making
two copies of the same namespace and then trying to keep them in sync
at the very least smells bad :-).
- The "new module" object has to be a subtype of ModuleType, b/c there
are lots of places that do isinstance(x, ModuleType) checks (notably
-- but not only -- reload()). Since a major goal here is to make it
possible to do cleaner deprecations, it would be really unfortunate if
switching an existing package to use the metamodule support itself
broke things :-).
- Lookups in the normal case should have no additional performance
overhead, because module lookups are extremely extremely common. (So
this rules out dict proxies and tricks like that -- we really need
'new_module.__dict__ is globals()' to be true.)

AFAICT there are three logically possible strategies for satisfying
that first constraint:
(a) convert the original module object into the type we want, in-place
(b) create a new module object that acts like the original module object
(c) somehow arrange for our special type to be used from the start

My options 1 and 2 are means of accomplishing (a), and my options 3
and 4 are means of accomplishing (b) while working around the
behavioural quirks of module objects (as required by the second
constraint).

The python-ideas thread did also consider several methods of
implementing strategy (c), but they're messy enough that I left them
out here. The problem is that somehow we have to execute code to
create the new subtype *before* we have an entry in sys.modules for
the package that contains the code for the subtype. So one option
would be to add a new rule, that if a file pkgname/__new__.py exists,
then this is executed first and is required to set up
sys.modules["pkgname"] before we exec pkgname/__init__.py. So
pkgname/__new__.py might look like:

import sys
from pkgname._metamodule import MyModuleSubtype
sys.modules[__name__] = MyModuleSubtype(__name__, docstring)

This runs into a lot of problems though. To start with, the 'from
pkgname._metamodule ...' line is an infinite loop, b/c this is the
code used to create sys.modules["pkgname"]. It's not clear where the
globals dict for executing __new__.py comes from (who defines
__name__? Currently that's done by ModuleType.__init__). It only works
for packages, not modules. The need to provide the docstring here,
before __init__.py is even read, is weird. It adds extra stat() calls
to every package lookup. And, the biggest showstopper IMHO: AFAICT
it's impossible to write a polyfill to support this code on old python
versions, so it's useless to any package which needs to keep
compatibility with 2.7 (or even 3.4). Sure, you can backport the whole
import system like importlib2, but telling everyone that they need to
replace every 'import numpy' with 'import importlib2; import numpy' is
a total non-starter.

So, yeah, those 4 options are really the only plausible ones I know of.

Option 1 and option 3 are pretty nice at the language level! Most
Python objects allow assignment to __class__ and __dict__, and both
PyPy and Jython at least do support __class__ assignment. Really the
only downside with Option 1 is that actually implementing it requires
attention from someone with deep knowledge of typeobject.c.

-n

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Antoine Pitrou]

On Sun, 30 Nov 2014 16:23:08 +1100
Chris Angelico  wrote:
> 
> Yes, GitHub is proprietary. But all of your actual code is stored in
> git, which is free, and it's easy to push that to a new host somewhere
> else, or create your own host. This proposal is for repositories that
> don't need much in the way of issue trackers etc, so shifting away
> from GitHub shouldn't demand anything beyond moving the repos
> themselves.

I hope we're not proposing to move the issue trackers to github,
otherwise I'm -1 on this PEP.

Regards

Antoine.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Paul Moore]

On 29 November 2014 at 23:27, Donald Stufft  wrote:
> In previous years there was concern about how well supported git was on 
> Windows
> in comparison to Mercurial. However git has grown to support Windows as a 
> first
> class citizen. In addition to that, for Windows users who are not well 
> aquanted
> with the Windows command line there are GUI options as well.

I have little opinion on the PEP as a whole, but is the above
statement true? From the git website, version 2.2.0 is current, and
yet the downloadable Windows version is still 1.9.4. That's a fairly
significant version lag for a "first class citizen".

I like git, and it has a number of windows-specific extensions that
are really useful (more than Mercurial, AFAIK), but I wouldn't say
that the core product supported Windows on an equal footing to Linux.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ben Finney]

Donald Stufft  writes:

> I think there is a big difference here between using a closed source
> VCS or compiler and using a closed source code host. Namely in that
> the protocol is defined by git so switching from one host to another
> is easy.

GitHub deliberately encourages proprietary features that create valuable
data that cannot be exported — the proprietary GitHub-specific pull
requests being a prime example.

So it is only true to say one can export the data to a different host if
one entirely abandons all those features which create GitHub-proprietary
data.

If you want to re-write the PEP to be clear you are only talking about
the repository hosting of GitHub, and *not* any of its extra features
that make it so attractive to use their proprietary service, I'd be
interested to see that.

On the other hand, if you want to promote those proprietary features as
part of the PEP, then it is disingenuous to claim the data can be easily
exported to a different host.

-- 
 \“Do not enter the lift backwards, and only when lit up.” |
  `\—elevator, Leipzig |
_o__)  |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Ben Finney]

Chris Angelico  writes:

> But what non-free software is required to use the community design
> processes? The GitHub client is entirely optional; I don't use it, I
> just use git itself. Using a free client to access a proprietary
> server isn't the same as using non-free software.

For that to remain true, there must be no advantage specifically to
GitHub except as a commodity repository host. So all the verbiage about
specific GitHub features in this PEP are undermined.

On the other hand, if GitHub is being recommended specifically because
of features that cannot be migrated elsewhere (such as its proprietary
pull requests), then it is not possible to argue that Git alone is
sufficient to realise these advantages.

-- 
 \ “DRM doesn't inconvenience [lawbreakers] — indeed, over time it |
  `\ trains law-abiding users to become [lawbreakers] out of sheer |
_o__)frustration.” —Charles Stross, 2010-05-09 |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Steven D'Aprano]

I have some questions and/or issues with the PEP, but first I'm going to 
add something to Nick's comments:

On Sun, Nov 30, 2014 at 11:12:17AM +1000, Nick Coghlan wrote:

> Beyond that, GitHub is indeed the most expedient option. My two main
> reasons for objecting to taking the expedient path are:
> 
> 1. I strongly believe that the long term sustainability of the overall open
> source community requires the availability and use of open source
> infrastructure. While I admire the ingenuity of the "free-as-in-beer" model
> for proprietary software companies fending off open source competition, I
> still know a proprietary platform play when I see one (and so do venture
> capitalists looking to extract monopoly rents from the industry in the
> future). (So yes, I regret relenting on this principle in previously
> suggesting the interim use of another proprietary hosted service)
> 
> 2. I also feel that this proposal is far too cavalier in not even
> discussing the possibility of helping out the Mercurial team to resolve
> their documentation and usability issues rather than just yelling at them
> "your tool isn't popular enough for us, and we find certain aspects of it
> too hard to use, so we're switching to something else rather than working
> with you to address our concerns". We consider the Mercurial team a
> significant enough part of the Python ecosystem that Matt was one of the
> folks specifically invited to the 2014 language summit to discuss their
> concerns around the Python 3 transition. Yet we'd prefer to switch to
> something else entirely rather than organising a sprint with them at PyCon
> to help ensure that our existing Mercurial based infrastructure is
> approachable for git & GitHub users? (And yes, I consider some of the core
> Mercurial devs to be friends, so this isn't an entirely abstract concern
> for me)


Thanks Nick, I think these are excellent points, particularly the 
second. It would be a gross strawman to say that we should "only" use 
software developed in Python, but we should eat our own dogfood whenever 
practical and we should support and encourage the Python ecosystem, 
including Mercurial.

Particularly since hg and git are neck and neck feature-wise, we should 
resist the tendency to jump on bandwagons. If git were clearly the 
superior product, then maybe there would be an argument for using the 
best tool for the job, but it isn't.

As for the question of using Github hosting, there's another factor 
which has been conspicuous by its absence. Has GitHub's allegedly toxic 
and bullying culture changed since Julie Horvath quit in March? And if 
it has not, do we care?

I'm not a saint, but I do try to choose ethical companies and 
institutions over unethical ones whenever it is possible and practical. 
I'm not looking for a witch-hunt against GitHub, but if the allegations 
made by Horvath earlier this year are true, and I don't believe anyone 
has denied them, then so long as GitHub's internal culture remains 
sexist and hostile to the degree reported, then I do not believe that we 
should use GitHub's services even if we shift some repos to git.

I have serious doubts about GitHub's compatibility with the ideals 
expressed by the PSF. Even if our code of conduct does not explicitly 
forbid it, I think that it goes against the principles that we say we 
aspire to.

Given Horvath's experiences, and the lack of clear evidence that 
anything has changed in GitHub, I would be deeply disappointed if Python 
lent even a smidgeon of legitimacy to their company, and I personally 
will not use their services.

I acknowledge that it's hard to prove a negative, and GitHub may have 
difficulty proving to my satisfaction that they have changed. (My 
experience is that company culture rarely changes unless there is a 
change in management, and even then only slowly.) Particularly given 
GitHub's supposed egalitarian, non-hierarchical, and meritocratic 
structure, that nobody apparently saw anything wrong with the bullying 
of staff and workplace sexism until it became public knowledge suggests 
that it is not just a few bad apples but a problem all through the 
company.



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Chris Angelico]

On Sun, Nov 30, 2014 at 8:54 PM, Nick Coghlan  wrote:
> On 30 November 2014 at 15:23, Chris Angelico  wrote:
>> Python is already using quite a bit of non-free software in its
>> ecosystem. The Windows builds of CPython are made with Microsoft's
>> compiler, and the recent discussion about shifting to Cygwin or MinGW
>> basically boiled down to "but it ought to be free software", and that
>> was considered not a sufficiently strong argument. In each case, the
>> decision has impact on other people (using MSVC for the official
>> python.org installers means extension writers need to use MSVC too;
>> and using GitHub means that contributors are strongly encouraged,
>> possibly required, to use GitHub); so why is it acceptable to use a
>> non-free compiler, but not acceptable to use a non-free host?
>
> Relying on non-free software to support users of a non-free platform
> is rather different from *requiring* the use of non-free software to
> participate in core Python community design processes.

But what non-free software is required to use the community design
processes? The GitHub client is entirely optional; I don't use it, I
just use git itself. Using a free client to access a proprietary
server isn't the same as using non-free software.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com