Re: Beancount Updates - 2015-06-29

[Martin Blais]

Fair enough.

The main problem is that I'm just not really into going back down the
rabbit hole and burn another few precious weekend days to setup new Windows
VMs and futz around with docker to test on various distros just to avoid
users having to type a single command to install dependencies, I already
burn that time, and I'm really busy. I deal with enough piping and plumbing
at work that on my own time I want to write brand new code fast and build
features or learn new things instead of spending all that time worrying
about packaging.

If you really care about this, and want to help, test your change on
Windows, Linux (Ubuntu, Debian, RedHat) and recent Mac OS X, and if all
pans out, I'll change it over. Otherwise my feeling is what's there -
distutils - is good enough.





On Thu, Oct 8, 2015 at 3:01 PM, Gary Peck 
wrote:

> Thanks. I'm not even sure how you can end up without setuptools installed
> with a modern python3 installation. I think as long as you require that the
> user has pip3 installed, through whatever means is appropriate to their
> OS/environment, it'll pull in setuptools.
>
> Gary
>
> On Wednesday, October 7, 2015 at 10:26:48 PM UTC-7, Martin Blais wrote:
>>
>> I created a ticket:
>>
>> https://bitbucket.org/blais/beancount/issues/72/consider-installing-with-setuptools-again
>>
>> It would need a lot more testing, e.g., on Windows, Mac, etc., in order
>> to make sure this works and if I recall all the crazy time I sunk into
>> this, I'm pretty sure I don't want to burn all that time all over again.
>> I'll reconsider it but it's at the very bottom of the list of things I need
>> to do. The price to pay to install the other dependencies explicitly is
>> really small IMO (and you don't have to install setuptools, which is a
>> plus).
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Oct 7, 2015 at 6:44 PM, Gary Peck  wrote:
>>
>>>
>>>
>>> On Thursday, July 2, 2015 at 3:14:33 PM UTC-7, Martin Blais wrote:

 On Thu, Jul 2, 2015 at 3:23 PM, Martin Blais  wrote:

>
> I'd been having problems with setup.py in that it's supposed to be
> able to install dependencies by default, so in theory you shouldn't have 
> to
> install anything but Beancount now. It hadn't been working before.
>
> Here's some background, in case anyone's interested: Python has two
> active supporting projects for packaging:
>
> (1) distutils, which comes with it in the stdlib, and
> (2) setuptools, which is newer and better, but which needs to be
> installed on top of the stdlib.
>
> There's a deep history of why that is and many projects have come and
> gone trying to make this better (e.g. distribute). The main issue with
> disutils is that it does not support automatic installation of 
> dependencies
> (it's really old but a ton of packages still depend on it, that's the 
> price
> of popularity). Setuptools does support it, the packager can declare
> dependencies and pip3 should be able to pull in and install all
> requirements automatically.  I could not get setuptools to install my
> dependencies due to a namespace collision with the 'parser' module (was my
> fault).
>
> Now that I've made changes, Beancount supports setuptools and installs
> dependencies automatically. If you don't have setuptools installed, it
> falls back on distutils (and you have to install the dependencies 
> manually,
> e.g. it's a one-liner:  "pip3 install python-dateutil lxml bottle ply").
>

 Okay, an update: the state of setuptools / distutils Python
 distribution tools is in a really sorry state indeed. I've spend much of my
 afternoon dealing with some more setuptools brokenness and running around
 its sad, sad source code. When I use

   python3 setup.py build_ext -i

 with setuptools installed, it puts the _parser.so library under
 src/python and not under src/python/beancount/parser where it should be, so
 that's a no go.

 This is using setuptools v15. Manually installing the latest v18 puts
 the .so in the right place but there is an annoying warning about not
 running it with pip3.

 #$&*# this. No seriously, I've had enough. I want to spend my time
 building features instead of packaging, and I've wasted the better part of
 a beautiful vacation day just dealing with setuptools and reading
 conflicting information on the web. I'm ditching support for setuptools.

 Beancount will use distutils, which is basic, comes with all versions
 of Python, and does the right thing by default. It's only drawback is that
 you need to install the dependencies manually. It won't do the dependencies
 automatically. It's a one-liner to install dependencies. No big deal in my
 book.

 Python seriously needs a brand new, rewritten from scratch packaging

Re: Beancount Updates - 2015-06-29

[Martin Blais]

I created a ticket:
https://bitbucket.org/blais/beancount/issues/72/consider-installing-with-setuptools-again

It would need a lot more testing, e.g., on Windows, Mac, etc., in order to
make sure this works and if I recall all the crazy time I sunk into this,
I'm pretty sure I don't want to burn all that time all over again. I'll
reconsider it but it's at the very bottom of the list of things I need to
do. The price to pay to install the other dependencies explicitly is really
small IMO (and you don't have to install setuptools, which is a plus).







On Wed, Oct 7, 2015 at 6:44 PM, Gary Peck 
wrote:

>
>
> On Thursday, July 2, 2015 at 3:14:33 PM UTC-7, Martin Blais wrote:
>>
>> On Thu, Jul 2, 2015 at 3:23 PM, Martin Blais  wrote:
>>
>>>
>>> I'd been having problems with setup.py in that it's supposed to be able
>>> to install dependencies by default, so in theory you shouldn't have to
>>> install anything but Beancount now. It hadn't been working before.
>>>
>>> Here's some background, in case anyone's interested: Python has two
>>> active supporting projects for packaging:
>>>
>>> (1) distutils, which comes with it in the stdlib, and
>>> (2) setuptools, which is newer and better, but which needs to be
>>> installed on top of the stdlib.
>>>
>>> There's a deep history of why that is and many projects have come and
>>> gone trying to make this better (e.g. distribute). The main issue with
>>> disutils is that it does not support automatic installation of dependencies
>>> (it's really old but a ton of packages still depend on it, that's the price
>>> of popularity). Setuptools does support it, the packager can declare
>>> dependencies and pip3 should be able to pull in and install all
>>> requirements automatically.  I could not get setuptools to install my
>>> dependencies due to a namespace collision with the 'parser' module (was my
>>> fault).
>>>
>>> Now that I've made changes, Beancount supports setuptools and installs
>>> dependencies automatically. If you don't have setuptools installed, it
>>> falls back on distutils (and you have to install the dependencies manually,
>>> e.g. it's a one-liner:  "pip3 install python-dateutil lxml bottle ply").
>>>
>>
>> Okay, an update: the state of setuptools / distutils Python distribution
>> tools is in a really sorry state indeed. I've spend much of my afternoon
>> dealing with some more setuptools brokenness and running around its sad,
>> sad source code. When I use
>>
>>   python3 setup.py build_ext -i
>>
>> with setuptools installed, it puts the _parser.so library under
>> src/python and not under src/python/beancount/parser where it should be, so
>> that's a no go.
>>
>> This is using setuptools v15. Manually installing the latest v18 puts the
>> .so in the right place but there is an annoying warning about not running
>> it with pip3.
>>
>> #$&*# this. No seriously, I've had enough. I want to spend my time
>> building features instead of packaging, and I've wasted the better part of
>> a beautiful vacation day just dealing with setuptools and reading
>> conflicting information on the web. I'm ditching support for setuptools.
>>
>> Beancount will use distutils, which is basic, comes with all versions of
>> Python, and does the right thing by default. It's only drawback is that you
>> need to install the dependencies manually. It won't do the dependencies
>> automatically. It's a one-liner to install dependencies. No big deal in my
>> book.
>>
>> Python seriously needs a brand new, rewritten from scratch packaging
>> system.
>>
>
> I fully understand your pain and frustration with setuptools, but I'm
> hoping you might reconsider. With a few small changes to setup.py, I have
> not been able to find an installation method that fails. If you let me know
> environments where the installation fails, I'm happy to try to fix those
> environments too. I'm trying to get others in my company to use beancount
> for our bookkeeping and it's really nice to be able to point people at a
> single installation command, as well as being able to use tools like pipsi
> to install beancount.
>
> I tested the following commands under Ubuntu 15.04 and (a) a virtualenv
> with python 3.4.3, pip 7.1.0, and setuptools 18.0.1, as well as (b) a
> pyvenv with python 3.4.3, pip 1.5.6, setuptools 12.2:
>
> - pip3 install -e . (this is recommended by the python packaging guide for
> development installs rather than using "python3 setup.py install" directly)
> - pip3 install .
> - python3 setup.py install (yes, you get an annoying warning about not
> running it under pip, but everything works)
> - python3 setup.py build_ext -i (copies _parser.so to
> src/python/beancount/parser)
>
> Here's a diff of the changes I had to make. I think the reason you were
> having trouble with _parser.so getting copied to the right place is because
> the first argument to Extension needs to be a python package path, not a
> filesystem path.
>
>
> diff -r 

Re: Beancount Updates - 2015-06-29

[Simon Michael]

On 7/2/15 12:23 PM, Martin Blais wrote:
 Thanks a lot Simon.
 Can I count you as one of my users now? ;-)

Alas no, that wouldn't be accurate yet :) I always want to learn more
about how beancount does things, it's just a matter of where to spend
time. Clear getting started docs and easy setup does lure me in, thanks
to that I have a working up to date beancount again, hurrah.

Probably another good thing to lure h/ledger users are demos/examples of
some things we can't easily do with those tools, that we could do with
beancount (ideally, coexisting with our existing workflow).


-- 

--- 
You received this message because you are subscribed to the Google Groups 
Ledger group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Beancount Updates - 2015-06-29

[Martin Blais]

On Wed, Jul 1, 2015 at 10:54 AM, Simon Michael si...@joyful.com wrote:

 Nice! Fewer clearer more cross-platform install steps is always good.


 Installation worked for me on osx just now. I may have had some
 prerequisites installed from before, such as python3 from brew.

 I thought I was done when bean-check ran, but just out of interest I ran
 bean-doctor from the next section and got a traceback showing that I
 still needed lxml. I confirmed that by re-reading your release notes and
 installed it with pip3 install lxml.


Ah! How could I forget. Thanks for pointing this out.
I added lxml to checkdeps.



 The next traceback mentioned
 datetime, so pip3 install python-datetime.


You mean python-dateutil, not python-datetime.
(datetime is part of the stdlib and should require no install.)



 Finally bean-doctor showed a
 nice list of dependencies and their status. I had bottle from before,
 and upgraded it with pip3 install bottle --upgrade, and ply sounds
 useful so pip3 install ply.


You need PLY for bean-query.


bean-doctor is nice, it helps reassure me that I won't be getting
 dependency-related tracebacks later while using the tool.


bean-doctor checkdeps is only required to run Beancount directly from the
clone.

I'd been having problems with setup.py in that it's supposed to be able to
install dependencies by default, so in theory you shouldn't have to install
anything but Beancount now. It hadn't been working before.

Here's some background, in case anyone's interested: Python has two active
supporting projects for packaging:

(1) distutils, which comes with it in the stdlib, and
(2) setuptools, which is newer and better, but which needs to be installed
on top of the stdlib.

There's a deep history of why that is and many projects have come and gone
trying to make this better (e.g. distribute). The main issue with disutils
is that it does not support automatic installation of dependencies (it's
really old but a ton of packages still depend on it, that's the price of
popularity). Setuptools does support it, the packager can declare
dependencies and pip3 should be able to pull in and install all
requirements automatically.  I could not get setuptools to install my
dependencies due to a namespace collision with the 'parser' module (was my
fault).

Now that I've made changes, Beancount supports setuptools and installs
dependencies automatically. If you don't have setuptools installed, it
falls back on distutils (and you have to install the dependencies manually,
e.g. it's a one-liner:  pip3 install python-dateutil lxml bottle ply).





 Here are some
 ideas make it even better:
 - always show the dependency list instead of a traceback


Hmm I've done such things in the past, and my experience is that it's often
better to just let an unexpected exception trickle through: if I put too
much magic in the runner and then _that_ magic fails, then you have an even
more cryptic error message: the original exception, and maybe another
exception occurring in the error detection code on top of it. Installing
dependencies is boiled down to something pretty simple at this point:

  pip3 install python-dateutil lxml bottle ply

That's it. Beancount isn't much different than other Python softwares and
the other packages don't wrap all their imports with try-except all over
the place. The only difference is that many users will be running from
source.  I prefer to keep things simple.  Nonetheless, I'll experiment with
the idea.



 - include the new lxml dependency in the list

Done.



 - indicate which libs are required

In docs and checkdeps output.



 - for not-yet-installed libs, suggest the pip command to install them

That's great idea. Done.
I've added a message to the checkdeps output.



 - maybe have each bean tool check for required dependencies at startup ?


Thanks a lot Simon.
Can I count you as one of my users now? ;-)

Thanks,



On 6/28/15 11:09 PM, Martin Blais wrote:
  2015-06-27
 
- Complete the design doc and made it public.

  2015-06-09
 
- Rewrote the Installation Instructions document completely to make it
  simpler. Some users had found it daunting.
  http://furius.ca/beancount/doc/install
 
- Tested Windows installation using Cygwin and added it to the
  installation
  instructions.
  http://furius.ca/beancount/doc/install


 --

 ---
 You received this message because you are subscribed to the Google Groups
 Ledger group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to ledger-cli+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 

--- 
You received this message because you are subscribed to the Google Groups 
Ledger group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Beancount Updates - 2015-06-29

[Martin Blais]

On Thu, Jul 2, 2015 at 3:23 PM, Martin Blais bl...@furius.ca wrote:

 On Wed, Jul 1, 2015 at 10:54 AM, Simon Michael si...@joyful.com wrote:

 Nice! Fewer clearer more cross-platform install steps is always good.


 Installation worked for me on osx just now. I may have had some
 prerequisites installed from before, such as python3 from brew.

 I thought I was done when bean-check ran, but just out of interest I ran
 bean-doctor from the next section and got a traceback showing that I
 still needed lxml. I confirmed that by re-reading your release notes and
 installed it with pip3 install lxml.


 Ah! How could I forget. Thanks for pointing this out.
 I added lxml to checkdeps.



 The next traceback mentioned
 datetime, so pip3 install python-datetime.


 You mean python-dateutil, not python-datetime.
 (datetime is part of the stdlib and should require no install.)



 Finally bean-doctor showed a
 nice list of dependencies and their status. I had bottle from before,
 and upgraded it with pip3 install bottle --upgrade, and ply sounds
 useful so pip3 install ply.


 You need PLY for bean-query.


 bean-doctor is nice, it helps reassure me that I won't be getting
 dependency-related tracebacks later while using the tool.


 bean-doctor checkdeps is only required to run Beancount directly from
 the clone.

 I'd been having problems with setup.py in that it's supposed to be able to
 install dependencies by default, so in theory you shouldn't have to install
 anything but Beancount now. It hadn't been working before.

 Here's some background, in case anyone's interested: Python has two active
 supporting projects for packaging:

 (1) distutils, which comes with it in the stdlib, and
 (2) setuptools, which is newer and better, but which needs to be installed
 on top of the stdlib.

 There's a deep history of why that is and many projects have come and gone
 trying to make this better (e.g. distribute). The main issue with disutils
 is that it does not support automatic installation of dependencies (it's
 really old but a ton of packages still depend on it, that's the price of
 popularity). Setuptools does support it, the packager can declare
 dependencies and pip3 should be able to pull in and install all
 requirements automatically.  I could not get setuptools to install my
 dependencies due to a namespace collision with the 'parser' module (was my
 fault).

 Now that I've made changes, Beancount supports setuptools and installs
 dependencies automatically. If you don't have setuptools installed, it
 falls back on distutils (and you have to install the dependencies manually,
 e.g. it's a one-liner:  pip3 install python-dateutil lxml bottle ply).


Okay, an update: the state of setuptools / distutils Python distribution
tools is in a really sorry state indeed. I've spend much of my afternoon
dealing with some more setuptools brokenness and running around its sad,
sad source code. When I use

  python3 setup.py build_ext -i

with setuptools installed, it puts the _parser.so library under src/python
and not under src/python/beancount/parser where it should be, so that's a
no go.

This is using setuptools v15. Manually installing the latest v18 puts the
.so in the right place but there is an annoying warning about not running
it with pip3.

#$*# this. No seriously, I've had enough. I want to spend my time building
features instead of packaging, and I've wasted the better part of a
beautiful vacation day just dealing with setuptools and reading conflicting
information on the web. I'm ditching support for setuptools.

Beancount will use distutils, which is basic, comes with all versions of
Python, and does the right thing by default. It's only drawback is that you
need to install the dependencies manually. It won't do the dependencies
automatically. It's a one-liner to install dependencies. No big deal in my
book.

Python seriously needs a brand new, rewritten from scratch packaging system.

-- 

--- 
You received this message because you are subscribed to the Google Groups 
Ledger group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.