Re: Importing transactions from bank.
You'll need to use something like this, https://gist.github.com/mterwill/7fdcc573dc1aa158648aacd4e33786e8#file-importers-chase-py, or make your own: https://docs.google.com/document/d/11EwQdujzEo2cxqaF5PgxCEZXWfKKQCYSMfdJowp_1S8/edit -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To post to this group, send email to beancount@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/41f2c7ca-6aae-42d7-9e87-45080536e393%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Issue #273: Trailing whitespace when printing with price (blais/beancount)
New issue 273: Trailing whitespace when printing with price
https://bitbucket.org/blais/beancount/issues/273/trailing-whitespace-when-printing-with
Michael Droogleever:
To reproduce:
```python
>>> from beancount.parser import printer
>>> from beancount.core import data
>>> import datetime as dt
>>> p1 = data.Posting("Account", data.Amount.from_string("10.00 CHF"), None,
>>> data.Amount.from_string("1 USD"), None, None)
>>> p2 = data.Posting("Account", data.Amount.from_string("-10.00 USD"), None,
>>> None, None, None)
>>> t = data.Transaction(None, dt.date(2017,1,1), '*', "", "Narration String",
>>> data.EMPTY_SET, data.EMPTY_SET, [p1, p2])
>>> print(printer.format_entry(t).replace(' ', '~'))
2017-01-01~*~"Narration~String"
~~Account~~~10.00~CHF~@~1~USD
~~Account~~-10.00~USD
```
The second posting has trailing whitespace to match the length of price, but
this causes a visual warning in many IDEs for trailing whitespace. e.g. in fava.
--
You received this message because you are subscribed to the Google Groups
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to beancount+unsubscr...@googlegroups.com.
To post to this group, send email to beancount@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/beancount/20180406194718.38336.50955%40celery-worker-108.ash1.bb-inf.net.
For more options, visit https://groups.google.com/d/optout.
Issue #272: D string format l10n (blais/beancount)
New issue 272: D string format l10n
https://bitbucket.org/blais/beancount/issues/272/d-string-format-l10n
Michael Droogleever:
Localisation of the D() Decimal str parser. Currently it only accepts numbers
with a '.' as the decimal separator and ',' as the thousands separator. e.g.
"1,123.45"
This has been mentioned frequently before, in the context of wider i18n issues,
but I could not find a specific issue for D. I am happy to create fix if simple
enough. More thousands separators could be added to the regex, but that is not
very sustainable.
Proposed fix, passes all existing beancount/core unittests:
```diff
--- a/beancount/core/number.py Thu Apr 05 01:01:06 2018 -0400
+++ b/beancount/core/number.py Fri Apr 06 00:07:17 2018 +0200
@@ -15,7 +15,10 @@
import types
import warnings
import re
+import locale
+# TODO determine if locale can be pulled from options
+# TODO determine if locale needs to be restoed to 'C' portable locale after
+locale.setlocale(locale.LC_ALL, '')
# Note: Python 3.3 is supposed to guarantee a fast "C" decimal implementation,
# as it comes with its source code. However, in practice, many distributions,
@@ -67,10 +71,10 @@
# errors that would occur from attempts to access incomplete data.
class MISSING: pass
# Regular expression for parsing a number in Python.
NUMBER_RE = r"[+-]?\s*[0-9,]*(?:\.[0-9]*)?"
-_CLEAN_NUMBER_RE = re.compile('[, ]')
+_CLEAN_NUMBER_RE = re.compile("[,']")
# pylint: disable=invalid-name
def D(strord=None):
@@ -91,6 +95,10 @@
if strord is None or strord == '':
return Decimal()
elif isinstance(strord, str):
+strord = strord.replace(' ', '')
+# Try using user's locale first
+if locale.getlocale() != (None, None):
+return Decimal(locale.delocalize(strord))
return Decimal(_CLEAN_NUMBER_RE.sub('', strord))
elif isinstance(strord, Decimal):
return strord
```
--
You received this message because you are subscribed to the Google Groups
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to beancount+unsubscr...@googlegroups.com.
To post to this group, send email to beancount@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/beancount/20180405222514.2706.57428%40celery-worker-110.ash1.bb-inf.net.
For more options, visit https://groups.google.com/d/optout.
Issue #243: ingest/importers/fileonly.py test_match failing (blais/beancount)
New issue 243: ingest/importers/fileonly.py test_match failing https://bitbucket.org/blais/beancount/issues/243/ingest-importers-fileonlypy-test_match Michael Droogleever: This is linked to #211. Believed to be a system specific issue. ``` #!python = FAILURES == __ TestFileOnly.test_match __ self = , filename = '/tmp/tmp3aqvnoxf' @unittest.skipIf(not file_type.magic, 'python-magic is not installed') @test_utils.docfile def test_match(self, filename): """\ DATE,TYPE,REF #,DESCRIPTION,FEES,AMOUNT,BALANCE 2014-04-14,BUY,14167001,BOUGHT +CSKO 50 @98.35,7.95,-4925.45,25674.63 2014-05-08,BUY,12040838,BOUGHT +HOOL 121 @79.11,7.95,-9580.26,16094.37 """ importer = fileonly.Importer( ['Filename: .*te?mp.*', 'MimeType: text/plain', 'Contents:\n.*DATE,TYPE,REF #,DESCRIPTION,FEES,AMOUNT'], 'Assets:BofA:Checking', basename='bofa') file = cache._FileMemo(filename) > self.assertTrue(importer.identify(file)) E AssertionError: False is not true fileonly_test.py:35: AssertionError --- Captured stdout call text/x-Algol68 ``` -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To post to this group, send email to beancount@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180317214321.33845.4754%40celery-worker-108.ash1.bb-inf.net. For more options, visit https://groups.google.com/d/optout.
Adding transactions to beancount ledger
I believe it is against the design of beancount, but is there any existing code which attempts to add transactions to an existing beancount file. Assuming the entries in the file are grouped by asset account, it would need to append the entry to the subsection of entries all from the same account. -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To post to this group, send email to beancount@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7e031f82-3826-49ed-9494-15582eb718af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Trouble running the unit tests
Just to add in case anyone else comes across this: You probably want to use `pip install -e ./path/to/beancount` if installing from a repo (see editable install) And Nose is deprecated as far as I know, but pytest is a replacement which is compatible with unittest and nose, and works for me: `pip install pytest` On Sunday, 25 February 2018 22:03:32 UTC+1, Jeff Brantley wrote: > > I want to try running the unit tests on Windows (native), but first, I > thought it wise to run them on Ubuntu to make sure I knew how to run them, > and to get a baseline idea of how many tests are supposed to run and > whether they are all passing. The first problem I'm hitting is that the > suite stalls after 302 tests at test_extract_examples, railing out a CPU > core and running indefinitely (at least a couple minutes before I killed > it). > > Before I attempt to debug the test---which employs some amount of > indirection to call out to a main function---I wanted to ask about two > things: > > 1. Is this a known or expected problem? > 2. Under what conditions/environment do I need to run the test? Or, > Martin, what do you do, and how flexible is that? > > In a nutshell (details later), my current setup involves a proper Ubuntu > 17.10 in a VM (not WSL), installing from source into a virtualenv, and then > invoking nosetests against the installation (not the repo). Trying it > against the repo yielded a bunch of errors that I did not investigate for > now. *Is this a reasonable arrangement? What is the appropriate setup for > running these tests?* > > *Details:* > Ubuntu 17.10 Desktop 64-bit running in VMWare Player > Existing packages updated. > Installed Python: 3.6.3 > sudo apt install python3-venv python3-pip python3-dev > Setup a python virtualenv using python3 -m venv ENVNAME > With the virtual environment activated: > pip install wheel (to avoid gripes about unable to build local wheels of > dependencies) > pip install nose > cd path/to/repo/beancount > pip install . > cd path/to/elsewere > nosetests -v beancount > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To post to this group, send email to beancount@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20e2fbe1-f785-4b13-ae89-7c93b2e7117a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Issue #215: Library of importers, for different filetypes and sources (blais/beancount)
Hi Martin, Thanks for the answer, it should have been obvious this was not a new idea, but I had sadly not come across LedgerHub nor its post-mortem in my reading; apologies for getting you to write it all out again. Seems you had a really good attempt but got a strong dose of realism about the amount of interest. Would probably need a critical mass of existing content to really get going. If the CSV importer refactor I attempted does get pulled in, I might have a go at a revival (but limiting it to beancount specific). I'd probably solve the anonymity issues by encouraging that patches be sent in by email which will then use a dummy account to create PRs..., and maybe manually recreate the exact same beancount data in all the various formats instead of trying to anonymise personal data, avoiding risking leaking something. Regression testing will be affected, but I don't think stability is that important. It might also be possible to find enough info online about various bank formats to create a good number importers from the get go. I'll post here if/when I do something. > Hi Michael, > > I suspect you may not be familiar with the history on this; there's a fair > amount of background. > > The "ingest" code that lives inside of Beancount today was born in a > different project I made, called LedgerHub, whose intent was precisely to > do this. You can read more about it here: > https://docs.google.com/document/d/11u1sWv7H7Ykbc7ayS4M9V3yKqcuTY7LJ3n1tgnEN2Hk/ > (design > doc) Beancount wasn't originally going to contain any importing code, it > was to be done there, and the dream was to have people contribute many > importers. > > It did not really take off; I have found that people do not share their > importers, or that there is insufficient overlap between all our > institutions that it's not worthwhile to share them. I wrote a detailed > post-mortem here: > https://docs.google.com/document/d/1Bln8Zo11Cvez2rdEgpnM-oBHC1B6uPC18Qm7ulobolM/ > > Moveover, anonymizing downloads from institutions is really time-consuming > and uncertain and I found myself unable to really do that, even for all the > importers I shared. The best way to test your importers is to run > regression tests on real files, and that is best done in the privacy of a > personal code repository. Finally, I think that even sharing the list of > the particular financial institutions you use can consist in a security > liability. > > I have since ported over all the "common" code to beancount.ingest, where > it lives today. The CSV importer that lives there is intended as an > example, though it's growing to become more and more useful (it's the basis > for my own CSV importers, for instance). > > If you really believe in the importer sharing idea I would encourage you > to start another repository and do this. This should not be integrated in > the Beancount repo, however. > > Thanks, and happy to discuss more on the mailing-list, > On Tuesday, 13 February 2018 21:38:54 UTC+1, Michael Droogleever wrote: > > New issue 215: Library of importers, for different filetypes and sources > > https://bitbucket.org/blais/beancount/issues/215/library-of-importers-for-different > > > Michael Droogleever: > > People are lazy. New users want to see software work for them before they > commit to using it. For most new beancount users, especially those that are > not already using a double entry accounting system, this involves importing > data exported from their financial institutions. Beancount does not include > many importers at the moment, there is no formal way of requesting for new > ones, and anyone who does make one for themselves has no obvious way of > sharing this with other users. The end result, frequently, is many unknown > repos containing individual importers. > > Suggestion: > > + Provide a simple way for those who do not wish to program an importer to > provide sample files that need to be imported (and give guidelines for > anonymising them). > > + Use this to make a list of importers which need to be made, ideally > sorted using some sort of vote system to determine interest. > > + Create a procedure, naming scheme, save location etc. for adding > importers. Invite people to make PRs to add importers. Simplify the process > and encourage tests. > > + Finally, advertise the list of importers included with beancount, > "batteries included", and provide simple generic guides on how to use them. > > The goal is to have a sizeable coverage of the major financial > institutions around the world and their file formats. This will hopefully > increase the interest in this software and plain text accounting. > > > --
Issue #215: Library of importers, for different filetypes and sources (blais/beancount)
New issue 215: Library of importers, for different filetypes and sources https://bitbucket.org/blais/beancount/issues/215/library-of-importers-for-different Michael Droogleever: People are lazy. New users want to see software work for them before they commit to using it. For most new beancount users, especially those that are not already using a double entry accounting system, this involves importing data exported from their financial institutions. Beancount does not include many importers at the moment, there is no formal way of requesting for new ones, and anyone who does make one for themselves has no obvious way of sharing this with other users. The end result, frequently, is many unknown repos containing individual importers. Suggestion: + Provide a simple way for those who do not wish to program an importer to provide sample files that need to be imported (and give guidelines for anonymising them). + Use this to make a list of importers which need to be made, ideally sorted using some sort of vote system to determine interest. + Create a procedure, naming scheme, save location etc. for adding importers. Invite people to make PRs to add importers. Simplify the process and encourage tests. + Finally, advertise the list of importers included with beancount, "batteries included", and provide simple generic guides on how to use them. The goal is to have a sizeable coverage of the major financial institutions around the world and their file formats. This will hopefully increase the interest in this software and plain text accounting. -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To post to this group, send email to beancount@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180213203852.8299.34125%40celery-worker-109.ash1.bb-inf.net. For more options, visit https://groups.google.com/d/optout.
