Re: [GNC] CSV matching produces duplicates?

[flywire]

Original post about turning off Bayesian matching prompts:

flywire wrote
> It's not that hard to match most transactions and add accounts using a
> spreadsheet or other tool leaving the rest for manual confirmation. It
> would be reasonable to have a setting so they could be imported straight
> in
> like any other software.

This is very specific about turning off something that seems illogical in a
csv import process. There are plenty of csv import questions and there may
well be good workarounds I'm not aware of.

*Many people* manipulate csv files and GnuCash already allows (requires ??)
pre-processed csv files. Files that can't be imported directly from the bank
because:

1. Mmm date format is not supported
2. DESCRIPTION IS UPPERCASE (from saving bits and COBOL) and Mixed Case Is
Preferred For Readability

also,

3. there are no accounts but a 20 line lookup table adds accounts to 80% of
transactions
4. GnuCash accounts can be entered into the lookup table to ensure a match
(and changed after import if required)

In a previous comment I noted, it is not my data so I don't know what
account to assign to many transactions. (To reduce processing I assign
unaccounted transactions to entertainment and change those that aren't.) I
imagine many people process transactions for others without knowing the full
details when they are imported.



--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

It is true in that particular case of yours that the internal account and the
name it is mapped to are the same. The difficulty is that the importer has to be
able to cope with situations where "Assets:Cheque" in the external record may
not necessarily be mapped onto the Assets:Cheque internal GnuCash account. I
have been in the situation where I have had multiple checqueing accounts with
the same bank for different purposes with in some cases nothing much to
distinguish them. There is no guarantee that the name an external provider gives
to an account will be the same name you may use in your GnuCash accounts and
that it will necessarily match although it usually does in most cases. The
importer has to cope as far as possible with all possible use cases not just
your particular case. 

That the account mapping procedure exists is testimony to the fact the use case
where an external account name which is the same as the internal GnuCash name
but is not the actual account the transaction should go to occurs sufficiently
often that the developers have taken the time and trouble to develop a procedure
to make this possible. It is a one off process if you complete the importing
process and then that external name is permanently mapped onto the appropriate
 internal account.  

Guncash is not written for one individual's use case but to meet the range of
use cases that a wide range of users do encounter are likely and do encounter.


David
On Mon, 2021-05-17 at 05:31 -0500, flywire wrote:
> Sample account that shouldn't need manual matching:
> 
> <
> http://gnucash.1415818.n4.nabble.com/file/t374675/GnuCashCsvImportMatchingExample.png
> > 
> 
> 
> 
> --
> Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see 
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
-- 
Dr David R Cousens
B.Sc, M.Prof. Acc., Ph.D., G.C.Ed

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

I can't read that file on my smartphone,  but what are we supposed to do
with it?  Ask our bankers to change their csv format to match it?

On Mon, May 17, 2021, 5:32 AM flywire  wrote:

> Sample account that shouldn't need manual matching:
>
> <
> http://gnucash.1415818.n4.nabble.com/file/t374675/GnuCashCsvImportMatchingExample.png>
>
>
>
>
> --
> Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[flywire]

Sample account that shouldn't need manual matching:


 



--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

In my experience, my bank uses the word 'check' to describe every check
that clears my bank account and my credit card company describes every
purchase at Amazon as 'Amazon'.  I could continue, but I certainly do not
want all transactions with those descriptions transferring to the same
respective accounts.  I think very few users of GnuCash would want that.
The GnuCash import wizards have been working for many years with Bayesian
matching giving varying results attempting to automatically match imported
transactions with existing transactions and to assign transfer accounts to
unmatched transactions. The imported transactions often have intentionally
vague descriptions describing, at best, the method of payment rather than
the purpose of the payment.  That fact makes the Bayesian matching
extremely difficult to accomplish accurately but with sufficient training
it eventually starts to work reasonably well.

If you really want Gnucash to transfer every 'check' to the same transfer
account, you can use Bayesian matching to train it eventually to do that in
most cases.  With the new Import Map editor you can improve the training to
work even more to your liking.  I think that would be adequate for most
users.

On Fri, Mar 26, 2021 at 9:31 PM David Cousens 
wrote:

>
> For the transfer account, the second account in a transaction being
> imported, GnuCash does in fact work that way, i.e. it imports all
> transactions in which the transfer account has not been specified to an
> Imbalance account for the currency of the primary account in the
> transaction. The presence of transactions in the Imbalance accounts is
> intended to trigger users that the transaction description is incomplete
> and
> that the correct account should be selected to replace the Imbalance
> account
> for the transaction.  In the majority of cases the transfer account is
> going
> to be an Income or Expense account. Where a transfer account is specified
> though, the label has to have been previously  mapped onto an internal
> account or a new internal account created and specified for it to be
> imported. The transaction record to be imported must specify a primary
> account label which is usually an asset or liability account and that label
> has to be mapped to an internal asset/liability account otherwise it does
> not know where to import it to.
>
> You could have an option whereby if a matching account label does not exist
> for the primary account for a transaction, GnuCash creates an account with
> the name. In the majority of cases for imported transactions, the primary
> account is going to be an asset or liability account. However, account
> structures in accounting have a structure which reflects how the business
> (even where it is simply personal accounting) operates and is financed and
> the asset and liability account structures are fairly strictly defined by
> the assets and liabilities you actually have. I would not want to have
> myaccounting program randomly creating new accounts in these categories
> every time it came across a label which had not been assigned to an
> existing
> internal account and I suspect that after experiencing it you may not
> either.
>
> A first time user new to accounting is always going to have trouble working
> out what goes where and why  which is why the GnuCash guide has a very
> brief
> introduction to accounting basics but nothing really replaces making
> mistakes.  Unfortunately most of us don't consult the documentation until
> we
> strike a problem, me included. The beauty of GnuCash is that if you do
> stuff
> it up you can at least correct it once your knowledge has built up.
>
>
>
> -
> David Cousens
> --
> Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>


-- 
David Carlson
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

For the transfer account, the second account in a transaction being
imported, GnuCash does in fact work that way, i.e. it imports all
transactions in which the transfer account has not been specified to an
Imbalance account for the currency of the primary account in the
transaction. The presence of transactions in the Imbalance accounts is
intended to trigger users that the transaction description is incomplete and
that the correct account should be selected to replace the Imbalance account
for the transaction.  In the majority of cases the transfer account is going
to be an Income or Expense account. Where a transfer account is specified
though, the label has to have been previously  mapped onto an internal
account or a new internal account created and specified for it to be
imported. The transaction record to be imported must specify a primary
account label which is usually an asset or liability account and that label
has to be mapped to an internal asset/liability account otherwise it does
not know where to import it to. 

You could have an option whereby if a matching account label does not exist
for the primary account for a transaction, GnuCash creates an account with
the name. In the majority of cases for imported transactions, the primary 
account is going to be an asset or liability account. However, account
structures in accounting have a structure which reflects how the business
(even where it is simply personal accounting) operates and is financed and
the asset and liability account structures are fairly strictly defined by
the assets and liabilities you actually have. I would not want to have
myaccounting program randomly creating new accounts in these categories
every time it came across a label which had not been assigned to an existing
internal account and I suspect that after experiencing it you may not
either.

A first time user new to accounting is always going to have trouble working
out what goes where and why  which is why the GnuCash guide has a very brief
introduction to accounting basics but nothing really replaces making
mistakes.  Unfortunately most of us don't consult the documentation until we
strike a problem, me included. The beauty of GnuCash is that if you do stuff
it up you can at least correct it once your knowledge has built up.



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[flywire]

David, your response doesn't change:

> For csv transactions GnuCash pretty well prompts with the correct account
> each time, so [provide a setting] to accept all prompts.



1) From my experience it's unreasonable to assume user can assign accounts
when they don't understand the transactions. So, it IS easier to fix later.
2) Worst case, matcher wouldn't work which isn't an issue if it's not being
used to assign accounts. Elsewhere there's a suggestion to allow matcher to
be retrained on current data meaning it would work, and better.



--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

Not sure that the developers could make that suggestion work.  It is much
harder to find records which have been imported to the wrong account after
the fact than it is to identify the correct account the first time it is
imported at the time it is imported.

The account names as specified in a CSV file import are only labels unless
the CSV record contains the GUUID that has been assigned and is associated
with an internal account name in the GNuCash program. Account Names
internally are only labels and the GUUID is used in all transaction
processing internally. These GUUIDs are only available with an association
to a specific internal GnuCash account where the records have been exported
specifically from a GNuCash book/file using the GNUCash CSV export format
which includes the internal GUUID of any accounts in the record as well as
the account name labels .

For CSV records originating elsewhere, GnuCash has no way of knowing which
internal account is specifically matched to a given  label in an import
record until you create that association by making and confirming a
selection.  Once you have successfully imported a record which contains a
specific label, that label is then mapped onto the GUUID associated with the
account you selected to match the label to from that point on. 

The importer has to be able to cope with the situation where the name your
bank may choose to give an account or category in their records may not be
the name you may choose to give it internally. Miss one space in the label
or change or add one character and the label is also a separate distinct
entity. That is why it is necessary to confirm and select the specific
internal account the first time a label is encountered in an import record.
The problem is that the importer not only has to cope with your specific use
case but with all possible use cases in some 189 countries world wide.

What might be  possible is to preselect the internal account whose account
name most closely matches the imported label in the drop down list of
internal account names in the selection dialog but that final confirmation
step would still be necessary. I suspect when it was written, since it is a
once only operation,  whoever wrote it opted to have the user manually
select the internal account matching he wants associated with a specific
imported label, likely well before the Bayesian matching procedure was put
in for matching the transfer accounts. 

While it may be possible to adapt the Bayesian matching  process to do this,
it is by no means a 5 minute exercise and , so far no-one on the main
development team has considered it a high enough priority to address. If it
has not been enetered as a feature request bug in the bugs data base, it is
also unlikely to be addressed unless someone takes an interest in it. That
requires that someone to have  an understanding of the code not only in the
specific area but as some parts of the code are used in many different
areas, an overall understanding of the code base. Ive never counted how many
million lines of code are in GnuCash.  It also does not stop the operation
of Gnucash and issues which do affect that are usually the highest priority.
It is a relatively minor user inconvenience. Until and unless someone in the
volunteer force of developers is prepared to consider addressing the problem
it is unlikely to change.

Even with OFXQFX imports where the banks usually specify a GUUID for each
account and each transaction, those externally assigned GUUIDs have to be
mapped onto the GUUIDs created and associated with an account name
internally the first time a transaction with an external GUUID is imported.
Your bank could change a GUUID associated with an account at any time of its
choosing. Once that mapping is done however importing is extremely reliable
with OFX/QFX because there is a defined standard institutions have to agree
to - some still manage to stuff it up though. 

With CSV there is no such defined standard and every developer in every
institution can choose to do their own thing so the situation is chaotic and
importers have to cope with a wide range of different and sometimes aberrant
behaviour on the part of the file creators.



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[flywire]

Yes, this topic broke off the original thread, something to do with me
replying from email.

A chart of accounts imports easily, as expected with no issues or nagging.

For csv transactions GnuCash pretty well prompts with the correct account
each time, so set to accept all prompts. There is no reason to "create a
huge mess of accounts" as the hierarchy could be included in the account
field if required but I don't think that is the case. Some software forces
account names to be unique so say rent would have to be an income or expense
account and say rental used for the other one and maybe this is why defaults
mapped well for me. This functionality in (almost all ??) similar programs.



--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[Adrien Monteleone]

Curious, if you fix them afterwards, outside of the matcher, how do you 
expect the matcher to learn what you have done?


The place to correct the matcher is in the matcher. Anything outside of 
that, the matcher has no cognizance of.


Or am I misunderstanding the request?

Regards,
Adrien

On 3/25/21 8:23 PM, flywire wrote:

The matching of the account names... only happens...


This doesn't change the suggestion of allowing the user to import the file,
with accounts as coded, and fix tagged records afterwards (like any other
software).

To confirm "Car" account is "Car" account, wtf


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

Maybe your message title is throwing us off.  If you want to have Gnucash
create new accounts on the fly during the import, that is almost possible
except that it would create a huge mess of accounts that were not properly
assigned to valid top level accounts (asset, liability, income, expense
and equity).

Gnucash assigns attributes to accounts based on those top level accounts.
 There is no mechanism to automate that.  It is possible to import a valid
chart of accounts,  but I have never done it myself and I have no clue how
it works.

On Thu, Mar 25, 2021, 8:24 PM flywire  wrote:

> > The matching of the account names... only happens...
>
> This doesn't change the suggestion of allowing the user to import the file,
> with accounts as coded, and fix tagged records afterwards (like any other
> software).
>
> To confirm "Car" account is "Car" account, wtf
>
>
>
>
> --
> Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[flywire]

> The matching of the account names... only happens...

This doesn't change the suggestion of allowing the user to import the file,
with accounts as coded, and fix tagged records afterwards (like any other
software).

To confirm "Car" account is "Car" account, wtf




--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

The matching of the account names in the input records (whether transfer
accounts or the main account for the transaction) to the GnuCash internal
accounts only happens for the first time those accounts occur in an input
data file. After that GnuCash remembers the assignment of external account
names to its internal account names and will only bring that dialog up when
it encounters an account name in the input file which has not previously
been matched. 

The other part of the matching procedures looks for duplicates, matching
records to be updated and when no transfer file has been imported in the
data file it tries to find similar record imported in the past and will if
it gets a sufficient match, assign the same transfer account to the record
being imported



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[flywire]

Yes, there would be no matching decisions if it was coded in the input file
but you won't be powering through it, GnuCash still requires user to verify
each [iirc] account (which is coded in the input file). I'm suggesting user
should be able to turn that checking off (since input file has been
checked) and during the input process GnuCash could tag any records it has
issues with.
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

Are you suggesting that sometimes you don't need to make any matching
decisions during the csv import process?  If so, it is already fairly easy
to power through the matching steps.

In my case,  there are nearly always many decisions to make on the 練 so
that GnuCash is usually "bugging" me to make a backup halfway through.

On Sat, Mar 20, 2021, 4:34 PM flywire  wrote:

> It's not that hard to match most transactions and add accounts using a
> spreadsheet or other tool leaving the rest for manual confirmation. It
> would be reasonable to have a setting so they could be imported straight in
> like any other software.
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

That is a pretty reasonable summary of the position with CSV imports. The
records in CSV do not contain the additional information that identifies a
particular transaction uniquely that is present in the OFX file so its
detection of duplicates is not as reliable as with OFX.  The Bayesian
process is used primarily in matching the transfer account to the data based
on previous assignments and not for the duplicate detection AFAIK. That is
primarily  based on a score of matches of the data in selected fields
between transactions in the date range around the date of the data being
imported.

When you get the no match message I think you can force the match editor to
come up (this is another popup dialog in addition to the Import Matcher
window). It has been a long time since I have had to use it  but if I
remember correctly it is something like a click or double click in any area
of the record other than the transfer account field. I don't have time to
check it out this morning unfortunately as I have an appointment but I will
try it out later today or tomorrow and report back. This will show you
transactions most closely matching the record being imported and you can
force selection of a particular match. I don't think this feature is very
well described in the documentation.  I wrote a lot of the current
documentation largely from experimentation with the program while matching
it to the code, but there were a few areas  where I hadn't managed to work
out exactly how they worked. When I have a bit of spare time I'll take
another look at that area.

My bank is pretty good with the date transaction range selection, so I can
set the 1st of the month to the last of the month and it wont download any
transactions either side of that. The only problems now are when i have
manually entered transactions with a date slightly different from the bank's
date. transaction clearing in banks locally is generally minutes these days
so it is rare for the dates to deliver. A few years back it was far more
likely.

Once the transactions have had the transfer account assigned, they usually
appear in the account register even though they have not yet been finally
imported. I use multiple screens so having the account register and the
import matcher window displayed side by side makes it easier.  I mainly rely
on reconciliation with the downloaded statements to find any really
difficult to locate duplicates.

David Cousens



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

David Cousens,

As Elmar pointed out, clicking the 'A' box  for an incoming transaction
forces the import even if there is an existing match.  That is worse than
just accepting every transaction as new and going back and deleting
duplicates later (this has essentially been my procedure for the past few
months of supposed training of the new Bayesian matcher).  I mostly use
manual OFX imports except for a few accounts where that is not offered.

He also points out that sometimes there is an existing match but the
importer does not find it or allow it to be manually matched during the
import.  I do see that too, although it is infrequent for me.

In any case, there is very little of the potential labor saving from the
automation actually happening



On Wed, Mar 3, 2021 at 6:38 PM David Cousens 
wrote:

> Elmar,
>
> If a transaction has been matched and is marked to be either updated (U
> column checkbox selected) or cleared (C column checkbox selected), you have
> to deselect the selected checkboxes in the U (up[date) and C (Clear)
> column(
> previously labelled the R column in earlier GC versions) and then select
> the
> checkbox in the A column for the transaction row to force it to import. It
> is then likely you will have to assign a transfer account before importing.
> The importer does learn particularly in the account matching part of the
> procedure and it is important for the learning process that this is done
> before the final import of the batch of transactions
>
> David.
>
>
>
> -
> David Cousens
> --
> Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>


-- 
David Carlson
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

Elmar,

If a transaction has been matched and is marked to be either updated (U
column checkbox selected) or cleared (C column checkbox selected), you have
to deselect the selected checkboxes in the U (up[date) and C (Clear) column(
previously labelled the R column in earlier GC versions) and then select the
checkbox in the A column for the transaction row to force it to import. It
is then likely you will have to assign a transfer account before importing.
The importer does learn particularly in the account matching part of the
procedure and it is important for the learning process that this is done
before the final import of the batch of transactions

David. 



-
David Cousens
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Cousens]

Elmar,
It is possible to do some limited tuning of the matching algorithm. Edit->
Preferences->Import although the adjustment of the parameters and what their
effects are is by no means intuitive or clear. The matching time window is hard
coded in so I regularly get matches to regularly repeated transactions in the
previous month when I am importing CSV data which is fortunately failry
infrequent these days. Particularly on CSV imports I check very carefully before
accepting GnuCash's matches. 

The OFX files have unique UUIDs on each transaction  whereas the CSV data
imports do not have unique identifiers that can be used to distinguish different
transactions and any hand entered entries will not have the external UUID
assigned by the bank.  The matcher is not prefect. but it does generally work
reasonably.
David Cousens
 On Wed, 2021-03-03 at 16:58 -0500, Elmar wrote:
> Version 4.4 from flatpak on linux mint 19.3  ( I think my sig needs to 
> be fixed, but I misremember how to do that)
> 
> Since my OFX import to my bank seems now broken, I have been reduced to 
> CSV imports.  And I find that the matcher does not match worth beans, 
> producing duplicates of tranasctions that in fact DO match within the 
> one month window.  Is this a know problem?  Is there a way to tune the 
> matcher better? - Elmar
> 
> 
> 
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see 
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
-- 
Dr David R Cousens
B.Sc, M.Prof. Acc., Ph.D., G.C.Ed
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[Elmar]

Even if I know that the match exists, and I tell it that it does, it 
replies that the match does not exist.  So I have to either leave it red 
so it doesn't import, or accept the duplicate.  Which means keeping both 
windows open (the match window and the account window), and manually 
checking the matches myself.  Which defeats the entire purpose of 
computer matching.


On 3/3/21 6:13 PM, David Carlson wrote:
Isn't it still possible to manually override the automatic behavior on 
a transaction by transaction basis during the various types of imports?


I know that is not wanted, but that is what we have to do to train the 
Bayesian matching.  I am frustrated that the new matcher released in 
the 3.x series is not learning to make matches that it did very well 
in the old matcher.  If anyone has suggestions in this regard I am all 
ears.


On Wed, Mar 3, 2021 at 4:46 PM Elmar > wrote:


No - just taken from the bank as CSV directly.  I have another post
where the OFX problem is described.

On 3/3/21 5:40 PM, Christopher Lam wrote:
> You mean transactions imported via ofx then reimported with CSV?
This
> may be https://bugs.gnucash.org/show_bug.cgi?id=798128
>
> For now, it'll be best to narrow down your CSV import to exclude
the
> already imported date range.
>
> On Thu, 4 Mar 2021, 5:59 am Elmar, mailto:etsc...@gmail.com>
> >> wrote:
>
>     Version 4.4 from flatpak on linux mint 19.3  ( I think my sig
>     needs to
>     be fixed, but I misremember how to do that)
>
>     Since my OFX import to my bank seems now broken, I have been
>     reduced to
>     CSV imports.  And I find that the matcher does not match
worth beans,
>     producing duplicates of tranasctions that in fact DO match
within the
>     one month window.  Is this a know problem?  Is there a way
to tune
>     the
>     matcher better? - Elmar
>
>
>
>     ___
>     gnucash-user mailing list
> gnucash-user@gnucash.org 
>
>     To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>     If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
>     -
>     Please remember to CC this list on all your replies.
>     You can do this by using Reply-To-List or Reply-All.
>

___
gnucash-user mailing list
gnucash-user@gnucash.org 
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.



--
David Carlson


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[David Carlson]

Isn't it still possible to manually override the automatic behavior on a
transaction by transaction basis during the various types of imports?

I know that is not wanted, but that is what we have to do to train the
Bayesian matching.  I am frustrated that the new matcher released in the
3.x series is not learning to make matches that it did very well in the old
matcher.  If anyone has suggestions in this regard I am all ears.

On Wed, Mar 3, 2021 at 4:46 PM Elmar  wrote:

> No - just taken from the bank as CSV directly.  I have another post
> where the OFX problem is described.
>
> On 3/3/21 5:40 PM, Christopher Lam wrote:
> > You mean transactions imported via ofx then reimported with CSV? This
> > may be https://bugs.gnucash.org/show_bug.cgi?id=798128
> >
> > For now, it'll be best to narrow down your CSV import to exclude the
> > already imported date range.
> >
> > On Thu, 4 Mar 2021, 5:59 am Elmar,  > > wrote:
> >
> > Version 4.4 from flatpak on linux mint 19.3  ( I think my sig
> > needs to
> > be fixed, but I misremember how to do that)
> >
> > Since my OFX import to my bank seems now broken, I have been
> > reduced to
> > CSV imports.  And I find that the matcher does not match worth beans,
> > producing duplicates of tranasctions that in fact DO match within the
> > one month window.  Is this a know problem?  Is there a way to tune
> > the
> > matcher better? - Elmar
> >
> >
> >
> > ___
> > gnucash-user mailing list
> > gnucash-user@gnucash.org 
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
> > https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
> >
>
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>


-- 
David Carlson
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[Elmar]

No - just taken from the bank as CSV directly.  I have another post 
where the OFX problem is described.


On 3/3/21 5:40 PM, Christopher Lam wrote:
You mean transactions imported via ofx then reimported with CSV? This 
may be https://bugs.gnucash.org/show_bug.cgi?id=798128


For now, it'll be best to narrow down your CSV import to exclude the 
already imported date range.


On Thu, 4 Mar 2021, 5:59 am Elmar, > wrote:


Version 4.4 from flatpak on linux mint 19.3  ( I think my sig
needs to
be fixed, but I misremember how to do that)

Since my OFX import to my bank seems now broken, I have been
reduced to
CSV imports.  And I find that the matcher does not match worth beans,
producing duplicates of tranasctions that in fact DO match within the
one month window.  Is this a know problem?  Is there a way to tune
the
matcher better? - Elmar



___
gnucash-user mailing list
gnucash-user@gnucash.org 
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.



___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] CSV matching produces duplicates?

[Christopher Lam]

You mean transactions imported via ofx then reimported with CSV? This may
be https://bugs.gnucash.org/show_bug.cgi?id=798128

For now, it'll be best to narrow down your CSV import to exclude the
already imported date range.

On Thu, 4 Mar 2021, 5:59 am Elmar,  wrote:

> Version 4.4 from flatpak on linux mint 19.3  ( I think my sig needs to
> be fixed, but I misremember how to do that)
>
> Since my OFX import to my bank seems now broken, I have been reduced to
> CSV imports.  And I find that the matcher does not match worth beans,
> producing duplicates of tranasctions that in fact DO match within the
> one month window.  Is this a know problem?  Is there a way to tune the
> matcher better? - Elmar
>
>
>
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.