Re: [GNC] importing account info
Alan, I think there have been some problems around the AQbanking direct importation of OFX data in 3.8 and changes in the associated library that have been reported in the user mailng list. The banks in my country (australia) are still a millenium or two out of date and don't provide direct connection OFX data except through very selected commercial accounting products they have partner relationships with so I have had no experience with this. They claim this is for security reasons but unfortunately we consumers have little control over their choices. i have voiced my objections but have been ignored. I have not yet noticed any problems in 3.8 importing the downloaded files from the bank but if the change back to 3.7 fixed yours then that is great. David - David Cousens -- Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] importing account info
David, thanks for all the analysis; I'm working through the information. I solved the problem by uninstalling 3.8 and installing 3.7; the info I needed flowed right in. It's most likely my machine, but I wonder if there's a bug. On 2/9/2020 2:32 PM, David Cousens wrote: Alan, As the message says, the imported record is in a currency/security that does not match the that of the account that GnuCash has assigned to it. This could mean either the commodity data in the input record is incorrect or the matcher already has stored map information which is assigning it an account which is in the wrong currency. It is unlikely the data from your broker may be incorrect unless the name of the security registeredon the exchange has been changed. Just to be sure I would check at least for the first record that the input OFX file has the commodity that is assigned to the account you expect the transactions's second split to be assigned to. "Invesco Preferred ETF" is not the name of an account but should be the name of the commodity/security specified in the input data. The GnuCash assigned account has a commodity name "iShares Russell 2000" which is the same as the account name, also wierd. I would think that the "iShares Russell 2000" account would likely have a USD ( or whatever is appropriate to your locale) currency. If it is not a n account of type stock that may be causing the confusion Did you manually choose the account "iShares Russel 2000" or did GnuCash supply that as a match? If it was the latter then somehow the account matching data may have been screwed up. I had some wierd account assignments when I first transition to V3 but haven't had that on the later v3 minor version transitions. If you choose Yes in the popup, you should get the account assignment dialog where you can assign the correct account then at least that record is correct. I would check the commodity of the account you wish to assign and make sure it matches the commodity identified in the qfx input data file first (in the Accounts tab, highlight the account, right click and select Edit Account from the pop up menu. The security/currency box should match the code in the input data). If assigning the correct transfer accounts allows that record to be imported and then repeats the same error on the next record, my guess is that the matcher data is screwed up. There are two approaches from here. Either manually correct each record by pressing Yes and assigning the correct account. A pain if you have a lot of records but if you complete the import process, the new data will be imported into the matching data for an account. The second approach is to use the Tools->Import Map Editor to edit the information that the account assignment is based on. This post http://gnucash.1415818.n4.nabble.com/GNC-import-map-editor-td4699101.html may contain some useful information. Section 6.15.8 in the help manual has a little information. It is pretty generic (I wrote it) as my understanding of this part of the code is still very limited. The matcher works by tokenizing information in the descrition field (not sure about the memo fields) and matching that to a list of tokens associated with transactions which have been imported in the past and matched to a specific account. Gnucash calculates a score for the number of matching tokens in the description and each account to which assignments of transaction splits weighted by the count of the frequency of each tokens appearance in transactions which have been assigned in past imports and assigns the highest scoring account. This is my current understanding of the account matching operation and it is only from a fairly cursory exploration of the code at this stage and a little but not much experimentation. I have intended to get back and explore it in more detail but life catches up with me sometimes. if you examine the token strings for the account Gnucash is incorrectly assigning and the tokens associated withthe account you want to be assigned along with the input record description you may be able to identify why the wrong account is being assigned and delete the appropriate tokens from the account. This has a high risk of stuffing up future account assignments. I would play with it on a copy of your data file and a small import data subset until you can get it to work if you take this route. The account matching data is stored in the XML data file so any work done on the copy will not automatically transfer back to your main datafile. If the datafile contains a lot of incorrect matching information as it is a Bayesian system continuing to fix the assigned accounts and completing the import will eventually fix things as it will change the weightings on tokens in the files. Maybe importing in smaller batches, fixing each batch up as you go may be the safest way to approach this. I would like to see a feature which would allow the existing transactions in selected accounts to be used in
Re: [GNC] importing account info
Alan, As the message says, the imported record is in a currency/security that does not match the that of the account that GnuCash has assigned to it. This could mean either the commodity data in the input record is incorrect or the matcher already has stored map information which is assigning it an account which is in the wrong currency. It is unlikely the data from your broker may be incorrect unless the name of the security registeredon the exchange has been changed. Just to be sure I would check at least for the first record that the input OFX file has the commodity that is assigned to the account you expect the transactions's second split to be assigned to. "Invesco Preferred ETF" is not the name of an account but should be the name of the commodity/security specified in the input data. The GnuCash assigned account has a commodity name "iShares Russell 2000" which is the same as the account name, also wierd. I would think that the "iShares Russell 2000" account would likely have a USD ( or whatever is appropriate to your locale) currency. If it is not a n account of type stock that may be causing the confusion Did you manually choose the account "iShares Russel 2000" or did GnuCash supply that as a match? If it was the latter then somehow the account matching data may have been screwed up. I had some wierd account assignments when I first transition to V3 but haven't had that on the later v3 minor version transitions. If you choose Yes in the popup, you should get the account assignment dialog where you can assign the correct account then at least that record is correct. I would check the commodity of the account you wish to assign and make sure it matches the commodity identified in the qfx input data file first (in the Accounts tab, highlight the account, right click and select Edit Account from the pop up menu. The security/currency box should match the code in the input data). If assigning the correct transfer accounts allows that record to be imported and then repeats the same error on the next record, my guess is that the matcher data is screwed up. There are two approaches from here. Either manually correct each record by pressing Yes and assigning the correct account. A pain if you have a lot of records but if you complete the import process, the new data will be imported into the matching data for an account. The second approach is to use the Tools->Import Map Editor to edit the information that the account assignment is based on. This post http://gnucash.1415818.n4.nabble.com/GNC-import-map-editor-td4699101.html may contain some useful information. Section 6.15.8 in the help manual has a little information. It is pretty generic (I wrote it) as my understanding of this part of the code is still very limited. The matcher works by tokenizing information in the descrition field (not sure about the memo fields) and matching that to a list of tokens associated with transactions which have been imported in the past and matched to a specific account. Gnucash calculates a score for the number of matching tokens in the description and each account to which assignments of transaction splits weighted by the count of the frequency of each tokens appearance in transactions which have been assigned in past imports and assigns the highest scoring account. This is my current understanding of the account matching operation and it is only from a fairly cursory exploration of the code at this stage and a little but not much experimentation. I have intended to get back and explore it in more detail but life catches up with me sometimes. if you examine the token strings for the account Gnucash is incorrectly assigning and the tokens associated withthe account you want to be assigned along with the input record description you may be able to identify why the wrong account is being assigned and delete the appropriate tokens from the account. This has a high risk of stuffing up future account assignments. I would play with it on a copy of your data file and a small import data subset until you can get it to work if you take this route. The account matching data is stored in the XML data file so any work done on the copy will not automatically transfer back to your main datafile. If the datafile contains a lot of incorrect matching information as it is a Bayesian system continuing to fix the assigned accounts and completing the import will eventually fix things as it will change the weightings on tokens in the files. Maybe importing in smaller batches, fixing each batch up as you go may be the safest way to approach this. I would like to see a feature which would allow the existing transactions in selected accounts to be used in generating the match data in addition to just the imported data. If this was resticted by date range one could selectively use the existing data for transactions to an account to provide training information as well as imported data. Good luck David Cousens hope this
Re: [GNC] importing account info
Mailman probably stripped it because it was HTML. The PNG came through fine though. Regards, Adrien > On Feb 9, 2020 w7d40, at 10:59 AM, Alan Schold wrote: > > Not sure why you couldn't see my screenshot. My program showed the attachment > to be 70.8 KB. Anyway, I saved it as a .png; maybe that will help. ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] importing account info
Not sure why you couldn't see my screenshot. My program showed the attachment to be 70.8 KB. Anyway, I saved it as a .png; maybe that will help. On 2/5/2020 3:04 PM, David Cousens wrote: Alan, Your screenshot is not attaching correctly, it is only 2 bytes. If you can attach it as a jpg or png file it may come through. >From what version of GnuCash did you upgrade to V3.8 ? The importer was completely rewritten in the upgrade from 2.6.x to V3. I had to completely retrain the Bayesian matching in the import matcher after the transition. I don't fully understand why as the information is stored in the xml datafile. Geert maybe able to explain what caused this. I think there may have been some changes to the parsing of tokens and maybe storage, but I am not sure of this. If the import matcher is incorrectly assigning an account, it is important that it is corrected in the import matcher and not later in the account register after importing as only assignments made during the import process are used in defining the import matcher database of tokenized information from the description field. That information is only stored when you complete the import process after assigning any unassigned transfer accounts and correcting any mistakes the automatic process made. At present no use is made of the information in existing transactions unless they have passed through the importer. I don't import QFX files but I do import OFX files where the structre is similar. After the initial problem going from 2.6.19 to 3.0 and retraining the matcher by importing data a month at a time and checking the account assignments carefully before completing the import, my original account assignments have largely been restored. Double clicking on a transaction in the import matcher window will bring up the account asiignment dialog. It is also possible to edit the information used for the matching process. If incorrect information has been stored there it can take some time and several imports to the same account to correct the information. Tools->Import Map Editor. You may need to exercise a great deal of caution and have a good idea of how the bayesian matching process works to successfully edit the data. So far I have looke at it but have not tried it. It usues combinations of the tokenized information to produce the overall weighting for a specific transfer account to be assigned. If it is only a single account that is causing a problem it should be possible to find the data which is causing the wrong assignment and delete it. AFAIK the Import Map Editor has not been documented yet. I have been upgrading some of the import process documentation but haven't got to this as yet and I am not at all familiar with the code in this area. I can't see what screen it is that is coming up but whether the account picker dialog or a list of the possible matches is displayed depends upon the state of the checkboxes labelle"A", "U" and "R" and it sounds like it was the latter nad not the former. If the A checkbox is checked, the transaction has been identified as not matching an existing transaction and flagged for import. It may have either a grren or gold background color. If green the matcher has assigned a transfer account and you will need to check it has made a correct choice. If gold it has not been able to match a transfer account and you will need to assign a transfer account to it. By default it is assigned to an Imbalance account if you do not assign an account before completing the import. IDouble clicking the transaction will bring up the account picker dialog. It is important for the training of the Bayesian match picker that the transfer account be assigned at this point. The dialog allows you to create new accounts if this should be necessary If the R checkbox is checked it has matched an existing transaction with the transaction to be imported and flagged it not to be imported and it will usually have a green background. The bar code displayed will give some idea of the quality of the match. Here double clicking will display the data for the imported transaction and the best matched transaction and any other close matches. You can select another of the close matches if it is the correct transaction to match. At this point you might decide that the import matcher has got it wrong in this case and the transaction needs to be imported as it isn't a match to any existing transaction. Cancel out of the display of matching transactions and click on the A checkbox to flag it for import. GnuCash will then attempt to assign a transfer account and the background will change to either green or gold. Again you can now accept or reject the assigned account or change it as above If the U checkbox is checked.it usually means any attempt at a match to any existing transaction had some limited success but not enough to mark the transaction as a match. It will have a red/pink background to indicate that the transaction requires
Re: [GNC] importing account info
Thanks, David, I'll get started on your suggestions. I updated from v 3.4 On 2/5/2020 3:04 PM, David Cousens wrote: Alan, Your screenshot is not attaching correctly, it is only 2 bytes. If you can attach it as a jpg or png file it may come through. >From what version of GnuCash did you upgrade to V3.8 ? The importer was completely rewritten in the upgrade from 2.6.x to V3. I had to completely retrain the Bayesian matching in the import matcher after the transition. I don't fully understand why as the information is stored in the xml datafile. Geert maybe able to explain what caused this. I think there may have been some changes to the parsing of tokens and maybe storage, but I am not sure of this. If the import matcher is incorrectly assigning an account, it is important that it is corrected in the import matcher and not later in the account register after importing as only assignments made during the import process are used in defining the import matcher database of tokenized information from the description field. That information is only stored when you complete the import process after assigning any unassigned transfer accounts and correcting any mistakes the automatic process made. At present no use is made of the information in existing transactions unless they have passed through the importer. I don't import QFX files but I do import OFX files where the structre is similar. After the initial problem going from 2.6.19 to 3.0 and retraining the matcher by importing data a month at a time and checking the account assignments carefully before completing the import, my original account assignments have largely been restored. Double clicking on a transaction in the import matcher window will bring up the account asiignment dialog. It is also possible to edit the information used for the matching process. If incorrect information has been stored there it can take some time and several imports to the same account to correct the information. Tools->Import Map Editor. You may need to exercise a great deal of caution and have a good idea of how the bayesian matching process works to successfully edit the data. So far I have looke at it but have not tried it. It usues combinations of the tokenized information to produce the overall weighting for a specific transfer account to be assigned. If it is only a single account that is causing a problem it should be possible to find the data which is causing the wrong assignment and delete it. AFAIK the Import Map Editor has not been documented yet. I have been upgrading some of the import process documentation but haven't got to this as yet and I am not at all familiar with the code in this area. I can't see what screen it is that is coming up but whether the account picker dialog or a list of the possible matches is displayed depends upon the state of the checkboxes labelle"A", "U" and "R" and it sounds like it was the latter nad not the former. If the A checkbox is checked, the transaction has been identified as not matching an existing transaction and flagged for import. It may have either a grren or gold background color. If green the matcher has assigned a transfer account and you will need to check it has made a correct choice. If gold it has not been able to match a transfer account and you will need to assign a transfer account to it. By default it is assigned to an Imbalance account if you do not assign an account before completing the import. IDouble clicking the transaction will bring up the account picker dialog. It is important for the training of the Bayesian match picker that the transfer account be assigned at this point. The dialog allows you to create new accounts if this should be necessary If the R checkbox is checked it has matched an existing transaction with the transaction to be imported and flagged it not to be imported and it will usually have a green background. The bar code displayed will give some idea of the quality of the match. Here double clicking will display the data for the imported transaction and the best matched transaction and any other close matches. You can select another of the close matches if it is the correct transaction to match. At this point you might decide that the import matcher has got it wrong in this case and the transaction needs to be imported as it isn't a match to any existing transaction. Cancel out of the display of matching transactions and click on the A checkbox to flag it for import. GnuCash will then attempt to assign a transfer account and the background will change to either green or gold. Again you can now accept or reject the assigned account or change it as above If the U checkbox is checked.it usually means any attempt at a match to any existing transaction had some limited success but not enough to mark the transaction as a match. It will have a red/pink background to indicate that the transaction requires some manual decision. A double click will bring up the dialog displaying
Re: [GNC] importing account info
Alan, Your screenshot is not attaching correctly, it is only 2 bytes. If you can attach it as a jpg or png file it may come through. >From what version of GnuCash did you upgrade to V3.8 ? The importer was completely rewritten in the upgrade from 2.6.x to V3. I had to completely retrain the Bayesian matching in the import matcher after the transition. I don't fully understand why as the information is stored in the xml datafile. Geert maybe able to explain what caused this. I think there may have been some changes to the parsing of tokens and maybe storage, but I am not sure of this. If the import matcher is incorrectly assigning an account, it is important that it is corrected in the import matcher and not later in the account register after importing as only assignments made during the import process are used in defining the import matcher database of tokenized information from the description field. That information is only stored when you complete the import process after assigning any unassigned transfer accounts and correcting any mistakes the automatic process made. At present no use is made of the information in existing transactions unless they have passed through the importer. I don't import QFX files but I do import OFX files where the structre is similar. After the initial problem going from 2.6.19 to 3.0 and retraining the matcher by importing data a month at a time and checking the account assignments carefully before completing the import, my original account assignments have largely been restored. Double clicking on a transaction in the import matcher window will bring up the account asiignment dialog. It is also possible to edit the information used for the matching process. If incorrect information has been stored there it can take some time and several imports to the same account to correct the information. Tools->Import Map Editor. You may need to exercise a great deal of caution and have a good idea of how the bayesian matching process works to successfully edit the data. So far I have looke at it but have not tried it. It usues combinations of the tokenized information to produce the overall weighting for a specific transfer account to be assigned. If it is only a single account that is causing a problem it should be possible to find the data which is causing the wrong assignment and delete it. AFAIK the Import Map Editor has not been documented yet. I have been upgrading some of the import process documentation but haven't got to this as yet and I am not at all familiar with the code in this area. I can't see what screen it is that is coming up but whether the account picker dialog or a list of the possible matches is displayed depends upon the state of the checkboxes labelle"A", "U" and "R" and it sounds like it was the latter nad not the former. If the A checkbox is checked, the transaction has been identified as not matching an existing transaction and flagged for import. It may have either a grren or gold background color. If green the matcher has assigned a transfer account and you will need to check it has made a correct choice. If gold it has not been able to match a transfer account and you will need to assign a transfer account to it. By default it is assigned to an Imbalance account if you do not assign an account before completing the import. IDouble clicking the transaction will bring up the account picker dialog. It is important for the training of the Bayesian match picker that the transfer account be assigned at this point. The dialog allows you to create new accounts if this should be necessary If the R checkbox is checked it has matched an existing transaction with the transaction to be imported and flagged it not to be imported and it will usually have a green background. The bar code displayed will give some idea of the quality of the match. Here double clicking will display the data for the imported transaction and the best matched transaction and any other close matches. You can select another of the close matches if it is the correct transaction to match. At this point you might decide that the import matcher has got it wrong in this case and the transaction needs to be imported as it isn't a match to any existing transaction. Cancel out of the display of matching transactions and click on the A checkbox to flag it for import. GnuCash will then attempt to assign a transfer account and the background will change to either green or gold. Again you can now accept or reject the assigned account or change it as above If the U checkbox is checked.it usually means any attempt at a match to any existing transaction had some limited success but not enough to mark the transaction as a match. It will have a red/pink background to indicate that the transaction requires some manual decision. A double click will bring up the dialog displaying the matching existing transaction information. You can select one of the matches if it is appropriate then accept it
[GNC] importing account info
I see several importing questions, but none speak to this problem. I hope this explanation is not too wordy. I download activity weekly from several trading accounts. I save them temporarily as .qfx files which I then import into the same accounts in GnuCash. This has worked perfectly until recently when I upgraded to 3.8. Since then GnuCash sees each import as belonging to some unrelated issue in my wife’s account. For each account I get the screen which I attached. Neither of the accounts referenced are the from/to account. If I choose “yes”, the same screen returns. If I choose “no”, one transaction is listed. This repeats for each transaction. I cannot get out of the process until all transactions are listed; then I can cancel (because the information will be saved to the wrong account & it is incomplete). I have tried deleting the incorrect account, but the program just finds another one. I can’t think of anything else I’ve done differently. ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
