Drag-n-drop the file on QGIS canvas! Regards Saber
On 10 January 2018 at 11:26, Jeroen Hovens <[email protected]> wrote: > At the moment I make the selection using the Select by expression tool. I > then choose one of the Gem2017 values (each object is an address, and has > one of 388 unique Gem217 values). I want to select all objects bwith one > unique Gem2017 value. > Making the selection is no problem, I can make any selection I want. > Making the selection usually takes less then a minute. > The problem is in copying specific selections to the clipboard or saving a > selection as a new file. Copying a selection of about 20.000 objects from > the 7,5 mln takes about a second. Unless the issue arises, then the copy > process takes forever > > I have no experience with Postgis or using databases. I tried saving the > csv file as a sqlite, but then had no idea how to re-open that file. The > dialogbox got me all confused ☹ > > > > > -----Oorspronkelijk bericht----- > Van: Richard Duivenvoorde [mailto:[email protected]] > Verzonden: woensdag 10 januari 2018 10:21 > Aan: Jeroen Hovens <[email protected]>; 'Ramon Andinach' < > [email protected]>; 'qgis-user' <[email protected]> > Onderwerp: Re: [Qgis-user] ctrl-c selection issue > > > Hi Jeroen, > > Loading a csv in QGIS is not optimal. I think csv does not have an index, > so when you create a small selection, QGIS has to run over all those 7,5 > mln object every time. > > Not sure how you do your 'selecting'? > Is it spatial or based on zip/postcode/buurtnr/wijknr? > > I think I would either load it in Postgis (if available) or save as > GeoPackage (better use QGIS 2.99 for that) or if all fails, create a > shapefile for it. AND then create indexes on desired columns. > > IF you are searching for strings like zipcodes in that list, I think tools > like grep (available in OSGeo4W too!) work best: > > Eg searching fro 3 zip codes is milliseconds work: > > grep '2022ZJ\|2211ZE\|1016XX' pc6hnr20170801_gwb.csv > > If all fails, give me a call. > > Regards, > > Richard Duivenvoorde > > On 10-01-18 00:33, Jeroen Hovens wrote: > > Hmm, I can’t use Edit > Paste As, because qgis does not finish the > > Edit > >> Copy command. > > > > > > > > The csv file is about 270 MB, 7,5 mln objects with 5 attributes, no > > x-y data. > > > > If you want to recreate the issue, the file can be found here: > > > > https://www.cbs.nl/nl-nl/maatwerk/2017/38/buurt-wijk-en-gemeente-2017- > > voor-postcode-huisnummer > > > > I run into this issue when selecting all objects with Attribute Gem217 > > = 310 > > > > 355 also has this issue. 321 and 356 work without a problem. > > > > > > > > I am using qgis here just as a means to chop the csv file into smaller > > selections which I can then use to edit, add attributes and data to > > and make maps of them. > > > > > > > > Gr. > > > > Jeroen > > > > > > > > *Van:* Qgis-user [mailto:[email protected]] *Namens > > *Ramon Andinach > > *Verzonden:* dinsdag 9 januari 2018 23:35 > > *Aan:* qgis-user <[email protected]> > > *Onderwerp:* Re: [Qgis-user] ctrl-c selection issue > > > > > > > > Just so we’ve got just about everything covered, does this happen for > > “Edit-> Paste As”? > > > > > > > > Also looking at the size of your original CSV, I’d go along with > > Jeroen, and try and import that csv into some other format first. > > > > If you can find a way of making the selection without going near the > > attribute table that will help too. > > > > > > > > > > > > Regards, > > > > Ramon. > > > > On 10 Jan 2018, at 06:26, Jeroen Hovens <[email protected] > > <mailto:[email protected]>> wrote: > > > > > > > > Thanks for the ideas. > > > > > > > > The option to save the selection as new csv file as Ramon suggested, > > doesn’t work; qgis goes into ‘not responding’ and no csv file is > > added to the layer list. However, a csv file is made, lacking a > > decent number of objects (first test 15095 of 18582 objects and > > second test is 15179 of 18582 objects, so missing about 3,5k > > objects). This csv file can be opened manually in qgis. > > > > And I checked: this options does work for selections where the > > ctrl-c option also worked. Then the new cvs layer is added to the > > layer list in no time. > > > > > > > > Is there some sort of log file in qgis that might give clues to the > > ‘not responding? > > > > > > > > @Kirk > > > > I did close and reopen qgis multiple times, even shut down the pc > > entirely, but to no effect. Selections that did allow for copying > > earlier, still work: I can make that selection again and ctrl-c > > works fine. Seems to me this rules out memory-issues. > > > > > > > > Gr. > > > > Jeroen > > > > > > > > > > > > *Van:* Qgis-user > > [mailto:[email protected]] *Namens *Kirk Schmidt > > *Verzonden:* dinsdag 9 januari 2018 22:20 > > *Aan:* [email protected] <mailto:[email protected]> > > *Onderwerp:* Re: [Qgis-user] ctrl-c selection issue > > > > > > > > After a few selections, you may have to close and reopen qgis to > > purge the memory as well. Importing a csv is memory intensive so > > another approach is to try saving as a spatial database (SQLite) or > > even a shapefile. Once written, you can load the spatial file and > > you should have lots of memory to keep selecting points. > > > > > > > > Kirk > > > > On 1/9/2018 4:58 PM, Ramon Andinach wrote: > > > > I’m not sure what the problem is, but if I were doing that > > repeatedly, I’d probably use Layer -> Save As, tick the Save > > Only Selected Features box and save it to CSV. Then muck about > > with excel later. > > > > > > > > This way ought to be a bit lighter on the memory. > > > > > > > > > > > > Ramon. > > > > On 10 Jan 2018, at 04:25, Jeroen Hovens <[email protected] > > <mailto:[email protected]>> wrote: > > > > > > > > Hi, > > > > > > > > Using qgis 2.14.18 I have been working today with a large > > csv file (7.5 mln objects). I just need selections of this > > database which I can then modify in excel. So I opened the > > attribute table and made a selection based on a specific > > value for an attribute, and copied that selection (about > > 30.000 objects) via the clipboard (ctrl-c) to a new > > Microsoft Excel file, then saved the excel file and repeated > > the process for another selection. This worked perfect for > > some time. However, some selections won’t copy: when I press > > the copy button the programs turn to ‘not responding’ and > > won’t snap out of it anymore. Selection size when this > > happens is about the same (about 20k to 35k objects). > > > > > > > > What could be the problem? Could the data be corrupted? How > > can I check what is wrong? > > > > > > > > Thank you > > > > Gr. > > > > Jeroen > > > > > > > > _______________________________________________ > > Qgis-user mailing list > > [email protected] <mailto:[email protected]> > > List info: https://lists.osgeo.org/ > mailman/listinfo/qgis-user > > Unsubscribe: > > https://lists.osgeo.org/mailman/listinfo/qgis-user > > > > > > > > > > > > > > > > _______________________________________________ > > > > Qgis-user mailing list > > > > [email protected] <mailto:[email protected]> > > > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > > > > Unsubscribe: > > https://lists.osgeo.org/mailman/listinfo/qgis-user > > > > > > > > > > -- > > > > Kirk Schmidt, MScF, BScF, RPF > > > > General Manager > > > > Nortek Resource Solutions Inc. > > > > RR # 1 > > > > Thorburn, NS > > > > B0K 1W0 > > > > Tel (902) 922.3607 > > > > Email: [email protected] <mailto:[email protected]> > > > > Web: www.nortekreources.com <http://www.nortekreources.com/> > > > > _______________________________________________ > > Qgis-user mailing list > > [email protected] <mailto:[email protected]> > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > > > > > > > > > > > > _______________________________________________ > > Qgis-user mailing list > > [email protected] > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > > > > > _______________________________________________ > Qgis-user mailing list > [email protected] > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user > -- Saber Razmjooei www.lutraconsulting.co.uk +44 (0)7568 129733
_______________________________________________ Qgis-user mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-user Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
