Hello Moritz,
It looks like I got some results, but I suspect there are still some
issues due to the warning messages.
I either kept getting that a file couldn't be found or that it had
trouble running the R-script.
My actions:
1. I found that it was looking in "C:\Program Files\QGIS
2.16.0\apps\Python27\Lib" for certain scripts but I also had another
location of the Python27 library "C:\Program Files\GRASS GIS
7.4.0\Python27\Lib". So I tried adding the second location via the
black terminal because I figured it needed the GRASS
versions/formats (with my lack of experience I am not sure if I
succeeded).
2. It was still failing, and I suspected this had to do with the qbwwv
voting method (see output). So I unchecked that option and got
results. I'am also sending a screenshot of the attribute table with
results.
Do you have any suggestions for improvement?
Soon I will test with a much bigger area for a land-use planning project.
/The last 2 command outputs:/
Running R now. Following output is R output.
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
Loading required package: caret
Loading required package: lattice
Loading required package: ggplot2
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.
Error in `$<-.data.frame`(`*tmp*`, vote_qbwwv, value = numeric(0)) :
replacement has 0 rows, data has 1965
Calls: $<- -> $<-.data.frame
Execution halted
ERROR: There was an error in the execution of the R script.
Please check the R output.
(Mon Jun 18 15:36:47 2018) Command finished (1 min 24 sec)
(Mon Jun 18 15:50:30 2018)
v.class.mlR -i --overwrite
segments_map=Segments_vector_Stats_Ben_test@haarlooj_Ben_Test
training_map=Training_Ben_Grass@haarlooj_Ben_Test
raster_segments_map=best5_myregion1_at_haarlooj_Ben_Test_rank1@haarlooj_Ben_Test
train_class_column=ecosystem output_class_column=vote
output_prob_column=prob classifiers=svmRadial,rf,C5.0 folds=5
partitions=10 tunelength=10 weighting_metric=accuracy
r_script_file=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_mlR-script3
processes=2
Running R now. Following output is R output.
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
Loading required package: caret
Loading required package: lattice
Loading required package: ggplot2
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.
Finished running R.
Loading results into attribute table
(Mon Jun 18 15:51:58 2018) Command finished (1 min 27 sec)
Best,
Jamille Haarloo
Department of Natural Resources and Environmental Assessment (NARENA)
Centre for Agricultural Research in Suriname (CELOS)
<http://www.celos.sr.org/>
Prof. Dr. Ir. Jan Ruinardlaan
AdeKUS University campus
Phone: 439982
On Thu, Jun 14, 2018 at 11:02 AM, Jamille Haarloo <[email protected]
<mailto:[email protected]>> wrote:
Hello Moritz,
Sorry about that. I am sending the zip of the location. The output
at the other machine (system cannot find the file specified) was due
to location/ directory issues and has been resolved, but also stops
executing the R-script saying there were more than 50 errors. Been
looking for similar cases online but haven't found many.
haarlooj_Ben_Test.zip
<https://drive.google.com/file/d/1wQqmreMbbd3oRfoVXWIIGJOG9L63miV6/view?usp=drive_web>
On Thu, Jun 14, 2018 at 5:19 AM, Moritz Lennert
<[email protected] <mailto:[email protected]>>
wrote:
P.S. Your vector data also does not contain any attribute
information as this is contained in a separate sqlite file. Just
another reason never to touch the contents of a GRASS database
directly, but only using dedicated tools. :-)
On 14/06/18 10:11, Moritz Lennert wrote:
Hi Jamille,
Sorry, but I've been hopping from meeting to meeting these
days and
haven't had the opportunity to look at your data.
Actually, the form you transmit them in is very
inconvenient. If you
want to transmit GRASS data to someone, you should either
zip an entire
location directory and send that, or export the data with
r.pack/v.pack.
In this form it is difficult to know which projection is
used and I have
to handcraft the data by copying the files into different
directories.
So, please resend your data in a format that I can easily use.
Best wishes,
Moritz
On 11/06/18 20:55, Jamille Haarloo wrote:
Hello Moritz,
I managed to edit my copy of v.class.mlR with the lines
you sent. But I
am still getting errors on two different machines.
I am sending you the data (hope I didn't miss anything)
and the command
outputs. I think there is sth wrong with my computer,
but also with one
of the files because of the error on the other machine.
_Computer I normally use:_
v.class.mlR -i --overwrite
segments_map=Segments_vector_Stats_Ben_test@haarlooj_Ben_Test
training_map=Training_Ben5@haarlooj_Ben_Test
raster_segments_map=best5_myregion1_at_haarlooj_Ben_Test_rank1@haarlooj_Ben_Test
train_class_column=Ecosystem output_class_column=vote
output_prob_column=prob classifiers=svmRadial,rf,C5.0
folds=5
partitions=10 tunelength=10 weighting_modes=smv,qbwwv
weighting_metric=accuracy
Running R now. Following output is R output.
During startup - Warning messages:
1: Setting LC_CTYPE=en_US.cp1252 failed
2: Setting LC_COLLATE=en_US.cp1252 failed
3: Setting LC_TIME=en_US.cp1252 failed
4: Setting LC_MONETARY=en_US.cp1252 failed
Loading required package: caret
Loading required package: lattice
Loading required package: ggplot2
There were 50 or more warnings (use warnings() to see
the first 50)
There were 50 or more warnings (use warnings() to see
the first 50)
Error in `$<-.data.frame`(`*tmp*`, vote_qbwwv, value =
numeric(0)) :
replacement has 0 rows, data has 1965
Calls: $<- -> $<-.data.frame
Execution halted
ERROR: There was an error in the execution of the R script.
Please check the R output.
_The output from the other machine:_
(Mon Jun 11 13:27:39 2018)
v.class.mlR -i --overwrite
segments_map=Segments_vector_Stats_Ben_test@haarlooj_Ben_Test
training_map=Training_Ben5@haarlooj_Ben_Test
raster_segments_map=best5_myregion1_at_haarlooj_Ben_Test_rank1@haarlooj_Ben_Test
train_class_column=Ecosystem output_class_column=vote
output_prob_column=prob classifiers=svmRadial,rf,C5.0
folds=5
partitions=10 tunelength=10 weighting_metric=accuracy
Running R now. Following output is R output.
Traceback (most recent call last):
File
"C:\Users\HaarlooJ\AppData\Roaming\GRASS7\addons/scri
pts/v.class.mlR.py <http://v.class.mlR.py>
<http://v.class.mlR.py>", line 639, in <module>
File
"C:\Users\HaarlooJ\AppData\Roaming\GRASS7\addons/scri
pts/v.class.mlR.py <http://v.class.mlR.py>
<http://v.class.mlR.py>", line 576, in main
shutil.copy(r_commands, r_script_file)
File "C:\Program Files\GRASS GIS
7.4.0\Python27\lib\subprocess.py", line 537, in check_call
retcode = call(*popenargs, **kwargs)
File "C:\Program Files\GRASS GIS
7.4.0\Python27\lib\subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Program Files\GRASS GIS
7.4.0\Python27\lib\subprocess.py", line 711, in __init__
errread, errwrite)
File "C:\Program Files\GRASS GIS
7.4.0\Python27\lib\subprocess.py", line 948, in
_execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file
specified
(Mon Jun 11 13:27:42 2018) Command finished (3 sec)
On Mon, Jun 11, 2018 at 5:47 AM, Moritz Lennert
<[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
Hi Jamille,
Le Fri, 8 Jun 2018 16:14:45 -0300,
Jamille Haarloo <[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> a écrit :
> Hello Moritz,
>
> This time I asked a vector to be created with
the stats and used this
> to extract training polygons in QGIS and
imported the training map in
> GRASS. I had to do some interventions regarding
the column names to
> make sure they are the same except for the class.
> I still get an error, and the only thing I could
trace is the fact
> that values are missing in some rows for both
vectors. I am not sure
> if I should correct this/ retry it all.
I haven't seen this before, so yes, please try to
eliminate the rows
with missing values. How did you get the feature
variables and how come
there are missing values ?
I don't have the time to test this right now, so I
prefer not to commit
as is, but you could try to edit your copy of
v.class.mlR to add the
four lines marked with a plus:
r_file.write('features <- read.csv("%s",
sep="%s", header=TRUE,
row.names=1)' % (feature_vars, separator))
r_file.write("\n")
+ r_file.write("features <- na.omit(features)")
+ r_file.write("\n")
r_file.write('training <- read.csv("%s",
sep="%s", header=TRUE,
row.names=1)' % (training_vars, separator))
r_file.write("\n")
+ r_file.write("training <- na.omit(training)")
+ r_file.write("\n")
This would eliminate all lines that have at least
one missing value.
Another option would be for you to send me the
data (segments
and training) privately, so that I can test.
Moritz
>
> This is the command output:
>
> (Fri Jun 08 15:48:28 2018)
>
> v.class.mlR -i --overwrite
>
segments_map=Segments_vector_Stats_Ben_test@haarlooj_Ben_Test
> training_map=Training_Ben5@haarlooj_Ben_Test
>
raster_segments_map=best5_myregion1_at_haarlooj_Ben_Test_rank1@haarlooj_Ben_Test
> train_class_column=Ecosystem
output_class_column=vote
> output_prob_column=prob
classifiers=svmRadial,rf,C5.0 folds=5
> partitions=10 tunelength=10
weighting_modes=smv,qbwwv
> weighting_metric=accuracy
>
classification_results=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_Classifier-results
>
accuracy_file=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_Classifier-accuracy
>
model_details=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_Classifier-module-runs
>
bw_plot_file=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_Classifier-performance
>
r_script_file=C:\Users\haarlooj\Documents\CELOS\v.class.mIRR_optional_output\Ben_test_R_script
> processes=3 Running R now. Following output is
R output.
> During startup - Warning messages:
> 1: Setting LC_CTYPE=en_US.cp1252 failed
> 2: Setting LC_COLLATE=en_US.cp1252 failed
> 3: Setting LC_TIME=en_US.cp1252 failed
> 4: Setting LC_MONETARY=en_US.cp1252 failed
> Loading required package: caret
> Loading required package: lattice
> Loading required package: ggplot2
> Loading required package: foreach
> Loading required package: iterators
> Loading required package: parallel
> During startup - Warning messages:
> 1: Setting LC_CTYPE=en_US.cp1252 failed
> 2: Setting LC_COLLATE=en_US.cp1252 failed
> 3: Setting LC_TIME=en_US.cp1252 failed
> 4: Setting LC_MONETARY=en_US.cp1252 failed
> During startup - Warning messages:
> 1: Setting LC_CTYPE=en_US.cp1252 failed
> 2: Setting LC_COLLATE=en_US.cp1252 failed
> 3: Setting LC_TIME=en_US.cp1252 failed
> 4: Setting LC_MONETARY=en_US.cp1252 failed
> During startup - Warning messages:
> 1: Setting LC_CTYPE=en_US.cp1252 failed
> 2: Setting LC_COLLATE=en_US.cp1252 failed
> 3: Setting LC_TIME=en_US.cp1252 failed
> 4: Setting LC_MONETARY=en_US.cp1252 failed
> Warning message:
> In nominalTrainWorkflow(x = x, y = y, wts =
weights, info =
> trainInfo, : There were missing values in
resampled performance
> measures. Error in `$<-.data.frame`(`*tmp*`,
vote_qbwwv, value =
> numeric(0)) : replacement has 0 rows, data has 1965
> Calls: $<- -> $<-.data.frame
> Execution halted
> ERROR: There was an error in the execution of
the R script.
> Please check the R output.
> (Fri Jun 08 15:49:32 2018) Command finished (1
min 4 sec)
>
>
>
> Best,
> Jamille
>
>
>
>
> On Thu, Jun 7, 2018 at 11:09 AM, Jamille Haarloo
> <[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
>
> > Hello Moritz,
> >
> > No worries. Thankful these modules are made
available for newbies
> > in RS like me and also happy these
interactions are possible for
> > learning. Hope to get back soon after some
adjustments.
> >
> > Best,
> > Jamille
> >
> > On Thu, Jun 7, 2018 at 10:44 AM, Moritz Lennert <
> > [email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
> >
> >> Thanks
> >>
> >> On 07/06/18 15:17, Jamille Haarloo wrote:
> >>
> >>> The first 20+ lines of Stats_Training_Ben_test:
> >>>
> >>>
cat,area,perimeter,compact_circle,compact_square,fd,WV_Benat
> >>>
imofo_1_min,WV_Benatimofo_1_max,WV_Benatimofo_1_range,WV_Ben
> >>>
atimofo_1_mean,WV_Benatimofo_1_stddev,WV_Benatimofo_1_varia
> >>>
nce,WV_Benatimofo_1_coeff_var,WV_Benatimofo_1_sum,WV_
> >>>
Benatimofo_1_first_quart,WV_Benatimofo_1_median,WV_Benatim
> >>>
ofo_1_third_quart,WV_Benatimofo_2_min,WV_Benatimofo_2_max,
> >>>
WV_Benatimofo_2_range,WV_Benatimofo_2_mean,WV_Benatimofo_2_
> >>>
stddev,WV_Benatimofo_2_variance,WV_Benatimofo_2_coeff_var,
> >>>
WV_Benatimofo_2_sum,WV_Benatimofo_2_first_quart,WV_
> >>>
Benatimofo_2_median,WV_Benatimofo_2_third_quart,WV_Benatimof
> >>>
o_3_min,WV_Benatimofo_3_max,WV_Benatimofo_3_range,WV_Benat
> >>>
imofo_3_mean,WV_Benatimofo_3_stddev,WV_Benatimofo_3_varianc
> >>>
e,WV_Benatimofo_3_coeff_var,WV_Benatimofo_3_sum,WV_
> >>>
Benatimofo_3_first_quart,WV_Benatimofo_3_median,WV_Benatim
> >>>
ofo_3_third_quart,WV_Benatimofo_4_min,WV_Benatimofo_4_max,
> >>>
WV_Benatimofo_4_range,WV_Benatimofo_4_mean,WV_Benatimofo_4_
> >>>
stddev,WV_Benatimofo_4_variance,WV_Benatimofo_4_coeff_var,
> >>>
WV_Benatimofo_4_sum,WV_Benatimofo_4_first_quart,WV_
> >>> Benatimofo_4_median,WV_Benatimofo_4_third_quart
> >>>
1144,3832.000000,1256.000000,5.723635,0.197144,1.729624,13,7
> >>>
6,63,46.4097077244259,9.98454911351384,99.69122100017,21.513
> >>>
9237092391,177842,40,47,53,40,138,98,90.2687891440501,15.250
> >>>
0825418009,232.565017531741,16.8940812061464,345910,81,92,
> >>>
100,15,61,46,40.8582985386221,7.82663897784868,61.2562776895
> >>>
802,19.1555675536767,156569,36,42,47,28,124,96,68.42536534
> >>>
44676,13.5774536655369,184.347248039801,19.8427200164517,262206,59,68,77
> >>>
1145,12092.000000,2282.000000,5.854120,0.192750,1.645226,13,
> >>>
94,81,51.386288455177,10.5294376761475,110.869057775874,20.4
> >>>
907534532914,621363,45,52,59,21,220,199,114.230731061859,23.
> >>>
3590328249442,545.644414516822,20.4489917973953,1381278,101,
> >>>
114,128,7,76,69,46.4219318557724,8.42747122371732,71.
> >>>
0222712265835,18.1540726264915,561334,42,48,52,17,198,181,
> >>>
97.2732385047966,22.492313569247,505.904169697333,23.
> >>> 1228176577445,1176228,84,97,110
> >>>
> >>> [...]
> >>
> >> ---------------------
> >>> All the lines of the output of v.db.select
> >>> Training_Ben2@haarlooj_Ben_Tes t:
> >>>
> >>> cat|id|Type|code
> >>> 1|4|B29|18
> >>> 2|5|B31|19
> >>> 3|3|B28|17
> >>>
> >>
> >>
> >> Again a lack of clear documentation on my
side: both the training
> >> and the segment info should contain the same
attributes, with only
> >> additional one column ('code' in your case)
present in the
> >> training data.
> >>
> >> It should be possible to do this
differently, i.e. provide the
> >> module with the features of all segments,
and only the id/cat of
> >> each training segment with the relevant
class and have the module
> >> merge the two, but this is not implemented, yet.
> >>
> >> I also just notice that you have the word
'Training' in both
names.
> >>
> >> The segment_file/segment_map contains the
info (cat + all feature
> >> variables) of all segments you wish to
classify, either in the
> >> form of a csv file or in the form of a
vector map with the info in
> >> the attribute table.
> >>
> >> The training_file/training_map contains the
info (cat + all
feature
> >> variables + class) of the training data.
Often this is an extract
> >> of the former, but not necessarily.
> >>
> >> All columns in the training file have to be
present in the segment
> >> file, except for the class column (your 'code').
> >>
> >> Sorry for the lack of docs. This module has
mostly been used
> >> internally here and so we are not always
aware of the unclear and
> >> missing parts. Having your feedback has been
very useful !
> >>
> >> Moritz
> >>
> >>
> >
