Hello,
A short reaction to my report of the bug in sort_routes.py as mentioned
earlier in this thread: the bug is still present in sumo 0.20.0. So when
applied, the list of sorted vehicles is inserted halfway the comments,
making the whole file getting commented out. A fix is simple, as I already
proposed, but I changed my fix a bit this time because I found an error in
my fix. When discussing the problem with DFRouter generating vehicles, I
already mentioned that I couldn't find the route distribution probabilities
in the vehicle file. I now understand that this is because these lines are
thrown away when applying sort_routes.py.
Attached is my version; I took the version from sumo 0.20.0, the only
change I made is in the else-part of the main function at the bottom. Sorry
that I don't have time to obtain commit rights and patch such fixes myself
right now, so therefore I just keep you informed.
Greetings,
Pieter
On 19 February 2014 20:31, Pieter Loof <[email protected]> wrote:
> Hello,
>
> Ah, that makes sense. However, the output also contains an error. The
> complete list of vehicle definitions is in a block comment, since the
> configuration comments at the start of the file are not properly closed
> before the vehicle definitions are written. I attached the file that
> sort_routes.py generated, I only removed all vehicles in between the first
> and the last one.
>
> In the python code I see that the original lines are copied until it reads
> "<routes". But in comment lines there is a flag definition "<routes-output"
> so the copying ends at that line unintendedly. Apart from that, all the
> route distributions and vehicle definitions in the file that dfrouter
> generated start after a line with "<additional>" instead of "<routes>". So
> I guess the python file should be altered a bit here. I fixed the python
> file for my case, I'm not sure how generic it is. But I attached the
> changed python file so you can have a look at it. I only changed the
> contents of the for-loop in the main method.
>
> Greetings,
> Pieter
>
>
> On 19 February 2014 16:03, Jakob Erdmann <[email protected]>wrote:
>
>> Hello,
>> you need to sort the emitters file (containing <vehicle> elements).
>> regards,
>> Jakob
>>
>>
>> 2014-02-19 12:15 GMT+01:00 Pieter Loof <[email protected]>:
>>
>> Hello,
>>>
>>> Thank you for the useful reactions, I see there is a tool for sorting
>>> the route file. However, when I run sort_routes.py I get the output "read 0
>>> elements. wrote 0 elements." and no errors. The output file that is
>>> generated contains some comments, but no route data. Now that I think of
>>> it, doesn't the python script need the vehicle file in order to be able to
>>> sort the routes?
>>>
>>> I read these reactions just after I sent a mail with an overview of my
>>> bug reports (which also includes this scenario as "sumo_segfault" but then
>>> with screenshot and reduced vehicle file size). So if the crash problem we
>>> are discussing is solved already then you can ignore this scenario that I
>>> just sent again in my bug overview message.
>>>
>>> Greetings,
>>> Pieter
>>>
>>>
>>> On 19 February 2014 09:16, Jakob Erdmann <[email protected]>wrote:
>>>
>>>> Hello,
>>>> the crash appears to be resolved by changes in the latest svn revision
>>>> (not just related to slight changes in driving dynamics).
>>>> regards,
>>>> Jakob
>>>>
>>>>
>>>> 2014-02-19 8:26 GMT+01:00 Jakob Erdmann <[email protected]>:
>>>>
>>>> Hello,
>>>>> thank for for reporting the crash and sending the files.. I was able
>>>>> to reproduce it with 0.19.0 and will look into the matter. As a workaround
>>>>> you may want to use the latest svn version which ran the scenario without
>>>>> crashing.
>>>>>
>>>>> Please note that the option --lanechange.duration which you are using
>>>>> is rather new and bugs were recently discovered: (
>>>>> http://sumo-sim.org/trac.wsgi/ticket/1152,
>>>>> http://sumo-sim.org/trac.wsgi/ticket/1153).
>>>>>
>>>>> Regarding the problem of unsorted vehicles in the dfrouter output,
>>>>> please look at the updated documentation here:
>>>>> http://sumo-sim.org/wiki/Demand/Routes_from_Observation_Points#How_to_include_the_files
>>>>> As Daniel mentioned we do not use DFRouter very often, which is why
>>>>> this annoyance is not yet fixed. (But here is a ticket:
>>>>> http://sumo-sim.org/trac.wsgi/ticket/1158 because I do think it is a
>>>>> defect)
>>>>> regards,
>>>>> Jakob
>>>>>
>>>>>
>>>>> 2014-02-17 14:35 GMT+01:00 Pieter Loof <[email protected]>:
>>>>>
>>>>> Hello,
>>>>>>
>>>>>> Giving the route and vehicle file to SUMO both via the "-r " flag in
>>>>>> the correct order solved the errors. But now I am running into a
>>>>>> different
>>>>>> problem again. When running SUMO I get a lot of warnings about discarded
>>>>>> vehicles because the route file is not sorted. Apart from that, the
>>>>>> simulation runs and exits normally. So again I run into the problem of
>>>>>> the
>>>>>> route file not being sorted, although it is generated by DFRouter and I
>>>>>> did
>>>>>> not alter it. Because thousands of vehicles are discarded, the scenario
>>>>>> output becomes unreliable for me.
>>>>>>
>>>>>> Now I guess a simple way to solve this is to put both the route and
>>>>>> vehicle file in the list of additional files in the correct order. This
>>>>>> simple solution works, the errors about the unsorted route file do not
>>>>>> occur. But when running the simulation now, the simulation suddenly
>>>>>> closes
>>>>>> with a segmentation fault after 1503 steps. This happens repeatedly and
>>>>>> both with and without GUI. Apart from some warnings about teleported
>>>>>> vehicles and changed depart speeds because of close by junctions, the
>>>>>> simulation seems to run normally up to the segmentation fault. So I
>>>>>> guess I
>>>>>> ran into another bug? I added the scenario for which the segmentation
>>>>>> fault
>>>>>> occurs. I simplified the scenario by removing some files and flags again
>>>>>> and checked that the fault still occurs. The scenario can be run by
>>>>>> running
>>>>>> sumo or sumo-gui on the provided configuration file.
>>>>>>
>>>>>> Greetings,
>>>>>> Pieter
>>>>>>
>>>>>>
>>>>>> On 5 February 2014 12:52, Jakob Erdmann
>>>>>> <[email protected]>wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>> I think I have finally figured out the underlying misunderstanding.
>>>>>>> Please read the following (new) documentation:
>>>>>>> http://sumo-sim.org/wiki/SUMO#Loading_order_of_input_files
>>>>>>>
>>>>>>> http://sumo-sim.org/wiki/Demand/Routes_from_Observation_Points#How_to_include_the_files
>>>>>>> regards,
>>>>>>> Jakob
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2014-02-04 Jakob Erdmann <[email protected]>:
>>>>>>>
>>>>>>> Hello,
>>>>>>>> regarding 1)
>>>>>>>> in config-files,
>>>>>>>> <additional-files value="file1 file2 file3"/> and
>>>>>>>> <additional-files value="file1,file2,file3"/> should both work.
>>>>>>>> What error message do you get?
>>>>>>>>
>>>>>>>> regarding 2)
>>>>>>>> most likely there is something wrong with the route file itself. It
>>>>>>>> should generally be possible to use the -r option for route files.
>>>>>>>> Only if the routes are unsorted (wrt departure time) you need to
>>>>>>>> use -a because -a does not perform incremental loading but loads the
>>>>>>>> whole
>>>>>>>> file instead.
>>>>>>>>
>>>>>>>> Note, that in your route file any <route> elements must come before
>>>>>>>> <vehicle route=""> elements referencing the former.
>>>>>>>>
>>>>>>>> regards,
>>>>>>>> Jakob
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-02-04 Pieter Loof <[email protected]>:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I just tried using the --save-configuration option for both
>>>>>>>>> DFRouter and
>>>>>>>>> SUMO. It succesfully creates a config file, but I ran into a few
>>>>>>>>> small
>>>>>>>>> problems.
>>>>>>>>>
>>>>>>>>> Problem 1:
>>>>>>>>>
>>>>>>>>> I know that the additional files in a .sumocfg file is a
>>>>>>>>> space-separator
>>>>>>>>> list, like this:
>>>>>>>>> <additional-files value="file1 file2 file3"/>
>>>>>>>>>
>>>>>>>>> Now, in whatever way I try to give the additional files as flag to
>>>>>>>>> a sumo
>>>>>>>>> command, it keeps saving it to the config file in the wrong
>>>>>>>>> format. As far
>>>>>>>>> as I know the flag for additional files should be a
>>>>>>>>> comma-separated list,
>>>>>>>>> like this:
>>>>>>>>> sumo -a file1,file2,file
>>>>>>>>>
>>>>>>>>> But when I run the command and look at the produced config file, I
>>>>>>>>> see that
>>>>>>>>> the commas are stll there, it looks like this:
>>>>>>>>> <additional-files value="file1,file2,file3"/>
>>>>>>>>>
>>>>>>>>> So there are still commas in between, and running SUMO with this
>>>>>>>>> config
>>>>>>>>> file gives errors that I do not receive when the list is
>>>>>>>>> space-separated. I
>>>>>>>>> tried a few different ways of formatting the "-a <FILE>" using
>>>>>>>>> commas or
>>>>>>>>> spaces as separator with and without enclosing braces, brackets or
>>>>>>>>> quotes,
>>>>>>>>> but the generated format is never correct. Am I doing something
>>>>>>>>> wrong?
>>>>>>>>>
>>>>>>>>> Problem 2:
>>>>>>>>>
>>>>>>>>> When generating the sumo config file, using the flag "-r <FILE>"
>>>>>>>>> to give
>>>>>>>>> the route file, then the route file is written as
>>>>>>>>> <route-files value=file/>
>>>>>>>>>
>>>>>>>>> But when running SUMO using the generated config file, errors
>>>>>>>>> occur about
>>>>>>>>> invalid references to routes. The only solution seems to be to
>>>>>>>>> give the
>>>>>>>>> route file under the "-a <FILE>" flag. If the route file should be
>>>>>>>>> an
>>>>>>>>> additional file, then please fix the config generation process or
>>>>>>>>> change
>>>>>>>>> the documentation about providing the route file.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Pieter
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Managing the Performance of Cloud-Based Applications
>>>>>>>>> Take advantage of what the Cloud has to offer - Avoid Common
>>>>>>>>> Pitfalls.
>>>>>>>>> Read the Whitepaper.
>>>>>>>>>
>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
>>>>>>>>> _______________________________________________
>>>>>>>>> sumo-user mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/sumo-user
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
#!/usr/bin/env python
"""
@file sort_routes.py
@author Jakob Erdmann
@author Michael Behrisch
@date 2011-07-14
@version $Id: sort_routes.py 15692 2014-02-22 09:17:02Z behrisch $
This script sorts the vehicles in the given route file by their depart time
SUMO, Simulation of Urban MObility; see http://sumo-sim.org/
Copyright (C) 2007-2014 DLR (http://www.dlr.de/) and contributors
This file is part of SUMO.
SUMO is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
"""
import sys
import os
import re
from xml.dom import pulldom
from xml.sax import handler
from xml.sax import make_parser
from xml.sax import handler
from optparse import OptionParser
def get_options(args=None):
USAGE = "Usage: " + sys.argv[0] + " <routefile>"
optParser = OptionParser()
optParser.add_option("-o", "--outfile", help="name of output file")
optParser.add_option("-b", "--big", action="store_true", default=False,
help="Use alternative sortign strategy for large files (slower but more memory efficient)")
options, args = optParser.parse_args(args=args)
if len(args) != 1:
sys.exit(USAGE)
options.routefile = args[0]
if options.outfile is None:
options.outfile = options.routefile + ".sorted"
return options
def sort_departs(routefilename, outfile):
routes_doc = pulldom.parse(sys.argv[1])
vehicles = []
for event, parsenode in routes_doc:
if event == pulldom.START_ELEMENT and (parsenode.localName == 'vehicle' or parsenode.localName == 'flow'):
vehicle = parsenode # now we know it's a vehicle or a flow
routes_doc.expandNode(vehicle)
if (parsenode.localName == 'vehicle'):
depart = int(float(vehicle.getAttribute('depart')))
vehicles.append((depart, vehicle.toprettyxml(indent="", newl="")))
elif (parsenode.localName == 'flow'):
begin = int(float(vehicle.getAttribute('begin')))
vehicles.append((begin, vehicle.toprettyxml(indent="", newl="")))
print('read %s elements.' % len(vehicles))
vehicles.sort()
for depart, vehiclexml in vehicles:
outfile.write(" "*4)
outfile.write(vehiclexml)
outfile.write("\n")
print('wrote %s elements.' % len(vehicles))
class RouteHandler(handler.ContentHandler):
def __init__(self, elements_with_depart):
self.DEPART_ATTR = {'vehicle' : 'depart', 'flow' : 'begin'}
self.elements_with_depart = elements_with_depart
self._depart = None
def setDocumentLocator(self,locator):
self.locator = locator
def startElement(self,name,attrs):
if name in self.DEPART_ATTR.keys():
self._depart = attrs[self.DEPART_ATTR[name]]
self._start_line = self.locator.getLineNumber()
def endElement(self,name):
if name in self.DEPART_ATTR.keys():
end_line = self.locator.getLineNumber()
self.elements_with_depart.append((self._depart, self._start_line, end_line))
def create_line_index(file):
print "Building line offset index for %s" % file
result = []
offset = 0
with open(file, 'rb') as f: # need to read binary here for correct offsets
for line in f:
result.append(offset)
offset += len(line)
return result
def get_element_lines(routefilename):
# [(depart, line_index_where_element_starts, line_index_where_element_ends), ...]
print "Parsing %s for line indices and departs" % routefilename
result = []
parser = make_parser()
parser.setContentHandler(RouteHandler(result))
parser.parse(open(routefilename))
print " found %s items" % len(result)
return result
def copy_elements(routefilename, outfilename, element_lines, line_offsets):
print "Copying elements from %s to %s sorted by departure" % (
routefilename, outfilename)
outfile = open(outfilename, 'w')
# copy header
for line in open(routefilename):
outfile.write(line)
if '<routes' in line:
break
with open(routefilename) as f: # don't read binary here for line end conversion
for depart, start, end in element_lines:
# convert from 1-based to 0-based indices
f.seek(line_offsets[start - 1])
for i in range(end - start + 1):
outfile.write(f.readline())
outfile.write('</routes>')
outfile.close()
def main(args=None):
options = get_options(args=args)
if options.big:
line_offsets = create_line_index(options.routefile)
element_lines = get_element_lines(options.routefile)
element_lines.sort()
copy_elements(options.routefile, options.outfile, element_lines, line_offsets)
else:
outfile = open(options.outfile, 'w')
close_line = ''
for line in open(options.routefile):
outfile.write(line)
if '<routes' in line:
close_line = '</routes>'
if '<additional' in line:
close_line = '</additional>'
if '<vehicle ' in line:
break
sort_departs(options.routefile, outfile)
outfile.write(close_line)
outfile.close()
if __name__ == "__main__":
main()
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
sumo-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sumo-user
