PuTUpdater methodology

The Optima PuTUpdater service is a stand alone application that updates the Optima DB every time the a new static public transport dataset in GTFS format is available. It works monitoring a file path: every time a change of the monitored file is detected, is triggered the update process.

The process is performed by Visum via COM interface.

Important:  As prerequisite, you must have installed the version of Visum compliant with your Optima version (see → Installing PuTUpdater).

The input of the process is composed by the Optima Visum model (.ver file) and a .zip file containing the GTFS data.

The output is an updated Optima DB.

The service can be configured to be:

Important:  You can handle this process only if you have a robust knowledge of Visum.

All the configuration parameters involved into the process are described in → PuTUpdater configuration.

The process proceed as shown:

  1. The new GTFS zip file is detected (see the GTFSFilePath configuration parameter).

  2. The result of the GTFS import (from now on called GTFS Model) is saved as a Visum file (.ver extension) (see the PuTSupplyDataDirectory configuration parameter).

  3. A new Visum Instance is run and the Optima private transport model is opened (see the OptimaBaseModelPath configuration parameter).

    Important:  In a Visum model, the pedestrian transport system used for commuting is of type PutWalk. This transport system must have the attributes code=PTWalk.

  4. The Put Supply import procedure is configured and run using the result of the GTFS import as source and the Optima private transport model as a target. The result is saved in a Visum model (from now on called Merged Model).

  5. The evaluation of the quality of the process is based on:

    • For the GTFS Model:

      1. The number of routes (NR) in GTFS is compared with the number of lines (NL) in the Visum model.

      2. The number of stops (NS) in GTFS with the number of stop areas (NSA) in the Visum model.

      If NR<>NL or NS<>NSA, the quality assessment is failed.

    • For the Merged Model: a set of quality KPIs (see → → Quality KPIs) is computed and compared with the thresholds defined in configuration (see the → THRESHOLDS configuration parameters). If any KPI value is above the threshold, the quality assessment is failed.

    For additional details about quality assessment, see → Quality assessment of the PuTUpdater process.

  6. Different things can happen:

    1. The service is configured to require the operator validation (see the AutomaticProcess configuration parameter). The result of the step 6 is saved as a Visum file in a configurable location (see the ToBeValidatedModelDirectoryconfiguration parameter).

    2. The service is configured to be automatic, but the quality KPIs are below the thresholds. The result of the previous step is saved as a Visum file in a configurable location (see the ToBeValidatedModelDirectory configuration parameter).

    3. The service is configured to be automatic and the quality assessment is passed successfully. Proceed to step 9.

  7. If the operator validation is requested (steps 7.A and 7.B), the operator:

  8. A new Visum instance is run and the Optima SaveAll script is automatically run.

    Important:  The SaveAll script is automatically included in the PuTUpdater package, and the user can ignore where it is stored.

  9. The TDE is configured and run and Optima public transport DB tables are updated (all DB updates are reverted in case of failure).

  10. The final step is to force the reload of the static PuT model.

Important:  If the process fails, it will try to restart with frequency set by the configuration parameter PollingIntervalInSeconds. For avoiding retry is necessary to remove the GTFS zip file.

A set of quality KPIs is computed and stored as User Defined Attributes (UDAs) in a Visum file (see the ToBeValidatedModelDirectory configuration parameter).

The KPIs are associated to three different Visum objects:

  • Stop points

  • Line routes

  • Line route items

The service can be configured with a set of thresholds on the quality KPIs to define the acceptance criteria for the model quality.

This is important when the service is configured to run in the fully automated mode (see the AutomaticProcess configuration parameter).

The configuration parameters are defined in the <Thresholds> node of the service configuration file appsettings.xml.

KPI Description Threshold

STOPPOINT_DISTANCE

Distance (in meters) between the stop point (in the Merged Model) and its stop area (in the GTFS Model).

STOPPOINT_MAX_DISTANCE

Maximum desired value for the STOPPOINT_DISTANCE KPI.

The quality assessment fails if any value above this threshold exist.

The suggested value is 150 (meters).
KPI Description Threshold

LINEROUTE_LENGTHRATIO

It is the ratio between:

  • The line route length in the Merged Model

  • The line route length in the GTFS Model

LINEROUTE_MAX_LENGTHRATIO

Maximum relative error between:

  • The Merged Model line route length A

  • The GTFS Model line route length B

The quality assessment succeeds if 1/LINEROUTE_MAX_LENGTHRATIO < A/B < LINEROUTE_MAX_LENGTHRATIO.

The suggested value is 1.1 (10% error).

LINEROUTE_NUMFALSENEGATIVESTOPS

It is the number of line route items of the line route having LINEROUTEITEM_ISFALSENEGATIVE=true.

LINEROUTE_MAXNUM_FALSENEGATIVE_STOPS

A "false negative stop" is a stop the line route is not stopping, at while it should.

LINEROUTE_MAXNUM_FALSENEGATIVE_STOPS is the maximum accepted value for the LINEROUTE_NUMFALSENEGATIVESTOPS.

The quality assessment fails if any line route has a value grater than the threshold.

Suggested value is 0.

LINEROUTE_LISTFALSENEGATIVESTOPS

It is the list the of line route items of the line route having LINEROUTEITEM_ISFALSENEGATIVE=true.

N.A.

LINEROUTE_NUMFALSEPOSITIVESTOPS

It is the number of line route items of the line route having LINEROUTEITEM_ISFALSEPOSITIVE=true.

LINEROUTE_MAXNUM_FALSEPOSITIVE_STOPS

A "false positive stop" is a stop the line route is stopping, at while it should not.

LINEROUTE_MAXNUM_FALSEPOSITIVE_STOPS is the maximum accepted value for the LINEROUTE_NUMFALSEPOSITIVESTOPS.

The quality assessment fails if any line route has a value grater than the threshold.

Suggested value is 0.

LINEROUTE_LISTFALSEPOSITIVESTOPS

It is the list of line route items of the line route having LINEROUTEITEM_ISFALSEPOSITIVE=true.

N.A.
KPI Description Threshold

LINEROUTEITEM_DISTANCE

Defined in the LineRouteItems table.

It is the distance (in meters) between:

  • The line route item (in the Merged Model)

  • The line route shape (in the GTFS Model)

LINEROUTEITEM_MAX_DISTANCE

Maximum desired value for the LINEROUTEITEM_DISTANCE KPI

The quality assesment fails if any value above this threshold exist.

The suggested value is 300 (meters).

LINEROUTEITEM_LENGTHRATIO

Defined in the LineRouteItems table.

It is the ratio between:

  • The line route item length (in the Merged Model)

  • The line route shape length (in the GTFS Model)

LINEROUTEITEM_MAX_LENGTHRATIO

Maximum relative error between:

  • The Merged Model line route item length A

  • The GTFS Model line route item length B

The quality assessment succeeds if 1/LINEROUTEITEM_MAX_LENGTHRATIO < A/B < LINEROUTEITEM_MAX_LENGTHRATIO.

The suggested value is 1.1 (10% error).

LINEROUTEITEM_ISFALSENEGATIVE

True if the line route item (in the GTFS Model) is not used (in the Merged Model).

LINEROUTEITEM_MAXNUM_FALSENEGATIVE

Maximum desired number of line route items having LINEROUTEITEM_ISFALSENEGATIVE=true.

The quality assessment fails if this number is exceeded.

Suggested value is 0.

LINEROUTEITEM_ISFALSEPOSITIVE

True if the line route item (in the Merged Model) is not used (in the GTFS Model).

LINEROUTEITEM_MAXNUM_FALSEPOSITIVE

Maximum desired number of line route items having LINEROUTEITEM_ISFALSEPOSITIVE=true.

The quality assessment fails if this number is exceeded.

Suggested value is 0.

The → Quality assessment of the PuTUpdater process is summarized and persisted in the file QualityAssessmentSummary.txt, stored in ValidatedModelDirectory.

The step 5 of the process is focused on the quality assessment.

The assessment involves four main objects:

  • OptimaBaseModel.ver

  • gtfs.zip

  • PuTSupplyDataDirectory\PuTSupplyData.ver

  • ToBeValidatedModel\ToBeValidatedModel.ver

These objects are stored in different locations:

 

  • OptimaBaseModelPath: the location of the target model that is updated with GTFS import and stored in OptimaBaseModelPath

    Example: OptimaBaseModel.ver

  • GTFSFilePath: the location of the GTFS feed provided by the transport authority, stored in gtfs.zip

    Example: gtfs.zip

  • PuTSupplyData: the location of the Visum model, obtained by importing the GTFS feed, is stored in PuTSupplyDataDirectory

    Example: PuTSupplyDataDirectory\PuTSupplyData.ver

  • ToBeValidatedModel: the location of the final result of the GTFS import into the current Visum model (OptimaBaseModelPath) that is stored in

    ToBeValidatedModelDirectory

    Example: ToBeValidatedModel\ToBeValidatedModel.ver

The result of quality assessment is stored in the QualityAssessmentSummary.txt file:

 

Put Updater Quality Check
2024-01-31T17:58:01.4189218+01:00
Files:
optimaBaseModel: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\Resources\OptimaBaseModel\OptimaBaseModel.ver
GTFSFilePath: GTFS/gtfs.zip
PuTSupplyData: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\PuTSupplyData\PuTSupplyData.ver
ToBeValidatedModel: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\ToBeValidatedModel\ToBeValidatedModel.ver
Info about incoming public transportation systems
PuTSupplyData: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\PuTSupplyData\PuTSupplyData.ver
Non-standard public transportation system codes detected:
code: 1
code: 2
code: 3
All lineroutes relying on such transportation systems cannot be imported.
****************************************************
Summary of GTFS importation
Number of Lines is preserved.
Number of StopAreas is preserved.
****************************************************
Summary of kpis evaluation
Stop point distance test passed!
False negative stops for lineroute test passed!
False positive stops for lineroute test passed!
Length ratio for lineroute test passed!
LineRouteItems distance test passed!
LineRouteItems length ratio test passed!

 

The first section of the file contains the timestamp of the file:

 

Put Updater Quality Check
2024-01-31T17:58:01.4189218+01:00

 

The second section contains the location of the four objects involved in the assessment:

 

Files:
optimaBaseModel: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\Resources\OptimaBaseModel\OptimaBaseModel.ver
GTFSFilePath: GTFS/gtfs.zip
PuTSupplyData: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\PuTSupplyData\PuTSupplyData.ver
ToBeValidatedModel: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\ToBeValidatedModel\ToBeValidatedModel.ver
Info about incoming public transportation systems
PuTSupplyData: C:\dev\ptv-optima-2\dot-net\PTVGroup.Optima.PuTUpdater.Tests\bin\Debug\net6.0-windows\PuTSupplyData\PuTSupplyData.ver

 

The third section contains the result of the assessment:

 

Non-standard public transportation system codes detected:
code: 1
code: 2
code: 3
All lineroutes relying on such transportation systems cannot be imported.
****************************************************
Summary of GTFS importation
Number of Lines is preserved.
Number of StopAreas is preserved.
****************************************************
Summary of kpis evaluation
Stop point distance test passed!
False negative stops for lineroute test passed!
False positive stops for lineroute test passed!
Length ratio for lineroute test passed!
LineRouteItems distance test passed!
LineRouteItems length ratio test passed!

 

In the example, there are some non-standard transportation system defined in the GTFS:

 

Non-standard public transportation system codes detected:
code: 1
code: 2
code: 3
All lineroutes relying on such transportation systems cannot be imported.

 

A possible reason could be that the transportation agency uses some extended definition of the route_type attribute in file route.txt, that is not present in the base Visum model. The possible solution is to modify the Visum model in order to add the new transportation system.

Below, another fragment of file where KPIs evaluation fails:

 

Put Updater Quality Check
2024-01-17T12:04:34.9372384+01:00
Files:
BasePrivateModelFileName: D:\Projects\PutUpdaterIntegrationYork\BaseModel\OptimaModelUpdate2022_Current_Master20230901_noZONES_noLines.ver
GTFSFilePath: D:\Projects\PutUpdaterIntegrationYork\GTFS\itm_yorkshire_gtfs.zip
PuTSupplyData: D:\Projects\PutUpdaterIntegrationYork\PuTSupplyData\PutSupplyData.ver
ToBeValidatedModel: D:\Projects\PutUpdaterIntegrationYork\ToBeValidated\ToBeValidatedModel.ver
Summary of kpis evaluation
****************************************************
Stop point distance test passed!
False negative stops for lineroute test NOT passed for 214 line routes
(threshold=0):
Id : NumberOfFalseNegative
19_72,19_72_2,> : 2
19_46,19_46_2,> : 1
19_46,19_46_1,> : 1
19_46,19_46,> : 1
81_39,81_39_9,> : 25

 

In this example some line routes are affected by defects.

For example:

  • For the route 19_72,19_72_2,> there are 2 missing stops into the GTFS feed

  • For the route 81_39,81_39_9,> there are 25 missing stops into the GTFS feed

Here you can consider two examples of validation handled by an operator (see steps 7.A and 7.B), using the Visum GUI and related to the KPIs:

  • STOPPOINT_DISTANCE

  • LINEROUTEITEM_DISTANCE

If the STOPPOINT_DISTANCE KPI exceeds the threshold for one or more stop points, they must be manually checked.

The inspection can be performed opening the Visum model containing the KPIs (see the ToBeValidatedModelDirectory configuration parameter), and looking to the Stop Areas with the stop points exceeding the distance.

In the figure you see the Network editor in the top panel and the list of Stop areas in the bottom one.

The stop areas are sorted by the maximum value of their STOPPOINT_DISTANCE KPI (A).

The stop points of the selected stop area are shown in the map enabling the extended marking feature (B).

In this case two of the three stop points are quite (400m) far away from their stop area location, and the KPI is showing almost certainly a map matching issue.

Here the stop area is located inside the Airport gate, and the two associated stops are on two one-way roads nearby; despite being far (~100m) from the stop area, the location of the stop point is correct.

If the LINEROUTEITEM_DISTANCE KPI exceeds the threshold, the line route items positions must be validated manually.

The validation can be performed opening both the Visum Model containing the KPI and the GTFS Model.

Then the line route shapes can be compared.

In this case the shape of a line route having a maximum LINEROUTEITEM_DISTANCE of ~14Km in the Merged Model (rigth) is compared with the shape of the same line route in the GTFS Model.

The two paths have a huge difference, indicating a major problem in the map matching, due to missing streets in the private transport model.

In the figure above, an example of a line route with line route items exceeding the distance threshold.

In this case though the map matching (on the right) could be considered valid because the original shape (on the left) has one long straight segment that has been matched on the closest path on the road network.