Reliability

PTV Optima Micro API provides several reliability indicators:

      "reliability": {
        "numRunsWithKPIs": 2,
        "numExpectedSimulationRuns": 2,
        "numWarnings": 0,
        "numErrors": 0,
        "reliabilityIndex": 0.2
    }
    

For more details see also:

• → Getting status, configuration or KPI results of a simulation group
• → Getting status, configuration or KPI results of a single simulation

PTV Optima Micro APIs also provide a reliabilityIndex, that is a number in between 0 and 1 (the higher, the better).

This index represents the overall quality of the results, considering:

• The warnings and errors occurred during the automatic real-time modeling
• The number of different random seeds used for the microscopic simulations.

The index can be scaled in the range of 0…100 points.

N is the number of random seed simulations as configured in → Optima Micro configuration > ptv-optima-microSettings.xml

The calculation proceed as shown:

1. The starting reliability index is 100 points.

2. If N<10, N=N - (10 *(10 – N))

   Tip:  This means that a maximum reliability of 100 points can only be achieved with the recommended number of 10 different random seeds.

3. The maximum reliability of an environment configured with N random seeds is N * 10 points.

4. Points are subtracted for modeling errors and warnings:

   • 25 points for every modeling error
   • 5 points for every modeling warning
   • No more than 50 points in total for errors and warnings
   • 10 points for every simulation that failed and did not provide any results.

The reliability index cannot distinguish severities in the same way as a human modeler would.

For example, a lane-blocking event that cannot be placed in a low traffic area at the exit of the sub-network has the same impact on reliability as a lane-blocking event in front of a central, high-demand intersection.

The other fields reported in the API are:

Field	Description
numRunsWithKPIS	It contains the number of completed (different random seed) micro simulations with results.
numExpectedSimulationRuns	It contains the N number of random seed simulations as configured in → Optima Micro configuration > ptv-optima-microSettings.xml
numWarnings	It contains the number of warnings considered in the calculation of the reliabilityIndex. Important:  This number does not distinguish between warnings occurred during preparation of all random seeds or during the individual processing of a random seed.
numErrors	It contains the number of errors considered in the calculation of the reliabilityIndex. Important:  This number does not distinguish between warnings occurred during the preparation of all random seeds or during the individual processing of a random seed.

Errors and warnings

Modeling errors reflect highly suspicious input data (for example, negative flows).

Modeling warnings reflect some issues in automatic modeling (for example, a lane-blocking event cannot be placed properly).

Most errors and warnings occur during the preparation of the automatic real-time modeling. This preparation is only performed once, disregarding the number of random seeds. However, it is also possible that errors or warnings occur during the individual processing of a random seed. In this case, the subtracted points are divided by N.

Errors and warnings within the reliability index refer to PTV Optima Micro logging and NOT the PTV Vissim internal logging.

The log files on the Optima Micro Worker servers can be examined for more details about errors or warnings; they are configured as follows: → Optima Micro configuration > ptv-optima-microSettings.xml.