Harmonizer methodology

A single measurement is a set of two numerical values: flow and speed. The set of these three values represents a single traffic state measurement through a sensor within a specific time interval.

At regular time intervals, data from the different configured sources is collected in the database tables, organized by street and transport system indexes. This is typically done by loading data from trafficstate tables.

As a first step, the measurements are validated and those measurements that can be safely considered incoherent are discarded. For example, measurements with negative values or zero flows with non-zero speed values.

Next, a reliability parameter and a weight based on the specific datasource are assigned to each measurement.

The filtering step is applied for each source and each transport system on each directional street separately.

The Harmonizer filters the different traffic data collected according to a defined set of rules and sets its reliability values.

The filtering rules to be applied are defined in an XML configuration file (see → Harmonizer configuration).

The pivoting procedure performs two major tasks:

1. A time series of data x_n is accumulated over the index n and its average A_n is computed iteratively as

   (A0 = 0)

   With the m-th order, the momentum M^m is computed as:

   (m > 0)

2. The actual incoming data is normalized with respect to reference values max, provided by the database table STRT, defining a pivoted value as

   

   where the maximum MAX is defined as a function of the average and of the m-th momentum defined above:

   

The order m of the momentum to be used, as well as the maximum number of elements to take into account for the computation of the pivoted average, are set in the <pivoting> node of the configuration file (see → Harmonizer configuration).

If the pivoting is scheduled, the output is written to the database table PIVOT.

Important:  The pivoting phase can be disabled by configuration.

Clustering is performed for each source and each associated transport system, on each directional street and lane separately.

In this step, the only data that is taken into account is within a plausible interval.

The common interval is defined as I = [x, x+W], where the width of the validity range W is defined in the <clustering> node (for every specific parameter) of the configuration file (see → Harmonizer configuration).

For each set of measurements, for each source and street, the algorithm selects, among the intervals I = [x, x+W] defined around each measurement x, the range that maximizes the sum of the reliabilities p_m (objective function) corresponding to each measurement value x_m falling in the same interval. If multiple sets have the same maximum value for the sum of the reliabilities, the software uses only the last set that has been evaluated for its further computations.

The set of measurements corresponding to the optimal interval is M.

Being W known and constant, the optimal interval is a function of its lower extreme x.

In case there is more than one value maximizing the objective function, the higher one of them is usually taken.

Example: interval I = [x, x+W] in one dimension

In this example, a graphical view of the evolution of the measurements interval is used to calculate the associated reliabilities for the values of speed.

The optimal interval is the interval with the higher value of reliability:

(Relx=sum(p_m)).

 

 

Example: interval I = [x, x+W] in two dimensions

As in the previous example, there are two dimensions, with two different values (W1 and W2) for interval width related to flow and speed.

This is the procedure used to evaluate an average over the entire set of data:

1. For each street, the harmonized measurement is the weighted mean of clustered measurements of all sources (the actual weight is the product of the reliability and weight of the source).
2. If m is a single value of the measurement (flow, speed), p_m the related reliability, and p_f the weight of the source, the harmonized value X is

3. The accuracy (attribute accu in the TDE table) associated with this value X is

   where the denominator is the number of measurements falling into the optimal clustered interval defined above.

The use of the aggregated results for KPI computation may lead to bad results if not all the lanes are monitored

Note: A speed measure no lanes will be also used in the simulation. in this case the aggregated result may be particularly hard to be interpreted.

The lane aggregation property can be configured "by model", therefore according to information collected in the network model.

If the property is set for a specific source, after this step, only data referencing the whole street is available and no lane-based information is stored as output.

The property is set if CLOC.AGGREGATE_LANES=TRUE.

If there is already some input data that refers to the whole street, then the aggregation process should overwrite the input data.

On the other hand, if the only available input is per street (no lane info), then the original data should be preserved regardless of the value of the aggregate attribute.

This aggregation step computes:

• Flow: as the sum of not null flows of each lane (flows equal to zero are aggregated).

• Speed: as the average speed of not null speeds of each lane (speeds equal to zero are averaged).

• Reliability: as the average of reliabilities.

In this step, the averages are simple arithmetic data.

All previous steps are performed for each transport system.

According to the specific configuration of each source, an extra step can be carried out to produce equivalent measurements.

Important:  If CLOC.AGGREGATE_LANES=FALSE and one or more lanes are not monitored, KPI computation results may not be reliable.

In the configuration, for each data source, you can specify if the equivalent computation must be performed or not. As input, for a given street and lane (considering the “no lane” case, too), it uses all traffic states of the given transport systems and computes an aggregated traffic state on the same street and lane suitably weighting the input information.

Specifically, it computes:

• Flow: as the weighted sum of not NULL flows, where the weight is provided by the transport system scaling factor (see EQVE field of the "Transport systems – TSYS table" on page 1).

  Therefore, by indicating with the flow of the transport system i , and with its EQVE value, the resulting flow will be:
  

• Speed: as the weighted average of not NULL speeds, where the weight is the not NULL flow of the corresponding transport system. Speeds without flow information will be discarded.

  Therefore, by indicating with the flow of the transport system i, and with the speed of the same transport system, the resulting speed will be:
  

Note that if all traffic states have NULL flow, the computed speed will be a simple arithmetic average.


• Reliability: as the average of the reliabilities.

The values X and accu , calculated for each street of the network and transport system, are the actual outputs of Harmonizer.

They can be:

• Stored into database tables as defined by the destinations; the default table is TSTA (→ Architecture and Data Model).
• Configured to be stored in the memory of another module; in this case, you can avoid storing the output in the DB.

In the first case, you can also configure an SQL statement to run before (pre-action) or after (post-action) storing data in the DB.