MTF main configuration

The main configuration is enclosed by a SchedulerConfiguration element:

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
  …
</SchedulerConfiguration>

Here is an example of a common configuration copied from a real project:

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
	<DefaultApplicationPath>C:\optima\MTF\TRE\TRE.exe</DefaultApplicationPath>
	<WebServiceBaseAddress>http://10.152.219.124:1801/mtf/</WebServiceBaseAddress>
	<DefaultConfigurationFile>C:\Optima\MTF\TRE\CommandLineTDE.csv</DefaultConfigurationFile>
	<Jobs>
		<!-- Simulation of next week -->
		<MidTermForecastNextDays Name="AshghalNextDays" Enabled="true">
			<DbConnectionString>Server=10.152.219.121;Port=5432;User Id=postgres;Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;Database=optima;CommandTimeout=3600;</DbConnectionString>
			<NumberOfDays>3</NumberOfDays>
			<CronSchedule>0 9 0 * * ? *</CronSchedule> 
			<!-- <CronSchedule>00 21 17 * * ? *</CronSchedule> -->
		</MidTermForecastNextDays>
	</Jobs>
</SchedulerConfiguration>
                

The SchedulerConfiguration element has a few direct sub-elements which define general configuration settings:

Jobs represent a scheduled sequence of actions (tasks). When the scheduled time has been reached, the list of tasks contained in the job is executed in the order defined by the tasks.

The set of jobs is a sub-element of SchedulerConfiguration.

It contains all the jobs in the following form:

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
    ....
  <Jobs>
    <JobConfiguration Name="Job1" Enabled="true">
     ....
    </JobConfiguration>
    <JobConfiguration Name="Job2" Enabled="true">
      …
    </JobConfiguration>
    …
  </Jobs>
</SchedulerConfiguration>
                

A JobConfiguration element represents a single job. It contains the job’s configuration and the list of tasks that belong to the job.

Each job uses XML attributes to define some of its properties:

Other parameters of jobs are defined by XML sub-elements:

The tasks that belong to a job are defined in a list of tasks, which is a sub-element of JobConfiguration elements.

  …
    <JobConfiguration Name="Example" Enabled="true">
      …
      <Tasks>
        <Task Name="Action" Order="1">
          …
        </Task>
      </Tasks>
    </JobConfiguration>
  …              

For details on tasks: → Manual task configuration.

You can associate a scheduling to three main elements:

• JobConfiguration

• MidTermForecastDayTypes

• MidTermForecastNextDays

Scheduling JobConfiguration

JobConfiguration is used for simulations during the rest of the day.

The CronSchedule tag is mandatory, otherwise it crashes while it is filling job configurations.

Example

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
	<WebServiceBaseAddress>http://localhost:1800/mtf</WebServiceBaseAddress>
	<Jobs>
		<!-- Simulation of the rest of the day: each our, from that moment until midnight -->
		<JobConfiguration Name="RestOfDay" Enabled="true">
			<CronSchedule>0 0 * * * ? *</CronSchedule>
			<AbortOnTaskFailure>true</AbortOnTaskFailure>
			<Tasks>
			<Task Order="1" Name="CreateTable">
				<ExecutionType>SQL</ExecutionType>
				<Parameters>
					<ConnectionString>Server=localhost;Port=5432;User Id=postgres;Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;Database=optima;CommandTimeout=3600;</ConnectionString>
					<Query>SELECT mtf_create_nextday('rlin_tsys_tre_rltm_{{{DateOf(now)}}}', {{{DateOf(now)}}});</Query>
				</Parameters>
			</Task>
			<TRE Order="2" Name="MidTerm">
			  <ApplicationPath>C:\Optima\MTF\TRE\TRE.exe</ApplicationPath>
			  <Parameters>
				<SDAY>{{{DateOf(now)}}}</SDAY>
				<SimInitHHMMSS>{{{now.ToString("HH") &amp; "0000"}}}</SimInitHHMMSS>
				<SimSpan>{{{(24 - now.Hour) * 3600}}}</SimSpan>
			  </Parameters>
			  <ExecutionType>NewProcess</ExecutionType>
			</TRE>
			</Tasks>
		</JobConfiguration>
	</Jobs>
</SchedulerConfiguration>

Scheduling MidTermForecastDayTypes

MidTermForecastDayTypes is used for simulations of day types.

The CronSchedule tag is not mandatory and it is assigned by default to the value (0 0 0 * * ? *).

In case you need your own CronSchedule, you can add it and the value that you select will replace the default value.

Example

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
	<WebServiceBaseAddress>http://localhost:1800/mtf</WebServiceBaseAddress>
	<Jobs>
		<!-- Simulation of day types -->
		<MidTermForecastDayTypes Name="DayTypes">
			<ApplicationPath>C:\Optima\MTF\TRE\TRE.exe</ApplicationPath>
			<DbConnectionString>Server=localhost;Port=5432;User Id=postgres;Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;Database=optima;CommandTimeout=3600;</DbConnectionString>
			<DayTypes>10,2</DayTypes>
		</MidTermForecastDayTypes>
	</Jobs>
</SchedulerConfiguration>

Scheduling MidTermForecastNextDays

MidTermForecastNextDays is used for the simulation of next days.

CronSchedule tag is not mandatory and it is assigned by default to the value (0 0 0 * * ? *).

In case you need your own CronSchedule, you can add it and the value that you insert will replace the default value.

Example

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
	<WebServiceBaseAddress>http://localhost:1800/mtf</WebServiceBaseAddress>
	<Jobs>
		<!-- Simulation of next week -->
		<MidTermForecastNextDays Name="Week" Enabled="true">
			<ApplicationPath>C:\Optima\MTF\TRE\TRE.exe</ApplicationPath>
			<DbConnectionString>Server=localhost;Port=5432;User Id=postgres;Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;Database=optima;CommandTimeout=3600;</DbConnectionString>
			<NumberOfDays>7</NumberOfDays>
		</MidTermForecastNextDays>
	</Jobs>
</SchedulerConfiguration>

The WildFly AS component of OPTIMA must be properly configured as MTF is typically running on a different host than that of WildFly’s.

It is necessary to set the <mtf> node in the configuration file:

/opt/ptv-optima-vv.n.xxxxptv-optima-as/standalone/configuration/optima/optima-configuration.xml.

This is a sample configuration node for MTF:

...
     <mtf>
        <serverHost>http://somehost:1801</serverHost>
     <mtf>
....
                

<serverHost> is composed by:

<protocol>://<hostname>:<port>

On this basis, WildFly contacts the MTF instance by appending also a constant path /mtf/rest/status.

MTF provides a number of specialized jobs for mid-term forecast computation. These specialized jobs automatically create their tasks based on their parameters. This makes it easy to schedule a mid-term forecast computation.

Mid-term forecast common parameters

Specialized jobs for a mid-term forecast configuration have some common parameters:

<ApplicationPath>C:\Optima\MTF\TRE\TRE.exe</ApplicationPath> 

<ApplicationPath>C:\Optima\MTF\TRE\TRE.dll</ApplicationPath>

For parameters that assign values to the automatically generated TRE tasks: → TRE task.

The execution of mid-term forecast jobs consists of 3 phases:

1. Execution of tasks listed in BeforeTasks
2. Execution of automatically generated tasks
3. Execution of tasks listed in AfterTasks

Next days forecasts

The typical mid-term forecast is a simulation of the next days (with respect to the date of its computation). The job MidTermForecastNextDays can simplify the schedule of this type of forecast.

MidTermForecastNextDays automatically creates a sequence of tasks, which can be grouped into 3 phases:

1. Execution of SQL tasks that create the tables needed for the simulation output
2. Execution of TRE tasks that perform the simulation
3. Execution of SQL tasks that delete old tables (containing data of past days)

The following is an example of a configuration for the next days of a mid-term forecast:

<?xml version="1.0"?>
<SchedulerConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sistemaits.com">
  <WebServiceBaseAddress>http://localhost:8080/mtf/</WebServiceBaseAddress>
  <Jobs>
    <MidTermForecastNextDays Name="Week" Enabled="true">
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <DbConnectionString>Server=localhost;Port=5432;User Id=postgres;
        Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;
        …
                

This type of job inherits all the common parameters (→ Mid-term forecast common parameters). Additionally, it has the following special parameters:

Day type forecasts

You can schedule simulations for lists of day types: MidTermForecastDayTypes makes this operation easy.

MidTermForecastDayTypes automatically creates a sequence of tasks, which can be grouped into 2 phases:

1. Execution of SQL tasks that create the tables needed for simulation output
2. Execution of TRE tasks that perform the simulation

The following is an example of a configuration for day types of a mid-term forecast:

    …
    <MidTermForecastDayTypes Name="DayTypes" Enabled="true">
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <DbConnectionString>Server=localhost;Port=5432;User Id=postgres;
        Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;
     …
                

The job inherits all the common parameters (→ Mid-term forecast common parameters). Additionally, it has the following special parameters:

Important:  The _sday and _dtyp tables are NOT created by TRE, but by the MTF scheduler according to the configuration. If the set of day types changes, the MTF configuration must be updated.

Important:  The configuration of MTF typically does not need anything more than specialized jobs for mid-term forecast (→ Special jobs for mid-term forecast computation). The manual task configuration described below can be useful for an advanced use of MTF, but is usually not required for typical configurations.

Tasks enable you to define actions that a job should perform. There are several types of tasks, depending on the type of action that should be performed.

Application features

Because tasks are mainly used for executing applications, all tasks inherit a set of parameters that describe application features. Depending on the type of task, each task can ignore some (or all) of these common properties if they are not useful for that specific type of task. For example, tasks for SQL queries do not need an application path and a configuration file.

The common application feature parameters are:

Parameters that will be passed to the application are defined by the Parameters parameter. The content of this parameter can be expressed in different ways:

• Each parameter can be a distinct XML sub-element, each having its value as content:

      …
        <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
        <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
        <Parameters>
          <Param1>Value1</Param1>
          <Param2>Value2</Param2>
        </Parameters>
      …
  

• A string can represent the parameters as they will be passed to the application:

      …
        <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
        <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
        <Parameters>--Param1=Value1  --Param2=Value2</Parameters>
      …
  

By default, the parameter format is --Param=Value. If a different format is used, you can define it with the ParameterFormat element in Parameters, using the placeholders [param] and [value] to set parameter and value positions. For example, if the format is -Param Value, you can set this format with a configuration like this:

    …
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <Parameters>
        <ParameterFormat>-[param]  [value]</ParameterFormat>
        <Param1>Value1</Param1>
        <Param2>Value2</Param2>
      </Parameters>
    …         

Or, when parameters are defined as a string:

    …
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <Parameters>-Param1 Value1 -Param2 Value2
        <ParameterFormat>-[param]  [value]</ParameterFormat>
      </Parameters>
    …         

When parameters are defined as XML elements, their format is used for creating the string that contains the parameters that will be passed to the application.

When parameters are defined as a string, their format is used for parsing the string and for extracting parameters and values.

You can assign special values to parameters so that their value is computed at runtime. This is useful when the value of a parameter depends on the time of execution (for example: A parameter needs the current date as its value). Special values contain fields that start with {{{ and end with }}}. The content of each field must be a Visual Basic.NET single-line expression whose result is a string. Variables and methods that can be used inside fields are defined in a configuration file (→ MTF additional configuration > Configuration file “DynamicStringSource.vb”).

Usually, a standard configuration provides the following variables and methods in dynamic string fields:

The following example shows the usage of fields:

    …
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <Parameters>
        <SDAY>{{{DateOf(now)}}}</SDAY>
        <Tomorrow>{{{AddDays(now,  1)}}}</Tomorrow>
        <Yesterday>{{{AddDays(now,  -1)}}}</Yesterday>
        <TimeMessage>Current time is {{{TimeOf(now)}}}…</TimeMessage>
        <JobName>This job is {{{jobName}}}!</JobName>
      </Parameters>
    …               

A similar example with parameters defined as a string is:

    …
      <ApplicationPath>C:\Optima\MTF\TRE\TRE-1.6.5.3\TRE.exe</ApplicationPath>
      <ConfigurationFile>C:\Optima\MTF\TRE\TRE-1.6.5.3\CommandLineTDE.csv</ConfigurationFile>
      <Parameters>--Date={{{DateOf(now)}}}  --Message="I am {{{jobName}}}"</Parameters>
    …               

Tasks inherit a basic set of parameters (→ Application features). Each task uses XML attributes to define some of its parameters:

Other parameters of tasks are defined by XML sub-elements:

An example of an SQL execution type task:

     …  
        <Task Name="CleanTable" Order="1">
          <ExecutionType>SQL</ExecutionType>
          <Parameters>
            <ConnectionString>Server=localhost;Port=5432;User Id=postgres;
              Password=postgres;ApplicationName=SISTeMA Mid-Term Forecast;
              Database=optima;CommandTimeout=3600;</ConnectionString>
            <Query>DELETE FROM example_table WHERE (now() -'1 day'::  interval) > data;</Query>
          </Parameters>
        </Task>
      …            

TRE tasks inherit all the parameters of generic tasks, but automatically set some of the values to simplify the usual configuration of tasks that involve TRE.

The following TRE task parameters are set automatically.

Additional TRE task parameters that are not set automatically are:

The identifier is used as a value of the parameter AppNameForIO in order to identify TRE instances. It is also appended to the web service address in order to easily locate the address of the TRE web service from its identifier.