Startup and Shutdown Scripts

The desire to have control of FME just before and just after translation has been discussed and debated for many years now. People have wanted to do different setup things before a translation begins, and different post-processing of log files or other translation artifacts after a translation ends.

At last this is possible, in a very powerful way with pre and post processing functions that provide the ability to execute a script both before and/or after a workspace is processed. The script has access to many of the translation artifacts (like counts, logfile, etc).

The possibilities are immense; some example applications are...

• Customized logging (to a file or to a databases even)
• Site-specific event logs
• Post-process renaming, moving or backup of files
• Pre-process testing of source and destination datasets

When Scripts Run

Startup scripts are executed after the mapping file has been completely parsed, and after the logfile has been opened, but before the readers have begun to do their processing. Shutdown scripts are executed after the writers have finished their processing and the logfile closed (or in the case of a translated terminated by error, after cleanup has taken place and the logfile closed).

Supported Scripting Languages

Startup and shutdown scripts can be in either Tcl or Python.

FME has a built-in Tcl interpreter, so no further installations are required. However (at the time of writing), to run a Python script requires the separate installation of Python.

If a script contains a procedure definition, the script must call the procedure it defined, otherwise the procedure itself will not be executed and no processing will occur.

Scripting with Mapping Files

In mapping files the scripts are accessed via directives. A directive can be used directly in a mapping file (if you have to ask how you shouldn't be using mapping files) or in a workspace header using Tools > Edit Header. The directives are:

• FME_BEGIN_TCL
• FME_END_TCL
• FME_BEGIN_PYTHON_ENC
• FME_END_PYTHON_ENC

The TCL syntax is:

FME_END_TCL <tcl script>

where <tcl script> is any valid Tcl code.

FME_END_TCL proc finalize {} { \
... tcl code here ...   \
};                          \
finalize;

FME_END_TCL source $(FME_MF_DIR_UNIX)/finalization.tcl

FME_END_TCL source {c:\my tcl\mytcl1.tcl}

Scripting with Workspaces

Scripts with FME Objects

Scripts in Custom Formats

Published Parameters

FME_END_TCL puts [array names FME_MacroValues];

FME_BEGIN_TCL puts $FME_MacroValues(DestDataset);

for name,val in FME_MacroValues.iteritems():
 logger.log( "   %s:%s" %(name,val))

Global Variables

Startup Scripts

Startup scripts may access the following global variables:
Global Variable     Contents
FME_MappingFileId     

• The value of the MAPPING_FILE_ID directive specified in the mapping file

FME_LogFileName     

• The filename of the logfile used by this translation. Will be blank if no logfile was used.

FME_MacroValues   

•  i.e. file delete $FME_MacroValues(DestDataset_XLS_ADO) Better than the older approach of using $(DestDataset_XLS_ADO)

Shutdown Scripts

Shutdown scripts may access the following global variables:
FME_Status     

• 0 if the translation failed and 1 if it was successful

FME_FailureMessage     

• The failure message if the translation failed, blank if the translation succeeded

FME_FeaturesRead(<featureType>)     

• Each element in this array, indexed by feature type, holds the number of features read for that feature type

FME_TotalFeaturesRead     

• The total number of features read

FME_FeaturesWritten(<featureType>)     

• Each element in this array, indexed by feature type, holds the number of features written for that feature type

FME_TotalFeaturesWritten     

• The total number of features written

FME_TotalCoordinates     

• The total number of coordinates output

FME_MappingFileId     

• The value of MAPPING_FILE_ID keyword if specified in the mapping file

FME_ElapsedTime     

• The elapsed time from just before the FME_BEGIN_TCL script was called until just before the FME_END_TCL was called.

FME_CPUTime     

• The CPU time from just before the FME_BEGIN_TCL was called until just before the FME_END_TCL was called.

FME_LogFileName     

• The filename of the logfile used by this translation. Will be blank if no logfile was used.

FME_StartingSeconds     

• The seconds since epoch when the translation was started (as returned by [clock seconds])

FME_StartingTimeStamp   

• The starting time of the translation, formatted as YYYY-MM-DD HH:MM:SS

FME_EndingSeconds   

• The seconds since epoch when the translation ended (as returned by [clock seconds]

FME_EndingTimeStamp     

• The ending time of the translation, formatted as YYYY-MM-DD HH:MM:SS

Passing Settings from Startup to Shutdown

Startup Script Failure

if [catch {set outputFile [open c:/temp/logfile.log {RDWR CREAT EXCL}]}] {
    puts stderr "ERROR - cannot open the log file."
} else {
    set gTranslationStartTime [clock format [clock seconds]]
    puts $outputFile "Translation started: $gTranslationStartTime
    close $outputFile
}

if {[file exists $outputFilename]} {
    error "The output file already exists - choose a different destination filename."
}

import os
if os.path.exists(outputFilename):
    raise Exception("The output file already exists - choose a different destination filename.)

Chaining Translations

 FME_END_TCL exec fme.exe anotherTranslation.fmw 2> NUL:

Examples

Q+A