Overview
This article investigates everything related to logging and FME Server. With each release of FME Server more and more information is available to FME Server Administrators using the Web Admin UI (http://<HostName>/fmeserver-admin/home). Information about FME Engines, Services, Jobs and Security is available in the Web Admin UI and most trouble-shooting can now be done there.
Occasionally it is necessary to look into the FME Server log files if using older versions of FME Server or if trouble-shooting in the Web Admin UI fails, and this article is intended to be resource when that becomes necessary.
This article describes the location of the log files and what each log file contains. Within this article we will assume that Apache Tomcat is the Web Application Server used by FME Server and this is the default in FME Server 2012 or later.
Below is an architecture diagram with an overview of all the critical log files. It helps give you an idea of which elements of the FME Server architecture produce which log files.
Managing Log Files
FME Server creates log files for every workspace it runs and for all of it’s components. These log files can accumulate and eventually take up significant disk space. In FME Server 2012 a sample workspace is provided in the
Utilities repository to delete outdated log files. There is a scheduled task for this workspace that can be enabled to clean up the logs on a regular interval. For earlier version of FME Server please see the article
Purging FME Server Log Files.
FME Server Log Files
Location of the Log Files
There is one root directory within which FME Server outputs virtually all log files found here:
<FMEServerDir>\Logs
Within this direcotory there are a series of log files and a number of sub-folders also containing log files. Below are the details of each of these directories and log files.
FME Logs
Location: <FMEServerDir>\Logs\<ServiceName>\FME_<value>.log
An FME log is created for every workspace run on FME Server and is placed in a sub-directory for the service it was run by.
Description
FME Server runs workspaces which generate FME logs just as FME Desktop does. FME log files log the details of the actual translation done by an FME Engine.
It is important to keep in mind that when workspaces are run on FME Server the FME log files are placed in the logs directory mentioned above and within that a sub-directory for the Service that was used to run that workspace.* This occurs regardless of any log file settings made in the workspace. For example if you run a workspace using the Data Download Service the FME log will be written here:
<FMEServerDir>\Logs\datadownload\FME_<value>.log
*Note: FME logs can now be accessed using the Web Admin UI. Go to the Jobs > History tab and click on a job. There is a link to view or download the FME log file.
When would I look in here?
When a workspace fails on FME Server the FME log may provide more information on the failure.
FME Engine Log
Location: <FMEServerDir>\Logs\fmeEngine_Engine#.log.
There is one FME_Engine log file for each Engine running on the server
Description
This log details information about startup and translation processes of FME Engines. You can also view the parameters the engine is using. By default every time you restart the FME Server service this log file will be re-created. Each FME command (e.g. workspace job), is logged to the FME Engine log along with the details of any pre and post commands included in the sub-section. This log helps us see the actual FME command line run and the sub-section used when running this command.
When to look in here?*
-
To see which jobs have been processed by a specific engine;
-
View the actual FME command line run and the sub-section used to run the command. This is useful for debugging; if a translation has failed you can run it from the FME Server Console using this command. This just removes another layer of processing which is good practice when debugging;
-
If you want to see the parameters an Engine is using to process workspaces.
*Note: In the current version of FME Server all of the above tasks can be done in the Web Admin UI by clicking on a specific job in the Jobs -> History tab.
Configuration
There is one log file for each running engine. Within here you can configure various aspects relating to the engine.
Configuration file: <FME Server Install DIR>\Server\fmeEngineConfig.txt.
Relevant Parameters
• FME_SERVER_LOG_FILE: Sets the name and full path of the FME Engine log files.
FME Server Core Log
Location: <FMEServerDir>\Logs\fmeServer_<value>.log
An fmeServer log will be created each time FME Server is restarted.
Description
This log is also created each time the FME Server is started, this log is used to store messages created by the transformation manager and the repository manager. Parameters relating to the database FME Server is using are also visible in here. The full life-cycle of a job submission i.e. submitted to the queue, processed by an FME Engine and completed are also shown.
When to look in here?
-
If we need to know the job_id of a specific job in the queue we can find it in the fmeServer log.*
-
Identify if a job has been sent to an engine and a response returned.
-
If you have moved FME Server to use a production database and are having problems you can check to see if a connection is being made within here.
*Note: In the current version of FME Server this task can be done in the Web Admin UI by going to the Jobs -> Queued tab.
Process Monitor Log
Location: processMonitor_<value>.log
One ProcessMonitor logs is created each time FME Server is started.
Description
A new copy of the processMonitor log is created in the FME Server logs directory each time the FME Server is started. This log file details the FME Server components which were started and if any components needed re-start at any time. If an FME Engine fails to start we would see this in the processMonitor log file along with an error message. A licensing issue with an FME Engine would show up in this log. Also, all FME Server translation jobs (i.e. workspace jobs) are shown in this log.
When to look in here?
-
If the FME Engines are not starting then you need to check in here to identify why. It is most likely a licensing issue.
-
You can identify the build number of the Engines you are using on the FME Server.
-
To view the life-cycle of the job: receiving the request, sending to the engine and a response on the state of the translation can be viewed in here.
Scheduler Logs
Location: <FMEServer>\Logs\scheduler\core\fmeScheduler__<value>.log.
Location: <FMEServer>\Logs\scheduler\tasks\FME_<value>.log
Description
-
One fmeScheduler log is created each time FME Server is restarted and one FME log is created each time a workspace is run by the scheduler. The fmeScheduler log file records the start-up and shut-down of the Scheduler component and every workspace run by the scheduler
-
FME log files created when a workspace is run by the scheduler.
When to look here:
-
If an issue arises with the Scheduler this log my help Safe Software support see the problem.
-
If a workspace is started by the Scheduler but fails to run the FME log may show why the workspace failed.
Service Logs
Each FME Server service creates its own log file within a sub-directory in the FME Server logs directory. Also any FME workspace run on FME Server will write its log into the logs directory of the service it was run by. Below are the details of each Service log directory.
Data Download
Location: <FMEServerDir>\Logs\datadownload
Description
Two different types of log file are created in here:
datadownload_<value>.log
• This log file is created each time the data download service is restarted. It details every request received and processed using the data download service. Other useful information displayed is the job id, success of the job, and number of features returned by the translation. There is also a link to the corresponding log file described below
FME_<Value>.log
• This log files is created for each translation processed by FME Server. The log file provides logging down to the workspace level. It is similar to the log produce when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here. By default a new log file is produced for each job that is submitted using the console.
Data Streaming
Location: <FMEServerDir>\Logs\datastreaming
Description
Two different types of log file are created in here:
datastreaming_<value>.log
-
A new one of these is created each time the data streaming service is restarted. It details every request received and processed using the data streaming service. Other useful information displayed is the job id, success of the job, and number of features returned by the translation. There is also a link to the corresponding log file described below
FME_<Value>.log
-
One of these log files is created for each translation processed by FME Server. The log file provides logging down to the workspace level. It is similar to the log produced when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here. By default a new log file is produced for each job that is submitted using the console.
Job Submitter Service
Location: <FMEServerDir>\Logs\jobsubmitterservice
Description
Two different types of log file are created in here:
jobsubmitterservice_<value>.log
-
A new one of these is created each time the job submitter service is restarted. It details every request received and processed using the job submitter service. Other useful information displayed is the job id, success of the job, and number of features returned by the translation. There is also a link to the corresponding log file described below.
FME_<Value>.log
-
One of these log files is created for each translation processed by FME Server. The log file provides logging down to the workspace level. It is similar to the log produce when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here. By default a new log file is produced for each job that is submitted using the console.
KML Network Link
Location: <FMEServerDir>\Logs\kmlnetworklink
Description
Only one log file is created in the kmlnetworklink folder: kmlnetworklink_<value>.log. Since the KML Network Link is an extension of the data streaming service the main logging is done in the data streaming folder (see section above). This file just logs the lifecycle of the request and whether or not it has been zipped to a KMZ file correctly.
OGC
Location: <FMEServerDir>\Logs\ogc
Description
Two different types of log file are created in here:
ogc_<value>.log
-
A new one of these is created each time the OGC service is restarted. It details every request received and processed using the OGC service. This includes both GetCapabilities and GET requests. If a user is having problems either retrieving the GetCapabilites file or submitting a request it is worth looking in here to view what happened to the request.
FME _<Value>.log
-
One of these log files is created for each GET request processed by FME Server (not for the GetCapabilites request). The log file provides logging down to the workspace level. It is similar to the log produce when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here. By default a new log file is produced for each job that is submitted using the console.
Server Job Submitter
Location: <FMEServerDir>\Logs\serverjobsubmitter
Description
Every time a workspace is run using the FMEServerJobSubmitter transformer in workbench a log file is created in this folder. The log file provides logging down to the workspace level. It is similar to the log produce when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here.
Web User Interface
Location: <FMEServerDir>\Logs\webui\webui.log
Description
This log file details output from the FME Server web user interface, both the standard and administrative are logged to the same file by default. It shows the registration of the services as a plug-in with the web application and any error or warning messages produced by the web application.
REST
Location: <FMEServerDir>\Logs\rest\
Description
Two different types of log file are created in here:
rest_<value>.log
-
This log is created each time the REST service is restarted and records requests to the REST serivce
FME_<Value>.log
-
If an FME workspace is run via the rest service a standard FME log file is created here.
Notifier
Locations: (all created each time FME Server Restarts)
<FMEServerDir>\Logs\notifier\core
<FMEServerDir>\Logs\notifier\subscribers\logger
<FMEServerDir>\Logs\notifier\subscribers\push
<FMEServerDir>\Logs\notifier\subscribers\email
Description
FME Server’s Notification service maintains two types of log here.
-
The Notifier core log records request to notification service and any subsequent push notifications to subscribers of a topic.
-
The Notifier subscriber logs records actions done by three types of subscribers, push, email and logging. Once of each type of log is created each time FME Server is restarted. These should not be confused with subscription FME logs described below.
FME Subscription
Location: <FMEServerDir>\ fmesubscription\FME_<value>.log
Creates one log for each workspace run by a subscriber to the notification service.
Description
FME logs found here are logs created by workspaces which have subscribed to FME Server’s notification service and were triggered by notification to a topic.
Catalog
Location: <FMEServerDir>\Logs\fmecatalog
Description
This log file details requests to FME Server’s Catalog Service.
SOAP
Location: <FMEServerDir>\Logs\soap\soap_<Value>.log
Description
This log file details every request that has come in using the SOAP servlet.
Spatial Direct (deprecated)
Location: <FMEServerDir>\Logs\spatialdirect
Description
This service is no longer part of FME Server 2012 or later versions. For older versions of FME Server the log files relating to spatial Direct will be populated within this folder.
Other Log Files
Console
Location: <FMEServerDir>\Logs\console
Description
This folder provides logging for any jobs run using the FME Server Console. The log file provides logging down to the workspace level. It is similar to the log produce when a workspace is run in FME Workbench. Crucially the number of features written out can be viewed in here. By default a new log file is produced for each job that is submitted using the console.
Installation
Location: <FMEServerDir>\Logs\installation\installation.log
Description
This log contains all the logging linked to the installation of FME Server. If you have had problems with the installation of FME Server you should look in here for clues. If FME Server installed correctly you do not need to worry about it.
Web Application Server Logs
Location varies: <FMEServerDir>\Utilities\tomcat\logs (FME Server 2012 Default)
The Web Application Server is always a third a party application used to run FME Server services although it may part of the FME Server installation. Application servers generally do their own logging and these logs can sometimes be helpful in trouble-shooting problems with FME Server Services. Starting in FME Server 2012 the default installation includes Apache Tomcat which will log any errors it experiences.