PythonCaller Transformer
The PythonCaller transformer lets you do things to features or groups of features using a Python script. Before using this transformer please be sure there is not already an FME transformer which does the task you want to do. Feel free to ask us in support (www.safe.com/support) if there is a transformer that can help you.
The PythonCaller can call a Function which you can use to process one feature at a time or a Class where you may want to do things to groups of features. In either case the EntryPoint parameter of the transformer is where you name the function or class in the script to run. The PythonCaller can use the full range of FME Objects including numerous methods and classes.
PythonCaller Example 1 (Function)
Attached workspace: PythonCallerFunction.fmw
As with all transformers FME Workbench will display a detailed transformer description when you have added the transformer to the canvas and have it selected. In this case we will use the sample code included in the description to add a timestamp to features. This is not an ideal example because this task could be done with the standard FME transformer the TimeStamper.
1. Add a PythonCaller to your workspace and select it. You should see the Transformer Description in a window below. If you don't see it there maybe a tab for it beside the log window or you can open it from the menu - View -> Window -> Transformer Description.
2. Open the transformer properties and click on the ellipsis to open the code editor. Notice there is already a Template Function and Class there for you. Delete the class because we are just using a function for now. Rename the function timestamper and copy in the code from example in the transformer description or from below. You may need to add some line returns and indentation to the code so that it looks like this (remember indents matter in python):
import fmeobjects
import time
# Template Function interface:
def timestampFeature(feature):
curTime = time.ctime(time.time())
feature.setAttribute("timestamp", curTime)
The script uses some FME objects methods so we need import fmeobjects and we are using the python time module so we need to import it as well. The function definition accepts FME feature as its only argument meaning all features will enter the function one by one for processing.
A new attribute is added to the feature with setAttribute method on the feature - this is actually an FME Objects method. 3. Click OK to dismiss the code editor. In the transformer you need set the
Entry Point to the name of the function
timestampFeature. Because we have added a new attribute called
timestamp we can expose it by entering its name in Attributes to Expose.
4. Use a creator to create some features to send you the PythonCaller and a Logger to look at the output. Every feature should have a timestamp attribute.
PythonCaller Example 2 (Function)
The FME Store includes a custom transformer called the
FuzzyStringComparer which uses the Python
difflib module to compare two string attributes and calculate a similarity ratio. In some ways this is a better example because it is something we cannot do with a regular FME transformer.
1. Open a blank workspace and add the FME Store transformer
FuzzyStringComparer by typing it on the canvas or by browsing the FME Store transformers in the Tranformer gallery. Right-click on the transformer and select
Embed which will allow you to edit the transformer and see its contents in the same workspace.
2. Once the
FuzzyStringComparer is embedded right click on it to
Edit and see the contents. Locate the PythonCaller transformer and edit its properties to access the script editor. Notice again we are using a function - this time to compare string attributes on a feature by feature basis. Again we are setting a new attribute which is called
FuzzyStringCompare.ratio. An
AttributeExposer is used later in the workflow to access the attribute we have created.
import difflib
def FuzzyStringCompare(feature):
string1 = feature.getStringAttribute('FuzzyStringCompare.string1').lower()
string2 = feature.getStringAttribute('FuzzyStringCompare.string2').lower()
ratio = difflib.SequenceMatcher(None,string1,string2).ratio()
feature.setRealAttribute('FuzzyStringCompare.ratio',ratio)
Important Note:
In the above example the function used to get an FME attribute value is from the now deprecated pyfme. If you are using fmeobjects the function to use would be feature.getAttribute()
PythonCaller Example 3 (Class)
Attached workpsace: PythonCallerClass.fmwAgain we can use the sample provided in the Transformer description for the PythonCaller. This time we will use a class rather than a function to calculate the area of all features. Once again this is not an ideal example because this task could be done with standard FME transformers.
1. Start a new blank workspace and use a Creator to create a polygon feature. Add a PythonCaller transformer and a logger to see the output.
2. Open the PythonCaller and open the script in the code editor. Copy in the code from the example in transformer description or from below. You may need to add some line returns and indentation to the code so that looks like this (remember indents matter in python):
import fmeobjects
class MyFeatureProcessor(object):
def __init__(self):
self.featureList = []
self.totalArea = 0.0
def input(self,feature):
self.featureList.append(feature)
self.totalArea += feature.getGeometry().getArea()
def close(self):
for feature in self.featureList:
feature.setAttribute("total_area", self.totalArea)
self.pyoutput(feature)
Again we are using a number of FME Objects methods so we need to import fmeobjects. On the input method we add each feature to a list and get the area from each feature which is added to the totalArea. On the close method we loop through each feature and add the total_area attribute to each one. It is important to note that if we want features to continue through the workspace they must be written out using the pyoutput() method.
3. Click OK to dismiss the code editor. In the transformer you need set the Entry Point to the name of the Class MyFeatureProcessor. Because we have added a new attribute called total_area we can expose it by entering its name in Attributes to Expose.
4. Run the workspace to ensure you have the total_area attribute on your feature. Open the C
reator transformer and increase the number of features. Run the worspace again to see that the total area of all features is being calculated.
Shutdown Python Script
A python script can be added to a workspace which will run when the workspace completes. For example you may want to copy files somewhere when a workspace is done, zip data, or send an email. The script runs after all of the transformers reader and writers in a workspace are finished but the script is still run a part of the FME process. You can access the FME parameter values in the same way as in the examples above using the
FME_MacroValues[] dictionary. A shutdown python script can also access a number of global variables which FME sets such as
FME_FeaturesRead, FME_FeaturesWritten and many more. For a complete list of the global variables please see the
FME Fundamentals Help under the topic FME_END_PYTHON.
Example of Shutdown Python Script
Attached workspace: ShutdownPython.fmwIn this example we add a shutdown python script to a workspace which sends an email using gmail when the workspace finishes. You can edit the script to use other mail servers is you wish. The subject of the email will say whether the workspace was successful or not and include the number of features written. A similar example can also be found in FME Workbench -> Help -> Startup and Shutdown Python Scripts.
1. Open any workspace and in the Workbench Navigator pane under
Workbench parameters - >Advanced you will see a parameter called
Shutdown Python Script. Here you can enter the Python code below which will can send the email using a gmail account. The comments in the code below signified with # explain most of the script. Notice two key points:
a) We are getting the user parameters from the
FME_MAcroValues dictionary in the same way as in the other scripts above. For example the gmail user name can be set when the workspace is run and comes from the published parameter like this: FME_MacroValues['GmailUser'].
b) Secondly, notice we use two global variables from FME:
FME_FeaturesWritten which is a python dictionary, and
FME_Status which is a boolean and tells us if the workspace succeed or failed.
# The smtplib gives us the emailing tools
import smtplib, fmeobjects
# Get the FME parameter values set when workspace run
to = FME_MacroValues['EmailTo']
gmail_user = FME_MacroValues['GmailUser']
gmail_pwd = FME_MacroValues['Password']
Subject = FME_MacroValues['SuccessSubject']
FailSubject = FME_MacroValues['FailSubject']
Emailfrom = FME_MacroValues['EmailFrom']
#Get number of features FME wrote
FeaturesWritten = str(FME_FeaturesWritten)
# If FME Fails then Change Subject
# FME_Status tells if FME was successful
status = FME_Status
if status == 0:
Subject = FailSubject
Message = 'Workspace Failed'
else:
Message = 'Workspace Successful the following features were written: ' + FeaturesWritten
# Sending the email
# Using the gmail smtp server
#You can edit this line to use your own email server
smtpserver = smtplib.SMTP("smtp.gmail.com",587)
smtpserver.ehlo()
smtpserver.starttls()
smtpserver.ehlo
smtpserver.login(gmail_user, gmail_pwd)
header = 'To:' + to + '\n' + 'From: ' + Emailfrom + '\n' + 'Subject:' + Subject +'\n'
print header
msg = header + '\n' + Message + '\n\n'
smtpserver.sendmail(gmail_user, to, msg)
print 'done!'
smtpserver.close()
2. Run the workspace using
File - Prompt and Run or the prompt and run button. Fill in the appropriate parameters. You will need your own gmail account user name (email address) and password and a
from email address which should be the same as the user. You can also fill in the
to email address and a subject line for failure and success. In the attached sample workspace the
Fail parameter is used to simulate workspace failure by send the features to the
Terminator transformer.
For a more complex shutdown python examples see the articles:
Notify an FME Server Topic on Success or Failure of a Workspace
(
http://fmepedia.safe.com/articles/How_To/Notify-an-FME-Server-Topic-on-Success-or-Failure-of-a-Workspace)
Example Workflow using FME, Python and Oracle
(
http://fmepedia.safe.com/articles/Samples_and_Demos/Example-Workflow-using-FME-Python-and-Oracle)