INSPIRE Protected Areas Demo for Sweden

Introduction

INSPIRE is a European Union directive that requires member states and agencies to share spatial data in order to better support environmental decision making. The key approach to enable this data exchange is the use of a common data model accessed using open standards such as OGC WFS (Open Geospatial Consortium Web Feature Services). This demo shows how FME can be used to implement this approach by reading from disparate data sources with different data models and transforming their schema to conform to the common INSPIRE schema for Protected Areas. This is done with a Loader workspace which reads the disparate sources, processes the required schema transformations and loads them into a PostGIS spatial database. It also demonstrates how FME can be used to construct an OGC web service to publish this data. A workspace was built to read from PostGIS and write out GML 3.2.1 data that complies with the INSPIRE Protected Areas schema. This was then published to the WFS service on FME Server.

Thus this prototype successfully demonstrates that FME Desktop can be used to design data transforms and FME Server can be used to publish them via OGC web services in order to support spatial data integration of different real world data sources into the common data structure defined in the INSPIRE protected area schema. All this was done without any programming or scripting. The only resources required were XML documents for the getCapabilities.xml, describeFeature.xml, a sample record for protected sites and a sample parent document that were adapted to serve as templates. It is hoped that this prototype will serve as an example implementation for other jurisdictions and inspire other users to consider using Spatial ETL tools such as FME to bridge the gap between their diverse data sources and the evolving complexities of the INSPIRE specifications with minimal risk and effort.

A special thanks to Metria[[1]], our partners in Sweden, for their central role and participation in this project. Also, thanks to Lantmäteriet [[2]], the Swedish mapping, cadastral and land registration authority for the demo source data.

Note that this demo requires deployment on FME Server to fully implement it. For a more basic example that demonstrates the load, transform and gml publication process FME is capabile of all within one workspace, it is recommended that you first investigate the INSPIRE Geographic Names Demo (Article#:000002067)

Source Data

The demo uses data from Natura2000 – a protected areas database for Europe, Swedish NVR – protected areas data from Swedish Environment, and Helcom, a database of protected areas for the Baltic region. All of these have different native schemas which had to be mapped to the INSPIRE protected areas schema.

Processing Steps

Loader workspace (Source to PostGIS):

1. Read the source database (Natura2000, NVR, Helcom or alternate) 2. Join all the required site fields together with a FeatureMerger based on site code. 3. Create any required field not present in source. 4. Process each feature through a SchemaMapper. This defines all the attribute mappings from the source schema to the destination Inspire protected areas schema. It is also used to create many of the required fields not present in the source and then sets default values for those fields. 5. Extract OGC geometry as GML 3.2 XML and store in a _geometry text attribute 6. Generate the XML for each child entity (one to many relationships) - such as protected site activity - with XMLTemplater. XMLTemplater uses an XML prototype of the subrecord entity and then substitutes attributes into that template using fme:get-attribute() and fme:get-xml-attribute functions located at the appropriate place within the template. This works like a document merge application. 7. Merge activities into protected site features containing the list of XML snippets for the activities and one large XML snippet for the geometry. 8. Write the complete protected site feature to the ProtectedSite table in the Inspire PostGIS database which is modelled after the Inspire Protected Areas schema.

Exporter workspace (PostGIS to GML and WFS):

1. Read the source Inspire ProtectedSite table from the PostGIS database. Use the extents supplied with the BBox parameters via the WFS getFeature request. 2. Generate the XML for each protected site feature with XMLTemplater substituting attributes into protectedSites_featureTemplate.xml using fme:get-attribute() and XML snippets using fme:get-xml-attribute() 3. Merge into a single feature containing the list of the protected site XML snippets 4. Generate the XML for the entire dataset with XMLTemplater substituting the protectedSites features XML into protectedSites_datasetTemplate.xml using fme:get-xml-list-attribute() 5. Test for schema validity against the Inspire schemas (this test is optional as disabling the test improves performance). 6. Write the completed XML doc using the text file writer

Note that in the FME Viewer the WFS data can easily be overlaid with other vector or raster data sources such as a WMS boundary layer to give context to the protected sites. If you have any problems, please contact support@safe.com and make sure you specify Inspire Protected Areas Demo in the subject line.

 

fmeserver.com DEMO

To try out this service, all you need is a standard OGC compliant WFS reader, such as FME Viewer. You will also need a username and password to access this service from our fmeserver.com demo site. Please contact support@safe.com and mention the protected areas INSPIRE demo and they will provide you with access credentials

For FME Viewer, follow these steps:

1. Download the INSPIRE application schemas for protected areas under the resources below: protectedAreasSchema.zip . Unzip these to a folder that is accessible by your viewer.

2. In FME Viewer, select File – Open dataset.

3. In the reader settings, choose WFS as the format and set the dataset to:
http://fmeserver.com:80/fmeogc/INSPIRE_PA/InspirePAexporter_OgcWfs.fmw

4. In the WFS parameters, set the Application Schema to point to: “ProtectedSitesFull.xsd”. Note that you will need both this file and all the others that came with protectedAreasSchema.zip to be in the same directory. You can download this from the bottom of this page.

5. Optionally, you may set the search envelope. If you don’t you will see data that uses the default search envelop. This is done to limit the data volumes resulting from any one transaction. Your search envelop should be somewhere between 10 and 28 degrees longitude for the x value and between 53 and 70 degrees latitude. Note that if you pick too large an area, you will be limited to 600 features per query, and if you pick too small an area or something out of range you may get no data at all.

6. Click on the Table list parameter and set it to: "ps-f:ProtectedSite {ProtectedSite}"

7. Leave all the other parameters with their default values and press ‘OK’ to see the protected sites data for your selection. Note that it will typically take one or two minutes for your request to be received, transmitted and rendered, depending on your internet access speed.

Demo Installation in your FMEServer

Loading Source data onto PostGIS ProtectedSite
 

1. Set up Loaders on an author machine that can see PostGIS on the Server.

2. Unzip source data to local sister directory called 'source'

3. Unzip the workspaces_resources_v7.0. This includes a resources folder, a schema folder

4. There is one loader for each of the source datasets. Open each of the 3 Loaders (N2000, NVR, Helcom) and edit as follows: a. Set source path to source FFS data path b. Adjust PostGIS user / password, server / database name c. Adjust the paths to the schema mapper csvs and to the xmlTemplates. d. Make sure table delete / truncate is set to no on destination feature type (unless this is the first Loader running in which case it should create table by default anyways).

5. Run each of the Loaders in succession. Check that the data is loaded after each run using Viewer reading PostGIS. You may want to run the Helcom Loader first as this is the smallest dataset and runs the fastest, so if you have any problems you won't waste much time. Note that the table name is critical for the WFS - it wont run if you change it. You can test load to ProtectedSite2 or ProtectedSites, but it won't work if you dont use 'ProtectedSite'. Also note that I use the public user to write but I use no user to read, which seems to work ok.

To publish to FME Server:

1. Extract and configure InspirePAexporterOgcWfs.fmw locally first.

2. Modify the PostGIS connnection info as needed. Modify the xmlTemplate paths.

3. Test run InspirePAexporterOgcWfs.fmw. Open the output gml file with Viewer using the protectedSitesFull.xsd as the target application schema and make sure you can see the output and attributes.If you have problems check to make sure encoding is unset on the Text  File writer. This is encoded in the dataset XML template and if these 2 dont agree then you may have trouble reading the GML.

4. Select File - publish to Server. Login as author.

5. Create a new repository for ProtectedAreas if not already available.

6. Deselect data source but publish all other resources including the 5 xml templates, getCapabilities_v2.0.xml and desribeFeature_v2.0.xml. Use the '+' multiple resource selection tool to do so. Note that all resources other than the source dataset will be local to the .fmw. Also make sure you publish all resources every time you republish the workspace, should you need to make any edits.

7. Select jobSubmitter, WFS for services to register with. Complete the publish wizard by clicking 'Publish'.

8. Go to the FME Server Web UI. Login as author.

9. Open the WFS - ProtectedAreas - InspirePAexporterOgcWfs.fmw configuration in the FMEServer WebUser UI.

10. Set the default bbox values (0 to 360 if you dont care) and set the WFS getCapabilites response file = getCapabilities_v2.0.xml

WFS describeFeature response file = describeFeature_v2.0.xml

11. Test this by clicking on the getCapabilities button. Click back on your browser.

12. Click on Show URL, copy to your buffer.

13. Open FME Viewer, open WFS data source and paste in the URL

14. Go to WFS reader parameters. Set application schema = ProtectedSitesFull.xsd. Note that you need the full directory of Inspire xsds locally accessible to wherever your WFS reader / FME Viewer is, not just the one ProtectedSitesFull.xsd file, since it includes a bunch of others. As far as I know, you dont need these xsds on the server but I could be wrong.

15. Select table = ps-f:ProtectedSite

16. Set extent selection - if you dont you will get the extents set when the Exporter was published. Dont try to select all the data or the server will likely crash and you will need to use the Server Admin UI to kill the job and avoid a continual job restart. You should probably set a max features on the exporter's PostGIS reader advanced settings (1000 or so).

17. Click ok and see if the Viewer can read your WFS server.

18. If you have problems test with the jobSbmitter. Also consult the logs under FMEServer\logs\ogc etc. Make sure all the required inspire xsds are included at the same location as ProtectedSitesFull.xsd. Also try reading with DataInspector.

19. Note that you shouldn't change the name of the workspace. It is hardcoded into the getCapabilities.xml. If you change it then you need to change it there as well.

20. The getCapabilities.xml also stores the repository name and the server name in the URLs for the various request responses, as follows: http://<serverName>:80/fmeogc/<repositoryName>/InspirePAexporterOgcWfs.fmw So dont forget to adjust all of these to be consistent.