Description

Fetches a target by performing a HTTP GET operation on a specified URL, storing the results in the specified target attribute.

Fetching Images

The ImageFetcher transformer can be used to retrieve images on an FTP site. Note that this transformer does not have authentication parameters, so the only way to provide a username/password is to enter it in the URL.

 

FTP Sites

Q) Can the HTTPFetcher be used to download a file from an FTP site?'


A) Yes.


Processing Results

When the attribute is retrieved, it may be used for further processing, however, it can be quite hard as it contains all the HTML tags and data not displayed in the browser such as content within <head></head> tags and <script></script>.

In order to extract the visible content you can use either some HTML parser such as BeautifulSoup, which also requires installing Python, or you can try to get rid of the tags with Regular Expressions.

 

Example

The attached example shows how to bring the attribute containing a multiline HTML code to a clean text visible in the browser. As a fun conclusion, it combines results with the map of the USA (50 most popular pages correspond well with 50 states) correlating state areas with page popularity.

User-added image