QGIS Planet

It’s been quite some time since I last had a chance to blog and a lot has happened since then. Not least of which is that QGIS 2.12 has now been released with a ton of new features that I’ve neglected to write about! To try and get things moving along here again I’m planning on writing a short series exploring how variables work in QGIS 2.12 and the exciting possibilities they unlock. First, let’s look into how variables can be used with QGIS map composer…

So, let’s get started! A new concept introduced in QGIS 2.12 is the ability to set custom variables for use in QGIS’ expression engine. The easiest way to do this is through the “Project Properties” dialog, under the “Variables” section:

Default project variables

You’ll see in the screenshot above that a blank project includes a number of read-only preset variables, such as @project_path and @project_title. (All variables in QGIS are prefixed with an @ character to differentiate them from fields or functions). You can add your own variables to this list by clicking the + button, as shown below:

Adding new variables to a project

Here I’ve added some new variables, @project_version and @author. Now, any of these variables can be used anywhere that you can use expressions in QGIS, including the field calculator, data defined symbology, labelling, map composer text, etc. So, you could make a map composer template with a label that includes the @author, @project_version and @project_path variables:

Variables in a composer label

Sure, you *could* also manually enter all these details directly into the label for the same result. But what happens when you have multiple composers in your project, and need to update the version number in all of them? Or you move your project to a new folder and need to make sure the path is updated accordingly? Manually updating multiple composers is a pain – make QGIS do the work for you and instead use variables! This would especially be helpful if you’re saving map composer templates for use across multiple projects or users. Using variables will ensure that the template is automatically updated with the right details for the current project.

Another neat thing about QGIS variables is that they can be inherited and overridden, just like CSS rules. Opening the options dialog will also show a Variables group for setting “Global” variables. These variables are always available for your QGIS installation, regardless of what project you’re working on at the time. If your workplace tends to reorganise a lot and constantly shuffle your department around, you could add a global variable for @work_department, so that changing the global variable value in one place will automatically filter through to any existing and future projects you have.

Global variables

And like I mentioned earlier, these variables are inherited through various “contexts” within QGIS. If I reopen the Project Properties dialog, you’ll see that a project has access to all the global variables plus the variables set within that specific project. In addition, by adding a variable with the same name to the Project variables the value of the Global variable will be overridden:

Overridden variables

There’s also a variable editor within each individual composer’s properties tab, so variables can also be set and overridden on a composer-by-composer basis within a project. It’s a really flexible and powerful approach which both simplifies workflows and also opens up lots of new possibilities.

Stay tuned for more on this topic – this topic has only just scratched the surface of how expression variables have changed QGIS! (You can also read part 2 and part 3)

Another sold-out event with a programme packed with useful, interesting and delightful talks. Fifty seven (57!) folk blew in from all over Scotland through a freezing rain but hot coffee and pastries were waiting in the Informatics Forum at the University of Edinburgh.

First up was an overview the current status of the QGIS project by Saber from Lutra Consulting. It was good for people new to QGIS and open-source to see how the project is organised and run and the direction it is taking. Pete, also from Lutra Consulting, then gave a quick summary of the bits of core functionality they have been working on including the new ruled based labelling system.
The group then split into two for 90 minute workshop sessions on cartographic labelling and advanced Atlas usage – a tough choice! Chris, from Ordnance Survey, presented a detailed how-to on the new ruled based labelling tools using some OS open data, interspersed with some slides on guidelines to good cartographic practices and labelling tips. The slides and material for this workshop are available here: https://dl.dropboxusercontent.com/u/185489368/QGIS-Scotland2015.pdf

Heikki, from thinkWhere, lead us through the process of using Atlas in Print Composer to automate map production for a series of maps containing a main context map and an inset overview map. Nothing better than doing something once and then being able to repeat it at the click of a button! The slides and material for this workshop are available here: https://github.com/HeikkiVesanto/Scottish_QGIS_User_Workshop

A break for lunch and a good hour of catching up with users from across all sectors – local government, central government, academia, forestry, planning consultancies, developers, student life and education. QGIS is popular and is obviously a flexible tool that meets many demands.

After lunch, Neil, from thinkWhere, organised a quick-fire “quirky QGIS quiz” with random questions from all aspects of FOSS4G demanding quick thinking for true/false answers. At least half the audience grabbed prizes courtesy of thinkWhere and Ordnance Survey.

What followed was a series of lightning talks on different aspects of using QGIS. Amy, from Cawdor Forestry, gave a brief overview of the plugins available in the QGIS plugin repository and highlighted some of her personal favourites. Paul, from Scottish Water, showcased some of the complex workflows created using SAGA, QGIS and the Processing Toolbox to model hydrological process. Ross, from Inverclyde Council, demonstrated the use of the QGIS Road Graph plugin to generate walking routes to school across a custom road and path network. Seb, from West Dunbartonshire Council, showed us how QGIS had put them in a happy place and showed how a “hearts and minds” campaign championing QGIS had changed the way they worked. Steve, fae Embra, gave some information on how to give back to the QGIS project through submitting Processing scripts and plugins to the repository. Don’t reinvent the wheel! Ross, from Angus Council, gave a quick demonstration of setting up a local plugin repository that could be used to share custom plugins or control access to plugins in an internet-less environment.

By this time, tea and cake was required and it gave everyone an opportunity to mix and ask questions of the speakers.

The last session was as series of longer talks started by Gemma, from Ordnance Survey, explaining how open-source software is used extensively at Ordnance Survey to underpin a lot of the cartographic processes and workflows. They use QGIS 2.8 LTR for stability and consistency across the business and a selection of plugins from both OS developers and the community. QGIS is used to generate all the cartographic styles sheets for the OS vector products: https://github.com/OrdnanceSurvey/OS-VectorMap-District-stylesheets

Tom, from EDINA, explained the processing of creating and delivering a QGIS training course to University staff and students. The first class sold out in next to no time and there is demand for more. Makes sense really, doesn’t it?

Steve, from GeoGeo, wrapped up the day with the kind of mapping we’d all like to be doing – high resolution elevation models, time series analysis of shadows, viewsheds across the Edinburgh skyline and analysis of rooftops for potential solar panels with sub 1m resolution LiDAR datasets. He uses a mix of QGIS, SAGA, Blender and other FOSS to inspiring effect. Check his Flickr stream: https://www.flickr.com/photos/stevefaeembra/

The day finished with Pete (Lutra Consulting) releasing into the open a new Search plugin for QGIS called Discovery (http://www.lutraconsulting.co.uk/products/discovery/) based on the PostGIS Search plugin from Tim Martin (Ordnance Survey). A very useful addition to any QGIS installation.

The day was sponsored by EDINA, thinkWhere and Ordnance Survey.

Links to slides will be coming shortly.

The task of projecting, or unfolding the spherical Earth onto a flat map, is an age old problem in cartography. Projection almost always introduces distortion, most projections cannot preserve angles, areas and distances at the same time, they may be conformal (angle-preserving), equal-area (area-preserving) or equidistant (distance preserving) but not all at once. The only exception is a Globe, which preserves angles, areas and distances perfectly. Thus a projection is a compromise.

The choice of projection depends on a map’s use, scale and audience. Conformal projections, for example, are preferred for nautical charts or small scale maps because they locally preserve angles necessary for navigation and survey drawings. Equal-area projections are best suited for maps of broad continental region as they preserve the relative sizes of countries, seas and oceans and allow comparison between regions. Finally, there are hybrid projections that minimise the distortion by merging conformal and equal-area projections, these can be used to create visually pleasing maps of the entire Earth (for a guide to selecting a map projection see Fig. 9 in Jenny (2011), link below).

But how does one measure the degree and type of distortion in a map projection?

One elegant method was developed in the 1880s century by the French cartographer Monsieur Nicolas Auguste Tissot, the Tissot’s Indicatrix (or Tissot’s Ellipse). This mathematical contrivance consists of a grid of infinitely small circles that measures the degree and type of distortion caused by projection. While Monsieur Tissot’s approach is mathematical, involving infinity small circles, his technique can be approximated overlaying a regular grid of large circles and crosses to a map.

The Indicatrix Mapper plugin for QGIS by Ervin Wirth and Peter Kun creates a Tissot’s Indicatrix by adding a vector layer of circles and crosses in a gridded pattern on a map. The degree and type of distortion of the Tissot’s Indicatrix reveals the class of map projection as follows: –

• If a projection is conformal, the area of circles and sizes of the crosses will change while the shapes of circles remain the same and intersection angle of the crosses will always meet at 90 degrees e.g. Mercator projection

• If a projection is equal-area, the area of the circles will remain the same while the shape of the circles change and intersection angle of the crosses will not always meet at 90 degrees e.g. Mollweide projection or Hammer projection

• If a projection preserves neither property, the area of the circles and their shape will change, and the intersection angle of the crosses will not always meet at 90 degrees e.g. Robinson

After adding the Indicatrix Mapper plugin to QGIS (menu Plugins – Manage and Install Plugins) first add a basemap using the OpenLayers plugin e.g. Bing Aerial layer, then click the Indicatrix Mapper icon and run the plugin using default settings. You can then select different projections (lower right in world icon QGIS) to see the effects of various protections on the Tissot Indicatrix. If the circles appear as squares after selecting a different projection, right click the Circles layer in the layers panel, then select the Rendering tab and deselect the Simplify geometry check box. Also, turn off the basemap layer when using different projections, unfortunately the OpenLayers plugin only supports Google Mercator projection (EPSG 3857). To create the basemap below, that can be displayed using different projections, I styled vector data downloaded from Natural Earth and OpenStreetMap.

Mercator Projection – the area of the circles and size of the crosses increase towards the poles but their shape remains the same.

Mollweide Projection – the area of the circles remain the same but their shapes are distorted, the crosses do not always intersect at 90 degrees.

Robinson Projection – both the area of the circles and intersection angle of the crosses circles vary.

It is important to note that a Tissot’s Indicatrix generated in QGIS is an approximation of mathematical ideal, we are not no longer dealing with infinity small circles. As a result, here will be some minor distortion visible towards the edge of a map independent of the projection used; notice that the circles in the Mercator projection nearest the poles are not quite symmetrical or the circles at the edge of the Mollweide projection do not appear to preserve area as they should. This anomalous distortion can be minimised by reducing the size and spacing of the circles and crosses created by the Indicatrix Mapper plugin. However, despite these limitations a Tissot’s Indicatrix elegantly reveals the distortion present. This is something to important to understand when when choosing a map projection.

References:

Last year we described a command line method that adds ESRI REST layers in QGIS. Well, a team at the Geometa Lab in the University of Applied Sciences Rapperswil (HSR) Switzerland, have released a plugin for QGIS that adds ESRI REST layers via a GUI (Github page). The plugin is experimental so you will need to tick the box “Show also experimental plugins” in the settings panel of the “Plugins – Manage and Install Plugins” dialogue in order to add the plugin to QGIS. The following URLs lists numerous REST layers in the plugin’s GUI:

http://services.arcgisonline.com/arcgis/rest/services

http://basemap.nationalmap.gov/arcgis/rest/services

http://services.nationalmap.gov/arcgis/rest/services

Reference:

REST API Connector Plug-in Wiki Page

In this tutorial I will show you how to create a Hillshaded topographic map in QGIS. We will be using Shuttle Radar Topography Mission (SRTM) data, a near global Digital Elevation Model (DEM) collected in February 2000 aboard NASA’s Space Shuttle Endeavour (mission STS-99). The mission used a X-Band mapping radar to measure the Earth’s topography, built in collaboration with the U.S. Jet Propulsion Laboratory, the U.S. National Imagery and Mapping Agency (now the National Geospatial-Intelligence Agency), and the German and Italian space agencies.

The raw radar data has been continuously processed and improved since it was first collected. Countless artefacts have been painstakingly removed and areas of missing data have been filled using alternate data sources. The version we will be using is the 1 Arc-Second Global SRTM dataset, an enhanced 30 meter resolution DEM that was released last year. It is a substantial improvement over the 3 Arc-Second / 90 meter SRTM data previously available for Ireland. SRTM elevation data can be downloaded from the United States Geological Survey’s EarthExplorer website.

When first loaded into QGIS (via Add Raster Layer), the DEM is displayed as a rather uninformative black and white image.

It is therefore necessary to apply a suitable colour ramp that accentuates topography. While it is possible to create your own colour ramp, or use one of the colour ramps provided by QGIS, superior colour ramps can be downloaded using Etienne Tourigny’s Color Ramp Manager (Plugins – Manage and Install Plugins). After the plugin is added to QGIS, go to the Plugins menu again and choose the Colour Ramp Manager.

In the window that pops up, choose the full opt-city package and click check for update. The plugin will then download the cpt-city library, a collection of over a hundred cartographic gradients (version 2.15). After the package downloads, quit the dialogue.

Back in QGIS, right click the DEM layer to bring up the Layer Properties dialogue. In the Style tab, change the render type from single band grey to single band pseudocolor. Then click new color ramp and new color ramp again, choose the cpt-city color ramp to bring up the cpt-city dialogue. Click topography and choose the sd-a colour ramp. While this is an excellent colour ramp, I find its colours are a bit too strong for my liking.

Still in the Layer Properties dialogue, change the min and max values to match your DEM’s lowest and highest elevations values and click classify, this applies the new colour ramp. Next, change the brightness to 30 and lower the contrast and saturation to -20. Click OK to apply the new style and quit the Layer Properties dialogue.

Next we need to create a Hillshade layer from the DEM, a 3D like visual representation of topographic relief. This is achieved via the menu Raster – Analysis – DEM (Terrain models). There is one small catch, the hillshading algorithm assumes the DEM’s horizontal units are in meters (they are decimal degrees). We need to enter a scale correction factor of 111120 (in the Scale ratio vert. units to horiz. box). Once that is all done, select an output path to save the generated hillshade and click OK. Generating a hillshade may take up to a minute depending on the size of your DEM.

After the hillshade is created, open its Layer properties dialogue. Change the min and max values to 125 and 255, increase its brightness to 45 and contrast to 20. Finally, switch the blending mode from normal to multiply. This allows the DEM beneath the hillshade to show though. Click OK to apply the new style.

If you followed these steps correctly you will have created a fine looking topographic map similar to the one below. It’s also possible to create contours but that’s a tutorial for another day.

Technical note:

There are two hillshading algorithms available in QGIS, one by Horne (1981) and another by Zevenbergen and Thorne (1987). Jones (1998) examined the quality of hillshading algorithms, he found the algorithm of Fleming and Ho€er (1979) is slightly superior to Horne’s (1981) algorithm. Zevenbergen and Thorne’s (1987) algorithm is a derivation of Fleming and Ho€er’s (1979) formula. QGIS uses Horne’s (1981) algorithm by default.

References:

Horn, B.K., 1981. Hill shading and the reflectance map. Proceedings of the IEEE, 69, 14–47.

Jones, K.H., 1998. A comparison of algorithms used to compute hill slope as a property of the DEM [PDF]. Computers & Geosciences, 24, 315–323.

EN | PT

Now and then I get too many map symbols (points) in the same place, and I thought how nice it would be if we could drag n’ drop them around without messing with their geometries position, just like we do with labels. That thought gave me an idea for a cool hack.

Choose your point layer and start by creating two new fields called symbX and symbY (Type: Decimal number; Size: 20; Precision: 5). Now go the layer properties and in the Style tab edit your symbol. For each level of your symbol select “map units” as the offset units, and set the following expression in the offset data define properties option:

CASE WHEN symbX IS NOT NULL AND symbY IS NOT NULL THEN
    tostring($x - symbX) + ',' + tostring($y - symbY)
ELSE
    '0,0'
END

Be aware that, if your coordinates have negative values, you need to adapt the code. E.g., If you have negative values in X you should use “tostring(symbX -$x)” instead.

Now, temporarly  label your layer with a small convenient text (I used a centered ‘+’ (plus sign) with a white buffer) and set its coordinates to data defined using the symbX and symbY Fields.

From this point on, when you use the move label tool, not only the label position change but also the actual symbol! Pretty cool, isn’t it?

Notice that the features geometries are not changed during the process. Also, remember that in this case you can also add leading lines to connect the symbols to the original position of the points.

Sea Surface Temperature data downloaded by Oceancolor Data Downloader.

The Oceancolor Data Downloader is a new plugin for QGIS from the Mapping and Geographic Information Centre of the British Antarctic Survey that downloads Oceancolor and Sea Surface Temperature data from NASA’s Oceancolor website. The plugin currently downloads three datasets:

• MODIS AQUA chlorophyll concentration
• SeaWiFS chlorophyll concentration
• MODIS AQUA night time Sea Surface Temperatures

The data accessed includes daily, 8 day, monthly and yearly composites, all of which can be saved to disk while downloading. Future plans for the plugin include additional access to other datasets such as ocean Net Primary Production, selection by bounding box, the ability to save in other formats, a progress bar etc.

I used the plugin to download global Sea Surface Temperatures for the 13th Jan 2015. I then used shapefiles from Natural Earth to create a simple basemap. I finally chose the IBCAO Polar Stereographic projection (EPSG: 3996) to create a map centred on the North Pole.

If you use the plugin to produce published research, please cite:

EN | PT

I had the need to add columns with the coordinates of polygons centroids. I came up with the following expressions to calculate X e Y, respectively:

xmin(centroid($geometry))
ymin(centroid($geometry))

The expression seems quite simple, but it toke me some time before I realize that, not having a x(geometry) and y(geometry) functions, I could use the xmin() and ymin() to get the coordinates of the polygons centroids. Since this wasn’t the first time I had to use this expressions, this post will work as a reminder for the future.

A very handy plugin for QGIS I use day to day is go2streetview by Enrico Ferreguti. The plugin adds an icon to the tool bar in QGIS and when selected I can click a road or street on a base map and a window will open that displays the Google Street or a Bing Maps Bird’s Eye view of the location. The camera’s direction and location is highlighted by a blue marker. I use the plugin when tracing boundaries of parks, open spaces and foot paths from aerial imagery. If the imagery is blurred or the view is obscured by trees, I click a point on a nearby street to see the location up close. The plugin works wherever Google Street view and Bing Birds Eye has coverage.

For example, in the screen-shot below notice there is a footpath leading to a bus shelter that’s not mapped by OpenStreetMap. I know where it is now, I will add it to my map.

Plugin: go2streetview

EN | PT

Recently I had the need to add labels to features with very close geometries, resulting in their collision.

Using data-defined override for label’s position (I have used layer to labeled layer plugin to set this really fast) and the QGIS tool to move labels, it was quite easy to relocate them to better places. However, in same cases, it was difficult to understand to which geometry they belonged.

I needed some kind of leading lines to connect, whenever necessary, label and feature. I knew another great plugin called “Easy Custom Labeling“, by Regis Haubourg, that did what I needed, but it would create a memory duplicate of the original layer, wish meant that any edition on the original layer wouldn’t be updated in the labels.

Since the data were stored in a PostgreSQL/Postgis database, I have decided to create a query that would return a layer with leading lines. I used the following query in DB manager:

SELECT
  gid,
  label,
  ST_Makeline(St_setSRID(ST_PointOnSurface(geom),27493), St_setSRID(St_Point(x_label::numeric, y_label::numeric),27493))
FROM
  epvu.sgev
WHERE
  x_label IS NOT NULL AND
  y_label IS NOT NULL AND
  NOT ST_Within(ST_Makeline(St_setSRID(ST_PointOnSurface(geom),27493), St_setSRID(St_Point(x_label::numeric, y_label::numeric),27493)),geom))

This query creates a line by using the feature centroid as starting point and the label coordinate as end point. The last condition on the WHERE statement assures that the lines are only created for labels outside the feature.

With the resulting layer loaded in my project, all I need is to move my labels and save the edition (and press refresh) to show a nice leading line.