OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Eighth GSoC Report – Multithreading on Processing

What do I have completed this week?

  • Created a mechanism to hold the result of the algorithm taking into account that there can be several algorithms running at the same time. The results are stored in a global variable called algResults. This variable is accessible via python console (e.g. “processing.algResults”).

algResults

  • The signals are connected in a different way in order to close the dialog and quit the thread when the algorithm is finished.
  • I have updated my fork and merged my work into the master branch.
  • Added the runalgIterating method to the AlgorithmExecutor with some changes to support the signals to set the progress
  • AlgorithmDialog refactored.
  • I have been searching for a solution to stop the algorithm execution and I think that is possible to create a sort of checkpoints inside the algorithm to check if we want to stop the execution and raise an exception to stop the thread event loop.

What am I going to achieve for the next week?

  • Allow the user to cancel the execution when running the runalgIterating.
  • Solve the problem with QGIS dependencies on Ubuntu and test the refactored algorithms.
  • Continue working on the algorithm iterating.

Is there any blocking issue?

  • I had to postpone the tests due to some problems installing QGIS dependencies on Ubuntu.
  • Didn’t had time to implement a solution to stop the execution of the algorithm.
Learn More

Seventh Report

What do I have completed this week?

  • Processing refactored in order to create a non-blocking implementation of the python console. The algorithm is processed in another thread asynchronously and the result is printed to the python console when the execution is finished.
  • AlgorithmDialog refactored to create a non-blocking dialog interface with a button to cancel the execution.
  • R algorithms refactored to support the progress signal.

What am I going to achieve for the next week?

  • Create a mechanism to hold the result of the algorithm taking into account that there can be several algorithms running at the same time.
  • Figure out a way to stop the execution of the algorithm. In third-party algorithms it can not be possible to just interrupt the thread loop and quit the thread. The only available option is to discard the result because the algorithms don’t have any kind of check-point that allows to verify once in while if we want to stop the execution. Kill the thread during the execution can be dangerous and it is not a good option.
  • Test the refactored algorithms.

Is there any blocking issue?

  • It took more time than I expect to create a non-blocking interface.
  • Quitting the thread execution turned out to be far more complicated due to the fact that the algorithms don’t provide any mechanism to allow quitting the thread loop during the execution.
Learn More

Sixth Report

What do I have completed this week?

  • Solved the issue with SAGA algorithms that makes the Toolbox crash.
  • Refactored Grass/Grass7 and OTB algorithms to work with signals.
  • Bug fixes.

What am I going to achieve for the next week?

  • Test Grass/Grass7 and OTB algorithms.
  • Add an option to cancel the execution of the algorithm and kill the thread.
  • Code refactoring.
  • Start implementing the runalgIterating.

Is there any blocking issue?

  • I had a very busy week due to my university deadlines.
  • An issue with my SAGA installation made me think that something was wrong with my implementation and made me lose some time.
  • The issue with the SAGA installation also prevented me from testing the algorithms.
Learn More

Fifth Report

What do I have completed this week?

  • Solved the issue with the outputs.
  • Did some tests with QGIS algorithms which seems to be working fine.
  • Refactored all QGIS algorithms in order to remove the progress bar parameter and use the signal instead.
  • I have changed the Buffer used by QGIS algorithms: the Buffer is now a subclass of QObject. This new implementation allows the buffer to have a parent (the algorithm) and get access to the algorithm’s signal.
  • SagaUtils refactored in the same way I did with the Buffer.
  • SAGA algorithms refactored in order to use the signals instead of the progress parameter (Not finished yet)

What am I going to achieve for the next week?

  • Solve the issue with SAGA algorithms.
  • Code refactoring in order to improve the previous implementation on the AlgorithmDialog.
  • Refactor Grass/Grass7.
  • Bug fixes.

Is there any blocking issue?

There is an issue with SAGA algorithms that makes the Toolbar crash.

Learn More

Crayfish 2.1: New features

New features keep being added to Crayfish. Now it is possible to export time variable grid as animation, add AnuGA results and visualise vectors on user-defined grids.

Export to animation

The ground works were done in Crayfish 2.0 for this feature. You can now generate animation from contours and vectors and export them as AVI. There are two methods of exporting to animation: basic and using QGIS print template (qpt).

With the basic option, you can define a title, a legend and a progress clock. Alternatively, for a smarter solution, you can set up a print composer with the benefit of all its rich features. The composer template (qpt) can be used as a frame layout for exporting your animation.

Below is an example of a multi-frame print composer template used to generate animation from a Crayfish layer.

AnuGA support

Crayfish 2.1 now supports SWW file format generated by AnuGA</li>.

SWW files generated by AnuGA in Crayfish (Click to enlarge)

Vector on user-defined grid

With this option, users can define a grid and Crayfish will interpolate values and displays results on the custom grid. Images below show vectors on the outputted mesh and user-defined grid.

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on a user-defined mesh in Crayfish (Click to enlarge)

Crayfish manual

With the ever-growing features in Crayfish, we have decided to dedicate a page on how to use Crayfish in QGIS. From the manual page, users can download a sample data and try the Crayfish features in QGIS.

Sponsors

We’d like to thank Maroondah City Council for sponsoring some of the great features in this release.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

How to use Function Editor in QGIS Field calculator

In QGIS 2.8, there is a new option for users to add their own python function in the Field calculator. This is an extremely useful feature enabling users to populate data within the attribute table using customised python function.

Nathan wrote a blog post about the feature and how to write a python with arguments. But in QGIS 2.8, the function editor does not correctly support functions without arguments.

In the example below, we are going to calculate proportion of area for each SAC (Special Areas of Conservation) in Great Britain to the total area of the layer.

Add GB_SAC_20130918_Dissolved.shp layer to QGIS, right-click on the layer and open attribute table. Make the layer editable and click on the Field calculator. We are now going to create a new column (propArea) and populate proportionate area of each SAC to the total area of the layer.

Under Function Editor tab, click on New file and type area for the name and save the file. For the content, copy and paste the following lines:

"""
A custom function to calculate total area of the GIS layer.
This function has no arguments.
"""

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

@qgsfunction(args='auto', group='Custom')
def total_area(x, feature, parent):
	return sum( f.geometry().area() for f in iface.activeLayer().getFeatures() )

Click on Run Script to add total_area function to your Custom function list.

Now, click on Expression tab and type:

$area / total_area(0)

As you can see, we have passed 0 as an argument. If you click OK, your QGIS will freeze! As there are many features in the layer, the expression, calculates total area for each row.

Lets make the script a bit more elegant. Firstly, we need to add caching, so that area will be calculated only once and cached for the rest of operation. Secondly, we can make the script a bit more generic, so that we can use it to get the area of other loaded layers in QGIS:

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

cache = {}
def _layer_area(layer):
	""" Internal method """
	if layer.name() not in cache:
		area = sum( f.geometry().area() for f in layer.getFeatures() )
		cache[layer.name()] = area
	return cache[layer.name()]

@qgsfunction(args='auto', group='Custom')
def total_area(layer_name, feature, parent):
	for layer in iface.mapCanvas().layers():
		if layer.name() == layer_name:
			return _layer_area(layer)
	return 0

Now, click on Expression tab and type:

$area / total_area('GB_SAC_20130918_Dissolved')

This time, it should be quicker!

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

ArcGIS REST API Connector Plugin for QGIS

ArcGIS REST Connector Plugin

Last year we described a command line method that adds ESRI REST layers in QGIS. Well, a team at the Geometa Lab in the University of Applied Sciences Rapperswil (HSR) Switzerland, have released a plugin for QGIS that adds ESRI REST layers via a GUI (Github page). The plugin is experimental so you will need to tick the box “Show also experimental plugins” in the settings panel of the “Plugins – Manage and Install Plugins” dialogue in order to add the plugin to QGIS. The following URLs lists numerous REST layers in the plugin’s GUI:

http://services.arcgisonline.com/arcgis/rest/services

http://basemap.nationalmap.gov/arcgis/rest/services

http://services.nationalmap.gov/arcgis/rest/services

Reference:

REST API Connector Plug-in Wiki Page

Learn More

Ti piace avere QGIS ben tradotto in italiano? Ora puoi contribuire

Avere tutto QGIS, incluso il programma, i manuali e il sito web, tradotti in italiano è una bella comodità; questo richiede uno sforzo notevole, per cui il tuo aiuto è essenziale. Fai una donazione tramite: http://qgis.it/#translation
Learn More

Crayfish 2.0: What's new!

After listening to user feedback we decided to do some major work on Crayfish. The changes include code refactoring, changes to the user interface, support for an additional file format, adding a vector and contour overlay, and a shiny new logo!

Crayfish 2.0

Time control

In the new version of Crayfish a time slider allows users to quickly browse through time. A drop-down menu allows the selection of an exact time.

Vector and contour overlay

In previous versions of Crayfish it was only possible to load contours and vectors from the same dataset. For example, it was not possible to show velocity vectors on top of depth contours. With the new Crayfish you can "unlock" the legend and choose different vectors or contours to be displayed. The video below demonstrates this in action.

Special times

Some datasets contain a special time-step outside the outputted time range. For example, Maximums and Minimums are stored at time 99999 in TUFLOW modelling package. Within the layer tree, additional time-steps items will now be shown if they exist within the dataset.

Additional file formats

We have added support for the XMDF file format. In addition, Hydro_AS 2D users should be able to open their files in the latest Crayfish.

New Python Module

We've refactored lots of code in the Crayfish library which makes it much easier to add support for further file formats and additional functionality. The Crayfish library now comes with a new Python module that allows easy manipulation with the mesh and results data - either in your custom scripts or within the QGIS Python console. For example, printing the coordinates of the nodes of a mesh together with their elevation takes just few lines of code:

import crayfish

m = crayfish.Mesh("/data/my_mesh.2dm")
o = m.dataset(0).output(0) # bed elevation data

for index, node in enumerate(m.nodes()):
print "Node XYZ: ", node.x, node.y, o.value(index)

New Profile tool plugin

If you use Profile tool plugin in QGIS, you can create a profile from the Crayfish layer and browse through the time. The profile gets updated as you change the output time.

Problems

If you have some feedback on our changes, suggestions for new functionality, or come across a bug, feel free to file a ticket on the issues page of the Crayfish github repository.

Sponsors

We’d like to thank Maroondah City Council for sponsoring some of the great features in this release.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Create great looking hillshaded maps in QGIS

Wicklow-Topo-original

In this tutorial I will show you how to create a Hillshaded topographic map in QGIS. We will be using Shuttle Radar Topography Mission (SRTM) data, a near global Digital Elevation Model (DEM) collected in February 2000 aboard NASA’s Space Shuttle Endeavour (mission STS-99). The mission used a X-Band mapping radar to measure the Earth’s topography, built in collaboration with the U.S. Jet Propulsion Laboratory, the U.S. National Imagery and Mapping Agency (now the National Geospatial-Intelligence Agency), and the German and Italian space agencies.

The raw radar data has been continuously processed and improved since it was first collected. Countless artefacts have been painstakingly removed and areas of missing data have been filled using alternate data sources. The version we will be using is the 1 Arc-Second Global SRTM dataset, an enhanced 30 meter resolution DEM that was released last year. It is a substantial improvement over the 3 Arc-Second / 90 meter SRTM data previously available for Ireland. SRTM elevation data can be downloaded from the United States Geological Survey’s EarthExplorer website.

When first loaded into QGIS (via Add Raster Layer), the DEM is displayed as a rather uninformative black and white image.

Wicklow-Topo-blackwhite

It is therefore necessary to apply a suitable colour ramp that accentuates topography. While it is possible to create your own colour ramp, or use one of the colour ramps provided by QGIS, superior colour ramps can be downloaded using Etienne Tourigny’s Color Ramp Manager (Plugins – Manage and Install Plugins). After the plugin is added to QGIS, go to the Plugins menu again and choose the Colour Ramp Manager.

In the window that pops up, choose the full opt-city package and click check for update. The plugin will then download the cpt-city library, a collection of over a hundred cartographic gradients (version 2.15). After the package downloads, quit the dialogue.

Back in QGIS, right click the DEM layer to bring up the Layer Properties dialogue. In the Style tab, change the render type from single band grey to single band pseudocolor. Then click new color ramp and new color ramp again, choose the cpt-city color ramp to bring up the cpt-city dialogue. Click topography and choose the sd-a colour ramp. While this is an excellent colour ramp, I find its colours are a bit too strong for my liking.

Still in the Layer Properties dialogue, change the min and max values to match your DEM’s lowest and highest elevations values and click classify, this applies the new colour ramp. Next, change the brightness to 30 and lower the contrast and saturation to -20. Click OK to apply the new style and quit the Layer Properties dialogue.

Wicklow-Topo-noShade

Next we need to create a Hillshade layer from the DEM, a 3D like visual representation of topographic relief. This is achieved via the menu Raster – Analysis – DEM (Terrain models). There is one small catch, the hillshading algorithm assumes the DEM’s horizontal units are in meters (they are decimal degrees). We need to enter a scale correction factor of 111120 (in the Scale ratio vert. units to horiz. box). Once that is all done, select an output path to save the generated hillshade and click OK. Generating a hillshade may take up to a minute depending on the size of your DEM.

Wicklow-Topo-hillshade

After the hillshade is created, open its Layer properties dialogue. Change the min and max values to 125 and 255, increase its brightness to 45 and contrast to 20. Finally, switch the blending mode from normal to multiply. This allows the DEM beneath the hillshade to show though. Click OK to apply the new style.

If you followed these steps correctly you will have created a fine looking topographic map similar to the one below. It’s also possible to create contours but that’s a tutorial for another day.

Wicklow-Topo

Technical note:

There are two hillshading algorithms available in QGIS, one by Horne (1981) and another by Zevenbergen and Thorne (1987). Jones (1998) examined the quality of hillshading algorithms, he found the algorithm of Fleming and Ho€er (1979) is slightly superior to Horne’s (1981) algorithm. Zevenbergen and Thorne’s (1987) algorithm is a derivation of Fleming and Ho€er’s (1979) formula. QGIS uses Horne’s (1981) algorithm by default.

References:

Horn, B.K., 1981. Hill shading and the reflectance map. Proceedings of the IEEE, 69, 14–47.

Jones, K.H., 1998. A comparison of algorithms used to compute hill slope as a property of the DEM [PDF]. Computers & Geosciences, 24, 315–323.

Zevenbergen, L.W. & Thorne, C.R., 1987. Quantitative analysis of land surface topography. Earth surface processes and landforms, 12, 47–56.
Learn More