OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Capturing geotagged photos with Input app

In this post, we will explore the new feature in the Input app which allows you to record and display photo’s geotag information (e.g. direction, longitude and latitude).

Photos’ metadata

When users take photos in the field, they often would like to record extra information which can help convey a better understanding and awareness of the surroundings. Recent mobile devices allow users to record photo direction, location, time stamp, camera settings, etc when taking photos. This information is often optional or restricted by default (due to privacy) within the mobile settings.

For surveying and data collection, this information can add extra dimension and context. For example, if you are surveying a bat nesting, it is useful to know the directions of the photos you have taken. This will help identifying the site easily in the subsequent site visit.

Geotag information is also useful metadata to have attached to your photos. There are other GIS or non-GIS applications which can read and interoperate the information.

In the recent version of the Input app, we have added a feature which allows you record and display the geotag information. Combined with QGIS styling, you can create a very informative symbology to display the information while you are capturing it in the field.

Before you start

To be able to capture geotag information, you need to enable your Camera app to use location. For that:

  • in Android, from Settings > Apps and notifications > (see all) Camera and under Permission ensure you allow Location

  • in iOS, from Settings > Privacy > Location Services > Camera and the Precise Location is enabled.

Quick start

If you want to record photos with directions as shown below, you can follow the following steps:

  • Log into Mergin (or sign up if you have not yet registered)
  • Brows to photo survey project
  • On the top right-click on Clone to make a copy of your project in your account. You can choose a different project name:

  • Download and install the Input app

  • Select My projects and log in using your Mergin credentials

  • Download the newly created project from the earlier step

  • Start recording points and add photos, you should see photo direction

More details

The project configuration is done in QGIS. The form has been set up to allow you take photos using Attachment widget. To learn more about different form edit tools in Input app and how to set the up in QGIS, see the Input app help pages.

There are multiple fields to extract geotag information from the photo, as soon as you add the photo either using the camera or the gallery:

  • Direction: read_exif_img_direction(@project_home + '/' + "photo")
  • Latitude: read_exif_latitude(@project_home + '/' + "photo")
  • Longitude: read_exif_longitude(@project_home + '/' + "photo")

As noted, all the above functions take the path to the photo (@project_home + '/' + "photo") and return different metadata related to the image.

In addition to the form, the layer styling has been modified to resemble the direction of the camera and field of view.

To set the camera icon and direction of the camera:

  • For the point styling, select SVG Marker
  • From the list of SVGs, select the camera icon. (ensure to embed the SVG to the project, otherwise it will appear as a question mark in the Input app.)
  • For the Rotation, select direction field

To add field of view:

  • Add another symbol layer to your point
  • For Symbol layer type select Geometry Generator
  • For Geometry type select Polygon / MultiPolygon
  • For the expression, type: wedge_buffer(center:=$geometry,azimuth:= "direction",width:=45,outer_radius:=0.0008)
  • For Symbol layer type, select Shapeburst Fill

Feedback and suggestions

Input, a field data collection app based on QGIS. Input makes field work easy with its simple interface and cloud-based sync. Available on Android and iOS.

Screenshots of the Input App for Field Data Collection

Get it on Google PlayGet it on Apple store

If you would like to add a new feature or suggestions to improve the app, do not hesitate to contact us on info@lutraconsulting.co.uk

Learn More

Tips & tricks for point clouds in QGIS

Just few weeks ago, QGIS 3.18 has been released - the first version to include support for point cloud data, thanks to the great support from the QGIS community in our joint crowdfunding campaign with North Road and Hobu.

We have received a lot of feedback from users and we would like to summarise the most common problems people have faced, and add a couple of tips on useful features that are otherwise easy to overlook.

1. I am unable to load LAS/LAZ files

If your QGIS installation does not recognize .las or .laz files (which are the two point cloud formats that we currently support in QGIS), the most likely source of the problem will be that the PDAL library needed for reading LAS/LAZ is missing in your installation. Unfortunately not all installers include it at this point. Particularly on Windows, there are several options how to install QGIS, and only one choice is the right one for PDAL support. At the time of writing (April 2021), you need to download installer from section called “Standalone installers from OSGeo4W testing packages (MSI)”, here’s a screenshot of what to look for:

On macOS the official all-in-one installers include point cloud support. On Linux, PDAL support depends on the particular distribution/packages, but probably most of them include point PDAL library.

2. Point cloud is completely flat in 3D

It may happen that you load a point cloud layer, then open the 3D map view and the point is displayed as a flat surface like on this screenshot:

The reason for this is that the 3D renderer is not enabled for your point cloud layer. To enable 3D renderer, open the Layer Styling panel (there’s F7 shortcut to open it!), then switch to the second tab (“3D View” - with a cube icon) and change the “No Rendering” option to some other option - for example “Classification” in case your point cloud is classified. You should then see your point cloud in 3D.

3. Point cloud is rendered twice - in 3D and in 2D (“flat”)

Commonly when people open 3D view with a point cloud, they may see the point cloud rendered twice, like in the following screenshot:

The reason is that both 3D rendering and 2D rendering of point cloud is enabled, and therefore the layer is also rendered as a 2D map texture on top of terrain (which is flat by default). An easy way how to fix this is to set 2D rendering of the point cloud layer to “Extent Only” in Layer Styling panel (in the first tab):

If the dashed rectangle marking the extent is still bothering you, it is possible to change the line symbol to use white (or transparent) colour.

Hopefully in near future we would address unexpected behaviour and layers with a 3D renderer defined would not be rendered as 2D.

4. I still can’t see my point cloud in 3D view

It could happen that if your point cloud is for a small area, yet the elevation of points is relatively high: when you first open 3D view or when you click “Zoom Full” button, the view may get zoomed too close and the actual point cloud data may be behind the camera. Try zooming out a bit to see if it helps. (This is a bug in QGIS - at this point “zoom full” ignores extra entities and only takes into account terrain.)

5. Enable “Eye Dome Lighting” in 3D view

For a much better 3D perception of your point cloud, try clicking the “Options” button (with a wrench icon) in the toolbar of 3D view and enable “Show Eye Dome Lighting” in the pop-up menu. This will apply extra post-processing that adds slight shading based on the positions of nearby points, and adds silhouettes when there is a sudden change in depth:

As soon as you zoom into your point cloud more, to the point when individual points can be seen, the eye dome lighting effect will start to disappear. You can try experimenting with the point size (in Layer Panel, 3D View tab) - increasing the point size will help.

- A view with point size 2.0 pixels

- The same view with point size 6.0 pixels

6. Try the new camera navigation mode in 3D view

In QGIS 3.18 we have added a new “Walk Mode” camera navigation mode that is much better suited for inspection of point clouds (compared to the default “Terrain Based” camera navigation mode). Open 3D map view configuration dialogue, pick “Camera & Skybox” tab and set it here:

Control                                          Action                                         
Mouse move Rotate camera
Mouse wheel Change movement speed
W / Up Move forward
S / Down Move backward
A / Left Move left
D / right Move right
Q / Page up Move up
E / Page dn Move down

7. Use elevation scaling and offset to your advantage

Sometimes it is useful to modify offset and/or scale of the elevation of points (Z values). For example, if the point elevations do not match your other 3D data, or maybe you have source data where X,Y coordinates are in meters and Z coordinate is in feet!

Another case when this can be useful, is when your point cloud data are further away from the terrain and the default “Terrain Based” navigation mode does not work nicely - it expects that data are near the terrain, and the camera rotates around a point terrain, which may feel strange when browsing point clouds. A workaround is to apply offset to the point cloud layer to move the points near the terrain. For example, this is a point cloud which is roughly at 200 meters elevation (the grey plane is the terrain):

When an offset of -200 is applied to the point cloud in Layer Styling panel, data show up much closer to the terrain and camera navigation feels more natural:

8. Try circular points in 2D maps

By default QGIS draws points as squares as this is the fastest option. But for higher quality output you may want to change point symbol style to circle in Layer Styling panel, which makes things look a little less jagged:

– using squares – using circles

At this point we always use square points in 3D views - in the future we will likely offer circular points in 3D views as well.

9. Give us your feedback

We would love to hear from you about your experience with point clouds in QGIS so far, what features you are missing or what problems you have encountered - feel free to drop us a mail at info@lutraconsulting.co.uk. If you think you have found a bug, best to file an issue for it in the QGIS GitHub repository.

With the QGIS 3.18 release, we know we are only at the beginning of a long journey to provide great point cloud support to the users. The decreasing cost of laser scanning (LIDAR) hardware and increasing availability of photogrammetric methods means that we will be be seeing point cloud data in GIS more and more often. There is still a lot of functionality missing for efficient work with raw point clouds, for easy processing of data and we are still using only a fraction of what the PDAL point cloud library offers. We are dedicated to provide first class support for point clouds in QGIS - however this cannot be done without funding. Therefore, if you would like to help us to push point clouds in QGIS to the next level, please do not hesitate to contact us at info@lutraconsulting.co.uk.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Publishing high quality Swisstopo WMTS with QGIS Cloud

The Swisstopo services can be used free of charge for commercial purposes since March 1st. In this blog article I want to show how these high quality data can be published on qgiscloud.com. Swisstopo-Services The Swisstopo WMTS services are tiled raster data. Due to the prior tiling, the loading times are significantly reduced. WMTS data are used when an up-to-date background is always required. For publication as a background layer on the QGIS Cloud Platform, the National Maps and the Swissimage orthophoto are very interesting.
Learn More

Support for Point Clouds Released in QGIS 3.18

QGIS 3.18 is finally here. This will be the first release of QGIS with native support for point cloud data. This work was made possible by generous contributions from the community.

Point cloud data in QGIS 3D (Data from UGKK SR, made by Tibor Lieskovsky)

Contributors

When we announced the campaign in August 2020, the response was overwhelming and within weeks, we managed to reach and then exceed the amount required.

Below is the list of contributors in no particular order:

Mapfly, Ujaval Gandhi from Spatial Thoughts, National Land Survey of Finland, Daniel Löwenborg, BNHR, Imapct GIS, Andreas Neumann, Kanton Solothurn, Switzerland, City of Vevey, Mapping Automation, Service de la géomatique du canton de Neuchâtel, Hans van der Kwast, 3DGeoCloud, Rudaz+Partner AG, Leonard Gouzin, TileDB, Inc.

We wish to also thank the many anonymous contributors who do not appear in this list.

Note: if you have contributed to the campaign but your name does not appear here, you may not have selected the option to be listed in the campaign form. Please contact us if you’d like to be listed.

In addition to financial contributions, we’d like to extend our gratitude to all those who helped spread the word, helped with the testing and provided feedback and sample data.

New features

Lutra Consulting North Road Hobu

This is a brief summary of new features from our joint work with North Road and Hobu:

  • Introduction of a new map layer type: a point cloud layer
  • Load LAS or LAZ files (either by drag’n’drop or by opening files in Data Sources Manager)
  • Load EPT datasets by pointing QGIS to their ept.json file (currently only supporting datasets on the local drive)
  • Support for rendering point cloud data in 2D and 3D map views
  • Apply various rendering styles in 2D and 3D:
    • “Attribute by Ramp” - draw data based on a single attribute and a color ramp (similar to “Graduated” styling for vector layers)
    • “RGB” - draw data using colors assigned to the points (combining red/green/blue attributes)
    • “Classification” - draw data using different colors for different classes (ground, buildings, vegetation, …), also allowing display of only desired classes
    • “Extent only” (2D only) - draw only bounding box of the point cloud
    • “Single color” (3D only) - draw all points with a single color
  • Set size and shape of points
  • Manually adjust scaling and offset of elevation (Z values) - if needed to match with elevation of other data
  • Point cloud layer properties dialog to see metadata of the point clouds
  • Identify tool supports point cloud layers and shows all attributes of picked points
  • 2D and 3D views only render a subset of the point cloud for the best performance for the given view (for geeks - this is thanks to indexing to octree data structure, using EPT format written by untwine tool packaged with QGIS)
  • Optimize the quality and performance of the 3D view using point budget configuration, which limits maximum amount of point rendered at any time (set to 1 million by default)
  • Enable eye-dome lighting in 3D views for much better depth perception of point clouds
  • New “Walk mode” camera navigation in 3D views - there is now a switch between the original “Terrain based” navigation mode and the new mode, which allow easier navigation through point cloud data

Please note that as this is the initial release (with over 10 thousand lines of new code related to point clouds), there may be still some rough edges here and there, or some data may not load or display correctly. In case you encounter any issues with the new functionality, please let us know - do not hesitate to create a new QGIS issue

Point cloud in action (Data from Helsinki City)

Future work

This has been the start of a larger effort to bring full support for point cloud data into QGIS. We, in collaboration with North Road and Hobu are developing requirements for integrating point cloud data processing and analysis, more data formats, better visualisation, profile tools etc. in future releases of QGIS.

If you’re interested in helping shape those requirements or funding such features, please contact us at info@lutraconsulting.co.uk.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Test QGIS with Point Clouds on Windows

In the past months, we have been busy adding support for point cloud data in QGIS (3.18). Read more on how to install and test this feature under MS Windows.

Installation

To be able to use this feature, you need extra packages and also the latest OSGeo4W installer.

Update

Since the original post, Jürgen Fischer has created a stand-alone MSI installer. You can download the new installer from the QGIS website. The installer is only for 64-bit platforms and does not support MS Windows 7.

Note that there have been several regressions and bugs with the first release of QGIS 3.18.0. The issues are being addressed and soon there will be an updated version available. The above link is only for those who are eager to test the point cloud data in QGIS.

Installation

To be able to use this feature, you need extra packages and also the latest OSGeo4W installer.

Note: This is a completely revamped and different packaging system than the current OSGeo4W installer. To avoid any clash with your current installation, it is recommended to use different paths for temporary download files and installation of the new packages. The new packaging only supports 64-bit platform.

1- Download and run the NEW OSGeo4W installer

2- Select the Advanced install and pick qgis-dev from the list of packages

Special thanks to Jürgen Fischer for his hard work on preparing the new packages for Windows.

Testing

Once installation is completed, try to run QGIS from the installation path (e.g. C:\OSGeo4W64\bin\qgis-dev.bat). You should be able to load LAS\LAZ file to your map from the Browser panel or the Data Source Manager.

The point cloud data can be visualised in 2D and 3D map canvas.

Point Clouds in QGIS under Windows

Please test and let us know if you encounter any problems when loading, viewing or styling point cloud data. The best way to do that is to create a new issue on GitHub: https://github.com/qgis/QGIS/issues

QGIS 3.18 will be released later this week (February 19), so grab your copy of QGIS today and give it a try, so that we can fix any remaining issues before the release!

Troubleshooting

Problem: I am unable to add any LAS/LAZ point cloud file

Solution: Ensure you have used the correct installer linked above. Development builds of QGIS in the ordinary OSGeo4W installer DO NOT include support for point clouds.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Case Study: FLO-2D QGIS Plugin

The case study presents the implementation of the QGIS FLO-2D Plugin project (5 minute read)

Introduction

FLO-2D is one of the most widely used commercially available flood models. FLO-2D is capable of simulating urban flooding in high resolution including storm drain systems.

FLO-2D

In 2016 the FLO-2D team invited us to develop a set of tools for optimising the flood model build process in QGIS. The resulting plugin allows hydraulic models to be built quickly by leveraging the wide range of tools available in the QGIS ecosystem.

The plugin can be downloaded by following the guidance in the plugin documentation

As the use of open source GIS grows within the water engineering sector, Lutra Consulting develops and maintains MDAL which makes the visualisation and post-processing of time-varying numerical model results possible in QGIS.

Contact us at info@lutraconsulting.co.uk if you’d like to discuss the benefits of integrating your flood modelling software more tightly with QGIS.

QGIS FLO-2D Plugin

Overview of Previous GUI tools: FLO-2D GDS

FLO-2D’s focus on urban modeling requires large datasets that may include several million grid cells. Each cell has between 4 and 10 attributes so datasets can often be in the range of several gigabytes.

Previously, Grid Developer System (GDS) was used to pre-process the spatial data used by FLO-2D. Being a 32-bit application, the GDS was only able to load up to 4 gigabytes of data and its programming framework was also no longer maintained by Microsoft. Modellers could only be built on Windows PCs.

All development and maintenance of the GDS was carried out by the FLO-2D team.

Behind the Implementation

Guiding principles behind the new QGIS Flo-2D plugin

We assessed the tools used at the time and in close collaboration with the FLO-2D team, optimised the workflow from the users’ point of view with emphasis on speed and simplicity.

To keep things simple for the users, we aimed to provide a level of abstraction so users would no longer need to be involved with the internal structure of FLO-2D solver input files. They could instead focus on real-world aspects affecting their models.

Together with the FLO-2D team we designed a solution that was based on 3 core ideas:

  • Modellers will use native QGIS tools and point/line/polygon layers to define real-world objects (e.g. domain, boundary conditions, levees, …)
  • These real-world objects will be converted automatically into the data structures required by the FLO-2D solver (although expert users can still modify these data structures if they so wish).
  • Additional productivity tools will be provided to allow users to speed-up time consuming tasks

Features of the plugin

Some of the key features of the solution are listed here:

  • Easy creation and handling of new models (all model data sits in a single GeoPackage file)
  • User friendly and intuitive digitizing and manipulation of model components (through simple GIS layers and dedicated tools) including:
    • 1D domain
    • Boundary and initial conditions
    • 1D channels and cross-sections
    • Levees
    • Rainfall
    • Infiltration areas
    • Storm drains
  • Various options for obtaining grid data such as elevation, roughness, reduction factors etc. from different sources (raster layers, external vector layers etc.)
  • Plotting profiles and editing time series data
  • Automated tools for schematizing input layers into the GDS format
  • Importing basic data from HEC-RAS models
  • Import / export functionality between GeoPackage and GDS format (*.DAT files)
  • Running external FLO-2D tools (FLO-2D Pro engine etc.) directly from QGIS

QGIS FLO-2D Plugin

Benefits for the FLO-2D developers:

Benefits for the developers of FLO-2D include:

  • Reduced development and maintenance costs, since much of the heavy lifting of the FLO-2D plugin is done by QGIS itself
  • By being part of the QGIS ecosystem, gaining opportunities to approach QGIS users in the flood risk industry to use FLO-2D software
  • The FLO-2D plugin is developed on GitHub, allowing the latest development technologies such as continuous integration, automatic testing and issue tracking to be used
  • Ability to solve upstream bugs in QGIS or MDAL due to the open-source nature of the projects
QGIS3

Benefits for FLO-2D users:

Some of the benefits realised by FLO-2D customers include:

  • Being able to work with their FLO-2D models using open source GIS on all major operating systems
  • A full GIS application to support their data pre-processing
  • Logical and intuitive workflows
  • FLO-2D results can now be visualised and post-processed natively in QGIS via mesh layer
  • Ability to use all native QGIS support and development channels in addition to FLO-2D support
  • Integration of internal workflows with powerful native QGIS features including projection support, GDAL/OGR integrations, background maps support (e.g. vector tiles), printed flood maps, etc.
  • Ability analyze results via QGIS’ Crayfish plugin and produce graphs and outputs

The plugin in action!

Further Reading

Do you have any questions or would like to see demo of QGIS Mesh Layer? Contact us at info@lutraconsulting.co.uk or schedule a demo call calendly.com/saber-razmjooei/15min

Key words

QGIS, plugin, python, migration, optimised, speed up, fast, hydraulic modelling, water, 2D, open-source, cost reduction, software development

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Quickly Get Site Photos Into Word

Preparing reports with lots of survey photos takes time - this plugin automates the process.

Nowadays, it’s really easy to take georeferenced photos on site visits. Tools like Input can capture photos, descriptions and location information and bringing it all into GIS is straight-forward. However.. I recently discovered that people still spend significant amounts of time organising their photos into reports outside of GIS. This small plugin automates the process of getting the photos and their descriptions into Microsoft Word.

Using Input to Survey My Garden

For this blog post I knocked-up a simple survey project based on the Field notes Mergin template. Points can have a photo, title and description amongst other fields. No comments on my gardening skills please!

When I sync the data back to QGIS, the attribute table looks something like this:

The Input Survey Layer's Table of Attributes

We’ll now use the HTML Table Exporter plugin to export the layer as an HTML table (this will let us get it into Word).

First install the plugin.

It’s an experimental plugin so you’ll first need to check the Show also experimental plugins option under Plugins > Manage and Install Plugins… > Settings.

When you have installed the plugin, open it using this button HTML Table Exporter Icon or via Plugins > HTML Table Exporter > Export table as HTML.

You should now see this:

HTML Table Exporter Dialog

Set Table to the layer you want to export. The image scaling option is described later, leave it at a small setting for the time being. Click OK to export and tell the plugin where to save the HTML file.

Now the data should be out of QGIS.. the next steps are in.. MS Word :o

Right click on the exported html file and open it with Microsoft Word:

Open Exported HTML With Word

Let’s see what it looks like:

First Export in Word

OK.. we’re getting somewhere! At this point you’ll want to:

  • Enable Print Layout mode in Word so you can see what the printed page will look like
  • Set the page orientation as desired (I chose landscape in the image above)
  • Experiment with image rotation if required (Right click photo > Size and position > Rotation)

If playing with rotation, just focus on rotating a single image, we’ll batch rotate later as required. For now you want to get a feel of whether you want to adjust the scale factor in QGIS (to make the images smaller or larger) to save you having to resize them individually in Word.

I’ve decided to increase the scale factor from 10% to 15% so will now re-export.

Beware that Word has an exclusive lock on the html file when it’s open so you need to close it in Word before you can export it again from QGIS.

I settled for 10% in the end so I could get multiple images on each page in portrait mode. After removing the columns I didn’t want, the table in Word now looks like this:

A Better Export to Word

To rapidly rotate images by 90 degrees, rotate the first one using Right click on photo > Size and position > Rotation then select subsequent photos and press the F4 key. This method is described in more detail here as well as other Word batch image rotation methods.

My document is almost finished. There are just a few small issues to iron out. Currently, the images are referenced by the Word document, not embedded. This means if I email the document to someone, the images will be missing. Let’s fix that by embedding the images in the word document.

First save the document as a Word document in Word’s native format (e.g. *.docx).

Next, locate the Edit Links to Files option:

Edit Links to Documents

Select all the linked images (the shift and arrow keys help here) and check the Save picture in document option and click OK:

Save Picture In Document

Save the document, your photos should now be embedded within the document.

If you find your word document gets huge, you can use the method here to quickly batch compress all images in the document.

Input is a free and open source field data collection and mobile GIS app based on QGIS.

Get it on Google PlayGet it on Apple store

If this guide saved you some time and you feel like doing something awesome for us, a review of Input on the Apple App Store or Google Play Store would be really well appreciated.

Learn More

Overview of QGIS 3.16 LTR and QGIS 3.18 Mesh Features

QGIS Mesh Layer now support DHI dfsu and HECRAS 6.0 format.

The releases of QGIS 3.16 LTR/QGIS 3.18, MDAL 0.8.0 and Crayfish 3.5.0 are planned for 19 February 2021. We are delighted to present the following improvements for the upcoming releases:

  • In-memory mesh datasets with persistence
  • Multi identify tool for mesh
  • Virtual dataset groups for mesh layer
  • 3D Mesh Layer Bug Fixes and speed optimizations
  • DHI’s dfsu format support (QGIS 3.18 only)
  • HECRAS 6.0 format support

If you’d like try the latest features, you can always install QGIS nightlies/master, which comes with all the latest features described in this blog post.

If you want to learn more about Mesh Layer in QGIS, read more here…

Mesh Calculator and Layer Improvements

The Virtual dataset groups for mesh layer and In-memory mesh datasets with persistence improvements greatly improves the workflows when using the Mesh Calculator in QGIS. Users can store the intermediate results into virtual layers that are recalculated on the fly (similarly to QGIS expressions for vector layers). The layers can be later persisted to any supported MDAL formats with write capabilities.

Multi identify tool for mesh feature allows to browse the temporal mesh data in more intuitive way and includes the extra information about the Mesh datasets loaded.

These features were sponsored by Artelia Group.

DHI’s dfsu format support

MDAL 0.8.0 supports loading of the external drivers. A first driver, available on Windows QGIS 3.18 only, is popular DFSU format by DHI, which is used to store MIKE 21 output results.

You can see how to configure and use QGIS to work with DFSU format on the DHI’s YouTube channel

Special thanks to the sponsor DHI this feature.

QGIS

We have added the following new features to QGIS to convert between mesh and vector/raster:

  • TIN Mesh creation
  • Ported most of the processing algorithms from Crayfish to QGIS core
  • 3D rendering improvements
  • Many Mesh Layer bugfixes

MDAL

  • Support for external drivers.
  • Fixed HECRAS vector datasets support
  • Packaging in conda

Crayfish

  • Fixed SAGA flow direction support
  • Fixed FFMPEG download link
  • Ported most of the processing algorithms from Crayfish to QGIS core

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Mergin and PostGIS in action

One of the challenges of data collection projects is consolidating all the data a central database, such as Postgres/PostGIS. Using PostGIS as a live survey layer is not usually recommended:

  • Security: exposing your database to the wider internet
  • Access and connectivity: internet connection is not always guaranteed for field survey projects

A more cumbersome way around this, is to convert your tables from PostGIS to a file based GIS layer (e.g. GeoPackage) and take the files with you to the field. This will create a new problem: keeping all the tables (from multiple surveyors) and the PostGIS table up-to-date.

During a survey to assess water access for villages in Limpopo province, South Africa, our friends at Kartoza have commissioned us to extend the Mergin service to support PostGIS. The Mergin service already supports file-based data synchronisation. The aim was to bridge the gap between the Mergin service and PostGIS so that the changes from Mergin immediately appear on PostGIS and vice versa.

To facilitate that, we further developed the Geodiff library to support Postgres driver. In addition, we developed mergin-db-sync tool to sync the tables from Postgres database with the Mergin service. The mergin-db-sync tool runs as a service (daemon) that keeps an eye on a particular Mergin project, and if there is a new version of the project, it will fetch the most recent changes and apply them to database tables in PostgreSQL. It works also the other way around at the same time: it looks for changes in the configured PostgreSQL schema and upload them in a new version of the Mergin project if any changes were detected. This service can be easily started on the local Intranet (where the PostgreSQL database is run) and therefore it does not need any adjustments to the firewall to allow access to the local database from public Internet.

DB-Sync schema

The above diagram details how Postgres/PostGIS synchronisation works with the Mergin service via the DB-Sync tool.

  • Tables 1 and 2 from the Postgres/PostGIS server are set up to work with the Mergin service
  • DB-Sync tool runs on a server on a regular interval to sync with the Mergin service
  • An offline version of Tables 1 and 2 are provided within the QGIS survey project on Mergin
  • Several surveyors download the project and add their data mostly while offline. The data are then synced with the Mergin.

From the surveyors’ point of view, the extra set up to sync with the Postgres/PostGIS does not affect their workflow. In fact, mergin-db-sync tool acts as another client syncing data to the Mergin project, therefore it is possible to see all the changes in the project log originated from mergin-db-sync tool.

The tool is available on GitHub with a permissive open source license (MIT). At this point it supports PostgreSQL, but the mechanism is fairly generic and support for other database engines may be added in the future without great effort. All the heavy lifting is done by the Geodiff library which has been significantly enhanced during the development of mergin-db-sync tool.

To try the tool, please follow the instructions on the project’s readme on GitHub. The easiest way is to use it in a Docker container.

If you have any issues or feedback to enhance the tool, you can file a ticket on the project repo.

If you’d like to set up DB-Sync tool with your Mergin survey projects, you can contact us on info@lutraconsulting.co.uk

Learn More

New version of Serval QGIS plugin

Raster selection tools and editing with QGIS expressions.

What’s new?

In the previous version Serval allowed for setting a constant value to a single raster cell at a time. The latest version brings in new raster cells selection tools and new ways of manipulating cells value:

  • QGIS expressions and
  • 3x3 low-pass filter.

Multi-bands rasters are fully supported - each band can be edited separately. Serval is meant to provide a convenient way to modify selected parts of a raster layer. It is not optimized to process entire images - use Raster Calculator for that.

Raster selection tools

In order to modify raster cells values users need to select them first with a line or polygon selection tool. The line selection tool has a configurable width and unit. Selections can also be easily imported/exported from/to a vector map layer.

Multiple selections are allowed and users can add or remove from existing selection using CTRL and SHIFT keys modifiers, respectively.

Selecting raster cells

Modifying rasters with QGIS expressions

In addition to setting a constant value, new cells value might be evaluated using QGIS expressions. Users can build their expressions using cells’ row and column variables, or x and y of cell center point.

In the Serval group of the builder users will find several expression functions for:

  • interpolating raster value from a mesh layer,
  • averaging attribute values from vector layer features intersecting the cell,
  • getting the nearest vector layer feature attribute,
  • interpolating z-value from the nearest 3D linestring feature.

Interpolation on a mesh layer

With the recent QGIS additions it is very easy to create mesh layers. Users can use a mesh layer for raster cell value interpolation. For example, interpolate_from_mesh() function will identify mesh dataset value and return it as a new raster value, optionally only when the value is higher than existing cell value. It is a convenient way for finding an embankment shape on a raster DTM - see pictures below.

Serval QGIS expression builder

Raster values interpolated on a mesh layer

Interpolation on 3D linestrings

Another useful function finds the nearest point on a 3D line feature and interpolate its z-value. It could be used for fine-tuning terrain elevation from a 3D profile line.

For example, if the existing embankment crown needs to be raised by an amount at one of its ends, a user could create a profile line, adjust the vertex elevation in the Vertex Editor and use it for the interpolation.

Raising embankment elevation based on 3D profile line

Creating custom expression functions

In the Expression builder users can define their own functions using other types of vector and raster layers.

Applying 3x3 low-pass filter

For eliminating peak values or local cell values averaging, users can use 3x3 low-pass filter.

Applying low-pass filter

For more details and examples, refer to the plugin’s Manual.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More