OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Virtual Point Clouds (VPC)

As a part of our crowdfunding campaign we have introduced a new method to handle a large number of point cloud files. In this article, we delve into the technical details of the new format, rationale behind our choice and how you can create, view and process virtual point cloud files.

Rationale

Lidar surveys of larger areas are often multi-terabyte datasets with many billions of points. Having such large datasets represented as a single point cloud file is not practical due to the difficulties of storage, transfer, display and analysis. Point cloud data are therefore typically stored and distributed split into square tiles (e.g. 1km x 1km), each tile having a more manageable file size (e.g. ~200 MB when compressed).

Tiling of data solves the problems with size of data, but it introduces issues when processing or viewing an area of interest that does not fit entirely into a single tile. Users need to develop workflows that take into account multiple tiles and special care needs to be taken to deal with data near edges of tiles to avoid unwanted artefacts in outputs. Similarly, when viewing point cloud data, it becomes cumbersome to load many individual files and apply the same symbology.

Here is an example of several point cloud tiles loaded in QGIS. Each tile is styled based on min/max Z values of the tile, creating visible artefacts on tile edges. The styling has to be adjusted for each layer separately:

An example of individual point cloud tiles loaded in QGIS, each styled differently

Virtual Point Clouds

In the GIS world, many users are familiar with the concept of virtual rasters. A virtual raster is a file that simply references other raster files with actual data. In this way, GIS software then treats the whole dataset comprising many files as a single raster layer, making the display and analysis of all the rasters listed in the virtual file much easier.

Borrowing the concept of virtual rasters from GDAL, we have introduced a new file format that references other point cloud files - and we started to call it virtual point cloud (VPC). Software supporting virtual point clouds handles the whole tiled dataset as a single data source.

At the core, a virtual point cloud file is a simple JSON file with .vpc extension, containing references to actual data files (e.g. LAS/LAZ or COPC files) and additional metadata extracted from the files. Even though it is possible to write VPC files by hand, it is strongly recommended to create them using an automated tool as described later in this post.

On a more technical level, a virtual point cloud file is based on the increasingly popular STAC specification (the whole file is a STAC API ItemCollection). For more details, please refer to the VPC specification that also contains best practices and optional extensions (such as overviews).

Virtual Point Clouds in QGIS

We have added support for virtual point clouds in QGIS 3.32 (released in June 2023) thanks to the many organisations and individuals who contributed to our last year’s joint crowdfunding with North Road and Hobu. The support in QGIS consists of three parts:

  1. Create virtual point clouds from a list of individual files
  2. Load virtual point clouds as a single map layer
  3. Run processing algorithms using virtual point clouds

Those who prefer using command line tools, PDAL wrench includes a build_vpc command to create virtual point clouds, and all the other PDAL wrench commands support virtual point clouds as the input.

Using Virtual Point Clouds

In this tutorial, we are going to generate a VPC using the new Processing algorithm, load it in QGIS and then generate a DTM from terrain class. You will need QGIS 3.32 or later for this. For the purpose of this example, we are using the LiDAR data provided by the IGN France data hub.

In QGIS, open the Processing toolbox panel, search for the Build virtual point cloud (VPC) algorithm ((located in the Point cloud data management group):

VPC in the Processing toolbox

VPC algorithm in the Processing toolbox

In the algorithm’s window, you can add point cloud layers already loaded in QGIS or alternatively point it to a folder containing your LAZ/LAS files. It is recommended to also check the optional parameters:

  • Calculate boundary polygons - QGIS will be able to show the exact boundaries of data (rather than just rectangular extent)

  • Calculate statistics - will help QGIS to understand ranges of values of various attributes

  • Build overview point cloud - will also generate a single “thinned” point cloud of all your input data (using only every 1000th point from original data). The overview point cloud will be created next to the VPC file - for example, for mydata.vpc, the overview point cloud would be named mydata-overview.copc.laz

VPC algorithm inputs, outputs and options

VPC algorithm inputs, outputs and options

After you set the output file and start the process, you should end up with a single VPC file referencing all your data. If you leave the optional parameters unchecked, the VPC file will be built very quickly as the algorithm will only read metadata of input files. With any of the optional parameters set, the algorithm will read all points which can take some time.

Now you can load the VPC file in QGIS as any other layer - using QGIS browser, Data sources dialog in QGIS or by doing drag&drop from a file browser. After loading a VPC in QGIS, the 2D canvas will show boundaries of individual files - and as you zoom in, the actual point cloud data will be shown. Here, a VPC loaded together with the overview point cloud:

VPC algorithm output

Virtual point cloud (thinned version) generated by the VPC algorithm

Zooming in QGIS in 2D map with elevation shading - initially showing just the overview point, later replaced by the actual dense point cloud:

VPC algorithm output in 2D maps

VPC output on 2D: displaying details when zooming in

In addition to 2D maps, you can view the VPC in a 3D map windows too:

If the input files for VPCs are not COPC files, QGIS will currently only show their boundaries in 2D and 3D views, but processing algorithms will work fine. It is however possible to use the Create COPC algorithm to batch convert LAS/LAZ files to COPC files, and then load VPC with COPC files.

It is also worth noting that VPCs also work with input data that is not tiled - for example, in some cases the data are distributed as flightlines (with lots of overlaps between files). While this is handled fine by QGIS, for the best performance it is generally recommended to first tile such datasets (using the Tile algorithm) before doing further display and analysis.

Processing Data with Virtual Point Clouds

Now that we have the VPC generated, we can run other processing algorithms. For this example, we are going to convert the ground class of the point cloud to a digital terrain model (DTM) raster. In the QGIS Processing toolbox, search for Export to raster algorithm (in the Point cloud conversion group):

VPC as an input to processing algorithms

VPC layer can be used as an input to the point cloud processing algorithm

This will use the Z values from the VPC layer and generate a terrain raster based on a user defined resolution. The algorithm will process the tiles in parallel, taking care of edge artefacts (at the edges, it will read data also from the neighbouring tiles). The output of this algorithm will look like this:

Converting a VPC layer to a raster

Converting a VPC layer to a DTM

The output raster contains holes where there were no points classified as ground. If needed for your use case, you can fill the holes using Fill nodata algorithm from GDAL in the Processing toolbox and create a smooth terrain model for your input Virtual Point Cloud layer:

Filling the holes in the DTM

Filling the holes in the DTM

Virtual point clouds can be used also for any other algorithms in the point cloud processing toolbox. For more information about the newly introduced algorithms, please see our previous blog post.

All of the point cloud algorithms also allow setting filtering extent, so even with a very large VPC, it is possible to run algorithms directly on a small region of interest without having to create temporary point cloud files. Our recommendation is to have input data ready in COPC format, as this format provides more efficient access to data when spatial filtering is used.

Streaming Data from Remote Sources with VPCs

One of the very useful features of VPCs is that they work not only with local files, but they can also reference data hosted on remote HTTP servers. Paired with COPCs, point cloud data can be streamed to QGIS for viewing and/or processing - that means QGIS will only download small portions of data of a virtual point cloud, rather than having to download all data before they could be viewed or analysed.

Using IGN’s lidar data provided as COPC files, we have built a small virtual point cloud ign-chambery.vpc referencing 16 km2 of data (nearly 700 million points). This VPC file can be loaded in QGIS and used for 2D/3D visualisation, elevation profiles and processing, with QGIS handling data requests to the server as necessary. Processing algorithms only take a couple of seconds if the selected area of interest is small (make sure to set the “Cropping extent” parameter of algorithms).

All this greatly simplifies data access to point clouds:

  • Data producers can use very simple infrastructure - a server hosting static COPC files together with a single VPC file referencing those COPC files.

  • Users can use QGIS to view and process point cloud data as a single map layer, with no need to download large amounts of data, QGIS (and PDAL) taking care of streaming data as needed.

We are very excited about the opportunities that virtual point clouds are bringing to users, especially when combined with COPC format and access from remote servers!

Thanks again to all contributors to our crowdfunding campaign - without their generous support, this work would not have been possible.

Contact us if you would like to add more features in QGIS to handle, analyse or visualise lidar data.

Learn More

Cesium Ecosystem Grant Win for QGIS 3D Tiles!

Success! Lutra and North Road have been rewarded a Cesium Ecosystem Grant to provide access to 3D tiles within QGIS. We will be creating the ability for users to visualise 3D Tiles in QGIS alongside other standard geospatial sources in both 3D and 2D map views.
3D Tiles Cesium integration ecosystem diagram
3D Tiles Cesium integration ecosystem
We are very excited about it, but to be included in the first cohort of awardees is also an added honour! We share this distinction with 3 other recipients:
The opportunity was brought to our attention by our friends over at Nearmap, which, along with the existence of this grant, shows how the geospatial community is working together by evolving the Open Source Economy. A movement close to our hearts and our core business. Working between commercial software and open-source, Cesium’s business model recognises the legitimacy of Open Source Software for use as a geospatial standard operating procedure by promoting openness and interoperability.
Our team of Nyall Dawson and Martin Dobias will create a new layer type, QgsTiledMeshLayer, allowing for direct access to Cesium 3D tile sources alongside the other supported geospatial layer types within QGIS. This will include visualisation of the tile data in both 3D and 2D map views (feature footprints). It will fulfill a critical need for QGIS users, permitting access to 3d data provided by their respective government agencies to work alongside all their other standard geospatial layers (vector, raster, point clouds). By making 3D Tiles a first class citizen in QGIS we help strengthen the case that those agencies should be providing their data in the Cesium format (as opposed to any proprietary alternatives).
Proposed Technical Architecture Cesium QGIS
Proposed Technical Architecture for Cesium 3D Tiles in QGIS
Here’s a breakdown of what we will be doing:
  • Develop a new QGIS layer type “QgsTiledMeshLayer”
  • Develop a parser for 3D Tiles format, supporting Batched 3D Model (with a reasonable set of glTF 2.0 features)
  • Develop a 3D renderer which dynamically loads and displays features from 3D Tiles based on appropriate 3D view level of detail. (A similar approach has already been implemented in QGIS for optimised viewing of point cloud data).
  • Develop a 2D renderer for 3D Tiles, which will display the footprints of 3D tile features in 2D QGIS map views. Just like the 3D renderer, the 2D renderer will utilise map scale information to dynamically load 3D tiles and display a suitable level of detail for the footprints.
  • Users will have full control over the appearance of the 2D footprints, with support for all of QGIS’ extensive polygon symbology options.
  • By permitting users to view the 2D footprints of features, we will promote use of Cesium 3D Tiles as a suitable source of cartographic data, eg display of authoritative building footprints supplied by government agencies in the Cesium 3D Tile format.

Through past partnerships, North Road and Lutra Consulting have developed and extended the 3D mapping functionality of QGIS. To date, all the framework for mature, performant 3D scenes including vector, mesh, raster and point cloud sources are in place. We are now ready to extend the existing functionality with Cesium 3D tiles support as QGIS 3D engine already implements most of the required concepts, such as out of core rendering and hierarchical level of detail (tested with point clouds with billions of points).

So there we go! Working together collaboratively with Lutra Consulting on another great addition to QGIS 3D Functionality thanks to Cesium Ecosystem Grants. Stay tuned on our social channels to find out when it will be released in QGIS.

Cesium Ecosystem grant Badge

 

Learn More

Native point cloud processing in QGIS

After the addition of support for visualising point clouds in the recent versions of QGIS, the next step was to add the processing tools so users can manage and analyse their data.

There are several 3rd party QGIS plugins (either proprietary or not fully open source) which allow users to interrogate and analyse lidar data. But with our latest work, we have introduced powerful point cloud algorithms to the QGIS Processing framework. All the algorithms are available out of the box in QGIS 3.32, with no need to install plugins.

In this blog post, we summarise the initial point cloud algorithms for QGIS Processing toolbox which will be available in QGIS 3.32 (to be released at the end of June 2023). This work was made possible by the generous donations to our crowdfunding.

Point Cloud algorithms in QGIS

First off a quick look at the new algorithms as shown in the Processing toolbox in three groups:

Point Cloud algorithms in QGIS processing toolbox

Point Cloud algorithms in QGIS processing toolbox
  • Convert formats: this will allow you to convert your point cloud data between LAS and LAZ formats for the time being. Other PDAL supported formats can be added later.
  • Export to raster: with this algorithm you can export point cloud to a regularly gridded raster data. It uses inverse distance weighting to assign raster cell values. Raster cells with no nearby points will get “no data” values (these holes may be removed by using “Fill nodata” raster algorithm).

Input point cloud layer file

Input point cloud layer file

Raster output using Intensity attribute of points

Raster output using Intensity attribute of points
  • Export to raster (using triangulation): this allows you to export Z data to a regularly gridded raster by interpolating between the points using triangulation. Note that this can be slower if you are dealing with a large dataset. If your point cloud is dense, you can export your ground points as a raster using the Export to raster algorithm.

Terrain raster output generated by point cloud triangulation

Terrain raster output generated by point cloud triangulation
  • Export to vector: to export point cloud to other vector formats. This is useful to export some of your data for software applications which do not support point cloud data and still use formats such as CSV, Shapefile, DXF.

Exporting point cloud (ground points) to Shapefile styled based on the elevation

Exporting point cloud (ground points) to Shapefile styled based on the elevation!
  • Assign projection: assigns a projection to a point cloud layer (if it is wrong or missing)
  • Build virtual point cloud (VPC): with this algorithm you can generate a virtual file (based on STAC specification) and load them as a single file in QGIS. There will be a separate blog post detailing this new exciting feature.
  • Clip: clip a point cloud layer by a vector polygon layer.

Input point cloud layer and a polygon coverage

Input point cloud layer and a polygon coverage

Result of the clipping algorithm

Result of the clipping algorithm
  • Create COPC: when you load a non-indexed point cloud layer in QGIS, it will take a while for the application to create the COPC index for your file. With this algorithm, you can create the index for all your files in a batch mode.
  • Information: displays information from a point cloud layer:
LAS           1.4
point format  6
count         56736130
scale         0.001 0.001 0.001
offset        431749.999 5440919.999 968.898
extent        431250 5440420 424.266
              432249.999 5441419.999 1513.531
crs           ETRS89 / UTM zone 34N (N-E) (EPSG:3046)  (vertical CRS missing!)
units         horizontal=metre  vertical=unknown

Attributes:
 - X floating 8
 - Y floating 8
 - Z floating 8
 - Intensity unsigned 2
 - ReturnNumber unsigned 1
 - NumberOfReturns unsigned 1
 - ScanDirectionFlag unsigned 1
 - EdgeOfFlightLine unsigned 1
 - Classification unsigned 1
 - ScanAngleRank floating 4
 - UserData unsigned 1
 - PointSourceId unsigned 2
 - GpsTime floating 8
 - ScanChannel unsigned 1
 - ClassFlags unsigned 1

Output from point cloud information algorithm

  • Merge: join multiple point cloud layers into a single file
  • Reproject: reproject the input file to a different coordinate reference system
  • Thin (by sampling radius): reduces the number of points within a certain radius

Thining point cloud (by sampling radius)

Thining point cloud (by sampling radius)
  • Thin (by skipping points): reduces the number of points by skipping nearby points
  • Tile: this algorithm generates a set of tiles based on the input point cloud layer and tile size
  • Boundary: generates a (multi) polygon from your point cloud data. The output file might contain holes depending on the density of your point cloud input data.

Extracting high vegetation and building polygons from an input point cloud layer

Extracting high vegetation and building polygons from an input point cloud layer
  • Density: outputs a raster file based on the number of points within each raster cell - useful for quality checking of point cloud datasets

Point density (number of points per 2x2 m) as a raster

Point density (number of points per 2x2 m) as a raster
  • Filter: it creates a new file based on the filter set as an expression. Note that most of the algorithms support on-the-fly filtering under the Advanced parameters.

Filtering of high vegetation class from an input point cloud layer

Filtering of high vegetation class from an input point cloud layer

Behind the scenes

All the heavy lifting of the point cloud processing is done by PDAL - a state of the art open source library for processing point clouds. PDAL provides a wide range of “readers”, “filters” and “writers” to build complex pipelines to process point clouds.

We have built a new standalone command line tool pdal_wrench on top of PDAL. It addresses two major issues that non-expert users typically face when working with PDAL:

  • Ease of use: not everyone finds it easy to manually craft JSON files with pipelines, study manuals of the many stages and read details about file formats involved.
  • Parallel execution: PDAL runs pipelines in a single thread, so only one CPU gets to do the work normally and users need to implement their own parallelism if they want to speed up processing.

The command line tool provides a simple set of commands that take care of everything. For example, to export a raster layer with elevations (DEM) with 1 meter resolution:

pdal_wrench to_raster --output=raster.tif --resolution=1 --attribute=Z data.las

The pdal_wrench tool does not depend on QGIS, so it can be easily used separately.

The commands are designed to run in parallel when there are multiple input files or when the input file is in COPC format. Depending on the algorithm, the work gets split spatially into square tiles (1000x1000 map units by default) for parallel processing, or individual files are processed in parallel. With a single ordinary LAS/LAZ file on input, there is currently no parallelism going on.

For commands that are sensitive to edge artifacts (such as export to raster), we take care of processing extra points outside of the extent of each tile (referred to as collar or buffer) to make sure the results are correct as if no tiling would be happening (see Martin Isenburg’s article for more details: https://rapidlasso.com/2015/08/07/use-buffers-when-processing-lidar-in-tiles/).

Future work

The current list of point cloud algorithms already allows users to do plenty of work. But more could be added to the toolbox - algorithms that are already supported by PDAL, but not exposed in QGIS: classification, noise removal, surface reconstruction, clustering, height above ground, colorizing and many more. If you are interested in more point cloud processing algorithms in QGIS, please contact us and we will be happy to add them to future QGIS releases.

Learn More

SLYR Update — June 2023

Welcome back, SLYR enthusiasts! We’re thrilled to share the latest updates and enhancements for our SLYR ESRI to QGIS Compatibility Suite that will dramatically streamline the use of ESRI documents within QGIS (and vice versa!). Our team has been hard at work, expanding the capabilities of SLYR to ensure seamless compatibility between the latest QGIS and ArcGIS releases. We’ve also got some exciting news for users of the Community Edition of SLYR! Let’s dive right in and explore the exciting new features that have been added to SLYR since our previous update…

Converting Raster Layers in Geodatabases

We’re pleased to announce that SLYR now offers support for converting raster layers within Geodatabases. With this update, users can effortlessly migrate their raster data from ESRI’s Geodatabases to QGIS, enabling more efficient data management and analysis.

This enhancement is only possible thanks to work in the fantastic GDAL library which underpins QGIS’ raster data support. Please ensure that you have the latest version of QGIS (3.30.3 or 3.28.7 at a minimum) to make the most of this feature.

Annotation and Graphic Layer Improvements

Text Annotations along Curves

For those working with curved annotations, we’ve got you covered! SLYR now supports the conversion of text annotations along curves in QGIS. With this enhancement, you’ll get accurate conversion of any curved text and text-along-line annotations from MXD and APRX documents. This has been a long-requested feature which we can now introduce thanks to enhancements coming in QGIS 3.32.

ArcGIS Pro Graphics Layer Support

SLYR now supports the conversion of ArcGIS Pro graphics layers, converting all graphic elements to their QGIS “Annotation Layer” equivalents. If you’ve spent hours carefully designing cartographic markup on your maps, you can be sure that SLYR will allow you to re-use this work within QGIS!

Curved text graphic conversion

Enhanced Page Layout Support

We’ve further improved the results of converting ArcGIS Pro page layouts to QGIS print layouts, with dozens of refinements to the conversion results. The highlights here include:

  • Support for converting measured grids and graticules to QGIS map grids
  • Enhanced dynamic text conversions:  Now, when migrating your projects, you can expect a smoother transition for dynamic text ensuring your layouts correctly show generated metadata and text correctly
  • Support for north arrows, grouped elements, legends and table frames.

Rest assured that your carefully crafted map layouts will retain their visual appeal and functionality when transitioning to QGIS!

Improved QGIS to ArcGIS Pro Conversions

SVG Marker Exports and Symbology Size

SLYR has introduced initial support for exporting SVG markers from QGIS to ArcGIS Pro formats. SVG graphics are a component of QGIS’ cartography, and are frequently used to create custom marker symbols. Unfortunately, ArcGIS Pro doesn’t have any native support for SVG graphics for marker symbols, instead relying on a one-off conversion from SVG to multiple separate marker graphics whenever an SVG is imported into ArcGIS Pro. Accordingly, we’ve implemented a similar logic in SLYR in order to convert SVG graphics to ArcGIS Pro marker graphics transparently whenever QGIS symbology is exported to ArcGIS. This enhancement allows for a seamless transfer of symbology from QGIS, ensuring that your converted maps retain their visual integrity.

Furthermore, the update includes support for exporting QGIS symbology sizes based on “map unit” measurements to ArcGIS Pro, resulting in ArcGIS Pro symbology which more accurately matches the original QGIS versions.

Rule-Based Renderer Conversion

The “Rule Based Renderer” is QGIS’ ultimate powerhouse for advanced layer styling. It’s extremely flexible, thanks to its support for nested rules and filtering using QGIS expressions. However, this flexibility comes with a cost — there’s just no way to reproduce the same results within ArcGIS Pro’s symbology options! Newer SLYR releases will now attempt to work around this by implementing basic conversion of QGIS rule-based renderers to ArcGIS Pro layers with “display filters” attached. This allows us to convert some basic rule-based configuration to ArcGIS Pro formats.

There’s some limitations to be aware of:

  1. Only “flat” rule structures can be converted. It’s not possible to convert a nested rule structure into something representable by ArcGIS Pro.
  2. While the QGIS expression language is very rich and offers hundreds of functions for use in expressions, only basic QGIS filter expressions can be converted to ArcGIS Pro rules.

Improved Conversion of Raster and Mesh Layers

Based on user feedback, we’ve made significant improvements to the conversion of QGIS rasters and mesh layers to ArcGIS Pro formats. Expect enhanced accuracy when migrating these types of data, ensuring a closer match between your QGIS projects and their ArcGIS Pro equivalents.

New tools

The latest SLYR release introduces some brand new tools for automating your conversion workflows:

Convert LYR/LYRX Files Directly to SLD

To facilitate interoperability, SLYR has introduced algorithms that directly convert ESRI LYR or LYRX files to the “SLD” format (Styled Layer Descriptor). This feature simplifies the process of sharing and utilizing symbology between different GIS software, allowing for direct conversion of ESRI symbology for use in Geoserver or Mapserver.

Convert File Geodatabases to Geopackage

We’re thrilled to introduce a powerful new tool in SLYR that enables a comprehensive conversion of a File Geodatabase to the Geopackage format. With this feature, you can seamlessly migrate your data from ESRI’s File Geodatabase format to the versatile and widely supported GeoPackage format. As well as the raw data conversion, this tool also ensures the conversion of field domains and other advanced Geodatabase functionality to their GeoPackage equivalent, preserving valuable metadata and maintaining data integrity throughout the transition. (Please note that this tool requires QGIS version 3.28 or later.)

 

All these exciting additions to SLYR are available today to SLYR license holders. If you’re after one-click, accurate conversion of projects from ESRI to QGIS, contact us to discuss your licensing needs.

As described on our SLYR page, we also provide some of the conversion tools for free use via the SLYR “Community Edition”. We’re proud to announce that we’ve just hit the next milestone in the Community Edition funding, and will now be releasing all of SLYR’s support for raster LYR files to the community edition! This complements the existing support for vector LYR files and ESRI style files available in the community edition. For more details on the differences between the licensed and community editions, see our product comparison.

Learn More

Integrate a QGIS Cloud Map into a Web Page

After a beautiful map has been created with QGIS and published on https://qgiscloud.com, the desire quickly arises to integrate this map into one’s own website. This way, visitors to the website could access the map directly and without having to go via https://qgiscloud.com. With a little HTML, this is easy to do. But let’s start from the beginning. Make a QGIS Cloud Map As always with QGIS Cloud, a map must first be published.
Learn More

QGIS Cloud: Publish Live Data in Web Map

The QGIS Cloud Support Team is often asked if it is possible to integrate data from external databases into the QGIS Cloud Web Map, so that the map information in the QGIS Cloud Web Map is always up-to-date. In this blog article we would like to show how this is possible with PostgreSQL. To publish data in the QGIS Cloud GDI, the data must always be stored in the personal QGIS Cloud database.
Learn More

FOSSGIS 2023

Sourcepole hat an der FOSSGIS 2023 in Berlin verschiedene Themen mit Vorträgen abgedeckt:

  • QGIS Web Client 2 (QWC2) - Neues aus dem Projekt
  • COPC, das neue cloudoptimierte Format für Point Clouds
  • Cloudoptimierte Formate für Kacheln und multidimensionale Rasterdaten
  • Geodatenverarbeitung mit Workflow-Engines
Learn More

Summary of the 3D features & enhancements in QGIS 3.30

QGIS 3.30 was released late last week and we are pleased to announce the new features introduced as a part of our latest crowdfunding campaign to improve 3D, point cloud and elevation data.

Thank you

First and foremost, thanks to the generous support from the community to fund our work. Here is the list of our contributors in no particular order:

IGN (INSTITUT NATIONAL DE L’INFORMATION GEOGRAPHIQUE ET FORESTIERE), National Land Survey of Finland, Danish Agency for Data Supply and Infrastructure, K2 Engineering GmbH, ProScape Consulting, Västra Götalandsregionen, Kristianstads kommun, IGN FI, L’Arrière Guichet, Septima, QWAST-GIS, ATAPEX s.r.o., REDcatch GmbH, F.A.R.M. Facilitazioni Agroecologiche Regionali Mobili, EPYMA TERRITORIO Y MEDIO AMBIENTE, SL, GEO EXPLORATION LTD, Bohannan Huston, Inc., Lidar Guys, Neuchâtel- Service de la géomatique, Wooding Geospatial Solutions, Ville de Vevey, QGIS User Group Switzerland, Ecophylla Consulting, Refactor, Locate Press, Alta ehf, Oester Messtechnik GmbH, RUDAZ+PARTNER AG, BayesMap Solutions LLC, GEOACE, Natalie Gyles, Andreas Neumann, Dougal Munro, Spatial Thoughts, Cicada Systems GIS Consulting, Cori Hermle, Powell Asset Mapping LLC, Darren Farmer, Greg Hall, Ecothought Pty Ltd, Gabriel Diosan, Bhutan QGIS Group, Ultimatum Finesse, Balanced Risk Strategies, Ltd, Concordia University, Burmis Studio Inc., Nicholas Hadaller, Angello Villatoro, Yoichi Kayama, Hennessy Amor Becerra Ayala, Flow Design Limited, BNHR.XYZ, Roberto Moyano, Benjamin Kuster, Goldspot, North River Geographic Systems, Inc, David W. Wormuth, Victor Graphics, Valley Spatial, Stephen Mather, SANTIAGO AURELIO MOTA, Kelly Crowell, Brian Duhan, Paddy Fisher, OSGEO:UK, Christian Gugl, GIP CRAIG - Centre Régional Auvergne-Rhône-Alpes de l’Infromation Géographqiue, Raphael Mabit, Tibor Lieskovský, Kerstin Fohlert, Zhan Li, Bernd Vogelgesang, Marlin Müller, Johannes Bonekamp, Stefan Giese, Fabian Faßnacht, QGIS Sweden user group, Falo, DAVID GARCIA HERNANDEZ, Lint Data and Geospatial, Cliff Sin Wai Lau, Grzegorz Sapijaszko, Łukasz Rapa, Alessandro Pintucci and Maarten Pronk.

Our gratitude also goes to those who want to remain anonymous.

Global map shading

Map data without global terrain shading

Global terrain shading for point clouds, dem and vectors

To see this feature in action, you can open QGIS project properties and under Terrain, there should be an option for Global Map Shading. You will need to first add a raster as your DEM under the Terrain section.

Elevations of all these layers are combined and the considered elevation is chosen depending on one of two methods:

  1. the highest elevation between raster, mesh or point cloud layers will be selected.
  2. elevation will be selected based on the order of layers in the layer panel.

Depending on the context and the use of the map the user can choose the more appropriate method.

For now, the shading methods implemented are the Eye Dome Lighting and the hill-shade. More methods could be added in the future - such as ambient occlusion.

The user can choose the elevation shading settings in a specific UI widget that can be found in two places:

  1. under the Project Properties, within the same tab of the project elevation settings.
  2. under the Styling Panel, a new tab is added for quick access to the user.

Global terrain shading settings in project properties

Global terrain shading settings in style panel

Profile elevation within print composer

This work was carried out by our collaborator North Road. The profile tool can embed elevation profiles within print layouts. It is possible to add beautifully styled profiles in your print outputs.

Embeding elevation profile in the print composer

To use elevation profiles in print layouts, simply click the Add Elevation Profile button icon elevation profile in print composer in the toolbar, and then to initialize it, copy profile configuration from an existing elevation profile from QGIS main window using the “Copy From Profile” button.

Elevation profile settings in the print composer

3D navigation improvements

With the new improvements, you can:

  1. move camera vertically using ctrl+shift+left mouse button
  2. keep zooming with the wheel while moving the mouse
  3. continue right mouse button zooming when pointer exits the viewport

Improve zoom-extents in 3D

The terrain’s and point cloud layers’ elevation range are taken into account so that the camera is not positioned below the scene’s contents, which was the case when using the terrain’s vertical scale setting to exaggerate the elevation differences.

Limit 3D scenes’ 2D extent

3D Views can now be limited to a specific 2D extent. The terrain is clipped and no 3D features beyond that extent are loaded, making it easy to render specific areas of big QGIS projects. The project’s 2D extent is used by default which can then be adjusted in each 3D view separately using the new General tab in 3D configuration.

Limiting 3D map extent

Limiting 3D map extent

Future updates

There will be more features planned for QGIS 3.32. We are finalising the Processing framework for point cloud data and it should be available on QGIS master in coming weeks.

Please do not hesitate to contact us if you have any suggestions to improve QGIS 3D and point cloud support.

Learn More

How Oslandia invests in OpenSource

You may be wondering where Oslandia’s name is coming from ? Or maybe you already know ? In this article we focus on the “OS” part of Oslandia : OpenSource !

Oslandia positions itself as IT expert in the field of OpenSource geographical information systems. QGIS is namely one of the proheminent opensource softwares for the geospatial industry. This position is a key element of our business model.

But do you know how we work behind the scene ? This article will give you an opportunity to discover some of our contributions to the OpenSource ecosystem.

Principles

Our general business model is based on projects we carry out for our clients. They fund us to design and implement solutions adapted to their needs and requirements. Part of these developments consist in core development of Opensource software. This allows us to contribute actively to FOSS4G components.

But this funding method makes it complicated to fund maintenance, or new exploratory developments, as well as communication, community management or other tasks necessary for healthy opensource projects.

As a consequence, we introduced at Oslandia a mechanism of internal OpenSource project grants.

These grants constitute self-investment from the company into the OpenSource ecosystem, and can be applied to new projects, research and development or existing projects.

This mechanism has multiple interests :

  • For opensource projects : maintenance and new contributions
  • For Oslandia : image and potential new business opportunities
  • For the team : work on projects that matter to them

These OpenSource grants consist in a large range of possible tasks, as we often say : “Opensource projects are not only code”. Instead of developers, we prefer the term contributors. Development, code review, maintenance, documentation, community management, communication, each collaborator can choose the type of task to focus on.

We differentiate software maintenance grants and opensource project grants. We call the latter “OpenSource mini-projects”

Software maintenance consists in refactoring, bugfixing, packaging, release management… All these tasks need dedicated time which is difficult to fund directly on client’s project.

Opensource mini-projects grants are specific opensource proposal which can be submitted by any collaborator on any subject. We then vote on the best proposal and the team can start working on the subject within the allocated budget.

Some numbers

We allocate around 5% of the global production time to software maintenance grants. Our Opensource maintenance grant for 2022 is therefore approximately 190 days of work. It mainly focus on QGIS, PostGIS, QWC2, Giro3D and a few other components we actively maintain.

We also allocate 5% of the global production time to opensource mini-projects grants. It represents an additional 190 days of work for 2022.

Oslandia therefore invests almost 400 days of work into the OpenSource ecosystem, outside of direct contributions for client’s projects.

Opensource Mini-projects

OpenSource mini-projects grants are submitted by Oslandia’s collaborators and focus on various task and thematics : innovation, development, design, prototyping, communication or any other kind of Opensource contribution.

Proposals have to define goals, deliverables, planning, team and needed budget. Then we evaluate the proposals given the following criteria :

  • proposal coherency ( e.g. deliverables vs budget )
  • alignment with Oslandia’s strategy
  • innovation level
  • business opportunities
  • fun and motivation
  • impacts in terms of communication
  • links with other projects at Oslandia
  • possibility of extra R&D funding

We then vote on best proposal and manage these mini-projects just as a client project.

Examples

QGIS

The maintenance grant on QGIS allowed us to work on the following tasks :

  • Bugfixing
  • Code review for PRs submitted by other developers
  • Code refactoring
  • Documentation
  • Packaging pipeline
  • OSGeo4W improvement

OpenSource mini-projects grants

During the year of 2022, we worked on the following mini-projects :

In 2023 we will continue to work on these projects, and others ! for example pg_featureserv, py3dtiles, infoclimat website, MapProxy,  pgRouting…

Conclusion

This investment mechanism allows Oslandia to be an opensource “pure player” and contribute actively to these OpenSource projects and to the OpenSource ecosystem as a whole.

Should you be interested in our contribution model, or if you have any question regarding our internal OpenSource grant program, do not hesitate to contact us : info@oslandia.com !

Learn More

[Blog] View and track changes in QGIS

With the recent changes to the Mergin Maps plugin for QGIS, you can visualise the local changes before synchronising your data. Read more in our blog post!
Learn More