OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

(Fr) Oslandia recrute : ingénieur(e) développement C++ / Python – OSL2011B

Sorry, this entry is only available in French.

Learn More

Google Summer of Code 2020 : QGIS 3D Improvements

In this year Google Summer of Code (GSoC), multiple new features were added to QGIS 3D. Belgacem Nedjima as the student with Martin Dobias and Peter Petrik as the mentors have implemented the following features into the 3D viewer of QGIS:

Feature 1 : Textures support

Now it is possible to add textures to vector layer data (buildings) in the 3D viewer: To do that you need to go the vector layer properties and select Realistic Textured (Phong) shading mode and select the image to be displayed on the surfaces by providing an image file to the diffuse texture field. texture-support-UI You can adjust the scale of the image and its rotation from the texture scale and texture fields respectively. texture-support-preview There is a workaround to seperate roofs from walls using rule based rendering by adding 2 seperate rules and selecting which facet (roof or a wall) to display and select which rendered facade to be used and the shading setting for each rule. texture-support-walls-roofs-UI Results for seperating roofs and walls texture-support-walls-roofs-preview

Feature 2 : Directional lights

QGIS 3.16 adds support for directional lights that you can think of as the sun. The default light in QGIS 3.16 is now directional which makes the lighting of the whole scene better. To use directional lights, open the 3D configuration dialog and from the lights tab you can configure how many lights you use. If you select a directional light, you will see the new improved way to provide the direction of the light (Thanks to Vincent Cloarec for the UI improvements). directional-lights-UI

Feature 3 : Scene export as .obj file

In the 3D viewer there is a new tool to export the scene as a .obj file and visualize or edit it in other software like Blender or MeshLab. To export the scene, from the 3D viewer select the 3D object icon and you will see the following dialog: export-UI You can select the name of the .obj file and where to save it (you need to select a folder because you will have multiple files if you export textures or color informations). You can specify the resolution of the terrain in case you’re using a DEM as well as the resolution of the exported textures. Here is a view of what an exported scene that has multiple rendering rules looks like in MeshLab: export-meshlab-preview

Feature 4 : Skybox rendering

You can add a skybox to your scene from the 3D configuration dialog and selecting from the 2 skybox formats provided:

  • Panoramic Texture: One 360 image is used to generate the skybox
  • Distinct Faces: 6 different faces describing the cube map of the skybox skybox-panoramic skybox-distinct-faces The result of applying a skybox to the scene: skybox-preview

Feature 5 : Shadow rendering (beta)

Althought it might not work for everyone, shadow rendering was added as a technical preview in QGIS 3.16. To try shadows, you need to open the 3D configuration dialog and enable shadows from the Shadows tab. shadows-settings-menu You can experiment with the shadow rendering parameters if you want to decrease the visual artifacts and make the shadows look better in the scene. Here is how the shadows look like in our scene: shadows-preview

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

QGIS 3.16 package on macOS

QGIS 3.16 on macOS coming with Proj6, GDAL3 and major fixes! macOS packaging mechanism is completely reworked for QGIS 3.16. This will bring QGIS on macOS on par with other platforms that already benefit from the new versions of PROJ and GDAL libraries - especially the greatly improved reprojection support.

QGIS nightly and QGIS 3.16

QGIS3.16

If you want to try the new packages, download and install the QGIS from nightly builds. There are still few weeks before QGIS 3.16 release for testing and bug-fixing, so make sure you report all your issues before the 23rd October 2020. Multiple packages now can be installed side-by-side, just rename or move the installed QGIS.app!

The earlier packages were based on Homebrew, but we didn’t have control over the versions of dependencies. We switched to the new system where we have full control, which is important for good quality releases.

The package/installer is not yet notarized by Apple, so you need to right-click on the QGIS.app icon and open it to overcome the security control of your macOS (only for 10.15+).

If you want to join the effort in testing and/or development of macOS packages, please drop me a mail on peter.petrik@lutraconsulting.co.uk We have a dedicated Slack channel to discuss the maintenance of the macOS packages.

What is in the all-in-one bundle?

The goal is to have all advanced functionality of QGIS prepared and ready to use after simple one-click installation.

QGIS and utilities

QGIS Desktop, of course, but also

  • QGIS server (try with /Applications/QGIS.app/Contents/MacOS/bin/qgis_mapserver)
  • QGIS process (try with /Applications/QGIS.app/Contents/MacOS/bin/qgis_process)
  • QtDesigner for custom forms (/Applications/QGIS.app/Contents/MacOS/bin/designer)
  • ogr2ogr and various other gdal utilities

FOSS4G libraries

  • Geos 3.8.1
  • Proj 6.3.2
  • GDAL 3.1.2
  • GRASS 7.8.3
  • SAGA 7.3.0

Python 3.7

with pip, so you can install the missing packages with command /Applications/QGIS.app/Contents/MacOS/bin/pip3 install <your package>

but, many packages are already preinstalled for you!

  • pipenv
  • requests
  • plotly
  • matplotlib
  • scipy
  • numpy
  • shapely
  • geopandas
  • gdal
  • h5py
  • pyproj
  • pillow

QGIS Processing

  • GRASS processing tools
  • GDAL processing tools
  • SAGA processing tools
  • OTB processing tools (needs external installation of OTB)

Data Providers

All basic providers

  • GeoPackage
  • Spatialite
  • DB2
  • WCS/WFS/OWS/WMS/WMTS
  • Vector Tiles
  • XYZ Tiles
  • OGR/GDAL
  • PostgreSQL
  • MDAL

But also:

  • ECW
  • MrSID
  • MSSQL
  • OracleDB

Acknowledgments

In Spring 2020, we prototyped the building of FOSS macOS libraries in completely controlled environment. Few weeks ago we have successfully finished the QGIS 2020 Grant “QGIS macOS Package Improvements”. This wouldn’t be possible without support from QGIS.org and its sponsors. And without proper testing and reporting of issues from our macOS power-users.

QGIS for iOS

Do you want to see your QGIS projects and data from your iPhone and iPad? Check InputApp

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Store and visualize your raster in the Cloud with COG and QGIS

We have recently been working for the French Space Agency ( CNES ) who needed to store and visualize satellite rasters in a cloud platform. They want to access the image raw data, with no transformation, in order to fullfill deep analysis like instrument calibration. Using classic cartographic server standard like WMS or TMS is not an option because those services transform datasets in already rendered tiles.

We chose to use a quite recent format managed by GDAL, the COG (Cloud Optimize Geotiff) and target OVH cloud platform for it provides OpenStack, a open source cloud computing platform.

How it works

A COG file is a GEOTiff file which inner structure is tiled, meaning that the whole picture is divided in fixed size tile (256 x 256 pixels for instance) so you can efficiently retrieve parts of the raster. In addition to the HTTP/1.1 standard feature range request, it is possible to get specific tiles of an image through the network without downloading the entire raster.

We used a service provided by OpenStack, called Object Storage to serve the COG imagery. Object storage allows to store and retrieve file as objects using HTTP GET/POST requests.

Why not WCS ?

Web Coverage Service standard could have been an option. A WCS server can serve raw data according to a given geographic extent. It’s completely possible to deploy a container or a VPS (Virtual Private Server) running a WCS Server in a cloud plateform. The main advantages of the COG solution over WCS Server is that you don’t have to deal with the burden of deploying a server, like giving it ressources, configuring load balancing, handle updates, etc…

The beauty of COG solution is its simplicity. It is only HTTP requests, and everything else (rendering for instance) is done on the client side.

Step by step

Here are the different steps you’d have to go through if you’re willing to navigate in a big raster image directly from the cloud.

First, let’s generate a COG file

gdal_translate inputfile.tif cogfile.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATE

Install your openstack-client, it can be achieved easily with Python pip install command line

$ pip install python-openstackclient

Next, configure your openstack client in order to generate an athentification token. To do so you need to download your project specific openrc file to setup your environment)

$ source myproject-openrc.sh
Please enter your OpenStack Password for project myproject as user myuser:
**********
$ openstack token issue                                 
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                                                                   |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| expires    | 2020-07-21T08:15:12+0000                                                                                                                                                                |
| id         | xxxx_my_token_xxxx
| project_id | 97e2e750f1904b41b76f80a50dabde0a                                                                                                                                                        |
| user_id    | 18f7ccaf1a2d4344a4e35f0d84eb065e                                                                                                                                                        |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

You are now good to push you COG file to the cloud instance

openstack object create MyContainer cogfile.tif --name cogfile.tif

Before starting QGIS, 2 environment variables need to be set.  (replace xxxx_my_token_xxxx with the token you’d just come to generate)

$ export SWIFT_AUTH_TOKEN=xxxx_my_token_xxxx
$ export SWIFT_STORAGE_URL=https://storage.sbg.cloud.ovh.net/v1/AUTH_$OS_PROJECT_ID

It can also be done directly from the QGIS Python console by setting those variable using the os.environ.

Finally, add a cloud raster data source in in QGIS

You can now navigating into your image directly reading it from the cloud

© CNES 2018, Distribution Airbus DS

Performances

While panning in the map, QGIS will download only few tiles from the image in order to cover the view extent. The display latency that you could see in the video depends essentially on:

  • The number of band of your image
  • The pixel size
  • Your internet connection (mine, the one use for the video, is not an awesome one)

Note that the white flickering that you could see when you move in the map and the raster is refreshed should be removed in next version of QGIS according to this QEP.

What’s next ?

Thanks so much to the GDAL and QGIS contributors for adding such a nice feature ! It brings lots of possibilities for organizations that have to deal with great number of big raster and just want to explore part of it.

We are already thinking about further improvments (ease authentification, better integration with processing…), so if you’re willing to fund them or just want to know more about QGIS, feel free to contact us at infos+data@oslandia.com. And please have a look at our support offering for QGIS.

Learn More

Publication de l&#8217;extension COVADIS RAPEA pour QWAT et QGEP

QWAT est une application open source de gestion des réseaux d’eau potable émanant des collectivités de Pully, le SIGE à Vevey, Morges et Lausanne.
QGEP est son homologue dédiée à la gestion des eaux usées et pluviales, initiée par le groupe utilisateur QGIS Suisse.

L’échange de données entre institutions est une pierre angulaire des politiques de l’eau. Ces échanges se basent sur des formats d’échanges standardisés. Ainsi les Cantons de Fribourg (format aquaFRI) ou de Vaud (format SIRE) conditionnent certaines subventions publiques à la transmission des données selon des formats pré-définis et permettent à ces échelons administratifs d’avoir une vision globale des réseaux humides.

Dans le cadre d’une expérimentation des outils QWAT (eau potable) et QGEP (eaux usées), Charentes Eaux a souhaité mettre en œuvre des extensions dédiées au standard d’échange de données sur les réseaux d’eau Français, le Géostandard Réseaux d’adduction d’eau potable et d’assainissement (RAEPA) défini par la Commission de validation des données pour l’information spatialisée (COVADIS).

Oslandia a été mandaté pour mettre en œuvre des instances de QWAT et QGEP, réaliser les extensions RAEPA pour chacun de ces outils, et aider Charente Eaux à charger les données des collectivités membres de ce syndicat mixte.

https://charente-eaux.fr/le-syndicat/qui-sommes-nous/

Le travail a été publié pour QWAT sous forme d’une extension standardisée dans le dépôt l’organisation QWAT https://github.com/qwat/extension_fr_raepa/

Pour QGEP, il n’existe pas encore de fonctionnalité pour gérer d’extension, le dépôt https://gitlab.com/Oslandia/qgep_extension_raepa/ contient donc les définitions de données et de vues à rajouter manuellement au modèle de données.

La compatibilité des modèles de données a été évaluée et le choix a été fait de ne faire que des vues dédiées à l’export de données. Il est techniquement possible de faire des vues éditables pour permettre le chargement de données via ces vues depuis des fichiers suivant le gabarit de données RAEPA. Le niveau de simplification et d’agrégation des listes de valeurs rend ce travail peu générique dans l’état actuel du géostandard (v1.1), il est donc plus pertinent à ce stade de réaliser des scripts de chargement sans passer par ce pivot dans le cas de Charente-Eaux

Learn More

Crowdfunding: Support for point cloud data in QGIS

We are delighted to announce that in collaboration with North Road and Hobu, we are running a crowdfunding campaign to implement native support for point cloud data in QGIS.

point cloud support in QGIS

With the proposed changes, you will be able to load, style and visualise your point cloud data in QGIS in 2D and 3D map views.

The work will be carried out by the trusted and highly skilled developers across PDAL and QGIS community (Lutra Consulting, North Road and Hobu) who have been at the forefront of some of the exciting features in Open Source projects.

If you or your organisation are point cloud data users, this is your chance to bring native support for your data in QGIS. With this work, you will be able to overlay your point cloud data to your other data (vector/raster). The addition of native support for point cloud data in QGIS will pave the way to support analytical tools for point cloud data in future.

The target amount is 49,000 € and the campaign will be active until 15 October 2020.

Please have a look at the dedicated page point cloud data support in QGIS for further details and help us spread the word!

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Vector tiles in QGIS 3.14

The new QGIS 3.14 version adds support for the native loading of vector tiles. The easiest way to load them is via the recently released plugin.
Learn More

Overview of QGIS 3.14 Mesh Features

Mesh layer is now supported for the 1D, 2D and 3D data frames. It also comes with integration to the temporal controller in QGIS 3.14.

The releases of QGIS 3.14, MDAL 0.6.1 and Crayfish 3.4.4 are planned for this Friday We are delighted to have made improvements for the upcoming release:

  • full integration with the new temporal controller
  • 1D meshes support (e.g. to support urban drainage system models)
  • New rendering and styling options for mesh layers
  • improving greatly rendering speed for large datasets
  • various other enhancements and bug-fixes

If you are eager to try the latest features, you can always install QGIS nightlies/master.

Special thanks to all contributors and the sponsors of these features

Do you want to use QGIS Mesh Layers in your projects? Read more…

Native QGIS Temporal Controller

The long awaited native temporal support in QGIS is finally materialised. We managed to port all QGIS Mesh Layer code to the new infrastructure and remove the custom time slider in the Mesh Layer properties/styling dialog. The native temporal support will greatly benefit users in the long run. We can now share the code components for time handing with raster, vector and WMS-T and other temporal data types. Also user can load multiple temporal layers and change time-domain for all of them at the same time.

mesh 3d in time

Unfortunately, the new temporal framework changed the API, so if you use QgsMeshLayer in your plugins, consult the QGIS documentation for required changes.

New MDAL logo and docs

We have started writing the proper documentation for all supported formats and also reference documentation for developers. We welcome all contributions to improve the documentations. Just go to https://www.mdal.xyz and click “Edit on GitHub”!

mdal logo

Support for 1d meshes

MDAL and QGIS now supports 1D meshes, currently for UGRID and 3Di formats. Other formats should be easy to add. If you are interested in supporting a format, drop us a line.

You can read the full technical description in the following QGIS Enhancement Proposal.

The 1D meshes can be styled with the various renderers, for example use of pressure data to calculate line width: variable with 1D mesh

Furthermore, Crayfish plugin has been extended to support such mesh data frames. For example, 1D plots in Crayfish with the snapping and routing along the network can be used: snapping 1D mesh

2D mesh rendering improvement

2D meshes streamlines/particles/arrows can be now styled by the colour ramp. You can better display streamlines and mesh vectoral data varaition: streamline colorram

Meshes in 3D map view

The mesh layer is now fully supported in the 3D map view. You can visualise your terrain and overlay vector arrows (e.g. velocity) and other mesh quantities (e.g. water surface) in the 3D map view.

meshes 3D

Other features and bugfixes

QGIS

  • The mesh rendering speed increased by 2-20 times depending on various mesh/zoom/map/data combinations
  • Ability to show only subset of dataset groups in the Mesh styling window
  • Support for reading multiple meshes in one file (e.g. UGRID)
  • Ability to resample vertex values to faces
  • Support for saving/loading of QML styles
  • Full usage of temporal controller, even in print templates, animation, …

MDAL

  • The GRIB and NetCDF datasets are now loaded twice as fast
  • Support for UGRID datasets with magnitude and direction
  • Support for reading of UGRID classified data and its usage in QGIS color ramps
  • Renamed the pointers and functions in the API for consistency
  • Created brand new logo and started the online-docs
  • Support XMS Tin

Crayfish

  • Ported to new temporal API
  • Added 1D plots
  • Fixed various UI bugs

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

(Fr) Financement mutualisé du logiciel libre: le cas QGIS

Sorry, this entry is only available in French.

Learn More

Working with vector tiles - Part 1

After a successful crowdfunding campaign to support Vector Tiles in QGIS, we have been busy adding new features for the upcoming release (3.14).

In this blog post, we are exploring different methods to add your vector tiles and style them.

Loading data - local files

If you have an MBTiles file containing vector tiles, you can simply drag and drop the layer in QGIS. Alternatively, you can connect to a local vector tile file(s) from the Data Source Manager or the Browser Panel. As an example, you can use this QGIS project which contains Switzerland vector tiles (maximum zoom level =14).

Vector tile file

You can also test this project on your mobile device. Download Input app (for iOS you need the TestFlight version of the app), head to Projects > Explore and download saber\blogpost-vectortile.

Loading data - web resources

You can access vector tiles served through the web. You need to have the URL and possibly API key to add them in your QGIS. In the example below, vector tiles from MapTiler are added to QGIS:

Vector tile file

To use MapTiler data, we recommend using their dedicated QGIS plugin, which comes with vector tiles in different styles. The plugin should be available on the QGIS plugin repository.

Styling data

Vector tiles contain points, line and polygon geometries. You can define a set of rules for features and apply style and label. Style and labelling can be dependent on the zoom level. Similar to vectors and rasters, styles for vector tiles can be stored in QML format. To set up filters, you can use the identify tool in QGIS to inspect geometries of your vector tile:

Vector tile file

Currently, the most common format to store vector tiles styles are Mapbox GL (json). The OpenMapTiles repo contains some excellent Mapbox GL styles. We initially developed a tool to convert the json files to QGIS style. The plugin has been further extended and developed by MapTiler to better handle the OpenMapTiles data schema. This will ensure future changes to the data schema will be addressed by the plugin.

Example of styles converted from Mapbox GL to QGIS from Maptiler QGIS plugin

Vector tile file

Vector tile file

Vector tile file

Vector tile file

In the next post, we will explore different options within QGIS or other 3rd party tools to generate vector tiles.

Credits

All the data used in this blog post was from OpenStreetMap server by Geofabrik. OpenMapTiles conversion tool was used to generate the tiles from OpenStreetMap to vector tiles. See software license for more details.

The following OpenMapTiles Mapbox GL styles Bright, Positron, Basic and Fiord were converted using mapbox2qgis and QGIS MapTiler plugin.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More