Tag: qgis

The QGIS 3.28 release is an extremely exciting release for all users who work in mixed software workplaces, or who need to work alongside users of ESRI software. In this post we’ll be giving an overview of all the new tools and features introduced in 3.28 which together result in a dramatic improvement in the workflows and capabilities in working with ESRI based formats and services. Read on for the full details…!

Before we begin, we’d like to credit the following organisations for helping fund these developments in QGIS 3.28:

• Naturstyrelsen, Denmark
• Provincie Gelderland, Netherlands
• Uppsala Universitet, Department of Archaeology and Ancient History
• Gemeente Amsterdam
• Provincie Zuid-Holland, Netherlands

FileGeodatabase (GDB) related improvements

The headline item here is that QGIS 3.28 introduces support for editing, managing and creating ESRI FileGeodatabases out of the box! While older QGIS releases offered some limited support for editing FileGeodatabase layers, this required the manual installation of a closed source ESRI SDK driver… which unfortunately resulted in other regressions in working with FileGeodatabases (such as poor layer loading speed and random crashes). Now, thanks to an incredible reverse engineering effort by the GDAL team, the open-source driver for FileGeodatabases offers full support for editing these datasets! This means all QGIS users have out-of-the-box access to a fully functional, high-performance read AND write GDB driver, no further action or trade-offs required.

Operations supported by the GDAL open source driver include:

• Editing existing features, with full support for editing attributes and curved, 3D and measure-value geometries
• Creating new features
• Deleting features
• Creating, adding and modifying attributes in an existing layer
• Full support for reading and updating spatial indexes
• Creating new indexes on attributes
• “Repacking” layers, to reduce their size and improve performance
• Creating new layers in an existing FileGeodatabase
• Removing layers from FileGeodatabases
• Creating completely new, empty FileGeodatabases
• Creating and managing field domains

On the QGIS side, the improvements to the GDAL driver meant that we could easily expose feature editing support for FileGeodatabase layers for all QGIS users. While this is a huge step forward, especially for users in mixed software workplaces, we weren’t happy to rest there when we  had the opportunity to further improve GDB support within QGIS!

So in QGIS 3.28 we also introduced the following new functionality when working with FileGeodatabases:

FileGeodatabase management tools

QGIS 3.28 introduces a whole range of GUI based tools for managing FileGeodatabases. To create a brand new FileGeodatabase, you can now right click on a directory from the QGIS Browser panel and select New – ESRI FileGeodatabase:

After creating your new database, a right click on its entry will show a bunch of available options for managing the database. These include options for creating new tables, running arbitrary SQL commands, and database-level operations such as compacting the database:

You’re also able to directly import existing data into a FileGeodatabase by simply dragging and dropping layers onto the database!

Expanding out the GDB item will show a list of layers present in the database, and present options for managing the fields in those layers. Alongside field creation, you can also remove and rename existing fields.

Field domain handling

QGIS 3.28 also introduces a range of GUI tools for working with field domains inside FileGeodatabases. (GeoPackage users also share in the love here — these same tools are all available for working with field domains inside this standard format too!) Just right click on an existing FileGeodatabase (or GeoPackage) and select the “New Field Domain” option. Depending on the database format, you’ll be presented with a list of matching field domain types:

Once again, you’ll be guided through a user-friendly dialog allowing you to create your desired field domain!

After field domains have been created, they can be assigned to fields in the database by right-clicking on the field name and selecting “Set Field Domain”:

Field domains can also be viewed and managed by expanding out the “Field domains” option for each database.

Relationship discovery

Another exciting addition in QGIS 3.28 (and the underlying GDAL 3.6 release) is support for discovering database relationships in FileGeodatabases! (Once again, GeoPackage users also benefit from this, as we’ve implemented full support for GeoPackage relationships via the “Related Tables Extension“).

Expanding out a database containing any relationships will show a list of all discovered relationships:

(You can view the full description and details for any of these relationships by opening the QGIS Browser “Properties” panel).

Whenever QGIS 3.28 discovers relationships in the database, these related tables will automatically be added to your project whenever any of the layers which participate in the relationship are opened. This means that users get the full experience as designed for these databases without any manual configuration, and the relationships will “just work”!

Dataset Grouping

Lastly, we’ve improved the way layers from FileGeodatabases are shown in QGIS, so that layers are now grouped according to their original dataset groupings from the database structure:

Edit ArcGIS Online / Feature Service layers

While QGIS has had read-only support for viewing and working with the data in ArcGIS Online (AGOL) vector layers and ArcGIS Server “feature service” layers for many years, we’ve added support for editing these layers in QGIS 3.28. This allows you to take advantage of all of QGIS’ easy to use, powerful editing tools and directly edit the content in these layers from within your QGIS projects! You can freely create new features, delete features, and modify the shape and attributes of existing features (assuming that your user account on the ArcGIS service has these edit permissions granted, of course). This is an exciting addition for anyone who has to work often with content in ArcGIS services, and would prefer to directly manipulate these layers from within QGIS instead of the limited editing tools available on the AGOL/Portal platforms themselves.

This new functionality will be available immediately to users upon upgrading to QGIS 3.28 — any users who have been granted edit capabilities for the layers will see that the QGIS edit tools are all enabled and ready for use without any further configuration on the QGIS client side.

Filtering Feature Service layers

We’ve also had the opportunity to introduce filter/query support for Feature Service layers in QGIS 3.28. This is a huge performance improvement for users who need to work with a subset of a features from a large Feature Service layer. Unfortunately, due to the nature of the Feature Service protocol, these layers can often be slow to load and navigate on a client side. By setting a SQL filter to limit the features retrieved from the service the performance can be dramatically increased, as only matching features will ever be requested from the backend server. You can use any SQL query which conforms to the subset of SQL understood by ArcGIS servers (see the Feature Service documentation for examples of supported SQL queries).

What’s next?

While QGIS 3.28 is an extremely exciting release for any users who need to work alongside ESRI software, we aren’t content to rest here! The exciting news is that in QGIS 3.30 we’ll be introducing a GUI driven approach allowing users to create new relationships in their FileGeodatabase (and GeoPackage!) databases.

At North Road we’re always continuing to improve the cross-vendor experience for both ESRI and open-source users through our continued work on the QGIS desktop application and our SLYR conversion suite. If you’d like to chat to us about how we can help your workplace transition from a fully ESRI stack to a mixed or fully open-source stack, just contact us to discuss your needs.

Our SLYR tool is the complete solution for full compatibility between ArcMap, ArcGIS Pro and QGIS. It offers a powerful suite of conversion tools for opening ESRI projects, styles and other documents directly within QGIS, and for exporting QGIS documents for use in ESRI software.

A lot has changed since our last SLYR product update post, and we’ve tons of very exciting improvements and news to share with you all! In this update we’ll explore some of the new tools we’ve added to SLYR, and discuss how these tools have drastically improved the capacity for users to migrate projects from the ESRI world to the open-source world (and vice versa).

ArcGIS Pro support

The headline item here is that SLYR now offers a powerful set of tools for working with the newer ArcGIS Pro document formats. Previously, SLYR offered support for the older ArcMap document types only (such as MXD, MXT, LYR, and PMF formats). Current SLYR versions now include tools for:

Directly opening ArcGIS Pro .lyrx files within QGIS

LYRX files can be dragged and dropped directly onto a QGIS window to add the layer to the current project. All the layer’s original styling and other properties will be automatically converted across, so the resultant layer will be an extremely close match to the original ArcGIS Pro layer! SLYR supports vector layers, raster layers, TIN layers, point cloud layers and vector tile layers. We take great pride in just how close the conversion results are to how these layers appear in ArcGIS Pro… in most cases you’ll find the results are nearly pixel perfect!

In addition to drag-and-drop import support, SLYR also adds support for showing .lyrx files directly in the integrated file browser, and also adds tools to the QGIS Processing Toolbox so that users can execute bulk conversion operations, or include document conversion in their models or custom scripts.

ArcGIS Pro map (mapx) and project (aprx) conversion

Alongside the LYRX support, we’ve also added support for the ArcGIS Pro .mapx and .aprx formats. Just like our existing .mxd conversion, you can now easily convert entire ArcGIS Pro maps for direct use within QGIS! SLYR supports both the older ArcGIS Pro 2.x project format and the newer 3.x formats.

Export from QGIS to ArcGIS Pro!

Yes, you read that correctly… SLYR now allows you to export QGIS documents into ArcGIS Pro formats! This is an extremely exciting development… for the first time ever QGIS users now have the capacity to export their work into formats which can be supplied directly to ESRI users. Current SLYR versions support conversion of map layers to .lyrx format, and exporting entire projects to the .mapx format. (We’ll be introducing support for direct QGIS to .aprx exports later this year.)

We’re so happy to finally provide an option for QGIS users to work alongside ArcGIS Pro users. This has long been a pain point for many organisations, and has even caused organisations to be ineligible to tender for jobs which they are otherwise fully qualified to do (when tenders require provision of data and maps in ArcGIS compatible formats).

ArcGIS Pro .stylx support

Alongside the other ArcGIS Pro documents, SLYR now has comprehensive support for reading and writing ArcGIS Pro .stylx databases. We’ve dedicated a ton of resources in ensuring that the conversion results (both from ArcGIS Pro to QGIS and from QGIS to ArcGIS Pro) are top-notch, and we even handle advanced ArcGIS Pro symbology options like symbol effects!

Take a look below how even very advanced ArcGIS Pro style libraries convert beautifully to QGIS symbol libraries:

ArcMap Improvements

While we’ve been focusing heavily on the newer ArcGIS Pro formats, we’ve also improved our support for the older ArcMap documents. In particular, SLYR now offers more options for converting ArcMap annotation layers and annotation classes to QGIS supported formats. Users can now convert Annotation layers and classes directly over to QGIS annotation layer or alternatively annotation classes can be converted over to the OGC standard GeoPackage format. When exporting annotation classes to GeoPackage the output database is automatically setup with default styling rules, so that the result can be opened directly in QGIS and will be immediately visualised to match the original annotation class.

Coming soon…

While all the above improvements are already available for all SLYR license holders, we’ve got many further improvements heading your way soon! For example, before the end of 2022 we’ll be releasing another large SLYR update which will introduce support for exporting QGIS projects directly to ArcGIS Pro .aprx documents. We’ve also got many enhancements planned which will further improve the quality of the converted documents. Keep an eye on this blog and our social media channels for more details as they are available…

You can read more about our SLYR tool at the product page, or contact us today to discuss licensing options for your organisation.

Thanks to the recent popularity of the “30 Day Map Challenge“, the month of November has become synonymous with beautiful maps and cartography. During this November we’ll be sharing a bunch of tips and tricks which utilise some advanced QGIS functionality to help create beautiful maps.

One technique which can dramatically improve the appearance of maps is to swap out rectangular inset maps for more organic shapes, such as circles or ovals.

Back in 2020, we had the opportunity to add support for directly creating circular insets in QGIS Print Layouts (thanks to sponsorship from the City of Canning, Australia!). While this functionality makes it easy to create non-rectangular inset maps the steps, many QGIS users may not be aware that this is possible, so we wanted to highlight this functionality for our first 30 Day Map Challenge post.

Let’s kick things off with an example map. We’ve shown below an extract from the 2032 Brisbane Olympic Bid that some of the North Road team helped create (on behalf of SMEC for EKS). This map is designed to highlight potential venues around South East Queensland and the travel options between these regions:

Circles featured heavily in previous Olympic bid maps (such as Budapest) where we took our inspiration from. This may, or may not, play a part in using the language of the target map audience – think Olympic rings!

Budapest Olympics 2024 Masterplan

Step by Step Guide to Creating a Circle Inset

Firstly, prepare a print layout with both a main map and an inset map. Make sure that your inset map is large enough to cover your circular shape:

From the Print Layout toolbar, click on the Add Shape button and then select Add Ellipse:

Draw the ellipse over the middle of your inset map (hint: holding down Shift while drawing the ellipse will force it to a circular shape!). If you didn’t manage to create an exact circle then you can manually specify the width and height in the shape item’s properties. For this one, we went with a 50mm x 50mm circle:

Next, select the Inset Map item and in its Item Properties click on the Clipping Settings button:

In the Clipping Settings, scroll down to the second section and tick the Clip to Item box and select your Ellipse item from the list. (If you have labels shown in your inset map you may also want to check the “force labels inside clipping shape” option to force these labels inside the circle. If you don’t check this option then labels will be allowed to overflow outside of the circle shape.)

Your inset map will now be bound to the ellipse!

Here’s a bit more magic you could add to this map – in the Main Map’s properties, click on Overviews and set create one for the Inset map – it will nicely show the visible circular area and not the rectangle!

Bonus Points: Circular Title Text!

For advanced users, we’ve another fun tip…and when we say fun, we mean ‘let’s play with radians’! Here we’re going to create some title text and a wedged background which curves around the outside of our circular inset. This takes some fiddly playing around, but the end result can be visually striking! Here we’re going to push the QGIS print layout “HTML” item to create some advanced graphics, so some HTML and CSS coding experience is advantageous. (An alternative approach would be to use a vector illustration application like Inkscape, and add your title and circular background as an SVG item in the print layout).

We’ll start by creating some curved circular text:

First, add a “HTML frame” to your print layout:

HTML frames allow placement of dynamic content in your layouts, which can use HTML, CSS and JavaScript to create graphical components.

In the HTML item’s “source” box, add the following code:

<svg height="300" width="350">
        <defs>
            <clipPath id="circleView">
                <circle id="curve" cx="183" cy="156" r="25" fill="transparent" />
            </clipPath>
        </defs>
        <path id="forText" d="M 28,150, C 25,50, 180,-32,290,130" stroke="" fill="none"/>
            <text x="0" y="35" width="100">
                <textpath xlink:href="#forText">
                    <tspan font-weight="bold" fill="black">Place text here</tspan>
                </textpath>
            </text>
             <style>
    <![CDATA[
      text{
        dominant-baseline: hanging;
        font: 20px Arial;
      }
    ]]>
  </style>
</svg>

Now, let’s add in a background to bring more focus onto the title!

To add in the background, create another HTML item. We’ll again create the arc shape using an SVG element, so add the following code into the item’s source box:

<svg width="750" height="750" xmlns="http://www.w3.org/2000/svg">
  <path d="M 90 70
           A 56 56, 0, 0, 0, 133 140
           L 150 90 Z" fill="#414042" transform=" scale(2.1) rotate(68 150 150) " />/>
</svg>

(You can read more about SVG  curves and arcs paths over at MDN)

So there we go! These two techniques can help push your QGIS map creations further and make it easier to create beautiful cartography directly in QGIS itself. If you found these tips useful, keep an eye on this blog as we post more tips and tricks over the month of November. And don’t forget to follow the 30 day Map Challenge for a smorgasbord of absolutely stunning maps.

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.

We have recently been working for the French Space Agency ( CNES ) who needed to store and visualize satellite rasters in a cloud platform. They want to access the image raw data, with no transformation, in order to fullfill deep analysis like instrument calibration. Using classic cartographic server standard like WMS or TMS is not an option because those services transform datasets in already rendered tiles.

We chose to use a quite recent format managed by GDAL, the COG (Cloud Optimize Geotiff) and target OVH cloud platform for it provides OpenStack, a open source cloud computing platform.

How it works

A COG file is a GEOTiff file which inner structure is tiled, meaning that the whole picture is divided in fixed size tile (256 x 256 pixels for instance) so you can efficiently retrieve parts of the raster. In addition to the HTTP/1.1 standard feature range request, it is possible to get specific tiles of an image through the network without downloading the entire raster.

We used a service provided by OpenStack, called Object Storage to serve the COG imagery. Object storage allows to store and retrieve file as objects using HTTP GET/POST requests.

Why not WCS ?

Web Coverage Service standard could have been an option. A WCS server can serve raw data according to a given geographic extent. It’s completely possible to deploy a container or a VPS (Virtual Private Server) running a WCS Server in a cloud plateform. The main advantages of the COG solution over WCS Server is that you don’t have to deal with the burden of deploying a server, like giving it ressources, configuring load balancing, handle updates, etc…

The beauty of COG solution is its simplicity. It is only HTTP requests, and everything else (rendering for instance) is done on the client side.

Step by step

Here are the different steps you’d have to go through if you’re willing to navigate in a big raster image directly from the cloud.

First, let’s generate a COG file

gdal_translate inputfile.tif cogfile.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATE

Install your openstack-client, it can be achieved easily with Python pip install command line

$ pip install python-openstackclient

Next, configure your openstack client in order to generate an athentification token. To do so you need to download your project specific openrc file to setup your environment)

$ source myproject-openrc.sh
Please enter your OpenStack Password for project myproject as user myuser:
**********
$ openstack token issue                                 
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                                                                   |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| expires    | 2020-07-21T08:15:12+0000                                                                                                                                                                |
| id         | xxxx_my_token_xxxx
| project_id | 97e2e750f1904b41b76f80a50dabde0a                                                                                                                                                        |
| user_id    | 18f7ccaf1a2d4344a4e35f0d84eb065e                                                                                                                                                        |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

You are now good to push you COG file to the cloud instance

openstack object create MyContainer cogfile.tif --name cogfile.tif

Before starting QGIS, 2 environment variables need to be set.  (replace xxxx_my_token_xxxx with the token you’d just come to generate)

$ export SWIFT_AUTH_TOKEN=xxxx_my_token_xxxx
$ export SWIFT_STORAGE_URL=https://storage.sbg.cloud.ovh.net/v1/AUTH_$OS_PROJECT_ID

It can also be done directly from the QGIS Python console by setting those variable using the os.environ.

Finally, add a cloud raster data source in in QGIS

You can now navigating into your image directly reading it from the cloud

© CNES 2018, Distribution Airbus DS

Performances

While panning in the map, QGIS will download only few tiles from the image in order to cover the view extent. The display latency that you could see in the video depends essentially on:

• The number of band of your image
• The pixel size
• Your internet connection (mine, the one use for the video, is not an awesome one)

Note that the white flickering that you could see when you move in the map and the raster is refreshed should be removed in next version of QGIS according to this QEP.

What’s next ?

Thanks so much to the GDAL and QGIS contributors for adding such a nice feature ! It brings lots of possibilities for organizations that have to deal with great number of big raster and just want to explore part of it.

We are already thinking about further improvments (ease authentification, better integration with processing…), so if you’re willing to fund them or just want to know more about QGIS, feel free to contact us at infos+data@oslandia.com. And please have a look at our support offering for QGIS.

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.