Latest news will appear here soon.

Tag: qgis

‘Add to Felt’ QGIS Plugin

The gift economy of Open Source is community driven and filled by folks with ideas that just go for it!

We at North Road are blessed that we get to join these creatives on their journey in order to get their products to you. Recently, the first QGIS flagship sponsor, Felt, engaged us to further strengthen their support for the up to 600,000 daily QGIS users to integrate their workflows between QGIS and Felt.

The result is the “Add to Felt” QGIS Plugin, which makes it super-simple to publish your QGIS maps to the Felt platform.

To get started, install the Add to Felt Plugin from the QGIS Plugin manager.

If you don’t have a free Felt account, you’ll need to sign up for one online (or from the Add to Felt plugin itself once you have installed it).

Within QGIS, users can easily publish their maps and layers to Felt. You can either:

  • Publish a single layer by right-clicking the layer and selecting “Share Layer to Felt” from the Export sub-menu
  • Publish your whole QGIS project/map by selecting the Project Menu, Export, “Add to Felt” action

Whilst Felt is loading up your map, you can continue working and it will let you know once your map is ready to open on Felt and share with others.

We are happy to let you know that the collaboration does not stop there! As with our SLYR tool, there is ongoing development as the requirements of the community and technology grow.  So install the Add to Felt Plugin via the QGIS Plugin manager, and let us know where you want it to go via the Add to Felt GitHub page.

Read more about it here:

Learn More

Cesium Ecosystem Grant Win for QGIS 3D Tiles!

Success! Lutra and North Road have been rewarded a Cesium Ecosystem Grant to provide access to 3D tiles within QGIS. We will be creating the ability for users to visualise 3D Tiles in QGIS alongside other standard geospatial sources in both 3D and 2D map views.
3D Tiles Cesium integration ecosystem diagram
3D Tiles Cesium integration ecosystem
We are very excited about it, but to be included in the first cohort of awardees is also an added honour! We share this distinction with 3 other recipients:
The opportunity was brought to our attention by our friends over at Nearmap, which, along with the existence of this grant, shows how the geospatial community is working together by evolving the Open Source Economy. A movement close to our hearts and our core business. Working between commercial software and open-source, Cesium’s business model recognises the legitimacy of Open Source Software for use as a geospatial standard operating procedure by promoting openness and interoperability.
Our team of Nyall Dawson and Martin Dobias will create a new layer type, QgsTiledMeshLayer, allowing for direct access to Cesium 3D tile sources alongside the other supported geospatial layer types within QGIS. This will include visualisation of the tile data in both 3D and 2D map views (feature footprints). It will fulfill a critical need for QGIS users, permitting access to 3d data provided by their respective government agencies to work alongside all their other standard geospatial layers (vector, raster, point clouds). By making 3D Tiles a first class citizen in QGIS we help strengthen the case that those agencies should be providing their data in the Cesium format (as opposed to any proprietary alternatives).
Proposed Technical Architecture Cesium QGIS
Proposed Technical Architecture for Cesium 3D Tiles in QGIS
Here’s a breakdown of what we will be doing:
  • Develop a new QGIS layer type “QgsTiledMeshLayer”
  • Develop a parser for 3D Tiles format, supporting Batched 3D Model (with a reasonable set of glTF 2.0 features)
  • Develop a 3D renderer which dynamically loads and displays features from 3D Tiles based on appropriate 3D view level of detail. (A similar approach has already been implemented in QGIS for optimised viewing of point cloud data).
  • Develop a 2D renderer for 3D Tiles, which will display the footprints of 3D tile features in 2D QGIS map views. Just like the 3D renderer, the 2D renderer will utilise map scale information to dynamically load 3D tiles and display a suitable level of detail for the footprints.
  • Users will have full control over the appearance of the 2D footprints, with support for all of QGIS’ extensive polygon symbology options.
  • By permitting users to view the 2D footprints of features, we will promote use of Cesium 3D Tiles as a suitable source of cartographic data, eg display of authoritative building footprints supplied by government agencies in the Cesium 3D Tile format.

Through past partnerships, North Road and Lutra Consulting have developed and extended the 3D mapping functionality of QGIS. To date, all the framework for mature, performant 3D scenes including vector, mesh, raster and point cloud sources are in place. We are now ready to extend the existing functionality with Cesium 3D tiles support as QGIS 3D engine already implements most of the required concepts, such as out of core rendering and hierarchical level of detail (tested with point clouds with billions of points).

So there we go! Working together collaboratively with Lutra Consulting on another great addition to QGIS 3D Functionality thanks to Cesium Ecosystem Grants. Stay tuned on our social channels to find out when it will be released in QGIS.

Cesium Ecosystem grant Badge

 

Learn More

SLYR Update — June 2023

Welcome back, SLYR enthusiasts! We’re thrilled to share the latest updates and enhancements for our SLYR ESRI to QGIS Compatibility Suite that will dramatically streamline the use of ESRI documents within QGIS (and vice versa!). Our team has been hard at work, expanding the capabilities of SLYR to ensure seamless compatibility between the latest QGIS and ArcGIS releases. We’ve also got some exciting news for users of the Community Edition of SLYR! Let’s dive right in and explore the exciting new features that have been added to SLYR since our previous update

Converting Raster Layers in Geodatabases

We’re pleased to announce that SLYR now offers support for converting raster layers within Geodatabases. With this update, users can effortlessly migrate their raster data from ESRI’s Geodatabases to QGIS, enabling more efficient data management and analysis.

This enhancement is only possible thanks to work in the fantastic GDAL library which underpins QGIS’ raster data support. Please ensure that you have the latest version of QGIS (3.30.3 or 3.28.7 at a minimum) to make the most of this feature.

Annotation and Graphic Layer Improvements

Text Annotations along Curves

For those working with curved annotations, we’ve got you covered! SLYR now supports the conversion of text annotations along curves in QGIS. With this enhancement, you’ll get accurate conversion of any curved text and text-along-line annotations from MXD and APRX documents. This has been a long-requested feature which we can now introduce thanks to enhancements coming in QGIS 3.32.

ArcGIS Pro Graphics Layer Support

SLYR now supports the conversion of ArcGIS Pro graphics layers, converting all graphic elements to their QGIS “Annotation Layer” equivalents. If you’ve spent hours carefully designing cartographic markup on your maps, you can be sure that SLYR will allow you to re-use this work within QGIS!

Curved text graphic conversion

Enhanced Page Layout Support

We’ve further improved the results of converting ArcGIS Pro page layouts to QGIS print layouts, with dozens of refinements to the conversion results. The highlights here include:

  • Support for converting measured grids and graticules to QGIS map grids
  • Enhanced dynamic text conversions:  Now, when migrating your projects, you can expect a smoother transition for dynamic text ensuring your layouts correctly show generated metadata and text correctly
  • Support for north arrows, grouped elements, legends and table frames.

Rest assured that your carefully crafted map layouts will retain their visual appeal and functionality when transitioning to QGIS!

Improved QGIS to ArcGIS Pro Conversions

SVG Marker Exports and Symbology Size

SLYR has introduced initial support for exporting SVG markers from QGIS to ArcGIS Pro formats. SVG graphics are a component of QGIS’ cartography, and are frequently used to create custom marker symbols. Unfortunately, ArcGIS Pro doesn’t have any native support for SVG graphics for marker symbols, instead relying on a one-off conversion from SVG to multiple separate marker graphics whenever an SVG is imported into ArcGIS Pro. Accordingly, we’ve implemented a similar logic in SLYR in order to convert SVG graphics to ArcGIS Pro marker graphics transparently whenever QGIS symbology is exported to ArcGIS. This enhancement allows for a seamless transfer of symbology from QGIS, ensuring that your converted maps retain their visual integrity.

Furthermore, the update includes support for exporting QGIS symbology sizes based on “map unit” measurements to ArcGIS Pro, resulting in ArcGIS Pro symbology which more accurately matches the original QGIS versions.

Rule-Based Renderer Conversion

The “Rule Based Renderer” is QGIS’ ultimate powerhouse for advanced layer styling. It’s extremely flexible, thanks to its support for nested rules and filtering using QGIS expressions. However, this flexibility comes with a cost — there’s just no way to reproduce the same results within ArcGIS Pro’s symbology options! Newer SLYR releases will now attempt to work around this by implementing basic conversion of QGIS rule-based renderers to ArcGIS Pro layers with “display filters” attached. This allows us to convert some basic rule-based configuration to ArcGIS Pro formats.

There’s some limitations to be aware of:

  1. Only “flat” rule structures can be converted. It’s not possible to convert a nested rule structure into something representable by ArcGIS Pro.
  2. While the QGIS expression language is very rich and offers hundreds of functions for use in expressions, only basic QGIS filter expressions can be converted to ArcGIS Pro rules.

Improved Conversion of Raster and Mesh Layers

Based on user feedback, we’ve made significant improvements to the conversion of QGIS rasters and mesh layers to ArcGIS Pro formats. Expect enhanced accuracy when migrating these types of data, ensuring a closer match between your QGIS projects and their ArcGIS Pro equivalents.

New tools

The latest SLYR release introduces some brand new tools for automating your conversion workflows:

Convert LYR/LYRX Files Directly to SLD

To facilitate interoperability, SLYR has introduced algorithms that directly convert ESRI LYR or LYRX files to the “SLD” format (Styled Layer Descriptor). This feature simplifies the process of sharing and utilizing symbology between different GIS software, allowing for direct conversion of ESRI symbology for use in Geoserver or Mapserver.

Convert File Geodatabases to Geopackage

We’re thrilled to introduce a powerful new tool in SLYR that enables a comprehensive conversion of a File Geodatabase to the Geopackage format. With this feature, you can seamlessly migrate your data from ESRI’s File Geodatabase format to the versatile and widely supported GeoPackage format. As well as the raw data conversion, this tool also ensures the conversion of field domains and other advanced Geodatabase functionality to their GeoPackage equivalent, preserving valuable metadata and maintaining data integrity throughout the transition. (Please note that this tool requires QGIS version 3.28 or later.)

 

All these exciting additions to SLYR are available today to SLYR license holders. If you’re after one-click, accurate conversion of projects from ESRI to QGIS, contact us to discuss your licensing needs.

As described on our SLYR page, we also provide some of the conversion tools for free use via the SLYR “Community Edition”. We’re proud to announce that we’ve just hit the next milestone in the Community Edition funding, and will now be releasing all of SLYR’s support for raster LYR files to the community edition! This complements the existing support for vector LYR files and ESRI style files available in the community edition. For more details on the differences between the licensed and community editions, see our product comparison.

Learn More

[Blog] View and track changes in QGIS

With the recent changes to the Mergin Maps plugin for QGIS, you can visualise the local changes before synchronising your data. Read more in our blog post!
Learn More

How to contribute to QField

QField is a community-driven open-source project. It is free to share, use and modify and it will stay like that. The very essence of a community is to help and support each other. And that’s where YOU come into play. To make it work we need your support!

For those who don’t know much about the concept of open source projects, a bit of background. Investing in open-source projects is a technical and ethical decision for OPENGIS.ch. Open source is a technological advantage, as we receive input from many developers worldwide who are motivated to work out the best possible software. It prevents our customers from vendor lock-in and allows complete ownership and control of the developed software. And finally, not only financially independent businesses and people should benefit from professional software but also those who might not have the financial means to pay for features, and licences.

You are not a developer, but you still like to use QField and support it? Good news. You don’t have to be a developer to use, contribute or recommend the app. There are plenty of things that need to be done to help QField to remain the powerful software it is right now and become even better. Here are a few suggestions on how you can give something back.

  1. Review the app ★★★★★ in google’s play store or apple’s app store .
  2. Let the world know about it! It doesn’t matter if you’re on Twitter, LinkedIn, Instagram or any other social media platform. Show and tell about where QField helped you. We appreciate every post and we promise to like, share and comment.
  3. Write about your experience and please let us know. Be it in your blog or as a new success story. Insights into field projects are extremely valuable. It helps us to make the app even more efficient for your work, and it helps others to understand the range of applications for QField.
  4. Register for a paid QFieldCloud account . QFieldCloud allows to synchronize and merge the data collected in QField. QFieldCloud is hosted by the makers of QField and by getting an account you help QField too.

Do you want to do something that is more hands-on and directly linked to the app? No problem.

  1. Help with the documentation . You can document features, or improve the documentation in English. Read the how-to guide to get started.
  2. And if you are multilingual you might consider translating the documentation or the app in your language .
  3. Become a beta tester and be the first to report a bug! When something doesn’t work properly it might be a bug. The quicker we know about it, the faster it can be resolved.
  4. You can ask and answer questions on gis.stackexchange and help others on the user discussions platform .
  5. If you are a developer and you want to get involved in QField development, please refer to the individual documentation for QField , QFieldCloud and QFieldSync .

And now finally for those of you who have the financial means, you can either sponsor a feature or subscribe to one of the monthly sponsorships . By doing so you help get freshly baked QField versions straight to everyone’s devices.

Nothing in it for you? In that case, just drop by to say thank you or have a hot or cold beverage with us next time you meet OPENGIS.ch at a conference and you might make our day!
Want to know more about the idea of community-driven open-source projects and the QGIS project in particular? Check out Nyall Dawson’s blog post about how to effectively get things done in open source !

Learn More

QGIS 3.28 improvements for working with ESRI formats and services

The QGIS 3.28 release is an extremely exciting release for all users who work in mixed software workplaces, or who need to work alongside users of ESRI software. In this post we’ll be giving an overview of all the new tools and features introduced in 3.28 which together result in a dramatic improvement in the workflows and capabilities in working with ESRI based formats and services. Read on for the full details…!

Before we begin, we’d like to credit the following organisations for helping fund these developments in QGIS 3.28:

  • Naturstyrelsen, Denmark
  • Provincie Gelderland, Netherlands
  • Uppsala Universitet, Department of Archaeology and Ancient History
  • Gemeente Amsterdam
  • Provincie Zuid-Holland, Netherlands

FileGeodatabase (GDB) related improvements

The headline item here is that QGIS 3.28 introduces support for editing, managing and creating ESRI FileGeodatabases out of the box! While older QGIS releases offered some limited support for editing FileGeodatabase layers, this required the manual installation of a closed source ESRI SDK driver… which unfortunately resulted in other regressions in working with FileGeodatabases (such as poor layer loading speed and random crashes). Now, thanks to an incredible reverse engineering effort by the GDAL team, the open-source driver for FileGeodatabases offers full support for editing these datasets! This means all QGIS users have out-of-the-box access to a fully functional, high-performance read AND write GDB driver, no further action or trade-offs required.

Operations supported by the GDAL open source driver include:

  • Editing existing features, with full support for editing attributes and curved, 3D and measure-value geometries
  • Creating new features
  • Deleting features
  • Creating, adding and modifying attributes in an existing layer
  • Full support for reading and updating spatial indexes
  • Creating new indexes on attributes
  • “Repacking” layers, to reduce their size and improve performance
  • Creating new layers in an existing FileGeodatabase
  • Removing layers from FileGeodatabases
  • Creating completely new, empty FileGeodatabases
  • Creating and managing field domains

On the QGIS side, the improvements to the GDAL driver meant that we could easily expose feature editing support for FileGeodatabase layers for all QGIS users. While this is a huge step forward, especially for users in mixed software workplaces, we weren’t happy to rest there when we  had the opportunity to further improve GDB support within QGIS!

So in QGIS 3.28 we also introduced the following new functionality when working with FileGeodatabases:

FileGeodatabase management tools

QGIS 3.28 introduces a whole range of GUI based tools for managing FileGeodatabases. To create a brand new FileGeodatabase, you can now right click on a directory from the QGIS Browser panel and select New – ESRI FileGeodatabase:

After creating your new database, a right click on its entry will show a bunch of available options for managing the database. These include options for creating new tables, running arbitrary SQL commands, and database-level operations such as compacting the database:

You’re also able to directly import existing data into a FileGeodatabase by simply dragging and dropping layers onto the database!

Expanding out the GDB item will show a list of layers present in the database, and present options for managing the fields in those layers. Alongside field creation, you can also remove and rename existing fields.

Field domain handling

QGIS 3.28 also introduces a range of GUI tools for working with field domains inside FileGeodatabases. (GeoPackage users also share in the love here — these same tools are all available for working with field domains inside this standard format too!) Just right click on an existing FileGeodatabase (or GeoPackage) and select the “New Field Domain” option. Depending on the database format, you’ll be presented with a list of matching field domain types:

Once again, you’ll be guided through a user-friendly dialog allowing you to create your desired field domain!

After field domains have been created, they can be assigned to fields in the database by right-clicking on the field name and selecting “Set Field Domain”:

Field domains can also be viewed and managed by expanding out the “Field domains” option for each database.

Relationship discovery

Another exciting addition in QGIS 3.28 (and the underlying GDAL 3.6 release) is support for discovering database relationships in FileGeodatabases! (Once again, GeoPackage users also benefit from this, as we’ve implemented full support for GeoPackage relationships via the “Related Tables Extension“).

Expanding out a database containing any relationships will show a list of all discovered relationships:

(You can view the full description and details for any of these relationships by opening the QGIS Browser “Properties” panel).

Whenever QGIS 3.28 discovers relationships in the database, these related tables will automatically be added to your project whenever any of the layers which participate in the relationship are opened. This means that users get the full experience as designed for these databases without any manual configuration, and the relationships will “just work”!

Dataset Grouping

Lastly, we’ve improved the way layers from FileGeodatabases are shown in QGIS, so that layers are now grouped according to their original dataset groupings from the database structure:

Edit ArcGIS Online / Feature Service layers

While QGIS has had read-only support for viewing and working with the data in ArcGIS Online (AGOL) vector layers and ArcGIS Server “feature service” layers for many years, we’ve added support for editing these layers in QGIS 3.28. This allows you to take advantage of all of QGIS’ easy to use, powerful editing tools and directly edit the content in these layers from within your QGIS projects! You can freely create new features, delete features, and modify the shape and attributes of existing features (assuming that your user account on the ArcGIS service has these edit permissions granted, of course). This is an exciting addition for anyone who has to work often with content in ArcGIS services, and would prefer to directly manipulate these layers from within QGIS instead of the limited editing tools available on the AGOL/Portal platforms themselves.

This new functionality will be available immediately to users upon upgrading to QGIS 3.28 — any users who have been granted edit capabilities for the layers will see that the QGIS edit tools are all enabled and ready for use without any further configuration on the QGIS client side.

Filtering Feature Service layers

We’ve also had the opportunity to introduce filter/query support for Feature Service layers in QGIS 3.28. This is a huge performance improvement for users who need to work with a subset of a features from a large Feature Service layer. Unfortunately, due to the nature of the Feature Service protocol, these layers can often be slow to load and navigate on a client side. By setting a SQL filter to limit the features retrieved from the service the performance can be dramatically increased, as only matching features will ever be requested from the backend server. You can use any SQL query which conforms to the subset of SQL understood by ArcGIS servers (see the Feature Service documentation for examples of supported SQL queries).

 

What’s next?

While QGIS 3.28 is an extremely exciting release for any users who need to work alongside ESRI software, we aren’t content to rest here! The exciting news is that in QGIS 3.30 we’ll be introducing a GUI driven approach allowing users to create new relationships in their FileGeodatabase (and GeoPackage!) databases.

At North Road we’re always continuing to improve the cross-vendor experience for both ESRI and open-source users through our continued work on the QGIS desktop application and our SLYR conversion suite. If you’d like to chat to us about how we can help your workplace transition from a fully ESRI stack to a mixed or fully open-source stack, just contact us to discuss your needs.

Learn More

24th Contributors QGIS Meeting in Firenze 2022

The international community of QGIS contributors got together in person from 18 to 22 August in parallel to OpenStreetMap State of The Map event and right before the FOSS4G . So there was a lot of open source geo power concentrated in the beautiful city of Florence in those days. It was my first participation and all I knew was that it’s supposed to be an unconference. This means, there is no strict schedule but space and opportunity for everyone to present their work or team up to discuss and hack on specific tasks to bring the QGIS project to the next level.

Introduction and first discussions

We were a group of six OPENGIS.ch members arriving mostly on Thursday, spending the day shopping and moving into our city apartment. In the evening we went to a Bisteccheria to eat the famous Fiorentina steak. It was big and delicious as was the food in general. Though, I am eating vegetarian since to compensate. On Friday we went to the Campus to meet the other contributors. After a warm welcome by the organizer, Rossella and our CEO and QGIS chair Marco Bernasocchi we did an introduction round where everyone mentioned their first QGIS version ever used. At this point, I became aware of the knowledge and experience I was sharing the room with. Besides this, I noticed that there was another company attending with several members, namely Tim Sutton’s Kartoza, which is also contributing a lot to QGIS. The first discussion was about QGIS funding model, vision, communication and on the new website in planning. This discussion then moved into some smaller groups including most of the long term contributors. I was looking around, physically and virtually, and tried to process all the new inputs and to better understand the whole QGIS world. Besides, I noticed my colleague Ivan having problems with compiling QGIS after upgrading to Ubuntu 22.04 which then motivated my other colleague Clemens to implement a docker container to do the compilation. Nevertheless, I postponed my Ubuntu upgrade. That evening we went out all together to have a beer or two and play some pool sessions and table football. Finally, the OPENGIS.ch crew navigated back home pairing a high-precision GNSS sensor with a mobile device running OpenStreetMap in QField . We arrived back home safely and super precise.

First tasks and coffee breaks

There was catering in the main hall covering breakfast, lunch and coffee breaks. It never took long after grabbing a cup of coffee to find yourself in a conversation with either fellow contributors or OpenStreetMap folks. I chatted with a mapper from Japan about mobile apps, an engineer from Colombia about travelling and a freelancer from the Netherlands about GDAL, to name 3 coffees out of many.

QGIS plugins website

After some coffee, Matthias Kuhn, our CTO and high-ranking QGIS contributor, asked me whether I could improve some ugly parts of QGIS plugins website . So I had my first task which I started working on immediately. The task was to make the site more useful on mobile devices which would be achieved by collapsing some unimportant information and even removing other parts. I noticed some quirks in the development workflow, so I also added some pre-commit hooks to the dev setup. Dimas Ciputra from Kartoza helped me finalize the improvements and merge them into master branch on github .

QGIS website downloads section

Regis Haubourg asked to help simplify the QGIS Downloads for Windows section on the main QGIS website. We played around in the browser dev tools until we thought the section looked about right. I then checked out the github repo and started implementing the changes. I need to say the tech stack is not quite easy to develop with currently, but there is a complete rework in planning. Anyway, following the pull request on github a lively discussion started which is ongoing by the time of writing. And this is a good thing and shows how much thought goes into this project.

Presentations

There were many interesting and sometimes spontaneous presentations which always involved lively discussions. Amy Burness from Kartoza presented new styling capabilities for QGIS, Tobias Schmetzer from the Bavarian Center for Applied Energy Research presented the geo data processing and pointed out issues he encountered using QGIS on this and Etienne Trimaille from 3liz talked about qgis-plugins-ci , just to name a few.

Amazing community

On Saturday evening a bus showed up at the campus and took us on a trip up to the hills. After quite a long ride we arrived at a restaurant high up with mind-blowing view of the city. I forgot how many rounds of Tuscan food were served, but it was delicious throughout. An amazing evening with fruitful conversations and many laughs.

The weather was nice and hot, the beers cold, the Tuscan food delicious and the contributors were not only popular Github avatars but really nice people. Thank you QGIS.

Learn More

A New Trick up QField’s Sleeve: Animated Maps

Starting with QField 2.2, users can fully rely on animation capabilities that have made their way into QGIS during its last development cycle. This can be a powerful mean to highlight key elements on a map that require special user attention.

The example below demonstrates a scenario where animated raster markers are used to highlight active fires within the visible map extent. Notice how the subtle fire animation helps draw viewers’ eyes to those important markers.

Animated raster markers is a new symbol layer type in QGIS 3.26 that was developed by Nyall Dawson . Supported image formats include GIF, WEBP, and APNG.

The second example below showcases more advanced animated symbology which relies on expressions to animate several symbol properties such as marker size, border width, and color opacity. While more complex than simply adding a GIF marker, the results achieved with data-defined properties animation can be very appealing and integrate perfectly with any type of project.

https://player.vimeo.com/video/732691644

You’ll quickly notice how smooth the animation runs. That is thanks to OPENGIS.ch’s own ninjas having spent time improving the map canvas element’s handling of layers constantly refreshing. This includes automatic skipping of frames on older devices so the app remains responsive.

Oh, we couldn’t help ourselves but take the opportunity to demonstrate how nice the QField feature form layout is these days in the video above ? To know more about other new features in QField 2.2, go and read the release page .

Happy field mapping to all!

The lovely animal markers used in the zoo example above were made by Serbian artist Arsenije Vujovic .

Learn More

Offline WMS - Benchmarking raster formats for QField

What are we looking for?

We would like to use WMS offline on QField. For that, we need to figure out what is the best way to get a raster from a WMS and which format is the most efficient (size and performance).

In this post we’ll show you is how to generate the ideal raster file from a WMS and the results of our efficiency tests for the the different raster formats.

WMS to GPKG

The simple way

If there is no limitation on the WMS or you need only a small region, here is the easiest process.

  1. Request the WMS and store a description file in XML :
gdal_translate "WMS:url" file.xml -of WMS
  1. Create a Geopackage from the information in the description file.
gdal_translate -of GPKG file.xml file.gpkg -co TILE_FORMAT=JPEG

That was quite simple, right?

The larger datasets way

If the command takes too much time, it means that it is trying to download too much data and could be caused by downloading higher resolution data than required.
The command might even completely fail if it contains a request for bigger data blocks thant the server allows.

Here is the process to get larger datasets in a simple way. Let’s use a real example:

  1. Use gdal_translate "WMS:https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv?request=getmap&service=wms&crs=EPSG:4326&format=image/jpeg&layers=gebco_latest&version=1.1.0" test.xml -of WMS
  2. Open the test.xml file for editing, here you’ll find the parameters of the WMS. We change the “SizeX” to 3600 and “SizeY” to 1800. By changing these parameters we lower the resolution. It is important to keep proportionality.
  3. Another thing we need to change are “BlockSizeX” and “BlockSizeY” that define the size of the tiles. We change both to 2048.
  4. Finally, use gdal_translate -of GPKG test.xml test.gpkg -co TILE_FORMAT=JPEG
  5. To make a Geopackage pyramid use gdaladdo GPKG:test.gpkg:gebco_latest. It will replace the Geopackage, if you want to keep the original one, you need to copy it first.

Now you have a raster Geopackage that you can use in QField.

Testing raster formats

Preparing the files

As first step we exported our test orthophoto WMS to a plain GeoTIFF using QGIS’ default behaviour.

Formatgdal_translategdaladdogpkg JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEG gpkg PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgpkg PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgpkg PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gpkg WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgpkg pyramid_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEGgdaladdo GPKG:C:\test\test_JPEG.gpkg:test_gpkg_JPEG gpkg pyramid_PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgdaladdo GPKG:C:\test\test_PNG.gpkg:test_gpkg_PNGgpkg pyramid_PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgdaladdo GPKG:C:\test\test_PNG_JPEG.gpkg:test_gpkg_PNG_JPEGgpkg pyramid_PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gdaladdo GPKG:C:\test\test_PNG8.gpkg:test_gpkg_PNG8gpkg pyramid_WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgdaladdo GPKG:C:\test\test_WEBP.gpkg:test_gpkg_WEBPJPEG2000gdal_translate -of JP2OpenJPEG “C:\test\ortho_test.tif” “C:\test\test_jpeg_2000.jpg"COG DEFLATEgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATECOG_JPEGgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog_JPEG.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=JPEGtifIn QGIS right click on the layer > export > save as > (see the details in the picture under the table)MBTgdal_translate -of MBTILES “C:\test\ortho_test.tif” “C:\test\test_mbt.mbtiles"Creation commands for all the tested formats

Rendering test results

We have tested many formats, here is a table with the results of the size and rendering speed in QGIS and QField.
To analyze the speed we used qgis_bench.exe -i 10 -p "C:\test\test.qgs" >> "C:\test\test.log.
Qgis_bench is a tool that renders a QGIS project a number of times to get performance measurements. The parameter -i is to define the iterations and -p is the project used which contains only the generated raster.

FormatExtent [m]File size [GB]Total_avgTotal_maxdevTotal_minTotal_stdevgpkg JPEG52'880/29'2300.4250.242255.7815.539244.984gpkg PNG52'880/29'2302.9412.002490.328152.142259.859gpkg PNG_JPEG52'880/29'2300.4250.125256.8756.750245.172gpkg PNG852'880/29'2301.4283.875296.40612.625271.250gpkg WEBP52'880/29'2300.3330.238348.10973.534256.703gpkg pyramid_JPEG52'880/29'2300.51.0093.4062.3970.688gpkg pyramid_PNG52'880/29'2303.01.2083.2812.0730.688gpkg pyramid_PNG_JPEG52'880/29'2300.61.4914.3442.8531.016gpkg pyramid_PNG852'880/29'2301.61.5084.3752.8670.969gpkg pyramid_WEBP52'880/29'2300.41.3334.9063.5730.766JPEG200052'880/29'2301.113.888136.109122.2220.219COG DEFLATE52'880/29'2303.6264.427273.09425.411239.016COG_JPEG52'880/29'2301.014.778131.172116.3941.734tif52'880/29'2306.42.3676.7344.3671.672MBT52'880/29'2304.40.4694.6414.1710Comparison of file size and rendering speed of different raster formats. “Total” columns are rendering times in [s]. Lower file size is more storage friendly, lower Total_avg is more performant.

Analysis

File size

The Geopackage WEBP (with and without pyramid) has the best result for file size, but it is not _yet_supported by QField (from 1.6) and is only slightly smaller than the JPEG variant.

Plain GeoTiff, MBTiles, Cloud Optimized GeoTIFF (COG - DEFLATE mode) and Geopackages with PNG generate by far the largest file sizes (up to 20x larger) and are thus not recommended.

Rendering speed

MBTiles are on average double as fast as JPEG Geopackages with pyramids which in turn are more than double as fast as GeoTIFF and 15x faster than COG.
Geopackages without pyramids are 200 to 400 times slower.

Conclusion

Even though MBTiles render faster than the Geopackage pyramid JPEG, they come with an almost 10x bigger storage requirement which makes us say that the best offline raster format supported by QField is Geopackage pyramid JPEG or if you need transparency and slightly smaller files Geopackage pyramid WebP.

If you need transparency before QField 1.6, the best results are achieved with Geopackage pyramid PNG_JPEG.

Learn More

QField 1.4 released - Happy new year

What a year’s start! After a very packed December publishing all the QGIS on the road videos and quietly releasing QField 1.3 - Ben Nevis we could have gone and relaxed over the holidays. But since we love QField so much we immediately started working on the next iteration. Now, after an intensive testing period, we are proud to announce the release of QField 1.4 - Olavtoppen.

Olavtoppen!? yes, the highest point of Bouvet Island, the remotest island on Earth. And sure enough, QField would follow you there!

As usual, get it on play store or download it from GitHub.

QField Crowdfunding Campaign

Before digging into all the new goodness that you will find in QField 1.4, let’s get a big “Thanks” out to everybody who supported our crowdfunding campaign for improved camera support and all our customers that agreed to open source the work we did for them.

If you like QField, want a new feature or would like to support the project, don’t hesitate to get in touch with us.

Usability enhancements

In QField 1.2 we started to improve on the usability of the user interface. We are constantly working on this with a usability expert to get the user interface to be even more appealing and user-friendly.

Besides lots of clean-up and polishing, QField received two major improvements, a portrait mode and a new welcome screen with recent projects.

Welcome screen with recent projects

QField is all about efficiency. While favourites folders in the file selector already give a great productivity boost, very often we work with the same 3-4 projects. This is why we redesigned the welcome screen to list the last five project used. And if you look carefully you might get a hint of what will be coming soon…

Portrait mode

QField now flawlessly works in portrait mode. We heard you say you needed a comfortable way to work in portrait mode, especially on smartphones. QField forms and button placements are now optimized to be easy to use with your thumbs.

New features

We keep on listening to your feedback and prioritize new features based on it. We did implement some minor features like allowing hiding legend nodes and printing to PDF using the current extent. But this time’s superstars are three highly expected features: Splitting of geometries, compass integration and, yes you guessed right, native camera and gallery app support!

Split Features

ezgif com-optimize
ezgif com-optimize

A new editing tool is available that allows for splitting existing features. This adds an even more powerful operation to an already impressive geometry editing tools set.

Compass integration

A long-awaited feature! QField now shows you on-screen in which direction you are looking, walking, driving, flying or warping direction. This makes it much easier and more pleasant to navigate in the field.

Screenshot\_20200115-154223\_QField Nightly
Screenshot_20200115-154223_QField Nightly

It is now possible to use your favourite camera app so that you have more control over how pictures are taken. It is also possible to select pictures which are already on your device by using the new gallery selector.

Pro Tip: You can use any camera app. For example, you can use the open camera app to create geotagged photos if your preinstalled system camera doesn’t save positioning information in EXIF data.

Pro Tip 2: You can use an image annotation app to add notes, sketches, drawings and so on to your images and then choose them from QField via the add from gallery button.

Antenna Height Correction

For high precision measurements, it’s possible to compensate your altitude by a fixed antenna height. This will then automatically adjust all the digitised altitude values.

JPEG 2000

Support for JPEG 2000 raster datasets was added. This lossy format offers a compression rate at par with proprietary formats like ECW or Mr SID.

Pro Tip: save your base maps in JPEG 2000 to save storage.

New Languages

Thanks to the hard work of our community, QField is now also available in Turkish and Japanese.

New packages

You say: wow that’s a lot! We say: there is more :)
We have upgraded our whole building infrastructure so that you can comfortably get even more QField goodness without having to uninstall your production ready QField.

Automated master builds

After each pull request is merged into our master code, a new package is created and automatically published on the playstore in a dedicated app called QField for QGIS - Unstable (Early Access) . Installing this app will allow you to always have the latest build of QField for testing and giving feedback. On your device, this app is completely separated from the production-ready QField and has a distinctive black icon so that you do not confuse it.

Pull request builds

QField is an extremely active project, and as you see we develop multiple functionalities and fixes at the same time. If you’re particularly interested in one of this, our continuous integration fairy builds and publishes new packages automatically at each commit directly to the pull request you are interested in. To see what we are currently working on, have a look at the pull request overview page .

Experimental Windows builds

Last but definitely not least, we’ve set up an Azure CI infrastructure to build QField for windows. For now, we still consider this experimental but we already had some very successful testing. If you are interested in testing out QField for windows you can get it here , remember it is experimental so don’t use it in production yet and give us as much feedback as possible :)

What’s next?

As you can imagine we’ve had a very busy start of 2020, but even more is to come soon with the next releases of QField. We’d like to thank again all companies and individuals that actively use QField and that invest in making QField even better. If you feel QField misses something you need or would like to support the project, don’t hesitate to get in touch with us.

Learn More