OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Visualization of Early QGIS Development

Years ago (2011), Nathan Woodrow did a visualization of code commits between QGIS 1.6 and 1.7 using Gource. I wanted to contrast the slow beginning of QGIS in 2002 with the flurry of activity in recent years. Gource can analyze a git repository and display the activity. The video below begins with the very slow start of QGIS development in 2002. Since displaying all of QGIS activity using Gource would result in a long video no one would want to watch, I took a look at the time period from 2002 to mid-2004, then jumped to June of 2021.
Learn More

About

Gary Sherman has a wide and varied background, including that of geologist, mining engineer, software engineer, publisher, and author. He has authored both technical books and articles, as well as fiction. In 2002, he founded the popular open source QGIS project and has published several books on the topic. For eight years, Gary was at the helm of Locate Press, publisher of books on open source software. Gary has published a number of novels as G.
Learn More

(Fr) Oslandia recrute : Ingénieur(e) développement d’applications SIG ( Python / SQL / QGIS ) – OSL2110A

Sorry, this entry is only available in French.

Learn More

Scoped storage in Input for Android

Android has forced app to use Scoped Storage for all app related data. If you are using Input app on Android please read carefully for the upcoming update (1.1) of the app in the Google Play Store.

What is Scoped Storage?

In summary, apps need to use a specific part of folders on Android devices to store app related data.

Currently, Input stores your QGIS project and some other settings (e.g. grid shift projection) on /Internal storage/INPUT. With the new Android requirements, the app related data should be stored on /Internal storage/Android/data/uk.co.lutraconsulting.

Update process

For the 1.1 release of Input on Android, there will be an extra process. This process will be a on-off action. When you launch the app after the upgrade, it will copy the data from /Internal storage/INPUT to /Internal storage/Android/data/uk.co.lutraconsulting. Depending on the size of your projects, this can take a couple of minutes. During the process you will see a screen similar to this one:

Input project migration

Best practices

The upgrade and copy process should work smoothly. But we suggest to take the following actions to ensure you will not lose any data during the process:

  • Sync all you changes: before upgrading the app, open Input and sync all your local changes to Mergin.

  • Storage space: if you work with several projects and large volume of data, make sure you have enough storage. The process will make a copy of your existing /Internal storage/INPUT without deleting it. So, you need at least the size of /Internal storage/INPUT storage available.

Troubleshooting

In case you have encountered any issues, you can take the following steps to fix the problem manually:

  • Lack of storage space:
    • Transfer the data through USB cable to your PC.
    • Make a back up of data on your PC.
    • Delete /Internal storage/INPUT on your Android phone/tablet
    • Transfer the data from PC through USB to /Internal storage/Android/data/uk.co.lutraconsulting
    Input project migration - storage warning

  • Missing data: the migration process does not delete /Internal storage/INPUT folder. It will rename it /Internal storage/INPUT_migrated. Similar to step above, you can copy the data to the PC and move them to /Internal storage/Android/data/uk.co.lutraconsulting. Alternatively, you can use a file browser app on your device to copy files around.

  • If you use Input app on a shared device, the migration process will transfer all the project data from /Internal storage/INPUT and marks the folder as /Internal storage/INPUT_migrated. Therefore, when the next user starts up the app, no data will be present. To fix the issue, you need to manually move the data from /Internal storage/INPUT_migrated to /Internal storage/Android/data/uk.co.lutraconsulting (as described above) for the other users on the device.
Input project migration problem

Need help

If you need further help, please join us on the community chatroom and we will be able to help you with the upgrade issue (or other Input/Mergin related problems.)

Learn More

Pecan Precision Farming with Input App

Precision farming opens an opportunity to increase the gain for a farmer up to 20% with fewer trees.

Making a sustainable profit from farming can be something like tossing a coin. However, today the advancements in digital technology provide the incredible benefit of reducing the unpredictability of farming in general. In pecan farming in particular, the proper know-how of a combination of multiple factors such as soil composition, environmental impact, climate, irrigation and sunlight management are vital for success. Hard data is essential for innovating key decisions. Riaan Burger, a South African farmer, successfully uses open source GIS software to tackle the issues of collecting and utilising such vital data.

Riaan on Farm

Riaan Burger on his pecan farm.

Becoming a Farmer

Riaan Burger’s journey started back in 2017 when, after 20 years’ working as an electrical engineer, he decided to embark on his own business in pecan nut farming. He is a self-taught farmer and co-owner of one of the biggest pecan farms in the district of Weenen, South Africa. At first, Riaan had only a rough idea of how to improve his newly bought 24 hectares of planted orchard.

After reading books on soil structure, type, depth and chemical analyses, he had soil samples taken to create a soil sample map. In his orchard, the trees were planted in 10x10 metres geometric layouts. After 20 years, the trees were already creating shade. Riaan knew that sunlight penetration is important and that pecan trees are susceptible to fungus, so he decided to take a closer look.

“The previous owner of the farm had already started chopping out every second tree. I was in the position where I could compare the yield from areas where trees had been chopped down, to areas where they had not been felled,” explains Riaan.

Initial stage

A photo of the farm taken by drone and initial stage of farming - soil maps.

Riaan realised that understanding the ambient conditions and mapping them to yield results was the right way to go. This was the beginning of his exciting journey in precision farming.

Collecting Data from the Field

It was no easy task for Riaan to get a complete picture. For two years, he only recorded the yield per day, and the relevant orchard block. Later, he used the services of a local consulting company which provided him with surveying applications to help him to set up a basic workflow.

“I record the weight of pecan nuts from each individual tree and I plot it on the map,” Riaan says. “And then afterwards I can see the different cultivars of trees, the weight of nuts they produced and the area in which they were produced. I previously had soil samples taken, so I’ve got soil sample maps. Now I can overlay a yield map over the soil sample map and correlate them”

Despite working well with the surveying app, Riaan was still in need of assistance to prepare data tables and then to export them to Excel sheets and to request map creation. After the consulting company moved from a third-party app to the in-house build solution, things got worse for Riaan: “It was a generic tool and a difficult app to work with. Displaying data was cumbersome. I wanted something that was quick and easy again. Ideally, I would like to capture data on the iPad and have it directly linked to my laptop – that is how I think it should work.”

So Riaan discovered the Input and Mergin suite, a surveying app based on QGIS and a cloud service for data synchronisation, respectively. Despite only a basic knowledge of QGIS and the necessity to learn and experiment, the effort immediately paid off!

“I’ve got Input, I’ve got a Mergin account and I’ve got QGIS. I now have full control of the process. I don’t need to fund a consultant and find that by 8 o’clock at night he has not yet responded to my query, and so I have to back it up with a WhatsApp call,” Riaan explains. “This is something I created entirely myself. I don’t need to ask someone to change this map, or add that icon, modify the legend or change the styling, I am in control of all of it! And I think there is a sense of satisfaction when one gets things going for oneself.”

Project in QGIS Project in Input

A farming project in QGIS desktop and Input app UI.

With the Input app, Riaan has gained control of the workflow – something not before experienced. He has gained the freedom to set the project according to his own individual needs. Without much effort, not only the data but the styling, an important factor for him, are synchronised.

“In QGIS I created a style where each bubble represents the kilograms of the yield and the colour represents the variety,” says Riaan. “Then I saw in Input that there is suddenly an extra page on my iPad. So now I have the bubbles and bubble colours on the iPad instantly, which I did not have in previous apps. Previously, I had to create it the long way around, but now it is suddenly on the iPad automatically. I enjoy having that available! In the past, I only had that information by the end of the harvest.”

Cutting Down Trees

The energy invested in creating workflow, capturing and analysing data paid off handsomely for Riaan. All his initial doubts were suddenly eliminated. He has all the information essential for making any important decision right at his fingertips.

“The first time you fell trees, you have nightmares! How on earth you can be cutting your income and destroying 20 years of growth? Now, with the necessary proof at hand, it is easier in your mind to motivate that tree felling is actually your source of income, that it is the right thing to do,” says Riaan. “You can see that the yield from the remaining trees is more than double than before. The trees are healthier, because of allowing sunlight in. Moreover, you also get less pressure from fungus, because pecan trees are easily susceptible to fungus. Now that you have good ventilation, because you have a draught through your orchard, you can see on the map what your exposure to fungus is.”

Higher density of trees Lower density of trees

Orchard with higher density of trees (top) and lower density (bottom).

Riaan continues: “I am not a sole owner, I have a partner. Now I have the data available, so that when we have a meeting I can explain to him this is what I did, this is what happened and this is the result. If you go to a bank manager for a loan, I mean, he really frowns at you when you tell him you are cutting down your trees which produce your income. Then you can explain to him why it is for the better.”

Yield increase

Increase in yield per tree after reduction of tree density.

Harvest Data Analysis and Beyond

The benefits of collecting and interpreting data carefully are limitless. Cutting down trees is just one example of the many advantages of careful data gathering and interpretation.

“Now I see that certain cultivars perform better than others this year,” adds Riaan. “So I can see if they consistently perform better or worse. I can decide, based on these results, what to do: if I need to add more nitrogen or improve soil pH, or whatever the case may be.”

User-friendly tools tailor-made to specific needs, such as the Input and Mergin suite, provided Riaan a way of collecting and analysing his specific type of farming data. His insights for necessary improvements also developed over the past few years.

“At first I did not have the distinction between individual trees and I even said to my partner” continues Riaan, “if only I’d started doing this in the very first season, I would have had 5 years of data to compare – but I didn’t. I will definitely continue to use this suite. It is valuable and I have been using it only for harvest. However, I see now that I can start recording where I have cut down a tree, so I have a record of when and where the tree was cut. I can also record my irrigation data.”

In spite of initial teething problems and the difficult decisions he had to take, Riaan’s success is proof that he has found the right way to go for his own specific pecan farming needs. He has even become an evangelist within the local farming community.

“I am not the only pecan farmer in our district. I am the biggest pecan farmer, but not the only one. I am trying to spread the word around. For example, my neighbour still does not want to cut down his trees. But I am getting there, showing him photos, proving that you have to be aggressive to open up the orchard to get sunlight in – and here is the proof that it has been working for me!”

Download Mergin Maps Today

Screenshots of the Input App for Field Data Collection

Get it on Google PlayGet it on Apple store

Learn More

Input version 1.0 release

Input app has reached a new milestone. Earlier in September, we have released version 1.0 of the app with many new features and enhancements.

Make syncs faster

When you work in a team with several collaborators adding data and photos to the project, the size of the project can get really big. Every time you try to sync your changes, photos from all users will be transferred to your phone. This can take a long while when there are several hundreds of photos collected by other collaborators.

Image below illustrates the difference between having selective sync or the default behaviour (selective sync is disabled):

With and without sync

The configuration file for enabling the selective sync is stored in mergin-config.json which should be placed in the root of your folder. By opening the file in a text editor and adding the following option, the sync will be enabled for photos within your project root folder:

{ "input-selective-sync": true }

Currently, the editing of the file is manual and through the text editor, but we plan to incorporate it within the Mergin plugin for QGIS.

To learn more about how to set up selective sync for your project, you can see the example project. Read more on our help pages for extra configuration options.

One too many!

It is often the case that you have a set of spatial features and you want to record some parameters every now and then. For example, there is a GIS layer representing the manholes and the surveyors carry out regular inspections of the manholes. Instead of duplicating the manhole layer and recording each inspection, you can create a non-spatial table and store each inspection as a new line.

1-N relations in Input

Another use-case for such a feature is that you’d like to attach multiple photos to a single feature.

Many photos to a single feature

To learn more about how to configure these types of projects in QGIS you can see the example projects (manhole example and multiple photos example). The documentation pages describes the logic and process in QGIS in more details.

Accuracy metadata

In addition to the display of the accuracy bubble in the app, we have recently added a whole set of new variables to capture the GPS accuracy, e.g. horizontal and vertical accuracy, ground speed and many more. See the help pages to find out how you can set up those variable within your form. Alternatively, you can clone the example project on Mergin website.

In addition to capturing GPS metadata, this feature can be used for geo-fencing: for example, you can only allow users to edit/capture data when they are physically (i.e. their GPS location) within a certain area:

Join our community

If you have any questions, would like to interact with the rest of community or want to give us your feedback, you can join the Slack community channel.

If you would like to add a new feature or have suggestions to improve the app, do not hesitate to contact us on info@lutraconsulting.co.uk

Learn More

Vector basemaps in QGIS

The new version of the MapTiler plugin pushes our maps from MapTiler Cloud almost to perfection
Learn More

Google Summer of Code 2021 : Virtual Raster Provider for QGIS

Read the guest post and congratulate Francesco Bursi, who successfully completed GSOC 2021 project to add virtual raster provider for QGIS with help of mentors Martin Dobias and Peter Petrik.

In this year’s Google Summer of Code (GSoC), I decided to work on the native QGIS raster calculator. Martin Dobias and Peter Petrik volunteered to mentor my work. I’ve been studying Civil Engineering and GeoInformatics at the University of Padua; here I had the opportunity to work both with a lot of GIS software including QGIS. I enjoyed working with QGIS almost immediately because of the possibility to perform complex analysis with a few clicks or with few python commands. Being passionate about programming and enthusiastic about Open Source, I realized that having the possibility to work together with some experienced developers and with an active community was really a great and unique opportunity, so I decided to apply to the GSoC.

GSOC & OSGeo

Virtual Raster Provider

The existing raster calculator is a powerful tool to perform map algebra that outputs a raster layer, before this work it was possible to take advantage of this tool only by saving the output of this operation as a file. The aim of this year GSoC was to allow users to perform their analysis without creating a new derived raster and taking up disk space and therefore have the result as an on-the-fly computational layer.

Let’s jump to an example and let’s say I want to compute the Chanopy Height Model (CHM), subtracting the Digital Terrain Model (DTM) from the Digital Surface Model (DSM).

I also want to perform some other analysis on the DTM since I want to compute the ideal elevation value for a particular tree planting (disclaimer: the elevation value used is example purposes only, moreover when planting trees you should take into account a lot of factors like slope, aspect, latitude. QGIS, by the way, can really be helpful in this kind of analysis). To do so I will start from the same data and I will create different on-the-fly layers for each calculation, in order to avoid the creation of different files I can take advantage of the new checkbox added to the raster calculator dialog. The computation of CHM is performed in the next screencast and the output layer name is, of course, CHM.

computation of CHM

I’ll end up with a new raster layer (CHM) that can be styled as a normal raster and that is not written as an output file to the disk. For some further analysis, from the DTM, I want to obtain the portion of the area with an elevation between 150 and 350 metres above the datum. By applying the following expression to DTM I’ll end up with a raster that has a value of 1 where conditions specified by the expression is TRUE and it will have value of 0 otherwise.

("dtm@1" > 150) AND ("dtm@1" < 350)

I did not select the output layer name intentionally. The resulting layer will be named after the expression used to generate the layer.

generation of CHM layer

Conditional Statement

I also had the opportunity to improve the raster calculator capabilities by adding the possibility to write expressions that involve conditional statements. Taking the already used example, let’s imagine I want to compute the CHM only for the areas of the DTM that are between 150 and 350 metres above the datum. It’s now possible to write an expression as the following one:

if ( ("dtm@1" > 150) AND ("dtm@1" < 350), CHM, -10)

This expression will output a raster with values of the CHM where the conditions are met and value of -10 if the conditions are not met. Since this is a final result of our analysis I’ll store this output as a file to the disk in the form of a GeoTIFF. I’d like to outline that the CHM used in the expression above and in the next screencast is an onn-the-fly computed raster, so it is possible to:

  • Take advantage of the virtual raster provider (on-the-fly computed raster) in other analysis with the raster calculator (and with other analysis tools);
  • Store the on-the-fly computed raster as a file.

Conclusion

I had fun and I struggled working with QGIS, but I learned a lot of new and interesting things. My pull requests were met with several constructive comments, suggestions and feedback. Some suggestions can be a starting point for future improvements.

  • An enhancement for the feature I’ve developed can be the possibility to take advantage of OpenCL acceleration as it has also been suggested in the dev mailing list;
  • Another enhancement that concerns the raster calculator and only partially the virtual raster provider would be the possibility to support the creation of output raster with multiple bands with the declaration of multiple formulas. I hope to continue to contribute to the QGIS project in the future.
Learn More

Organising Charitable Collection Routes with Offline Mobile Maps

Significant time saved when route maps distributed with Input and Mergin.

This case study was originally written in Czech. The Czech version can be found here.

Every year, teams of volunteers walk door-to-door through the Czech town of Litomyšl collecting charitable donations. Event organisers define routes for the various volunteer teams by marking-up paper maps with pens. The process has a number of issues both in the making and usage of the maps which organisers worked to overcome by making the maps digital using open source GIS software.

Maps were developed using QGIS and made available on volunteers’ phones using the Input app. Volunteers are now able to easily orientate themselves on maps which clearly show their routes. Organisers have reduced the time it takes to update routes and distribute these to volunteers.

Veronika Peterková works for the Litomyšl Parish Charity, a non-profit organisation providing health and social services to people in need since 1993.

Veronika describes the charity’s activities: “We provide home medical services and nursing care to the residents of Litomyšl and its surrounding villages. This includes helping families where the healthy development of a child is at risk and providing respite stays for clients who are otherwise cared for by their families at home. We provide care for about 1000 clients a year.”

She added: “We also coordinate the activities of volunteers who visit the elderly, help with tutoring children and with various leisure and cultural activities.”

One of the parish charity’s biggest fundraising events is the “Tříkrálová sbírka” (Three Kings Collection), a door-to-door carol-singing collection taking part around the 6th of January each year.

Tříkrálová sbírka Litomyšl

Volunteers participating in the Three Kings Collection.

“The Three Kings Collection is the largest national volunteer event in the Czech Republic. In the Litomyšl region alone, nearly 300 volunteers are involved each year with the carol-singers collecting over 500,000 Czech crowns (~20,000 EUR) in sealed boxes. The proceeds are intended to help the sick, the disabled, the elderly, mothers with children in need and other in-need groups in the local area.” Veronika explains.

The Three Kings Collection is organised by Caritas Czech Republic and at least 10% of its proceeds are allocated for humanitarian aid abroad.

charita logo

The Challenge

Veronika is responsible for planning routes for the carol-singers so they efficiently visit households in the Litomyšl area. Singers are split into groups and paper maps are provided which show groups which households to visit.

Old map © mapy.cz

An example of previous paper maps, image courtesy of Farní charita Litomyšl.

The above maps were produced by printing screenshots from a national web mapping provider and marking-up printouts for each of the 50 teams using marker pens.

This method proved to have a number of issues as Veronika describes: “On maps of larger areas, house numbers were not always visible due to the scale. This made it even harder for coordinators not familiar with the area to orient themselves, leading to confusion. Coordinators also found it hard to keep the maps dry and undamaged during unfavourable weather. If new groups signed-up afterwards or others opted-out, we’d have to redo/redivide the areas which would be very time-consuming as the maps would need to be marked-up manually once again.”

The Solution and Implementation

Veronika wanted to try a new solution for organising the 2021 Three Kings Collection with the goal of making volunteer tasks clearer and less reliant on paper maps. She wanted the new solution to allow her to:

  • reduce work through the reuse of maps in future Three Kings Collection events
  • easily update maps if new groups sign in/out and areas need editing
  • allow carol singers to see exactly where they are on the map
  • gradually replace paper maps while still allowing the use of paper maps where preferred
  • group and colour buildings to be visited on the computer
  • record a building’s use (e.g. commercial) to direct volunteers more effectively
  • clearly show how areas are assigned so anyone can see who is responsible for a given area

In addition, Veronika wanted the solution to be affordable and work offline without volunteers needing internet connectivity in the field.

Peter Petrík, a regular participant of the Litomyšl Three Kings Collection suggested Veronika try using the Input app for coordinating the collection in 2021. Peter works for Lutra Consulting, the company behind Input and Mergin.

He showed Veronika how to create the maps in QGIS, a free and open source mapping software. Using map data from OpenStreetMap, they created a project showing the buildings to be visited, coloured by their associated volunteer group number.

qgis map © OpenStreetMap contributors

Houses grouped by team in QGIS, image courtesy of Farní charita Litomyšl.

The styled map was uploaded to Mergin, a collaborative mapping platform, making it readily available for viewing interactively on volunteer’s phones using the Input mobile app. Both QGIS and Input integrate closely with Mergin which meant that maps could be adjusted in QGIS with the resulting changes being visible to volunteers shortly thereafter.

Outcomes

Veronika reflects on the solution: “The solution met all our requirements and the maps we’ve prepared can easily be reused in upcoming events, saving us time. The fact that the new maps were made publicly accessible means volunteers can just download them using Input which makes distributing and updating them very easy.”

qgis map © OpenStreetMap contributors

Volunteer routes and position information shown in Input, screenshot courtesy of Farní charita Litomyšl.

She adds: “All the districts we wanted to visit were distinguished from each other by colour and we were also pleased to be able to clearly mark the areas not to be visited like industrial areas by colouring them in grey.”

Unfortunately COVID meant that Veronika’s plans changed as she explains: “Using these new methods we were able to prepare for the 2021 Three Kings Collection in a short time. Unfortunately however, the COVID situation meant we could not go out on the streets to use the new maps as intended. We hope that in 2022 we’ll be able to more closely evaluate the positives and negatives of the field aspect of the project.”

She adds: “We already see it’s now much easier to allocate areas of the town to our volunteers in a clear and fair manner using QGIS. Producing printed maps for those who prefer them is also now easy and the maps look much more professional. Those who only wanted to use the Input app could see the same information as on the paper maps, but had the advantage of being able to pinpoint their exact location and clearly see the house numbers of each building.”

new map © OpenStreetMap contributors

Example printed map created for volunteers wanting also paper maps, image courtesy of Farní charita Litomyšl.

She concludes: “Overall we found the solution user-friendly, and appreciated being able to discuss the process with Lutra Consulting who helped us solve issues as required. About a third of our volunteers are interested in using Input, which I consider positive.”

The Litomyšl Parish Charity are on Facebook and Instagram.

Download Mergin Maps Today

Screenshots of the Input App for Field Data Collection

Get it on Google PlayGet it on Apple store

Learn More

QGIS Cloud: Starting a new feature series for QGIS Cloud Pro customers

We are pleased to announce that today we have published a new release of QGIS Cloud. Besides a whole bunch of bug fixes, we have also introduced new features for QGIS Cloud Pro customers. Starting with this release, we will be releasing more features for our QGIS Cloud Pro customers in the coming weeks. The following new features are available for QGIS Cloud Pro users starting today. Import Layer: In the Layers & Legend tool you will now find the possibility to import layers.
Learn More