Posts by Lutra Consulting

This is a part of series of blog posts to update QGIS community with the outcome of the funding we had raised during late 2021 to improve elevation and point clouds in collaboration with North Road and Hobu. For other updates see part 1 and part 2.

Profile tool

With the new integrated profile tool, you can generate cross sections of point clouds, raster, vector and mesh data. For more information on this tool, you can see the excellent video introduction by North Road who implemented this part of the project.

To be able to view profiles from different data types, there is now a dedicated Elevation settings under layer properties. Users can set the elevation source, style and some other configurations. You can then enable elevation profile widget window by going to the main menu in QGIS, View > Elevation Profile.

Support for COPC

Cloud Optimized Point Cloud (COPC) is a new format for point cloud data and QGIS 3.26 comes with support for it (for both local files and data hosted on remote servers).

COPC is a very exciting addition to the ecosystem, because it is “just” a LAZ file (a format well established in the industry) that brings some interesting extra features. This means all software supporting LAZ file format will also be able to read COPC files without any extra development. If you are familiar with Cloud Optimized GeoTIFF (COG) for rasters, COPC is an extension of the same concept for point cloud data. Read more at https://copc.io/

Ordinary LAS/LAZ files have an issue that it is not possible to efficiently read a subset of data without reading the entire file. This is less of an issue when processing point cloud data, but much more important for point cloud viewers, which typically show only a small portion of the data (e.g. zoomed in to a particular object or zoomed out to show the entire dataset). For that reason, viewers need to index (pre-process) the data before being able to show it - QGIS also needs to do the indexing when a point cloud file is first loaded. The new feature that COPC brings is that data is re-organized in a way that reading just some parts of data is efficient and easy. Therefore when loading COPC files, QGIS can immediately show them without any indexing (that takes time and extra storage).

In addition to that, COPC files can be efficiently used also directly from remote servers - clients such as QGIS can only request small portions of data needed, without the need to download the entire file (that can have size of many gigabytes). This makes dissemination of point cloud data easier than before - just make COPC files available through a static server and clients are ready to stream the data.

A small note: until now, QGIS indexed point cloud files to EPT format upon first load. From QGIS 3.26 we have switched to indexing to COPC - it has the advantage of being just a single file rather than lots of small files in a directory. If you have point cloud data indexed in EPT format already, QGIS will keep using EPT index (rather than indexing also to COPC).

Classified renderer improvements

Classified renderer for point clouds has been improved to:

• Show only classes that are in the dataset (instead of hard-coded list) & show also non-standard classes
• Show percentage of points for each class
• Work also for other attributes (return number, number of returns, point source and few other classes)

Vector transparency in 3D scene

This improvement is not part of the crowdfunding campaign and was exclusively funded by the Swedish QGIS user group, but it is somehow relevant to the audience of this blog post!

With this feature, you can set polygon transparency in 3D scenes.

Want to see more features?

We are trying to improve QGIS to handle point clouds for visualisation and analysis. If you would like certain features to be added to QGIS, do not hesitate to contact us on info@lutraconsulting.co.uk with your idea(s).

This is a part of series of blog posts to update QGIS community with the outcome of the funding we had raised during late 2021 to improve elevation and point clouds in collaboration with North Road and Hobu. For other updates see part 1 and part 3.

Point cloud filtering

With this feature, you can filter the point cloud data based on the classes or any other attributes. This is very similar to filtering available for vector layers.

Filtering data allows you superimpose for example building on top of the raster representation of your point cloud data:

Examples of filtering you can use:

• Classification = 2 - only show ground points

• ReturnNumber = 1 - only show the first return or ReturnNumber = NumberOfReturns for the last return

• Z >= 10 AND Z <= 50 - only show a slice from the range of elevations

The filtering window also displays statistics of some of the parameters.

2D/3D Camera sync

When you navigate in the 2D map, you often want to see the 3D map view also updated and vice versa. This feature also allows you to view the extent and camera angle of your 3D map view on 2D map.

New point clouds styling method

There is a new 3D styling mode for point cloud which follows the 2D styling. This means you do not need to apply the same styling, e.g. Classification twice: once for the 2D map view and another for the 3D map view. Once you set the 3D style to follow 2D map, any changes in 2D map style will be automatically displayed in 3D map.

Camera and navigation improvements

This feature was funded by QGIS.org to improve 3D map navigation. Users can now better move, rotate both the map and camera. The 3D map navigation is now more inline with other applications like Google Earth.

This is a part of series of blog posts to update QGIS community with the outcome of the funding we had raised during late 2021 to improve elevation and point clouds in collaboration with North Road and Hobu. For other updates see part 2 and part 3.

A big thanks!

This work was made possible with generous donations and support by the individuals and organisations below (not in a particular order):

Stuart Smith, BayesMap Solutions, Tibor Lieskovský, Balanced Risk Strategies, Yoichi Kayama, Basel Land Registry and Surveying Office (GVA), Rudaz + Partner, Jakub Fuska, Richard Barnes, Spatial Thoughts, Hans van der Kwast, António Pestana, Richard Lorion, Eagle Resources, Suresh Muthukrishnan, 12P Consulting, Alta, JCIS Consultants, Brenna Hughes, Amt für Geoinformation Basel-Landschaft, Darren Farmer, F.A.R.M. Facilitazioni Agroecologiche Regionali Mobili, Ali Nayeri, Land Vorarlberg, Landesamt für Vermessung und Geoinformation, QGIS User Group Switzerland, Robert Thunen, Twomile Heavy Industries, Inc., Roberto Moyano, Jens Grehl, Pēteris Daknis, Rob Willson (Ecophylla Consulting), Daniel Löwenborg, Ville de Vevey, Alfredo Toledo (Suriyaco), QTIBIA Engineering, Ian Burrows (FAS), Pascal Obstetar, Lidar Guys, Mapping Automation, LLC, Featherstone Survey and Civil, Peter Schmitz, Fernando Michel Tuesta Chichipe, Hugo Sørensen, Bernie Connors, Watershed Research and Training Center, MBS Environmental, Andreas Neumann, Adrian Matter, Mapfly, Enso, João Gaspar, Eric van Dijk, City of Uster, Switzerland, QGIS Usergroup Denmark, STAEREA, Ostschweizerische Gesellschaft für Höhlenforschung, Department of Environment, Land, Water and Planning (Victoria), IGN FI, Travis Flohr, Amt für Wald beider Basel, Matthew Bodnar, Surface libre, OSGeo:UK, National Land Survey of Finland,Natural Resources Canada, Fonds Brukhalter, Arbeitsgemeinschaft Höllochforachung AGH, gis experts, BNHR, Rogue Geoscience Ltd., USACE CRREL and Ian Huitson.

In addition to the list above, we thank several anonymous donors who chose not to be listed.

If you have made a donation towards this work and your name or your organisation name does not appear here, please contact us (info@lutraconsulting.co.uk).

3D view manager

Previously, if you closed a project with a 3D map view, the 3D map view and all its settings were lost when you reopen that project. So in QGIS 3.24 we’ve added a “3D map view manager” that takes care of listing, removing, renaming and duplicating 3D map views in your projects! We’ve also added a new “3D Map Views” menu, which contains all your created 3D map views for easy access.

To summarise, these are the advantages of this new feature:

• Saving 3D map views within QGIS project (similar to other settings) and being able to retrieve the 3D view after closing (either the view or the project)
• 3D map view manager: which allows you to duplicate, rename and delete 3D map views

Dock/undock 3D views

3D map canvas panel was difficult to move, resize and often resulting in unwanted docking. With QGIS 3.24 we added the ability to switch 3D maps from a dockable widget to a top-level window (and back to a dock widget), so that these map views can now be managed, resized and moved just like a standard application window. In addition, you can now use 3D map view in full screen mode.

Respect Z ordering of point clouds in 2D

We’ve added an option to render point clouds according to their Z-order in 2D map views. With the new bottom-to-top ordering option enabled, points with larger Z values will cover lower points – resulting in the appearance of a true orthographic photo. There’s also an option for reverse sorting (top-to-bottom), where the scene appears as if viewed from below. This feature is available in QGIS 3.24

The image below displays the default Z ordering of a LAS file when loaded in QGIS:

The same layer with the ordering of Z switched to bottom-to-top:

Visualisation of point cloud as solid surfaces

With this feature you can render point cloud layer in the 3D view as solid surfaces generated by triangulation. The triangulation is available for all the 3D point cloud renderers: unique color, ramp color, classification and RGB. This feature will be available in QGIS 3.26 and you can try it in the current QGIS nightly/master.

Mergin and the Input App used for efficient mapping and recording of weed clearance in environmentally important wetland of Swan Bay.

“Input can in theory handle everything you’d ever want for a mapping tool!”

Greg Parry is President of the Swan Bay Environment Association in the Borough of Queenscliff, Victoria, Australia. This semi-retired marine ecologist runs a one-man ecological consulting business, Marine Ecological Solutions.

Bare sand and seagrass meadows off the beach at Edward’s Point.

Importance of Swan Bay

Swan Bay is a significant marine wetland with an area of 30 km2, situated near the entrance to Port Phillip Bay on the south-central coast of the Province of Victoria. This is an important bird habitat, supporting the life of about 200 different bird species. It has been recognised as an area of international importance under the Ramsar Convention. Every October, Swan Bay is visited by thousands of migratory shorebirds. More than 3 000 Black Swans can be seen in the bay in Summer and Autumn. The seagrass in Swan Bay is also an important habitat for a variety of fish and other marine life.

Drone image of Swan Bay, Australia

Aims of the Swan Bay Environment Association

The natural flora and fauna in the area are under threat by environmental weeds. Among the 300 different plant species, about 50 % are alien. Invasive environmental weeds harm native plants and animals, the natural landscape and threaten the biodiversity of indigenous species. The three main invasive weeds in Swan Bay are Italian Buckthorn, African Boxthorn and Polygala myrtifolia (myrtle-leaf milkwort from South Africa).

African Boxthorn

A team of volunteers from among the 120 members of the Environment Association will assist with the removal of invasive weeds. Some of these are so large or spiny they need to be cleared by contractors. Others require spraying, while many will be cleared manually by a team of volunteers.

Pre- and post-weeding of the same area

Need for Effective Mapping

This is where Dr Parry’s expertise comes in – together with Mergin and the Input App. He realises the urgent need for effective mapping and planning, to co-ordinate the work of volunteers. If this is not done, there is “inadequate follow-up, so it is a waste of their time and energy”. He emphasises, “We need better records of when places are weeded and how they are weeded.”

The role of accurate mapping is two-fold: firstly to create a historical record of past weeding and to assess weed density, and, secondly, to identify the areas for future weeding.

Dr Parry has classified the following categories for weeding as:

1. areas suitable only for contractors requiring heavy equipment, (RED on map)
2. areas suitable for contractors or volunteers – more open areas, usually where the worst weeds have already been removed by contractors, (ORANGE on map)
3. areas suitable for volunteers – few weeds requiring diligent searching, and many man-hours but limited manual labour, (GREEN on map)
4. areas where drainage has increased weed infestations so that drainage should be tackled before weed removal makes sense. (BLUE on map)

Classification of 4 different weeding categories

Dr Parry realises the need to store the information accurately (how much work has been done, the man-hours and number of people, etc.) and then to analyse it, so as to achieve a more effective job by volunteers in future.

Map showing all areas where weeds have been removed between 2015 and mid-2021. Underlying these ‘weed density’ polygons are polygons showing areas that were weeded in different years between 2015-2021.

Conclusion

Dr Parry finds that Input has all the features required for this project. It will be very useful for volunteers in the field, especially as it is usable by both iOS and Android. He thinks it is “remarkably cheap”, compared to the software he had previously been using.

He has some experience with GIS and, after watching a YouTube instructional video, has managed to incorporate QGIS and set it up without many problems. Once he has fine-tuned the set-up, he is sure that he will be able to enlist many more willing volunteers for this important undertaking.

Back in 2005, Dr Parry had used a Magellan Mobile Mapper hand-held device for mapping, which was bulky and cost about $7 500. “Now all of the capacity of that system is available on your phone!” He is motivated to incorporate the user-friendly features of the Input App in order to achieve his main long-term objective:

“This information, I think, will be very helpful in improving co-ordination of weeding efforts within the borough and ensuring that resources are used efficiently. Over a few years, we will get a much better concept of the resources required to do the job in total. I’d summarise it to say we should be a bit more strategic about it, so as to be more effective.”

Download Mergin Maps Today

Nick shares problems faced by his distributed GIS team and how Mergin solved them.

Nick was searching for a way for his remote team to safely edit the GIS layers of their fibre network designs at the same time. Solutions he’d tried were either unsafe for concurrent editing, not feature-rich enough or had prohibitive licence costs. Nick now uses Mergin to collaborate with his colleagues around the world in near real time.

Nick Whittaker is Director of Highbeech Consultancy, a company providing specialised fibre optic network design services. Highbeach is based in England and operates in markets around the world.

Around 9 years ago, Highbeech started helping their clients develop city-wide fibre-to-the-home network design strategies. Nick’s team developed network simulation models which they used to understand the performance and likely construction costs of different designs fed by both public and municipality GIS data. The models proved a success as several clients commissioned construction of the resulting designs.

Nick’s team were now tasked with acting as technical liaison to the engineering companies carrying out the build in the US.

Nick Whittaker

The Challenge

Nick now needed a way for the different project partners to safely view and edit the design at the same time: “This was working sequentially, not collaboratively - we needed to be able to work in parallel!”

“One of the engineering companies we were advising had requirements to have staff in different parts of the US and at present, everything we were doing with the designs ran on-premises. People could use VPNs but working on designs was a nightmare” Nick explained.

Nick found working on designs via VPN was too slow to be workable and instead looked for a solution which would bring designs closer to the staff needing to work on them.

Cloud storage solutions such as OneDrive, Google Drive or Dropbox are unsuitable for collaboration on GIS data as they allow files that would normally be locked and accessed one at a time to be modified simultaneously by multiple users. This situation commonly results in data loss.

“We were having to limit ourselves to one person working on the design at a time and having to notify each other when it was safe for the next person to take over - file locking by email. This was working sequentially, not collaboratively - we needed to be able to work in parallel!” he added.

Nick searched for better solutions for allowing the various teams to work together quickly and safely at the same time.

“In the past we’d used ArcGIS Server which worked well as a collaboration platform when everyone was in the same office but wouldn’t be suitable in this case. We’d also tried ArcGIS Online but found it frustrating as it was too cut-down compared with ArcGIS Server.” Nick said.

Licencing was also an important factor for Nick: “Some solutions we looked at were plugins to other suites, for example, AutoCAD. If I tell my client that we’ll do a project in AutoCAD, they may tell me they can’t afford an AutoCAD licence.”

“While searching for collaborative GIS platforms I came across the Mergin plugin for QGIS.”

QGIS is open source GIS software with a large number of extensions called plugins.

“GIS is widely used in fibre network design and I used ArcGIS primarily. I first heard about QGIS when using FiberPlanIT as it’s implemented as a QGIS plugin. Within about 12 months I was doing the majority of my GIS tasks in QGIS and now consider myself a QGIS convert, purely because I see the power, potential and capability of its community.”

A fibre network design shown in QGIS

Implementation and Outcomes

After taking time to evaluate Mergin within Highbeech, Nick proposed it as a collaboration platform to the US-based engineering company.

“Initially they were unsure how to set it up so I offered to do that and to administer it for them. Within 2-3 weeks they had 20 guys using it and within 3 months I’d migrated control of it over to them.” Nick said.

“The value is all about collaborating in near real time rather than the days it used to take to do things”

Nick’s team can now see the changes their US-based client makes to the design as they make them and jump in/out as required to perform validations and make corrections and changes as required.

“The time zone difference with the US now works to our advantage - we even tell our clients we can turn their 8 hour working day into a 16 hour working day.”

“Once our client’s fielding team has finished for the day, we work through their data, performing checks and validations and everything’s done by the time they’re back in the office the next day. This forms much of what Highbeech does on the project nowadays and this wouldn’t be possible without Mergin.”

When asked what worked particularly well about Mergin, Nick said: “the seamless way you can be working with someone on a call, make a change to a design and have it appear in front of them within seconds. This is incredible and something that happens on a regular basis.”

“The value is all about collaborating in near real time rather than the days it used to take to do things. That’s the greatest strength of both Mergin and Input.”

Input is a mobile app that allows GIS projects to be viewed and edited in the field.

When asked about how collaborative working might change in the future, Nick said: “We live in a world where an ever increasing number of people choose to work from home. For those working in fibre network design, that’s only possible with collaborative platforms such as Mergin.”

Nick has since introduced other clients in the US and UK to Mergin which supports their collaborative GIS efforts together.

Download Mergin Maps Today

We are pleased to announce the success of our crowdfunding campaign to improve point cloud and elevation tools in QGIS. Thanks to the generous pledges from QGIS community, we have exceeded the target (including the stretch goal).

We are very excited and looking forward to developing those features in the upcoming QGIS releases in collaboration with North Road and Hobu.

Thanks again to all those who have contributed to the campaign. Without your support, these major developments would have not been possible. We will publish a blog post with the list of contributors in due course.

To stay tuned with the latest development, you can visit QGIS code repository or visit our blog for news and updates.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows.

Input App helps to identify burial grounds in a citizen science project in the Netherlands.

“QGIS is my Number One tool for all my work. Through a podcast, I heard about Mergin and the Input App - I immediately installed it and have been playing around with it ever since. It has worked wonderfully and flawlessly. We haven’t had any problem with it so far. It was the perfect solution for this specific project!”

21st-century tools reveal 2000 years of our past history

Konan Pruiksma, born in the Netherlands, is an archaeologist and GIS (Geographic Information System) Specialist experienced in Data Analysis, Relational Databases and Geostatistics, who is making a career in exploring our cultural heritage – in particular the vast wealth buried in the fields of the Netherlands. As an employee of Tijdlab, he was approached by Heritage Quest (Erfgoed Gezocht) and Leiden University/Erfgoed Gelderland to assist in their citizen science project, in which volunteers participate in archaeological research on the Veluwe and Utrechtse Heuvelrug.

Area of already well-known burial mounds created in QGIS

Secrets of Burial Mounds

The Dutch landscape is dotted with ancient burial mounds, Celtic fields and cart tracks, some dating back to the 3rd and 2nd millennium BC. Bodies were first cremated and then the ashes were buried in these mounds. In the Middle Ages, mounds were also used for navigational purposes and even for gallows. The ashes of important people were probably honoured with a solitary mound, whereas other mounds contained the ashes of whole families or of many individuals.

Two burial mounds

Aims of the Heritage Quest Project

The aims of the Heritage Quest project are two-fold, focusing on two views of the concept of citizen science:

1. The public viewpoint - to introduce as many citizens as possible to an awareness of the unique archaeological heritage in the Netherlands literally under their feet. This encourages better protection and conservation of this ancient and fragile heritage. Citizen science lessons are even planned for the classroom, to make children aware of their archaeological heritage and how to conduct scientific research.

2. The scientific viewpoint - to collect as much information as possible about the as yet unknown archaeological treasure trove. With the aid of LiDAR map data, huge areas become visible, which were previously hidden by vegetation and thus undiscovered. Later, with the participation of citizen volunteers, a large amount of field data can be collected, which archaeologists simply do not have the manpower to gather.

Volunteers using Input App in the field

LiDAR Assists in New Discoveries

LiDAR maps – high-resolution models of ground elevation created by a laser scanner, GPS and INS systems mounted on a small aircraft – are made accessible to the project by the Dutch government. Over the past 2 years, these LiDAR maps of areas of suspected burial mounds were analysed by over 6 500 volunteers, even by children, who searched over 600 000 maps and identified many thousands of possible new discoveries. In addition to burial mounds, Celtic fields (agricultural fields about 2 500 years old) and deep linear depressions left by the wheels of carts or wagons are clearly visible in the sandy soil of the Utrechtse Heuvelrug and Veluwe on LiDAR maps.

This manual work of many volunteers has another great benefit. The identified objects from LiDAR maps were used as a teaching dataset for a neural network that could potentially do a similar task automatically in the near future for different sites.

LiDAR image showing burial mounds and tracks

As every map was inspected by at least 15 different participants, the difference in the probability of potential barrows become clear immediately; some hills are identified by all participants, while others only by a few. It is probable that hills recognised by more people have a higher chance of being burial mounds and not natural hillocks. Following this reasoning, there are about 6 000 hills that have a high potential of being ancient burial mounds. However, this needs to be checked in the field.

Volunteers Gather Data in the Field

In summer 2021, the field work began and this is where Mergin and Input App came into play.

“For this project, I immediately thought of Mergin and Input App and it worked from the get-go,”

says Konan Pruiksma regarding the ideal tools he chose for data collection in the field. Volunteers install the Input App to their phones and see the potential burial mounds or Celtic fields on the map. They navigate to a point of interest, digitise the point and fill in the required information, such as photos and notes, in a form. Such field work will continue to be carried out over the next year or more.

Burial mounds to be verified and form to fill in by volunteers

Once they have all the information about the location in Input App, they synchronise the data back to the Mergin Cloud. Konan, as a field manager, sees that data are synchronised on Mergin Cloud Dashboard, as well as who made the changes and when. If he needs to update anything, he can do it even from his office and let volunteers refresh the map. The collected data are stored in the PostGIS database via a docker container for further analysis. Konan uses QGIS Plugin for Mergin to download the field data gathered and to analyse them. When the point has been confirmed as a possible burial mound in the field a certain number of times, it is removed from the volunteers’ maps on Input App and reported to the professional team of archaeologists.

“Input App is user-friendly and can be used on any smartphone. This makes it possible to be used by volunteers with only minimal instruction. Which is great, because with over 6 000 locations to be inspected, we need all the help we can get!” explains Konan. “The App furthermore enables volunteers and archaeologists alike to locate the barrows which are usually hidden beneath vegetation. In the field, these low rises are often poorly visible, which is the reason they have not been identified until now. Before we used Input App, it often took us a very long time just to be able to locate the hill highlighted on the LiDAR map.”

Konan doing analysis of collected field data on his laptop

Information Gleaned from Soil of Burial Mounds

This team further investigates the mound by borehole surveying and the removal of soil samples. The mounds are generally not excavated, only if there is a danger of them being destroyed. Archaeologists prefer to keep these archaeological remains intact.

By coring with a 7 cm auger, a thin soil profile can be extracted from the barrow. In this way, archaeologists can get a small glimpse of the different layers that are present beneath the soil without destroying them by excavation. This provides invaluable information about our prehistoric ancestors. Radiocarbon (or Carbon-14) dating can provide accurate dating of the contents of these prehistoric mounds.

Konan says, “We previously did not know that there were so many burial mounds in the Netherlands!” He explains how information is obtained from the soil, without the need for excavation: “If we find charcoal, it is almost 100 % sure that it is a burial mound. We can learn what the burial rituals were of our ancestors, how they lived and what they ate.”

Conclusion

There are inestimable benefits of collecting and interpreting data accurately. The vast number of burial mounds in the Netherlands would not be able to be detected were it not for the assistance of volunteers, combined with LiDAR maps and tools such as QGIS or Input App. At present, there is a team of 20 volunteers at work on the Veluwe project, but this number should increase in the future, as more citizens become interested in doing citizen science in their environment. As Input App is user-friendly and very intuitive, a minimum amount of training is needed for volunteers, many of whom are students or senior citizens not au fait with digital technology.

Download Mergin Maps Today

Android has forced app to use Scoped Storage for all app related data. If you are using Input app on Android please read carefully for the upcoming update (1.1) of the app in the Google Play Store.

What is Scoped Storage?

In summary, apps need to use a specific part of folders on Android devices to store app related data.

Currently, Input stores your QGIS project and some other settings (e.g. grid shift projection) on /Internal storage/INPUT. With the new Android requirements, the app related data should be stored on /Internal storage/Android/data/uk.co.lutraconsulting.

Update process

For the 1.1 release of Input on Android, there will be an extra process. This process will be a on-off action. When you launch the app after the upgrade, it will copy the data from /Internal storage/INPUT to /Internal storage/Android/data/uk.co.lutraconsulting. Depending on the size of your projects, this can take a couple of minutes. During the process you will see a screen similar to this one:

Best practices

The upgrade and copy process should work smoothly. But we suggest to take the following actions to ensure you will not lose any data during the process:

• Sync all you changes: before upgrading the app, open Input and sync all your local changes to Mergin.

• Storage space: if you work with several projects and large volume of data, make sure you have enough storage. The process will make a copy of your existing /Internal storage/INPUT without deleting it. So, you need at least the size of /Internal storage/INPUT storage available.

Troubleshooting

In case you have encountered any issues, you can take the following steps to fix the problem manually:

• Lack of storage space:
  • Transfer the data through USB cable to your PC.
  • Make a back up of data on your PC.
  • Delete /Internal storage/INPUT on your Android phone/tablet
  • Transfer the data from PC through USB to /Internal storage/Android/data/uk.co.lutraconsulting
  

• Missing data: the migration process does not delete /Internal storage/INPUT folder. It will rename it /Internal storage/INPUT_migrated. Similar to step above, you can copy the data to the PC and move them to /Internal storage/Android/data/uk.co.lutraconsulting. Alternatively, you can use a file browser app on your device to copy files around.

• If you use Input app on a shared device, the migration process will transfer all the project data from /Internal storage/INPUT and marks the folder as /Internal storage/INPUT_migrated. Therefore, when the next user starts up the app, no data will be present. To fix the issue, you need to manually move the data from /Internal storage/INPUT_migrated to /Internal storage/Android/data/uk.co.lutraconsulting (as described above) for the other users on the device.

Need help

If you need further help, please join us on the community chatroom and we will be able to help you with the upgrade issue (or other Input/Mergin related problems.)

Precision farming opens an opportunity to increase the gain for a farmer up to 20% with fewer trees.

Making a sustainable profit from farming can be something like tossing a coin. However, today the advancements in digital technology provide the incredible benefit of reducing the unpredictability of farming in general. In pecan farming in particular, the proper know-how of a combination of multiple factors such as soil composition, environmental impact, climate, irrigation and sunlight management are vital for success. Hard data is essential for innovating key decisions. Riaan Burger, a South African farmer, successfully uses open source GIS software to tackle the issues of collecting and utilising such vital data.

Riaan Burger on his pecan farm.

Becoming a Farmer

Riaan Burger’s journey started back in 2017 when, after 20 years’ working as an electrical engineer, he decided to embark on his own business in pecan nut farming. He is a self-taught farmer and co-owner of one of the biggest pecan farms in the district of Weenen, South Africa. At first, Riaan had only a rough idea of how to improve his newly bought 24 hectares of planted orchard.

After reading books on soil structure, type, depth and chemical analyses, he had soil samples taken to create a soil sample map. In his orchard, the trees were planted in 10x10 metres geometric layouts. After 20 years, the trees were already creating shade. Riaan knew that sunlight penetration is important and that pecan trees are susceptible to fungus, so he decided to take a closer look.

“The previous owner of the farm had already started chopping out every second tree. I was in the position where I could compare the yield from areas where trees had been chopped down, to areas where they had not been felled,” explains Riaan.

A photo of the farm taken by drone and initial stage of farming - soil maps.

Riaan realised that understanding the ambient conditions and mapping them to yield results was the right way to go. This was the beginning of his exciting journey in precision farming.

Collecting Data from the Field

It was no easy task for Riaan to get a complete picture. For two years, he only recorded the yield per day, and the relevant orchard block. Later, he used the services of a local consulting company which provided him with surveying applications to help him to set up a basic workflow.

“I record the weight of pecan nuts from each individual tree and I plot it on the map,” Riaan says. “And then afterwards I can see the different cultivars of trees, the weight of nuts they produced and the area in which they were produced. I previously had soil samples taken, so I’ve got soil sample maps. Now I can overlay a yield map over the soil sample map and correlate them”

Despite working well with the surveying app, Riaan was still in need of assistance to prepare data tables and then to export them to Excel sheets and to request map creation. After the consulting company moved from a third-party app to the in-house build solution, things got worse for Riaan: “It was a generic tool and a difficult app to work with. Displaying data was cumbersome. I wanted something that was quick and easy again. Ideally, I would like to capture data on the iPad and have it directly linked to my laptop – that is how I think it should work.”

So Riaan discovered the Input and Mergin suite, a surveying app based on QGIS and a cloud service for data synchronisation, respectively. Despite only a basic knowledge of QGIS and the necessity to learn and experiment, the effort immediately paid off!

“I’ve got Input, I’ve got a Mergin account and I’ve got QGIS. I now have full control of the process. I don’t need to fund a consultant and find that by 8 o’clock at night he has not yet responded to my query, and so I have to back it up with a WhatsApp call,” Riaan explains. “This is something I created entirely myself. I don’t need to ask someone to change this map, or add that icon, modify the legend or change the styling, I am in control of all of it! And I think there is a sense of satisfaction when one gets things going for oneself.”

A farming project in QGIS desktop and Input app UI.

With the Input app, Riaan has gained control of the workflow – something not before experienced. He has gained the freedom to set the project according to his own individual needs. Without much effort, not only the data but the styling, an important factor for him, are synchronised.

“In QGIS I created a style where each bubble represents the kilograms of the yield and the colour represents the variety,” says Riaan. “Then I saw in Input that there is suddenly an extra page on my iPad. So now I have the bubbles and bubble colours on the iPad instantly, which I did not have in previous apps. Previously, I had to create it the long way around, but now it is suddenly on the iPad automatically. I enjoy having that available! In the past, I only had that information by the end of the harvest.”

Cutting Down Trees

The energy invested in creating workflow, capturing and analysing data paid off handsomely for Riaan. All his initial doubts were suddenly eliminated. He has all the information essential for making any important decision right at his fingertips.

“The first time you fell trees, you have nightmares! How on earth you can be cutting your income and destroying 20 years of growth? Now, with the necessary proof at hand, it is easier in your mind to motivate that tree felling is actually your source of income, that it is the right thing to do,” says Riaan. “You can see that the yield from the remaining trees is more than double than before. The trees are healthier, because of allowing sunlight in. Moreover, you also get less pressure from fungus, because pecan trees are easily susceptible to fungus. Now that you have good ventilation, because you have a draught through your orchard, you can see on the map what your exposure to fungus is.”

Orchard with higher density of trees (top) and lower density (bottom).

Riaan continues: “I am not a sole owner, I have a partner. Now I have the data available, so that when we have a meeting I can explain to him this is what I did, this is what happened and this is the result. If you go to a bank manager for a loan, I mean, he really frowns at you when you tell him you are cutting down your trees which produce your income. Then you can explain to him why it is for the better.”

Increase in yield per tree after reduction of tree density.

Harvest Data Analysis and Beyond

The benefits of collecting and interpreting data carefully are limitless. Cutting down trees is just one example of the many advantages of careful data gathering and interpretation.

“Now I see that certain cultivars perform better than others this year,” adds Riaan. “So I can see if they consistently perform better or worse. I can decide, based on these results, what to do: if I need to add more nitrogen or improve soil pH, or whatever the case may be.”

User-friendly tools tailor-made to specific needs, such as the Input and Mergin suite, provided Riaan a way of collecting and analysing his specific type of farming data. His insights for necessary improvements also developed over the past few years.

“At first I did not have the distinction between individual trees and I even said to my partner” continues Riaan, “if only I’d started doing this in the very first season, I would have had 5 years of data to compare – but I didn’t. I will definitely continue to use this suite. It is valuable and I have been using it only for harvest. However, I see now that I can start recording where I have cut down a tree, so I have a record of when and where the tree was cut. I can also record my irrigation data.”

In spite of initial teething problems and the difficult decisions he had to take, Riaan’s success is proof that he has found the right way to go for his own specific pecan farming needs. He has even become an evangelist within the local farming community.

“I am not the only pecan farmer in our district. I am the biggest pecan farmer, but not the only one. I am trying to spread the word around. For example, my neighbour still does not want to cut down his trees. But I am getting there, showing him photos, proving that you have to be aggressive to open up the orchard to get sunlight in – and here is the proof that it has been working for me!”

Download Mergin Maps Today

Input app has reached a new milestone. Earlier in September, we have released version 1.0 of the app with many new features and enhancements.

Make syncs faster

When you work in a team with several collaborators adding data and photos to the project, the size of the project can get really big. Every time you try to sync your changes, photos from all users will be transferred to your phone. This can take a long while when there are several hundreds of photos collected by other collaborators.

Image below illustrates the difference between having selective sync or the default behaviour (selective sync is disabled):

The configuration file for enabling the selective sync is stored in mergin-config.json which should be placed in the root of your folder. By opening the file in a text editor and adding the following option, the sync will be enabled for photos within your project root folder:

{ "input-selective-sync": true }

Currently, the editing of the file is manual and through the text editor, but we plan to incorporate it within the Mergin plugin for QGIS.

To learn more about how to set up selective sync for your project, you can see the example project. Read more on our help pages for extra configuration options.

One too many!

It is often the case that you have a set of spatial features and you want to record some parameters every now and then. For example, there is a GIS layer representing the manholes and the surveyors carry out regular inspections of the manholes. Instead of duplicating the manhole layer and recording each inspection, you can create a non-spatial table and store each inspection as a new line.

Another use-case for such a feature is that you’d like to attach multiple photos to a single feature.

To learn more about how to configure these types of projects in QGIS you can see the example projects (manhole example and multiple photos example). The documentation pages describes the logic and process in QGIS in more details.

Accuracy metadata

In addition to the display of the accuracy bubble in the app, we have recently added a whole set of new variables to capture the GPS accuracy, e.g. horizontal and vertical accuracy, ground speed and many more. See the help pages to find out how you can set up those variable within your form. Alternatively, you can clone the example project on Mergin website.

In addition to capturing GPS metadata, this feature can be used for geo-fencing: for example, you can only allow users to edit/capture data when they are physically (i.e. their GPS location) within a certain area:

Join our community

If you have any questions, would like to interact with the rest of community or want to give us your feedback, you can join the Slack community channel.

If you would like to add a new feature or have suggestions to improve the app, do not hesitate to contact us on info@lutraconsulting.co.uk