OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Track the Vendée Globe race with Python and QGIS

Create and visualize GIS data on the progress of the Vendée Globe 2024 race from official spreadsheets.
Learn More

Revue de presse du 29 novembre 2024

Une GeoRDP bourrée de hashtags, en plein #30DayMapChallenge, à lire avec son smartphone orienté vers le Sud
Learn More

GeoParquet in QGIS – smaller & faster files for the win!

tldr; Tired of working with large CSV files? Give GeoParquet a try!

“Parquet is a powerful column-oriented data format, built from the ground up to as a modern alternative to CSV files.” https://geoparquet.org/

(Geo)Parquet is both smaller and faster than CSV. Additionally, (Geo)Parquet columns are typed. Text, numeric values, dates, geometries retain their data types. GeoParquet also stores CRS information and support in GIS solutions is growing.

I’ll be giving a quick overview using AIS data in GeoPandas 1.0.1 (with pyarrow) and QGIS 3.38 (with GDAL 3.9.2).

File size

The example AIS dataset for this demo contains ~10 million rows with 22 columns. I’ve converted the original zipped CSV into GeoPackage and GeoParquet using GeoPandas to illustrate the huge difference in file size: ~470 MB for GeoParquet and zipped CSV, 1.6 GB for CSV, and a whopping 2.6 GB for GeoPackage:

Reading performance

Pandas and GeoPandas both support selective reading of files, i.e. we can specify the specific columns to be loaded. This does speed up reading, even from CSV files:

Whole fileSelected columns
CSV27.9 s13.1 s
Geopackage2min 12s 😵20.2 s
GeoParquet7.2 s4.1 s

Indeed, reading the whole GeoPackage is getting quite painful.

Here’s the code I used for timing the read times:

As you can see, these times include the creation of the GeoPandas.GeoDataFrame.

If we don’t need a GeoDataFrame, we can read the files even faster:

Non-spatial DataFrames

GeoParquet files can be read by non-GIS tools, such as Pandas. This makes it easier to collaborate with people who may not be familiar with geospatial data stacks.

And reading plain DataFrames is much faster than creating GeoDataFrames:

But back to GIS …

GeoParquet in QGIS

In QGIS, GeoParquet files can be loaded like any other vector layer, thanks to GDAL:

Loading the GeoParquet and GeoPackage files is pretty quick, especially if we zoom into a small region of interest (even though, unfortunately, it doesn’t seem possible to restrict the columns to further speed up loading). Loading the CSV, however, is pretty painful due to the lack of spatial indexing, which becomes apparent very quickly in the direct comparison:

(You can see how slowly the red CSV points are rendering. I didn’t have the patience to include the whole process in the GIF.)

As far as I can tell, my QGIS 3.38 ‘Grenoble’ does not support writing to or editing of GeoParquet files. So I’m limited to reading GeoParquet for now.

However, seeing how much smaller GeoParquets are compared to GeoPackages (and also faster to write), I hope that we will soon get the option to export to GeoParquet.

For now, I’ll start by converting my large CSV files to GeoParquet using GeoPandas.

More reading

If you’re into GeoJSON and/or PyGeoAPI, check out Joana Simoes’ post: “Navigating GeoParquet: Lessons Learned from the eMOTIONAL Cities Project”

And if you want to see a global dataset example, have a look at Matt Travis’ presentation using Overture data:

Learn More

Suivre le Vendée Globe 2024 depuis un SIG

Créer et visualiser les données SIG de l'avancement de la course du Vendée Globe 2024 à partir des tableurs officiels.
Learn More

(Fr) [Story] Oslandia x QWC : épisode 1 / 8

Sorry, this entry is only available in French.

Learn More

(Fr) [Story] Oslandia x QWC : épisode 6 / 8

Sorry, this entry is only available in French.

Learn More

Disaster Management with QGIS and the InaSafe Plugin

Enhancing Disaster Management with QGIS and the InaSafe Plugin

In the ever-evolving landscape of disaster management, having the right tools to anticipate and respond to disasters can be the difference between preparedness and chaos. One such tool that has been making waves is InaSafe, an open-source plugin for QGIS designed to produce realistic natural hazard impact scenarios for better planning and response.



At the GISSA Gauteng session earlier this year, I demonstrated how InaSafe can empower disaster management professionals. My talk focused on how the tool can be used in practical, real-world scenarios, specifically through the analysis of flood events. Here’s a breakdown of my key points and insights from the session.

What is InaSafe?

InaSafe is a plugin that allows users to conduct impact assessments based on hazard and exposure data, like floods, earthquakes, or volcanic eruptions. The tool was developed to provide actionable information to help in making informed decisions during disaster management planning. One of InaSafe’s most powerful aspects is its ability to model different disaster scenarios and provide detailed analyses of their impacts on communities, infrastructure and the environment.



Website: InaSafe.org

Manual and Resources: InaSafe User Manual


The Power of Simple and Complex Analyses



During the session, I performed two different types of flood analysis to highlight the versatility of InaSafe.


Key Benefits for Disaster Management

  1. Real-World Applications: InaSafe isn’t just theoretical—it’s actively used in disaster-prone regions around the world, from Southeast Asia to Africa. Governments, NGOs, and disaster response teams leverage it to improve preparedness and risk mitigation strategies.
  2. Customisable Outputs: InaSafe allows users to generate tailored impact reports. These reports can include detailed statistics, maps, and charts, making it easy to communicate risks and necessary actions to decision-makers and stakeholders.
  3. Community Driven: The tool is open-source and free, with a global community contributing to its development. This ensures that InaSafe stays up to date with the latest disaster management needs and best practices.



How to Get Started with InaSafe

If you're interested in exploring the power of InaSafe for your own disaster management needs, here’s a simple guide to get started:

Install the Plugin: InaSafe is available as a free plugin in QGIS. You can install it directly through the QGIS plugin manager.

Explore the Plugin: Familiarise yourself with the tool by exploring the various hazard and exposure datasets available.

Conduct a Simple Analysis: As demonstrated in the session, start with a basic hazard layer (e.g., flood, earthquake) and use the plugin to assess impacts on your region of interest.

Dive Deeper with Complex Analyses: For those looking to conduct more advanced studies, InaSafe supports custom inputs, more detailed parameters and sophisticated simulations.


1. Simple Analysis


We began with a basic flood scenario, demonstrating how easy it is to run an analysis using InaSafe once the plugin is installed in QGIS. For this exercise, I used predefined hazard and exposure layers, quickly generating an impact report. The simplicity of this analysis underscores InaSafe's accessibility for users, even those new to QGIS or GIS analysis. In just a few clicks, you can produce meaningful insights about potential flood impacts, helping first responders prioritise actions.


2. Complex Analysis


I then shifted to a more intricate analysis, integrating multiple datasets and refining parameters to simulate a more realistic and high-stakes flood scenario. This demonstrated InaSafe's capacity to handle complex scenarios where greater detail is needed—such as assessing the potential displacement of populations, the impact on critical infrastructure like roads and hospitals, or the number of affected households. The results of these analyses showed the breadth and depth of InaSafe’s capabilities. The tool can be used to assess anything from small, localised events to large-scale disasters with widespread impacts.


Wrapping Up



As we face an increasing number of natural disasters, tools like InaSafe are essential in helping us prepare, respond, and ultimately save lives. Whether you're a GIS professional or a disaster management planner, InaSafe provides a powerful, accessible platform to better understand the impacts of disasters before they strike. With its combination of simplicity and depth, it’s a tool that can truly transform the way we approach disaster management.

Learn More

Introducing QChat: a chat within QGIS!

In 2024, it's time to leave Teams and IRC behind to enter a new era where you can chat directly from QGIS with other GIS fellows. #GISChat #GISTribe
Learn More

QChat

Tchatter en direct avec ses confrères et consœurs, à l'intérieur même de QGIS avec QChat
Learn More

(Fr) Du nouveau pour [CityBuilder] CityForge

Sorry, this entry is only available in French.

Learn More