OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

3.4.0 - Ebo

Changes

34

🚀 Features

  • A nifty processing toolbox has been added into QField, with countless algorithms available to edit features and geometries while in the field
  • Geofencing functionality has landed in QField, allowing users to use vector layers to define geofenced areas to warn users or restrict areas where digitizing can occur
  • Project variables are now shown and customizable within the variable editor
  • The ratio and resolution settings for photos captured by the QField camera
  • Date and positioning details can be imprinted onto photos captured by the QField camera

✨ Improvements

  • A visual revamp of the variables panel provides more space for variable content and easier interaction
  • QField’s growing documentation is now searchable directly within QField through the search bar, simply type ? followed by keyword(s)
  • Users can now flag folders as ‘favorites’ when browsing local datasets and projects
  • The cloud projects list now comes with a nifty search bar
  • Significant improvements in attachment upload to the QFieldCloud servers (if this was a problem in the past, try again!)
  • Improvements to the QField camera geotagging functionality
  • On mobile devices, the home screen has a shutdown button to insure authentication details are cleared
  • Projects can now have a default active layer set for each map theme

🪟 Windows improvements

Learn More

3.4.1 - Ebo

What's Changed

  • Add orientation (aka compass direction) detail when watermarking/stamping photos
  • Fix typo in stamping toaster notification
Learn More

Plugin Update – August, 2024

In last August, there were 24 new plugins published in the QGIS plugin repository.

Highlight

“Help us create the world’s most advanced open database on litter, brands & plastic pollution.”

This sentence welcomes us to the website of the OpenLitterMap project, which aims at providing tools for any and all citizens to capture data on litter worldwide. This data can now be directly accessed in QGIS for visualisation and analysis purposes, thanks to the efforts of the plugin’s author NaturalGIS. Well done to everyone involved, and we wish all the luck to this great project.

Overview

Here follows the quick overview in reverse chronological order. If any of the names or short descriptions catches your attention, you can find the direct link to the plugin page in the table below:

Theme Switcher
This plugin adds a popup to easily switch between layer themes.
Clickhouse_Connector
This plugin connects to Clickhouse.
Yarding Distance
This plugin calucurates the “Yarding Distance” (average of distance from Polygon to Points).
RealEarth
This plugin allows users to directly access SSEC RealEarth web services public catalog of near real-time satellite imagery and related ancillary data through the OGC standard web services of WMTS and WFS. A login to RealEarth is recommended to extend data volume quota before watermarking occurs, but is not required.
SkyGIS
This is a plugin to download files from Skydeck, process it in QGIS and upload the results back to Skydeck portal.
3D Arcs
Tool to convert 2D lines to 3D Arcs.
Transit Reachability Analyser
Using OpenTripPlanner to calculate public transport reachability from a starting point to all stops in a GTFS feed.
QMapOD
Cartographie d’enquêtes O/D sous QGIS / Spatialite.
PackageStyler
Style all loaded layers in the GPKG in a few clicks.
AutonomousGIS_GeoDataRetrieverAgent
An autonomous agent framework to select geospatial data and then fetch data by generating and executing programs with self-debugging.
merqantile
Easy visualisation of XYZ tile bounds.
Feature Transfer GIS Tool
Feature Transfer Tool provides a seamless way to copy and paste features between layers.
Select Lines
Select Lines.
Layer Atlas
Discover and share geospatial layers easily within QGIS.
Verificar_Sobreposicao
Verifica sobreposição de feição.
Chainage Tool
This tool provides utility to convert line to chainage points.
Raster Value Regular
Smooth and interpolate grid from a Raster Layer using RegularGridInterpolator from scipy, then apply values to a vector layer as attribute.
Earth Observation Pavement Analysis
This plugin prepares the data sets to train, validate and assess earth observation imagery for pavement analysis.
OpenLitterMap
Processing provider to download raw data from the OpenLitterMap (https://openlittermap.com) project.
TilePick
Easy load raster or point cloud tiles from index vector layer or map canvas position.
WCS 2
A OGC WCS 2.0 / EO-WCS Client to download spatio-temporal subsets from time-series datacubes.
Disaster Risk Management IADB Toolbox
Processing provider that integrates various disaster risk management tools into QGIS.
PhotoViewer360
PL: Wtyczka umożliwiająca import i wizualizację zdjęć panoramicznych. ENG: Plugin for importing and visualising local panoramic images.
Online Map Linker
This plugin makes links from points to online map.
Learn More

(Fr) Variabilisez vos profils QGIS avec QDT

Sorry, this entry is only available in French.

Learn More

Building spatial analysis assistants using OpenAI’s Assistant API

Earlier this year, I shared my experience using ChatGPT’s Data Analyst web interface for analyzing spatiotemporal data in the post “ChatGPT Data Analyst vs. Movement Data”. The Data Analyst web interface, while user-friendly, is not equipped to handle all types of spatial data tasks, particularly those involving more complex or large-scale datasets. Additionally, because the code is executed on a remote server, we’re limited to the libraries and tools available in that environment. I’ve often encountered situations where the Data Analyst simply doesn’t have access to the necessary libraries in its Python environment, which can be frustrating if you need specific GIS functionality.

Today, we’ll therefore start to explore alternatives to ChatGPT’s Data Analyst Web Interface, specifically, the OpenAI Assistant API. Later, I plan to dive deeper into even more flexible approaches, like Langchain’s Pandas DataFrame Agents. We’ll explore these options using spatial analysis workflow, such as:

  1. Loading a zipped shapefile and investigate its content
  2. Finding the three largest cities in the dataset
  3. Selecting all cities in a region, e.g. in Scandinavia from the dataset
  4. Creating static and interactive maps

To try the code below, you’ll need an OpenAI account with a few dollars on it. While gpt-3.5-turbo is quite cheap, using gpt-4o with the Assistant API can get costly fast.

OpenAI Assistant API

The OpenAI Assistant API allows us to create a custom data analysis environment where we can interact with our spatial datasets programmatically. To write the following code, I used the assistant quickstart and related docs (yes, shockingly, ChatGPT wasn’t very helpful for writing this code).

Like with Data Analyst, we need to upload the zipped shapefile to the server to make it available to the assistant. Then we can proceed to ask it questions and task it to perform analytics and create maps.

from openai import OpenAI

client = OpenAI()

file = client.files.create(
  file=open("H:/ne_110m_populated_places_simple.zip", "rb"),
  purpose='assistants'
)

Then we can hand the file over to the assistant:

assistant = client.beta.assistants.create(
  name="GIS Analyst",
  instructions="You are a personal GIS data analyst. Write and rund code to answer geospatial analysis questions",
  tools=[{"type": "code_interpreter"}],
  model="gpt-3.5-turbo",  # or "gpt-4o"
  tool_resources={
    "code_interpreter": {
      "file_ids": [file.id]
    }
  }  
)

Then we can start asking questions and giving our assistant tasks:

thread = client.beta.threads.create()

message = client.beta.threads.messages.create(
  thread_id=thread.id,
  role="user",
  content="Please load the zipped shapefile and describe the content"
)

assistant > The shapefile has been successfully loaded as a GeoDataFrame with the following details:

- The GeoDataFrame has 243 entries.
- It contains a total of 38 columns with various data types including integer, float, and object types.
- One of the columns is a geometry column representing the spatial information.

If you have any specific analysis or visualizations you would like to perform on this data, feel free to let me know!

The docs have a nice example for how to stream the answers back, e.g. into a Jupyter notebook. I’ve used it to create a small convenience function: 

def ask_another_question(question):
    message = client.beta.threads.messages.create(
        thread_id=thread.id,
        role="user",
        content=question
    )
    with client.beta.threads.runs.stream(
        thread_id=thread.id,
        assistant_id=assistant.id,
        event_handler=EventHandler(),
    ) as stream:
        stream.until_done()

ask_another_question("Please list the three largest cities")

assistant > code_interpreter

# Sort the GeoDataFrame by the population in descending order
largest_cities = gdf.sort_values(by='pop_max', ascending=False).head(3)[['name', 'pop_max']]
largest_cities
assistant > The three largest cities in the dataset, along with their maximum population numbers, are as follows:

1. Tokyo: 35,676,000
2. New York: 19,040,000
3. Mexico City: 19,028,000

These cities have the highest recorded maximum population numbers in the dataset. Let me know if you need any further information or analysis!
ask_another_question("extract all scandinavian cities ")

assistant > To extract all the Scandinavian cities from the dataset, we can filter the GeoDataFrame based on the countries typically considered part of Scandinavia. These countries include Norway, Sweden, Denmark, Finland, and sometimes Iceland. Let's filter the GeoDataFrame to extract the Scandinavian cities.
assistant > code_interpreter

# List of countries typically considered part of Scandinavia
scandinavian_countries = ['Norway', 'Sweden', 'Denmark', 'Finland', 'Iceland']

# Filter the GeoDataFrame for Scandinavian cities
scandinavian_cities = gdf[gdf['adm0name'].isin(scandinavian_countries)][['name', 'adm0name']]

scandinavian_cities.head()  # Displaying the first few Scandinavian cities
assistant > Here are some of the Scandinavian cities extracted from the dataset:

1. Reykjavík, Iceland
2. Oslo, Norway
3. Helsinki, Finland
4. København (Copenhagen), Denmark
5. Stockholm, Sweden

These are just a few examples of Scandinavian cities present in the dataset. If you would like the complete list or perform any specific analysis on these cities, feel free to let me know!
ask_another_question(
    "please create a world map with all cities (use gray circles) "
    "with scandinavian cities highlighted as red circles. "
)

assistant > To create a world map with all cities represented as gray circles and Scandinavian cities highlighted as red circles, we will use the GeoPandas library for mapping. Let's go ahead and create the map.
assistant > code_interpreter

import matplotlib.pyplot as plt

# Plot all cities with gray circles and Scandinavian cities with red circles
fig, ax = plt.subplots(figsize=(12, 8))

# Plot all cities in gray circles
gdf.plot(ax=ax, color='lightgrey', markersize=5)

# Plot Scandinavian cities in red circles
scandinavian_cities.plot(ax=ax, color='red', markersize=50, edgecolor='black', label='Scandinavian Cities')

# Set the title
ax.set_title('World Map with Scandinavian Cities Highlighted', fontsize=16)

# Add legend
ax.legend()

# Display the map
plt.show()
assistant > It seems that there was an error while plotting the map because the GeoDataFrame `scandinavian_cities` does not have the necessary numeric data to plot the map directly.
...
plt.show()

output >

assistant > Here is the world map with all cities represented as gray circles and Scandinavian cities highlighted as red circles. The map provides a visual representation of the locations of the Scandinavian cities in relation to the rest of the cities around the world. If you need any further assistance or modifications, feel free to let me know!

To load and show the image, we can use: 

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

def show_image():
    messages = client.beta.threads.messages.list(thread_id=thread.id)

    for m in messages.data:
        if m.role == "user":
            continue
        if m.content[0].type == 'image_file':
            m.content[0].image_file.file_id
            image_data = client.files.content(messages.data[0].content[0].image_file.file_id)
            image_data_bytes = image_data.read()
            with open("./out/my-image.png", "wb") as file:
                file.write(image_data_bytes)
            image = mpimg.imread("./out/my-image.png")
            plt.imshow(image)
            plt.box(False)
            plt.xticks([])
            plt.yticks([])
            plt.show() 
            break

Asking for an interactive map in an html file works in a similar fashion.

You can see the whole analysis workflow it in action here:

This way, we can use ChatGPT to perform data analysis from the comfort of our Jupyter notebooks. However, it’s important to note that, like the Data Analyst, the code we execute with the Assistant API runs on a remote server. So, again, we are restricted to the libraries available in that server environment. This is an issue we will address next time, when we look into Langchain.

Conclusion

ChatGPT’s Data Analyst Web Interface and the OpenAI Assistant API both come with their own advantages and disadvantages.

The results can be quite random. In the Scandinavia example, every run can produce slightly different results. Sometimes the results just use different assumptions such as, e.g. Finland and Iceland being part of Scandinavia or not, other times, they can be outright wrong.

As always, I’m interested to hear your experiences and thoughts. Have you been testing the LLM plugins for QGIS when they originally came out?

Learn More

(Fr) [Story] Oslandia x QWC : épisode 1 / 8

Sorry, this entry is only available in French.

Learn More

MovingPandas 0.19 released!

This release is the first to support GeoPandas 1.0.

Additionally, this release adds multiple new features, including:

For the full change log, check out the release page.

We have also revamped the documentation at https://movingpandas.readthedocs.io/ using the PyData Sphinx Theme:

On a related note: if you know what I need to change to get all Trajectory functions listed in the TOC on the right, please let me know.

Learn More

(Fr) [Équipe Oslandia] Florent, développeur SIG

Sorry, this entry is only available in French.

Learn More

Reports from the winning grant proposals 2023

With the QGIS Grant Programme 2023, we were able to support six proposals (four in the first round and two in the second round) that are aimed to improve the QGIS project, including software, infrastructure, and documentation. The following reports summarize the work performed in the first four proposals:  

  1. QGIS Bug Tracker cleanup (#266)  – Report
     We have identified and closed ~291 tickets, among them:
    • 162 bugreports and feature requests which were already fixed or implemented
    • 29 bugreports and feature requests which are invalid (data issues, wrong use of functionality, etc)
    • 57 duplicate bugreports and feature requests
    • 5 won’t fix bugreports
    • 5 bugreports were converted to feature requests
    • 33 tickets were closed (does not contain steps to reproduce, test data and no feedback was provided within several month)
    • Additionally we ensured that all tickets has correct tags assigned to to make them easier to find.
  2. Porting to C++ and harmonization of Processing algorithms (#271) – Report
    The QGIS Porting to C++ and Harmonisation of Processing Algorithms grant is now complete.
    • Existing Processing algorithms Voronoi Polygons and Delaunay Triangulation have been ported to C++ and now use GEOS instead of the unmaintained Python module.
    • Two algorithms for generating XYZ tiles (directory and MBTiles variants) have been ported to C++ using a safer and cleaner multi-threading approach.
    • The Align Rasters tool, which was not exposed to Processing, has been removed and a new Processing algorithm with the same functionality has been added.
    • The existing Raster Calculator algorithm has been ported to C++. The algorithm now has two variants: a toolbox version that works the same way as before, and a modeler version that uses the same approach to input naming as the GDAL raster calculator.
  3. Add vertical CRS handling to QGIS (#267) – Report
    • As of QGIS 3.34, QGIS can now create and handle vertical and compound CRSes.
    • In QGIS 3.34 coordinate transforms were reworked so that they function correctly with vertical transformation, if both the source and destination CRS have vertical components.
    • In QGIS 3.36 the coordinate reference selection widgets were updated to offer choices of 2d only, compound, or vertical only CRSes.
    • In version 3.38, we introduced a new setting for QGIS projects, for their vertical reference system. Users can control this through project properties, and it’s accessible via PyQGIS and via associated expression variables (eg @project_vertical_crs) for use in print layouts.
    • Similarly, in 3.38 we introduced the API support for map layers to have a vertical CRS. (This was not exposed to users in 3.38 though)
    • In QGIS 3.40 so far we have exposed the vertical CRS setting for vector layers to users (via the layer properties dialog), allowing users to specify the associated vertical CRS for these layers. The vertical CRS is respected in elevation profile plots, in Identify tool results, and in 3D Map views (assuming the 3D map is created with an associated vertical CRS).
    • There is an open pull-request for 3.40 to expose the vertical CRS for point cloud layers in a similar way, with the vertical CRS being respected in elevation profiles, identify tool results, and 3D map views
    • We have open pull requests for 3.40 to show layer vertical CRS information in the layer properties “information” pages, and add expression variables at the layer scope (eg @layer_vertical_crs).
  4. Improve test result handling on QGIS CI (#268) – Report
    Any tests which fail a rendering comparison will write a descriptive comment to the PR. The comment details which render tests failed, where they are in the code, and includes some helpful pointers to downloading the full test report and the QGIS developer documentation. We also implemented lots of improvements in running the tests locally and how the render test reports are generated and presented to developers.

Thank you to everyone who participated and made this round of grants a great success and thank you to all our sustaining members and donors who make this initiative possible!

Learn More

GRASS GIS 8.4.0 released

The GRASS GIS 8.4.0 release provides more than 520 improvements and fixes with respect to the release 8.3.2.

The post GRASS GIS 8.4.0 released appeared first on Markus Neteler Consulting.

Learn More