OSGeo Statement: EU Cyber Resilience Act

QGIS Planet

Importing CSV files into PostgreSQL using the DB Manager in QGIS

There is very useful tool in QGIS that can import very large CSV files into PostgreSQL rapidly and reliably. The DB Manager’s “Import Vector Layer” tool. Contrary to its highly misleading title it can import CSV files as well. Open the DB Manager (menu Database – DB Manager). Then select the database where you want to store your table and click the “Import layer/file” icon.

Icon_to_ClickFrom the Import Vector Layer GUI, locate our CSV file on disk and enter the name of your new table in the Table box and click OK. Yes, it’s that simple. Proceeding this, you may need to select an text encoding scheme, files created on Windows often use ISO-8859-1 (Latin-1) instead of UTF-8 encoding. In my case, I was able to import a large statistical data set describing the energy efficiency of 525,500 Irish homes (432 megabytes) into PostgreSQL in ~15 minutes. After the CSV file is imported, you can optionally add it to your project using the DB Manager, right-click the table and select Add to Canvas. Don’t use the “Add PostGIS Layers” menu, it’s not a PostGIS layer.

Import_GuiAnd one more useful tip. You can convert Tab delimited text to CSV using QGIS. Load a Tab delimited text file into QGIS using the Add Delimited Text Layer GUI, then right click the imported file in the layer panel and save it as a CSV file.

Learn More

Go2streetview plugin for QGIS

A very handy plugin for QGIS I use day to day is go2streetview by Enrico Ferreguti. The plugin adds an icon to the tool bar in QGIS and when selected I can click a road or street on a base map and a window will open that displays the Google Street or a Bing Maps Bird’s Eye view of the location. The camera’s direction and location is highlighted by a blue marker. I use the plugin when tracing boundaries of parks, open spaces and foot paths from aerial imagery. If the imagery is blurred or the view is obscured by trees, I click a point on a nearby street to see the location up close. The plugin works wherever Google Street view and Bing Birds Eye has coverage.

For example, in the screen-shot below notice there is a footpath leading to a bus shelter that’s not mapped by OpenStreetMap. I know where it is now, I will add it to my map.

Street View

Plugin: go2streetview

Learn More

QGIS Layer Tree API (Part 3)

In the two previous blog posts we have learned how to query a layer tree and how to modify it. All these new APIs are available since QGIS 2.4 release. Today we will look into how to make layer trees available in GUI and connect them with a map canvas.

Model/View Overview

As we have seen earlier, a layer tree is a usual hierarchical data structure composed from nodes of two types - layers and groups. They do not provide any GUI functionality as they live in the QGIS core library. In order to visualize a layer tree, we will use Model/View approach from Qt framework. Readers not familiar with those concepts are recommended to read the Qt Model/View overview first.

Layer tree model/view

There is QgsLayerTreeModel class (derived from QAbstractItemModel) which as you may have guessed provides a model to access a layer tree. Instance of this class may be used in any QTreeView class, however it is recommended to use it together with QgsLayerTreeView because of the extra convenience functionality offerred by the custom view class.

Here is an example how to create another view of current project’s layer tree (try that in QGIS Python console):

from qgis.gui import *

root = QgsProject.instance().layerTreeRoot()
model = QgsLayerTreeModel(root)
view = QgsLayerTreeView()
view.setModel(model)
view.show()

Any changes that happen to the layer tree structure are automatically monitored by the model/view classes. After running the example above, try changing a layer’s name or add/remove/reorder some layers - all those actions will be immediately visible in all views.

As you can see, the layer tree view is just one way how to visualize the underlying layer tree - in the future it may be possible to have different ways to show the layer tree, for example using Qt’s QML framework (with all sorts of animated transitions known from mobile apps).

Plugin developers can access the layer tree view in the main window of QGIS through the following interface call:

view = iface.layerTreeView()

More About the Model

The model is meant to be flexible and it is possible to use it in various contexts. For example, by default the model also provides legend for the layers in the tree. This however may not be wanted in some cases. Similarly, sometimes the layer tree should be read-only while in other context it is desired that the user can reorder layers or change their names. These preferences can be passed to the model with flags:

model = QgsLayerTreeModel(root)
model.setFlag(QgsLayerTreeModel.AllowNodeReorder)
model.setFlag(QgsLayerTreeModel.AllowNodeChangeVisibility)

The setFlag() method has optional second parameter “enabled” (True by default). Flags can be also set all at once with setFlags() method which expects a combination of flags joined by binary OR operator. There are also flags() and testFlag() methods to query the current flags.

The QgsLayerTreeModel class also provides routines for conversion between QgsLayerTreeNode instances and corresponding QModelIndex objects used by Qt Model/View framework - index2node() and node2index() may come handy especially when working with views.

There is also some functionality related to legend display - it has been greatly extended in QGIS 2.6 release and we will try to cover that in a future blog post.

More About the View

As mentioned earlier, it is possible to use any QTreeView instance in combination with QgsLayerTreeModel to show the layer tree, but it is highly recommended to use QgsLayerTreeView class (a subclass of QTreeView) because of the additional functionality it provides (and more may be added in the future):

  • Easier handling of selection. Normally one needs to work with selection through view’s selection model. The QgsLayerTreeView class adds higher level methods that operate with QgsLayerTreeNode objects like currentNode() or with QgsMapLayer objects like currentLayer().

    def onChange(layer):
  QMessageBox.information(None, "Change", "Current Layer: "+str(layer))

# connect to the signal
view.currentLayerChanged.connect(onChange)
# change selection to the top-most layer (onChange will be also called)
view.setCurrentLayer( iface.mapCanvas().layers()[0] )

  • Display of context menu. It is possible to assign a menu provider to the view (subclass of QgsLayerTreeViewMenuProvider) - its createContextMenu() implementation will return a QMenu object with custom actions whenever user right-clicks in the view. Additionally, there is a factory class QgsLayerTreeViewDefaultActions that can create commonly use actions for use in the menu, such as “Add Group”, “Remove” or “Zoom to Layer”. The following example shows how to create a provider with one action that shows the current layer’s extent:

    class MyMenuProvider(QgsLayerTreeViewMenuProvider):
  def __init__(self, view):
    QgsLayerTreeViewMenuProvider.__init__(self)
    self.view = view

  def createContextMenu(self):
    if not self.view.currentLayer():
      return None
    m = QMenu()
    m.addAction("Show Extent", self.showExtent)
    return m

  def showExtent(self):
    r = self.view.currentLayer().extent()
    QMessageBox.information(None, "Extent", r.toString())

provider = MyMenuProvider(view)
view.setMenuProvider(provider)

Interaction with Canvas

The layer tree classes and map canvas class are separate components that are not dependent on each other. This is a good thing and a great step forward, because until QGIS 2.2 there was big internal monolithic QgsLegend view class that handled everything and it was directly connected to map canvas and various other components, making it impossible to reuse it elsewhere. With the new layer tree API this has been solved, now it is possible use map canvas without an associated layer tree view or vice versa - to use a layer tree view without map canvas. It is even possible to get creative and use one layer tree with several map canvas instances at once.

Layer tree and map canvas can be connected via QgsLayerTreeMapCanvasBridge class. It listens to signals from the given layer tree hierarchy and updates canvas accordingly. Let’s see how we could create a new map canvas that would show the same layers as the main canvas does:

canvas = QgsMapCanvas()
root = QgsProject.instance().layerTreeRoot()
bridge = QgsLayerTreeMapCanvasBridge(root, canvas)
canvas.zoomToFullExtent()
canvas.show()

That’s it! We have tied the new canvas with project’s layer tree. So any actions you do in the layer tree view (for example, add or remove layers, enable or disable layers) are automatically passed to the new canvas. And of course this does not work just with project’s layer tree - you could use any custom layer tree in your layer tree model/view or canvas.

The bridge class has advanced functionality worth mentioning. By default the ordering of layers in canvas is according to the order in the layer tree (with first layer in the tree being at the top), though there are API methods to override the default order.

For convenience, the bridge by default also configures some settings of the canvas (this can be disabled if necessary):

  • enable on-the-fly reprojections if layers have different coordinate reference system (CRS)
  • setup destination CRS and map units when first layers are added
  • zoom to full extent when first layers are added

Summary

I hope you have enjoyed the three blog posts introducing QGIS layer tree API. They should cover everything you need to know in order to start using the new functionality. In a future post we will have a look at the new legend API that nicely complements the layer tree API - stay tuned!

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

Labels leading lines with QGIS and Postgis

EN | PT

Recently I had the need to add labels to features with very close geometries, resulting in their collision.

Capturar_3

Using data-defined override for label’s position (I have used layer to labeled layer plugin to set this really fast) and the QGIS tool to move labels, it was quite easy to relocate them to better places. However, in same cases, it was difficult to understand to which geometry they belonged.

Capturar_2

I needed some kind of leading lines to connect, whenever necessary, label and feature. I knew another great plugin called “Easy Custom Labeling“, by Regis Haubourg, that did what I needed, but it would create a memory duplicate of the original layer, wish meant that any edition on the original layer wouldn’t be updated in the labels.

Since the data were stored in a PostgreSQL/Postgis database, I have decided to create a query that would return a layer with leading lines. I used the following query in DB manager:

SELECT
  gid,
  label,
  ST_Makeline(St_setSRID(ST_PointOnSurface(geom),27493), St_setSRID(St_Point(x_label::numeric, y_label::numeric),27493))
FROM
  epvu.sgev
WHERE
  x_label IS NOT NULL AND
  y_label IS NOT NULL AND
  NOT ST_Within(ST_Makeline(St_setSRID(ST_PointOnSurface(geom),27493), St_setSRID(St_Point(x_label::numeric, y_label::numeric),27493)),geom))

This query creates a line by using the feature centroid as starting point and the label coordinate as end point. The last condition on the WHERE statement assures that the lines are only created for labels outside the feature.

Capturar_1

With the resulting layer loaded in my project, all I need is to move my labels and save the edition (and press refresh) to show a nice leading line.

guidelines

Learn More

A new QGIS tool (based on ogr2ogr) to import vectors in PostGIS, the fast way

In QGIS there are many tools that can be used to import vectors inside a PostGIS database, each one has pros and cons: SPIT core plugin: available since long ago but now seems to be a unmaintained tool and therefore will be probably removed in a future QGIS release. It  has the advantage to allow […]
Learn More

CartoDB wins best “high-growth web entrepreneur” at the 2014 European Web Entrepreneur of the Year Awards

CartoDB, the FOSS powered web mapping solution, was honoured at the 2014 European Web Entrepreneur of the Year Awards along with three other companies at Dublin’s Web Summit on November 7th. The awards, presented by a European Commission backed body, were announced after a six month competition that involved public voting across four categories. CartoDB won the award for best “high-growth web entrepreneur”. CartoDB now has a growth rate of over 15% per month and customers in over 30 countries.

In September, a partner company of CartoDB, Kudos Ltda., released a plug-in that allows QGIS users to view, create, edit and delete data stored on their CartoDB accounts. Here is a map I created with the help of the new CartoDB plug-in that shows mountains and hills across the contiguous United States in the form of a heat map.

CartoDB and QGIS illustrate the exciting convergence between web hosted and desktop GIS, where interactive maps created in QGIS can be quickly published on the web and viewed by a worldwide audience.

FinalCartoDB

Learn More

PyQGIS Resources

Here is a short list of resources available when writing Python code in QGIS. If you know of others, please leave a comment. Blogs/Websites In alphabetical order: GIS StackExchange Kartoza Linfiniti Lutra Consulting Nathan Woodrow Nyall Dawson Twitter #pyqgis Documentation Choose the version to match your QGIS install PyQGIS Cookbook QGIS API Example Code Existing plugins can be a great learning tool Code Snippets in the PyQGIS Cookbook Plugins/Tools Script Runner: Run scripts to automate QGIS tasks Plugin Builder: Create a starter plugin that you can customize to complete your own plugin Plugin Reloader: Allows you to reload a plugin from within QGIS pb_tool: Tool to compile and deploy your plugins Books PyQGIS Programmers Guide Geospatial Desktop: GIS Scripting (PDF)
Learn More

Getting Started Writing QGIS Python Plugins

This blog post is a QGIS plugin tutorial for beginners. It was written to support a workshop we ran for the Scottish QGIS user group here in the UK and aims to be a simple step-by-step guide.

In this tutorial you will develop your first QGIS plugin - a Map Tool for selecting the closest feature within multiple loaded vector layers. Knowledge of Python is recommended but not required.

The Goal

Before we get started let’s look at where we’re going.

We will develop a plugin that implements a new QGIS Map Tool. The Identify Features QGIS Identify Tool and Pan Map QGIS Pan Tool tools are both examples of QGIS Map Tools. A Map Tool is a tool which performs an action when used with the map canvas.

We will create a new Select Nearest Feature Map Tool Nearest Feature Tool which will sit in the plugins toolbar.

Our Select Nearest Feature Map Tool will allow the user to select the feature nearest a mouse click on the canvas. For example, clicking here:

Example Canvas Click

would select the following polygon:

Example Polygon Selection

The Starting Point

Before getting started:

The QGIS Plugin Builder plugin was used to create a base plugin which we’ll modify to fit our requirements.

This base plugin can be found in the zip file mentioned above under code/01__Empty Plugin/NearestFeature

code/01__Empty Plugin contains a batch file install.bat that can be used to copy the plugin into your QGIS plugins folder, making it available to QGIS.

Let’s now load and run this simple base plugin in QGIS.

  • Run install.bat
  • Restart QGIS if already open
  • Open the Plugin Manager: Plugins > Manage and Install Plugins
  • Enable the Nearest Feature plugin

A new action Nearest Feature Tool should now be visible in the plugins toolbar which opens the following dialog:

Empty Dialog

Creating a Basic Map Tool

When activated, our plugin currently shows a simple dialog (functionality provided by the Plugin Builder plugin. We’re going to adapt it to instead activate a Map Tool.

A basic Map Tool is included within the zip file mentioned above. It can be found in nearest_feature_map_tool.py in the Additional Files folder.

  • Copy nearest_feature_map_tool.py into the NearestFeature folder and open it in an editor.
  • Note that many of the code segments (highlighted in gray) below link to relevant parts of the API docs. Those links will open in a dedicated browser tab.

nearest_feature_tool.py defines a new NearestFeatureMapTool class (line 28) which inherits (is based on) QgsMapTool, the QGIS Map Tool class. Its __init__() method expects to be passed a reference to a QgsMapCanvas. This canvas reference is passed to the constructor of the underlying QgsMapTool class on line 32 and then stored on line 33 for later use. The QGIS API documentation describes the functionality made available by QgsMapTool.

On line 34 we define a simple, different-looking cursor (a QCursor based on Qt.CrossCursor) later used to indicate that the Map Tool is active.

Our class definition features a method called activate(). Notice the API documentation for QgsMapTool already defines a method with the same name. Any methods defined as virtual methods in the API documentation can be overwritten or redefined as they have been within this file. Here we have overwritten the default implementation of activate().

The activate() method is called when the tool is activated. The new cursor based on Qt.CrossCursor defined above is set with a call to QgsMapCanvas.setCursor().

For the moment, when activated, our Map Tool would simply change the cursor style - that’s all.

Great - next we’ll get our plugin to use the new Map Tool.

Connecting the Basic Map Tool

In this section we will modify the plugin to make use of our new Map Tool.

  • Open nearest_feature.py in a text editor.

We need to first import the NearestFeatureMapTool class before we can use it.

  • Add the following code towards the top of the file just before os.path is imported:
from nearest_feature_map_tool import NearestFeatureMapTool

Next we will create a new instance of the NearestFeatureMapTool class and store a reference to it in self.nearestFeatureMapTool.

  • Add the following code to the initGui() method just before the call to self.add_action() taking care to ensure the indentation is correct:
# Create a new NearestFeatureMapTool and keep reference
self.nearestFeatureMapTool = NearestFeatureMapTool(self.iface.mapCanvas())

Notice that a reference to the map canvas has been passed when creating the new NearestFeatureMapTool instance.

The run() method is called when our plugin is called by the user. It’s currently used to show the dialog we saw previously. Let’s overwrite its current implementation with the following:

# Simply activate our tool
self.iface.mapCanvas().setMapTool(self.nearestFeatureMapTool)

The QGIS map canvas (QgsMapCanvas class) provides the setMapTool() method for setting map tools. This method takes a reference to the new map tool, in this case a reference to a NearestFeatureMapTool.

To ensure that we leave things in a clean state when the plugin is unloaded (or reloaded) we should also ensure the Map Tool is unset when the plugin’s unload() method is called.

  • Add the following code to the end of the unload() method:
# Unset the map tool in case it's set
self.iface.mapCanvas().unsetMapTool(self.nearestFeatureMapTool)

Now let’s see the new map tool in action.

  • Save your files
  • Run install.bat to copy the updated files to the QGIS plugin folder
  • Configure the Plugin Reloader plugin to reload the NearestFeature plugin using its configure button, Configure Plugin Reloader Tool

Configure Plugin Reloader Dialog

  • Reload the Nearest Feature plugin using the Reload Plugin Tool button.
  • Click the Nearest Feature Tool button

When passing the mouse over the map canvas the cursor should now be shown as a simple cursor resembling a plus sign. Congratulations - the Map Tool is being activated.

Map Tool State

When you use the Identify Features Map Tool you’ll notice that its button remains depressed when the tool is in use. The button for our map tool does not yet act in this way. Let’s fix that.

The action (QAction) associated with our plugin is defined in the initGui() method with a call to self.add_action().

self.add_action() actually returns a reference to the new action that’s been added. We’ll make use of this behaviour to make the action / button associated with our Map Tool toggleable (checkable).

  • Modify the call to add_action() as follows:
action = self.add_action(
            icon_path,
            text=self.tr(u'Select nearest feature.'),
            callback=self.run,
            parent=self.iface.mainWindow())

action.setCheckable(True)

We now use the reference to the new action to make it checkable.

The QgsMapTool class has a setAction() method which can be used to associate a QAction with the Map Tool. This allows the Map Tool to handle making the associated button look pressed.

  • Add the following line to the end of initGui():
self.nearestFeatureMapTool.setAction(action)
  • Save your files and run install.bat
  • Reload the Nearest Feature plugin using the Reload Plugin button
  • Click the Nearest Feature Tool button

The button should now remain pressed, indicating that the tool is in use.

  • Activate the Identify Features Identify Features Tool tool

The Nearest Feature button should now appear unpressed.

Handling Mouse Clicks

The QgsMapTool class has a number of methods for handling user events such as mouse clicks and movement. We will override the canvasReleaseEvent() method to implement the search for the closest feature. canvasReleaseEvent() is called whenever the user clicks on the map canvas and is passed a QMouseEvent as an argument.

We will now write some functionality which:

  1. Loops through all visible vector layers and for each:
    • Deselects all features
    • Loops through all features and for each:
      • Determines their distance from the mouse click
      • Keeps track of the closest feature and its distance
  2. Determines the closest feature from all layers
  3. Selects that feature

 

  • Add the following method to the NearestFeatureMapTool class:
def canvasReleaseEvent(self, mouseEvent):
    """
    Each time the mouse is clicked on the map canvas, perform
    the following tasks:
        Loop through all visible vector layers and for each:
            Ensure no features are selected
            Determine the distance of the closes feature in the layer to the mouse click
            Keep track of the layer id and id of the closest feature
        Select the id of the closes feature
    """

    layerData = []

    for layer in self.canvas.layers():

        if layer.type() != QgsMapLayer.VectorLayer:
            # Ignore this layer as it's not a vector
            continue

        if layer.featureCount() == 0:
            # There are no features - skip
            continue

        layer.removeSelection()

The layers() method of QgsMapCanvas (stored earlier in self.canvas) returns a list of QgsMapLayer. These are references to all visible layers and could represent vector layers, raster layers or even plugin layers.

We use the type() and featureCount() methods to skip non-vector layers and empty vector layers.

Finally we use the layer’s removeSelection() method to clear any existing selection. layerData is a list that we’ll use in a moment.

Our plugin now clears the selection in all visible vector layers.

  • Open the Shapefiles included in the Data folder.
  • Make a selection of one or more layers.
  • Reload the plugin and ensure it is working as expected (removing any selection).

Accessing Features and Geometry

We now need access to each feature and its geometry to determine its distance from the mouse click.

  • Add the following code to canvasReleaseEvent() within the loop over layers:
# Determine the location of the click in real-world coords
layerPoint = self.toLayerCoordinates( layer, mouseEvent.pos() )

shortestDistance = float("inf")
closestFeatureId = -1

# Loop through all features in the layer
for f in layer.getFeatures():
    dist = f.geometry().distance( QgsGeometry.fromPoint( layerPoint) )
    if dist < shortestDistance:
        shortestDistance = dist
        closestFeatureId = f.id()

info = (layer, closestFeatureId, shortestDistance)
layerData.append(info)

The mouse click event (a QMouseEvent) is stored in mouseEvent. Its pos() method returns a QPoint describing the position of the mouse click relative to the map canvas (x and y pixel coordinates). To calculate its distance to each feature we’ll need to first convert the mouse click position into real world (layer) coordinates. This can be done using a call to QgsMapTool.toLayerCoordinates() which automatically deals with on-the-fly projection and returns a QPoint in layer coordinates.

The features of a vector layer can be accessed using the layer’s getFeatures() method which returns (by default) a list of all QgsFeature in the layer that we can iterate over using a simple loop.

With access to features we can easily gain access to geometry using QgsFeature.geometry(). The QgsGeometry class has a number of spatial relationship methods including distance() which returns the distance to a second QgsGeometry passed as an argument.

In the code above we loop over all features, keeping track of the feature id of the closest feature using QgsFeature.id(). The shortest distance and closest feature id are stored in shortestDistance and closestFeature. When we are finished iterating through all the features in this layer, we store a note of the layer, its closest feature id and associated distance into layerData.

Note that we convert layerPoint (a QgsPoint) into a QgsGeometry so we can use it directly in spatial relationship operations such as QgsGeometry.distance().

Completing canvasReleaseEvent

We’re almost done. At this point layerData is a list of tuples, one for each vector layer containing:

  1. A reference to the layer
  2. The id of the closest feature within that layer
  3. The distance of that closest feature from the mouse click

Now we can simply sort layerData by distance (its 3rd column) and make a selection based on the layer and feature in the first row of layerData.

  • Add the following code to canvasReleaseEvent() outside the outer for loop:
if not len(layerData) > 0:
    # Looks like no vector layers were found - do nothing
    return

# Sort the layer information by shortest distance
layerData.sort( key=lambda element: element[2] )

# Select the closest feature
layerWithClosestFeature, closestFeatureId, shortestDistance = layerData[0]
layerWithClosestFeature.select( closestFeatureId )

The code above returns early if no workable vector layers were found. It sorts layerData (the list of tuples) by the 3rd element (the distance).

The code then calls QgsVectorLayer.select() to select the closest feature by its feature id.

The plugin should now be finished.

  • Reload the plugin
  • Ensure it works as expected.

Summary

Within this tutorial we’ve worked briefly with the following parts of the QGIS API:

  • Map Tools
  • Map Canvas
  • Vector Layers
  • Features
  • Geometry

Hopefully this has been a useful tutorial. Please feel free to contact us with any specific questions.

You may also like...

Mergin Maps, a field data collection app based on QGIS. Mergin Maps makes field work easy with its simple interface and cloud-based sync. Available on Android, iOS and Windows. Screenshots of the Mergin Maps mobile app for Field Data Collection
Get it on Google Play Get it on Apple store
Learn More

A Quick Guide to Getting Started with PyQGIS on Windows

Getting started with Python and QGIS can be a bit overwhelming. In this post we give you a quick start to get you up and running and maybe make your PyQGIS life a little easier. There are likely many ways to setup a working PyQGIS development environment—this one works pretty well. Contents Requirements Installing OSGeo4W Setting the Environment Python Packages Working on the Command Line IDE Example Workflow Creating a New Plugin Working with Existing Plugin Code Troubleshooting
Learn More

QGIS Development with Plugin Builder and pb_tool

The Plugin Builder is a great tool for generating a working plugin project that you can customize. One of the main tasks in the development cycle is deploying the plugin to the QGIS plugin directory for testing. Plugin Builder comes with a Makefile that can be used on Linux and OS X to aid in development. Depending on your configuration, the Makefile may work on Windows. To help in managing development of your projects, we’ve come up with another option—a Python tool called pb_tool, which works anywhere QGIS runs.
Learn More