The PostgreSQL Connection Service File pg_service.conf is nothing new. It has existed for quite some time and maybe you have already used it sometimes too. But not only the new QGIS plugin PG service parser is a reason to write about our love for this file, as well we generally think it’s time to show you how it can be used for really cool things.

What is the Connection Service File?

The Connection Service File allows you to save connection settings for each so-called “service” locally.

So when you have a database called gis on a local PostgreSQL with port 5432 and username/password is docker/docker you can store this as a service called my-local-gis.

# Local GIS Database for Testing purposes

host=localhost port=5432 dbname=gis user=docker password=docker

This Connection Service File is called pg_service.conf and is by client applications (such as psql or QGIS) generally found directly in the user directory. In Windows it is then found in the user’s application directory postgresql.pg_service.conf. And in Linux it is by default located directly in the user’s directory ~/.pg_service.conf. 

But it doesn’t necessarily have to be there. The file can be anywhere on the system (or on a network drive) as long as you set the environment variable PGSERVICEFILE accordingly:

export PGSERVICEFILE=/home/dave/connectionfiles/pg_service.conf 

Once you have done this, the client applications will search there first – and find it.

If the above are not set, there is also another environment variable PGSYSCONFDIR which is a folder which is searched for the file pg_service.conf.

Once you have this, the service name can be used in the client application. That means in psql it would look like this:

~$ psql service=my-local-gis
psql (14.11 (Ubuntu 14.11-0ubuntu0.22.04.1), server 14.5 (Debian 14.5-1.pgdg110+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

gis=#

And in QGIS like this:

If you then add a layer in QGIS, only the name of the service is written in the project file. Neither the connection parameters nor username/password are saved. In addition to the security aspect, this has various advantages, more on this below.

But you don’t have to pass all of these parameters to a service. If you only pass parts of them (e.g. without the database), then you have to pass them when the connection is called:

$psql "service=my-local-gis dbname=gis"
psql (14.11 (Ubuntu 14.11-0ubuntu0.22.04.1), server 14.5 (Debian 14.5-1.pgdg110+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

gis=#

You can also override parameters. If you have a database gis configured in the service, but you want to connect the database web, you can specify the service and explicit the database:

$psql "service=my-local-gis dbname=web"
psql (14.11 (Ubuntu 14.11-0ubuntu0.22.04.1), server 14.5 (Debian 14.5-1.pgdg110+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

web=#

Of course the same applies to QGIS.

And regarding the environment variables mentioned, you can also set a standard service.

export PGSERVICE=my-local-gis

Particularly pleasant in daily work with always the same database.

$ psql
psql (14.11 (Ubuntu 14.11-0ubuntu0.22.04.1), server 14.5 (Debian 14.5-1.pgdg110+1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

gis=#

And why is it particularly cool?

There are several reasons why such a file is useful:

• Security: You don’t have to save the connection parameters anywhere in the client files (e.g. QGIS project files). Keep in mind that they are still plain text in the service file.
• Decoupling: You can change the connection parameters without having to change the settings in client files (e.g. QGIS project files).
• Multi-User: You can save the file on a network drive. As long as the environment variable of the local systems points to this file, all users can access the database with the same parameters.
• Diversity: You can use the same project file to access different databases with the same structure if only the name of the service remains the same.

For the last reason, here are three use cases.

Support-Case

Someone reports a problem in QGIS on a specific case with their database. Since the problem cannot be reproduced, they send us a DB dump of a schema and a QGIS project file. The layers in the QGIS project file are linked to a service. Now we can restore the dump on our local database and access it with our own, but same named, service. The problem can be reproduced.

INTERLIS

With INTERLIS the structure of a database schema is precisely specified. If e.g. the canton has built the physical database for it and configured a supernice QGIS project, they can provide the project file to a company without also providing the database structure. The company can build the schema based on the INTERLIS model on its own PostgreSQL database and access it using its own service with the same name.

Test/Prod Switching

You can access a test and a production database with the same QGIS project if you have set the environment variable for the connection service file accordingly per QGIS profile.

You create two connection service files.

The one to the test database /home/dave/connectionfiles/test/pg_service.conf:

[my-local-gis]
host=localhost
port=54322
dbname=gis-test

And the one for the production database /home/dave/connectionfiles/prod/pg_service.conf:

[my-local-gis]
host=localhost
port=54322
dbname=gis-productive

In QGIS you create two profiles “Test” and “Prod”:

And you set the environment variable for each profile PGSERVICEFILE which should be used (in the menu Settings > Options… and there under System scroll down to Environment

or

If you now use the service my-local-gis in a QGIS layer, it connects the database prod in the “Prod” profile and the database test in the “Test” profile.

The authentication configuration

Let’s have a look at the authentication. If you have the connection service file on a network drive and make it available to several users, you may not want everyone to access it with the same login. Or you generally don’t want any user information in this file. This can be elegantly combined with the authentication configuration in QGIS.

If you want to make a QGIS project file available to multiple users, you create the layers with a service. This service contains all connection parameters except the login information.

This login information is transferred using QGIS authentication.

You also configure this authentication per QGIS profile we mentioned above. This is done via Menu Settings > Options… and there under Authentication:

(or directly where you create the PostgreSQL connection)

If you add such a layer, the service and the ID of the authentication configuration are saved in the QGIS project file. This is in this case mylogin. Of course this name must be communicated to the other users so that they can also set  the ID for their login to mylogin.

Of course, you can use multiple authentication configurations per profile.

QGIS Plugin

And yes, there is now a great plugin to configure these services directly in QGIS. This means you no longer have to deal with text-based INI files. It’s called PG service parser:

It finds the connection service file according to the mentioned environment variables PGSERVICEFILE or PGSYSCONFDIR or at its default location.

As well it’s super easy to create new services by duplicating existing ones.

And for the Devs

And what would a blog post be without some geek food? The back end of this plugin is published on PYPI and can be easily installed with pip install pgserviceparser and then be used in Python.

For example to list all the service names. 

>>> import pgserviceparser
>>> pgserviceparser.service_names()
['my-local-gis', 'another-local-gis', 'opengisch-demo-pg']

Optionally you can pass a config file path. Otherwise it gets it by the mentioned mechanism.

Or to receive the configuration from the given service name as a dict.

>>> pgserviceparser.service_config('my-local-gis')
{'host': 'localhost', 'port': '54322', 'dbname': 'gis', 'user': 'docker', 'password': 'docker'}

There are some more functions. Check them out here on GitHub or in the documentation.

Well then

We hope you share our enthusiasm for this beautiful file – at least after reading this blog post. And if not – feel free to tell us why you don’t in the comments 

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.

At OPENGIS.CH, we’ve been working lately on improving the DXF Export QGIS functionality for the upcoming release 3.38. In the meantime, we’ve also added nice UX enhancements for making it easier and much more powerful to use!

Let’s see a short review.

DXF Export app dialog and processing algorithm harmonized

You can use either the app dialog or the processing algorithm, both of them offer you equivalent functionality. They are now completely harmonized!

Export settings can now be exported to an XML file

You can now have multiple settings per project available in XML, making it possible to reuse them in your workflows or share them with colleagues.

All settings are now well remembered between dialog sessions

QGIS users told us there were some dialog options that were not remembered between QGIS sessions and had to be reconfigured each time. That’s no longer the case, making it easier to reuse previous choices.

“Output layer attribute” column is now always visible in the DXF Export layer tree

We’ve made sure that you won’t miss it anymore.

Possibility to export only the current map selection

Filter features to be exported via layer selection, and even combine this filter with the existing map extent one.

Empty layers are no longer exported to DXF

When applying spatial filters like feature selection and map extent, you might end up with empty layers to be exported. Well, those won’t be exported anymore, producing cleaner DXF output files for you.

Possibility to override the export name of individual layers

It’s often the case where your layer names are not clean and tidy to be displayed. From now on, you can easily specify how your output DXF layers should be named, without altering your original project layers.

We’ve also fixed some minor UX bugs and annoyances that were present when exporting layers to DXF format, so that we can enjoy using it. Happy DXF exporting!

We would like to thank the Swiss QGIS user group for giving us the possibility to improve the important DXF part of QGIS

Sorry, this entry is only available in French.

Focused on stability and usability improvements, most users will find something to celebrate in QField 3.2

Main highlights

This new release introduces project-defined tracking sessions, which are automatically activated when the project is loaded. Defined while setting up and tweaking a project on QGIS, these sessions permit the automated tracking of device positions without taking any action in QField beyond opening the project itself. This liberates field users from remembering to launch a session on app launch and lowers the knowledge required to collect such data. For more details, please read the relevant QField documentation section.

As good as the above-described functionality sounds, it really shines through in cloud projects when paired with two other new featurs.

First, cloud projects can now automatically push accumulated changes at regular intervals. The functionality can be manually toggled for any cloud project by going to the synchronization panel in QField and activating the relevant toggle (see middle screenshot above). It can also be turned on project load by enabling automatic push when setting up the project in QGIS via the project properties dialog. When activated through this project setting, the functionality will always be activated, and the need for field users to take any action will be removed.

Pushing changes regularly is great, but it could easily have gotten in the way of blocking popups. This is why QField 3.2 can now push changes and synchronize cloud projects in the background. We still kept a ‘successfully pushed changes’ toast message to let you know the magic has happened

With all of the above, cloud projects on QField can now deliver near real-time tracking of devices in the field, all configured on one desktop machine and deployed through QFieldCloud. Thanks to Groupements forestiers Québec for sponsoring these enhancements.

Other noteworthy feature additions in this release include:

• A brand new undo/redo mechanism allows users to rollback feature addition, editing, and/or deletion at will. The redesigned QField main menu is accessible by long pressing on the top-left dashboard button.
• Support for projects’ titles and copyright map decorations as overlays on top of the map canvas in QField allows projects to better convey attributions and additional context through informative titles.

Additional improvements

The QFieldCloud user experience continues to be improved. In this release, we have reworked the visual feedback provided when downloading and synchronizing projects through the addition of a progress bar as well as additional details, such as the overall size of the files being fetched. In addition, a visual indicator has been added to the dashboard and the cloud projects list to alert users to the presence of a newer project file on the cloud for projects locally available on the device.

With that said, if you haven’t signed onto QFieldCloud yet, try it! Psst, the community account is free

The creation of relationship children during feature digitizing is now smoother as we lifted the requirement to save a parent feature before creating children. Users can now proceed in the order that feels most natural to them.

Finally, Android users will be happy to hear that a significant rework of native camera, gallery, and file picker activities has led to increased stability and much better integration with Android itself. Activities such as the gallery are now properly overlayed on top of the QField map canvas instead of showing a black screen.

Sorry, this entry is only available in French.

According to the definition of software quality given by french Wikipedia

An overall assessment of quality takes into account external factors, directly observable by the user, as well as internal factors, observable by engineers during code reviews or maintenance work.

I have chosen in this article to only talk about the latter. The quality of software and more precisely QGIS is therefore not limited to what is described here. There is still much to say about:

• Taking user feedback into account,
• the documentation writing process,
• translation management,
• interoperability through the implementation of standards,
• the extensibility using API,
• the reversibility and resilience of the open source model…

These are subjects that we care a lot and deserve their own article.

I will focus here on the following issue: QGIS is free software and allows anyone with the necessary skills to modify the software. But how can we ensure that the multiple proposals for modifications to the software contribute to its improvement and do not harm its future maintenance?

Self-discipline

All developers contributing to QGIS code doesn’t belong to the same organization. They don’t all live in the same country, don’t necessarily have the same culture and don’t necessarily share the same interests or ambitions for the software. However, they share the awareness of modifying a common good and the desire to take care of it.

This awareness transcends professional awareness, the developer not only has a responsibility towards his employer, but also towards the entire community of users and contributors to the software.

This self-discipline is the foundation of the quality of the contributions of software like QGIS.

However, to err is human and it is essential to carry out checks for each modification proposal.

Automatic checks

With each modification proposal (called Pull Request or Merge Request), the QGIS GitHub platform automatically launches a set of automatic checks.

Example of proposed modification

Result of automatic checks on a modification proposal

The first of these checks is to build QGIS on the different systems on which it is distributed (Linux, Windows, MacOS) by integrating the proposed modification. It is inconceivable to integrate a modification that would prevent the application from being built on one of these systems.

The tests

The first problem posed by a proposed modification is the following “How can we be sure that what is going to be introduced does not break what already exists?”

To validate this assertion, we rely on automatic tests. This is a set of micro-programs called tests, which only purpose is to validate that part of the application behaves as expected. For example, there is a test which validates that when the user adds an entry in a data layer, then this entry is then present in the data layer. If a modification were to break this behavior, then the test would fail and the proposal would be rejected (or more likely corrected).

This makes it possible in particular to avoid regressions (they are very often called non-regression tests) and also to qualify the expected behavior.

There are approximately 1.3 Million lines of code for the QGIS application and 420K lines of test code, a ratio of 1 to 3. The presence of tests is mandatory for adding functionality, therefore the quantity of test code increases with the quantity of application code.

In blue the number of lines of code in QGIS, in red the number of lines of tests

There are currently over 900 groups of automatic tests in QGIS, most of which run in less than 2 seconds, for a total execution time of around 30 minutes.

We also see that certain parts of the QGIS code – the most recent – are better covered by the tests than other older ones. Developers are gradually working to improve this situation to reduce technical debt.

Code checks

Analogous to using a spell checker when writing a document, we carry out a set of quality checks on the source code. We check, for example, that the proposed modification does not contain misspelled words or “banned” words, that the API documentation has been correctly written or that the modified code respects certain formal rules of the programming language.

We recently had the opportunity to add a check based on the clang-tidy tool. The latter relies on the Clang compiler. It is capable of detecting programming errors by carrying out a static analysis of the code.

Clang-tidy is, for example, capable of detecting “narrowing conversions”.

Example of detecting “narrowing conversions”

In the example above, Clang-tidy detects that there has been a “narrowing conversion” and that the value of the port used in the network proxy configuration “may” be corrupted. In this case, this problem was reported on the QGIS issues platform and had to be corrected.

At that time, clang-tidy was not in place. Its use would have made it possible to avoid this anomaly and all the steps which led to its correction (exhaustive description of the issue, multiple exchanges to be able to reproduce it, investigation, correction, review of the modification), meaning a significant amount of human time which could thus have been avoided.

Peer review

A proposed modification that would validate all of the automatic checks described above would not necessarily be integrated into the QGIS code automatically. In fact, its code may be poorly designed or the modification poorly thought out. The relevance of the functionality may be doubtful, or duplicated with another. The integration of the modification would therefore potentially cause a burden for the people in charge of the corrective or evolutionary maintenance of the software.

It is therefore essential to include a human review in the process of accepting a modification.

This is more of a rereading of the substance of the proposal than of the form. For the latter, we favor the automatic checks described above in order to simplify the review process.

Therefore, human proofreading takes time, and this effort is growing with the quantity of modifications proposed in the QGIS code. The question of its funding arises, and discussions are in progress. The QGIS.org association notably dedicates a significant part of its budget to fund code reviews.

More than 100 modification proposals were reviewed and integrated during the month of December 2023. More than 30 different people contributed. More than 2000 files have been modified.

Therefore the wait for a proofreading can sometimes be long. It is also often the moment when disagreements are expressed. It is therefore a phase which can prove frustrating for contributors, but it is an important and rich moment in the community life of a free project.

To be continued !

As a core QGIS developer, and as a pure player OpenSource company, we believe it is fundamental to be involved in each step of the contribution process.

We are investing in the review process, improving automatic checks, and in the QGIS quality process in general. And we will continue to invest in these topics in order to help make QGIS a long-lasting and stable software.

If you would like to contribute or simply learn more about QGIS, do not hesitate to contact us at infos+qgis@oslandia.com and consult our QGIS support proposal.

Sorry, this entry is only available in French.

Sorry, this entry is only available in French.