J-Term 2021, Week 2: Getting Group By Up and Running
This week I got my plugin, “Middlebury Minimal GIS,” up and running with a single algorithm: Group By. I started by exporting the Group By model to a Python script and seeing what needed to be changed. When I originally learned that I could export models to Python scripts, I was actually disappointed because I thought my work would be too easy. If QGIS could take care of the conversion for me, then what was my role?
I quickly learned that exporting as a .py file was only the very beginning of the process. The code was there, but the formatting made it difficult to read, and there were almost no comments. There were also a few changes Prof. Holler and I had talked about making to the tool’s parameters, which I needed to figure out how to implement.
My first step was to format things nicely and add comments so that I and others could better understand what was going on. This required some learning on my part, as I had never written a QGIS Processing algorithm from scratch. I’d used the @alg decorator to convert regular Python scripts into Processing scripts, but if you want to package your script into a plugin, you have to do things the hard way. I relied on this guide from the QGIS documentation, as well as the script template from the QGIS Processing Toolbox (available under the Python menu in the Processing Toolbox as “Create New Script From Template…”), in order to figure out what the different methods of the QgsProcessingAlgorithm class do.
Once I had my script nicely formatted and commented, I made a couple of relatively easy fixes. I changed the input parameter from a QgsProcessingParameterVectorLayer to a QgsProcessingParameterFeatureSource in order to enable the user to check “Selected features only” if there is an active selection on the input layer. I also changed the “Dissolve Geometry” option to be negative by default. These changes made, I figured my algorithm would run smoothly and I would be all set.
Little did I know that my task was far from over. The Group By algorithm uses the Execute SQL tool from QGIS to do its main work. The input query from the model is a complicated concatenation of different expressions based on the values of the model’s inputs. However, after making some small changes to the script, this query no longer worked. This was particularly problematic for me because not only had I never written any SQL, but I also had very little idea what this complex concatenation was doing. I spent some quality time staring at the string being passed into the Execute SQL tool, but it was all Greek to me. So I took a step back and opened up the expression editor in the Execute SQL dialog. Looking up some of the operators and expressions I found in the string, I found syntax explanations and was better able to understand how the query was being built. But I was still missing one enormous piece of the puzzle: I didn’t know what an SQL query even looked like.
After some fruitful Googling, I realized that these queries actually looked much simpler than I would have imagined, using self-explanatory terms like “select” and “group by”. Finally I had enough knowledge to write a model query for the Group By tool based on an arbitrary set of parameters. I ran the query from the Execute SQL dialog, and it worked! From there it wasn’t too difficult to write some Python code to concatenate together a query string based on the parameters selected in the Group By tool. After testing my code, I took the final step of adding backticks (`) around each field name in my query string to ensure that the query would work even with “bad” field names (such as names containing problematic characters like colons). I had added a step to the algorithm that prints the query to the feedback dialog, so I could easily tell what string I was passing into Execute SQL.
Now it was, at last, time to package the script into a plugin! This process was, thankfully, quite straightforward. I used the Plugin Builder plugin to create a Processing Provider plugin, and Plugin Builder built all the necessary files. From there I just had to rename a couple of files and classes (I hadn’t quite known what I was doing when I entered names for my plugin and algorithm in the Plugin Builder menus) and then copy and paste the code for my algorithm into the groupby_algorithm.py file. I reloaded my plugin using the Plugin Reloader plugin (also a must-have for QGIS plugin development!) and tested it out. For maybe the first time ever in my programming career, it worked perfectly right off the bat. There’s a “Middlebury Minimal GIS” provider in my Processing Toolbox, and if I click the drop-down arrow, I can open my Group By tool. Next week I’ll be converting the Distance & Direction tool into a Python script, which will join the Group By algorithm in my new plugin.