Wednesday, February 10, 2021

HydroloGIS turns 16 + joining Aries

Today it’s HydroloGIS’ birthday. We count 16 years of passionate open source engagement in the fields of GIS and environmental engineering, against the odds of those that told us in the face that you can’t live on open source and share your work and your knowledge for free. 

Well, in our 16th birthday we have one big news to share. It is with great pleasure that we announce the start of our collaboration with the team of Artificial Intelligence for Environment and Sustainability (in short ARIES) of the Basque Center for Climate Change. 
With this collaboration we finally close a circle going back to our roots: integrated modelling for the environment. This is how we started over 15 years ago, developing and maintaining the open source project of the Hortonmachine. Having the possibility to join what we think is the most important open source project addressing integrated modelling, is huge for us. 
Silvia will be working on water related modelling, mostly hydrology and hydraulics. Andrea will be joining the klab core engine team as a GIS expert.

So don’t be confused if you find us on the Aries page. We are still HydroloGIS, but with a beautiful breeze of research that flows steadily through our souls. :-)
And so that it is clear, the Hortonmachine project will be strengthened by this collaboration (klab already uses the Hortonmachine modules) and the SMASH and Geopaparazzi projects will move on as they were doing before, in case you were worried.

Ahhh, life's goood!

Friday, January 15, 2021

SMASH 1.6.3 is out - experimental Postgis

We just released SMASH 1.6.3 on the stores. It should be available within the day.

This is basically a bugfix release, but we decided to also release one feature that will be very important in the future: postgis support.

The biggest issue with the last release is that we introduced an error trapping engine to allow the user to better send issues of the application to the developers. The problem we didn't notice for soem obscure reason is that everytime an online map tile would not be properly downloaded, this would result into a crash report. Your many email explained well the situation :-)))  This version fixes this.

We also enhanced the WMS source creator (which still needs much love) with the possibility to select the image format:


The note taking workflow just got a bit simpler. You can just tap on the text and the edit field is empty if the note had never been edited. This allows for less taps, which is great out in the field.

 Last but not least, we started adding PostGIS support. This will be a 2 steps effort. Step one (this one) is to add full online support. This means that you now can access the remote db view from the layers list:

Let's assume you have a test db running in your network, something like:

Then you can add a new remote db as:

you can enter the table or trigger a refresh, which will also help you in testing the connection:

you can also optionally add a where condition, if you need to load only a subset of the data.

Once the db is completed, it will stay in your list of available db datasets. Just add it as a layer to teh map using the rightmost button:

which will result in:

and on the map:

well, this looks kind of ugly. You might have seen in some previous post that SMASH supports styling via SLD specification. Well, adding style the same as we do for geopackage brings us to this:

Nice right?

Now this is a vector layer that can be selected:

The above view allows also for table data editing.

And also geometric editing is allowed:

So, why is this the first of two steps? Well, out in the field we often do not have network access, so we will need to create a solution that would allow to create a local geopackage cache, which then eventually can sync back to the remote db.

But we decided to first release the direct online postgis access, since that still has some nice usecases in situations in which the network is available. Once we have this one working stable, we will move on to the next step. So, if you are into Postgis and want to help, please test and report issues.



Wednesday, January 6, 2021

Hortonmachine 0.10.0 released - get a grip on your basins size

It sure is a while that I don't post about the Hortonmachine. While we work a lot on it and with it, it always kind of stays in background. Well today we decided to make a new release due to some tools that will be used in production environments. And I think one is definitely worth to mention.

Some of you might have used the netnumbering module to extract subbasins on a network. The module also accepts monitoring points to allow splitting the basin in fixed points. 

For a watershed like this:

The result is something like:

Now, some models, as for example Riccardo Rigon's Geoframe, need a bit more control over the basin's size. This is why, on Riccardo's request, we added the possibility to set a desired output basins size and  buffer threshold, to handle some degree of flexibility.

It has been quite some fun and recursive exercise to aggregate in the best possible way basins that are too small. We added plantuml mindmap outputs to be able to better check the way the hierarchy is getting merged. This has been really helpful, in particular to check large basins and also to check on monitoring point positions, which is where basins are meant to keep a fixed border.

 Here the mindmap of the above basin, with references to areas and fixed basins:

and finally the aggregated basins mindmap assuming a desired basin size of 100000 cells (10*10 meters here) and a 20% of threshold:

and the resulting subbasins map:

The result is quite nifty, given the fact that the only basins that do not obey to the 20% buffer are either basins that are fixed and at the very top (so they can't be merged with anyone) or basin 6 which is "blocked" between 2 fixed basins, so same as before, no way to enlarge it by aggregation.

Well, you can find a release containing the tools at the usual release download site of the Hortonmachine here.

Enjoy! We surely did :-)


Monday, November 30, 2020

SMASH 1.6.1 is out geo-tiffs and -fences

Today we released SMASH 1.6.1 to the stores.

The main reason we are doing this is because there was a startup bug that was freezing SMASH, mostly on Xiaomi devices. If you experienced something like this:

...well, update and it will be gone. 


That said, we also have some significant additions:

Geotiff support

Up to now we had tiff support needing world files and projection file to be present. Now geotiff tag reading is supported, so "real" geotiffs can be read. This is quite handy, since the image file gets back to be a single one again. Same as for other imagery types it supports overall transparency and single color removal, which allows for some nice technical map overlays (for example):


Since were we live the COVID lockdown imposed people to only walk a certain distance from home, we decided to lend a helping hand. You now can add fences and make them ring when you enter or exit the radius. 

 The fence option is available from the bottom toolbar:

Once you hit it, it inserts a new fence in the current map center:

Defining the fence with no sound will result in a grey colored fence to show that it is "inactive":

To modify an existing fence, just place the map center inside its circle and then long tap the fence button. The fence dialog will appear again, this time with an additional possibility to remove the fence.

Now that the fence is active (on enter and exit sound set), the color is red and there are two borders. The green one reminds that on entering there will be a sound, and the blue one that on exit there will be a sound.



Just so you know, fences are not saved into the project, but only in the application preferences. This means that on uninstall or app cache removal..  they will disappear.

Log lines themes

While this is possibly my favorite feature of this release, I have written about it in my last post here, so just check there for descriptions and images.

Other small features

Magnet icon for center on gps: if you are in automatic center on gps mode (double tap on gps icon), it will be visible through a small magnet:

Location permission at startup now allows to access the settings to activate directly background gps usage even when the app is not in use. 

Note sharing now also contains the complete OSM url (not sure how that could have been missing up to now):

Project browsing has been made more readable. Not sure how about you, but in my case I just create a new project and using the default proposed name, I simply add something meaningful at the end of it, after an underscore. This allows me to order my projects alphabetically also by timestamp. The problem is that when I want to load an older project, I find myself in this:

The smash_timestamp take the whole line and makes everything unreadable. So now you can activate the upper right filter and the timestamp will be removed from the name and added as additional info ( day and time if available):

I guess that is it for this release. Enjoy!

Tuesday, November 10, 2020

A better view on GPS logs in SMASH

It is a while now that I am collecting gps logs done by car and bike to get a better idea on how to "correct" a bit jumping points. This is how the Kalman filter was added to the GPS points pipeline as additional information of choice (not always you want the corrected information).

As usual, when working on stuff, you need to get a better view over things, which usually means also to get a qualitative idea of what is going on. This is how we got to gradient coloring of the gps lines.

The result is actually quite cool and gives a good insight of what happened and when. So we decided that this will enter the next release of SMASH.

How does it work?

The GPS log properties panel now has an additional combobox that permits the selection of a color palette for a particular information.

If a palette is selected it overrides the solid color of the log, applying the palette to the line.

The result in the case of elevation is something like:

Which gives a pretty good idea about where the peak is. Also, assuming the temporal direction of the log, the downhill parts of the log are drawn with dotted border.

You might want to know where the steep parts are, right? That is what slope can help you in:

A maybe less interesting vision is the one colored by speed:

And last but not least, it is possible to view the log by its accuracy. This can be quite interesting for surveyors, if they decide to pair to the survey a complete log, which will then tell them where accuracy was better. Obviously in the below map it is simple to see where the galleries are:

I find this view of the GPS logs quite intriguing. Some might say that it would be better to have this information before a trip... and they are right. Which is why we added this coloring option also to gpx data layers. If the gpx contain the elevation information, then the elevation and slope palette are available:

I hope you'll also enjoy it... in the next release :-)

Tuesday, October 20, 2020

SMASH 1.6.0 is out - all the love for geopackage

A couple of days ago we released SMASH 1.6.0 to the stores. So why is this post coming only now?

Because Google store changed its policy for apps that use background location and is making me life impossible to get a release to the store. Since I am still waiting to understand why I am doing what wrong, I decided to make this post anyways and place the apk of the android version as a download.

Basically this release brings out all the love we have for the geopackage format.

Geopackage geometry editing

We finally have a first version of SMASH with vector editing. Last week I shared a video in which editing has been showcased. Well, not much has changed since that day, so to have a look at editing, just have a look at this. I guess it is better than 500 screenshots I would place here below  :-)

Geopackage export

We have been thinking a lot about the geopackage format. We were tempted to migrate the internal geopaparazzi/SMASH format to geopackage directly, but then we decided it would be better to keep the current format to maintain backward compatibility and compatibility with geopaparazzi.

Since many people see Geopackage also as a good "transfer" format from mobile to PC, we added a new export option. It is now possible to export the complete survey as a single geopackage with everything in it.


A minor change (but one I like) is that the GPS notification message on Android is dynamic. It shows a lot of nice information about the current state of the GPS, position and logging status.

Where to download for Android?

Until the issue with the google store is not sorted out, please get your copy of SMASH 1.6.0 HERE.

Friday, September 25, 2020

SMASH 1.5.0 is out - raster enhancements, CRS, exports and measuring tool

Today we released SMASH 1.5.0 for Android and IOS to the markets.

Quite some work has been done in the raster and projection system backbone.

Raster data and CRS

When it comes to geopackage a lot is allowed. Until now SMASH (as geopaparazzi) allowed to view only tile data generated using the google tiling scheme. And moreover data was visible only at zoom levels for which the tiles had been prepared. This is a problem inherited in many workflows from the preparation of mbtiles datasets. Often in these cases zoomlevel beyond the original image's resolution are generated bloating the database to gigabyte size even for small regions.

Gdal is smart and by default generates only one zoomlevel at the optimal image resolution and using the best tile schema. And yes, also the projection might be different from the usual epsg 3857 used in the google tile scheme. 

Supporting this has been important, since it allows to create geopackages of big imagery maintaining more or less the same size. So now we have two ways to load geopackage tile datasets.

In the properties view of geopackage layers you will have more options now:

Apart of the opacity now there are two more options. The first enables the possibility to load a wider range of tiles schemas. Here an example of an ortophoto reprojected with gdal and packaged as geopackage:

This features has a larger memory footprint as it loads the tiles in memory as an image. So if the dataset is too large for your device you might need to go back to loading it with classic tiling scheme.

Since we are at it, what is the second option? It allows to select a color to make transparent. This is very useful when there is the need to overlay technical maps over an elevation model or ortophoto. But it also solves a situation like this:

These are two geopackage layers of a reprojected ortophoto. When both loaded the black collar of one covers the other.  It is possible to set the black collar to transparent:

Which results in:

which looks already better. Actually, when the jpeg geopackage tiles are generated, there is some interpolation and antialiasing going on, which makes part of the color not completely black, hence the remaining pixels. But if we load the original geotiffs and remove the collar from the upper image, the result is nice and clean (and also way faster to load);

This is how it looks like if we remove white from the image below:

So what about reprojecting data?

Yes, we added a nice CRS view in the settings:

from which we also can add new projections by epsg code:

The projection information is downloaded from the internet and the projection is registered:

Available projections are used to reproject vector datasets (geopackage layers and shapefiles) onto the map. This works very nicely.

If a dataset is loaded into the layerview and its projection is not known, SMASH will download and register it:

If the projection is not recognised, the user will be prompted to supply the epsg code for the projection:

While vector datasets are nicely reprojected, with raster it is a different story. At the moment only the raster boundingbox is reprojected to place the image in the right spot, but the result can be also quite bad. So while it looks nice in one corner:

it can be bad in the other:

So this is more a fallback workaround for the case you forgot to prepare a dataset, it is not really a solution yet.

So, handle with care!

GPS, to filter or not to filter

In previous SMASH releases we introduced the Kalman filtered GPS signal. It was available only for gps logs that were recorded. To make things more consistent, the behavior has been extended to all GPS dependent features. So the filtered GPS setting

now has some impact. As a reminder, in case of gps logs original and filtered GPS signal are BOTH always saved into the database and the user can choose to look at the original and/or filtered log. This still applies. 

But now the GPS position layer obeys to the setting. And when you insert a note in the gps position, the one defined in the settings will be used. 

This might sound scary but it actually is not. Just remember to have your settings right.

To aid the user, the "other" position is shown using a tin line and a dot starting from the current used position:


For example when driving through a tunnel, the original position on android can be jumpy:

New Toolbar

Since we are starting to prepare for geometry editing in geopackages, we started to think about a proper toolbar already. We came up with a small handler in the lower right corner:

which allows to activate the bottom toolbar:

The feature info tool has already been migrated from the right drawer down here.

And a new tool has been added in this release: the measurement tool. For those coming from geopaparazzi, I am sure they were missing it. With this tool you can drag over the map and have a qualitative measurement of the distance:

Other features

SMASH has two new export formats: GPX and KML. These export the whole survey project to a single GPX/KML file:

If instead you need to export one single log to gpx (since that is a great exchange format for SMASH users, being able to overlay it), you can do that sliding a log from the log view:

Last but not least, during logging now some time and distance (normal and filtered... note the difference when driving through several tunnels with a jumpy GPS) related information are shown:

That is all for version 1.5.0. Enjoy!!