Geopaparazzi Survey Server

Would you believe it? Only a month ago I was showing the Geopaparazzi Profiles Server developed by the guys at GeoAnalytic and now I am here again to write about the Geopaparazzi Survey Server (GSS).

What does Survey Server even mean? :-)

Well, while the Profile concept is a sophisticated way to handle survey data, background data, forms, spatialite databases and their way of synchronization, the GSS si something much smaller. But in our opinion it reflects the workflow of many, many surveyors and groups of surveyors.

The Geopaparazzi Survey Server (GSS) is a web application that allows geopaparazzi users to synchronize their project data with a central server.

Its companion is an Android app named Geopaparazzi Survey Server Sync (GSSS) available on google play. The app can connect to geopaparazzi projects and synchronize the data contained using the unique device ID to upload the data to the server.

Any device that connects to the server with its ID, will be accepted and if not available, the new id is automatically inserted in the central db.

So this is not about spatialite datasets, but only about geopaparazzi project files, that can contain notes, complex form notes, GPS logs and images.

The server application

The server application is packaged as a docker images and can be installed in the blink of an eye ( well, if your connection is fast). Once you install it, you get a nice and simple web application with a login

a dashboard, a mapview with the possibility to load/unload the data of your surveyors, zoom to it and check information of your notes, gps logs and images:

The surveyors' device that connects to the server is inserted in the surveyors list, if it is not registered already. There a human readable name can be assigned and some basic contact:

The mobile app

The mobile app has been done for android. FYI, we are also working on a desktop version app. There are several reasons for which we decided to go with an external app instead of adding this to geopaparazzi itself. The most important thing is that several cloud synchronization applications are getting born around geopaparazzi these days. We will probably have to let these ideas mature and then at some point it will be possible to converge on the best methodology.

But right now I find it more respectful to have an external app that uses its own way to collect the data from the geopaparazzi projects and send them to the server instance.

So let's have a short look at the Geopaparazzi Survey Server Sync, GSSS :-)

Well, it is a simple simple app with the possibility to load geopaparazzi projects. In it you can see what data are available for upload, i.e. dirty data:

Since the app uses the device id as surveyor id, some simple configurations need to be done:

Once the project file is loaded, the device id is ensured and the server url is inserted... well, just push the upload button! On successful upload the list of notes will be empty and your survey can go on using geopaparazzi.

Installation and training

A complete installation and quickstart guide is available here. Check it out to see all features available.

If you want to get a small training first hand from the developers, we will be giving a workshop about geopaparazzi and GSS at the following locations in the near future:

• Geopaparazzi Workshop at Foss4G EU in Portugal. Want to join us?
• Geopaparazzi Workshop at Foss4G in Dar es Salaam. Want to join us?
• Geopaparazzi Workshop at Foss4G Asia. Well, to be honest we haven't submitted the workshop yet, we'll see if it get's accepted :-)

We would also be very happy to be involved with this stack in projects in developing countries, where data collection and centralization is necessary.

If you are interested, please reach out to us.

Cloud Profiles - The Open Source Reference Server Implementation

Today it is with great joy that I just copy/paste an email that arrived in the geopaparazzi mailinglist. I already announced it, but this is the proper announcement by the guys from GeoAnalytic Inc. So I just leave this to you all, as written by Brent Fraser:


Hi All,Geopaparazzi 5.6.0 is now available at Google's Play Store (and GitHub as an APK).  One of my favorite new features is Cloud Profiles. Its purpose is to ease the data management tasks by automatically downloading project files, Spatialite databases and other files when connected to a 
"compatible" web server.    
 
And there's more!  When you activate a Profile, Geopaparazzi will set the basemap and attach the Spatialite databases ready for viewing in the Map View.    

So what about the "compatible" web server?  Since the Cloud Profiles list is a JSON text file, you could hand coded it and use a plain old web server like Apache or IIS.  Or do some server-side scripting if you want to get fancy. We've got an Open Source project started to be a "reference server" for this.  hopefully it will grow to provide more Geopaparazzi services, but right now it just servers up Cloud Profiles. Have a look at the blog post of more details:
 
https://geoanalytic.github.io/a-reference-server-for-geopaparazzi-cloud-profiles/    

There is a public version set up so you can experiment with Cloud Profiles (it delivers two demo data-sets: Vienna and Bolzano).  To try it out:   

1. Open up Geopaparazzi 5.6.0 on your device  
2. Go into Settings (using the gear icon on the action bar)  
3. Go into Cloud server settings  
4. Edit the "Cloud Profiles URL" to be https://geo.trailstewards.com/myprofiles/  (don't forget to have the trailing slash in the above URL). You don't need any values for the Username, Password, or Cloud Project server.  
5. Go back to the main dashboard screen and press the big Import icon (the one on the bottom left)  
6. Press the Cloud Profiles at the bottom of the Import options.  
7. Press the icon beside the Profile you would like to download.  Some of the files are large-ish so it is best to be connected via WiFi.    
8. After the Profile has downloaded, it's time to "activate" it:  
• On the main dashboard screen, press the menu icon on the far left of the action bar and select "Profiles"  
• You should see the Profile you downloaded listed there.  
• Press the Profile's gear icon to change the properties  
• Slide the Activate profile switch to the right to activate it  
• Go back to the main dashboard screen and press the Map View icon.    

You should see a basemap with and overlay as defined by your selected profile!  

Best regards,  
Brent Fraser  
GeoAnalytic Inc.


This is a great contribution to the geopaparazzi community and makes me feel that the geopaparazzi project is on the right track to be a good free and open source project.

Enjoy!






 

Geopaparazzi 5.6.0 is out - work with cloud profiles to handle surveys and data!

This release brings a ton of fixes and compatibility enhancements for newer Android versions.

And then it brings cloud profiles. And I think this one is huge! :-)

Cloud Profiles

In the last months a new company entered the geopaparazzi community and they have been doing nice contributions. The guys from Geoanalytic were working with us on the implementation of a more structured version of the cloud projects synchronization: Cloud Profiles!

We have been working on this already at the Bonn code sprint together with Cesar from the company Scolab. And now we are finally at a first release that supports this concept.

Cloud Profiles are a great way of easing Geopaparazzi’s data handling tasks. When a web server is configured to serve Cloud Profiles, Geopaparazzi can automatically download Projects, Basemaps, Spatialite Overlays, forms for Notes, and other files. When a user activates a downloaded Profile, Basemaps are made available, Overlays are attached to the Map View and layers are set to display.

Read more about it here.

The Geoanalytic guys have also been so nice to write a reference geopaparazzi profile server that can be used as a starting point. You can read more about it and download it from here.

The real power behind cloud profiles is that you can use your own server for this, but you could also make it much simpler by using Generic Cloud Server. You can set up your own Cloud Profile server by putting your files on a generic cloud file server like Dropbox or Google Drive, and editing a Cloud Profiles list like the one above and putting it on the cloud file server as well.

The very important fix

The most important fix (in my opinion) is the one resolving the GPS Location Limitations on Android Oreo. Basically it wasn't possible to log gps tracks with the screen off.

Technically it has been necessary to create a notification icon for geopaparazzi, so you will now have that visible when geopaparazzi is active.

The positive side is that:

1. you will always know when geopaparazzi is active
2. you can always see information about your position directly in the notification area

Other features and fixes

Values in settings. The settings screen now shows the actual values:

Buttons size. Small buttons are hard to see and press while on the trail. While it can be difficult to show a lot of information on a small screen, where there is room, you can now change the button and text size. This applies to the notes view for now:

Dynamic hints. Dashboard button hints are dynamic where possible. You can now see from there how many notes and logs you have in store. Just long-tap on the buttons:

Notes settings. The notes settings view is now accessible from the notes list (it was hidden in the gps data list menu). Access it from the palette icon:

PDF export. The PDF export now allows to export a subset of notes instead of everything contained in the project. The user can select the notes he/she likes to export and those will be converted in the pdf version.

Linked resources. It is now possible to view not only images stored in a Spatialite database when they are related to (geospatial) features but also for example PDF. 

Note that while a user can take pictures in the field and link them to a feature, in the case of PDF, the resource has to be linked before, i.e. it can be just viewed only from geopaparazzi.

Remove all. In the basemaps view it is now possible to remove all maps in one tap. This is really helpfull for those that are used to load maps through the load-folder option and then need a lot of time to remove them to have things more readable.

Mapurl service. One sad note is that the Tanto Mapurl service, which was used to download automagically configured mapurls based on WMS services, is no longer maintained and has therefore been removed also from geopaparazzi.

Fixes for:

• images taken within Geopaparazzi are not geotagged.
• unable to open form image thumbnails. 
• many other fixes have been done, the complete list is here.

Geopaparazzi 5.5.0 is out

I know this should have been about Hortonmachine tools for gvSIG, but as it happens, the job came in between and we are now releasing Geopaparazzi 5.5.0.

Two lines are mandatory, since there are a few new features. They are very few actually, since this should have been only a bugfix release.

Therefore many bugs have been fixed. Please continue to send them in via the ACRA integration.

Major features:

1. integration of the Geoss2Go project to create profiles in geopaparazzi. I wrote a complete description of the tool in this post. Now this is all available through the "Profiles" entry in the dashboard menu.
2. use of vector icons for dashboard and map tools, so that they adapt to style changes. Important for customized versions. This makes a great user experience, since these icons adapt very well to different resolutions and devices. As a drawback, all the icons had to be replaced and re-done. The once you will find most different are for sure those of the editing tools. Below a few screenshots:

Enjoy!

JGrasstools back in black: The Horton Machine - Part 1

I have to admit that when in far 2002, working at the University of Trento, Faculty of Environmental Engineering, we published the first release of The Horton Machine, I never thought we would ever change that name. The Horton Machine was a collection of around 40 GRASS modules written in C and dedicated to advanced Hydrology and Geomorphology. They represented the effort of the passed 10 years (now around 20) of professor Riccardo Rigon and his team.

At that time Riccardo and I were dreaming about a nice to use GUI for GRASS that would allow GRASS to be used more outside of the academic domain. In 2003 we started the JGrass project with that objective: create a userfriendly GUI for GRASS. The reaction of the GRASS community was bad, mostly (so they said) because Java was not open source. Also QGis was coming and becoming the natural choice for being an interface to GRASS.

Not at all in the mood of religious wars in 2006 we decided to join the java tribe and moved our resources to support the uDig project, where we happily lived and developed for many many years. We kind of stayed between two worlds, still using GRASS and its mapsets, but living in the userfriendly java world. :-)

At that time the processing libraries for Hydrology and Geomorphology (as well as LiDAR and forestry later on) were extracted into a library that could be used in standalone mode or inside uDig. That library, as a logical follow-up, got the name JGrasstools.

Had I only known better! In some (rare, but still!) occasions I and other JGrasstools developers have been asked why we still use the name JGrasstools, since we are not directly "bound" to GRASS. Well, I have been fighting over this a few times and had no hard feeling about this, apart of the huge work that it would have been to change everything. 

The last time it happened, was at the Foss4g conference in Paris. At the end of a great presentation given by Silvia about the tools we developed for forestry management using LiDAR data, (again) a member of the GRASS community asked the same old question in the questions interval dedicated to the presentation: Why do you still call it JGrasstools....?

This was the final straw for me. I still have to understand why people do certain things, but one thing was sure for me: we had to change that name, to make some of the GRASS community members sleep sweet dreams, and to be finally free!

So this is it. After 15 years of continuous development on the JGrasstools core, we go back to our origins: The Horton Machine.

The Horton Machine is now something more than just Hydrology and Geomorphology, there are other projects that support interaction with the mobile digital field mapping app Geopaparazzi, a module that supports interaction with spatial databases (also on android), the LESTO modules developed with the team of professor Giustino Tonon at the Free University of Bolzano... and beyond other things... also the plugins for the Desktop GIS gvSIG.

It has been quite an exercise to make this namespace migration and has taken me days between code refactoring, domain registration, maven publishing updates, documentation updating (still much work to be done there) and and and... but it is done now. Many will sleep sweet dreams, and I will be the first. And maybe at the next conference someone will ask a question related to the content of the presentation. Don't know what? A hint: "How did you get such a high single tree extraction rate from LiDAR data with your tools?" ;-)

 

 What will happen now?

Before the 13 international gvSIG conference we will do the first HortonMachine branded release and the together with it the connected release if gvSIG plugins.

Also maven releases of the modules will be done. At the time JGrasstools was at version 0.8.1. The HortonMachine will most probably start at 0.9.0. It sure should have been a major number, but well, we still need to reach the first major. :-)

In the next post we will show you what you will find in the release. Stay tuned!

Geopaparazzi 5.4.0 is out

We just released Geopaparazzi 5.4.0 into the wild!

It all started with a bug fix to have a 5.3.2 that would be able to read the latest mapsforge maps. For some unknown reason the backward compatibility had been lost and geopaparazzi was no longer able to visualize newer mapsforge maps. Being the most important map used, that had to be fixed.

Then we noticed that also GPS import was broken. In fact the new plugin system had a bug. That also had to be fixed.

Then we fixed some smaller usability issues that had been reported, but this also was ok for a 5.3.2 bugfix release.

What made us decide to go for a 5.4.0, was a fantastic project we are doing with other members of the gvSIG Association for INEGI. INEGI is the National Institute of Statistics and Geography of Mexico and recently we won a tender to supply a whole GIS stack for Desktop-Server-Mobile.

At HydroloGIS we are obviously taking care of the mobile part and it is thrilling to know that INEGI invested in Geopaparazzi/gvSIG Mobile and that hundreds of technicians will use it to do their digital field surveys.

That project already brought in the previous release an important change which is the plugin system.

Recently it also added to geopaparazzi two important features:

1) support for unique values map theming in spatialite databases

Geopaparazzi now supports in readonly mode unique value styled spatialite layers.

We are used to a simple style in geopaparazzi:

Now you can do things like these...

Polygons:

Lines:

Points:

Looks quite nifty :-)

But now the problem... how do we make such a styled database?

We created a tool inside the JGrasstools Spatialtoobox, which means it will also get to gvSIG. Right now there are compatibility issues between the spatialite drivers of JGrasstools and gvSIG. We will probably get there for the 2.4.0 release.

In the meanwhile people interested can pick the standalone version of the JGrasstools' Spatialtoolbox, which can be downloaded here.

Just run the script jgt-spatialtoolbox.sh and look for the right module:

Now pay attention. This will import a folder of shapefiles into a newly created (or existing) spatialite database. If the shapefile has a style defined in an SLD file, then that will be used. So use your favorite SLD editor and create a unique values theme, and it will be inserted in the new database, geopaparazzi ready!

2) pdf export

The second interesting feature is the PDF export of form based notes.

For those users that couldn't care less of GIS and complex stuff, but only want to have a nice ordered list of the surveyed data, the PROJECT FORMS TO PDF button is the right choice:

This will export the content of the form based notes to PDF, trying to group the information nicely into tables, adding any image contained in the note.
Just to have an idea, this is how 2 random pages would look like:

That's it, huge thanks to INEGI for pushing this project and making geopaparazzi better!!

First release of the JGrasstools and Geopaparazzi plugins for gvSIG

Ther are finally here. We were able to finalize, test and package the JGrasstools and Geopaparazzi plugins for gvSIG and make a first official release out of them.

It is a first shy 0.1.0, but it contains tons of functionalities ready to be tested by the community.

To download and install the plugins, please have a quick look at this document.

Not much more to say, everything is in the linked quickstart guide... enjoy!