The Geopaparazzi family upgrades... new server and apps!

In July we released the Geopaparazzi Survey Server and it has been such a great success for ourselves, that we started to miss several features in it and had to develop them.

While doing so we noticed that the Eclipse RAP platform didn't seem to be particularly flexible and supported so we tried a switch to Vaadin... and never looked back since. The whole app has been ported and important functionalities have been added, being the 2 main ones:

The form builder

It is ages now that users ask for a visual form builder. The gvSIG Association has built one for gvSIG that works nicely. But then it still needs to be loaded to the device. Since we started to have a server, we felt this should all be there and allow for easy connection and download.

I think this first version of the form builder is already looking great and very usable:

A more detailed documentation about it can be found in the manual.


The mobile companion app is now able to get the list of forms and install them into the geopaparazzi folder:

The project data download

In order to be able to get project data from the server, it is now possible to load datasets on the server:

These dataset are then available to the mobile app:

The project data are downloaded into a dedicated folder and the geopaparazzi user needs to load them manually. So this is not so sophisticated as it is done in the profiles server.

Export database

While it is best to connect to the online database through the port 9092, which is exposed by the server, in some environments it might not be possible (ex. Heroku allows only one port). In this case it is possible to export the database by downloading it.

Send a log

The log sending has been enhanced to allow the user to add some comment that can help the developers to understand the issue:

Geopaparazzi 5.6.2 released

A new version of geopaparazzi has been released with some bugfixes, enhancements and language updates. This version now allows the user to have multiple forms json files. geopaparazzi will load them all into the add notes view.

Do you own a reMarkable? This could help you...

reMarkable HyUtilities is a simple application that helps the reMarkable user  mainly with 3 things that I have been missing. Maybe some of them are available to windows or apple users, but they suck for linux users.

1. upload of templates. At every update the templates are reset to factory, so my personalized ones are gone. I needed a quick tool to restore them. ssh and scp and even FileZilla are good, but I needed something that didn't require my brain.
2. upload of graphics. Same as for templates. I customized all possible graphics (splash, reboot, etc), so I need to restore them.
3. backup. This is important. At the time being there is no way to the raw data. The story that pushed me to this development was a user that reported his reMarkable stolen and was afraid that the thief would delete everything from it before being able to somehow disconnect the cloud account and save the data. Since we are making big use of the device, this is a HUGE issue.

First startup

Download the application from the release page and unzip it on your disk.

To run the application launch the start.bat or start.sh file. Note that this needs java to be installed on your system.

Once launched, you should see something like:



the first time you run it you will need to insert:

• the host, i.e. the ip address of your reMarkable device
• the user: root
• the password to your device
• the folder on your disk to use as a base for your data. Notice that it is mandatory to have 3 folders already in it:
• templates: put there all your template images, can be empty
• graphics: put there all your custom graphics, can be empty
• backup: can be empty

Templates

If working mode templates is selected, then the local templates should be shown.

If you put the refresh button, the remote ones will be shown.



Push the Upload local button and the app will upload your templates to the device.

Graphics

The working mode graphics works exactly the same way as the templates mode.

Backup

The backup mode is a download only mode at the moment. It enables the backup button:



If that one is pushed, the application will:

• compress all your data on the device
• download the tar.gz archive to the local folder
• remove the archive from the device again

Note that the backup action can be very long going if you have many data on the device. Let the app the time to finish.

Restart device

There is also a restart device button. Once uploads have been done, it is better to restart your device. It can be done manually or through the button.

Warning

On the project page there are big warnings, please read them!

Enjoy!

Geopaparazzi 5.6.1 is out

Usually when I have time to make bugfixes I also make a release after that. Last time I did fixes I then got caught in the work crossfire and forgot to. I didn't notice, since on my devices I have the most advanced testing devel version.

Yesterday Silvia asked me why the fix she asked for wasn't there yet! :-)
So I noticed. Well, I added a couple of more fixes to the closed list and here we go.

This is mostly a bugfix release, with a few minor usability enhancements that recently a very advanced geopaparazzi user started to report. I am not able to catch up with all his reports, but some of them are in.

Here we go:

Enhancements

• better feedback about form name and positioning mode (GPS or Map Center) in actionbar
• all exports now follow all the same pattern. They are exported in the geopaparazzi/export folder and their name is made if the project name + type + timestamp of export
• project PDF export now has the possibility to export only a selected portion of notes
• activate button for profiles is now on the main cardview
• better proportion of forms in portrait mode
• tile sources icon is now always visible in actionbar
• dashboard enhancements: visualize number of notes and logs, open notes list on long tap
• save button in forms now a floating action button to remind user to save

Fixes

• the force locale didn't have English to allow a user to go back to default
• fixes on profiles not being visible in landscape mode
• fix for last background map used not reloaded on app restart
• fix for crash on pushing back button in form notes
• fix for crash on issue empty sql query in advanced view
• fix for issue with saving forms with empty fields
• avoid data loss in form notes when user exists with back button. Now the user is warned.

 And of course, language updates.

 Enjoy!

Geopaparazzi Survey Server

Would you believe it? Only a month ago I was showing the Geopaparazzi Profiles Server developed by the guys at GeoAnalytic and now I am here again to write about the Geopaparazzi Survey Server (GSS).

What does Survey Server even mean? :-)

Well, while the Profile concept is a sophisticated way to handle survey data, background data, forms, spatialite databases and their way of synchronization, the GSS si something much smaller. But in our opinion it reflects the workflow of many, many surveyors and groups of surveyors.

The Geopaparazzi Survey Server (GSS) is a web application that allows geopaparazzi users to synchronize their project data with a central server.

Its companion is an Android app named Geopaparazzi Survey Server Sync (GSSS) available on google play. The app can connect to geopaparazzi projects and synchronize the data contained using the unique device ID to upload the data to the server.

Any device that connects to the server with its ID, will be accepted and if not available, the new id is automatically inserted in the central db.

So this is not about spatialite datasets, but only about geopaparazzi project files, that can contain notes, complex form notes, GPS logs and images.

The server application

The server application is packaged as a docker images and can be installed in the blink of an eye ( well, if your connection is fast). Once you install it, you get a nice and simple web application with a login

a dashboard, a mapview with the possibility to load/unload the data of your surveyors, zoom to it and check information of your notes, gps logs and images:

The surveyors' device that connects to the server is inserted in the surveyors list, if it is not registered already. There a human readable name can be assigned and some basic contact:

The mobile app

The mobile app has been done for android. FYI, we are also working on a desktop version app. There are several reasons for which we decided to go with an external app instead of adding this to geopaparazzi itself. The most important thing is that several cloud synchronization applications are getting born around geopaparazzi these days. We will probably have to let these ideas mature and then at some point it will be possible to converge on the best methodology.

But right now I find it more respectful to have an external app that uses its own way to collect the data from the geopaparazzi projects and send them to the server instance.

So let's have a short look at the Geopaparazzi Survey Server Sync, GSSS :-)

Well, it is a simple simple app with the possibility to load geopaparazzi projects. In it you can see what data are available for upload, i.e. dirty data:

Since the app uses the device id as surveyor id, some simple configurations need to be done:

Once the project file is loaded, the device id is ensured and the server url is inserted... well, just push the upload button! On successful upload the list of notes will be empty and your survey can go on using geopaparazzi.

Installation and training

A complete installation and quickstart guide is available here. Check it out to see all features available.

If you want to get a small training first hand from the developers, we will be giving a workshop about geopaparazzi and GSS at the following locations in the near future:

• Geopaparazzi Workshop at Foss4G EU in Portugal. Want to join us?
• Geopaparazzi Workshop at Foss4G in Dar es Salaam. Want to join us?
• Geopaparazzi Workshop at Foss4G Asia. Well, to be honest we haven't submitted the workshop yet, we'll see if it get's accepted :-)

We would also be very happy to be involved with this stack in projects in developing countries, where data collection and centralization is necessary.

If you are interested, please reach out to us.

Cloud Profiles - The Open Source Reference Server Implementation

Today it is with great joy that I just copy/paste an email that arrived in the geopaparazzi mailinglist. I already announced it, but this is the proper announcement by the guys from GeoAnalytic Inc. So I just leave this to you all, as written by Brent Fraser:


Hi All,Geopaparazzi 5.6.0 is now available at Google's Play Store (and GitHub as an APK).  One of my favorite new features is Cloud Profiles. Its purpose is to ease the data management tasks by automatically downloading project files, Spatialite databases and other files when connected to a 
"compatible" web server.    
 
And there's more!  When you activate a Profile, Geopaparazzi will set the basemap and attach the Spatialite databases ready for viewing in the Map View.    

So what about the "compatible" web server?  Since the Cloud Profiles list is a JSON text file, you could hand coded it and use a plain old web server like Apache or IIS.  Or do some server-side scripting if you want to get fancy. We've got an Open Source project started to be a "reference server" for this.  hopefully it will grow to provide more Geopaparazzi services, but right now it just servers up Cloud Profiles. Have a look at the blog post of more details:
 
https://geoanalytic.github.io/a-reference-server-for-geopaparazzi-cloud-profiles/    

There is a public version set up so you can experiment with Cloud Profiles (it delivers two demo data-sets: Vienna and Bolzano).  To try it out:   

1. Open up Geopaparazzi 5.6.0 on your device  
2. Go into Settings (using the gear icon on the action bar)  
3. Go into Cloud server settings  
4. Edit the "Cloud Profiles URL" to be https://geo.trailstewards.com/myprofiles/  (don't forget to have the trailing slash in the above URL). You don't need any values for the Username, Password, or Cloud Project server.  
5. Go back to the main dashboard screen and press the big Import icon (the one on the bottom left)  
6. Press the Cloud Profiles at the bottom of the Import options.  
7. Press the icon beside the Profile you would like to download.  Some of the files are large-ish so it is best to be connected via WiFi.    
8. After the Profile has downloaded, it's time to "activate" it:  
• On the main dashboard screen, press the menu icon on the far left of the action bar and select "Profiles"  
• You should see the Profile you downloaded listed there.  
• Press the Profile's gear icon to change the properties  
• Slide the Activate profile switch to the right to activate it  
• Go back to the main dashboard screen and press the Map View icon.    

You should see a basemap with and overlay as defined by your selected profile!  

Best regards,  
Brent Fraser  
GeoAnalytic Inc.


This is a great contribution to the geopaparazzi community and makes me feel that the geopaparazzi project is on the right track to be a good free and open source project.

Enjoy!






 

Geopaparazzi 5.6.0 is out - work with cloud profiles to handle surveys and data!

This release brings a ton of fixes and compatibility enhancements for newer Android versions.

And then it brings cloud profiles. And I think this one is huge! :-)

Cloud Profiles

In the last months a new company entered the geopaparazzi community and they have been doing nice contributions. The guys from Geoanalytic were working with us on the implementation of a more structured version of the cloud projects synchronization: Cloud Profiles!

We have been working on this already at the Bonn code sprint together with Cesar from the company Scolab. And now we are finally at a first release that supports this concept.

Cloud Profiles are a great way of easing Geopaparazzi’s data handling tasks. When a web server is configured to serve Cloud Profiles, Geopaparazzi can automatically download Projects, Basemaps, Spatialite Overlays, forms for Notes, and other files. When a user activates a downloaded Profile, Basemaps are made available, Overlays are attached to the Map View and layers are set to display.

Read more about it here.

The Geoanalytic guys have also been so nice to write a reference geopaparazzi profile server that can be used as a starting point. You can read more about it and download it from here.

The real power behind cloud profiles is that you can use your own server for this, but you could also make it much simpler by using Generic Cloud Server. You can set up your own Cloud Profile server by putting your files on a generic cloud file server like Dropbox or Google Drive, and editing a Cloud Profiles list like the one above and putting it on the cloud file server as well.

The very important fix

The most important fix (in my opinion) is the one resolving the GPS Location Limitations on Android Oreo. Basically it wasn't possible to log gps tracks with the screen off.

Technically it has been necessary to create a notification icon for geopaparazzi, so you will now have that visible when geopaparazzi is active.

The positive side is that:

1. you will always know when geopaparazzi is active
2. you can always see information about your position directly in the notification area

Other features and fixes

Values in settings. The settings screen now shows the actual values:

Buttons size. Small buttons are hard to see and press while on the trail. While it can be difficult to show a lot of information on a small screen, where there is room, you can now change the button and text size. This applies to the notes view for now:

Dynamic hints. Dashboard button hints are dynamic where possible. You can now see from there how many notes and logs you have in store. Just long-tap on the buttons:

Notes settings. The notes settings view is now accessible from the notes list (it was hidden in the gps data list menu). Access it from the palette icon:

PDF export. The PDF export now allows to export a subset of notes instead of everything contained in the project. The user can select the notes he/she likes to export and those will be converted in the pdf version.

Linked resources. It is now possible to view not only images stored in a Spatialite database when they are related to (geospatial) features but also for example PDF. 

Note that while a user can take pictures in the field and link them to a feature, in the case of PDF, the resource has to be linked before, i.e. it can be just viewed only from geopaparazzi.

Remove all. In the basemaps view it is now possible to remove all maps in one tap. This is really helpfull for those that are used to load maps through the load-folder option and then need a lot of time to remove them to have things more readable.

Mapurl service. One sad note is that the Tanto Mapurl service, which was used to download automagically configured mapurls based on WMS services, is no longer maintained and has therefore been removed also from geopaparazzi.

Fixes for:

• images taken within Geopaparazzi are not geotagged.
• unable to open form image thumbnails. 
• many other fixes have been done, the complete list is here.