Monday, December 17, 2012

What if... this was a rasterlite database...?

What if this was a rasterlite database loaded as background map in Geopaparazzi?

Would you be happy? I am :)

Wednesday, December 12, 2012

How to load spatialite layers in Geopaparazzi?

I have to apologize for one really important missing piece in mi previous post: How do you get to view you Spatialite layers????

Thanks to all the nice users that made me kindly notice :)

I dedicated a page on the Geopaparazzi wiki to that. You can find it here.

Sunday, December 9, 2012

Geopaparazzi 3.4.0 is out! Please welcome the Spatialite brothers... and NFC support

At the begin of November Sandro Furieri, known for being the father of the Spatialite project (but also known for being our very personal presidential Fury of the Italian GFOSS association), wrote me an email about a release of spatialite for android.

Well, I was in a real busy moment and struggeling with the preparation of the presentation of the upcoming GFOSS day, the Osgeo Italian chapter gathering of the tribes.

Since I was doing a presentation about a history of the mobile trend, particularly related to Android and GFOSS software, I had to do something. So I spent soem night to integrate spatialite in geopaparazzi. Well, during the presentation I gave at the conference:

I only showed a small joke screenshot, where geopaparazzi was dialoging with Fury to do some transformations (see slide 50).

But from that moment I could not stop to think about a raster+vector format for data exchange between different operating systems and devices.

With Spatialite it would be possible to finally overlay not only points and simple lines, but also polygons and large datasets. The spatial index would help, i knew it would.

So we started to implement a new layer in geopaparazzi: the spatialite dataset layer.

An it seemed to work nicely with the Natural Earth datasets.

As a sidenote, since the map viewport in geopaparazzi is bound to use OSM mercator, Spatialite has to reproject the original dataset into that one.

This also produced a short developer tutorial which can be found here.

We tested this new spatialite enabled version on different devices and out in the Finnish flat/wetlands when checking back watershed boundaries extracted through the JGrasstools libraries (well, that will most probably be the next story). It works incredibly well even if it is not really optimized yet, which is why we decided to release it already.

This is how it looked like when we overlayed on a scanned Finnish orienteering map the results of our hydrologic/hydraulic study: old, new and extracted watershed boundaries and channel lines.

Ah yes, did you notice the new icon under the zoom buttons? That is a query button. Through that one you can draw query boxes on the map and the attrubutes of the intersecting features will pop-up:

Well, this opens up a whole new world for geopaparazzi and mobile development in general. And it will even more when the rasterlite part of Spatialite will be supported for Android!

A big thank goes to Sandro for helping out optimising the spatial queries and for fostering the development through realtime support.

Sure, this is the major news here, but there is one more huge addition to this release. We now support reading of RFID tags through NFC and bluetooth devices. For this feature an activity has been exposed for developers, but normal users can access this feature simply through the forms:

You can enable the tag reader widget and you will get a button, which, once pushed, will open this scanner activity:

The nice thing about this is that it is able to scan tags both through NFC and a bluetooth device connected to geopaparazzi.
Once you see your tag id scanned, push ok and the form textfield will be filled with the tag id.

One downside of the addition of Spatialite to the game, is that adding the native libraries and necessary stuff, brought geopaparazzi from less than 2 megabytes to almost 9 megabytes. We figured that people would not care particularly, since the added value is huge.

Well, I think that is all for this release. Enjoy!

Saturday, December 8, 2012

Android: Incomplete location object, missing timestamp or accuracy - ERROR

 Recently, when using my own Mock GPS Provider, I was getting an exception on my Google Nexus phone:

Incomplete location object, missing timestamp or accuracy

this almost drove me mad, since I could not find any solution out in the net.

Also it seemd to happen only on Jelly Bean Android devices.

So I had to do what I usually do not like a lot... dive in the Android source code. Luckily it was an easy one.

The LocationManager does a check on the Location object (isComplete method). There we have the mess. A check is made for

if (mElapsedRealtimeNanos == 0)

but I am not able to set that variable anywere before JB.
So if you create a Location with an SDK prior to JB and then run the code on JB, it will throw you an exception.

There is a nice method that, as javadoc states, helps backwards compatibility:

  1. /**
  2.      * Helper to fill incomplete fields.
  3.      *
  4.      * Used to assist in backwards compatibility with
  5.      * Location objects received from applications.
  6.      *
  7.      * @see #isComplete
  8.      * @hide
  9.      */
  10.     public void makeComplete()

So what you need to do to make sure your mock LocationManager doesn't break, is simply check at runtime if the method exists through reflection:

Method locationJellyBeanFixMethod = Location.class.getMethod("makeComplete");
if (locationJellyBeanFixMethod != null) {

and if it does, invoke it on the "incomplete" location object.

Since it is anyway a Mock class, I have no problem with that solution. I just wonder if that is the way to go. I assume not, since in that case it would be documented somewhere.

But well, it works. Hope that helps someone out there.

Monday, November 19, 2012

Il Repertorio Nazionale dei Dati Territoriali

JGrassTechTips inserisce oggi nella sua homepage il link del Repertorio Nazionale dei Dati Territoriali (RNDT). Attualmente questo strumento sta muovendo i primi passi, ma noi desideriamo che entri a fare parte della “cassetta degli attrezzi” di ogni geomatico e, in prospettiva, di ogni cittadino italiano. Non siamo da soli, ma insieme a:

Saturday, November 10, 2012

Geopaparazzi 3.3.0 is out: free you painting artist heart!

We just released Geopaparazzi 3.3.0 in the market.

There are a couple of enhancements and a feature I think most people will like.
The is version's changelog is the following:
  • added sketch pad for forms(the feature I think most people will like)
  • added textarea for forms
  • move export folder outside of geopap folder
  • fixed OSM notes info
  • sms are now sent through the system app
  • better geocoding
  • better about window 
Let's give some more info about those that need...

1) the sketch pad

If you browse through the menus, you will note that you are now able to add a new type of note:

and also when you add notes without gps:

Well, you now can add finger drawn sketches as notes to your map. For now the sketch pad is quite simple, it allows for some colors and some line widths. But it also has.... UNDO/REDO capabilities!! I really want to thank Almond Mendoza for his tutorials on creating a nice sketch pad.
An example? Assume you are an engineer that needs to create a quick schema of what he sees inside a manhole:

Just draw it, save it and it is on your map!

2) geocoding 

In the geocoding part, the routing gui has been cleaned up to have a better user experience. Remeber that narrow gui, where on small screens you were not even able to read the results? Yeah, that one.

3) sms sending

A user reported that the sms were sent without considering special offers they have with their carriers. So if seems that direct sending of sms goes through an extra channel.

Therefore for all geo-sms message sending we now popup the system's sms application, so the user can decide to add some text and decide the destination.

For the panic button, since when we use it we are happy to pay those few cents if it happens, we left the automatic, one tab sms sending.

Well, that is more or less it.

one sad PS: this was the last version supporting android 2.2. With the next release NFC will get into the library, which will require android 2.3+. I am sorry about that, but we need to keep moving with the innovating parts.

Thursday, November 8, 2012

GIS clipping benchmark: jgrasstools & uDig (on JTS & geotools)

I hate this. I hate it when I see a callout for a benchmark and am not able to not give it a try. And I hate it even more when I have tons of work to do. But yeah, I love the thrill :)

So here it goes. This morning Markus called out for the GIS Clipping benchmark contest revisited. Well, I somehow felt invited as part of the "other FOSS GIS", so I had to run for it. :)

On thing I noticed at once, is that the reported processing times were quite high. I had no proof, but in the last years I worked (also on editing) withmany 1.5 giga shapefiles in uDig. And to be honest I aslo found it was the only software (FOSS and non) that was able to handle the thing properly without making one get mad.

I wanted to have a look how the geotools/jgrasstools/uDig Spatial Toolbox would perform. I am not a real benchmark guy, but well, I can run some modules to have an idea about more or less what it takes. So I grabbed the testdata and gave them a run.

I have also to say that I ceated a bit because we didn't have a clipping tool inside the spatial toolbox up to now, since in uDig this would be done with the Axios tools. So I quickly wrote one, the VectorClipper.

The machine I tested this on is a 2 years old windows 7 64bit machine with 8 gigs of RAM and java 1.6.
It has 4 cores.

The first run was really simple, reading in the features, create an index to quickly get the features that might be involved in the clipping and clip one by one.

Result using one core: ~201 seconds

Interesting. GRASS takes about 5 minutes, but that is ok (or better, great), since it also does topology. QGIS seems to take about 5 minute, which looks a bit like bad performance to me, considered that our worst case scenario takes a bit over 3 minutes.

Anyways, let's focus on making things better.

The VectorClipper can run in multithreading mode. So I enabled it for my 4 cores.

Result using 4 cores: ~152 seconds

Hey, hey. Nice!
But We can do better than that.

Since the creation of the spatial index was taking quite some time, I decided to drop that one, and use instead JTS prepared geometries, which would make the intersects query faster.

So without an index but with fast intersect query I got:

Result using 1 core: ~130 seconds
Result using 4 core: ~110 seconds

So I though: what if I was wrong about the index thing?
So I added the index back...

Result using 4 core: ~102 seconds

Then I noticed that in the multithreading part I had some lists that got copied instead of just passed as indexes. Once that was fixed I got a nice:

Result using 4 core: ~82 seconds

Which is where I was very satified and stopped my tests.

So the above could be called the geotools/JTS/jgrasstools benchmark, since I was running the module from my development environment.

Running this inside uDig has some more overhead, so it made sense to test that, since that is what the user would get.

Here the result I got:

as you can see from the console output, the processing took about 117 seconds, which is anyways a good performance, considered the available benchmarks.

I can't take much credit on this, all the work is done by JTS and geotools, so the credit goes there.

I loaded the used jgrasstools libs on the site, so they could be used in uDig. If you need help in getting started with uDig's spatial toolbox, have a look at this video.

Just for the interest/fun of doing it, I ran this on amazon AWS on an cc2.8xlarge instance with 32 cores. It took about 25 seconds, 2 of which for reading the data, 7 of which for writing the result, and most f the rest of the time for creating the index.

Sunday, October 14, 2012

The new uDig video tutorial series

The following comes from the udig-news blog, since I wrote it I will report it also here.

Today we activated a youtube channel for uDig:

Here we want to collect different video material about uDig.

Some members of the community have agreed on creating each week a new tutorial about any feature not yet described.

Need some examples?

Ever needed to change language of the gui?
Or maybe you are new to GIS and are wondering about how to load common gis data formats?
What about query data in uDig?

More advanced users might need to rasterize a shapefile or vectorize a raster?
Scientific users might want to get started with the Spatial toolbox to do some analyses, maybe start with the extraction of a watershed from a DEM?

Anyone can contribute to the channel and anyone can ask for a tutorial about a certain feature.
On this page information about ongoing tutorial production is kept.
If you need a tutorial about any uDig feature, check if it is not already listed there, and if it is not, join the mailinglist and ask kindly. :)

Tuesday, October 9, 2012

Geopaparazzi 3.2.5 is out - mostly fixes and localization

Well there have been a couple of bugfix release that didn't need any announcement, but for this one I think it is good to let you know.

One nice guy has added a French translation of the project to our crowdin translation platform. Thanks Fred whoever you are :) !

Please let me know if you are working on translations, I can cut a release for you to see how it results in the user interface.
Since we were there, we updated also the Italian translation.

On request of some users some infos were added to the notes and gps logs:

1) the notes now show also the lat/long of the point

2) the gps log properties now show the start and end timestamp

A major bug has been fixed when merging gps logs together. Now merging is safe:

Last but not least, the GPS indicator has been fixed. It has been reported to be a bit unstable on certain phones which resulted in being unable to record logs or notes.

Enoy and please report back any problem. Thanks!

Thursday, September 20, 2012

Geopaparazzi 3.2.0 is out - tons of news

A new version of Geopaparazzi is out in the market: 3.2.0 (or at least it should be soon, lately google play takes some time to get in sync)

The list of news is quite nice:
  • huge enhancement of forms with support for multiple tabs, fragments for tablets, images and map screenshots, date widgets...
  • integration of forms with kmz export
  • style enhancements on kmz export
  • send simple notes and bookmarks info via sms
  • geocoding via google services
  • osm routing via
  • measure tool is back
  • opening of geoSMS by geopaparazzi in offline mode
  • editing mode for geonotes with forms
  • support for rendering themes through style maps for offline maps
  • battery level visible in map mode
  • gps connection status visible in map mode 

Again I will try to reference to the WIKI, so that you get used to check out the documentation we keep updated with much effort.


Forms have been enhanced quite a bit when we started to create survey forms for the earthquake issues and for post flooding surveys. There is now support for enhanced user experience on tablets via multiple tabs and fragments, multiple pictures can be saved and also map screenshots can be added to the forms.

A complete explanation can be found here.


KMZ export got enhanced in two ways:

  • the new form is supported as table view
  • the style from the device is kept


For a quick exchange of basic data it is now possible to send bookmarks and notes (not forms) via sms. The same sms can be opened on an other device and imported. If there are too many data, they are split over multiple sms.


Support has been added for geocoding and routing. Those features need a network connection.

The options are explained here.


Since our migration to mapsforge, the measure tool had been removed due to technical issues. Several users complained about that, so we had to get it back somehow.


Geopaparazzi is now able to open geoSMS in offline mode directly from the sms. That is handy when you are not online (which in my case is almost always during surveys).


Notes that were inserted with a form attached, can now be reopened ad edited. Even if that doesn't show bit screenshots, this is one of the most important features if you are doing complex surveys with forms.


uDig now directly supports the import of geopaparazzi data. A howto can be found here.

That's all folks! Enjoy!

Monday, June 4, 2012

Geopaparazzi 3 is out - explained through wiki-links

I learned through the years that jumping up major versions too often is not good. And I applied what I learned. Well, this time I was forced to, I swear. This had to be geopaparazzi 3. No 2.6.2, no 2.7.0. Just 3.

And  to show that to you all, I also upgraded the whole documentation wiki. And since most of the people out there do not even know that there is such an effort, I will make this release based on its entry.

So, let's walk through the changes that this release contains. The first two changes are definitely the most important ones and depend on the fact that we dropped osmand as mapping engine to go with mapsforge. The decision mostly came from the need to work with vector maps, but resulted in some other big advantage.


It is now possible to load TMS tilesets into geopaparazzi. One can create tilesets from GIS data layers (also with jgrasstools) and load them in geopaparazzi. Tilesets can be in TMS or GOOGLE mode, and both remote or local.

Professionals will be particularly happy with this feature, since one can create maps from the overlay of complex GIS layers, as our small showcase shows.


Mapsforge works with vector maps. And so does Gaopaparazzi now (obviously). Vector maps are great, because they take much less space than tiles and one is able to carry along the whole map of his nation on the sdcard at each zoom level. That is an amazing feature. Never more missing tiles during your surveys!

Mapsforge generates and maintains a set of standard vector maps. Those guys are really amazing!

Recently the osmandguys started generating vector maps rendering after the opencycle maps.Those maps are great, because they also contain isolines.


The OSM tools god some enhancement, mainly related to the ability to update the osm tags once they are extended on the main website. A tutorial is now available.


The bookmarks list has finally been enhanced to look a bit nicer. Note that it now gives the possibility to enable certain bookmarks with a proximity alert, which will ring when one get's near the point of interest.


The forms part has been enhanced with some new tag, which can be usefull for more complex forms.


New import and export features have been added. It is now possible to import and export from GPX, export to KMZ, export to some cloudbased service. An utility to import and export bookmarks has been added, which makes it easy to move bookmarks around projects.


Geopaparazzi exposes also a very simple API to load projects into the web and download online projects definitions as well as download actual projects. Currently we have no server to make this public, so you will have to believe in what we wrote here.


We didn't want to wait any longer to release this version, so we are behind with the translations. We have instructions for potential translators. Please help translating geopaparazzi in your language!


Until this release only BeeGIS was able to seamlessly import data into a real GIS software to allow further analysis.

Since the GRASS community sprint in Prague there is one new amazing software that is able to import geopaparazzi data. There is no info page available yet, but the module is already usable through the GRASS extensions engine. The module works with GRASS 7.

And that's it for this release. Enjoy|!

Wednesday, May 16, 2012

Geopaparazzi Web Map Tiles enhancements

Well, why should geopaparazzi hang behind uDig? :)

Ok, there are a number of reasons, but in this case, it doesn't have to be like that.

A small piece of history
Some of you might know that I am now part if the Italian TANTO geospatial (and not only) information/education blog.
In April I decided to go and visit the one guy that actually founded that blog in the most distant place in Italy from where I live: Andrea Borruso from Palermo.

We spent a couple of days daydreaming under the heavy effects of muffolette, cassatelle, cannoli and the grand finale of the sfince di San Giuseppe.

The recurring question was: how do we bring a custom raster into geopaparazzi? The potential is there, but we need to be able to load custom map!

Well, at the end we figured that the best thing would be to support local (as well as remote) TMS.
Andrea is black belt of TMS and was able to provide me with some tilesets.

So I started to implement the possibility to load the tiles on the phone and describe them with a small file, so that they would appear as tile sources.

We will throw out this new version as soon as the documentation is ready and some good tutorial on how to quickly handle data and stuff. as usual, if you wanna help, please contact us, we badly need support.

But in the meanwhile I really want to share some examples of the results we were able to achieve.

1) tiles of the ortofoto of sicily, zoomed on Palermo

2) an archeological area airbourne survey, done with drones. Imagery provided by the Consorzio TICONZERO. Interested how they are doing this? Have a look here.

3) technical map, overlayed with the results of a risk analysis (perfect for outdoor geologists and engineers)

Do you remember seeing this image somewhere already? Maybe loaded in uDig? :)

4) Last but not least, a scanned historical map (shiver down my spine)

Ok, need to stop here.
Basically soon it will be possible to load almost any kind of raster map into geopaparazzi, which is an amazing enhancement. Now all that's left, is to wait for the next release :)

This post is born as a parallel post (as in two different ways to tell the same story by the people that lived it) of this post.

Saturday, May 12, 2012

uDig web map tiles enhancements

Many of you for sure know that uDig has the ability to show Web Map Tiles.

From the import dialog you select Web Map Tiles:

push the next button and find a long list of available web map tile servers (which we should probably check).

There is also the possibility to select a custom server. This is very handy, but had 2 problems:
- it could not load local folders of files
- it understood only the google tile schema numbering, which means no TMS

Well, this is the first enhancement. Those two problems were solved. Mind that you will have to check the right option:

The second enhancement will make uDig Spatial Toolbox users happy. With the next uDig release we will also release the module that creates TMS folders from a set of layers. I do not want to go into detail shere, but a new module will allow to export a set of styled layers as tiles folder. Those tiles come with the complete description of the zoom levels and so on.

In fact since that file is the only thing needed to load a folder of tiles, it made sense to add a small item to the list, the JGrasstools TMS folder:

And there we have it. Push the button, push, push the button and:

This will be extrememly helpful for those like us that use uDig with BeeGIS field extensions out in the field on netbooks. The base dataset gets extremely lightweight to load and rasters can be prepared to be fast and easy.

Sunday, April 15, 2012

uDig feature movies

Have you ever had to check something like 2000 features? A quick check, nothing difficult. But 2000 or more features.
You start panning around, after a while you do not know what you passed already, your eyes get glossy, you loose control, bad things happen, and some of you ask themself why they can't just be writing code instead.

The only solution I could think of, was some kind of movie that would crawl through the features and zoom to each one, something you could stop and resume.

Well, the next uDig will finally have that tool:

I guess this doesn't need much explanation. If you have doubts, just ask.

One nice thing for the future might be to also add amap image export facility, which would make it possible to print an image of each feature?

Sunday, April 8, 2012

Extreme Easter Geopaparazz-ing Recipe


1) Choose the right day and the right place

2) Choose the right tool (Geopaparazzi of course!!)

3) Survey everything, no matter how hard

4) Let no obstacle stop you, let your mind float!

5) Tag on OSM even the most difficult locations

6) Fight for survey-val

7) Get home and do a regular (and well earned) Fruehshoppen with white sausages, brezel and hefe-weizen beer!

Happy Easter!!


Small update: the geopaparazzi kmz export. Amazing view if it does not snow that much :)

Wednesday, March 28, 2012

The geopap-browser is on its way

Well, it is not often that I throw myself in the web development world, but geopaparazzi needed a simple way to upload/download projects for happy collaboration.

So aided by the simplicity of the play framework and my treasurebox TANTO team, I started the journey to create the geopap-browser.

The aim was to create a web prototype to showcase the functionalities of the webproject possibilities of geoapaprazzi. So in the next release you will be able to export a project to a configured server:

The browser application itself is quite simple.
We have a main page listing the uploaded projects (with simple filter possibility):

And then there is a project view that gives a nice summary of the data contained in the project. It is also possible to see the profiles of the traces and... download the points and lines as shapefiles.

So... we are working on it, stay tuned!

Sunday, February 12, 2012

Geopaparazzi 2.6.0 is out - OSM, Mixare and GeoSMS

[for an Italian version of this article, click here]

We just released Geopaparazzi 2.6.0 to the market.

I was really looking forward to this release because it contains a ton of new stuff some were expecting. Let's try to see some of it.

1) Openstreetmap support

This is likely the most important change for this version. We now have the possibility to create OSM points of interest and synchronize them with the OSM accoutn online.

First one has to activate OSM support in the preferences:

If OSM support is activated, once geopaparazzi restarts, the user is prompted to download the OSM tags. This is a library of symbols and definitions of OSM tags.

The map view will then prefsent a small OSM-logo handle at the right side of the map. If that one is pulled, it shows the OSM categories available.

Entering one of the category, the various tags are presented:

And when selected, the user gets the form with the definition of the selected tag. Once filled out and pressed ok at the bottom of the form, the point is saved and kept ready for upload to OSM.

From the categories view one can upload the OSM points to his account. The application just askes for a description for the changeset it is going to upload.

If the user and password were inserted properly in the preferences, geopaparazzi will connect to a WPS service written and made available by Luca Delucchi. The WPS will take care to upload the points to the defined OSM account. A big thanks goes to Luca, who helped me to understand the necessary OSM internals. He alos wrote the actual WPS service, made it available for all geopaparazzi users and also prepared the OSM tags library.

2) OGC GeoSMS support

This version has support for the recently released OGC GeoSMS specification. This works in two ways: for sending geosms, but also for incoming geosms.

The sms preferences allow to "enable sms catcher", which will handle incoming geosms, while the panic number is instead the number that will be used to send geosms to.

Let's start with outgoing geosms'. Geopaparazzi has the bottom "panic" bar. From there a user can send a panic message, which will send to the configured number a geosms containing the positions following OGC specs and a message asking for help. There is a second button that can be used to send a customized geosms. It will ask the user for a cutom text to add to the location definition.

What happens instead when a GeoSMS is recieved? If the catcher is enabled in the settings, geopaparazzi will parse the incoming message and recognize the GeoSMS format. It will notify the user that such a message has arrived:

The notification makes it then possible to open the GeoSMS with one of the available applications on the phone, that can handle the position information.

So once selected the notification:

And opening it for example with a well know application we can see that I was preparing some screenshots for my talk at the Italian GRASS and GFOSS user meeting in Trieste:

3) augmented reality with mixare

In this version we added the ability to view all points of interest contained in the current geopaparazzi view in mixare. I already described the feature in this post.
The entry point is the menu in the map view:

4) create a new project

While this has been a bit odd in the past, it now follows a more natural workflow. You are asked for the name of the new project and geopaparazzi creates the project and loads it.

5) bookmarks

One this I was missing, was a very easy way to add bookmarks to geopaparazzi. So we added a check on the presence of a bookmarks.csv file in the project folder (by default geopaprazzi). If the file is there and it contains points in the form:

Hotel Trieste, 45.642043,13.780791
Grassday Trieste,45.65844,13.79320

those points will be loaded in the application and appear in the bookmarks list.

This feature lead me to have a ton of bookmarks and defined the need for a way to filter them. So the bookmarks list now has a textfield that filters the bookmarks by name:

6) center on gps

After having risked to make car accidents just to pan while driving in order to have the actual position visible, we added a function to have the gps alwasy inside the current window, through automatic panning. It can be switched on in the gps preferences:

7) enhanced charting with zoom to markers

Just to remind that one, I wrote already here about it.

8) osmdroid doesn't kill tiles any more

Some developers from the osmdroid (the engine we use to display the maps) community finally solved the bug that in certain low memory conditions were killing all the OSM tiles, leaving the user with no more maps. This was one of the worst bugs ever, since it had happend to several user that they found themself out in the field with no network and no more maps. This has now finally been solved!!!

9) possibility to use network based position

This was asked by several users so we thought we could add it. In the setting it is now possible to ask geopaparazzi to use network based position definition instead of GPS based. Even if I do not like it that much, I agree that it is handy sometimes.