Geopaparazzi 3 is out - explained through wiki-links

I learned through the years that jumping up major versions too often is not good. And I applied what I learned. Well, this time I was forced to, I swear. This had to be geopaparazzi 3. No 2.6.2, no 2.7.0. Just 3.

And  to show that to you all, I also upgraded the whole documentation wiki. And since most of the people out there do not even know that there is such an effort, I will make this release based on its entry.

So, let's walk through the changes that this release contains. The first two changes are definitely the most important ones and depend on the fact that we dropped osmand as mapping engine to go with mapsforge. The decision mostly came from the need to work with vector maps, but resulted in some other big advantage.

CUSTOM MAP TILES

It is now possible to load TMS tilesets into geopaparazzi. One can create tilesets from GIS data layers (also with jgrasstools) and load them in geopaparazzi. Tilesets can be in TMS or GOOGLE mode, and both remote or local.

Professionals will be particularly happy with this feature, since one can create maps from the overlay of complex GIS layers, as our small showcase shows.

VECTOR MAPS

Mapsforge works with vector maps. And so does Gaopaparazzi now (obviously). Vector maps are great, because they take much less space than tiles and one is able to carry along the whole map of his nation on the sdcard at each zoom level. That is an amazing feature. Never more missing tiles during your surveys!

Mapsforge generates and maintains a set of standard vector maps. Those guys are really amazing!

Recently the osmandguys started generating vector maps rendering after the opencycle maps.Those maps are great, because they also contain isolines.

OSM TOOLS

The OSM tools god some enhancement, mainly related to the ability to update the osm tags once they are extended on the main website. A tutorial is now available.

BOOKMARKS AND PROXIMITY ALERT

The bookmarks list has finally been enhanced to look a bit nicer. Note that it now gives the possibility to enable certain bookmarks with a proximity alert, which will ring when one get's near the point of interest.

FORMS AND TAGS

The forms part has been enhanced with some new tag, which can be usefull for more complex forms.

IMPORT AND EXPORT

New import and export features have been added. It is now possible to import and export from GPX, export to KMZ, export to some cloudbased service. An utility to import and export bookmarks has been added, which makes it easy to move bookmarks around projects.

CLOUD PROJECTS

Geopaparazzi exposes also a very simple API to load projects into the web and download online projects definitions as well as download actual projects. Currently we have no server to make this public, so you will have to believe in what we wrote here.

TRANSLATIONS

We didn't want to wait any longer to release this version, so we are behind with the translations. We have instructions for potential translators. Please help translating geopaparazzi in your language!

CONNECTIONS TO OTHER SOFTWARE

Until this release only BeeGIS was able to seamlessly import data into a real GIS software to allow further analysis.

Since the GRASS community sprint in Prague there is one new amazing software that is able to import geopaparazzi data. There is no info page available yet, but the module is already usable through the GRASS extensions engine. The module works with GRASS 7.


And that's it for this release. Enjoy|!

OMS3 Grass modules Wrapper at Prague GRASS-Hackfest

Last week I took part at the GRASS Community Sprint. As every sprint I took part at, it has been an amazing experience and it has been good to catch up with "old" grass developers I knew and to get introduced to some really great "new" developers.

At the sprint I wanted to wrap all the grass modules in an OMS3/jgrasstools wrapper. Well, I had only two days to code on it, so I am not exactly where I wanted to, but lot's of stuff is already working and I can show you some stuff already loaded in the OmsBox:

Running r.watershed from within uDig:



And the result:



If you want to know what OMS3, jgrasstools and the omsbox are, you are invited to have a look at the presentation I gave at the Geoinformatics FCE CTU 2011.

While in the screenshots you can see the modules inside the OmsBox (just so it looks nice), the result is a very small java package that depends on no big library or toolbox, so that the whole java world can take advantage of this. Amen!


I would also like to thank personally the sponsors that helped out:
GFOSS.it Associazione Italiana per l'Informazione Geografica Libera
Stefan Sylla, sylla-consult, Frankfurt, Germany
R3 GIS, Merano, Italy
FOSSGIS e.V., D-A-CH

First small JGrass tools migrate to uDig

You might be thinking I am crazy, but I was waiting for this post since at least 2 years now, i.e. from the day I decided to merge JGrass into the uDig community. I never did regret the move, but I had hard times to get at least the general purpose GIS stuff of JGrass into uDig. But that last hurdle seems to be gone now.

So let me do a quick summary of what got already into the udig core (i.e. next udig releases will show them) and what will come next:

1) one of the most important one for me was having support for the GRASS format. And here it is finally.

You can now import grass locations into uDig directly and visualize/use the rasters contained.



Also you can define colorrules and (!) categories.



2) The csv import tool

Not much to say, you have a comma separated values file that contains point coordinates, go to the import wizard, check csv import



configure how you want the resulting feature layer and you are done



3) Coverage export

It is now possible to export coverage layers to geotiff or esrii ascii maps. Starting from the export wizard



it is also possible to perform a reprojection during the export.



4) View line orientations

This might be of few help to some, but might be life/time saving to others. I much to often need to know the orientation of the lines that were digitalized. By selecting a line layer and opening the operations menu



this is now shown to you through arrows




5) The coverage info tool

The info tool now also understands coverages. Therefore if you pic with the info tool over a map, now also the coverage layers are taken into account




6) The profile tool

A profile tool for raster maps is now available. It is available under the info tools (the one with the red icon, indicating that there is a problem with the icon, has to be solved still)



Once activated, the selected layer's coverage is read and we can use the tool to draw lines (same as with the distance tool). The profile view opens up and the resulting profile is shown as it proceeds. Red markers are shown in the click points.



There is also a second mode to use the profile tool, which is through an existing feature.
To use that you have to select the feature layer and the raster layer you want to use for the elevations. Then you select the line feature you want to use for the profile.
From the operations menu choose: Feature on Coverage profile
You will get something like the following, having a marker in every node of the line




I feel good, I knew that I would!
Next are all the advanced style features and the netcdf format support.

JGrass loves Worldwind

Lately I had the need to look a bit more deep into Nasa World Wind.

With the help of the amazing examples of Patrick Murris I have been able to connect a map from a GRASS workspace to be automatically visualised and tiled for WW.

The funky thing is that while tiled, a cache is created, so it is slow only the first time.



and the same with transparency:




I am still looking for a way to substitute the elevation model of WW with a custom one on the fly.

For now this is all about prototyping and documenting, but it looks great.
Would be good to have it embeded in JGrass/uDig as its default 3D viewer?
Who knows... :)

The day r.proj stood still...

I use GRASS, from time to time I develop GRASS, all in all I love GRASS.
But there are things of GRASS that I hate... those small things that lead me to start with JGrass long time ago.

An example? r.proj!

Does it really have to be a pain to reproject a couple of maps into a new location?

Shouldn't it be something like:

1) select the maps you want to reproject, rightclick on the selection and use a nice command to reproject



2) define a name for the new location and a new mapset, choose the new coordinate reference system




3) sit back and watch that thing work and enjoy the new location appearing in the file browser



4) enjoy the new location with its reprojected maps




Yes, I think that's the way it should be... and it also will be in JGrass aligned with uDig 1.2.

The day GRASS maps learned to fly through my projections

On of the things I was dreaming from bringing the GRASS raster format to geotools, was the fact to be able to reproject them on the fly. Be able to reproject them without all this issues around r.proj that I used so many time. Just select a box with the coordinate reference system and voila', distorce my map around the world... and now it is there, growing to be in the JGrass development version... and you have to see this...


1) the adige basin in EPSG:32632, UTM zone 32N



2) EPSG:3003, Gauss boaga (no evident change, just coordinates change)



3) the almighty EPSG:4326... the World Geodetic System 1984...



Tonight I will sleep quite :)

And soon the export to reprojected geotiff will come...


As good Simone makes me notice, I should thank those guys that helped me with that odd thing that is coverage. Some knowledge came from both (in alphabetic order :)) from:

• Geomatys
• Geosolutions

How to get GRASS related preferences into your code

What to do when you need the mapset that is currently set for doing calculations or even the path to the GRASS GISBASE?

Go and buy some at your preferred store:

           ScopedPreferenceStore preferences = (ScopedPreferenceStore) ConsoleEditorPlugin
                   .getDefault().getPreferenceStore();

           String mapsetString = preferences.getString(PreferencesInitializer.CONSOLE_ARGV_MAPSET);

           String gisbaseString = preferences
                   .getString(PreferencesInitializer.CONSOLE_ARGV_GISBASE);

How to do calculations on maps: asking GRASS for help - part II

Perhaps one of the most used tools in GRASS for those that work with rasters, is the r.mapcalc. This tool gives you a powerful way to do algebra on raster maps.

Let's have a look at it wearing the JGrass gui.

First search the tool in the raster tools:


Selecting the tool you will get the calculator window:


As you can see, there is an area into which to put the equation you want to do on the maps. The equation supports a minimal syntax highlighting. Important is the fact that existing maps are written in bold and italics, so if they are not, probably you are making a typo error and you can check the name of the map before executing it.

Also there is a resulting map text field, which represents the name of the created map.

Another nice thing is the combobox that keeps track of the launched commands, also throught the JGrass session. The last 20 entries are saved over time:



In the first dialog the equation is the following:
if(bacino_chiese_pit <>2500, bacino_chiese_pit , null())
which translated means:
In the pixels of the map, in which the value is <> 2500 put the original value of the map, else put a novalue.

This is a very simple example, this tools can be really exploited with nested commands and strange stuff.

And the result is as expected:



Enjoy!

How to create a contour map: asking GRASS for help - part I

As you probably know, JGrass can execute GRASS command, which makes it a really powerfull funbox.

So what if I want to extract a shapefile of contour lines from a DTM?

def outputName = "contours"
def outFolder = "/Users/moovida/TMP/"
def inputDem = "bacino_chiese_pit"
grass r.contour --o input=$inputDem output=$outputName step=250
grass v.out.ogr input=$outputName type=line dsn=$outFolder olayer=$outputName layer=1 format=ESRI_Shapefile
grass g.remove vect=$outputName

And we go from:


through:


to the needed layer:

New JGrass tools to query and correct raster maps

d.what.rast

With JGrass under the info tools in the gui you can find the "Query tool" or in GRASS language d.what.rast.

By selecting one or more maps you are able to query a particular coordinate to get the raster value for that, together with some useful position info.



Now the same can be achieved from the console with more than one coordinate in order to get a list of coordinates and values to be used in the script:

jgrass {
$values = d.what.rast
--igrass-in bacino_brenta_pit
--coords "692055.62,5103501.69,694218.62,5105772.84"
 --printrowcols "true"
--otable-out "UITABLE#x#y#value#row#col"
}

will gain:



r.correct

In the same info icon menu you can find the Correct tool, or better known as r.correct.
Even if some of you might find it useless, some of us will need it for everyday's work. It gives the possibility to correct manually pixel values of a raster map.

For example here I have a map with a river (which pixels contain the channelnumber) network over an elevation model map. Note that the river map has novalues were there is no river.

If I click with the r.correct tool on a point on the river network, a dialog with a table containing the raster values opens and I am able to change those values and save the map (overwriting the old or creating a new one).

Note also that you have buttons all around the table that let you navigate the raster, if there are other values around you want to change.

The same of above by image:

1) click the point on the network in the area you want to change:


2) change the value of the pixel you want to change. In this case let's just split the network arm from the main network. Split means set a value to novalue, which by default is -9999.0:


3) save and hit refresh to redraw the map. The river is split:

How to read a grass raster and script on it

I just added a r.read command to the console commands list. Currently it can read grass raster.

The console scripting code:

jgrass {
$map = r.read --igrass-in bacino_brenta_pit --ograss-out *
}

rasterData = map.getJGrassRasterData();

rows = rasterData.getRows();
cols = rasterData.getCols();

println rows + " " + cols;

rows--;
cols--;
for (i in 0..rows) {
for (j in 0..cols){
 print rasterData.getValueAt(i, j) + " ";
}
println "";
}

Please note that groovy really doesn't need type declaration.


Well on the raster data you could do some nice calculus:

How to create a command OR Presenting the actions-commands-console-openmi-uibuilder taskforce

You want to implement a new model/algorithm/command/action/do-something (MACAD) in JGrass?

Here more or less the full list of things to do. More or less is due to the fact that I assume you already know the JGrass or at least the Udig or at very least the Eclipse RCP development environment. Also some sort of OpenMI knowledge should be here. All in all I'm not sure how many will find this useful. Hmmm... I'm sure at least one of the chapters could be.
Why I'm doing this? Because next week I have to present a whole bunch of the JGrass stuff at the EGU and I need some docu. Enjoy :)


UPDATE: please note that now the steps 1 and 2 are done the clean rcp way throught extension points. Read here for more info.

1) Create your MACAD

My example will treat an OpenMI based model, i.e. h.pitfiller.

The model extends ModelsBackbone, which implements OpenMI's interface ILinkableComponent.

Create such a MACAD and implement the needed methods.

2) Tell the console engine that the new model is there

In the list of available MACADs add your new model, so that the engine knows that your MACAD exists.
The line looks like the selected in the image below:



which basically tells us what the model's name and class are.
Also information is given about the arms the MACAD has. In this case 2 arm, which define the input elevation map and the output depitted map.

Note that at this point the model h.pitfiller can already be used in the JGrass console as
h.pitfiller --igrass-elevation elevation --ograss-pit pit


3) create a gui xml file definition for the MACAD (the UIBuilder of JGrass)

Create a file with the same name of the MACAD and xml extention and fill it with the following:





The first line will create a label of text descr, a textfield and a button that opens a map selection window.
The second creates a label of text descr and a textfield for the output map name.

When ok is pressed, the widget creates a command string build by the name of the command tag and the repr of the fileds tags, substituting the # with the user input:
h.pitfiller --igrass-elevation userinput --ograss-pit userinput


4) create an rcp action to launch the command

Once the gui definition is there, we need something to trigger the action.

Eclipse RCP's actionset comes to help us. I create a plugin inside which I keep all the gui for console commands and create an action. In the below image you can see the whole context with adifferent menus and action groups.


Please note the textfield class, inside which I put: eu.hydrologis.jgrass.ui.actions.h_pitfiller

The class MUST have the same name as the MACAD you want to create!
Let's create the class by clicking over the hyperlinked text class which will create the action class with all the needed implementation of interfaces and methods.


And now the same class with the needed changes:



Apply the same changes and that is all. If you didn't do so, put the xml gui definition file inside the same package as the action class. Take a look at the screenshot above to see how it should look inside of the actions package.

5) try it out!

Launch JGrass and search in the menu bar your new item. In my case there are a lot, as I already have a whole bunch of organised MACADs in my configuration. However, here I go with pitfiller:



and the autogenerated gui looks like:


6) what happens when I click ok?

The uibuilder creates the commandline representation of the command line execution.
After that the command is passed to the console engine and a backtrace console window is attached to the process in order to deal with output and errormessages.

So the running h.pitfiller command looks like the following:

---

The same could have been done with any GRASS command, apart of the fact that you will not need to create the MACAD yourself :)

Alright, so let's see how to do the same with r.in.gdal, the GRASS native high power raster data import tool.

What you have to do different from before, is that when you create the action class, you need to tell the execution facility class that it is a GRASS command:


Create the xml file with the options you need:



and fully enjoy your new command:

Perhaps all do not know that... the interface JGrassConstants holds important path pieces

Since lots of things change as time passes by, the GRASS location and mapset path pieces are hold in a facility interface called JGrassConstants.
So instead of writing WIND to define the active region file, you can (or better should, or better must) use JGrassConstans.WIND

Here some of the most important parts:

/** folder of the JGrass database structure */
  public final String PERMANENT_MAPSET = "PERMANENT";

  /** folder of the JGrass database structure */
  public final String DEFAULT_WIND = "DEFAULT_WIND";

  /** folder of the JGrass database structure */
  public final String PROJ_INFO = "PROJ_INFO";

  /** folder of the JGrass database structure */
  public final String PROJ_WKT = "PROJ_INFO.WKT";

  /** folder of the JGrass database structure */
  public final String PROJ_UNITS = "PROJ_UNITS";

  /** folder of the JGrass database structure */
  public final String WIND = "WIND";

  /** folder of the JGrass database structure */
  public final String MYNAME = "MYNAME";

  /** folder of the JGrass database structure */
  public final String FCELL = "fcell";

  /** folder of the JGrass database structure */
  public final String CELL = "cell";

  /** folder of the JGrass database structure */
  public final String CATS = "cats";

  /** folder of the JGrass database structure */
  public final String HIST = "hist";

  /** folder of the JGrass database structure */
  public final String CELLHD = "cellhd";

  /** folder of the JGrass database structure */
  public final String COLR = "colr";

  /** folder of the JGrass database structure */
  public final String CELL_MISC = "cell_misc";

  /** folder of the JGrass database structure */
  public final String CELLMISC_FORMAT = "f_format";

  /** folder of the JGrass database structure */
  public final String CELLMISC_QUANT = "f_quant";

  /** folder of the JGrass database structure */
  public final String CELLMISC_RANGE = "f_range";

  /** folder of the JGrass database structure */
  public final String CELLMISC_NULL = "null";

Docs: theJGrass workspace

Overview of a GRASS database structure:




Most important files and folders for now

• WIND - contains the active processing region and the resolution
• PROJ_INFO - contains the information about the projection
• PROJ_UNITS - contains the information about projection units used
• cell, fcell - contain the raster files
• vector - contain the vector data since GRASS 6
• sites_list - contain the sites type data, deprecated from GRASS 6 on, but maintained in JGrass

File structure of GRASS Location

A GRASS raster map consists of several files in several subdirectories in a mapset, organized as follows:

• cellhd/: map header including projection code, coordinates representing the spatial extent of the raster map, number of rows and columns, resolution, and information about map compression;
• cell/, fcell/ or grid3/: generic matrix of values in a compressed, portable format which depends on the raster data type (integer, floating point or 3D grid);
• hist/: history file which contains metadata such as the data source, the command that was used to generate the raster map, or other information provided by the user;
• cats/: optional category file which contains text or numeric labels assigned to the raster map categories;
• colr/: optional color table;
• cell_misc/: optional timestamp, range of values, quantization rules (for floating point maps) and null (no-data) files;

GRASS vector maps are stored in several separate files in a single directory. While the attributes are stored in either a DBF file, a SQLite file or in an external DBMS (PostgreSQL, MySQL, ODBC), the geometric data are saved as follows:

• head: vector map ASCII header with information about the map creation (date and name), its scale and threshold;
• coor: binary geometry file which includes the coordinates of graphic elements (primitives) that define the vector feature;
• topo: binary topology file describes the spatial relationships between the map's graphic elements;
• hist: history ASCII file with complete commands that were used to create the vector map, as well as the name and date/time of the map creation;
• cidx: binary category index file which is used to link the vector object IDs to the attribute table rows;
• dbln: ASCII file which contains definition(s) of link to attribute storage in database (DBMS).

Taken from the GRASS developer's manual

Docs: the active region

Since our migration towards the udig community I've been struggling around mostly with two concepts that didn't fit exactly into the picture of the existing udig:

• the active region concept
• the GRASS workspace
  

While the second is not yet solved in my mind, the active region concept has come to its (hopefully) final version.

Let's go through it:

1) what is the active region (AR)?

The AR is a defined portion of the territoy inside which calculations can be performed. It is defined by a bounday (n,s,w,e) and a particular resolution (in both northern and eastern directions) and a number of rows and columns, that are directly bound to the resolution (rows and columns can be only integer numbers, so the resolution has to recalculated if the boundary calculus with the resolution doesn't gain integer rows and cols).

The AR concept is particularly useful for example when you have very large maps that don't even fit in you pc's memory and you want to do calculations on a smaller portion of the map or on the whole map, but with lower resolution. In these cases the AR can be set to a particular portion of the map or the resolution can be lowered and the models and algorythms from that moment on will do calculations only on that region, as if the map was contained only inside of it.

2) How to use the AR?

The AR is visually displayed by the Active Region Graphic, which can be found in the catalog view with the other mapgraphic objects.


Once you drag that object onto the map, the first time you will be asked for the mapset you want to use the active region from (for informations about what a JGrass workspace-location-mapset-etc is, have a look here).




Once choosen one mapset, the active region magically appears.


From that moment on an AR definition occurred and every algorythm and module that needs it can retrieve it and use it. The user doesn't have to care, it is the developer's care to include this concept in the calculation models.

3) setting the AR

There are two ways to set the active region.

a) setting the AR by its properties panel

The AR properties panel is rather straight forward. You can set the mapset from which to take the AR from, but please pay attention about the fact that the AR is consistent with the maps you are working on, i.e. don't use maps from different mapsets or locations, since you could gain very fancy results :) Always the same thing: always try to know what you are doing.

You can set boundaries and resolutions. Resolutions are recalculated everytime to be in shape woth the rows-cols consistency.

You can also set some style proprties, like color and transparency.

And last but not least you can also set the region to a particular map, raster or vector. in the case of a raster map, its fileregion (i.e. its boundaries and resolution) is taken and the AR is set to that. In the case of a vector map the region is set to its boundary, but then, since vector maps do not have a resolution definiton, the region is adjusted to snap its boundaries and resolution to the prior AR. This is mandatory to be able to coninue to work on a consistent way on the raster grid.




b) setting the AR by the box tool

The second way to set the AR is the Active Region tool that can be found in the toolbar. At the moment of writing it has the same icon of the selection tool, I hope to find some time to change it soon.



With this tool you will be able to just draw a box and the active region will be set to that box. In fact the selected box is adapted to snap to the prior AR and naturally will have the resolution of the prior AR.