GeoServer REST API vs. SDE-Layers

Yesterday i gave the GeoServer REST API a try to register approx. 250 Raster and vector layers all coming from an SDE-Database. My debian6 server hosts postgres 8.4 with SDE 10.0 and Geoserver 2.1.2 (tomcat6)…Previously I installed the sde- extension for geoserver

From my experience the most important issue on publishing hundreds of layers from an GDB is a 100 percent consistent naming convention. Note that in ArcGIS SDE-GDB the cool feature-dataset structure is not represented through the JAVA sde-driver, meaning that the ability to group layers must be possible trough their names…

My aim was to do all basic geoserver-gui tasks in a simple batch file from my win7 client:

  • create raster stores
  • register raster layers
  • register vector layers from existing store

The geoserver doc has some examples on how to interact with the REST API. I also found this project site (opengeoportal wiki)  very useful. My example uses curl…

Raster-Registration based on textfile containing sde layer names (newLayerListRaster.txt):

Result is:

  • raster store (as named in textfile)
  • raster layer with default style (as named in textfile)
@echo off
 SET username="????"
 SET password="????"
 SET sdeusername="????"
 SET sdepassword="????"
 SET sdehost="127.0.0.1"
 SET sdeport="????"
 SET geoserverhostandpath="????/geoserver"
 SET workspace="sde"
 SET list=%1
 echo ---GeoServer Raster Registration ArcSDE---
 echo REQUEST...
 echo arg1=NewStoreName: --from list--
 echo arg2=SDE-RasterTableName(DB.SCHEMA.TABLENAME): --from list--
 echo.
 echo RESPONSE...
 FOR /F %%i in (%list%) do curl -XPOST -u %username%:%password% -H "Content-Type:text/xml" -d "<coverageStore><name>%%i</name><type>ArcSDE Raster</type><enabled>true</enabled><url>sde://%sdeusername%:%sdepassword%@%sdehost%:%sdeport%/#%%i</url></coverageStore>" http://%geoserverhostandpath%/rest/workspaces/%workspace%/coveragestores
 FOR /F %%i in (%list%) do curl -XPOST -u %username%:%password% -H "Content-Type:text/xml" -d "<coverage><name>%%i</name><nativeFormat>ArcSDE Raster</nativeFormat><requestSRS><string>EPSG:4326</string></requestSRS><responseSRS><string>EPSG:4326</string></responseSRS><parameters><entry><string>OVERVIEW_POLICY</string><string>QUALITY</string></entry></parameters></coverage>" http://%geoserverhostandpath%/rest/workspaces/%workspace%/coveragestores/%%i/coverages
 echo.

 

Vector Registration (store must exist):

Result is:

  • vector layer (as named in textfile)
@echo off
 SET username=????
 SET password=????
 SET geoserverhostandpath=????/geoserver
 SET workspacename=????
 SET datastoreName=????
 SET list=%1
 echo ---GeoServer Vector-Layer Registration in existing Store for ArcSDE ---
 echo REQUEST...
 echo arg1=SDE-VectorTableName(DB.SCHEMA.TABLENAME): --from list--
 echo arg2=newStyleName: --from list--
 echo.
 echo RESPONSE...
 @echo on
 FOR /F %%i in (%list%) do curl -XPOST -u %username%:%password% -d {"featureType":{"name":"%%i"}} -H "Content-Type:application/json" http://%geoserverhostandpath%/rest/workspaces/%workspacename%/datastores/%datastoreName%/featuretypes
 @echo off
 echo.
 rem curl -XPOST -u %username%:%password% -H "Content-type: text/xml" -d "<style><name>%stylename%</name><filename>%styleName%.sld</filename></style>" http://%geoserverhostandpath%/rest/styles
 echo.

These two examples can easily be adopted to define projection settings and other layer settings. Just edit the xml or json part of the XPOST request. A good starting point is to define a layer fully by hand in the geoserver gui and look into the resulting layer.xml or coverage.xml in the geoserver data dir. The structure of these xml’s is quite straightforward. Then try to place the tags you want into the batch request…

The next thing i want to automate is style handling and assigning specific styles to layers… One approach is to extend the layerlist from our batch routine with a second or third attribute delimited by e.g. semicolon, holding style info…

Single commands: create style > upload sld file for style > assign style to layer

  • curl -u admin:geoserver -XPOST -H 'Content-type: text/xml' -d '<style><name>newStyleName</name><filename>newStyleName.sld</filename></style>' http://?????/geoserver/rest/styles
    
  • curl -u admin:geoserver -XPUT -H 'Content-type: application/vnd.ogc.sld+xml'-d '@newStyleName.sld' http://?????/geoserver/rest/styles/schwemmland
    
  • curl -u admin:geoserver -XPUT -H 'Content-type: text/xml' -d '<layer><defaultStyle><name>newStyleName</name></defaultStyle></layer>' http://?????/geoserver/rest/layers/LayerName
    

Conclusion:

Depending on the amount of layers the REST API can save a lot of time.

Another setup of a geo-aware application server

Requirements:

  • Traditional webGIS functions
  • Spatial DB
  • Vector and Raster Data Storage
  • Geodata Integration via web Interface
  • Open source software

Solution:

  • Use openlayers for frontend functions, add value with some magic JS (jquery or extJS)
  • Use tomcat & geoserver as server backend
  • Use Postgresql & postgis as a spatial datawarehouse
  • Use geowebcache for serving cached basemaps and overlays
  • Alternatively use ArcSDE with postgres if your customer wants to manage all geodata with ArcGIS  -> licence issues
  • Hold data in DB and on file system
  • Use JSP and J2EE Servlets with tomcat for building the webGIS app
  • Rely on well-established GIS frameworks like geotools, gdal/ogr or JTS for doing GIS jobs over the web
  • Use Debian as OS

As of July 2012 the following versions of software tools are available:

Server-software:

  • Tomcat7
  • Geoserver 2.1.4
  • Postgres 9.1.4
  • Postgis 2.0
  • ArcSDE 10.1
  • JDK 7
  • Gdal 1.9
  • Geowebcache 1.3

Frontend tools:

  • OpenLayers 2.12
  • Jquery 1.7.2
  • extJS 4.1

Think of all the requirements each piece of software has against any other underlying software…

Step1

Debian 6 (squeeze) 64bit, minimal system

–No X server running, disable ipv6

–Use hardening-tools and security updates

Step2

Install JDK 64bit delivered by apt (non-free)

Step3

Install Postgres 9.0 if you plan to use ArcSDE 10.1

Step4

Install Postgres 9.1 if plan to use PostGIS

Step5

Install tomcat6 delivered by apt

–Look at http://sebthom.de/142-installing-tomcat-6-debian-squeeze/

–Set tomcat to Port 80 by using iptables

—-iptables -A INPUT -i eth0 -p tcp –dport 80 -j ACCEPT

—-iptables -A INPUT -i eth0 -p tcp –dport 8080 -j ACCEPT

—-iptables -A PREROUTING -t nat -i eth0 -p tcp –dport 80 -j REDIRECT –to-port 8080

–set JAVA options according to your amount of RAM

Step6

Install geoserver 2.1.3

–Use war file

–Deploy with tomcat

Step7

Install geowebcache 1.3

–Use war file

–Deploy with tomcat

Step8

Install ArcSDE 10.1 via network using ArcGIS Desktop toolbox

–Look at esri-docs

–Choose whether you want to go completely esri, by using the esri geometry type (and raster type) exclusively inside Postgres, OR if you plan to use both: PostGIS geometry and esri geometry datatypes

Step9

Install GDAL 1.9

–Compile from source

–Choose your needed format drivers

–Install and choose your preferred bindings (JAVA, Python,…)

Step10

Configure your system

–Bring in your data

–Configure geoserver (e.g. install sde-plugin)

—-Style layers

–Configure geowebcache (register pre-cached tilesets from ArcGIS server, or create new ones)

–Configure postgres

—-Setup user management schema for webapp

—-Setup essential app config tables (layer related to app)

—-Setup multiple viewer configurations

—-Implement a web admin tool for controlling the webGIS app

–Develop JSP’s and J2EE servlets for app logic (Eclipse)

–Develop frontend client and connect to servlets