Browsing WeoGeo Market Using the ESRI Silverlight API

Updated: This demo application now running here. I will update this demo periodically, as time permits, so keep checking back.

At the 2010 ESRI Federal User Conference, WeoGeo announced the availability of a toolbar for interacting with WeoGeo Market and private libraries from within ArcMap. This, combined with Dan Dye’s series of posts showing how to use the WeoGeo REST API with Python got me thinking about how easy it would be to integrate with ESRI’s clients for the ArcGIS Server REST API. All of my clients (it seems) are using the Silverlight API these days so I am spending a lot of time with it and decided to use it as my testbed.

My goal was simple, I wanted to browse the WeoGeo Market for any data sets in the current map extent, be able to select one from a list, and have its preview image display in the proper location on my Silverlight map.

WeoGeo provides the tools needed to store spatial data online, in the cloud, and sell/disseminate it from there. It gives a user the ability choose options such as spatial reference, data format, geographic extent and other such parameters when they download. If you are a data provider that is selling data, WeoGeo can host it and handle the sales transaction for you. WeoGeo has more advanced data management capabilities as well but they are beyond the scope of this post, although I may delve into them in future posts.

You can find and order data from WeoGeo completely online via their web site but, as the toolbar demonstrates, the WeoGeo API enables integration into the environment in which you work. A key part of the WeoGeo process is the ability to preview data sets to ensure that they are what you need. WeoGeo provides low-resolution KML images as well as other preview images to accomplish this via the web site. Those preview images are also exposed via the WeoGeo API. I decided to hook into the PNG images that are normally delivered via KML.

The WeoGeo datasets API provides the means to browse data that is available within a geographic extent. This is accomplished through a GET request described here. There are many parameters available to allow you to filter data sets but I stuck with the standard parameters this time around. The response depends on what you ask for. GET /datasets.weo will return WeoGeo’s XML document, or weo file. GET /datasets.json will return a JSON formatted listing. For this attempt, I went with JSON.

The call returns a list of data sets. The JSON for a single data set looks like this:

[sourcecode lang=”javascript”]
{
"votes": 0,
"royalty_model": "UNCREDITED",
"rating": 0.0,
"provider_discount_expire_option": null,
"permalink": "spatialed_sample_sql_server_data",
"name": "Sample SQL Server Data",
"step_kamap": 0.0717937948,
"spatial_resolution_in_meters": 3233.80355656636,
"user": {"votes": 0, "rating": null, "username": "spatialed"},
"center_lat": 45.16073438845,
"west_kamap": -179.1333928049,
"south_kamap": -134.2235717723,
"number_of_layers": 3,
"uncompressed_misc_files_size": 0,
"token": "37a39b7d-af4e-d55d-9aa6-1aaff9353594",
"north_kamap": 224.6980287277,
"center_long": 0.3283989648,
"layers": ["Layer_1", "Layer_2", "Layer_3"],
"data_type": "OTHER",
"data_created_on": null,
"from_appliance?": false,
"provider_discount_expires_at": null,
"primary_tag": "SQL",
"south": 18.9234204317,
"projection": "GEO",
"market": "Complete",
"file_format": "GeoTIFF",
"east_kamap": 179.7882076951,
"datum": "WGS84",
"children_count": 0,
"uploaded_at": "2009/06/01 22:23:52 -0400",
"uncompressed_data_files_size": 75000770,
"provider_discount_rate": 0,
"price_type": "FIXED",
"parents_count": 0,
"north": 71.3980483452,
"max_price": 5.04,
"y_conv": "1.0000000000",
"west": -179.1333928049,
"description": "u003Cpu003ESQL Server 2008 Spatial data for the United States. The data is supplied as a zipped SQL Server Backup file (Sample_USA.bak.zip) and contains the following tables:u003C/pu003Ernu003Cbru003Ernu003Culu003Ernu003Cliu003EUS Counties u003Ciu003E[polygons]u003C/iu003Eu003C/liu003Ernu003Cliu003EUS States u003Ciu003E[polygons]u003C/iu003Eu003C/liu003Ernu003Cliu003EUS Zipcodes u003Ciu003E[polygons]u003C/iu003Eu003C/liu003Ernu003Cliu003EUS CensusBlockGroups u003Ciu003E[polygons]u003C/iu003Eu003C/liu003Ernu003Cliu003EUS GeoNames u003Ciu003E[points]u003C/iu003Eu003C/liu003Ernu003Cliu003EUS Highways u003Ciu003E[linestrings]u003C/iu003Eu003C/liu003Ernu003C/ulu003Ernu003Cpu003EEach table contains a column of type geometry (geom) and a column of type geography (geog). The geom column is in a spherical Albers Equal Area projection (the sphere is WGS 84 Authalic, the units are in meters ). The geog column is in WGS 84 ellipsoidal coordinates.u003C/pu003Ern rn ",
"x_conv": "1.0000000000",
"status": "Approved",
"spatial_resolution": 0.04114097905,
"scales": "357765655;205940642;118545611;68238409;39280075;22610789",
"provider_min_margin": 0.0,
"hosted": true,
"east": 179.7901907345,
"provider_max_discount": 0.0,
"provider_margin": 0.0
}
[/sourcecode]

From this, I can parse anything I need to know about the data set. In order to handle this, I created a class in C# that can deserialize this JSON into a .Net object. A snippet of that class is here:

[sourcecode lang=”csharp”]
[DataContract()]
public class BrowseDataset
{
[DataMember(Name = "token")]
public string Token { get; set; }
[DataMember(Name = "name")]
public string Name { get; set; }
[DataMember(Name = "description")]
public string Description { get; set; }
[DataMember(Name = "north_kamap")]
public double North { get; set; }
[DataMember(Name = "south_kamap")]
public double South { get; set; }
[DataMember(Name = "east_kamap")]
public double East { get; set; }
[DataMember(Name = "west_kamap")]
public double West { get; set; }
[DataMember(Name = "data_type")]
public string DataType { get; set; }
[DataMember(Name = "file_format")]
public string Format { get; set; }
[DataMember(Name = "data_created_on")]
public string CreateDate { get; set; }
[DataMember(Name = "provider_margin")]
public double FullPrice { get; set; }
[DataMember(Name = "hosted")]
public string Hosted { get; set; }
[DataMember(Name = "projection")]
public string Projection { get; set; }
[DataMember(Name = "datum")]
public string Datum { get; set; }
[DataMember(Name = "center_lat")]
public double Latitude { get; set; }
[DataMember(Name = "center_long")]
public double Longitude { get; set; }
[DataMember(Name = "children_count")]
public int Children { get; set; }
[DataMember(Name = "parents_count")]
public double Parents { get; set; }
[DataMember(Name = "user")]
public BrowseUser User { get; set; }
}
[/sourcecode]

As can be seen, I used DataContract and DataMember attributes to map the JSON values to properties. I also simplified the property names. If you take the time to compare, you’ll notice that I don’t handle all of the information. The dataset JSON contains more information than I needed for simply browsing so I am filtering out some of it. This doesn’t reduce traffic across the wire but does cut down on memory usage in my Silverlight client.

For my purposes, I decided to extend the ESRI WMS sample for the Silverlight API. I chose this mainly because I am working in it at the moment for another project. To that UI, I added a button that will fetch a WeoGeo dataset listing for my current map extent.

When that button is clicked, this event handler is fired:

[sourcecode lang=”csharp”]
private void btnFetch_Click(object sender, RoutedEventArgs e)
{
weo.ProxyUrl = PrefixProxy("http://xxxx").AbsoluteUri;
weo.BrowseDatasetCompleted += new BrowseDatasetCompletedHandler(weo_BrowseDatasetCompleted);
ESRI.ArcGIS.Client.Geometry.Envelope env = MyMap.Extent;
weo.getDatasetList(1, env.YMax, env.YMin, env.XMax, env.XMin);
}
[/sourcecode]

“weo” is an instance of a wrapper class I created to encapsulate to WeoGeo interaction.

You will notice that I set up a proxy URL. Silverlight and Flash both are designed to only call back to the server from which they are being served. Any external servers must have a crossdomain.xml file to allow calls from these clients. This means that any external server you may want to use must have one of these files in order to receive calls from Flash or Silverlight clients. Fortunately for this application, this “security” feature can be circumvented by calling to a proxy handler back on our server which brokers requests and responses to and from external servers. The ESRI sample included a proxy handler that suited my needs so I just used that.

The PrefixProxy (also from the ESRI sample) method includes some Silverlight-specific code so I kept it in my Silverlight Page class. I build a dummy call using the “http://xxxx” token so that my WeoGeo wrapper class doesn’t need to be Silverlight-specific. The wrapper then replaces the token with the correct call to the WeoGeo server. The wrapper’s getDatasetList method looks like this:

[sourcecode lang=”csharp”]
public void getDatasetList(int page, double north, double south, double east, double west)
{
request.DownloadStringCompleted += new DownloadStringCompletedEventHandler(request_DownloadStringCompleted);

try
{

string url = string.Format(_datasetBrowse,_protocol, _library, page.ToString(), north.ToString(), south.ToString(), east.ToString(), west.ToString());

url = this.ProxyUrl.Replace("http://xxxx", url);
//System.Windows.MessageBox.Show(url);
request.DownloadStringAsync(new Uri(url));
}
catch (WebException ex)
{
throw ex; //handled elsewhere
}
}
[/sourcecode]

The WebClient class in Silverlight differs from the standard .Net WebClient in that it only makes asynchronous calls. So I attached a handler for the DownloadStringCompleted event. I then format the tokenized proxy URL with the correct URL to the WeoGeo Market and make the call. The _datasetBrowse variable is simply the template for the call and is defined as:

[sourcecode lang=”csharp”]
private string _datasetBrowse = @"{0}{1}/datasets.json?page={2}&north={3}&south={4}&east={5}&west={6}";
[/sourcecode]

One note about this call: WeoGeo expects the values for the extent to be in the WGS84 (EPSG:4326) spatial reference. The ESRI Silverlight API does not do coordinate transformations internally so, if your map is not in WGS84, you will need to transform your extent before making the call to the WeoGeo API. In my case, I kept my life simple by using the Blue Marble WMS service that is served by JPL and is in WGS84 (this is a great resource and @JeffHarrison was dead-on for reminding me of that fact) .

The DownloadStringCompleted event handler looks like this:

[sourcecode lang=”csharp”]
void request_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
try
{
string s = e.Result;
byte[] b = Encoding.Unicode.GetBytes(s);
MemoryStream st = new MemoryStream(b);
DataContractJsonSerializer ds = new DataContractJsonSerializer(typeof(BrowseDatasets));
BrowseDatasets retval = (BrowseDatasets)ds.ReadObject(st);
foreach (BrowseDataset d in retval.items)
{
d.Library = _library;
}
this.BrowseResults = retval;
BrowseDatasetCompleted(retval);
}
catch (Exception ex)
{
//System.Windows.MessageBox.Show(ex.ToString());
BrowseDatasets erDs = new BrowseDatasets();
erDs.items = new List<BrowseDataset>();
BrowseDatasetCompleted(erDs);
}
}
[/sourcecode]

In essence, it takes the JSON response, deserializes it into a list of BrowseDataset objects and then raises the wrapper’s BrowseDatasetCompleted event to pass the object back to the Silverlight application. In this case, an exception produces an empty list.

Back in the Silverlight application, the BrowseDatasetCompleted event handler looks like this:

[sourcecode lang=”csharp”]
void weo_BrowseDatasetCompleted(Weo4Net.Data.BrowseDatasets ds)
{
if (ds.items.Count > 0)
{
this.weoDatasetList.Items.Clear();
this.weoDatasetList.DisplayMemberPath = "Name";
foreach (BrowseDataset bds in ds.items)
{
this.weoDatasetList.Items.Add(bds);
}
this.weoDatasetList.Visibility = Visibility.Visible;
}
else
{
this.weoDatasetList.Visibility = Visibility.Collapsed; //hide if no data sets
}
}
[/sourcecode]

We could do anything we want here but I am simply adding the data sets to a list box so they can be selected and previewed. At this point, we are ready to put some stuff on the map.

The ESRI Silverlight API provides a layer class called an ElementLayer. This gives you the ability to add one or more UIElement objects to the map to get a very interactive UI. In Silverlight, if you can see it, it’s probably a UIElement so this is how you can add buttons, media, pictures, etc. onto your map. I used an ElementLayer to display the preview images from WeoGeo.

Each BrowseDataset object contains the North, South, East and West properties that define the geographic extent of the data and, thus, the preview image. When the user selects a dataset in the list, I use this information to add an Image element to the layer and define where it should be displayed. That is all handled in the list’s SelectionChanged event handler like this:

[sourcecode lang=”csharp”]
private void weoDatasetList_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
ElementLayer elyr = MyMap.Layers["weoPreviewLayer"] as ElementLayer;
BrowseDataset ds = this.weoDatasetList.SelectedItem as BrowseDataset;
elyr.Children.Clear(); //clear any previous previews
string url = String.Format("http://weodata.weogeo.com/dataset_tiles/{0}/kml.png", ds.Token);
Image img = new Image();
BitmapImage bmi = new BitmapImage(new Uri(url));
img.Source = bmi;
//set the ElementLayer.Envelope attribute of the Image
ESRI.ArcGIS.Client.Geometry.Envelope env = new ESRI.ArcGIS.Client.Geometry.Envelope(ds.East, ds.South, ds.West, ds.North);
ElementLayer.SetEnvelope(img, env);
elyr.Children.Add(img);
}
[/sourcecode]

As can be seen, this is designed to display one preview at a time. The main trick here is that the ElementLayer class defines an attached property called “Envelope” to define the geographic extent of the UIEelement. In XAML, it would look like this:

[sourcecode lang=”xml”]
<Image Stretch="Fill" Source="http://weodata.weogeo.com/dataset_tiles/fc69c451-4714-8250-90d4-91b74528127e/kml.png" esri:ElementLayer.Envelope="-78.0496954009,34.9499119752,-71.9494760266,40.0496953721" />
[/sourcecode]

In code, you use the static ElementLayer.SetEnvelope method to set this value for your UIElement. With that, you can now pan/zoom to an extent that you want, click a button to fetch a dataset listing, and select select individual datasets in the list to see the preview images.

I’ll probably continue to explore the WeoGeo API with this application over time but this is my first pass. I have it running on one of our servers to play with but here is a screen capture: