UESPWiki:Oblivion Map Design

All the Google map specific documentation on this page refers to the older and now deprecated v2 maps API. The new map can be found here

This article covers various aspects of the design of UESP's online Oblivion map using the Google Maps API. This is currently a functional Oblivion Map, although improvements continue to be made.

The Basic Map

Google Maps With An Oblivion Map

By default Google Maps uses its own tile server to serve up images of various types from Earth (satellite, road maps, etc...). We will have to use a custom map in order to display the Oblivion map data instead of Earths. This tutorial (archived) on custom maps shows the basic steps needed. The main thing is the addition of a custom tile function which translates the requested coordinates and zoom level into a tile image, for example:

 CustomGetTileUrl = function(Coordinate, Zoom) {
      return "tile_" + Coordinate.x + "_" + Coordinate.y + "_" + Zoom + ".jpg"
 }
 

If the input coordinate was (22, 44) and zoom was 3, this function would return: tile_22_44_3.jpg

The input coordinate and zoom values are described as follows:

• Coordinate origin at (0,0) with increasing values to the right and down.
• Zoom levels start at 0 (the farthest zoomed out) and increase from there (arbitrary limit, as many as needed).
• The expected number of tiles per zoom level is: 2^Zoom. So at zoom level 0 there should be only 1 tile while at zoom level 9 the map is split into 512x512 tiles.
• If using the Mercator projection the tile origin (0,0) is at coordinate (90,-180). There are projection methods to convert to/from tile and latitude/longitude coordinates.

Creating the Oblivion Map

Using the information in the previous section on how the Google Maps API works is possible to figure out how to arrange our Oblivion map tiles in order to get them assembled into a map image. This will quickly become the most complicated step in the process of making a working map. Basic steps in creating the Oblivion map tiles are listed below:

Export Local Map Data

Using the Create Local Maps command in the World menu of the Construction Set. Select Exterior Cells and choose Tamriel (we may have to export the other exterior cells later). The CS seems to have a problem exporting map data as it crashes as soon as it runs out of RAM. For example, on my machine with 2GB of RAM I could generally only export 500 cells or so before it crashed. This forces us to export the map in sections by using the NW CellX, NW CellY, SE CellX and SE CellY fields. The overall cell limits in Oblivion are (-64,59) in the NW to (69,-69) in the SE (134x129 cells). Note that a good amount of those cells are outside of the 'real' Oblivion area (as much as half perhaps). I exported cells going from left to right in manageable sections (i.e., from (-64, 59) to (-60, -69)). If the CS crashes make sure you are not missing any tiles.

Exported Map Tiles

The exported map data will be saved as uncompressed 256x256 DDS files in the Data\textures\Maps\Oblivion folder of your Oblivion path (unsure if you have to create this before hand or not). The output filename format is:

  [CellName].[CellX].[CellY].dds

  ex: Tamriel.-23.01.dds

The cell values are a minimum of 2 characters in size, so 1 will be output zero padded as 01 while -1 will be output as just -1 without padding. If you output the entire range of cells you'll end up with 134x129=17286 images around 256kb in size each.

The output images actually show slightly more than a complete cell each as they overlap with adjacent cells as follows:

• Right edge contains 15 pixels of overlap with the cell to the right.
• Top edge contains 14 pixels of overlap with the cell to the top.
• Bottom edge contains 1 pixel of overlap with the cell to the bottom.
• Left edge has no overlap

If you eliminate this overlap you'll be left with a 241x241 image that will show the cell border on all 4 edges. It may be necessary to remove one pixel from the right and top edges to end up with a 240x240 image although the 241x241 image seems to tile fine.

Converting Map Tiles

A number of conversions steps must be performed in order to get the exported map tiles into a format compatible with the Goggle Maps API.

1. Convert to JPG with a quality level around 20%. This reduces the average image size from 256kb to 17kb. You can compress higher but it begins to degrade the image noticeably past 20%.
2. Trim the cell overlap in each image ending up with a 241x241 (or 240x240) image.
3. Resize each image back to 256x256. We could cut/paste images to get the 256x256 size needed by the API but it is much faster to just resize the images. Resizing also smooths out imperfections in the image and make it look slightly better anyways.
4. Rename the file to match that needed by the API. The Y coordinates between Oblivion and Goggle Maps are reversed and we need to set the origin to a non-negative value. For simplicity we'll set the origin to (0,0) although this may change later.

At this point we should have necessary images to test the map at the largest zoom level. We can test this locally by simply setting the custom map tile function to point to the images on our hard drive.

Generating Zoom Levels

We have got the highest zoom level from manipulating the exported Oblivion map tiles but we have to now generate all lower zoom level map tiles as well. Since zooms are just a power of two this is relative simple: just combine 4 adjacent map tiles and reduce the image size by 50%. Things only get tricky at the right and bottom edges since our overall tile size (134x129) is not a perfect power of two. This can be solved by using a default image, like an empty water tile, for any missing image. We just keep repeating the zoom process until we are left with a 1x1 map tile size.:

For convienence we'll place each zoom level in its own directory using a tile filename format like: zoom15/tamriel-34-24-15.jpg.

In the end we should have some 23,000 images in 8 different zoom levels totalling some 400MB which will allow us to make a working map able to zoom in and out.

Simple Test Map

The basic page for displaying the Oblivion map is shown below:

<html><head><title>Oblivion Test Map</title>
    <script src="http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAdCxcZPL1FlURx_ZuWGDXPBQgXfZEq_9lK90U8L4elbuzKplcARR3-FuKzQZL-QHw0_mnLYqOXQ7VxA"
      type="text/javascript"></script>
</head>
<body onload="load()" onunload="GUnload()">
    <div id="map" style="width:800px; height:600px;"></div>
    <script type="text/javascript">
    //<![CDATA[

    function load() {
      if (GBrowserIsCompatible()) {
        var map = new GMap2(document.getElementById("map"));
        map.addControl(new GLargeMapControl());

        var copyCollection = new GCopyrightCollection('Map Data');
        var copyright = new GCopyright(1, new GLatLngBounds(new GLatLng(-90, -180), new GLatLng(90, 180)), 8, "uesp.net");
        copyCollection.addCopyright(copyright);

        function CustomGetTileUrl(a,z) {
          return "zoom" + z + "/tamriel-"+a.x+"-"+a.y+"-"+z+".jpg";
        }

        var tilelayers = [new GTileLayer(copyCollection, 9, 16)];
        tilelayers[0].getTileUrl = CustomGetTileUrl;

        var custommap = new GMapType(tilelayers, G_SATELLITE_MAP.getProjection(), "Tamriel", {errorMessage:_mMapError});+
        map.addMapType(custommap);
        map.setCenter(new GLatLng(85.021, -179.6), 11, custommap);
      }
    }
    //]]>
    </script>
</body></html>

Things to note in this version:

• The zoom levels of 9 (zoomed out) to 16 (maximum zoom) are used.
• Map tile origin is at (0,0)
• The default projection (Mercator) results in some strange coordinates due to our choice of tile origin. The center of the map is roughly at (85.021, -179.6). The latitude coordinate (85.021) is particularly sensitive since we're near the North pole with this projection.
• No checks for valid coordinates in the custom tile function will result in blank (gray) areas where image files are missing.

Flat Projection

As seen in the test map example we run into some minor problems due to the default mercator projection used by the Google Maps API. This is a standard projection for displaying flat maps of a spherical object, like Earth. Since the Oblivion game map is not based on a spherical world a flat map projection would be better. We'll use this tutorial which gives an implementation for the Euclidean map projection type.

Using a Euclidean projection we find a more reasonable behavior of the Y coordinate (latitude). In the previous Mercator projection the map extents were roughly (84.99,-180) to (85.05, -179.25). Note the very small latitude range due to the proximity of the North pole. In the Euclidean projection our map extents are a more reasonable, and square, (89.62, -180) to (90, -179.25).

Another alternative is to place the map tile origin closer to the center of the map. While this is possible it complicates the tile naming since each zoom level will have a different origin index. However, we will very likely be using a non-zero tile origin in order to add other continents of Tamriel (like Daggerfall and Morrowind).

More Features

There are a number of other map features that are useful as described below:

Map Bounds

We're already limiting the zoom levels by creating a custom map tile layer but we still need to limit the latitude/longitude to keep within the Oblivion map bounds. The custom map tutorial includes a section on how to accomplish this using a move listener event. There are two additional lines needed in the checkBounds function since we're near the origin:

  if (X > 0.0)   {X = AminX;}
  if (Y < 0.0)   {Y = AmaxY;}

Adding these before the other X/Y bounds checks will prevent the map from scrolling incorrectly at the top and left edges. Note that this is only needed if we're using the map tile origin of (0,0).

Overview Map

Another good tutorial on how to add a small overview map.

Invalid Tile Checking

The current version of the map doesn't check if a given map tile is valid or not. This results in missing images being displayed as blank gray space. We can modify the tile function to check for an invalid tile index and use a custom map tile in that case:

 function CustomGetTileUrl (Coordinate, Zoom) {
   if (Zoom >= 9 && Zoom <= 16) {
        var MaxX = Math.floor(133/Math.pow(2, 16 - Zoom));
        var MaxY = Math.floor(129/Math.pow(2, 16 - Zoom));

        if (Coordinate.x < MaxX && Coordinate.y < MaxY && Coordinate.x >= 0 && Coordinate.y >= 0) {
             return "zoom" + Zoom + "/tamriel-" + Coordinate.x + "-" + Coordinate.y + "-" + Zoom + ".jpg"; 
        }
   }
   return "oorimage.jpg";
 };

The oorimage.jpg is a 256x256 image to be displayed for any map tile outside of the Oblivion map bounds.

More Complete Map Example

See the source code for a complete map using the features explained above. The map itself is working well and now we just need to add some actual location markers to it.

Map Locations

Required Data

In order to show locations on the map the following data will be needed:

• Location name
• Oblivion map location (X/Y/Z)
• Cell X/Y (may not be needed if it can be derived from the location X/Y)
• Description (may be needed for some locations)
• Editor and form ID (probably not required)
• Location type (dungeon, house, landmark, shrine, etc...)
• Latitude/Longitude (can likely be derived from the location X/Y)

Most of this information can be directly exported from Oblivion.esm using a custom utility (the Construction Set cannot export any of this information).

Exporting Data From Oblivion

The relevant location data needed from Oblivion.esm is found deep within the top level WRLD group.

• WRLD Group
  • WRLD Children (match by FormID of world space, Tamriel=0x3C)
    • CELL Child Group
      • CELL Persist Group
        • REFR records for persistent locations (ones with XTEL subrecords)
    • Exterior Cell Groups
      • Exterior Sub-Cell Groups
        • CELL Children Groups
          • Persistent Children Group
            • REFR location records (ones with XTEL subrecords)
          • Temporary Children Group
            • REFR location records (ones with XTEL subrecords)

The location information is stored within a RERF (reference) record that has a XTEL (teleport) subrecord which gives the destination location.

• REFR Record
  • NAME Subrecord (form ID)
  • EDID Subrecord (editor ID)
  • FULL Subrecord (full location name)
  • XTEL Subrecord (teleport information)
    • Destination Door Reference ID (formid)
    • Destination Location (float X, Y, Z)
    • Destination Angle in Radians (float AX, AY, AZ)
  • DATA Subrecord (reference data)
    • Location (float X, Y, Z)
    • Angle in Radians (float AX, AY, AZ)

Exported Location Data

The exterior location data exported from Oblivion.esm using the ObEdit utility looks like (some columns removed):

FormID	WorldSpace	LocX	LocY	LocZ	AngleX	AngleY	AngleZ	DestFormID	DestName	DestLocX	DestLocY	DestLocZ	DestAngleX	DestAngleY	DestAngleZ
0x0000BAEF	Tamriel	-67407.4	61902.5	10713.7	0	0	-1.5	0x0000BAE2	Chapel%20of%20the%20Brethren	2131.59	1618.05	6894.9	0	0	-1.57
0x000CA84D	Tamriel	-31584.4	26890.2	3657.52	-0.27	0.3	-0.74	0x000CA830	Ceyatatar%20Gorihame	-4773.42	-1463.89	-191.32	0	0	1.41
0x000C50D4	Tamriel	44363.1	52565.2	134.56	0	0	-2.36	0x000C50A5	The%20South%20East%20Tunnel	-1346.9	-4076.69	46.61	0	0	-0.03
0x000AD44D	Tamriel	-172914	-17250.4	5036.92	0	0	-3.08	0x000818BC	Garlas%20Agea	-1011.99	-7037.76	132.44	0	0	0
0x000AD3E4	Tamriel	83790.5	68266.1	4689.6	0	0	-0.18	0x00082AE8	Nagastani	-1518.46	-881.26	-187.79	0	0	7.59E-007
0x000AD389	Tamriel	-95423.3	76174.4	13620.8	0	0	-2.08	0x000A229D	Broken%20Promises%20Cave	-3516.17	2630.26	106.58	2.56E-007	-0.1	1.5
0x000AD374	Tamriel	132381	-132589	4002.08	0	0	2	0x000A6E76	Onyx%20Caverns	-4585.76	569.51	697.23	0	0	1.5
etc...

Exporting exterior locations from all worldspaces gives us 920 records.

Map Markers

It turns out that Oblivion actually uses special map marker references to display the icons on the map. These markers are references of the MapMarker (0x10) static object and contain a full location name (FULL subrecord) as well as a location type (TNAM subrecord). Marker references are identified by the empty XMRK subrecord.

There are 451 exterior map markers in Oblivion.esm although some of these will duplicate locations exported using the teleport information. We can merge the two sets of location data by looking for locations very close to each other (within 1500 units) with the same, or similar, names.

Overall Design

There are a number of methods possible for serving up the map locations to us.

• Internal JavaScript -- The locations are stored in an array in the JavaScript format on the actual map page. This is the simplest design and would work well for a small number of locations. Unfortunately, once the number of locations begins to increase it will result in a very large and hard to maintain file. Since we will likely have 1000s of locations in the future this is not a good design for us.
• PHP Using Data Files -- This design has the map script query a PHP script on the uesp.net server for a list of locations within a particular area. The PHP script reads locations from a data file (XML, or CSV for example) and responds with the appropriate XML data. The map script then parses this data to get the marker information to display. This greatly increases the complexity since we're now dealing with ansynchronous data. It has the advantage of keeping the location data stored on the server which makes it easier to manage and only the necessary location data is transferred. This allows us to have virtually any amount of location data.
• PHP Using Database -- The same as the previous method but using a database (MySQL) for storing the location data. This simplifies the PHP somewhat as we don't have to worry about the performance scaling of the file format used for the location data. This is the method we'll be using.

Table Design

The following are a description of the minimum fields needed for storing the location information.

Field	Type	Description
ID	INT (autoincrement)	Unique identifier for the location.
FormID	UNSIGNED INT	FormID for the location.
EditorID	TINYTEXT	Editor ID of the location if it has one.
WorldSpace	TINYTEXT	World space name the location is found in.
ObLocX	float	Position of the location in Oblivion world coordinates.
ObLocY	float
ObLocZ	float
DestFormID	UNSIGNED INT	FormID of the teleport destination.
DestName	TINYTEXT	Name of the teleport destination.
Type	INT	Type of location (town, dungeon, mine, etc...).
Level	INT	At what zoom level should the location be displayed in (higher values for more important or larger locations).

Basic PHP Location Script

A simple PHP script for accessing the locations in the database and returning them in a XML format is not too complex (taken from MapKI):

<?php
$host = 'localhost';
$user = '...';
$pass = '...';
$dbname = 'uesp_map_data';

if (!$db = mysql_connect($host, $user, $pass)) {
        echo 'Could not connect to mysql';
        exit;
}
if (!mysql_select_db($dbname, $db)) {
        echo 'Could not select database';
        exit;
}
      // Date in the past
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); 
      // always modified
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
      // HTTP/1.1
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
      // HTTP/1.0
header("Pragma: no-cache");
      //XML Header
header("content-type:text/xml");

$query = "SELECT ID, ObLocX, ObLocY, ObLocZ, DestName FROM test_map";
$query = mysql_query($query);
echo "<locations>";
while ($row=mysql_fetch_assoc($query)){
        echo '<location id="'.$row['ID'].'" name="'.$row['DestName'].'" X="'.$row['ObLocX'].'" Y="'.$row['ObLocY'].'" X="'.$row['ObLocZ'].'"/>';
}
echo "</locations>";
?>

This script simply returns all the locations in the database which is fine for testing purposes. A more complete version will only return locations within a particular map area or locations matching an input search string.

Converting Oblivion Locations

The Oblivion map data uses its own coordinate system while the map uses latitudes and longitudes. In order to display the locations on the map we will have to convert between the two systems. We'll need a number of settings in order to get the conversion functions:

• Map tile origin (currently set to (0,0))
• Number of map tiles at maximum zoom (134 x 129)
• Map projection (currently using Euclidean)
• Zoom levels used (currently 9 to 16)
• Extents of the map tiles in Oblivion coordinates

• Upper Left (-64,59) = (-262145, 245759) = 4096*(-64, 60)
• Lower Right (69,-69) = (286719, -282625) = 4096*(70, -69)

• Oblivion cell size (4096 Oblivion units)
• Map bounds in Latitude/Longitude (not as accurate but makes it easier)

• Lower Left = (89.62,-179.9999)
• Upper Right = (89.9999, -179.25)

Using these values the conversion functions are:

    function ConvertObLocToLng(X, Y) {
         return -180.0 + ((X/4096.0 + 64)/65536.0 * 360.0);
    }
    
    function ConvertObLocToLat(X, Y) {
         return 90.0 - ((59.0 - Y/4096.0)/65536.0 * 180.0);
    }

Basic Map Script Changes

The map script needs some additions in order to request the location data from the PHP script and read the resulting location XML data (taken from MapKI):

function getMarkers(){
    var urlstr="read.php";
    var request = GXmlHttp.create();
    
    request.open('GET', urlstr , true); // request XML from PHP with AJAX call
    request.onreadystatechange = function () {
         if (request.readyState == 4) {
              var xmlDoc = request.responseXML;
              locations = xmlDoc.documentElement.getElementsByTagName("location");
              markers = [];

              if (locations.length){
                   for (var i = 0; i < locations.length; i++) { // cycle thru locations
                        var Lat = ConvertObLocToLat(locations[i].getAttribute("ObLocX"), locations[i].getAttribute("ObLocY"));
                        var Lng = ConvertObLocToLng(locations[i].getAttribute("ObLocX"), locations[i].getAttribute("ObLocY"));
                        markers[i] = new GMarker(new GLatLng(Lat,Lng));
                        markers[i].infowindow = "This is " + locations[i].getAttribute("DestName");
                        markers[i].markerindex = i;
                        markers[i].db_id = locations[i].getAttribute("ID");
                        map.addOverlay(markers[i]);
                   }
              }
         }
    }
    request.send(null);
}

All this function does is to request all locations from the PHP script when it is called and to display them once they are received.

Versions

Software Updates

v0.3.1, 6 July 2008

• All three UESP maps completely moved to new, merged code
• Minor improvements:
  • Accept locx+locy as input instead of oblocx+oblocy
  • "Link to this location" now provides URL using locx+locy instead of lat+long
  • Zoom optional, instead of required, for locx+locy links (defaults to umMapLinkZoomedValue for all URLs where other params are provided)

v0.3.0, 23 June 2008

• Merged code for all three UESP google API maps: Oblivion, Shivering Isles, and Morrowind.
  • Note that the new version of the code is only available for the moment at the above links (which for Oblivion and Shivering Isles are not the same as the links used everywhere else on the site). When the new versions are confirmed to be working (within a few days), the old URLs will be forwarded to the new URLs, and the wiki map links will be updated.
  • This also means that although the map editors for all three maps are now functional, any edits made to the Oblivion map at User:Daveh/Map will not take effect on the pre-existing Oblivion map (changes made at User:Daveh/SiMap partially take effect on the pre-existing SI map).
• Fixed problem with Safari browser failing to view map
• Introduced alphabetization to the location list
• Introduced ability to customize label positions relative to map markers, and changed default position to middle right
• Added SearchTags parameter
• Added new categories for map markers (although new icons needed for most categories)

v0.2.1, 28 August 2006

• Added 263 Nirnroot locations (display level 15, marker type 13)

v0.2.0, 23 August 2006

• Added the centeron parameter to the map page which allows one to quickly zoom to a named location.
• Added info popup windows when a location is clicked.
• Added the WikiPage field to locations. Initial values for all locations have been set to the location name for all locations that have a matching wiki page (about half of them).

v0.1.1, 12 August 2006

• Added smarter location loading handling. Loaded locations now stay on the map area unless they go outside the current visible area. This reduces the number of locations that have to be loaded each time the map is panned or zoomed.
• Added 'divs' for the search results header and footer.

v0.1.0, 11 August 2006

• First release

Data Updates

• 13 January 2008: general cleanup of SI map markers --Nephele^Talk
  • Disabled all map markers for "Dementia Garden," "Mania Garden," and ordered Fringe worlds.
  • Disabled extra markers for Aichan Prison and Corpserot Prison (i.e., the teleportation markers that never appear on player's map).
  • Set DisplayLevel = 17 for all extra doors on houses.
  • Fixed most (if not all) wiki page links to now point to valid wiki pages.
  • Adjusted DisplayLevel for many other markers to make marker distribution more uniform and less cluttered at most zoom levels (most doors moved to zoom=16; settlements moved to zoom=13).
• 13 January 2008: updated all Fringe and New Sheoth tiles for the SI map. Now using using higher resolution images from individual worlds, as created by Timenn --Nephele^Talk
• 11 January 2008: added all exterior Obelisks of Order and Crystal Chests to SI map --Nephele^Talk

General Map

The original version of the OB map was specifically created for OB. As of version 0.3, the map code has been generalized so that it can be reused for all desired maps. Specifications for the Morrowind map are provided at Morrowind Map Design.

Isolating Game Specifics

The following items are specific to the map being displayed:

• Tiles -- The simplest method would simply override the custom map tile function used by the Google maps API.
• Maps Bounds -- Both in game cells/coordinates, zoom levels, and in map latitude/longitudes.
• Coordinate Conversion -- Converting to/from game coordinates to map latitude/longitudes.
• Location Database -- Each map will likely use a different database/table of locations and which may possibly contain varied data.
• Images -- Including 'null' images and map icons.
• Marker Types -- Types of map locations will change from game to game.
• Link to Wiki -- Default namespace and options to override default namespace.
• Map Labels -- May require updating to display game specific data.

Oblivion Map Specifications

Marker Types

ID	Type	Icon
0	None
1	Camp
2	Cave
3	City
4	Ayleid Ruin
5	Fort Ruin
6	Mine
7	Landmark
9	Settlement
10	Daedric Shrine
11	Oblivion Gate
20	Ayleid Well
21	Doom Stone
22	Rune Stone
23	Wayshrine
30	Nirnroot
31	Treasure
40	Door
41	House
42	Shop
43	Chapel
44	Tavern
45	Guild
46	Castle
100 (default)	Other

Shivering Isles Map Specifications

Worldspaces

The following worldspaces are to be considered as exterior in SI:

• Bliss
• Crucible
• Dementia Garden
• Mania Garden
• Palace Grounds
• Realm of Sheogorath
• The Fringe

Marker Types

ID	Type	Icon
0	None
1	Camp
2	Cave
3	City
4	Door†
5	Ruin
7	Landmark
9	Settlement
25	Obelisk of Order
31	Treasure
32	Crystal Chest
40	Door
41	House
42	Shop
43	Chapel
44	Tavern
46	Castle
100 (default)	Other

† This icon is only used on the in-game Shivering Isles map for the ordered version of the Door to Cyrodiil. Although the in-game icon is probably a bug, might as well make it available for that one icon if desired.

ToDo

• Copyright license
• Proper marker types for all locations
• More marker types? (guilds, store types)
• Distinguish between door and map markers (eliminate map markers?)

Other Stuff

• Initial Notes -- Some initial notes on the Google maps API
• ConvertNames.py -- Python script to convert CS exported filenames to one used by the online map system.
• CreateZoomLevels.py -- Python script to create all zoom levels from the base export.
• ResizeObMap.pspScript -- PaintShopPro v8.0 macro for resizing the base layer map tiles to the proper size.
• getmaplocs.php -- Full script for retrieving the map locations from the database.
• Database Table -- Current MySQL map location table structure.