Changes to CONCEPT to support Spatial Modeling

If not already done, PROJ4, GEOS, and PostGIS must be installed. PROJ4 and GEOS must be installed prior to installing PostGIS. If PostGIS is already installed and PROJ4 or GEOS is not installed, you will have to recompile PostGIS after installing the PROJ4 and GEOS packages. PROJ4 can be downloaded from www.remotesensing.org/proj. GEOS can be downloaded from geos.refractions.net. PostGIS can be downloaded from postgis.refractions.net/download.php. If you need to recompile PostGIS, make sure that the PostGIS “Makefile” has USE_GEOS=1, USE_PROJ=1, and USE_STATS=1.

Once the additional spatial software is installed, you will need to initialize your data base to support spatial modeling:

% cd <POSTGRES_INSTALL_HOME>/contrib/<POSTGIS_INSTALL_HOME>

% psql –f postgis.sql <PROJECT>

% psql –f spatial_ref_sys.sql <PROJECT>

Note, each time you initialize or reinitialize <PROJECT> using the CONCEPT control script (i.e., concept/concept), you will need to run the ‘psql’ commands to load PostGIS functionality.

The CONCEPT execution scripts will initialize the POSTGIS when a new project is created, but the environment variable POSTGIS_DIR must be set prior to concept execution:

setenv POSTGIS_DIR <POSTGRES_INSTALL_HOME>/contrib/<POSTGIS_INSTALL_HOME>

Prior to running the spatial components

Prior to running any of the spatial modeling capabilities, you will need to download the spatial coverages that are used to prepare area source spatial surrogates. Some of the coverages are also used for QA purposes by the CONCEPT Point Source Model. These files can be downloaded via anonymous ftp at ftp.epa.gov in the directories EmisInventory/emiss_shp2003/canada (~550 MB) and EmisInventory/emiss_shp2003/us (~5.6 GB). The data from the EPA directories should can be stored anywhere though in my case, I put the data in the directories $CONCEPT_HOME/../concept_projects/<project>/spatial/us and $CONCEPT_HOME/../concept_projects/<project>/spatial/canada. Regardless of where you put the data, you will need to modify the file $CONCEPT_HOME/../concept_projects/<project>/<scenario>/spatial_coverages.txt so that the directory structure conforms to your installation. You will also need to modify $CONCEPT_HOME/../concept_projects/<project>/<scenario>/run_control.txt to point to spatial_coverages.txt for your installation. Of note, when the spatial surrogates processing code is released, you will receive a new version of spatial_coverages.txt.

You should also note that I have included three shapefiles that EPA has not posted to its ftp site: us_tribalbnds; statesp020; and countyp020. These shapefiles are in the directory $CONCEPT_HOME/../concept_projects/<project>/spatial/us. us_tribalbnds contains all tribal areas that are greater than one square mile for the 50 states. statesp020 contains the political boundaries of the 50 states. countyp020 contains the political boundaries of all the counties in the 50 states.

Description of SPATIAL_COVERAGES.TXT

This files contains the names of the shapefiles that will be imported into CONCEPT for use in developing spatial surrogates. This file is an ASCII, colon (i.e., “:”) delimited file and has the following format:

Field 1 – fully qualified location and name of shapefile to import (text);

Field 2 – arbitrary and unique internal name used by CONCEPT (text);

Field 3 – fully qualified location and name of the map projection file for the shapefile (text) (if this is a number, then it is the map projection SRID that exists in PUBLIC.SPATIAL_REF_SYS);

Field 4 – country_code identifier (text) (this can be a field name in the shapefile; SUBSTR(<fieldname>,ss,2) where <fieldname> is a field from the shapefile which contains the country_code and ss identifies the start position of the country_code; DEFAULT=’??’ where ?? identifies the default country code [e.g., US]; OVERLAY=<shapefile> where <shapefile> is a unique internal name of a shapefile that will be overlaid to obtain the country_code; or MISSING which states that the current shapefile is missing the country_code [currently not implemented]);

Field 5 – state_county_fips identifier (text) (this can be a field name in the shapefile; SUBSTR(<fieldname>,ss,5) where <fieldname> is a field from the shapefile which contains the state_county_fips and ss identifies the start position of the state_county_fips; DEFAULT=’??’ where ?? identifies the default state_county_fips [e.g., 21002]; OVERLAY=<shapefile> where <shapefile> is a unique internal name of a shapefile that will be overlaid to obtain the state_county_fips; or MISSING which states that the current shapefile is missing the state_county_fips [currently not implemented]);

Field 6 – tribal_code identifier (text) (this can be a field name in the shapefile; SUBSTR(<fieldname>,ss,5) where <fieldname> is a field from the shapefile which contains the tribal_code and ss identifies the start position of the tribal_code; DEFAULT=’??’ where ?? identifies the default tribal_code [e.g., 0083]; OVERLAY=<shapefile> where <shapefile> is a unique internal name of a shapefile that will be overlaid to obtain the tribal_code; or MISSING which states that the current shapefile is missing the tribal_code currently not implemented]);

Field 7 – if YES, this shapefile is used to QA point sources against the FIPS state and county code.

Running CONCEPT Grid Definition Model

In order to take advantage of the CONCEPT spatial capabilities, you must run the CONCEPT Grid Definition Model and import spatial coverages before you can run the CONCEPT Area Source Model or the CONCEPT Point Source Model. In order to run the CONCEPT Grid Definition Model, you need to execute the following (this assumes that the run control file has been updated to include the grid definition parameters and that the run control parameters have been read [i.e., concept import_control … has been run]):

% concept run_grid_definition_model –n <project> -s <scenario> -d <directory>

The run control text that has been provided contains the grid definition parameters for the National 36 km RPO grid except that the grid cells are at a 4 km resolution (aye, this makes for a large grid). It should take the CONCEPT grid definition model no more than 10 minutes to build the grid structure.

Running Spatial Component of the CONCEPT Point Source Model

With the modifications included herein, the CONCEPT Point Source Model is currently configured to automatically run the spatial components necessary for the point source modeling. Hence, the user runs the CONCEPT Point Source Model as he/she has done so before this release.

Specific Changes to CONCEPT code

The following text provides a brief description of the modifications that were made to existing code to support spatial modeling.

concept/concept

· Added creation of the SPATIAL schema

· Added gis_check_projection.sql

· Consider changing all variable=$(…) to variable=`…` (I think that this is a more standard shell execution scheme)

· Added RUN_GRID_DEFINITION_MODEL execution stream

· Added IMPORT_SPATIAL_COVERAGES execution stream

· Added CREATE_SPATIAL_SURROGATES execution stream

concept/src/import/import_sp.sh

· Added import of stored procedures necessary for spatial processing

concept/src/import/import.sh

· Dropped importing “point_grid_xref”

concept_projects/beta2/scenario1/run_control.txt

· XCellSize, YCellSize, OriginX, OriginY: INTEGER to NUMERIC fields

· Added variables: GridName (VARCHAR[20]), GridDatum (VARCHAR[20]), Grid1Lat (NUMERIC), Grid2Lat (NUMERIC), GridLonOrigin (NUMERIC), GridLatOrigin (NUMERIC)

concept/src/import/import_control.pl

· Added support to import new RUN_CONTROL.TXT variables

concept/src/db/run_control.sql

· Added support to create new RUN_CONTROL.TXT variables

· Changed GRID_PROJECTION from VARCHAR(10) to VARCHAR(20)

concept/src/db/nei_point.sql

· changed x_coordinate and y_coordinate from NUMERIC(12,2) to NUMERIC(16,6) to support more precision in LATLON

concept/src/db/rpo_xrefs.sql

· RPO_SR – added “basis” and “parent_basis” each of type NUMERIC to support tracking the actual surrogate values (e.g., population count, miles of roadway) for each surrogate (did not supply precision since the numbers could be all over the place)

· POINT_GRID_XREF – added x_coordinate and y_coordinate (native map projection units) of type NUMERIC(16,6); x_coordinate_lon and y_coordinate_lat (LATLON units) of type NUMERIC(16,6); and default of type INTEGER where 0=point okay (is on grid structure and in county), FIPS check successful; 1=is outside of emissions modeling grid; 2=failed FIPS check (is on grid but outside of county); 3=FIPS check not possible to make

concept/src/point/point_model.sql

· Added call to PointLocalGridCells() to find emissions modeling grid coordinates of the emissions release points