calculate river slope from coarse DEM

Here is one way to calculate the slope of a stream (and long profile) from a coarse DEM.

(1) If the DEM is huge, clip it to only the area of interest, to speed up the processing. You can do this by creating a polygon outlining the area, adding it to the map, and then using extract by mask (Spatial Analyst, Extraction, Extract by Mask).

(2) It may help to visualize things by making a Hillshade.

(3) Under Spatial Analyst, Hydrology, there are three steps.

(a) Fill

(b) Flow direction

(c) Flow accumulation

(4) Now you want to extract the stream by choosing an upstream point, then selecting all points with a flow accumulation greater than that value, using the conditional to turn the map to 0s and 1s to delineate the stream, and then turn that stream into a vector.

Map algebra, Single Output Map
Conditional

Hydrology
Stream to feature (may be optional and affect choice in Step 5)

(5) Select equally spaced points along this line either by using Hawth’s tools / Geospatial Modeling Environment or this previous post.

(6) Extract Values to Points, Export data

(7) You can open the .dbf in Excel or Gnumeric to inspect the slope.

There’s also this:
Tarboton, D. G., R. L. Bras, and I. Rodriguez-Iturbe. 1991. On the Extraction of Channel Networks from Digital Elevation Data. Hydrological Processes. 5: 81-100.

Clip a Raster to a polygon - GIS

Make a new polygon with the shape you want.
Toolbox:
Data Management Tools>Raster>Raster Processing>Clip
Your raster in the "input raster"
Your polygon is the Output Extent
(check the "Use Input Features for Clipping Geometry (Optional)" box)

Create points from a line (ArcGIS)

I need two text files, one with the x coordinates and another with the y coordinates of a stream (to put into matlab). I have a polyline shapefile of the stream in ArcGIS. Don't know if this is the easiest way, but it works :)

1. In ArcCatalog, make a new point file with the same coordinate system as your line file.
2. In ArcGIS, start editing your POINT file, and select the line you want to make poins for (so that it is highlighted green).
3. Under the editor pull down (in the editor tool bar) select to 'divide'.
Choose the second option: 'place points separated by every __ units' (the units are your map units... you can check this under dataframe properties).
4. Stop editing.
5. Open the 'Add XY Coordinates' tool. (ArcToolbox>Data Management Tools>Features)
Choose your point shapefile and say OK.
6. Once it's complete, open the attribute table for your point shapefile.
7. Under attribute table options, choose 'export' and save your file as a .dbf file.
8. Open this file in excel (excel is the default program to open .dbf files anyway).
9. For my matlab application I need two files, one with just x coordinates, and one with just y coordinates, so I made two copies of this file and deleted all but the POINT_X column in one and all but the POINT_Y column in the other. Then for each I deleted the title row, changed it to scientific notation and saved the file as a text(ms dos) file.
10. Wahoo! maybe there is an easier way to do this, but it works :)

mapping software alternatives

Up until last week, I didn't know about any viable alternatives to arcGIS.

But here are a couple (one still under development...)

GeoMapApp

http://www.geomapapp.org/

Open Earth Framework

http://oef.geongrid.org/

Both are being developed at academic institutions, the first at Lamont-Doherty and the second at UCSD.

I haven't used them very much yet, but I hope to check them out more in the near future. 

Quaternary faults, reproject

Quaternary Faults in California, maintained by California Geological Survey:
http://www.consrv.ca.gov/cgs/information/publications/Pages/QuaternaryFaults_ver2.aspx

(1) Download .e00 files
(2) Import from Interchange File

(3) Data Management Tools -> Projections and Transformations -> Feature -> Project
Fill in:
Input Dataset
Input Coordinate System (automatic)
Output Dataset (put what you want, specify folder you want)
Output Coordinate System (for me: NAD_1983_UTM_Zone_10N)
Geographic Transformation (for me: (guessed) NAD_1927_To_NAD_1983_NADCON)

Aside:
wth is NADCON
http://www.ngs.noaa.gov/TOOLS/Nadcon/Nadcon.shtml

dealing with .e00 files v.2

Converting .e00 files. An easier way than here:

In ArcCATALOG

View -> Toolbars -> ArcView 8x Tools -> Conversion Tools
Import from Interchange File

data frames, neatlines, graticules

In ArcMap sometimes you want to add lat/lon ticks around your map, add a scale bar, etc.

This stuff is all filed under Page Layout and Map Composition. You see most of this stuff under the Layout View, not the Data View. (Buttons in lower left corner of map window toggle these.)

data frame: most fundamental element in a map document and in ArcMap user interface.

neatline: border line commonly drawn around the extent of a map to enclose the map, keeping all o the information pertaining to that map in one "neat" box

graticule: the grid of intersecting lines, esp. of latitude and longitude on which a map is drawn. vs. gridicule.

Accessing the Grids and Graticules wizard:

1. Click the View menu and click Data Frame Properties.
2. Click the Grids tab.
3. Click the New Grid button.

Adding grids and graticules (reference systems)

Working with data frames
Adding data frames
Customizing data frames (you can rotate them)
How to do inset maps (extent rectangles)

spatial hydrology data

I've been trying to download river spatial data (for arc) for a while now, and finally have come up with some sort of resolution. But the problem seems to be information overload. I still haven't figured out if there's a way to, say, query "Truckee River" and have the trace of only that river returned.

Instead, here are some convoluted ways to get data of Quads or defined map areas.

(1) USGS maintains the National Hydrography Dataset.

(2) The Data Availability page describes ways to obtain the data.

(3) It appears that USGS is in the middle of transitioning to a new map viewer/data downloader interface.

New one: http://viewer.nationalmap.gov/viewer/

It is easy to download hydrology data by quadrangle, if you click on the map, it tells you the quadrangle and the available data. (Using the download data tool.)

(4) One way to figure out all of the quadrangles covered by a particular river is to use the Geographic Names Information System.

(5) The page for the river will list each quadrangle, which you can then locate in the Map Viewer.

(6) You send in a request for a data and it may take several hours to be returned. (Still waiting for some I ordered a couple hours ago.) It is not very efficient.

(7) Another way is to download pre-defined subregions. These datasets are ~250 MB zipped and ~750 MB unzipped. It is information overkill, but you can also download it right away. The old map viewer seems to be better in identifying the 4-digit identifier (first 4 of the 8 digits).

FTP for predefined regions:
ftp://nhdftp.usgs.gov/SubRegions/

Only works in Internet Explorer! Not Firefox or Chrome! Also has a README in a .doc format! gah.

I'm not sure how much closer this gets me to the trace of a single river, but possibly this information will be useful to someone somewhere.

Here's their tutorial.
http://nhd.usgs.gov/tutorials.html

Note:

The last section of the tutorial, on Navigating, actually lays out how to save the trace of a river. First you need to click on Analysis in the Utility Network Analyst toolbar, and click Options, Results, and check Selection.

Then "perform navigation," set an Edge Flag at the source of the river (this is located in the Geographic Names Information System entry, or at least the correct quadrangle). Then use the Utility Network Analyst to navigate to all points downstream.

You will be able to save it to the map and export the data as a shapefile.

Read the last page of the tutorial linked above.

Export Table to ASCII

Under the Spatial Statistics Tools, under Utilities, there is a tool called Export Feature Attribute to ASCII.

This tool will write out a .dbf Attribute Table to ASCII, thereby negating my previous bitching about it. You can choose which columns to write out, and you can choose the delimiter.

Great, Way to make me feel dumb, arc. I can still complain that this tool is hidden where it is - why not just have a button for Export .dbf, or Save As? Why?

Oh well, now we know.

refreshing and labeling

In ArcCatalog, if you've just created a new layer in ArcMap, it may not appear until you refresh the list. To refresh the list, use F5.

Labels

You can label features (ugh, an arc word...) with their attributes (ugh, another arc word). There is a toolbar for it.

You might need to be in an open Edit Session for this to work.

View -> Toolbars -> Labeling

The first icon is the Label Manager.

You can label with more than one attribute field. Click on the Expression button under Text String. You can Append more than one field name in the label. (e.g. you can label a fluvial terrace with both its unique id number and also the terrace grouping Qt1, Qt2, etc.)

Edit the size and color and etc. to your liking.

The end.

Text in ArcGIS is nontrivial (labels vs. annotation vs. graphic text), but labeling is relatively straight-forward.

Overview of working with text

linear referencing and hatch marks

Linear referencing: point location along the line as an alternative to expressing the location using an xy coordinate. Useful, for example, for measuring distance along a stream.

An Overview of Linear Referencing

Here are steps outlining a way to put hatch marks (with distance labels) on a polyline (e.g. following a stream).

1. When you create the polyline shapefile in arc catalog, you need to check the box for "Coordinates will contain M values. Used to store route data."

2. Make the polyline (start edit session, click click click, double click to end, stop edit session)

3. In the arc toolbox -> linear referencing tools -> create routes  [routes?]

a. Input line features: the polyline layer
b. Route identifier field: (you can use id or if you created another field like "route." If there's only one line, that's ok.)
c.  Output route feature class (it fills in a default)
d.  Coordinate Priority: where is the start of the polyline? In my case, lower_left.
e.  click OK - this will create another later eg. channel_CreateRoutes

4. I'm still fuzzy on why you need an entirely new layer, but anyway... it is on this new layer that you can specify hatch marks.

This page nicely outlines the 19 (!!!) steps necessary to display hatch marks on that layer:

Displaying hatches

1. Right-click the layer in the table of contents that you want to hatch and click Properties.
2. Click the Hatches tab.
3. Check Hatch features in this layer.
4. Type an appropriate hatch interval.
5. Click Hatch Def(1).
6. The Hatch Definition view becomes active.
7. Type an appropriate line length.
8. Right-click the hatch class and click Add Hatch Definition.
9. The Hatch Definition view becomes available.
10. Type an appropriate multiple of the hatch interval.
11. Type an appropriate line length.
12. Check Label these hatches.
13. Right-click the hatch class and click Add End Hatch Definition.
14. The End Hatch Definition view becomes active.
15. Type an appropriate end hatch tolerance if necessary.
16. Type an appropriate line length.
17. Check Label these hatches.
18. Click OK.
19. Poke your eye out with a pen.

5. Peripherally related: It may also be useful to you to divide a line into segments: Splitting line features

6. Next up: putting something useful along the route: Event Tables

in the beginning...

Sometimes you have to start from the beginning, there can be useful information in the beginning.

LaTeX 
"haha, if you're not smart enough to figure this out, too bad for you."

http://en.wikibooks.org/wiki/LaTeX/Introduction
http://en.wikibooks.org/wiki/LaTeX/Importing_Graphics 

Arc
"haha, we don't give a fck about user experience, too bad for you."

http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=An_overview_of_the_Geodatabase

dealing with .e00 files

.e00 files are from an older version of arc. a simpler time. but now it is complicated to open them.

update 7/26/2010:
Easier way:

In ArcCATALOG

View -> Toolbars -> ArcView 8x Tools -> Conversion Tools
Import from Interchange File

**** old stuff below ***

I ended up importing some .e00 files that were in a .tar.Z file (1996 USGS Open report) in a very roundabout manner, and later found some other more straightforward procedures (but can't figure out how to get those to work with my version of Arc). I'll just post them all here.

1. Official explanation on arc site:  Importing an ArcInfo interchange file

2. Mac person's explanation:
there is a arctool box feature for importing arc/info exchange (.e00) files. Toobox-> Coverage -> Conversion -> To Coverage -> Import exchange files. I was able to unzip the tar file using the Mac Archive Utility.

3. My convoluted method on dumb PC (involving much sighing and swearing):

1. download tar.Z to PC
2. ftp to unix account
3. uncompress and untar in unix

uncompress *.Z
tar -xvf *.tar (I may have left off one of the flags to get it to work)

4. ftp back to PC
5. use Import71.exe (wtf, really) to import the .e00 files to the newer version (see below)

my [source]

Importing .e00 files using Import71.exe, ArcView 9:

a. Before you start, note that this import tool does not deal well with folder paths that have space in their names (e.g., like "My Documents"). Therefore, prior to using this tool, you should create a new folder for that file in your C: drive. E.g., if I was importing a TxDot county file for Williamson County called urban246.e00, I should create a folder called williamson in the c:\temp\ folder, and copy my urban246.e00 file to that folder.

b. Locate the Import71.exe file - it should be in the C:\arcgis\bin\ folder (or use the Windows search tool to find it)

c. Run the Import71.exe file by double-clicking on it - a dialog box should appear

d. Browse to the location of your .e00 file and select that file and its directory path as the input file (e.g., c:\temp\barbara\williamson\urban246.e00)

e. For your output file, simply specify the name you want for the newly imported file (e.g., txdot) - no directory path is needed, the file will import to the same location as the .e00 file

f. Be patient - the import process may take 10 minutes or so. You will know when it is done because a small box saying Import Complete will appear over the Import71 dialog box.

g. After you see the Import Complete notice, start ArcView again and check that the coverage displays properly. A "coverage" is an older GIS data format.

contours

Simple:

Toolbox -> Spatial Analyst Tools -> Surface -> Contour

Specify input raster, output name, contour interval.

Creating contours in spatial analyst

x, y, z table for a bunch of points

This is how I got an xyz table of a bunch of points. (There is probably a better way out there.)

1. Using arc catalog, create a shapefile of points. Then add columns to the attribute table, x, y
2. In arcmap, start an edit session and click away on the points.
3. End and save edit session, I think.
4. Right click on shapefile in Table of Contents column and open Attribute Table. All of the points should be there.
5. Right click on the x column
6. Click Calculate Geometry
7. Choose X coordinate of Point, Use coordinate system of the data source (UTM meters)
8. Repeat with x column and Y coordinate of the Point
9. Use Toolbox -> Spatial analyst Tools -> Extraction -> Extract Values From Points
Input point features (point shapefile)
Input raster (DEM)
Output point features (choose your name)
Click OK
10. This should create a new column in the table. Now you can save the table and export the .dbf if you wish, and do whatever you want with it.

Overview of Extraction tools

aspect-slope map plus other stuff

While hunting around for some guidance in producing a slope map (a matter of a few mouse clicks), I came across this page [aspect-slope map], which is useful because it gave an example of how to implement the following things:

Styles (predefined colors, symbols, map elements, etc.)
Slope map (with link to how it works)
Aspect map
Reclassify: redefine the default number and width of display bins
Map Algebra

.dbf files

ArcGIS output may be saved as a .dbf file, with no immediately apparent way to save as a .txt, or even to copy and paste the cells to a simple spreadsheet program (why? why?).

I had no idea what to do about this so I just downloaded a small program that converts the .dbf to .csv.

http://www.fileguru.com/DBF-To-CSV/download

I did this a couple days ago so now I have no recollection of how it worked, I just remember that it worked, and I copied the data into a google docs spreadsheet. Open Office won't even install on my computer, how lame is that?

If anyone else knows a more intelligent way to do this, please share.

Somebody else talks about .dbf and .xls:
http://gisatvassar.blogspot.com/2008/02/excel-dbf-files-and-arcgis-92.html

Note, 7/14/2010, This is the way to do it:
http://matlabor.blogspot.com/2010/07/export-table-to-ascii.html

Elevation statistics of traced polygons

I wanted to find the mean, min, max, stddev elevation of a bunch of traced terraces, but this can be done with any input files of a zone dataset (i.e. shapefile of polygons) and a value raster (i.e. raster of elevation values).

1. Make the zone dataset, in my case, make a shapefile of polygons [make polygon layer]

2. Add field(s) to the Attribute Table to define the zones. I have a separate ID for each polygon and also the field "Name" for the groupings Qt1, Qt2, etc. To add fields, open the shapefile in ArcCatalog and in the Fields tab, just click in the table to add the Field Name and Data Type (must be integer or text).

Then edit the field to define your zones. First add the zone dataset to the map.
Then, Editor toolbar -> Start edit session -> right-click on your shapefile layer in the leftbar -> Open Attribute Table -> click in the table and fill in.

Stop edit session and Save changes.

3. Zonal analysis by attribute
Spatial Analyst toolbar -> Zonal Statistics
Fill in stuff in the pop up window:
Zone dataset: shapefile of polygons
Zone field: ID or Name or whatever your defining field is
Value raster: elevation dataset
Check or uncheck Ignore NoData and Join output table to zone layer (I like to join)
Chart statistic: I checked Mean but got min, max, range, mean, std, sum in the output table
Specify output table

4. Click OK, the output table pops up, and you're Done.

make polygons

To make some @#$*%$@#% polygons in arcgis (e.g. trace fluvial terrace deposits):

1. Open Arc Catalog
2. In the Location bar, navigate to the folder where you want to put the polygons (shapefile)
3. File -> New -> Shapefile
4. Name it and choose Feature Type -> Polygon
5. If you want, edit the Coordinate System
Select -> Projected Coordinate Systems ->
(UTM NAD 1983 10 N for local stuff)

In ArcCatalog you can right-click a shapefile and edit Properties - e.g add Fields and specify the DataType (text, long integer, double, etc.). But I think the shapefile can't be open in ArcMap if you want to edit it.

6. Add the (empty) layer to the ArcMap window by dragging or using the AddData button
7. Start an edit session (Edit Toolbar)
8. Task dropdown menu: Create New Feature
9. Target dropdown menu: choose your shapefile - if the target drop down is empty, it might be a bug, restart ArcMap and ArcCatalog

10. Use the Sketch button (pencil icon), click and double-click to make vertices and close polygon
11. Stop edit session, Save

12. *Get mean elevation of each polygon from an elevation raster (TBD)

hillshade from seamless data

It's ArcGIS time. First, let's reflect on how Arc is the anti-google in terms of user-friendliness. It's a constant source of exasperation and I am here to document my exasperation and hopefully avoid some in the future.

Starting from the beginning, here's an example if you want to download high resolution topographic data (1/9 arc second, where 1 arc second at the equator is ~31 m)  from the USGS Seamless Server, reproject it to UTM, and then make a hillshade.

1. Download stuff from seamless - http://seamless.usgs.gov/

Click on the "view and download united states data" image [link].

a. On the right hand side of the screen:
In the Display Tab, under Layer Extents, specify NED 1/9 arc second (National Elevation Dataset). This shows where that data exists.
In the Download Tab, click 1/9 arc sec, assuming that's what you want (default is 1 arc).

b. On the left hand side of the screen, under the Downloads section, define your area:
The way I like to do it is with the 2nd button, Define download area by coordinates, because then you know exactly what the boundaries are, if you need to mosaic it later or whatever. I like to get my lat/lon coordinates from Google Earth.

c. Fill in the window that pops up and then click "Add Area", another window will pop up telling you your request is in a queue and then the file will be downloaded to your default location. If the area you specify is large (I don't know how large), it will be divided into separate files.

Note: Sometimes one or more of the files will stall during downloading, and since the naming convention is incomprehensible (an 8-digit string, seemingly random), it is hard to figure out what part you are missing. Oh well.

2. Unzip all files in the directory you want

3. If there are multiple files, make a mosaic from the separate files

In ArcMap,

a. Make sure the Spatial Analyst extension turned on (Tools -> Extensions -> Spatial Analyst)
(!$#%!#$)%!#$%& that this is not on by default)

b. Click on red tool box (or bring up the toolbar with View ->Toolbars ->Tools)

Click Data management Tools -> Raster -> Raster Dataset -> Mosaic to New Raster

if you don't specify an extension, it will default to GRID

4. Convert from lat/lon to UTM

In Data Management Tools,
Projections and Transformations -> Raster -> Project Raster -> UTM

Input: make sure to erase the default output name .img extension, and make under 13 characters (bug in GIS)

usual convention for new projection: name_prj

Select -> Projected Coordinate Systems

NAD 1983 Zone 10N (for local CA stuff)

Scroll down to Resampling technique-> change to CUBIC (the default method tends to create striping)

5. Restart ARC because it is possible that it thinks it is still in lat/lon

6. Make hillshade

In tools:
Spatial Analyst Tools -> Surface -> Hillshade

You should have a nice hillshade of the topo data.