Open Sourced - From Analysis to Communication in the Age of Web Mapping - Report Week

Special Topics - Project 4 Week 4

The final final.  This project ended with the creation of a Power Point slide presentation of the analysis we performed in previous weeks.  Creating the presentation was fairly straight-forward until I got to the audio part.  I tried several different ways to get the audio to start up automatically but I never was able to accomplish that.  After recording the whole presentation three times in different ways I finally gave up.  The rest of it was simple enough until I went into QGIS  to create a finished map.  For analysis week I didn't need a finished map from QGIS, just a map that had had the analysis done and the layers created.  It should have been an easy enough task, but every time I made one change in the layout view it would change all kinds of things.  I was very careful to switch over to the data window and turn the appropriate groups on or off before unlocking the groups in the layout view, and yet it still would change things in ways I never could figure out.  Finally I got the map to the point where I wasn't too embarrassed by it so I did a quick save and export before anything else could happen.

All in all this last project was rather interesting, but I thought it was a bit much for a final project to have so much new stuff thrown at us with very little guidance in using it.  Had each of the programs been worked into the previous lessons it wouldn't have been so bad, but at the very end it was rather frustrating.

But it's all over and done now and time to move on.

Here are the links for both my Web map and my Power Point presentation.

http://students.uwf.edu/mr80/STGIS/OkDesert.html

http://students.uwf.edu/mr80/STGIS/FoodDeserts_MRoth.pptx

Final Project

Photo Interpretation and Remote Sensing - Final

The final project for this class was to use what was learned in all the previous classes to answer a question.  I chose to go with the suggested project for which the data was provided.  The question being, Is urbanization to blame for the decrease in clarity of the waters of Lake Tahoe.  

To answer this question a composite image was created of individual bands of a 2010 Landsat 5 TM image in ArcMap, then imported into Erdas Imagine where spectral signatures were created and a supervised classification was run.  This image was then returned to ArcMap to compare to a 1992 National Land Cover Database image.  The comparison showed urbanization is likely the cause of the problem.

It's a relief to finally finish this course as it was the most difficult of them all.  I think part of the problem was I kept looking for things to be black and white, and quite frequently they were more best guess.  It was a difficult concept for me to adjust to.  In the end I think I did okay, and somewhere along the way I also came to enjoy it.  I think that was at the very end, when I knew it was almost over and I'd never likely need to do this again.  But at least now I can look back at this with fond memories.  

GIS Portfolio

GIS Internship - Final Assignment

My final assignment was to create a GIS Portfolio with a little information about me, my goals, my resume and and examples of my work.  It was a little difficult selecting what maps to use for examples since so many showed off some awesome skills, but I think the ones I selected provide a good picture of what has been learned during this past year.

Here's is a link to my portfolio:

http://students.uwf.edu/mr80/MRothPortfolio.pdf

Open Sourced - From Analysis to Communication in the Age of Web Mapping - Analyze Week 2

Special Topics - Project 4 Week 3

 I downloaded the 2010 census shapefile from FGDL, and used the Florida Counties shapefile from Prepare Week to create my county shape to clip the census shapefile to, and then created my study area and centroid shapefiles from that within QGIS.  I then used Google Earth to identify grocery stores within that study area and created a KLM file from that search and opened that in ArcMap and saved the results to a shapefile, then added my centroid shapefile.  With these two layers I then ran the Analyst tool Near to locate those stores within 1 mile of the census tract centroids and add that data to the centroid attribute table.

With the centroid layer modified in ArcMap I opened the database file in Excel and saved it as a .csv file then went back into QGIS and added the table as a Delimited Text Layer and joined it to the Study Area layer.  This added those new distance fields to the Study area layer.  From this field I was able to select those records in my Study Area that did not have a store within a 1 mile radius of the tract centroid and create a Food Desert shapefile from that.  Reversing the selection I then created a Food Oasis shapefile from the rest.

Moving on to MapBox I uploaded my zipped Food Desert and Grocery Store shapefiles to new Tilesets, then created a basemap from the basic style, changed a few elements, then added the tilesets to the basemap.  After that I added the Study Area to ArcMap and investigated the results of different classes and classification methods and finally settled on Jenks Natural Breaks with 4 classes as the most reasonable distribution, then switched to Colorbrewer for the HEX and RGB color codes for the color scheme I wanted to use.  Back in MapBox I created a Group for the food desert layer and made three duplicates of the layer to represent the four classes, then added filters and changed colors so each layer would represent the correct class.

The last step was to create a webmap in Leaflet.  To do this I copied the text file from Analyze Week 1 and made the adjustments in the text file that were appropriate for the differences between maps.  Basically, all the things that needed to be added or adjusted for Analyze Week 1 also needed to be adjusted for Analyze Week 2. With that done I save the text file as a HTML on my I drive.  This is the link for it:  http://students.uwf.edu/mr80/STGIS/OkDesert.html

The Study Area encompasses the Greater Fort Walton Beach area in South Okaloosa County, Florida, including the surrounding rural area.  Within this area are 10 grocery stores that service most of the urban area, but not all, and there is not one grocery store within the rural area.

Fort Walton Beach is not my home town but I am pretty familiar with it and was actually surprised to see so little of it falls within a food desert.  I didn’t realize there were so many grocery stores within town.  I was also surprised to discover there were no grocery stores at all within the rural area outside of town.  I would have expected there to be at least one. 

Open Sourced - From Analysis to Communication in the Age of Web Mapping - Analyze Week 1

Special Topics - Project 4 Week 2

Analyze Week 1 introduced us to Mapbox, Leaflet and web mapping.  Using the data we created last week in QGIS, this week we uploaded our food desert and grocery store shapefiles to Mapbox, then symbolized it.  This was a little more complicated than in ArcMap.  In Mapbox, first copies needed to be made of the food desert layer for every class we had, then the class and color needed to be assigned to each layer.  There were a few ways this data could be derived.  I chose to use ArcMap and experiment a bit with the classes in the layer symbology window until I felt I had a reasonable distribution of the classes, then I made a note of those ranges.  Then I went back to Mapbox and added that many layers.  Next I went to the Color Brewer and selected the color scheme I liked best and specified the number of classes.  I made note of the HEX and RGB codes for those colors.  Back in Mapbox I started with the lowest copy of the food desert layer and in the Select data window associated that layer with the lowest range of data based on the POP2000 field of the attribute table for all the layers, then went back through all the layers and in the Style window used the RGB codes noted from the Color Brewer to assign the correct symbol color to each of the layers.

Next we moved to Leaflet to create a web map to be hosted on our UWF I Drive.  We started with the source of the Leaflet map as a base template, then adjusted the code to meet our requirements.  This included changing the location of the style sheet and script files, changing the center point so the map would open centered on Pensacola, FL, editing pop-ups, polygons and circles, adding a legend and a geocoder to make the map searchable.  Adding the legend required us to add a long section of code that was provided, then changing the population ranges and the HEX codes we had noted earlier.  I also adjusted the size of the map view and scale so almost my whole study area would show when the map opened.  This file was then saved to the I Drive as an html file.

It was an interesting assignment, but a lot of new stuff to deal with for a final project.  Here is the link to the web map:

Lower Escambia County, FL Food Desert

GIS Day Around the Kitchen Table

GIS Internship - Week 13

This year on GIS Day I was in New Mexico visiting my daughter so I decided to make my GIS day an opportunity to share GIS with her.  My daughter is a park ranger and has been looking for openings at parks on the east coast, preferably in Virginia because she'd like to go back to school and get her Masters in History, focusing on Constitutional History.  One of the things she has said over and over is that she'd like to be close enough to make trips up to DC so she can go to the Library of Congress to do research. So of course the very first thing I did was share a fellow classmate's post about her GIS Day visit to the Library of Congress.  My daughter was very jealous and very impressed.  She was also surprised to hear they had material from the medieval age since that preceded not only the founding of this country, but also its discovery. So she is very anxious now to discover what that material is and why it was included in the library's collection.

Once I got her to quit drooling over the library I showed her the Residential Location Study I did for my final project for Applications in GIS.  She got a kick out of seeing I had done it for a mother and daughter moving to St. Augustine, a place I've told her many times I'd like to go visit with her, and the mother would be working at the county GIS office and the daughter as a park ranger at Castillo de San Marcos National Monument.  I summarized the process of creating the location study and we talked about the different criteria we could use to help her chose a location to start looking for houses once she finds a job back east.

Last we used google maps to plan her drive from New Mexico to Florida, selecting good stopping points along the way and investigating the hotels in those areas where she can spend the night with her two furry babies when she drives out to spend Christmas with me.

We had a good time and several giggles, and she said she thought she understood much better now what I've been studying, but it really wasn't anything to take pictures of.

Open Sourced - From Analysis to Communication in the Age of Web Mapping - Prepare Week

Topics - Project 4 Week 1

The first week of our final project was spent learning Quantum GIS (QGIS) and about Food Deserts.  Part A taught how to use QGIS by adding layers, setting a coordinate reference system, clipping layers, and grouping layers in the main window.  Grouping the layers allows for multiple frames in the Print Composer, QGIS' Layout View.  Just as in ArcMap's Layout View, Print Composer is where map elements, such as legends, scale bars, etc., are added.  Unlike ArcMap, the data frames in QGIS must be locked if they are not being manipulated or what is done in one frame will affect what is in another.  This was less of a problem in Part A than it was in Part B.

In Part B the layers were grouped by the results of our analyses.  Using the census tracts of Escambia County a study area was created, then created centroids for those polygons.  Using the Join tool in the

Study Area layer a join with the Near.csv file was performed to add the NEAR_DIST field from that table to the Study area attribute table.  From this field we were able to create selection sets for those census groups that were within and those that were without Food Deserts.  Statistics were done on each of these layers to determine the percent of the population that fell into each category.  With the statistics done the layers were duplicated where necessary and grouped for their different data frames.  In this second map there were more layers to contend with and forgetting to turn layers on and off  in the main window and lock and unlock the data frames in the Print Composer window resulted in a lot of repetitive steps.  But in the end it all came out right.  Except I still haven't figured out how to put a neatline around the entire map.

Supervised Classification

Photo Interpretation and Remote Sensing - Mod 10

This week we created a land use map of Germantown, Maryland using Supervised Classification tools in Erdas.  Training sites were created using a combination of polygons and the Growing Properties tool.  With these tools, features were selected and defined so an algorithm would know what class to assign to each pixel.  We started by defining 14 classes, then merged them down to 8 using the Recode tool.    Once that was done a Class Name column was added and the classes renamed, then an area column was added to the attribute table to calculate the area covered by each class.  To finish the map off it was added to ArcMap and symbolized appropriately.  

Statistical Analysis with GIS - Analyze Week

Special Topics - Project 3 Weeks 2 & 3

For the analyze portion of this project we ran numerous iterations of the Ordinary Least Squares (OLS) regression starting with 29 independent variables and removing one with each iteration.  To

determine which variable to remove the first three of six checks were performed in combination.  Each variable was tested for Probability, Value Inflation Factor (VIF), Coefficient, and Importance.  If the answer to all these questions were "No" the OLS was repeated with that variable removed.  If just one question returned a "Yes" the variable remained.  This process was to be repeated until either all the variable that failed the test were removed or the Adjusted R-Squared value was as high as it could get.  The Adjusted R-Squared value was expected to come in between 0.0-1.0, but during this process mine seldom came close.

Once all the variables were removed the next step was to check the Jarque-Bera Statistic score to check for bias.  If the p-value was less than 0.05 and has an asterisk next to it the model is biased.  To analyze the data to find the skewed results a Scatterplot Matrix graph was created.  This graph also included a histogram of each variable provided a second way to view the data.  For each variable that was skewed the OLS was run again with that variable removed and the Jarque-Bera score checked for improvement.

Check 5 was the first time we viewed the map for results.  A part of the OLS process is the creation of a layer of Standard Residual values.  The Standard Residual categorized the residual values making them comparable between different models.  The residual is the difference between the density of meth labs the model predicted would be in a census tract and the density that actually exists.

Check 6 was to see how well the model was predicting the dependent variable.  It seemed okay to me.

Unsupervised Classification

Photo Interpretation & Remote Sensing - Mod 9

This week's less in Unsupervised Classification was pretty straightforward and simple, though somewhat time consuming.  Most of the work was done in Erdas Imagine starting with running the Unsupervised Classification tool on a high resolution aerial photograph of the UWF campus.  This

resulted in a thematic raster that allowed us to simplify the image into fewer classes by selecting the pixels and changing them.  Our task was originally to classify the image into four categories; Trees, Grass, Building/Roads, and Shadows.  This seemed simple enough until some of the clusters affected multiple features.  In order to deal with this problem a Mixed class was added as well and anything that affected more than one feature could be added to that.  Sometimes the affect on a second feature was so limited it made more sense to keep that class as the first feature.  Once we had the image classified we merged the classes so we had only the five specified.

Next we added fields to the attribute table for Class Name and Area.  This allowed us to calculate the area for each of the classes so we could determine the percentage of permeable verses impermeable surfaces.  Permeable surfaces include Grass and Trees, impermeable, Buildings\Roads.  The other two classes included a combination of both, so before we could make that calculation we first had to calculate what percentage of each of those classes were made of each surface.

Thermal & Multispectral Analysis

Photo Interpretation & Remote Sensing - Mod 8

This week's assignment was to use image manipulation and interpretation techniques to identify a feature from an image using the thermal infrared band as part of the analysis.  The selection was made in Erdas Imagine using the TM Thermal Infrared Composite band combination of red for the thermal layer Band 6, green for Band 3 and blue for Band 2.  In ArcMap the same image was symbolized with a composite combination of red for Band 1, green for Band 2 and blue for Band 3 for a more real world image, which made the feature clearer and more easily identified as an airport.  The coordinate for this feature are 30° 28' 34.8306' N, 86° 31' 7.3224' W and allowed it to be identified as Destin - Ft. Walton Beach Airport (VPS) in northwest Florida.

Statistical Analysis with ArcGIS - Prepare Week

Special Topics - Project 3 Week 1

The week starts off with a new project:  Statistical Analysis of Methamphetamine Laboratory Busts in West Virginia.  For Prepare Week we had to do a bit of reading up on Methamphetamine, it's

history and its users.  We also created a basemap to use in upcoming weeks to display and report on the analysis of the data we were supplied or downloaded in order to assist the government and law enforcement in anticipating future criminal activity.

The study area of census tracts for Kanawa and Putnum Counties was provided along with a point file of Charleston Meth Labs.  The cities, roads, river, and counties of West Virginia were downloaded from the US Census Bureau's Tiger Shapefile site.

In preparation for the analysis process to occur next week a spatial join was performed on the census tracts and meth lab layers in order to combine the attributes of both tables into one.  Once the attribute tables were combined the unnecessary fields were turned off and this is what we were left with:

Some of the attribute names were a little difficult to interpret.  Best guess is sometimes the best you can do.  It will be interesting to see what we do with them next week.

Image Preprocessing 2: Spectral Enhancement and Band Indices

Photo Interpretation and Remote Sensing - Mod 7

This week was very challenging.  We worked with histograms a lot this week learning how to analyze them in order to interpret images.  Our final exercise was to identify certain areas on the map based on histogram information.  That was very challenging, but by the end I had a little better understanding of this week's lesson.  Here are the maps created:

Mountain Top Removal - Report Week

Special Topics - Project 2 Week 3 & 4

This week was the culmination of all the prep work and analysis performed in previous weeks.  We had to publish our group Mountain Top Removal (MTR) Analysis map on ArcGIS Online UWF Org, complete our Story Map Journal and add our published map to that, along with a link to this blog.

Mountain Top Removal (MTR) is a method of mining for coal that destroys a mountaintop or ridgeline.  All too frequently the plans described in work permits are significantly different from the actually mining activities that occur on the ground.    Remotely sensed data has been used to investigate evidence of human caused changes to landscape as a result of MTR and to compare what was permitted to what has actually taken place.  The data and methods were provided by SkyTruth, a non-profit agency monitoring MTR in the Appalachian Coal Mining Regions within the states of Tennessee, Kentucky, Ohio, Virginia and West Virginia.  This dataset was created by Group 3 of the University of West Florida’s (UWF) Online GIS Certification Program 2016 class and will be shared with SkyTruth for a comparison study.  Group 3 students include; Rachel Hamaty, Maggie Roth, Charmaine Hingada and Austin Adkison.

Satellite imaging was chosen as an independent and cost-effective method of identifying, mapping and quantifying landscapes disrupted and altered by MTR.  2010 Landsat data was used to create this dataset of polygons covering those areas of MTR in eastern Kentucky.  It is a compilation of Landsat images LT50190332010243EDC00 and LT50190342010243EDC00.  The combined accuracy is 97%, with a total acreage of 131,144 acres.  This dataset contains only those areas 40 acres or larger.  Areas within 50 meters of roads and rivers or within 400 meters of major rivers and highways have been removed.  With those exceptions, only those areas that intersect with mountain ridges have been included.

The Story Map Journal is basically a compilation of slides with a column on the side for descriptive text.  My Story Map Journal, A Journal of Mountain Top Removal, starts with an introduction of MTR, then background of MTR and the role of GIS in defining and analyzing it, an image of the study area my group was responsible for, an image of our analysis that we published though ArcGIS Online, and finally a discussion through a link to this blog.  Below is the link to my Story Map Journal:

http://arcg.is/2daNCgz

Image Processing 1: Spatial Enhancement and Radiometric Correction

Photo Interpretation and Remote Sensing - Mod 6

This week we used radiometric and spatial enhancements to enhance an image and reduce striping.  

The first step was to perform a Fourier transformation in order to run some of the Fourier tools in a Fourier Transform Editor.  Prior to running this step the image was just a big white blur with a few black splotches until it was zoomed to 1:150000.  After running the Fourier Transform Editor tools it was a complete image that could be zoomed to its extent, but it still had striping.  I tested numerous tools trying to find that right combination that would lessen the stripes without diminishing the clarity of the image but nothing I tried work.  Finally I settle on an image that still had all the stripes it came out of the Fourier Transform Editor with, but the clearest image I had managed to achieve.  The tools I used to accomplish this after the Fourier Transform Editor were the Convolution tools Sharpen and Haze Reduction in ERDAS Imagine and in ArcMap I used the Spatial Analyst Focal Statistics tool with a width and height of 3 and a Statistics type of Range.

Mountain Top Removal - Analyze Week

Special Topics - Project 2 Week 2

This week we used 2010 Landsat data to create a polygon of current Mountain Top Removal (MTR) areas in the Appalachian Coal Regions of West Virginia and surrounding states.  The first step was to create a single raster dataset from 7 Landsat bands using the Data Management Raster Processing tool Composite Bands, saving it as an .img file.  The Spatial Analyst Extraction tool Extract by Mask was used to clip the composite raster to the study area, again saving to an .img file to use in ERDAS Imagine.

In ERDAS, an Unsupervised Classification was performed on the clipped image with a number of 50 classes.  Areas of MTR were then identified by picking a pixel on this classified image and changing the Class designation for that pixel in Class_Name field in the attribute table to MTR, and the color to Red.  Once all the MTR areas were identified and changed the rest of the pixels were assigned a class designation of NonMTR and the color changed to Dark Green.

This saved image was then added to ArcMap and reclassified with the Spatial Analyst Reclassify tool by populating the New Values field with a 1 for the MTR classes and left blank for the NonMTR classes and the missing values assigned to No Data, creating a new raster of polygons around the MTR areas.

Intro to Electromagnetic Radiation (EMR)

Photo Interpretation & Remote Sensing - Mod 5a

This lesson had us work in ERDAS IMAGINE for the first time which was rather different.  We brought a raster image in and selected an area of Washington State to export to ArcMap using the Inquire Box and created a subset image.  After running the process we moved to ArcMap to manipulate the image properties and legend to display the information appropriately.  Below is the result.

Mountain Top Removal - Prepare Week

Special Topics - Project 2 Week 1

Mountain Top Removal (MTR) is a method of mining for coal that destroys a mountaintop or ridgeline.  This project will explore MTR in the Appalachian Coal Region of West Virginia using data and methods provided by SkyTruth.org to investigate evidence of human-caused changes to landscape cause by MTR using remotely sensed data.  Since this is such a large area to cover, the project was divided into four groups.  I am part of Group 3.

Starting with four Digital Elevation Models (DEMs) that were merged using the Raster to New Mosaic tool and clipped to the Study Area with Extract by Mask tool, a hydrology dataset was created.  This was accomplished using the following Spatial Analyst Hydrology tools:

• Fill - to modify the raster to prevent water flow from pooling up anywhere in the Study Area.
• Flow Direction - to assign each pixel a value representing the direction of water flow across that cell.
• Flow Accumulation - to calculate the total number of other cells that flow into it.
• Con - defines the threshold of flow accumulation values that qualify something as a stream.  Prior to running this tool 1% of the pixels of the original clipped DEM needed to be calculated in order to quantify which conditions a pixel must meet in order to be classified as a stream.  
• Stream to Feature - creates polyline vector feature from the Con tool output raster.
• Basin - delineates the drainage areas in the raster, giving each a unique value
• Raster to Polygon - creates a polygon shapefile from the raster.

This is the basemap that will be worked from in the upcoming Analyze and Report weeks.  Also a part of this week's assignment was to create a story map of the MTR process and a Story Map Journal to be filled in over the next couple weeks.  These are the links for both:

Story Map:  http://arcg.is/2daFw7K

Story Map Journal:  http://arcg.is/2daNCgz

Ground Truthing and Accuracy Assessment

Photo Interpretation and Remote Sensing - Mod 4

Using the map from last weeks lab we investigated random areas of the map to verify the classification schemes we had selected.  Since we weren't able to field check the areas we used Google maps to do so.

 We started by creating a point shapefile, then starting an editing session to add 30 points to the file.  I chose the Stratified Random Sample Protocol to select the locations for my points.

Once I had the points selected I used the Identify tool to determine their coordinates in Degrees, Minutes and Seconds and entered that data in Google.  The image in Google was much clearer than the TIFF we worked from originally.  Even in those areas where I wasn't able to get a street view I was able to see the area clear enough to tell if I had gotten it right or not.  I was not surprised to discover I made the most mistakes on the islands in the river.  They were very difficult to distinguish in the TIFF so I guess at most of them.  In Google I was better able to tell there were no trees so what I had marked as Forested Wetlands was actually Non-Forested Wetlands.  The only real surprise was to find a house with a nice lawn, trees and even a few flower gardens in what had once been just a sandy area.  Overall my accuracy came to 67%

Network Analyst - Report Week

Special Topics - Project 1 Week 3

This week was putting together the final results of the previous two weeks.  It seemed a simple task, but turned out to be rather daunting.  Eight maps had to be created.  Two of them were added to a pamphlet for the hospital evacuation routes.  Those maps and the next three were pretty simple though a little time consuming.  The next three were supply routes from the National Guard armory to each of the three shelters.

The next map was the most difficult.  This was the map with multiple routes leading out of Downtown Tampa to a shelter.  There was a lot information that had to go on that map including the color coded routes, street names, arrows pointing the correct direction.  This alone wouldn't have been too bad.  But it was supposed to be finished off in Adobe Illustrator.  I spent a considerable amount of time realizing I no longer remember how to use that program.  Finally I had to give up and just finish it in ArcMap.

The last map was also supposed to be done in AI, and  I thought since it was a simpler map that might work, but still had no luck.   This time I finished it off in Power Point, but I'm not real happy with the results.