Sharing Tools

GIS Programming - Mod 11

Our final assignment for this course was Sharing Tools.  We started with a script that was already written and a tool that was already made.  The script had a couple paths that were hard coded to non-existent files or locations so our first step was changing those paths to parameters defined by the tool.  To do that we used the sys.argv[] function, which is very similar to GetParameterAsText() as it always returns string objects.

 With the variables set to pull the paths from the tool parameters the tool then ran correctly and added random points and put buffers around them, so we moved on to clean up the tool a little bit.

The dialog box has a help window that explains the purpose of the tool and the requirements for all the parameters, but it doesn't just magically appear with the creation of a tool, so we entered it by editing the Item Description in ArcCatalog.  Since this tool was a combination of two existing tools, Create Random Points and Buffer, the explanations could pretty much be copied from those tools with just a little tweaking.

Once the script and the tool were both cleaned up and grade A professional looking the script was imported to the tool so the tool could more easily be shared, requiring only the passing of the toolbox.  And to make sure nobody could sneak in and sabotage all my hard work on this my last GIS Programming assignment, I password protected it with a diabolically clever password nobody would ever figure out in a million gazillion years.

While I really enjoyed the entire course, I think what I enjoyed most was learning how to create custom tools.  It frustrates me to use a program and always wonder how it knows what to require of me and what to do with the information I give it.  Creating custom tools gave me a behind the scenes, behind the screen, view of what is going on.  Now I have a sense of what goes into the making of dialog boxes and what the limitations and possibilities can be.  I think that is something that can come in very handy in the future, especially for making GIS more usable to others who don’t have time to learn the program.  They only have to learn how to enter the data required, and they’re making their own maps.

Another thing I really enjoyed was debugging.  I just don’t feel I have a deep enough knowledge yet to really be very successful at it.  Hopefully once I get back to work I’ll have the opportunity to use Python and get to really dig deep into it. 

Creating Custom Tools

GIS Programming - Mod 10

This week we modified a script to create a custom tool and a toolbox.  We modified a stand-alone script to create a tool to perform the clip function on multiple features based on a single clip boundary file. We did this by adding the new script to the new toolbox we created, added a description and selected the "Store relative path names" option.  Next we set the tool parameters and the options that go along with them.  

Once the tool was prepared we went back into the script to make the adjustments that were required to take it from a stand-alone script to one that would work within ArcMap.  To do this we had to replace the hard-coded values in the script with parameters passed by the tool using arcpy.GetParameter () and adding the appropriate index value for each variable into the argument and replacing the print statements with arcpy.AddMessage () statements in a for loop.

To do this we followed these basic steps:

1.       Create a new toolbox in a location it can be selected along with the script.

2.       Add the stand-alone script to the toolbox.

3.       Add a Description and select the “Store relative path names” option.

4.       Select the script as the Script File.

5.       Run to make sure it works.

6.       Enter the parameters and their options in the Properties box of the tool

7.       Update the script to change hard-coded values with arcpy.GetParameters(#),  replacing # with the appropriate index value for that particular parameter.

8.       Update the script to change print statements to arcpy.AddMessage statements, making sure everything to be printing is inside the argument parenthesis and any commas are changed to the plus sign.

9.       Any variables that need to pass a string and use results from arcpy.GetParameter function must be converted to a string

10.   Verify “Store relative path names” option is checked in the tool’s Properties, then select the script and the toolbox and compress them to a zipped folder.

Working With Rasters

GIS Programming - Mod 9

The assignment this week was to write a script that would create a raster output that would identify certain areas with a particular set of parameters.  Those parameters were; slope, aspect, and land cover type.  We accomplished this by piecing together many of the scripts we created in our exercise using multiple modules, functions and the one method available to the raster object, .save.  

As usual we started by importing our modules and classes and setting our environments.  We had the option of creating a geodatabase to save our final raster to or just saving it to a file.  I chose to create a geodatabase because I wanted to refresh myself on the process since I think it's something I will use often.

There are different tools available in ArcMap depending on the licensing agreement you have.  Many of the tools we use are Spatial Analyst tools which requires a higher level of licensing than the basic ArcMap package.  The script we wrote was mostly contained within an If statement that would make sure the Spatial Analyst Extension was available before running the script, and an Else if it wasn't to print a message to the user letting them know the license wasn't available.

 In between those if and else statements we checked out the license to use the Spatial Analyst tools, remapped and reclassified the land cover raster, recalculated the slope and aspect of the elevation raster and combined all five temporary rasters, then used the .save method to save them to a permanent file in the geodatabase.  Once done with all of that the Spatial Analyst extension could be checked back in and the script ended.

This assignment was pretty straightforward.  Most of the steps were defined in the exercise so it was just a matter of going back through that to see what we had done.  The one thing that caused me problems was I had copied a portion of the script exactly as it had appeared in the exercise, even though there were a few things about it I didn't understand.  When my script was complete and I went to ArcMap to see what I had done, I found I had large patches of land that had no color.  I went back through my script several times looking for the culprit and searched the discussion groups and consulted with classmates to try to determine the problem.  Eventually it was discover that one of the parameters "NODATA" in the reclassifying step that had never been explained was what was causing the problem.  Once I took that out everything came out as it was supposed to.  And then hours later I found out how it original was was how it actually was supposed to be so I had to go back through everything and fix it all back to the way I originally had it.

To help keep track of where the script was in the process of running, a print statement was put in prior to each step.  This not only helped to keep track of where the script was in the process, it was also a way to trace back where a problem was if the script failed.  These results show the progress of the script, right to the successful end.  It is a very useful step in the process of running the script, but very frustrating to deal with when creating the flow chart, since it adds so many extra elements to the chart.  I often wonder if a generic note on the bottom of the page stating each step was preceded by a print statement would be good enough, but never think to ask.

You Can't Get There From Here

GIS Programming - Participation Assignment #2

You can’t get there from here.  It’s a comment frequently heard.  Often it’s said with a grin or a smirk, sometimes with chagrin.  But when it’s a matter of life or death or the loss of all you own, it’s not so funny.  Narrow streets and curvy road can be a serious impediment for large emergency vehicles like fire engines.  Even wider streets with cars parked on both sides can be difficult to navigate.  These are issue that concern fire departments around the world.  How you get there from here could save a life, or lose it. 

This article I found on sciencedirect.com, published by Procedia Social and Behavior Science from the 11^th International Conference of the International Institute for Infrastructure Resilience and Reconstruction (I3R2) : Complex Disasters and Disaster Risk Management, titled “Models and Applications of Firefighting Vulnerability”  talks about the many investigations that have been done by researchers in regards to the use of GIS in firefighting, but weren’t comprehensive enough so another study was called for.

This study focused on the scope of firefighting models in regards to applications in the system and showing how much more GIS can be used with new information technology environments with the use of state-of-the-art devices and more in-depth information from the fire scenes.  The four models were based on fire vulnerability types, considering firefighting vulnerabilities and critical factors through the different stages of the fire, from the notification at the fire station to the fire scene, through the firefight and back to the station.  The study was conducted in four cities in South Korea.

This is the link for the article if you’d like to read it in detail:

http://www.sciencedirect.com/science/article/pii/S1877042816300192

Working With Geometries

GIS Programming - Mod 8

This week in working with geometries we created a text file and wrote lines of data to it from a rivers polyline shapefile.  This text file is the result.  Each segment of line for each object is listed individually.  The first number is the object ID, the second number is a vertex ID to show which segment of the object it belongs to.  Next are the xy coordinates and last is the river name.  This data was derived using a SearchCursor and a couple for loops, then written to the text file with a .write () statement.

Also as part of the assignment was writing the pseudocode and a flowchart for the script.

Exploring and Manipulating Spatial Data

GIS Programming - Mod 7

This week we created a geodatabase using the CreateFileGDB_ management function, then created a list of shapefiles in our Data folder and copied them to our geodatabase using the CopyFeatures_ management function.

With our geodatabase set we then created a SearchCursor for the city names and populations for those cities within the cities feature class that were county seats in the state of New Mexico.  This was the only real problem I had with the assignment.  I was having a little difficulty figuring out how I was going to populate the dictionary two steps ahead, but I decided not to worry about that step until after I finished this step.  In the exercises we performed a SearchCursor and it was pretty straight forward, but we were only looking to search the attributes of one field in the table, whereas for the assignment we were dealing with two fields.  It took me a while to figure out the SearchCursor was basically going to build a list of the two fields based on the attributes of the third field.  Once I realized that I was able to figure out my syntax and move on.  I created an empty dictionary, then populated it within a for loop by pulling the fields from the list I created with SearchCursor and updating the dictionary with them.

Getting all this down in the flowchart was a bit of a chore.  Not so much the different steps, but adding all the print statements was kind of a pain since we had to add one before and after each step.  I ended up combining the ones that came between steps and just hope that will be good enough.

Geoprocessing with Python

GIS Programming - Mod 6

Module 6 had us running three geoprocessing tools with a Python script.  AddXY added xy coordinates to a hospitals point shapefile, Buffer generated a 1000 meter buffer around the points, then Dissolve dissolved the buffer file into a single feature.  

After each tool I printed a message with the GetMessages function to show the successful completion of each tool.

With that successfully done I created my flow chart and the assignment was done.

Geoprocessing in ArcGIS

GIS Programming - Mod 5

This week we learned about geoprocessing tools, models and scripts. We started out creating a new toolbox then, within that, creating a model using ModelBuilder.  Our task was to clip the soils layer to the extent of the basin layer and remove from that selection those soils that were classified "Not prime farmland."  To do this I dragged the soil and basin layers from the table of contents to the model and the Clip tool from the Toolbox menu, added the parameters to the Clip tool and ran the model.  As soon as I clicked run I realized I hadn’t set my workspace and scratch environments or adjusted the path for my Output Feature Class so I had to delete the output from the geodatabase of a previous lesson, set the workspace and scratch place environments to the correct one, verify the Overwrite option was selected, then rerun the model. 

With that portion working correctly I dragged Select to the model and added the output of my clip as the input, defined the output and created an SQL expression to select only the specified attributes to add to the selection set.  Using the results of the Select tool I used Erase to remove those attributes from the Clip output.  Before running this step I right clicked on the output and selected Add to Display so my output would show up in the table of contents and I could check the attribute table to make sure the selection worked correctly.  The above shows I did it right so I was able to export the model as a Python script and moved on to the next part of the assignment.

Exporting a model to Python allows the script to work within ArcMap, but in order for it to work outside of ArcMap as a stand-alone script a few things need to be changed.  Because the input files came directly from the map's table of content they didn't have a path associated with them.  In order for the script to stand alone the path needed to be added to those variables.  The next change was telling the script to overwrite the output since that had been set within the map workspace, not the model.

 Now my script was able to stand alone as though I had written the whole thing from the beginning. From this script I was able to create a script tool within the toolbox.

Debugging and Error Handling

GIS Programming - Mod 4

Debugging and error handling are crucial aspects of programming.  Mistakes happen and when then do it's very important to know how to fix them.  Our assignment this week was to work on three scripts that had errors in them.  The first script was to print out the names of all the fields in an attribute table of a shape file.  It had a couple typos that needed to be fixed, then ran fine.

Script 1 Results

The second script was to print out the names of all the layers in the data frame in a map.  This one was a little more complex.  In this script we had to correct 8 errors of different types, from correcting paths, to typos, to adding or removing arguments from statements.

Script 2 Results

The last script was the most difficult in that it was a difficult concept to understand.  The intent of the script was to return an error statement for Part A and certain information from the data frame of a map for Part B.  Part B had no errors in it, but we had to get through Part A before we could get there.  Instead of correcting the errors in Part A we had to use a try-except statement to allow the script to run through the error portion without failing, but returning a relevant error message, then move on to the second portion of the script.  To accomplish this I ran the script to see where my first error was then investigated that.  I saw I was missing an argument in a statement so I placed my try before that line.   At this point I wasn’t really sure how the try-except worked so tried the except after the for statement to see what that would do.  I ran the script again and came up with another error.   I realized at this point that nothing after setting mapdoc could work right because everything was based off mapdoc being correct so I moved my except to the end of Part A and added the second error type to the argument and ran the script.  This time my script ran correctly, but I realized by moving the except where I did I shouldn’t be getting that second error any more so I removed that and tried again and it was good.

Script 3 Results

We also had to do a flow chart for one of the scripts.  I chose to do it for Script 1.

Flow Chart of Script 1

Python Fundamentals Part 2

GIS Programming - Mod 3

Woohoo!  What a blast.  So far programming in Python is like sitting around playing with a jigsaw puzzle and getting credit for it.  This week's lesson was a bit more complicated.  This week we started with a code that was partially written, with a few mistakes in it.  We had to correct the mistakes for the code to create a dice game based on the players' name length.  

Results of Python Script

After correcting the existing code we then had to use the modules, methods, functions and conditional statements we had learned about in both this week's and previous weeks' lessons to generate a list of 20 random numbers ranging from 0-10, then select one unlucky number to remove from the list and print a statement saying what would be removed and how many times.  That was a very exciting, but extremely intimidating process.  Even after reading the instructions for the assignment twice I still missed a lot of the information and clues that were provided until I went back a third or a forth or even a fifth time.  

I didn't have any difficulties with the portion of the assignment we were required to take notes on; removing numbers from the list.  My difficulties came from the previous section where we had to print statements saying how many, if any numbers were to be removed from the list.

Flowchart of Python Script

 It was a fairly

simple process until my first test came back with the result of removing only one instance of my unlucky number.  That was when I realized  I was gong to need an elif statement in there too, otherwise the sentence would read that it was taking the unlucky number out "1 times." and that wasn't right.  Figuring the syntax out for that was pretty easy, but it took me quite a while to figure out how to express the elif statement in the flowchart.  All the examples I had seen were basically just if/else statements so I had to play with it for a bit to figure out what made sense for the if/elif/else combination. 

Another confusion I had with the flowchart was how to handle not starting at the beginning.  It was left to us to decide if we would include the code that had been provided for us or just start where we created the code ourselves.  I decided to start where I started, but I didn't want to leave the impression where I started was the beginning of the script so after showing we imported random I added a break to the flow line and hoped that would make sense.

While I learned a lot from this lesson I think one of the most important things I learned is that I need to reorganize my previous lessons so certain aspects of them are more easily accessed.  If this week's lesson is any indication it looks like I'll need to go back quite a bit to refresh and clarify how to do certain things.  There is so much to learn in each lesson it just isn't possible to absorb it all at once.

Python Fundamentals Part 1

GIS Programming - Mod 2

This week was another fun assignment.  We had to write a Python script that resulted in the length of our last name being multiplied by three, but the only information we could enter manually was a variable of a string of our full name.  Every other step required a function or method to pull the information out of that original string and manipulate it down to where we could get the length and multiply it.  Some of the methods and functions we used to accomplish this were split, indexing and len.

We also had to create a flowchart of our process.  The flowchart was a little more difficult for me to d

 I think mostly because I'm not very familiar with Power Point and I had some difficulty getting things to work the way I wanted them to.  The most annoying problem I had was trying to switch the layout from landscape to portrait so I could spread my boxes out a bit more, but I was never able to figure that out.  Otherwise it was fairly straight forward.

Intro to Python

GIS Programming - Mod 1

This week's assignment was a lot of fun.  We were introduced to Python, the preferred programming language for ArcGIS since 2010.  This assignment didn't require much from us, it was basically just an introduction to the programming language and the different methods for working with it such as IDLE and PythonWin.  The main point of the assignment was to learn how to run an existing script called "CreateModuleFolders.py" to create folders for all the mods of this course.

I followed the instructions exactly and had no problems with this.  When I got to the fourth step, the actual running of the script, I saw there were a few different methods that could be used.  From within the PythonWin window where I was viewing the script the easiest way to run the script would have been to click the little icon of the running man.  But I wanted to watch the actual process of the folders being created so I went back to Windows Explorer and double clicked on the file.  There was a little blip on the screen and then it was done.  So it turned out there was nothing to see after all, but the job was done.

This is such a cool little program I thought I would go ahead and use it to create the same folders on my hard drive.  I made a copy of the script so I wouldn't mess up the original, then changed the path from S to C, but it put it on my network C rather than my hard drive.  Oh, well.  I'm sure once I download Python onto my hard drive I'll solve that little problem.  In the meantime, I'm really excited to use it for setting up my folders for my applications class too, just as soon as I figure out what folders I'll need for that class.

I know this is just the first assignment and it was a simple one, but I just know I'm going to love this class.  I can tell already it's going to be a lot of fun, and even the mistakes will be a blast to troubleshoot and figure out.