Sharing Tools

The final week's lab required that we open and edit a script, updating it so that the hard-coded variables for input boundary feature and output file location parameters were set using the sys.argv[ ] code instead.  This code uses a number system beginning with 1 for the first parameter, rather than 0 which is where parameters set with GetParameter and GetParameterAsText functions use.

The next step was to edit the tool's description in ArcCatalog to make the tool more user-friendly and informative.  We added Dialog Explanations for each parameter, which shows up in the dialog box when using the tool as seen in the screenshot here on the left.  On the right is the map created by the tool which created random points and then put a buffer around each one.

The last step was to embed the script into the tool so it could be shared without adding a separate script file, and so that it could be protected with a password, which we did. 

Two other steps which are important are making sure that relative file paths are stored (check box), and that the .py extension is made visible in ArcCatalog, via the Options box.

Module 10: Custom tools

 

The ability to turn a script into a custom tool was this week's topic.  To do that, the following basic steps were followed:

1.      Create and save a .py script

2.      Create a custom toolbox for storing it

3.      Add a script tool to the custom toolbox with the AddScript wizard, selecting the saved .py script

4.      Modify the code in the script so it can receive the parameters set by the tool dialog box

5.      Set up the parameters in the tool’s properties

6.      Edit script using GetParameterAsText and GetParameter functions to set up parameter Dialog box for script tool

 

Here is a screenshot of the parameter Dialog box that allows input to the script tool.  Here there are 4 parameters, two of them already showing default locations for input and output files.  These are examples of workspace type parameters.  Two feature class parameter types are also listed: clip boundary feature and input features.  The latter was set to have a MultiValue property, since more than one .shp was used in this situation. 

 

 

 

Before the script tool ran, we edited it once more, adding AddMessages statements so that they would print in the results window, as shown below.

 

This lab was especially useful in showing how all the work of creating scripts can be used in a more simple, direct, and integrated way in ArcMap.

Module 9: Rasters

Last week we worked with Vectors, this week we learned how to write scripts for Raster data.  The functions and classes found in the arcpy.sa module are used to list, describe, create, and modify rasters.

ListRaster is used to find out what rasters exist in a workspace - Esri GRID and geodatabase rasters do not have file extensions; .img, .tif, and .jpg are returned for other image formats.

Describe returns general and specific properties of raster datasets.  The elements that can be described are datasets, bands, and catalogs.  Different properties are available depending on which element is being described.

Once a raster object is created, it can be used in other Python statements and map algebra statements.  Properties include band count, cell height and width, spatial reference, pixel type, and more.  The raster object has only one method possible - save.  This is used to make a temporary raster object permanent after ArcMap has closed.

In order to be able to use the tools in the arcpy.sa module directly, all the functions can be imported.  This can be helpful especially when using map algebra operators.

Several classes in the arcpy.sa module were introduced, including Remap and Neighborhood.  There are 8 others.

This week's lab asked us to create a raster from 2 existing rasters: elevation and landcover.  To start, we wrote code to see if the spatial analyst extension was available, then checked out the spatial analyst extension.  Next we reclassified the landcover raster so that 3 forest landcover classifications were changed to 1.  Then we created temporary rasters for elevation that had slope between 5-20 degrees, and aspect between 150-270 degrees.  The combined temporary rasters were saved to a final raster in a newly created geodatabase, and the spatial analyst extension was checked back in.  Below is the result, seen in ArcMap.

Using GIS for landscape archaeology

Sam Turner, Jim Crow, “Unlocking Historic Landscapes: Two Pilot Studies Using Historic Landscape Characterisation” in Antiquity 84(2010): 216 – 229

http://antiquity.ac.uk/ant/84/ant840216.htm

This article explains how GIS is being used to recreate historic landscapes and to analyze and interpret their development.  Historic Landscape Characterisation (HLC) is the term for “mapping landscape with particular reference to its historic character and development”.  The archaeologists in this example are using GIS to explore two locations in the Aegean, focusing on field systems and how they evolved over time.

Recently there has been a shift to focus less on either of the two prevailing approaches to the study of past, present and future landscapes - economic/functional or social/symbolic, and instead to look at “multifunctionality”.  HLC uses satellite images, aerial photography, and historic maps to “map, analyze, compare and contrast the perceptions of a wide range of people working within the landscape” using GIS.

HLC is not new, but in the past data storage had been an issue when working with such large projects.  GIS provides a solution, allowing a range of map sources, the ability to adjust scale, and the analysis tools which enhance interpretation.   With GIS, features of the historical landscape at a particular period are bundled together, creating visible groupings and patterns that characterize that period.  It is possible to do retrogressive analysis, and to add explanatory text connected to a database.  Also, GIS is flexible and adaptable to projects and research questions of any size or topic.

The two examples of the use of GIS in HLC in this article are from Greece and Turkey.  In Greece, the fields are terraced, and modeling can expose patterns of land use over time.  In Turkey, coaxial fields were used, and again, maps show a chronological progression of land development.

I chose this article for this assignment because it is along the lines of a project I’d like to put together once I’ve learned enough to do it. 

 

 

 

Working with Geometries

 

In Module 8, we explored how to work with geometries - points, lines and polygons.  The use of geometry tokens to simplify some operations was one topic; SHAPE@LENGTH and SHAPE@XY were two of the tokens we worked with. We also learned how to parse points and polylines in order to get at the values within the geometry. The last part of the lesson focused on reading and writing geometries.

 

Below are the results of a script we worked on this week.  Each line contains data about a point or vertex in a feature.  The shapefile that provided the data was of rivers in Hawaii.  The script retrieved information including the object ID, x and y coordinates, and name of the river for each point, then created a new text file and wrote the information to that text file.

 

 

 

 

I had some trouble with the last part of the lab, writing to the text file.  My two issues were that I didn't put the writing part of the script inside the most nested loop - I had it all the way out, not indented.  Also I was using double quotes around by variables, so I kept getting the name to print and not the values of the variables.

Cursors and Dictionaries

This week we investigated ways to explore and manipulate spatial data.  The Describe function allows the user to discover what data exists as well as to examine the properties of that data.  Tuples and dictionaries were introduced.  We then went on to learn how to manipulate data by using cursors to access and iterate over rows in a table, or to insert or delete records.  One use of the SearchCursor function we practiced in the lab was in creating a SQL statement, which in our case gave us the name and population of all county seats in New Mexico.  We used this table to create a dictionary of keys (cities that were county seats) and values (their population).  One other topic this week was working with text, which was not part of the lab.

This screenshot shows the results of a script which created a geodatabase, copied data (shapefiles) into that geodatabase, selected the name, feature and population fields of the cities feature class, then narrowed that down to just those cities which were county seats.  The last part of the script created a dictionary and populated it with this information and printed it.

It took a while to get the correct syntax for the SQL statement, since the correct type and number of brackets and quotation marks is critical.  However, I got stuck on the part of the code where the dictionary was to print the name and population.  Finally, with some excellent tips and guidance, I was able to make it work.

Python and Geoprocessing

 

 

This week's lesson focused on writing Python script for geoprocessing tools. While these tools can be used in ArcMap, writing custom scripts can increase the potential of tools. 

The lecture, readings, and exercise that prepared us to write our own script included material on importing Arcpy modules, classes, functions and tools.  We practiced setting a workspace, creating variables, working with parameters, learning syntax for commonly used code, working with messages, and using a few geoprocessing tools.  Then we wrote our own geoprocessing code in order to create a 1000 meter buffer around hospitals, dissolving the overlapping perimeter lines.  We also set the XY coordinates for each hospital. After each geoprocessing tool was run, a line of code asked for a message to be printed, allowing the user to see if each step had run successfully. The image above shows that the script ran successfully in PythonWin.  In ArcMap, running the script produced the new shapefiles and added them as layers to the map.

This week's reading assignment was Chapter 5, and for me it was the most essential one so far, giving me several "ah-ha" moments as I finally understood some of the concepts and language that have been frustrating me up to this point.  Up until now, I didn't really get how the code we were writing actually made things run.  Things are clicking a bit better now.

Geoprocessing in ArcGIS

 

 

In Module 5, the lesson addressed the following objectives:

• creating a toolbox
• creating a tool using ModelBuilder
• setting model parameters
• exporting a script from ModelBuilder
• updating a model-derived script to work outside of ArcMap
• creating a script tool

The task was to use ModelBuilder to clip a "soils" layer to the extent of another layer, "basin", then to select soils within the clipped output that were not prime farmland and to erase them.  The image above shows the result.

 

Using ModelBuilder was straightforward and fun (compared to writing the script from scratch), and it helped me visualize the process so that when we did have to update the script to work outside of ArcMap, I actually understood what was written.  The three tools, Clip, Select, and Erase, were linked with inputs and outputs whose parameters we manipulated.  This also clarified the process for me. 



Fixing things

 This is the result of running the first code once I had fixed a few syntax errors.  Little things like capitol vs. lower-case letters, or an added letter in a word make a difference, and code won't run if it's incorrect.

 This script had eight errors to fix.  I commented-out all but the top and ran through the code bit by bit, looking at the error messages in the interactive window and correcting the code based on the message.  Some errors were syntax errors, others were problems with objects and functions.

This script shows the result of using a try-except statement to catch an error.  It took a while to get the except message written and placed correctly, but it finally worked.

The lab this week was very helpful, and I'm glad it came earlier on in the course.  I wish I had learned it for last week's lab!

Using lidar to uncover archaeological landscapes at Angkor

Article: “Uncovering archaeological landscapes at Angkor using lidar”

Damian H. Evans, Roland J. Fletcher, Christophe Pottier, et al

www.pnas.org/cgi/doi/10.1073/pnas.1306539110

While there has been a long history of archaeological investigation of the structures associated with Angkorian civilization, our understanding of the complex system of civic-ceremonial centers, high-density and low-density urban environments, water management systems, and agricultural space has been incomplete and flawed.  One major reason for this is the thick forest that covers the structural remains.  With the development of lidar, the vegetation is, in effect, removed.  This article describes the use of lidar and ArcGIS processing in Cambodia at the site of the medieval Khmer Empire complex of Angkor.

In 2012, a block of territory covering the forested area within most of the Angkor World Heritage site was scanned to map variations in surface topography in both horizontal and vertical planes.  Locations were recorded using GPS.  Data points were processed using a method specifically developed for archaeological surveys in forest environments.  These were processed into DEMs, hillshade models, and local relief models using ArcGIS, and there were then analyzed and interpreted.   Archaeologists then went out into the field to verify the results using data loaded into portable GIS units. 

To illustrate a portion of the analytical part of this process, the article includes two visual examples.  In Fig. 2 two layers are presented, one a digital orthophoto mosaic showing elevation from the lidar digital surface model, and a layer showing the extruded lidar DTM with 2x vertical exaggeration.  Modern roads and canals are also shown.  In Fig. 3, a lidar DTM, a conventional satellite image showing limited features due to the vegetation, and a map of previously documented archaeological features shows the extent to which lidar has revealed new structures.

This use of GIS processing, with the precision of GPS location technology and the ability of lidar to strip away vegetation and reveal even slight variations in the ground surface, has transformed our knowledge of the extent and layout of the Angkorian complex.  First, it revealed that the urban center was at least 35 square km in area, rather than the 9 square km that was previously thought to be its extent.  It also revealed that the urban landscapes are not confined within the enclosed or walled city, but extend far beyond.  Much of the extended urban landscape features conform to what was known to be a predictably aligned formation in a grid pattern, but lidar revealed some new types of urban features which have no apparent agricultural, occupational, or hydrological function.

As a result of this study, crucial areas have now been mapped and analyzed, and our understanding of the layout and extent of the Khmer civilization at Angkor, and the factors associated in its decline, has greatly increased.

If, then, while, else

If I had all the time in the world, then I would be able to handle this assignment much better.
If I had been taught even a little bit of coding in my educational past, then I would not be so confused and inept.
While I am in school this summer I will try my hardest, even though that isn't cutting it at the moment.
Else I will fail, and that is not an option.
Print result.

 

This week we learned about conditional statements and loops, and then got to try them out.  First we imported the module random, then fixed some code that was already written so that it printed out the loop above concerning dice being thrown.  Then we created a loop to print a random list of 20 integers between 0 - 10, and then another conditional statement and loop to remove a certain number from the list. 

 

Never have I spent so much time trying and trying and trying, all for only partial success, and that only with some coaching by a very helpful instructor.  I'm sure that if I had more time, and could really practice each thing we learned this week in small pieces, several times each with variations, this would make more sense, and perhaps I would have been successful at the last piece of code.  However, I ran out of time.  Life intervened.  The lawn had to be mowed before it rained again, and my kids needed their mother back.  So it goes.  I'll keep trying, for what it's worth.

 

 

 

 

 

Mod2 - Last Name script

 

 

 

This week's task was to write a script. The screenshot above shows the result.  To make the script, several steps were taken which allowed us to practice some of the functions and methods we learned in this week's lecture and exercise, including writing a string, splitting it and creating a list, printing one item on the list using indexing, finding the length of my last name, and multiplying that number by 3.

Slowly, the fundamental concepts from this week's lesson are sinking in, and this lab was excellent practice.  It's certainly a challenge, but there is something satisfying when it works after trial and error.  The most frustrating thing is PythonWin, which refuses to do anything but flash briefly on the screen.  A nice person at the Help desk clued me in to a quick fix, which is to type wait = input ("PRESS ENTER TO CONTINUE) as the last line of code.  That's the only way I could test my script in PythonWin and see the result, or get a screenshot.  The Help desk person couldn't figure out what was going on with PythonWin, even by going on my computer remotely.  Apparently this is not an uncommon problem, since I found quite a bit on it by doing a quick web search.

Folders, by Python

Folders, by Python

This list of folders was created by running a script created by UWF staff in PythonWin.  In the lab we were supposed to be able to see the details of the script, by opening the .py file and choosing "Edit with Python Win.  Something wasn't working correctly for me, because although PythonWin opened, I was never able to view the script.  Clearly it ran, but I never got to check it out.  I'll see if I can make it work another time, but the window of opportunity for schoolwork has ended for me this week.