Games I'm in

One of my many hobbies is playing table-top games. Of course, there is not that much time anymore, so these tend to be played only with imaginary friends (i.e. over the internet). And it does change its character over the internet (i.e. message boards and email). Because at that point it is more shared writing of fiction then a game. When I was deployed, a Marine in my unit enjoyed teasing me about this. After a point, when he commented that I seemed to feed this bit of entertainment, I replyed "if this is what you are ragging me on, I'm doing pretty good."

The premise of role-playing games is you play a character. And the character has a background which provides abilities, characteristics and motivation.

Like books and movies, there are many types of games, for different motivations for players. Some are more escapist, and the focus is on combating opponents in contests that follow set rules. These often involve lots of dice rolling. Dungeon & Dragons is the most well known of this type. Others are narrative, where the goal is to tell a good story and the characters are encouraged to take on weaknesses to advance a story line. FATE (Spirit of the Century) is like this. Others are focused on the characters, where the characters are written and played so they can be explored in depth. And in these later two, after the character is created, it almost disappears into the background, since the numbers do not matter as much as the description.

The idea is (unless one is escapist, or is playing a game that intentionally comedic) to see the world from another point of view, where you are faced with a different sets of abilities, skills, weaknesses, liabilities, strengths and knowledge. And you can make decisions that have consequences (in-game consequences that is).

I am in three games right now. In age:

Kanon: A non-commissioned officer in an army in a world that is mostly low-fantasy. Actually, because he is a very ordinary NCO, this is pre-industrial revolution. Other characters actually look more low-fantasy. There are actually two military branches in this scenario. One is focused on individual skill and its members have a upper class bearing to them. Kanon is part of the other one, whose focus is there professionalism. Kanon is sure that he is in the better branch. As he is also a racial minority (human) of a race that a large portion of which are in slavery, professional pride becomes a large part of how he engages the world. (http://www.kalyr.com, using Fudge (Wikipedia)

Adelson: a newly-minted university professor (biology) in 1920's United States. It turns out, finishing grad school and starting academia, brought him right back home. Which makes his mother very happy. The scenario (which he does not know anything about yet) is horror. What he knows is that there is an environmentalist vs. logging company fight not far from his university that he has been asked to study. (Call of Cthulhu, Wikipedia) )

Jason: a spaceship engineer/pilot who has just finished his enlistment in a space navy. And now needs work so he is signed up on a merchant ship. Actually, I don't know much about him yet. Yes, there is a character sheet and backstory, but the other players don't engage in much conversation, so there has not been much of a chance for him to have a personality yet. (Traveller (Wikipedia), using Fudge)

There is room for players in these games (well, not Cthulhu) if it is of interest.

In Memoriam: Keith Herber Post-Interview Chat Video

Yog RadioVision - In Memoriam: Keith Herber Post-Interview Chat Video

Posted using ShareThis

In memoriuam at Yog-Sothoth.com

Keith Herber was an author of stories and game scenarios based on the setting of stories by H. P. Lovecraft, commonly referred to as the Cthulhu Mythos. He passed away suddenly last Friday. By all accounts, people remember him for his guidance in working through the collaborative process of writing as well as being an editor deeply concerned with the quality of the works that passed through him to print.

The craft of writing can be difficult. There is a need to write not just to tell a story, but to make a setting that is complete enough that the story has a place to live and flow. To create a game setting is even harder. You have to make the setting deep and rich enought that the reader can imagine a place in the world, and a group playing has room to roam. And you have to create the mood, such that those playing

I'm involved in a game based on one of Keith Herber's last works. We've just started, and as we settle into the personalities we've created, anticipating the path ahead, and wondering what Keith Herber has created for us. And remembering him.

PSO: Music by the sea

One of my hobbies is backpacking.  I enjoy the opportunities to be in the natural world, to accept it on its own terms (including storms), and to be able to listen.  And when you think of such things in music, the first piece that comes to mind is Beethoven Symphony No. 6, and the Fantasia rendering of green pastors, interupted by storms, and return to green pastures.  Except, the real, living world is not like that.  Even in its most peaceful and idyllic, everything is just much more, . . . , random.  And anyplace that is alive has sounds (whether or not you are listening properly is another question) and the representation of the natural world by sweeping melodies, evokes an emotion or a picture rather then represents its true nature.  Last night's Dharma at Big Sur by John Adams was different.

Dharma provided a different sort of view of the outdoors.  And listening to it brings you through a full day.  There is the beginning of the day, and you can almost hear the low murmurs of surf and nature entering your subconscious as you slowly wake up in the morning.  As the sun rises, more and more of the world awakes, with waking birds adding clarity above (but not replacing) the ambient sounds that continue. And finally, you wake up yourself, looking forward to the day and adventures ahead of you with sights and sounds both familiar and not (the gongs provide the different sort of sound that keeps things interesting.).

But my friends also know that water and I go together when I go outdoors.  And so Dharma has this too.  It also has the raw power of nature crashing.  But even here, its not just that you can hear the waves crashing against the rocks on the shore, or the thunder and lighting, but you hear the constant rain that is in the backdrop, and the wind's whistle bends as it goes by.

It is a different part of life.  I mentioned how John Adams Short Ride in a Fast Machine

seemed to reflect the greatness in even the doing of everyday tasks in a very man-made environment.  But this seemed almost organic and a living and growing thing.  And like a good backpacking trip, I almost expect it to be different the next time I hear it.

PSO: Wow, Fast fingers

I remember hearing about this concert a few weeks ago. I had been sent links to YouTube videos with Yuja Wang, and hints that she was a pianist with amazing technique, a virtuoso. But those are words I've heard before, and did not pay much attention.

For all the talk of technique and raw talent, I believe that there is more to being proficient then ability, it is the ability to apply that talent to a context interacting with others that provides value above and beyond the application of talent. I had a discussion about this in regard to a book I did not like all that much. It was a claim at a person could grow in skill to a level way beyond any of those around her. My claim has always been is that ability past a certain point requires a person be interacting with others who can complement her gifts and talents, and point the way towards greater heights.

Where I lose interest in virtuosi is when they seem to only shine alone. And the concerti where a soloist and the orchestra take turns playing are not as interesting as when the orchestra and soloist play off each other.

From the point of view of looking for virtuoso, Ms. Wang did not disappoint. I was amused during the intermission of one man who said to her "Wow." But, for all the accolades that Yuja Wang has received over skill, there was more she could show during the Prokofiev. From the opening, where the orchestra seemed like it was sneaking in to the piano's opening, to numerous transitions throughout, the changes in focus from piano to orchestra seemed smooth, with the level of intensity and energy matching at the point of transition, both the piano and orchestra parts fitting together. Supporting each other as in conversation, not merely dropping out in an exchange of soliloqueys. And sharing the stage does not make the soloist lessor, but made Ms. Wang's talent shine.

And this impresses me more than the ample demonstration of her skills, even more the arrangement of Flight of the Bumblebee encore that she played. That her and the orchestra playing together make each other look better. And people like that, not only do they look good. You are glad they do and hope to see them again.

Juno: Movie review

(Wikipedia)

Juno is the story of a teenager who is pregnant. In 2007 when it came out, there was much controversy, as everyone on the pro-life/pro-choice/teenage pregnancy/values issues had a go at it. But I never saw it until now.

I was all expecting it to be one of those movies with the spunky teenage girl and clueless adults around her. But it was not. It is a teenage girl, dealing with the world through her mistakes and with the help of people around her.

Juno is a high school girl, who has gotten pregnant. The other major characters are the boy (surprise!), her best friend, and her parents. While initially she was going to have an abortion (apparently, her best friend had helped a few other girls in their school through the process) Juno changes her mind at the clinic and tries to give up her baby through adoption. The couple that is planning on adopting the baby become the other main characters in the movie.

Instead of the sassy teenager, Juno is introspective and comes to terms with what she recognizes as a mistake. And her focus is on dealing with it. There are others around her like the school nurse, some of the other classmates who look down on her. But those closest react differently. Her parents first reaction is disbelief, mostly blaming themselves for being bad parents. But they too, after first beating themselves up, switch into dealing with the situation, getting Juno through her pregnancy and the adoption.

What I liked about it was how they handle adversity. Yes, Juno and her parents (and the father) are hard on themselves for messing up. But the focus is on what has to happen next. The parents making sure that lessons are learned. The daughter learning that her parents are paying attention and care about her. And they guide her, with what wisdom they have. And in the end they have her respect.

None of the principles thinks they have it all together. Just people who have made mistakes, and need to figure out life. And in a movie, that is good.

Beaver Falls Coffee and Tea: Guatamalan Huehuetenango

Beaver Falls Coffee & Tea Blog

Beaver Falls Coffee & Tea is a coffee shop nearby Geneva College in Beaver Falls, PA. It was started by a couple of alumni who just didn't quite leave, basically the stereotypical college coffee shop. But last year they have started the next step, roasting the beans. We occasionally get beans from them, and this time we got some Guatemalan Huehuetenango (which were Direct Trade).

Why is this something of note? It turns out, like most food, coffee is time sensitive. The green beans are shipped from there origins, but all the steps after that effect the taste, starting from when they are roasted. It matters how long from roasting they are brewed, it also matters how long between grinding and brewing,and of course, how long they are exposed to direct heat (i.e. either take the coffee off the brewer and pour it in a thermos within 1/2 hour or dump it.)

So a year ago, Beaver Falls Coffee and Tea started roasting. The other roaster in the Pittsburgh area is La Prima, but they have no outlets in the North Hills (I know, I've asked their marketing director and after much discussion among the staff, came up blanks.) So we made a point of using them, even asking them for different style of roasts to try the range of roasts (light to dark).

We got the Guatemalan this week. It is a medium-light roast. Based on prior experience, we French-pressed it at 7 minutes (for most coffees, we French Press for 5 minutes). At this point, it was still a little light, but we were at the point where the bitterness in the bean was coming out, so I think we have the brew time just right. But the beans were probably too lightly roasted for my tastes.

The dark roasts are probably easier to quality control and these have gone well (which is why if you get coffee at places that ship their roasted beans from somewhere else, you should get dark roasts). The medium and lighter roasts have more nuances, but our experiences with Beaver Falls Coffee & Tea is that they are inconsistent, and their mediums and lights are often too light (not as much taste)

UPDATE: Batch 2 was this afternoon. This time I ground the beans to medium grind (rather then coarse like I usually do for French Press) and had a bit more beans in there. Better. I get more of the taste. Again, at 7 min, I can taste just a hint of the bitterness that tells me that I better not let the beans brew any longer. Of course, now I have to use a second filter to filter out the grounds, but this mostly works. But we'll probably make a point to get medium-dark roasted in the future.

Of Cowards and Conversation at the New York Times

Of Cowards and Conversation, as summary by Eric Etheridge

I'm in a hotel room, very tired, listening to CNN No Bull with with Campbell Brown. And I'm writing, very dangerous to write in public while tired, especially if race relations is within spitting distance of the topic.

The setting is two-fold. The New York Post has an editorial cartoon satarizing the shooting of a chimpanzee who mauled a child in Connecticut, depicting the chimpanzee as the economic stimulus bill. Al Sharpton and others are railing against it claiming the chimpanzee is actually representing a black man, namely President Obama. The other event was Attorney General Eric Holder gave a talk where he claimed that the U.S. “in things racial we have always been and continue to be, in too many ways, essentially a nation of cowards.”

My feeling, in short, is the fuss over the cartoon is a farce in itself, and that Holder is right. Because it is incredibly difficult to have an honest conversation about race relations. I'd say I've only been part of one, at an alumni board meeting at my graduate department. Like real conversations there were opinions, even disagreements,. And like good conversations there was exploring, depth and the exchange and understanding of points of view. And an example of the dielectic. When this particular conversation was over, one of the more vocal people there openly commented that what we just did was shocking because it was an honest conversation about race relations. Left unsaid was the fact we probably could not expect to experience another one anytime soon. It is very easy for conversations to become a self-justification session, which tends to ruin the whole thing. (most conversations about people trying to announce their own sanctification are rather boring)

My wife and I have accepted the fact that if we have kids, they may not follow our paths. And our hope (well, certainly mine) is that in whatever direction our kids go, we will have older people in our circle who can be with them, so they may have a mature example of whatever it is they do. But in my own life, I've been noticing that my circle is less and less broad in scope. It's not married life, I've just don't have the energy to explore that I used to. And as big as my world is, the only way I will add to it is if it is indeed complementary to what is there.

These, like many choices, is a decision that is made. In this case, openly and honestly. It has the advantage there are some points that have a very big point of entry. But it is a reality that I don't have as broad a circle as I used to. And the world that I can share in life is smaller then it used to be. And that is a bit sad.

QOTD: We did not mean to go to the opera on Valentine's Day

(The following is a very loose paraphrase)

K: What did you do yesterday
L: Well, we went to the opera. You see, when S asked me about Saturday or Sunday, I just picked Saturday night without looking at the calendar.
S: I kept on telling you "Are you sure?"
L: You can't just hint, you have to spell it out!
S: So, we went to the opera on Valentine's Day.
L: We've been so good about never doing anything on Valentine's day

K: That is so you

Fresh Air talks about Afghanistan - February 4, 2009

Fresh Air is one of my favorite radio programs. It is a show of interviews. And Terry Groves is a good interviewer, who has the ability to ask the questions at the heart of the matter, and uses the interview to draw out the story the interviewee has to tell.
This episode revolves around the current campaign in Afghanistan and centers around two interviews. The first interview is with Sarah Chayes (Taliban Terrorizing Afghanistan), a former NPR reporter who was assigned to Afghanistan in 2001, and has stayed there to run an NGO (Non-Governmental Organization) that is establishing an industry in Afghanistan. The second is with Ahmed Rashid (Taliban Activity Up In Pakistan), a Pakistani journalist. Terry Groves asks them about the conditions they see in Central Asia, what they think about the U.S. troop increase promised by President Barack Obama, and what they think about the future.

Some highlights:

• Both of them are highly supportive of the idea to add 30,000 additional American troops into Afghanistan.  The biggest issue they both see is the lawlessness and lack of security, and 30,000 additional American troops will go a long way.
• Both of them are almost fawning in admiration for U.S. General Petraeus, the new head of the U.S. Central Command, for his leadership in Iraq as the head of U.S. forces in Iraq, and for Petraeus' understanding of counter-insurgency.  Listening to them, it reminded me of a Sargeant who unshamingly praised General Patraeus as a scholar-warrior, and this was when Petraeus was just starting in his post in Iraq.
• Cheyas sees as the most difficult problem in Afghanistan is the corruption in the Afghanistan government. And she sees President Karzai as part of the problem.  It is not just the fact that there is corruption.  It is the severity (not extent) of the corruption that is wearing out the people.  And they remember that it was the corruption among the rulers that made people welcome the Taliban many years ago in the first place.
• Both of them blame the previous American administration for not pressing the Afghanistan or Pakistan governments to improve in governance all this time.  That one reason for the lack of progress was because no progress was required, the Americans would support the Afghan and Pakistani governments regardless of their progress.

I especially find it interesting that both welcome the additional U.S. military forces in the region.  They both see the U.S. military as the most capable and beneficial force in the area.  And one that has learned much over the past few years.  But they are few in number, and they need other expertise to help the real issue, improving the Afghanistan ability to govern themselves.  Which is not something that is reasonably expected to be a job that goes to the U.S. military.  (of course, they get that job like all other jobs.)

Book Review: Desktop GIS: Mapping the Planet with Open Source by Gary Sherman

Desktop GIS: Mapping the Planet with Open Source by Gary Sherman

My review

rating: 4 of 5 stars
Desktop GIS covers Open Source software for use as a Geographic Information Systems (GIS). In particular, it covers the following programs and libraries:

- Quantum GIS
- uDig
- GRASS(and its Java front end JGrass http://www.jgrass.org)
- PROJ.4
- GDAL/OGR
- PostGIS
- FWTools
- GMT

At the beginning of the book, the author outlines three classes of users. A casual user who only needs to look at data found from elsewhere, an intermediate user who visualizes but also creates or converts data, and an advanced user who has the need to do spatial analysis. To cover all of these is an ambitious goal, which is further diluted by the authors felt need to cover the entirety of open source mapping in one book.

What the author does well is to identify tools, and explains what can do what with enough to get you started. So Quantum GIS and uDig are the viewers, able to read almost any GIS format (in particular the readily-available ESRI Shapefiles as well as PostGIS). GDAL and OGR that can convert anything to anything (including delimited text. These are often distributed as FWTools). PROJ.4 that converts projections from one to another (and is embedded in everything). PostGIS which is the spatial database that enables spatial analysis. And GRASS, which is the full-fledged can-do-everything-but-is-hard-to-learn tool. And then some random programs that either do something completely different (e.g. OSSIM, which is an imagery analysis tool) or can make a picture of a map with lots of options (e.g. GMT).

What he provides are the basics for the casual or intermediate user. The advanced GIS analyst would only have a taste of what GRASS can do, but would not know what can be done with it. Similarly, while the intermediate user will get a sense of what PostGIS can do, the lack of space to cover spatial extensions to SQL supported by PostGIS loses its value to the advanced user. What could have made this book better was more focus. If the author was compelled to have a survey of all open source mapping, it may have gone into an appendix with a few paragraphs for each of the miscellaneous tools. But for the book, one good focus would have been the GIS stack comprising of data storage, data analysis and data viewing. Basically the open source counterpart to the ArcGIS/Oracle with Spatial Extensions. And everything that does not play a role in the stack, gets pushed into the appendix.

What could have been done? The book tended to be organized by tool. But once past the casual user, almost all tasks required multiple tools. I would have gone:

1. Viewing data (raster, vector, introduction to QGIS, uDig, GRASS)
2. Converting data/data formats (GDAL/OGR, Maybe PROJ.4)
3. Creating/Editing data (digitizing, importing)
4. Spatial databases (PostGIS)
5. Geoprocessing/spatial analysis - GRASS, PostGIS, R-spatstats
6. Tools integration (QGIS/uDig with GRASS/PostGIS)
7. Scripting
8. Customization

and everything that did not fall into this gets a page in an appendix.

Knowing where to start is a big help. Most of the websites either focus on one product, or try to teach everything as being equally important. Gary Sherman at least identifies the main building blocks. (QGIS or uDig, GDAL/OGR/PROJ.4, PostGIS, GRASS) and gives enough to get started. And this can be very helpful, so at least the starting analyst knows where to start.

What would be next? For the person still working within the GIS stack (as opposed to a completely different topic, like imagery analysis which is OSSIM's territory) there are a few obvious topics.

1. PostGIS - Spatial databases with SQL. Maybe even connections with ArcGIS. Even a short (10 pg) appendix would have done wonders here.
2. GRASS - The author devotes an additional appendix to this. But to do this right, you probably need to refer to Open Source GIS: A GRASS GIS Approach
3. R spatial statistics packages. Applied Spatial Data Analysis with R would cover this.

Mostly, a good book to get started in GIS using Open Source tools. Casual users would be well served. Intermediate users would get started and can find the rest using the internet. Advanced users are going to miss alot (to the point they don't even realize that these tools were a worthy alternative).



View all my reviews at Goodreads.

Loading lat/long data into PostGIS

Base scenario. I have a datatable with longitude and latitude data that I want to look at using spatial tools. And I have shape files for the metropolitan area that I want to use them with. So, in other words I have the following.

1. ESRI shapefiles
2. Data table with latitude and longitude

The plan is to load the data table into PostGIS, then I can pull them into a GIS system like uDig or Quantum GIS or various ESRI products. Because PostGIS can do transforms, but ESRI shapefiles are what they are, the procedure is as follows:

1. Identify SRID (spatial reference ID) for the ESRI shapefiles. This should include units (feet, meters, or degrees)
2. Identify a corresponding SRID that used lat long degrees
3. Load data table into PostGIS
4. Add geometry (points) data to datatable, including transformation from degrees to feet.
5. Optional: export PostGIS to ESRI Shapefile if needed.

1. Identify SRID (spatial reference ID) for the ESRI shapefiles. This should include units (feet, meters, or degrees)

Shapefiles usually come as a set of files inside a directory. When they come from an official source, one of these should be a *.prj file. This is an example of one

PROJCS["NAD_1983_StatePlane_Pennsylvania_South_FIPS_3702_Feet",
GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983", SPHEROID["GRS_1980",6378137.0,298.257222101]],
PRIMEM["Greenwich",0.0],
UNIT["Degree",0.0174532925199433]], PROJECTION["Lambert_Conformal_Conic"], PARAMETER["False_Easting",1968500.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-77.75],
PARAMETER["Standard_Parallel_1",39.93333333333333],
PARAMETER["Standard_Parallel_2",40.96666666666667],
PARAMETER["Latitude_Of_Origin",39.33333333333334],
UNIT["Foot_US",0.3048006096012192]]

The first line identifies the projection, using something that should approach a WKT ("Well Known Text") name of a projection. In PostGIS, when the PostgreSQL database was geographically enabled, a table "spatial_ref_sys" was created that has a list of all SRID with names. Searching on this table should reveal a projection that has a name very similar to this one. That will define the target SRID for the whole project. The quick way to do this is to write a query:

SELECT srid, auth_name, auth_srid, srtext, proj4text
FROM spatial_ref_sys
WHERE srtext LIKE '%Pennsylvania%'
ORDER BY srtext;

In this case, we find:

2272; "PROJCS["NAD83 / Pennsylvania South (ftUS)", GEOGCS["NAD83", DATUM["North_American_Datum_1983",SPHEROID["GRS 1980", 6378137,298.257222101, AUTHORITY["EPSG","7019"]],

Note that in this case the key parts were NAD83, Pennsylvania South, and ft.

2. Identify a corresponding SRID that used long lat degrees
Another search through the "spatial_ref_sys" table finds only projections that are associated with feet or meters. So a broader search should be used to get


SELECT srid, auth_name, auth_srid, srtext, proj4text
FROM spatial_ref_sys
WHERE srtext LIKE '%NAD83%'
ORDER BY srtext;

This eventually reveals one projection that

4269;"GEOGCS["NAD83",DATUM["North_American_Datum_1983",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]], AUTHORITY["EPSG","6269"]], PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]], UNIT["degree",0.01745329251994328, AUTHORITY["EPSG","9122"]], AUTHORITY["EPSG","4269"]]"

The key here is that the 'UNIT' is "degree" as opposed to "Survey feet" or "Meters".

3. Load data table into PostGIS

First, by this point the PostgreSQL database was created using a template that has PostGIS enabled. I created an SQL file with all the Create Table commands ahead of time. They look like this:

CREATE TABLE table2004
(
case_no character(7) NOT NULL,
week integer,
"month" character(3),
eventdate date,
bg_lat numeric(12,6),
bg_long numeric(12,6),
serial2004 serial NOT NULL,
CONSTRAINT table2004_pk PRIMARY KEY (serial2004)
)
WITH (OIDS=TRUE);
ALTER TABLE table2004 OWNER TO userid;

Key point is that I need something an integer field that is a primary key, or at some points indexing becomes a problem. So I made a serial field to be an index (because my case_no field is a text field so was unsuitable). Another issue is the geographic indexing also likes an OID field, so I set OIDS = TRUE. Most current PostgreSQL documentation mentions that this is deprecated, but PostGIS databases would be an exception.

Next, a geographically (geometry) enabled field is added. PostGIS has a PL/pgsql function named "AddGeometryColumn" that does this. Note that earlier, I established that the target SRID is 2272. Next, because there will be spatial searches, I will create an index on the geometry column.

SELECT AddGeometryColumn('public', 'table2004', 'longlat', 2272, 'POINT', 2);

CREATE INDEX idx_table2004_longlat ON table2004
USING GIST (longlat);

Next, the data is loaded. I had exported the data table to a csv file, then used VIM to surround each line with an INSERT INTO statement so it looked like this:

INSERT INTO table2004 (week, month, eventdate, case_no, bg_lat, bg_long) VALUES( 1,'JUL','07/01/2004', 6666097,40.382649,-79.803297);

This loads the data into PostgreSQL. The next step is to actually create the geographic data.

4. Add geometry (points) data to datatable, including transformation from degrees to feet.

Now that there is a geometry column, and the lat/long information in the database as a source, the point needs to be added as a point. This is done through an UPDATE statement, using the PL/pgsql function "transform" "PointFromText". Note that the SRID that corresponded to a latlong representation was 4269. Also note that POINT is expressed in x, y. So longitude comes before latitude.

UPDATE table2004 SET longlat = transform(PointFromText('POINT(' || bg_long || ' ' || bg_lat || ')', 4269), 2272);

Now, this is done, and the table can be viewed using a GIS viewer that can connect to a spatial database.

Some of the data is messy, in particular, I was not able to get lat/long for all the addresses. So I created a View that removed these from the result (or I get a bunch of points at 0, 0

CREATE OR REPLACE VIEW table2004work AS
SELECT table2004.week,table2004.month,table2004.eventdate, table2004.case_no,table2004.bg_lat,table2004.bg_long, table2004.serial2004, table2004.longlat
FROM table2004
WHERE table2004.bg_lat > 1 AND table2004.bg_long < (-1);

ALTER TABLE table2004work OWNER TO userid;
COMMENT ON VIEW table2004work IS 'Remove 0,0 data from map';

So this provides a cleaner dataset, and it is also accessible from a GIS viewer.

5. Optional: export PostGIS to ESRI Shapefile if needed.

To export from the PostGIS database to an ESRI Shapefile (if it needed to be given to someone who only need to look at it), the PostgreSQL database should be running. Then pgsql2shp can be run. This should be done after creating a data directory for the shape file to exist. From the terminal, cd into the target directory and run

pgsql2shp -f table2004working.shp datatable table2004work

This creates a shapefile with .shp, .dbg and .shx files. Note that you may need to create a .prj file as well, but this can be done from within a GIS viewer.


Note: Much of this was found through the great documentation available at Refractions PostGIS site and the Boston GIS website, which has multiple tutorials on PostGIS and PostgreSQL.

[Edit]  In the 'srtext' field, the PROJCS identifies the SRID as belonging to a 'projection' which is a curved surface put on a flat plane.  When there is no projection, it is a pure GEOGCS (geographic coordinate system) and it is in degrees lat/long

Represent (The Open Blog at the New York Times)

Represent (from Open: All the news that's fit to printf() )

More than eight million people live in the five boroughs of New York City. They have more than 150 elected legislative representatives, from the local level to Congress. Keeping tabs on the people who represent you is a difficult task.
Represent

That’s the idea behind Represent, an interactive feature we launched in beta last week. Using your address as a starting point, Represent figures out which political districts you live in and who represents you at different levels of government. It draws maps that show how where you live fits into the political geography of the city. And using information collected from around the Web, it presents a customized activity stream that tracks what the people who represent you are doing.

Hat tip to Paul Ramsey (The Clever Elephant) for pointing to this.

I've been teach myself the use of Geographic Information Systems (GIS). Specifically, the use of spatially enabled database. The general idea is take a dataset that includes locations (such as addresses) and display and analyze the spatially. The display part is actually pretty easy. From a list of addresses there are many tools to get a longitude and latitude, then create an ESRI shapefile. I've been using uDig to view shapefiles.

But the real goal is to do analysis. You could link a GIS viewer with a database to view items based on some parameter. uDig, QGIS, and ArcView all have this capability. But the real way to do this is to connect this to a spatial database so you can use all of your standard reporting tools. And you could produce reports based on any set of criteria (such as buffers or overlays) Commercially, the provider of GIS tools is ESRI with their ArcGIS family of systems. And if you want to do more then just look at the data, there are a very few database management systems that could handle spatial data. Oracle, Informix, DB2, SQL Server (I think the very latest version has spatial capability), and PostGIS. For the commercial systems, they don't come fully spatially enabled out of the box, you have to buy the more expensive versions. PostGIS is the exception. It is freely distributable. Which is important if you are just exploring and don't have money. Or you are looking at applications that are for someone who does not have money.

Anyway, it turns out the technology team at the New York Times has been doing a variety of projects with the website, using technology to provide many different visualizations of the information (not just stories) in the New York Times. Some of the good ones are the Faces of the Dead project, which provides stories about individual servicemen and women killed in Iraq, and the Election Results Map, which graphically displays the election results in a variety of different ways, which tells a story much differently then you can using only words.



The Represent project is another project. It takes an address, then determines what districts you live in (local, state federal) and who represents you. Then it goes and searches the New York Times, and a variety of data sources to let you know what they have been up to, how they have voted recently, and what is coming up (in their public schedule. And the tools they use was driven by the fact they needed to use an open source (freely available) spatial database, which means PostGIS. (They use GeoDjango web framework to build the website, which includes extensions to Django web framework that allow the Object Relational Mapper (ORM) to work with spatial data fields.

Why is this cool? Well, it is what some call a mash-up, it is a way to re-purpose and reuse data, in this case news articles. And in the case of the New York Times, create something that cannot exist anywhere else, to deliver timely, local information. Just what a newspaper is supposed to do.

PSO: American composers and a new day

(also can be found at http://pittsburghsymphony.blogs.com/outside/)

This weekend's concert featured Gabriela Montero, fresh off a little Washington DC performance of Air and Simple Gifts in a small venue of 2 million people and a worldwide TV audience. It promised to be something interesting.

Bookended by Berber and Mendelson, Ms. Montero played Rhapsody in Blue by Gershwin and the live performance premier of Air and Simple Gifts by John Williams.  Rhapsody in Blue is probably best known for being played in commercials. Air and Simple Gifts, for its place in the recent 2009 U.S. Presidential Inauguration. For both, Ms. Montero made hearing these pieces live a treat. If your concept of Rhapsody in Blue is the driving theme of a TV commercial, the PSO performance presented many sides to the theme. From the hard driving we're used to hearing, to a light dancing on piano, to strings and horns playing off each other, and winds taking the theme exposed on their own. There is even room for a banjo. And listening to a piano improvise her parts is a treat, especially when someone like Ms. Montero, whose a skilled improvisationist, is the one on stage.

Air and Simple Gifts is also transformed in a live performance. While on TV watching the inauguration it sounded like a straight copy of the Copland Appalachian Spring, live shows many more layers that seem like mere rumbling on the televised version. You can hear the counterpoint in the strings as clarinet and piano take on the main Simple Gifts theme. The piano part becomes an integral weave throughout the piece instead of a murmur under the strings and wind.

I was amused at the program's description of George Gershwin in relation to his contemporary classical composers. It describes Gershwin as a jazz composer dabbling in classical composition. Certainly he is better known for his jazz and folk-style music, along with his many musicals. But this description also fits another composer, who is better known for writing music that is played in a popular culture setting, whose tunes also embed themselves in popular consciousness, setting the theme for a work, or even an event. John Williams, composer for film, and Presidential inaugurals. The comparison is greater when one looks at the criticisms of the piece in various newspapers (not about the playing, but the piece itself). Similar to Gershwin, John Williams is best known for writing music for a popular setting, in his case movie music. His music sticks and conjures up strong images
in your mind. What images do I here in Air and Simple Gifts? There is a quiet beginning of a new day, then waking up for the work to do with the Appalachian Spring theme with the clarinet. And the rest of the piece giving that sense that there is much work to be done by all, good work, but lots of it. And for this piece, at this time, that sounds about right.

A concert with modern American pieces just fits this week, with an inauguration just behind us. And it seemed to show the contrast between a brasher, more cocky America of the not-too-distant past, and one that has many challenges ahead in a new day. It made for a worthy performance. Of course, there was the also timely improvisational encore at the end of the first half, when Ms. Montero took requests for a theme to play off of. And what followed was many spirited variations on 'Here we go Steelers, here we go!' Very timely indeed.

Massaging text with Vim

I'm teaching a database design class this semester. And while I could do most of it using a database I already know (and I have been doing the samples so far using SQLite. I'm sure just about all the students are using MS Access), I'm also taking the opportunity to learn PostgreSQL. (I'm using PostgreSQL because I have a project that will require a spatial database at some point. And the only real options are Oracle and PostGIS, an extension of PostgreSQL. Since funding is at a premium, PostGIS it is.) PostgreSQL is a bit of a step for me, because it is a server-based database, and requires administration. So stuff to learn.

The other thing I need to learn is how to get data into it. MS Access, SQLite and Derby all have mechanisms for pulling in your average CSV file. But the general way that databases are populated (and backups made) is via SQL files. But data does not come in SQL files, they come in delimited text. So the way to do this is take a text file, then build SQL around the data.

The procedure is straight forward. After the tables are created in the database (using CREATE TABLE statements) you just read in a text file (usually with a .sql extension just so you can recognize it when you see it again). And the text file is not hard either. It starts out as
INSERT INTO tablename(fieldname1, fieldname2) VALUES ('value1', 'value2');

Now, it is not too hard to convert the delimited text into the 'value1', 'value2' format using a bunch of search and replace operations (and accounting for things like escaped "'" and the like). The trick is getting the rest of the text around it. It could be as simple as a cut and paste. Multiple pastes. But some of these files had 100000s of lines. That is a lot of [CTRL]-[v]

Like a lot of people whose fortune is it to program, I've used a number of text editors to write code. Microsoft Visual Studio, Eclipse, JEdit are my current set. But one editor that I've always kept around is Vim. And it is for things like this I do so.

So, I have something I need to do for a few 100000 times. One way to do this would be to write a script. Not too hard, read a line of text, use it to build a new line of text with all the SQL in the proper place. Write a file. But it is a bit of overhead with having the interpreter running and managing the file handles, etc.

But another way to do this is just write a macro or script inside the editor. But this would not be fun in Visual Basic. Vim, like MS Word, has a macro recorder. Basically, it records a bunch of key strokes and allows you to replay them however many times you need to. So I start by doing one line. Then I put the INSERT INTO tablename(fieldname1, fieldname2) VALUES ( on its own line at the beginning of thie file (before the first line of data). "ayy puts this into a memory register 'a' within Vim (not the system clipboard. I tried this, but it is much slower.) Now the magic starts.

q1 (starts recording in register '1')
J (joins the following line with the current line)
[CTRL]-[a] (moves cursor to the end of the line and switches to insert text mode)
); (finishes off the line of SQL)
[ESC] (ends insert text mode)
j (moves down one line)
^ (goes to beginning of current line)
"aP" (inserts contents of register 'a')
q (stops recording the macro)

Now, to use this, I test this a couple of times to make sure it does what I want.


@1 (replay macro '1')

Looks good.

100@1 (macro '1' 100 times)

Or

500000@1 (macro '1' 500000 times)

and go to bed.

PSO: John Adams looking through history

I have not been to a concert in almost a month. It almost seems like going through withdrawal. And soon some reminders started coming. My friend Mimi let me know that she was playing in the pre-concert program. And my reply was, of course, I'll be there.



AppalAsia is one of those experiments that happens when musicians from different backgrounds decide to try something different. In this case traditional chinese instruments along with Appalachian folk instruments. Mimi played the Erhu (chinese instrument with bow drawn across strings), Jeffwas on the lap dulcimer and Sue played banjo. What came out was something like jazz, where the players listen to each other and adapt. Each hearing the others play, reacting to the quality of the music each was producing and adapting their own playing to it. It would be unfair to call the music chinese, or Appalachian, it was a hybrid of the two, done in the traditions of jazz.



Prior to performing each piece, composer/conductor John Adams introduced the pieces. There were two pieces, selections from the opera Nixon in China, and the Dr Atomic Symphony. What does it mean for a composer to introduce his own work? We get to hear why the piece was written, what he was thinking when he wrote the piece, and the flow of the piece. So, for Nixon in China we know the attitudes of the principle parts, the setting of the Nixon's arriving and being greeted, how Nixon is not paying attention while in the greeting line, of Mao reminiscing, of Chiang's imperiousness, and Chou's introspection.

Before Dr Atomic, John Adams introduction gave us something more, he let us know what voices various parts of the orchestra were taking on. The aggressive General, the Tewa Indian maid, the panic of the storm, and Dr. Oppenheimer's anguish over what is being unleashed on the world.

Why the story? Most of the pieces in the classical canon are not so tied to recent history, even the recent history of when they were written. In John Adams case, not just with these pieces, but with topics such as terrorist attacks on cruise ships and the 2001 World Trade Center attacks Adams directly referred to the potential reaction to these sometimes raw topics in his introduction to "Nixon in China" as being an "opera for Republicans and Communists." But he also said during his introduction to Dr. Atomic Symphony was that he writes about the history he has lived, and these were the big events, the cold war with its specter of atomic death and the rise of terrorism, the interconnectedness of globalization, the rise of America's impact on the world, and the times of transition we are in now as others have grown in the world America had nurtured.

I wonder if it is an American trait to do this sort of introspection. In Nixon in China the third act has the Nixon's talking about the most mundane aspects of being witness to history, while Mao and Chiang focus on how they overcame hardship in their struggle in the birth of Communist China. And our artists write works such as this, casting the mighty and powerful as dealing with the mundane, a contrast to the recent grandour of the presentation of the last Summer Olympics in Beijing. (not that there are not over the top representations of America, but the American desire to look at the greatest of figures as mundane and earthy seems different.)

My wife has remarked that living life together is like living a history book, touched by war, natural disasters, the thrill of a political campaign. And it is history from a mundane level, not the glorious stories of heroes (although there are many such), but people, responding in the positions they are in, in the good and the bad. Like John Adams, in my own little way, I appreciate the opportunities to remember and record the history around us. And maybe tell stories in years to come.

Data Analysts Captivated by R's power: New York Times

New York Times article by Ashlee Vance

This article is about the open source statistical environment, R (http://r-project.org).  It is an implementation of the S language created by John Chambers for the purpose of data analysis.  There is another implementation, the commercial package S-Plus, generally considered to be number three among statistics packages (after SAS and SPSS).  

The fact that it is open source means one thing, that people can inspect the source code, determine its correctness, and add to it to improve the implementation.  In practice, R has become a playground for statistics researchers.  Because they can inspect the inner workings of R, they know every little step being done, and they can make it do exactly what they want.  Generally, people won't change what is already there without very good reason, that is justified to the maintainers.  And numerous test suites exist that ensure the integrity of the code (at least the core parts).  Many packages exist for R, in many cases written by academics who release the package to go along with papers they have published and books they have written.  These are also heavily tested, as the authors stake their academic reputation on these packages (and the distribution of the packages makes it much easier to sell their books and encourages people to read and cite their journal articles because it is easier to use the methods when there is already software readily accessible.).

One thing that open source software also attracts are critics.  Especially when there is a commercial competitor.  The article quotes people from The SAS Institute, the top statistical package around.  One quote is:

SAS says it noticed R's rising popularity at universities, despite educational discounts on its own software, but it dismisses the technology as being of interest to a limited set of people working on very hard tasks. 

"I think it addresses a niche market for high-end data analysts that want free, readily available code," said Anne H. Milley, director of technology product marketing at SAS.   She adds, "We have customers who build engines for aircraft.  I am happy they are not using freeware when I get on a jet."

In this case, both paragraphs mislead the issue.  What open source software attracts are people who (1) need to know exactly what the software is doing and (2) have needs that were not apparent to the writers of the software.

In the Ms. Milley's comment, the question is, do the people who build engines for aircraft know more about what the software should be doing, or does SAS know more about the software should be doing.  If the engine designers know what the software should be doing, maybe they should be the ones writing it (and testing and validating and verifying).  The SAS code is a black box, to people who are very smart and do not need or want a black box.  In particular, the acceptability of numeric code should not depend on how much you paid for it, but on the testing and validation done.  And this is done through inspection of the code or a test suite.  R's code and its packages can be inspected.  SAS's code cannot.   And the test suite may very well be freely available too.  Because the researchers who initially developed the methodology had to prove its correctness in the open to the academic community who peer reviewed the work when it was first done.  And nowadays, that work was first done in R.

The second group are people who know more about the subject matter then the commercial software builders.  First a digression.  Most of the people who have written code for R are academics and researchers in statistics (academic or corporate).  One of the obvious contrasts is S-Plus, the commercial implementation of S.  Most of the programmers who write S-Plus have backgrounds in computer programming.  So the result is S-Plus is generally regarded as faster and makes better use of resources.  But methodology gets developed in R first and the methods are more correct, because the subject matter experts actually wrote the code.  And this is true across the board.  Many niche areas, the methodology is written in R, because the subject matter is too small for the mainstream statistical programmers at SAS to put their time into.  And this is in addition to the "high-end" uses that SAS has.  Because the people at SAS don't have time to learn the nuances to every use that requires statistical environments.  Or the subtleties of every application.  They have to program to the mean, and to people who only want a black box that only spits back numbers.

What is the biggest obstacle to open source software acceptance in numeric uses?  The requirement for certifications.  SAS has the money to certify their product for use in regulatory purposes.  What remains is the need to certify the environment around the statistical package, which companies that are involved in regulated activities must then do.  And some of the work for those who use R has been done as well in the document R: Regulatory Compliance and Validation Issues - A Guidance Document for the Use of R in Regulated Clinical Trial Environments http://www.r-project.org/doc/R-FDA.pdf.

My own involvement?  I once wrote some code in an R project, R-GLPK.  It provided documentation and examples for the use of a linear programming package from within R.  Why?  Because it was conceivable that people performing data analysis would, in the midst of the analysis, use linear programming to produce intermediate results.  Or that an analysis may use intermediate results as inputs into a linear program.  Or that R may just happen to be the platform other work was done an now someone would need to solve a linear program. Or any of a multitude of things.  That you don't go to SAS for.  So an open source package that connects to something that does something well just made things that much easier.

Latodami Nature Center, North Park, PA January 1, 2009

2	Downy Woodpecker
1	Hairy Woodpecker
3	American Crow
2	Black-capped Chickadee
2	Tufted Titmouse
3	White-breasted Nuthatch
2	Song Sparrow
1	White-crowned Sparrow
3	American Goldfinch

The Day of Battle: The War in Sicily and Italy by Rick Atkinson.

The Day of Battle: The War in Sicily and Italy, 1943-1944 by Rick Atkinson

My review

rating: 4 of 5 stars
Atkinson follows his previous work "Army at Dawn" (about the Allied invasion of North Africa in WWII) with the account of the Allied invasion of Italy that followed. The book focuses on the growth of the American Army in both number and experience. After North Africa there were 4 combat experienced American divisions. With green divisions arriving in Europe in preparation for the invasion of Western Europe that was sure to come, the Americans had the task of learning how to wage war. And that meant learning lessons in North Africa and Italy, then passing them on to the soldiers preparing in England.

Atkinson takes a series of campaigns, and looks at them from the point of view of the commanders and the soldiers. The decisions that were made out of caution or aggressiveness. Soldiers who had their first experiences of brutality, the mistakes that lead to either defeat, death or mutilation, And the plain hardness required to survive. You get the feel of desperation, from general to private, and sort of get what motivates them to do things that sometimes look outright stupid and self-destructive, because they had to act on what they knew, not what they did not.

Atkinson also goes beyond looking at the British and Americans, but also touches on the colonials, the Free French from North Africa and the British colonies (India, Gurkhas, etc.) the fact that he touches on them makes this seem that much more comprehensive.

I liked the fact that Atkinson does not present anyone as heroes or demons. Leaders were not presented as merely geniuses, idiots, or gloryseekers. Individual soldiers were not merely heroic or monsters, but everyone there had conflicting motivations, abilities and pressures. And all trying to function in the chaos of war.

View all my reviews.

Thank You, from the American Red Cross

From the American Red Cross

A Handpicked Team for a Sweeping Shift in Foreign Policy - New York Times

A Handpicked Team for a Sweeping Shift in Foreign Policy by David Sangler, New York Times

President-elect Obama today introduced a set of cabinet appointees for positions involved in national security. Among them are Sen. Hillary Clinton (a rival candidate for the Presidency), Mr. Gates (current Secretary of Defense under a Republican Presidency) and Gen. Jones (USMC, ret.) The range of reactions has reached a bit of predictability. Progressives/liberals complaining that these are people who have been working in this field and therefore do not represent change. The Republican National Committee talking about how these picks are taking their places alongside the Four Horsemen of the Apocylpse (I'm only slightly facetious here). And various greybeards observing that President-elect Obama has a pattern of picking pragmatists for all of his policy positions. And Obama supporters who have observed that the entire Obama campaign was based on pragmatism planks over ideology.

But looking at those senior picks of Jones and Gates, neither of whom is there any reason to believe are Obama loyalists, the question is how can they work together. All of the picks (to include Sen. Clinton, Dr. Rice and the others) have strong personalities and definitive beliefs. President-elect Obama has stated that he intends his senior leadership meetings to be one of vigorous debate, in stark contrast to the current policy of removing all cabinet members that showed dissent. But it is also observed that people who disagree all the time don't work together all that well.

It turns out there is a unifying theme among the top nominees. The theme are the elements of national power, described as DIME (diplomacy, information, military, economic). Gen. Jones and Secretary Gates have been heard on many occasions advocating the need to strengthen the non-military elements of national power. Sec. Gates going so far as to say he would willingly give up part of the budget of the Department of Defense to strengthen the Department of State. (and reminding JCS Chief Admiral Mullen he made similar statements when he was PACOM). This is also the driving factor of candidate Obama's national security platform when running. So, essentially, the top-level nominees and the President-elect (with the exception of Sen. Clinton, whom I do not know about) have each independently advocated the same change in the overall approach to the national security of the United States. For all their differences (and it is expected that intelligent, capable people have differences), the fact that they have subscribed to the same paradigm makes the rest details.

Roger Ebert's Journal: Death to film critics! Hail to the CelebCult!

Roger Ebert column at the Chicago Sun-Times, November 26, 2008
Death to film critics! Hail to the CelebCult!

Roger Ebert is the movie reviewer at the Chicago Sun-Times, made famous by being part of the movie review team of Siskal and Ebert (later Ebert at the Movies). This column talks about the art of being a movie critic. It's trigger.

The crowning blow came this week when the once-magisterial Associated Press imposed a 500-word limit on all of its entertainment writers. The 500-word limit applies to reviews, interviews, news stories, trend pieces and "thinkers." Oh, it can be done. But with "Synecdoche, New York?"

. . .

Worse, the AP wants its writers on the entertainment beat to focus more on the kind of brief celebrity items its clients apparently hunger for. The AP, long considered obligatory to the task of running a North American newspaper, has been hit with some cancellations lately, and no doubt has been informed what its customers want: Affairs, divorces, addiction, disease, success, failure, death watches, tirades, arrests, hissy fits, scandals, who has been "seen with" somebody, who has been "spotted with" somebody, and "top ten" lists of the above. (Celebs "seen with" desire to be seen, celebs "spotted with" do not desire to be seen.)

So, what is a film critic for? Is it just to tell about the gawking of the actors and actresses who play the character on the screen? In another forum, one of my book reviews is being severely criticized, because I have stated that my primary criteria to judging a book is the believability of the characters (after I grant to the author the scenario and situation), not the politics of the book. Ebert says this much better.

Why do we need critics? A good friend of mine in a very big city was once told by his editor that the critic should "reflect the taste of the readers." My friend said, "Does that mean the food critic should love McDonald's?" The editor: "Absolutely." I don't believe readers buy a newspaper to read variations on the Ed McMahon line, "You are correct, sir!" A newspaper film critic should encourage critical thinking, introduce new developments, consider the local scene, look beyond the weekend fanboy specials, be a weatherman on social trends, bring in a larger context, teach, inform, amuse, inspire, be heartened, be outraged.

I could only hope to move in the general direction of that goal. And mourn every sign of its passing that comes my way.

PSO: Ohlsson plays Beethoven

Last week, Mr. Ohlsson was featured in New York Times review of a recital in New York City. Reading this piqued my interest, and I was looking forward to this Sunday's concert.

With the beginning of Beethoven Piano Concerto No. 4 seemed to start different. Instead of the usual build from the orchestra, Mr. Ohlsson introduced the theme on the piano. And the exposed introduction on the piano was followed by the richness of the full orchestra. For the rest of the piece we were treated to impressive piano, intricate, delicate and amazing fingerwork. Well contrasted with the fullness of sound in the orchestra.

I have a belief that people and roles are not interchangeable. Which is a little strange, since my profession is one that is forced to work with people who are anonymous (except perhaps in terms of skill sets.) But even if from an abstract level we can work as if people could be interchangeable, at the point where something actually happens, it always seems different. People working alongside each other seem to adjust to each others strengths and weaknesses, and even if the job titles are fixed, the actual jobs seem to be just a little different from one team to another.

And Ohlsson playing Beethoven shows the uniqueness. It never seems like the piano and orchestra are ever competing, because the orchestral parts display a richness that is hard to imagine coming from a piano, and the piano parts have intricacy and detail that is hard to imagine coming from the multitude of voices of an orchestra. It was a wonder and a joy to behold.

Book meme: Head First Java

The rules:

* Grab the nearest book.
* Open it to page 56.
* Find the fifth sentence.
* Post the text of the sentence in your journal along with these instructions.
* Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

From Head First Java 2nd ed. by Sierra and Bates

"But when you're talking about memory allocation issues, your Big Concern should be about how many objects (as opposed to object references) you're creating, and how big they (the objects) really are.

Veterans Day 2008

Sargent Mackenzie

Book Review: The Audacity of Hope by Barack Obama

The Audacity of Hope by Barack Obama

My review

rating: 5 of 5 stars
The Audacity of Hope is not a biography, it is a discussion of politics in the United States. It is not a denunciation of the current political system, but a description of it and how and why it developed the way it has.



Obama walks through a number of topics, the two-party system, values, the constitution, politics, opportunity, faith, and race, and writes about how each is related to the political process. He discusses the actual issues that are involved, how various groups within the United States view the various sides of the issues, and he discusses on how these issues are addressed during political campaigns.



The theoretical part is instructive. Among other things, Obama taught constitutional law in one of the top law schools in the country. That is not a qualification in itself, but he notes that it exposes him to intelligent people who will challenge details of his understanding, and force him to clarify the corners of what he believes. The chapter on constitutional law is probably one of the better descriptions of the views and implications of the strict constructionist view of law, and why Obama opposes it (Strict construction implies that the law should be taken as written. The opposition arguement "judicial review" is the law (especially founding fathers) tells us how to think, not what to think, the writers of the constitution actually disagreed on many things, so the concept of "founders intent" in "what to think" is meaningless. Since the founders left behind considerable commentary (paralleled by the congressional record in the case of laws), the commentary can be used in understanding "how to think" and give an understanding of how the law should be applied to the situation at hand.) )



The more practical components stem from his experiences on state and federal level campaigns (Illinois and U.S. Congress) and his experience at the community level as an organizer with a number of churches on the south side of Chicago. It is highly practical, how politics work at street level. It is probably one of the more comprehensive explanations of why special interests are so important to political campaigns, especially as the area covered by the campaign gets larger.



His main point about special interests is they provide two things, money and political workers on the ground. A normal candidate cannot provide these things by him/herself because a mere candidate base of support starts as local, where the candidate has been working in the past. So all political candidates work with people who can provide large sums of funds, and motivate people who would be willing to speak on behalf of the candidate in various parts of the country, and to their neighbors through phone banks and canvassing. For the Democrats, the main groups that provide this manpower are labor unions, environmental groups, and prochoice groups. For Republicans, the key special interest groups are the religious right, local chambers of commerce, the NRA and anti-tax organizations. And the messages are "spun" by media in this context.



So it is instructive on how this understanding of the political process played out in the campaign. Not just how the sides were taken (the Republican advocacy of Strict construction, the Republican use of Gov. Palin to keep the key special interest groups involved in the campaign, the McCain/Palin emphasis on not raising taxes.) but how Obama structured his campaign. The emphasis on small donations rather then large donations. The refusal to take money from lobbyists or allow lobbyists to take positions in the campaign. The heavy recruiting of volunteers not associated with unions, environmental or prochoice groups. So if a lobbyist comes to advocate a position, that lobbyist cannot say that the group he represents provided the decisive contribution to the Obama campaign, in either money or workers. The lobbyist must advocate the issue on its face.



Audacity of Hope is a primer on the U.S. political system, both as theory, and how it actually works and why. It also provides a framework to look at the American political system at work. And, it provides thoughts on how the more self-destructive tendencies can be mitigated. The next question is how this works in practice.


View all my reviews.

Book Review: Faith of My Fathers by John McCain

Faith of My Fathers: A Family Memoir by John McCain

My review

rating: 3 of 5 stars
Faith of My Fathers is the book about the making of a military officer, whose author was preparing for the 2000 Republican Presidential primary campaign. McCain talks about his father and grandfather, both successful Navy officers (an Admiral who became CINCPAC, and a submarine Captain).



McCain discusses his relationship with the Navy in the context of this family. The fact that as a young man, he was not so serious about the Navy, that it started out looking like just a job. In this, he acknowledged that he probably disappointed his family as well as others who cared deeply about the Navy as an institution.



He writes that his sense of duty and seriousness grew as he served and gained responsibility. First, with his midshipman cruise, where he went to sea on a deployed ship. Then as he had multiple tours of duty on a carrier.



With all of his mentors, and experiences, he makes it to the second part, his service in the Vietnam war then being a prisoner of war of the Vietnamese. With his now maturing sense of duty to country and fellow servicemen, he undergoes interrogation by the Vietnamese, and the offers to break faith with his fellow prisoners or for freedom in return for false statements. And he intimidates that it was his developing sense of duty and honor, that was not present at the beginning of his Naval Academy times, that gave him the strength to bear it, and return with honor.



In it essence it is a story of how a pattern of dealing with adversity and responsibility turned into strength of character, in time to face a test of character. It is a very deeply personal story. It suffers from this degree of personal intimacy, because it lacks generalization. This may have cost him during the 2000 campaign, when political supporters of George W. Bush spread rumors that his time as a POW, detailed in the book, actually made McCain a "Manchurian Candidate." While the Bush supporters campaign was very efficiently run, I would have wished this book could have been a better foil to the 2000 smear campaign.



The deeply personal nature of this makes it different then the books related by Afghanistan and Iraq veterans (military and otherwise) that I have read. Granted, it is because I tend to read the books that are closer to my experience, but most of what I have read deals with the need to work with the locals, the difficulty of working with people who were not open and honest with you. Of knowing that even people who were in fact supporting you, could not be open with you because of the hazards they faced when you walked away for the day. Because of this, the current generation writes more about having to depend on each other, on those they trained with, shared their struggles, and the lessons they all learned together. Yes, all learned much about themselves, but even that was done in the context of others in service. Faith of My Fathers almost suffers because it focuses so much inward.


View all my reviews.

The Obama Ground Game, from the ground

[these are notes, so someday I can tell my grandkids what it was like as part of the Obama campaign]

I started following the Obama campaign during the primary season, in the days leading up to the Pennsylvania Primary. A Democratic Primary campaign that was expected to be a lock for Sen. Hilary Clinton had turned interesting, with the first two contests split between Sen. Clinton and Sen. Obama. Sen. Clinton's big push was against Sen. Obama's inexperience, proclaiming that when the primaries were over, Sen. McCain and the Republicans would launch into a no-gloves smearfest, and Sen. Clinton could handle it while Sen. Obama could not. She predicted that the Republicans would leave no stone unturned, or distorted, to make these attacks. And since the Republicans have been smearing her for 10 years, the Democrats already knew she could survive.

As part of these attacks, Sen. Clinton's campaign brought up the Reverend Jeremiah Wright. A veteran, with a tour in the Marine Corps, then a tour in the Navy during the Vietnam war era, Rev. Wright was Pastor of Trinity United Church of Christ, which is where Sen. Obama accepted the Christian faith and became part of a christian community. Rev. Wright was also known for his fire and anger at the United States with all of its injustices. And Sen. Clinton attacked Sen. Obama for this.

Sen. Obama had two options. He could ignore the attack, hoping it would not stick. Or he could forsake his pastor, denouncing Rev. Wright's views. He decided something different. He acknowledge the hate of his pastor, then denounced it, stating that it acknowledged truth, but denied what America could be. And, in Philadelphia, he presented a vision for an America that could be ("More Perfect Union"), completely in keeping with the themes of his political life.

I ended up voting for Obama in the primary, then connected with the campaign. Months later, as Sen. Obama began winning more primary contests, and the Sen. Clinton campaign died down, the Obama campaign in Pennsylvania started up again, looking towards the national election.

I live in the North Hills, affluent north suburbs of Pittsburgh. The area is heavily conservative. In addition, the entire southwest Pennsylvania region is striking in its insularity and lack of diversity, visible to anyone who has to work with the region as a whole (my experience is with public safety and emergency services). And with a multi-racial candidate, this was shown to be an issue with whisper campaigns and unattributed e-mails that circulated throughout the primary season.

I first started with voter's registration in Cranberry, then in the North Hills. The local field organizer for Butler County (a student from California taking a semester off), as part of our introduction, explained that it was never expected that Butler County would vote for Obama. But the goal was to cut into its dominance, and let the Butler Republican Party know that the Democrates would work in Butler County. The field organizer for the North Hills (recent graduate, taking time off before starting a teaching job in PA where he grew up) said the same. For Pennsylvania, it could be expected the Democrats would dominate Philadelphia and the city of Pittsburgh, the Republicans would dominate central PA. The question would be the ability of the Democrats to develop support in the Philadelphia and Pittsburgh suburbs. And as campaign volunteers in the north suburbs of Pittsburgh, we would see it all.

Voter's registration went on during the remainder of the summer. It was good natured, especially in dominantly Republican Butler County. We probably registered as many Republicans as Democrats in Cranberry. But the point was to make a presence, and let people know that Democrats were not stand-ins for the devil (which was a serious issue there.)

By late-July, the North Hills team was started with a field organizer who grew up in the North Hills, and a few couples who would serve as volunteer organizers, dividing the six townships in northern Allegheny County. We continued voter's registration, but as Obama reached the point that he won the Democratic nomination, we started calling voter's and canvassing (knocking on doors), asking them about their views on Obama and McCain, recruiting volunteers from supporters and talking to Clinton supporters, encouraging them in their support of Clinton and the Democratic Party in the runup to the Democratic National Convention. And we laid the groundwork for a bigger organization to come. From a group that started in late July at about 20 volunteers for the six townships in the North Hills, it was expected to multiply ten-fold after the convention.

Right after the convention, and after going to a rally in Beaver County with Obama and VP-Candidate Sen. Biden, I left for three weeks to volunteer in Texas and Louisiana in response to Hurricane Gustav and Ike. The volunteer coordinator for Franklin Park/Marshall teases me for being so lazy to leave the campaign in September.

When I returned, the campaign was a buzz of activity. The campaign had a North Hills office, right off McKnight Road (the big commercial road for the North Hills) There were always volunteers there on the phone, with extra ones for the evening. There were now two additional staff. Of course, this was after the Republican Convention. With the standard convention bounce, polls considered the race even. Which was expected as Sen. McCain was the only Republican who was different enough from the Republican party to have a good shot at the middle. And while the conservative wing of the party, especially those who spoke for evangelical christians, openly disliked him, having Sarah Palin as candidate for VP gave them cause to embrace and support the Republican candidates. With so many volunteers, I elected to canvass when I had time, because I was comfortable with going around, knocking on doors of strangers, even if I was alone. Because I figured the North Hills of Pittsburgh did not compare to Afghanistan or a disaster area.

Knocking on doors had lots of responses. There were the houses with Obama supporters who asked "are there other Obama supporters here?" The houses where people said "good work" and "good luck". Old ladies who were so delighted to have someone that came to talk to them. People who were delighted to have a Presidential campaign contact them and wanted to help in any way. Veterans who did not support Obama, but when learning I was in Afghanistan, we would greet each other as brothers. There were people who wanted to talk about health care, or taxes or the economy. We would usually have a few fact sheets and specific issues. And the Obama-Biden campaign would posts videos on YouTube about issue areas as well as current campaign speeches by the candidates and supporters around the country. One day Caroline Kennedy Schlossberg (daughter of John F. Kennedy) came by to give us a pep talk and join us canvassing. And so many people who asked "do you think he (Obama) has a chance?" (my reply "he has a chance, but we have work to do.") The most memorable conversations were a few mothers of soldiers or soon to be soldiers. We spent alot of time talking about Afghanistan, and why I thought that Obama would be a good commander-in-chief for their children. Several asked if Obama would get us out of the war. Of course, unlike some candidates, Obama does not promise a unilateral, immediate pullout. But I could tell them that he will use the military appropriately, and he will also bring other aspects of national power (DIME) to bear, not just the military.

And the campaign also posted McCain-Palin ads and recordings of campaign phone calls. The McCain-Palin campaign had made the choice to go almost all negative, to the point where their campaign site had only the briefest of outlines of their position and platform. There were the houses where they heard from church that Obama was a Muslim (Obama is a church going Christian, there is no Muslim in the world who would call Obama a Muslim. If Obama was not Christian, he could not be attacked on the basis of his pastor.) There were those who asked if he worked with terrorists (based on being part of a Republican organized board to improved schools in Chicago. The Republican organizers included a former member of the 60's radical Weather Underground.) Others were told he went to school in a madrassas (He went to public schools in Indonesia. And the ones that wondered if he was indeed a citizen and truly eligible to be President (Barack Obama was born in Hawaii. Sen. Obama had earlier in the year prepared a memorandum that supported that the Panama Canal-born John McCain was considered 'natural born citizen' as opposed to one who citizenship was conferred to ensure McCain's citizenship would not be an issue during the Presidential campaign.) And there were the people who would slam the door in any Democrates face. During this time these views were being promoted on radio shows, automatic telephone calls and email campaigns. All of which the Obama-Biden campaign would make available on the internet so we could address the smear campaign.

By the week before the election, the weekend canvassing was too big to work out of the Obama campaign office, so we staged out of the local congressional campaign office. Then we used staging areas at the volunteer coordinators (three for the North Hills). The weekend before the election, 200 volunteers had canvassed in the North Hills at some point. While McCain was 'guarantee victory', the Obama campaign was preaching "Don't let up" And we did not. With that many volunteers, we were meeting each other all over the place. Over the weekend I took four shifts. I saw three other groups of Obama canvassers, two groups with SEIU (a union that was supporting Obama) and a group supporting Republican Congressional Candidate Hart. And I know that there was at least one group with Democratic Congressional Candidate Altmire around me as well. There were thanks and encouragement from Obama supporters and those who did not want to say who they supported, and heckles from McCain supporters. On Saturday night there was a big volunteer conference call, that included Sen. Obama himself giving a pep talk to the volunteers over the phone. ( *ring ring* this is the Obama campaign. Please hold the line for a few minutes for a conference call with Sen. Obama.)

The last day was election day. After voting, I spent the rest of the day knocking on doors, checking if people had voted. It was just like the old school Chicago precinct captain election day ritual of walking around the neighborhood to make sure people had voted. There were packets of walk sheets that the volunteer coordinator house. People were dropping off food, snacks and drinks for the volunteers, and others were standing by to provides for anyone who needed it.

And then the wait. Everyone had discounted polls talking about 15% advantage for Obama in PA, preferring to focus on all of McCain and Palin campaigning in the area, and our memories were of when PA was considered even. I took the time off to go home and rest. And around 8:30 my wife and I went back to the volunteer organizer's house to watch the results. The next couple hours went from casual acceptance to astonishment. Many of the states were pretty well known. Pennsylvania was the first shock, because we almost did not believe that Obama would be that much ahead to call so early (Obama ended up winning by 10%). We discounted almost every piece of news before 9PM. There was Indiana not being called for either for the longest time. The announcement that Democrats swept the New England congressional races. Obama winning New Mexico. Colorado still being in contention. Then Ohio being called for Obama. And after the unified media had Obama at 200 electorial votes, with California, Oregon, Washington and Hawaii yet to close, everyone knew that Obama would reach 270. The various network announcers were tripping over themselves dealing with the obvious (CA, OR, WA and HI were all going to be for Obama without question) without actually saying it before the polls in those states closed. Of course, everyone else could do the same math, and all the various campaign offices were calling us, sharing in the excitement without actually saying anything because it may jinx it. And then, at 11 EST, the west coast polls closed. And the six media networks (ABC, CBS, NBC, CNN, FOX, AP) all declared that Obama would win those states, and their tally had Obama over 270 electorial votes (even without Florida, Virginia, North Carolina or Missouri) And much gladness, and feeling of a job well done.