Parenting 26 months: For your entertainment, tonight's concert is . . .

This month T has been learning how to use his voice.  And he practices by singing. Unfortunately, he does not come with a volume control.  But he gets lots of points for singing with enthusiasm.

Some favorite performance pieces include Old MacDonald has a farm and Twinkle, Twinkle, Little Star.  But the absolute favorite is Da hai-guxiang.  He has been asking grandma (lao-lao) to sing this for him at naptime.  Now, he wants to do the singing.  And from watching videos he knows that a proper singer needs a microphone, so if he can get hold of any microphone shaped object (markers are the main standby) and an audience he bursts into song.

He also likes reading.  Now, part of the nighttime reading routine is we read a book, but he reads at least one.  Christmas saw the Who is coming to our house? as one he quickly learned most of the words to.  Other books that he reads in part include "Freddie Frog is Hungry",  "Little Pea," and "That's Not My Dinosaur!"   And then there are the books that he reads out loud in full.  "Brown Bear, Brown Bear, What do you see?" and "I am Sam"

My wife has been packing up older toys to give away.  He had a fun time going through the boxes playing with some of them, but he has never been all that serious about toys.  When his older cousin visited for Thanksgiving one of his first comments was that we did not have any toys (slight exaggeration, he is only four).  While he does play, we are far and away still his favorite playtime accessory (followed very closely by grandparents).

Book review of Statistics in a Nutshell 2nd ed. by Sarah Boslaugh

Statistics in a Nutshell by Sarah Boslaugh
My rating: 4 of 5 stars

One of the biggest problems faced in teaching statistics is the gap between learning the methods to actually using them. Statistics classes that are based on learning formulas fail due to the disconnect between learning formulas and the reality that very rarely are these methods used by implementing the formulas that are so painstakingly taught. But learning statistics as a set of steps or functions in a computer package only gets a little further. The real goal should be what methods should be used and why. The how is almost secondary. Statistics in a Nutshell focuses on the what and the why. I would not use this to learn how to perform a technique or its formulas, but this is where to go to understanding how the various methods of statistical analysis should be used and their qualities. It is meant to be read, not just studied, and as such it holds a different place than other statistics texts.

When I first learned statistics, the focus was on learning formulas that calculated various values. But the problems I could work on were only toys, and it took so long that we did not get into much of understanding what we were dong. Now with readily available software packages, the temptation is to focus on the mechanics of implementing a procedure on a set of data and reading the computer output. But software documentation and even books that teach statistics fall into the trap of focusing on how a method works then applying it and not as much on why. Part of this is because of the pressure of having to cover topics, but the fact that the methods are presented in isolation, without their application context so it is rare to grapple with the question of how to know what needs to be done and instead the focus is on how to do it.

Statistics in a Nutshell is the other kind of book. I was taught that for computer programming for any language you wanted a book that focused on methods, but also a book that focused on morals, the why you use a language feature. This is the morals book for statistical programming. You read it, not to learn how to calculate statistical output or implement visualizations, but to think about what method or visualization is appropriate to help understand the data and environment and to communicate those truths to an audience.

Because of the expectation that any course that teaches statistics gives the students a toolkit, this would never be a good book for teaching a course. But in the real world, what is more important is that you understand what these statistical methods are and why you use one over the other. So for the data analyst or a student who needs an overview of everything this is ideal. It would also be ideal for someone who may not have the time to go into detailed study of statistical methods, but needs to interpret the results or work with statisticians and data analysts. This book will help interpret what you get and ask the right questions to both understand statistical results and perhaps point those who are doing the analysis in the right direction so that they are answering the right questions.

Note: I received a free electronic copy of Statistics in a Nutshell through the O'Reilly Press Bloggers program.

View all my reviews

Engineers can read! A book review of Against the Gods by Peter Bernstein

Against the Gods: The Remarkable Story of Risk by Peter L. Bernstein
My rating: 4 of 5 stars

One problem with the teaching of math and statistics as practiced in the U.S. is that it often seems like a series of topics that are sprung out of whole cloth with no context. Against the Gods has two parts: a history of how views of risk developed within western civilization and then a examination of the tools and (mis)use of risk managements in modern finance. In doing so it paints a picture of not only what the principles of probability and risk management are, but why they were developed and how they are used (and misused).

I had assigned this book as part of a course in decision analysis within an engineering department to a mix of upperclassmen and graduate students. Most found the first have to be fairly uninteresting. But the payoff came later as Bernstein tracked the growth of the development of probability to its application in insurance then to financial instruments in general By the end of the book we were discussing the purpose of modern financial instruments in terms of risk management using both modern examples and the 15th century patrons of renaissance explorers. And seeing how not understanding the principles and purposes behind the techniques leads to trouble, many of my students said the book gave a greater appreciation for the probability and statistics they have been learning.

And a gratifying note, in their report, many of the students stated that they did not read outside of their technical books, but after this experience they developed an appreciation for non-fiction and planned on looking for more such books to read in the years to come.

View all my reviews

Book Review: 21st Century C by Ben Klemens

21st Century C: C Tips from the New School by Ben Klemens
My rating: 4 of 5 stars

All engineering students take a one semester course fairly early in their education. My experience as a professor is that most of them promptly forget it and never make use of it again. I get alternately amused and discouraged at the lengths some of the students I work with go to avoid programming again. Because that one semester course teaches a programming language, but does not teach someone how to program. This book is meant to take someone through that step. It is sometimes hard to follow (my heavy programming in C was years ago) but it gets someone to where they can be productive.

When someone is referred to me an claims to have some programming background, I have learned to ask a simple question: have you programmed with libraries? I usually get a blank stare, and I know that this person may have taken a class, but cannot do anything useful. What this is how to make programming in C useful. So it builds in working with the various utilities that make C programming useful (I learned pkg-config and profiling with valgrind with this book. And before I used to be able to edit Makefiles but I would not try to write one from scratch) And, of course, working with C using some fairly useful libraries (if I am programming I am general pulling data from databases and using numerical methods, so I am rather pleased at the use of SQLite and GSL as example libraries).

The various tools are very useful. Programming without learning to use the various utilities such as debuggers, library managers, packaging, etc. is an exercise in frustration (and is how the standard introduction to programming course is set up).

Am I a convert? I still regard Python and R as my preferred environments, and I'm not swayed by his arguments that C is a "punk" language. I still find that all the little cruft such as pointers and their management something I'm glad to avoid. But I also regard one of the best features of Python and R the fact that I can dive into C when I need something fast and efficient. So being able to do this more effectively is good to improve my toolkit.

The book is not really for pure beginners. If you did not already have some background in C, you need to get that somewhere else. I got in trouble with some of the examples, so it helped that I have used many of these tools before (even if not all that effectively). It also is opinionated, presenting one way to do things. (of course, that is very useful to someone starting out, knowing one good way is better than having a dozen options in front of you when you don't know what is what). But it can be very good for someone who needs to go from "I have learned C" to "I know how to effectively use C."

Note: I received a free electronic copy of this book from the O'Reilly Bloggers program.

View all my reviews

Public transit: 2012 update

List of cities/metro areas that I have used public transit

Washington, DC
Chicago, IL
Boston, MA
Champaign-Urbana, IL
New York City, NY
Denver, CO
Vienna, Austria
London, United Kingdom
Singapore, Singapore
Bangkok, Thailand
Los Angelos, CA
San Francisco, CA
Seattle, WA
San Jose, CA
Portland, OR
Philadelphia, PA
Charlotte, NC
Montreal, QC
Phoenix, AZ
Berlin, DE

Lessons observed from using cases in teaching decision analysis

I taught a course in Decision Models last semester. While there was a text, instead of relying on the text for the assignments, I used cases to provide practice in applying the concepts.

I chose two cases from the INFORMS Transactions on Education, Home Depot and the OVA1 case. In addition, I had them read a book "Against the Gods" by Bernstein, and a personal case. The class as a mix of undergraduate and graduate students. All of the undergraduates were US born while the graduate students were more varied in background including US born, Chinese, Indian, Turkish, and one Western European. A few had work experience.

The Home Depot case was a multi-attribute decision making case. I had the students form teams of two and take a point of view in the case: HomeDepot, competitor, the local government, or local residents. While the case writers used one method, I had them structure according to SMART, which is what was used in out textbook and the class. Some notes. The students often had trouble staying in a point of view. This led to some strange arguments during the course of the case report. Also it made the assessment of values difficult for the students, as observed in their discussions and the values they assigned to attributes. Noone had background in the issue, except for one student who witnessed a similar debate in his hometown. They found assigning numbers to values difficult, But one result that I pointed out, also noted by the case writers, was they were remarkably in agreement in their final recommendations, even though their choices of. numbers varied wildly. As noted in the book and by the case writers, this was one of the justifications for the value of structured decision analysis methods in a highly subjective environment. I think that this exercise gave the course validity.

We also covered the eventual outcome. As the case writers expected, they generally came to the same conclusions as the decision in real life. But the eventual outcome s not very good, and in real life it is being revisited, so we could talk about if the decision was good even if the outcome was not as good as hoped for.

The OVA1 case was on medical decision making, and teams could be the doctor, the patient, or an insurance company. There was one person considering medical school in the course, This was used as an exercise in developing decision trees and assigning utility in the face of uncertainty. in addition, this was before they covered the value of information, so they essentially, they were developing the techniques of determining the value of information as they went along. They did much better with keeping with their assigned role this time around. And it took work, but they figured out the expected value of information method, which made teaching it later very easy. Also, the case got across the idea that decisions look very different when looking over a large number of instances (doctor and insurance company) compared to when you looked at a sample of one (patient).

The real question is the use of cases instead of homework. In an engineering department, this would naturally be one of the less quantitative courses. However, I presented it as an integrative course, meaning that the other courses teach methodology that are used to support decisions. Even with the case that was supposed to be based on a personal issue (I gave them some leeway by saying that it could be based on something or someone they personally know instead of just themselves)they commented on how hard it was to find data or determine utility in a quantifiable form. I think that presenting this in the form of cases made them work harder at it and gave them a deeper understanding than if I had them do a homework questions from the book where the numbers were given and they did not have to think about where they came from. I had many people comment in their reports that this class was making them think about many issues differently. (especially the personal case, where most of them even thought to include such considerations of their romantic relationships in their evaluation of decisions.)

Parenting 25 months: I can do it

Here is a drum to work with, a photo by LugerLA on Flickr.

Highlight of the month was Thanksgiving. We hosted my sisters, along with my brother-in-law and nephew. It made for a crowded holiday. It was also a lot of fun as my nephew got to be the big cousin again and we went around to the places we like to play. In particular, we took everyone to the Carnegie Museum of Natural History, where T is a frequent and experienced visitor.

The major development this month is a lot more talking. Like most toddlers, it is mostly repetition. And he has taken advantage of many sources of repetition. In addition to the four of us (two of us + my in-laws) he draws from videos but especially books and iphone apps. Especially the books and apps, because of the repetition. We have been surprised by the vocabulary he has picked up that we could find no source except the iphone apps that he likes to play with.

His favorite books are the Little Pea/Hoot/Pig series by Amy Rosenthal. Actually, we let him do the talking when it comes time to read. He does a pretty good job of getting most of it, with the skipping of words declining weekly. What is most amusing is when he is pretty sure what is happening, but does not know all the words, so he improvises. We know what he means :-)

His newest catch phase comes from Potty by Leslie Patricelli. At the end, after a successful potty engagement, the toddler says "I did it!" Somehow, T has also learned "I can do it" along the way, and has started using it at any reasonable opportunity. There are worse phrases a 2 year old can learn, so we don't mind this one.

Installing Ubuntu 12.10 on Dell Dell Precision WST3500,Xeon (E5640) Nvidia Quadro NVS295

Ok, this is getting way too hard, I better keep track of everything that is going on.


1.  To boot from USB, The instructions on the Dell splash screen (which goes past to fast to note) says to press F12 when splash screen is up to change the boot order.  Now you can boot from USB or CD drive and install Ubuntu.

2.  The boot up will go to a blank screen because the NVIDIA graphics drivers are unsupported.  Hold the shift key during boot up.  Then in the startup script that shows up, look for the line that starts with 'linux'  Find the option 'quick splash'.   Replace it with 'nomodeset' 

Ok, now Ubuntu starts.  But you have to do do (2) everytime unless you can get the NVIDIA driver installed. 

3.  Download the appropriate driver from the NVIDIA website.

4.  You cannot install a new graphics driver while x-windows in on (i.e. trying to do this through the Ubuntu Software Center was a failure).  So get to a real terminal (not just xterm) and shutdown the x-server using the following instructions (Note: I just shut down the x-server using this.  Installing the nvidia drivers comes next)

http://neelmohile.wordpress.com/2011/11/02/how-to-manually-install-latest-nvidia-drivers-on-ubuntu-oneiric-11-10/

4.  Follow the instructions here on installing the driver.  Note that you also need to have the linux-headers installed.  And when I did this, it specified linux-headers-3.5.0-17-generic  (not 3.5.0-18-generic)

  sudo add-apt-repository ppa:ubuntu-x-swat/x-updates
  sudo apt-get update
  sudo apt-get install nvidia-current nvidia-settings

3.1  Don't forget to restart the X server using sudo service lightdm start

Inside Team Romney's whale of an IT meltdown - Ars Technica

Inside Team Romney's whale of an IT meltdown: ORCA, the Romney campaign's "killer" app, skips beta and pays the price

It was supposed to be a "killer app," but a system deployed to volunteers by Mitt Romney's presidential campaign may have done more harm to Romney's chances on Election Day—largely because of a failure to follow basic best practices for IT projects.

. . .

IT projects are easy scapegoats for organizational failures. There's no way to know if Romney could have made up the margins in Ohio if Orca had worked. But the catastrophic failure of the system, purchased at large expense, squandered the campaign's most valuable resource—people—and was symptomatic of a much bigger leadership problem.

This is quickly turning into a how to on how NOT to run an IT project.  Project ORCA was the Romney campaign Get out the Vote platform, developed by Microsoft and an unnamed consultant. (ORCA was a play on the Obama version, NARWHALE, because killer whales are a predator of narwhales)  It was developed in seven months, but more importantly, it was developed in relative obscurity, with volunteers who have computer development experience stating that their briefings were more like rah-rah sessions and technical questions that they raised (like 'was this stress-tested?') went un-answered.

1.  Security through obscurity.  The reason given for not having more open testing (e.g. expose it to the internet and put a load on it) was to avoid having the system be hacked.  Most open source advocates state that you want to have your system be exposed and tested by friendly adversaries (e.g. a Red Team made up of campaign volunteers) because they will find faults that you can then fix, security or otherwise.

2.  Users are part of the system.   The users of ORCA had nearly a literal 0 involvement in development of the system. Volunteers who were expected to use ORCA were expected to obtain the app the Monday before elections.  The instructions for the app (which people got Monday night) turned out to be a 60 page PDF.  Those that received and read it complained that it included numerous mistakes that could have been caught with proof reading.  And another surprise for people looking at Android and Blackberry App Stores was that this was a web app.

3.  Users are part of the system part 2.  The deployment of those who were part of the ORCA managed GOTV campaign did not account for any realities on the ground.  Issues like multiple precincts with co-located voting locations, distance from volunteer to the assigned polling place, and the instructions given (or not given) to poll watchers (e.g. they neglected to mention that you need to have a poll watcher certificate to hang out at a polling place or you will be suspected of voter intimidation).

4.  Models, systems, and data are nice, but the value is on what decisions you can make out of it.  When they Romney campaign gave an interview proudly presenting ORCA as their response to the Obama campaign data analysts, they said that the main use was to redirect resources from areas where they determined they had good turnout to areas that had low turnout.  But what are you going to do?  That would mean moving people across the state on short notice?  If you had resources you could have used in an area, why were you not using them before?  i.e. high tech is flashy, but what are you going to do with it?

From what I can tell from various articles and interviews (and blogs by technically adept people who were Romney campaign volunteers) was that Project ORCA may have really been a technical marvel, as those involved still claim, but it was a project management failure.  Rather ironic coming from a campaign and a party that prides itself on business sense.

Book Review: Python for Data Analysis by Wes McKinney

Python for Data Analysis by Wes McKinney
My rating: 5 of 5 stars

For some time now I have been using R and Python for data analysis. And I have long ago discovered the Python technical stack of ipython, NumPy, Scipy, and Matplotlib and I thought I knew what I was doing. I even dipped my toe into pandas as my data structure for analysis. But Python for Data Analysis showed me entire worlds of improvement in my workflow and my ability to work with data in the messy form that is found in the real world.

Python, like most interpreted languages, is slow compared to compiled languages. But there is a technical stack that started with the NumPy libraries and has grown to include Scipy, Matplotlib (graphing), ipython (shell) and pandas you get high quality and fast algorithm and data structure Fortran and C libraries underneath Python. But while these libraries are designed to be used together, documentation tends to be only about one at a time, and very little puts it all together as an integrated whole. McKinney's Python for Data Analysis fills that gap.

Even though I have been using iPython, NumPy, Scipy and Matplotlib for years, and pandas for about half a year, going through this book makes me feel like I was a rank novice. I learned how to efficiently use the shell as a development tool, to the point I have stopped automatically using the ipython notebook or pydev (eclipse) when starting new projects and I use the shell instead, because its introspection and debugging capabilities made it much easier to work. I had started using pandas for a data structure because I liked the similarities with R data frames, this book showed me where pandas goes well beyond that. With matplotlib I could make specific plots, this book showed me how to use the pandas interface to make them a natural part of the workflow (even if it is not yet at the level of a grammer such as ggplots)

Python for Data Analysis does not just teach how to use the Python scientific stack, it also teaches a workflow for technical computing. And this is beyond what you can get from reading off the web, it probably really requires the opportunity to work alongside someone who knows what they are doing to see the practices that makes them productive. As such, I would recommend it for anyone who does scientific and technical computing, whether in the sciences, engineering, finance, or other areas where quantitative computing using Python is done.

Disclaimer: I received a free electronic copy of this book from the O'Reilly Blogger Program.

View all my reviews

Two years! How did that happen?

Happily practicing piano, a photo by LugerLA on Flickr.

If year one was dominated by colic, year two was completely different. We have ourselves an almost ideal toddler. We joke that when we take him out he is a walking advertisement for having a baby. He is happy, likes exploring, babbles, and does not go out of control, and is generally enjoying himself and being with us.

My wife is fully into the chinese mom thing of drilling letters and short words. I am more lazy and worry about things like counting animals and fruits and taking him to museums. The art museum staff get amused as he does things like walk up the stairs (there are a lot of stairs) and pick out pieces of artwork to stare at for a while (and just generally want to walk and run around. The Carnegie Museum actually has a very good run around room. Very convenient when you have a toddler along) (the Museum of natural history and science center are full of kids, so they are not nearly as impressed) And when we have lunch together, I regularly get comments about how well behaved he is when we are eating.

Favorites:

Movie and songs to dance to: Sound of Music. He can sing and dance along with most of the songs.

Book: Brown bear, brown bear what do you see? He not only turns pages, but he can read it to us. And he can improvise at random moments. Runner up: Little Pea

Song to sing: The alphabet song. We actually were not trying to teach him this. I used this as a song to wash hands too. Now he uses it whenever he really does not know what to do. Runner up: Twinkle Twinkle Little Star.

Toy: Choo-choo. We have a little wooden train set. Plain as can be from Ikea. Actually, mommy and daddy are his favorite toys. Grandpa is not too far behind.

Candy: Bleh! Yuck! Somehow, we have a two year old that does not have a sweet tooth. Today, with a cake right in front of him with nothing in the way, he went for the apples because he wanted a snack.

Food: Bananas and Cherios are constants. But he does eat almost anything (but maybe not on any given day)

Drink: He has recently learned how to say chocolate milk. We consider this to be the big guns if he is upset and we are serious about getting him out of it.

TV: Ni hao Kai Lan! is still the top, but Mickey Mouse Clubhouse is now on the request list.

Ways to get out of naps: He is still not a good sleeper, during the day or night. Recent tricks are to say '(I want to) Study' which is guaranteed to work with grandma and grandpa. Singing the alphabet song is also a good way to convince us to not to try to get him to do something. Clearly, this is a toddler who is figuring out his parents and grandparents buttons.

Having grandparents around has been a great help and I think everyone is enjoying each others company. A very good year.  Now we are not worried so much about survival or health (although he still gets sick and is always on the lookout for something to eat), but our thoughts turn to the raising of a child.

Parenting 23 months: So much to explore

As T has been getting older, he has become a lot more mobile. That means he gets exercise. He goes to a neighborhood park every day and runs around with Grandpa. (Which means he is slowly leaning out. Weight is the same, but not as puffy everywhere like he used to be.) And he can choose what to spend his time looking at some more.

Dinosaurs are becoming a big thing. Early exposure was through Ni Hao Kai Lan!, as this is the title character's favorite thing. But now we have regular trips to the Carnegie Museum of Natural History. And because Pittsburgh had a dinosaur everywhere period (like Chicago did one summer with cows), it is easy to find dinosaur things all over the place.



Other toys are Legos (he is getting the hang of his Duplo blocks) and trains set. Fortunately, he has not discovered branding, so we can get away with generic things.



His playtime is becoming more varied as well. While he used to like to dance to Sound of Music and other videos, now he tries to sing. Of course, he only sings the words he knows and pauses for the other words, so it takes a while to figure out what he is singing sometimes. But sometimes it is rather impressive the words he does know.

Vocabulary in general has been picking up. We were worried about the iPhone use. But it turns out his favorite app has been a flashcard app, and he is actually learning the words (meaning he uses them outside the app). Animals, foods, vehicles, and shapes have entered his vocabulary. And he can somewhat read one of his books, "Brown bear, brown bear, what do you see?" It gets fun when he improvises based on something in front of him. Once he was looking at a picture and went around the picture using the "___ ___, ___ ___, what do you see? I see ____ looking at me!" formula.

Socialization is still slow. He likes to play with the kids of the neighborhood, because they are old enough that he is like a moving doll to them. But he does not play with kids his age (ok, that is pretty rare anyway). While he likes going out with us and he will go off and explore (e.g. in museums or stores) he does not like crowds and gets pretty clingy around them.

Next up, the two year point!

Forcing a patina on a carbon steel Mora knife

I got a new toy in the mail. A Mora MG Companion. Mora knives are known as inexpensive, but reasonable quality, fixed blade knives for outdoors or light utility work. I used to have a Benchmade Fixed Griptilian knife to serve as my very-hard-to-break knife. But while it has been pressed into service to do such tasks as batoning wood and much food prep, reality is I don't do the kind of camping where a good quality, reasonably thick fixed bladed knife was useful. And instead of it languishing in my car trunk, I identified a worthy owner who has started outdoors excursions worthy of it. And I replaced it with the MG Companion.

The Mora MG Companion seems to be the successor to the Mora Clipper. Known as the cheapest quality fixed blade knife for use in outdoor survival (meaning light woodwork and food prep for backpackers and hikers). Also very light, so lightweight backpackers who wanted a fixed blade knife (because of strength and safety) would either get a small neck knife or a Mora. It is everything expected. Lightweight, handles like a rugged kitchen utility knife. And cheap so I don't worry about beating up on it. One problem. It has a carbon steel blade. This is ok in the kitchen where you can clean it right after use, but in the outdoors it is susceptible to rust. One way to handle it is to make the steel oxidize on purpose before it can rust. That is a patina.



There are a lot of descriptions on how to do such a thing in the internet. Other than use (because it would develop a patina naturally, if you used it for a few years and prevented it from rusting first) the way is to apply acid to the knife. While blood works (from meat!), normally you find fruits. Popular food items to stick a knife into include apples, oranges, and potatoes. Another way to do this is to apply vinegar. And this is what I did.

1. Apply vinegar to a lanyard. I used the strap from a conference ID holder. I put a small amount of rice vinegar into a bowl, and used the lanyard to take it out of the bowl.
2. Wrap lanyard around knife blade
3. Wrap knife in a paper towel (to keep from making a mess)
4. Wait one hour.

 And voila! a carbon steel knife with a patina. So instead of shiny steel, it is a grey-blue-brown pattern. And I had to sharpen the knife when I was done. I think I rather like the fact that it is not shiny anymore. Some internet forums comment that the resale value on the knife will drop, but for a < $20 knife, that is not exactly a consideration. Especially since I have every intent to beat on this every now and then. So now, this knife lives in my car, ready for going hiking on a moments notice, or being around when we need another knife to cut food (without scaring everyone when I pull it out!).

 

Book Review: PostgreSQL: Up and Running by Regina Obe and Leo Hsu

PostgreSQL: Up and Running by Regina Obe
My rating: 4 of 5 stars

Compared to most books that introduce a full featured relational database management system, PostgreSQL: Up and Running is surprisingly short. It achieves this by being focused on its purpose, and doing it well. It is aimed at someone who knows a little bit about databases, although not necessarily a full featured client-server one, and brings them up to speed on both database server administration in general and PostgreSQL administration in general. It continues to introduce you to features that makes PostgreSQL special, making this a very valuable book to someone coming in to PostgreSQL from some other database, whether that database is simpler or a peer competitor. And since my background is from simpler databases, it works for someone like me.

First, this is not a book for someone who knows nothing about databases. Things such as database design, basic database concepts, and SQL is not covered, with the assumption that you get this knowledge somewhere else before coming here. (In my case, I learned on MS Access and SQLite) So what someone with this background may not have is how to work with a server based database management system.

This also has many sections that discuss some special features of PostgreSQL. The one that I thought was most interesting was the fact that you could write functions in one of many languages. Since I do much of my technical programming in Python, seeing how it is done was intriguing. (I've known that PostgreSQL has PL/Python, but the descriptions that I had seen in the past were obtuse and I had no idea how it would work.)

But this book also knows where to stop. Once I understand a topic to a base level, I'm used to looking to the web to learn more. And the authors consider the PostgreSQL Online Journal as an extension of the book, introducing an advanced topic, then pointing the reader to a quality source as a step in learning more. One advantage of this is that it keeps this book from becoming an intimidating door stopper.

In short, PostgreSQL: Up and Running has decided it will serve as a market that are not complete neophytes, but only bring people up to the point where they are better served by a rich set of advanced resources available on the web. And since this is probably a pretty good description of me, I found this to be a very good resource.

Note: I received a free electronic copy of PostgreSQL: Up and Running as part of the OReilly Blogger Program

View all my reviews

A new (to me) Black & Decker Workmate 125.

My latest toy. A Black & Decker Workmate 125. We got this from a garage sale from someone down the street. The only thing he did with it was to take it out of the box. All the parts were still in the bags. When I looked at the instructions, I was not surprised. The instructions were almost unusable. (if you want to try to put it together, I suggest looking for the video on YouTube first). Like many reviewers, I found that many of the parts don't fit quite right, and there is a pin on the handle that adjusts the jaws that I don't really have in (a awl from a Swiss Army Knife works here in a pinch). Now, my wife says, I have to use it.

Book Review: Introduction to Regular Expressions by Michael Fitzgerald

Introducing Regular Expressions by Michael Fitzgerald
My rating: 4 of 5 stars

This is not the first time I've tried to learn Regular Expressions. But other than some basic syntax, it never clicked. But I think this book provided the kind of introduction I needed to get me to not just know the syntax, but to get an understanding of how regular expressions work and how to learn its power. In short, it is not a reference, but a book that teaches me how to learn. And that is what I had not had before.

I've come across regular expressions on numerous occasions in the course of figuring out how to do tasks in various programming languages and tools, but I always found the idea of actually learning it daunting. And when opportunities came up, I generally figured out some Frankenstein combination of tools, functions, and macros that could get the job done.

What Michael Fitzgerald did was not to just give the mechanics of regular expressions, but his exercises lead you through exploring how it works by adding (or subtracting) various expressions so you understand the effects of each feature that is being put to use. The chapters add concepts that demonstrate the power of regular expressions that go well beyond the simple searching that many text or word processing packages use, starting with basic searches and pattern matching and adding markups, boundaries and anchors, back references, groups, alternatives, character classes, unicode, quantifiers, and lookarounds. Each step built up understanding and by the time I got to the last chapter I realized that I actually understood much of the new (to me) concepts that give regular expressions capabilities beyond search and replace that I was used to having in my tools.

Another useful feature of the book is the introduction of numerous tools to help work with regular expressions. He covers a few websites that help test regular expressions as well as features or add-ins for a number of text editors. He also covers how regular expressions are used inside some Unix tools such as grep, sed, and awk.

One thing that is confusing are various dialects of regular expressions. As he goes and switches between websites, Fitzgerald mentions that some of the features don't work in all the websites or tools covered. But he does not explain how to tell the difference. I half remember hearing about various dialects of regular expressions (for example, Fitzgerald mentions that grep does not have full capabilities) but some identification of this may have been helpful, so you can match the various tools you are using with the development environments you work with. So instead of observing that different tools are different, he could have identified some major dialects, and matched the tool to the dialect of regular expression (e.g. grep, perl, java, javascript). Even better, pick a dialect for the book, state up front that the book is based on a single dialect (in keeping with the fact that this is a pure introduction), and identify key areas where dialects may differ as you go.

Overall, highly recommended introduction to regular expressions. In my work in data analysis, I am at the point where I realize that I need to be able to process text automatically as the volume is too much to make doing this manually practical. I already surprise my colleagues with my ability to automate much analysis (they generally use MS Excel). Regular Expressions will enable what is indistinguishable to magic.

Disclaimer: I received a free electronic copy of this book as part of the O'Reilly Blogger program.
Introducction to Regular Expressions by Michael Fitzgerald





View all my reviews

Book Review: Safe C++ by Vladimir Kushnir

Safe C++: How to Lower the Bug Count of your C++ code by Vladimir Kushnir
My rating: 2 of 5 stars

Safe C++ bills itself as providing ways of avoiding common programming mistakes in C++ through a set of rules. While the author makes a convincing case that these rules are better than other sets of rules (for example, his rules are supposed to be affirmatively stated, i.e. a set of things to do rather a a set of thou shalt nots), these are still only a set of rules which are combined to his own library (a mix of testing framework and some replacements for some modules in the standard library).  And as such was somewhat disappointing.

As a disclosure, I am primarily a scientific/technical programmer, and I spend most of my time in the Scipy/Numpy Python stack and R.  And freely drop from there into C/C++/Java/Fortran as needed for performance or other reasons.  And I have completely bought into the idea that depending on compilers to find bugs is foolhardy, to do it right means to unit test everything.

Coming from a dynamic language and unittesting background, I found the first chapter on the use of the compiler to catch errors to be eyebrow raising at best.  And after several pages of argument, the problems turns out to be type conversions and the the use of enums, which are integers in disguise. Which are not problems in dynamic languages where you don't make type assumptions, and if you use unittesting, you don't get caught by passing around values that do not mean what you think they mean (because you catch that during the unittest).

Going through the rest, much of it revolves around using his C++ library (available through github and also printed in the book).  And these have two major categories.  One is a testing framework, which is a pretty good start on a unittesting framework of the type I'm used to.  The other category are essentially replacements to parts of the standard library, which get around flaws that he sees.  Reading the first type I started wondering why he did not just pick a unittesting framework/library and make that his example instead of building a partial one of his own.  The second, what he says sounds right to me, and is a worthy statement to make and learn, but it gets lost in everything else.

Kushnir is not the only writer that discusses common problems in programming C++ along with some rules/guidelines for programmers to avoid them.  But I am not convinced by his assertion that his set of rules is better than others.  And I suspect that one would be better served by a version of this book that dealt with using more full featured testing frameworks and more intelligent use of the standard libraries than this book which is built around his library.

Note:  I received a free electronic copy of this book through the O'Reilly Blogger program.


Safe C++: How to Avoid Common Mistakes by Vladimir Kushnir


View all my reviews

Parenting 21 months: Mommy school is in session

It is the middle of summer, and with two academics in the house that means mommy school is in full swing. The foundation of recognizing all the letters (upper case) and numbers (1-10) has been well established, now the task at hand are three letter words. In addition to the landing study area, T gets quizzed using the alphabet letters and number we have at bathtime and the letter mats we have in the bedroom. Kidding aside, we noticed that what T remembers are the ends of words and phrases. So when there are words that end in the same letter/sound he gets confused. First letters are not a problem. We also noticed this with letter (that end with the 'eee' sound) and songs (he says the ends of phrases).

Another scholarly activity is reading. He will work through a book and there is a clear correspondence with the words saying and the page that is open. Clearly, this is mostly a result of memorizing the book, but there is some actual reading going on. His favorite is "Brown bear, brown bear, what do you see?". This was also the first book where he did the page turning while we read, so it is apparently still his favorite book.

Building PostGIS on Ubuntu from source

I wanted to revisit PostGIS for an update on a project.  In the current version, we built a data table of distances, but I really want to be able to build this table based on a road network, i.e. if I have a road shapefile, pick two nodes and calculate the distance.  And sending many thousands of requests of this type over the internet to build the distance matrix is not very practical, especially if there are multiple road networks that need to be considered as well.

There are pre-built installers for Postgresql for all the major operating systems (Windows, OS X, Linux), but they seem to be version dependent.  i.e. it matters what version of the operating system you have, because the pre-built versions make assumptions about shared libraries and such (I think that is what is happening).  Plain Postgresql does not seem to be that much of a problem, but when adding in extensions (like PostGIS) it is.  So instead of fighting it, I decided to build the PostGIS stack on Linux.

Some issues along the way.

1.  Using ./configure -> make -> make install seemed to work smoothly (Postgresql 9.1, PostGIS 2.0.  note that I already have GEOS and PROJ libraries on my system)  But I needed to take a careful look at the config.log, because then I would have noticed that it could not find GTK+, dblatex and CUnit.

2.  To build the graphical shp2pgsql-gui plugin for Postgresql pgAdminIII, configure requires a --with-gui option.

3.  For GTK+, I had this on my system, but it was GTK3, and the pgAdmin actually requires GTK 2 development libraries.

4.  For CUnit, it requires the CUnit development libraries.

5.  After it is installed, shp2pgsql-gui is in the /usr/lib/postgresql/9.1/bin/ directory, not /usr/sbin as some places suggest.  So I needed to make a symbolic link using
   sudo ln -s /usr/lib/postgresql/9.1/bin/shp2pgsql-gui /usr/bin/shp2pgsql-gui

6.  Then I needed to add appropriate entries to the plugins.ini file, which is actually in /usr/share/pgadmin3/plugins.d directory

I took the entry found at GIS Stackoverflow
http://gis.stackexchange.com/questions/16181/how-can-i-enable-shapefile-gui-loader-in-pgadmin3

Now after this, it builds, all tests pass and the PostGIS graphical loader is working.

Book Review: Clojure Programming by Emerick, Carper, and Grand

Clojure Programming by Chas Emerick
My rating: 4 of 5 stars

When I first started reading Clojure Programming, the first impression I had was that I've never seen a programming language introduction try so hard to tell me that my programming skills are going to grow significantly as a result of learning this programming language. This book is probably not for beginners, but working through this gives insights into why certain programming constructs work the way they do and how functional programming constructs fit with someone who is used to procedural and object oriented programming. While the efforts it makes to declare its superiority towards other languages gets annoying, the practice in using the REPL actually helps me understand how to better use tools I use in other languages.

My background is in scientific computing. And while I have used some standard languages such as Java, C, C++, and even Fortran, my first choice in languages tends to be either R or Python. The R environment is admittedly ugly, but in Python I typically use tools that surround the scientific ecosystem of ipython/Numpy/Scipy/Matplotlib. My programs tend to be structured into a data gathering and manipulation portion (MVC) then an analytics portions dominated by a series list comprehensions and map-apply and finally presentations of results. In this context, I first heard of Clojure because the Incanter: Data Sorcery project, which was an attempt to create a Lisp-based statistical computing and graphics platform. So I approached Clojure Programming with a view of its suitability for technical computing, or a way of learning functional programming techniques in a deeper fashion for use in R or Python data analytics.

It somewhat disappoints in this. When Clojure Programming put effort into demonstrating its unique points, such as the REPL, I kept on thinking "I already have this in Python/ipython" (in particular the ipython shell gives considerably more capability than the standard shell) When reaching the Numerics chapter, it seemed to be a discussion of standard data types (and when trying to discuss Python numerical types, not acknowledging Numpy data types and discussing performance is ignoring the real world.) My main complaint is that the discussions of language and environment features feels like I'm being presented with a long list of details, without having first having a general sense of what programming would be like.  In many ways, it is similar to learning scientific programming in Python.  You generally learn general programming first, and later on you learn the techniques for data analysis, many of which are based on functional programming practices.

What the book does well is explain deeply the principles behind the language and functional programming.  In works through them with cross references in the text wherever they are necessary.  And it explains why these principles are useful.   In particular with the REPL, which if I had not had experience with the ipython shell would have been quite different then other shells.

What this leads me to conclude is that this book would be very good for someone with a solid grasp of Java programming, with its ecosystem and libraries, and wanted to move into functional programming.  Clojure Programming provides a different workflow since it does compile/evaluate as you go, allowing you to see quickly the results of changes without any performance penalty. And having a different programming paradigm than object-oriented adds a set of tools that can be brought to bear under a different situation.

This was not quite what I was hoping for, but that may be because the tools and language features of Python/ipython were already overlapping with Clojure.  It does fulfill the promise of making me a more knowledgable programmer.  And I hope that working through Clojure Programming also makes me a better one.

Disclaimer:  I received a free electronic copy of Clojure Programming as part of the O'Reilly Press Blogger program.


View all my reviews