Thursday, November 22, 2012

Installing Ubuntu 12.10 on Dell Dell Precision WST3500,Xeon (E5640) Nvidia Quadro NVS295

Ok, this is getting way too hard, I better keep track of everything that is going on.

1.  To boot from USB, The instructions on the Dell splash screen (which goes past to fast to note) says to press F12 when splash screen is up to change the boot order.  Now you can boot from USB or CD drive and install Ubuntu.

2.  The boot up will go to a blank screen because the NVIDIA graphics drivers are unsupported.  Hold the shift key during boot up.  Then in the startup script that shows up, look for the line that starts with 'linux'  Find the option 'quick splash'.   Replace it with 'nomodeset' 

Ok, now Ubuntu starts.  But you have to do do (2) everytime unless you can get the NVIDIA driver installed. 

3.  Download the appropriate driver from the NVIDIA website.

4.  You cannot install a new graphics driver while x-windows in on (i.e. trying to do this through the Ubuntu Software Center was a failure).  So get to a real terminal (not just xterm) and shutdown the x-server using the following instructions (Note: I just shut down the x-server using this.  Installing the nvidia drivers comes next)

4.  Follow the instructions here on installing the driver.  Note that you also need to have the linux-headers installed.  And when I did this, it specified linux-headers-3.5.0-17-generic  (not 3.5.0-18-generic)

  sudo add-apt-repository ppa:ubuntu-x-swat/x-updates
  sudo apt-get update
  sudo apt-get install nvidia-current nvidia-settings

3.1  Don't forget to restart the X server using sudo service lightdm start

Saturday, November 10, 2012

Inside Team Romney's whale of an IT meltdown - Ars Technica

Inside Team Romney's whale of an IT meltdown: ORCA, the Romney campaign's "killer" app, skips beta and pays the price

It was supposed to be a "killer app," but a system deployed to volunteers by Mitt Romney's presidential campaign may have done more harm to Romney's chances on Election Day—largely because of a failure to follow basic best practices for IT projects.

. . .

IT projects are easy scapegoats for organizational failures. There's no way to know if Romney could have made up the margins in Ohio if Orca had worked. But the catastrophic failure of the system, purchased at large expense, squandered the campaign's most valuable resource—people—and was symptomatic of a much bigger leadership problem.

This is quickly turning into a how to on how NOT to run an IT project.  Project ORCA was the Romney campaign Get out the Vote platform, developed by Microsoft and an unnamed consultant. (ORCA was a play on the Obama version, NARWHALE, because killer whales are a predator of narwhales)  It was developed in seven months, but more importantly, it was developed in relative obscurity, with volunteers who have computer development experience stating that their briefings were more like rah-rah sessions and technical questions that they raised (like 'was this stress-tested?') went un-answered.

1.  Security through obscurity.  The reason given for not having more open testing (e.g. expose it to the internet and put a load on it) was to avoid having the system be hacked.  Most open source advocates state that you want to have your system be exposed and tested by friendly adversaries (e.g. a Red Team made up of campaign volunteers) because they will find faults that you can then fix, security or otherwise.

2.  Users are part of the system.   The users of ORCA had nearly a literal 0 involvement in development of the system. Volunteers who were expected to use ORCA were expected to obtain the app the Monday before elections.  The instructions for the app (which people got Monday night) turned out to be a 60 page PDF.  Those that received and read it complained that it included numerous mistakes that could have been caught with proof reading.  And another surprise for people looking at Android and Blackberry App Stores was that this was a web app.

3.  Users are part of the system part 2.  The deployment of those who were part of the ORCA managed GOTV campaign did not account for any realities on the ground.  Issues like multiple precincts with co-located voting locations, distance from volunteer to the assigned polling place, and the instructions given (or not given) to poll watchers (e.g. they neglected to mention that you need to have a poll watcher certificate to hang out at a polling place or you will be suspected of voter intimidation).

4.  Models, systems, and data are nice, but the value is on what decisions you can make out of it.  When they Romney campaign gave an interview proudly presenting ORCA as their response to the Obama campaign data analysts, they said that the main use was to redirect resources from areas where they determined they had good turnout to areas that had low turnout.  But what are you going to do?  That would mean moving people across the state on short notice?  If you had resources you could have used in an area, why were you not using them before?  i.e. high tech is flashy, but what are you going to do with it?

From what I can tell from various articles and interviews (and blogs by technically adept people who were Romney campaign volunteers) was that Project ORCA may have really been a technical marvel, as those involved still claim, but it was a project management failure.  Rather ironic coming from a campaign and a party that prides itself on business sense.

Friday, November 09, 2012

Book Review: Python for Data Analysis by Wes McKinney

Python for Data AnalysisPython for Data Analysis by Wes McKinney
My rating: 5 of 5 stars

For some time now I have been using R and Python for data analysis. And I have long ago discovered the Python technical stack of ipython, NumPy, Scipy, and Matplotlib and I thought I knew what I was doing. I even dipped my toe into pandas as my data structure for analysis. But Python for Data Analysis showed me entire worlds of improvement in my workflow and my ability to work with data in the messy form that is found in the real world.

Python, like most interpreted languages, is slow compared to compiled languages. But there is a technical stack that started with the NumPy libraries and has grown to include Scipy, Matplotlib (graphing), ipython (shell) and pandas you get high quality and fast algorithm and data structure Fortran and C libraries underneath Python. But while these libraries are designed to be used together, documentation tends to be only about one at a time, and very little puts it all together as an integrated whole. McKinney's Python for Data Analysis fills that gap.

Even though I have been using iPython, NumPy, Scipy and Matplotlib for years, and pandas for about half a year, going through this book makes me feel like I was a rank novice. I learned how to efficiently use the shell as a development tool, to the point I have stopped automatically using the ipython notebook or pydev (eclipse) when starting new projects and I use the shell instead, because its introspection and debugging capabilities made it much easier to work. I had started using pandas for a data structure because I liked the similarities with R data frames, this book showed me where pandas goes well beyond that. With matplotlib I could make specific plots, this book showed me how to use the pandas interface to make them a natural part of the workflow (even if it is not yet at the level of a grammer such as ggplots)

Python for Data Analysis does not just teach how to use the Python scientific stack, it also teaches a workflow for technical computing. And this is beyond what you can get from reading off the web, it probably really requires the opportunity to work alongside someone who knows what they are doing to see the practices that makes them productive. As such, I would recommend it for anyone who does scientific and technical computing, whether in the sciences, engineering, finance, or other areas where quantitative computing using Python is done.

Disclaimer: I received a free electronic copy of this book from the O'Reilly Blogger Program.

View all my reviews

I review for the O'Reilly Blogger Review Program