Tuesday, January 15, 2013

PSO: A life with companions for the journey

It has been a while since I've been to the symphony.  On occasion I get asked why go through the trouble and expense of hearing symphonic music live. Every time I go to a concert it I remember.  It is the richness of the sound, the interaction between the instruments, and the range of the instruments, as they call out and respond to each other.

For this concert was a piece written by a composer with an equally illustrious friend and musician who would play it. There are a few ways this could play out, but with the maturity of age, what comes across is the range of expression.  Some somber, some light and almost whimsical, and some parts bursting with excitement.  And you could see this in Noseda and Dindo on stage. Noseda with his expressiveness ranging from delicate to so much energy I thought he would fly from his podium.  Dindo's feeling in his playing reflected in his expressions showing as he played.

And that is how you hope that friendships will be.  Full of memories, some somber, some whimsical, some bursting with excitement.  Not everything in life should be burdened with the requirement to be deep and full of meaning, but the realities of life's journey means parts of it may very well be, and you want to have friends and companions along the way.  And there are parts of life which do not have meaning beyond the moment, which companions to share in the ephemera.  But even better is when you have companions along the way in both the special and the mundane, in life's milestones and the somber and contemplative parts of life.  Because your relationships should be with people in their fullness, not to be casually cast aside.  And the cello concerto was like that as well, full of the richness of life in its grandeur and the mundane weaved together like a life to be lived will among friends that deserved to be remembered until the end of your days.

Sunday, January 06, 2013

2012 Python meme

I think the official rules are here.  I don't have a Twitter feed, so I'll just put this tag in my labels.
  • copy-paste the questions and answer them in your blog
  • tweet it with #2012pythonmeme hashtag

1. What’s the coolest Python application, framework, or library you have discovered in 2012?

Hmm, I got to know alot about pandas this year, but I discovered it before.  So I have to say pweave.  I learned the whole concept of combining the narrative discussion, code, and output in literate programming through Sweave in R, and I love the idea of doing the same in Python.  Combined with Pandoc, this means I can write explanation, code, and results, have the data or code change, and my slides or documents are updated automatically.  And that makes a lot of things work better.

2. What new programming technique did you learn in 2012?
Using list comprehensions and other functional programming based patterns in data analysis.  Most of the programming I do tends to be data analysis (because the students and post-docs I work with are not skilled in this), and my code has been turning more functional over the year.

3. Which open source project did you contribute to the most in 2012? What did you do? 
Not much in the way of direct contributions.  I filled out bug reports with both pandas and R Studio.  I actually enjoyed the interactions I had with developers (both have developers employed by companies that are focused on these products) as we trouble shot the code and developed working solutions.

4. Which Python blog or website did you read the most in 2012?
I cheat and use Planet SciPy and get all of the scientific programming in Python blogs.

5. What are the top things you want to learn in 2013?
Working with both simulation libraries (SimPy) and optimization modeling languages (Pyomo or PuLP) and working them into a real problem.

6. What is the top software, application, or library you wish someone would write in 2013?  
Round trip between iPython Notebook and Python with Markdown.  Right now I think I have to choose between workflows, and there are some nice parts about each that I would like to keep.

I could say more Python 3 migration, but there are so many pieces that have to be moved, this would hardly result in a top N list for any reasonable value of N.

Personal experiments on workflow, note takers and task organizers

This semester I'm undertaking a few experiments in productivity.  Now, most of my work as a professor does involve computers, either developing and implementing models and methodology, preparing course lectures and materials, and communicating, and managing others.  And this spring, I have a fairly challenging schedule from an organizational standpoint.  In addition to the standard course and research projects I am working with a new (to me) graduate student and post-doc, I am running the department senior project course (with the department chair to back me up), and I am involved in one of those long-term career milestones.  So part of this past winter break was spent setting up systems to improve productivity.  Some minor workflow issues.  And some larger ones.

1.  Using Pandoc, Markdown, and pweave for class preparation.  

I long ago decided on Beamer as my preferred method for preparing presentations.  Using LaTeX essentially forces the use of styles instead of fiddling with page layout (or having your styles get overridden without notice) and I can focus on content rather than form.  But there is alot of scaffolding that goes into any LaTeX document.  Enter Markdown.  I already use Markdown with R using knitr, which allows me to keep both the documentation about a method along with its coded implementation and the output of the code in a single document with a syntax that is not that far off what you would naturally do if you had to format a document using plain text (e.g. using Underlining and blank lines to set off headings or '*' to identify emphasized words.)  So Pandoc is a utility that allows me to use Markdown format to generate slides in HTML5 or as PDF files (via Beamer).  And pweave does for Python what knitr and Sweave do for R, combine the documentation of algorithms, the implementation, and the output in a single document where the output will automatically update as the implementation or the input data changes.

So now, I can build slides in Markdown that includes the lecture, the math model, a Python implementation of the math model, and the output of the implementation.  And since Markdown is a lot less complex than LaTeX, hopefully do this faster than I was before.

2.  Using Mercurial as a hosted version control system.  

I was running a project that was based on a simulation being built over a period of years by a post-doc.  However, he was really the only one working on the code.  It was at the point where I wanted to do some side experiments with our model, so I wrote another implementation in Python. (note, using SimPy, it took me two weeks).  But now he has moved on, and I'm still working on the project with another post-doc and grad student. So, since they are new, and we are jointly working on the model to apply it to a new system, I've taken the model from Dropbox (shared folders) to distributed version control, namely Mercurial on Bitbucket.

While Git is the more popular (at least among hipster elite programmers), Mercurial uses commands similar to Subversion (which the post-doc already knew) and tools were readily and obviously available for both Windows and Linux.

I've been using Mercurial and Bitbucket for my own projects including coding and writing papers in LaTeX for a while now, but this is the first project of my own that I have other people working with the same repository.  So I'm training them along the way.

3.  Google Tasks as a task organizer

I've started using Google Tasks. At one point I was using Remember the Milk, but I stopped because it was not integrating well with Google Calendar (which is how I do my scheduling). (RTM put an entry into every calendar day, which means I had to open my calendar to see if the marker was real or empty)

So with Google tasks, I can create a task in the Google Calendar sidebar, in the sidebar of Google mail, or there is an iOS app (GoTasks). The interface is not as pretty as Remember the Milk, but it works better with my Calendar. And the iOS version is not crippled (RTM is limited to one update a day on the free version).  The main criteria here was the fact that it worked on all of my computers (I use Windows, Mac OS X, and Linux in different settings) as well as my iPhone and iPad (because I often go to meetings and trips with just my iPad and not my laptop)

4.  Tomboy as a notes organizer.

This past fall I was working with an undergraduate who learned the use of MS OneNote while a summer associate with McKinsey and I was impressed about how well he could organize and use these free form notes. I had been using Google Docs to do this, but the need to get on the internet, fire up the browser, and then work was sometimes too much, and it was hard to organize.

Tomboy is a fairly lightweight application.  It provides a formattable and linkable text area, but not as heavy as most desktop applications.  Tomboy can also work with a cloud storage account such as Dropbox or Ubuntu One to keep the data in the cloud, to be synced whenever your computer is on-line.  So now, I have set up a notes folder for the senior design project, with subfolders for each team, a folder for my major research project (with the grad student and post-doc), and my other project.  And I know have collections of short notes that are each dated, identified with keywords, and linked to each other as much as needed.  And I can access and edit them from each of my computers, my iPhone and iPad.

So, here is the great experiment, cobbled together in the midst of all the other work and writing I was doing over the winter break.  And next week is when the fun starts.