A recap on work flow, note takers and task organizers

Some experiments I have been doing with workflow over the past semester as introduced on this post back in January. 

1.  Using Pandoc, Markdown, and pweave for class preparation.  

I did the notes for my Logistics and Supply Chain course completely in Markdown.  This included slides for in class presentation, writing homework solutions, and everything else with the class preparation.  I wrote a Makefile that took care of everything.  Basically what it meant is that to create slides for class prep I wrote it as a text file.  It does force some simplicity because there is no ability to tweak a slide or an image.  But that is probably a good thing because it forces efficiency.

One nice advantage of using pweave for homework and exam solutions is that it makes it easy to have multiple versions of an exam.  If I have equations and steps in place, I can change a few parameters, and maybe an assumption or two and I have a new problem with not much extra work.  Also I did a lot of cut and pasting of code.  The code I wrote for presenting in class became the code that I used to solve the homework.  And since I let them see the code (and I also copied over the equations needed to solve the homework) it connected the homework solution to the lectures.  One issue is that so few of my students were competent programmers in any language that the Python code was not all that helpful to them (other than proving that I did indeed solve the problem)  But this was a definite success.

2.  Using Mercurial as a hosted version control system.  

I had to work with a new team on one project I was working on, so the first thing I did was to move the  former post-doc's work into a version control (Mercurial) and put it on a hosted repository (bitbucket).  And we spent a month refactoring the entire thing to separate out the code from the site specific bits (because we were going to apply it to a new location).  The repository meant that we were able to be aggressive in refactoring (yes, I did have to completely through out a day of work at one point because what I was thinking was not going to work.  All I did was wipe my directory and reclone the repository)  The others got into the hang of putting everything in the repository as well.  Now, the comments are not all that good on the part of the post-doc and grad student working with me, and we tended to work in separate areas so it was not true collaborative programming, but it was not bad.

I plan on continuing this over the summer with a few students who are learning agent based modeling together.  The intent is that if all of us are putting our models in the repository, we can ask questions like "why does ___ not work" and we can have their model to look at.  We'll see how that works.

3.  Google Tasks as a task organizer  

This worked out pretty well.  I ended up using Google tasks as a note taker (more on that later)  I was off an on regarding being consistent, but there were periods where I had a lot going on that this to-do list being available on all of my computers, my iphone and ipad was very useful.  Keep doing this.

4.  Tomboy as a notes organizer. 

Note taking sounds like a no-brainer.  Not.  This one pretty much died.  In February Canonical eliminated an API that enabled use of Ubuntu One as a storage place.  So I ended up storing the notes data on Dropbox (so my Windows and Linux machines were synced).  But the iPad app that integrated with Tomboy no longer worked, so if I was using the iPad, the notes were not integrated.  And this pretty much meant I was not using Tomboy notes.  Sometimes I would create a task in Google tasks and create a note if I needed something.  But generally this went unused.

I'm going to try this again.  But since I realize now that having it on my iPad and syncing to my computers is the key to the whole affair, my main focus was on the iPad integration.  So I am looking at Simplenote for the note-taking. It is originally designed for iPhone/iPad, and there are applications on Windows and Linux that integrate with it (and also the Google Chrome browser)

Personal experiments on workflow, note takers and task organizers

This semester I'm undertaking a few experiments in productivity.  Now, most of my work as a professor does involve computers, either developing and implementing models and methodology, preparing course lectures and materials, and communicating, and managing others.  And this spring, I have a fairly challenging schedule from an organizational standpoint.  In addition to the standard course and research projects I am working with a new (to me) graduate student and post-doc, I am running the department senior project course (with the department chair to back me up), and I am involved in one of those long-term career milestones.  So part of this past winter break was spent setting up systems to improve productivity.  Some minor workflow issues.  And some larger ones.

1.  Using Pandoc, Markdown, and pweave for class preparation.  

I long ago decided on Beamer as my preferred method for preparing presentations.  Using LaTeX essentially forces the use of styles instead of fiddling with page layout (or having your styles get overridden without notice) and I can focus on content rather than form.  But there is alot of scaffolding that goes into any LaTeX document.  Enter Markdown.  I already use Markdown with R using knitr, which allows me to keep both the documentation about a method along with its coded implementation and the output of the code in a single document with a syntax that is not that far off what you would naturally do if you had to format a document using plain text (e.g. using Underlining and blank lines to set off headings or '*' to identify emphasized words.)  So Pandoc is a utility that allows me to use Markdown format to generate slides in HTML5 or as PDF files (via Beamer).  And pweave does for Python what knitr and Sweave do for R, combine the documentation of algorithms, the implementation, and the output in a single document where the output will automatically update as the implementation or the input data changes.

So now, I can build slides in Markdown that includes the lecture, the math model, a Python implementation of the math model, and the output of the implementation.  And since Markdown is a lot less complex than LaTeX, hopefully do this faster than I was before.

2.  Using Mercurial as a hosted version control system.  

I was running a project that was based on a simulation being built over a period of years by a post-doc.  However, he was really the only one working on the code.  It was at the point where I wanted to do some side experiments with our model, so I wrote another implementation in Python. (note, using SimPy, it took me two weeks).  But now he has moved on, and I'm still working on the project with another post-doc and grad student. So, since they are new, and we are jointly working on the model to apply it to a new system, I've taken the model from Dropbox (shared folders) to distributed version control, namely Mercurial on Bitbucket.

While Git is the more popular (at least among hipster elite programmers), Mercurial uses commands similar to Subversion (which the post-doc already knew) and tools were readily and obviously available for both Windows and Linux.

I've been using Mercurial and Bitbucket for my own projects including coding and writing papers in LaTeX for a while now, but this is the first project of my own that I have other people working with the same repository.  So I'm training them along the way.

3.  Google Tasks as a task organizer

I've started using Google Tasks. At one point I was using Remember the Milk, but I stopped because it was not integrating well with Google Calendar (which is how I do my scheduling). (RTM put an entry into every calendar day, which means I had to open my calendar to see if the marker was real or empty)

So with Google tasks, I can create a task in the Google Calendar sidebar, in the sidebar of Google mail, or there is an iOS app (GoTasks). The interface is not as pretty as Remember the Milk, but it works better with my Calendar. And the iOS version is not crippled (RTM is limited to one update a day on the free version).  The main criteria here was the fact that it worked on all of my computers (I use Windows, Mac OS X, and Linux in different settings) as well as my iPhone and iPad (because I often go to meetings and trips with just my iPad and not my laptop)

4.  Tomboy as a notes organizer.

This past fall I was working with an undergraduate who learned the use of MS OneNote while a summer associate with McKinsey and I was impressed about how well he could organize and use these free form notes. I had been using Google Docs to do this, but the need to get on the internet, fire up the browser, and then work was sometimes too much, and it was hard to organize.

Tomboy is a fairly lightweight application.  It provides a formattable and linkable text area, but not as heavy as most desktop applications.  Tomboy can also work with a cloud storage account such as Dropbox or Ubuntu One to keep the data in the cloud, to be synced whenever your computer is on-line.  So now, I have set up a notes folder for the senior design project, with subfolders for each team, a folder for my major research project (with the grad student and post-doc), and my other project.  And I know have collections of short notes that are each dated, identified with keywords, and linked to each other as much as needed.  And I can access and edit them from each of my computers, my iPhone and iPad.

So, here is the great experiment, cobbled together in the midst of all the other work and writing I was doing over the winter break.  And next week is when the fun starts.