The Impact of Generative AI in the Development of Mathematics

After the INFORMS-Pittsburgh event on Reliable AI in Healthcare (https://luma.com/bjrab6ek), among other things, conversation turned to the recent proof of the Navier-Stokes Millennium Prize Problem For a proper response, I recommend After Math by Silvia De Toffoli and Eamon Duede. And this is not a sudden reaction, it is part of an ongoing conversation this summer on AI and Mathematics that includes the First Proof Project which is on its third batch, the Leiden Declaration on Artificial Intelligence and Mathematics, and the reaction to the OpenAI proof of Navier-Stokes in A Severe Misalignment of AI in Mathematics, written by a group of 25 Fields Medalists (the highest honor in mathematics, awarded every four years to a few chosen mathematicians). So for a more thought out response, go to A Severe Misalignment or the Leiden Declaration. For a philosophy of Math, go to Terence Tao ICM 2026 Public Lecture - Mathematics in the age of AI These are my reactions as a practitioner of the mathematical sciences who is not nearly at that level, but my work applies mathematics in settings that non-mathematicians can use to get their work done.

First, a discussion on what it means to do a mathematical proof. (Terence Tao does a much more complete job). A mathematical proof is a logical series of arguments that starts from a given condition and demonstrates a final conclusion must be true. In the modern day, most people get their first exposure to this in high school geometry. The ancient Greek philosophers regarded the ability to do mathematical proofs to be absolutely essential for one to be able to participate in any form of logical argument. And in anything of any complexity, we find that we need to prove many intermediate results along the way. (there are referred to as 'lemmas'. Think of them as little proofs, suitable for beginning mathematical researchers.) And it is not enough that a mathematically correct proof is done, people have to understand it to understand its implications, both on the direct question at hand but also related concepts. The computer scientist (and mathematician) Richard Hamming stated "The purpose of computing is insight, not numbers", which is a core understanding in my own field of operations research.

In modern mathematics, this shows in if a proof is accepted. It is not enough for a proof to have artifacts that state that it is correct, this is not like a homework assignment. It is important that it can be understood. Terence Tao calls this the whiteboard test, you have to be able to explain the proof in front of an audience at a whiteboard. An example of this is the proof of the four color theorem. The Four color theorem states that no more than four colors are needed to color any map so that no two adjacent regions have the same color. It had been believed true for a century, based on observation of people making maps with a minimum of colors. It was first proven using computer in 1976, but it was not possible to check the computer generated proof by hand. Not until 1989 was a new proof developed that was accepted (and along the way it found and correct errors found in the 1971 proof along the way and found by others).

One of my graduate school professors at Northwestern, Collette Collard, greeted us with a statement that mathematics was beautiful and useful. (I am in operations research, mathematics is a means of understanding and improving operations). So a proof that no-one can understand is neither beautiful nor useful. So, in something as abstract as this, the rework to make a proof understandable has a normal side effect of making it beautiful. And, more important to someone like me whose mathematics is in the pursuit of other goals, the understanding leads to being useful. Because in the process of working through and understanding a proof, many other concepts are developed and proven along the way. And those other concepts also become points of departure for exploration. And potential impact. But a proof that is a deux ex machina, or created out of nothingness from an AI, an alien, or a deity, does not provide those insights for the person who shepherded the computer in discovering the proof, or those who read and (do not) understand the proof. This type of proof is a monument or idol, it can be looked on in awe, but it does no-one any good and is neither beautiful nor useful.

Can AI be taught to make proofs beautiful and useful? We have ways of checking that a proof is correct (the LEAN programming language and proof assistant, LEAN is also notable for programmers because it enables provably correct computer code), but it does not check for goodness or usefulness, or if it allows for insight, which is the point of the exercise for operations researchers like myself, or for society who invests in and desires to benefit from mathematics, even theoretical math which frequently becomes practical in unexpected directions with the passage of time. We have already had computer assisted proofs where we can check the endpoint, but we have not been able to teach the computer to go beyond correctness to organizing the proof so that insights are generated along the way which would make the mathematics useful.

I have sometimes been a judge at the ISEF (International Science and Engineering Fair) in mathematics. (Pittsburgh used to be one of the rotating hosts). The Carnegie Mellon University and University of Pittsburgh math faculty, who formed the core of the judges, explained to the rest of us that they would judge the quality of the math, the rest of us were there to assess the ability of the students to describe their math to normal people (as mere engineering and applied math PhDs were closer to normal people than research mathematicians). As mathematics does not require expensive or extensive equipment or resources, these kids were as good as anyone in the world. My approach to judging these projects was not to focus on the end result, but to identify what were the key insights that were needed along the way, and ask the participant how they attained that insight. And I knew I got it right when the participant launched into an animated discussion about that point. Because that is how math (and science) works, exploration (often in some direction) and insights into our world that come along the way, pointing to new directions of exploration or application.

For a mathematical proof, the burden after it is created is the same as any other proof. Do we understand it? What insights do we gain from the proof or effort? If there is no insights, so what? (which is the same thing my business partners would say if I brought such a thing to them)

Why learn theory

Why learn theory? We were bringing our daughter back from an arts camp in northern Michigan. As we talked about music, her lessons and teachers, pieces, performances, and ensembles, we also talked about her music practice (music theory and music history) class. And as a musician, while music practice was a welcome and sometimes entertaining break, she asked why we learn theory.

All practical domains have an underlying theory. Theory is how a domain understands its environment and how practitioners and researchers interact with its environment. For example, knowing the fire triangle/tetrahedron enables firefighters approaching a scene to assess how best to control and contain a fire while ensuring the safety of people in the area. Planning looks a capability and capacity and how to employ these to turn strategic and operational goals into resource allocation and actions over time.

What does theory give you? First, it helps explain why things work or do not work. Theory provides the basis to understand success and learn from failures. So that each observation is not simply a succeed/fail assessment, but you learn lessons on what made it work and how it can be done better. And each event/observation is also an opportunity to examine the theory and make it more detailed (with experience and wisdom comes a more nuanced and detailed theory. e.g. physics)

What are alternatives to theory (always know the alternative to any concept)? The most common is 'how we have always done it" or "this is our experience". Next is "loudest person in the room" and its relative "highest paid person in the room". But experience is highly situational, and without theory you do not know what is the key detail that made the experience what it was. The appeal to authority and judgement is also highly situational - people's expertise is in large part based on the experiences they have had (one reason that best practices for developing executives and general/flag officers is to give them experiences in a wide range of settings over the course of their early careers). Theory creates a framework to have conversations and identify what the questions that need answering and how to create and evaluate alternative solutions.

In my own area of operations research, which has led me to artificial intelligence and it generative AI form, theory is what I use to evaluate alternative approaches to addressing a problem. I like to say I know how to break every tool I've ever used. In this new era of Generative AI, I start projects with new business partners and domains by figuring out what about the problem domain generative AI gets wrong. (my business partners say that I am secretly happy when they find mistakes or cause problems with generative AI tools. They are wrong. I'm openly happy) And theory helps look at those problems and work out ways to modify those tools and make them work in the setting at hand. Even as the world changes around me.

Reflections on the Advent of OR: Using Generative AI in Analytics and Agile Operations Research

In December 2025 I participated in the Advent of OR (https://adventofor.com) which was a 24 day exercise that guided participants through an optimization project. And instead of just solving problems and creating models, the Advent of OR walked through an entire project life cycle, using the INFORMS Analytics Framework.

While I am not a student or early career who was the target audience, I took part, and I had three goals.

1. Use a new programming toolkit. I used VS Code with R and Quarto.  I usually use R Studio and I wanted to try R on VS Code.  And I think Quarto is the future replacing Jupyter Notebooks for Python and a natural evolution from R Markdown.

2. Practice in optimization. In the Operations Research world,  I am NOT an optimization person. My thesis was applied probability (queuing) and my methods research has been in simulation (one stream in ranking & selection and another stream in Bayesian methods for input modeling)

3. Using Generative AI. I wanted to see how generative AI does in an operations research project.  And I wanted to do it right in a setting where I can give it references to guide it.  Note: I have found that Generative AI favors descriptive statistics, machine learning, and hypothesis based statistics over other forms of analytics, so it needs some guidance.

Toolkit

I had to set up VS Code with the R extensions, Quarto (and extension), ompr and the glpk with associated R ROI packages, and to make sure everything worked, I download the repository for OR_using_R by Tim Anderson.  Then to render the book (meaning I made sure all the code ran) I had to install texlive with xetex and extra fonts.  Generative AI (I had Gemini CLI installed) was very helpful in all of the system administration tasks since it could figure out what was needed every time there was an error message.

Data analysis and Optimization

Working with the data sets, it read in the data, (I had to give it some corrections along the way to help it recognize the data types.  When the data files were read in, it recognized that the data sets did not correspond in granularity.  In the R markdown file it created, in addition to generating the code that read in the data and created summaries, it also identified a number of questions and concerns about the data and created questions for the stakeholder.  This was a good set of questions that corresponded to what others put forward.  

It also did well with the optimization.  Given an optimization textbook, I first asked the generative AI for a mathematical formulation based on the project description. Similarly, it created a process for determining what kind of problem this was and worked through that process to determine that this was a linear programming optimization problem.

Next was a LP formulation using OMPR.  The first formulation was straight forward.  I went and had the GenAI break out the formulation into its own R script to enforce a separation of concerns between the data handling, optimization model, and output processing.

I generally also ask for docstrings as I go, and the Gen AI did this for both the model as well as various handling functions. I generally read the docstrings to ensure they say what I expected them to say. When it did not, since the docstrings were written based on the code, I took it to mean that the code was not right (this exposed a mistake in the initial formulation of the LP in OMPR).  Similarly, I had the Gen AI write unit tests for the constraints and a mock problem to test the optimization.

Agile Operations Research

One of the aspects of having the Advent of OR over 24 days is that it rotates topics between the art of modeling, implementing and managing models, and interactions with stakeholders.  There are a couple of very important points. First is that interacting with stakeholders is not something that is at the beginning and end of project and ignored in the middle.  There needs to be stakeholder engagement throughout the modeling process.  A second point that has come out in the conversations on LinkedIn is that the most common cause of project failure across data analytics are communication failures, in particular between the analyst and the end customer. While this can have many causes (including management inserting themselves in between the analyst and end customer), as analysts we must have that direct interaction from the beginning of the project (business problem formulation in the INFORMS Analytics Framework)

In the early stages of the project, one factor we need to face is failure of imagination. The first level for analytics is that our stakeholders often do not know what is possible across the full range of analytics.  Often a problem is presented as a request for a tool, but for the results of the project to have any value, it has to address the end problem, so business problem formulation has to start with the end problem, determine what kind of information from data can help the decision makers address the problem, and then we can start discussing what methods can provide results in the form that will be useful.  Currently, because of media hype, the initial request can be for a dashboard, or a predictive model, or a generative AI tool.  As operations research analysts we can also bring to bear statistics, forecasting, optimization, simulation, and queueing; and different ways of applying those methods to give different kinds of results that can be delivered to decision makers to make better decisions.

After the business problem formulation, the next big change in the project will occur when presenting the first minimum viable model to the end user. This is the first model that uses a minimum acceptable subset of the data and model that covers the most essential aspects of the smallest version of the problem. The reason this is important is before this, all conversations are abstract and theoretical. The first time a model with outputs is presented to an end user, the end user will start to imagine how they would use these results in real situations that have happened in the past. And they will start telling about all of the considerations they account for, the information they need to gather to make decisions, and who they need to consult and coordinate with. And this can change the entire project.  And from experience, I do not think it matters how much work is done at higher levels to define the project, the first time a model is presented to an end user the project will change so that the outcomes can be usable to the business. So it is best to make that happen as early as possible so that change causes the least disruption to the work in progress.

The idea of rapid cycles of iteration and feedback from the customer, and the willingness to accept changes to the project due to that interaction are the hallmarks of agile development methodologies in the software development world. Having regular rounds of model iteration where additional elements are added to the model, and getting feedback from stakeholders to confirm that the project is on the right track to produce something useful. And just like the software development world has experienced, this is more likely to lead to useful product, and actually faster than attempting to follow a rigid path that leads to something irrelevant.  

Conclusion

The Advent of OR proved to be a valuable exercise, offering a full-cycle project experience that highlighted two critical modern aspects of Operations Research: the integration of Generative AI and the necessity of an Agile approach. Generative AI demonstrated significant utility in accelerating system setup and basic modeling tasks, freeing up the analyst for higher-level problem-solving. More importantly, the experience reinforced that project success hinges on continuous, direct stakeholder engagement, mirroring the principles of Agile development. By prioritizing early delivery of a Minimum Viable Model, analysts can gain crucial feedback that aligns the project with real business needs, ultimately reducing the risk of communication-based failure and ensuring the final product is relevant and utilized.

Why take opportunities for public speaking as an analytics professional

For many of us in technical fields, public speaking often feels like a skill we left behind in school or perhaps dusted off for job interviews, especially if our roles involved training or teaching. Once we're in the professional world, the focus tends to shift solely to our day-to-day tasks, and public speaking opportunities seem to dwindle. However, effective communication is crucial for professional growth, and unfortunately, workplaces don't always provide sufficient feedback on technical presentations.

This is where engaging with local professional communities can be incredibly valuable. While I've had the privilege of speaking at professional society conferences, I've also found immense benefit in giving talks within local technical organizations. Many metropolitan areas are familiar with these as "Meetups," named after the platform that serves as their online home. These local speaking engagements offer distinct advantages compared to academic talks or large industry conferences.

Low-Stakes Practice Environment

---

One significant benefit of giving technical talks locally is the opportunity for low-stakes public speaking practice. These communities are typically smaller, comprising individuals genuinely interested in professional development. Because many members also use these meetings as a platform to share their own insights, the environment is inherently supportive and sympathetic. It's a space free from the competitiveness that can sometimes arise when individuals are trying to build a reputation or feel they're in direct competition. This fosters a very friendly atmosphere for honing your presentation skills, where attendees genuinely want to see you succeed.

Sharpening Your Communication

---

Secondly, preparing a talk for a public audience compels you to think critically about what truly matters. In a work setting, it's easy to gloss over foundational concepts because everyone involved in a project is assumed to have that background. In a public forum, you're required to identify the essential information and ensure you cover it as necessary background. This is particularly true for work-related topics when you might need to use public datasets (as most companies don't permit the use of proprietary data for more informal talks). This process forces you to consider what's important for your audience and what's technically crucial. It's an excellent exercise in organizing your thoughts and effectively communicating them, a skill that translates seamlessly back to your work when you realize not everyone on your team has the same background knowledge.

Building Professional Community

---

Finally, these local groups are instrumental in fostering community. Recent articles in local Pittsburgh publications have highlighted the increasing difficulty of forming social connections after school, and professional colleagues, while valuable, often don't entirely fill this gap due to shorter average tenures at companies and the inherent limitations of work-only relationships. Professional organizations offer the unique advantage of being specific enough to align with shared interests, yet broad enough to expose you to ideas beyond your immediate work. Giving a talk provides a natural reason for others to engage with you, sparking discussions and building relationships that can extend far beyond any single job.

(DRAFT) What do university departments provide to the employers of their students (data science)

 I gave a talk at the 2025 INFORMS (Institute for Operations Research and the Management Sciences) Analytics+ conference (i.e., industry practice focused as opposed to research focused) on Where Should the Analysts Live: Organizing Analytics within the Enterprise. The talk was a result of many organizations asking if analytics should be managed within companies centralized or de-centralized.  One of the topics that came up is the fact that much of the practice of data science is learned on the job.  For some people, they may ask if this is the job of universities. I would argue that the practice of data science is so large that this is an impossible ask. I do so from the perspective of someone who for a while was an industry focused professor within an R1 engineering department.

First, what is data science?  Drew Conway still gives the best definition that I have seen in the data science Venn Diagram

Math/stats are the full range of analytical methods as well as the scientific method (the 'science' of data science).  Hacking skills are the computer programming, software engineering, and data engineering specific to working with data (as opposed to what is generally emphasized by academic computer science). Substantive expertise is the subject domain of the work, but it also includes the specifics of the company such as understanding its markets, its customers, and its strategy.

Math/stats is in principle the domain of our university departments.  But university departments are specialists (and research faculty are hyper specialists.  There are two problems with expecting university departments to cover the full range of math/stats that may be needed at a particular company.  First, university departments focus on a particular domain, so it is not expected that they cover other areas of data analysis that a company may need based on their particular interests. Second, they have limited time and unless you are at a very large state university with a particular mission to cover the full range of a subject area, the faculty of a small or medium size department cannot cover the full range of topics that are associated with a given field of knowledge.  So departments create undergrad or graduate programs to cover a foundation, then allow students to specialize (in areas that the department can cover with the faculty they have).  As a non-tenure stream professor, I would explain to students that departments hire to cover a wide range of their field, so they generally do not have much duplication. But each department has to make a conscious choice for what they cover and not cover every time they make a hiring decision.

So what is a university promising with their graduates?  The base set of knowledge and methods (and methods are more important than knowledge, because it is easy to refresh knowledge, you actually need practice with methods), for STEM (and social sciences) the scientific method that creates understanding through iterative experimentation and statistical analysis of experimental results. And most crucially, the capability of learning a technical area. This ability to learn is arguably the most important part of this whole exercise.  Because the world is a big place, and a 17 year old high school student will not be able to predict what the next 40 years will be like. So where a 22 year old college graduate is capable of will be nothing like what she will do over the course of a career. It is hard to develop this ability without college. High school tends to be focused on what you know.  And it is too easy in most jobs to just do what you are doing now, unless you already have the experiences of having to learn new/different domains.  For example, in most STEM and the social sciences, statistics is a side knowledge domain. But for those who go into data science, the fact that they learned statistics makes learning applied machine learning easy.  And the scientific method, while it may not be the thing you think about when you think about engineering or economics, is ingrained into the methods by which they see the world.  It is relatively easy to teach skills, it is much hard to teach mindset or the ability to learn new ways to think.

Is there anything different about artificial intelligence? Actually, yes, which makes it easy to learn for STEM and social science trained people, but also dangerous.  By definition (see  Section 238(g) of the National Defense Authorization Act of 2019) any version artificial intelligence are those which perform tasks without significant human oversight, or that can learn from experience and improve performance when exposed to data sets. In particular, it means that the creators of an artificial intelligence system or model do not have to know how the system that the AI is being added to works. For those in the mathematical sciences (e.g. mathematics, statistics, applied math, operations research, computer science), this is incomprehensible. Even the most theoretical researcher has a core belief that any application of mathematical models involves representing important aspects of the system in mathematical form.  But this makes AI (such as machine learning) relatively easy to use in practice, and this has a low barrier to entry.  But if someone, like a company, actually has subject matter expertise relevent to the problem at hand, not incorporating that expertise into the model is lost value.

Is it enough to be able to learn new skills as needed?  No, we also have to be able to learn to think differently.  The most prominent example is Generative AI. For those who only have knowledge and skills, Generative AI is a completely new thing.  For those who are able to come up with new ways of thinking, Generative AI a combination and extention of deep neural nets, natural language processing, and reinforcement learning trained on the published internet.  And its strengths and weaknesses are not random facts akin to gotchas, but are based on characteristics related to its origins. And knowing that makes a world with Generative AI different, but something that we can use.   This past week I went to a seminar on quantum computing. The mathmatics are completely beyond me. but I could understand enough to recognize the reason for its promise, what is lacking, and some sense of what are some key intermediate steps that have to happen if it ever reaches the promise that many talk about.  And this practice of being faced with completely new subject domains is something I do frequently.

So what can companies expect from the graduates that come from their university partners (whether through former relationships or merely through hiring in the community).  Sometimes it is a collection of specific skills. But more important, a college graduate comes with a testiment that person is able to learn a range of skills and knowledge that are part of a cohesive whole and put them to use. And having done so once, will be able to do it again over a 40 year career.

On simplicity in data science communications

Everything should be made as simple as possible, but no simpler  paraphrase of A. Einstein

 I spent this past week at the 2024 INFORMS Analytics conference.  One of the major themes across both the speakers, the panels (of which I was a panelist), and conversations in the halls was soft skills for data analysts/Data Scientist.  Data scientists are subject to the same stereotype that is used with all those who are technical specialists, that they lack soft skills and this prevents our end stakeholders from understanding and being able to take advantage of the knowledge and capability we bring to our organizations.  And that the most commonly provided solution is to ask the specialists to simplify the delivery.  But, other than an excuse for those with business backgrounds to beat up on the geeks, I don't know if this is the right solution. A better direction would be, as John-Eric Bonilla described it, the data scientist acts as a translator, the person who takes the aggregated insight of the data and of the subject matter experts throughout the organization and translate that into the framework of the decision maker.  This is a tall order, but this is the reason that Drew Conway in his Data Science Venn Diagram gives subject matter expertise equal weight with the math/stats and the computer skills that get so much prominence in these discussions.

When I was deployed in Afghanistan, a brief that I prepared was being pushed up to the Commanding General, ISAF- Afghanistan. Members of the General's staff was present at the last brief, and their comment was that it was a good brief, but I needed to redo it in their format. And they gave me a highly specified template. Now, I could be judgmental and say that it met no conceivable definition of "simple", but I won't. Because I realized immediately that the template had two functions.  First, to shortstop any Powerpoint Rangers and save the General from Death by Powerpoint, because a commanding General in a combat zone is a busy person and does not have time for that.  But also because the purpose of the template was to present the information inside the framework that that particular General processes information for the purpose of making a decision.  And that specificity of presentation, that the recipient can process the information using the framework they have as an expert in their area and make a decision, is the goal of technical communications, such as data science.

The common recommendation to data scientists is that we need to simplify our work for presentation to our decision maker audience.  And the reason that we are given this message is that our decision maker audiences do not need or want our technical explanations and they cannot understand complex topics.  But this view of our decision maker stakeholders is demeaning.  So far, in my career I have found the decision makers that motivate my work to be intelligent, subject experts in their own right, and fully capable of understanding detail and nuance.  But the key is not to remove subtlety and detail (which is the reason this person is in the position of decision maker), but to present the subtlety and detail that is important.  Certainly, the tendency of technical experts to want to focus on the story of their work does not help either. The answer (IMHO) lies in the use of frameworks.  Every specialty community that I know of has frameworks that are used to organize and communicate information. Examples are the range of SITREP formats used by specific emergency response and military communities, the 9-line medevac report, frameworks used for reporting patient condition in specific circumstances in the medical community.  And individual leaders have developed a framework to make the decisions, even if this framework is masked in intuition. And in the ideal case, that record of good decisions is the reason they are in their position.

That makes the key of data communication is to understand the decision making framework used by experts in this situation. In the case of the then Commanding General - ISAF, this framework was formalized by the General's staff, so that all briefs going to him were presented in that framework. And the General can fit all of the provided information into his internal decision making framework.

When this framework has not been formalized, the key is direct communication between the data analyst and the decision maker (or surrogate). The data scientist needs to communicate with the decision maker, or someone who knows how the decision maker things (either intuitively, or because they are members of the same professional community who analyzes information in a standard way) to understand how the decision maker thinks.  Then, this identifies both the type of information and the criteria that will be used to make the decisions.  The data scientist task becomes either identifying the data needed to provide this information, or to use the data that is available to come as close as possible to the information required.  And this unlocks the value of the data scientist, without diminishing either the role or capability of the decision maker.

There are a few ways for this to fail.  First is from the data scientist side. Many technical experts have no desire to learn the decision maker process. This is often accompanied by beliefs that the technical facts make the needed action self evident. Then from the other side, there are those who think that they give commands to people and the people doing the work should be able to get it done without needed resources. Both fall under the heading of lack of communications between the analyst and the customer, which is universally known to be the most common cause of data project failure.  The role of the data analytics manager is to ensure that constant communications is maintained and to intervene if not. (There are managers who think their role is to be a broker. But this also breaks communications and does not help change the most common cause of failure in data analytics projects)

Is being a translator easy? No.  But I have found on my projects is that the data scientist is often the first person who realizes all of the people who are actually involved in an activity, because the data scientist is tracing all of the data elements. So the data scientist needs to learn everyone's language to get a good picture on what is actually happening, and communicate to the decision maker in a format the decision maker can understand. Yes, this is hard (80% of data projects fail, and while vendors use this to market products, those who investigate that number say it is mostly communications). But we are not the only people who have to take complex information and transmit it to a decision maker in a form they can understand and use to make decisions.  The UX community does this too.  And, often they do it well.  So can we.

Using blogs and news articles as class mini-cases - How discrete-event simulation can help project prison populations

How discrete-event simulation can help project prison populations (SAS Subconscious Musings)

My experiment this semester is more intensive use of news articles as subjects for in-class discussion of examples of applications of what we are learning.  While I have done this in the past, this semester I made it a deliberate plan to discuss one article a week in each class.  So far this semester in my simulation and decision models classes, I have covered reactions to the W. Africa Ebola outbreak, Game 7 of the 2014 World Series, Gamergate, flu vaccinations, commercial manned space transport, pulling a goalie in hockey, cargo shipping, business expansion, business divestiture, automation of manufacturing, health care system operations, among other things.

I identify articles through the use of RSS feed aggregators. My news feed includes a number of feeds from a range of business school professors focusing on supply chain and operations management issues.  I follow the CDC MMWR as well as the journal Health Affairs to get health care related articles.  And the New York Times front page and Google News are good for a lot of different stories.  The key is finding an article where the reporter was good enough to discuss the various options that were available and enough details that you can figure out the values of various actors involved.

The key a good class case article is that there are potentially reasonable alternatives to discuss.  In the decision models course, the discussion revolves around identifying the courses of action available, the sequence in which decisions need to be made and information becomes available, assessing the attributes (values) of the people involved, then assessing how they may assess probabilities of various events.

For the simulation course, the focus on case discussions is on understanding how a decision maker in the article may use the simulation, then we do a whiteboard exercise where we draw out an event graph diagram to model that system, focusing on what needs to be included (states, events) based on the decision maker needs. The goal is to discuss modeling in a specific context, so we can talk about what needs to be included, and what does NOT need to be included in the model to fit the particular purpose.  The contrast is to the textbook homework problems, which generally provide a very specific context and set of details which have be included in the model to answer the homework problems.  Textbook problems generally do not include thinking about modeling in such a way to determine what is the right question and how to simplify the model to address the question.

Last week we looked at the decisions made by the North Carolina Sentencing commission.  Unlike most cases, in this case we happen to know for a fact that a simulation was used in the decision making process.

Our discussion began with purpose: why would the North Carolina Sentencing Commission be interested in a simulation of prison population. We came up with the need to plan prison space, make arrangements with neighboring states to house NC prisoners, and to allocate resources to monitor parolees.

Next, a discussion of what would the simulation need to track to fulfill the purpose of the NCSC. This would include the number of prisoners and the number of parolees. And the time remaining for each prisoners sentences. Then, we have the state of the system being the prisoners and parolees, and the terms of sentencing. (we decided not to discuss the size of the prison, since that is something that was being determined).

The last part of the discussion was where the typical homework or exam problem started, diagramming the events tracked by the system, how each events changes the system state, and then how to generate delays in the simulation.

The purpose of the exercise was to discuss modeling. Not in terms of how you build a model from a system description, but to think through how to model and make modeling trade-offs given the decision that needs to be made about a specific system.  The cost of this discussion is time, doing this results in us not completing a semester syllabus of a class that is quite analytical. But, as textbooks usually begin modeling examples with a system description and a purpose, I think it adds to the course and I think the compromise is worth it.

Thanks to Natalia, Jeff, and Leo from SAS for our conversations about this particular SAS case at the INFORMS conference. It enriched the class discussion to know what was happening behind the scenes of the Subconscious musings blog article.

Data Points by Nathan Yue: Book review

Data Points: Visualization That Means Something by Nathan Yau
My rating: 4 of 5 stars

This book is not about how to create data visualizations, it is about how you use visualizations to communicate data. In that respect it is not trying to be a book about tools, but a book on aesthetics, it focuses on how you evaluate different combinations of visualization options for communicating different types of information about data, not just a number of rules. In this respect, it goes considerably deeper and profound about how people comprehend and interpret visualizations than a set of pithy rules masquerading as common sense. In this respect, it is a successor to Tufte in an age where being able to try alternative visualizations and even having consumers interact with the visualizations is cheap.

The book is not a description of various types of visualizations, even though it has such descriptions and discussion of comparative assessments. It is a book on how to think about the message(s) you are trying to communicate, and how to do so in ways that can engage the reader at many layers of depth where simple messages are easily grasped, and complex messages can be absorbed with their relations and implications. Along the way he discusses the relative strengths of using different types of visual cues to communicate information (position, length, angle, direction, shapes, area, volume, saturation, hue), which is much deeper than saying 'bar charts are better than pie charts' (which is an argument that a post-doc tried to engage in with me once). After a brief introduction, he proceeds to show you by example after example of the comparative qualities of each cue, and also how they can be used in combination to show multiple levels of information and relationships.

One of my biggest insights from 'Data Points' is actually not discussed in the book. The book gives you the understanding you need to evaluate the range of combinations of means of presenting data. But about halfway through I realized that the discussion and philosophy of combining these visualizations has a name. Wilkinson's Grammer of Graphics. I have learned the ggplot implementation of Grammer of Graphics, and I favor it above other plotting families in R and Python as being more flexible and giving you more control over the result. The discussion in Data Points explains why Grammer of Graphics is important, it provides an interface for exploring combinations of aesthetics (visual cues) to communicate aspects of complex data sets. And with this, it will probably change how I present and teach visualization for data analysis.

View all my reviews

Interactive Data Visualization for the Web by Scott Murray: Book review

Interactive Data Visualization for the Web by Scott Murray
My rating: 4 of 5 stars

This is really a d3.js (Data Driven Documents) book, since the entire book is about using the d3 Javascript library for data visualization. For me, coming in with a background in data analysis but no practical Javascript background, this provided a good overview of many Javascript/web concepts required then the d3 library.

My background in data visualization is R (ggplot and lattice) and Python (Matplotlib). I find Excel too limiting and Matlab inflexible. But what I'm used to are descriptions of visualizations that are then implemented. d3 is different, it makes you specify the form of the visualization and makes you work at a very low level (which is what ggplot and Matplotlib are trying to avoid). But this provides room for great creativity, and learning how to do this is the point of this book.

This book does a good job of introducing enough of the background technologies of Javascript and the web to make d3.js useful. This was always the reason I never got around to actually using Javascript, I never learned all the things you needed to know to make it useful. For example, I now think I understand JSON well enough that I can see using it as a data store for data frames instead of CSV or SQLite or pickles (Python).

I come at this as someone who already does data analysis, but may need more ways of delivering the products of the analysis to others. And this has promise as a means of this delivery through a browser, which is already comfortable for most. And this book does a good job of working both the subject of d3.js and the web ecosystem that it is a part of to get to someplace useful.

Disclaimer: I received a free electronic copy of this book as part of the O'Reilly Blogger Program

View all my reviews at Goodreads.

Book review of Statistics in a Nutshell 2nd ed. by Sarah Boslaugh

Statistics in a Nutshell by Sarah Boslaugh
My rating: 4 of 5 stars

One of the biggest problems faced in teaching statistics is the gap between learning the methods to actually using them. Statistics classes that are based on learning formulas fail due to the disconnect between learning formulas and the reality that very rarely are these methods used by implementing the formulas that are so painstakingly taught. But learning statistics as a set of steps or functions in a computer package only gets a little further. The real goal should be what methods should be used and why. The how is almost secondary. Statistics in a Nutshell focuses on the what and the why. I would not use this to learn how to perform a technique or its formulas, but this is where to go to understanding how the various methods of statistical analysis should be used and their qualities. It is meant to be read, not just studied, and as such it holds a different place than other statistics texts.

When I first learned statistics, the focus was on learning formulas that calculated various values. But the problems I could work on were only toys, and it took so long that we did not get into much of understanding what we were dong. Now with readily available software packages, the temptation is to focus on the mechanics of implementing a procedure on a set of data and reading the computer output. But software documentation and even books that teach statistics fall into the trap of focusing on how a method works then applying it and not as much on why. Part of this is because of the pressure of having to cover topics, but the fact that the methods are presented in isolation, without their application context so it is rare to grapple with the question of how to know what needs to be done and instead the focus is on how to do it.

Statistics in a Nutshell is the other kind of book. I was taught that for computer programming for any language you wanted a book that focused on methods, but also a book that focused on morals, the why you use a language feature. This is the morals book for statistical programming. You read it, not to learn how to calculate statistical output or implement visualizations, but to think about what method or visualization is appropriate to help understand the data and environment and to communicate those truths to an audience.

Because of the expectation that any course that teaches statistics gives the students a toolkit, this would never be a good book for teaching a course. But in the real world, what is more important is that you understand what these statistical methods are and why you use one over the other. So for the data analyst or a student who needs an overview of everything this is ideal. It would also be ideal for someone who may not have the time to go into detailed study of statistical methods, but needs to interpret the results or work with statisticians and data analysts. This book will help interpret what you get and ask the right questions to both understand statistical results and perhaps point those who are doing the analysis in the right direction so that they are answering the right questions.

Note: I received a free electronic copy of Statistics in a Nutshell through the O'Reilly Press Bloggers program.

View all my reviews

Book Review: Programming Python 4e by Mark Lutz

Programming Python by Mark Lutz

My rating: 4 of 5 stars


I've been using Python since I was a graduate student. While I have known of earlier versions of this tome it had never occurred to me to go through this. Because my focus in computer programming is not in the systems administration or IT related skills like most of O'Reilly Press customers seem to be. My interest is in scientific programming and data analysis. Now that I have had some time and occasion to use this, I find that the content is indeed weighted at an audience whose function is computer programming (as opposed to using computers for something else), but what this book really provides is an education on how to accomplish tasks in idiomatic Python, not just a programming how-to book.



Programming Python (PP4E) goal is NOT to teach programming in the Python language (that task was for another book by Lutz, Learning Python). The stated goal was to teach practical use of Python, and that necessarily means using the standard libraries. But even at 1600+ pages you can't do everything. So this book focuses on (IT) systems administration, GUI development and web programming using Python 3.x (not covering Python 2.x). None of these are what I do. But, there are some tasks that are covered that can be very useful, so in reviewing this book I focused on Graphic User Interface (GUI) and text processing.



What I found as I worked through these sections was that PP4E was not a reference as it lacks systematic coverage of topics. And it is not just a tutorial, although it definitely follows a crawl, walk, run sequence as it covers the topics. What it does do well is cover how to think and how to make design decisions. So for the GUI section it focuses on tkinter, and it does cover various widgets, window managers, etc., where it shows its colors is when it discusses how the parts work and how you make choices between alternatives (e.g. window managers). There are some warts with its focus on Python 3. There are several chapters in the GUI section that require the Python Imaging Library (PIL). But based on the PIL web page, PIL is not ported to Python 3.x. So it probably means that Lutz used a pre-release port without telling us.



Similarly, regular expressions is something I've never quite gotten. And I would not be able to master use of the re module here, since it lacks a usual lists of functions with examples that a reference or a tutorial would have. But what it does do is to cover how regular expressions fit along with other string operations, when and why to use match, search, find, findall, compile. When to compile and how to think through building a regular expression sequence. You would not learn how to use regular expressions here (I would never be able to develop the examples from what is in the book), but you will learn how to think through them, and bring that when you go back to a real reference or instructional book.



One aspect that is very annoying is the examples quickly become more complex than the material that leads up to it. So instead of crawl, walk, run; it takes on the feel of crawl - sprint, with a basic introduction then quickly moving into a complex and mature code. Some of this is a result of a goal of trying to be deep in everything. So be warned.



I received a free electronic copy of Programming Python from O'Reilly Press as part of their blogger review program.




View all my reviews at Goodreads

Book Review: Data Analysis with Open Source Tools by Philipp Janert

Data Analysis with Open Source Tools by Philipp K Janert

My rating: 5 of 5 stars


This is a book that is how to think about data analysis, not only how to perform data analysis. Like a good data analysis, Janert's book is about insight and comprehension, not computation. And because of this it should be a part of any analysts bookshelf, set apart from all the books that merely teach tools and techniques.

The practice of data analysis can get a bad rap, especially by those who think that data analysis is only statistics. Most books on data analysis don’t help because they focus on using the features of a particular tool, leading to the view that data analysis is following a recipe from a cookbook. This book subverts this by being principally of how to think about data analysis, and providing examples using different tools (primarily R and Python, but he uses other examples as well)

Among other topics, Janert covers graphing, single and multi-variable analysis, probability, data modeling, statistics, simulation, component analysis, reporting, financial modeling and predictive analytics. In each section he starts by explaining the concepts, what it is for, and (just as important) what each topic is not. Working through it you get a sense of not just what and how of the various tools and methods discussed, but why they are used as well as some ways these techniques are misapplied.

Janert also illustrates the methods using some data analysis environments. Principally R and Python (with Numpy, Scipy and Matplotlib), but also other tools such as Gnuplot and the Gnu Scientific Library. What is helpful here is the focus is on what techniques and capabilities are needed in the tool, not the tool itself. Instead of being a cheerleader for a particular tool, Janert discusses in his appendix the qualities that make environments such as Matlab, R and Python good data analysis environments. However, this focus means that he does not teach any particular tool. If you want to learn how to use a particular tool for data analysis, you are better off getting a book on R or Python (or Matlab, Excel, etc.)


The book page at O'Reilly.com is here: Data Analysis with Open Source Tools





View all my reviews

Conference notes: INFORMS 2010

I was in Austin, TX for the Institute for Operations Research and Management Sciences (INFORMS) annual conference. I had gone to this conference through grad school, and a few times since. This time was a little different.

There are many reasons to go to conferences. To keep up with your field. To follow recent developments. It is an efficient way of learning new developments (compared to only reading a paper). To present your work. But it is also a way to keep in touch with old friends (and if you are in a profession for any length of time, there should be friends at conferences). And in my case, the obvious conversation topics were:
"How old is he? Why are you here?" and "Pictures!"

As far as the professional side went, I had my talk. It went well. And a good chunk of the audience left right after my talk (I was the first talk of the session, so this meant that people were in the room for my talk, then they went to where they really wanted to be for the rest of the session time slot).

But what made the conference different was my talk was not important (ok, so maybe that has not changed). I was part of two talks at the conference, and this was the boring one. The interesting one was my student's. And that went well. Good audience reaction and commentary. And definite interest. Which bodes well when he does this again next year, with visions of graduating dancing in our heads. The goal for this conference was not about me, but all about him. Having him give a talk. And meet all the researchers who wrote the papers that he has been reading, poring over, and extending. To talk about what they have done and what he is doing now. Developing confidence that what he is working on is indeed something reasonably new, a contribution to the body of work, and something that can be of interest when it is done. It was a good feeling, watching my student interact with people I know (and many I did not). While talking to my own advisor, another, more senior researcher was commenting that it would be my advisors first 'grandchild'. I have to agree, talking about my student is a lot like what being proud of your own child is supposed to be like. So, in a sense, I have two thriving children, and one getting ready to take flight.

Why I am a researcher

I was asked yesterday about the value of a Ph.D. It is a fair question. When I was in graduate school, occasionally a first year student would say something to the effect that he was in grad school because he wanted to make money. We all informed him that he assuredly had the skills to do very well, with an MBA (equivalent) from the excellent business school down the block at our university. And that he would make more money for considerably effort then the road that he was starting on. I did not start my professional life as an academic, so this is a question that in a sense has an answer.

After I finished college I went to D.C. to a policy school for a professional master's degree and to work. (because all Master's in policy students usually work while in school). Over the next four years I worked for a legislature, a non-profit, a government agency and at a government contractor. I did this as a quantitative analyst and as a logistician. And it was fun. I worked on environmental studies and projects that were at the cutting edge of applied economics and modeling. There is work I did that I could see in the news over a decade later as they came to fruition or the issues they addressed began to be considered by commercial enterprises. I had few projects that were routine. There was always the sense that you were developing something new, and while there may be ways to do that, what you would find along the way was not always obvious in foresight.

But I also realized that the only reason I was working on these projects was because of my boss(es) along the way. And this is not what work would always be like. Looking ahead, I realize that there were really two ways to have a career doing new things like this. One was to work for twenty years and establish yourself at the top of your field (like my last boss). The other was to get a Ph.D.

And this was the value of the Ph.D. Not the knowledge gained along the way, but the opportunities to work on things that were new and hard. At one meeting with non-researchers I've explained I do not want to be working on problems unless someone else has examined it and failed to find a resolution. And at this time, I can say I have this. Will this continue? As a non-tenure track academic I have a good place working on difficult applied problems, where there are both technical and social issues that need to be solved (as I've commented to one doctor, if the social issue is what is preventing the situation from being resolved, it is an issue that needs to be solved.) Is there a place long term for this? One of the criticism of academia is that it focuses on problems that are theoretically hard rather then hard in practice, even when the reason it is hard in practice is because of the lack of development in theory. In Operations Research, there is pride in working on hard problems, but the definition of hard is based on the technical difficulty of the math, while I tend to use the definition that smart and talented people have tried and failed as my benchmark. Whether there is a place for that remains to be seen.

WinBUGS and JAGS differences

One of my graduate students and I have been working on input modeling in a setting where the amount of historical data can range from a lot to almost no (i.e. 0, 1, 2) occurrences. So both of us have been learning Bayesian methods and developing means to work with our setting. Part of this involves working with the standard tools and building blocks for Bayesian methods, in particular Monte Carlo Markov Chains (MCMC). To implement these methods, you can program your own, or use some standard modeling frameworks. Some of these frameworks are essentially programming libraries. But the primary ones are Bayesian inference Using Gibbs Sampling (BUGS) derivatives. There are two families: WinBUGS/OpenBUGS and JAGS.

WinBUGS is the direct successor to BUGS, with OpenBUGS being the open source next generation. BUGS is both a software package as well as a model specification language for MCMC models. One issue with WinBUGS/OpenBUGS is that it is written in Component Pascal. WinBUGS depended on the BlackBox component builder, which is only available in MS Windows (as does OpenBUGS, but OpenBUGS is essentially the open source next generation of WinBUGS once it is done). JAGS is an opensource implementation of the BUGS model specification and command language and is written in C++ on top of some open source libraries. JAGS exists specifically because of the inability to port WinBUGS. A review of the literature shows that WinBUGS/JAGS/various other libraries, all have areas of strength and weakness. Research groups working extensively with Bayesian methods use all of the them at various times.

But, this is our first project. So one thing we did was to examine three frameworks while we were learning Bayesian methods. I primarily used JAGS, one graduate student worked with WinBUGS, a third graduate student (the strongest programmer) used MCMCpack (an R package).

Our intent was to use either WinBUGS or JAGS when we actually implemented our methods on real data, so the question was which to use. Some obvious differences that came up.

1. When running using R, the RWinBUGS interface opens the WinBUGS application to run BUGS model. We anticipate having to run this on ~50 separate data sets a day as part of our methodologies. This time of interface is unstable (both theoretically, and practically as my student's computer ends up hanging a lot)
2. WinBUGS is a lot more particular about the BUGS model specification. Or, since the WinBUGS people are the ones who developed the BUGS model specification, it is probably better to say that JAGS is much more forgiving. Some gotchas that came up included:

• When specifying distributions, parameters in WinBUGS could not be expressions. Only single variables. I.e. something like (num1*num2) is not allowed. So we have to perform all computations, then specify the distribution separately. JAGS allows passing an expression as a parameter.
  
• WinBUGS does not have an exponent operator (either '^' or '**') JAGS does. Since one parameter that is important and common is 1/variance, this is very useful when the data is expressed in terms of standard deviation. We have to use expressions like (num1*num1) all over the place.
• JAGS from R is seamless (R2jags, rjags). You call JAGS, and it returns without opening any IDEs. You can see the JAGS output in the terminal, but JAGS runs in command line, then is done. No IDE left to close. (R2jags was designed to work the same way as RWinBUGS, but I think it is much more elegant)
• The JAGS documentation includes some more differences, but we have not hit those yet.

So, for this project, given that we have a certain amount of risk based on the fact that we are figuring out Bayesian methods and MCMC along the way, we're going to go with JAGS as our engine. Someday we'll get better at using WinBUGS/OpenBUGS and MCMCpack, but not today.

QOTD: INFORMS 2009 Part 1

P1: What do you do?
. . .
P1: What does your wife do?
. . .
P1: You are going to have brilliant babies!

P2: (while standing between two baggage claim terminals for 5+ minutes) What baggage claim are the bags from ___ coming?
Me: I know nothing.
. . .
Me to P2: You know, I try really hard to cultivate the 'I know nothing' look, obviously I'm not doing nearly a good enough job at it.
P2: You already got your bag.

P3: "Buenos dias (more stuff in Spanish). Ooh, (aside to coworker) I just started speaking in spanish to him."

P4: I did not know anyone else was doing work on ___. How come I don't know who you are?!
. . .
P4: (to a section chair) I want to have a session on ___ (next year). (to me) You will have something, right?
Me: My student will have something ready.

Me to P5: I'm here to do what traditionally gets done at conferences, meeting someone who works one block away from me.

Teaching QOTD: This is how it will be for the rest of the semester

I'm teaching the introductory Operations Research undergraduate course. (typically taught to first semester juniors and the core usually is introduction to linear programming.) Even though it is at 8AM the first two weeks have been very lively. Out of ~60 students, there are probably 15+ that take an active part in class discussion. I have been able to teach the material almost completely using question and answer. It is not quite Socratic method, but it is very close (and lets be real, even law schools don't truly use Socratic method.)

Until today (day 1 of week 3). Most of the class was pretty dead. The usual chatter of people arriving is subdued. And I am down to a few reliables to answer questions. I felt like I was actually lecturing for the first time this semester. By 15 minutes I acknowledge the obvious and we take it nice and slow for the rest of the class.

Later today I was chatting with our undergraduate coordinator (who has a good read on the undergrads. She is a teaching professor so they are her focus.) And she told me that yes, this class is particularly good. And that now that we are in week three, the initial excitement and energy of the new semester has worn off, and this is how it will be for the rest of the semester. *sigh*

QOTD: Engineers seem to have an interesting, and somewhat unique, definition of fun

Engineers seem to have an interesting, and somewhat unique, definition of fun." Ask your doctor or a local public health official what defines a good time, and you are not likely to hear about the logistics of mass antibiotic distribution or mathematical models of influenza transmission. Larry Wein's opus over the last decade has been, to use a hackneyed phrase, deadly serious, and yet he notes that a personal criterion for his engagement in a project is that it be enjoyable. For those of us in public health emergency response, this provides an important insight into the workings of the engineering mind, since it is precisely those things that we often avoid (i.e., the nitty gritty, and by that I mean quantitative, evaluation of the feasibility and performance of the programs we plan and carry out in the name of health protection) that appeal to Prof. Wein and a relatively small number of colleagues.

From Nathaniel Hubert, Commentary on "Commentary on Homeland Security: From
Mathematical Models to Policy Implementation" by Lawrence M. Wein, Operations Research, OR Forum

A Long Line for a Shorter Wait at the Supermarket

Queueing makes the New York Times. Something else for me to point at when someone asks "what do you do?"

A Long Line for a Shorter Wait at the Supermarket (New York Times)