Sunday, December 09, 2012

Lessons observed from using cases in teaching decision analysis

I taught a course in Decision Models last semester. While there was a text, instead of relying on the text for the assignments, I used cases to provide practice in applying the concepts.

I chose two cases from the INFORMS Transactions on Education, Home Depot and the OVA1 case. In addition, I had them read a book "Against the Gods" by Bernstein, and a personal case. The class as a mix of undergraduate and graduate students. All of the undergraduates were US born while the graduate students were more varied in background including US born, Chinese, Indian, Turkish, and one Western European. A few had work experience.

The Home Depot case was a multi-attribute decision making case. I had the students form teams of two and take a point of view in the case: HomeDepot, competitor, the local government, or local residents. While the case writers used one method, I had them structure according to SMART, which is what was used in out textbook and the class. Some notes. The students often had trouble staying in a point of view. This led to some strange arguments during the course of the case report. Also it made the assessment of values difficult for the students, as observed in their discussions and the values they assigned to attributes. Noone had background in the issue, except for one student who witnessed a similar debate in his hometown. They found assigning numbers to values difficult, But one result that I pointed out, also noted by the case writers, was they were remarkably in agreement in their final recommendations, even though their choices of. numbers varied wildly. As noted in the book and by the case writers, this was one of the justifications for the value of structured decision analysis methods in a highly subjective environment. I think that this exercise gave the course validity.

We also covered the eventual outcome. As the case writers expected, they generally came to the same conclusions as the decision in real life. But the eventual outcome s not very good, and in real life it is being revisited, so we could talk about if the decision was good even if the outcome was not as good as hoped for.

The OVA1 case was on medical decision making, and teams could be the doctor, the patient, or an insurance company. There was one person considering medical school in the course, This was used as an exercise in developing decision trees and assigning utility in the face of uncertainty. in addition, this was before they covered the value of information, so they essentially, they were developing the techniques of determining the value of information as they went along. They did much better with keeping with their assigned role this time around. And it took work, but they figured out the expected value of information method, which made teaching it later very easy. Also, the case got across the idea that decisions look very different when looking over a large number of instances (doctor and insurance company) compared to when you looked at a sample of one (patient).

The real question is the use of cases instead of homework. In an engineering department, this would naturally be one of the less quantitative courses. However, I presented it as an integrative course, meaning that the other courses teach methodology that are used to support decisions. Even with the case that was supposed to be based on a personal issue (I gave them some leeway by saying that it could be based on something or someone they personally know instead of just themselves)they commented on how hard it was to find data or determine utility in a quantifiable form. I think that presenting this in the form of cases made them work harder at it and gave them a deeper understanding than if I had them do a homework questions from the book where the numbers were given and they did not have to think about where they came from. I had many people comment in their reports that this class was making them think about many issues differently. (especially the personal case, where most of them even thought to include such considerations of their romantic relationships in their evaluation of decisions.)

No comments: